Fuzzy cognitive networks (FCNs) arose from traditional fuzzy cognitive maps (FCMs) to have the advantage of guaranteed convergence to equilibrium points, thus being more suitable than conventional FCMs for a variety of pattern recognition and system identification tasks. Moreover, recent developments led to FCNs with functional weights (FCNs-FW), as a significant FCNs enhancement in terms of storage requirements, efficiency and less human intervention requirements. In this paper we proceed further by introducing hybrid deep learning structures, interweaving FCNs-FW with well established deep neural network (DNN) representative structures and apply the new schemes on a variety of pattern recognition and time series prediction tasks. More specifically, after discussing general issues related to the construction of deep learning structures using FCNs-FW we present three hybrid models, which combine the FCN-FW with convolutional neural networks (CNNs), echo state networks (ESNs) and AutoEncoder (AE) schemes, respectively. The hybrid schemes are tested on diverse benchmark data sets and prove that FCN-FW based hybrid schemes perform equally well or better than state-of-the-art DNN-based schemes, paving thus the way for using cognitive networks to deep learning representative structures.