Achieving asymptotic and concurrent fault diagnosis in hydraulic system remains a challenging endeavor due to the inherent attributes of the hidden occurrence, simultaneous manifestation, coupling, and limited sample size. To address the above issues, this paper proposes a hierarchical multi-output fault detection and diagnosis framework, namely, HMDF, based on a hierarchical learning strategy to leverage an improved convolutional neural network (CNN) and support vector machine (SVM). Both a multi-channel CNN and a multi-branch CNN are employed to extract and downscale features collected by the sensors at diverse sampling frequencies first, and then, such features are subsequently subjected to classification using SVM. The hierarchical learning strategy enables the identification of different fault states, both at the component and the intra-component level. Additionally, a modified whale optimization algorithm (WOA) is also utilized to optimize the classification process of SVM. Extensive experiments are conducted to test the proposed HMDF with the hydraulic system datasets. Results show that HMDF achieves a diagnostic accuracy of up to 98.9% for the dataset, surpassing traditional methods reliant on manual extraction of time-frequency features, and it also exhibits superior classification performances with a small sample size. The HMDF is expected to offer a generalized framework for the multi-output fault detection and diagnosis in hydraulic systems and other complex components.