With the rapid development of industrial informatization and deep learning technology, modern data-driven fault diagnosis (MIFD) methods based on deep learning have been continuously emphasized by the industry. However, most of these methods require sufficient training samples to achieve the desired diagnostic effect, but the scarcity of fault samples in the actual industrial environment leads to the limited development of MIFD methods. In addition, due to the changes of equipment operating conditions and production requirements, data-driven fault diagnosis methods often need to face the cross domain problem of cross load or even cross different equipment. In this paper, a parameter optimization and feature metric-based fault diagnosis method with few samples, called model agnostic matching network model, is designed for the problem of sparse fault samples and cross-domain between data sets in real industrial environments. The method combines both a parameter-based optimization meta-learning network, which extracts optimization information adapted to different domains, and a metric-based meta-learning network, which extracts metric information for similarity discriminations. The experimental result show that the method outperforms the current baseline method for the 5-shot fault diagnosis problem of rolling bearings under limited data conditions and achieves an accuracy of up to 94.4% in cross-equipment diagnosis experiments from rolling bearings to gas regulators, indicating the feasibility of the method. The features are visualized by T-SNE to show the validity of the model.