To construct a more effective model with good generalization performance for inter-site autism spectrum disorder (ASD) diagnosis, domain adaptation based ASD diagnostic models are proposed to alleviate the inter-site heterogeneity. However, most existing methods only reduce the marginal distribution difference without considering class discriminative information, and are difficult to achieve satisfactory results. In this paper, we propose a low rank and class discriminative representation (LRCDR) based multi-source unsupervised domain adaptation method to reduce the marginal and conditional distribution differences synchronously for improving ASD identification. Specifically, LRCDR adopts low rank representation to alleviate the marginal distribution difference between domains by aligning the global structure of the projected multi-site data. To reduce the conditional distribution difference of data from all sites, LRCDR learns the class discriminative representation of data from multiple source domains and the target domain to enhance the intra-class compactness and inter-class separability of the projected data. For inter-site prediction on all ABIDE I data (1102 subjects from 17 sites), LRCDR obtains the mean accuracy of 73.1%, superior to the results of the compared state-of-the-art domain adaptation methods and multi-site ASD identification methods. In addition, we locate some meaningful biomarkers: Most of the top important biomarkers are inter-network resting-state functional connectivities (RSFCs). The proposed LRCDR method can effectively improve the identification of ASD, and has great potential as a clinical diagnostic tool.