Collaborative Problem Solving (CPS) is a widely disseminated, neurobiologically based, trauma-sensitive treatment for children’s challenging behavior. Measuring treatment integrity is critical to support implementation and continued research on the approach. This article presents the development and psychometric evaluation of an observational system, the CPS Manualized Expert-Rated Integrity Coding System (CPS-MEtRICS), for measuring CPS treatment integrity. Audio recordings of in-home treatment sessions (159 CPS and 82 treatment-as-usual) were independently rated by pairs of seven trained coders for integrity and the affective aspect of client–provider relationship. Results showed satisfactory interrater reliability (intraclass correlations [ICCs] 0.64–0.86). Additionally, the integrity scores on the CPS-MEtRICS discriminated between providers delivering CPS and treatment-as-usual, while these two groups did not differ in general client–provider affective bond, supporting the construct and discriminant validity of ratings using the system. This study provided evidence for the reliability of the CPS-MEtRICS and the validity of its integrity ratings, which can be used for research on CPS and which will provide the foundation for other CPS integrity measurement tools useful in practice settings. This article also provides a model that can be used when developing integrity measurement systems for other therapeutic approaches.