Although the observation and assessment of psychotherapeutic competences are central to training, supervision, patient care, quality control, and life‐long practice, structured instruments are used only occasionally. In the current study, an observation‐based tool for the Assessment of Core CBT Skills (ACCS) was translated into German and adapted, and its psychometric properties were pilot evaluated. Competence of therapists‐in‐training was assessed in a random sample of n = 30 videos on cognitive behavioural therapy including patients diagnosed with hypochondriasis. Two of three raters independently assessed the competences demonstrated in the entire, active treatment sessions (n = 60). In our sample, internal consistency was excellent, and interrater reliability was good. Convergent validity (Cognitive Therapy Scale) and discriminant validity (Helping Alliance Questionnaire) were within the expected ranges. The ACCS total score did not significantly predict the reduction of symptoms of hypochondriasis, and a one‐factorial structure of the instrument was found. By providing multiple opportunities for feedback, self‐reflection, and supervision, the ACCS may complement current tools for the assessment of psychotherapeutic competences and importantly support competence‐based training and supervision.