Competition among individuals is a natural mode of determining who is fittest. While in nature, economics, and sports, it is common to infer ability or aptitude from the outcome of competitions, our knowledge on its effects in regard to psychological/educational assessment is scarce. In the present pilot study, we explore a measurement approach for assessing individual differences in interpersonal, face-to-face competitions, based on a set of cognitively demanding, competitive, fast-paced, two-opponent tasks. For initial task evaluation, we conducted comprehensive reliability and construct validation analyses, considering cognitive ability, motivation, and personality measures. Moreover, using structural equation models we conducted a simultaneous factorization of the tasks with the other validation measures. The results suggest that the newly developed tasks measure both cognitive ability (intelligence) as well as a competition-specific component. The competition-specific component was positively associated with experience in competitive gaming and negatively correlated with neuroticism. While the pattern of validities was promising, the measurements’ reliabilities were yet unsatisfactory. Implications for future research as well as the design of competition-based measurements are discussed.