Measuring selective attention in a speeded task can provide valuable insight into the concentration ability of an individual, and can inform neuropsychological assessment of attention in aging, traumatic brain injury, and in various psychiatric disorders. There are only a few tools to measure selective attention that are freely available, psychometrically validated, and can be used flexibly both for in-person and remote assessment. To address this gap, we developed a self-administrable, mobile-based test called “UCancellation” (University of California Cancellation), which was designed to assess selective attention and concentration and has two stimulus sets: Letters and Pictures. UCancellation takes less than 7 minutes to complete, is automatically scored, has multiple forms to allow repeated testing, and is compatible with a variety of iOS and Android devices. Here we report the results of a study that examined parallel-test reliability and convergent validity of UCancellation in a sample of 104 college students. UCancellation Letters and Pictures showed adequate parallel test reliability ( r = .71–.83, p < 0.01) and internal consistency (ɑ = .73–.91). It also showed convergent validity with another widely used cancellation task, d2 Test of Attention ( r = .43–.59, p < 0.01), and predicted performance on a cognitive control composite ( r = .34–.41, p < 0.05). These results suggest that UCancellation is a valid test of selective attention and inhibitory control, which warrants further data collection to establish norms. Supplementary Information The online version contains supplementary material available at 10.3758/s13428-021-01765-5.