BackgroundPain Squad is an evidence-based, freely available iOS app designed to assess pain in children with cancer. Once research-based technologies such as Pain Squad are validated, it is important to evaluate their performance in natural settings to optimize their real-world clinical use.ObjectiveThe objective of this study was to evaluate the implementation effectiveness of Pain Squad in a natural setting.MethodsParents of 149 children with cancer (aged 8-18 years) were contacted to invite their child to participate. Participating children downloaded Pain Squad on their own iOS devices from the Apple App Store and reported their pain using the app twice daily for 1 week. Participants then emailed their pain reports from the app to the research team and completed an online survey on their experiences. Key implementation outcomes included acceptability, appropriateness, cost, feasibility, fidelity, penetration, and sustainability.ResultsOf the 149 parents contacted, 16 of their children agreed to participate. More than a third (6/16, 37.5%) of participating children returned their pain reports to the research team. Adherence to the pain assessments was 62.1% (mean 8.7/14 assessments). The 6 children who returned reports rated the app as highly feasible to download and use and rated their overall experience as acceptable. They also reported that they would be willing to sustain their Pain Squad use over several weeks and that they would recommend it to other children with cancer, which suggests that it may have potential for penetration.ConclusionsWhile Pain Squad was well received by the small number of children who completed the study, user uptake, engagement, and adherence were significant barriers to the implementation of Pain Squad in a natural setting. Implementation studies such as this highlight important challenges and opportunities for promoting the use and uptake of evidence-based technologies by the intended end-users.