Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by various sensory perceptual and cognitive issues that lead to far-reaching challenges in autistics' social and daily lives. Research in HCI is inclined to take an important role in providing support to autistic individuals. However, due to the sensitivity of this user population and the HCI research community's lack of experience in working with neurodiverse users, researchers struggle while conducting and reporting on user studies with autistics. Aiming at analyzing autistics' involvement in design processes and the struggles encountered, here we present a critical review of user experience studies with autistic users conducted between 2010 and 2016. Using qualitative and quantitative approaches together, we revealed common practices and major problems. We provide a twofold contribution to the HCI field: A guideline for conducting studies with users with autism as well as suggestions on how to report these studies.