Children with complex congenital heart disease are less active than recommended for optimal health, with social and physical environments important determinants. The purpose of this study was to examine the physical activity perceptions of children with complex congenital heart disease and their parents to identify social and physical environment intervention targets. A semi-structured discussion guide elicited physical activity perceptions from children (26 boys, 19 girls, 6.0–12.4 years) with complex congenital heart disease (single ventricle n = 42) and their parents during three child and three parent focus groups and 41 interviews. Interviews and focus groups were audio-recorded and transcribed verbatim for inductive thematic analysis. Children and parents identified home, peer and health environments as impacting on their children’s physical activity participation. Peer environments, such as school or daycare, were supportive by providing physical activity facilities and enabling fun with peers and time outdoors. At home, parent and sibling interactions both encouraged and discouraged physical activity. The children’s unique health environment fostered physical activity uncertainty, discouraging activity despite minimal or no physician recommendations to restrict physical activity. Children with complex congenital heart disease and their parents recognize the importance of physical activity and fun with friends. Physical activity uncertainty contributes to their inactive lifestyles despite minimal restrictions from health professionals. Positive clinical encouragement and health environment interventions that better support physical activity are required.