Involving children in the design process of interactive technology can greatly enhance its likelihood of successful adoption. However, children's input and ideas require careful interpretation to reach viable designs and technical specifications, which poses a significant challenge to an adult design research team. In this paper we discuss our approach to managing the complexity of combining concepts and ideas that were generated through participatory design work with the practical, technical, ethical and theoretical constraints of developing a technologically enhanced learning environment for children with and without Autism Spectrum Conditions. We found that the nature of this design problem did not lend itself to be rationally reduced to produce a single solution, but required an understanding of interpretive and speculative approaches for us to be able to cope with the complexity of requirements. We describe a workshop in which members of the design team used such approaches to develop a design brief that is faithful to the children's input. By making this process transparent, we aim to contribute to the methodology of using such designerly approaches in combination with participatory and human-centred methods to develop interactive technology.