Systematic Fuzz Testing Techniques on a Nanosatellite Flight Software for Agile Mission Development

Abstract

The success of the CubeSats space missions depends on the ability of performing properly in a harsh environment. A key component in space missions is the flight software, which manages all of the processes executed by the satellite on its onboard computer. Literature shows that CubeSat missions suffer high infant mortality, and many spacecraft failures are related to flight software errors, some of them resulting in complete mission loss. Extensive operation testing is the primary technique used by CubeSats developers to ensure flight software quality and avoid such failures. The "New Space" requirements pressure to add "agility" to the software development, which could limit the capacity to test. While advanced and beneficial software testing techniques are found in the software engineering field, CubeSat software solutions mostly rely on unit testing, SILS and HILS. In this work fuzz testing techniques were developed, implemented and evaluated as a manner to expedite operational testing of CubeSats while maintaining their completeness. The impact of the tools were evaluated by using the three new 3U CubeSats under development at the University of Chile. We identified twelve bugs not covered by classic testing strategies, in less than three days. These failures were reported, fixed and characterized by the developers in eight sprint sessions. Our results indicate that fuzz testing improved the completeness of flight software testing through automation and with almost non development interruption. Although our approach have been tested on the SUCHAI flight software, it is applicable to systems that follow a similar architecture.