Unmanned aircraft are complex cyber physical systems. As such, software is a vital part of the aircraft. The midterm perspective of safe integration of unmanned aircraft in our airspace requires such a software‐driven system to be fail‐safe. Today, all classes of unmanned aircraft heavily rely on software, especially since successively increasing degrees of autonomy have already been shown for onboard decision‐making and automation of mission task elements. One important software component of an unmanned aircraft is its guidance software. Unlike an assistance system for supporting a human pilot on a manned aircraft, autonomous guidance software is supposed to allow for human independent but safe plan execution, in case the control link to the operator is interrupted. Thus, to ensure safety objectives are achieved without the conventional fall‐back layer of the human pilot in the loop, such cases of lost data and controlling links require special attention. First, this chapter will discuss fail‐safe systems implications for unmanned aircraft systems from a software requirements perspective. Second, we discuss software challenges and possibilities to achieve autonomy in flight from lessons learned with our flying rotorcraft research testbed. We analyze software in the loop tests mapped to an abstract set of orthogonal capabilities. Furthermore, we utilize formal requirement modeling and model checking techniques to complement traditional testing. As a result, we present a methodology and tools that are not constrained to the unmanned system at hand.