The techniques offormal specification and refinement of specifications into implementations are beginning to have an impact on the development of computer software. The ideal of software engineering is that programs may be proved correct with respect to their specifications. In principle, this is achievable for many applications. However, techniques for the specification and development ofAI applications are not so well defined at present. Part ofthe problem is that it is not always clear what should be specified. Motivated by an informal legal liability study, we present a requirements analysis of the aspects that should be covered by the formal specification. The discussion is focused on the specification ofa medical decision support system, but the arguments are generally valid. In particular, we will argue that it is especially important to take a more holistic approach and consider the specification of both the system and its environment.