Extant levels of automation (LoAs) taxonomies describe variations in function allocations between the driver and the driving automation system (DAS) from a technical perspective. However, these taxonomies miss important human factors issues and when design decisions are based on them, the resulting interaction design leaves users confused. Therefore, the aim of this paper is to describe how users perceive different DASs by eliciting insights from an empirical driving study facilitating a Wizard-of-Oz approach, where 20 participants were interviewed after experiencing systems on two different LoAs under real driving conditions. The findings show that participants talked about the DAS by describing different relationships and dependencies between three different elements: the context (traffic conditions, road types), the vehicle (abilities, limitations, vehicle operations), and the driver (control, attentional demand, interaction with displays and controls, operation of vehicle), each with associated aspects that indicate what users identify as relevant when describing a vehicle with automated systems. Based on these findings, a conceptual model is proposed by which designers can differentiate LoAs from a human-centric perspective and that can aid in the development of design guidelines for driving automation.