Auditory processing disorder, defined here as a deficit in the way sounds are analyzed by the brain, has remained a controversial topic within audiology for decades. Some of the controversy concerns what it is called. More substantively, even its existence has been questioned. That view has likely emerged because there has not been sufficient rigor in determining when difficulty in understanding speech has been the consequence of some type of auditory processing deficit, versus when it is the consequence of a cognitive deficit or a language deficit. This article suggests that the field use the term “listening difficulty” as an umbrella term to indicate a reported deficit in recognizing sounds or understanding speech, one possible cause of which is an auditory processing disorder. Other possible causes are hearing deficits, cognitive deficits, and language deficits. This article uses a plausible, and hopefully noncontroversial, model of speech understanding that comprises auditory processing, speech processing, and language processing, all potentially affected by the degree of attention applied and the listener’s memory ability. In a fresh approach to the construction of test batteries, the stages of the model are linked to tests designed to assess either all or selected parts of the processes involved. For two of the stages, a listener’s performance is quantified as the additional signal to noise ratio that he or she needs to function equivalently to his or her age peers. Subtraction of the deficits revealed by each test enables the contributions of each processing stage to a listening deficit to be quantified. As a further novel contribution, the impact of memory and attention on each test score is quantitatively allowed for, by an amount that depends on each test’s dependence on memory and attention. Attention displayed during the test is estimated from the fluctuations in performance during the test. The article concludes with a summary of the research that must be conducted before the structured tests can be used to quantify the extent to which different potential causes of listening difficulties are responsible for real-life difficulties in an individual child.