Phishing emails constitute a major public health problem, linked to negative health outcomes due to fraud and exploitation. Because of their sheer volume and because phishing emails are designed to deceive, purely technological solutions such as filters only go so far, leaving human judgment as the last line of defense against phishing. However, in part because it is difficult to phish people under controlled laboratory conditions, little is known about the cognitive and neural mechanisms underlying phishing susceptibility. There is therefore a critical need to develop an ecologically valid measure of phishing susceptibility that can be used in the lab to test cognitive models of phishing detection. In this work, we present such a task: PEST, the Phishing Email Suspicion Test, in which participants rate a series of phishing and non-phishing emails according to their level of suspicion. By comparing suspicion scores for each email to its real-world efficacy (assessed in a field experiment in an independent group of participants), we find support for the ecological validity of PEST in that phishing emails that were more effective in the real world were more effective at deceiving people in the lab. By modeling behavior in PEST, we find evidence that the suspicion level of emails is assessed using a comparison process in which the current email is compared with previously encountered emails to determine its suspicion level. Together our task and model provide a framework for studying the cognitive neuroscience of email phishing detection in the lab.