Background Phishing is a major threat to the data and infrastructure of healthcare organizations and many cyberattacks utilize this socially engineered pathway. Phishing simulation is used to identify weaknesses and risks in the human defences of organizations. There are many factors influencing the difficulty of detecting a phishing email including fatigue and the nature of the deceptive message. Method A major Italian Hospital with over 6000 healthcare staff performed a phishing simulation as part of its annual training and risk assessment. Three campaigns were launched at approx. 4-month intervals, to compare staff reaction to a general phishing email and a customized one. Results The results show that customization of phishing emails makes them much more likely to be acted on. In the first campaign, 64% of staff did not open the general phish, significantly more than the 38% that did not open the custom phish. A significant difference was also found for the click rate, with significantly more staff clicking on the custom phish. However, the campaigns could not be run as intended, due to issues raised within the organization. Conclusions Phishing simulation is useful but not without its limitations. It requires contextual knowledge, skill and experience to ensure that it is effective. The exercise raised many issues within the Hospital. Successful, ethical phishing simulations require coordination across the organization, precise timing and lack of staff awareness. This can be complex to coordinate. Misleading messages containing false threats or promises can cause a backlash from staff and unions. The effectiveness of the message is dependent on the personalization of the message to current, local events. The lessons learned can be useful for other hospitals.