Workplace violence (WPV) by patients and visitors is a hazard in many emergency departments (ED), with serious consequences for both staff and patients. Patients with a migratory background seem to be prone to being involved in WPV. We therefore reviewed all reports of ED staff who experienced WPV over a 4-year period (2013–2016). We analyzed data on the reasons for the incident, the time of day, the manner of violence, the consequences, and the migratory background of the aggressor. In total, 83 cases of WPV were reported over a four-year period. The average age of the violent person was 33.1 years; in 35 cases (42.0%), aggressors were younger than 30 years old, 53 (63.8%) were male, 49 (59%) were of Swiss nationality, and 35–40% had a migratory background. The odds ratio of people originating from a low- to middle-income country versus those originating from a high-income country was 1.8. Furthermore, 45.8% of the patients arrived by ambulance (n = 38) and 19 patients (22.9%) were self-presenting. Most cases (92.8%) involved verbal aggression, but in more than half of the cases, physical assault (56.6%) was also reported. In addition, 43 (51.8%) of the events occurred during the night. Results also showed that 42 (50.6%) of patients who were involved in WPV were under the influence of alcohol and 29 (34.9%) suffered from psychiatric disorders. Security personnel and police were involved in 53 (63.9%) and 47 (56.6%) cases, respectively. Twenty patients (24.1%) were sedated and 16 (19.3%) were restrained. In 18 cases (21.7%), the psychiatrist ordered compulsory hospitalization in a psychiatric institution. Taken together, WPV is a relatively common event in our ED and persons with a migratory background are involved more often relative to their frequency of ED visits.