Handover has major implications for patient care. The handover process between ambulance and emergency department (ED) staff has been sparsely investigated. The purpose of this paper is, based on a literature review, to identify and elaborate on the major factors influencing the ambulance to ED handover, and to bring suggestions on how to optimize this process. A literature search on handovers to EDs was performed in PubMed, Embase, Web of Science and Cochrane databases. A total of 18 papers were included. Issues regarding transfer of information are highlighted. Newer studies suggest that implementing a structured handover format holds the possibilities for improving the process. Electronic equipment could play a part in reducing problems. Cultural and organizational factors impact the process in different ways. The professions perceive the value and quality of information given differently. Giving and taking over responsibility is an important issue. The handover of patients to the ED has the potential to be improved. Cultural issues and a lack of professional recognition of handover importance need to be approached. Multidisciplinary training in combination with a structured tool may have a potential for changing the culture and improving handover.