Objective Emergency medical services (EMS) personnel frequently use the Glasgow Coma Scale (GCS) to assess injured and critically ill patients. This study assessed the accuracy of EMS providers’ GCS scoring as well as the improvement in GCS assessment with the use of a scoring aid. Methods This randomized, controlled study was conducted in the emergency department (ED) of an urban, academic trauma center. Emergency medical technicians or paramedics who transported a patient to the ED were randomly assessed one of nine written scenarios, either with or without a GCS scoring aid. Scenarios were created by consensus of expert attending emergency medicine, EMS, and neurocritical care physicians with universal consensus agreement on GCS scores. Chi-square and student’s t-tests were used to compare groups. Results Of 180 participants, 178 completed the study. Overall, 73/178 (41%) participants gave a GCS score that matched the expert consensus score. GCS was correct in 22/88 (25%) of cases without the scoring aid. GCS was correct in 51/90 (57%) of cases with the scoring aid. Most (69%) of total GCS scores fell within one point of the expert consensus GCS score. Differences in accuracy were most pronounced in scenarios with a correct GCS of 12 or below. Sub-component accuracy was: eye 62%, verbal 70%, and motor 51%. Conclusion In this study, 60% of EMS participants provided inaccurate GCS estimates. Use of a GCS scoring aid improved accuracy of EMS GCS assessments.