Background Emergency department (ED) visits show a high volatility over time. Therefore, EDs are likely to be crowded at peak-volume moments. ED crowding is a widely reported problem with negative consequences for patients as well as staff. Previous studies on the predictive value of weather variables on ED visits show conflicting results. Also, no such studies were performed in the Netherlands. Therefore, we evaluated prediction models for the number of ED visits in our large the Netherlands teaching hospital based on calendar and weather variables as potential predictors. Methods Data on all ED visits from June 2016 until December 31, 2019, were extracted. The 2016–2018 data were used as training set, the 2019 data as test set. Weather data were extracted from three publicly available datasets from the Royal Netherlands Meteorological Institute. Weather observations in proximity of the hospital were used to predict the weather in the hospital’s catchment area by applying the inverse distance weighting interpolation method. The predictability of daily ED visits was examined by creating linear prediction models using stepwise selection; the mean absolute percentage error (MAPE) was used as measurement of fit. Results The number of daily ED visits shows a positive time trend and a large impact of calendar events (higher on Mondays and Fridays, lower on Saturdays and Sundays, higher at special times such as carnival, lower in holidays falling on Monday through Saturday, and summer vacation). The weather itself was a better predictor than weather volatility, but only showed a small effect; the calendar-only prediction model had very similar coefficients to the calendar+weather model for the days of the week, time trend, and special time periods (both MAPE’s were 8.7%). Conclusions Because of this similar performance, and the inaccuracy caused by weather forecasts, we decided the calendar-only model would be most useful in our hospital; it can probably be transferred for use in EDs of the same size and in a similar region. However, the variability in ED visits is considerable. Therefore, one should always anticipate potential unforeseen spikes and dips in ED visits that are not shown by the model.