The results of brain connectivity analysis using reconstructed source time courses derived from EEG and MEG data depend on a number of algorithmic choices. While previous studies have investigated the influence of the choice of source estimation method or connectivity measure, the effects of the head modeling errors or simplifications have not been studied sufficiently. In the present simulation study, we investigated the influence of particular properties of the head model on the reconstructed source time courses as well as on source connectivity analysis in EEG and MEG. Therefore, we constructed a realistic head model and applied the finite element method to solve the EEG and MEG forward problems. We considered the distinction between white and gray matter, the distinction between compact and spongy bone, the inclusion of a cerebrospinal fluid (CSF) compartment, and the reduction to a simple 3-layer model comprising only the skin, skull, and brain. Source time courses were reconstructed using a beamforming approach and the source connectivity was estimated by the imaginary coherence (ICoh) and the generalized partial directed coherence (GPDC). Our results show that in both EEG and MEG, neglecting the white and gray matter distinction or the CSF causes considerable errors in reconstructed source time courses and connectivity analysis, while the distinction between spongy and compact bone is just of minor relevance, provided that an adequate skull conductivity value is used. Large inverse and connectivity errors are found in the same regions that show large topography errors in the forward solution. Moreover, we demonstrate that the very conservative ICoh is relatively safe from the crosstalk effects caused by imperfect head models, as opposed to the GPDC.