Composite insulation materials are an inseparable part of numerous electrical devices because of synergy effect between their individual parts. One of the main aims of the presented study is an introduction of the dielectric properties of nanoscale magnesium oxide powder via Broadband Dielectric Spectroscopy (BDS). These unique results present the behavior of relative permittivity and loss factor in frequency and temperature range. Following the current trends in the application of inorganic nanofillers, this article is complemented by the study of dielectric properties (dielectric strength, volume resistivity, dissipation factor and relative permittivity) of epoxy-based composites depending on the filler amount (0, 0.5, 0.75, 1 and 1.25 weight percent). These parameters are the most important for the design and development of the insulation systems. The X-ray diffraction patterns are presented for pure resin and resin with optimal filler amount (1 wt %), which was estimated according to measurement results. Magnesium oxide nanoparticles were also treated by addition of silane coupling agent (γ-Glycidoxypropyltrimethoxysilane), in the case of optimal filler loading (1 wt %) as well. Besides previously mentioned parameters, the effects of surface functionalization have been observed by two unique measurement and evaluation techniques which have never been used for this evaluation, i.e., reduced resorption curves (RRCs) and voltage response method (VR). These methods (developed in our departments), extend the possibilities of measurement of composite dielectric responses related to DC voltage application, allow the facile comparability of different materials and could be used for dispersion level evaluation. This fact has been confirmed by X-ray diffraction analyses.