Electroconductive cement-based composites are modern materials that are commonly used in many industries such as the construction industry, among others. For example, these materials can be used as sensors for monitoring changes in construction, grounding suspension, and resistance heating materials, etc. The aim of the research presented in this article is to monitor the impact of carbon particle character on cement-based electroconductive composites. Four types of graphite were analyzed. Natural and synthetic types of graphite, with different particle sizes and one with improved electrically conductive properties, were tested. For the analysis of the electrical conductivity of powder raw materials, a new methodology was developed based on the experience of working with these materials. Various types of graphite were tested in pure cement paste (80% cement, 20% graphite) as well as in a composite matrix, which consisted of cement (16.8%), a mixture of silica sand 0–4 mm (56.4%), graphite filler (20.0%) ground limestone (6.7%) and super plasticizers (0.1%). The resistivity and physical-mechanical properties of the composite material were determined. Furthermore, the resistivity of the test samples was measured with a gradual decrease in saturation. It may be concluded that graphite fillers featuring very fine particles and high specific surface are most suitable and most effective for creating electrically conductive silicate composites. The amount, shape and, in particular, the fineness of the graphite filler particles thus creates suitable conditions for the creation of an integrated internal electricity-conductive network. In the case of the use of a coarse type of graphite or purely non-conductive fillers, the presence of an electrolyte, for example, in the form of water, is necessary to achieve a low resistivity. Samples with fine types of graphite fillers achieved stable resistivity values when the sample humidity changed. The addition of graphite fillers caused a large decrease in the strength of the samples.