This study presents data on the use of shungite ore (the Bakyrchik deposit, Kazakhstan) and its concentrate as fillers in elastomer composites based on nitrile butadiene rubber. In addition to carbon, these shungite materials contain oxides of Si, Fe, K, Ca, Ti, Mn, and Al. The shungite concentrate was obtained through a flotation process involving five stages. The chemical composition analysis of these natural fillers revealed that during flotation, the carbon content increased 3.5 times (from 11.0 wt% to 39.0 wt%), while the silicon oxide content decreased threefold (from 49.4 wt% to 13.6 wt%). The contents of oxides of K, Ca, Ti, Mn, and Al decreased by less than 1%, and iron oxide content increased by 40% (from 6.7 wt% to 9.4 wt%). The study explored the impact of partial or full replacement of carbon black (CB) of P 324 grade with the shungite ore (ShO) and the shungite concentrate (ShC) on the vulcanization process and the physical–mechanical properties of the rubber. It was found that replacing CB with ShO and ShC reduces Mooney viscosity ML (1 + 4) 100 °C of the rubber compounds by up to 29% compared to the standard CB-filled sample. The use of the shungite fillers also increased scorch time (ts) by up to 36% and cure time (t90) by up to 35%. The carbon content in the shungite fillers had little influence on these parameters. Furthermore, it was demonstrated that replacing 5–10 wt% of CB with ShO or ShC improves the tensile strength of the rubber. The results of the flotation enrichment process enable the assessment of how these shungite fillers affect the properties of the composites for producing rubbers with specific characteristics. It was also found that substituting CB with ShO or ShC does not significantly affect the rubber’s resistance to standard oil-based media. The findings indicate that Kazakhstan’s shungite materials can be used as fillers in rubber to partially replace CB.