The reported study concerns the introduction of renewable raw materials into the formulation of rigid polyurethane foams in the quest for the sustainable development of polymer composites. In this study, rigid polyurethane foam composites were prepared using 75 wt.% of rapeseed oil-based polyol and 15 parts per hundred parts of polyol (php) of natural fillers such as chokeberry pomace, raspberry seeds, as well as hazelnut and walnut shells. The influence of the used raw materials on the foaming process, structure, and properties of foams was investigated using a FOAMAT analyzer and a wide selection of characterization techniques. The introduction of renewable raw materials limited reactivity of the system, which reduced maximum temperature of the foaming process. Moreover, foams prepared using renewable raw materials were characterized by a more regular cell structure, a higher share of closed cells, lower apparent density, lower compressive strength and glass transition temperature, low friability (<2%), low water absorption (<1%), high dimensional stability (< ± 0.5%) and increased thermal stability. The proper selection and preparation of the renewable raw materials and the rational development of the polyurethane recipe composition allow for the preparation of environmentally-friendly foam products with beneficial application properties considering the demands of the circular economy in the synthesis of rigid foams.