In this paper, native cellulose I was subjected to alkaline treatment. As a result, cellulose I was transformed to cellulose II and some nanometric particles were formed. Both polymorphic forms of cellulose were modified with poly(ethylene glycol) (PEG) and then used as fillers for polyurethane. Composites were prepared in a one-step process. Cellulosic fillers were characterized in terms of their chemical (Fourier transformation infrared spectroscopy) and supermolecular structure (X-ray diffraction), as well as their particle size. Investigation of composite polyurethane included measurements of density, characteristic processing times of foam formation, compression strength, dimensional stability, water absorption, and thermal conductivity. Much focus was put on the application aspect of the produced insulation polyurethane foams. It was shown that modification of cellulosic filler with poly(ethylene glycol) has a positive influence on formation of polyurethane composites—if modified filler was used, the values of compression strength and density increased, while water sorption and thermal conductivity decreased. Moreover, it was proven that the introduction of cellulosic fillers into the polyurethane matrix does not deteriorate the strength or thermal properties of the foams, and that composites with such fillers have good application potential.