Fibrillary cellulose and nanocellulose are widely applied as fillers for the improvement of the mechanical properties, thermostability, and biodegradability of polymers. Here, an opposite concept is suggested which is to use cellulose for structure‐forming and directing the scaffold, whereas polymers serve for its modification. Cellulose possesses excellent properties, but they are badly affected by water; its mechanical strength drops sharply, cellulosic materials swell, become dimensionally unstable, and susceptible to fungal degradation. The deterioration of properties gives no way of using methods of wet chemistry for cellulose treatment. To obviate this problem, dimensionally stable aerogels are prepared from microfibrillated cellulose. Their stability allows them to be treated in solutions of isocyanate. Polyurethane synthesized in situ is attached covalently to cellulose through hydroxyl groups. Polymer coating further stabilizes the 3D microfibrillar network, Young's modulus increases by more than two orders of magnitude and differences in the properties of aerogels in the dried and wetted states are minimized. Improvement of the mechanical strength makes cellulose aerogels better suitable for various applications than present cellulosic materials.