The transient dimensional changes during hygro-expansion and hydro-expansion of freely and restrained dried, softwood and hardwood sheets and fibers is monitored, to unravel the governing micro-mechanisms occurring during gradual water saturation. The response of individual fibers is measured using a full-field global digital height correlation method, which has been extended to monitor the transient hydro-expansion of fibers from dry to fully saturated. The hygro- and hydro-expansion is larger for freely versus restrained dried and softwood versus hardwood handsheets. The transient sheet-scale hydro-expansion reveals a sudden strain and moisture content step. It is postulated that the driving mechanism is the moisture-induced softening of the so-called ”dislocated regions” in the fiber’s cellulose micro-fibrils, unlocking further fiber swelling. The strain step is negligible for restrained dried handsheets, which is attributed to the ”dislocated cellulose regions” being locked in their stretched configuration during restrained drying, which is supported by the single fiber hydro-expansion measurements. Finally, an inter-fiber bond model is exploited and adapted to predict the sheet-scale hygro-expansion from the fiber level characteristics. The model correctly predicts the qualitative differences between freely versus restrained dried and softwood versus hardwood handsheets, yet, its simplified geometry does not allow for more quantitative predictions of the sheet-scale hydro-expansion.