In situ TEM straining allows probing deformation mechanisms of ultrafine-grained and nanocrystalline metals. While obtaining statistically meaningful information about microstructural changes using conventional bright-field/dark-field imaging or diffraction is time consuming, automated crystal orientation mapping in TEM (ACOM-TEM) enables tracking orientation changes of hundreds of grains simultaneously. We use this technique to uncover extensive grain rotations during in situ tensile deformation of a freestanding, ultrafine-grained aluminum film (thickness 200 nm, mean grain size 180 nm). During loading, both the fraction of grains that undergo rotations and the magnitude of their rotations increase with strain. The rotations are partially or fully reversible in a significant fraction of grains during unloading, leading to notable inelastic strain recovery. More surprisingly, the direction of rotation remains unchanged for a small fraction of grains during unloading, despite a sharp reduction in the applied stress. The ACOM-TEM measurements also provide evidence of reversible and irreversible grain/twin boundary migrations in the film. These microstructural observations point to a highly inhomogeneous and constantly evolving stress distribution in the film during both loading and unloading.