Understanding the kinetic pathways of self-assembly in block copolymers (BCPs) has been a long-standing challenge, mostly due to limitations of in situ monitoring techniques. Here, we demonstrate an approach that uses optical birefringence, determined by spectroscopic ellipsometry (SE), as a measure of domain formation in cylinder- and lamellae-forming BCP films. The rapid experimental acquisition time in SE (ca. 1 sec) enables monitoring of the assembly/disassembly kinetics of BCP films during solvent-vapor annealing (SVA). We demonstrate that upon SVA, BCP films form ordered domains that are stable in the swollen state, but disorder upon rapid drying. Surprisingly, the disassembly during drying strongly depends on the duration of solvent exposure in the swollen state, explaining previous observations of loss of order in SVA processes. SE thus allows for decoupling of BCP self-assembly and disordering that occurs during solvent annealing and solvent evaporation, which is difficult to probe using other, slower techniques.