Latex, an aqueous dispersion of sub-micron polymer particles, is widely used as polymer binder in waterborne coatings and adhesives. Drying of a latex is inhomogeneous, during which the spatial distribution of particles is non-uniform and changes with time, usually resulting in a compromise of the integrity of a dried film. To study drying inhomogeneity of latex, we developed a system integrating optical coherence tomography (OCT) with gravimetric and video analysis (OCT-Gravimetry-Video method) to non-destructively monitor the drying process of non-film-forming latexes consisting of hard polystyrene spheres over time. OCT structural and speckle images of the latex’s internal structure show the packing process of particles, the detachment of latex and the formation of apparent shear bands in cross-sectional views. Video recordings show the formation of cracks and the propagation of the drying boundary in the horizontal direction. The drying curve, measured by gravimetry, shows the drying rate and the water content of the latex at each drying stage. Furthermore, we find that the particle size affects packing and cracking phenomena remarkably. The OCT-Gravimetry-Video method serves as a general and robust approach to investigate the drying process of waterborne latex system. This method can be employed for fundamental studies of colloids and for evaluations of industrial latex products.