The drying process of bitumen emulsion largely dominates the strength development of emulsion-based mixtures for pavement structure, thus it can be used to judge the quality of bitumen emulsion. However, the drying behaviour of bitumen emulsions was seldom considered. The emulsion drying and film formation theory are employed to study the drying process of different bitumen emulsions with a thin layer. Results indicated the drying process of bitumen emulsion can be divided into three stages: (a) an initial high evaporation rate stage; (b) an intermediate stage with a rapidly decreasing evaporation rate; (c) a final stage with a very small evaporation rate. The boundaries among the three stages can be identified by studying the water evaporation rate. Three drying parameters, i.e., the critical volume fractions of bitumen defining the boundaries among the three stages and the maximum packing fraction of bitumen droplets, are proposed to quantitatively characterize the drying behavior of bitumen emulsion. High values of these parameters indicate a bitumen emulsion that has rapid drying behavior. Therefore, these parameters are independent of the emulsifier type, but they are highly dependent on the bitumen’s droplet size. These drying parameters increase with a decrease in bitumen droplet size. Therefore, bitumen emulsion with a smaller size distribution of bitumen droplets can have a more rapid drying behavior, which is recommended in real engineering.