The relationships between the atomic arrangements, electronic structures, and energetics of three sesquioxides, As 2 O 3 , Sb 2 O 3 , and Bi 2 O 3 , are systematically investigated by first-principles lattice-dynamics calculations. Dynamically stable structures of the defective fluorite family are searched for by including atomic displacements along imaginary modes of lattice vibrations that appear in high-symmetry structures. Experimentally reported crystal structures (arsenolite-As 2 O 3 , α-Sb 2 O 3 , β-Sb 2 O 3 , and β-Bi 2 O 3) are found to be formed by symmetry breaking and local atomic distortion in this way. Moreover, some dynamically stable structures that have not yet been revealed by experiments are discovered. Their electronic structures and the magnitudes of their band gaps are obtained. In the dynamically stable low-symmetry structures, valence electrons localize and form an asymmetric charge distribution along the direction of an empty anion site of the defective fluorite structure. This is a common characteristic of a "lone pair" of cations. The formation of the lone pair is less significant in the order As 2 O 3 , Sb 2 O 3 , Bi 2 O 3 , which is the same trend as the lattice distortion. The formation of the lone pair plays an essential role in determining the structures, stability, and properties of these sesquioxides.