Dielectric spheres of various size may sustain electromagnetic whispering-gallery modes resonating at optical frequencies with very narrow linewidths. Arbitrary small deviations from the spherical shape typically shift and broaden such resonances. Our goal is to determine such shifted and broadened resonances. A boundary-condition perturbation theory for the acoustic vibrations of nearly circular membranes was developed by Lord Rayleigh more than a century ago. We extend this theory to describe the electromagnetic excitations of nearly spherical dielectric cavities. This approach permits to avoid dealing with decaying quasi-normal modes. We explicitly find the frequencies and the linewidths of the optical resonances for arbitrarily deformed nearly spherical dielectric cavities, as power series expansions by a small parameter, up to and including and including second order terms. We thoroughly discuss the physical conditions for the applicability of perturbation theory.