The development of novel materials and structures for efficient second‐order nonlinear micro/nano devices remains a significant challenge. In this study, we demonstrate the remarkable enhancement of second‐harmonic generation (SHG) and even cascaded sum frequency generation (CSFG) in whispering gallery mode microspheres made of surface‐crystallized glass with a 6‐μm Ba2TiSi2O8 crystal layer. Attributed to the core‐shell design, the Ba2TiSi2O8 located on the surface can be efficiently coupled with whispering gallery modes, resulting in a highly efficient micron‐scale cavity‐enhanced second‐order optical nonlinearity. Greatly enhanced SHG of the microcavity was observed, which is up to 80 times stronger than that of non‐resonant bulk sample. Furthermore, owing to the wavelength non‐selectivity of random quasi‐phase matching, we demonstrated ultra‐wideband SHG with a strong response ranging from 860 nm to 1600 nm and high‐contrast polarization characteristics. The glass‐ceramic based microsphere cavity also boosts the cascading optical nonlinearity, manifested by a two‐magnitude enhancement of CSFG. This work delineates an efficient strategy for boosting nonlinear optical response in glass‐ceramics, which will open up new opportunities for applications in photonics and optical communications.This article is protected by copyright. All rights reserved