Monolayer semiconductors are atomically thin quantum wells with strong confinement of electrons in two-dimensional (2D) plane. Here, we experimentally study the out-of-plane polarizability of excitons in hBN-encapsulated monolayer WSe2 in strong electric fields of up to 1.6 V/nm (16 MV/cm). We monitor free and bound exciton photoluminescence peaks with increasing electric fields at a constant carrier density, carefully compensating for unintentional photodoping in our double-gated device at 4K. We show that the Stark shift is < 0.4 meV despite the large electric fields applied, yielding an upper limit of polarizability αz to be ~ 10 -11 Dm/V. Such a small polarizability, which is nearly two orders of magnitude smaller than the previously reported value for MoS2, indicates strong atomic confinement of electrons in this 2D system and highlights the unusual robustness of free excitons against surface potential fluctuations.