Interfacial perpendicular magnetic anisotropy, which is characterized by the firstorder (K1) and second-order (K2) anisotropies, is the core phenomenon for nonvolatile magnetic devices. A sizable K2 satisfying a specific condition stabilizes the easy-cone state, where equilibrium magnetization forms at an angle from the film normal. The easy-cone state offers intriguing possibilities for advanced spintronic devices and unique spin textures, such as spin superfluids and easy-cone domain walls. Experimental realization of the easy-cone state requires understanding the origin of K2, thereby enhancing K2. However, previously proposed origins of K2 cannot fully account for experimental results. Here we show experimentally that K2 scales almost linearly with the work-function difference between the Co and X layers in Pt/Co/X heterostructures (X = Pd, Cu, Pt, Mo, Ru, W, and Ta), suggesting the central role of the inversion asymmetry in K2. Our result provides a guideline for enhancing K2 and realizing magnetic applications based on the easy-cone state.