Classically, vertical reference frames were realized as national or continent-wide networks of geopotential differences derived from geodetic leveling, i.e., from the combination of spirit leveling and gravimetry. Those networks are affected by systematic errors in leveling, leading to tilts in the order of decimeter to meter in larger networks. Today, there opens the possibility to establish a worldwide unified vertical reference frame based on a conventional (quasi)geoid model. Such a frame would be accessible through GNSS measurements, i.e., physical heights would be derived by the method of GNSS-leveling. The question arises, whether existing geodetic leveling data are abolished completely for the realization of vertical reference frames, are used for validation purposes only, or whether existing or future geodetic leveling data can still be of use for the realization of vertical reference frames. The question is mainly driven by the high quality of leveled potential differences over short distances. In the following we investigate two approaches for the combination of geopotential numbers from GNSS-leveling and potential differences from geodetic leveling. In the first approach, both data sets are combined in a common network adjustment leading to potential values at the benchmarks of the leveling network. In the second approach, potential differences from geodetic leveling are used as observable for regional gravity field modeling. This leads to a grid of geoid heights based on classical observables like gravity anomalies and now also on leveled potential differences. Based on synthetic data and a realistic stochastic model, we show that incorporating leveled potential differences improves the quality of a continent-wide network of GNSS-heights (approach 1) by about 40% and that formal and empirical errors of a regional geoid model (approach 2) are reduced by about 20% at leveling benchmarks. While these numbers strongly depend on the chosen stochastic model, the results show the benefit of using leveled potential differences for the realization of a modern geoid-based reference frame. Independent of the specific numbers of the improvement, an additional benefit is the consistency (within the error bounds of each observation type) of leveling data with vertical coordinates from GNSS and a conventional geoid model. Even though we focus on geodetic leveling, the methods proposed are independent of the specific technique used to observe potential (or equivalently height) differences and can thus be applied also to other techniques like chronometric or hydrodynamic leveling.