Seismic surface wave dispersion inversion is used widely to study the subsurface structure of the Earth. The dispersion property is usually measured by using frequency-phase velocity (f-c) analysis and by picking phase velocities from the obtained f-c spectrum. However, because of potential contamination the f-c spectrum often has multimodalities at each frequency for each mode. These introduce uncertainty and errors in the picked phase velocities, and consequently the obtained shear velocity structure is biased. To overcome this issue, in this study we introduce a new method which directly uses the spectrum as data. We achieve this by solving the inverse problem in a Bayesian framework and define a new likelihood function, the energy likelihood function, which uses the spectrum energy to define data fit. We apply the new method to a land dataset recorded by a dense receiver array, and compare the results to those obtained using the traditional method. The results show that the new method produces more accurate results since they better match independent data from refraction tomography. This real-data application also shows that it can be applied efficiently since it removes the need to pick phase velocities, and with relatively little adjustment to current practice since it uses standard f-c panels to define the likelihood. We therefore recommend using the energy likelihood function rather than explicitly picking phase velocities in surface wave dispersion inversion.