The formation mechanism of extreme waves in the coastal areas is still an open contemporary problem in fluid mechanics and ocean engineering. Previous studies have shown that the transition of water depth from a deeper to a shallower zone increases the occurrence probability of large waves. Indeed, more efforts are required to improve the understanding of extreme wave statistics variations in such conditions. To achieve this goal, large scale experiments of unidirectional irregular waves propagating over a variable bottom profile considering different transition water depths were performed. The validation of two highly nonlinear numerical models was performed for one representative case. The collected data were examined and interpreted by using spectral or bispectral analysis as well as statistical analysis. The higher probability of occurrence of large waves was confirmed by the statistical distributions built from the measured free surface elevation time series as well as by the local maximum values of skewness and kurtosis around the end of the slope. Strong second-order nonlinear effects were highlighted as waves propagate into the shallower region. A significant amount of wave energy was transmitted to low-frequency modes. Based on the experimental data, we conclude that the formation of extreme waves is mainly related to the second-order effect, which is also responsible for the generation of long waves. It is shown that higher-order nonlinearities are negligible in these sets of experiments. Several existing models for wave height distributions were compared and analysed. It appears that the generalised Boccotti’s distribution can predict the exceedance of large wave heights with good confidence.