A novel data analysis technique based on the "empirical mode decomposition" and Hilbert transform is applied to the analysis of edge fluctuation data and Mirnov coil data from the ADITYA tokamak ͓S. B. Bhatt et al., Ind. J. Pure Appl. Phys. 27, 710 ͑1989͔͒. It is shown that the edge fluctuations can be well represented by a finite number of about 10 discrete modes. Their instantaneous energies show intermittent bursts and the high frequency modes are nonstationary. The technique is further developed to study three-mode interactions and employed to show that triplet interactions are statistically significant among high frequency modes of the fluctuation data. A general critique of the method is presented and its utility for the analysis of nonlinear phenomena in complex systems like plasma turbulence is highlighted.