This study proposes a computer cryptographic system through performing the chaotic modulation on the intrinsic mode functions with a non-dyadic number of the encrypted signals. First, the empirical mode decomposition is applied to an input signal to generate a set of intrinsic mode functions. Then, these intrinsic mode functions are categorised into two groups of signals. Next, a type 1 polyphase is employed to represent each group of signals. These polyphase components are combined to generate a non-dyadic number of polyphase components. Second, the chaotic modulation is applied to these combined polyphase components for performing the encryption in the time frequency domain. To reconstruct the original signal, first, the chaotic demodulation is applied to the encrypt components to reconstruct the combined polyphase components. Then, the original groups of intrinsic mode functions are reconstructed through the type 2 polyphase representation and the original signal is reconstructed. Compared with the chaotic filter bank system, the proposed approach enjoys the nonlinear and adaptive property of the empirical mode decomposition. Therefore, a better security performance can be achieved particularly for the non-stationary signals. Compared with the conventional chaotic modulation approach, the proposed system allows performing the cryptography in the time frequency domain.