This paper focuses on the numerical investigation of a fractal modification of capillary oscillator by using a coupling technique based on the two-scale transformation and the global residue harmonic balance method. This fractal oscillator can be transformed as the classical capillary oscillator with the help of the two-scale transformation. We further obtain an approximated oscillator by using Taylor approximation. The approximations or frequencies are given by applying the global residue harmonic balance method without discretization. Numerical sensitive analysis of the approximations about different parameters is considered in detail. Compared results with Runge–Kutta method and homotopy perturbation method are given to illustrate the efficiency and stability of the present technology.