In this paper, a new population based hybrid meta-heuristic algorithm called the Hybrid Whale-Particle Swarm Optimization Algorithm (HWPSO) for solving complex optimization problems has been proposed. The proposed algorithm tries to overcome the limitations associated in a Particle Swarm Optimization (PSO) exploration phase i.e. Stagnation Effect, by hybridizing with a whale optimization algorithm (WOA) in a novel way as WOA has been reported to have very good exploration capability. During hybridization, two novel techniques have been employed, namely 'Forced' Whale and 'Capping' Phenomenon. The 'Forced' WOA is introduced in exploration phase and it enables the WOA to guide PSO for better local optima avoidance and concurrently a new 'Capping' phenomenon is employed which restricts the WOA search mechanism during exploitation phase, thereby converging the solution faster to a global optimum value. The performance of the proposed HWPSO has tested on 18 benchmark mathematical functions and 3 Electronics Design Optimization problems. The simulation results indicate that the proposed algorithm outperforms many state of the art algorithms by achieving a better optima with a very low standard deviation for most of the benchmark functions used. The effectiveness of the proposed HWPSO has been also validated by statistical as well as complexity analysis. The performance of HWPSO algorithm is also found to be satisfactory in all three cases of electronics design optimization problems, which have been further validated with standard design tools and found to be in close agreement with each other.