This study introduces an effective population-based optimization algorithm, namely the Golden Search Optimization Algorithm (GSO), for numerical function optimization. The new algorithm has a simple but effective strategy for solving complex problems. GSO starts with random possible solutions called objects, which interact with each other based on a simple mathematical model to reach the global optimum. To provide a fine balance between the explorative and exploitative behavior of a search, the proposed method utilizes a transfer operator in the adaptive step size adjustment scheme. The proposed algorithm is benchmarked with 23 unimodal, multimodal, and fixed dimensional functions and the results are verified by a comparative study with the well-known Gravitational Search Algorithm (GSA), Sine-Cosine Algorithm (SCA), Grey Wolf Optimization (GWO), and Tunicate Swarm Algorithm (TSA). In addition, the nonparametric Wilcoxon's rank sum test is performed to measure the pair-wise statistical performance of the GSO and provide a valid judgment about the performance of the algorithm. The simulation results demonstrate that GSO is superior and could generate better optimal solutions when compared with other competitive algorithms.