Designing onshore wind farms presents unique challenges related to interactions between terrain and landscape characteristics. This research focuses on optimizing the layout of onshore wind farms while considering the effect of the terrain and land characteristics. Three real onshore site areas in South Sulawesi, Indonesia, are selected for wind farm design and optimization. A novel optimization algorithm, the Multiobjective Modified Electric Charged Particles Optimization (MOMECPO), is introduced to minimize both the Levelized Cost of Electricity (LCOE) and noise levels. This algorithm employs a new game theory-based indexing method to effectively sort the Pareto solution set. The results show that the proposed algorithm enhances the exploration and exploitation capabilities of the solutions obtained. Our optimal solutions demonstrate that MOECPO achieves LCOE values of 6.78, 7.73, and 5.56 US cents/kWh for Sites 1, 2, and 3, respectively. Correspondingly, noise levels are recorded at 53.71 dBA, 52.53 dBA, and 55.25 dBA for the same sites. These values outperform seven other comparative algorithms, with NSGA achieving the closest performance among them, yielding LCOE values of 6.865, 7.815, and 5.579 US cents/kWh, and noise levels of 53.858, 52.556, and 55.197 dBA for Sites 1, 2, and 3, respectively. As for the terrain effect, our findings reveal that sites with complex terrains tend to have higher AEP and lower LCOE due to the steeper slopes. However, this site also experiences increased noise levels because of the higher energy production.