Soft computing techniques are frequently used in modeling and control applications of nonlinear systems. However, the fuzzy cognitive map method, which is one of the soft computing techniques, is rarely used in control applications as a main controller. In this study, a fuzzy cognitive map based PD controller structure is introduced and used for the stabilization of an inverted pendulum system which is a nonlinear, unstable, and under-actuated system. The proposed controller has two inputs which are the error and the change of error. In the proposed PD controller structure, the crisp input values are fuzzified to be handled in a fuzzy cognitive map process. Then, causal relationships between fuzzified inputs and a control output are defined by using weight parameters. Finally, the crisp control output value which will be applied to the system is obtained by using an activation function. The types of membership functions used for the fuzzification process and the activation function determine the nonlinear characteristics of the proposed fuzzy cognitive map based PD controller. The proposed controller has three tuning parameters which are one output gain and two weight parameters. To show the effectiveness and robustness of the proposed fuzzy cognitive map based PD controller, simulation studies are performed on an inverted pendulum system. Additionally, the performance of the proposed controller is compared with a PD controller. All controller parameters are determined by using a genetic algorithm. Comparison results indicate that the proposed fuzzy cognitive map based PD controller shows better control performance than the classical PD controller.