In this paper, we consider the infinite time horizon LQR optimal control problem for the linearized Boussinesq system. The goal is to justify the approximation by penalization of the free divergence condition in this context. We establish convergence results for optimal controls, optimal solutions and Riccati operators when the penalization parameter goes to zero. These results are obtained under two different assumptions. The first one treats the linearization arround a sufficiently small stationary state and an arbitrary control operator (possibly of finite rank), while the second one does no longer require the smallness of the stationary state but requires to consider controls distributed in a subdomain and depending on the space variable.