We consider a class of constrained stochastic optimal control problems with applications to an illiquid stock position build-up. Using a geometric Brownian motion model, we allow the drift to be purchase-rate dependent to characterize "price impact" of heavy share accumulation over time. The constraint is the fund availability. That is, the expected fund availability has an upper bound. We use a Lagrange multiplier method to treat the constrained control problem. Because a closed-form solution is virtually impossible to obtained, we develop approximation schemes, which consist of inner and outer approximations. The inner approximation is a numerical procedure for obtaining optimal strategies based on a fixed parameter of the Lagrange multiplier. The outer approximation is a stochastic approximation algorithm for obtaining the optimal Lagrange multiplier. Convergence analysis together with numerical examples are provided.