We propose an optimal portfolio problem in the incomplete market where the underlying assets depend on economic factors with delayed effects, such models can describe the short term forecasting and the interaction with time lag among different financial markets. The delay phenomenon can be recognized as the integral type and the pointwise type. The optimal strategy is identified through maximizing the power utility. Due to the delay leading to the non-Markovian structure, the conventional Hamilton-Jacobi-Bellman (HJB) approach is no longer applicable. By using the stochastic maximum principle, we argue that the optimal strategy can be characterized by the solutions of a decoupled quadratic forward-backward stochastic differential equations(QFBSDEs). The optimality is verified via the super-martingale argument. The existence and uniqueness of the solution to the QFBSDEs are established. In addition, if the market is complete, we also provide a martingale based method to solve our portfolio optimization problem, and investigate its connection with the proposed FBSDE approach. Finally, two particular cases are analyzed where the corresponding FBSDEs can be solved explicitly.