Anchor piles are wildly adopted in mooring systems. However, there are still challenges in predicting the failure mode and pullout capacity of the pile. Previous Finite Element (FE) analysis was all based on the traditional small strain FE analysis. The whole pullout process of the pile cannot be simulated. Hence, the failure mode of the pile is hard to be understood, especially under small loading angles to the horizontal. In addition, theoretical analysis received limited attention in analyzing the failure mode and pullout capacity of anchor piles under inclined loading. In the present work, both Large Deformation Finite Element (LDFE) and theoretical analyses are performed to investigate the failure mode and pullout capacity of anchor piles. The effectiveness of the LDFE analysis is at first verified by model test data. Then, a theoretical method is proposed to predict the Optimal Loading Point (OLP), failure mode and pullout capacity of the pile under inclined loading. Comparative study is also performed between LDFE and theoretical analyses. It is concluded: (1) the proposed theoretical method is effective in predicting the OLP, failure mode and pullout capacity of anchor piles; (2) the OLP is not affected by the length-to-diameter ratio and the failure direction of β>5°, while very sensitive to the soil strength and the loading angle; (3) there is a critical loading angle, below which the pile will be pulled out vertically; and (4) the lateral capacity at the OLP could be more than twice that at the top of the pile.