Cutting force prediction is very important to optimize machining parameters and monitor machining state. In order to predict cutting force of sculptured surface machining with ball end mill accurately, tool posture, cutting edge, contact state between cutter, and workpiece are studied. Firstly, an instantaneous motion model of ball end mill for sculptured surface is established. The instantaneous milling coordinate system and instantaneous tool coordinate system are defined to describe the position and orientation of tool, and the transformation matrix between coordinate systems is derived. Secondly, by solving three boundaries around engagement of cutter and workpiece, a cutter-workpiece engagement model related to tool posture, milling parameters, and tool path is established. It has good adaptability to the variable tool axis relative to the machining surface. Finally, an algorithm of thickness about an instantaneous undeformed chip is researched, and a prediction model of cutting force is realized with microelement cutting theory. Also, the model is suitable for sculptured surface machining with arbitrary tool posture and feed direction. The accuracy of the proposed prediction model was verified by a series of experiments.