In recent years, many researchers have aimed to construct robotic soft grippers that can handle fragile or unusually shaped objects without causing damage. This study proposes a smart textile-composite actuator and its application to a soft robotic gripper. An active fiber and an inactive fiber are combined together using knitting techniques to manufacture a textile actuator. The active fiber is a shape memory alloy (SMA) that is wire-wrapped with conventional fibers, and the inactive fiber is a knitting yarn. A knitted textile structure is flexible, with an excellent structure retention ability and high compliance, which is suitable for developing soft grippers. A driving source of the actuator is the SMA wire, which deforms under heating due to the shape memory effect. Through experiments, the course-to-wale ratio, the number of bundling SMA wires, and the driving current value needed to achieve the maximum deformation of the actuator were investigated. Three actuators were stitched together to make up each finger of the gripper, and layer placement research was completed to find the fingers’ suitable bending angle for object grasping. Finally, the gripping performance was evaluated through a test of grasping various object shapes, which demonstrated that the gripper could successfully lift flat/spherical/uniquely shaped objects.