Humans exploit the inherent biomechanical compliance in their fingers to achieve stability and dexterity during many manipulation tasks. The compliance is a result of muscles and tendons (series compliance) and flexible joints (parallel compliance). While the effects of series compliance have been studied in many robotic systems, research on the effects of joint compliance arranged in parallel with the actuators is limited. In this paper, we first demonstrate, through mathematical modeling, that introducing parallel compliance improves stability and robustness in the presence of time delay in a generic robotic joint. We also provide guidelines to balance the benefits of added stability with the increased actuator load when implementing parallel compliance in robotic joints. The result is experimentally validated on a one-degree-of-freedom tendon-driven joint. We build upon this result to design two 2-DOF tendon-driven fingers with parallel compliance, and develop an impedance control law for object manipulation with the fingers. The experimental results demonstrate, for the first time, the advantages of introducing parallel compliance in robotic hands during dexterous manipulation tasks, specifically in achieving smoother trajectory tracking, improved stability, and robustness to impacts.