The emergence of the E-health system has brought convenience to many chronically ill patients and elderly people with limited mobility. With the help of the E-health system, patients can upload their physiological data timely and get a diagnosis at home, which is more convenient and efficient as they do not have to line up in hospitals. In order to ensure this convenience while protecting patients’ privacy, many schemes have been proposed which can help patient and medical server authenticate each other. However, considering these patients’ inconvenience, sometimes family members need to participate in the patient’s treatment process. So, the E-health system needs to provide a secure communication platform for the family members. At present, most of the authentication schemes for the E-health system only focus on the secure communication between the patient and the medical server, while ignoring the participation of family members. Moreover, in the E-health system, the permissions of family members and patient should be different, and the medical server needs to distinguish their permissions efficiently. In order to overcome these problems, we propose a patient family binding and authentication privacy protection scheme for the E-health system. In the scheme proposed by us, the medical server can efficiently assign different permissions to the family member and patient. And our scheme can allow patient to authorize their family members freely, and the increase in the number of family members will not impose additional burden on the server. At the same time, the authentication between the family member and the medical server does not require the participation of the patient. In addition, by comparing with other related schemes, we prove that our scheme has suitable efficiency and security performance in the E-health system.