Digital technology and innovation have transformed healthcare systems in multiple dimensions. In the dawn of this era, AI is revolutionizing the tackling of healthcare challenges by detecting and predicting medical diagnoses. On one hand, care and assistive robots are getting ready to fill the shortages of medical staff and carers in developed nations. The need for such care support is more relevant in Western society, as the older generation prefers to live independently. Such a system can provide quality independent living. However, this system deals with sensitive data of the users, so it is critical to ensure that the engagement, interaction, and access of user data are securely controlled and the privacy of the users is protected. The first line of defense in protecting the user's sensitive information is to securely identify and authenticate the user so that access is monitored and controlled. In this paper, a seamless multifactor secure-by-design authentication system using face recognition and a smartwatch is developed so that a continuous form of authentication is maintained when the user is around the robot. This ensures the robotic system is not open for its services unless the authorized user is present.