Smart home is one of the key applications of the Internet of Things (IoT), which allows users to control the smart devices in their houses through the Internet. However, a smart home system also faces severe challenges in terms of privacy and confidentiality when users are allowed to remotely access it. Despite the recent research efforts on authentication schemes to improve the security aspects of a smart home, there are still unsolved problems. On the one hand, most of the existing schemes focus on secure authentication and communication via a trusted third party without taking its privacy leakage into consideration. On the other hand, many protocols enable the users to directly authenticate themselves to a large number of smart devices in the smart home network, which is often inefficient and inconvenient. To cope with these issues, we propose a smart home system model based on Internet services, like if this then that (IFTTT), and design an anti-tracking mutual authentication scheme with a key agreement element in it. Specifically, our scheme introduces an IFTTT home gateway as the control commands' executor and the security guard to allow a user to remotely access a smart home system privately. The proposed scheme employs the elliptic curves' cryptography (ECC) algorithm, nonces, XOR, and cryptographical hash functions to achieve mutual authentication with security features, such as anonymity and perfect forward security. The security analysis and performance comparison results demonstrate that the proposed scheme achieves secure and private authentication.INDEX TERMS Smart home, user authentication, anti-tracking, key agreement, BAN-logic.