Through sensor networks, agriculture can be connected to the IoT, which allows us to create connections among agronomists, farmers, and crops regardless of their geographical differences. Faulty sensor detection is critical in IoT. When a sensor becomes faulty, missing data and/or bad data is provided to the control and management systems, which may lead to potential malfunction or even system failures. Because of this, a sensor fault detection mechanism must be implemented in an IoT system to eliminate this potential fault. This paper focuses on the implementation of a faulty sensor detection mechanism using data correlation among multivariate sensor readings, which is called Multivariate Faulty Sensor Detection Mechanism (Multi-FSDM) in a smart agriculture system. The smart agriculture system is attached with multi-variate sensors, which are moisture, temperature, and water sensor. These sensors are connected to Arduino UNO, which is equipped with an ESP8266 Wi-Fi module for internet connectivity. ThingsBoard is selected as the IoT cloud platform. The sensor readings are collected periodically and send to the cloud via the internet. Multi-FSDM calculates the correlation between each sensor reading to determine the health condition of each sensor. When all sensors are in good condition, all sensor readings are correlated with each other. However, when any sensor becomes faulty, sensor readings become uncorrelated. Once uncorrelated sensor readings occur, this means a faulty sensor is detected. Based on the findings, it is proven that Multi-FSDM can detect each sensor state on the smart agriculture system either in a good or faulty condition. When a sensor becomes faulty, Multi-FSDM detects and determines the faulty sensor successfully.