Multi-Resident Activity Recognition in a Smart Home Using RGB Activity Image and DCNN

A Review of the Recent Developments in Integrating Machine Learning Models with Sensor Devices in the Smart Buildings Sector with a View to Attaining Enhanced Sensing, Energy Efficiency, and Optimal Building Management

“…The authors of these scientific articles make use of different types of sensors in their analyses. These include wearable sensors [18,74]; environmental sensors [73,74]; motion sensors [18,75]; a two-dimensional acoustic array [27]; a Wireless Sensor Network (WSN) [23] and sensor networks [76]; temperature sensors [1,63,73,77]; photosensors [1,63]; Passive Infra-Red Sensors (PIR) [73,75]; sensors for humidity and for evaluating the carbon dioxide concentration [1,77]; microphones [77]; cameras [18]; occupancy information sensors [1]; electricity meters [1,75]; accelerometers [5,63]; sensors of IoT devices [38]; an altimeter, a gyroscope and a barometer [63]; sensors mounted on different objects [75]; an unobtrusive sensing module [14]; and binary and ubiquitous sensors [29].…”

“…With respect to the reasons for implementing Neural Networks for classification integrated with sensor devices in smart buildings, these are mainly related to the recognition/classification of human activity in the papers [1,5,14,18,23,27,29,63,[73][74][75][76][77]. In some of these papers, human activity recognition has as a final purpose the detection and prediction of abnormal behavior [75], monitoring the activities of elderly who are living alone [14,63], classification of the gender of occupants in a building [5], and monitoring the activities of elderly who are living in smart homes care [18,77].…”

“…The authors of [1] made use of the Support Vector Machine technique and compared it with the Decision Tree and the Artificial Neural Networks techniques. In [73], a Deep Convolutional Neural Network (DCNN) approach was implemented, and this was compared with the Naïve Bayes (NB) and the Back-propagation (BP) algorithms. In [38], the authors made use of a Bayesian Network approach that was subsequently compared with the Decision Tree and Monolithic Bayesian Network methods.…”

“…The performance metrics considered in the scientific papers that use Neural Networks for classification purposes integrated with sensor devices in smart buildings include: Confusion Matrix [18,38,73]; F1 Score [18,73,74,76]; Accuracy [1,5,18,27,29,38,73,74,76]; Root Mean Square Error (RMSE) [23,75,77]; Precision [29,38,73,74,76]; Recall [29,38,73,74,76]; Standard Deviation (STD) [1,63]; Mean Absolute Percentage Error (MAPE) [23,77]; Mean Squared Error (MSE) [77]; Coefficient of Determination (R 2 ) and Mean Absolute Error (MAE) [23]; Specificity [14,73]; Sensitivity (SN), Area Under the Receiver Operating Characteristic Curve (AUC) [14]; and Maximum, Minimum, Median, Mode, Kurtosis, Skewness, Intensity, Difference, Root-Mean-Square (RMS), Energy, Entropy and Key Coefficient [63].…”

“…Passive Infrared (PIR) and temperature environmental sensors were used by Tan et al [73] with a view to recognizing and classifying, in an unobtrusive manner, the activity of multiple inhabitants within the same smart home. The authors proposed a method based on analyzing the sensor-acquired Red-Green-Blue (RGB) images by means of a Deep Convolutional Neural Network (DCNN), which was trained and tested using the Cairo open dataset.…”