Advances in wireless sensor networks have enabled the monitoring of daily activities of elderly people. The goal of these monitoring applications is to learn normal behavior in terms of daily activities and look for any deviation, i.e., anomalies, so that alerts can be sent to relatives or caregivers. However, human behavior is very complex, and many existing anomaly detection systems are too simplistic which cause many false alarms, resulting in unreliable systems. We present Holmes, a comprehensive anomaly detection system for daily in-home activities. Holmes accurately learns a resident's normal behavior by considering variability in daily activities based not only on a per day basis, but also considering specific days of the week, different time periods such as per week and per month, and collective, temporal, and correlation based features. This approach of learning complicated normal behaviors reduces false alarms. Also, based on resident and expert feedback, Holmes learns semantic rules that explain specific variations of activities in specific scenarios to further reduce false alarms. We evaluate Holmes using data collected from our own deployed system, public data sets, and data collected by a senior safety system provider company from an elderly resident's home. Our evaluation shows that compared to state of the art systems, Holmes reduces false positives and false negatives by at least 46% and 27%, respectively.