With the availability of various publicly available and personal sensors, recording and profiling of activities of daily living (ADL) is becoming a reality. The sensors are omnipresent -in smartphones, smartwatches, and smartglasses and even in the environment around us in the form of peer smartphones or even infrastructure sensors such as bluetooth low energy beacons. However, there are various challenges pertaining to the sensor data processing, which makes creation of activities of daily life tracker challenging. In this work, we discuss about some of these challenges. We also discuss about some ADL tracking systems that we have developed and how we have addressed some of the challenges in building these systems. We further discuss about how various ADL trackers can be combined into a framework which can allow individuals to select a custom set of ADLs for self-tracking.