There are diverse sensor applications built into different personal devices, which have the ability to record data related to various aspects of the user. With the ever increasing popularity and lowering costs of such personal devices such as Smart Phones, collecting data from the mobile sensors available in these devices becomes feasible. A wealth of information can be gleaned from such data collected from these sensors which reveals various aspects of the individual's behaviour and activity. Existing approaches for analyzing such data mainly focuses on inferring semantic context and detecting associations from such data. For example, GPS enabled devices allow users to record their movements in the form of spatio-temporal stream points, and meaningful information can be extracted based on different research objectives.In this paper, we have investigated a computation framework in order to identify users' activity categories and their event's associations from GPS trajectory data. This framework has several progressive stages and is designed based on different approaches in each stage, which will facilitate to analyse people's everyday lifestyles that are related to outdoor behaviours. Moreover, we have proposed an approach to improve the performance of the semantic annotation process of this framework, by combining different sources of mobile sensor data (i.e. GPS and audio data). The proposed framework and approaches have been validated on actual data sets which include the Microsoft's Geolife data set and a data set collected by ourselves.