Ontology-based knowledge driven Activity Recognition (AR) models play a vital role in realm of Internet of Things (IoTs). However, these models suffer the shortcomings of static nature, inability of self-evolution and lack of adaptivity. Also, AR models cannot be made comprehensive enough to cater all the activities and smart home inhabitants may not be restricted to only those activities contained in AR model. So, AR models may not rightly recognize or infer new activities. In this paper, a framework has been proposed for dynamically capturing the new knowledge from activity patterns to evolve behavioural changes in AR model (i.e. ontology based model). This ontology based framework adapts by learning the specialized and extended activities from existing user-performed activity patterns. Moreover, it can identify new activity patterns previously unknown in AR model, adapt the new properties in existing activity models and enrich ontology model by capturing change representation to enrich ontology model. The proposed framework has been evaluated comprehensively over the metrics of accuracy, statistical heuristics and Kappa Coefficient. A well-known dataset named DAMSH has been used for having an empirical insight to the effectiveness of proposed framework that shows a significant level of accuracy for AR models.