Online identification of time-variant parameters without knowledge of external loads is an important but challenging task for structural health monitoring and vibration control. In this study, a two-stage approach, named extended Kalman filter with forgetting factor matrix under unknown inputs (EKF-FFM-UI), is proposed for simultaneously identifying the time-variant parameters and external loads. In stage 1, an extended Kalman filter under unknown inputs (EKF-UI) approach previously proposed by the authors is employed for estimating the structural states and unknown loads. This EKF-UI approach is solely suitable for time-invariant system identification. Therefore, the aim of stage 2 is to improve this approach for the purpose of possessing tracking capability. In this stage, the acceleration responses are first reconstructed by using the differential equation of motion and employed for improving the accuracy of estimated structural states. A forgetting factor matrix is introduced into the priori estimation error covariance matrix to track time-varying parameters. The square errors between the measurements and the corresponding estimates are defined as an index and used for detecting the damage time instant. Then, a covariance resetting technique is employed to assure that such changes in structural parameters can be efficiently captured. A shear-type building structure without/with magneto-rheological (MR) dampers and a fixed beam structure are used as numerical examples for validating the effectiveness of the proposed approach. Experimental tests on a six-story building model are also conducted. Results show the time-varying parameters and unknown inputs can be simultaneously identified with acceptable accuracy.