Physical capability (PC) is conventionally evaluated through performance-based clinical assessments. We aimed to transform a battery of sensor-based functional tests into a clinically applicable assessment tool. We used Exploratory Factor Analysis (EFA) to uncover the underlying latent structure within sensor-based measures obtained in a population-based study. Three hundred four community-dwelling older adults (163 females, 80.9 ± 6.4 years), underwent three functional tests (Quiet Stand, QS, 7-meter Walk, 7MW and Chair Stand, CST) wearing a smartphone at the lower back. Instrumented tests provided 73 sensor-based measures, out of which EFA identified a fifteen-factor model. A priori knowledge and the associations with health-related measures supported the functional interpretation and construct validity analysis of the factors, and provided the basis for developing a conceptual model of PC. For example, the “Walking Impairment” domain obtained from the 7MW test was significantly associated with measures of leg muscle power, gait speed, and overall lower extremity function. To the best of our knowledge, this is the first time that a battery of functional tests, instrumented through a smartphone, is used for outlining a sensor-based conceptual model, which could be suitable for assessing PC in older adults and tracking its changes over time.