Older adults residing in long-term care often experience declines in physical and cognitive functions despite the access to in-house physical activity (PA) programs. This study aimed to evaluate the associations of PA with physical function and global cognitive function in older adults living in long-term care, while examining potential sex differences. A cross-sectional analysis of baseline data from a two-arm cluster randomized controlled trial was conducted, involving 67 participants (34 men, 33 women). PA levels were assessed using tri-axial accelerometers. Physical function, including muscular strength, postural sway, and Short Physical Performance Battery (SPPB) and cognitive function were measured. Spearman correlation analysis revealed no significant associations between PA metrics and muscular strength, postural sway, or global cognitive function across the entire samples (p ≥ 0.091). Multiple linear regression models were developed for the entire sample, males, and females to examine the associations between PA and physical function measures and global cognitive function. After adjustments for confounders, light PA was significantly associated with higher SPPB sub-scores (gait: β = 0.600, p < 0.001; sit-to-stand: β = 0.574, p < 0.001), faster usual gait speed (β = 0.659, p = 0.012), and shorter sit-to-stand times (β = −0.305, p = 0.041) across the whole sample. Similar significant associations were observed in males between light PA and SPPB scores (total: β = 0.319, p = 0.040; gait: β = 0.532, p < 0.001; sit-to-stand: β = 0.417, p = 0.009), usual gait speed (β = 0.762, p = 0.017), and sit-to-stand times (β = −0.677, p < 0.001). In females, a significant association was found between light PA and global cognitive function (β = 0.319, p = 0.012) after adjusting for confounders. This study highlights sex differences in the association of accelerometer-determined PA with physical and cognitive function in older adults living in long-term care, with LPA showing beneficial effects, especially for physical function in males and for cognitive function in females.