Background and Objectives: It is thought that muscle and bone interact only on a biomechanical level, however, some research is now emerging that links bone and muscle on a cellular level. The aim of this study was to explore associations between physical function, muscle mass and bone density in community-dwelling elderly men with sarcopenia. A secondary goal was to analyze if muscle morphology was associated with bone density and physical functioning. Materials and Methods: Body composition was measured by dual-energy X-ray absorptiometry (DXA). Bone density was evaluated according to WHO criteria. Sarcopenia was diagnosed according to European Working Group on Sarcopenia in Older People (EWGSOP) criteria: low muscle mass and low muscle strength or low physical performance. Microbiopsy of musculus vastus lateralis was performed with a disposable muscle microbiopsy system. The perimeter and cross-sectional area of muscle fibers were calculated using image analysis software in whole slide images; type of fibers and their distribution were evaluated as well. Results: A total of 151 men, 60 years or older were included in this study. Mean age of the subjects was 72.9 ± 8.02 years. Sarcopenia was diagnosed in 45 (29.8%) men. Multiple significant correlations were found between bone mineral density, lean mass, appendicular lean mass, arm and leg lean mass, gait speed, balance test and handgrip strength in sarcopenic men. Lean mass was associated with femoral neck BMD (bone mineral density; r = 0.418, p = 0.006) and handgrip strength (r = 0.553, p < 0.001). In the sarcopenia group, 25 muscle biopsies were examined. In 9 sarcopenic men with T-scores equal or below −2.5, the muscle fiber area had a significant correlation with the balance test (r = 0.73, p = 0.025). Conclusions: In men with sarcopenia, low lean muscle mass was associated with low femoral neck BMD and low muscle strength. In sarcopenic men with osteoporosis, low muscle fiber area was associated with low scores in a balance test.