In the stage of aging society and population aging, the social needs of the elderly are widely discussed by researchers. Especially driven by the demand of tele-medical treatment and tele-rehabilitation therapy, it is vital for the elderly to integrate into virtual communities by combining social virtual reality (VR) with different medical services and entertainment needs. In addition, affected by the COVID-19 epidemic, it is more difficult for people to have face-to-face contacts. With more remote consultation, entertainment and virtual social connectivity, the application of social VR is more urgent and valuable. However, there is little discussion on the acceptability and influencing factors of social VR among the elderly at present. Therefore, in order to get further data, we used (1) early stage semi-structured interviews and then (2) Unified Theory of Acceptance and Use of Technology (UTAUT) questionnaires for investigation. One hundred fourteen elderly people aged 60–89 living in the metropolitan area of Taipei were taken as the subjects. To help them understand the situation and state of using social VR, these elderly people were asked to use a head-mounted display (HMD) to experience social VR games. The preliminary results showed that the elderly had obvious preference for entertainment (32.4%) and medical treatment (31.3%). The interview showed that this was related to the physiological condition or medical needs of the age range. In order to further understand how social VR would affect the social life of the elderly, we proposed the further demand structure of UTAUT Model based on the interview of both experts and the elderly. The model structures include (1) Performance Expectancy, (2) Perceived Enjoyment, (3) Social Influence, (4) User Attitude, (5) Behavioral Intention, and so on. These structures were applied to conduct interviews and questionnaires to find out the influence extent and relevance of the elderly on different structural needs, and suggestions were given accordingly. The results of the above interviews showed that (1) the elderly thought that the functions of entertaining and interacting of social VR could increase their social opportunities, and also meet medical needs (teleconference, cognitive decline, etc.), (2) the closeness of social relations (between family members, friends, doctors, and places), and also affect the relevance of Perceived Enjoyment (β = 0.77, p = 0.000 < 0.05). The results of these phenomena and interviews showed the interplay between the demand structures and their special relevance. They also indicated that as to social VR technologies, various demands and functional issues of the elderly need to be considered, and these demands would appear in the subtle usage, and different social VR interfaces and functions would emerge based on their special living ways and physical and psychological demands.