Mobile robots have many applications in the modern world. The autonomy of robots is increasing, but critical cases like search and rescue missions must involve the possibility of human intervention for ethical reasons and safety. To achieve effective human–robot interaction, the operator needs to have a sense of agency (SoA) over the activities of the robot. One possible way to increase one's SoA in remote control could be the use of VR technology. The remote control situation has some important features, so indicators of SoA need to be reproduced there independently. In our study, participants controlled a mobile robot using either a monitor or a VR-headset as an output device. In both cases, active control was contrasted with passive observation of the robot's movement. In each trial, participants estimated the distance traveled by the robot—a putative implicit indicator of SoA. A significant difference between subjective distance estimates was found in the active and passive conditions with the monitor, but not in the active and passive conditions with VR. The effect obtained in the monitor conditions suggests that distance estimates can be used as an implicit indicator of SoA in robot remote control. We believe that the lack of difference between the active and passive conditions in VR was caused by motion sickness due to a mismatch of visual and vestibular sensory cues, leading to a weakened SoA.