Abstract-Mobile robots are increasingly entering the real and complex world of humans in ways that necessitate a high degree of interaction and cooperation between human and robot. Complex simulation models, expensive hardware setup, and a highly controlled environment are often required during various stages of robot development. There is a need for robot developers to have a more flexible approach for conducting experiments and to obtain a better understanding of how robots perceive the world. Mixed Reality (MR) presents a world where real and virtual elements co-exist. By merging the real and the virtual in the creation of an MR simulation environment, more insight into the robot behaviour can be gained, e.g. internal robot information can be visualised, and cheaper and safer testing scenarios can be created by making interactions between physical and virtual objects possible. Robot developers are free to introduce virtual objects in an MR simulation environment for evaluating their systems and obtain a coherent display of visual feedback and realistic simulation results. We illustrate our ideas using an MR simulation tool constructed based on the 3D robot simulator Gazebo.

Abstract. The objective of this research is to apply markerless Augmented Reality (AR) techniques to aid in the visualisation of robotic helicopter related tasks. Conventional robotic AR applications work well with markers in prepared environments but are infeasible in outdoor settings. In this paper, we present preliminary results from a real time markerless AR system for tracking natural features in an agricultural scene. By constructing a virtual marker under a known initial configuration of the robotic helicopter, camera and the ground plane, the camera pose can be continuously tracked using the natural features from the image sequence to perform augmentation of virtual objects. The experiments are simulated on a mock-up model of an agricultural farm and the results show that the current AR system is capable of tracking the camera pose accurately for translational motions and roll rotations. Future work includes reducing jitter in the virtual marker vertices to improve camera pose estimation accuracy for pitch and yaw rotations, and implementing feature recovery algorithms.

Abstract. The development cycle of an Unmanned Aerial Vehicle (UAV) system can be long and challenging. Mixed Reality (MR) simulations can reduce cost, duration and risk of the development process by enabling the replacement of expensive, dangerous, or not yet fully developed components with virtual counterparts. However, there has been little validation of such hybrid simulation methods in practical robot applications. This paper evaluates the use of MR simulations for prototyping a UAV system to be deployed for a dairy farming monitoring task. We show that by augmenting the robot's sensing with a virtual moving cow using an extensible Augmented Reality (AR) tracking technique, MR simulations could help to provide efficient testing and identify improvements to the UAV controller. User study findings reveal the importance of both virtual and MR simulations to robot development, with MR simulations helping developers transition to development in a more physical environment.

Abstract. A Mixed Reality (MR) simulation aims to enable robot developers to create safe and close-to-real world environments from a mixture of real and virtual components for experimenting with robot systems. However, the reliability of the simulation results and its usefulness in solving practical problems remain to be validated. This paper presents an evaluation of an MR simulator by examining its use for the development of a robotic screw remover system. Quantitative evaluation compares the robot's trajectories produced in our MR simulation with those from a real world experiment, yielding results that indicate the MR simulation reliably represents the real world. A user study was conducted and the results demonstrate that the MR simulator gives users a stronger confidence of accurate results in comparison to a virtual simulator.