Mapping the environment is the basis for measurement tasks, planning processes, monitoring over time, and decision-making. Moreover, the inspection of engineering structures, roads and railways, water and wastewater, and energy infrastructure is essential to ensure safety and sustainability. For an automated mapping and inspection process, the use of robotic systems is indispensable. Due to the complexity of the environments of structures and the requirements for inspection, the automated operation requires real-time navigation, specific sensor solutions, and automated interpretation of the data. In this study, we present various concepts of autonomous measurement robotics for mapping and inspecting challenging environments and damage types. Robotic systems based on unmanned aerial vehicles, unmanned ground vehicles, and unmanned underwater vehicles are addressed for various scenarios including pipes, dams, bridges, tunnels, coastal areas from the air, and underwater structures. After identifying the demands and needs and presenting the state-of-the-art methods and systems and their identifying limitations, we present ways for an autonomous operation of robotic systems equipped with specific sensors for LiDAR-based subsurface damage detection of cavities or delaminations, underwater laser scanning, high-resolution bathymetry, and multispectral LiDAR for moisture detection. This includes real-time data interpretation, iterative inspection strategies, automated data interpretation, and the strategy for the integration of the human operator. In conclusion, this work gives an overview of existing methods and systems and their limitations for mapping and inspection tasks. Moreover, we show how advanced sensor systems can be used as part of autonomous robotic systems to overcome current limitations for high-quality mapping and inspection of challenging environments and structures.