Incremental nonlinear dynamic inversion (INDI) is a sensor-based control law design strategy that is based on the principles of feedback linearization. Contrary to its nonincremental counterpart (nonlinear dynamic inversion), this design method does not require a complete onboard model of the airframe dynamics and is therefore more robust against regular perturbations arising from aerodynamic variations. Therefore, INDI brings a natural design approach to desirable flying qualities. However, robustness against singular perturbations, which may arise due to transport delays, elastic airframe effects, or other types of badly modeled or unknown dynamics, is a known challenge for INDI-based control laws. Therefore, this paper addresses the question of robust stability and performance for INDI and its linear form (incremental dynamic inversion [IDI]) in the context of mixed regular and singular perturbations. This is done through analytical insights and by performing quantitative robustness assessments based on the structured singular value framework. Additionally, inversion loop augmentation solutions are investigated using robust synthesis techniques to improve the robustness characteristics of basic IDI designs.