Due to the high positioning uncertainty and low stiffness of robots, significant geometrical deviations occur in robotic incremental sheet forming (ISF) which requires an optical in-process tool pose measurement. To achieve a measuring uncertainty of less than 50 $$\upmu $$
μ
m for the position components and less than 0.05$$^\circ $$
∘
for the orientation components in the machining volume, a multi-sensor system based on shadow imaging sensors is proposed. To calculate the six-degree-of-freedom tool pose, the sensor system measures the positions of three LEDs attached to the tool. The LEDs emit light with different colors, and the associated shadows are separately detected by using color filters. In addition, each sensor consists of a shadow-casting mask and a monochrome camera. Experimental results show that the systematic error dominates the measurement uncertainty budget and needs to be calibrated, while the random error is one order of magnitude smaller than the required measuring uncertainty. Furthermore, the color filters reduce the cross-sensitivities between the signals to an acceptable level. Final experiments on a robot demonstrate plausible pose measurements indicating the feasibility to apply the multi-sensor system in robotic ISF.