“…Other research efforts focus on visionbased systems to inform the digital representation and adapt tooling in accordance with the deviation between the physical realm and the digital representation (Dörfler, Rist and Rust, 2013;Schwartz et al, 2014;Brugnaro et al, 2016;Ercan Jenny et al, 2020;Nicholas et al, 2020). More recently, researchers began looking into how this established feedback loop can be utilized as a conversational method for designing within more interactive design-fabrication workflows (Zoran and Paradiso, 2013;Weichel et al, 2015;Capunaman, 2020;Lee and Llach, 2020) Within this body of work, we propose a computational framework for high fidelity reconstruction of the work object to inform subsequent fabrication operations and register the digital twin within the robot's coordinate system within tooling tolerances. Even though similar vision-based approaches to capture work environment exists in high precision fabrication scenarios such as the aerospace industry, these solutions are often black-boxed, task-specific and Figure 1 An example application of GFRP façade panels on SFMoMA Expansion Building.…”