Recent development in computer-based manufacturing and inspection has necessitated extended knowledge and usage of geometric tolerances as carriers of design intent. The aim of applying geometrical tolerances in design is to provide function-oriented precise description of part geometry where the conventional size tolerance system fails to address. In view of the current development of computer-aided systems, applying geometric tolerances opens a new research front. This article examines the challenges in applying geometric tolerance information to carry the design intent to other downstream manufacturing processes and intelligently integrate the whole system. Based on the observed practical capabilities and literature studies, it is concluded that the current computer-aided design (CAD) systems cannot effectively provide the appropriate use of geometric tolerances. This article highlights the existing challenges and proposes a scheme of algorithm development for appropriate use of tolerance symbols and conditions at the design specification stage. This, in the long run, enables the CAD model to carry the design intent and opens a window of opportunity for intelligently integrating manufacturing systems.