Camera positioning system is a critical member of a nanoindentation tester characterizing the mechanical properties such as hardness, creep, surface roughness, or elastic modulus of a material sample. This paper presents a design optimization for a dragonfly-inspired compliant joint. This joint is used to drive the camera positioning system. A new hybrid approach of Taguchi method, adaptive neuro-fuzzy inference system (ANFIS), and Jaya algorithm is developed to solve the multi-objective optimization problem. The Taguchi method is used to build the numerical data and to find the best membership functions for the ANFIS structure by minimizing the root mean squared error. Then, the weight factor of each objective function is determined by established equations well. Subsequently, a structure of ANFIS is developed to map the design parameters and responses. Sensitivity analysis of each controllable parameter is analyzed by the statistical method. Finally, Jaya algorithm is initialized to find the optimal solution. The results found that the optimal displacement, frequency, and stress are about 12581.11 μm, 67.76 Hz, and 333.68 MPa, respectively. The proposed hybrid optimization algorithm is a robust and effective optimizer and considered as soft computing technique for engineering optimization problems.