Inverse kinematics problems (IKP) are ubiquitous in robotics for improved robot control in widespread applications. However, the high non-linearity, complexity, and equation coupling of a general six-axis robotic manipulator pose substantial challenges in solving the IKP precisely and efficiently. To address this issue, we propose a novel approach based on neural network (NN) with numerical error minimization in this paper. Within our framework, the complexity of IKP is first simplified by a strategy called joint space segmentation, with respective training data generated by forward kinematics. Afterwards, a set of multilayer perception networks (MLP) are established to learn from the foregoing data in order to fit the goal function piecewise. To reduce the computational cost of the inference process, a set of classification models is trained to determine the appropriate forgoing MLPs for predictions given a specific input. After the initial solution is sought, being improved with a prediction error minimized, the refined solution is finally achieved. The proposed methodology is validated via simulations on Xarm6—a general 6 degrees of freedom manipulator. Results further verify the feasibility of NN for IKP in general cases, even with a high-precision requirement. The proposed algorithm has showcased enhanced efficiency and accuracy compared to NN-based approaches reported in the literature.