The distribution of ranging errors of time of arrival techniques fails to satisfy zero means and equal variances. It is one of the major causations of position error of least square-based localization algorithm. The optimization of time of arrival ranging is defined as a nonlinear programming problem. Then, time of arrival ranging error model and geometric constraints are used to define the initial values, objective functions, and constraints of nonlinear programming, as well as to detect line of sight and nonline of sight. A three-dimensional localization algorithm of an indoor time of arrival-based positioning is proposed based on least square and the optimization algorithm. The performance of the ranging and localization accuracies is evaluated by simulation and field testing. Results show that the optimized ranging error successfully satisfies zero mean value and equal variances. Furthermore, the ranging and localization accuracies are significantly improved.