Competition is an essential driving factor that influences forest community sustainability, yet measuring it poses several challenges. To date, the Competition Index (CI) has generally been the tool of choice for quantifying actual competition. In this study, we proposed using the Total Overlap Index (TOI), a CI in which the Area Overlap (AO) index has been adapted and modified to consider the “shading” and “crowding” effects in the vertical dimension. Next, based on six mixed forest plots in Xiaolong Mountain, Gansu, China, we assessed the results to determine the TOI’s evaluation capability. Individual-tree simulation results showed that compared to the modified Area Overlap index (AOM), the TOI has superior quantification capability in the vertical direction. The results of the basal area increment (BAI) model showed that the TOI offers the best evaluation capability among the four considered CIs in mixed forest (with Akaike Information Criterion (AIC) of 1041.60 and log-likelihood (LL) of −511.80 in the model fitting test, mean relative error of −28.67%, mean absolute percent error of 117.11%, and root mean square error of 0.7993 in cross-validation). Finally, the TOI was applied in the Kaplan–Meier survival analysis and Cox proportional-hazards analysis. The Kaplan–Meier survival analysis showed a significant difference between the low- (consisting of trees with the TOI lower than 1) and high-competition (consisting of trees with the TOI higher than 1) groups’ survival and hazard curves. Moreover, the results of the Cox proportional-hazards analysis exhibited that the trees in the low-competition group only suffered 34.29% of the hazard risk that trees in the high-competition group suffered. Overall, the TOI expresses more dimensional information than other CIs and appears to be an effective competition index for evaluating individual tree competition. Thus, the competition status quantified using this method may provide new information to guide policy- and decision-makers in sustainable forest management planning projects.