The Global Navigation Satellite System (GNSS) radio occultation (RO) technique is widely used to observe the atmosphere for applications such as numerical weather prediction and global climate monitoring. The ionosphere is a major error source to RO at upper stratospheric altitudes and a linear dual-frequency bending angle correction is commonly used to remove the first-order ionospheric effect. However, the residual higher-order ionospheric error (RIE) can still be significant so that it needs to be further mitigated for high accuracy applications, especially from 30 km altitude upward where the RIE is most relevant compared to the decreasing magnitude of the atmospheric bending angle. In a previous study we quantified RIEs using an ensemble of about 700 quasi-realistic end-to-end simulated RO events, finding typical RIEs at the 0.1 to 0.5 rad noise level, but were left with 26 exceptional events with 5 anomalous RIEs at the 1 to 10 rad level that remained unexplained. In this study, we focused on investigating the causes of the high RIE of these exceptional events, employing detailed along-raypath analyses of atmospheric and ionospheric refractivities, impact parameter changes, and bending angles and RIEs under asymmetric and symmetric ionospheric structures. We found that the main causes of the high RIEs are a combination of physics-based effects, where asymmetric ionospheric conditions play the 10 primary role, more than the ionization level driven by solar activity, and technical ray tracer effects due to occasions of imperfect smoothness in ionospheric refractivity model derivatives. We also found that along-ray impact parameter variations of more than 10 to 20 m are well possible due to ionospheric asymmetries, and depending on prevailing horizontal refractivity gradients are positive or negative relative to the initial impact parameter at the GNSS transmitter. Furthermore, mesospheric RIEs are 15 found generally higher than upper stratospheric ones, likely due to being closer in tangent point heights to the ionospheric E layer peaking near 105 km, which increases RIE vulnerability. In future we will further improve the along-ray modeling system to fully isolate technical from physics-based effects and to use it beyond this work for additional GNSS RO signal propagation studies.