In this study, we address two technical challenges to enhance golf swing trajectory accuracy using a wrist-worn inertial sensor: orientation estimation and drift error mitigation. We extrapolated consistent sensor orientation from specific address-phase signal segments and trained the estimation with a convolutional neural network. We then mitigated drift error by applying a constraint on wrist speed at the address, backswing top, and finish, and ensuring that the wrist's finish displacement aligns with a virtual circle on the 3D swing plane. To verify the proposed methods, we gathered data from twenty male right-handed golfers, including professionals and amateurs, using a driver and a 7-iron. The orientation estimation error was about 60% of the baseline, comparable to studies requiring additional sensor information or calibration poses. The drift error was halved and the single-inertial-sensor tracking performance across all swing phases was about 17 cm, on par with multimodal approaches. This study introduces a novel signal processing method for tracking rapid, wide-ranging motions, such as a golf swing, while maintaining user convenience. Our results could impact the burgeoning field of daily motion monitoring for health care, especially with the increasing prevalence of wearable devices like smartwatches.