Background: Long-term coronavirus disease 2019 (long COVID) is associated with physio-somatic (chronic fatigue syndrome and somatic symptoms) and affective (depression and anxiety) symptoms. The severity of the long COVID physio-affective phenome is largely predicted by increased peak body temperature (BT) and lowered oxygen saturation (SpO2) during the acute infectious phase. This study aims to delineate whether the association of BT and SpO2 during the acute phase and the long COVID physio-affective phenome is mediated by neurotoxicity (NT) resulting from activated immune-inflammatory and oxidative stress pathways. Methods: We recruited 86 patients with long COVID (3–4 months after the acute phase) and 39 healthy controls and assessed serum C-reactive protein (CRP), caspase 1, interleukin (IL) 1β, IL-18, IL-10, myeloperoxidase (MPO), advanced oxidation protein products (AOPPs), total antioxidant capacity (TAC), and calcium (Ca), as well as peak BT and SpO2 during the acute phase. Results: Cluster analysis revealed that a significant part (34.9%) of long COVID patients (n = 30) show a highly elevated NT index as computed based on IL-1β, IL-18, caspase 1, CRP, MPO, and AOPPs. Partial least squares analysis showed that 61.6% of the variance in the physio-affective phenome of long COVID could be explained by the NT index, lowered Ca, and peak BT/SpO2 in the acute phase and prior vaccinations with AstraZeneca or Pfizer. The most important predictors of the physio-affective phenome are Ca, CRP, IL-1β, AOPPs, and MPO. Conclusion: The infection–immune–inflammatory core of acute COVID-19 strongly predicts the development of physio-affective symptoms 3–4 months later, and these effects are partly mediated by neuro-immune and neuro-oxidative pathways.