The use of neonatal hearing screening has enabled the identification of congenital unilateral sensorineural hearing loss (USNHL) immediately after birth, and today there are several intervention options available to minimize potential adverse effects of this disease, including cochlear implantation. This study aims to analyze the characteristics of the inner ear of a homogeneous group of congenital non-syndromic USNHL to highlight the features of the inner ear, which can help in clinical, surgical, and rehabilitative decision-making. A retrospective chart review was carried out at a tertiary referral center. Systematic diagnostic work-up and rigorous inclusion–exclusion criteria were applied to 126 children with unilateral hearing impairment, leading to a selection of 39 strictly congenital and non-syndromic USNHL cases, undergoing computed tomography (CT) and magnetic resonance (MR) imaging studies. The frequency and type of malformations of the inner ear in USNHL and unaffected contralateral ears were assessed, with an in-depth analysis of the deficiency of the cochlear nerve (CND), the internal auditory canal (IAC) and the cochlear aperture (CA). Inner ear anomalies were found in 18 out of 39 (46%) of the USNHL patients. In 1 subject, the anomalies were bilateral, and the CND resulted in the predominant identified defect (78% of our abnormal case series), frequently associated with CA stenosis. Only 3 out of 14 children with CND presented stenosis of the IAC. CND and CA stenosis (and to a much lesser extent IAC stenosis) are a frequent association within congenital and non-syndromic USNHL that could represent a distinct pathological entity affecting otherwise healthy infants. In the context of a diagnostic work-up, the evaluation with CT and MRI measurements should take place in a shared decision-making setting with thorough counseling. Both imaging techniques have proven useful in differentiating the cases that will most likely benefit from the cochlear implant, from those with potentially poor implant performance.