We place empirical constraints on the yields from zero-and low-metallicity core collapse supernovae (CCSNe) using abundances measured in very metal-poor (VMP; [Fe/H] ≤ −2) Damped Lyman Alpha Absorbers (DLAs). For some abundance ratios, VMP DLAs constrain the metal yields of the first SNe more reliably than VMP stars. We compile a large sample of high-S/N VMP DLAs from over 30 years of literature, most with high resolution spectral measurements. We infer the median IMFaveraged CCSNe yield from the median values from the DLA abundance ratios of C, N, O, Al, Si, S, and Fe (with respect to Fe and O). Critically, we assume that the DLAs are metal-poor enough that they represent galaxies in their earliest stages of evolution, when CCSNe are the only nucleosynthetic sources of the metals that we analyze. We compare five sets of widely adopted zero-and low-metallicity theoretical yields to the empirical yields derived in this work. We find that the five theoretical yield sets agree with the DLA yields for [Si/O], [Si/Fe], [S/O], and [S/Fe]. The theoretical models predict a very large range of N yields, and only one of the models, Heger & Woosley (2010, hereafter HW10), reproduced the DLA value of [N/Fe] and [N/O], and only one other model, Limongi & Chieffi (2018, hereafter LC18), reproduced [N/O].We briefly investigate the adoption of a SN explosion landscape (where certain initial stellar masses collapse into black holes without contributing to nucleosynthesis) onto HW10 and find the predictions are comparable to when there is no landscape. On the other hand, fixing explosion energy to progenitor mass results in wide disagreements between the predictions and DLA abundances. We also investigate the adoption of a simple, observationally motivated Initial Distribution of Rotational Velocities (IDROV) for LC18, which has rotation velocity is a free parameter, and find a slight improvement in agreement with the observations.