Counterions at charged rod-like polymers exhibit a condensation transition at a critical temperature (or equivalently, at a critical linear charge density for polymers), which dramatically influences various static and dynamic properties of charged polymers. We address the critical and universal aspects of this transition for counterions at a single charged cylinder in both two and three spatial dimensions using numerical and analytical methods. By introducing a novel Monte-Carlo sampling method in logarithmic radial scale, we are able to numerically simulate the critical limit of infinite system size (corresponding to infinite-dilution limit) within tractable equilibration times. The critical exponents are determined for the inverse moments of the counterionic density profile (which play the role of the order parameters and represent the inverse localization length of counterions) both within mean-field theory and within Monte-Carlo simulations. In three dimensions, we demonstrate that correlation effects (neglected within mean-field theory) lead to an excessive accumulation of counterions near the charged cylinder below the critical temperature (condensation phase), while surprisingly, the critical region exhibits universal critical exponents in accord with the mean-field theory. Also in contrast with the typical trend in bulk critical phenomena, where fluctuations are strongly enhanced in lower dimensions, we demonstrate, using both numerical and analytical approaches, that the mean-field theory becomes exact for the 2D counterion-cylinder system at all temperatures (Manning parameters), when number of counterions tends to infinity. For finite particle number, however, the 2D problem displays a series of peculiar singular points (with diverging heat capacity), which reflect successive de-localization events of individual counterions from the central cylinder. In both 2D and 3D, the heat capacity shows a universal jump at the critical point, and the energy develops a pronounced peak. The asymptotic behavior of the energy peak location is used to locate the critical point, which is also found to be universal and in accordance with the mean-field prediction.