We investigate the interaction of visible light with the solid matters of semiconducting oxide nanorods (NRs) of zinc oxide (ZnO), indium tin oxide (ITO), and zinc tin oxide (ZTO) at the single nanomaterial level. We subsequently identify an intriguing, material-dependent phenomenon of optical rotation in the electric field oscillation direction of the scattered light by systematically controlling the wavelength and polarization direction of the incident light, the NR tilt angle, and the analyzer angle. This polarization rotation effect in the scattered light is repeatedly observed from the chemically pure and highly crystalline ZnO NRs, but absent on the chemically doped NR variants of ITO and ZTO under all measurement circumstances. We further elucidate that the phenomenon of polarization rotation detected from single ZnO NRs is affected by the NR tilt angle, while the phenomenon itself occurs irrespective of the wavelength and incident polarization direction of the visible light. Combined with the widespread optical and optoelectronic use of the semiconducting oxide nanomaterials, these efforts may provide much warranted fundamental bases to tailor material-specific, single nanomaterial-driven, optically modulating functionalities which, in turn, can be beneficial for the realization of high-performance integrated photonic circuits and miniaturized bio-optical sensing devices.