Recent advances in the determination of crystal structures and studies of optical properties of gold nanoclusters in the size range from tens to hundreds of gold atoms have started to reveal the grand evolution from gold complexes to nanoclusters and further to plasmonic nanoparticles. However, a detailed comparison of their photophysical properties is still lacking. Here, we compared the excited state behaviors of gold complexes, nanolcusters, and plasmonic nanoparticles, as well as small organic molecules by choosing four typical examples including the Au10 complex, Au25 nanocluster (1 nm metal core), 13 diameter Au nanoparticles, and Rhodamine B. To compare their photophysical behaviors, we performed steady-state absorption, photoluminescence, and femtosecond transient absorption spectroscopic measurements. It was found that gold nanoclusters behave somewhat like small molecules, showing both rapid internal conversion (<1 ps) and long-lived excited state lifetime (about 100 ns). Unlike the nanocluster form in which metal–metal transitions dominate, gold complexes showed significant charge transfer between metal atoms and surface ligands. Plasmonic gold nanoparticles, on the other hand, had electrons being heated and cooled (~100 ps time scale) after photo-excitation, and the relaxation was dominated by electron–electron scattering, electron–phonon coupling, and energy dissipation. In both nanoclusters and plasmonic nanoparticles, one can observe coherent oscillations of the metal core, but with different fundamental origins. Overall, this work provides some benchmarking features for organic dye molecules, organometallic complexes, metal nanoclusters, and plasmonic nanoparticles.