We summarize how future measurements of electromagnetic (EM) probes at the Relativistic Heavy Ion Collider (RHIC), in connection with theoretical analysis, can advance our understanding of strongly interacting matter at high energy densities and temperatures. After a brief survey of the important role that EM probes data have played at the Super Proton Synchrotron (SPS, CERN) and RHIC to date, we identify key physics objectives and observables that remain to be addressed to characterize the (strongly interacting) Quark-Gluon Plasma (sQGP) and associated transition properties at RHIC. These include medium modifications of vector mesons via low-mass dileptons, a temperature measurement of the hot phases via continuum radiation, as well as γ-γ correlations to characterize early source sizes. We outline strategies to establish microscopic matter and transition properties such as the number of degrees of freedom in the sQGP, the origin of hadron masses and manifestations of chiral symmetry restoration, which will require accompanying but rather well-defined advances in theory. Increased experimental precision, an order of magnitude higher statistics than currently achievable, as well as a detailed scan of colliding species and energies are mandatory to discriminate between theoretical interpretations. This increased precision can be achieved through hardware upgrades to the large RHIC detectors (PHENIX and STAR) along with at least a factor of ten increase in luminosity over the next few years, as envisioned for RHIC-II.