This is a comprehensive review of the combination of scanning probe microscopy (SPM) with various optical spectroscopies, with a particular focus on Raman spectroscopy. Efforts to combine SPM with optical spectroscopy will be described, and the technical difficulties encountered will be examined. These efforts have so far focused mainly on the development of tip-enhanced Raman spectroscopy, a powerful technique to detect and image chemical signatures with single molecule sensitivity, which will be reviewed. Beyond tip-enhanced Raman spectroscopy and/or topography measurements, combinations of SPM with optical spectroscopy have a great potential in the characterization of structure and quantitative measurements of physical properties, such as mechanical, optical, or electrical properties, in delicate biological samples and nanomaterials. The different approaches to improve the spatial resolution, the chemical sensitivity, and the accuracy of physical properties measurements will be discussed. Applications of such combinations for the characterization of structure, defects, and physical properties in biology and materials science will be reviewed. Due to the versatility of SPM probes for the manipulation and characterization of small and/or delicate samples, this review will mainly focus on the apertureless techniques based on SPM probes.