Extracellular vesicles (EVs) are complex biological nanoparticles endogenously secreted by all eukaryotic cells. EVs carry a specific molecular cargo of proteins, lipids, and nucleic acids derived from cells of origin and play a significant role in the physiology and pathology of cells, organs, and organisms. Upon release, they may be found in different body fluids that can be easily accessed via noninvasive methodologies. Due to the unique information encoded in their molecular cargo, they may reflect the state of the parent cell and therefore EVs are recognized as a rich source of biomarkers for early diagnostics involving liquid biopsy. However, body fluids contain a mixture of EVs released by different types of healthy and diseased cells, making the detection of the EVs of interest very challenging. Recent research efforts have been focused on the detection and characterization of diagnostically relevant subpopulations of EVs, with emphasis on label-free methods that simplify sample preparation and are free of interfering signals. Therefore, in this paper, we review the recent progress of the label-free optical methods employed for the detection, counting, and morphological and chemical characterization of EVs. We will first briefly discuss the biology and functions of EVs, and then introduce different optical label-free techniques for rapid, precise, and nondestructive characterization of EVs such as nanoparticle tracking analysis, dynamic light scattering, atomic force microscopy, surface plasmon resonance spectroscopy, Raman spectroscopy, and SERS spectroscopy. In the end, we will discuss their applications in the detection of neurodegenerative diseases and cancer and provide an outlook on the future impact and challenges of these technologies to the field of liquid biopsy via EVs.