The microtubule associated protein tau forms filamentous assemblies in the cytosol of neurons in Alzheimer's disease and other neurodegenerative diseases. Assemblies of tau have been proposed to transit between cells of the brain in a 'prion-like' manner, resulting in templated aggregation of native tau in recipient neurons. Interactions between tau assemblies, surface receptor LRP1 and heparan sulphate proteoglycans promote the uptake of tau assemblies to membrane-bound vesicles. A subsequent escape from these vesicles is postulated for assemblies to enter the cytosol and contact cytosolic tau pools. However, the process by which tau assemblies enter the cytosol is poorly defined. Here we establish assays that permit the study of tau entry in real time and at physiological concentrations. Modulation of entry by genetic or pharmacologic means alters levels of seeded aggregation, confirming the role of cytosolic entry as the rate-limiting, upstream step in seeded aggregation. Entry to HEK293, a commonly used reporter cell line, depended on clathrin-mediated pathways with late endosomal Rab7 GTPase involvement. In contrast, entry to primary neurons was via a clathrin- and dynamin-independent route that was sensitive to cholesterol levels. Extraction of cholesterol from neurons increased tau entry to the cytosol, consistent with cholesterol's established role in maintaining vesicle stability. Importantly, reducing cholesterol levels increased seeded aggregation in both primary neurons and organotypic slice culture models of tau pathology. Finally, we find no evidence that tau assemblies mediate their own entry to the cytosol by membrane rupture. Our results establish cytosolic entry as a distinct event from uptake and is upstream and essential to seeded aggregation. They further describe a cholesterol-sensitive, clathrin-independent pathway of tau entry to the cytosol of neurons.