Worldwide increases in fluvial fine sediment are a threat to aquatic animal health. Fluvial fine sediment is always a mixture of particles whose mineralogical composition differs depending on the sediment source and catchment area geology. Nonetheless, whether particle impact in aquatic organisms differs between mineral species remains to be investigated. This study applied an in vitro approach to evaluate cytotoxicity and uptake of four common fluvial mineral particles (quartz, feldspar, mica, and kaolin; concentrations: 10, 50, 250 mg L−1) in the rainbow trout epithelial gill cell line RTgill-W1. Cells were exposed for 24, 48, 72, and 96 h. Cytotoxicity assays for cell membrane integrity (propidium iodide assay), oxidative stress (H2DCF-DA assay), and metabolic activity (MTT assay) were applied. These assays were complemented with cell counts and transmission electron microscopy. Regardless of mineral species, particles ≤2 µm in diameter were taken up by the cells, suggesting that particles of all mineral species came into contact and interacted with the cells. Not all particles, however, caused strong cytotoxicity: Among all assays the tectosilicates quartz and feldspar caused sporadic maximum changes of 0.8–1.2-fold compared to controls. In contrast, cytotoxicity of the clay particles was distinctly stronger and even differed between the two particle types: mica induced concentration-dependent increases in free radicals, with consistent 1.6–1.8-fold-changes at the 250 mg L−1 concentration, and a dilated endoplasmic reticulum. Kaolin caused concentration-dependent increases in cell membrane damage, with consistent 1.3–1.6-fold increases at the 250 mg L−1 concentration. All effects occurred in the presence or absence of 10% fetal bovine serum. Cell numbers per se were marginally affected. Results indicate that (i.) natural mineral particles can be cytotoxic to gill epithelial cells, (ii.) their cytotoxic potential differs between mineral species, with clay particles being more cytotoxic, and (iii.) some clays might induce effects comparable to engineered nanoparticles.