The non-destructive measurement of a subsystem within a larger quantum system is crucial for error correction during quantum computation, simulation, and metrology, and for studying open quantum system dynamics. In many quantum technologies based on trapped atoms, measurement is performed by imaging all atoms simultaneously, a process that is typically slow and that decoheres the entire quantum system. Here, we use a strongly coupled optical cavity to read out the state of a single tweezer-trapped 87 Rb atom within a small tweezer array. Measuring either atomic fluorescence or the transmission of light through the cavity, we detect both the presence and the state of an atom in the tweezer, requiring only tens of microseconds for state-selective detection, with state preparation and measurement infidelities of roughly 0.5% and atom loss probabilities of around 1%. Using a two-tweezer system, we find measurement on one atom within the cavity causes no observable hyperfine-state decoherence on a second atom located tens of microns from the cavity volume.