Precise temperature measurements on systems of few ultracold atoms is of paramount importance in quantum technology, but can be very resource-intensive. Here, we put forward an adaptive Bayesian framework that substantially boosts the performance of cold atom temperature estimation. Specifically, we process data from release-recapture thermometry experiments on few potassium atoms cooled down to the microkelvin range in an optical tweezer. We demonstrate that adaptively choosing the release-recapture times to maximise information gain does substantially reduce the number of measurements needed for the estimate to converge to a final reading. Unlike conventional methods, our proposal systematically avoids capturing and processing uninformative measurements. Furthermore, we are able to produce much more reliable estimates, especially when the measured data are scarce and noisy. Likewise, the resulting estimates converge faster to the real temperature in the asymptotic limit. Our method can be adapted to enhance the precision and resource-efficiency of many other techniques running on different experimental setups, thus opening new avenues in quantum thermometry.