We revisit here the issue of thermally assisted Quantum Annealing by a detailed study of the dissipative Landau-Zener problem in presence of a Caldeira-Leggett bath of harmonic oscillators, using both a weak-coupling quantum master equation and a quasi-adiabatic path-integral approach. Building on the known zero-temperature exact results (Wubs et al., PRL 97, 200404 (2006)), we show that a finite temperature bath can have a beneficial effect on the ground-state probability only if it couples also to a spin-direction that is transverse with respect to the driving field, while no improvement is obtained for the more commonly studied purely longitudinal coupling. In particular, we also highlight that, for a transverse coupling, raising the bath temperature further improves the ground-state probability in the fast-driving regime. We discuss the relevance of these findings for the current quantum-annealing flux qubit chips.