This study explores the impact of crystalline fraction on the mechanical losses of amorphous tantalum oxide (tantala, Ta2O5) thin films intended for gravitational wave detectors. We use ion beam sputtering (IBS) technique to prepare a series of samples, which are then subjected to controlled thermal annealing to achieve varying degrees of crystallized fraction. The microscopic structure of the annealed samples is characterized by combining different analytical techniques. Our investigation reveals that the amorphous films comprise randomly distributed crystalline grains, whose density and average size depends on the duration of thermal treatment. To assess mechanical losses of the coatings, a Gentle Nodal Suspension (GeNS) system is applied. Remarkably, a substantial reduction of approximately 20% in the coating’s mechanical loss angle with respect to annealed amorphous coatings is observed for samples exhibiting a crystalline fraction of around 5%. This improvement may lead to the definition of alternative thermal treatments to improve the mechanical performances of coatings for gravitational wave detectors or other highly sensitive optical experiments. However the reduction in mechanical losses comes at the expense of an increase in optical scattering. The possibility of reducing the optical losses to the level required by gravitational interferometers by modifying the grain size distribution via appropriate annealing treatments is discussed.