In the present work, we study the anharmonicity of the unfilled CoSb3 skutterudite from the pressure and temperature dependence of the Raman-active modes. A full assignment of the Raman-active modes of CoSb3 was performed for the first time. Significant changes in the Raman spectrum are observed above 27 GPa and attributed to the insertion of antimony atoms within the cages under high external pressure. We report the lattice dynamics of SbCo4Sb12 by DFT calculations and find very low energy optical modes due to the antimony guest atoms with the cage framework. We report the isothermal Grüneisen parameters of the Raman-active modes, which are larger and approach 2 for the highest energy modes. From analysis of the temperature and pressure dependence of the Raman-active modes, we found that the implicit volume contribution is the dominant contribution for the highest energy Raman-active modes whereas both the implicit volume contribution and the explicit anharmonic contribution have same magnitude for the other Raman-active modes.