This paper explores the influence of compensation on employee retention of the banking institutions in Tanzania. The stratified sampling technique was used to ensure representativeness of each bank category. Eleven banks were purposely selected and simple random sampling was applied to draw a sample of 370 employees. The quantitative data were collected using questionnaire and analyzed using Binary logistic regression. The results indicated that compensation is significantly influencing employee retention with a ‘p’ value of 0.001. It was further revealed that compensation attribute (fair salary) has significant influence on employee retention. The study recommends to the bank managers to develop and implement retention policies that contemplate fair salaries as this is a most valued compensation attribute. Furthermore, the bank managers need to pay great attention to right retention policies in order to improve retention of employees.