Theoretical studies of thermoelectric properties using ab initio electronic structure calculations help not only to understand existing experimental data but also to predict new materials which can be potentially good thermoelectrics. However, in these studies it is inevitable to employ some approximations. It is therefore important to verify their reliability. To this end, we have investigated the validity of the rigid band approximation (RBA), commonly used in calculating thermopower (S) in doped (sometimes heavily) narrow band gap semiconductors. We have considered two important systems: half-Heusler (HH) HfCoSb and PbTe. We calculate band structures of pure and doped systems (using quasi-periodic approximation-QPA) by employing density-functional method. We then use Boltzmann transport theory to calculate thermopower using both RBA and the band structure with QPA. We find that band structures do not change significantly when isovalent impurities are present excepting in specific cases. However, charged impurities (relevant to the doping case) providing carriers can change the host band structure appreciably. We find that impurities in general remove existing degeneracies which tend to reduce the RBA value of | S |. The reduction is significant in both HfCoSb and PbTe when charged defects are present.