By thermoelectric power generation we mean the creation of electrical power directly from a temperature gradient. Semiconductors have been mainly used for this purpose, but these imply the use of rare and expensive materials. We show in this review that ion-exchange membranes may be interesting alternatives for thermoelectric energy conversion, giving Seebeck coefficients around 1 mV/K. Laboratory cells with Ag|AgCl electrodes can be used to find the transported entropies of the ions in the membrane without making assumptions. Non-equilibrium thermodynamics can be used to compute the Seebeck coefficient of this and other cells, in particular the popular cell with calomel electrodes. We review experimental results in the literature on cells with ion-exchange membranes, document the relatively large Seebeck coefficient, and explain with the help of theory its variation with electrode materials and electrolyte concentration and composition. The impact of the membrane heterogeneity and water content on the transported entropies is documented, and it is concluded that this and other properties should be further investigated, to better understand how all transport properties can serve the purpose of thermoelectric energy conversion.