Let G be a finite abelian group, let E be a subset of G, and form the Cayley (directed) graph of G with connecting set E. We explain how, for various matrices associated to this graph, the spectrum can be used to give information on the Smith normal form. This technique is applied to several interesting examples, including matrices in the Bose-Mesner algebra of the Hamming association scheme H(n, q). We also recover results of Bai and Jacobson-Niedermaier-Reiner on the critical group of a Cartesian product of complete graphs.2010 Mathematics Subject Classification. 05C50.