In this paper, we will study the chromatic number of a family of Cayley graphs that arise from algebraic constructions. Using Lang-Weil bound and representation theory of finite simple groups of Lie type, we will establish lower bounds on the chromatic number of a large family of these graphs. As a corollary we obtain a lower bound for the chromatic number of certain Cayley graphs associated to the ring of n × n matrices over finite fields, establishing a result for the case of SLn parallel to a theorem of Tomon [26] for GLn. Moreover, using Weil's bound for Kloosterman sums we will also prove an analogous result for SL2 over certain finite rings.2010 Mathematics Subject Classification. 20D60, 20G40.