Educators recognize that group work and physical involvement with learning materials can greatly enhance the understanding and retention of difficult concepts. As a result, math manipulatives-such as pattern blocks and number lines-have increasingly been making their way into classrooms and children's museums. Yet without the constant guidance of a teacher, students can easily become distracted, confused, or frustrated. This paper describes how math games with tangible user interfaces can address this need. Tangible user interfaces allow the students to work together in a physical environment, while the underlying computer system provides the guidance that a teacher would. This paper discusses pedagogical principles and an approach to designing and developing games that utilize tangible technologies. A library of functions, designed specifically for tracking visual tags in math games, is presented. The paper concludes with examples of math games that have been prototyped with this library.