Artificial Intelligence (AI) is at the top of the agenda for education leaders today in educating the next generation across the globe. However, public understanding of AI technologies and how to define AI literacy is under-explored. This vision poses upcoming challenges for our next generation to learn about AI. On this note, an exploratory review was conducted to conceptualize the newly emerging concept "AI literacy", in search for a sound theoretical foundation to define, teach and evaluate AI literacy. Grounded in literature on 18 existing peer-reviewed articles, this review proposed four aspects (i.e, know and understand, use, evaluate, and ethical issues) for fostering AI literacy based on the adaptation of classic literacies. This study sheds light on the consolidated definition, teaching, and ethical concerns on AI literacy, establishing the groundwork for future research such as competency development and assessment criteria on AI literacy.