<span lang="EN-GB">Communication is very important for people including adults and babies to exchange their thoughts, views and information. They can express their thoughts verbally and non-verbally. Baby sign language is non-verbal type that consists of a set of standard and organised form of hand gestures motion to represent different word. This language allows the children who have hearing impairment problem to communicate with others. In addition, it can also be taught to normal babies to convey their need before they have the ability to speak. It is very challenging for adults to understand babies and children with hearing impaired problem. Thus, a real-time hand gesture recognition system has been developed for adult to interactively learn the baby sign language. Firstly, the face region was detected in order to use the skin colour model of the face to detect the hand. After that, the hand shape was determined by finding the palm centre and convex hull. Convexity defects which representing gaps between the fingers were used to determine the number of fingertips whereas the movement of the hand was detected based on motion analysis. By setting the background colour to blue and adding extra light in front of the laptop, the system successfully obtained an accuracy higher than 85% rate for sign “mom”, “eat” and “milk” when tested on Malay, Chinese and Indian races in Malaysia.</span>