A security system based on biometrics of a person depends on the features of a person. These can be physical features or behavioral features. User authentication is necessary for security control system and intelligent human computer interface, so that only the real user accesses it. We present a method for the authentication of person with help of multimodal data which includes face and speech. Agent based system has been used for fusion of these data which gives us a more reliable, efficient and secure biometric system. A method for face detection for recognition of faces has been given. For speech recognition Artificial Neural Network has been used. Both these combined together identify the speaker. This multimodal biometric method will give highly reliable secure system.