In this paper, we evaluate the security properties of multi-factor biometric authentication (MFBA) and define the notion of User Privacy, where the biometrics is assumed as a set of features that can be either ordered or unordered depending on the biometric modality. We propose efficient schemes for MFBA, which do not incorporate secure sketches, thus, different template extraction methods are applicable. We formally describe the security model for MFBA, where the computations are performed in the encrypted domain but without requiring a decryption key for the authentication decision. Thus, leakage of the secret key of any system component does not affect the security of the scheme as opposed to the current biometric systems involving cryptographic techniques.