This research investigates the performance of edge and texture features, and Artificial Neural Network (ANN) and Support Vector Machine (SVM) classifiers for traditional house recognition. Malaysia is well known for its wealthy segments of cultural heritage and one of it is the traditional Malay houses. Capturing images of houses from the natural scenes are affected by illumination changes, scale and rotation invariants that makes the recognition process very challenging. Feature and classifier selection plays an important role in this recognition process. Thus, two comparative studies have been conducted, that are, between edge and texture features and between ANN and SVM classifiers to determine which combination will produce better recognition performance. Experimental results show that edge features with ANN achieve good recognition result.