Abstract-Traffic sign detection and recognition is a difficult task, especially if we aim at detecting and recognizing signs in images captured under poor conditions. Complex backgrounds, obstructing objects, inappropriate distance of signs, shadow, and other lighting-related problems may make it difficult to detect and recognize signs in both rural and urban areas. In this paper we propose and test a system that employs image pre-processing, color filtering, color segmentation for traffic sign detection at the detection stage, feature extraction and trained neural networks for unique identification of signs at the recognition stage. The traffic sign detection and recognition system has been tested on actual roadside images taken under poor conditions. The images were selected in order to test the efficiency of the system under challenging conditions of inappropriate distance, traffic sign size, poor lighting and complex background. Suggestions are made for improving the performance of the system.