Lane detection is important for autonomous vehicles. For this reason, many approaches use lane boundary information to locate the vehicle inside the street, or to integrate GPS-based localization. Advanced driverassistance systems are developed to assist drivers in the driving process reducing road accidents. In this work, we present an end-to-end system for lane identification, clustering and classification, based on two cascaded neural networks, that runs in real-time. The first step is camera calibration which is used to remove the effect of lens distortion. Then a canny edge detection algorithm finds the edges of the images. Then the region of interest (ROI) is selected. The ROI is actually based on the rectangular shape appearing at the bottom of the image. ROI removes the unwanted region in the image. The potential lane markers are then determined using the Hough transform to analyze lane boundaries. Once the lane pixels are found, these pixels are continuously scanned to obtain the best linear regression analysis.It is qualified to be applied on highways and urban roadways. It also has been successfully verified in sunny, and rainy conditions for both day and night.