Left ventricle (LV) segmentation is essential to identify the cardiac functions for treating cardiovascular disorders. Cardiovascular magnetic resonance (CMRI) imaging is a non‐invasive technique for diagnosing cardiovascular diseases. CMRI is widely used to assess the functional integrity of the left and right ventricles for detecting changes in myocardial structure. Clinical parameters of the LV are often retrieved from CMRI scans, such as LV volumes, and ejection fraction. Moreover, manually segmenting cardiac diseases and evaluating such functions is time‐consuming and difficult for medical professionals. Deep learning networks require a lot of time, cost, and knowledge. To overcome this issue, a novel Edge and Shape feature‐based Fully Convolutional Neural Network (ES‐FCN) has been proposed for automatic LV segmentation. The ES‐FCN model segments MRI images based on edge maps instead of using gray‐scale images, which accelerates the performance of the FCN. The fuzzy‐based canny edge detection algorithm leverages fuzzy logic to detect structural changes in the LV and generate binary images. Additionally, binary‐valued kernels are used for convolution operations, where the binary values are influenced by biases derived from edge map shape descriptors. In ES‐FCN, bias values are learned from ground truth segmentation. The proposed ES‐FCN model achieves the Jaccard indices of 0.9484 ± 0.0188 for the ACDC dataset and 0.9476 ± 0.0237 for the LVSC dataset, and the dice index of 0.9319 ± 0.0188 for the ACDC dataset and 0.9314 ± 0.0237 for LVSC dataset respectively. The experimental results also reveal that the proposed ES‐FCN model is faster and requires minimal resources compared to state‐of‐the‐art models.