Brain-Computer Interface (BCI) provides a direct communicating pathway between the human brain and the external environment. In the BCI systems, electroencephalography (EEG) signals are used to represent different cognitive patterns corresponding to various limb movements or motor imagery (MI) activities. However, EEG signals are multichannel in nature that require explicit information processing to alleviate the computational complexity of the BCI system. This paper represents a novel channel selection method to find effective EEG channels using Dynamic Channel Relevance (DCR) score. The proposed approach explores the minimum redundancy maximum relevance paradigm for selecting pertinent channels from raw ones. Firstly, the EEG signals are preprocessed using the Savitzky-Golay filter, and a sliding window approach is used to decompose them into chunks of fixed length. Further, three priorly known channels are used as candidate solutions for detecting new correlated channels. The selected channels are used to extract spatial-temporal features using the Multivariate Empirical Mode Decomposition (MEMD) method. The extracted features are used to discriminate four MI tasks (left hand, right hand, tongue, and foot) using the Support Vector Machine (SVM). The experiment is validated on three public EEG datasets (BCI Competition IV-2008 -IIA, BCI Competition IV-dataset 1, BCI competition III -dataset IVa). The results show that the proposed method achieved a superior classification accuracy (85.4% on dataset 1, 80.33% on dataset 2, and 85.20% on dataset 3) with a lesser number of channels compared to state-of-the-art methods. In addition, our method significantly reduced the computation time compared to other published results without compromising the classification accuracy. Topographical mapping between the selected channels and the cognitive regions showed that the central, frontal, and parietal lobe contributes to the execution of various MI tasks during physical activities.