A brain tumor is an abnormal growth of a mass or cell in the brain. Early diagnosis of the tumor significantly increases the chances of successful treatment. Artificial intelligence-based systems can detect the tumor in early stages. In this way, it could be possible to detect a tumor and resolve this problem that may endanger human life early. In the study, the partial correlation-based channel selection formula was presented that allowed the selection of the most prominent feature that differ from the other studies in the literature. Additionally, the multi-channel convolution structure was proposed for the feature network phase of the Faster R-CNN architecture. In the proposed model, the most prominent features were obtained from the multi-channel selection structure in the feature network phase with the channel selection formula in the channel selection layer. The architecture was applied for the early detection of possible brain tumors, which are a severe risk for human life. Within the present study, the brain tumor was classified applying the proposed multi-channel Faster R-CNN based model with three different open-access datasets. VGG-16, Faster R-CNN, DenseNet-201, Resnet-50, and SRN models, which are popular deep learning architectures, were applied to the same problem to compare the results and demonstrate the efficiency of the proposed model. Accuracy, sensitivity, and processing times of the applied methods were measured to demonstrate the models' performance and efficiency. As a result, the highest accuracy rates were obtained using the proposed model as 98.31%, 99.6%, and 99.8% for three datasets. In addition, it was compared with related studies in the literature to demonstrate the proposed model's applicability. The proposed model's accuracy and performance proved to be higher than in the other studies.