Customer conveys their opinion in natural language about an entity. Applying sentiment analysis to those reviews is a very complex task. The significance terms that are influencing the polarity of a review are not examined. The terms that are having contextual meaning are not recognized which are present across multiple sentences in a review. To address the above two issues, we have proposed an Attention-based Convolution Bi-directional Recurrent Neural Network (ACBRNN). In this model, two convolution layer captures phrase-level feature, while Self-Attention in the middle assigns high weight to the significant terms and Bi-directional GRU performs a conceptual scanning of review through forward and backward direction. We have conducted four different experiments viz., Unidirectional, Bidirectional, Hybrid and Proposed model on IMDB dataset to show the significance of the proposed model. The proposed model has obtained an F1 score of 87.94% on IMDB dataset which is 5.41% higher than CNN. Thus the proposed architecture performs well while comparing with all other baseline models.