This paper describes our approach to solve Semeval task 3: EmoContext; where, given a textual dialogue, i.e., a user utterance along with two turns of context, we have to classify the emotion associated with the utterance as one of the following emotion classes: Happy, Sad, Angry or Others. To solve this problem, we experiment with different deep learning models ranging from simple LSTM to relatively more complex attention with Bi-LSTM model. We also experiment with word embeddings such as ConceptNet along with word embeddings generated from bi-directional LSTM taking input characters. We fine tune different parameters and hyper-parameters associated with each of our model and report the micro precision, micro recall and micro F1-score for each model. We identify the Bi-LSTM model, along with the input word embedding taken as the concatenation of the embeddings generated from the bidirectional character LSTM and ConceptNet embedding, as the best performing model with a highest micro-F1 score over the test set as 0.7261.