Simultaneous localization and mapping (SLAM) have been an essential requirement for the autonomous operation of mobile robots for over a decade. However, in the wake of recent developments and successes of deep neural networks and machine learning, the conventional task of SLAM is gradually being replaced by Semantic SLAM. Extracting semantic information (such as object information) from sensory data can enable the robot to distinguish different environmental regions beyond the conventional grid assignments of free and occupied. This level of scene awareness is essential for performing higher-level navigation and manipulation tasks and enhancing human-robot interactions. This paper presents an integrated framework that not only builds such maps of indoor environments but also facilitates the execution of 'Go to object' tasks with high-level user input. We also present a method to extract meaningful endpoints of navigation based on object class. Our modular stack leverages well-known object detectors (YOLOv3), RGB-D SLAM techniques (RTAB-Mapping) and local navigation planners (TEB) to perform ObjectGoal navigation tasks. We also validate the results of experiments in real environments.