Object detection from image is more challenging and integral part in the inter-discipline area of computer vision. The computer vision is highly attractive in many applications like human pose estimation, instance segmentation, recognizing action, disease predictions object prediction and many more applications. The traditional method of detecting objects from the images is done using bounding boxes with labels. It suffers from the overlapping of the boxes with various smaller objects, which leads to accuracy issues in detection problems. Hence, machine learning techniques are used to detect the relevant objects from the image using center point to avoid the nonmaximal suppression in bounding box. To accurately identify images, an U-Net architecture based object detection method is proposed. In this model, it effectively uses semantic level segmentation and instance segmentation. This system effectively identifies all the objects present in the given image using the efficient hybrid segmentation models and Gromov Hausdroff distance measure. For experimentation, two data sets are used for evaluation of the model to identify all categories of objects from the image. The proposed model achieves an accuracy of 91.8% and reliable when compared to existing effective object detection algorithms like fully convolution network (FCN), YOLO (you only look once) and mask region based-convolutional neural network (mask R-CNN) model.