Osteosarcoma is the most normal kind of cancer that arises in bones, which appears on the surface to resemble earlier types of bone cells that assist in forging new bone tissues, but the tissue in osteosarcoma is weaker and softer than normal bone tissue. The usage of automated techniques for the detection of osteosarcoma has the potential to mitigate the obligations and burdens confronted by pathologists owing to its abundant quantity of cases. Artificial intelligence (AI) has an emerging progress in diagnostic pathology. In recent years, numerous studies using deep learning (DL) techniques to histopathological images (HI) have been published. While several studies claim higher accuracy, they might lack generalization and fall into the pitfall of overfitting owing to the wide range of HI. The study objective is to enhance the diagnosis and detection of osteosarcoma by employing computer-assisted detection (CAD) and diagnoses (CADx). Technique like convolutional neural networks (CNN) make better prognoses for patient conditions and considerably reduce the surgeon's workload. CNN needs to be trained on the massive quantity of data to accomplish a remarkable performance. Therefore, the study presents a novel Group Teaching Optimization Algorithm with Deep Learning-Driven Osteosarcoma Detection on Histopathological Images (GTOADL-ODHI) technique. The purpose of the GTOADL-ODHI technique is to examine the HIs for the detection and classification of osteosarcoma. To accomplish this, the GTOADL-ODHI algorithm applies the Gaussian filtering (GF) method for image pre-processed to become rid of the noise. Besides, the capsule network (CapsNet) model is utilized for the extractor of the feature vector. Furthermore, the hyperparameter selection of the CapsNet approach takes place using the GTOA. Finally, the self-attention bidirectional long short-term memory (SA-BiLSTM) model can be employed for osteosarcoma recognition and classification. The widespread experimental analysis of the GTOADL-ODHI method is tested on the benchmark datasets. The simulation validation reported the optimum solution of the GTOADL-ODHI algorithm related to existing systems concerning distinct aspects.