Osteoarthritis (OA) of the knee is a common cause of activity restriction and physical impairment in elderly people. Early identification and treatment can help delay the progression of OA. Physicians' visual examination rating is objective, varies across interpretation, and is heavily reliant on their expertise. We use two machine learning approaches (CNN) in this article to automatically estimate the severity of knee OA as described by the Kallgren- Lawrence (KL) grading system. To begin, we use a customized one-stage YOLOv2 network to recognize kneecap based on the size of knee joints scattered in X-ray pictures with poor contrast. Second, we use a new customizable arbitrary loss to fine-tune its most famous Cnn architectures, spanning ResNet, VGG, and DenseNet versions, as well as InceptionV3, to categorize the collected knee joint pictures. To be more explicit, we provide a stronger penalty to misrepresentation with a greater difference between the predicted and actual KL grade, driven by the ordinal character of the knee KL grading assignment. The Osteoarthritis Institute (OAI) collection is used to evaluate the basic X-ray pictures. Under the Jaccard index criterion of 0.75, we acquire a mean Jaccard index of 0.858 and a recall of 92.2 percent for knee joint identification. The fine-tuned VGG-19 model with the provided linear loss achieves the greatest generalization ability of 96.7 percent and mean standard deviation (MAE) of 0.344 on the knee KL grading task. Both knee joint identification and knee KL assessment are at the cutting edge of technology Keywords: Osteoarthritis (OA), Deep Learning, X-rays, CNN