The vision of this study is to find a way for increasing phosphorus release from bone char. An incubation experiment was conducted to study the effect of co-applying different nitrogen fertilizer types with bone char (BC) on the availability and distribution of phosphorus in calcium carbonate-rich soil. The experiment contains the following treatments: soil without any nitrogen fertilizer (BC only), soil + ammonium sulfate (BC + AS), soil + ammonium nitrate (BC + AN), and soil + urea (BC + U). Bone char was added to all treatments at a dose of 4 g kg−1 soil. Co-applying bone char with all nitrogen fertilizers caused a significant decrease in pH and increased significantly phosphorus availability in the soil. The concentrations of soil available phosphorus increased from 8.05 mg kg−1 soil for BC treatment to 8.99, 8.90, and 10.16 mg kg−1 soil for BC + AS, BC + AN, and BC + U treatments, respectively, at the end of incubation. Significant increases in soil available phosphorus were observed with increasing incubation periods in all treatments. The effectiveness of the treatments on the soil available phosphorus increase was in the order of BC + U > BC + AS > BC + AN > BC. Nitrogen fertilization treatments significantly increased the NaHCO3-Pi concentrations compared to the BC treatment. Urea application to the soil increased significantly the NaOH-Pi fraction compared to other treatments at day 10 of incubation. Changes in HCl-Pi and Res-P fractions were non-significant in all treatments under nitrogen fertilization. According to the findings, co-applying bone char with nitrogen fertilizers is an agronomic practice that improves phosphorus availability in calcium carbonate-rich soil, thence it is preferable to add urea rather than other nitrogen fertilizers. This study explores a sustainable management strategy to find cost-effective and environmentally friendly alternatives to phosphate fertilizers.