Conventional fertilizer management can destroy the structure of soil. Replacing chemical fertilizers with organic fertilizers can improve soil quality and nitrogen use efficiency. We aimed to study the effects of organic fertilizer substitutions for chemical nitrogen fertilizer on soil fertility and nitrogen use efficiency in order to clarify the effectiveness of the available nutrient management measures in improving soil quality and increasing foxtail millet yield. A field experiment was carried out over two consecutive years, and a total of six treatments were set up: no fertilizer (CK), chemical nitrogen fertilizer alone (N), the substitution of 25% of chemical nitrogen fertilizer with bio-organic fertilizer (N25A1), the substitution of 25% of chemical nitrogen fertilizer with fermented mealworm manure (N25B1), the substitution of 50% of chemical nitrogen fertilizer with bio-organic fertilizer (N50A2), and the substitution of 50% of chemical nitrogen fertilizer with fermented mealworm manure (N50B2). The results of this study show the following: (1) Compared with chemical nitrogen fertilizer, the substitution of organic fertilizer for nitrogen fertilizer reduced the bulk density and solid phase of the soil, and it increased the total porosity, water content, liquid phase, and gas phase of the soil. (2) Compared with nitrogen fertilizer, the use of an organic fertilizer increased the contents of nitrate nitrogen, ammonium nitrogen, and total nitrogen in the soil by 13.59~52.56%, 4.47~18.27%, and 4.40~12.09%, respectively. The content of alkaline nitrogen increased by 1.70~32.37%, and the contents of soil available potassium, available phosphorus, and organic matter also increased. (3) The activities of sucrase, urease, glutaminase, and asparaginase were improved by replacing chemical nitrogen fertilizer with organic fertilizer. The N25 treatments performed better than the N50 treatments, and fermented mealworm manure performed better than biological organic fertilizer. (4) A moderate application of organic fertilizer (N25) can increase the grain yield, ear weight, grain weight, and 1000-grain weight of foxtail millet, whereas excessive application of organic fertilizer (N50) can reduce foxtail millet yield. (5) Replacing chemical nitrogen fertilizer with organic fertilizer can improve the agronomic use efficiency, physiological efficiency, biased productivity, harvest index, and apparent use efficiency of nitrogen fertilizer. In this study, the substitution of 25% of chemical nitrogen fertilizer with fermented mealworm manure was the best combination for restoring crop productivity and soil quality.