College Japanese teaching is the cradle of Japanese professional development. With the rising frequency of interactions with Japan in our country in the fields of politics, economy, trade, and other fields, Japanese as a professional discipline is exhibiting popularization, universalization, and folkization features. However, the ever-emerging trend of Japanese as an application tool for worldwide communication has rendered professional Japanese instruction in colleges insufficient. To meet the needs and growth of society, as well as to address the problems of teacher shortages and a lack of attention to students’ fundamental knowledge in the reform of Japanese language teaching in colleges, the study uses an expert system as the theoretical foundation and combines BP neural network technology to design an auxiliary teaching system with a friendly interface, strong versatility, and extensibility for Japanese language teachers and students. Teachers can use this technique to organize the test by classifying and summarizing the test questions based on the knowledge points and complexity of the questions. Students can utilize this system to learn on their own, and by identifying weak links in their knowledge points, they can practice more effectively and create a multiplier impact with half the work. Finally, the whole system is designed and implemented in accordance with the software development process. It has been demonstrated that the system can provide realistic results and has good application value after a huge quantity of data testing and operation.