Fault injection attacks pose a serious threat to many cryptographic devices. The security of most cryptographic devices hinges on a key block called modular division (MD) over a prime field. Although a lot of research has been done to implement the MD over a prime field in hardware efficiently, studies on secure architecture against fault injection attack are very few. A few of the studies that focused on secure architecture against fault injection attack can only detect faults but not locate faults. In this regard, this paper designs a novel secure architecture for the MD over a prime field, which can not only detect faults, but also can locate the error processing element. In order to seek the best optimal performance, four word-oriented systolic structures of a main function module (MFM) were designed, and three error detection schemes were developed based on different linear arithmetic codes (LACs). The MFM structures were combined flexibly with the error detection schemes. The time and area overheads of our architecture were analyzed through the implementation in an application-specific integrated circuit (ASIC), while the error detection and location capabilities of our architecture were demonstrated by C++ simulation, in comparison to two existing methods. The results show that our architecture can detect single-bit error (SBE) with 100% accuracy and locate the erroneous processing element (PE), and correctly identify most of the single PE errors and almost all of the multi-PE errors (when there are more than three erroneous PEs). The only weakness of our architecture is the relatively high time and area overhead ratios.