In this paper, the problem of fixed-time attitude consensus control is addressed for a group of rigid spacecraft in the presence of inertia uncertainties and external disturbances. By applying the adaptive technique and neural network approximation technique to handle the disturbances and uncertainties, an adaptive neural network-based distributed protocol is proposed to achieve attitude consensus control for multiple rigid spacecraft. The proposed distributed attitude consensus protocol is composed of a group of distributed fixed-time observers for followers to estimate the leader's information and an adaptive neural network-based fixed-time sliding mode control law to realize attitude tracking control. Rigorous proofs are provided to demonstrate that the estimation errors of the proposed observers are convergent in a fixed time. Further, it is also proven that attitude tracking errors reach some adjustable regions in a fixed time under the proposed attitude consensus protocol. Numerical simulations are conducted to illustrate the performance of the proposed distributed attitude consensus protocol.