High Curie temperature of 900 K has been reported in Cr-doped AlN diluted magnetic semiconductors prepared by various methods, which is exciting for spintronic applications. It is believed that N defects play important roles in achieving the high temperature ferromagnetism in good samples. Motivated by these experimental advances, we use a full-potential density-functional-theory method and supercell approach to investigate N defects and their effects on ferromagnetism of (Al,Cr)N with N vacancies (VN). We investigate the structural and electronic properties of the VN, single Cr atom, Cr-Cr atom pairs, Cr-VN pairs, and so on. In each case, the most stable structure is obtained by comparing different atomic configurations optimized in terms of the total energy and the force on every atom, and then is used to calculate the defect formation energy and study the electronic structures. Our total energy calculations show that the nearest substitutional Cr-Cr pair with the two spins in parallel is the most favorable and the nearest Cr-VN pair makes a stable complex. Our formation energies indicate that VN regions can be formed spontaneously under N-poor condition because the minimal VN formation energy equals -0.23 eV or Cr-doped regions with high enough concentrations can be formed under N-rich condition because the Cr formation energy equals 0.04 eV, and hence real Cr-doped AlN samples are formed by forming some Cr-doped regions and separated VN regions and through subsequent atomic relaxation during annealing. Both of the single Cr atom and the N vacancy create filled electronic states in the semiconductor gap of AlN. N-vacancies enhance the ferromagnetism by adding 1µB to the Cr moment each, but reduce the ferromagnetic exchange constants between the spins in the nearest Cr-Cr pairs. These calculated results are in agreement with experimental observations and facts of real Cr-doped AlN samples and their synthesis. Our first-principles results are useful to elucidating the mechanism for the ferromagnetism and exploring high-performance Cr-doped AlN diluted magnetic semiconductors.