Hyperspectral anomaly detection (HAD) as a special target detection can automatically locate anomaly objects whose spectral information are quite different from their surroundings, without any prior information about background and anomaly. In recent years, HAD methods based on the low rank representation (LRR) model have caught much attention, and achieved good results. However, LRR is a global structure model, which inevitably ignores the local geometrical information of hyperspectral image. Furthermore, most of these methods need to construct dictionaries with clustering algorithm in advance, and they are carried out stage by stage. In this paper, we introduce a locality constrained term inspired by manifold learning topreserve the local geometrical structure during the LRR process, and incorporate the dictionary learning into the optimization process of the LRR. Our proposed method is an one-stage algorithm, which can obtain the low rank representation coefficient matrix, the dictionary matrix, and the residual matrix referring to anomaly simultaneously. One simulated and three real hyperspectral images are used as test datasets. Three metrics, including the ROC curve, AUC value, and box plot, are used to evaluate the detection performance. The visualized results demonstrate convincingly that our method can not only detect anomalies accurately, but also suppress the background information and noises effectively. The three evaluation metrics also prove that our method is superior to other typical methods.