Analog meters equipped with one or multiple pointers are wildly utilized to monitor vital devices' status in industrial sites for safety concerns. Reading these legacy meters autonomously remains an open problem since estimating pointer origin and direction under imaging damping factors imposed in the wild could be challenging. Nevertheless, high accuracy, flexibility, and real-time performance are demanded. In this work, we propose the Vector Detection Network (VDN) to detect analog meters' pointers given their images, eliminating the barriers for autonomously reading such meters using intelligent agents like robots. We tackled the pointer as a two-dimensional vector, whose initial point coincides with the tip, and the direction is along tail-to-tip. The network estimates a confidence map, wherein the peak pixels are treated as vectors' initial points, along with a two-layer scalar map, whose pixel values at each peak form the scalar components in the directions of the coordinate axes. We established the Pointer-10K dataset composing of realworld analog meter images to evaluate our approach due to no similar dataset is available for now. Experiments on the dataset demonstrated that our methods generalize well to various meters, robust to harsh imaging factors, and run in real-time.Impact Statement-From the automatic meter recognition perspective, this work proposes an accurate, flexible, and readyto-use pointer detection algorithm that contributes to dealing with legacy pointer-type meters, which still serve in industrial sites to date and require extensive human inspection. Adopting this technique to automatic agents could save human labor and promote the effectiveness and efficiency of getting data. It also inspires particular object detection problems, wherein the object could be considered a vector whose position is concerned, and its orientation also matters. The proposed Vector Detection Network is agnostic to the semantic meaning of the vectors it detects. Thus it could also be used to detect objects of this kind, such as the branch of plants, helping the agriculture robots do accurate trimming.