Abstract. We present a novel system for vision based anomaly detection in real-time environments. Our system uses an event-based vision sensor consisting of asynchronously operating pixels that is inspired by the human retina. Each pixel reports events of illumination changes, are processed in a purely event-based tracker that pursues edges of events in the input stream. The tracker estimates are used to determine whether the input events originate from anomalous or regular data. We distinguish between the two cases with a Growing Neural Gas (GNG), which is modified to suite our event-based processing pipeline. While learning of the GNG is supervised and performed offline, the detection is carried out online. We evaluate our system by inspection of fast-spinning cogwheels. Our system achieves faster than real-time speed on commodity hardware and generalizes well to other cases. The results of this paper can be applied both to technical implementations where high speed but little processing power is required, and for further investigations into event-based algorithms.