Human agents can acquire knowledge and learn through argumentation. Inspired by this fact, we propose a novel argumentation-based machine learning technique that can be used for online incremental learning scenarios. Existing methods for online incremental learning problems typically do not generalize well from just a few learning instances. Our previous argumentation-based online incremental learning method outperformed state-of-the-art methods in terms of accuracy and learning speed. However, it was neither memory-efficient nor computationally efficient since the algorithm used the power set of the feature values for updating the model. In this paper, we propose an accelerated version of the algorithm, with polynomial instead of exponential complexity, while achieving higher learning accuracy. The proposed method is at least 200× faster than the original argumentation-based learning method and is more memory-efficient.