Online handwriting classification has become an open research problem as it serves as a preliminary step for handwriting recognition systems and applications in several other fields. This paper aims to extend the current trends and knowledge with multiple contributions in handwriting classification using Spatio-temporal information. Firstly, it enriches the annotations of several publicly available online handwriting datasets, SenseThePen, IAM-onDB, IAMonDo, to be used for online handwriting classification and recognition tasks, i.e., stroke, sequence, and line level. The enriched annotations of the datasets extend their functionality for online handwriting classification at different levels for further research analysis. In addition to enrichment, it also unifies the annotation level across the datasets, which enables the research community to benchmark proposed methods for comparative analysis using multiple datasets. All the datasets with enriched annotations are made publicly available for the research community as part of the IAMonSense dataset. Moreover, this paper presents a comprehensive benchmark of these datasets using multiple deep neural networks such as traditional convolutional IAMonSense: Multi-level Handwriting classification neural networks, graph neural networks, attention-based neural networks, and transformers. These benchmarks can be used later on for further development in the field of online handwriting classification.