Growing battery use in energy storage and automotive industries demands advanced Battery Management Systems (BMSs) to estimate key parameters like the State of Charge (SoC) which are not directly measurable using standard sensors. Consequently, various model-based and data-driven approaches have been developed for their estimation. Among these, the latter are often favored due to their high accuracy, low energy consumption, and ease of implementation on the cloud or Internet of Things (IoT) devices. This research focuses on creating small, efficient data-driven SoC estimation models for integration into IoT devices, specifically the Infineon Cypress CY8CPROTO-062S3-4343W. The development process involved training a compact Convolutional Neural Network (CNN) and an Artificial Neural Network (ANN) offline using a comprehensive dataset obtained from five different batteries. Before deployment on the target device, model quantization was performed using Infineon’s ModusToolBox Machine Learning (MTB-ML) configurator 2.0 software. The tests show satisfactory results for both chosen models with a good accuracy achieved, especially in the early stages of the battery lifecycle. In terms of the computational burden, the ANN has a clear advantage over the more complex CNN model.