Surveillance and adjustments of fillbase and fillage are crucial for optimizing pump performance and maximizing production efficiency. Proper management of fillbase not only promotes longevity by preventing unnecessary wear and tear, but also helps in optimizing fluid handling efficiency, energy efficiency, rod-load management, and overall production of rod pumps. The primary objective of this paper is to propose a comprehensive and automated solution for surveillance and optimal management of fillbase with high number of wells. Our proposed method powered by combination of machine learning models enable efficient, data-driven monitoring and fillbase tuning. The pump cards (Position-Load on the rod) generated by the wellhead controllers along with other operational parameters including pump-speed, wellhead-pressure, temperature, etc. are first processed by a LightGBM classification model to classify cards as good or bad. If the card is bad, the operators are notified to help with supervision and taking remedial actions, and if the card is good, fillbase is predicted by a XGBoost regression model and the values are adjusted automatically based on the predictions. Different machine learning models were assessed for classifying pump cards and predicting fillage and fillbase. The best models with their respective hyperparameters were then tested in the field. These models were evaluated in real-time by senior operation specialists and feedback was collected. The card classification model had an accuracy score of approximately 99% on the test data and showed similar performance during real-time monitoring. The Fillbase model had R2 values of 98% and the fillbase model correctly detected the fillbase setting in around 95% of cases, as confirmed by senior operation specialists and production engineers. The controller calculates the fillage based on the given fillbase setting and there is an average deviation of around 4% from the fillage predicted by our model. The fluid rates recorded by the controllers after adjusting the fillbase were compared to well test rates, and there was a significant decrease (15%) in mean absolute error after implementing the ‘Auto Fill’ workflow, which serves as a quantitative validation for our models. Additionally, the model can adjust the fillbase settings even in the event of sudden changes in operating conditions, reducing the need for frequent manual interventions by operators. Using data driven systems and powerful machine learning models, the proposed approach aims to automate surveillance and fillbase management, reducing the need for manual effort from operators and production engineers. This process also enhances consistency in setting fillage or fillbase and allows for automated speed control of pumps, improving production and energy efficiency. The system is specifically designed to optimize operational productivity in large oil field settings with numerous wells by streamlining surveillance and fillbase adjustment processes.