Combining the Power of IoT and Big Data to Unleash the Potential of Digital Oil Field

Abstract

The objective of this paper is to demonstrate the process of unleashing the potential of digital oil fields by combining the power of Big Data platform with the Internet of Things (IoT). This new method enables efficient machine learning training utilizing Big Data and real-time scoring against mathematical models for predicting future outcomes. Digital oil fields have a diverse set of IoT devices that measure important field metrics in real-time, such as downhole pressure, temperature and oil rate. A typical digital oil well is equipped with many equipment such as Multiphase Flow Meter, Electrical Submersible Pump, and Permanent Downhole Monitoring Systems. Those equipments have several sensors generating a huge volume of data every second. In order to enable data scientists to analyze this huge amount of data streaming from various data sources, a data engineering pipeline was built. This pipeline combines data from various real-time and historical data repositories along with a master relational database in order to provide a consistent and clean analytics database for data scientists. This method saves data scientists the trouble of manually preparing and cleaning data from different datasets. Furthermore, by utilizing the analytics database cluster for machine learning, data scientists were able to use bigger data sets for training their models which can improve the accuracies of the models. As part of the solution, a scoring engine was built which consumes real-time data feed from the digital oil fields and performs real-time predictions and scoring utilizing machine learning models. The new architecture significantly improved the productivity of data scientists by allowing them to focus on building models and not to have to worry about data plumbing and deployment of the model to the field. Moreover by utilizing bigger data sets, models accuracies was improved considerably. Finally by integrating the models with IoT real-time data stream, field engineers can see and act on the models’ predictions in a timely manner. This architecture and methodology combines different technology domains (IoT and Big Data) with unique solution to bring value to the Oil and Gas producing & production business function.

The role of data management in the Industrial Internet of Things

“…This new method enables efficient machine learning training utilizing Big Data and real-time scoring against mathematical models for predicting future outcomes. AlBar et al 43 stated that digital oil fields have a variety of IoT devices that measure critical field metrics in real time, such as downhole pressure, temperature, and oil rate. A typical digital oil well is equipped with equipment such as a multiphase flow meter, electrical submersible pump, and permanent downhole monitoring systems.…”

“…To overcome the issues of big data and utilize the analytics database cluster for machine learning, a scoring engine was proposed, which would consume a real-time data feed from the digital oil fields and perform real-time predictions and scoring, utilizing machine learning models. 43 Al-Naser and Al-Habib 44 discussed the challenges of oil and gas exploration and proposed an IR4 method for a fourth industry revolution in oil and gas. The methods are composed of three pillars: first, alignment is required among entities within a corporate to overcome challenges and establish a roadmap with common goals; second, awareness needs to be raised among all stakeholders; and third, empowerment is required, including in computing infrastructure, data access, and across domain collaboration.…”

Diagnosis of Operating Conditions of the Electrical Submersible Pump via Machine Learning

“…These technologies empower oil and gas facilities with detection equipment to enable real-time monitoring of wells and other assets in an oil field. The existence of such tools with digital world connection is essential for the industry [ 1 ].…”