A Data Professionals Community

Big Data IoT Transforming The Industry

With digital oilfield programs focusing on optimizing key E&P work processes, two of the hottest emerging digital technology trends, Big Data and the Internet of Things (IoT), are poised to make additional digital inroads into the business.


Google Books defines Big Data as, “A blanket term for any collection of datasets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications.” The IoT and its counterpart industrial IoT (IIoT) are defined by Autodesk Fusion Connect as, “The interconnection of uniquely identifiable embedded computing devices within the existing Internet infrastructure.” Key issues surround the emerging role of Big Data and the IoT in the industry.

Consumerization of IT

Underlying the industrial application of Big Data and the IoT is that today’s digital technology market is increasingly driven by consumer applications of IT. This is forcing cultural changes into industrial business environments, including increased expectations for personal use of hyper-connected, continuously advancing products and real-time access to information. Five years ago “bring your own device” would have been viewed as a naïve and unrealistic expectation, primarily driven by the “new generation” of oilpatch workers. Today companies acknowledge the reality of personal choice, and management teams are struggling with how to make “bring your own device” work in a secure manner.

There are going to be many more issues driven by consumerization of IT. For example, will the exponential growth in consumer IoT outflank IIoT to become the primary technology driver? The industry has seen this play in other digital technology areas. Consumerization is powerfully enabled by its ability to be distributed and to scale both up (through connectivity, clouds, etc.) and down (lower costs, ever greater permeation). Rapidly declining costs with rapidly expanding capabilities can create business imperatives for adoption of consumer- based technology. Moreover, distributed information systems are a better match to the physical world, distributed assets and real-time operations such as oil and gas. The practical application of Big Data and IoT to E&P will inevitably need to account for this fundamental technology trend.

Increasing scale and complexity

As oil and gas companies take in petabytes of data daily, improving business performance will critically depend upon the ability to identify and understand trends; correctly interpret geotechnical, production and operations data; and make effective operational decisions. The ability to access and draw insights from datasets is at the heart of profitability in an industry where success relies on properly forecasting future performance and keeping costs low. The challenge is that the Big Data volumes, potentially greatly expanded by implementation of the IoT, will produce an information system of such scale and complexity that it threatens to overwhelm existing systems. The data volume is growing, but a company cannot afford to get lost in datasets.

Companies face big challenges in quickly manipulating large volumes of data and mining them for relevant information. Analysis and insights must be delivered in near real time to support decision-making in key areas such as drilling and production optimization. Companies are tracking both structured data and unstructured data. Oil and gas companies must also deal with data from external sources to track service crews, truck traffic, equipment and water usage. The requirements for Big Data and the IoT are not coming from just one part of the industry but pose a challenge across the entire spectrum…Continue Reading


This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More