The Oil & Gas industry is under incredible competitive pressure right now. The price of crude oil is volatile and sending shockwaves through the industry.

As a result, organisations are re-thinking their operational processes with a big focus on enhancing efficiency and innovation – not only is it important to cut costs, but to find ways to drive new areas of growth. And, a major asset currently being explored is data.

The industry has an avalanche of information – from seismic exploration data to geological scans dating back decades, to data on supply chain and distribution and data on commercial markets, pricing and risk. But how are Oil & Gas organisations using it now and into the future?

In my conversations with leaders in this sector, there are three key data trends in this sector right now. These include:

1)     Better scale for seismic data – the industry is under immense pressure to find new sources of oil and gas. Collecting and analysing data from mapping the ocean floor is the go-to process for this business critical issue allowing them to look for patterns and contours that might indicate the presence of valuable resources. However, with file sizes from the scans and the cumulative volume of data stored growing exponentially year by year, a big scale out platform is required to add greater capacity. When it comes to seismic exploration, the images are far more detailed than they were even 3-5 years ago; a single scan can return individual file sizes of three to four terabytes, which means Oil & Gas firms have to boost their capacity by petabytes each year to cope with the volume of data they’re creating. And indeed, as the sophistication of the search algorithms grows, there’s increasing value in looking back at archived scans again – retrieving data off tape to see if an old scan has a hidden mystery that can be made plain through modern analysis techniques. This means they can now have even more scale available to include all archive data in a more accessible storage tier than the slow tape backups many have been on. Ultimately, this is all about ensuring that if a drill site is selected, it is more likely to be profitable and less likely to cost millions for limited output; something that is increasingly vital as the pool of ‘easy’ oil and gas resources are exhausted. Some oil companies are already leading the way with this using file-based storage to save and retrieve very large amounts of data related to exploration, extraction and production of oil and gas.

2)     Data lakes – increasingly, Oil & Gas companies are finding that they have mountains of data in different repositories. This makes analytics complex and performance slow. Bringing these sources together into a single repository will support the development of a new class of analytics applications. While it might seem like a complex task to do this level of consolidation, there are new, straightforward ways this can be achieved. A data lake approach will allow businesses to achieve this level of consolidation while benefitting from reduced OpEx costs and improved efficiency and performance. For example, one of our European-based, international oil customers is using storage in a transformative way, collecting data in a single, common platform that can be scaled up and down as necessary and managed easily, significantly reducing IT costs.

3)     Analytics – these first two trends inform the third, which is that of making use of analytics to drive other efficiencies in the sector. In particular this is around running more sophisticated search analytics algorithms on seismic data, but the Oil & Gas industry has a wealth of other data – surveillance data on large industrial sites, which could be used to enhance security or key processes, machine data from vast networks of sophisticated processing, extraction and distribution technology, which could be used to inform predictive maintenance and reduce costly and dangerous unscheduled downtime. Properly exploiting this data is a rising priority for firms facing off an increasingly competitive and volatile commercial environment.

The big data needs of the Oil & Gas sector continue to be set by commercial imperative: use the data available to be more efficient and effective and cut potential and actual waste out of its daily operations. The leaders of tomorrow will be those swiftest to capitalize on the opportunity data represents. In few places is the phrase ‘data is the new oil’ more true than in the oil and gas industry itself.