Tag Archives: oilfield

Managing Data For The Sake Of Managing Data Or Are You Making a Difference?

Leave a reply

A client and now dear friend of mine told me once “We are not managing data for the sake of data management, we are doing it to support the business.” We connected immediately, and I took this as a sign that she would achieve great things for her company.

Supporting the business is the only reason to justify an IM group in an E&P company. But how does an Information Management connect (and prove the value of) enterprise initiatives that may take years to complete, to business operations that fluctuate with commodity prices?

Let’s look at the typical experience of many companies in the past few years:

When oil prices hovered for a lengthy period at approximately $100 a barrel, most businesses prioritized exploration and production to find new plays as fast as possible. Drill faster, complete faster, produce sooner, and find more. In this “growth” mode data came in, fast and furious. Companies threw in serious money to gather and analyze every data.

However, when oil prices hit $26 a barrel, “survival” mode kicked in. Most companies renegotiated their contracts and loans while trying to maintain base oil or gas production (revenue) at the least cost possible. Meeting or exceeding production targets became existential, not just good for business. Here in this mode, some data gathering slowed significantly, while the focus on producing wells and its facilities heightened.

Two entirely different sets of processes, completely different sets of priorities, could force totally different data management projects. In ‘growth’ mode, the focus was on the speed of processing directional surveys, logs, perforation, costs, and frac data. In ‘survival mode,’ the focus changed to Wells’ and facilities’ performance and integrity.

TECHNICAL DATA

All technical data is critical to an oil and gas company and should be available, boom or bust. It is also, entirely understandable that, in a world of limited resources, projects with the highest impact to the business are prioritized first. Shifting IM priorities with the change in commodity prices or change of business focus is not simple.

However, a good EIM strategy will support the business in any mode, growth, survival, or any other mode, with ease. The good news is, it is entirely possible to have such an EIM strategy, simply by focusing efforts towards organizational goals through growth and lean times alike. Also, today’s advancements in technology allow for increased agility in organizational response. But you got to have a strategy.

Once a strategy is defined and embraced, every information management project, for both structured and unstructured information, must advance the ball towards the goal, or just be killed. This is not as easy as it sounds, of course. It requires expertise and the dedicated effort. Prioritizing efforts, identifying weaknesses, choosing the right technology, all can help your organization grow faster in growth mode, as well as to swim, rather than tread water in survival mode.

Has your organization defined a strategy yet? Are they working to support the business, or are they just managing data for the sake of data management?

For greater clarity on your position, call or email us to schedule a complimentary strategy appraisal with one of our consultants.

To Build Fit Enterprise Solutions, Be Physical …

Leave a reply

The British and the Americans speak the same language. But, say “I have a flat” to a British, and it means something completely different than said to an American. The former would congratulate you, and the latter would feel sorry for you. Flat in the UK means an apartment. Flat in Houston means a flat tire. The same 4 words, arranged in the exact same way, in what is ostensibly the same language, and yet either speaker would confuse their audience, if the audiences were transposed.

It is the same thing in business – if you cross different corporate cultures or even inter-organizational boundaries, industry terminology might sound the same but mean very different things. Sometimes we think we are communicating, but we are not.

Why is this a problem? Because it is not possible to build an enterprise data management solution to serve all departments without addressing variations in expectations for the same word. Especially if the term in question is one that defines your organization’s values and activities.

“Sometimes we think we are communicating, but we are not”

In the corporate world of Energy E&P, the word “completion” means different things to the different departments. If you mention a “Completion” to a Landman, he will assume you are referring to the subsurface horizon for his leases (it is more complex than this, but for the sake of this argument we need not dive into details). If a “Completion” is referenced to a Production Engineer, she immediately thinks of the intersection of a wellbore and a reservoir horizon. To a Completion Engineer, the same term means the process of completing a well after the well has reached final depth.

As organizations’ data management practice become more matured, they start to make their way towards the right of the EIM MM (Enterprise Information Management Maturity Model). Centralized solutions such as Master Data Management (MDM) are important and are designed to serve ALL departments to break as many silos as possible.

Naturally, to create a centralized solution that addresses needs across the enterprise, you must first reach consensus on how to build that solution. The solution must ensure that the data is NOT LOST, NOT OVERWRITTEN and is FULLY CAPTURED and useful to EVERYONE. What is the best way to reach consensus without the risk of losing data?

Get Physical

To answer the above question, many agree that information systems need to be built based on the physical reality to gather granular data …

By basing your data on the physical world and capture granular data as practically possible, you not only make it possible to capture all related information but also possible to report it in any combination of grouping and queries. See the example in figure 1.

Focus on Enterprise Needs and Departmental needs will follow…

I have seen systems that ignore wellbore data yet store only completions per well. At other clients, I have seen systems that take short cuts by storing wells, wellbore and wellbore completion data in one line (this necessitates overwriting old completion data with new everytime there is a change), these are “fit-for-purpose” systems.  These are not enterprise level solutions, but rather serve departmental needs.

Too often systems are designed for the need of one group/department/purpose rather than for the need of the company as a whole. However, if the needs of the whole are defined and understood, both company and groups will have what they need and then some.

Let’s look at an example to clarify this position:

Figure 1 Multi lateral well

Figure 1 Multi lateral well

In Figure 1 above, how would you store the data for the well in your organization or your department? Would you define the data captured as one well, three bores, and three completions? Or maybe two completions? One?
Depending on your department or organizational definitions, any of the above definitions could be fit-for-purpose correct. Accounting systems might keep track of ONLY one completion if it made Payroll and Tax sense. While Land may only keep track of 2 completions if the bores are in two zones. An engineer would track three completions and will be specific to one completion per wellbore. The regulatory department may want you to report something entirely different.
How do we decide the number of completions so that the information is captured accurately, yet remains useful to a Landman, Accountant, Engineer, and Geoscientist? Build based on the physical reality and stay granular.
In Figure 1, physically speaking, we see one well with three paths (3 wellbores). Each bore has its own configuration that open to the reservoir (completions). In total, this well has three different ‘Completions’,  one ‘Completion’ for each of the horizontal bores.
Accounting can query how many different cost centers the well has, and depending on the production (and other complex rules) the answer could be three but it could be 1.  Depending on the lease agreement, Landman could get a result of one or 3 completions. An engineer can also easily query and graph this data to find the three pathways, and determine each completion job per wellbore.
While it could be argued that data needs to be presented differently to each department, the underlying source data must reflect the physical truth. After all, we cannot control what people call things and certainly cannot change the lingo.

Part 3 of 3: Are we progressing? Oil & Gas Data Management Journey the 2000s

1 Reply

The 1990’s shopping spree for applications produced a spaghetti of links between databases and applications while also chipping away the petro professional’s effective time with manual data entry. Then, a wave of mega M&As hit the industry in late 90s early part of the 2000s.

Mega M&As (mergers and acquisitions) continued into the first part of the 2000s, bringing with them—at least for those on the acquiring side – a new level of data management complexity.

With mega M&As, the acquiring companies inherit more databases and systems, and many physical boxes upon boxes of data. This influx of information proved to be too much at the outset and companies struggled – and continue to struggle – to check the quality of the technical data they’d inherited. Unknown at the time, the data quality issues present at the outset of these M&As would have lasting effects on current and future data management efforts. In some cases it gave rise to law suites that were settled in millions of dollars. 

Early 2000s

Companies started to experiment with the Internet.  At that time, that meant experimenting with simple reporting and limited intelligence on the intranet.  Reports were still mostly distributed via email attachments and/or posted in a centralized network folder.

I am convinced that it was the Internet that  necessitated cleaning technical data and key header information for two reasons: 1) Web reports forced the integration between systems as business users wanted data from multiple silo databases on one page. Often times than not, real-time integration could not be realized without cleaning the data first 2)  Reports on the web linked directly to databases exposed more “holes” and multiple “versions for the same data”;  it revealed how necessary it was to have only ONE VERSION of information, and that  had better be the truth.

The majors were further ahead but for many other E&P companies, Engineers were still integrating technical information manually, taking a day or more to get a complete view and understanding of their wells, excel was the tool moslty. Theoretically, with these new technologies, it should be possible to automate and instantaneously give a 360 degree-view of a well, field, basin and what have you. However, in practice it was a different story because of poor data quality.  Many companies started data cleaning projects, some efforts were massive, in tens of millions of dollars, and involved merging systems from many past acquisitions.

In the USA, in addition to the internet, the collapse of Enron in October 2001 and the Sarbanes–Oxley Act enacted in July 30, 2002, forced publicly traded oil and gas companies to document and get better transparency into operations and finances. Data management professionals were busy implementing their understanding of SOX in the USA. This required tightener definitions and processes around data.

Mid 2000s

By mid-2000s, many companies started looking into data governance. Sustaining data quality was now in the forefront.  The need for both sustainable quality data and data integration gave rise to Well Master Data Management initiatives. Projects on well hierarchy, data definitions, data standards, data processes and more were all evolving around reporting and data cleaning projects. Each company working on its own standards, sharing success stories from time to time.  Energetics, PPDM and DAMA organizations came in handy but not fully relied on.

Late 2000s

When working on sustaining data quality, one runs into the much-debated subject of who owns the data?  While for years, the IT department tried to lead the “data management” efforts, they were not fit to clean technical oil and gas data alone; they needed heavy support from the business. However the engineers and geoscientists did not feel it was their priority to clean “company-wide” data.

CIOs and CEOs started realizing that separating data from systems is a better proposition for E&P.  Data lives forever while systems come and go. We started seeing a movement towards a data management department, separate and independent from IT, but working close together. Few majors made this move in mid 2000s with good success stories others are started in late 2000s. First by having a Data Management Manager reporting to the CIO (and maybe dotted line to report to a business VP) then reporting directly to the business.

Who would staff a separate data management department?  You guessed it; resources came from both the business and IT.  In the past each department or asset had its own team of technical assistants “Techs” who would support their data needs (purchase, clean, load, massage…etc.) Now many companies are seeing a consolidation of “Techs” in one data management department supporting many departments.

Depending on how the DM department is run, this can be a powerful model if it is truly run as a service organization with the matching sense of urgency that E&P operations see. In my opinion, this could result in cheaper, faster and better data services for the company, and a more rewarding career path for those who are passionate about data.

Late 2008 and throughout 2009 the gas prices started to fall, more so in the USA than in other parts of the world. Shale Natural Gas has caught up with the demand and was exceeding it.  In April 2010, we woke up to witness one of the largest offshore oil spill disasters in history. A BP well, Macondo, exploded and was gushing oil.

For companies that put all their bets on gas fields or offshore fields, they did not have appetite for data management projects. For those well diversified or more focused on onshore liquids, data management projects were either full speed or business as usual.

 2010 to 2015 ….

Companies that had enjoyed the high oil prices since the 2007 started investing heavily in “digital” oilfields.  More than 20 years had passed since the majors started this initiative (I was on this type of project with Schlumberger for one of the majors back in 1998). But now it is more justifiable than ever. Technology prices have come down, systems capacities are up, network reliability is strong, wireless-connections are reasonably steady and more. All have come together like a prefect storm to resurrect the “smart” field initiatives like-never before. Even the small independents were now investing in this initiative. High oil prices were justifying the price tag (multiple millions of dollars) on these projects. A good part of these projects is in managing and integrating real time data steams and intelligent calculations.

Two more trends appeared in the first half of the 2010s:

  • Professionalizing the petroleum data management. Seemed like a natural progression now data management departments are in every company. The PPDM organization has a competency model that is worth looking into. Some of the majors have their own models that are tied to their HR structure. The goal is to reward a DM professional’s contribution to business’ assets. (Also please see my blog on MSc in Petroleum DM)
  • Larger companies are starting to experiment and harness the power of Big Data, and the integration of structured with unstructured data. Meta data and managing unstructured has become more important than ever.

Both trends have tremendous contributions that are yet to be fully harnessed.  The Big Data trend in particular is nudging data managers to start thinking of more sophisticated “analysis” than they did before .  Albeit one could argue that Technical Assistants that helped engineers with some analysis, were also nudging towards data analytics initiatives.

In December 2015, the oil price collapses more than 60% from its peak

But to my friends’ disappointment, standards are still being defined. Well hierarchy, while is seems simple to the business folks, getting it all automated and running smoothly across all types and locations of assets  will require the intervention of the UN.  With the data quality commotion some data management departments are a bit detached from the operations reality and take too long to deliver.

This concludes my series on the history of Petroleum Data Management. Please add your thoughts would love to hear your views.

For Data Nerds

  1. Data ownership has now come full circle, from the business to IT and back to business.
  2. The rise of Shale and Coal-bed Methane properties, fast evolution of field technologies are introducing new data needs. Data management systems and services need to stay nimble and agile. The old ways of taking years to come up with a usable system is too slow.
  3. Data cleaning projects are costly, especially when cleaning legacy data, so prioritizing and having a complete strategy that aligns with the business’ goals are key to success. Starting with well-header data is a very good start, aligning with what operations really need will require paying attention to many other data types, including mealtime measurements.
  4. When instituting governance programs, having a sustainable, agile and robust quality program is more important than temporarily patching problems based on a specific system.
  5. Tying data rules to business processes while starting from the wellspring of the data is prudent to sustainable solutions.
  6. Consider outsourcing all your legacy data cleanups if it takes resources away from supporting day to day business needs. Legacy data cleaning outsources to specialized companies will always be faster, cheaper and more accurate.
  7. Consider leveraging standardized data rules from organizations like PPDM instead of building them from scratch. Consider adding to the PPDM rules database as you define new ones. When rules are standardized data, sharing exchanging data becomes easier and cost effective.  

Part 2: Are we progressing? Oil & Gas Data Management Journey

3 Replies

In my previous blog, I looked back to the 1960s, 70s, and 80s, and how E&P technical data was generated and stored. For those three decades, data management was predominantly and virtually exclusively on paper. As I looked to the 90s, I found them packed with events that affected all areas of data value chain, from generation to consumption to archival.

Early 90s: Driving Productivity Forward

The early 90s continued one dominant theme from the late 1980s: the relentless drive for increased productivity throughout the business. This productivity focus coincided with three technological advancements that made their way into the industry. First of all, dropping costs of hardware with their growing capacity meant that computers became part of each office with meaningful scientific programs on them. Second, the increased capabilities of “networks” and “server/client” opened up new possibilities by centralizing and sharing one source of data. Third, proven success of relational databases and the SQL offered sophisticated ways to access and manipulate more data.

All this meant that, by the early 90s, engineers and the majority of geoscientists were able to do an increasing portion of their work on their own computers. At that time, the world of computer was divided into two; UNIX for G&G professionals and PC for the rest. Despite the divide of technologies, increases in productivity were tangible. Technology had proven itself useful and helpful to the cause, and was here to stay.

Petroleum Geoscience- and Engineering- specific software applications started springing up in the market like Texas wild flowers in March. Although some companies built seismic and log interpretation software back in the 70s using Cray super computers and on DEC mini computers, not many could afford an $800,000 computer (yes, one computer that is) with limited capacity. “I remember selling software on time share for CGG back in the 80s” my friend commented, “companies had to connect to expensive super computers on extremely slow connections” he adds.  So when the computer became affordable and with the right power for E&P technical applications, the software market flourished.

The industry was thirsty for software and absorbed all of what was produced on the market and then some; operators who could afford it created their own. The big service companies decided they were not going to miss out. Schlumberger acquired Geoquest in 1992 for its seismic data processing services and tools, then also acquired Finder, Eclipse and a long string of other applications.

The only problem with all these different software applications was that they existed standalone; each application had its own database and did not communicate with another. As a result, working on each hydrocarbon asset meant multiple data entry points or multiple reformatting and re-loading. This informational and collaborative disconnect between the different E&P applications was chipping away the very productivity and efficiency the industry was desperate to harness.

Nevertheless, the standardization of defining, capturing, storing and exchanging E&P data was starting to be of interest to many organizations. PPDM in Canada and later POSC in the USA (now Energetics) were formed in 1988 and 1990 respectively. PPDM’s mission at the time was focused on creating an upstream data model that could be utilized by different applications. POSC’s mission was broader; to develop a standardized E&P data model and data exchange standards.

Schlumberger had a solution for its own suite of applications; it offered both Geoframe and Finder as answers to the data mess with Finder being the master database that fed Geoframe with information, and Geoframe integrated the various software applications together.

Mid-90s: Making Connections

In the mid-90s, Halliburton acquired Landmark Graphics and unveiled the OpenWorks platform for its suites of applications in April 1997 at the AAPG. Their market positioning? Integrated reservoir management and a data management solutions. OpenWorks offered similar data integration to GeoFrame but with its own set of scientific software. Geoframe and OpenWorks would butt heads for years to come, both promoting their vision of data management and integrated workflows. It seemed that the larger companies were either a Schlumberger or Landmark shop.

In 1997, the Open Spirit Alliance funded by a consortium (Schlumberger, Shell and Chevron) was born and interoperability was its mission. PrismTech was to develop and market an application integration framework that any company could utilize, it was to be open. Open Spirit platform was officially launched at the SEG in 1998.

Late 90s: Big Industry Changes

Come the late 90s, another drop in oil prices combined with other macroeconomics appeared to trigger a surge in “mega” M&A activities starting with Exxon acquiring Mobil in 1998, BP acquiring Amoco in 1999, and then Conoco acquiring Philips in 2000, these mega acquisitions continued through early 2000s.

All this M&A in the 90s added complexity to what was already a complex technical dataflow environment.

For the data nerds

  • In the 90s, the industry rapidly evolved from hand-written scout tickets, and hand-drawn maps to electronic data.
  • The “E&P software spring” produced many silo databases. These databases often overlapped in what they stored creating multiple versions of the same data.
  • The IT department’s circle of influence was slowly but surely expanding to include managing E&P data. IT was building data systems, supporting them, uploading data to them and generating reports.
  • Engineers and Geoscientist still kept their own versions of data, but in MANY locations now. While hardcopies were the most trusted form (perceived to be the most reliable), technical data was also stored in disks, network drives, personal drives and in various applications’ databases and flat files. It compounded the data management problems of the years prior to computerization of processes.
  • Relational databases and SQL proved to be valuable to the industry. But it was expensive to support a variety of databases; many operators standardized and requested systems on Oracle (or SQLServer later).
  • Systems not on relational databases either faded away to the background or converted to relational databases that were accepted by operators.
  • Two standard data models emerged PPDM and POSC (now Energetics) and one data integration platform from the OpenSpirit (now part of the Tibco suite).
  • Geos and engineers validated and cleaned their own data (sometimes with the help of Geotechs or technical assistants) prior to their analyses.

 Stay tuned for the Millennium, and please add your own memories (and of course please correct me for what is not accurate ….)

Are we progressing? Oil & Gas Data Management Journey…

3 Replies

Last month, I had dinner with a long-term friend who is now part of a team that sets strategic technical plans for his E&P employer. Setting strategies requires a standardized view of technical & financial data across all assets, in this case, multinational assets around the world. This data is required at both granular and helicopter level.  One of the things he mentioned was “I have to start by fixing data standards. I am surprised how little progress data-management standards have made since the POSC days in the mid 90s.”

How did Data Management evolve in oil & gas? Are we repeating mistakes? Are we making any progress? Here is what my oil and gas friends and I remember in this first part of a three-part series.  Please join me on this journey down memory lane and add your own thoughts.

The 1960s & 70s

Perhaps we can call these times, mainframe times.  Mainframes started to make their way into our industry around the mid-60s. At that time, they were mostly used to maintain accounting data. Like most data at this time, E&P accounting data was manually entered into systems, and companies employed large data-entry staff to input. Any computational requirement of the data was through feeding  programs through “punch cards”.

Wireline logs (together with Seismic data) were one of the very first technical data that required the use of computers, mainly at the service provider’s Computer Centers and then at the large offices of the largest major operators. A friend of mine at Schlumberger remembers the first log data processing center in Houston opening about 1970. In the mid-70’s more oil city offices (Midland, Oklahoma City, etc.) established regional computing centers. Here, wireline log data, including petrophysical and geological processing, was “translated” from films into paper log graphics for clients.

A geophysicist friend remembers using mainframe computers to read seismic tapes in the mid-70s. He said, “Everything was scheduled. I would submit my job, consisting of data and many Punch Cards, into boxes to get the output I needed to start my interpretation. That output could be anything from big roll of papers for seismic sections to an assemblage of data that could then be plotted. Jobs that took 4 hours  to process on a mainframe in the 70’s are instantaneous today”

The Society of Exploration Geophysicist (SEG) introduced and published data formatting standard SEG_Y in 1975.  SEG-Y formats are still utilized today.

The need to use a standard, well number identification process became apparent as early as 1956. Regulatory agencies started assigning API numbers to wells in the late 60s in the USA. The concept of developing world wide global well ID numbers is still being discussed today with some organizations making good progress.

The 2nd half of the 70s, pocket calculators and mini computers made their way to the industry. With that some computations could be done at the office or on the logging truck at the field without the need for Mainframes.

The 1980s

Early 80s. With the proven success of 3D seismic introduced by ExxonMobil, large and special projects started heavily processing 3D seismic on Mainframes. However, the majority of technical data was still mainly on paper. Wireline logs were still printed on paper for petrophysicists  to add their handwritten interpretations. Subsurface maps were still drawn, contoured and colored by hand. Engineering data came in from the field on paper and was then recorded on a running paper table. A reservoir engineer remembers   “We hired data clerks to read field paper forms and write the data in table (also on paper)”.

As personal computers  (PCs) made their way into the industry, some large companies started experimenting,  albeit they lacked the personal side since PCs were numbered and located in a common area. Employees were only given occasional access to them. These were also standalone computers, not networked. Data transfer from one PC to another happened via floppy disk. It was during this time that engineers were first exposed to spreadsheets (boy did they love those spreadsheets! I know I do)

Mid-80s. March 1986, oil prices crashed, a 55% drop over few days. In the three years following the crash, the industry shed staff the way cats shed hair. The number of petroleum staff dropped from approximately 1,000,000 employed staff to approximately 500,000 in three years.

oil price

Late 80s. But what seemed bad for the industry, may have also done the industry a favor. The oil price crash may have actually accelerated the adoption of technology. With a lot less staff, companies were looking for ways to accomplish more with less staff.

A geologist friend remembers using Zmap as early as 1988, which was the beginning of the move towards predominantly computer-based maps and technical data.

For data nerds: 

  • Engineers and geo professionals were responsible for maintaining their own data in their offices.
  • Although not very formal, copies of the data were maintained in centralized “physical” libraries. Data was very important in the “heat of the moment” after the project is complete, that data is someone else’s issue. Except there was no “someone else” yet.
  • This system produced many, many versions of the same data (or a little variation of it) all over. This data was predominantly kept on physical media and some kept on floppy disks which were mostly maintained by individuals.
  • From the 60s through to the end of the 80s, we can say there were mostly two global standards, one for the seismic data formatting – SEG-Y – and the other for log data – LAS (Log Ascii Standard). Any other standards were country- or company-specific.

I would love to hear from you if you feel I have missed anything or if you can add to our knowledge of how technical E&P data was managed during the above period.

Stay tuned for the 90s …