Automation initiatives show untapped potential in drilling optimization

By Stephen Whitfield, Senior Editor

Automation is the name of the game in today’s drilling industry. As new systems are being tested and implemented on the rig, drillers and operators are ceding to these systems various processes that were previously the domain of their human personnel.  

For instance, research is being done on the use of deep reinforcement learning (RL) for automated drilling operations, with a machine learning model acting on its own to optimize various parameters – such as ROP, bit wear and weight on bit (WOB) – in different drilling environments. It’s anticipated the model can eventually become a valuable tool in relieving drillers of some of the complex decisions they have to make. 

“The drilling process is very complicated,” said Xu Huang, Research Engineer at Baker Hughes. “The driller is continuously adjusting or changing parameters like the weight on bit. They want to maximize ROP. They want to reach total depth. They don’t want severe vibrations. You want to control your depth of cut (DOC) so you don’t see any cutter damage. It is a process optimization problem. We’ve found that tools like deep reinforcement learning are quite suitable for solving these types of complicated process issues.” 

Process automation is another type of automation that is increasingly being accepted and integrated into daily drilling operations. Automating repetitive manual tasks, such as traditionally manual slip-to-weight (S2W) and weight-to-slip (W2S) sequences, has shown improved procedural compliance. This reduces variability in performance among drillers, both on the same rig and across multiple rigs. In some cases, the success of these systems are leading operators and drilling contractors to examine the effectiveness and consistency of their procedures, and even whether they are implementing the right procedure at various points in the drilling process.

“As an industry, the conversation around automation is changing a little bit,” said Chris Christopher, Senior Product Development Manager at Patterson-UTI Drilling Company. “Historically when we talked about an automation product, there would be questions about time or cost savings. What I’m starting to see now is the conversation revolves more around what’s an appropriate time for this process or this procedure? Are we choosing the right procedures? Are people not adhering to the right procedures?”

Automated systems, when combined with robotics, are also enabling companies to deliver greater consistency in executing various processes on the rig, such as pipe handling. For example, NOV is leveraging learnings from a pilot project – physically removing the driller’s cabin from the rig floor on its test rig in Navasota, Texas – to make a step change in terms of where and how the driller interacts with new automation technology.  

“As soon as you start automating something, you immediately increase the consistency of the operation,” said Brandt Lanzet, Product Line Manager at NOV. “Once we get the consistency and the utilization of these automation systems, that’s where we can really go to work optimizing that and then really pushing performance.”

How deep learning can optimize processes

Building a roadmap of algorithms to create an automated drilling system is a complicated, multi-objective processing optimization problem. Drillers need to adjust WOB and RPM continuously and in a timely manner to maximize ROP and total footage, as well as prevent severe vibration and maintain downhole tool durability. Any model developed for automation must take these complexities into account. 

Baker Hughes undertook an R&D project testing a virtual drilling agent using a deep RL model. The model was developed to automatically make decisions during drilling and effectively optimize drilling parameters. Mr Huang outlined the development of the RL model during a presentation at this year’s IADC/SPE International Drilling Conference in Galveston, Texas. 

The deep RL model uses a deep deterministic policy gradient algorithm, which is designed to operate over continuous action spaces like those seen in a drilling environment, in order to optimize the drilling process. It uses a reward model, meaning that scores (rewards) are given for how it responds to prompts. These rewards guide the model to produce desired outcomes, effectively training the model as a proxy for human preferences. A positive reward indicates that the model produced a desired outcome, while a negative reward indicates the outcome was not desired. 

For the model’s training, the rewards of decisions at each step in the drilling process were a function of ROP, downhole vibration tendency, bit wear and the risk of tool failure. For each step in which the model produced a desired outcome in each of these categories, it would receive a positive score. Separate modules evaluating the reward of each metric were implemented and trained using field and laboratory data. The model was also tested in different drilling environments in the laboratory, including hard and abrasive rock, and embedded rock. 

“What we’re trying to do here is train the model to be its own drilling agent,” Mr Huang said. “Actually, it’s a neural network. This agent, given a certain state, will tell you what action to do next. Then you go on to the next state, and it tells you what to do next. The goal is to train this agent so that we can maximize the reward.” 

The design of the reward functions was critical in developing the model, as it plays a significant role in the model’s learning ability. For the ROP metric, the reward function offered a positive score if the ROP obtained as a result of a given decision was greater than the baseline ROP. The bit wear was also monitored in each step of the drilling process. If the bit wear reached a given threshold value, pull-out-of-hole (POOH) was triggered. If the POOH occurred before the bit reached a target drilling depth, the model received a significant negative reward, owing to the cost and NPT associated with tripping in and out.

The reward function for tool failure primarily looked at DOC. An uncontrolled DOC can lead to the overloading of the bit cutting structure in hard drilling environments, Mr Huang said. In transitional drilling applications, the DOC can fluctuate and increase unexpectedly, leading to potentially severe stick-slip. In directional drilling applications, the DOC and ROP may need to be controlled tightly to improve tool face control and bit steerability, he added. 

Physics-based models were used to evaluate the cutting structure’s durability. The rock-cutter interaction and cutting force were calculated using a 3D drilling model. The cutter’s impact stress under extreme conditions was also evaluated using dynamic finite element analysis, a simulation technique that measures how a structural system – in this case, the cutter – behaves when it is subjected to dynamic loads. From these models, the Baker Hughes team created the reward function for tool failure – a negative reward was assigned if the resultant DOC under a given action was above a pre-defined threshold. 

The vibration reward function was straightforward – if stick-slip was identified after a given decision, a negative reward was given. 

The RL model successfully found the optimal drilling actions in different simulated environments. For laboratory testing of the ROP reward function, a drilling environment corresponding to an ideal pure abrasive wear drilling application with no vibrations or bit durability issues was simulated. The model was able to execute a gradual and continuous drop of ROP under maximized WOB, indicating the bit wear progressed as expected. It also successfully optimized its own decision making and reduced overall drilling time by 50% compared with the expected drilling time under the given conditions.

When adding the bit wear reward function to the laboratory testing, the RL model successfully learned to maximize ROP and limit WOB in order to avoid a negative reward for controlling DOC. During the test run, the bit wore progressively, and the WOB was guided to gradually increase in order to maintain the DOC. The ROP also remained at a nearly constant value during the test run. The RL model then showed in laboratory testing that it could also adjust ROP and WOB to avoid a negative stick-slip reward.

Automating connection processes

Patterson-UTI recently partnered with a major operator in the Permian Basin to develop an automation sequence within an existing third-party software program for connections. Connections are a prime candidate for automation, in part because of the combination of detailed metrics and repetitive processes, Mr Christopher said at the 2024 IADC Drilling Onshore Conference in Houston on 16 May. The presentation was an extension of a previous presentation given at the 2024 IADC/SPE International Drilling Conference in March.

The project focused on the automation of S2W and W2S sequences in drilling. Historically, best practices for these sequences were individually maintained. However, the operator wished to utilize automation to help standardize and ensure adherence to those best practices; it was believed this would help mitigate bit damage.

The operator provided a drilling connection and survey procedure for its rigs working in the Permian to help Patterson-UTI develop the automation. This procedure outlined the steps from the drilling of a stand to the point where the bit returns to the bottom and drilling rates resume at full capacity. The procedure specified steps and parameters by hole section, covering processes from W2S, including drilling off WOB, setting slips and re-engaging from slips to drilling. 

For Patterson-UTI, one of its main goals while developing the automation was to build in enough flexibility so that it could suit the operator’s connection procedure without requiring software customization. This means it had to be designed to accommodate both the operator’s procedures and the recommended connection practices outlined in the IADC Drilling Manual. This flexibility, Mr Christopher said, would allow Patterson-UTI to scale the automation to different operators if needed.

“The key for us in developing best practices is the consistency. We’re drilling for consistency,” he said. “Those best practices are hard-earned from operators and from contractors. What automation lets you do is you make a configuration, click a button, and you can see those practices are adhered to.” 

Using the operator’s drilling connection and survey procedure, Patterson-UTI’s engineers created a logical flow within the software, specifying the sequence of operations, automation levels, operating parameters, hazards, data requirements and time frames. This flow was used to define the project’s automation requirements. 

Drillers interacted with the control system primarily through HMI screens, manual switches and joysticks in the driller’s cabin, along with a third-party EDR. The HMI communicates with equipment like mud pumps, top drives and drawworks through a PLC connected to the equipment’s variable frequency drive. To manage complex tasks and data flows, an edge server complemented the PLC, interfacing with the EDR. The manual connection process was mapped to delineate the driller’s interactions with the control system, including prerequisites, inputs and outputs.

A comprehensive requirements document was drafted, outlining the scope, data flows, prerequisites, equipment behavior expectations, shutdown conditions and testing protocols for the W2S and S2W processes. This document also directed the collaborative efforts to develop automation – Mr Christopher said collaboration between Patterson-UTI and the operator was critical to the project’s success. 

“You have to build a partnership, which goes hand in hand with trust,” he said. “Without the trust, you can’t get the utilization numbers you need. And without those numbers, you won’t know if your automation is having a strong impact. You need a partner, and you need frequent communication.” 

Product testing was done in three sequential phases: initial development, quality control (QC) and field testing. 

For the first two phases, Patterson-UTI used a drilling simulator in a lab environment. The lab simulation facilitated emulating mud pumps, top drive operations and drawworks movements. Several servers ran distinct rig operating software and interacted with PLCs and specific electric motors to replicate rig equipment functionalities. Mr Christopher said this integrated setup, including third-party EDR systems, allowed for seamless data transmission across platforms. 

Patterson-UTI’s QC engineers created a series of test cases derived from the requirements document, ensuring the new application harmoniously integrates without impeding existing system functions. Upon completion of the first round of QC, failed tests were addressed and categorized based on urgency. Subsequent QC testing verified the critical issues that had been previously identified, after which the software was earmarked for field testing. 

For the field test, trained technicians were on hand during rig transitions to implement control system updates and for troubleshooting. A dedicated team at Patterson-UTI’s corporate headquarters in Houston also monitored the tests to gauge performance and pinpoint potential issues. Weekly analytical reports captured essential KPIs, insights and prospective enhancements.

Because of the importance of training rig personnel on the new automation sequence prior to field testing, a schedule was created and shared between the operator and drilling contractor, providing a structure for the companies to discuss software updates and training strategies. The timing of software updates was selected in collaboration to align with rig walks and/or rig moves, minimizing downtime and allowing for troubleshooting. Training occurred during initial deployment and included all shifts and tour changes. Further, training was planned to follow the initial training to refresh the material and cover any new features or lingering questions, along with remote support.

“Training became really critical as we moved forward,” Mr Christopher said. “We trained day and night. We made user guides. We made laminated cheat sheets showing how to configure things. We created these strong communication paths and called them out regularly.”

In the end, both the operator and drilling contractor derived tangible benefits from the implemented measures. Mr Christopher noted uniform connection performance across all drillers and faster onboarding for new drillers. Also, it was observed that the drillers had more time to concentrate on advanced drilling tasks, such as well control and safety.

Initial findings indicated substantial improvements in S2W and W2S durations, all while ensuring rigorous process adherence. Initial utilization resulted in a compliance of 84% for S2W and 77% for W2S, compared with a 75% target. Notably, during scaling, Patterson-UTI observed 75% compliance was achieved after just six weeks — earlier than the two-month objective. 

The automation technique has since been implemented on 30 Patterson-UTI rigs operated by the E&P company that participated in the test project and another operator working in the Permian region. 

Taking people off the rig floor

To highlight the potential for automation to improve rig site safety, in 2023 NOV removed the driller’s cabin from the rig floor of its test land rig in Navasota, Texas, installing it instead on the ground next to the rig. At the time, the company said the move was a step toward realizing its vision for a manless rig floor, where automation and robotics systems can be leveraged to move the driller and crews away from potentially hazardous situations on the rig floor. 

That project has served as a validation of various technologies that will be important, ultimately, to enabling greater automation of land rigs, Mr Lanzet said. One of the first key enablers of the driller’s cabin move was Aura, an integrated information hub. It consolidates various screens showing data from different equipment sources (such as the CCTV cameras, EDR, an MWD system, or an MPD system) into a single customizable screen. The screen displays data and visuals from multiple sources pertinent to the current stage of the drilling operation, including downhole simulations, digital well plans, and KPIs related to red zone monitoring. 

By integrating monitors from different operations and equipment, the system allows the driller to view every component of the operation more easily and quickly, he explained. “When you go to a rig and step into the driller’s cabin, you can see that every third party on that rig has set up a separate screen inside the driller’s cabin. It overloads the driller, and his attention is being divided rather than focused. That is really the direction that we’re trying to get away from. Aura integrates third-party connections into the screen, so the right information is given to the driller at the right time.”  

Other key enabling technologies included the robotics systems designed to manage pipe handling, as well as the pipe-handling automation systems that control the robotics. Without personnel operating near potentially hazardous equipment, the driller can fully shift his or her focus to other parts of the operation, thus improving efficiency, Mr Lanzet said. 

“With the robotics, now the driller isn’t worried about people. He’s just worried about other machines, and he has views of those machines. The cool thing is, now that we’ve integrated robotics into a land rig, not only have robots reduced exposure to the red zone, but they’ve also provided mechanization that wasn’t there before. Once you have that level of mechanization, now we can provide automated pipe-handling sequences. Whereas the driller was having to do things manually with the pipe-handling operation, now there’s just a series of confirmation steps that the robotics are doing.” 

Multi Machine Control is the control system that automates pipe-tripping and stand-building processes for the drill pipe and casing. By letting the automation systems orchestrate the drill floor equipment, more rig floor operations can be completely manless. 

Measurement tools to validate the delivered efficiency are also part of NOV’s digital ecosystem on a manless rig floor. Among them is Rig Cadence, a software program that uses predefined benchmarks and thresholds to detect deviations in the performance of iron roughnecks, top drives, drawworks and slips. It then provides corrective solutions to optimize the operation of machine movements and improve system cycle times. The system offers real-time monitoring and analysis, as well as reports on pipe-handling operations, including tripping in, tripping out and connections. 

While the cabin floor removal has only been enacted on NOV’s test rig, the company is looking to take the lessons learned to customers. To support the technology, NOV created its Performance Centers in 2023. The centers, which are located in Houston, the UK and Norway, provide remote support and performance recommendations for drilling and pipe-handling operations. 

A total of 20 land and offshore rigs — all of which are running NOV’s automation technologies — have been enrolled since the launch of the Performance Centers. Some of these rigs have set weight-to-weight (W2W) and slip-to-slip (S2S) connection time records, according to NOV. For instance, a land rig in Alaska’s North Slope achieved its fastest W2W time of 11.16 minutes for an intermediate section. In the North Sea, a semisubmersible achieved a record W2W average of 3.41 minutes. Also, a drillship offshore Guyana set a record W2W average of 7.86 minutes.  DC