- Press Release Distribution

Positive reinforcements help algorithm forecast underground natural reserves

Texas A&M researchers designed a new reinforcement-based system that automates the prediction of subsurface environments.

( Texas A&M University researchers have designed a reinforcement-based algorithm that automates the process of predicting the properties of the underground environment, facilitating the accurate forecasting of oil and gas reserves.

Within the Earth's crust, layers of rock hold bountiful reservoirs of groundwater, oil and natural gas. Now, using machine learning, researchers at Texas A&M University have developed an algorithm that automates the process of determining key features of the Earth's subterranean environment. They said this research might help with accurate forecasting of our natural reserves.

Specifically, the researchers' algorithm is designed on the principle of reinforcement or reward learning. Here, the computer algorithm converges on the correct description of the underground environment based on rewards it accrues for making correct predictions of the pressure and flow expected from boreholes.

"Subsurface systems that are typically a mile below our feet are completely opaque. At that depth we cannot see anything and have to use instruments to measure quantities, like pressure and rates of flow," said Siddharth Misra, associate professor in the Harold Vance Department of Petroleum Engineering and the Department of Geology and Geophysics. "Although my current study is a first step, my goal is to have a completely automated way of using that information to accurately characterize the properties of the subsurface."

The algorithm is described in the December issue of the journal Applied Energy.

Simulating the geology of the underground environment can greatly facilitate forecasting of oil and gas reserves, predicting groundwater systems and anticipating seismic hazards. Depending on the intended application, boreholes serve as exit sites for oil, gas and water or entry sites for excess atmospheric carbon dioxide that need to be trapped underground.

Along the length of the boreholes, drilling operators can ascertain the pressures and flow rates of liquids or gas by placing sensors. Conventionally, these sensor measurements are plugged into elaborate mathematical formulations, or reservoir models, that predict the properties of the subsurface such as the porosity and permeability of rocks.

But reservoir models are mathematically cumbersome, require extensive human intervention, and at times, even give a flawed picture of the underground geology. Misra said there has been an ongoing effort to construct algorithms that are free from human involvement yet accurate.

For their study, Misra and his team chose a type of machine-learning algorithm based on the concept of reinforcement learning. Simply put, the software learns to make a series of decisions based on feedback from its computational environment.

"Imagine a bird in a cage. The bird will interact with the boundaries of the cage where it can sit or swing or where there is food and water. It keeps getting feedback from its environment, which helps it decide which places in the cage it would rather be at a given time," Misra said. "Algorithms based on reinforcement learning are based on a similar idea. They too interact with an environment, but it's a computational environment, to reach a decision or a solution to a given problem."

So, these algorithms are rewarded for favorable predictions and are penalized for unfavorable ones. Over time, reinforcement-based algorithms arrive at the correct solution by maximizing their accrued reward.

Another technical advantage of reinforcement-based algorithms is that they do not make any presuppositions about the pattern of data. For example, Misra's algorithm does not assume that the pressure measured at a certain time and depth is related to what the pressure was at the same depth in the past. This property makes his algorithm less biased, thereby reducing the chances of error at predicting the subterranean environment.

When initiated, Misra's algorithm begins by randomly guessing a value for porosity and permeability of the rocks constituting the subsurface. Based on these values, the algorithm calculates a flow rate and pressure that it expects from a borehole. If these values do not match the actual values obtained from field measurements, also known as historical data, the algorithm is penalized. Consequently, it is forced to correct its next guess for the porosity and permeability. However, if its guesses were somewhat correct, the algorithm is rewarded and makes further guesses along that direction.

The researchers found that within 10 iterations of reinforcement learning the algorithm was able to correctly and very quickly predict the properties of simple subsurface scenarios.

Misra noted that although the subsurface simulated in their study was simplistic, their work is still a proof of concept that reinforcement algorithms can be used successfully in automated reservoir-property predictions, also referred as automated history matching.

"A subsurface system can have 10 or 20 boreholes spread over a two- to five-mile radius. If we understand the subsurface clearly, we can plan and predict a lot of things in advance, for example, we would be able to anticipate subsurface environments if we go a bit deeper or the flow rate of gas at that depth," Misra said. "In this study, we have turned history matching into a sequential decision-making problem, which has the potential to reduce engineers' efforts, mitigate human bias and remove the need of large sets of labeled training data."

He said future work will focus on simulating more complex reservoirs and improving the computational efficiency of the algorithm.


Hao Li of the University of Oklahoma was a contributor to this work. This research is funded by the United States Department of Energy.


Transformed by light: Fast photochromism discovered in an inexpensive inorganic material

Transformed by light: Fast photochromism discovered in an inexpensive inorganic material
Isn't it convenient when office building windows adaptively darken according to the intensity of sunlight? Or when standard glasses turn into sunglasses under the sun and switch back as you enter a building? Such feats are possible thanks to photochromic materials, whose optical (and other) properties change radically when irradiated by visible or ultraviolet light. Today, virtually all fast-switching photochromic materials are made using organic compounds. Unfortunately, this makes them considerably expensive and complex to synthesize, requiring multi-step processes that are difficult to scale up for mass production. So, despite the myriad ...

Oxidation processes in combustion engines and in the atmosphere take the same routes

Oxidation processes in combustion engines and in the atmosphere take the same routes
Thuwal/Helsinki/Leipzig. Alkanes, an important component of fuels for combustion engines and an important class of urban trace gases, react via another reaction pathways than previously thought. These hydrocarbons, formerly called paraffins, thus produce large amounts of highly oxygenated compounds that can contribute to organic aerosol and thus to air pollution in cities. An international research team has now been able to prove this through laboratory experiments with state-of-the-art measurement technology at the University of Helsinki and the Leibniz Institute for Tropospheric ...

Machine learning aids in simulating dynamics of interacting atoms

Machine learning aids in simulating dynamics of interacting atoms
LOS ALAMOS, N.M., February 23, 2021--A revolutionary machine-learning (ML) approach to simulate the motions of atoms in materials such as aluminum is described in this week's Nature Communications journal. This automated approach to "interatomic potential development" could transform the field of computational materials discovery. "This approach promises to be an important building block for the study of materials damage and aging from first principles," said project lead Justin Smith of Los Alamos National Laboratory. "Simulating the dynamics of interacting atoms is a cornerstone of understanding and developing new materials. Machine learning methods are providing computational scientists new tools to accurately and efficiently conduct these atomistic ...

Abundance of iron drives cell death and could inform novel treatments for neuroblastoma

Abundance of iron drives cell death and could inform novel treatments for neuroblastoma
Neuroblastoma is a cancer that develops in nerve tissue, most commonly in the glands around the kidneys. The gene MYCN is overexpressed in 20-25% of neuroblastoma, and MYCN-amplified neuroblastoma contributes to a considerable percentage of pediatric cancer-related deaths. Anthony Faber, Ph.D., and a team of researchers at VCU Massey Cancer Center were awarded a grant from the American Cancer Society to study how MYCN and an abundance of iron can drive cancer cell death in neuroblastoma and potentially be targeted with novel treatments. This award is the first part of a potential two-stage grant worth a combined total of $600,000. "Iron is a double-edged sword in a cancer cell. It can help the cancer grow and survive, but it also creates ...

Seeing schizophrenia: X-rays shed light on neural differences, point toward treatment

Seeing schizophrenia: X-rays shed light on neural differences, point toward treatment
Schizophrenia, a chronic, neurological brain disorder, affects millions of people around the world. It causes a fracture between a person's thoughts, feelings and behavior. Symptoms include delusions, hallucinations, difficulty processing thoughts and an overall lack of motivation. Schizophrenia patients have a higher suicide rate and more health problems than the general population, and a lower life expectancy. There is no cure for schizophrenia, but the key to treating it more effectively is to better understand how it arises. And that, according to Ryuta Mizutani, professor of applied biochemistry at Tokai University in Japan, means studying the structure of brain tissue. Specifically, it means comparing the brain tissues of schizophrenia patients with those ...

Stanford researchers identify four causes of "Zoom fatigue" and their simple fixes

Even as more people are logging onto popular video chat platforms to connect with colleagues, family and friends during the COVID-19 pandemic, Stanford researchers have a warning for you: Those video calls are likely tiring you out. Prompted by the recent boom in videoconferencing, communication Professor Jeremy Bailenson, founding director of the Stanford Virtual Human Interaction Lab (VHIL), examined the psychological consequences of spending hours per day on these platforms. Just as "Googling" is something akin to any web search, the term "Zooming" has become ubiquitous and a generic verb to replace videoconferencing. Virtual meetings have skyrocketed, with hundreds of millions happening daily, as social distancing protocols have kept people apart physically. In ...

Scientists found in marine mold substance that antidotes paraquat

Scientists found in marine mold substance that antidotes paraquat
Biologically active compounds from the marine fungus Penicillium dimorphosporum protect cells from paraquat, the highly toxic herbicide with no remedy, and might enhance the action of some drugs. The fungus was isolated from soft coral collected in the South China Sea during an expedition on the Akademik Oparin research vessel. Scientists of Far Eastern Federal University (FEFU) and G. B. Elyakov Pacific Institute of Bioorganic Chemistry reported the results in Marine Drugs. Paraquat a herbicide compound highly toxic for animals and humans. About a hundred countries, including the United States, apply it for crop cultivation and weed control. Dozens of countries, including Russia, have banned the ...

School of Community Health Sciences publishes study on sugar-sweetened beverage taxes

A new research study out of the University of Nevada, Reno's School of Community Health Sciences has just been published by the American Journal of Public Health and addresses state preemption of local sugar-sweetened beverage (SSB) taxes, issuing an emerging public health threat. Assistant Professor Eric Crosbie examines commercial determinants of health and public health policy, specifically in industries like tobacco and food and beverage. "The beverage industry is aggressively attempting to preempt sugar-sweetened beverage taxes at the state level to prevent the diffusion of progressive policies at the local level throughout the United States," Crosbie, an affiliate of the University's Ozmen Institute for Global Studies, said. "Once preemption laws are enacted, ...

Give the heart a ketone? It may be beneficial

There is growing evidence that ketone bodies may be beneficial to heart disease patients regardless of the method of delivery used to increase ketone delivery to the heart. A Journal of the American College of Cardiology review paper examines emerging evidence regarding ketone bodies' effects on the heart and the potential for ketone therapy as a cardiovascular intervention in heart disease patients. In recent years ketone bodies entered the popular lexicon through the "keto diet," which consists of a very low carbohydrate and high fat diet that endeavors to force the body into ketosis. This is a metabolic ...

Monoclonal antibodies against MERS coronavirus show promise in phase 1 NIH-sponsored trial

Monoclonal antibodies against MERS coronavirus show promise in phase 1 NIH-sponsored trial
WHAT: A randomized, placebo-controlled Phase 1 clinical trial of two monoclonal antibodies (mAbs) directed against the coronavirus that causes Middle East respiratory syndrome (MERS) found that they were well tolerated and generally safe when administered simultaneously to healthy adults. The experimental mAbs, REGN3048 and REGN3051, target the MERS coronavirus (MERS CoV) spike protein used by the virus to attach to and infect target cells. The mAbs were discovered and developed by scientists at the biopharmaceutical company Regeneron, located in Tarrytown, New York. The trial was sponsored by the National Institute of Allergy and Infectious Diseases (NIAID), part of the National Institutes of Health. The trial was the first to test the experimental antibodies Hyperlink Code ...


Unequal parenthood impacts may explain academia's publication gender gap

Potential biomarker predicts the risk of kidney transplant rejection in patients

A gearbox for tumor cell identity changes

University of Chicago study uncovers inhibitory role of 'Ter cells' in cancer therapies

Asteroid dust found in crater closes case of dinosaur extinction

Politicized pandemic shaped compliance with social distancing

Ultra-rapid detection of SARS-CoV-2 in public workspace environments

Improving road safety to tackle crime

New research at UH Rainbow studies the impact of face masks on heart ra

New technology shows potential to improve potency and durability benefits in gene therapy

Scientists describe earliest primate fossils

Revive the map: 4D building reconstruction with machine learning

Record-high Arctic freshwater will flow to Labrador Sea, affecting local and global oceans

Diabetes patients use of mobile health app found to improve health outcomes, lower medical costs

Study identifies strengths and challenges of responding to dual disasters

A combination therapy for treating severe neurological childhood disorders

Bearded seals are loud - but not loud enough

New study charts the complexity of SARS-CoV-2 neutralization

Scientists suggested using 'defective' diamonds in x-ray optics

Data transfer system connects silicon chips with a hair's-width cable

Uncovers the molecular mechanism behind synapse loss in Alzheimer's disease

Celebrating Black chemists and chemical engineers

Study finds digital treatment for osteoarthritis is superior to traditional routine care

A gene provides both protection and destruction

Pushing computing to the edge by rethinking microchips' design

Flu vaccination this season likely to be highest ever

'Micropopulism' may be turning education into a battlefield in the culture wars

Study finds low rate of COVID-19 among dental hygienists

Sulfur: the consequences

NCI study finds people with SARS-CoV-2 antibodies may have low risk of future infection

[] Positive reinforcements help algorithm forecast underground natural reserves
Texas A&M researchers designed a new reinforcement-based system that automates the prediction of subsurface environments.