(Press-News.org) Philadelphia, PA--A computer model is a representation of the functional relationship between one set of parameters, which forms the model input, and a corresponding set of target parameters, which forms the model output. A true model for a particular problem can rarely be defined with certainty. The most we can do to mitigate error is to quantify the uncertainty in the model.
In a recent paper published in the SIAM/ASA Journal on Uncertainty Quantification, authors Mark Strong and Jeremy Oakley offer a method to incorporate judgments into a model about structural uncertainty that results from building an "incorrect" model.
"Given that 'all models are wrong,' it is important that we develop methods for quantifying our uncertainty in model structure such that we can know when our model is 'good enough'," author Mark Strong says. "Better models mean better decisions."
When making predictions using computer models, we encounter two sources of uncertainty: uncertainty in model inputs and uncertainty in model structure. Input uncertainty arises when we are not certain about input parameters in model simulations. If we are uncertain about true structural relationships within a model—that is, the relationship between the set of quantities that form the model input and the set that represents the output—the model is said to display structural uncertainty. Such uncertainty exists even if the model is run using input values as estimated in a perfect study with infinite sample size.
"Perhaps the hardest problem in assessing uncertainty in a computer model prediction is to quantify uncertainty about the model structure, particularly when models are used to predict in the absence of data," says author Jeremy Oakley. "The methodology in this paper can help model users prioritize where improvements are needed in a model to provide more robust support to decision making."
While methods for managing input uncertainty are well described in the literature, methods for quantifying structural uncertainty are not as well developed. This is especially true in the context of health economic decision making, which is the focus of this paper. Here, models are used to predict future costs and health consequences of options to make decisions for resource allocation.
"In health economics decision analysis, the use of "law-based" computer models is common. Such models are used to support national health resource allocation decisions, and the stakes are therefore high," says Strong. "While it is usual in this setting to consider the uncertainty in model inputs, uncertainty in model structure is almost never formally assessed."
There are several approaches to managing model structural uncertainty. A primary approach is 'model averaging' in which predictions of a number of plausible models are averaged with weights based on each model's likelihood or predictive ability. Another approach is 'model calibration', which assesses a model based on its external discrepancies, that is, output quantities and how they relate to real, observed values. In the context of healthcare decisions, however, neither of these approaches is feasible since typically more than one model is not available for averaging, and observations on model outputs are not available for calibration.
Hence, the authors use a novel approach based on discrepancies within the model or "internal discrepancies" (as opposed to external discrepancies which are the focus of model calibration). Internal discrepancies are analyzed by first decomposing the model into a series of subunits or subfunctions, the outputs of which are intermediate model parameters that are potentially observable in the real world. Next, each sub-function is judged for certainty based on whether its output would equal the true value of the parameter from real-world observations. If a potential structural error is anticipated, a discrepancy term is introduced. Subsequently, beliefs about the size and direction of errors are expressed. Since judgments for internal discrepancies are expected to be crude at best, the expression of uncertainty should be generous, that is, allowed to cover a wide distribution of possible values. Finally, the authors determine the sensitivity of the model output to internal discrepancies. This gives an indication of the relative importance of structural uncertainty within each model subunit.
"Traditional statistical approaches to handling uncertainty in computer models have tended to treat the models as 'black boxes'. Our framework is based on 'opening' the black box and investigating the model's internal workings," says Oakley. "Developing and implementing this framework, particularly in more complex models, will need closer collaboration between statisticians and mathematical modelers."
INFORMATION:
Source Article:
When Is a Model Good Enough? Deriving the Expected Value of Model Improvement via Specifying Internal Model Discrepancies
http://epubs.siam.org/doi/pdf/10.1137/120889563
SIAM/ASA Journal on Uncertainty Quantification, 2(1), 106 (Online publish date: February 6, 2014). The paper is available for free download at the link above through December 31, 2014.
About the authors:
Mark Strong is a clinical senior lecturer in public health and the Deputy Director of Public Health Section at the School of Health and Related Research at the University of Sheffield, and Jeremy Oakley is a professor of statistics in the School of Mathematics and Statistics at the University of Sheffield.
About SIAM
The Society for Industrial and Applied Mathematics (SIAM), headquartered in Philadelphia, Pennsylvania, is an international society of over 14,000 individual members, including applied and computational mathematicians and computer scientists, as well as other scientists and engineers. Members from 85 countries are researchers, educators, students, and practitioners in industry, government, laboratories, and academia. The Society, which also includes nearly 500 academic and corporate institutional members, serves and advances the disciplines of applied mathematics and computational science by publishing a variety of books and prestigious peer-reviewed research journals, by conducting conferences, and by hosting activity groups in various areas of mathematics. SIAM provides many opportunities for students including regional sections and student chapters. Further information is available at http://www.siam.org.
Overcoming structural uncertainty in computer models
What is good enough to aid health economics decision making?
2014-04-01
ELSE PRESS RELEASES FROM THIS DATE:
Want spring allergy relief? Avoid stress
2014-04-01
ARLINGTON HEIGHTS, Ill. (April 1, 2014) – Stress doesn't cause allergies, but easing your mind might mean less allergy flare-ups this spring. According to a study published in the April issue of Annals of Allergy, Asthma & Immunology, the scientific journal of the American College of Allergy, Asthma and Immunology, allergy sufferers with persistent stress experience more allergy flares.
"Stress can cause several negative effects on the body, including causing more symptoms for allergy sufferers," said allergist Amber Patterson, MD, lead study author and ACAAI member. ...
Scientists ID genes that could lead to tough, disease-resistant varieties of rice
2014-04-01
As the Earth's human population marches toward 9 billion, the need for hardy new varieties of grain crops has never been greater.
It won't be enough to yield record harvests under perfect conditions. In an era of climate change, pollution and the global spread of pathogens, these new grains must also be able to handle stress. Now, researchers at Michigan Technological University have identified a set of genes that could be key to the development of the next generation of super rice.
A meta-data analysis by biologist Ramakrishna Wusirika and PhD student Rafi Shaik has ...
Pause the paunch and halt the hair loss
2014-04-01
In particular, the protein that activated hair follicle growth was shown to also inhibit fat production.
The world first research confirmed that changes in the hair growth cycle led to fluctuations in the thickness of the underlying fat layer of the skin – essentially meaning that the skin can regulate fat production.
The research was led by Professor Fiona Watt at King's College London in collaboration with Professor of Dermatology Rodney Sinclair from the University of Melbourne and Epworth Hospital.
Professor Sinclair said these findings could potentially be ...
Higher risk of death from skin cancer among men living alone
2014-04-01
There are differences in prognosis in cutaneous malignant melanoma depending on cohabitation status and gender, according to a new study published in the scientific periodical Journal of Clinical Oncology. Single men of all ages are more likely to die of their disease.
Cutaneous malignant melanoma is the most aggressive form of skin cancer. The disease is one of the fastest growing cancers among Caucasian (white) populations and is an escalating health problem even among young individuals. Swedish researchers from Karolinska Institutet and Linköping University have now, ...
Universal syllables
2014-04-01
Take the sound "bl": how many words starting with that sound can you think of? Blouse, blue, bland... Now try with "lb": how many can you find? None in English and Italian, and even in other languages such words either don't exist or are extremely rare. Human languages offer several examples of this kind, and this indicates that in forming words we tend to prefer certain sound combinations to others, irrespective of which language we speak. The fact that this occurs across languages has prompted linguists to hypothesize the existence of biological bases of language (inborn ...
1.1 million Americans caring for recently wounded veterans, study finds
2014-04-01
More than 1.1 million spouses, parents and friends are caring for the injured and disabled who have served in the U.S. military since Sept. 11, 2001, often doing so without a formal support network and putting their own well-being at risk, according to a new RAND Corporation study.
The largest-ever study of military caregivers -- commissioned by the Elizabeth Dole Foundation -- finds that Americans who are taking care of veterans who served after 9/11 are younger than other caregivers, are usually employed outside the home and are more likely to care for someone who has ...
Customers prefer restaurants that offer nutrition facts and healthful foods
2014-04-01
Customers are more likely to frequent restaurants that provide both healthful foods and nutrition information, according to researchers at Penn State and the University of Tennessee.
"The Affordable Care Act has mandated that chain restaurants -- those with more than 20 restaurants -- provide nutrition information to customers," said David Cranage, associate professor of hospitality management. "Many restaurants had been fighting this legislation because they thought they would lose customers if the customers knew how unhealthy their food was. In this study, we found ...
New yeast species travelled the globe with a little help from the beetles
2014-04-01
Researchers from the National Collection of Yeast Cultures (NCYC) at the Institute of Food Research (IFR) have identified a new globe-trotting yeast species that lives on tree-associated beetles. This new species demonstrates the importance of preserving biodiversity, as yeasts like this may help efforts to develop renewable fuel sources in the future.
Preserving biodiversity must go beyond plants and animals and also preserve the microbial life. Threats to habitats, for example through oil exploration, could destroy forever potential solutions to global challenges locked ...
Simple changes in ICU can help heart attack patients: Study
2014-04-01
To improve recovery for heart attack patients, hospitals should maintain normal day and night cycles for those patients during the first few days after the attack, say University of Guelph researchers.
Their new study shows for the first time that interrupting diurnal rhythms impairs healing immediately after a heart attack, said Prof. Tami Martino of the Department of Biomedical Sciences.
Researchers already knew that circadian rhythms, or day-night cycles, can affect timing of a heart attack. This is the first study to show the importance of circadian rhythms during ...
Dog watch
2014-04-01
Dogs are individual personalities, possess awareness, and are particularly known for their learning capabilities, or trainability. To learn successfully, they must display a sufficient quantity of attention and concentration. However, the attentiveness of dogs' changes in the course of their lives, as it does in humans. The lead author Lisa Wallis and her colleagues investigated 145 Border Collies aged 6 months to 14 years in the Clever Dog Lab at the Vetmeduni Vienna and determined, for the first time, how attentiveness changes in the entire course of a dog's life using ...
LAST 30 PRESS RELEASES:
Washington coast avian flu outbreak devastated Caspian terns, jumped to seals
Mice tails whip up new insights into balance and neurodegenerative disease research
New study: Earthquake prediction techniques lend quick insight into strength, reliability of materials
Vitamin D during pregnancy boosts children’s bone health even at age seven
Use of “genetic scissors” carries risks
Does work-related stress compromise cardiovascular health?
New research may lead to potatoes that are less reliant on nitrogen fertilizers
Do commercial ties influence ESG ratings?
Study assesses "gendered space" in financial institutions in Pakistan
Chinese herbal medicine’s potential in preventing dementia
Firms that read more perform better
Tightly tied waist cord of saree underskirt may pose cancer risk, warn doctors
10% of children in high-burden tuberculosis settings may develop the disease by age 10
Health experts push for the elimination of a ‘remarkably harmful toxin’
University of Tennessee, Lockheed Martin expand Master Research Agreement
Testing thousands of RNA enzymes helps find first ‘twister ribozyme’ in mammals
Groundbreaking study provides new evidence of when Earth was slushy
International survey of more than 1600 biomedical researchers on the perceived causes of irreproducibility of research results
Integrating data from different experimental approaches into one model is challenging – this study presents a community-based, full-scale in silico model of the rat hippocampal CA1 region that integra
SwRI awarded grant to characterize Las Moras Springs watershed
Water overuse in MATOPIBA could mean failure to meet up to 40% of local demand for crop irrigation
An extra year of education does not protect against brain aging
Researchers from Uppsala and Magdeburg obtain an ERC Synergy Grant to advance cancer immunotherapy
Deaf male mosquitoes don’t mate
Recognizing traumatic brain injury as a chronic condition fosters better care over the survivor’s lifetime
SwRI’s Dr. James Walker receives Distinguished Scientist Award from Hypervelocity Impact Society
A mother’s health problems pose a risk to her children
Ensuring a bright future for diamond electronics and sensors
The American Pediatric Society selects Dr. Maria Trent as the Recipient of the 2025 David G. Nichols Health Equity Award
The first 3D view of the formation and evolution of globular clusters
[Press-News.org] Overcoming structural uncertainty in computer modelsWhat is good enough to aid health economics decision making?