Study: Addressing 'mischievous responders' would increase validity of adolescent research

'Jokester youths' can disrupt surveys and impact research, policy, and public perceptions

2014-05-15

(Press-News.org) WASHINGTON, D.C., May 15, 2014 ─ "Mischievous responders" play the game of intentionally providing inaccurate answers on anonymous surveys, a widespread problem that can mislead research findings. However, new data analysis procedures may help minimize the impact of these "jokester youths," according to research published online today in Educational Researcher, a peer-reviewed journal of the American Educational Research Association (AERA).

VIDEO: Author Joseph P. Robinson-Cimpian discusses key findings: http://youtu.be/WFFaA74sygI.

"Inaccurate Estimation of Disparities Due to Mischievous Responders: Several Suggestions to Assess Conclusions", by Joseph P. Robinson-Cimpian of the University of Illinois at Urbana-Champaign, suggests that mischievous responders are present in commonly used data sets and that their presence may severely alter impressions of relative risk between groups of adolescents. The article introduces novel sensitivity-analysis procedures for investigating and reducing the bias that mischievous responders often introduce in research on adolescents.

Mischievous responders are youths who provide extreme and potentially untruthful responses to multiple questions on self-administered questionnaires (SAQs). By providing misleading responses that they think are funny, these responders, even in small numbers, can lead researchers to wildly incorrect conclusions. In turn, the conclusions can lead to ineffective policymaking and may perpetuate negative stereotypes about marginalized groups.

In his article, Robinson-Cimpian demonstrates that even a very small proportion of mischievous responders can lead to exaggerated risk estimates for lesbian, gay, bisexual, and questioning (LGBQ), transgender, and disabled youths, with regard to drug usage, suicidal thoughts, and school disengagement. For example, potentially mischievous responders caused LGBQ–heterosexual disparities in recent cocaine/crack use to inflate from 4 percentage points to 12 percentage points.

To identify mischievous responders, Robinson-Cimpian's method relies on detecting patterns of unusual answers provided by survey respondents.

"If we find that youths reporting to be gay are more likely than those reporting to be straight to say that they are blind and deaf and extremely tall and parenting multiple children all at the same time, then we might question whether the data are valid," said Robinson-Cimpian. "Just like these jokester youths think it's funny to say that they are gay and blind, they also think it's funny to say that they are suicidal, engage in sexually risky behavior, and take drugs. And this can dramatically affect our estimates of risk."

SAQs are often used to estimate the psychological and health disparities between groups, including adoptees and non-adoptees, sexual minorities and non-minorities, racial/ethnic minorities and non-minorities, and individuals with and without disabilities, among others. The anonymity often provided by SAQs makes them an important tool in adolescent research, but Robinson-Cimpian cautions that researchers must take steps to help ensure the data are valid.

"Within the past decade, much research has been criticized for failure to replicate and for exaggerated results," wrote Robinson-Cimpian in his article. He also said, "If we want sound research and policy, we need to have sound data. The procedures introduced here have broad relevance to research and can be widely, and easily, implemented."

The proposed approach does not require additional data collection, which makes it more practical, cost-effective, and applicable to already collected data. Robinson-Cimpian's technique is unique in that it can be used in situations where it is impossible or unethical to verify information by comparing adolescents' responses with those of their parents or by conducting in-person follow-up interviews with the respondents.

The four-step sensitivity analysis proposed by Robinson-Cimpian requires first identifying youths who provide high numbers of low-frequency responses and then comparing estimated disparities when including and excluding these youths.

While it is possible that some respondents will be mistakenly identified as mischievous, or that some mischievous respondents will not provide enough low-frequency responses to be identified as mischievous, there is great potential for both scientific and practical damage when no analysis is performed to assess data validity and ensure robust results, according to Robinson-Cimpian. This includes perpetuated stigma against minority groups.

For instance, this type of sensitivity analysis may have prevented well-intentioned research from being used by groups that claim LGBTQ identification leads to substance abuse and other risky behaviors, and thus advocate against using school resources to help LGBTQ youths, wrote Robinson-Cimpian in his article.

INFORMATION: Funding Note Robinson-Cimpian's research was partially funded through the National Academy of Education/Spencer Postdoctoral Fellowship Program.

About the Author Joseph P. Robinson-Cimpian is assistant professor of quantitative and evaluative research methodologies at the University of Illinois at Urbana-Champaign.

About AERA The American Educational Research Association (AERA) is the largest national professional organization devoted to the scientific study of education. Founded in 1916, AERA advances knowledge about education, encourages scholarly inquiry related to education, and promotes the use of research to improve education and serve the public good. Find AERA on Facebook and Twitter.

This news release is available online: http://www.aera.net/Newsroom/NewsReleasesandStatements/StudyAddressingMischievousRespondersWouldIncreaseValidityofAdolescentResearch/tabid/15514/Default.aspx

ELSE PRESS RELEASES FROM THIS DATE:

Neural pathway to parenthood

2014-05-15

Good news for Dads: Harvard researchers say the key to being a better parent is – literally – all in your head. In a study in mice, Higgins Professor of Molecular and Cellular Biology and Howard Hughes Investigator Catherine Dulac have pinpointed galanin neurons in the brain's medial preoptic area (MPOA), that appear to regulate parental behavior. If similar neurons are at work in humans, it could offer clues to the treatment of conditions like post-partum depression. The study is described in a May 15 paper published in Nature. "If you look across different animal ...

Getting chemo first may help in rectal cancer

2014-05-15

PROVIDENCE, R.I. [Brown University] — First things first. If cancer patients are having trouble tolerating chemotherapy after chemoradiation and surgery, then try administering it beforehand. Reordering the regimen that way enabled all but six of 39 patients to undergo a full course of standard treatment for rectal cancer, according to research to be presented at the American Society for Clinical Oncology annual meeting in Chicago. Studies have shown that only about 60 percent of rectal cancer patients comply with postoperative chemotherapy, said lead researcher Dr. Kimberly ...

MIPT experts reveal the secret of radiation vulnerability

2014-05-15

The scientists - Boris Kuzin, Ekaterina Nikitina, Roman Cherezov, Julia Vorontsova, Mikhail Slezinger, Olga Zatsepina, Olga Simonova, Grigori Enikolopov and Elena Savvateeva-Popova - studied Drosophila flies, in whose genome weak mutations of two different genes were combined. The paper is published in the PLoS One. They concluded that these mutations synergistically strengthen their mutual phenotypic expression. In other words, the aggregate effect of these mutations is much greater than that which can be produced by one of them individually. The mutant flies bred by ...

MIPT scientists develop algorithm for anti-aging remedy search

2014-05-15

The scientists – Alexander Zhavoronkov, Anton Buzdin, Andrey Garazha, Nickolay Borisov and Alexey Moskalev– have based the new research on their previously-developed methods in the study of cancer cells. Each cell uses particular schemes of molecular interaction, which physiologists call intercellular signaling pathways. A signaling pathway is a chain of sequential events of interaction between certain molecules which make the cell respond to stimulation. For example, hormone molecules first interact with the cell's membrane receptors, then the receptors engage with the ...

Next frontier: How can modern medicine help dying patients achieve a 'good' death?

2014-05-15

(TORONTO, Canada – May, 15, 2014) -- The overall quality of death of cancer patients who die in an urban Canadian setting with ready access to palliative care was found to be good to excellent in the large majority of cases, helping to dispel the myth that marked suffering at the end of life is inevitable. "Fear of dying is something almost every patient with advanced cancer or other life-threatening illness faces, and helping them, to achieve a "good death" is an important goal of palliative care," says Dr. Sarah Hales, Coordinator of Psychiatry Services, Psychosocial ...

Study shows young men increasingly outnumber young women in rural Great Plains

2014-05-15

Lincoln, Neb., May 15, 2014 -- In many rural communities hard hit by decades of population declines, young men increasingly outnumber young women, a new study of Kansas and Nebraska census data shows. In places with 800 or fewer residents, the proportion of young men increased by an average of nearly 40 percent as people went from their teens to their 20s. Those findings suggest leaders should consider the needs of young women in their economic and community development plans, said Robert Shepard, a University of Nebraska-Lincoln doctoral candidate ...

Most NHL players peak by age 29: Study

2014-05-15

A new University of British Columbia study identifies when the clock runs out on an NHL player's peak performance, giving team executives insight into how best to build a roster. The study by Sauder School of Business professor James Brander found that the performance of forwards peaks between the ages of 27 and 28. Defencemen are best between 28 and 29, and the performance of goaltenders varies little by age. The forthcoming study to be published in the Journal of Quantitative Analysis in Sports also reveals that players performed close to their peak levels for a ...

Learning from sharks

2014-05-15

This news release is available in German. Custom-tailored antibodies are regarded as promising weapons against a multitude of serious illnesses. Since they can accurately recognize specific structures on the surface of viruses, bacteria or cancer cells, they are already being deployed successfully in cancer diagnostics and therapy, as well as against numerous other diseases. The stability of the sensitive antibodies is a decisive factor in every step, from production and storage to therapeutic application. A team of researchers headed by Dr. Matthias J. Feige and ...

Where have all the mitochondria gone?

2014-05-15

It's common knowledge that all organisms inherit their mitochondria – the cell's "power plants" – from their mothers. But what happens to all the father's mitochondria? Surprisingly, how – and why – paternal mitochondria are prevented from getting passed on to their offspring after fertilization is still shrouded in mystery; the only thing that's certain is that there must be a compelling reason, seeing as this phenomenon has been conserved throughout evolution. Now, Dr. Eli Arama and a team in the Weizmann Institute's Molecular Genetics Department have discovered special ...

Sugar implicated in cardiovascular disease risk independent of weight gain

2014-05-15

Researchers from New Zealand's University of Otago have uncovered evidence that sugar has a direct effect on risk factors for heart disease, and is likely to impact on blood pressure, independent of weight gain. Research Fellow with Otago's Department of Human Nutrition Dr Lisa Te Morenga, Professor Jim Mann and colleagues have conducted a review and meta-analysis of all international studies that compared the effects of higher versus lower added sugar consumption on blood pressure and lipids (blood fats or cholesterol) – both of which are important cardiovascular risk-factors. They ...

Study: Addressing 'mischievous responders' would increase validity of adolescent research

ELSE PRESS RELEASES FROM THIS DATE:

LAST 30 PRESS RELEASES: