(Press-News.org) SALT LAKE CITY, Aug. 14, 2015 - Software may appear to operate without bias because it strictly uses computer code to reach conclusions. That's why many companies use algorithms to help weed out job applicants when hiring for a new position.
But a team of computer scientists from the University of Utah, University of Arizona and Haverford College in Pennsylvania have discovered a way to find out if an algorithm used for hiring decisions, loan approvals and comparably weighty tasks could be biased like a human being.
The researchers, led by Suresh Venkatasubramanian, an associate professor in the University of Utah's School of Computing, have discovered a technique to determine if such software programs discriminate unintentionally and violate the legal standards for fair access to employment, housing and other opportunities. The team also has determined a method to fix these potentially troubled algorithms.
Venkatasubramanian presented his findings Aug. 12 at the 21st Association for Computing Machinery's Conference on Knowledge Discovery and Data Mining in Sydney, Australia.
"There's a growing industry around doing resume filtering and resume scanning to look for job applicants, so there is definitely interest in this," says Venkatasubramanian. "If there are structural aspects of the testing process that would discriminate against one community just because of the nature of that community, that is unfair."
Machine-learning algorithms
Many companies have been using algorithms in software programs to help filter out job applicants in the hiring process, typically because it can be overwhelming to sort through the applications manually if many apply for the same job. A program can do that instead by scanning resumes and searching for keywords or numbers (such as school grade point averages) and then assigning an overall score to the applicant.
These programs also can learn as they analyze more data. Known as machine-learning algorithms, they can change and adapt like humans so they can better predict outcomes. Amazon uses similar algorithms so they can learn the buying habits of customers or more accurately target ads, and Netflix uses them so they can learn the movie tastes of users when recommending new viewing choices.
But there has been a growing debate on whether machine-learning algorithms can introduce unintentional bias much like humans do.
"The irony is that the more we design artificial intelligence technology that successfully mimics humans, the more that A.I. is learning in a way that we do, with all of our biases and limitations," Venkatasubramanian says.
Disparate impact
Venkatasubramanian's research determines if these software algorithms can be biased through the legal definition of disparate impact, a theory in U.S. anti-discrimination law that says a policy may be considered discriminatory if it has an adverse impact on any group based on race, religion, gender, sexual orientation or other protected status.
Venkatasubramanian's research revealed that you can use a test to determine if the algorithm in question is possibly biased. If the test -- which ironically uses another machine-learning algorithm -- can accurately predict a person's race or gender based on the data being analyzed, even though race or gender is hidden from the data, then there is a potential problem for bias based on the definition of disparate impact.
"I'm not saying it's doing it, but I'm saying there is at least a potential for there to be a problem," Venkatasubramanian says.
If the test reveals a possible problem, Venkatasubramanian says it's easy to fix. All you have to do is redistribute the data that is being analyzed -- say the information of the job applicants -- so it will prevent the algorithm from seeing the information that can be used to create the bias.
"It would be ambitious and wonderful if what we did directly fed into better ways of doing hiring practices. But right now it's a proof of concept," Venkatasubramanian says.
INFORMATION:
In addition to Venkatasubramanian, the research also was conducted by University of Arizona Computer Science assistant professor Carlos Scheidegger, Haverford College Computer Science assistant professor Sorelle Friedler, University of Utah doctoral student John Moeller and Haverford undergraduate student Michael Feldman.
University of Utah College of Engineering
72 S. Central Campus Dr., Room 1650 WEB
Salt Lake City, UT 84112
801-581-6911 fax: 801-581-8692
http://www.coe.utah.edu
Discovered in 1982, Helicobacter pylori (H. pylori) is a disease-causing bacterium that survives in our stomachs despite the harsh acidic conditions. It is estimated that one in two people have got it, though most won't ever experience any problems. Even so, it is considered one of the most common bacterial infections worldwide and a leading cause of dyspepsia, peptic ulceration and gastric cancer.
Through unique evolutionary adaptations, H. pylori is able to evade the antiseptic effect of our stomach acid by hiding within the thick acid-resistant layer of mucus that ...
Washington, DC - August 14, 2015 - Tungsten is exceptionally rare in biological systems. Thus, it came as a huge surprise to Michael Adams, PhD., and his collaborators when they discovered it in what appeared to be a novel enzyme in the hot spring-inhabiting bacterium, Caldicellulosiruptor bescii. The researchers hypothesized that this new tungstoenzyme plays a key role in C. bescii's primary metabolism, and its ability to convert plant biomass to simple fermentable sugars. This discovery could ultimately lead to commercially viable conversion of cellulosic (woody) biomass ...
What happens when a drought in Florida estuaries causes a rise in the salt levels in water? Fewer wild oysters appear on restaurant menus, for starters.
New research from Northeastern University marine and environmental sciences professor David Kimbro and graduate student Hanna Garland, published in PLOS ONE, links the deterioration of oyster reefs in Florida's Matanzas River Estuary (MRE) to a population outbreak of carnivorous conchs and ...
OAK RIDGE, Tenn., Aug. 14, 2015 - Another barrier to commercially viable biofuels from sources other than corn has fallen with the engineering of a microbe that improves isobutanol yields by a factor of 10.
The finding of the Department of Energy's BioEnergy Science Center, published in the journal Metabolic Engineering, builds on results from 2011 in which researchers reported on the first genetically engineered microbe to produce isobutanol directly from cellulose.
Isobutanol is attractive because its energy density and octane values are much closer to gasoline and ...
Americans favor diplomatic and economic strategies over military involvement in foreign policy, according to a new national survey conducted by The Associated Press-NORC Center for Public Affairs Research. Americans also question whether or not the United States should be the world's chief problem solver, even as a myriad of issues across the globe are identified as important for the next president to address. The nationwide poll of 1,167 adults collected data from June 25 to July 7 using AmeriSpeak, the probability-based panel of NORC at the University of Chicago. Interviews ...
Early-stage Non-small-cell Lung Cancer (NSCLC) is asymptomatic and difficult to detect since no blood test for NSCLC is currently available. In a new study, Chen-Yu Zhang and Chunni Zhang's group at Nanjing Advanced Institute for Life Sciences, Nanjing University identified a panel of five serum microRNAs (miRNAs) as the potential biomarker for NSCLC diagnosis.
The study is published this week in the journal EBioMedicine.
MiRNAs are a family of small, single-stranded non-coding RNAs that are critical regulators of numerous diseases, and their expression patterns have ...
Researchers at Columbia University's National Center for Disaster Preparedness (NCDP) and the University of Washington have published a new study focused on the public health implications of climate change. The article explores climate change impacts on human health in the U.S. Gulf Coast and has implications for this and other coastal regions that are particularly vulnerable to climate change. The study appears in the International Journal of Environmental Research and Public Health (August 11, 2015). The Open Access article is available here: http://bit.ly/1gAVqVe
This ...
Rockville, Md. -- A new study may have unlocked understanding of a mysterious part of the brain -- with implications for neurodegenerative conditions such as Alzheimer's. The results, published in Translational Vision Science & Technology (TVST), open up new areas of research in the pursuit of neuroprotective therapies.
Glaucoma is a neurodegenerative disease where patients lose seemingly random patches of vision in each eye. This random pattern of vision loss is in stark contrast to loss from a brain tumor or stroke, which causes both eyes to develop blind spots in the ...
The liver is unique among organs in its ability to regenerate after being damaged. Exactly how it repairs itself remained a mystery until recently, when researchers supported by the National Institutes of Health discovered a type of cell in mice essential to the process. The researchers also found similar cells in humans.
When healthy liver cells are depleted by long-term exposure to toxic chemicals, the newly discovered cells, known as hybrid hepatocytes, generate new tissue more efficiently than normal liver cells. Importantly, they divide and grow without causing cancer, ...
Physicists from Ludwig-Maximilians-Universität (LMU) in Munich studied the interaction of light with tiny glass particles.
A team of physicists and chemists from Ludwig-Maximilians-Universität Munich (LMU) and the Laboratory of Attosecond Physics (LAP) at the Max Planck Institute of Quantum Optics (MPQ), from the Institute of Physics of the University of Rostock, and from the Freie Universität Berlin studied the interaction between strong laser pulses and glass nanoparticles, which consist of multiple millions of atoms. Depending on how many atoms were ...
