(Press-News.org) Researchers from the University of Southampton are designing incentives for collection and verification of information to make crowdsourcing more reliable.
Crowdsourcing is a process of outsourcing tasks to the public, rather than to employees or contractors. In recent years, crowdsourcing has provided an unprecedented ability to accomplish tasks that require the involvement of a large number of people, often across wide-spread geographies, expertise, or interests.
The world's largest encyclopaedia, Wikipedia, is an example of a task that can only be achieved through crowd participation. Crowdsourcing is not limited to volunteer efforts. For example, Amazon Mechanical Turk (AMT) and CrowdFlower are 'labour on demand' markets that allow people to get paid for micro-tasks, as simple as labelling an image or translating a piece of text.
Recently, crowdsourcing has demonstrated effectiveness in large-scale, information-gathering tasks, across very wide geographies. For example, the Ushahidi platform allowed volunteers to perform rapid crisis mapping in real-time in the aftermath of disasters such as the Haiti earthquake.
One of the main obstacles in crowdsourcing information gathering is reliability of collected reports. Now Dr Victor Naroditskiy and Professor Nick Jennings from the University of Southampton, together with Masdar Institute's Professor Iyad Rahwan and Dr Manuel Cebrian, Research Scientist at the University of California, San Diego (UCSD), have developed novel methods for solving this problem through crowdsourcing. The work, which is published in the academic journal PLOS ONE, shows how to crowdsource not just gathering, but also verification of information.
Dr Victor Naroditskiy of the Agents, Interaction and Complexity group at the University of Southampton, and lead author of the paper, says: "The success of an information gathering task relies on the ability to identify trustworthy information reports, while false reports are bound to appear either due to honest mistakes or sabotage attempts. This information verification problem is a difficult task, which, just like the information-gathering task, requires the involvement of a large number of people."
Sites like Wikipedia have existing mechanisms for quality assurance and information verification. However, those mechanisms rely partly on reputation, as more experienced editors can check whether an article conforms to the Wikipedia objectivity criteria, has sufficient citations, etc. In addition, Wikipedia has policies for resolving conflicts between editors in cases of disagreement.
However, in time-critical tasks, there is no established hierarchy of participants, and little basis for judging credibility of volunteers who are recruited on the fly. In this kind of scenario, special incentives are needed to carry out verification. The research presented in the PLOS ONE paper provides such incentives.
Professor Iyad Rahwan of Masdar Institute in Abu Dhabi and a co-author of the paper, explains: "We showed how to combine incentives to recruit participants to verify information. When a participant submits a report, the participant's recruiter becomes responsible for verifying its correctness.
Compensations to the recruiter and to the reporting participant for submitting the correct report, as well as penalties for incorrect reports, ensure that the recruiter will perform verification."
Incentives to recruit participants have previously been proposed by Dr Manuel Cebrian from UCSD, and a co-author of the paper, to win the DARPA Red Balloon Challenge, where teams had to locate 10 weather balloons positioned at random locations throughout the United States. In that scheme, where the person who found the balloons received a pre-determined compensation, for example $1,000, his recruiter received $500 and the recruiter of the recruiter got $250. Dr Manuel Cebrian says: "The results on incentives to encourage verification provide theoretical justification for the incentives used to win the Red Balloon Challenge."
### END
Making crowdsourcing more reliable
2012-10-11
ELSE PRESS RELEASES FROM THIS DATE:
Testosterone increases honesty
2012-10-11
Testosterone is considered THE male hormone, standing for aggression and posturing. Researchers around Prof. Dr. Armin Falk, an economist from the University of Bonn, have now been able to demonstrate that this sex hormone surprisingly also fosters social behavior. In play situations, subjects who had received testosterone clearly lied less frequently than individuals who had only received a placebo. The results have just been published in the Public Library of Science's international online journal "PLoS ONE."
The hormone testosterone stands for typically male attributes ...
Melanoma - The wolf in sheep's clothing
2012-10-11
Melanoma is so dangerous because it tends to metastasize early on. New treatment approaches utilize, among other things, the ability of the immune defense to search out and destroy malignant cells. Yet this strategy is often only temporarily effective. A research team under the direction of Bonn University has discovered why this is the case: In the inflammatory reaction caused by the treatment, the tumor cells temporarily alter their external characteristics and thus become invisible to defense cells. This knowledge forms an important foundation for the improvement of ...
High levels of blood-based protein specific to mesothelioma
2012-10-11
NEW YORK, October 11, 2012 – Researchers at NYU School of Medicine have discovered the protein product of a little-known gene may one day prove useful in identifying and monitoring the development of mesothelioma in early stages, when aggressive treatment can have an impact on the progression of disease and patient prognosis.
"This gene produces a protein, fibulin-3, that is present in levels four to five times higher in the plasma of patients with mesothelioma compared to levels in asbestos-exposed patients or patients with several other conditions that cause tumors ...
The best of both catalytic worlds
2012-10-11
Catalysts are substances that speed up the rates of chemical reactions without themselves being chemically changed. Industrial catalysts come in two main types - heterogeneous, in which the catalyst is in a different phase from the reactants; and homogeneous, in which catalyst and the reactants are in the same phase. Heterogeneous catalysts are valued for their sustainability because they can be recycled. Homogeneous catalysts are valued for their product selectivity as their properties can be easily tuned through relatively simple chemistry.
Researchers with the U.S. ...
Mine your business: Text mining insights from social media
2012-10-11
NEW YORK - October 10, 2012 - Thanks to blogs, online forums, and product review sites, companies and marketers now have access to a seemingly endless array of data on consumers' opinions and experiences. In principle, businesses should be able to use this information to gain a better understanding of the general market and of their own and their competitors' customers.
Yet this wealth of consumer-generated content can be both a blessing and a curse. A new approach, described in a study by Oded Netzer, the Philip H. Geier Jr. Associate Professor at Columbia Business School, ...
Light might prompt graphene devices on demand
2012-10-11
HOUSTON – (Oct. 10, 2012) – Rice University researchers are doping graphene with light in a way that could lead to the more efficient design and manufacture of electronics, as well as novel security and cryptography devices.
Manufacturers chemically dope silicon to adjust its semiconducting properties. But the breakthrough reported in the American Chemical Society journal ACS Nano details a novel concept: plasmon-induced doping of graphene, the ultrastrong, highly conductive, single-atom-thick form of carbon.
That could facilitate the instant creation of circuitry – ...
Fly like an eagle: New launch and recovery system takes UAV into the future
2012-10-11
A shipboard-capable system designed to support both the launch and recovery of the Scan Eagle unmanned aerial vehicle (UAV) successfully completed final demonstration flight testing Sept. 27 at a testing range in eastern Oregon.
Sponsored by the Office of Naval Research (ONR), the Compact Launch and Recovery System (CLRE) will provide a small-scale solution for the unmanned surveillance craft's operations.
"This system's shipboard capability is unique," said John Kinzer, who manages ONR's Air Vehicle Technology Program. "It's more compact than other systems, so you ...
Improving nanometer-scale manufacturing with infrared spectroscopy
2012-10-11
One of the key achievements of the nanotechnology era is the development of manufacturing technologies that can fabricate nanostructures formed from multiple materials. Such nanometer-scale integration of composite materials has enabled innovations in electronic devices, solar cells, and medical diagnostics.
While there have been significant breakthroughs in nano-manufacturing, there has been much less progress on measurement technologies that can provide information about nanostructures made from multiple integrated materials. Researchers at the University of Illinois ...
The good, the bad, and the guilty: Anticipating feelings of guilt predicts ethical behavior
2012-10-11
From politics to finance, government to education, ethics-related scandals seem to crop up with considerable regularity. As whistleblowers and investigative journalists bring these scandals to light, one can't help but wonder: Are there specific character traits that predispose people to unethical behavior?
Converging evidence suggests that the answer could be guilt proneness.
In a new article in Current Directions in Psychological Science, a journal of the Association for Psychological Science, researchers Taya Cohen and Nazli Turan of Carnegie Mellon University and ...
A planetary nebula gallery
2012-10-11
This gallery shows four planetary nebulas from the first systematic survey of such objects in the solar neighborhood made with NASA's Chandra X-ray Observatory. The planetary nebulas shown here are NGC 6543, also known as the Cat's Eye, NGC 7662, NGC 7009 and NGC 6826. In each case, X-ray emission from Chandra is colored purple and optical emission from the Hubble Space Telescope is colored red, green and blue.
In the first part of this survey, published in a new paper, twenty one planetary nebulas within about 5000 light years of the Earth have been observed. The paper ...