Crowdsourced testers prefer new cyber search method

(Press-News.org) Computer scientists at Case Western Reserve University have developed a new tool to search and fetch electronic files that saves users time by more quickly identifying and retrieving the most relevant information on their computers and hand-held devices.

Anonymous testers recruited through crowdsourcing preferred the new search tool nearly two-to-one over a keyword-based lookup interface and the most commonly available lookup search interface using Google.

Side-by-side comparisons showed the scientists' Conjunctive Exploratory Navigation Interface (CENI), which combines two search modes and a more comprehensive way to organize and tag data, is more effective than looking up items by matched keywords alone.

CENI, an on-screen portal where users access data by browsing through menus of topics and typing in keywords, provides a more focused search and retrieves the most pertinent information.

In one test, for example, a keyword search came up with 89 responses to a question: "What are the typical vision problems associated with diabetes?" CENI came up with the most applicable 13 by selecting appropriate menus.

The study is now online in the open-access Journal of Medical Internet Research at http://www.jmir.org/2014/2/e45/.

"Most people have an iPhone or laptop that stores a wide variety of information and, often, you can't find it when you need it, even though you know it's there," said GQ Zhang, professor of electrical engineering and computer science, division chief of Medical Informatics at Case Western Reserve and an author of the study. "Or, you go to a website where the content has been divided under different areas, and what you're looking for fits several. If you choose one area but whoever filed the data chose another area, you may not find that information."

CENI overcomes this limitation by allowing data to be tagged into as many areas as relevant, and provides an interface and system that leverages multiple tags for each single data item.

Zhang and Licong Cui, a PhD student in Zhang's lab, developed CENI.

Prototype On Health Website

They have a working prototype designed specifically for the health resource website, NetWellness. This not-for-profit site allowed the public to ask health professionals at Case Western Reserve, Ohio State University and the University of Cincinnati health-related questions. More than 60,000 questions and answers are searchable using CENI. The interface is currently not available to the public.

Health information is highly sought after. A Pew Foundation survey found that 80 percent of Internet users have searched for health information, and 60 percent used that information to help make health-care decisions.

But a study in the Journal of the American Medical Association concluded that accessing health information using simple terms on such search engines as Google and Yahoo was inefficient. Less than a quarter of the searches led to relevant information, the study found.

This kind of search, called "lookup," can overwhelm the user with a long list of document links the user must then sift through, Zhang said. "If results do not show up in the first couple of pages, they are lost because the user is not going to go through millions of links manually."

CENI combines lookup and another search method, called "exploratory navigation." The exploratory mode enables users who lack a specific target or have trouble forming descriptive lookup terms to use menus of topics to navigate and explore information.

Exploratory navigation, however, has limitations if each item is tagged with a single topic. On the NetWellness site, for example, a question concerning kidney disease because the patient suffers pain in the kidney region, especially after taking certain medicines, may be tagged under a single topic: kidney disease, pain or the medicine.

To make questions and answers less hit-or-miss, "we designed a simple approach called 'multi-tagging' that allows the system to tag as many topics as it takes to cover the bases," Zhang said. The menus produced would enable a person suffering from multiple issues, like the one in the question above, to click on any of several available topics and quickly narrow the search to questions and answers most relevant to them.

To be useful to a greater audience, the scientists built text analytics into CENI and annotated tag words to account for synonyms and related concepts, Cui explained. "So if you didn't explicitly type a tag word," she said, "you would still get the content."

"And you can find something if you use only descriptive properties," Zhang said. "If you're searching for a needle and can only remember small pointy metal thing, you should still be able to find it."

Anonymous Testers

To evaluate the interface for consumer use, the researchers recruited 90 testers through Amazon Mechanical Turk, a crowdsourcing marketplace. Rebecca Carter, a PhD student in epidemiology and biostatistics at Case Western Reserve School of Medicine, assisted in the crowdsourcing design.

The testers were asked to perform nine tasks, with six searches in lookup mode and three in exploratory. Overall, 96 percent found relevant answers using the researcher's CENI interface, 89 percent using NetWellness' keyword-based search and 87 percent using Google.

They rated the new interface easier to use than the others.

"I liked being able to easily select multiple topics to narrow my search quickly," said one tester. "I found it to be the easiest search method because it decreased the number of unrelated search results."

A second said, "I could click on any number of general topics and type in something to narrow down the search and quickly get what I was searching for."

"It helps even a less-experienced user to find and target the topics and answers he is looking for," said another.

Although most respondents said the interface was easier to use, some of them, nonetheless, preferred the other modes because they were more familiar.

For More Than Health

Cui currently uses CENI on her laptop, to organize, search and fetch all of her files. "This technology isn't specific to the NetWellness' health question content, but can be used to organize and retrieve most any kind of information," she said.

Zhang and Cui have filed for a patent on the approach.

For a sports fan who wants to monitor the latest news and events, few articles in sports sections actually use the word "sport." So a keyword search using "sports" would most likely miss the majority of relevant information. Using the CENI approach, "sports" would be expanded to include the specific names of different sports, key players, major events and more, the researchers say. But the user would just select "sports" in the topic menu to collect the list of article links.

"It could be used to transform the organization of many websites using multi-tagging," Zhang said. "And, down the road, it could be used to reorganize digital contents on our desktop computers or portable devices, to make it easier for information retrieval."

INFORMATION: END