New chatbot can explain apps and show you how they access hardware or data
Hey GUI helps you find information on apps and their user interfaces with a simple conversation rather than complex tools or code
2021-07-01
(Press-News.org) Chatbots have already become a part of our everyday lives with their quick and intuitive way to complete tasks like scheduling and finding information using natural language conversations. Researchers at Aalto University have now harnessed the power of chatbots to help designers and developers develop new apps and allow end users to find information on the apps on their devices.
The chatbot -- 'Hey GUI' (pronounced goo-ee), short for Graphical User Interface, which will be presented at ACM Designing Interactive Systems 2021 on 1 July -- can answer questions by showing images and screenshots of apps, or through simple text phrases.
"Hey GUI eliminates the need for coding skills or technical expertise. Anyone can use it to search for information on, for example, which one of my apps is asking for permission to use the camera, or where is the search button on this screen", says postdoctoral researcher Kashyap Todi of Aalto University.
Important findings for chatbot developers and researchers
Different kinds of applications and user interfaces contain vast amounts of information, which can be used for different needs and tasks. In their study, the researchers set out to find the most desirable chatbot features and how users would have conversations with them.
Three target groups were considered in the experiment:
designers, software developers, and general users. Over 100 designers, developers, and ordinary users responded to the survey.
"Our goal was to get a good understanding of what capabilities and functions different kinds of users would find most beneficial when using this new type of chatbot, as well as how they would formulate questions -- especially what kinds of words and phrases they would use. This is an important first step towards developing chatbots that can help users find information about apps using simple conversations", says Kashyap.
As a result of the survey, 21 key features and more than 1300 of the most common questions put forward by users were collected and analysed to create Hey GUI, allowing users to find information within user interfaces using simple conversations, based on the most frequent expressions collected in the survey.
"Designers can get inspiration, for instance, by asking Hey GUI to find good online shop designs. The chatbot responds with several screenshots from existing apps in the catalogue. Software developers could ask a chatbot what typical colour codes are used for login buttons, and the chatbot responds with several options", Kashyap adds.
In questions from ordinary users, there was a special emphasis on matters related to privacy and accessibility information, or getting help about their apps, such as asking which apps display maps.
The research prototype is currently under development. The common questions gathered from the survey are available at the website. (https://osf.io/g97my/)
The study was conducted in collaboration with the universities of Luxembourg and Bayreuth.
INFORMATION:
For more information, see the project page: https://userinterfaces.aalto.fi/hey_gui
Read the full article published at ACM DIS 2021: https://dl.acm.org/doi/10.1145/3461778.3462124
Watch a short video with a demonstration of the chatbot: https://youtu.be/F5MdXITm9RY
Watch a long video (six minutes) with a demonstration of the chatbot: https://www.youtube.com/watch?v=vB_OH-Eky-Q
[Attachments] See images for this press release:
ELSE PRESS RELEASES FROM THIS DATE:
2021-07-01
Hundreds of different bacterial species live in and on leaves and roots of plants. A research team led by Julia Vorholt from the Institute of Microbiology at ETH Zurich, together with colleagues in Germany, first inventoried and categorised these bacteria six years ago. Back then, they isolated 224 strains from the various bacterial groups that live on the leaves of thale cress (Arabidopsis thaliana). These can be assembled into simplified, or "synthetic" plant microbiomes. The researchers thus laid the foundations for their two new studies, which were just published in the journals Nature Plants and Nature Microbiology.
Volume control of the plant response
In the first study, the researchers investigated ...
2021-07-01
A team of scientists has devised a more accurate way to predict the effects of climate change on plants and animals -- and whether some will survive at all.
Frequently, ecologists assess an organism's fitness relative to the climate by quantifying its functional traits.
"These are physical properties you can measure -- height, diameter, the thickness of a tree," said UC Riverside biologist Tim Higham. "We believe more information is needed to understand how living things will respond to a changing world."
The team, led by Higham, outlines an alternative model for researchers in an article ...
2021-07-01
NEW YORK, NY (July 1, 2021)--Lowering levels of a hormone called PTHrP can prevent metastases and improve survival in mice with pancreatic cancer and could lead to a new way to treat patients, according to a study from cancer researchers at Columbia University Vagelos College of Physicians and Surgeons and Herbert Irving Comprehensive Cancer Center and with collaborators at the University of Pennsylvania.
When patients are first diagnosed with pancreatic cancer, the cancer usually has spread to other organs. Because of these metastases, nearly all patients will succumb to their cancer within one year of diagnosis, but no drugs exist to prevent metastasis.
In an effort to find treatments, cancer researchers at Columbia--led by ...
2021-07-01
What if a microscope allowed us to explore the 3D microcosm of blood vessels, nerves, and cancer cells instantaneously in virtual reality? What if it could provide views from multiple directions in real time without physically moving the specimen and worked up to 100 times faster than current technology?
UT Southwestern scientists collaborated with colleagues in England and Australia to build and test a novel optical device that converts commonly used microscopes into multiangle projection imaging systems. The invention, described in an article in today's Nature Methods, could open new avenues ...
2021-07-01
SAN FRANCISCO, CA--June 28, 2021--While vaccines are doing a remarkable job of slowing the COVID-19 pandemic, infected people can still die from severe illness and new medications to treat them have been slow to arise. What kills these patients in the end doesn't seem to be the virus itself, but an over-reaction of their immune system that leads to massive inflammation and tissue damage.
By studying a type of immune cells called T cells, a team of Gladstone scientists has uncovered fundamental differences between patients who overcome severe COVID-19 and those who succumb to it. The team, working together with researchers from UC San Francisco and Emory University, also found that dying patients harbor relatively large numbers of T cells able to infiltrate the lung, which may contribute ...
2021-07-01
BUFFALO, N.Y. -- Patients with Type 2 diabetes who were prescribed SGLT2 inhibitors lost more weight than patients who received GLP-1 receptor agonists, according to a University at Buffalo-led study.
The research, which sought to evaluate the difference in weight loss caused by the antidiabetic medications -- both of which work to control blood sugar levels -- found that among 72 patients, people using SGLT2 inhibitors experienced a median weight loss of more than 6 pounds, while those on GLP-1 receptor agonists lost a median of 2.5 pounds.
The findings, published last month ...
2021-07-01
The adult brain is more malleable than previously thought, according to researchers from the Interdisciplinary Center Herzliya. They trained a 50-year-old man, blind from birth, to "see" by ear, and found that neural circuits in his brain formed so-called topographic maps - a type of brain organization previously thought to emerge only in infancy. This finding reported recently in END ...
2021-07-01
Rochester, Minn. -- Mayo Clinic researchers have discovered that genetic variants in a neuro-associated gene called SPTBN1 are responsible for causing a neurodevelopmental disorder. The study, published in Nature Genetics, is a first step in finding a potential therapeutic strategy for this disorder, and it increases the number of genes known to be associated with conditions that affect how the brain functions.
"The gene can now be included in genetic testing for people suspected of having a neurodevelopmental disorder, which may end the diagnostic odyssey these people and their families have endured," says Margot Cousin, Ph.D., a translational ...
2021-07-01
UNIVERSITY PARK, Pa. -- In the late 1800s, scientists were stumped by the "yellow cells" they were observing within the tissues of certain temperate marine animals, including sea anemones, corals and jellyfish. Were these cells part of the animal or separate organisms? If separate, were they parasites or did they confer a benefit to the host?
In a paper published in the journal Nature in 1882, biologist Sir Patrick Geddes of Edinburgh University proffered that not only were these cells distinct entities, but they were also beneficial to the animals in which they lived. He assigned them to a new genus, Philozoon -- from the Greek phileo, meaning 'to love ...
2021-07-01
Giving women in India's Madhya Pradesh state greater digital control over their wages encouraged them to enter the labor force and liberalized their beliefs about working women, concluded a new study co-authored by Yale economists Rohini Pande and Charity Troyer Moore.
The study, published in the American Economic Review, found that a relatively simple intervention directed to poor women -- providing them access to their own bank accounts and direct deposit for their earnings from a federal workfare program, along with basic training on how to use local bank kiosks -- increased the amount ...
LAST 30 PRESS RELEASES:
[Press-News.org] New chatbot can explain apps and show you how they access hardware or data
Hey GUI helps you find information on apps and their user interfaces with a simple conversation rather than complex tools or code
