Carnegie Mellon, Disney researchers develop acoustically driven controls for smartphones

PITTSBURGH--As people find ever more inventive uses for smartphones, touchscreens sometimes fall short as control surfaces. Researchers at Carnegie Mellon University and Disney Research have developed an inexpensive alternative - a toolbox of physical knobs, sliders and other mechanisms that can be readily added to any device. The researchers drew inspiration from wind instruments in devising these mechanisms, which they call Acoustruments. The idea is to use pluggable plastic tubes and other structures to connect the smartphone's speaker with its microphone. The device can then be controlled by acoustically altering sounds as they pass through this system. Just as a simple slide whistle or flute can produce expressive music, these Acoustruments can add a wide range of functionality to a smartphone, including proximity and pressure sensors. And because no electrical circuitry is involved, the plastic Acoustruments can be made rapidly and inexpensively. "We're providing people with tangible interactivity at basically no cost," said Gierad Laput, a Ph.D. student in Carnegie Mellon's Human-Computer Interaction Institute (HCII). Laput is the lead author of a study that will be presented April 22 at CHI 2015, the Conference on Human Factors in Computing Systems, in Seoul, Korea, and has been named a Best Paper by the conference organizers. "Using smartphones as computers to control toys, appliances and robots already is a growing trend, particularly in the maker community," Laput added. "Acoustruments can make the interactivity of these new 'pluggable' applications even richer."

People who want to control a smartphone application while their eyes need to be focused elsewhere, for instance, might be less distracted by a tangible knob or button made possible with Acoustruments than with trying to glance at a touchscreen. Applications that use smartphones as virtual reality displays make it impossible to use the touchscreen controls at all; Acoustruments enable users to make adjustments without disrupting the virtual reality experience. The CMU and Disney researchers have used Acoustruments to build an interactive doll, which responds when its tummy is poked; a smartphone case that can sense when it has been placed on a table or is being hand carried; and an alarm clock that provides physical on/off and snooze buttons. Wind instruments take a sustained source of sound from a mouthpiece and then alter it by changing the cavity - with a series of holes, as in a flute, or by changing its size, as with a trombone. Acoustruments operate in a similar manner; the smartphone speaker produces continuous "sweeps" of ultrasonic frequencies; interactions that block, open holes or change the length or diameter of the plastic tubes connecting the speaker to the microphone alter this acoustic signal. Experiments by the CMU and Disney researchers showed that Acoustruments can achieve 99 percent accuracy in controlling the device. Acoustruments can be made with 3-D printers, with injection molds, or even by hand in some cases, Laput said. The ultrasonic frequencies are inaudible to people and the pluggable structures are designed to block out interference from external noise.

INFORMATION:

In addition to Laput, the research team included Eric Brockmeyer of Disney Research Pittsburgh and two HCII faculty members, Scott Hudson and Chris Harrison. The research was sponsored by Disney Research.

About Carnegie Mellon University Carnegie Mellon is a private, internationally ranked research university with programs in areas ranging from science, technology and business, to public policy, the humanities and the arts. More than 13,000 students in the university's seven schools and colleges benefit from a small student-to- faculty ratio and an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration and innovation. A global university, Carnegie Mellon has campuses in Pittsburgh, Pa., California's Silicon Valley and Qatar, and programs in Africa, Asia, Australia, Europe and Mexico.

About Disney Research Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific, and technological innovation to advance the company's broad media and entertainment efforts. Disney Research is managed by an internal Disney Research Council co-chaired by Disney-Pixar's Ed Catmull and Walt Disney Imagineering's Bruce Vaughn, and including the directors of the individual labs. It has facilities in Los Angeles, San Francisco, Pittsburgh, Boston and Zurich. Research topics include Computer Graphics, Video Processing, Computer Vision, Robotics, Radio and Antennas, Wireless Communications, Human-Computer Interaction, Displays, Data Mining, Machine Learning and Behavioral Sciences.