AI learns to type on a phone like humans
Computational model precisely replicates eye and finger movements of touchscreen users -- could lead to better auto-correct and keyboard usability for unique needs
Touchscreens are notoriously difficult to type on. Since we can't feel the keys, we rely on the sense of sight to move our fingers to the right places and check for errors, a combination of efforts we can't pull off at the same time. To really understand how people type on touchscreens, researchers at Aalto University and the Finnish Center for Artificial Intelligence (FCAI) have created the first artificial intelligence model that predicts how people move their eyes and fingers while typing.
The AI model can simulate how a human user would type any sentence on any keyboard design. It makes errors, detects them -- though not always immediately -- and corrects them, very much like humans would. The simulation also predicts how people adapt to alternating circumstances, like how their writing style changes when they start using a new auto-correction system or keyboard design.
'Previously, touchscreen typing has been understood mainly from the perspective of how our fingers move. AI-based methods have helped shed new light on these movements: what we've discovered is the importance of deciding when and where to look. Now, we can make much better predictions on how people type on their phones or tablets,' says Dr. Jussi Jokinen, who led the work.
The study, to be presented at ACM CHI on 12 May, lays the groundwork for developing, for instance, better and even personalized text entry solutions.
'Now that we have a realistic simulation of how humans type on touchscreens, it should be a lot easier to optimize keyboard designs for better typing -- meaning less errors, faster typing, and, most importantly for me, less frustration,' Jokinen explains.
In addition to predicting how a generic person would type, the model is also able to account for different types of users, like those with motor impairments, and could be used to develop typing aids or interfaces designed with these groups in mind. For those facing no particular challenges, it can deduce from personal writing styles -- by noting, for instance, the mistakes that repeatedly occur in texts and emails -- what kind of a keyboard, or auto-correction system, would best serve a user.
The novel approach builds on the group's earlier empirical research, which provided the basis for a cognitive model of how humans type (watch the new four-minute video END
The AI model can simulate how a human user would type any sentence on any keyboard design. It makes errors, detects them -- though not always immediately -- and corrects them, very much like humans would. The simulation also predicts how people adapt to alternating circumstances, like how their writing style changes when they start using a new auto-correction system or keyboard design.
'Previously, touchscreen typing has been understood mainly from the perspective of how our fingers move. AI-based methods have helped shed new light on these movements: what we've discovered is the importance of deciding when and where to look. Now, we can make much better predictions on how people type on their phones or tablets,' says Dr. Jussi Jokinen, who led the work.
The study, to be presented at ACM CHI on 12 May, lays the groundwork for developing, for instance, better and even personalized text entry solutions.
'Now that we have a realistic simulation of how humans type on touchscreens, it should be a lot easier to optimize keyboard designs for better typing -- meaning less errors, faster typing, and, most importantly for me, less frustration,' Jokinen explains.
In addition to predicting how a generic person would type, the model is also able to account for different types of users, like those with motor impairments, and could be used to develop typing aids or interfaces designed with these groups in mind. For those facing no particular challenges, it can deduce from personal writing styles -- by noting, for instance, the mistakes that repeatedly occur in texts and emails -- what kind of a keyboard, or auto-correction system, would best serve a user.
The novel approach builds on the group's earlier empirical research, which provided the basis for a cognitive model of how humans type (watch the new four-minute video END