Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone System
Abstract
Optical see-through head-mounted displays (OST HMDs) are a popular output medium for mobile Augmented Reality (AR) applications. To date, they lack efficient text entry techniques. Smartphones are a major text entry medium in mobile contexts but attentional demands can contribute to accidents while typing on the go. Mobile multi-display ecologies, such as combined OST HMD-smartphone systems, promise performance and situation awareness benefits over single-device use. We study the joint performance of text entry on mobile phones with text output on optical see-through head-mounted displays. A series of five experiments with a total of 86 participants indicate that, as of today, the challenges in such a joint interactive system outweigh the potential benefits.
Media
First-person views of the conditions in Experiment 2 on the effects of combined typing on a joint optical see-through head-mounted display (OST HMD) smartphone system. Left: typing on a standard keyboard (condition BASELINE). Center: typing on a full-screen keyboard displaying the text at a fixed position in the user’s field of view (condition HUD). Right: typing on a full-screen keyboard displaying the text spatially registered above the smartphone in an Augmented Reality view (condition AR). The visual artifacts (blur, aberrations, white stripes) are due to capturing the OST HMD screen with a camera.
Publication
Jens Grubert, Lukas Witzani, Alexander Otte, Travis Gesslein, Matthias Kranz & Per Ola Kristensson.
Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone System. In IEEE Transactions on Visualization and Computer Graphics, vol. 30, no. 8, pp. 5830-5846, Aug. 2024.
DOI |
arXiv preprint