Cross-Device Shortcuts

Seamless Attention-guided Content Transfer via Opportunistic Deep Links between Apps and Devices

ACM ICMI 2023
Marilou Beyeler*, Yi Fei Cheng*, and Christian Holz
*equal contribution
Cross-Device Shortcuts teaser image

Cross-device shortcuts directly link content from an app running on one device into an app on another—content-aware, manifesting following the user’s attention switches, and presented via feed forward. (A) This user selects a picture on a phone and then attends to her laptop, creating a cross-device shortcut that is revealed through an in-app preview and afford drag&drop. (B) Cross-device shortcuts work across phones, laptops, and tablets. (C) Feed-forward previews appear at the UI-element level, in-app level, or operating-system level depending on content type and the receiving app’s capability for accepting it.

Abstract

Although users increasingly spread their activities across multiple devices—even to accomplish a single task—information transfer between apps on separate devices still incurs non-negligible effort and time overhead. These interaction flows would considerably benefit from more seamless cross-device interaction that directly connects the information flow between the involved apps across devices. In this paper, we propose cross-device shortcuts, an interaction technique that enables direct and discoverable content exchange between apps on different devices. When users switch their attention between multiple engaged devices as part of a workflow, our system establishes a cross-device shortcut—a deep link between apps on separate devices that presents itself through feed-forward previews, inviting and facilitating quick content transfer. We explore the use of this technique in four scenarios spanning multiple devices and applications, and highlight the potential, limitations, and challenges of its design with a preliminary evaluation.

Video

Reference

Marilou Beyeler*, Yi Fei Cheng*, and Christian Holz. Cross-Device Shortcuts: Seamless Attention-guided Content Transfer via Opportunistic Deep Links between Apps and Devices. In International Conference on Multimodal Interaction 2023 (ACM ICMI).