High-Precision Translation and Rotation Sensing on Passive Surfaces

Guy Luethi*, Andreas Fender*, and Christian Holz
*equal contribution
DeltaPen teaser image

(a) DeltaPen is a digital pen that resolves precise translation and rotation (yaw, tilt) on uninstrumented surfaces. (b) Our device requires no surrounding cameras or specialized sensing surfaces and instead has all sensing hardware integrated. Two optical flow sensors and a pressure sensor enable high-precision pen input, which is complemented with an IMU as well as components for supporting vibrotactile feedback. DeltaPen processes optical flow from our dual-sensor design to derive precise translation and yaw input, monitors pressure input to detect precise contact with a surface, and senses gravity to obtain tilt. (c) Besides precise translation, DeltaPen also senses rotational movements for input. In this photo sorting application example, the user can drag pictures and rotate them at the same time.


We present DeltaPen, a pen device that operates on passive surfaces without the need for external tracking systems or active sensing surfaces. DeltaPen integrates two adjacent lens-less optical flow sensors at its tip, from which it reconstructs accurate directional motion as well as yaw rotation. DeltaPen also supports tilt interaction using a built-in inertial sensor. A pressure sensor and high-fidelity haptic actuator complements our pen device while retaining a compact form factor that supports mobile use on uninstrumented surfaces. We present a processing pipeline that reliably extracts fine-grained pen translations and rotations from the two optical flow sensors. To asses the accuracy of our translation and angle estimation pipeline, we conducted a technical evaluation in which we compared our approach with ground-truth measurements of participants’ pen movements during typical pen interactions. We conclude with several example applications that leverage our device’s capabilities. Taken together, we demonstrate novel input dimensions with DeltaPen that have so far only existed in systems that require active sensing surfaces or external tracking.



Guy Luethi*, Andreas Fender*, and Christian Holz. DeltaPen: A Device with Integrated High-Precision Translation and Rotation Sensing on Passive Surfaces. In Proceedings of ACM UIST 2022.


We are thankful to PixArt Imaging Inc. for their support with the optical sensors that we integrated into DeltaPen’s dual-sensor design.