WildPPG
μDPad
A Large-Scale Multimodal PPG and IMU Dataset for Wrist-Worn Microgesture Recognition
ACM ICMI 2026Department of Computer Science, ETH Zürich
Abstract
Gestures provide a natural interaction modality for mobile and wearable devices. However, most prior work focuses on macro gestures involving large wrist or arm movements, which are fatiguing and socially conspicuous. In contrast, microgestures, subtle, low-effort finger movements that can be performed one-handed and without visual attention, offer a promising alternative but remain significantly harder to sense and recognize. In this work, we present a large-scale multimodal dataset for wrist-based microgesture recognition, collected with an integrated wearable prototype that combines an IMU with an 8-channel spatial PPG sensor configuration similar to one emerging in commercial fitness watches. Because such devices typically do not expose raw optical signals, our goal is to provide the community with a resource for studying whether spatial PPG can capture the minute tissue deformations associated with microgestures. Our dataset comprises 25 hours of synchronized IMU and PPG recordings from 68 participants, including 9 hours of aligned video annotations, and captures substantial variability in users, hand poses, and execution styles. To establish reference performance, we benchmark a range of recognition pipelines and present a simple multimodal baseline. This baseline already shows that adding spatial PPG improves microgesture detection over IMU-only sensing, reaching 83.7% accuracy with a 3.7% false-positive rate, while also underscoring that microgesture recognition remains substantially harder than recognizing coarser macro-gesture vocabularies. By releasing the dataset, we aim to enable more substantive algorithmic advances toward practical, low-effort, and socially acceptable microgesture interaction on future wrist-worn devices.
Reference
Lars Hauptmann, Dominik Hollidt, Xintong Liu, Manuel Meier, and Christian Holz. μDPad: A Large-Scale Multimodal PPG and IMU Dataset for Wrist-Worn Microgesture Recognition. In International Conference on Multimodal Interaction 2026 (ACM ICMI).
Sensing Hardware
μDPad hardware prototype integrating a 6D IMU and an 8-channel spatial PPG sensor. The circular photodiode layout captures local deformation patterns under the device that help disambiguate directional thumb swipes.
Spatial PPG Sensing
Spatial PPG sensing used by μDPad. Interpolating eight photodiode readings over a one-second gesture window reveals non-uniform reflectivity changes (a deformation “pattern”) that complements wrist IMU features for classifying directional thumb microgestures. Green and IR PPG channels are encoded into the green and red RGB values.
Gesture Set
Overlaid signals from our gesture set. The opaque line is the barycenter of all corresponding signals from three different participants. The transparent lines are recordings with a high similarity to the barycenter.
Recognition Pipeline
μDPad pipeline. A window-based multimodal classifier maps IMU+PPG windows to class probabilities, which are decoded over time with an HMM to enforce plausible transitions and improve robustness for online microinteraction input.
Results
Comparison of Machine Learning Classifiers
| Model | Accuracy | F1 Score |
|---|---|---|
| ResNet | 82.1% ± 2.9 | 74.3% ± 2.0 |
| U-Net | 80.3% ± 3.4 | 71.6% ± 2.6 |
| Transformer | 62.0% ± 5.8 | 48.0% ± 2.0 |
| Random Forest | 61.9% ± 8.4 | 48.9% ± 2.5 |
| SVM | 64.7% ± 7.9 | 53.5% ± 2.9 |
| μDPad (early fusion) | 83.4% ± 2.9 | 76.6% ± 2.3 |
| μDPad (ours) | 83.7% ± 2.6 | 77.0% ± 2.3 |
| μDPad (fine-tuned) | 88.0% ± 5.5 | 82.6% ± 7.3 |
Model architecture comparison (mean ± std.). The VGG-style window-based model used by μDPad performs best on this dataset, outperforming alternative deep architectures and feature-based baselines.
Ablation of Sensing Modalities
| Modalities | Accuracy | F1 Score |
|---|---|---|
| Acc | 80.8% ± 2.5 | 72.8% ± 1.9 |
| Acc + PPG | 80.4% ± 2.7 | 72.3% ± 2.9 |
| Gyro | 73.1% ± 5.5 | 63.8% ± 3.6 |
| Gyro + PPG | 75.0% ± 3.9 | 64.3% ± 3.4 |
| Acc + Gyro | 82.9% ± 3.0 | 76.1% ± 2.4 |
| PPG | 47.1% ± 5.3 | 35.3% ± 2.0 |
| Acc + Gyro + PPG | 83.7% ± 2.6 | 77.0% ± 2.3 |
Sensor-modality ablation (mean ± std.) with PPG green+IR channels. IMU features dominate overall accuracy, while the complementary information from spatial PPG improves performance for some IMU-only failure cases.
Per-Participant Gains
Relative accuracy gains in leave-one-subject-out evaluation. Participants are grouped by IMU-only accuracy percentiles (6 participants per bin). Spatial PPG yields the largest relative benefit for low-accuracy users, while fine-tuning improves performance across all bins.
Dataset Scaling
Accuracy of μDPad with increasing number of participants in the training split.
Applications
Example microinteraction scenarios enabled by μDPad. Left: always-available, low-effort input in everyday contexts without touching a screen. Right: prototype mappings for music control and lightweight game input using directional swipes, taps, and pinch-hold combined with wrist rotation.
