SituationAdapt
Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM Reasoning.
ACM UIST 2024
Research areas
- Context-aware adaptive interfaces for Mixed Reality
- Egocentric perception from body-worn sensors
- Decoding human input and novel interactive devices
- Personalization-based modeling and optimization of human interfaces
- Digital biomarkers for Mobile Health and Affective Computing
- Machine learning from sensor signals
Context-aware adaptive interfaces for Mixed Reality
InteractionAdapt
Interaction-driven Workspace Adaptation for Situated Virtual Reality Environments.
ACM UIST 2023
Cross-Device Shortcuts
Seamless Attention-guided Content Transfer via Opportunistic Deep Links between Apps and Devices.
ACM ICMI 2023
ViGather
Inclusive Virtual Conferencing with a Joint Experience Across Traditional Screen Devices and Mixed Reality Headsets.
ACM MHCI 2023
Reality Rifts
Wonder-ful Interfaces by Disrupting Perceptual Causality.
ACM CHI 2023
InfinitePaint
Painting in Virtual Reality with Passive Haptics Using Wet Brushes and a Physical Proxy Canvas.
ACM CHI 2023
Causality-preserving Asynchronous Reality
Where everything remains real except the flow of time.
ACM CHI 2022. Best paper award.
The Chaotic Behavior of Redirection
Revisiting Simulations in Redirected Walking.
IEEE VR 2022. Best paper award.
SoundsRide
Affordance-Synchronized Music Mixing for In-Car Audio Augmented Reality.
ACM UIST 2021. Best paper award.
AirConstellations
In-Air Device Formations for Cross-Device Interaction via Multiple Spatially-Aware Armatures.
ACM UIST 2021
TransforMR
Pose-Aware Object Substitution for Composing Alternate Mixed Realities.
IEEE ISMAR 2021
SurfaceFleet
Exploring Distributed Interactions Unbounded from Device, Application, User, and Time.
ACM UIST 2020
DreamWalker
Substituting Real-World Walking Experiences with a Virtual Reality.
ACM UIST 2019
Mise-Unseen
Using Eye Tracking to Hide Virtual Reality Scene Changes in Plain Sight.
ACM UIST 2019
RealityCheck
Blending Virtual Environments with Situated Physical Reality.
ACM CHI 2019
SeeingVR
A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision.
ACM CHI 2019
VRoamer
Generating On-The-Fly VR Experiences While Walking inside Large, Unknown Real-World Building Environments.
IEEE VR 2019
Sparse Haptic Proxy
Touch Feedback in Virtual Environments Using a General Passive Prop.
ACM CHI 2017
Imaginary Reality Gaming
Ball Games Without a Ball.
ACM UIST 2013
Imaginary Devices
Gesture-Based Interaction Mimicking Traditional Input Devices.
ACM MobileHCI 2013
Egocentric perception from body-worn sensors
EgoPressure
Hand Pressure and Pose Estimation from Egocentric Vision.
CVPR 2025. Accepted as Highlight.
EgoSim
Egocentric Simulation of Multiple Body-Worn Cameras.
NeurIPS 2024
TouchInsight
Uncertainty-aware Touch and Text Input from Egocentric Vision.
ACM UIST 2024
MANIKIN
Biomechanically Accurate Neural Inverse Kinematics for Human Motion Estimation.
ECCV 2024
EgoPoser
Egocentric Pose Estimation from Sparse Observations in the Wild.
ECCV 2024
Ultra Inertial Poser
Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging.
ACM SIGGRAPH 2024
Structured Light Speckle
Joint Egocentric Depth Estimation and Contact Detection.
ACM UIST 2023
HOOV
Hand Out-Of-View Tracking for Mixed Reality using Inertial Sensing.
ACM CHI 2023
AvatarPoser
Articulated Full-Body Pose Tracking from Sparse Motion Sensing.
ECCV 2022
Gaze comes in handy
Predicting and Preventing Erroneous Hand Actions in AR-Supported Manual Tasks.
IEEE ISMAR 2021
Privacy-Preserving Ego-Motion Estimation
Using an Extremely Low-Resolution Camera.
IEEE RAL 2020
Imaginary Phone
Learning Imaginary Interfaces by Transferring Spatial Memory from a Familiar Device.
ACM UIST 2011
Decoding human input and novel interactive devices
Relative Position Tracking on Moving Nodes without Anchors
Fusing UWB Ranging and Inertial State Estimation.
IEEE IROS 2024
TouchInsight
Uncertainty-aware Touch and Text Input from Egocentric Vision.
ACM UIST 2024
Structured Light Speckle
Joint Egocentric Depth Estimation and Contact Detection.
ACM UIST 2023
Cross-Device Shortcuts
Seamless Attention-guided Content Transfer via Opportunistic Deep Links between Apps and Devices.
ACM ICMI 2023
HOOV
Hand Out-Of-View Tracking for Mixed Reality using Inertial Sensing.
ACM CHI 2023
HandyCast
Bimanual Input for VR in Space-Constrained Settings via Smartphone.
ACM CHI 2023
DeltaPen
High-Precision Translation and Rotation Sensing on Passive Surfaces.
ACM UIST 2022
TouchPose
Hand Pose and Depth Estimation from Capacitive Sensing.
ACM UIST 2021
CapContact
Super-resolution Contact Areas from Capacitive Touchscreens.
ACM CHI 2021. Best paper award.
TapID
Rapid Touch Interaction in Virtual Reality using Wearable Sensing.
IEEE VR 2021. Best demonstration award.
Flashpen
A High-Fidelity and High-Precision Multi-Surface Pen for Virtual Reality.
IEEE VR 2021
Haptic PIVOT
On-Demand Handhelds in Virtual Reality.
ACM UIST 2020
Omni
Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback.
ACM UIST 2020
Tilt-Responsive Techniques for Digital Drawing Boards
ACM UIST 2020
Virtual Reality Without Vision
A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds.
ACM CHI 2020. Honorable mention award.
GazeConduits
Calibration-Free Cross-Device Collaboration through Gaze and Touch.
ACM CHI 2020
CapstanCrunch
A Haptic VR Controller with User-supplied Force Feedback.
ACM UIST 2019. Best demonstration honorable mention award.
Cross-Device Taxonomy
Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices.
ACM CHI 2019
Sensing Posture-Aware Pen+Touch Interaction on Tablets
ACM CHI 2019. Honorable mention award.
TORC
A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction.
ACM CHI 2019
Project Zanzibar
A Portable and Flexible Tangible Interaction Platform.
ACM CHI 2018. Best paper award.
Haptic Revolver
Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller.
ACM CHI 2018. Honorable mention award.
CLAW
A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality.
ACM CHI 2018
Enabling People with Visual Impairments to Navigate Virtual Reality
Using a Haptic and Auditory Cane Simulation.
ACM CHI 2018
Haptic Links
Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation.
ACM CHI 2018
SurfaceConstellations
A Modular Hardware Platform for Ad-Hoc Reconfigurable Cross-Device Workspaces.
ACM CHI 2018
PolarTrack
Optical Outside-In Device Tracking that Exploits Display Polarization.
ACM CHI 2018
Finding Common Ground
A Survey of Capacitive Sensing in Human-Computer Interaction.
ACM CHI 2017
Sparse Haptic Proxy
Touch Feedback in Virtual Environments Using a General Passive Prop.
ACM CHI 2017
NormalTouch and TextureTouch
High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers.
ACM UIST 2016. Honorable mention award.
DuoSkin
Rapidly Prototyping On-Skin User Interfaces Using Skin-Friendly Materials.
ACM ISWC 2016
On-Demand Biometrics
Fast and Convenient Cross-Device Login.
ACM CHI 2016
Pre-Touch Sensing for Mobile Interaction
ACM CHI 2016
Tracko
Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction.
ACM UIST 2015
Biometric Touch Sensing
Seamlessly Augmenting Each Touch with Continuous Authentication.
ACM UIST 2015
Bodyprint
Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts.
ACM CHI 2015
Fiberio
A Touchscreen that Senses Fingerprints.
ACM UIST 2013. Best paper award.
Imaginary Reality Gaming
Ball Games Without a Ball.
ACM UIST 2013
GravitySpace
Body Pose Tracking on a Pressure-Sensing Floor.
ACM CHI 2013. Honorable mention award.
Bootstrapper
Recognizing Tabletop Users by their Shoes.
ACM CHI 2012
Implanted User Interfaces
ACM CHI 2012
Imaginary Phone
Learning Imaginary Interfaces by Transferring Spatial Memory from a Familiar Device.
ACM UIST 2011
Data Miming
Inferring Spatial Object Descriptions from Human Gesture.
ACM CHI 2011
Multitoe
High-Precision Interaction with Back-Projected Floors Based on High-Resolution Multi-Touch Input.
ACM UIST 2010
The Generalized Perceived Input Point Model
How to Double Touch Accuracy by Extracting Fingerprints.
ACM CHI 2010
Relaxed Selection Techniques
for Querying Time-Series Graphs.
ACM UIST 2009
Personalization-based modeling and optimization of human interfaces
Cybersickness Detection Beyond Subjectivity
Continuous EEG-based Estimation using Multitaper Spectrums.
IEEE TVCG 2025
Continual Human-in-the-Loop Optimization
ACM CHI 2025. Honorable mention award.
Redefining Affordance
via Computational Rationality.
ACM IUI 2025
MARLUI
Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIs.
ACM EICS 2024
Synchronous vs. Asynchronous Reality
A controlled study on task delivery.
IEEE TVCG 2024
Controllers or Bare Hands?
A controlled study of input techniques on performance and exertion in VR.
IEEE TVCG 2023
ComforTable User Interfaces
Passive surfaces benefit input performance and lower exertion and strain in VR.
IEEE ISMAR 2022
Demographic & Behavioral Correlates of Cybersickness
A Large Lab-in-the-Field Study of 837 Participants.
IEEE ISMAR 2022
TapType
Ten-finger text entry on everyday surfaces via Bayesian inference.
ACM CHI 2022
Understanding Multi-Device Usage Patterns
Physical Device Configurations and Fragmented Workflows.
ACM CHI 2022
Understanding Touch
ACM CHI 2011
The Generalized Perceived Input Point Model
How to Double Touch Accuracy by Extracting Fingerprints.
ACM CHI 2010
Digital biomarkers for Mobile Health and Affective Computing
Cybersickness Detection Beyond Subjectivity
Continuous EEG-based Estimation using Multitaper Spectrums.
IEEE TVCG 2025
Nightbeat
Heart Rate Estimation From a Wrist-Worn Accelerometer During Sleep.
IEEE JBHI 2025
WildPPG
A Real-World PPG Dataset of Long Continuous Recordings.
NeurIPS 2024
Respiro
Continuous Respiratory Rate Monitoring During Motion via Wearable Ultra-Wideband Radar.
IEEE-EMBS BHI 2024
SympCam
Remote Optical Measurement of Sympathetic Arousal.
IEEE-EMBS BHI 2024
Emotion Recognition for Chatbots
Multimodal analyses for Human-Chatbot Interaction in the Wild.
ACM ICMI 2024
Tri-Spectral PPG
Robust cardiac monitoring by fusing multiple sensing wavelengths.
IEEE BSN 2024
Multi-Site Photoplethysmography
Robust heart rate detection via multi-site signal fusion.
IEEE EMBC 2024
Unsupervised Sleep Quality Estimation
Learning Cardiac Activity for Predicting Sleep Quality.
IEEE EMBC 2024
Predicting Perceived Sleep Quality
Interpretable Models based on Cardiovascular and Behavioral Features.
PLOS ONE 2024
Chatbots With Attitude
Enhancing Chatbot Interactions Through Dynamic Personality Infusion.
ACM CUI 2024
Detecting Users' Emotional States
Affective State Awareness during Passive Social Media Use.
ACM IMWUT 2024
GPT-3's Personality Dimensions
Personality expressed during human-chatbot interactions.
ACM IMWUT 2024
Modeling Multiple Sclerosis
Wearable and mobile monitoring to assess disease state.
npj Digital Medicine 2024
Biomarkers for daily fatigue
Continuous wearable monitoring for fatigue modeling.
Cell Press iScience 2024
Video-based Sympathetic Arousal
Non-contact estimation of electrodermal activity.
Biomedical Optics Express 2023
BeliefPPG
Uncertainty-aware Heart Rate Estimation from PPG signals via Belief Propagation.
UAI 2023
Personality Trait Recognition in the Wild
Based on Characteristics of Typing on Smartphones.
IEEE TAffC 2023
SSVEP Visual Stimuations vs. User-Friendliness in Virtual Reality
IEEE BCI 2023
Physiological Responses to Fear, Frustration, and Insight
A study in Virtual Reality.
IEEE TVCG 2022
Cognitive Fatigability Assessment Test
A novel instrument to assess perceived fatigue.
DIGITAL HEALTH 2022
Autonomic dysfunction in Multiple Sclerosis
Continuous wearable monitoring to assess MS severity.
Multiple Sclerosis Journal 2022–Experimental, Translational and Clinical 2022
Affective State Prediction
From Smartphone Touch and Sensor Data in the Wild.
ACM CHI 2022
Virtual reality reduces COVID‑19 vaccine hesitancy
A randomized trial in the wild on 232 participants.
Nature Scientific Reports 2022
A self-administered virtual reality intervention increases COVID-19 vaccination intention
Vaccine 2021
Predicting Fatigue from Fatigability
A phone-based in-the-wild Study with Multiple Sclerosis Patients.
ACM IMWUT 2021
The Effect of Vibratory Stimuli
Non-invasive intervention for motor signs of Parkinson's disease.
Journal of Neurology, Neurosurgery & Psychiatry 2020
Assessing Fatigability in Multiple Sclerosis Patients
A Rapid Tapping Task for Phone-based Mobile Health.
ACM CHI 2020
Naptics
Continuous Blood Pressure Monitoring during Sleep.
ACM IMWUT 2018
Smartphone Pulse Oximetry
Phone displays for selective illumination and cameras as sensors.
IEEE EMBC 2018
Glabella
Smart Glasses for Continuously Monitoring Blood Pressure.
ACM IMWUT 2017. Distinguished paper award.
DualBlink
Detecting, Tracking, and Actuating Blinking For Alleviating Dry Eyes and Computer Vision Syndrome.
ACM IMWUT 2017
Machine learning from sensor signals
Shifting the Paradigm
Achieving Shift-Invariancy for Deep Learning on Time Series.
ICLR 2025
WildPPG
A Real-World PPG Dataset of Long Continuous Recordings.
NeurIPS 2024
Quasi-periodic Data Augmentation
From Chaos to Order in Contrastive Learning for Time Series.
NeurIPS 2023
BeliefPPG
Uncertainty-aware Heart Rate Estimation from PPG signals via Belief Propagation.
UAI 2023