EgoHRV

Continuous Heart Rate Variability Estimation from Egocentric Systems

ECCV 2026
Berken Utku Demirel and Christian Holz
Department of Computer Science, ETH Zürich, Switzerland
EgoHRV

Abstract

Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used noninvasive marker of autonomic regulation under stress. HRV reflects small timing differences between successive heartbeats and has so far been out of reach for egocentric platforms, where motion and noise in gaze video mask exactly this fine-grained timing. We propose EgoHRV, a method that estimates HRV as well as heart rate (HR) from the gaze cameras that are already integrated into egocentric headsets. Our pipeline combines a 3D backbone with a novel low–high decomposition module that extracts the blood volume pulse (BVP) signal from gaze video. Our cross-domain pretraining aligns the frequency-domain representations of contact-based and camera-derived signals. This alignment gives EgoHRV the temporal precision to recover HRV from the subtle fluctuations in gaze video. EgoHRV achieves state-of-the-art accuracy for HR and HRV estimation from egocentric video, and its uncertainty-aware design improves downstream behavioral modeling. Integrating our HRV estimates and confidence measures into EgoExo4D’s proficiency estimator raises accuracy by 17.8%. Beyond skill, continuous HRV estimation also opens egocentric systems to stress- and arousal-aware estimation tasks. Code

Reference

Berken Utku Demirel and Christian Holz. EgoHRV: Continuous Heart Rate Variability Estimation from Egocentric Systems for Autonomic Response and Skill Assessment. In European Conference on Computer Vision 2026 (ECCV).

Physiology from egocentric eye tracking cameras

Egocentric devices already use inward-facing cameras for eye tracking.
EgoHRV uses the same cameras to estimate heart rate (HR) and heart rate variability (HRV), without adding a contact sensor at test time.

HRV depends on the timing between consecutive heartbeats. This makes it harder to estimate than HR: small timing errors caused by head motion, illumination changes, or noisy video can strongly affect the result.

EgoHRV estimates these beat intervals from gaze video at roughly 4-second resolution and also predicts when its estimates are uncertain.

EgoHRV

Figure 1: EgoHRV estimates changes in cardiac activity throughout an egocentric recording. In the bouldering example, HR and beat-to-beat variability change across planning, climbing, and recovery.

From gaze video to beat intervals

The pipeline has three parts.

  1. Gaze video → BVP.
    A 3D convolutional network processes temporally differenced eye-camera frames. A low–high decomposition block reduces motion-related noise before reconstructing the blood-volume-pulse (BVP) signal.

  2. BVP → inter-beat interval.
    We pretrain an IBI estimator using contact BVP and ECG datasets. The model operates on Fourier magnitudes, which makes the representation less dependent on phase differences between contact sensors and cameras. IMU measurements are used to estimate prediction uncertainty.

  3. Physiology → downstream tasks.
    PhysFusion combines the predicted cardiac features and their uncertainty with video features. We use this for proficiency estimation on EgoExo4D.

EgoHRV pipeline

Figure 2: EgoHRV first reconstructs a BVP signal from gaze video, predicts inter-beat intervals and uncertainty using a cross-domain model, and uses the resulting physiological features in PhysFusion for downstream prediction.

Why estimate every few seconds?

Previous egocentric HR estimation commonly averages predictions over 60-second windows.

That removes a large part of the variation that happens within the minute. On egoPPG-DB, 60-second averaging retains about 59.8% of the total HR variance, meaning roughly 40% is removed by averaging.

For activities with changing effort or motion, that difference matters.

What this adds to egocentric sensing

EgoHRV turns an existing headset sensor into a source of cardiac information.

The method provides a continuous sequence of beat-interval estimates that can be aligned with what the wearer is doing, together with an estimate of when those predictions should be trusted.

We use proficiency estimation as one example. The same representation can be used in future work on workload, fatigue, arousal, and other tasks where physiological state complements visual observations.

HRV is an indicator of autonomic regulation and can be associated with stress, but it is not itself a direct stress label.