Gaze-in-Wild
Most algorithms that label eye movements as fixations, pursuits or saccades were built for lab recordings in which the head is held still. In everyday activity the head moves too, and its rotation changes the velocity signals those algorithms rely on, yet there was no labeled dataset of natural, head-free gaze to train or test them on.
This paper introduces that dataset, recorded with a hardhat that carries a binocular eye tracker, a motion sensor for head rotation and a stereo depth camera, all calibrated together in space and time. Trained labelers marked fixations, pursuits, saccades and blinks, and the paper trains a Random Forest and recurrent networks on eye and head velocities as baselines, scored with both sample-level and event-level metrics.
Main results
- 19 participants performed everyday tasks (walking, catching a ball, visual search and making tea), and about 2 hours and 15 minutes of recordings were hand-labeled into gaze events, with agreement between labelers of 0.74 (sample-based Cohen's kappa).
- On held-out participants, a Random Forest reached a kappa of 0.63 and a bidirectional recurrent network 0.61.
- The classifiers reached about 87% of human performance on fixations and saccades, but pursuit detection (kappa 0.37) was 49% below human agreement (0.73).