RIT-Eyes
Video-based eye trackers find the pupil, iris and sclera in infrared images of the eye, and neural networks do this well, but they need many hand-labeled images to train. Rendered images come with labels for free, yet networks trained on earlier renderings lose accuracy on real eyes, which suggests the renderings miss some of the variation in real ones.
RIT-Eyes models the eye in more detail than earlier generators: an aspherical cornea, a pupil whose size reshapes the iris texture, the inner corner of the eye, the bright-pupil effect, eyelids that follow gaze, blinks, and reflections from the environment and from eyeglasses. The paper renders datasets that copy the camera setups of the public NVGaze and OpenEDS datasets, plus one that spans the positions possible on a head-mounted eye tracker, and tests which differences between datasets limit how well segmentation networks generalize.
Main results
- Segmentation networks trained on the renderings and tested on held-out head models reached a mean IoU above 95 (RITnet 96.31 and SegNet 95.96 on the NVGaze-style renderings).
- Trained on a narrow range of camera positions, the networks still reached 88.71 (RITnet) and 86.11 (SegNet) on renderings that cover a much wider range.
- Tested on real OpenEDS images, the mean IoU fell to 73.94 for RITnet and 80.31 for SegNet, mainly because the sclera was segmented poorly.