A wide range of real-world applications, including computational photography (e.g., portrait mode and glint reflections) and augmented reality effects (e.g., virtual avatars) rely on estimating eye position by tracking the iris. Once accurate iris tracking is available, we show that it is possible to determine the metric distance from the camera to the user without the use of a dedicated depth sensor.

This, in turn, can improve a variety of use cases, ranging from computational photography, over virtual try-on of properly sized glasses, and hats to usability enhancements that adopt the font size depending on the viewer’s distance.

Example of virtual avatar using iris tracking.
Font size changing based on eye distance.

Iris tracking is a challenging task to solve on mobile devices, due to limited computing resources, variable light conditions, and the presence of occlusions, such as hair or people squinting. Often, sophisticated specialized hardware is employed, limiting the range of devices on which the solution could be applied.

MediaPipe determining distance from user to the camera.

Today, we announce the release of MediaPipe Iris, a new machine learning model for accurate iris estimation. Building on our work on MediaPipe Face Mesh, this model is able to track landmarks involving the iris, pupil, and the eye contours using a single RGB camera, in real-time, without the need for specialized hardware.

Through use of iris landmarks, the model is also able to determine the metric distance between the subject and the camera with relative error less than 10% without the use of a depth sensor. Note that iris tracking does not infer the location at which people are looking, nor does it provide any form of identity recognition. Thanks to the fact that this system is implemented in MediaPipe – an open-source cross-platform framework for researchers and developers to build world-class ML solutions and applications – it can run on most modern mobile phones, desktops, laptops and even on the web.

See more HERE.

Source: Google AI Blog

Related posts: