Temporal data processing from webcam eye tracking using artificial neural networks

Malakhova, K.Y., Shelepin, E.Y., Malashin, R.O.

Full text «Opticheskii Zhurnal»

Full text on elibrary.ru

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Малахова Е.Ю., Шелепин Е.Ю., Малашин Р.О. Применение искусственных нейронных сетей, учитывающих временную динамику, для анализа движения глаз без специального оборудования // Оптический журнал. 2018. Т. 85. № 3. С. 77–80. http://doi.org/10.17586/1023-5086-2018-85-03-77-80

Malakhova E.Yu., Shelepin E.Yu., Malashin R.O. Temporal data processing from webcam eye tracking using artificial neural networks [in Russian] // Opticheskii Zhurnal. 2018. V. 85. № 3. P. 77–80. http://doi.org/10.17586/1023-5086-2018-85-03-77-80

For citation (Journal of Optical Technology):

E. Yu. Malakhova, E. Yu. Shelepin, and R. O. Malashin, "Temporal data processing from webcam eye tracking using artificial neural networks," Journal of Optical Technology. 85(3), 186-188 (2018). https://doi.org/10.1364/JOT.85.000186

Abstract:

The technology for determining eye gaze direction on a monitor screen is considered by analyzing the images received from a video camera directed at a user without using additional equipment. We propose a combined convolutional neural network extracting high-level features of images with a neural network of long short-term memory that takes into account the temporal dynamics of oculomotor activity. To train the model, a representative database of video sequences with reference information concerning eye direction was collected. Experiments confirmed that accounting for temporal information increased the accuracy of recording the eye direction.

Keywords:

еye tracking, artificial neural networks, video sequences analysis

OCIS codes: 150.0150

References:

1. W. Sewell and O. Komogortsev, “Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network,” in Proceedings of CHI, 2010, pp. 3739–3744.
2. E. Demjén, V. Abosi, and Z. Tomori, “Eye tracking using artificial neural networks for human computer interaction,” Physiol. Res. 60(5), 841 (2011).
3. A. Krizhevsky, I. Sutskever, and E. G. Hinton, “ImageNet classification with deep convolutional neural networks,” Adv. Neural Inf. Process. Syst. 2, 1097–1105 (2012).
4. A. George and A. Routray, “Real-time eye gaze direction classification using convolutional neural network,” in International Conference on Signal Processing and Communications (SPCOM), 2016.
5. K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Matusik, and A. Torralba, “Eye tracking for everyone,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2176–2184.
6. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput. 9(8), 1735–1780 (1997).