La Gioconda and the indeterminacy of smile recognition by a person and by an artificial neural network

Zhukova, O.V., Malakhova, K.Y., Shelepin, Y.E.

Full text «Opticheskii Zhurnal»

Full text on elibrary.ru

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Жукова О.В., Малахова Е.Ю., Шелепин Ю.Е. Джоконда и неопределенность распознавания улыбки человеком и искусственной нейронной сетью // Оптический журнал. 2019. Т. 86. № 11. С. 40–50. http://doi.org/10.17586/1023-5086-2019-86-11-40-50

Zhukova O.V., Malakhova E.Yu., Shelepin Yu.E. La Gioconda and the indeterminacy of smile recognition by a person and by an artificial neural network [in Russian] // Opticheskii Zhurnal. 2019. V. 86. № 11. P. 40–50. http://doi.org/10.17586/1023-5086-2019-86-11-40-50

For citation (Journal of Optical Technology):

O. V. Zhukova, E. Yu. Malakhova, and Yu. E. Shelepin, "La Gioconda and the indeterminacy of smile recognition by a person and by an artificial neural network," Journal of Optical Technology. 86(11), 706-715 (2019). https://doi.org/10.1364/JOT.86.000706

Abstract:

This paper presents a comparative analysis of the possibilities of smile recognition by a person and by an artificial neural network under conditions of indeterminacy. The main brain-activity patterns are studied by functional magnetic-resonance tomography. There are fundamental limitations inherent to natural and artificial neural networks, and therefore generalizations of the results of recognizing test images are obtained under below-threshold and above-threshold conditions. The probability of recognizing a smile is thus fairly high under ordinary conditions, but it decreases under indeterminacy conditions (threshold and noisy images) both in humans and in artificial neural networks. For instance, the recognition of a smile in La Gioconda’s facial expression by a person and by an artificial neural network occurs with probability 0.69. We assume that the most important operating principle in both networks is a matched-filtering mechanism as a measure of how well the presented image corresponds to a pattern learned by the neural network—in particular, a smile.

Keywords:

smile, artificial neural network, recognition, large neural network, brain-activity pattern

Acknowledgements:

The neurophysiological part of this study was carried out with the financial support of the scientific research project Psychophysiological and Neurolinguistic Aspects of the Process of Recognition of Verbal and Nonverbal Patterns, a project of the Russian Science Foundation (No. 14-18-02135).
The modeling on a convolutional deep-learning neural network was carried out with the financial support of the Program of Fundamental Scientific Research of State Academies in 2013–2020 (GP-14, Section 63), I. P. Pavlov Institute of Physiology.

OCIS codes: 100.4996, 170.6960, 330.5020

References:

1. F. Rozenblatt, Principles of Neurodynamics (The Perceptron and the Theory of Brain Mechanisms) (Mir, Moscow, 1965).
2. D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex,” J. Physiol. 160, 106–154 (1962).

3. K. Fukushima and S. Miyake, “Neocognitron. A self-organizing neural network model for a mechanism of visual pattern recognition,” in Competition and Cooperation in Neural Nets (Springer Heidelberg, Berlin, 1982), pp. 267–285.
4. U. Guclu and M. A. van Gerven, “Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream,” J. Neurosci. 35, 10005 (2015).
5. N. Kriegeskorte, “Deep neural networks: a new framework for modeling biological vision and brain information processing,” Annu. Rev. Vision Sci. 1, 417–446 (2015).
6. O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in Proceedings of the British Machine Vision Conference (BMVC) (2015).
7. E. G. Malakhova, “The processing of visual information in artificial and biological neural networks,” in Neurotechnology, Yu. E. Shelepin and V. N. Chikhman, eds. (Izd. VVM, St. Petersburg, 2018), pp. 338–349.
8. N. N. Krasil’nikov and Yu. E. Shelepin, “Masking as the result of matched filtering,” Fiziol. Chel. 22(5), 99–103 (1996).
9. Yu. E. Shelepin, O. V. Borachuk (Zhukova), S. V. Pronin, A. K. Kharauzov, P. P. Vasiliev, and V. A. Fokin, “The face and nonverbal methods of communication,” Peterburg Psikhol. Zh. 9, 1–43 (2014).
10. O. V. Zhukova, “Reconfiguration regularities of a large neural network of the human brain in the recognition of faces under indeterminacy conditions,” Author’s Abstract of Candidate’s Dissertation (SPbGU, St. Petersburg, 2017).
11. N. F. Podvigin, F. N. Makarov, and Yu. E. Shelepin, Elements of the Structural–Functional Organization of the Visual System (Nauka, Leningrad, 1986).
12. K. Tanaka, H. Saito, Y. Fukada, and M. Moriya, “Coding visual images of objects in the inferotemporal cortex of the macaque monkey,” J. Neurophysiol. 66(1), 170–189 (1991).
13. Yu. E. Shelepin, A. K. Kharauzov, S. V. Pronin, O. A. Vakhrameeva, V. N. Chikhman, V. A. Fokin, and N. Foreman, “Using neuroimaging methods to localize mechanisms for making decisions concerning the ordering of textures,” J. Opt. Technol. 78(12), 808–816 (2011) [Opt. Zh. 78(12), 57–69 (2011)].
14. C. F. Cadieu, H. Hong, D. L. Yamins, N. Pinto, D. Ardila, E. A. Solomon, N. J. Majaj, and J. J. DiCarlo, “Deep neural networks rival the representation of primate IT cortex for core visual object recognition,” PLoS Comput. Biol. 10, e1003963 (2014).
15. T. Kohonen, “Analysis of simple self-organizing process,” Biol. Cybern. 44, 135–140 (1982).
16. K. Fukushima, “Neural network model for selective attention in visual pattern recognition and associative recall,” Appl. Opt. 26(23), 4985–4992 (1987).