Categorization of objects and scenes by a neural network whose input modules are pre-trained to decode spatial texture inhomogeneities

Явна Д.В., Бабенко В.В., Горбенкова О.А., Плавельский И.В., Вороная В.Д., Столетний А.С. Категоризация объектов и сцен нейронной сетью, входы которой предварительно обучены декодированию пространственных неоднородностей текстуры // Оптический журнал. 2023. Т. 90. № 1. С. 37–48. http://doi.org/10.17586/1023-5086-2023-90-01-37-48

Yavna D.V., Babenko V.V., Gorbenkova O.A., Plavelsky I.V., Voronaya V.D., Stoletniy A.S. Categorization of objects and scenes by a neural network whose input modules are pretrained to decode spatial texture inhomogeneities [in Russian] // Opticheskii Zhurnal. 2023. V. 90. № 1. P. 37–48. http://doi.org/10.17586/1023-5086-2023-90-01-37-48

D. V. Yavna, V. V. Babenko, O. A. Gorbenkova, I. V. Plavelsky, V. D. Voronaya, and A. S. Stoletniy, "Classification of objects and scenes by a neural network with pretrained input modules to decode spatial texture inhomogeneities," Journal of Optical Technology. 90(1), 20-25 (2023). https://doi.org/10.1364/JOT.90.000020

Scope of research. Investigation of the possibility of using neural network models of second-order visual mechanisms as input data for neural network classifiers. Second-order visual mechanisms make it possible to detect spatial inhomogeneities in contrast, orientation, and spatial frequency in an image. These mechanisms are traditionally considered by visual researchers as one of the stages of early visual processing; their role in the perception of textures has been well studied. The purpose of the work is to study whether the use of classifier input modules previously trained to demodulate spatial modulations of brightness gradients will contribute to the categorization of objects and scenes. Method. Neural network modeling was used as the main method. At the first stage of the study, a set of texture images was generated, which is used to train neural network models of second-order visual mechanisms, and these models were trained. At the second stage, samples of objects and scenes were prepared, on which classifier networks were trained. Previously trained models of second-order visual mechanisms with frozen weights were placed at the input of these networks. Main results. The second order information, presented as a map of instantaneous values of the modulation function of contrast, orientation and spatial frequency in the image, may be sufficient to identify only some classes of scenes. In general, within the framework of the proposed neural network architectures, the use of modulation function values for solving the problem of object classification turned out to be ineffective. Thus, the hypothesis that second-order visual filters encode features that allow identifying an object was not confirmed. This result makes it necessary to test an alternative hypothesis that the role of second-order filters is limited to participation in the construction of saliency maps, and the filters themselves are windows through which information comes from the outputs of first-order filters. Practical significance. The possibility of using second-order models of visual mechanisms in computer vision systems was assessed.

Acknowledgment: the study was financially supported by the Russian Foundation for Basic Research, project № 18-29-22001 MK "An investigation of neurocognitive technologies of attentional control and formation of mental representations of visual web content".