Training an improved recurrent attention model using an alternative reward function

Malashin, R.O.

Full text «Opticheskii Zhurnal»

Full text on elibrary.ru

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Малашин Р.О. Обучение улучшенной модели рекуррентного внимания с использованием альтернативной функции вознаграждения // Оптический журнал. 2021. Т. 88. № 3. С.18–23. http://doi.org/10.17586/1023-5086-2021-88-03-18-23

Malashin R.O. Training an improved recurrent attention model using an alternative reward function [in Russian] // Opticheskii Zhurnal. 2021. V. 88. № 3. P.18–23. http://doi.org/10.17586/1023-5086-2021-88-03-18-23

For citation (Journal of Optical Technology):

R. O. Malashin, "Training an improved recurrent attention model using an alternative reward function," Journal of Optical Technology. 88(3), 127-130 (2021). https://doi.org/10.1364/JOT.88.000127

Abstract:

A recurrent attention model is considered for application to image classification tasks. Modifications to the approach to improve its accuracy are described. Utilization of the reward in the form of negative cross entropy, which increases the informative value of the reinforcement signal, is proposed for neural network training. A deeper architecture of the attention control subnetwork and an asynchronous actor–critic algorithm are also used. Experiments based on the MNIST and CIFAR datasets are conducted, which confirm the effectiveness of the proposed modifications. Experiments using learned classifiers are also conducted, which demonstrate the complexity of the simultaneous attention control and selection of an embedded classifier for analysis of the chosen patch.

Keywords:

attention control, image classification, reinforcement learning

OCIS codes: 100.4996, 100.4999, 100.4996

References:

1. Minh V., Nicolas H., Graves A., Kavukcouglu K. Recurrent models of visual attention // NIPS Proc. 2014. URL: https://papers.nips.cc/paper/2014/file/09c6c3783b4a70054da74f2538ed47c6-Paper.pdf (accessed 03.05.2020).
2. Bellver M., Giro-i Nieto X., Marques F., Torres J. Hierarchical object detection with deep reinforcement learning // Deep Reinforcement Learning Workshop. 2016. URL: https://imatge.upc.edu/web/sites/default/files/pub/cBellver.pdf (accessed 03.11.2020).
3. Wang X., Yu F., Gonzalez J. SkipNet: Learning dynamic routing in convolutional Networks. 2017. URL: https://arxiv.org/pdf/1711.09485.pdf (accessed 03.11.2020).
4. Wang Z., Sarcar S., Liu J., Zheng Y., Ren X. Outline objects using deep reinforcement learning. 2018. URL: https://arxiv.org/abs/1804.04603 (accessed 03.11.2020).
5. Samyua J., Lord A., Lee N., Torr P. Learn to pay attention // ICLR. 2018. URL: https://arxiv.org/abs/1804.02391 (accessed 03.05.2020).
6. Xiang Li, Wenhai Wang, Xiaolin Hu, Jian Yang. Selective Kernel Networks. 2019. URL: https://arxiv.org/abs/1903.06586 (accessed 03.10.2020).
7. Xiang Li, Wenhai Wang, Xiaolin Hu, Jian Yang. Selective Kernel Networks. 2020. URL: https://arxiv.org/pdf/2004.08955.pdf (accessed 03.11.2020).
8. Vaswani A., Shazeer N., Parmar N., et al. Attention is all you need. 2017. URL: https://arxiv.org/abs/1706.03762 (accessed 03.11.2020).
9. Dosovitskiy A., Beyer L. An image is worth 16×16 words: Transformers for image recognition at scale. 2020. URL: https://arxiv.org/abs/2010.11929 (accessed 03.11.2020).
10. Carion N., Massa F., Synnaeve G., Usunier N., et al. End-to-end object detection with transformers 2020. URL: https://arxiv.org/abs/2005.12872 (accessed 03.11.2020).
11. Malashin. R. Principle of least action in dynamically configured image analysis systems // JOT. 2019. V. 86. № 11. P. 678–685.
12. Chen T., Wang Z., Li G., Lin L. Recurrent attentional reinforcement learning for multi-label image recognition. 2017. URL: http://proceedings.mlr.press/v70/mcgill17a.html (accessed 03.11.2020).