Edge-directed image interpolation as neuromorphic decoding of their sampling representations

For Russian citation (Opticheskii Zhurnal):

Анциперов В.Е., Кершнер В.А. Пограничная интерполяция изображений как метод нейроморфного декодирования их выборочных представлений // Оптический журнал. 2025. Т. 92. № 1. С. 53–67. http://doi.org/10.17586/1023-5086-2025-92-01-53-67

Antsiperov V.E., Kershner V.A. Edge-directed image interpolation as neuromorphic decoding of their sampling representations [in Russian] // Opticheskii Zhurnal. 2025. V. 92. № 1. P. 53–67. http://doi.org/10.17586/1023-5086-2025-92-01-53-67

For citation (Journal of Optical Technology):

Abstract:

Subject of study is the synthesis of data processing and analysis procedures in the form of event streams for image encoding and decoding tasks based on neuroiconics. Aim of study. A model of neuromorphic encoding and subsequent decoding of video data, algorithms for image recovery and object detection. Method. A sampling representation of images is used. Based on the statistics of the sampling representation, a generative encoder model is formed, which is a joint distribution of input and encoded data in the form of a mixture of components. To model the mechanisms of lateral inhibition, the structure of a system of receptive fields is introduced. Main results. The modeling of the mechanisms of primary neuro-processing of video data in the periphery of the visual system is implemented, neuromorphic type procedures for encoding and restoring images are synthesized. The experience of numerical testing and optimization of the developed algorithms has shown that it is possible to avoid computational problems associated with processing massive data and adapt the approach to modern neural network tasks. Practical significance. The synthesized procedures can be used in modern communication systems, as well as in related tasks of searching, identifying, etc. objects in digital images. In addition, the proposed approach can be used as a basis for the analysis/synthesis of other neuromorphic information systems focused on working with data streams.

Keywords:

neuromorphic systems, neuroiconics, sampling representation, neural coding, receptive field system, adaptive filtering, edge directed image interpolation

OCIS codes: 100.2960, 100.3020, 330.4060

References:

1. Lee R. Big data, cloud computing, and data science engineering. 1st ed. Switzerland: Springer, 2020. С. 214. https://doi.org/10.1007/978-3-030-24405-7
2. Bull D.R., Zhang F. Intelligent image and video compression: Communicating pictures. 2nd ed. London: Academic Press, 2021. P. 608.
3. Tschannen M., Agustsson E., Lucic M. Deep generative models for distribution-preserving lossy compression // Proc. of the 32nd Intern. Conf. Neural Information Processing Systems — NIPS. Montréal, Canada. December 3–8, 2018. Р. 5933–5944. https://doi.org/10.48550/arxiv.1805.11057
4. Анциперов В.Е. Генеративная модель автокодировщиков, самообучающихся на изображениях, представленных выборками отсчетов // Автоматика и телемеханика. 2022. № 12. С. 108–140. https://doi. org/10.31857/S0005231022120091
Antsiperov V.E. Generative model of autoencoders self-learning on images represented by count samples // Automation and Remote Control. 2022. V. 83. № 12. P. 1959–1983. https://doi.org/10.1134/S00051179220120098
5. Шелепин Ю.Е. Введение в нейроиконику. СПб.: Троицкий мост, 2017. С. 352.
Shelepin Yu.E. Introduction to neuroiconics: Monograph [in Russian]. St. Petersburg: "Troitsky Most" Publ., 2017. P. 352.
6. Шелепин Ю.Е., Луцив В.Р., Коротаев В.В. Оптические технологии и зрительная картина мира: иконика и нейроиконика // Оптический журнал. 2022. Т. 89. № 8. С. 3–7. http://doi.org/10.17586/1023-5086-2022-89-08-03-07
Shelepin Yu., Lutsiv V., Korotaev V. Optical technologies and the visual picture of the world: Iconics and neuroiconics // J. Opt. Technol. 2022. V. 89. P. 434–436. https://doi.org/10.1364/JOT.89.000434
7. Явна Д.В., Бабенко В.В., Горбенкова О.А. и др. Категоризация объектов и сцен нейронной сетью, входы которой предварительно обучены декодированию пространственных неоднородностей текстуры // Оптический журнал. 2023. Т. 90. № 1. С. 37–48. http:// doi.org/10.17586/1023-5086-2023-90-01-37-48
Yavna D.V., Babenko V.V., Gorbenkova O.A., et al. Classification of objects and scenes by a neural network with pretrained input modules to decode spatial texture inhomogeneities // J. Opt. Technol. 2023. V. 90. № 1. P. 20–25. https://doi.org/10.1364/JOT.90.000020
8. Васильев В.Н., Тибилов А.С., Шелепин Ю.Е. Подавление непрерывных шумов фоторецепторов — палочек под действием отрицательной обратной связи, формируемой горизонтальной клеткой // Оптический журнал. 2020. Т. 87. № 6. С. 24–33. http://doi.org/10.17586/1023-5086-2020-87-06-24-33
Vasil’ev V.N., Tibilov A.S., Shelepin Yu.E. Suppressing the continuous noise of photoreceptor rods under the action of negative feedback formed by a horizontal cell // J. Opt. Technol. 2020. V. 87. № 6. P. 342–348. https:// doi.org/10.1364/JOT.87.000342
9. Streit R.L. Poisson point processes. Imaging, tracking, and sensing. Springer, 2010. P. 273. https://doi. org/10.1007/978-1-4419-6923-1
10. Antsiperov V., Kershner V. Retinotopic image encoding by samples of counts // Pattern Recognition Applications and Methods, ICPRAM 2021–2022. Lecture Notes in Computer Science / Ed. De Marsico, et al. Springer, Cham., 2023. P. 52–75. http://doi.org/10.1007/978-3-031-24538-1_3
11. Latecki L.J., Lakamper R., Eckhardt T. Shape descriptors for non-rigid shapes with a single closed contour // Proc. IEEE Conf. Computer Vision and Pattern Recognition CVPR. 2000. (Cat. No. PR00662). June 13–15, 2000. Р. 424–429. https://doi.org/10.1109/CVPR.2000.855850
12. Schiller P.H., Tehovnik, E.J. Vision and the visual system. Oxford: Oxford University Press, 2015. Р. 408. http:// doi.org/10.1093/acprof:oso/9780199936533.001.0001
13. Gauthier J.L., Field G.D., Sher А., et al. Receptive fields in primate retina are coordinated to sample visual space more uniformly // PLoS Biol. 2009. V. 7. № 4. P. e1000063. https://doi.org/10.1371/journal. pbio.1000063
14. Shamir O., Sabato S., Tishby N. Learning and generalization with the information bottleneck // Theoretical Computer Sci. 2010. V. 411. № 29–30. P. 2696–2711. https://doi.org/10.1016/j.tcs.2010.04.006
15. Кемпбелл Ф.В., Шелепин Ю.Е. Возможности фовеолы в различении объектов // Сенсорные системы. 1990. Т. 4. № 2. С. 181–185.
Campbell F.W., Shelepin Yu.E. The foveola capacity for object recognition [in Russian] // Sensornye Sistemy [Sensory Systems]. 1990. V. 4(2). P. 181–185.
16. Zapp S.J., Nitsche S., Gollisch T. Retinal receptivefield substructure: Scaffolding for coding and computation // Trends in Neurosciences (Regular ed.). 2022. V. 45. № 6. P. 430–445. https://doi.org/10.1016/j.tins. 2022.03.005
17. Young G.A., Smith R.L. Essentials of statistical inference. Cambridge, UK; New York: Cambridge University Press, 2005. V. 16. P. 225.
18. Marr D., Hildreth E. Theory of edge detection // Proc. Royal Soc. of London. Ser. B. Biological Sci. 1980. V. 207. № 1167. P. 187–217. https://doi.org//10.1098/ rspb.1980.0020
19. Kok C.W., Tam W.-S. Edge-directed interpolation. Digital image interpolation in MatLab. 1st ed. Wiley: IEEE Press, 2019. P. 191–209. https://doi.org/ 10.1002/9781119119623.ch7