Processing the output signals of joint transform correlators using a pre-trained convolutional neural network

Goncharov D.S., Zlokazov E.Y., Petrova E.К., Starikov R.S.

For Russian citation (Opticheskii Zhurnal):

Гончаров Д.С., Злоказов Е.Ю., Петрова Е.К., Стариков Р.С. Обработка выходных сигналов корреляторов совместного преобразования с применением предварительно обученной свёрточной нейронной сети // Оптический журнал. 2025. Т. 92. № 2. С. 87–95. http:// doi.org/10.17586/1023-5086-2025-92-02-87-95

Goncharov D.S., Zlokazov E.Yu., Petrova E.K., Starikov R.S. Processing the output signals of joint transform correlators using a pre-trained convolutional neural network [in Russian] // Opticheskii Zhurnal. 2025. V. 92. № 2. P. 87–95. http://doi.org/10.17586/1023-5086-2025-92-02-87-95

For citation (Journal of Optical Technology):

Abstract:

Research subject. The paper presents the results of the study of coherent optical-digital correlator variants of the coherent optical-digital joint transform correlator with neural network post-processing of output signals. Purpose of the work. Experimental testing of the possibilities of application of deep learning neural network for signal processing of optical-digital diffractive joint transform correlator for pattern recognition problem. Method. A convolutional neural network was applied to process the output signals of an optical-digital diffractive image correlator. The resolution of recognized images at the input of the correlator was 256×256 pixels, the resolution of the fragments of correlation functions fed to the input of the neural network was 32×32 pixels. The data dimensionality reduction provided in this way allows combining the speed of optical processing with the flexibility of the neural network method. Main results: Two setups of a joint transform correlator based on modern models of spatial light modulators have been realized: one based on a digital micromirror modulator and one based on a liquid crystal modulator. To classify correlator signals, a convolutional neural network is applied, pre-trained on numerically generated correlation responses using invariant correlation filters, which provide a given characteristic shape of autocorrelation peak and were synthesized on sets of images different from those used in the experiments. It is demonstrated that in both cases the processing of correlator signals with the help of neural network allows to perform recognition of input images. Practical significance. The results can be applied to the construction of high speed image recognition systems for various purposes.

Keywords:

joint transformation correlator, 1f correlator, image recognition, convolutional neural network, space-time modulator of optical radiation

Acknowledgements:

the work was carried out with the financial support of the Russian Science Foundation, Grant № 23-12-00336

OCIS codes: 070.4550, 070.5010, 100.1160, 100.3005, 100.3008, 100.4550, 100.5010, 100.4996, 200.4260

References:

1. Weaver C., Goodman J. Technique for optically convolving two functions // Appl. Opt. 1966. V. 5. P. 1248. https://doi.org/10.1364/AO.5.001248
2. Rau J. Detection of differences in real distributions // JOSA. 1966. V. 56. P. 1490. https://doi.org/10.1364/ JOSA.56.001490
3. Selected papers on optical pattern recognition using joint transform correlation. Bellingham, WA, USA: Alam M. SPIE, 1999. V. MS157. 658 p.
4. Vijaya Kumar B., Mahalanobis A., Juday R. Correlation pattern recognition. Cambridge UK: Cambridge University Press, 2005. 404 p.
5. Figue J., Refregier Ph., Rajbenbach H., Huignard J.-P. Neural optoelectronic correlator for pattern recognition // Proc. SPIE. 1991. V. 1564. P. 550. https://doi.org/10.1117/12.49757
6. Casasent D. An optical correlator feature extractor neural net system // Opt. Eng. 1992. V. 31. P. 971. https://doi.org/10.1117/12.57138
7. Dadeshidze V., Kompanets I., Lindunen I., Vasiliev A. Adaptive neuromorphic optical correlator with binary phase-only filters // Proc. SPIE. 1994. V. 2051. P. 426. https://doi.org/10.1117/12.166047
8. Javidi B., Li J., Tang Q. Optical implementation of neural networks for face recognition by the use of nonlinear joint transform correlators // Appl. Opt. 1995. V. 34. P. 3950. https://doi.org/10.1364/AO.34.003950
9. Barge M., Heggarty K., Idan Y., Chevallier R. 64-channel correlator implementing a Kohonen-like neural network for handwritten-digit recognition // Appl. Opt. 1996. V. 35. P. 4655. https://doi.org/10.1364/AO.35.004655
10. Lu T., Hughlett C., Zhou H., Chao T.-H., Hanan J. Neural network post-processing of grayscale optical correlator // Proc. SPIE. 2005. V. 5908. P. 590810. https://doi.org/10.1117/12.615573
11. Froehly L., Courvoisier F., Brunner D., Larger L., Devaux F., Lantz E., Dudley J., Jacquot M. Advancing Fourier: space-time concepts in ultrafast optics, imaging, and photonic neural networks // J. Opt. Soc. Am. A. 2019. V. 36. P. C69. https://doi.org/10.1364/JOSAA.36.000C69
12. Wetzstein G., Ozcan A, Gigan S., Fan S., Englund D., Soljačić M., Denz C., Miller D., Psaltis D. Inference in artificial intelligence with deep optics and photonics // Nature. 2020. V. 588. P. 39. https://doi.org/10.1038/s41586-020-2973-6
13. The multiply and Fourier transform unit: A microscale optical processor. White Pages. Updated: Wilson J. 12/12/2020. www.optalysys.com, ссылка взята 25.07.2024
14. Cartlidge E. Photonic computing for sale // Optics & Photonics News. 2023. V. 34. P. 26. https://www.optica-opn.org/home/articles/volume_34/january_2023/features/photonic_computing_for_sale/
15. Haijia Chen, Shaozhen Lou, Quan Wang, Peifeng Huang, Huigao Duan, Yueqiang Hu. Diffractive deep neural networks: Theories, optimization, and applications featured // Appl. Phys. Rev. 2024. V. 11. P. 021332. https://doi.org/10.1063/5.0191977
16. Birch P., Akter H., Young R., Chatwin C. Coherent correlator design analysis for the implementation of deep learning networks // J. Opt. Soc. Am. A. 2024. V. 41. P. 272. https://doi.org/10.1364/JOSAA.504956
17. Goncharov D., Starikov R. Improving correlation method with convolutional neural networks. arXiv:2004.09430 [cs.CV] (2020)
18. Shaulskiy D., Evtikhiev N., Zlokazov E., Starikov S., Starikov R., Petrova E., Molodtsov D. Variants of light modulation for MINACE filter implementation in 4-F correlators // Proc. SPIE. 2015. V. 9598. P. 95980T. https://doi.org/10.1117/12.2190700
19. Evtikhiev N., Shaulskiy D., Zlokazov E., Starikov R. Variants of minimum correlation energy filters: comparative study // Proc. SPIE. 2012. V. 8398. P. 83980G. https://doi.org/10.1117/12.919644