A method for reconstructing grayscale images from 3D scene holograms based on a physically informed generative-adversarial neural network

Kiriy S.A., Rymov D.А., Svistunov A.S., Starikov R.S., Shifrina A.V., Rodin, V.G., Cheremkhin, P.А.

For Russian citation (Opticheskii Zhurnal):

Кирий С.А., Рымов Д.А., Свистунов А.С., Стариков Р.С., Шифрина А.В., Родин В.Г., Черёмхин П.А. Метод реконструкции полутоновых изображений с голограмм 3D-сцен на основе физически осведомлённой генеративно-состязательной нейросети // Оптический журнал. 2026. Т. 93. № 12. С. 70–79. http://doi.org/10.17586/1023-5086-2026-93-12-70-79

Kiriy S.A., Rymov D.A., Svistunov A.S., Starikov R.S., Shifrina A.V., Rodin V.G., Cheremkhin P.A. A method for reconstructing grayscale images from 3D scene holograms based on a physically informed generative-adversarial neural network [in Russian] // Opticheskii Zhurnal. 2026. V. 93. № 12. P. 70–79. http://doi.org/10.17586/1023-5086-2026-93-12-70-79

For citation (Journal of Optical Technology):

Abstract:

The subject of the study is the investigation of the possibilities of using neural network approaches to reconstruct grayscale images of three-dimensional scene sections from single holograms and evalu ating the quality of the resulting reconstructions. The proposed method based on a physically aware generative-adversarial neural network, provides both image recovery and improved reconstruction quality relative to existing approaches. The aim of the work is to improve holographic image restoration quality by considering physical features of hologram formation. The methodology involves modeling the hologram recording process using the Fresnel diffraction approximation and training a generative adversarial neural network using synthesized and optically recorded digital holograms. A physical criterion has been introduced into the generator error function — a comparison of the original hologram with the calculated one based on reconstruction, and additional modifications have been applied. The input is a single hologram, and the method produces a set of reconstructed scene sections at the output. The main results. The proposed method ensured the recovery of images from holograms with a high level of quality. The value of the structural similarity index reaches 0.89, which is higher compared to the neural network method using the U-Net architecture. Practical significance. This method can be used for high-quality and high-speed image recovery from digital holographs, in cluding microscopy, 3D visualization and image recognition.

Keywords:

digital holography, image reconstruction, generative adversarial network, deep le- arning, physics-informed models, grayscale images, computer-generated holography, 3D scene

Acknowledgements:

the study was financially supported by the Russian Science Foundation, Project № 22-79-10340-П.

OCIS codes: 090.1995, 110.4280, 230.5160, 090.1760

References:

1. Schnars U., Falldorf C., Watson J., Jüptner W. Digital holography and wavefront sensing: Principles, tech niques and applications. Berlin Heidelberg: Springer Verlag, 2015. 226 p.

2. Huang M., Han Y., Liu Y. et al. Phase-shifting-free structured illumination for enhanced resolution in off axis digital holographic microscopy // Opt. Laser Technol. 2025. V. 192. P. 113772. DOI: 10.1016/j.optlastec. 2025.113772

3. Dudaie M., Barnea I., Nissim N., Shaked N. On-chip label-free cell classification based directly on off-axis holograms and spatial-frequency-invariant deep lear- ning // Sci. Rep. 2023. V. 13. № 1. P. 12370. DOI: 10.1038/s41598-023-38160-3

4. Юдин Н.Н., Павлов П.В., Зиновьев М.М. и др. Оцен ка усталостных повреждений авиационного фторор ганического стекла методами цифровой голографии // Оптический журнал. 2021. Т. 88. № 2. С. 20–26. DOI:10.17586/1023-5086-2021-88-02-20-2

Yudin N.N., Pavlov P.V., Zinov’ev M.M. et al. Assess ment of fatigue damage of fluoroorganic aircraft glass using digital holography methods // J. Opt. Technol. 2021. V. 88. P. 72–76. DOI: 10.1364/JOT.88.000072

5. Черёмхин П.А., Рымов Д.А., Свистунов А.С. и др. Нейросетевые методы в цифровой и компьютерной голографии. Обзор // Оптический журнал. 2024. Т. 91. № 3. С. 62–78. DOI:10.17586/1023-5086-2024 91-03-62-78

Cheremkhin P.A., Rymov D.A., Svistunov A.S. et al. Neural-network-based methods in digital and comput er-generated holography: a review// Journal of Optical Technology. 2024. V. 91. № 3. P. 170–180. DOI: 10.1364/ JOT.91.000170

6. Leith E.N., Upatnieks J. Wavefront reconstruction with diffused illumination and three-dimensional ob jects // J. Opt. Soc. Am. 1964. V. 54. P. 1295–1301. DOI: 10.1364/josa.54.001295

7. Yamaguchi I., Zhang T. Phase-shifting digital holog raphy // Opt. Lett. 1997. V. 22. P. 1268–1270. DOI: 10.1364/ol.22.001268

8. Stoykova E., Kang H., Park J. Twin-image problem in digital holography — A survey (Invited Paper) // Chin. Opt. Lett. 2014. V. 12. P. 060013–060024. DOI: 10.3788/col201412.060013

9. Situ G. Deep holography // Light. Adv. Manuf. 2022. V. 3. P. 1–23. DOI: 10.37188/lam.2022.013

10. Рымов Д.А., Черёмхин П.А., Стариков Р.С. Нейросе тевая реконструкция сцен с цифровых голограмм на основе извлечения амплитуды и фазы // Оптический журнал. 2022. Т. 89. № 9. С. 11–19. DOI:10.17586/1023 5086-2022-89-09-11-19

Rymov D.A., Cheremkhin P.A., Starikov R.S. Neural network enabled holographic image reconstruction via amplitude and phase extraction // Journal of Optical Technology. 2022. V. 89. № 9. P. 511–516. DOI: 10.1364/ JOT.89.000511

11. Xu X., Luo W., Ren Z., Song X. Intelligent detection and recognition of marine plankton by digital hologra phy and deep learning // Sensors. 2025. V. 25. № 7. P. 2325. DOI: 10.3390/s25072325

12. Svistunov A.S., Rymov D.A., Starikov R.S., Cherem khin P.A. HoloForkNet: digital hologram reconstruction via multi-branch neural network // Applied Sci ences. 2023. V. 13. № 10. P. 6125. DOI: 10.3390/ app13106125

13. Manisha, Mandal A.C., Rathor M. et al. Randomness assisted in-line holography with deep learning // Sci Rep. 2023. V. 13. P. 10986. DOI: 10.1038/s41598-023 37810-w

14. Drozdov M.K., Rymov D.A., Svistunov A.S. et al. Light weight neural network for holographic reconstruction of pseudorandom binary data // Technologies. 2025. V. 13. P. 474. DOI: 10.3390/technologies13100474

15. Khan A., Zhijiang Z., Yu Y. et al. GAN-Holo: Genera tive adversarial networks-based generated holography using deep learning // Complexity. 2021. V. 2021. P. 1–7. DOI: 10.1155/2021/6662161

16. Lu Z., Cao Y., Liu M. et al. Image-to-image translation for improved digital holographic reconstruction based on a generative adversarial network learning frame work // Optics and Laser Technology. 2023. V. 166. P. 109654. DOI: 10.1016/j.optlastec.2023.109654

17. Kiriy S.A., Rymov D.A., Svistunov A.S. et al. Genera tive adversarial neural network for 3D-hologram re construction // Laser Physics Letters. 2024. V. 21. № 4. P. 045201. DOI: 10.1088/1612-202X/ad26eb

18. Creswell A., White T., Dumoulin V. et al. Generative adversarial networks: An overview // IEEE Signal Processing Magazine. 2018. V. 35. P. 53–65. DOI: 10.1109/MSP.2017.2765202

19. Ronneberger O., Fischer T., Brox T. U-Net: Convolu tional networks for biomedical image segmentation // Int. Conf. Med. image Comput. Comput. Interv. 2015. V. 9351. P. 234–241. DOI: 10.1007/978-3-319-24574 4_28

20. Verrier N., Atlan M. Off-axis digital hologram recon struction: Some practical considerations // Applied Optics. 2011. V. 50. P. H136–H146. DOI: 10.1364/AO. 50.00H136

21. Wang Z., Bovik A.C., Sheikh H.R., Simoncelli E.P. Im age quality assessment: from error visibility to struc tural similarity // IEEE Trans. Image Process. 2004. V. 13 (4). P. 600–612. DOI: 10.1109/TIP.2003.819861

22. Miyato T., Kataoka T., Koyama M., Yoshida Y. Spectral normalization for generative adversarial networks // Proceedings of ICLR. 2018. P. 1–26. http://arxiv.org/ abs/1802.05957

23. Ulyanov D., Vedaldi A., Lempitsky V. Instance normal ization: The missing ingredient for fast stylization // Arxiv. 2016. http://arxiv.org/abs/1607.08022