DOI: 10.17586/1023-5086-2020-87-06-51-56

Depth map denoising using bilateral filter and progressive CNNs.

Li Shuaihao, Zhu Weiping, Zhang, Bin, Yang Xinfeng, Chen Min

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Shuaihao Li, Weiping Zhu, Bin Zhang, Xinfeng Yang, Min Chen Устранение шумов в картах глубин с использованием билатерального фильтра и прогрессивных свёрточных нейронных сетей // Оптический журнал. 2020. Т. 87. № 6. С. 51–56. http://doi.org/10.17586/1023-5086-2020-87-06-51-56

Shuaihao Li, Weiping Zhu, Bin Zhang, Xinfeng Yang, Min Chen Depth map denoising using bilateral filter and progressive CNNs [in English] // Opticheskii Zhurnal. 2020. Т. 87. № 6. С. 51–56. http://doi.org/10.17586/1023-5086-2020-87-06-51-56

For citation (Journal of Optical Technology):

Shuaihao Li, Weiping Zhu, Bin Zhang, Xinfeng Yang, and Min Chen, "Depth map denoising using a bilateral filter and a progressive CNN," Journal of Optical Technology 87(6), 361-364 (2020). https://doi.org/10.1364/JOT.87.000361

Abstract:

With the advantages of low cost and real-time acquisition of depth and color maps of the object, Time-of-Flight (ToF) camera has been used in 3D reconstruction. However, due to the hardware shortage of ToF sensors, the depth maps obtained by ToF camera has a lot of noise, which limits its subsequent application. Therefore, it is necessary to denoise the depth maps by software method. We propose an algorithm for denoising depth maps by combining the bilateral filter and Progressive Convolution Neural Networks (PCNN). The algorithm takes a single depth map as input. Firstly, the first individual network of the PCNN is used to denoise the depth map, and then the bilateral filter and the second individual network of the PCNN are used to further process, so that the edge information of depth maps can be retained on the basis of fine denoising. Finally, we have carried out experiments on the popular Middlebury dataset. The experimental results show that the proposed algorithm is obviously superior to the traditional methods.

Keywords:

depth map denoising, 3D reconstruction, bilateral filter, Progressive convolution neural networks

OCIS codes: 100.0100 100.2980 100.6890

References:

1. Schuon S., Theobalt C., Davis J., Thrun S. Lidarboost: depth superresolution for ToF 3d shape scanning // IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Florida. 23 June 2009. P. 343–350.

2. Shuaihao Li, Qianqian Li, Min Chen et al. 3D reconstruction of oil refinery buildings using a depth camera // Chemistry & Technology of Fuels & Oils. 2018. V. 54(5). P. 613–624.

3. Halber M., Funkhouser T. Fine-to-Coarse Global Registration of RGB-D Scans // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society. Hawaii. 22 July 2017. P. 1755–1764.

4. Wang N., Zhang Y., Li Z. et al. Pixel2Mesh: Generating 3D mesh models from single RGB Maps // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Utah. 20 June 2018. P. 52–67.

5. Yan S., Wu C., Wang L. et al. DDRNet: Depth Map denoising and refinement for consumer depth cameras using cascaded // Proceedings of the European Conference on Computer Vision (ECCV). Munich. 12 September 2018. P. 151–167.

6. Tomasi C., Manduchi R. Bilateral filtering for gray and color maps // IEEE International Conference on Computer Vision. Bombay. 7 January 1998. P. 839–846.

7. Greene N., Heckbert P. Creating raster OmnimaxMaps from multiple perspective views using the elliptical weighted average filter // IEEE Proc Computer Graphics & Applications. 1986. V. 6(6). P. 21–27.

8. Zhe L.V., Wang F.L., Chang Y.Q. et al. Region-adaptive Median Filter // Journal of System Simulation. 2007. V. 19(23). P. 5411–5414.

9. Ito K. Gaussian filter for nonlinear filtering problems // IEEE transactions on automatic control. 2000. V. 45(5). P. 910–927.

10. Vincent P., Larochelle H., Lajoie I. et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion // Journal of Machine Learning Research. 2010. V. 11(12). P. 3371–3408.

11. Ren H., El-Khamy M., Lee J. Image super resolution based on fusing multiple convolution neural networks // IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii. 21 July 2017. P. 54–61.

12. Dong C., Loy C.C., He K. et al. Image super-resolution using deep convolutional networks // IEEE Transaction on Pattern Analysis and Machine Intelligence. 2016. V. 38(2). P. 295–307.

13. Rumelhart D.E., Hinton G.E., Williams R.J. Learning representations by back-propagating errors // Nature. 1986. V. 323(6088). P. 533.

14. Scharstein D., Pal C. Learning conditional random fields for stereo // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Minneapolis. MN. 19 June 2007. P. 1–8.

15. Coleman S. Black holes as red herrings: topological fluctuations and the loss of quantum coherence // Nuclear Physics B. 1988. V. 307(4). P. 867–882.

16. Jia Y., Shelhamer E., Donahue J. et al. Caffe: convolutional architecture for fast feature embedding // Proceedings of the 22nd ACM International Conference on Multimedia. Orlando. Florida. 4 November 2014. P. 675–678.