УДК: 004.932.72

Robust multi-object tracking using deep learning framework

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Sh. Ch. Pang, Anan Du, Zh. Zh. Yu Robust multi-object tracking using deep learning framework [на англ. яз.] // Оптический журнал. 2015. Т. 82. № 8. С. 33–47.

Sh. Ch. Pang, Anan Du, Zh. Zh. Yu Robust multi-object tracking using deep learning framework [in English] // Opticheskii Zhurnal. 2015. V. 82. № 8. P. 33–47.

For citation (Journal of Optical Technology):

Sh. Ch. Pang, Anan Du, and Zh. Zh. Yu, "Robust multi-object tracking using deep learning framework," Journal of Optical Technology. 82(8), 516-527 (2015). https://doi.org/10.1364/JOT.82.000516

Abstract:

Object appearance representation is crucial to any visual tracker. In order to improve the appearance representation, we propose a deep compact and high-level appearance representation applied to a multi-object tracking algorithm, which is called Deep Multi-object Tracking. In this paper, we adopt the deep learning framework to offline obtain generic image features with auxiliary natural images and online fine-tune our Deep Multi-object Tracking system to adapt to appearance changes of the moving objects. Besides, we have fully considered the temporal information denoting the dynamic duration time of each object. Based on the temporal information and particle filter, our Deep Multi-object Tracking algorithm can effectively generate an online learning and updating model to form a discriminative appearance scheme to achieve successful multi-object tracking. Experiments show that the proposed Deep Multi-object Tracking performs well both indoor and outdoor, where the objects undergo large pose, scale, occlusion, and illumination variations in complex scenes including abnormal ones.

Keywords:

multi-object tracking, deep learning, temporal information, particle filter, online learning

Acknowledgements:

This work has been supported by Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant No.20120061110045) and the Project of Science and Technology Development Plan of Jilin Province, China (Grant No. 20150204007GX).

OCIS codes: 100.4996, 100.4999, 110.4155, 150.4065

References:

1. Reid D. An algorithm for tracking multiple targets // IEEE Trans. Autom. Comtrol. 1979. V. 24. № 6. P. 843–854.
2. Isard M., Blake A. Condensation-conditional density propagation for visual tracking // Int. J. Comput. Vision. 1998. V. 29. № 1. P. 5–28.
3. Okuma K., Taleghani A., Freitas N.D., Little J., Lowe D. A boosted particle filter: Multitarget detection and tracking // Proc. Eur. Conf. Comput. Vision. 2004. P. 28–39.
4. Li Y., Ai H., Yamashita T., Lao S., Kawade M. Tracking in low frame rate video: A cascade particle filter with discriminative observers of different lifespans // Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit. 2007. P. 1–8.
5. Shan C., Tan T., Wei Y. Real-time hand tracking using a mean shift embedded particle filter // Pattern Recognit. 2007. V. 40. № 7. P. 1958–1970.
6. Comaniciu D., Ramesh V., Meer P. Kernel-based object tracking // IEEE Trans. Pattern Anal. Mach. Intell. 2003. V. 25. № 5. P. 564–577.
7. Dalal N., Triggs B. Histograms of oriented gradients for human detection // Proc. IEEE CVPR. 2005. P. 886–893.
8. Wu B., Nevatia R. Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors // Int. J. Comput. Vis. 2007. V. 75. № 2. P. 247–266.
9. Huang C., Wu B., Nevatia R. Robust object tracking by hierarchical association of detection responses // Proc. Eur. Conf. Comput. Vision. 2008. P. 788–801.
10. Xing J., Ai H., Lao S. Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses // Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit. 2009. P. 1200–1207.

11. Henriques J.F., Caseiro R., Batista J. Globally optimal solution to multi-object tracking with merged measurements // Proc. IEEE Int. Conf. Comput. Vision. 2011. P. 2470–2477.
12. Wu Z., Kunz T.H., Betke M. Efficient track linking methods for track graphs using network-flow and set-cover techniques // Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit. 2011. P. 1185–1192.
13. Yang B., Nevatia R. An online learned CRF model for multi-target tracking // Proc. IEEE CVPR. 2012. P. 2034–2041.
14. Milan A., Roth S., Schindler K. Continuous energy minimization for multi-target tracking // Pattern Analysis and Machine Intelligence. IEEE Transactions on. 2014. V. 36. № 1. P. 58–72.
15. Wu X., Zhu X., Wu G.Q. Data mining with big data // Knowledge and Data Engineering, IEEE Transactions on. 2014. V. 26. № 1. P. 97–107.
16. Torralba A., Fergus R., Freeman W.T. 80 million tiny images: A large data set for nonparametric object and scene recognition // Pattern Analysis and Machine Intelligence. IEEE Transactions on. 2008. V. 30. № 11. P. 1958–1970.
17. Wang N., Yeung D.Y. Learning a deep compact image representation for visual tracking // Advances in Neural Information Processing Systems. 2013. P. 809–817.
18. Vincent P., Larochelle H., Bengio Y. Extracting and composing robust features with denoising autoencoders // Proceedings of the 25th international conference on Machine learning. ACM. 2008. P. 1096–1103.
19. Potapov A., Batishcheva V., Peterson M. Limited generalization capabilities of autoencoders with logistic regression on training sets of small sizes // Artificial Intelligence Applications and Innovations. Springer Berlin Heidelberg. 2014. P. 256–264.
20. Morales J.L., Nocedal J. Remark on Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization // ACM Transactions on Mathematical Software (TOMS). 2011. V. 38. № 1. P. 7.
21. Duan K., Keerthi S.S., Chu W. Multi-category classification by soft-max combination of binary classifiers // Multiple Classifier Systems. Springer Berlin Heidelberg. 2003. P. 125–134.
22. Wu Y., Lim J., Yang M.H. Online object tracking: A benchmark // Computer Vision and Pattern Recognition (CVPR). 2013 IEEE Conference on. IEEE. 2013. P. 2411–2418.
23. Zhang L., Maaten L. Structure preserving object tracking // Computer Vision and Pattern Recognition (CVPR). 2013 IEEE Conference on. IEEE. 2013. P. 1838–1845.