Classification of emotional stress and physical stress using facial imaging features

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Kan Hong Classification of emotional stress and physical stress using facial imaging features (Классификация эмоционального и физического напряжения посредством анализа признаков изображения лица) [на англ. яз.] // Оптический журнал. 2016. Т. 83. № 8. С. 77–83.

Kan Hong Classification of emotional stress and physical stress using facial imaging features (Классификация эмоционального и физического напряжения посредством анализа признаков изображения лица) [in English] // Opticheskii Zhurnal. 2016. V. 83. № 8. P. 77–83.

For citation (Journal of Optical Technology):

Kan Hong, "Classification of emotional stress and physical stress using facial imaging features," Journal of Optical Technology. 83(8), 508-512 (2016). https://doi.org/10.1364/JOT.83.000508

Abstract:

A novel algorithm based on Eulerian magnification and empirical mode decomposition (EM-EMD) is proposed to classify emotional stress and physical stress. Different from previous stress recognition algorithms, EM-EMD does not model the relationship between physiological stress parameters and thermal imprints, but establishes the classification model using different thermal signal features under different types and statuses of stress. It first amplifies blood vessel signals in the human forehead. Later, the proposed algorithm performs frequency division processing on the emotional stress signal and the physical stress signal according to time scale characteristics of the data. Finally, it establishes a classification model of emotional and physical stress using the Gaussian mixture model classifier. Experimental results demonstrated that the EM-EMD algorithm could achieve 85% classification accuracy and could provide a practical method model for future industrial applications. It also shows that the classification rate of the proposed algorithm is better than the conventional classification method. As far as we know, the proposed EM-EMD algorithm is a successful classification model of emotional and physical stress through non-contact imaging.

Keywords:

stress classification, emotional stress, physical stress

Acknowledgements:

The authors are highly thankful for the financial support of National Natural Science Foundation of China (61304111), Beijing Natural Science Foundation (4153059), National Program on Key Basic Research Program of China under grant 2014CB744904, and Science and technology project of Education Department of Jiangxi Province under grant GJJ150798.

OCIS codes: 100.0100

References:

1. Natural Tech: The Adrenal Stress Profile. http://www.natratech.com/Products/saliva_test.htm. Accessed 7 March 2012.
2. Read G.F. Immunoassays of steroids in saliva // Steroid Biochem. 1985. V. 22. Is. 33. P. 437–438.
3. Healey J.A., Picar R.W. Detecting stress during real-world driving tasks using physiological sensors // IEEE Trans. Intell. Transp. Syst. 2005. V. 6. Is. 2. P. 156–166.
4. Barreto A., Zhai J., Adjouadi M. Non-intrusive physiological monitoring for automated stress detection in human–computer interaction // IEEE International Workshop on Human–Computer Interaction. 2007. V. 4796. Is. 1. P. 29–38.
5. Shi Y., Nguyen M.H., Blitz P., French B., Fisk S., DelaTorre F., Smailagic A., Siewiorek D.P., Kumar S. Personalized stress detection from physiological measurements // International Symposium on Quality of Life Technology. 2010. V. 1. Is. 1. P. 28–29.
6. Qiang W., Sourina O., Nguyen M.K. Fractal dimension based neuro-feedback in serious games // Visual Computer. 2011. V. 27. Is. 4. P. 299–309.

7. Stephanos I., Vittorio G., Arcangelo M. Thermal infrared imaging in psychophysiology: potentialities and limits // Psychophysiology. 2014. V. 51. Is. 10. P. 951–963.
8. Pavlidis I., Eberhardt N.L., Levine J.A. Human behavior: seeing through the face of deception // Nature. 2002. V. 415. Is. 6867. P. 35–36.
9. Pavlidis I., Dowdall J., Sun N., Puri C., Fei J. Interacting with human physiology // Computer Vision & Image Understanding. 2007. V. 108. Is. 1. P. 150–170.
10. Pavlidis I., Tsiamyrtzis P., Shastri D., Wesley A., Yan Z. Fast by nature – how stress patterns define human experience and performance in dexterous tasks // Scientific Reports. 2012. V. 2. Is. 3. P. 305–305.
11. Calvin K.L., Duffy V.G. Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement // Occupational Eorgonomics. 2007. V. 7. Is. 2. P. 83–94.
12. Drummond P. The effect of adrenergic blockade on blushing and facial flushing // Psychophysiology. 1997. V. 34. Is. 2. P. 163–168.
13. Ebisch S.J., Aureli T., Bafunno D., Cardone D., Romani G.L. Mother and child in synchrony: thermal facial imprints of autonomic contagion // Biological Psychology. 2012. V. 89. Is. 1. P. 123–129.
14. Nozawa A., Tacano M. Correlation analysis on alpha attenuation and nasal skin temperature // Journal of Statistical Mechanics Theory & Experiment. 2009. V. 9. Is. 1. P. 415–430.
15. Anbar M. Assessment of physiologic and pathologic radiative heat dissipation using dynamic infrared imaging // Annals of the New York Academy of Sciences. 2002. V. 972. Is. 972. P. 111–118.
16. Jarlier S., Grandjean D., Delplanque S., NDiaye K., Cayeux I., Velazco M.I., Sander D., Vuilleumier P., Scherer K.R. Thermal analysis of facial muscles contractions // IEEE Transactions on Affective Computing. 2011. V. 2. Is. 1. P. 2–9.
17. Nhan B.R., Tom C. Classifying affective states using thermal infrared imaging of the human face // IEEE Transactions on Biomedical Engineering. 2010. V. 57. Is. 4. P. 979–987.
18. Kan H., Sheng H. Real time stress assessment using thermal imaging // The Visual Computer. 2015. V. 10. Is. 1. P. 1–9.
19. Kreibig S.D. Autonomic nervous activity in emotion: A review // Biological Psychology. 2010. V. 84. Is. 3. P. 394–421.
20. Nummenmaa L., Glerean E., Hari R., Hietanen J.K. Bodily maps of emotions // Proceedings of the National Academy of Sciences. 2013. V. 111. Is. 2. P. 646–651.
21. Sharma N., Dhall A., Gedeon T., Goecke R. Thermal spatio-temporal data for stress recognition // Eurasip Journal on Image & Video Processing. 2014. V. 2014. Is. 1. P. 1–12.
22. Takahashi T., Kageyama Y., Goto Y., Ishhii M., Nishida M. Detection of amusement feeling by using skin temperature on cheeks // IEEJ Transactions on Electronics Information & Systems. 2014. V. 134. Is. 3. P. 353–354.
23. Kan H., Peter Y. Detection and classification of stress using thermal imaging technique // SPIE 3rd International Conference on Proceedings of the Crime Detection and Prevention. 2009. V. 7486. P. 1–9.
24. Logunova E.V., Shelepin Y.E. Study of the role of spatial-frequency filtering of images when evaluating the age and interpreting the emotional expression of faces // Journal of Optical Technology. 2015. V. 82. Is. 10. P. 694–699.
25. Dowdall J., Pavalidis I.T., Tsiamyrtzis P. Coalitional tracking // Computer Vision & Image Understanding. 2007. V. 106. Is. 2. P. 205–219.
26. Hao Yu.W., Michael R., Shih E., Guttag J. Eulerian video magnification for revealing subtle changes in the world // ACM Transactions on Graphics. 2012. V. 31. Is. 4. P. 1–8.
27. Ne H., Shen Z., Sr L., Mc W., Shih H.H. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis // Proceedings of the Royal Society. A. Mathematical Physical & Engineering Sciences. 1998. V. 454. Is. 1971. P. 903–995.
28. Zhaohua W., Norden E.H. Ensemble empirical mode decomposition A noise assisted data analysis method // Advances in Adaptive Data Analysis. 2009. V. 1. P. 1–41.
29. Jinde Z., Junsheng C., Yu Y. Modified EEMD algorithm and its applications // Journal of vibration and shock. 2013. V. 32. Is. 21. P. 1–21.
30. Mohammadi H., Saeidi R. Speaker identification performance enhancement using Gaussian mixture model with GMM classification post-processor // IEEE International Conference on Signal Processing and Communication. 2007. P. 24–27.
31. Miller R., Plessow F., Kirschbaum C., Stalder T. Classification criteria for distinguishing cortisol responders from nonresponders to psychosocial stress: evaluation of salivary cortisol pulse detection in panel designs // Psychosom Med. 2013. V. 75. Is. 9. Р. 832–840.