Comparison of the influence of optical and acoustic communication channels on interlocutors

Zhukova, O.V., Shchemeleva, О.V.

Full text «Opticheskii Zhurnal»

Full text on elibrary.ru

Publication in Journal of Optical Technology

For Russian citation (Opticheskii Zhurnal):

Жукова О.В., Щемелева О.В. Соотношение влияния оптических и акустических каналов связи на собеседников // Оптический журнал. 2020. Т. 87. № 10. С. 38–49. http://doi.org/10.17586/1023-5086-2020-87-10-38-49

Zhukov O. V. and Shchemeleva O. V. Comparison of the influence of optical and acoustic communication channels on interlocutors [in Russian] // Opticheskii Zhurnal. 2020. V. 87. № 10. P. 38–49. http://doi.org/10.17586/1023-5086-2020-87-10-38-49

For citation (Journal of Optical Technology):

O. V. Zhukova and O. V. Shchemeleva, "Comparison of the influence of optical and acoustic communication channels on interlocutors," Journal of Optical Technology. 87(10), 590-597 (2020). https://doi.org/10.1364/JOT.87.000590

Abstract:

This study is devoted to the search for influential markers of optical and acoustic communication channels on the interlocutors in face-to-face and back-to-back communication. The rhythm patterns in different frequency ranges, including the projection of facial muscles on electroencephalogram and electromyogram data, are analyzed. The patterns of the electroencephalogram rhythms are shown to have similar and distinctive features in different communication modes. The maximum number of distinctions between various communication modes with interlocutors situated back-to-back was observed in Δ and γ rhythms in the brain. When the interlocutors are positioned face-to-face, a powerful nonverbal component increases the power of β and γ rhythms and levels out the distinctions between different communication modes. The frontal (F7 and F8) and occipital (O1 and O2) areas of the brain are the most informative and demonstrate significant distinctions between different communication modes at different interlocutor positions relative to each other.

Keywords:

electroencephalogram, nonverbal and verbal communication, facial expressions, communication, brain rhythms

OCIS codes: 100.4996, 170.6960, 330.5020

References:

1. Yu. E. Shelepin, Introduction to Neuroiconica (Troitskii most, St. Petersburg, 2017).
2. G. Buzsaki, Rhythms of the Brain (Oxford University Press, Oxford, 2006).
3. Yu. D. Kropotov, Quantitative EEG, Cognitive Induced Potentials of the Human Brain and Neurotherapy (Zaslavskiy Yu. A., Donetsk, 2010).
4. E. A. Kostandov and E. A. Cheremushkin, “Changes in the lowand high-frequency oscillations of the EEG -band in the intervals between meaningful visual stimuli,” Hum. Physiol. 39(4), 339–345 (2013) [Fiziol. Chel. 39(4), 5–12 (2013)].
5. V. A. Ponomarev, M. V. Pronina, and Yu. D. Kropotov, “Dynamics of the EEG spectral density in the , , and bands in the visual Go/NoGo task,” Hum. Physiol. 43(4), 366–376 (2017) [Fiziol. Chel. 43(4), 13–24 (2017)].
6. J. Rimmele, J. Gross, S. Molholm, and A. Keitel, Brain Oscillations in Human Communication (Frontiers Media, Lausanne, 2018), pp. 1–4.
7. P. R. Montague, G. S. Berns, J. D. Cohen, S. M. McClure, G. Pagnoni, and M. Dhamala, “Hyperscanning: simultaneous fMRI during linked social interactions,” Neuroimage 16(4), 1159–1164 (2002).
8. T. D. Duane and T. Behrendt, “Extrasensory electroencephalographic
induction between identical twins,” Science 150, 367 (1965).
9. A. Pérez, G. Dumas, M. Karadagg, and J. A. Duñabeitiach, “Differential brain-to-brain entrainment while speaking and listening in native and foreign languages,” Cortex 111, 303–315 (2019).
10. T. Kinoshita, H. Tanaka, K. Yoshino, and S. Nakamura, “Measuring
affective sharing between two people by EEG hyperscanning,” in Adjunct of the 2019 International Conference on Multimodal Interaction (ICMI) (2019), article 3.
11. O. V. Zhukova, Yu. E. Shelepin, O. V. Shchemeleva, P. P. Vasil’ev, and G. A. Moiseenko, “Influence of verbal and nonverbal signals on an interlocutor’s electroencephalogram,” J. Opt. Technol. 85(8), 455–462 (2018) [Opt. Zh., 85(8), 13–21 (2018)].
12. R. N. Vigario, “Extraction of ocular artifacts from EEG using independent component analysis,” Electroencephalogr. Clin. Neurophysiol. 103, 395–404 (1997).
13. T. P. Jung, S. Makeig, M. Westerfield, J. Townsend, E. Courchesne,
and T. J. Sejnowski, “Removal of eye artifacts from visual eventrelated
potentials in normal and clinical subjects,” Clin. Neurophysiol. 111, 1745–1758 (2000).
14. E. P. Tereshchenko, V. A. Ponomarev, Yu. D. Kropotov, and A. Müller,
“Comparative efficiencies of different methods for removing blink artifacts in analyzing quantitative electroencephalogram and eventrelated potentials,” Hum. Physiol. 35(2), 124–131 (2009) [Fiziol. Chel. 35(2), 124–131 (2009)].
15. F. Perrin, J. Pernier, O. Bertrand, and J. F. Echallier, “Spherical splines for scalp potential and current density mapping,” Electroencephalogr. Clin. Neurophysiol. 72, 184–187 (1989).
16. E. Niedermeyer, “The normal EEG of the waking adult,” in Electroencephalography: Basic Principles, Clinical Applications, and Related Fields (Lippinkot Williams and Wilkins, Philadelphia, 2005), pp. 167–192.
17. F. Lopes da Silva, “EEG: origin and measurement,” in EEG–fMRI,
Physiological Basis, Technique and Applications (Springer Verlag, Berlin, 2010), pp. 19–38.
18. S. D. Muthukumaraswamy, “Mu rhythm modulation during bservation
of an object-directed grasp,” Cognit. Brain Res. 19, 195–201 (2004).
19. B. Güntekin and E. Basar, “Facial affect manifested by multiple scillations,” Int. J. Psychophysiol. 71 (1), 31–36 (2009).
20. A. L. Giraud and D. Poeppel, “Cortical oscillations and speech processing: emerging computational principles and operations,” Nat.
Neurosci. 15, 511–517 (2012).
21. N. Ding, L. Melloni, H. Zhang, X. Tian, and D. Poeppel, “Cortical
tracking of hierarchical linguistic structures in connected speech,” Nat. Neurosci. 19, 158–164 (2016).
22. G. G. Knyazev, J. Y. Slobodskoj-Plusnin, and A. V. Bocharov, “Eventrelated delta and theta synchronization during explicit and emplicit
emotion processing,” Neuroscience 164(4), 1588–1600 (2009).
23. E. A. Kostandov and E. A. Cheremushkin, “Induced synchronization/
desynchronization of the theta and alpha cortical electrical activity to a face stimuli with increasing visual working memory load,” Zh. Vyssh. Nervn. Deyat. im. I. P. Pavlova 61(1), 35–46 (2011).
24. B. C. M. Van Wijk, P. J. Beek, and A. Daffertshofer, “Neural synchrony within the motor system: what have we learned so far?” Front. Hum. Neurosci. 6, 252 (2012).
25. N. E. Crone, D. L. Miglioretti, B. Gordon, and R. P. Lesser, “Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band,” Brain 121, 2301–2315 (1998).
26. V. A. Ponomarev, “Hidden sources of electroencephalogram and
potentials connected to events and their significance,” in Doctor of
Sciences thesis (N. P. Bekhtereva Institute for Human Brain Research
of Russian Academy of Sciences, St. Petersburg, 2016).
27. W. Lutzenberger, F. Pulvermüller, T. Elbert, and N. Birbaumer, “Visual
stimulation alters local 40 Hz responses in humans: an EEG study,” Neurosci. Lett. 183, 39–42 (1995).
28. F. B. Campbell and Yu. E. Shelepin, “Foveola abilities in object ecognition,” Sens. Sist. 4(2), 181–185 (1990).