Search by item HOME > Access full text > Search by item

JBE, vol. 27, no. 5, pp.738-750, September, 2022

DOI: https://doi.org/10.5909/JBE.2022.27.5.738

Brain Correlates of Emotion for XR Auditory Content

Sangin Park, Jonghwa Kim, Soon Yong Park, and Sungchul Mun

C.A E-mail: auditory stimulus, XR interface, brain connectivity, human factors, emotion

Abstract:

In this study, we reviewed and discussed whether auditory stimuli with short length can evoke emotion-related neurological responses. The findings implicate that if personalized sound tracks are provided to XR users based on machine learning or probability network models, user experiences in XR environment can be enhanced. We also investigated that the arousal-relaxed factor evoked by short auditory sound can make distinct patterns in functional connectivity characterized from background EEG signals. We found that coherence in the right hemisphere increases in sound-evoked arousal state, and vice versa in relaxed state. Our findings can be practically utilized in developing XR sound bio-feedback system which can provide preference sound to users for highly immersive XR experiences.



Keyword: auditory stimulus, XR interface, brain connectivity, human factors, emotion

Reference:
[1] Apple Glasses: here's everything we know so far, TechRadar News, https://www.techradar.com/news/apple-glasses(accessed Jul. 03, 2022).
[2] Apple Quietly Released One of The Most Impressive AR Room-mapping Tools, Road to VR - Virtual Reality News, https:// www.roadtovr.com/apple-roomplan-ar-mapping (accessed Jul. 03, 2022).
[3] H. Huang and C.-F. Lee, “Factors affecting usability of 3D model learning in a virtual reality environment,” Interactive Learning Environments, Vol.30, No.5, pp. 848-861, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.1080/10494820.2019.1691605
[4] Y. Xing et al., “Web XR User Interface Research: Design 3D Layout Framework in Static Websites,” Applied Sciences, Vol.12, No.11, pp. 5600, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.3390/app12115600
[5] DeepMotion Animate 3D, https://www.deepmotion.com (accessed Jul. 7, 2022).
[6] OpenBCI and Varjo Partner to Bring Neurotechnology to Spatial Computing, https://varjo.com/company-news/openbci-and-varjo-partner- to-bring-neurotechnology-to-spatial-computing (accessed Jul. 7, 2022).
[7] S. Mun, E. S. Kim, and M. C. Park, “Effect of Mental Fatigue Caused by Mobile 3d Viewing on Selective Attention: An Erp Study,” International Journal of Psychophysiology, Vol.94, No.3, pp.373-381, 2014, (accessed Jul. 7, 2022). doi: https://doi.org/10.1016/j.ijpsycho.2014.08.1389
[8] V. Putkinen et al., “Decoding Music-Evoked Emotions in the Auditory and Motor Cortex,” Cerebral Cortex, Vol.31, No.5, pp.2549-2560, 2020, (accessed Jul. 7, 2022). doi: https://doi.org/10.1093/cercor/bhaa373
[9] L. Taruffi, S. Skouras, C. Pehrs, and S. Koelsch, “Trait Empathy Shpaes Neural Responses Toward Sad Music,” Cognitive, Affective, & Behavioral Neuroscience, Vol.21, No.1, pp.231-241, 2021, (accessed Jul. 8, 2022). doi: http://doi.org/10.3758/s13415-020-00861-x
[10] L. Taruffi et al., “Effects of Sad and Happy Music on Mind-Wandering and the Default Mode Network,” Scientific Reports, Vol.7, pp. 14396, 2017, (accessed Jul. 8, 2022). doi: https://doi.org/10.1038/s41598-017-14849-0
[11] S. Koelsch et al., “Neocortical Substrates of Feelings Evoked with Music in the ACC, Insula, and Somatosensory Cortex,” Scientific Reports, Vol.11, No. 1, pp. 10119, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1038/s41598-021-89405-y
[12] B. Lee and W. Son, “Evaluation of Human Factors in Extended Reality Content,” Information and Communications Magazine, The Korean Institute of Commucations and Information Sciences, Sempember- 2021 (accessed Jul. 8, 2022).
[13] G. Marion, G. M. D. Liberto, and S. A. Shamma, “The Music of Silence: Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics,” The Journal of Neuroscience, Vol.41, No. 35, pp. 7435-7448, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1523/JNEUROSCI.0183-21.2021
[14] S. Sangnark et al., “Revealing Preference in Popular Music Through Familiarity and Brain Response,” IEEE Sensors Journal, Vol.21, No. 13, pp. 14931-14940, 2021, (accessed Jul. 8, 2022). doi: https://doi.org/10.1109/JSEN.2021.3073040
[15] E. Fox, “Perspectives from affective science on understanding the nature of emotion,” Brain and Neuroscience Advances, Vol.2, pp. 1-8, 2018, (accessed Sep. 1, 2022). doi: https://doi.org/10.1177/2398212818812628
[16] H. Engelbregt et al., “Effects of binaural and monaural beat stimulation on attention and EEG,” Experimental Brain Research, Vol.239, pp. 2781-2791, 2021, (accessed Sep. 1, 2022). doi: https://doi.org/10.1007/s00221-021-06155-z
[17] S. Park and M. Whang, “Infrared camera-based non-contact measurement of brain activity from pupillary rhythms,” Frontiers in Physiology, Vol.9, pp. 1400, 2018, (accessed Sep. 1, 2022). doi: https://doi.org/10.3389/fphys.2018.01400
[18] L. A. Schmidt and L. J., “Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions,” Cognition & Emotion, Vol.15, No. 4, pp. 487-500, 2001, (accessed Sep. 1, 2022). doi: https://doi.org/10.1080/02699930126048
[19] J. H. Kim et al., “A research on EEG coherence variation by relaxation,” Korean Journal of the science of Emotion & sensibility, Vol.13, No. 1, pp. 121-128, 2010 (accessed Jul. 8, 2022).
[20] S. Mun, et al., “SSVEP and ERP Measurement of Cognitive Fatigue Caused by Stereoscopic 3d,” Neuroscience Letters, Vol.525, No.2, pp.89-94, 2012, (accessed Jul. 8, 2022).doi: https://doi.org/10.1016/j.neulet.2012.07.049
[21] S. Mun, M.-C. Park, and S. Yano, “Performance Comparison of a SSVEP BCI Task by Individual Stereoscopic 3d Susceptibility," International Journal of Human–Computer Interaction,” Vol.29, No.12, pp.789-797, 2013, (accessed Jul. 8, 2022). doi: https://doi.org/10.1080/10447318.2013.765289
[22] S. Mun, E.-S. Kim, and M.-C. Park, “Effect of mental fatigue caused by mobile 3D viewing on selective attention: an ERP study," International Journal of Psychophysiology,” Vol.94, No.3, pp.373-381, 2014, (accessed Jul. 8, 2022). doi: https://doi.org/10.1016/j.ijpsycho.2014.08.1389
[23] S. Mun and M.-C. Park, “Affective Three-Dimensional Brain–Computer Interface Created Using a Prism Array-Based Display,” Optical Engineering, Vol.53, No.12, pp.123105, 2014, (accessed Jul. 8, 2022). doi: https://doi.org/10.1117/1.OE.53.12.123105
[24] S. Mun, M.-C. Park, and S. Yano, “Evaluation of Viewing Experiences Induced by a Curved Three-Dimensional Display,” Optical Engineering, Vol.54, No.10, pp.103104, 2015, (accessed Jul. 8, 2022). doi: https://doi.org/10.1117/1.OE.54.10.103104
[25] S. Mun, et al., “Effects of Mental Workload on Involuntary Attention: A Somatosensory ERP Study,” Neuropsychologia, Vol.106, pp.7-20, 2017, (accessed Jul. 8, 2022). doi: https://doi.org/10.1016/j.neuropsychologia.2017.08.021
[26] S. Park, L. Kim, J. Ha, and S. Mun, “Infrared Webcam-Based Non-Contact Measurement of Event-Related Potentials from Event-Related Pupillary Responses: An Approach Focused on Mental Workload,” Journal of Computational Design and Engineering, Vol.9, No. 4, pp.1327-1347, 2022, (accessed Jul. 8, 2022). doi: https://doi.org/10.1093/jcde/qwac059

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved