Micro-expression detection in ASD movies: a YOLOv8-SMART approach

Yutong Gu; Hanni Li; Jiarong Liu; Chenxi Liu; Yuxuan Li; Chen Li; Ning Xu

doi:10.55092/bi20250002

Technique Report

Open Access

Expand

Micro-expression detection in ASD movies: a YOLOv8-SMART approach

download PDF

Yutong Gu¹, Hanni Li¹, Jiarong Liu¹, Chenxi Liu¹, Yuxuan Li¹, Chen Li¹^,∗, Ning Xu²^,∗

¹ School of Biomedical Engineering, Northeastern University, Shenyang, China

² School of Art and Design, Liaoning Petrochemical University, Fushun, China

* lichen@bmie.neu.edu.cn; xuning201096@hotmail.com

Volume
Volume 1 Issue 1, 2025
Citation
Gu Y, Li H, Liu J, Liu C, Li Y, et al. Micro-expression detection in ASD movies: a YOLOv8-SMART approach. Biomed. Inform. 2025(1):0004, https://doi.org/10.55092/bi20250002.
DOI
10.55092/bi20250002
Copyright
Copyright2025 by the authors. Published by ELSP.

Abstract

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder in which individuals often face social difficulties as well as language and communication challenges. Micro-expressions are extremely brief changes in facial expression. Moreover, the micro-expressions exhibited by individuals with ASD frequently represent an accurate reflection of their internal feelings. Therefore, using the Cinemetrics method to extract micro-expressions from ASD patients in movies and targeting them for detection can help doctors make early diagnosis of ASD patients. In this paper, we establish a dataset of micro-expressions of ASD patients in movies, use the improved YOLOv8-SMART algorithm for target detection, and compare it with other target detection algorithms without improvement. The comparison results prove that our algorithm effectively improves the recognition of micro-expressions, which provides reference value for future practical applications in the task of micro-expression recognition in ASD patients.

Keywords

autism spectrum disorder; Cinemetrics; micro-expressions; movies; YOLOv8-SMART; target detection algorithms

Preview

view pdf

References

[1] Geschwind DH. Genetics of autism spectrum disorders. Trends Cognit. Sci. 2011, 15(9):409–416.
[2] Prevalence of autism spectrum disorders-Autism and developmental disabilities monitoring network. MMWR Surveill Summ 2009, 58:1–20.
[3] Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, et al. Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). The Lancet 2006, 368(9531):210–215.
[4]Autism. World Health Organization. 2023. Available: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders (accessed on 5 October 2024).
[5] Eissa N, Al-Houqani M, Sadeq A, Ojha SK, Sasse A, et al. Current enlightenment about etiology and pharmacological treatment of Autism Spectrum Disorder. Front. Neurosci. 2018, 12:304.
[6] Li X, Pfister T, Huang X, Zhao G, Pietikäinen M. A spontaneous micro-expression database: Inducement, collection and baseline. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China, April 22–26, 2013, pp. 1–
[7]Ekman P, Friesen WV. Nonverbal leakage and clues to deception. Psychiatry, 1969, 32(1): 88–106.
[8] Quang NV, Chun J, Tokuyama T. CapsuleNet for micro-expression recognition. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France, May 14-18, 2019, pp. 1–7.
[9] Tsivian Y. Digital Tools in Media Studies; Michael Ross, Manfred Grauer, Eds. Cinemetrics, part of the humanities’ cyberinfrastructure; Bielefeld: Transcript Verlag, 200
[10] Jung JJ, You E, Park SB. Emotion-based character clustering for managing story-based contents: A cinemetric analysis. Multimedia Tools Appl. 2013, 65(1):29–45.
[11] Internet Archive. Movies and Videos. Internet Archive. Available: https://archive.org/details/movies. (accessed on 10 October 2024).
[12] Genovese A, Butler MG. Clinical assessment, genetics, and treatment approaches in Autism Spectrum Disorder (ASD). Int. J. Mol. Sci. 2020, 21(13):4726.
[13] Kanner L. Autistic disturbances of affective contact. Nervous Child 1943, 32:217–253.
[14] Mezzacappa A, Lasica P-A, Gianfagna F, Cazas O, Hardy P, et al. Risk for Autism Spectrum Disorders according to period of prenatal antidepressant exposure: A systematic review and meta-analysis. JAMA Pediatrics 2017, 171(6):555–563.
[15] American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. 5th ed., American Psychiatric Publishing, 2013.
[16] Quang NV, Chun J, Tokuyama T. CapsuleNet for micro-expression recognition. In 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France, May 14–18, 2019, pp. 1–7.
[17] Haggard EA, Isaacs KS. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy. In Methods of Research in Psychotherapy, Boston: Springer US, 1966, pp. 154–165.
[18]Ekman P, Friesen WV. Facial action coding system: A technique for the measurement of facial movement. Palo Alto. 1978(3).
[19] Li X, Pfister T, Huang X, Zhao G, Pietikäinen M. A spontaneous micro-expression database: Inducement, collection and baseline. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China, April 22–26, 2013, pp. 1–6.
[20] Li X, Hong X, Moilanen A, Huang X, Pfister T, et al. Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans. Affective Comput. 2018, 9(4):563–577.
[21] Devangini P, Hong X, Zhao G. Selective deep features for micro-expression recognition. In 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, December 4–8, 2016, pp. 2258–2263.
[22] Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. Advances in neural information processing systems 2017, 30.
[23] Jaiswal A, AbdAlmageed W, Wu Y, Natarajan P. CapsuleGAN: Generative adversarial capsule network. In Proceedings of the European conference on computer vision (ECCV) workshops, Munich, Germany, September 8–14, 2018, pp.1586–1601.
[24] Ultralytics. Ultralytics. Github. 20 Available: https://github.com/ultralytics/ultralytics (accessed on 1 October 2024).
[25]Wu T, Dong Y. YOLO-SE: Improved YOLOv8 for remote sensing object detection and recognition. Applied Sciences. 2023, 13(24):12977.
[26]Ma Siliang, Xu Yong. MPDIoU: A loss for efficient and accurate bounding box regression. arXiv 2023, arXiv: 2307.07662.
[27] Yutong Gu, Hanni Li, Yuxuan Li, Jiarong Liu, Chenxi Liu, et al. A movie role based video dataset for Autism Spectrum Disorder micro-expression detection. In Proceedings of Conference on Machine Vision, Image Processing and Imaging Technology (MVIPIT 2024), Zhangjiakou, China, September 13–15, 2024.
[28]Ekman P, Friesen WV. Facial action coding system. APA PsycTests. 1978.
[29]Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, et al. The Extended Cohn-Kanade Dataset (CK+). Pittsburgh: Carnegie Mellon University, 2010. Available: http://www.pitt.edu/~emotion/ck-spread.htm (accessed on 10 October 2024).
[30]McPheeters ML, Warren Z, Sathe N, Bruzek JL, Krishnaswami S, et al. A systematic review of medical treatments for children with Autism Spectrum Disorders. Pediatrics 2011, 127(5):e1312–e1321.
[31]Samata SR, Gonda X, Tarazi FI. Autism Spectrum Disorder: Classification, diagnosis and therapy. Pharmacol. Ther. 2018, 190:91–104.