Emotion Detection and Student Engagement in Distance Learning During Containment Due to the COVID-19
DOI:
https://doi.org/10.21123/bsj.2023.8698Keywords:
Convolutional Neural Network (CNN); COVID-19; Distance learning; Emotion detection; E-learning; Student engagementAbstract
Distance learning is one of the teaching and learning approaches adopted after the COVID-19 pandemic. The task of getting learners interested in class is difficult for the professors. In this research, a mechanism has been developed to estimate student engagement levels and emotions. Visual data from recorded videos of students participating in learning courses are utilized due to the availability of multiple methods for measuring student engagement levels. The data from the videos recorded and sent by students is processed to determine the extent of student engagement and identify their emotions. The system has been implemented and tested, enabling the evaluation of student attention. Several algorithms and techniques have been used to implement our prototype as CNN. A private dataset has been created to train and evaluate the model. The results show that it is possible to measure participation, learn about feelings, and use them to make decisions in favor of student outcomes and improve teaching and learning methods. This technology can be applied in other scenes, such as self-driving and security, with a minor adjustment.
Received 10/03/2023
Revised 22/07/2023
Accepted 24/07/2023
Published Online First 20/09/2023
References
Berry LA, Kowal KB. Effect of Role-Play in Online Discussions on Student Engagement and Critical Thinking. Online Learn J 2022; 26(3): 4-21; https://doi.org/10.24059/olj.v26i3.3367.
Elshami W, Taha MH, Abdalla ME, Abuzaid M, Saravanan C, Al Kawas S. Factors that affect student engagement in online learning in health professions education. Nurse Educ Today. 2022 Mar 1; 110: 105261; https://doi.org/10.1016/j.nedt.2021.105261.
Al-Sakkaf A, Omar M, Ahmad M. Social Worked-Examples Technique to Enhance Student Engagement in Program Visualization. Baghdad Sci J . 2019; 16(2): 0453; https://doi.org/10.21123/bsj.2019.16.2(SI).0453.
Alruwais N, Zakariah M. Student-Engagement Detection in Classroom Using Machine Learning Algorithm. Electronics .2023; 12(3): 731; https://doi.org/10.3390/electronics12030731.
Ali MM, Hassan N. Defining concepts of student engagement and factors contributing to their engagement in schools. Creat Educ. 2018 Oct 25; 9(14): 2161-70; https://doi.org/10.4236/ce.2018.914157.
Astin AW. Student involvement: A developmental theory for higher education. J Coll Stud Pers. 1984 Jul 25; 25(4): 297-308; https://psycnet.apa.org/record/1999-01418-006.
Chen Y, Zhou J, Gao Q, Gao J, Zhang W. MDNN: Predicting Student Engagement via Gaze Direction and Facial Expression in Collaborative Learning. CMES – Comput Model Eng Sci. 2023 Jul 1; 136(1): 381-401. https://doi.org/10.32604/cmes.2023.023234.
Maluenda-Albornoz J, Berríos-Riquelme J, Infante-Villagrán V, Lobos-Peña K. Perceived Social Support and Engagement in First-Year Students: The Mediating Role of Belonging during COVID-19. Sustainability. 2022 Dec 29; 15(1): 597; https://doi.org/10.3390/su15010597.
Tomaszewski W, Xiang N, Huang Y, Western M, McCourt B, McCarthy I. The Impact of Effective Teaching Practices on Academic Achievement When Mediated by Student Engagement: Evidence from Australian High Schools. Educ Sci. 2022 May 20; 12(5): 358; https://doi.org/10.3390/educsci12050358.
Ladino Nocua AC, Cruz Gonzalez JP, Castiblanco Jimenez IA, Gomez Acevedo JS, Marcolin F, Vezzetti E. Assessment of Cognitive Student Engagement Using Heart Rate Data in Distance Learning during COVID-19. Educ Sci. 2021 Sep 14; 11(9): 540; https://doi.org/10.3390/educsci11090540.
Gallinat A. Student engagement in the management of accelerated change. Learn Teach. 2018 Mar 1; 11(1): 35–56; https://doi.org/10.3167/latiss.2018.110103.
Coates H. A model of online and general campus‐based student engagement. Assess Eval High Educ. 2007 Apr 1; 32(2): 121–41; https://doi.org/10.1080/02602930600801878.
Altuwairqi K, Jarraya SK, Allinjawi A, Hammami M. A new emotion–based affective model to detect student's engagement. J King Saud Univ Comput. 2021 Jan 1; 33(1): 99-109; https://doi.org/10.1016/j.jksuci.2018.12.008.
Schlechty PC. Engaging Students: The Next Level of Working on the Work. John Wiley & Sons; 2011. 224 p. 2011 Feb 16; https://www.wiley.com/en-gb/Engaging+Students%3A+The+Next+Level+of+Working+on+the+Work-p-9781118015520.
D'Mello S, Graesser A. Dynamics of affective states during complex learning. Learn Instr. 2012 Apr 1; 22(2): 145–57; https://doi.org/10.1016/j.learninstruc.2011.10.001.
Whitehill J, Serpell Z, Lin YC, Foster A, Movellan JR. The faces of engagement: Automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect Comput. .2014 Apr 10; 5(1): 86-98; https://doi.org/10.1109/TAFFC.2014.2316163.
Jang M, Park C, Yang HS, Kim JH, Cho YJ, Lee DW, et al. Building an automated engagement recognizer based on video analysis. ACM/IEEE Proc Int Conf Hum Robot Interact. 2014 Mar 3 (pp. 182-183). Bielefeld Germany. https://doi.org/10.1145/2559636.2563687
Li J, Ngai G, Leong HV, Chan SC. Multimodal human attention detection for reading from facial expression, eye gaze, and mouse dynamics. ACM SIGAPP Appl Comput Rev. 2016 Nov 4; 16(3): 37-49; https://doi.org/10.1145/3015297.3015301.
Monkaresi H, Bosch N, Calvo RA, D'Mello SK. Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans Affect Comput. 2016 Jan 6; 8(1): 15-28; https://doi.org/10.1109/TAFFC.2016.2515084.
Abdellaoui B, Moumen A, El Idrissi YE, Remaida A. Face detection to recognize students' emotion and their engagement: A systematic review. ICECOCS. 2020 Dec 2 (pp. 1-6). IEEE; https://doi.org/10.1109/ICECOCS50124.2020.9314600.
Kumari J, Rajesh R, Pooja KM. Facial expression recognition: A survey. Procedia Comput Sci. 2015 Jan 1; 58: 486-91; https://doi.org/10.1016/j.procs.2015.08.011
Omar MR, Husni H. Teen-Computer Interaction: Building a Conceptual Model with Thoughts-Emotion-Behaviour. Baghdad Sci J. 2019 Jun 20;16(2(SI): 0485; https://doi.org/10.21123/bsj.2019.16.2(SI).0485.
Mellouk W, Handouzi W. Facial emotion recognition using deep learning: review and insights. Procedia Comput Sci. 2020 Jan 1; 175: 689-94 ; https://doi.org/10.1016/j.procs.2020.07.101.
Villar BF, Viñas PF, Turiel JP, Marinero JC, Gordaliza A. Influence on the user's emotional state of the graphic complexity level in virtual therapies based on a robot-assisted neuro-rehabilitation platform. Comput Methods Programs Biomed. 2020 Jul 1; 190: 105359; https://doi.org/10.1016/j.cmpb.2020.105359.
Abdurrasyid A, Indrianto I, Susanti MN. Face detection and global positioning system on a walking aid for blind people. Bull Electr Eng Inform. 2022 Jun 1; 11(3): 1558-67; https://doi.org/10.11591/eei.v11i3.3429.
Henrie CR, Halverson LR, Graham CR. Measuring student engagement in technology-mediated learning: A review. Comput Educ. 2015 Dec 1; 90: 36-53; https://doi.org/10.1016/j.compedu.2015.09.005.
Karimah SN, Hasegawa S. Automatic engagement estimation in smart education/learning settings: a systematic review of engagement definitions, datasets, and methods. Smart Learn Environ. 2022 Dec; 9(1): 1-48; https://doi.org/10.1186/s40561-022-00212-y.
Shao M, Pham-Hung M, Alves SFDR, Snyder M, Eshaghi K, Benhabib B, et al. Long-Term Exercise Assistance: Group and One-on-One Interactions between a Social Robot and Seniors. Robotics. 2023 Jan 6; 12(1): 9; https://doi.org/10.3390/robotics12010009.
Thomas C, Sarma KP, Gajula SS, Jayagopi DB. Automatic prediction of presentation style and student engagement from videos. Comput Educ Artif Intell. 2022 Jan 1; 3: 100079; https://doi.org/10.1016/j.caeai.2022.100079.
Remaida A, Moumen A, El Idrissi YE, Abdellaoui B, Harraki Y. The use of personality tests as a pre-employment tool: A comparative study. InSHS Web of Conferences. SHS Web Conf. 2021; 119: 05007; https://doi.org/10.1051/shsconf/202111905007.
Sabri Z, FAKHRI Y, MOUMEN A. The Effects of Gamification on E-learning Education: Systematic Literature Review and Conceptual Model. Statistics. Optim Inf Comput. 2022 Feb 8; 10(1): 75-92; https://doi.org/10.19139/soic-2310-5070-1115.
Al-Fallooji AS, Al-Azawei A. Predicting Users' Personality on Social Media: A Comparative Study of Different Machine Learning Techniques. Karbala Int J Mod Sci. 2022; 8(4): 617-30; https://doi.org/10.33640/2405-609X.3262
Al-Shareeda MA, Manickam S. COVID-19 vehicle based on an efficient mutual authentication scheme for 5G-enabled vehicular fog computing. Int J Environ Res Public Health. 2022 Nov 24; 19(23): 15618; http://dx.doi.org/10.3390/ijerph192315618
Mejjad N, Cherif EK, Rodero A, Krawczyk DA, El Kharraz J, Moumen A, et al. Disposal Behavior of Used Masks during the COVID-19 Pandemic in the Moroccan Community: Potential Environmental Impact. Int J Environ Res Public Health. 2021 Apr 20;18(8):4382; https://doi.org/10.3390/ijerph18084382
Gloster AT, Lamnisos D, Lubenko J, Presti G, Squatrito V, Constantinou M, et al. Impact of COVID-19 pandemic on mental health: An international study. PloS one. 2020 Dec 31; 15(12): e0244809; https://doi.org/10.1371/journal.pone.0244809
Musolino S. Families, Relational Scenarios and Emotions in the Time of the COVID-19 Pandemic. Ital Sociol Rev. 2020 Dec 6; 10(3S): 737; https://doi.org/10.13136/isr.v10i3S.396
Matthewman S, Huppatz K. A sociology of Covid-19. J Sociol. 2020 Dec; 56(4): 675-83; https://doi.org/10.1177/1440783320939416
Mohammed BA, Al-Shareeda MA, Manickam S, Al-Mekhlafi ZG, Alreshidi A, Alazmi M, et al. FC-PA: Fog Computing-based Pseudonym Authentication Scheme in 5G-enabled Vehicular Networks. IEEE Access. 2023 Feb 22; 11: 18571-81. https://doi.org/10.1109/ACCESS.2023.3247222
Al-Mekhlafi ZG, Al-Shareeda MA, Manickam S, Mohammed BA, Qtaish A. Lattice-based lightweight quantum resistant scheme in 5G-enabled vehicular networks. Math. . 2023 Jan 12; 11(2): 399; https://dx.doi.org/10.3390/math11020399
Al-Shareeda MA, Manickam S. Man-in-the-middle attacks in mobile ad hoc networks (MANETs): Analysis and evaluation. Sym. 2022 Jul 27; 14(8): 1543; https://doi.org/10.3390/sym14081543
Singh AK, Srivastava S. Development and Validation of Student Engagement Scale in the Indian Context. Global Bus Rev 2014; 15(3): 505-515; https://doi.org/10.1177/0972150914535137
Gupta S, Kumar P, Tekchandani R. A multimodal facial cues based engagement detection system in e-learning context using deep learning approach. Multimed Tools Appl. 2023 Feb 10: 1-27; https://doi.org/10.1007/s11042-023-14392-3
Awais M, Raza M, Singh N, Bashir K, Manzoor U, Islam SU, et al. LSTM-based emotion detection using physiological signals: IoT framework for healthcare and distance learning in COVID-19. IEEE Internet Things J. 2020 Dec 10; 8(23): 16863-71; https://doi.org/10.1109/JIOT.2020.3044031
Thiruthuvanathan M, Krishnan B, Rangaswamy M. Engagement detection through facial emotional recognition using a shallow residual convolutional neural networks. Int J Intell Eng Syst. 2021 Mar 1; 14(2): 236-47; https://doi.org/10.22266/ijies2021.0430.21
Rodriguez P, Ortigosa A, Carro RM. Detecting and making use of emotions to enhance student motivation in e-learning environments. Int J Contin Eng Educ Life Long Learn. 2014 Jan 1; 24(2): 168-83; https://doi.org/10.1504/IJCEELL.2014.060156
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Baghdad Science Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
