Friday 4 September Daniele Di Mitri defended his dissertation on “Multimodal Tutor: Adaptive Feedback from Multimodal Experiences” to an enthusiastic audience.
The dissertation describes the ideation, prototyping and empirical testing of the Multimodal Tutor. The Multimodal Tutor supports the learning of practical skills and makes use of sensor data and neural networks. The system is not intended to replace humans, but it can temporarily substitute or provide support. Di Mitri's research was made possible in part by the Interreg Euregion Meuse-Rhine project SafePAT and contributes to improving the interaction and collaboration between humans and artificial intelligence. In this case it is about recognizing and understanding human actions and learning to reduce behavioral and social signals.
In one of his empirical investigations, Daniele focuses on the use of the Multimodal Tutor during CPR (Cardiopulmonary Resuscitation). In this form of resuscitation, both breathing and circulation must be restarted. The medical simulation procedure requires precise levels of body coordination and extensive practical experience. The Multimodal Tutor recognizes training errors such as incorrect posture or a locking of the arms and provides feedback.
Seeking balance in generalisation and personalisation
One of the finest pieces of Daniele’s work with his promotor, Prof dr. Marcus Specht (CEL’s scientific director and leader of 4TU.CEE) was, “From signals to knowledge” where he conceptualised a MLeAM (Multimodal Learning Analytics Model) for the trending research field of MMLA. MLeAM advocates the optimal use of multimodal data to support learning activities through intelligent tutoring and learning analytics. In his defense, Di Mitri foregrounded the big-5 Cs (challenges) of MMLA: two of these big 5 Cs are data annotation and data exploitation. In his defense, he accentuates the importance of seeking a balance in generalisation and personalisation to overcome bias in model training. This is particularly true in the light of CEL’s MMLA project (a collaboration with the Learning Sciences Lab, National Institute of Education, Singapore) where we sought to harness AI-driven learning analytics to provide real time feedback to inform teaching and learning practices.
Read more
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted Learning, 34(4), 338-349.
Daniele Di Mitri’s defense is also available online.