Biblio

Recent scientific results show that audio feedback associated with body movements can be fundamental during the development to learn new spatial concepts [1], [2]. Within the weDraw project [3], [4], we have investigated how this link can be useful to learn mathematical concepts. Here we present a study investigating how mathematical skills changes after multisensory training based on human-computer interaction (RobotAngle and BodyFraction activities). We show that embodied angle and fractions exploration associated with audio and visual feedback can be used in typical children to improve cognition of spatial mathematical concepts. We finally present the exploitation of our results: an online, optimized version of one of the tested activity to be used at school. The training result suggests that audio and visual feedback associated with body movements is informative for spatial learning and reinforces the idea that spatial representation development is based on sensory-motor interactions.

This paper describes a system for embodied conversational agents developed by Inmerssion and one of the applications—Young Merlin: Trial by Fire —built with this system. In the Merlin application, the ECA and a human interact with speech in virtual reality. The goal of this application is to provide engaging VR experiences that build rapport through storytelling and verbal interactions. The agent is fully automated, and his attitude towards the user changes over time depending on the interaction. The conversational system was built through a declarative approach that supports animations, markup language, and gesture recognition. Future versions of Merlin will implement multi-character dialogs, additional actions, and extended interaction time.