Multi-sensory teaching method: Arithmetic and geometry can be studied through music, dance

Reviewed by James Ives, M.Psych. (Editor)Feb 15 2019

Arithmetic and geometry can be studied through music and dance: this is the new multi-sensory teaching method identified by the European project WeDraw, coordinated by IIT-Istituto Italiano di Tecnologia. The new teaching method gave birth to four serious games, where fractions, angles and 3d trasformations are associated with sounds, vibrations and body movements. More than 100 children tested the new technologies, showing that the use of multisensory serious games improves mathematical learning in children at elementary school. A video can be watched on WeDraw YouTube channel.

Teachers usually considered the sight as the prevalent sensory channel to transmit mathematical concepts, leaving out hearing and touch, seen as marginal. Monica Gori's group at IIT conducted psychophysical and developmental psychology studies which showed a different sensorial situation: before 8-10 years of age children are not showing integration between sensory modalities using a preferred sensory channel to perceive and to learn specific concepts, for example vision for space, audition for time and touch for object size. The WeDraw project started from such scientific results, aiming at creating and evaluating a novel technology based on multiple sensory modalities.

The project WeDraw started in 2017, selected as first in the European Commission H2020 ICT "Technologies for Learning and skills" call, out of 92 submitted proposals. It lasted 2 years and involved more than 200 teachers within Italy, UK and Ireland to identify the most suitable sensory modalities to teach specific mathematical concepts in classrooms.

The WeDraw serious games are: "RobotAngle", which concerns the concept of angles and their operations and it will be soon available for schools on a dedicated website; "Cartesian garden" is about the cartesian plane; "Spaceshape" is dedicated to 2D and 3D objects and their rotation in a virtual area; a fourth one is dedicated to fractions.

"WeDraw gave us the possibility to define a novel embodied and enactive pedagogical approach. weDRAW technologies contribute to innovative pedagogical approaches, enabling teachers to design activities which will be tailored to the specific needs of their students, even when they are visually impaired" comments Monica Gori, coordinator of the WeDraw project and team leader of U-Vip Lab at IIT, in Genova (Italy)

WeDraw involved teachers in a participatory design process to iteratively inform the development of the technology and associated pedagogical activities. The identified mathematical concepts were: fractions, Cartesian plane, angle understanding, 2D-3D transformations.

Then researchers worked on the development of the WeDraw technologies, focusing on the realization of libraries of software modules for the analysis of nonverbal motoric and affective behavior of children and for real-time control of auditory, haptic, and visual feedback. They also worked to integrate hardware and software platform supporting multiple inputs and output devices, which are scalable to different learning environments. Then they designed the different serious games.

The technological solutions developed in WeDraw were validated at psychophysical and pedagogical level, by developing specific tests with primary school children and also with visually impaired children, in order to evaluate their motor, arithmetic, and geometrical skills. Studies also included pre and post-tests around the specific mathematical constructs. Both typically developing and impaired children (from 6 to 10 years of age) were involved in the evaluation. Results showed, for example, that the Spaceshape game improves the understanding of fractions and transformations from 2D to 3D in 7-year-old children.

The new WeDraw technologies are expected to have a direct impact on society, especially for enhancing the teaching and learning processes, for increasing educational possibilities for pupils with special (dis)abilities, and for increasing the capacity to retain pupils in the areas threatened by lack of specialized schools.

In particular, some serious games will be available for schools thanks to the collaboration of IIT and project partners: the Cartesian Garden will be commercialized by the SME Ignition Factory, while a WeDraw app including the RobotAngle game will be freely available in Italy on the website of the publishing house De Agostini Editore.