SciTransfer
TELMI · Project

AI-Powered Music Learning Platform Using Motion Sensors and Video Feedback

digitalTestedTRL 5

Imagine learning violin from a world-class teacher, but instead of meeting them once a week, you have a smart system that watches your posture, listens to your playing, and tracks your bow movements — giving you real-time tips just like having a teacher looking over your shoulder. TELMI built exactly that: a multi-sensor platform combining cameras, motion trackers, and audio analysis that captures how master violinists play and then coaches students to improve on their own. They also created a public database of expert performances that anyone can study, like a YouTube for violin technique but with full 3D motion data.

By the numbers
EUR 2,617,425
EU funding for development
5
consortium partners
3
countries involved (ES, IT, UK)
2
industry partners including SMEs
13
total project deliverables
40%
industry partner ratio in consortium
The business problem

What needed solving

Music education still relies heavily on infrequent one-on-one lessons followed by long unsupervised practice sessions, leading to high student dropout. Students practicing alone have no way to get objective feedback on their technique, posture, or bowing — they essentially practice blind until the next lesson. Current digital tools rarely go beyond simple audio and video recording, missing the rich movement data that teachers observe in person.

The solution

What was built

TELMI built a complete multi-modal music learning platform with sensor hardware (EMF sensors, IR cameras, audio capture), a system platform iterated through 3 versions (first prototype, improved second version, final platform), and a public multimodal database with an HTML5 web visualizer for browsing and downloading expert performance recordings.

Audience

Who needs this

Online music education platforms (e.g., apps teaching instruments remotely)Music conservatories and academies looking to supplement traditional teachingMusical instrument companies building smart practice accessoriesSports and performing arts training companies needing motion analysis feedbackPhysical rehabilitation clinics using movement analysis for patient coaching
Business applications

Who can put this to work

EdTech / Online Music Education
SME
Target: Online music learning platforms and apps

If you are an EdTech company offering music lessons online and struggling with high student dropout rates — TELMI developed a multi-modal learning platform combining audio, video, and motion analysis that provides real-time augmented feedback to students practicing alone. The system was built with 5 partners across 3 countries, iterated through multiple prototype versions, and includes a public multimodal database of expert performances for benchmarking student progress.

Musical Instrument Manufacturing
mid-size
Target: Instrument makers adding smart practice tools to their product lines

If you are a musical instrument manufacturer looking to add digital learning tools to differentiate your products — TELMI created data acquisition hardware using EMF sensors, IR cameras, and audio capture that can be integrated into practice environments. The final prototypes were fully integrated with database, user interface, and feedback systems, ready for product adaptation beyond the original violin use case.

Performing Arts Education
any
Target: Music conservatories and performing arts schools

If you are a conservatory or music school looking to improve student retention and scale your teaching capacity — TELMI built a complete system platform covering student-teacher, student-only, and collaborative learning scenarios. The platform was tested across 3 countries with pedagogical evaluation, and includes a publicly available reference database of multimodal recordings that institutions can use for curriculum integration.

Frequently asked

Quick answers

What would it cost to license or adopt this technology?

The project had a total EU contribution of EUR 2,617,425 shared across 5 partners over 3 years. Licensing terms would need to be negotiated with the coordinator (Universidad Pompeu Fabra). The public database component is freely available, but the integrated platform and sensor systems would likely require a commercial agreement.

Can this scale beyond violin to other instruments or disciplines?

The project used violin as a case study, but the underlying multi-modal technology (audio, video, motion capture, EMF sensors, IR cameras) is instrument-agnostic. The platform architecture supports student-teacher, student-only, and collaborative learning scenarios, making it adaptable to other performance-based disciplines like dance, sports training, or physical rehabilitation.

What is the IP situation — who owns the technology?

The consortium includes 2 industry partners and 3 universities across Spain, Italy, and the UK. IP ownership would follow the Horizon 2020 grant agreement, typically shared among partners based on contribution. Contact the coordinator at Universidad Pompeu Fabra to clarify licensing availability for specific components.

How mature is the technology — is it ready to deploy?

The project delivered multiple prototype iterations: first working prototypes, second improved versions based on user feedback, and final platforms of data acquisition systems fully integrated with the database and feedback systems. A final version of the TELMI System Platform was delivered, suggesting the technology reached demonstration level.

What data and infrastructure does this require?

The system uses EMF sensors, IR cameras, and audio capture for data acquisition. A public multimodal database was created with an HTML5 web-based visualizer for browsing recordings. The platform requires the sensor hardware plus server infrastructure for the database and feedback engine.

Has this been validated with real users?

Based on available project data, the second version of prototypes was explicitly improved based on user feedback about usability and practical concerns during recordings. The project included pedagogical evaluation of effectiveness, and the consortium included both technical and pedagogical partners working in tight collaboration.

Are there regulatory concerns for use in educational settings?

Based on available project data, the system captures multimodal recordings (audio, video, motion) of students, which would require compliance with GDPR and educational data protection regulations. The public database component was designed for research use, so commercial deployment would need additional privacy and consent measures.

Consortium

Who built it

The TELMI consortium is a compact group of 5 partners across Spain, Italy, and the UK, with a healthy 40% industry ratio (2 industry partners, both SMEs) alongside 3 universities. This mix signals genuine market interest — the SME involvement means commercial players saw enough potential to commit resources. The coordinator, Universidad Pompeu Fabra in Barcelona, is a well-known research university with strong music technology expertise. For a business looking to adopt or license this technology, the small consortium size means fewer parties at the negotiation table, and the SME partners likely have practical insights into commercialization pathways.

How to reach the team

Universidad Pompeu Fabra (Barcelona, Spain) — reach the music technology research group through their institutional website or the TELMI project page.

Next steps

Talk to the team behind this work.

Want an introduction to the TELMI team to discuss licensing or integration? SciTransfer can arrange a direct connection with the coordinator and relevant consortium partners.