Imagine if your VR headset could actually understand what you're saying and see what you're looking at at the same time. Instead of clicking buttons, you just talk to the digital world and it reacts naturally, like a real person would. It's like giving a brain to virtual spaces so they can translate languages and trigger visual effects based on your voice.
Current XR experiences often rely on clunky menus or limited voice commands that don't understand the visual context. This creates a friction-filled user experience that slows down the adoption of AR and VR in professional and entertainment sectors.
Pretrained XR models combining language and vision AI, and two specific conversation agents for VR conferences and training assistants.
If you are a virtual event organizer dealing with language barriers in global meetings — this project developed multimodal AI models that enable virtual conferences with automated translation and virtual agents. This allows attendees from anywhere in the world to participate in a shared virtual environment more naturally.
If you are a theater production house dealing with static stage effects — this project developed AR VFX triggered by predetermined speech. This allows for an augmented theater experience where visual effects are synchronized with the actors' voices in real-time.
If you are a training provider dealing with inefficient manual guidance in VR — this project developed a Training Assistant virtual agent. This provides a more natural human-to-machine interaction to support users in their daily tasks.
Based on available project data, specific commercial pricing is not mentioned, although the project received an EU contribution of EUR 4,786,875 for development.
The project is designed for industrial application through the development of pretrained next-generation XR models and validation across three distinct use cases.
Based on available project data, specific licensing terms are not provided, but the project results include a set of pretrained models and applications.
The project focuses on the convergence of NLP and CV to drive both AR and VR systems, suggesting a focus on software-level integration for XR back-ends.
The project period runs from 2022-10-01 to 2025-12-31, indicating that final results and validated applications will be available by the end of 2025.
The consortium is heavily industry-weighted with 50% industry participation (5 companies), including 4 SMEs. This balance, combined with 1 university and 2 research centers across 5 countries, suggests a strong focus on commercial viability and practical application rather than purely theoretical research.
Contact MAGGIOLI SPA in Italy for licensing and partnership inquiries.
Contact us to find the right partner for integrating these multimodal XR models into your product line.
