Imagine wearing VR goggles that don't just follow a script, but actually see and hear what you're doing and talk back to you like a real coach. It's like having a smart assistant that understands the room and changes its instructions on the fly based on your mistakes or needs. This technology lets virtual characters act and react naturally to help people learn complex tasks or recover from injuries.
Current XR training and therapy tools rely on rigid scripts that cannot adapt to a user's unique mistakes or unplanned questions. This leads to unnatural experiences and limited effectiveness in complex learning or medical environments.
A multimodal XR platform featuring AI chatbots for aphasia and metacognition, a voice-integrated VR fire safety trainer, and an automated IFC-to-Unreal Engine 5 pipeline for BIM reviews.
If you are a clinic dealing with stroke patients with language impairments — this project developed aphasia and metacognition agents that provide personalized guidance. This allows for automated analysis of rehabilitation sessions to improve patient recovery.
If you are a training company dealing with rigid, scripted safety drills — this project developed a VR fire-extinguisher prototype with open-ended dialogues. This ensures trainees are guided by a context-aware AI rather than a fixed sequence of events.
If you are a design firm dealing with complex 3D model reviews — this project developed a system that loads IFC geometry into Unreal Engine 5 with LLM-extracted metadata. This enables real-time, language-driven design reviews of architectural assets.
Based on available project data, there is no information regarding the pricing or cost of the developed system.
The project aims to deliver scalable, personalised XR and set global standards for adaptive XR, though specific scaling metrics are not provided.
Based on available project data, specific IP and licensing terms are not disclosed, though it emphasizes EU compliance and an ethical framework.
The system integrates with Unity and Unreal Engine 5, and uses a TCP/IP-based Python API for data exchange.
The project period runs from 2024-01-01 to 2026-12-31.
The consortium is well-balanced for commercialization, featuring 12 partners across 8 countries. With 5 SMEs and an industry ratio of 42%, there is a strong bridge between the 4 universities and 3 research institutes and the actual market application.
Contact DFKI GmbH in Germany
Contact us to connect with the LUMINOUS consortium for pilot integration.
