Extended Reality (XR) entered mainstream awareness when applications like Pokémon Go and Geocaching brought digital information into the physical world. These playful experiences made augmented reality familiar to millions - yet only few people realize how transformative XR can be far beyond entertainment. In industrial environments, operators of off‑highway machinery work in low‑visibility and high‑risk conditions where clear information is crucial. XR expands their situational awareness by transforming sensor data into intuitive visual guidance directly within their field of view. As Europe strengthens its sovereign capabilities in IIoT, edge computing, AI and XR, Horizon Europe initiatives like THEIAXR help translate these technologies from early consumer fascination into practical, high‑value applications for industry.
THEIAXR: Bringing XR into off-highway applications
THEIAXR is a Horizon Europe research and innovation project that started in January 2023 and ran for three years. Led by TTControl, the project brought together a consortium from six European countries including Prinoth, Kalmar, VTT Technical Research Centre of Finland Ltd, TU Graz, TU Dresden, Stuttgart Media University, the University of Luxembourg, Haption, Creanex Oy and TTTECH to explore how Extended Reality can improve human–machine interaction in mobile machinery. THEIAXR is one of the first Horizon Europe initiatives to examine how XR can be applied in off‑highway machinery, addressing harsh working conditions, unreliable connectivity and the demanding cognitive load that operators face every day.
A defining element of the project was its strong human-centered and transdisciplinary co-design approach. More than 500 end users contributed throughout the entire development process - from requirements definition to prototype evaluation and final validation. Their operational expertise, safety expectations, and daily routines shaped the design in ways that conventional engineering workflows could not achieve.
This methodology established a key guiding principle for THEIAXR: XR technologies must enhance situational perception without introducing additional cognitive or physical load for operators. This requirement anchored the development process across all work packages.
One early insight illustrates the relevance of this approach. Many established XR solutions rely on head-mounted displays or windshield projections. In off-highway environments, operators rejected both options. Wearing a headset for extended shifts was considered impractical, and windshield projections proved unsuitable because windshields are frequently opened for ventilation during operation. These findings required the project teams to redefine how machine states, sensor signals and environmental cues should be translated into the operator’s field of view-without restricting comfort, performance or workflow.
The resulting interaction concepts extend the operator’s natural perception by merging sensor data, camera inputs, and machine intelligence into clear, context‑aware visual guidance. By grounding the development consistently in real user experience, THEIAXR ensures that XR technologies support safety, reliability and acceptance in human–machine interaction across off‑highway applications.
From concept to field use: XR demonstrators and applications
As project coordinator, TTControl - a joint venture of TTTECH and HYDAC - oversaw the development, integration and validation of the XR concepts within THEIAXR. Leveraging its expertise in human-machine interaction, embedded computing and operator assistance technologies, the company ensured that these concepts were transferred effectively into real testing environments.
To support this work, TTControl developed the visualization and interaction system infrastructure designed to host and integrate XR technologies directly within mobile machinery cabins. Working closely with TTTECH, the team ensured that all XR functions were seamlessly implemented in the project demonstrators and fully operational under real-world conditions.
For the snow grooming demonstrator, TTControl additionally contributed its FusionAI computing platform, which brought an edge-computing architecture into the project. Here, FusionAI serves as the brain of vehicle architecture, linking data acquisition, processing and output pathways.
It offers a multi-screen, multi-interface platform, on which all perception and processing tasks can operate entirely at the edge, even when connectivity is limited.
This edge-based approach was a critical enabler for the snow-grooming demonstrator. By integrating an embedded edge computing platform, machines can operate independently of external computing resources, keeping all data on the machine and reducing or eliminating transfers to cloud or Internet services. This significantly enhances safety and security, while offering a scalable and versatile solution suitable for domains ranging from agriculture to construction and municipal operations.
Key benefits include:
- Higher performance, with low latency on machine processing ensuring responsive guidance in operational environments.
- Cost efficiency, as multi modal XR capabilities can run on a single platform rather than several distributed systems.
- Future readiness, enabled by a software-defined framework that supports ongoing feature development and automation roadmaps.
In combination, these elements provided the technical foundation for deploying XR technologies in demanding off‑highway operating environments, forming the basis for the project’s demonstrators.
Snow-grooming Demonstrator: Strengthening operator awareness in harsh winter environments.
In snow‑grooming, operation take place in some of the most demanding off‑highway environments, shaped by limited visibility, frequent whiteouts, and constantly changing terrain. Within this context, a projection concept rendered directional cues, warnings, and speed information directly onto the snow surface ahead of the vehicle - positioned via GPS‑assisted localization and a Unity‑based environmental model. For the validation phase, TTControl collaborated with Prinoth, a long‑standing customer and member of the Autonomous Operation Cluster (AOC). Prinoth’s snow grooming vehicles provided the real-world platform needed to test how XR concepts can be embedded into off‑highway machinery and aligned with authentic operator workflows. This cooperation reflects TTControl’s close engagement with industry partners within the AOC, where application‑oriented technologies for assisted and semi‑autonomous functions are explored jointly.
Two Additional XR Demonstrators: Exploring broader operational scenarios
Alongside this FusionAI based demonstrator, THEIAXR also implemented two additional XR use cases, each focused on different operational environments and user needs.
- Supporting remote reachstacker operation: In port and terminal environments, reachstacker operations must be carried out with high precision in tight and constantly changing environments. XR clarified distances, spatial relations, and potential obstacles for remote operators relying on multiple camera feeds – maintaining awareness, supporting precision moves and helping reduce collision risk in tight layouts.
- Making digital design data clearer on construction sites: Excavator operators must align digital terrain models with real-world conditions while managing blind spots and constantly shifting work areas.
Presenting essential reference geometry and height information within the natural field of view helped operators to understand the working area more quickly and react to people or obstacles around the machine, improving finish grading and in‑field design tasks.
TTTECH Group: research translated into industrial relevance
The THEIAXR project shows that Extended Reality can strengthen operator awareness when information is delivered clearly, intuitively, and in line with real-world workflows. Several results are already informing industrial product development and academic research, supporting wider adoption of XR concepts beyond the project context.
By integrating these concepts directly into machine environments, THEIAXR illustrates how XR can support safe and confident operation across demanding off‑highway settings. This perspective reflects the TTTECH Group’s commitment to advancing dependable technologies for industrial edge computing, intelligent assistance systems and human‑machine interaction — enabling solutions that make complex machinery more intuitive and safer to operate.
Stay informed about the future of XR, industrial edge technologies, and safe autonomous operations:
- Explore TTTECH CEO Talks and technology updates:
- Automation to autonomy: From vision to execution (Guest: Martin Pottendorfer, TTControl)
- Advancements in AgTech, smart farming trends, and precision agriculture (Guest: Christiana Seethaler, TTControl)
- Software-defined solutions: Openness, security and real-time performance (Guest: Christoph Trappl, TTTECH Digital Solutions)
- Discover TTControl’s expertise in HMI and safety critical systems
- Read the related press release: Extended Reality improves human–machine interaction
- THEIAXR: end of project video
- THEIAXR website
