LPVR-DUO in an Airborne Helicopter

In-Flight VR

Imagine soaring through the skies as a pilot, testing the limits of a helicopter’s capabilities while feeling the rush of wind and turbulence. Now imagine that you don’t see the real world outside and the safe landing pad that your helicopter is approaching but a virtual reality (VR) scene where you are homing in on a ship in high seas. The National Research Council Canada (NRC) and Defence Research and Development Canada (DRDC) have brought this experience to life with their groundbreaking Integrated Reality In-Flight Simulation (IRIS).

IRIS is not your ordinary simulator; for one, it’s not sitting on a hexapod, it’s airborne. It’s a variable-stability helicopter based on the Bell 412 that can behave like other aircraft and can simulate varying weather conditions; combine that with a VR environment and you have a tool that allows safely training operations in the most adverse conditions. In particular it is used for Ship Helicopter Operating Limitations (SHOL) testing.

Mission-Critical Application with LPVR-DUO

The LPVR-DUO system is what makes VR possible on this constantly moving platform. This cutting-edge AR/VR tracking system seamlessly merges the inertial measurements taken by the headset with the helicopter’s motion data and a camera system mounted inside the cabin to provide the correct visuals to the pilot. The challenges of using cameras to track the VR headset inside the tight environment of the helicopter while lighting conditions are ever-changing are overcome by using an ART SmartTrack 3 system. This system follows an arrangement of reflective markers attached to the pilot’s helmet. The VR headset is attached to the helmet in such a way that the pilot can wear it as if it were a pair of night vision goggles. Put together, this allows displaying a virtual world to the pilot, even in the most extreme maneuvers.

To ensure an authentic experience, the IRIS system incorporates real-time turbulence models, meticulously crafted from wind tunnel trials. These turbulence effects are seamlessly integrated into the aircraft’s motion and into the VR scene, providing pilots with precise proprioceptive and vestibular cues. It’s a symphony of technology and innovation in the world of aviation testing.

In-Cockpit Implementation

The optical tracking system relies on highly reflective marker targets on the helmet to track movement in three dimensions. Initially, only five markers were installed, strategically placed for optimal tracking. But the pursuit of perfection led NRC to create custom 3D-printed low-reflectivity helmet molds, allowing them to mount a dozen small passive markers. This significantly improved tracking reliability in various lighting conditions and allowed for a wider range of head movement.

Recently, NRC put this remarkable concept to the test with actual flight trials. The response from pilots was nothing short of exhilarating. They found the system required minimal adaptation, exhibited no noticeable lag, and, perhaps most impressively, didn’t induce any motion sickness. Even the turbulence effects felt incredibly realistic. Surprisingly, the typical VR drawbacks, such as resolution and field of view limitations, had minimal impact, especially during close-in shipboard operations. It’s safe to say that IRIS has set a new standard for effective and immersive aviation testing.

Publication of Results

The NRC team presented their results at the Vertical Flight Society’s 79th Annual Form in two papers [1]  and [2] and they also have a blog post on their site.

NOTE: Image contents courtesy of Aerospace Research Centre, National Research Council of Canada (NRC) – Ottawa, ON, Canada

High-performance Use Cases of LPVR & Varjo Headsets

Components of a VR/AR Operating System

Augmented and virtual reality technology helps boost worker productivity in various fields such as automotive, aerospace, industrial production, and more. Whereas the context of these applications is usually fairly specific, some aspects are common to many of these use cases. In this article, we will specifically explore the topic of pose tracking of Varjo head mounted displays (HMDs) based on LP-RESEARCH’s LPVR operating system. We will further on show two customer use cases that utilize LPVR in different ways.

In a typical VR/AR setup, you find three main subsystems as shown in the illustration below:

With our LPVR operating system, we connect these three building blocks of an VR/AR system and make them communicate seamlessly with each other while providing a simple, unified interface to the user. Depending on the specific use case, users might select different types of hardware to build their VR/AR setup. Therefore LPVR offers a wide range of interface options to adapt to systems from various manufacturers.

LPVR Flavors

LPVR operates in different flavors, we can group end applications into two categories:

  • LPVR-CAD – Static AR/VR setups, where multiple users operate and collaborate in one or more joint tracking volumes. These tracking volumes can be situated in different locations.
  • LPVR-DUO – AR/VR systems that are located in a vehicle or on a motion platform: such systems have special requirements, especially on the tracking side. If, for example, you would want to track a headset inside a car, displaying a virtual cockpit anchored to the car frame, and a virtual outside world fixed to a global coordinate system, means of locating the car in the world and referencing the HMD locally in the car frame are required.

 

In the following paragraphs, we will introduce two customer use cases that cover these two basic scenarios.

Large-scale Industrial Design at Hyundai

– Varjo XR-3 at Hyundai Design Center with optical markers attached. Image credit: Hyundai

For the Korean automotive company Hyundai Motor Corporation, we created a large, location-based virtual reality installation at their research and development center in Namyang, Korea. The system is used to showcase, amend and modify prototype and production-ready automobile designs.

This application uses optical outside-in tracking and LP-RESEARCH’s LPVR-CAD solution to track up to 20 users wearing head-mounted displays. While LPVR allows a mix of different headset types to operate in the same tracking volume, the Varjo XR-3 gives the most outstanding performance to inspect objects in high resolution and great detail. Additionally to an HMD, users carry hand controllers for a total of more than 40 tracked objects in a space close to 400 sqm.

– Hyundai’s collaborative virtual reality design experience. Image credit: Hyundai

Responsiveness is achieved by using LPVR-CAD to combine data from the inertial measurement unit built into the headsets and information from the optical tracking system. The optical system uses 36 infrared cameras to track the 160 markers attached to the HMDs and hand controllers. Perfectly smooth and uninterrupted position and orientation data of each user’s HMD is achieved by using LP-RESEARCH’s sensor fusion algorithms.

Depending on the type of headset, users either wear a backpack PC, connect to a host wirelessly or use an extra-long cable to connect directly to a rendering PC outside the tracking volume.

“Currently, we are actively utilizing VR from the initial development stage to the point of development. In the future, we plan to increase accessibility and usability by simplifying equipment using wireless HMDs. For this, improving the speed and stability of wireless internet is essential, which we plan to address by introducing 5G. In addition, LP RESEARCH’s technology is essential for multi-user location sharing within a virtual space.” – SungMook Kang, Visualization Specialist, Hyundai Motor Company

Next-level Automotive Entertainment with CUPRA

Imagine.. playing Mario cart, your hands are gripping the wheel, and you are in Neo Tokyo, on a race track. Futuristic buildings keep flying by while you race ahead, drifting into long turns and leaving your competitors behind you.

Now imagine you are no longer in your living room, you are sitting in an actual race car, buzzing around in an empty parking lot. Instead of looking through the windshield with your own eyes, you are wearing a Varjo XR-3 HMD. What you see outside the car is a virtual world, it’s Neo Tokyo.

– The view through the Varjo XR-3 headset. Image credit: CUPRA

As the car moves on the parking lot, you move inside the virtual world. When you move your head inside the car’s cockpit, the motions of your head are accurately tracked.

– Varjo XR-3 inside the cabin of the Urban Rebel. Image credit: Fonk Magazine

– Cupra’s Urban Rebel drifting on the test course

Together with the Norwegian company Breach VR, we have implemented this experience for the automotive company CUPRA. CUPRA is relentlessly pushing the technology of their vehicles into the future, striving to provide a novel driving experience to their customers.

Tracking of the vehicle and the Varjo XR-3 inside the vehicle is achieved with LP-RESEARCH’s automotive tracking systems LPVR-DUO. As the headset’s gyroscope sensors record the superimposed motion data of the car and the user inside the car, a specialized sensing system, and the algorithm are required to separate the two.

The result of this cascade of exceptional technology is a compellingly immersive driving experience of the future. The combination of an outstanding visualization device like the Varjo XR-3, LPVR state-of-the-art tracking, BreachVR’s 3D software and design and, last but not least, the incredible CUPRA race cars make for an exciting ride that you’ll greatly enjoy and never forget. Come and join the ride!

Check this blog blog post in the Varjo Insider Blog.

Check out our Instagram for further use cases with Varjo’s HMDs: @lpresearchinc

Design of an Efficient CAN-Bus Network with LPMS-IG1

Introduction to Designing an Efficient CAN-Bus Network

This article describes how to design an efficient high speed CAN-bus network with LPMS-IG1. We offer several sensor types with a CAN bus connection. The CAN bus is a popular network standard for applications like automotive, aerospace and industrial automation where connecting a large number of sensor and actuation units with a limited amount of cabling is required.

While creating a CAN bus network is not difficult by itself, there are a few key aspects that an engineer should follow in order to achieve optimum performance.

Efficient CAN-Bus Network Topology

A common mistake when designing a CAN bus network is to use a star topology to connect devices to each other. In this topology the signal from each device is routed to a center piece by connections of similar length. The center piece is connected to the host to acquire and distribute data to the devices of the network.

For reaching the full performance of a CAN bus network, we strongly discourage using this topology. Most CAN bus setups designed in this way will fail to work reliably and at high speed.

The CAN bus standard’s fundamental concept is to work best in a daisy chain configuration, with one sensor unit or the data acquisition host being the first device in the chain and one device being the last in the network.

Maximum CAN-Bus Speed and Cable Length

A key aspect for the design of an efficient high speed CAN-bus network is to correctly adjust bus cable lengths. The bus line running past each device is to be the longest connection in the network. Each sensor needs to be connected to the bus by a short stub connection. A typical length for such a stub connection is 10-30cm, whereas the main bus line can have a length of hundreds of meters, depending on the desired transmission speed.

Speed in bit/s Maximum Cable Length
1 Mbit/s 20 m
800 kbit/s 40 m
500 kbit/s 100 m
250 kbit/s 250 m
125 kbit/s 500 m

Note that a CAN bus network needs to be terminated using a 120 Ohm resistor at each end. This is especially important for bus length of more than 1-2m and should be considered as general good practice.

LPMS-IG1 CAN-Bus Configuration

One of our products with a CAN bus interface option is our LPMS-IG1 high performance inertial measurement unit. LPMS-IG1 can be flexibly configured to satisfy user requirements. It has the ability to output data using the CANopen standard, freely configurable sequential streaming or our proprietary binary format LP-BUS. These and further parameters can be set via our IG1-Control data acquisition application.

Some CAN bus data loggers that rely on the CANopen standard require users to provide an EDS file to automatically configure each device on the network. While we don’t support the automatic generation of EDS files from our data acquisition applications, depending on the settings in IG1-Control or LPMS-Control, it is possible to manually create an EDS file as described in this tutorial.

In this article we give a few essential insights into how to design an efficient high speed CAN-bus network with LPMS-IG1. If you would like to know more about this topic or have any questions, let us know!

Collaboration with Pimax

We are happy to announce a collaboration with the head-mounted display (HMD) manufacturer Pimax. Pimax HMDs feature very high resolution (up to 8K pixels) displays and an industry-leading field-of-view (max. 200°). By default, Pimax HMDs support SteamVR tracking and therefore are limited to relatively small tracking volumes.

We developed a special driver that allows our LPVR middleware LPVR-CAD and LPVR-DUO to work with Pimax headsets. Using LPVR, the headsets can now be used within a large-scale, location-based context, in connection with outside-in optical systems such as ART (Advanced Real-Time Tracking).

As Pimax is planning to implement UltraLeap hand tracking in their HMDs in the future, we are confident that we will also be able to extend our inside-out tracking algorithm to their devices.

The video above shows the basic functionality of tracking a Pimax HMD using LPVR and an optical tracking system. The headset’s motions are represented in SteamVR. For this demonstration the tracking volume is relatively small, but can be extended easily by using more outside-in tracking cameras.

This video was kindly provided to us by evoTec Solutions. Evotec is a new company in Switzerland that focuses on virtual reality (VR) solutions for corporations. Contact them for further information!

Big in Korea

Location-based Virtual Reality for Automotive Design

Figure 1 – Using LPVR-CAD large room-scale tracking, 3D design content is visualized on VIVE Pro HMDs

In cooperation with Korean automotive design solutions provider AP-Solutions, we created a large location based virtual reality installation at the Hyundai research and development center close to Seoul, Korea. The system is used to showcase, amend and modify prototype and production-ready automobile designs (Figure 1).

LPVR Large Room Scale Tracking Engine

Figure 2 – Each VIVE Pro HMD is equipped with optical tracking markers and an LPMS-CU2 IMU. The IMUs are covered with black tape to avoid reflections of infrared light.

The system uses optical tracking together with LP-Research’s LPVR solution to track up to 20 users wearing Vive Pro Head-mounted Displays (HMD). Each user carries a VIVE hand controller for a total of 40 tracked objects in a space close to 400sqm.

Responsiveness is achieved by using LPVR (Figure 2) to combine LPMS IMU data and a software package to achieve optimum performance. The optical system uses 36 infrared cameras to track the 160 markers attached to the HMDs and hand controllers. The position and orientation data of each user’s HMD is combined using LP-Research’s algorithm.

The content of the virtual space is rendered using a CAD software package running on backpack PCs worn by each of the 20 users. The PCs communicate and coordinate via a central server.

Korean News Coverage

Images courtesy of Hyundai Motor Group Newsroom.

1 2 3 4