Picture this: an action-packed Indiana Jones adventure, but instead of a movie set, it’s happening at our booth! This year at Cine Gear 2024, Catalyst Virtual was invited to power a ground-breaking and unique virtual production concept. Combining the power of digital worlds, robotics, and a practical set, we created an unforgettable visitor experience where show participants could be a part of the action - and walk away with a cinematic video of the experience, to boot!
We had the opportunity to team up with some of our favorite companies in the industry: ROE Visual and their industry-leading LED wall panels, SISU Cinema Robotics and their industry-leading robotic arms, NAC Effects sourcing the mechanical & physical props, VP Toolkit dialing in workflow and lighting and Form Studios' Shawn Herold providing the environment and creative direction. Together, we delivered an unforgettable in-camera VFX virtual production live on a convention show floor. We're incredibly proud of what we were able to accomplish with this first-of-its-kind setup, and we wanted to talk a bit more about how it came to be.
On-the-Spot Creativity
The concept was simple: integrate a real-life Tuk-Tuk on a motion-tracked turn-table into an Unreal Engine scene, matching the rotation to an in-game vehicle driving on a path, while using a robot arm to shoot the action. Cast a trained actor to drive, allow show attendees to ride the Tuk-Tuk, and let them walk away with a cinematic video minutes afterward. Easy!... right?
Here at Catalyst Virtual, we've been working hard to make the footprint of virtual production stages as small as possible. With our mobile stage kit, we pair a powerful render node with a road case and a mobile network, resulting in a ready-to-roll package containing the core components of an ICVFX virtual production. Always up for a challenge, we shipped two of our mobile stage kits across the country to the Warner Bros. sound stage in Burbank, California. These would be the engines powering this adventure.
Our Neon Gold render node, which contains 2x of the latest Nvidia GPUs, was the featured player. Leveraging Unreal Engine and the new multi-processing options introduced in 5.3, we were able to isolate different sections of the render to individual GPUs, and composite them together in the final output. By separating the inner and outer frustum rendering processes, we massively boost the performance of our virtual background—almost doubling it. It’s like giving each part of the scene its own supercharged engine, ensuring everything runs smooth and looks fantastic.
Hooking up to the ROE Visual LED wall was a breeze thanks to our pre-configured platform. Think of it like assembling a high-tech LEGO set - everything just clicks into place! In no time at all, we were ready to move on to integrating all of the other elements of our activation into the Unreal Engine scene. This was where the real challenge began!
Technology Integration
Despite the show taking place on a professional sound stage, working under the conditions of an open convention floor created a number of non-optimal working conditions. Avoiding light spill on the LED wall is always critical to a realistic look - no easy task in a large, bright room. Ian Fursa of VP Toolkit worked his lighting magic to finesse these conditions into a production-grade result despite the obstacles. Aperture provided the gear that made everything pop.
We also had to consider that our audience at the show may never have seen an ICVFX production in action before. We decided to run our render systems at a substantially higher resolution than we normally would on a real production, so the audience would have more eye-candy while they watched the final result being filmed. Our Neon Gold didn't even break a sweat!
To control this whole setup, we plugged multiple laptops into our network. These operator laptops controlled the backgrounds using a separate instance of Unreal Engine and acted as remote controls for the LED wall Brompton processors. With everything connected, we could make quick changes to environments and creative content on the fly — because who doesn’t love a bit of spontaneity?
The Setup
The SISU Robot was our MVP, positioning and moving the camera with surgical precision. We connected the SISU robot to our network, configuring a virtual "subnet" to isolate the motion control data and match the IP scheme already used by the robot. SISU robots output a Live Link signal natively, which Unreal Engine render nodes and editors pick up on to display the motion in-engine. Since we had two robots running demos at our booth, we had to create a second subnet to avoid any crossed signals. Before doing this, motion from different robots would flow to both stages, creating movement where it shouldn't have! It's like giving each robot its own playground so they don't fight over the swings. The robot wasn't just standing around either; it triggered the driving sequence in Unreal Engine and recorded the tuk-tuk gimbal encoder data, making sure everything was in sync.
The tuk-tuk, brought in and set up by NAC Effects, was mounted on a gimbal to sync its' movements seamlessly into Unreal Engine. SISU hooked up the gimbal to an Arduino board with an encoder to get this real-time rotational data. This setup was supposed to sync the virtual elements with the tuk-tuk’s movements perfectly. But, as tech often does, the encoder and Arduino board started to drop data. So, we had to change our game plan: instead of the rotational data running in real-time, we recorded the data in Unreal as a sequence and played it back to match the background, triggered by the robot. Think of it as a high-tech dance rehearsal where we nailed every move and then hit replay for the perfect performance. This demonstrates the advantages of robotic control perfectly - precise and reproducible results, every time.
With all the pieces in place in record time, we ran dry-run after dry-run, refining the setup dynamically along the way to get everything perfect.
Our final setup is a great example of the flexibility that can be afforded to a virtual production when the right foundation is in place. Using a single Neon Gold render node, paired with a laptop and a small, flexible network, we were able to operate the entire setup.
This setup connected our entire on-site team with the technology, making it easier and faster to be creative (because we needed it). This efficient arrangement enabled us to deliver top-notch results quickly.
Visitor Experience
The activation was a smash hit! Everyone wanted their own Indiana Jones moment, lining up like an amusement park for a piece of the action. After the ride, attendees got personalized videos of their adventure emailed to them by SISU. That rigged up an automated exporting process, complete with cut scenes to really sell the idea. Everybody loved being able to see immediate results, which really showcases the advantages of Virtual Production in a fun way.
Collaborative Innovation
Our success at Cine Gear 2024 was a true team effort. Form Studios brought their creative vision, Sisu Robots delivered precision and reliability, Roe’s LED walls provided the stunning visuals, and NAC Effects set up our tuk-tuk with impeccable skill. This collaboration exemplifies how working together can push the boundaries of what’s possible.
Looking Ahead
A huge shoutout to Form Studios, Sisu, Roe, and NAC Effects for making this possible. We’re already cooking up ideas for next year and can’t wait to wow you all over again.
Here’s to another year of epic adventures at Cine Gear!
For more info on our services and tech, visit our website.
