Revolutionizing VFX with GenVFX: How Groove Jones is Leading the Charge in AI-Driven Visual Effects

In recent months, Artificial Intelligence (AI) has taken the world by storm with its rapid advancements. However, in the world of Visual Effects (VFX), artists have been quietly utilizing machine-learning tools for years to revolutionize workflows. This comes as no surprise since VFX has always thrived on technological innovation. Now, with the rise of Generative AI (GenAI), the industry is poised for its next groundbreaking leap — much like how CG dinosaurs once redefined storytelling in Jurassic Park.

At Groove Jones, we’ve been embracing this wave of AI development, using it to elevate our studio’s offerings while staying true to our artistic roots. By leveraging AI, we’ve been able to create those unforgettable “WOW” moments more efficiently than ever, enhancing both the creative process and speed of delivery.

What is GenVFX? It is the Intersection of GenAI and Machine Learning

Our approach centers on GenVFX — a suite of tools that combines the imaginative power of Generative AI (GenAI) with the efficiency of Machine Learning (ML). These two technologies complement each other, creating a new ecosystem that addresses traditional challenges in VFX production while unlocking unprecedented creative possibilities.

• What GenAI Brings to the Table: GenAI excels in generating original content—images, animations, and videos—acting as a creative collaborator that enhances artists’ work.
• The Role of Machine Learning: ML provides the backbone for automating repetitive tasks, bringing efficiency and refinement to VFX workflows.

• Enhanced Stock Plates: AI can generate perfect green or blue screen elements for keying and compositing. For example, we’ve gone a step further by introducing “generative extras” with animated behaviors in VR experiences, such as with the Apple Vision Pro, featuring hyper-realistic 3D avatars with lifelike expressions.
• Hybrid Background Plates: GenAI enables the creation of stunning matte paintings and environmental plates, which are easily composited into projects to elevate production design.
• Neural Diffused Render Engines: AI-powered render engines use depth maps, or gray-box renders to craft final shots with meticulous control. We’ve been leveraging open-source AI tools for test renders and look development, and now we’re scaling this for video and CG assets.

• Relighting Capabilities: AI-driven relighting adjusts the lighting of original footage with remarkable precision—no reshoots required. This tool offers unprecedented creative flexibility, from altering the mood to simulating different times of the day.
• Style-to-Video Transformations: This feature allows us to reimagine the style or content of an entire video, opening incredible possibilities to expand storylines and worlds within a single production.

Machine Learning (ML) in Our Workflow

Unlike GenAI, ML refines and automates processes, saving time and reducing production costs. At Groove Jones, we currently deploy ML for:

• Denoising: Removing noise and grain from renders, especially useful for low-light sequences.

• Optical Flow Object Tracking: More precise and faster motion tracking for 3D objects, enhancing complex edits.
• Retiming: Creating smooth slow-motion effects with reduced artifacts, eliminating the need for costly reshoots.
• Streamlined Rotoscoping: Automating tedious tasks like subject masking using advanced segmentation models.
• UpRes: Elevating low-resolution assets to 4K while maintaining critical details.

Groove Jones in Action: Redefining Production

We are using GenFX to do some fantastic things that years ago would have taken a large team of artists and engineers years to accomplish. For example, In “Forrest Gump” (1994), Robert Zemeckis and the VFX team at ILM took over six months to create a 6-second shot of Tom Hanks interacting with JFK in the film. Ironically, the Academy Award-winning team (Best Visual Effects) made a huge mistake in this shot, resulting in the President’s tie changing colors between takes in the same scene.

Our GenVFX approach enables us to accomplish what once required months of effort in a fraction of the time. As an example:

Historical Preservation Meets GenVFX

We recently utilized GenVFX tools to support historian Bryan Rigg’s work documenting German Jews’ survival during World War II. Originally thought to be a handful, Rigg revealed that there were over 150,000 Jewish men who survived. This included decorated veterans, high-ranking officers, generals, and admirals. In the 1990s, Rigg traveled throughout Austria and Germany, capturing hundreds of hours of their stories on a VHS recorder. We started with authentic video interviews of Holocaust survivors telling their stories. We began working on an initiative to turn this historical media into an educational tool powered by AI and our background in interactive development and VFX. Rigg’s extensive video interviews from the 1990s were captured on VHS, and we applied AI-driven transcription, translation, voice synthesis, and lip-syncing to create fully localized and accurate educational tools.

By employing a specialized cloud-based dashboard, experts could provide real-time feedback on AI transcriptions, enabling a reinforcement-learning loop for ongoing improvement. This pioneering work would have taken years to complete with traditional methods but was achieved in a fraction of the time using GenVFX.

Expanding Horizons with GenVFX

• Accelerated Storyboarding: GenAI-powered tools, like Stable Diffusion, allow us to experiment with mood boards or quick client-aligned storyboards. This reduces pre-production time while ensuring the client’s vision is met before diving into complex production stages.

• Modernized Backgrounds and Assets: Our integration of GenAI-generated assets with traditional VFX workflows produces seamless visual effects. For photorealistic matte paintings or enhanced 3D props, GenAI helps give teams a creative leg-up.
• Virtual Try-Ons and Wardrobe Fixes: Virtual wardrobe adjustments now apply LoRA flux training, allowing us to modify costumes or replace logos digitally—in ways previously possible only with post-production paint-outs.
• Generative Voiceovers and Lip-Syncing: GenAI tools streamline dialogue or voiceover generation. We’ve trained AI to match voice actors for narrations, as seen in projects like the Invesco QQQ NCAA production or the American Heart Association campaigns, eliminating rescheduling delays during narration updates.

Balance is Key: The Human-AI Partnership

While the power of GenVFX is undeniable, at Groove Jones, we believe in finding balance. AI tools are meant to amplify artistry, not replace it. Here’s how we strike that balance:

• Investing in Talent: Our artists are trained to use AI as a tool to push their creative boundaries while leveraging their artistic intuition.
• A Hybrid Workflow: Combining AI advancements with trusted VFX workflows ensures optimal results for our clients.

Looking Ahead

The rise of GenVFX is redefining the boundaries of visual storytelling. AI allows us to create faster and more efficiently than ever without compromising creativity or quality. At Groove Jones, we’re proud to drive this evolution while remaining faithful to the artistry that defines the VFX industry.

By harnessing the strengths of Generative AI and Machine Learning, we’re shaping a future where technology and imagination combine to create at the speed of thought. Let Groove Jones help you craft unforgettable, cutting-edge experiences for your audience. Contact us today!