Volumetric video can capture an actor's full three-dimensional performance and hand the director a movable, re-lightable digital asset—but only if the production is prepared for integration challenges that industry research still describes as unresolved. The technology records subjects in 3D, creating digital models that can be viewed from any angle at any time, and it offers real creative advantages over green screen work, particularly in actor freedom and post-production camera flexibility. The trade-off is that existing volumetric capture solutions are, in the assessment of a feasibility study by Distortion Creative Group and CAMERA at the University of the Bath, "currently not fit for purpose in the commercial environment." This guide walks through the professional workflow, compares the approach against traditional VFX, and sets out the technical hurdles a producer should budget for before committing.
Introduction to Volumetric Video in Film & TV Production
Volumetric video is a capture technique that records a three-dimensional moving object—people, objects, or entire environments—rather than a flat image. As filmmaking guidance on the technology explains, it records subjects in 3D and creates digital models that viewers can see from any angle at any time. In a film and television context, that means a performance is not locked to the camera positions chosen on the day; the resulting asset can be re-framed, re-lit, and composited into scenes long after the shoot ends.
The technology has grown up alongside virtual production, which brings real-time graphics engines such as Unreal Engine into film production by combining 3D camera tracking—using devices such as RedSpy to understand where the camera sits within 3D space—with live-rendered environments. Distortion Creative Group reports using this technology commercially since 2020, across everything from university lectures to narrative films, and is now studying how volumetric capture fits into those extended reality and virtual production workflows. Sony has built its own Volumetric Capture Studio around multiple high-resolution cameras and depth sensors to create photorealistic 3D human performances, capturing actors in full 360-degree detail so directors can make dynamic camera movements and viewpoint changes in post-production.
The Volumetric Video Workflow: Capture, Processing, and Integration
A professional volumetric pipeline breaks into three stages, and understanding where the effort sits in each one is essential to planning a production around it.
Capture. The subject performs inside a dedicated capture volume ringed by multiple cameras or depth sensors that record depth and color data simultaneously. Sony's studio system uses multiple high-resolution cameras and depth sensors together to build photorealistic 3D human performances. The specific setup varies by system, but it generally involves a dedicated capture stage—though software such as Depthkit has been developed as a flexible option for creators without expensive capture stages, pairing with affordable depth cameras like the Intel RealSense to make volumetric capability accessible to more creators.
Processing. The raw multi-camera data is converted into a three-dimensional moving model of the performance. This is where much of the technical weight sits: the captured depth and color streams must be fused into a usable 3D asset, and as discussed below, current compression and processing techniques are a known weak point for visual quality and speed.
Integration. The finished asset is brought into the production's real-time engine or post-production environment—Unreal Engine for virtual production workflows, or compositing tools for final assembly—where it can be placed in virtual environments, re-framed, and rendered from arbitrary viewpoints. Sony describes its system as integrating with existing film production pipelines and supporting both real-time preview and post-production workflows. In practice, this integration stage is precisely where the industry's open problems cluster: the Distortion Creative Group and University of Bath feasibility study exists because fitting volumetric capture into extended reality and virtual production workflows remains an identified R&D gap rather than a solved problem.
Volumetric Video vs. Traditional VFX: A Decision Framework
The clearest way to decide between volumetric capture and established techniques is to compare them dimension by dimension. The framework below draws on documented capabilities and limitations of each approach across the criteria that most affect a producer's or director's choice.
| Criterion | Volumetric Video | Traditional VFX (e.g., Green Screen) |
|---|---|---|
| On-set capture process | Captures depth and color data simultaneously using multiple cameras or depth sensors, producing a three-dimensional moving object. Setup varies by system but generally requires a dedicated capture stage. | Captures live-action footage against a uniform background for later compositing, isolating the subject for background replacement—which limits immediate 3D context. |
| Actor performance freedom | Actors can deliver uninterrupted takes without worrying about camera setups, performing naturally within the capture volume—though freedom is constrained by the physical boundaries and technical limits of the stage. | Performances are often constrained by precise positioning, eye-lines, and interaction with non-existent elements; actors must imagine their environment, which can impact spontaneity. |
| Real-time on-set preview | Preview systems exist—Depthkit shows live previews to adjust lighting and camera angles instantly—but they typically operate at reduced quality levels that inadequately represent final output. | Preview quality depends on the compositing setup; the technique is mature enough that what is monitored on set maps predictably to the final composite. |
| Post-production flexibility | Enables dynamic camera movements and viewpoint changes in post-production, since the performance exists as a full 360-degree 3D asset. | The final framing is largely fixed at capture; changing perspective after the shoot typically requires re-shooting or full CGI reconstruction. |
| Integration with existing pipelines | Recognized as a significant technical challenge; existing solutions are currently not fit for purpose in the commercial environment, and ongoing research aims to improve seamless integration. | Compositing and CGI pipelines are mature and widely supported across industry-standard software; while complex, they are well-understood and optimized. |
| Potential for visual artifacts | Current compression techniques often introduce artifacts that compromise professional-grade visual quality, requiring significant post-processing. | Issues like green screen spill, inaccurate keying, or unnatural lighting integration occur, but extensive tools and experience exist to address them. |
Read across the rows, the pattern is consistent: volumetric video trades pipeline maturity and predictable quality for performance freedom and post-production camera flexibility. A production whose value depends on a spontaneous, unbroken actor performance inside a virtual world gains the most; a production that needs guaranteed final-pixel quality on a fixed schedule gains the least.
Current Technical Challenges and Limitations
Four hurdles dominate the current state of the technology, and each one has a direct production consequence.
Commercial viability. The blunt assessment from the Distortion Creative Group and CAMERA at the University of Bath feasibility study is that existing volumetric capture solutions are "currently not fit for purpose in the commercial environment." That gap is what their research project—aimed at integrating volumetric capture into extended reality and virtual production workflows—was established to fill, which tells a producer two things: the tools available today require in-house technical expertise, and better-integrated options are actively in development.
Compression artifacts. Analysis of volumetric video in film production reports that current compression techniques often introduce artifacts that compromise the visual quality necessary for professional film production. These artifacts affect final visual fidelity and push additional work into post-processing.
Insufficient real-time processing. The same analysis finds that real-time processing capabilities remain insufficient for interactive on-set applications, limiting directors' ability to make immediate creative decisions during filming. Where preview systems do exist, they typically run at reduced quality levels that inadequately represent the final output—creating uncertainty in production planning and creative direction.
Quality consistency. Technical teams continue to be challenged by quality consistency across different lighting conditions and capture environments, meaning results achieved in one studio setup may not transfer reliably to another.
None of these are reasons to rule the technology out, but together they define the realistic profile of a volumetric production today: one that carries R&D risk, needs specialist support, and cannot yet rely on on-set previews as a trustworthy proxy for the final image.
Making the Call: When to Embrace Volumetric Video
Film and TV professionals should consider volumetric video for projects prioritizing actor performance freedom, dynamic virtual environments, and unique creative expression, provided they are prepared to invest in overcoming current integration and quality challenges. The decision boundary is straightforward: if the project's creative core is an uninterrupted, naturalistic performance that must live inside a virtual environment and be re-framed freely in post-production, volumetric capture offers something green screen and full CGI cannot easily replicate. If the project instead depends on schedule certainty, predictable final-pixel quality, and a pipeline every vendor already knows, established VFX remains the safer path.
Studios themselves are signaling where the demand sits: industry reporting notes that production houses are actively seeking integrated solutions that serve multiple content formats and distribution channels, with interest extending beyond narrative filmmaking into virtual production for television, advertising, and extended reality experiences. That demand is aimed squarely at reducing the iterative back-and-forth typically required in CGI integration—precisely the workflow friction volumetric video promises to remove once its integration problems are solved.
Before committing a project, a production team should validate the technology against a concrete benchmark: successful integration of volumetric assets into a real-time engine (e.g., Unreal Engine) with acceptable visual fidelity and minimal post-processing artifacts. Running that test early—on a representative scene, under the production's actual lighting conditions—turns the technology's documented uncertainties into a measurable go/no-go signal rather than a mid-production surprise.
