Every phone video is shot from exactly one viewpoint. Yet when we watch a clip of someone being interviewed, we can effortlessly imagine the same moment from the side, from behind, or from above: we build a rough 3D world in our heads and replay its motion from wherever we like. Camera redirection asks a machine to do the same, synthesizing a new video of a dynamic scene along a user-specified camera trajectory from a single monocular recording. Today, effects like bullet-time replays still require multi-camera rigs; doing it from one casual video would put that capability in everyone’s pocket.

We are excited to share FreeOrbit4D, our SIGGRAPH 2026 paper on this problem. FreeOrbit4D is training-free: it composes off-the-shelf models with no fine-tuning, and it handles the regime where existing methods break down, namely large-angle redirection, where the target camera swings 120° or even 180° away from where the video was shot. The key idea is to first recover a foreground-complete 4D proxy, an explicit point-cloud representation of the scene in which the moving object is reconstructed on all sides, including the sides the camera never saw, and then use that proxy as a geometric scaffold for a video diffusion model.

Input video
Redirected by FreeOrbit4D

Why large-angle redirection is hard

A monocular video is a very thin slice of a 4D world. It observes each surface from a single direction, and anything facing away from the camera is simply never recorded. Small viewpoint changes can be handled by reconstructing the visible surfaces and re-rendering them, as NeRF- and Gaussian-splatting-style methods do. But once the target camera moves far from the source trajectory, the unseen surfaces dominate the frame, and there is nothing in the input to anchor them.

Recent work leans on video diffusion models to fill this gap, and largely follows one of two recipes. Implicit control methods encode the target camera as a learned embedding or a text prompt and inject it into the generator. This gives only soft controllability: text cannot describe a precise orbit, learned conditions frequently ignore the requested path (a “turn to the back” instruction often does not turn to the back), and training these models needs paired multi-view video data that is expensive and scarce. Explicit warping methods instead estimate depth, warp the observed pixels to the target view, and let a generator inpaint the holes. Camera control is now precise, but the warped geometry only ever contains what was visible, so the object arrives at the new viewpoint with half of itself missing, and the generator has to hallucinate the rest frame by frame, with no 3D support to keep it consistent over time.

Neither recipe delivers both precise camera control and complete visibility of unseen regions, and both are needed for faithful large-angle redirection.

A foreground-complete 4D proxy

Our observation is that reconstructing a dynamic scene from monocular video and completing the occluded geometry of an object are fundamentally different tasks. The first needs temporally consistent, scene-level reasoning; the second needs multi-view understanding of object shape. So we handle them in separate coordinate spaces, with models that are good at each, and then unify the results.

Overview of FreeOrbit4D: global scene reconstruction and canonical object completion are fused into a foreground-complete 4D proxy, rendered into depth maps that condition a video diffusion model.
Overview. Global scene reconstruction recovers the background and a partial foreground in world space; canonical object completion reconstructs the full foreground from synthesized multi-view images. After alignment, depth rendered from the unified 4D proxy along the target trajectory conditions a video diffusion model, together with a reference frame and a text prompt.

Decoupled 4D reconstruction. In global scene space, a dynamic-aware feed-forward network (PAGE-4D, a temporally-aware extension of VGGT) lifts every frame of the source video into a point map registered in a single world coordinate system. SAM 2 masks, initiated by one click on the first frame or automatically from a detector, split these points into a static background and a per-frame foreground. This foreground is geometrically incomplete: it only contains surfaces the source camera saw. In parallel, in canonical object space, we cut the foreground out of each frame and feed the sequence to an object-centric multi-view video diffusion model (SV4D 2.0), which synthesizes four temporally synchronized novel views at 90° azimuth intervals. Running VGGT on the source view plus the four synthesized views gives a complete foreground point cloud for every frame, but one that knows nothing about where the object sits in the scene.

Correspondence-aware alignment. The two foregrounds have to be merged, and the trick that makes this easy is that both were derived from the same source image. A pixel in frame t maps to a 3D point in the partial cloud and to a 3D point in the complete cloud, so we get dense, pixel-synchronized 3D–3D correspondences for free. Monocular lifting cannot pin down absolute depth scale, so we do not fit the two clouds point-by-point; instead we use the partial cloud only to determine where the object is and how big it is, estimating a per-frame scale and translation while keeping the complete cloud’s geometry intact. Because monocular depth also jitters from frame to frame, we smooth the aligned object’s centroid trajectory with a bidirectional Kalman filter under a constant-velocity model. The output is the foreground-complete 4D proxy: the static background plus a complete, correctly placed, temporally coherent foreground at every time step.

Geometry-conditioned video synthesis. Finally, we render the proxy along the target camera trajectory into per-frame depth maps and hand them to a depth-conditioned video diffusion model (Wan2.2-VACE), along with the first source frame as an appearance reference and a text prompt. Depth is not full geometry, but it compactly encodes the 3D layout and visibility from the target viewpoint, which is exactly what the generator needs to follow the prescribed camera precisely and stay consistent across frames. The whole pipeline runs on a single NVIDIA A40 and processes a 45-frame clip at 832×480 in roughly 50 minutes.

Results

The interactive 4D point clouds for several scenes, including the robot demo below, can be rotated and scrubbed through time on the project page.

Input video
Redirected by FreeOrbit4D

Because the proxy is a real 3D representation, the same reconstruction can be replayed along any number of trajectories. Below, each input video is redirected to two different target cameras.

Input
Trajectory #1
Trajectory #2
Input
Trajectory #1
Trajectory #2

The scenes span articulated human motion, humanoid robots, animals and vehicles, with real-world footage from DAVIS and online videos as well as synthetic clips generated by Veo and Sora. For evaluation we deliberately choose extreme trajectories, with yaw or pitch rotations of 120° or 180° from the initial viewpoint.

Input video
Redirected by FreeOrbit4D

We compare against five recent camera-controlled video-to-video methods: ReCamMaster, TrajectoryCrafter, EX-4D, GEN3C and CogNVS. The “Swing” sequence below is a hard case, with fast foreground motion and thin swing ropes, viewed from a camera that orbits 120° around the scene. ReCamMaster and EX-4D lose the structure of the human body, producing blurred limbs and ghosting; TrajectoryCrafter, GEN3C and CogNVS drift geometrically and semantically as the camera rotates. FreeOrbit4D keeps sharp detail and stable geometry throughout, because the completed foreground tells the generator what should be there.

Input
ReCamMaster
TrajectoryCrafter
EX-4D
GEN3C
CogNVS
FreeOrbit4D (ours)

Quantitatively, FreeOrbit4D ranks first on five of the six VBench dimensions and achieves the best DINO-SIM, CLIP-SIM and FID-V among all methods. But we found that automatic metrics do not really measure whether a method followed the camera: ReCamMaster and TrajectoryCrafter score reasonably while visibly deviating from the requested path. So we also ran a user study with 20 participants across 10 sequences, rating overall preference, camera-motion accuracy and temporal stability on a 1–5 scale. There the gap is much larger.

Method DINO-SIM ↑ CLIP-SIM ↑ FID-V ↓ Overall ↑ Motion ↑ Stability ↑
ReCamMaster 0.37 0.75 2.6 2.0 2.5 2.0
TrajectoryCrafter 0.47 0.79 2.0 2.8 3.2 2.9
EX-4D 0.28 0.69 3.2 2.0 2.5 2.0
GEN3C 0.43 0.75 2.3 2.4 3.5 2.3
CogNVS 0.34 0.78 3.0 2.0 3.2 1.9
FreeOrbit4D 0.65 0.84 1.7 4.6 4.5 4.5

FID-V is reported ×10². The three right-hand columns are user-study ratings (1–5). See the paper for the full VBench and FVD-V numbers.

Ablations confirm that both halves of the design matter. Naively feeding the synthesized multi-view images together with the source video into a single feed-forward reconstructor collapses correspondences across time and produces ghosting; removing multi-view generation degrades every metric; and removing the Kalman smoothing noticeably hurts coherence.

Beyond redirection

Because the proxy is an explicit, editable point cloud rather than a latent inside a network, it turns out to be useful for more than moving the camera.

Appearance propagation. Edit a single reference frame with an image editor (we use Qwen-Image-Edit), say a zebra print on the car or an anime restyle, and the proxy’s depth scaffolds carry that edit consistently across every novel viewpoint of the redirected video.

Original
Appearance editing: color / pattern
Original
Appearance editing: style

4D geometry manipulation. The point cloud can be edited directly in space-time: scale the foreground, or composite objects reconstructed from different videos into one shared scene, and then synthesize a plausible redirected video from the modified geometry. We think this points toward using the recovered proxy for 4D asset creation and synthetic data generation, where dense, consistent geometry and annotations come along for free.

Original
Geometry editing: scaling
Original
Geometry editing: composition

Limitations

FreeOrbit4D assumes a single dominant foreground object and a mostly static background. It can handle several objects as long as each is observed well enough over time, but heavy mutual occlusion or dynamic backgrounds remain difficult for current object-centric models. As a modular system built from off-the-shelf parts, errors upstream in segmentation or multi-view synthesis propagate downstream, although that same modularity means each component can be swapped for a stronger one as it appears. And the multi-stage pipeline prioritizes quality over speed; at tens of minutes per clip, it is not yet interactive.

Read more

The paper appears in SIGGRAPH 2026 Conference Papers. The project page has interactive 4D reconstructions you can rotate and scrub, plus full comparisons on all sequences, and the code is on GitHub.

@inproceedings{cao2026freeorbit4d,
  title     = {{FreeOrbit4D}: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete {4D} Reconstruction},
  author    = {Wei Cao and Hao Zhang and Fengrui Tian and Yulun Wu and Yingying Li and Shenlong Wang and Ning Yu and Yaoyao Liu},
  booktitle = {SIGGRAPH Conference Papers},
  year      = {2026},
  url       = {https://doi.org/10.1145/3799902.3811122}
}

This research is supported by the National Artificial Intelligence Research Resource (NAIRR) Pilot under award NAIRR250199, with computational resources from Delta and DeltaAI at NCSA through ACCESS allocations.