Video
Overview
We introduce FreeFix, a fine-tuning-free approach designed to eliminate artifacts and boost the rendering quality of 3D Gaussian Splatting (3DGS) in extrapolated views. Existing methods often require costly fine-tuning on domain-specific 3D datasets, which also compromise a model's generalization. FreeFix leverages pre-trained image diffusion models (IDMs) as a zero-shot prior. By utilizing an interleaved 2D-3D refinement strategy and a novel per-pixel confidence guidance, our method achieves high-fidelity, multi-view consistent results that are comparable to or even superior to state-of-the-art fine-tuning-based approaches across diverse datasets like Waymo and Mip-NeRF 360.
FreeFix Architecture
FreeFix employs an interleaved 2D-3D refinement strategy to ensure multi-frame consistency without the computational burden of video diffusion models. For a given extrapolated view, the system first renders an initial image from the 3DGS, which is then refined by an image diffusion model guided by a per-pixel confidence mask derived from Fisher information. This mask identifies uncertain regions to target for improvement while preserving high-fidelity areas. These refined 2D views are then integrated back into the 3D scene to update the 3DGS parameters. Additionally, we incorporate multi-level and overall guidance to provide structural hints and maintain consistency, especially in texture-less regions like the sky or ground.
Comparisons
