Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories

Susung Hong1 Johanna Karras1 Ricardo Martin-Brualla2 Ira Kemelmacher-Shlizerman1,2
1University of Washington
2Google Research
CVPR 2025
 Paper  Code

Abstract

The fields of 3D reconstruction and text-based 3D editing have advanced significantly with the evolution of text-based diffusion models. While existing 3D editing methods excel at modifying color, texture, and style, they struggle with extensive geometric or appearance changes, thus limiting their applications. We propose Perturb-and-Revise, which makes possible a variety of NeRF editing. First, we perturb the NeRF parameters with random initializations to create a versatile initialization. We automatically determine the perturbation magnitude through analysis of the local loss landscape. Then, we revise the edited NeRF via generative trajectories. Combined with the generative process, we impose identity-preserving gradients to refine the edited NeRF. Extensive experiments demonstrate that Perturb-and-Revise facilitates flexible, effective, and consistent editing of color, appearance, and geometry in 3D without model retraining.

3D Object Editing

3D Object Editing Example 1
3D Object Editing Example 2
3D Object Editing Example 3

3D Scene Editing (🐻 ➡ 🐻‍❄️)

Bear transformation example

Method Overview

Method figure

Perturb-and-Revise takes a source NeRF and an edit prompt as input and produces the edited result through: (1) versatile initialization via parameter perturbation, (2) multi-view consistent score distillation, and (3) refinement with the identity-preserving gradient.

Parameter Perturbation

Parameter perturbation figure

Effect of parameter perturbation. In this example, we aim to make a NeRF model of a standing person sit down using just the word "sitting." The scene converges quickly even with large perturbations (η = 0.6), while complete regeneration yields blurry rendering results given the same number of optimization steps.

Objaverse Object Editing

Objaverse Editing 1
Objaverse Editing 2
Objaverse Editing 3
Objaverse Editing 4
Objaverse Editing 5
Objaverse Editing 6

Fashion Object Editing

Fashion Editing 1
Fashion Editing 2
Fashion Editing 3
Fashion Editing 4
Fashion Editing 5
Fashion Editing 6

Real Scene Editing

Real Scene Editing

Citation

@article{hong2024perturb,
    title={Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories},
    author={Hong, Susung and Karras, Johanna and Martin-Brualla, Ricardo and Kemelmacher-Shlizerman, Ira},
    journal={arXiv preprint arXiv:2408.00760},
    year={2024}
}