Skip to content

Conversation

bchretien
Copy link

MCMC-based models often suffer from very large splats that reduce the overall quality, as partially discussed in #252 and nerfstudio-project/nerfstudio#3548. This PR adds the same logic as in the default strategy, that is pruning (or in this case relocating) splats that are too large. I used the same naming convention and default values as for the default strategy for consistency.

@bchretien
Copy link
Author

If it makes things clearer, I can rename prune_scale3d to relocate_scale3d.

@N-Demir
Copy link

N-Demir commented Mar 6, 2025

Have you ran this and seen if it fixes the issues mentioned on those datasets?

@bchretien
Copy link
Author

@N-Demir: I only ran it on my datasets (that I can't share) and it improves the situation. Some users on the Discord server tested it and it worked for them as well.

@CameronFraser
Copy link

can also confirm this dramatically improves the quality of the exported ply files, this needs to be in nerfstudio asap

@ichsan2895
Copy link
Contributor

ichsan2895 commented Apr 9, 2025

Sorry, I tested it with two setup:

  1. ns-train splatfacto-mcmc --pipeline.datamanager.batch-size 3 with gsplat 1.4.0
  2. ns-train splatfacto-mcmc --pipeline.datamanager.batch-size 3 with this PR of gsplat

But the result of 2nd setup is worse. Both quantitative (PSNR, LPIPS, SSIM) and visually qualitative.

So far, Tested it with garden, bicycle, and stump dataset from mip360.

1st setup:
image

2nd setup:
image

Dunno why it does not give good result from me. FYI. I have not tested it with native gsplat simple_trainer.py mcmc.

Additional info:

Ubuntu 22.04.3 LTS
Python 3.10.12
Torch 2.1.2+cu118
RTX 4090 on Runpod cloud environment
Nerfstudio commit d5bdd45b5bfd30e928e8b90d6735b2e38ae79a59

@ichsan2895
Copy link
Contributor

ichsan2895 commented Apr 10, 2025

python3 examples/simple_trainer.py mcmc --batch-size 3:

1st = Gsplat 1.4.0
image

2nd = This PR of GSplat
image

Additional benchmark of MCMC from @MrNeRF in this thread: #409. It uses batch-size 1.
Untitled

@bchretien
Copy link
Author

@ichsan2895 thanks for the feedback! So maybe keep it disabled by default, so that users can enable it for scenes where this improves things? In my case, drone flights for which we look down and not far away (large splats are often located on the border of the scene but "spill" in the focused part).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants