Video Comparisons

Table of contents

  1. Simple-ZipNeRF
    1. Effect of Augmentations
    2. Comparisons with Competing Models
  2. Simple-NeRF
    1. Effect of Augmentations
    2. Comparisons with Competing Models
    3. Comparisons with Ablations
  3. Simple-TensoRF
    1. Effect of Augmentations
    2. Comparisons with Competing Models
  4. Miscellaneous
    1. Lenticular effect due to Shape-Radiance ambiguity in NeRF
    2. Effect of Mass Concentration Loss in TensoRF Augmented Model
    3. TensoRF Augmented Models with only one of fewer components or lower resolution

Simple-ZipNeRF

Effect of Augmentations

Scene: MipNeRF360 / bicycle; We observe that the renders of ZipNeRF contain severe floaters. Our augmentation reduces floaters significantly, but contains severe blur in cycle spokes, the grass and the background greenery. Simple-ZipNeRF obtains the best of both worlds, reducing floaters and maintaining sharpness.
Scene: MipNeRF360 / kitchen; Here we visualize the effect in ZipNeRF renders by reducing the sampling domain first and then reducing the hash table size in the augmented model. Reducing the sampling domain removes most of the floaters close to the camera, but there exist undesired depth discontinuties in the scene. Reducing the hash table size induces smoothness in the augmented model which helps in reducing such undesired depth discontinuities.
Back to top

Comparisons with Competing Models

Scene: MipNeRF360 / bonsai; We observe that ZipNeRF fails to reconstruct the bonsai and creates floaters.
Scene: MipNeRF360 / counter; We find that ZipNeRF introduces severe floaters by placing objects close to the camera.
Scene: MipNeRF360 / stump; We observe that ZipNeRF fails to reconstruct the tree stump at the center of the video. Further, the background objects are also not reconstructed properly.
Back to top

Simple-NeRF

Effect of Augmentations

Smoothing Augmentation

Scene: LLFF / flower; We observe that the renders of NeRF contain several floaters. Our augmentation reduces floaters significantly, but fails to learn sharp depth edges at the end of the flower. Simple-NeRF reduces the floaters while retaining sharp depth edges. The remaining smooth depth edges in Simple-NeRF are in the disoccluded regions.

Lambertian Augmentation

Scene: LLFF / room; Here, we visualize the effects of view-dependent radiance by rendering the same objects from varying viewpoints. This is similar to the right video in "View-Dependent Appearance" section on the original NeRF website. For reference, we show the scene as observed from the moving camera in the top left corner of each video.
We observe that DS-NeRF is unable to learn the geometry of the object on the table. Instead, it changes the color of the table based on the viewpoint to give an illusion of the object (lenticular effect). Our Lambertian augmentation does not suffer from this shape-radiance ambiguity since it predicts color independent of the viewpoint. Supervising the main NeRF from the Lambertian augmentation helps Simple-NeRF better learn the geometry of the object on the table.
Back to top

Comparisons with Competing Models

DS-NeRF v/s Simple-NeRF

Scene: RealEstate / 00004; We find that DS-NeRF predictions are blurred and lack detail. Simple-NeRF predictions are sharper and more detailed.
Scene: RealEstate / 00006; We find that DS-NeRF predictions are blurred and lack detail. Simple-NeRF predictions are sharper and more detailed.

DDP-NeRF v/s Simple-NeRF

Scene: RealEstate / 00000; We observe ghosting artifacts on top of the chair nearest to the camera in DS-NeRF rendered video.
Scene: RealEstate / 00003; We find that DDP-NeRF predictions are blurred in the regions containing trees and twigs, whereas Simple-NeRF predictions are sharper.
Scene: LLFF / room; We observe that DDP-NeRF predictions contain severe distortions. Many of these distortions have been mitigated by Simple-NeRF.
Although we still observe a few distortions in Simple-NeRF predictions, they are probably due to the objects visible in less than two input views and also due to the difficulty of the scene. With specular objects, it may be difficult to learn the 3D geometry accurately with three input views only.

RegNeRF v/s Simple-NeRF

Scene: LLFF / flower; We observe floater artifacts in RegNeRF predictions, which are mitigated by Simple-NeRF.
Scene: LLFF / orchids; We observe floater artifacts in RegNeRF predictions, which are mitigated by Simple-NeRF.

FreeNeRF v/s Simple-NeRF

Scene: LLFF / fortress; We find that FreeNeRF predictions contain a hole in the table and the table surface regions are at different depths. Simple-NeRF predictions appear much more plausible without the above mentioned artifacts.

ViP-NeRF v/s Simple-NeRF

Scene: RealEstate / 00001; We observe distortions in the shape of the edges in the roof and the floor on the right in ViP-NeRF predictions. Simple-NeRF mitigates these significantly.
Back to top

Comparisons with Ablated Models

Scene: LLFF / fortress; We find that disabling the Smoothing Augmentation leads to more floaters in the predictions.
Scene: LLFF / room; Here, we visualize the radiance/color of the same 3D objects as the viewpoint varies. Disabling Lambertian Augmentation leads to severe Shape-Radiance Ambiguity among the chairs on the left.
Scene: LLFF / fortress; We notice that using Identical NeRF models as Augmentations also leads to floaters in the predictions.
Scene: LLFF / room; Here, we visualize the radiance/color of the same 3D objects as the viewpoint varies. Using Identical Augmentation instead of Lambertian Augmentation leads to Shape-Radiance Ambiguity around the chairs on the left.
Back to top

Simple-TensoRF

Effect of Augmentations

Smoothing augmentation

Scene: LLFF / flower; We observe translucent floaters around the flower in the renders of DS-TensoRF. While our augmentation reduces floaters significantly, it fails to learn sharp depth edges at the end of the flower. Simple-TensoRF reduces the floaters while retaining sharp depth edges. The remaining smooth depth edges in Simple-TensoRF are in the disoccluded regions.
Back to top

Comparisons with Competing Models

TensoRF vs Simple-TensoRF

Scene: LLFF / fern; We observe that Simple-TensoRF predictions are significantly better than that of TensoRF.
Scene: RealEstate / 00003; We observe that TensoRF fails to learn correct geometry of the scene and instead replicates multiple copies of the objects. Simple-TensoRF does not suffer from such issues.

DS-TensoRF vs Simple-TensoRF

Scene: LLFF / flower; We observe that Simple-TensoRF reduces floaters as compared to DS-TensoRF.
Scene: RealEstate / 00001; DS-TensoRF fails to properly reconstruct the textures on the left door. It also fails to properly reconstruct the geometry of the bottom part of the door on the right. Simple-TensoRF produces significant better renders in both regions.
Back to top

Miscellaneous

NeRF: Lenticular effects due to Shape-Radiance ambiguity

Scene: LLFF / room
Scene: LLFF / trex

The above two videos visualize how the radiance/color of the same 3D objects changes as the viewpoint varies, as learned by DS-NeRF. Observe how the objects in the scene change their position as the viewpoint changes. Specifically, observe the object on the table in the room scene and the roof in the trex scene. DS-NeRF is learning incorrect geometry but explains the observed images by simply varying the color of a different object.

Back to top

Simple-TensoRF: Augmented Model with and without Mass Concentration Loss

Scene: LLFF / horns; We show the effect of mass concentration loss on the augmented model of Simple-TensoRF. We observe several translucent floaters in the left video, which can lead to incorrect depth supervision to the main TensoRF. Adding the mass concentration loss reduces such translucent floaters leading to better depth supervision.
Back to top

Simple-TensoRF: Augmented Model with only one of fewer number of components or lower resolution

Scene: LLFF / horns
Scene: LLFF / room

In the left video, reducing only the number of components in the augmented model leads to tiny floaters, since the model can exploit the higher resolution. In the right video, reducing only the resolution of the augmented model leads to block floaters, since the model can exploit the higher number of components to create floaters at voxel edges.

Back to top