awesome-implicit-representations

shows how to fit images, audio signals, and even solve simple Partial Differential Equations with the SIREN architecture

shows how to fit a neural radiance field, allowing novel view synthesis of a single 3D scene

shows how you can leverage gradient-based meta-learning to generalize across neural implicit representations

Learn how you can use globally conditioned neural implicit representations as self-supervised correspondence learners, enabling robotics imitation tasks

(Park et al. 2019)

(Mescheder et al. 2019)

(Chen et al. 2018)

Sal: Sign agnostic learning of shapes from raw data

(Atzmon et al. 2019) shows how we may learn SDFs from raw data (i.e., without ground-truth signed distance values)

Implicit Geometric Regularization for Learning Shapes

(Gropp et al. 2020) shows how we may learn SDFs from raw data (i.e., without ground-truth signed distance values)

, , concurrently proposed hybrid voxelgrid/implicit representations to fit large-scale 3D scenes

(Sitzmann et al. 2020) demonstrates how we may parameterize room-scale 3D scenes via a single implicit neural representation by leveraging sinusoidal activation functions

(Chibane et al. 2020) proposes to learn unsigned distance fields from raw point clouds, doing away with the requirement of water-tight surfaces

proposed to learn an implicit representations of 3D shape and geometry given only 2D images, via a differentiable ray-marcher, and generalizes across 3D scenes for reconstruction from a single image via hyper-networks. This was demonstrated for single-object scenes, but also for simple room-scale scenes (see talk)

Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision

(Niemeyer et al. 2020), replaces LSTM-based ray-marcher in SRNs with a fully-connected neural network & analytical gradients, enabling easy extraction of the final 3D geometry

(Mildenhall et al. 2020) proposes positional encodings, volumetric rendering & ray-direction conditioning for high-quality reconstruction of single scenes, and has spawned a large amount of follow-up work on volumetric rendering of 3D implicit representations. For a curated list of NeRF follow-up work specifically, see

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

(Lin et al. 2020), demonstrates how we may train Scene Representation Networks from a single observation only

(Yu et al. 2020) proposes to condition a NeRF on local features lying on camera rays, extracted from contact images, as proposed in PiFU (see "from 3D supervision")

(Yariv et al. 2020) demonstrates sphere-tracing with positional encodings for reconstruction of complex 3D scenes, and proposes a surface normal and view-direction dependent rendering network for capturing view-dependent effects

(Sitzmann et al. 2021) proposes to represent 3D scenes via their 360-degree light field parameterized as a neural implicit representation

(Saito et al. 2019) Pifu first introduced the concept of conditioning an implicit representation on local features extracted from context images. Follow-up work achieves photo-realistic, real-time re-rendering

(Oechsle et al.)

(Niemeyer et al. 2019) first proposed to learn a space-time neural implicit representation by representing a 4D warp field with an implicit neural representation

(Deng et al. 2021) makes conditional implicit neural representations equivariant to SO(3), enabling the learning of a rotation-equivariant shape space and subsequent reconstruction of 3D geometry of single objects in unseen poses

Neural Sparse Voxel Fields

Applies a similar concept to neural radiance fields

(Yu et al. 2020) proposes to condition a NeRF on local features lying on camera rays, extracted from contact images, as proposed in PiFU (see "from 3D supervision")

leverages features learned by Scene Representation Networks for weakly supervised semantic segmentation of 3D objects

leverages features learned by occupancy networks to establish correspondence, used for robotics imitation learning

learns latent state space for robotics tasks using neural rendering, and subsequently expresses policies in that latent space

uses neural implicit geometry representation for learning a robot self-model, enabling space occupancy queries for given joint angles

leverages neural fields & vector neurons as an object-centric representation that enables imitation learning of pick-and-place tasks, generalizing across SE(3) poses

(Saito et al. 2019) proposed to locally condition implicit representations on ray features extracted from context images

(Sitzmann et al. 2019) proposed meta-learning via hypernetworks

(Sitzmann et al. 2020) proposed gradient-based meta-learning for implicit neural representations

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

(Lin et al. 2020) show how to learn 3D implicit representations from single-image supervision only

(Tancik et al. 2020) explored gradient-based meta-learning for NeRF

(Mildenhall et al. 2020) proposed positional encodings

(Sitzmann et al. 2020) proposed implicit representations with periodic nonlinearities

(Tancik et al. 2020) explores positional encodings in an NTK framework

(Stanley et al. 2007) first proposed to parameterize images implicitly via neural networks

(Sitzmann et al. 2020) proposed to generalize across implicit representations of images via hypernetworks

(Bemana et al. 2020) parameterizes the Jacobian of pixel position with respect to view, time, illumination, etc. to naturally interpolate images

Learning Continuous Image Representation with Local Implicit Image Function

(Chen et al. 2020) proposed a hypernetwork-based GAN for images

uses FILM-conditioned MLP as an image GAN

(Niemeyer et al. 2021)

(Guo et al. 2020)

(Yu et al. 2021)

Implicit Geometric Regularization for Learning Shapes

(Gropp et al. 2020) learns SDFs by enforcing constraints of the Eikonal equation via the loss

(Sitzmann et al. 2020) proposes to leverage the periodic sine as an activation function, enabling the parameterization of functions with non-trivial higher-order derivatives and the solution of complicated PDEs

(Lindell et al. 2020)

(Jiang et al. 2020) performs super-resolution for spatio-temporal flow functions using local implicit representaitons, with auxiliary PDE losses

(Schwarz et al. 2020)

(Chan et al. 2020)

(DeVries et al. 2021) Leverage a hybrid implicit-explicit representation, by generating a 2D feature grid floorplan with a classic convolutional GAN, and then conditioning a 3D neural implicit representation on these features. This enables generation of room-scale 3D scenes

uses FILM-conditioned MLP as an image GAN

(Skorokhodov et al. 2020)

Learning Continuous Image Representation with Local Implicit Image Function

(Chen et al. 2020)

(Anokhin et al. 2020)

(Karras et al. 2021)

(Shaham et al. 2020) leverages a hybrid implicit-explicit representation for fast high-resolution image2image translation

(Deng et al. 2020) represents an articulated object as a composition of local, deformable implicit elements

awesome-NeRF

List of implicit representations specifically on neural radiance fields (NeRF)