awesome-scene-understanding

Scene understanding dataset and paper list

A curated collection of papers and datasets for research and development in scene understanding using computer vision techniques.

😎 A list of awesome scene understanding papers.

GitHub
729 stars
48 watching
93 forks
last commit: 25 days ago
Linked from 1 awesome list

3d-sceneawesomecomputer-visiondeep-learningindoor-scenesscene-understanding

Awesome Scene Understanding / Survey
Neural Fields in Robotics: A Survey
Advances in Data-Driven Analysis and Synthesis of 3D Indoor Scenes
State-of-the-art in Automatic 3D Reconstruction of Structured Indoor Environments
[project]
Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
RGBD Datasets: Past, Present and Future
[project]

Awesome Scene Understanding / Dataset / Realistic Dataset
ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes
[project]
ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data
[code] 669 2 months ago
Zillow Indoor Dataset: Annotated Floor Plans With 360Ëš Panoramas and 3D Room Layouts
[code] 176 almost 2 years ago
HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures
[project]
OASIS: A Large-Scale Dataset for Single Image 3D in the Wild
[project]
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera
[project]
The Replica Dataset: A Digital Replica of Indoor Spaces
[code] 1,016 4 months ago
Matterport3D: Learning from RGB-D Data in Indoor Environments
[project]
Joint 2D-3D-Semantic Data for Indoor Scene Understanding
[project]
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
[project]
SceneNN: a Scene Meshes Dataset with aNNotations
[project]
SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite
[project]
SUN3D: A Database of Big Spaces Reconstructed using SfM and Object Labels
[project]
Indoor Segmentation and Support Inference from RGBD Images
[project]

Awesome Scene Understanding / Dataset / Synthetic Dataset
Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation
[project]
R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding
[project]
FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenes
GeoSynth: A Photorealistic Synthetic Indoor Dataset for Scene Understanding
[code] 40 12 months ago
MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis
[project]
3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics
[project]
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
[project]
OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
[project]
Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling
[project]
InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset
[project]
SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?
[project]
Semantic Scene Completion from a Single Depth Image
SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
[project]
The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes
[project]

Awesome Scene Understanding / Holistic Scene Understanding / Perspective Image
Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture
[project]
Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes
[code] 106 3 months ago
Holistic 3D Scene Understanding from a Single Image with Implicit Representation
[project]
Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
[code] 423 8 months ago
PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points
Hoilistc++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense
[project]
Complete 3D Scene Parsing from an RGBD Image
Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation
[project]
Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image
[project]
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene
[project]
Im2CAD
[project]
DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding
[project]
Emptying, Refurnishing, and Relighting Indoor Spaces
[project]
Scene Parsing by Integrating Function, Geometry and Appearance Models
Understanding Indoor Scenes using 3D Geometric Phrases
Recovering Free Space of Indoor Scenes from a Single Image
Efficient Exact Inference for 3D Indoor Scene Understanding
Efficient Structured Prediction for 3D Indoor Scene Understanding
Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces
Thinking Inside the Box: Using Appearance Models and Context Based on Room Geometry

Awesome Scene Understanding / Holistic Scene Understanding / Panoramic Image
PanoContext-Former: Panoramic Total Scene Understanding with a Transformer
PanelNet: Understanding 360 Indoor Environment via Panel Representation
DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
[code] 89 over 2 years ago
HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features
[Code] 108 almost 2 years ago
Automatic 3D Indoor Scene Modeling from Single Panorama
Pano2CAD: Room Layout From A Single Panorama Image
PanoContext: A Whole-room 3D Context Model for Panoramic Scene Understanding
[project]

Awesome Scene Understanding / Room Layout Estimation / Perspective Image
CAD-Estate 106 about 1 year ago
Matterport3D-Layout
ScanNet-Layout 33 over 4 years ago
Polygon Detection for Room Layout Estimation using Heterogeneous Graphs and Wireframes
[code] 26 10 months ago
ST-RoomNet: Learning Room Layout Estimation From Single Image Through Unsupervised Spatial Transformations
Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image
[code] 103 over 2 years ago
RoomStructNet: Learning to Rank Non-Cuboidal Room Layouts From Single View
GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planes
[Matterport3D Layout Dataset]
Structural Deep Metric Learning for Room Layout Estimation
General 3D Room Layout from a Single View by Render-and-Compare
[project]
Smart Hypothesis Generation for Efficient and Robust Room Layout Estimation
Flat2Layout: Flat Representation for Estimating Layout of General Room Types
Thinking Outside the Box: Generation of Unconstrained 3D Room Layouts
RoomNet: End-to-End Room Layout Estimation
Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation
[project]
A Coarse-to-Fine Indoor Layout Estimation (CFILE) Method
DeLay: Robust Spatial Layout Estimation for Cluttered Indoor Scenes
Learning Informative Edge Maps for Indoor Scene Layout Prediction
Rent3D: Floor-Plan Priors for Monocular Layout Estimation
[project]
Box In the Box: Joint 3D Layout and Object Reasoning from Single Images
Estimating the 3D Layout of Indoor Scenes and its Clutter from Depth Sensors
[project]
Recovering the Spatial Layout of Cluttered Rooms

Awesome Scene Understanding / Room Layout Estimation / Panoramic Image
ZInD 176 almost 2 years ago
MatterportLayout 63 almost 4 years ago
LayoutMP3D 27 over 4 years ago
No More Ambiguity in 360â—¦ Room Layout via Bi-Layout Estimation
Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for 360 Room Layout Reconstruction
iBARLE: imBalance-Aware Room Layout Estimation
GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network 📷
Shape-Net: Room Layout Estimation from Panoramic Images Robust to Occlusion using Knowledge Distillation with 3D Shapes as Additional Inputs
U2RLE: Uncertainty-Guided 2-Stage Room Layout Estimation
Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness
Code 48 12 months ago [ ]
360-MLC: Multi-view Layout Consistency for Self-training and Hyper-parameter Tuning 📷
[Project]
3D Room Layout Estimation from a Cubemap of Panorama Image via Deep Manhattan Hough Transform
[Code] 50 over 1 year ago
3D Room Layout Recovery Generalizing across Manhattan and Non-Manhattan Worlds
PSMNet: Position-aware Stereo Merging Network for Room Layout Estimation 📷
[code] 3 almost 2 years ago
Self-supervised 360Ëš Room Layout Estimation
[code] 13 over 2 years ago
LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Network
Deep3DLayout: 3D Reconstruction of an Indoor Layout from a Spherical Panoramic Image
[project]
Transferable End-to-end Room Layout Estimation via Implicit Encoding
[project]
OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas
[code] 2 over 3 years ago
LED2-Net: Monocular 360Ëš Layout Estimation via Differentiable Depth Rendering
[project]
SSLayout360: Semi-Supervised Indoor Layout Estimation from 360 Panorama
Single-Shot Cuboids: Geodesics-based End-to-end Manhattan Aligned Layout Estimation from Spherical Panoramas
[project]
Manhattan Room Layout Reconstruction from a Single 360 image: A Comparative Study of State-of-the-art Methods
[code] 222 about 3 years ago
Training and Post Processing 3D Room Layout Beyond the Manhattan World Assumption
Joint 3D Layout and Depth Prediction from a Single Indoor Panorama Image
AtlantaNet: Inferring the 3D Indoor Layout from a Single 360 Image Beyond the Manhattan World Assumption
[project]
Corners for Layout: End-to-End Layout Recovery from 360 Images
[project]
DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
[project]
HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation
[code] 325 8 months ago
Layouts from Panoramic Images with Geometry and Deep Learning
[code] 46 over 4 years ago
LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image
[code] 419 almost 4 years ago
Efficient 3D Room Shape Recovery From a Single Panorama
[code] 111 almost 8 years ago

Awesome Scene Understanding / Floorplan
FRI-Net: Floorplan Reconstruction via Room-wise Implicit Representation 🎲
[code] 13 5 months ago
PolyRoom: Room-aware Transformer for Floorplan Reconstruction 🎲
[code] 27 5 months ago
PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models 🎲
[project]
Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries 🎲
[project]
Floorplan Restoration by Structure Hallucinating Transformer Cascades 📷
MVLayoutNet: 3D Layout Reconstruction with Multi-View Panoramas 📷
Extreme Structure From Motion for Indoor Panoramas Without Visual Overlaps 📷
[code] 35 almost 3 years ago
MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans 🎲
Scan2Plan: Efficient Floorplan Generation from 3D Scans of Indoor Scenes 🎲
Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path 🎲
[project]
Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans 📷
[project]
DeepPerimeter: Indoor Boundary Estimation from Posed Monocular Sequences 🎲
FloorNet: A unified framework for floorplan reconstruction from 3D scans 📷
[project]

Awesome Scene Understanding / Floorplan / Floorplan Vectorization
VectorFloorSeg: Two-Stream Graph Attention Network for Vectorized Roughcast Floorplan Segmentation
[code] 44 about 1 year ago
Parsing Line Segments of Floor Plan Images Using Graph Neural Networks
Residential floor plan recognition and reconstruction
Versailles-FP dataset: Wall Detection in Ancient Floor Plans
Deep Floor Plan Recognition using a Multi-task Network with Room-boundary-Guided Attention
[project] 284 10 months ago
CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis
[code] 307 almost 2 years ago
Raster-to-Vector: Revisiting Floorplan Transformation
[project]

Awesome Scene Understanding / Floorplan / Visual Localization
SPVLoc: Semantic Panoramic Viewport Matching for 6D Camera Localization in Unseen Environments
[project]
LaLaLoc++: Global Floor Plan Comprehension for Layout Localisation in Unvisited Environments
[code] 23 about 2 years ago
LASER: LAtent SpacE Rendering for 2D Visual Localization
LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments

Awesome Scene Understanding / Primitive / Junction
Manhattan Junction Catalogue for Spatial Reasoning of Indoor Scenes

Awesome Scene Understanding / Primitive / Line Segment and Wireframe
Volumetric Wireframe Parsing from Neural Attraction Fields 📷
code 55 8 months ago [ ]
NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction from Multi-view Images 📷
[project]
DeepLSD: Line Segment Detection and Refinement with Deep Image Gradients
[Code] 488 5 days ago
Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning
Learning to Construct 3D Building Wireframes from 3D Line Clouds 🎲
[Code] 41 almost 2 years ago
HoW-3D: Holistic 3D Wireframe Perception from a Single Image
[Code] 37 over 2 years ago
Semantic Room Wireframe Detection from a Single View
[code] 72 over 1 year ago
Towards Real-time and Light-weight Line Segment Detection
[code] 541 over 1 year ago
Hole-robust Wireframe Detection
Fully Convolutional Line Parsing
[code] 156 about 2 months ago
ELSD: Efficient Line Segment Detector and Descriptor
SOLD2: Self-supervised Occlusion-aware Line Description and Detection
[code] 548 11 months ago
Line Segment Detection Using Transformers without Edges
[code] 212 5 months ago
PlueckerNet: Learn to Register 3D Line Reconstructions
[code] 37 over 3 years ago
LGNN: A Context-aware Line Segment Detector
TP-LSD: Tri-Points Based Line Segment Detector
[code] 143 about 4 years ago
Deep Hough-Transform Line Priors
[code] 164 5 days ago
Deep Hough Transform for Semantic Line Detection
[code] 345 about 2 years ago
Holistically-Attracted Wireframe Parsing
[code] 301 9 months ago
Learning to Reconstruct 3D Manhattan Wireframes from a Single Image
[code] 70 4 months ago
End-to-End Wireframe Parsing
[code] 506 4 months ago
PPGNet: Learning Point-Pair Graph for Line Segment Detection
[code] 172 over 5 years ago
Learning Attraction Field Representation for Robust Line Segment Detection
[code] 297 over 5 years ago
Novel Single View Constraints for Manhattan 3D Line Reconstruction
Learning to Parse Wireframes in Images of Man-Made Environments
[code] 217 over 2 years ago
A Novel Linelet-Based Representation for Line Segment Detection
MCMLSD: A Dynamic Programming Approach to Line Segment Detection
Lifting 3D Manhattan Lines from a Single Image
LSD: A Fast Line Segment Detector with a False Detection Control

Awesome Scene Understanding / Primitive / Outdoor Architecture
HEAT: Holistic Edge Attention Transformer for Structured Reconstruction
[Project]
Structured Outdoor Architecture Reconsruction by Exploration and Classification
[Project]
Roof-GAN: Learning to Generate Roof Geometry and Relations for Residential Houses
[Code] 41 over 3 years ago
Vectorizing World Buildings: Planar Graph Reconstruction by Primitive Detection and Relationship Inference
[Project]
Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction
[Project]

Awesome Scene Understanding / Primitive / Plane
MonoPlane: Exploiting Monocular Geometric Cues for Generalizable 3D Plane Reconstruction
[code] 10 29 days ago
UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos 📷
AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings 📷
[project]
PlaneRecTR: Unified Query learning for 3D Plane Recovery from a Single View
[Code] 24 3 months ago
NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstruction 📷
[Code] 65 about 1 year ago
PlaneFormers: From Sparse View Planes to 3D Reconstruction 📷
[project]
PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos 📷
[Project]
PlaneRecNet: Multi-Task Learning with Cross-Task Consistency for Piece-Wise Plane Detection and Reconstruction from a Single RGB Image
[code] 79 over 2 years ago
PlaneTR: Structure-Guided Transformers for 3D Plane Recovery
[code] 93 almost 2 years ago
Planar Surface Reconstruction From Sparse Views 📷
[project]
Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
[code] 52 about 3 years ago
Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction
[code] 40 over 3 years ago
Peek-a-Boo: Occlusion Reasoning in Indoor Scenes with Plane Representations
[project]
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding
[code] 364 10 months ago
PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image
[project]
Recovering 3D Planes from a Single Image via Convolutional Neural Networks
[code] 96 over 2 years ago
PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
[project]

Awesome Scene Understanding / Vanishing Point
Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction
[code] 99 27 days ago
Transformer Based Line Segment Classifier with Image Context for Real-Time Vanishing Point Detection in Manhattan World
Deep Vanishing Point Detection: Geometric Priors Make Dataset Variations Vanish
VaPiD: A Rapid Vanishing Point Detector via Learned Optimizers
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution
[Code] 179 4 months ago

Backlinks from these awesome lists: