In this episode of the Talking Papers Podcast, I hosted Dejan Azinović to chat about his paper "Neural RGB-D Surface Reconstruction”, published in CVPR 2022.
In this paper, they take on the task of RGBD surface reconstruction by using novel view synthesis. They incorporate depth measurements into the radiance field formulation by learning a neural network that stores a truncated signed distance field. This formulation is particularly useful in regions where depth is missing and the color information can help fill in the gaps.
PAPER TITLE
"Neural RGB-D Surface Reconstruction"
AUTHORS
Dejan Azinović Ricardo Martin-Brualla Dan B Goldman Matthias Nießner Justus Thies
ABSTRACT
In this work, we explore how to leverage the success of implicit novel view synthesis methods for surface reconstruction. Methods which learn a neural radiance field have shown amazing image synthesis results, but the underlying geometry representation is only a coarse approximation of the real geometry. We demonstrate how depth measurements can be incorporated into the radiance field formulation to produce more detailed and complete reconstruction results than using methods based on either color or depth data alone. In contrast to a density field as the underlying geometry representation, we propose to learn a deep neural network which stores a truncated signed distance field. Using this representation, we show that one can still leverage differentiable volume rendering to estimate color values of the observed images during training to compute a reconstruction loss. This is beneficial for learning the signed distance field in regions with missing depth measurements. Furthermore, we correct for misalignment errors of the camera, improving the overall reconstruction quality. In several experiments, we show-cast our method and compare to existing works on classical RGB-D fusion and learned representations.
RELATED PAPERS
📚 NeRF
📚 BundleFusion
LINKS AND RESOURCES
💻 Project Page
💻 Code
To stay up to date with Dejan's latest research, follow him on:
👨🏻🎓 Dejan's personal page
🎓 Google Scholar
🐦 Twitter
👨🏻🎓LinkedIn:
Recorded on April 4th 2022.
CONTACT
If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: talking.papers.podcast@gmail.com
SUBSCRIBE AND FOLLOW
🎧Subscribe on your favourite podcast app
📧Subscribe to our mailing list
🐦Follow us on Twitter
🎥YouTube Channel
#talkingpapers #CVPR2022 #NeuralRGBDSurfaceReconstruction #SurfaceReconstruction #NeRF #3DVision #ComputerVision #AI #DeepLearning #MachineLearning #deeplearning #AI #neuralnetworks #research #artificialintelligence
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com
📧Subscribe to our mailing list: http://eepurl.com/hRznqb
🐦Follow us on Twitter: https://twitter.com/talking_papers
🎥YouTube Channel: https://bit.ly/3eQOgwP