Talking Papers Podcast

Guy Gafni - NerFACE

February 03, 2022 Itzik Ben-Shabat Season 1 Episode 4
Talking Papers Podcast
Guy Gafni - NerFACE
Show Notes

Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction

Guy Gafni      Justus Thies      Michael Zollhöfer     Matthias Nießner   

Project page:


We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or head-poses is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photo-realistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.

📚Representing Scenes as Neural Radiance Fields for View Synthesis
📚Deep Video Portraits
📚Nerfies: Deformable Neural Radiance Fields
📚AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis

If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email:


00:07 Intro
00:27 Authors
01:16 Abstract / TLDR
02:54 Motivation
12:24 Related Work
13:20 Approach
17:10 Results
27:05 Conclusions and future work
32:12 Outro

#talkingpapers #CVPR2021 #NeRF
#machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence #FacialAvatars

Recorded on April, 2nd 2021.

🎧Subscribe on your favourite podcast app:
📧Subscribe to our mailing list:
🐦Follow us on Twitter:
🎥YouTube Channel:

🎧Subscribe on your favourite podcast app:

📧Subscribe to our mailing list:

🐦Follow us on Twitter:

🎥YouTube Channel: