Neural Rerendering in the Wild

scene under varying appearance

• render scene into deep buffer, render buffers(neural textures) to photos
• introducing appearance vector and semantic mask
  • seperate vp, appearance conditions and transient objects (auto decoder framework)
  • pretrain encoder for stable training, jointly finetune both networks
  • train a seperate semantic labeling network