A Variational U-Net for Conditional Appearance and Shape Generation by Patrick Esser, Ekaterina Sutter, Björn Ommer.
Abstract:
Deep generative models have demonstrated great performance in image synthesis. However, results deteriorate in case of spatial deformations, since they generate images of objects directly, rather than modeling the intricate interplay of their inherent shape and appearance. We present a conditional U-Net [30] for shape-guided image generation, conditioned on the output of a variational autoencoder for appearance. The approach is trained end-to-end on images, without requiring samples of the same object with varying pose or appearance. Experiments show that the model enables conditional image generation and transfer. Therefore, either shape or appearance can be retained from a query image, while freely altering the other. Moreover, appearance can be sampled due to its stochastic latent representation, while preserving shape. In quantitative and qualitative experiments on COCO [20], DeepFashion [21, 23], shoes [43], Market-1501 [47] and handbags [49] the approach demonstrates significant improvements over the state-of-the-art.
The abstract fails to convey the results described in the paper. Try:
The animated versions are based on the single image on the left.
There is a Github site with training data: https://github.com/CompVis/vunet which carries this short description:
The model learns to infer appearance from a single image and can synthesize images with that appearance in different poses.
My answer to anyone who objects to Sarah Huckabee Sanders or other members of the current regime in Washington being the subjects of this technique: Jim Acosta video.
This is war friends and you don’t win wars by praying for the other side to be more courteous.