Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing


Aiyu Cui
Daniel McKee
Svetlana Lazebnik

University of Illinois at Urbana-Champaign

ICCV 2021

[Paper]
[Talk]
[Slides]
[Bibtex]
[Github]


News


Abstract

This paper proposes a flexible person generation framework called Dressing in Order (DiOr), which supports 2D pose transfer, virtual try-on, and several fashion editing tasks. Key to DiOr is a novel recurrent generation pipeline to sequentially put garments on a person, so that trying on the same garments in different orders will result in different looks. Our system can produce dressing effects not achievable by existing work, including different interactions of garments (e.g., wearing a top tucked into the bottom or overit), as well as layering of multiple garments of the same type (e.g., jacket over shirt over t-shirt). DiOr explicitly encodes the shape and texture of each garment, enabling these elements to be edited separately. Joint training on pose transfer and inpainting helps with detail preservation and coherence of generated garments. Extensive evaluations show that DiOr outperforms other recent methods like ADGAN in terms of output quality, and handles a wide range of editing functions for which there is no direct supervision.

Selected applications that DiOr supports:

Pose-guided person generation   Tucking in (try-on)   Content removal   Texture transfer  
Garment layering (try-on)   Content insertion   Garment reshaping

Dataset and Preprocessing

We use deepfashion dataset (inshop). We obtain pose estimations by OpenPose and human parsing by SCHP.

This model can be trained on a single 12GB Graphic card for 256x176 images within 1-2 days.



Try-on Applications

DiOr supports a couple of novel virtual try-on applications, including:

Tucking in

Users can control whether they want to tuck in or not by specifying the dressing order.

Garment Layering

Our DiOr system supports users to both add base layers under the existing outfits and keep layering garments outside existing outfits. Users can achieve different looks by different dressing order to freely express their fashion tastes!

Editing Applications

DiOr also supports a number of fashion editing tasks, including:

Content Removal

Users can remove the unwanted contents on garments (e.g., logos) with DiOr system!

Print Insertion

Users can insert external prints (e.g. doge dog or Bulbasaur Pokemon) on the outfits!

Texture Transfer

Don't like the garment texture? Swap it with another texture from garments or wild pictures!

Reshaping

Not happy with the garment's shape either? Use DiOr to reshape it into the desired shape!


Check our paper to see what else DiOr can do!



Citation

If you find this work is helpful, please cite us as [Bibtex]



Thanks to Unnat Jain for sharing the template of this project page.