Image Based Virtual Try-On (VITON)

The goal of image based VITON is to fit a given cloth image to a person in a reference image. The majority methods include two steps. First a warping network is trained for aligning the human body and the cloth. Then a generator is applied to fuse the warped cloth and the reference image with target person.

There are two main challenges for image based VITON. The first challenge is the generated images should be realistic and natural enough to avoid dissonance. GAN-based methods usually have difficulty in generating correct garment folds, natural light and shadow, or realistic human bodies. Diffusion models can generate more realistic images. The pre-trained latent diffusion models can be used to ensure high realism of generated images and natural try-on effects. The second critical challenge is how to preserve as much as possible the garment detail features, such as complicated text, textures, colours, patterns and lines. In this article, we selected two diffusion based methods, i.e. StableVITON and OOTDiffusion.

StableVITON[1]

Large pre-trained diffusion models show promising performance on generalisation. They are also used for virtual try-on task. A significant challenge is to preserve the cloth details while use the knowledge of the pre-trained diffusion model. [1] proposed to learn the semantic correspondence between…