# cv_practive2 **Repository Path**: limengxu/cv_practive2 ## Basic Information - **Project Name**: cv_practive2 - **Description**: virtual try on project - **Primary Language**: Python - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2022-05-24 - **Last Updated**: 2022-06-06 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Inference using colab # README **Note:** To finish our second project of CV lessons, we modified the implementation of the paper : [DeepFashion_Try_On (ACGPN)](https://github.com/switchablenorms/DeepFashion_Try_On) Please refer to the original repository for details. If there are any copyright infringement, please let us know. [[Dataset Partition Label]](https://drive.google.com/open?id=1Jt9DykVUmUo5dzzwyi4C_1wmWgVYsFDl) [[Sample Try-on Video]](https://www.youtube.com/watch?v=BbKBSfDBcxI) [[Checkpoints]](https://drive.google.com/file/d/1UWT6esQIU_d4tUm8cjxDKMhB8joQbrFx/view?usp=sharing) [[Dataset_Test]](https://drive.google.com/file/d/1tE7hcVFm8Td8kRh5iYRBSDFdvZIkbUIR/view?usp=sharing) [[Dataset_Train]](https://drive.google.com/file/d/1lHNujZIq6KVeGOOdwnOXVCSR5E7Kv6xv/view?usp=sharing) [[Paper]](https://arxiv.org/abs/2003.05863) ## How to run it to obtain the virtual try-on function, only with ACGPN is not enough. We need to preprocess the given images and then feed them into the ACGPN model. ### optional: - download the [testing dataset](https://drive.google.com/uc?id=1tE7hcVFm8Td8kRh5iYRBSDFdvZIkbUIR) from google drive and unzip it under the `Data_preprocessing` folder. ### mandatory: - download about 200M sized file named "pose_iter_440000.caffemodel" from [google drive](https://drive.google.com/uc?id=1hOHMFHEjhoJuLEQY0Ndurn5hfiA9mwko), and put it under the `pose` folder. this file is a pretrained model used by [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose), a real-time multi-person key-point detection library for body, face, hands, and foot estimation. We will use it to predict poses and generate pose key-points from the model image. - download about 255M sized file named "lip_final.pth" from [google drive](https://drive.google.com/file/d/1E5YwNKW2VOEayK9mWCS3Kpsxf-3z04ZE/view?usp=sharing) and put it under the working space directly. this file is another one of the pretrained models of [Self Correction for Human Parsing](https://github.com/GoGoDuck912/Self-Correction-Human-Parsing) , an out-of-box human parsing representation extractor. - download a 4.7M sized file named "u2netp.pth" from [google drive](https://drive.google.com/file/d/1rbSTGKAE-MTxBYHd-51l2hMOQPT_7EPy/view?usp=sharing) and a 173.6M sized file named "u2net.pth" from [google drive](https://drive.google.com/file/d/1ao1ovG1Qtx4b7EoskHXmi2E9rp5CHLcZ/view?usp=sharing), then put it separately under the `U_2_Net/saved_models/u2netp` and `U_2_Net/saved_models/u2net` folder. These are two pretrained model of [U2Net](https://github.com/xuebinqin/U-2-Net) , a simple yet powerful deep network architecture for salient object detection (SOD). - download our ACGPN pretrained model , a 499M sized file, from [google drive](https://drive.google.com/uc?id=1UWT6esQIU_d4tUm8cjxDKMhB8joQbrFx). If you want to train your own model, please refer to the original github repository as showed above. This pretrained model is based on the VITON dataset, and the results of are only guaranteed in VITON dataset only. Then, unzip `ACGPN_checkpoints.zip` under the `checkpoints` folder. - put your model image under the `Data_preprocessing/inputs/img` directory, please limit the number to **1**. - put your cloth image under the `Data_preprocessing/inputs/cloth` directory,please limit the number to **1**. - execute the script: `U_2_Net/process.py` - check out the results in the `results` folder. ### note: - Due to hardware shortage, our work does not support cuda, so we alter the original code for the use of cpu. - we disable the multithread function of the U2net because of some strange errors during calling `infer` function on line 20 in file `process.py`. To enable it, just set the `num_workers`parameter to a positive number on line 65 in file `u2net_run.py` - you can also try it on colab: [colab-badge]: [![Open In Colab][colab-badge]](https://colab.research.google.com/github/levindabhi/ACGPN/blob/master/ACGPN_and_SieveNet.ipynb) ## License The use of this software is RESTRICTED to **non-commercial research and educational purposes**. ## Citation ``` @InProceedings{Yang_2020_CVPR, author = {Yang, Han and Zhang, Ruimao and Guo, Xiaobao and Liu, Wei and Zuo, Wangmeng and Luo, Ping}, title = {Towards Photo-Realistic Virtual Try-On by Adaptively Generating-Preserving Image Content}, booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2020} } ``` ## Dataset **VITON Dataset** This dataset is presented in [VITON](https://github.com/xthan/VITON), containing 19,000 image pairs, each of which includes a front-view woman image and a top clothing image. After removing the invalid image pairs, it yields 16,253 pairs, further splitting into a training set of 14,221 paris and a testing set of 2,032 pairs. You can find the VITON Dataset [here](https://www.kaggle.com/datasets/rkuo2000/viton-dataset). ## PS for more questions, please contact us: 191250167@smail.nju.edu.cn