Awesome-Generative-Image-Composition

COCOEE

(within-domain, single-ref): 500 background images from MSCOCO validation set. Each background image has a bounding box and a foreground image from MSCOCO training set

TF-ICON test benchmark

(cross-domain, single-ref): 332 samples. Each sample consists of a background image, a foreground image, a user mask, and a text prompt

FOSCom

(within-domain, single-ref): 640 background images from Internet. Each background image has a manually annotated bounding box and a foreground image from MSCOCO training set

(within-domain, multi-ref): 220 background images and 30 unique foreground objects from 15 categories

MureCom

(within-domain, multi-ref): 640 background images and 96 unique foreground objects from 32 categories

Begin by installing the dependencies listed in

Segment Anything

Additionally, install

: Used for CLIP score and FID score calculations

ViT-H SAM model

: Utilized to estimate foreground masks for reference images and generated composites

: Employed in DINO score computation

coco2017_gmm_k20

: Utilized to compute the overall quality score

COCOEE benchmark

Prepare the alongside your generated composite results. Ensure that your composite images have filenames corresponding to the background images of the COCOEE dataset, as illustrated below:

Pengzhi Li, Qiang Nie, Ying Chen, Xi Jiang, Kai Wu, Yuhuan Lin, Yong Liu, Jinlong Peng, Chengjie Wang, Feng Zheng: " " arXiv preprint arXiv:2403.12658 (2024)

Yicheng Yang, Pengxiang Li, Lu Zhang, Liqian Ma, Ping Hu, Siyu Du, Yunzhi Zhuge, Xu Jia, Huchuan Lu: " " arXiv preprint arXiv:2411.17223 (2024)

Shaoan Xie, Yang Zhao, Zhisheng Xiao, Kelvin C.K. Chan, Yandong Li, Yanwu Xu, Kun Zhang, Tingbo Hou: " " arXiv preprint arXiv:2312.03771 (2023)

Yulin Pan, Chaojie Mao, Zeyinzi Jiang, Zhen Han, Jingfeng Zhang: " " arXiv preprint arXiv:2403.19534 (2024)

Yibin Wang, Weizhong Zhang, Jianwei Zheng, Cheng Jin: " " ACM MM (2024)

Shilin Lu, Yanzhu Liu, Adams Wai-Kin Kong: " " ICCV (2023)

Roy Hachnochi, Mingrui Zhao, Nadav Orzech, Rinon Gal, Ali Mahdavi-Amiri, Daniel Cohen-Or, Amit Haim Bermano: " " arXiv preprint arXiv:2302.10167 (2023)

Zitian Zhang, Frederic Fortier-Chouinard, Mathieu Garon, Anand Bhattad, Jean-Francois Lalonde: " " arXiv preprint arXiv:2410.08168 (2024)

Zhekai Chen, Wen Wang, Zhen Yang, Zeqing Yuan, Hao Chen, Chunhua Shen: " " ECCV (2024)

Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, Yedid Hoshen: " " ECCV (2024)

Gemma Canet Tarrés, Zhe Lin, Zhifei Zhang, Jianming Zhang, Yizhi Song, Dan Ruta, Andrew Gilbert, John Collomosse, Soo Ye Kim：" " ECCV (2024)

Weijing Tao, Xiaofeng Yang, Biwen Lei, Miaomiao Cui, Xuansong Xie, Guosheng Lin: " " arXiv preprint arXiv:2409.10090 (2024)

Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, He Zhang, Wei Xiong, Daniel Aliaga: " " CVPR (2024)

Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, Hengshuang Zhao: " " CVPR (2024)

Vishnu Sarukkai, Linden Li, Arden Ma, Christopher Re, Kayvon Fatahalian: " " WACV (2024)

Ziyang Yuan, Mingdeng Cao, Xintao Wang, Zhongang Qi, Chun Yuan, Ying Shan: " " ACM MM (2024)

Bo Zhang, Yuxuan Duan, Jun Lan, Yan Hong, Huijia Zhu, Weiqiang Wang, Li Niu: " " arXiv preprint arXiv:2308.10040 (2023)

Xin Zhang, Jiaxian Guo, Paul Yoo, Yutaka Matsuo, Yusuke Iwasawa: " " arXiv preprint arXiv:2306.07596 (2023)

Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, Fang Wen: " " CVPR (2023)

Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, Daniel Aliaga: " " CVPR (2023)

Sumith Kulal, Tim Brooks, Alex Aiken, Jiajun Wu, Jimei Yang, Jingwan Lu, Alexei A. Efros, Krishna Kumar Singh: " " CVPR (2023)

Lingxiao Lu, Bo Zhang, Li Niu: " " arXiv preprint arXiv:2309.15508 (2023)

Tianle Li, Max Ku, Cong Wei, Wenhu Chen: " " TMLR (2023)

Jinghao Zhou, Tomas Jakab, Philip Torr, Christian Rupprecht: " " arXiv preprint arXiv:2312.12419 (2023)

Mohamad Shahbazi, Liesbeth Claessens, Michael Niemeyer, Edo Collins, Alessio Tonioni, Luc Van Gool, Federico Tombari: " " arXiv preprint arXiv:2401.05335 (2024)

Rahul Goel, Dhawal Sirikonda, Saurabh Saini, PJ Narayanan: " " CVPR (2023)

Rahul Goel, Dhawal Sirikonda, Rajvi Shah, PJ Narayanan: " " CVPR Workshop (2023)

Verica Lazova, Vladimir Guzov, Kyle Olszewski, Sergey Tulyakov, Gerard Pons-Moll: " " WACV (2023)

Jiaxiang Tang, Xiaokang Chen, Jingbo Wang, Gang Zeng: " " NIPS (2022)

Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, Zhaopeng Cui: " " ICCV (2021)

Boxiao Pan, Zhan Xu, Chun-Hao Paul Huang, Krishna Kumar Singh, Yang Zhou, Leonidas J. Guibas, Jimei Yang: " " arXiv preprint arXiv:2401.10822 (2024)