OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision

♠️Cong Wei, ♣Zheyang Xiong, ♠️Weiming Ren, †Xinrun Du, ♠️Ge Zhang, ♠️Wenhu Chen
♠️University of Waterloo, ♣University of Wisconsin Madison, †M-A-P,
Paper 🤗 Training Data(OmniEdit-Filtered-1.2M) Code

OMNI-EDIT

We present OMNI-EDIT, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly. Our contribution is in two folds:

(1) OMNI-EDIT is trained by utilizing the supervision from seven different specialist models to ensure task coverage.

(2) we utilize importance sampling based on the scores provided by large multimodal models (like GPT-4o) instead of CLIP-score to improve the data quality.

OmniEdit

Overview of the OMNI-EDIT training pipeline. The pipeline consists of four stages: (1) task-specific specialist models are trained for diverse editing tasks; (2) these specialist models are used to generate a large, high-resolution, multi-aspect-ratio dataset; (3) a cost-efficient distilled large multi-modal model (LMM) assigns importance weights to each pair of image-editing data; and (4) the final generalist model is trained on the weighted dataset.

OMNI-EDIT-Filtered-1.2M

We synthesize the large scale dataset through specialist distillation and cost-efficient LMM filtering. Our released version contains 1.2M pairs covering seven different skills like addition, swaping, removal, attribute modification, background change, environment change and sytle transfer. The dataset has been filtered with VIEScore.

OmniEdit

Our dataset has the most diverse, highest-quality image editing pairs of any resolution.

OmniEdit

Editing high-resolution multi-aspect images with OMNI-EDIT. OMNI-EDIT is an instruction-based image editing generalist capable of performing diverse editing tasks across different aspect ratios and resolutions. It accurately follows instructions while preserving the original image’s fidelity.

Citation

If you find our paper useful, please cite us with: 


@article{wei2024omniedit,
  title={OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision},
  author={Wei, Cong and Xiong, Zheyang and Ren, Weiming and Du, Xinrun and Zhang, Ge and Chen, Wenhu},
  journal={arXiv preprint arXiv:2411.07199},
  year={2024}
}