WACV-2023 MixGen：A New Multi-Modal Data Augmentation
MixGen: Multimodal Data Augmentation via Image Linear Interpolation and Text Concatenation
Abstract
The paper proposes MixGen, a joint data-augmentation method for vision–language representation learning that forms new image–text pairs by interpolating images and concatenating captions while preserving their semantic alignment. The approach is evaluated on four frameworks: CLIP, ViLT, ALBEF, and TCL.
Introduction
Augmenting image–text pairs is difficult because common transforms can drop information or introduce ambiguity—for instance, after flipping an image, its original caption may no longer be appropriate. Effective augmentation therefore requires changes that retain information and do not alter semantics.
MixGen
Two images are linearly interpolated (blended), and their corresponding sentences are concatenated to produce new training samples for augmentation.
Experiments
<HR align=left color=#987cb9 SIZE=1>