Auto-ARM: An Autonomous Adaptive Mask Refinement Mechanism for Enhancing Naturalness in Virtual Try-On Models

Abstract

Virtual Try-on (VTON or VITON) technology has become a cornerstone of e-commerce, offering users an immersive and personalized shopping experience. Recent advancements in diffusion models have improved the quality of try-on images. However, these models still rely heavily on the accuracy of input try-on masks, which are the masked regions used to instruct VTON models to generate the target clothing on. Furthermore, such approaches apply rule-based methods to produce try-on masks, which lack flexibility and can lead to distortion or incomplete clothing replacement, especially with unnatural poses or mixed clothes. To address these limitations, we introduce Auto-ARM, an innovative framework that employs an attention-based U-Net architecture with Attention Gate (AG) to dynamically refine the try-on masks based on the target outfit. This novel approach not only significantly enhances the generalization of mask-dependent VTON models but also delivers superior qualitative and quantitative results. Auto-ARM achieves state-of-the-art performance on benchmarks such as VITON-HD and DressCode, proving its potential for high-quality, real-world VTON applications.