ResAdapter : Domain Consistent Resolution Adapter for Diffusion Models

Recent advancement in text-to-image models (e.g., Stable Diffusion) and corresponding personalized technologies (e.g., DreamBooth and LoRA) enables individuals to generate high-quality and imaginative images. However, they often suffer from limitations when generating images with resolutions outside of their trained domain. To overcome this limitation, we present the Resolution Adapter (ResAdapter), a domain-consistent adapter designed for diffusion models (e.g., SD and the personalized model) to generate images with unrestricted resolutions and aspect ratios. Unlike other multi-resolution generation methods that process images of static resolution with post-process, ResAdapter directly generates images with the dynamical resolution. This perspective enables the efficient inference without repeat denoising steps and complex post-process operations, thus eliminating the additional inference time. Enhanced by a broad range of resolution priors without any style information from trained domain, ResAdapter with 0.5M generates images with out-of-domain resolutions for the personalized diffusion model while preserving their style domain. Comprehensive experiments demonstrate the effectiveness of ResAdapter with diffusion models in resolution interpolation and exportation. More extended experiments demonstrate that ResAdapter is compatible with other modules (e.g., ControlNet, IP-Adapter and LCM-LoRA) for images with flexible resolution, and can be integrated into other multi-resolution model (e.g., ElasticDiffusion) for efficiently generating higher-resolution images.