This was part of Computational Imaging

Diffusion in Style

Sabine Süsstrunk, EPFL

Friday, August 9, 2024



Slides
Abstract: Our Diffusion in Style (ICCV 2023, WACV 2024) method adapts Stable Diffusion to any desired style using only a small set of target images. It is based on the key observation that the style of the images generated by Stable Diffusion is tied to the initial latent tensor,i.e., the initial noise the images are generated with. Not adapting this initial latent tensor to the style makes fine-tuning slow, expensive,and impractical, especially when only a few target style images are available. In contrast, fine-tuning is much easier if the initial latenttensor is also adapted. Our Diffusion in Style is orders of magnitude more sample-efficient and faster. It also generates more pleasing images than existing approaches.