Presenter: Zhengzhong Tu, Assistant Professor, Texas A&M University

Abstract: Large generative diffusion models have revolutionized text-to-image generation and offer immense potential for conditional generation tasks such as image enhancement, restoration, and editing. However, their adoption is hindered by the high computational cost during training and adaptation, as well as the difficulty to control the generation process. In this talk, I will first introduce our previous works on efficient modeling of image enhancement problems, followed by our recent progress to leverage pre-trained diffusion models to solve multiple image restoration problems. Moreoever, we will also present our conditional diffusion distillation frameworks for higher-fidelity and faster image generation. To fundamentally investigate the efficiency bottleneck, we have studied the scaling properties of latent diffusion models with an emphasis on their sampling efficiency. Our findings unveil a surprising trend: when operating under a given inference budget, smaller models frequently outperform their larger equivalents in generating high-quality results. We hope our preliminary studies on diffusion model adaptation and controllability can facilitate wider applications on real-world products.

Bio: Zhengzhong Tu is currently an Assistant Professor of Computer Science at Texas A&M University. His research interests are generative AI, multimodal AI, and their real-world applications such as computational photography, autonomous driving, and robotics. Previously, he was a research engineer at Google Research, working on on-device computational imaging and machine learning. Before that, he spent four years in his PhD studies at University of Texas at Austin, advised by Professor Alan Bovik. He received his Master’s and Bachelor’s degrees from Fudan University. Zhengzhong has authored over twenty papers in top-tier computer vision and machine learning venues. His contributions have been acknowledged through various recognitions, such as the CVPR 2022 Best Paper Nomination Award, highlights in Google Research’s annual blog, and featuring in Google I/O media outlets.

Democratizing Diffusion Models