RIML Lab

💠 Compositional Learning Journal Club

✅ This Week's Presentation:

🔹 Title: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching

🔸 Presenter: Arash Marioriyad

🌀 Abstract:
Diffusion models have achieved significant success in text-to-image generation. However, alleviating the misalignment between text prompts and generated images remains a challenging issue.
This presentation will focus on two observed causes of misalignment: concept ignorance and concept mis-mapping. To address these issues, we will discuss CoMat, an end-to-end diffusion model fine-tuning strategy that uses an image-to-text concept matching mechanism.
Using only 20K text prompts to fine-tune SDXL, CoMat significantly outperforms the baseline SDXL model on two text-to-image alignment benchmarks, achieving state-of-the-art performance.

📄 Paper:
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching

Session Details:
- 📅 Date: Sunday, 8 September 2024
- 🕒 Time: 3:30 - 5:00 PM (GMT+3:30)
- 🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

www.tg-me.com/ca/telegram/com.RIMLLab/131

3.8K viewsArash Marioriyad, Sep 6, 2024 at 18:55

💠 Compositional Learning Journal Club

✅ This Week's Presentation:

🔹 Title: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching

🔸 Presenter: Arash Marioriyad

🌀 Abstract:
Diffusion models have achieved significant success in text-to-image generation. However, alleviating the misalignment between text prompts and generated images remains a challenging issue.
This presentation will focus on two observed causes of misalignment: concept ignorance and concept mis-mapping. To address these issues, we will discuss CoMat, an end-to-end diffusion model fine-tuning strategy that uses an image-to-text concept matching mechanism.
Using only 20K text prompts to fine-tune SDXL, CoMat significantly outperforms the baseline SDXL model on two text-to-image alignment benchmarks, achieving state-of-the-art performance.

📄 Paper:
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching

Session Details:
- 📅 Date: Sunday, 8 September 2024
- 🕒 Time: 3:30 - 5:00 PM (GMT+3:30)
- 🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

BY RIML Lab