Stable diffusion: resources and discussion

Got questions, comments, links, or want to chat about Stable Diffusion? Do it here! Here’s some links to help get you started:

40 Likes

Shameless plug: Diffusion Models [Aakash Nain, Sayak Paul, Rishabh]

12 Likes

Here’s something cool that @yiyimarz did too:

14 Likes

Thanks for mentioning my project here! I feel so honored:) I’ve learned so much from fast.ai courses and am super excited to join this upcoming class!

10 Likes

Concepts like textual inversion or dreambooth to me are possibly the most exciting and powerful extensions to stable diffusion. Being able to inject custom representations (with only 3-5 images!) into the text-to-image model and optimizing towards seemingly any novel concept provides incredible control over content generation.

7 Likes

This subject is really interesting, given the vast amount of links here and within links to further papers/ pages perhaps a score process could be defined to indicate the ones that explain the subject best in the simplest terms, for me so far I give a vote for ‘Lilian Weng’, the ‘Yang Song’ intrigues me by the title only so far so next on my reading list.

After reading Yang Song paper I give that a vote also, note the first link in Jeremy’s original post is from that paper. In this paper Yang Song writes a commentary on the connections to different models in this sphere of knowledge.

I must comment that I am new to this topic and my suggestions are only that. Each link above reveals the number of it’s access clicks but does not say how useful the experience has been to the reader.

5 Likes

I got interested two weeks ago after I chanced upon @ilovescience 's youtube Diffusion Study Group #1 - EleutherAI ; so glad this is happening :slight_smile:

9 Likes

I found this twitter thread particularly useful at giving me a good high level view of SD, and particularly around the “latent diffusion” idea, together with this other which is linked in the original one.

It gave me a good idea of the pieces involved and a basis for what the endgame is

8 Likes

Is someone implementing SD training/finetunning on fastai?

6 Likes

I like Outlier’s videos a lot; the previous one on diffusion was very good too, but requires a lot of let-me-rewind-20s-and-listen-to-that-again.
Going to watch this new one, thanks!

1 Like
4 Likes
2 Likes

Just wanted to highlight a few cool new diffusion model techniques released this week!

MAKE-A-VIDEO: TEXT-TO-VIDEO GENERATION WITHOUT TEXT-VIDEO DATA
paper: https://makeavideo.studio/Make-A-Video.pdf
project: https://makeavideo.studio/

TRAINING-FREE STRUCTURED DIFFUSION GUIDANCE FOR COMPOSITIONAL TEXT-TO-IMAGE SYNTHESIS
paper: https://openreview.net/pdf?id=PUIqjT4rzq7
key idea: Improve SD prompt-adherence using cross-attention

DREAMFUSION: TEXT-TO-3D USING 2D DIFFUSION
paper: https://openreview.net/pdf?id=FjNys5c7VyY
project: https://dreamfusionpaper.github.io/
key idea: diffusion model for text to 3d

7 Likes

I thought this article about using it for compression was super interesting as well!

6 Likes

How do you finetune SD? use dreambooth?

1 Like

After seeing this tweet by Tanishq I knew something had to be done…

Using the HF SD Dreambooth training collab I added Jeremy as a new concept to SD. It took just 5 images of Jeremy that I found online, < 5 mins of total training time on a V100, and < 10s per image to generate these.

jeremy_rendering





24 Likes

Just wondering are these pre-requisites for the course? :sweat_smile: Or simply just posting some links ahead of time?

1 Like

simply just posting some links ahead of time

7 Likes

TabDDPM: Modelling Tabular Data with Diffusion Models

5 Likes

This link does not seem to work

Here is a snapshot from 27 september

2 Likes