Stable diffusion 1.5 was released today runwayml/stable-diffusion-v1-5 · Hugging Face . But it is in ckpt file format. Any idea how to load it using diffusers api ?
Have a look at lessons 9A and 10, where we do exactly that. Also, try clicking the “Use in Diffusers” button on the page you linked to.
Let us know if you have any questions or issues!
Just as a note, it’s my understanding that it’s not really the 1.5 model that Stability AI is working with This is the original Stability 1.2 model further trained by RunwayML to provide better inpainting support.
But as Jeremy mentioned, it is covered in the videos but there’s also a code sample on how to use the model in the model card page itself here:
I tried running the lesson 9 notebook with this 1.5 release and didn’t see any improvements in outputs FYI.
It does seem to work better for inpainting (at least based on videos I saw on YouTube) but I don’t believe generated image quality would be improved since it’s based on the StabilityAI 1.2 model rather than the 1.4 model which is what almost everybody is using …
Ok so loading with string 'runwayml/stable-diffusion-v1-5". I was wondering if there is a way to load from the ckpt files also. I want to load some fine tuned dreambooth model using diffuser library.
I believe the original CompVIS code loads the .ckpt file directly. You should be able to find out how they did that by looking at the source code here:
Also, it appears that I was wrong about the Runway model being the 1.2 Stability AI model that was trained further by RunwayML. That’s what the model card said (and still says) but there are other claims that this is StabilityAI’s 1.5 model or that it’s a beta of StabilityAI’s 1.5 model. I have no idea which one it is but I guess we’ll eventually find out