Several Issues while trying the Stable Diffusion deep Dive code from Lesson 9A

I was trying to replicate the code from the Stable diffusion deep dive code. And when I tried to run it locally on my GTX 1650ti 4gb ram it kind of worked but it gave me an empty image and a warning

RuntimeWarning: invalid value encountered in cast images = (image * 250).round().astype(“uint8”)

The Github link to my code with the black image generated: Link

The only change I made was to add “.to(torch_device)” to t and latents as it was giving me an error(some variables are not on the same device). This is on the last line of the loop. Everything else is same.

with autocast('cuda'):
    for i, t in tqdm(enumerate(scheduler.timesteps)):
        latent_model_input =[latents] *2)
        sigma = scheduler.sigmas[i]
        latent_model_input = scheduler.scale_model_input(latent_model_input, t)
        latent_model_input =
        # to preduct residual noise
        with torch.no_grad():
            noise_pred = unet(latent_model_input, t, encoder_hidden_states = text_embeddings).sample
        #perform guidance
        noise_pred_uncond, noise_pred_text = noise_pred.chunk(2)
        noise_pred = noise_pred_uncond + guidance_scale * (noise_pred_text - noise_pred_uncond)
        latents = scheduler.step(noise_pred,,

Then I tried to run the original code on a free paperspace machine and it gave me an Out of Memory error. Even though, it had 8gbs of RAM. What should I do to make it run anywhere?

Hi @Adhe11

I am also facing the same issue. Have you figured out what the issue is and resolve it?

If yes, let me know how did you resolve it