I was trying to replicate the code from the Stable diffusion deep dive code. And when I tried to run it locally on my GTX 1650ti 4gb ram it kind of worked but it gave me an empty image and a warning
RuntimeWarning: invalid value encountered in cast images = (image * 250).round().astype(“uint8”)
The Github link to my code with the black image generated: Link
The only change I made was to add “.to(torch_device)” to t and latents as it was giving me an error(some variables are not on the same device). This is on the last line of the loop. Everything else is same.
with autocast('cuda'):
for i, t in tqdm(enumerate(scheduler.timesteps)):
latent_model_input = torch.cat([latents] *2)
sigma = scheduler.sigmas[i]
latent_model_input = scheduler.scale_model_input(latent_model_input, t)
latent_model_input = latent_model_input.to(torch_device)
# to preduct residual noise
with torch.no_grad():
noise_pred = unet(latent_model_input, t, encoder_hidden_states = text_embeddings).sample
#perform guidance
noise_pred_uncond, noise_pred_text = noise_pred.chunk(2)
noise_pred = noise_pred_uncond + guidance_scale * (noise_pred_text - noise_pred_uncond)
noise_pred=noise_pred.to(torch_device)
latents = scheduler.step(noise_pred,t.to(torch_device),latents.to(torch_device)).prev_sample
Then I tried to run the original code on a free paperspace machine and it gave me an Out of Memory error. Even though, it had 8gbs of RAM. What should I do to make it run anywhere?