Pipei2i error: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor)

Got this error running this notebook: diffusion-nbs/stable_diffusion.ipynb at master · fastai/diffusion-nbs · GitHub

At this step:

prompt = "Wolf howling at the moon, photorealistic 4K"
images = pipei2i(prompt=prompt, num_images_per_prompt=3, init_image=init_image, strength=0.8, num_inference_steps=50).images
image_grid(images, rows=1, cols=3)

Error was:

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same

Here’s a screenshot of the full trackback:

Anyone know how to fix this one?

I’m running on a https://lambdalabs.com/ A100 instance.

I changed the first line in that notebook to the following to get a missing dependency:

%pip install diffusers transformers fastcore fastdownload

Found the answer here: Lesson 9 official topic - #27

@bernhard_jung says:

You need a diffusers version higher than 0.4.1, because this patch is needed for fp16 to work.

1 Like

So running this in the first cell should work:

%pip install -U diffusers transformers fastcore fastdownload

It doesn’t though - that still gave me diffusers 0.4.1.

This DID work:

%pip install -U https://github.com/huggingface/diffusers/archive/refs/heads/main.zip

That installs the latest main branch of the diffusers GitHub repo.

1 Like

Better fix inspired by Lesson 9 official topic - #66

%pip install https://github.com/huggingface/diffusers/archive/e895952816dc4a9a04d1d82fb5f8bdee59341c06.zip

That grabs a specific commit which definitely includes the fix.


Same, had to install from GitHub as both pip and conda are still on 0.4.1

Seems default version from pip is now 0.4.2 so should work with fresh install or upgrade.


Yeah here are their release notes, from 18 hours ago: Release v0.4.2: Patch release · huggingface/diffusers · GitHub

@simonw Did it fix the problem?
I have the following versions:

diffusers                         0.4.2
fastai                            2.7.5
huggingface-hub                   0.10.1
transformers                      4.23.1

and still get the

Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same

Quick fix: change torch_dtype to float32 and it works.

from diffusers import StableDiffusionImg2ImgPipeline
from fastdownload import FastDownload

pipei2i = StableDiffusionImg2ImgPipeline.from_pretrained(
    torch_dtype=torch.float32, # Changed row

Yes it does! Thanks!

From what I understand of their docs here, this is mainly for places where you have memory constraints.

“Note: If you are limited by GPU memory and have less than 10GB of GPU RAM available, please make sure to load the StableDiffusionPipeline in float16 precision instead of the default float32 precision as done above.”

I imagine that most of us should probably be ok to use the float32, esp if using cloud instances, but I wonder if it’s also slower then to generate the images etc. Probably the way it is right now allows for faster iteration.

In any case, I used the requirements.txt (recently merged in) for my pip install, then restarted the kernel (while using JarvisLabs) and it worked for me with the notebook as is.


That’s right @strickvl, both less memory usage and faster computation with fp16!


@jeremy Seems like there are a lot of issues related to the versions of packages. Do you want to keep this in the back of your mind and maybe release a stable configuration later on?

A requirements.txt file was merged onto the repo (yesterday, I think), which should be what you’re looking for in terms of ‘stable’ versions of packages. Hopefully that can be updated as needed.

1 Like

Hey all, we just did a new release in diffusers. Please let us know if you still face this issue.


Yes. I installed diffusers 0.5.1, and (after restarting the Kernel), the error went away.


Also, for the
prompt = “a photograph of an astronaut riding a horse”

the horse also went away (at first)… :slight_smile:

1 Like

There seems to be similar issue with the inpainting pipeline.
Have to use autocast currently to get around it :slight_smile:
If anyone wants to play around with it, here is the code

from diffusers import StableDiffusionInpaintPipeline
from PIL import ImageDraw

pipe_inpaint = StableDiffusionInpaintPipeline.from_pretrained(
p = FastDownload().download("https://i.redd.it/hdxi9adiquw21.jpg")#credit-https://www.reddit.com/r/funny/comments/blw0hq/a_true_photobomb/
init_image = Image.open(p).convert("RGB").resize((485, 323))
#mask is black and white(region to be inpainted)
msk = init_image.resize((485, 323))
drawMsk = ImageDraw.Draw(msk)
drawMsk.rectangle([(0, 0), (485, 323)],outline=(0, 0, 0), fill=(0, 0, 0))#black bkg
drawMsk.rectangle([(350, 170), (480, 250)],outline=(255, 255, 255), fill=(255, 255, 255))#white
from torch import autocast
prompt = "rocks in the water"
with autocast("cuda"):
    images = pipe_inpaint(prompt=prompt, init_image=init_image, mask_image=msk,strength=0.7,guidance_scale=8, num_inference_steps=70).images

1 Like

Hi, maintainer of diffusers here :wave:

Thanks for reporting it! This issue with image2image and inpainting pipeline has been fixed in v0.4.2 please upgrade your version of diffusers and then this should run in fp16 without any issues.


This course will cover a lot of cutting edge stuff - the idea of having a stable configuration isn’t really compatible with what we’re doing here… Expect sharp edges!

Also, a goal of this course is for students to get comfortable with cutting edge research - which includes dealing with all the little software issues which come up in research code.