I keep getting this “RuntimeError: expected scalar type Float but found BFloat16” as I was running Jeremy Howard new video on hackers guide on LLM. To be specific after using the model from “'TheBloke/OpenOrca-Platypus2-13B-GPTQ”.
I am using free version of Google Colab to run this experiment. What can be the reason for this error message?
/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:1417: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see Generation ) warnings.warn(
RuntimeError Traceback (most recent call last)
in <cell line: 1>() ----> 1 res = gen(mk_pmt(ques_ctx),300)
7 frames
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input) 112 113 def forward(self, input: Tensor) → Tensor: → 114 return F.linear(input, self.weight, self.bias) 115 116 def extra_repr(self) → str:
RuntimeError: expected scalar type Float but found BFloat16