what changes are needed in learn.lr_find and in learn.fit if I want to use learn.half()??
in model definition if i specify learn.half(), I get errors when I use learn_lr_find:
RuntimeError: Expected object of type Variable[torch.cuda.HalfTensor] but found type Variable[torch.cuda.FloatTensor] for argument #1 ‘other’
Are you sure you are 100% updated with the master branch?
I am not sure. I will check later in the day. Running a script which may die if I update the repo now.
Is there an example in the repo where learn.half() is used?
My local repo is updated, my pytorch is also updated to 0.4.0.
I am looking to use fp16 processing on a notebook for the caravana dataset.
But it is not simply learn.half() that does the trick some more lines need a change?
Thanks for your help
I did some trial runs of the notebook measure_fp16.ipynb in the imagenet-fast.
I noticed that the model was made into fp16 model by specifying:
model = models.__dict__[arch]().cuda().half()
I also noticed that the input data was made into fp16 (I guess halfTensor) by specifying:
x_var = torch.autograd.Variable(x).cuda().half()
The notebook runs well and you see time comparison between FP32 processing and FP16 processing.
Coming back to my project (a variation of code for Carvana on dataset of Kaggle_2018_ScienceBowl), code accepts learn.half()…but does not automatically making input data into fp16.
in imagenet-fast repository there is a line like data = get_data(8,bs*4).half() which does not work because ImageClassifierData object has no attribute “half”
Is that code under construction?
Well, fastai is under construction
did you upgrade using the updated master branch?
Yes I did.
train.half() does not immediately error out. However since there is no working code for data = get_data(8,bs*4).half() , the code errors out as you start learn.fit(). Error about data not being half tensor.
I’m also trying to use learn.half(), and using standard training on the cats/dogs dataset from lesson 1 I get the following error:
RuntimeError: Expected object of type Variable[torch.cuda.HalfTensor] but found type Variable[torch.cuda.FloatTensor] for argument #1 'target.
I haven’t seen a fully worked example of using
learn.half() yet, and would appreciate it if anyone finds one.
I also found this problem. I guess the problem is that to use
half() your data needs to be float16. But if I load my data with ImageClassifierData, I am not sure how to change that. I suppose there is a function, but I couldn’t find one.