Generate empty dataloader for production

I want to port a trained model to somewhere else but I would like to do it using pth file rather than pkl. However, in order to perform predictions using pth there I have to specify a dataloader in order to create the leaner using learn = cnn_learner(dls, resnet34, metrics=[accuracy]).

Is there any way to generate a dummy dataloader to pass it to cnn_learner in order to perform learn.predict(img)?

¿Why not use a random image with fake labels? Then you can create a test dataloader for tour data. This post may help you.

1 Like

@Joan you should do a learn.export and change the file extension to use the test_dl @vferrer pointed to

1 Like

Ok! I am struggling a little bit on this. I am unsure if I understood this properly.
If I call learn.export I will have a pkl file. You are telling me to change the extension from pkl to pth after saving the model?

Another question is that I’m trying to run the whole prediction in CPU. I am working with single images.
In this sense I tried all the tricks I found to send the model to make the predictions on CPU but I still have jobs when run nvidia-smi. My code so far looks like this (using pthfiles):

dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
                   get_x=ColReader(0, pref=f'{path}/'),
                   splitter=RandomSplitter(),
                   get_y=ColReader(1))
dls = dblock.dataloaders(df, bs=16, num_workers=8)
dls = dls.cpu()
defaults.use_cuda = False
learn = cnn_learner(dls, resnet34, metrics=accuracy)
img = PILImage.create(os.path.join(test_path, filename))
x, = first(dls.test_dl([img]).to('cpu'))
learn = learn.load('model')
learn.model.to('cpu')
learn.dls.cpu()
torch.device('cpu')
print(torch.cuda.current_device())
pred,pred_idx,probs = learn.predict(img)

As you may see, I am strongly trying to send the data to cpu but I cannot find the error…

I suggest you to read the 2nd chapter as the fastbook.
basically all you have to do is

learn.export() #do this and you will get export.pkl, copy it to your server

In your server

learn_inf = load_learner(path/'export.pkl')
learn_inf.predict(img)
2 Likes

Thanks for the feedback!
Actually I was aware of this but at some point got lost because I was trying to deploy on myBinder and getting also CAM output working.
Anyway, it works nicely with the fastbook pipeline.

1 Like