Migrating from PyTorch


I am trying to use the Migrating notebook from fastai2 to try to use some PyTorch code in fastai2.

It seems that everything work until I tried to fit_one_cycle when I got an error about pbar . Disable it does not seem to solve the issue. Here is the code with the error.

I did a fresh install of fastai2, openslide-python (1.1.1) and fastprogress (0.2.2)

Any ideas how to solve the problem? Thanks

P.S.: This question was first here but I opened a new issue bc is kind of unrelated

It looks like the DataLoader could not pull the len of your dataset. Can you try len(ds) (where ds is what you give the DataLoader)?

Thanks @sgugger for the answer. It seems that the custom dataset build has an argument (mode) that, if not specified, does not generate the ds. However, things seems a little bit more complex than default migrating. I made a summary of what I think it does:

BACKGROUND: This pipeline uses openslide library to tile a very big image (WSI - a tissue histology image) and make a prediction in a weakly supervised way. The authors mean with this that they only have an overall label rather than a per tile label. The workflow goes more or less like:

  1. Perform predictions directly on all the tiles generated and got probabilities
  2. Reorder the tiles with the greatest probability
  3. Keep only the tiles with the greatest probability and generate a subset of images with the overall label (converting the problem to a fully supervised fashion)
  4. Perfrom training on this subset and update weights and optimizer

So, if I understood the code correctly, this pipeline perform a first prediciton step that is not suitable for fastai2 Dataloaders to work out of the box. Am I right? Do you think is there any possibility to implement this kind of pipeline in fastai2? Maybe is just enough to give the dataloader a first prediction and then the subsequent steps in the fastai2 pipeline will work?
Thanks :slight_smile: