Check this out link which meant use this untar_data(URLs.MNIST_SAMPLE)
MNIST_SAMPLE
in short, the lib uses MNIST_SAMPLE , which has a different format.
1 Like
IPython Debugger gets lost in internal code after set_trace
After pressing n(ext) it just keeps going down the rabbit hole of its internal functions and I cannot get past set_trace().
you need to remove the tgz at the end of string to not get that error.
use this —> untar_data(‘https://s3.amazonaws.com/fast-ai-imageclas/mnist_png’)
1 Like
Thanks but that wasn’t really what I meant. I wanted to move my trained model to another CPU only machine to perform predictions. The learn.load method can only be called on an instantiated learner object. In order to instantiate a learner I have to provide it with a DataBunch and an architecture as a minimum. I have hacked around this with the following:
import torch
from fastai.vision import get_image_files, ImageDataBunch, get_transforms, ConvLearner, models, open_image
from pathlib import PosixPath
path_data = PosixPath('/Users/jeremyblythe/Downloads/dummy')
fnames = get_image_files(path_data)
pat = r'/([^/]+)_\d+.jpg$'
data = ImageDataBunch.from_name_re(path_data, fnames, pat, ds_tfms=get_transforms(), size=224, bs=10, flip_lr=False)
learn = ConvLearner(data, models.resnet34)
learn.model.load_state_dict(torch.load('/Users/jeremyblythe/Downloads/stage-2.pth', map_location='cpu'))
img = open_image('/Users/jeremyblythe/Downloads/test-A_29.jpg')
p = img.predict(learn)
data.classes = ['B', 'A', 'E', 'Y']
print(data.classes[p.argmax()])
My dummy directory contains one sample of each class. Note that when loading a model trained on a GPU machine into a CPU machine you cannot use learn.load as map_location='cpu' is required. Also note that you need to restore data.classes to the match the order from the original training.
The above hack has allowed me to train on a p2.xlarge in AWS and then bring the model back to my Mac to run predictions.
I think I saw a couple other people asking how to do something like this, @matwong and @danielegaliffa (questions G,H perhaps?) - I’m not saying this is a nice way to do it but it seems to work.
Wondering if anyone has a better, less hacky, way to do this?
1 Like
Can anyone tell me how to download a dataset from the UCI ML repository and use it? I have created a topic for it: How to download a dataset from a URL?
Any help is appreciated.
How do I get the number of images in each class, from the ImageDataBunch object.
Many thanks for the reminder - fixed now.
1 Like
oh sry for tagging.
thankyou for the information
1 Like
thank you!
Yes. I have reinstall fastai in new environment. The other one wasn’t letting me upgrade for 1.0.6. So I finally got it working . Thank you.
1 Like
I was just wondering the same thing. That would be very useful for building out the model without having to create a while new folder structure for it.
I also encountered the same FWIW.
I tried to run a duplicate copy of lesson1 jupyter notebook and initially it ran with no problem. But when i tried to rerun it, path.ls() gave me only
['images'] as o/p. I restarted the server, tried on the original notebook also but I’m getting the same o/p. Since I downloaded the dataset twice maybe that might be causing this. But I don’t know how to access ‘/storage/oxford-iiit-pet’ on cloud and delete the multiple downloads. Any idea on this?
I seem to getting this error regarding using TTA. ImageClassificationDataset has not attribute ‘tfms’ .
Please I am trying to get the total number of training images as well as the number in each class.
How do I do that?
Maybe you didn’t include any transforms in your DataBunch?
Can you provide a link? Thanks!
You are right, I made a typo. It didn’t raise any error.