Anyone else see this?
I’m having some trouble with running the following line when following along lesson 1:
data.show_batch(rows=3, figsize=(7,6))
This appears to hang. If I force it to stop (by pressing the stop button), I get the following traceback:
KeyboardInterrupt Traceback (most recent call last)
<ipython-input-34-66824b983385> in <module>
----> 1 data.show_batch(rows=3, figsize=(7,6))
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/site-packages/fastai/basic_data.py in show_batch(self, rows, ds_type, **kwargs)
179 def show_batch(self, rows:int=5, ds_type:DatasetType=DatasetType.Train, **kwargs)->None:
180 "Show a batch of data in `ds_type` on a few `rows`."
--> 181 x,y = self.one_batch(ds_type, True, True)
182 n_items = rows **2 if self.train_ds.x._square_show else rows
183 if self.dl(ds_type).batch_size < n_items: n_items = self.dl(ds_type).batch_size
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/site-packages/fastai/basic_data.py in one_batch(self, ds_type, detach, denorm, cpu)
162 w = self.num_workers
163 self.num_workers = 0
--> 164 try: x,y = next(iter(dl))
165 finally: self.num_workers = w
166 if detach: x,y = to_detach(x,cpu=cpu),to_detach(y,cpu=cpu)
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/site-packages/fastai/basic_data.py in __iter__(self)
73 def __iter__(self):
74 "Process and returns items from `DataLoader`."
---> 75 for b in self.dl: yield self.proc_batch(b)
76
77 @classmethod
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
629 while True:
630 assert (not self.shutdown and self.batches_outstanding > 0)
--> 631 idx, batch = self._get_batch()
632 self.batches_outstanding -= 1
633 if idx != self.rcvd_idx:
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _get_batch(self)
608 # need to call `.task_done()` because we don't use `.join()`.
609 else:
--> 610 return self.data_queue.get()
611
612 def __next__(self):
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/multiprocessing/queues.py in get(self, block, timeout)
92 if block and timeout is None:
93 with self._rlock:
---> 94 res = self._recv_bytes()
95 self._sem.release()
96 else:
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/multiprocessing/connection.py in recv_bytes(self, maxlength)
214 if maxlength is not None and maxlength < 0:
215 raise ValueError("negative maxlength")
--> 216 buf = self._recv_bytes(maxlength)
217 if buf is None:
218 self._bad_message_length()
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/multiprocessing/connection.py in _recv_bytes(self, maxsize)
405
406 def _recv_bytes(self, maxsize=None):
--> 407 buf = self._recv(4)
408 size, = struct.unpack("!i", buf.getvalue())
409 if maxsize is not None and size > maxsize:
/nix/store/k8rccr5k6hvlzbk187gz3xshxmfk4fyr-python3-3.7.2-env/lib/python3.7/multiprocessing/connection.py in _recv(self, size, read)
377 remaining = size
378 while remaining > 0:
--> 379 chunk = read(handle, remaining)
380 n = len(chunk)
381 if n == 0:
KeyboardInterrupt:
Doing a little research, it seems like I may be running into the following issue:
That thread is quite long, but it doesn’t seem like there is a clear answer.
I was able to work around it by changing the line:
data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs
).normalize(imagenet_stats)
to
data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs, num_workers=0
).normalize(imagenet_stats)
You can see how I am setting num_workers=0.
It would be great if there were a fix and not just a workaround.
I’m building my Python environment using Nix, so this should be reproducible. Here is my .nix file:
(If you’ve never used nix before, this can be built by running nix-build default.nix. It produces a script that can be run ./result/bin/run-fastai-jupyter to start Jupyter.)
Here are some relevant versions:
- fastai: 1.0.46
- pytorch: 1.0.0
- Python: 3.7 (I’ve also tried with 3.6, but I was experiencing the same error)
- CUDA: 9.2
I’m on Ubuntu 18.04.2 LTS (Bionic Beaver).
edit: I just tried with pytorch-1.0.1, and that seems to make this issue go away. ImageDataBunch.from_name_re now seems to work without hanging.
Hey @Vineeth
I am assuming that you want your image file names aligned with your predictions.
I am supposing that you have a Learner named learner.
Then, you can get the file names which you used for prediction with:
filenames = learner.data.test_ds.items
and your predictions for example will be something like:
preds, _ = learner.get_preds(ds_type=DatasetType.test)
and then you can essentially zip the two lists or create a dataframe with pandas with something like:
df = pd.DataFrame.from_dict({"filename": filenames, "predictions":preds})
1 Like
Hey Fabian,
I am working on a similar problem; I was able download and use a good dataset provided by DeepSolar Project:
So far, only their validation data is available, but it’s more than enough to train and validate a model. Let me know if you would like to know more about the steps I took.
Edit: solved my problem. If you’re getting a connection error downloading the dataset, make sure you’ve enable internet access, for which you need to verify your phone # via SMS code. There’s a link on the the right hand side of the notebook within the settings frame.
This is the second time Im trying to run the lab. First time it worked fine, now im getting the following error - “NameError: name ‘create_cnn’ is not defined”. Any ideas as to why that is happening now?
1 Like
I have the same problem as well right now… I just run the same scripts yesterday and it worked fine, but now it prompts “NameError: name ‘create_cnn’ is not defined”…
Hi guys,
First at all thanks for you hard work with this course.
I ran this notebook with no issues. But now I want use trained model on single image to give me prediction of dog or cat breed.
something like this from last year course:
trn_tfms, val_tfms = tfms_from_model(arch, sz)
ds = FilesIndexArrayDataset([fn], np.array([0]), val_tfms, PATH)
dl = DataLoader(ds)
preds = learn.predict_dl(dl)
np.argmax(preds)
Unfortunately this is not working anymore as fastai changed a lot in the mean time.
I tried find solution for this in this thread but with no luck.
I found trn_tfms, val_tfms = tfms_from_model(arch, sz) is now trn_tfms, val_tfms = get_transforms(arch, sz)
But I have no luck with other lines
Any advice will be much appreciated.
Thanks
Roman
I tried lesson 01 on colab.How can I download the classifier I trained from colab for personal use?
Hello, I am trying to create my own image dataset and having some trouble.
I noticed we create the files through
urls = Array.from(document.querySelectorAll('.rg_di .rg_meta')).map(el=>JSON.parse(el.textContent).ou);
window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));
which adds all the images to a csv file.
However, download_images(path/file, dest, max_pics=200) pulls from path/file and path/file is a txt file as defined by
folder = 'grizzly'
file = 'urls_grizzly.txt'
&
path = Path('data/bears')
dest = path/folder
dest.mkdir(parents=True, exist_ok=True)
How does this work?
The Javascript grabs a list of urls and stores them into a txt file, from which you will upload to your jupyter notebook.
The path to the data will be under the “data/bears” folder. Grizzly, teddys, and black will be the subfolders, and as well, categories of your data.
The dest.mkdir call, will create those directories for you. Then the download_image calls, will then grab all of the images listed in the urls.txt file.
You just have to make sure you call the corresponding cells so the downloaded images get placed in the correct folders.
The javascript grabs a list of urls and stored them in a CSV file not a txt file though, as defined by
window.open('data:text/csv;charset=utf-8,'
Yeah just make sure to rename it with mv on linux, or right click it on Windows to rename it. Then it should work properly.
1 Like
Hi guys,
I found solution for making prediction for your own picture.
Here is my code:
dataClasses = data.classes
def load_model():
sz = 224
arch = models.resnet50
path = '/home/roman/FastAI/Lesson1' #path where my notebook is stored
classes = dataClasses
data = ImageDataBunch.single_from_classes(path, classes, tfms=get_transforms(), size=sz).normalize(imagenet_stats)
learn = create_cnn(data, arch)
learn.load('stage-2-50') #name of you saved model
return learn
img = open_image("/home/roman/FastAI/Lesson1/images/lurcher.jpeg") #path with your image
trained_model = load_model()
pred_class, pred_idx, outputs = trained_model.predict(img)
pred_class #this will give you prediction for your image
1 Like
Thanks @utkb and @SiewLin. I am able to run the assignment now. I was using 1.0.48 version of fastai (through AWS SageMaker) hence had to replace with cnn_learner. One thing to note in case others run into same problem - I had to restart the kernel for the change to take into effect.
Hi,
I’m using colab, I want to use Fast.ai v3 but it has fast.ai v1 (1.0.46) installed. How to upgrade to that?
Because of that cnn_learner wasn’t working and had to change it to create_cnn, to make it work.
Please let me know how to upgrade.
Hi @apoorv16, just run this line before running other codes:
!curl https://course.fast.ai/setup/colab | bash
Were you able to figure this out?
I tried it with clouderizer, it worked well…