I have a question about untar_data. The documentation comment makes it sound like I can specify a path via fname, but I don’t see anything in the code for it that takes in a path variable. How do I change the destination path? Looking at the code from lesson 1.
1 Like
Yeah, this was discussed briefly in another thread, but I’m not sure what the answer is to this.
1 Like
When working through chapter 4 from the book I got to the part of visualising the 3 using pandas and thought I would try to test the same approach with an image I downloaded directly from the internet. A URL link didn’t directly work so I googled how to do that and followed instructions on stackoverflow:
import requests
from io import BytesIO
cat_url = 'https://toppng.com/uploads/preview/cat-11525956124t37pf0dhfz.png'
response = requests.get(cat_url)
img = Image.open(BytesIO(response.content))
This allowed me to visualise the imported image but the next step failed
cat_t = tensor(img)
df2 = pd.DataFrame(cat_t[4:40, 4:40])
Likely because the tensors have different dimensions and I had wondered slightly off script to check that I know what actually going on.
And I had no idea what to do next so I thought I’d better search for some help on fastai functions:
Which brought me to your post!!
When running Google Colab and Kaggle with doc(get_image_files) I get the not overly helpful response:
And when trying to pip install nbdev I get the following type of error. Should I be trying to install nbdev on Google Colab or Kaggle? It would be nice to have the link to source directly available so maybe there is a way to have the links displayed?
Sorry for the long post but being able to link to the docs directly from colab would really help if it is possible?
I’ve found one solution to my problem. Use Paperspace and the links work out of the box. So much easier to stay in the flow of fastai rather than googling help and becoming derailed…
Yes this is another reason this is my preferred platform.
You can also just leave a tab open with docs.fast.ai in it, and use the search functionality on that site to find what you’re looking for. One benefit of that is that it’ll show you tutorials as well as the main API docs.
2 Likes
As the paperspace stuff is already covered in the conversation above, I can try to hint you a bit in the direction of how to reason about fixing the error that you ran into.
If I understand correctly, the code failed at the pandas dataframe creation step (defining df2).
- First thing to notice is that the
cat_thas a shape of[859, 840, 4], as opposed to the mnist example with[28, 28] - I’m assuming this is a PNG image of dimension 859px by 840px with 4 channels (Red, Green, Blue, Alpha), hence the total 4 channels.
- The MNIST example is 28px by 28x px with just 1 channel, that’s why it only has 2 axes instead of 3 as in your image.
- When you take a slice of the
cat_ttensor incat_t[4:40, 4:40], you should get a tensor of shape[36, 36, 4]back, as you’re only slicing on the 0th & 1st axis. - A pandas dataframe holds tabular data, in other words it’s a 2 dimensional matrix/table like data, which means in our case that it can only be created out of a rank-2 tensor (no. of axes = 2, length of shape = 2)
- The
cat_t[4:40, 4:40]is a rank 3 tensor. To verify this, trycat_t[4:40, 4:40].ndimorlen(cat_t[4:40, 4:40].shape) - What we can do is to filter out using only 1 channel from this image, so the slice can be fed into the pandas dataframe.
- I’d suggest picking any on channel when building the slice, after which you can check the rank (has to be a rank 2 tensor) and feed to the pandas df creation. (eg
cat_t[4:40, 4:40, 0].ndim)
Let me know how it goes. I’ll not post the exact code, but the solution is in the text above. I hope this points you in the direction of resolving the error that you ran into. 
4 Likes
Thank you Suvash! You have a new admirer 
I went from from this image:
To this:
And along the way learnt how to use Paperspace, simulate date, filter out one channel in an image of the simulated data, check ranks of tensors and upload image files to a notebook. Now I would like to control the output size but feel satisfied with my few learnings this last week.
3 Likes
Yay !
edit: solved by completely shutting down instance and re running all cells
Hello everyone,
My goal is to clone the fast.ai repo into an AWS instance (conda_python3). I have the instance set up and the repo cloned.
In order to have some things defined I had to add:
conda install -c fastchan fastai
import fastai
pip install fastai --upgrade
pip install nbdev
Something else I tried that failed was:
pip install -e "fastai[dev]"
But received:
ERROR: fastai[dev] is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).
Note: you may need to restart the kernel to use updated packages.
I had to remove “.all” from
from fastai.vision.all import *
from nbdev.showdoc import *
set_seed(2)
However, when I try to run the cells I receive:
NameError: name 'set_seed' is not defined
Any advice is greatly appreciated. I also tried restarting the kernel
pip install -e “fastai[dev]”
— The above installation command to be used when doing a git clone of fastai repository and then doing a pip install like so:
git clone GitHub - fastai/fastai: The fastai deep learning library
pip install -e “fastai[dev]”
Regarding this problem, ‘I had to remove “.all”’. I suspect your installation is an older version of fastai. Please check the installed version by running,
pip show fastai
It should be 2.7.7. If it is not then please run
pip install --upgrade fastai
2 Likes
Thank you! I tried:
!git clone https://github.com/fastai/fastai
!pip install -e “fastai[dev]”
Which successfully cloned a folder. But whenever I try to use
pip install -e “fastai[dev]”
I received:
I was able to move forward without this but I would like to learn best practices and I am worried it will cause a headache later.
1 Like
You’re missing the ! at the start of that command.
Is there a structured ‘SUPER beginner guide’ for dummies like me?
I’m at lesson 2, but already drown in complexity
- can’t set up pytorch to use GPU on my local machine - using pip/conda - killed many hours for this
- can’t verify mobile at kaggle (support doesn’t respond), so can’t follow the book there
- paperspace, despite providing a GPU, runs twice slower than CPU on my local machine
- using google collab. But what is fastBOOK? I thought the lib is fastAI?? Why does fastbook tries to access my gdrive? and then later fastSETUP??? fastCHAN???
- why do I need to register at Azure to search for images? Switched to _ddg - now the function call never finishes (42minutes and counting)
- what’s the difference between PILImage and Image.open()?
- what is nbdev?
- then it got weirder - and much more complex - HuggingFace, which evolved into WSL, boilerplate app.py without any understanding, fastsetup/mamba/fastchan/nbdev, and of course the detour into html and JavaScript
besides all that, I’m totally overwhelmed by Jupiter extensions, lots of libraries installation - which I have no idea about, lots of new functions from the library that haven’t got into context yet
RED BUTTON
sorry for venting, I’m just overwhelmed and super frustrated
Hi @michael-pru - I hear your frustration and it can be overwhelming at times. I highly recommend you go to the Live coding sessions starting from number 1. It starts off with the basics and the idea was that it would be a step-by-step guide for absolute fastai beginners like you and me (3 months ago). It does diverge from this original intent and goes off on a few deep tangents but should help you get started.
Don’t do both Google Colab and Paperspace. Pick one and I recommend Paperspace because the Live Coding sessions use this.
I really hope the live coding sessions restart at some time in the future because I made the most progress in the shortest space of time by following along. Word of warning. These sessions aren’t a structured course and highly dependent on the questions from people who attended.
Also the setup and recommended first steps changed deep into the sessions because things were still being optimised. If you want to jump to Session 17 to setup up more automatically this session will help you do that. Going from Session 1, however, you’ll set up Paperspace from scratch which is also really helpful. I recommend the going from Session 1 and learning the easy way later but you can choose your path.
6 Likes
Thanks! That’s what I was looking for.
Also decided to switch from Win to Linux - which made everything easier.
Managed to make CUDA available, learned what fastsetup is, starting to get the hang of environments, package managers, jupyter notebooks.
Very helpful series
2 Likes
I have trouble understanding some code of 04_mnist. I don’t know how did the code below combine the images into 3D tensor… torch.stack takes a list, and then how did the tensor magically create another 2 dimensions ? I don’t see we stack seven_tensors 3 times…
The list seven_tensors already contains 28 \times 28 data, which get stacked on top of each other and result in a 6265 \times 28 \times 28 vector. Check:
len(seven_tensors)
>> 6265
and
seven_tensors[0].shape ## The shape of one object in the `seven_tensors` list
>> torch.Size([28, 28])
Does this help? 
2 Likes
thanks a lot. after some hardcore reading on rank, axis…I figured it out too
1 Like
Hello!
I am doing the first lesson 'Is it a bird? Creating a model from your own data"
And I don’t know what to do with this error
pil_img = PILImage.create('bird.jpg')
is_bird,_,probs = learn.predict(pil_img)
I have the following error - AttributeError: read
The file exists. The path is correct. All versions are updated.
The text of the error
AttributeError Traceback (most recent call last)
/tmp/ipykernel_17/665255836.py in
----> 1 is_bird,_,probs = learn.predict(img)
2 print(f"This is a: {is_bird}.“)
3 print(f"Probability it’s a bird: {probs[0]:.4f}”)
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in predict(self, item, rm_type_tfms, with_input)
319 def predict(self, item, rm_type_tfms=None, with_input=False):
320 dl = self.dls.test_dl([item], rm_type_tfms=rm_type_tfms, num_workers=0)
→ 321 inp,preds,_,dec_preds = self.get_preds(dl=dl, with_input=True, with_decoded=True)
322 i = getattr(self.dls, ‘n_inp’, -1)
323 inp = (inp,) if i==1 else tuplify(inp)
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in get_preds(self, ds_idx, dl, with_input, with_decoded, with_loss, act, inner, reorder, cbs, **kwargs)
306 if with_loss: ctx_mgrs.append(self.loss_not_reduced())
307 with ContextManagers(ctx_mgrs):
→ 308 self._do_epoch_validate(dl=dl)
309 if act is None: act = getcallable(self.loss_func, ‘activation’)
310 res = cb.all_tensors()
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in _do_epoch_validate(self, ds_idx, dl)
242 if dl is None: dl = self.dls[ds_idx]
243 self.dl = dl
→ 244 with torch.no_grad(): self._with_events(self.all_batches, ‘validate’, CancelValidException)
245
246 def _do_epoch(self):
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in with_events(self, f, event_type, ex, final)
197
198 def with_events(self, f, event_type, ex, final=noop):
→ 199 try: self(f’before{event_type}'); f()
200 except ex: self(f’after_cancel{event_type}‘)
201 self(f’after_{event_type}’); final()
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in all_batches(self)
203 def all_batches(self):
204 self.n_iter = len(self.dl)
→ 205 for o in enumerate(self.dl): self.one_batch(*o)
206
207 def _backward(self): self.loss_grad.backward()
/opt/conda/lib/python3.7/site-packages/fastai/data/load.py in iter(self)
125 self.before_iter()
126 self.__idxs=self.get_idxs() # called in context of main process (not workers/subprocesses)
→ 127 for b in _loadersself.fake_l.num_workers==0:
128 # pin_memory causes tuples to be converted to lists, so convert them back to tuples
129 if self.pin_memory and type(b) == list: b = tuple(b)
/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in next(self)
519 if self._sampler_iter is None:
520 self._reset()
→ 521 data = self._next_data()
522 self._num_yielded += 1
523 if self._dataset_kind == _DatasetKind.Iterable and \
/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _next_data(self)
559 def _next_data(self):
560 index = self._next_index() # may raise StopIteration
→ 561 data = self._dataset_fetcher.fetch(index) # may raise StopIteration
562 if self._pin_memory:
563 data = _utils.pin_memory.pin_memory(data)
/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
32 raise StopIteration
33 else:
—> 34 data = next(self.dataset_iter)
35 return self.collate_fn(data)
36
/opt/conda/lib/python3.7/site-packages/fastai/data/load.py in create_batches(self, samps)
136 if self.dataset is not None: self.it = iter(self.dataset)
137 res = filter(lambda o:o is not None, map(self.do_item, samps))
→ 138 yield from map(self.do_batch, self.chunkify(res))
139
140 def new(self, dataset=None, cls=None, **kwargs):
/opt/conda/lib/python3.7/site-packages/fastcore/basics.py in chunked(it, chunk_sz, drop_last, n_chunks)
228 if not isinstance(it, Iterator): it = iter(it)
229 while True:
→ 230 res = list(itertools.islice(it, chunk_sz))
231 if res and (len(res)==chunk_sz or not drop_last): yield res
232 if len(res)<chunk_sz: return
/opt/conda/lib/python3.7/site-packages/fastai/data/load.py in do_item(self, s)
151 def prebatched(self): return self.bs is None
152 def do_item(self, s):
→ 153 try: return self.after_item(self.create_item(s))
154 except SkipItemException: return None
155 def chunkify(self, b): return b if self.prebatched else chunked(b, self.bs, self.drop_last)
/opt/conda/lib/python3.7/site-packages/fastai/data/load.py in create_item(self, s)
158 def retain(self, res, b): return retain_types(res, b[0] if is_listy(b) else b)
159 def create_item(self, s):
→ 160 if self.indexed: return self.dataset[s or 0]
161 elif s is None: return next(self.it)
162 else: raise IndexError(“Cannot index an iterable dataset numerically - must use None.”)
/opt/conda/lib/python3.7/site-packages/fastai/data/core.py in getitem(self, it)
456
457 def getitem(self, it):
→ 458 res = tuple([tl[it] for tl in self.tls])
459 return res if is_indexer(it) else list(zip(*res))
460
/opt/conda/lib/python3.7/site-packages/fastai/data/core.py in (.0)
456
457 def getitem(self, it):
→ 458 res = tuple([tl[it] for tl in self.tls])
459 return res if is_indexer(it) else list(zip(*res))
460
/opt/conda/lib/python3.7/site-packages/fastai/data/core.py in getitem(self, idx)
415 res = super().getitem(idx)
416 if self._after_item is None: return res
→ 417 return self._after_item(res) if is_indexer(idx) else res.map(self._after_item)
418
419 # %% …/…/nbs/03_data.core.ipynb 53
/opt/conda/lib/python3.7/site-packages/fastai/data/core.py in _after_item(self, o)
375 raise
376 def subset(self, i): return self._new(self._get(self.splits[i]), split_idx=i)
→ 377 def _after_item(self, o): return self.tfms(o)
378 def repr(self): return f"{self.class.name}: {self.items}\ntfms - {self.tfms.fs}"
379 def iter(self): return (self[i] for i in range(len(self)))
/opt/conda/lib/python3.7/site-packages/fastcore/transform.py in call(self, o)
206 self.fs = self.fs.sorted(key=‘order’)
207
→ 208 def call(self, o): return compose_tfms(o, tfms=self.fs, split_idx=self.split_idx)
209 def repr(self): return f"Pipeline: {’ → '.join([f.name for f in self.fs if f.name != ‘noop’])}"
210 def getitem(self,i): return self.fs[i]
/opt/conda/lib/python3.7/site-packages/fastcore/transform.py in compose_tfms(x, tfms, is_enc, reverse, **kwargs)
156 for f in tfms:
157 if not is_enc: f = f.decode
→ 158 x = f(x, **kwargs)
159 return x
160
/opt/conda/lib/python3.7/site-packages/fastcore/transform.py in call(self, x, **kwargs)
79 @property
80 def name(self): return getattr(self, ‘_name’, _get_name(self))
—> 81 def call(self, x, **kwargs): return self._call(‘encodes’, x, **kwargs)
82 def decode (self, x, **kwargs): return self._call(‘decodes’, x, **kwargs)
83 def repr(self): return f’{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}’
/opt/conda/lib/python3.7/site-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
89 def _call(self, fn, x, split_idx=None, **kwargs):
90 if split_idx!=self.split_idx and self.split_idx is not None: return x
—> 91 return self._do_call(getattr(self, fn), x, **kwargs)
92
93 def _do_call(self, f, x, **kwargs):
/opt/conda/lib/python3.7/site-packages/fastcore/transform.py in do_call(self, f, x, **kwargs)
95 if f is None: return x
96 ret = f.returns(x) if hasattr(f,‘returns’) else None
—> 97 return retain_type(f(x, **kwargs), x, ret)
98 res = tuple(self.do_call(f, x, **kwargs) for x in x)
99 return retain_type(res, x)
/opt/conda/lib/python3.7/site-packages/fastcore/dispatch.py in call(self, *args, **kwargs)
118 elif self.inst is not None: f = MethodType(f, self.inst)
119 elif self.owner is not None: f = MethodType(f, self.owner)
→ 120 return f(*args, **kwargs)
121
122 def get(self, inst, owner):
/opt/conda/lib/python3.7/site-packages/fastai/vision/core.py in create(cls, fn, **kwargs)
123 if isinstance(fn,bytes): fn = io.BytesIO(fn)
124 if isinstance(fn,Image.Image) and not isinstance(fn,cls): return cls(fn)
→ 125 return cls(load_image(fn, **merge(cls._open_args, kwargs)))
126
127 def show(self, ctx=None, **kwargs):
/opt/conda/lib/python3.7/site-packages/fastai/vision/core.py in load_image(fn, mode)
96 def load_image(fn, mode=None):
97 “Open and load a PIL.Image and convert to mode”
—> 98 im = Image.open(fn)
99 im.load()
100 im = im._new(im.im)
/opt/conda/lib/python3.7/site-packages/PIL/Image.py in open(fp, mode, formats)
2919 exclusive_fp = True
2920
→ 2921 prefix = fp.read(16)
2922
2923 preinit()
/opt/conda/lib/python3.7/site-packages/PIL/Image.py in getattr(self, name)
539 )
540 return self._category
→ 541 raise AttributeError(name)
542
543 @property
AttributeError: read