Here is the link to today’s livestream:
The notebooks for today are (in order):
- 02_MNIST
- 02_SGD
- 02_Deployment
6 Likes
Question: How do I get my labels in fastaiv2? Basically I would like to match the filename with the class in the get_y.
In fastaiv1 I used to do ImageList.from_df and then .label_from_df(cols=‘class’) but I could not found in the fastaiv2 documentation how to do it.
animals = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
splitter=RandomSplitter(0.2),
get_y=???)@mgloria you want ColReader, 03_Multi_Label shows this (Planets) 
(along with a few different ways, we’ll walk through them all next week)
1 Like
Thanks @muellerzr! I get though an error I believe due to the fact that some of the images were corrupt. I unlinked them but they are still of course mentioned in the csv which I believe could be the issue.
animals = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=ColReader(4, pref=f'{path}/train/'),
splitter=RandomSplitter(0.2),
get_y=ColReader(3))
batch_tfms=[*aug_transforms(size=224, max_warp=0), Normalize.from_stats(*imagenet_stats)]
dbunch = animals.databunch(df, batch_tfms=batch_tfms)
error:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~/SageMaker/custom-miniconda/miniconda/envs/fastai2/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
2896 try:
-> 2897 return self._engine.get_loc(key)
2898 except KeyError:
pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
KeyError: 4
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
<ipython-input-17-21ecc1e0890b> in <module>
----> 1 dbunch = animals.databunch(df, batch_tfms=batch_tfms)
Is it really due to this? If so, is there a way to get back the name of the unlinked files so that I can remove those rows from the df. I feel it is a very common situation having some corrupt images.
I’d try defining a get_x and a get_y explicitly and seeing from there
1 Like
Had to make a quick fix:
4 Likes
I can hear you on the stream.
is rank the same as dimension?
you can have a 2 dimensions which is rank 1.
This corresponds to the maximal number of linearly independent columns of a matrix. Not sure if this is what mean by rank?
Update: Rank in linear algebra is different from the rank in deep learning.
Good discussion on overfitting: Determining when you are overfitting, underfitting, or just right?
3 Likes
if possible can you show how to package a model as a docker component. thanks
1 Like
I am unfamiliar with docker so I would not be a good resource with that, I am sorry!
1 Like
Here is the validation set article by Rachel:
1 Like
@muellerzr please could you tell when should one use item transform and batch transform. I am not clear about those two techniques
You should use item to prepare everything for a batch. Your initial resizeing will go there along with conversion to a tensor. Basically preparing for what your model expects
Batch transforms are anything from there (more resizing, warping, etc)
So would it be correct to assume that item transform is a prerequisite for batch level transform
Correct!
Hi harikrishnanrajeev hope all is well!
This link is for a docker tutorial, it is one of the best I have found. https://training.play-with-docker.com/ops-s1-hello/ you can practice building docker components in their virtual environment, it is great for getting a good understanding of Docker. https://training.play-with-docker.com/
This link is a good starter for deploying your model https://course.fast.ai/deployment_render.html once you have built it you can deploy it on any platform that supports Docker, not just render.com.
The snippets of code described by muellerzr in the deployment tutorial can be inserted straight into the deployment starter app.
Cheers mrfabulous1 
6 Likes
is this what we need to install ? (I think the notebooks currently don’t have this update).
in 02_MNIST.ipynb the following line gpu_tfms = [Cuda(), IntToFloatTensor(), Normalize()] returns the following error. (Not sure if the name has changed.) (could you share tips on how you go about solving this). Thank you very much have learnt so much already.