https://drive.google.com/open?id=1XuZntehm5x5azwylrqqTZ4LRAzM2YMT1
link to my colab notebook lesson 2 image_model. its not complete as I got some error in between the code.
Its a bit unorganised though.
You are passing in suffix parameter or your image folder has something which it shouldnāt have or its not a image
In my case, it was that there were some other folders like MacOSX
Let me now if you query is resolved,
If it isnāt then this has been discussed before by me, just give a quick search on the forum
Thanks
Ok so mate, I was taking a look and saw your path pointing to PATH = 'data/planet/'. Are you sure that you have a data folder? Coz I couldnāt find the command mkdir data anywhere.
I am using google colab. Why I am getting this error: No such file or directory: ādata/dogscats/trainā
can anyone help? I have downloaded the dataset and unzipped it as shown in the lesson.
Two suggestions:
-
When you look at data/dogscats/ make sure you see sub-directories: train/ and valid/
-
Is your PATH variable set correctly? Within Jupyter, executing ā!ls $PATHā should return the contents of data/dogscats/, including train/ and /valid.
EDIT: I restarted the notebook and now it works. I realized something was up when I tried to go back and rerun with cats/dogs and it was no longer working. Iāll leave this post up instead of deleting it in case someone else gets this error and searches for it, though.
Is training the model on your own data supposed to be as simple as making a new set of folders that contain images corresponding to two new things and then changing the PATH variable? Or are there other parts you need to change too? I did those steps, but I am getting an error (RuntimeError: cuda runtime error (4) : unspecified launch failure).
Iām wondering if there is something wrong with my data that could be causing things to fail or if I am missing some other steps in adapting the code to work for new data. I did make a sample folder with just a few files that are all actual images though, and I still get the same error, so I think itās something more than just having corrupted files or something.
1 Like
I followed this command to clone or copy the folder to my Azure DL vm (ubuntu vm)
"git clone https://github.com/fastai/courses.git but after that i notices it contains old file but whatever u have shared contains correct file(the https://github.com/fastai/fastai/tree/master/courses/dl1)
Old path is mentioned in the wiki page.http://wiki.fast.ai/index.php/Github
Which one is correct?
Now that Iāve used the lesson 1 notebook on my own images, how would I use the model to sort images?
Y. Iām new to the course, and just ran into this obstacle. Similar to Nyx and Paul, Iāve waited for 2 days to receive a response back - to no avail. So, like Paul, Iāve just submitted a HelpDesk ticket. Hopefully thatāll do the trick ā¦ Iāll let you know!
How did you out your data in to run the test? I cannot figure it out.
About getting automatically the value of the best learning rate
The learn.lr_find () function is great but looking visually for the best value of the learning rate on the graph seems to be a bit of an old-fashioned technique, no ? 
Has anyone created a script that automatically gives this best value after using learn.lr_find () ?
I am stuck exactly where Luke is, can anyone help me out?
Hi all,
I tried search the forum and this thread for an answer, but it seems to be very few things associated, at least as I could find.
I am currently a complete beginner when it comes to Deep Learning, however I have a solid background of programming. I would like to ask for a piece of advice here. I started looking at the first lecture, and after googling about at the costs of AWS (e.g. paperspace) I came to realise that it is not very cheap in the end anyway, especially with cost increasing proportional to use.
I have a freshly built (watercooled) rig, that I would then like to run my Deep Learning server (DLS) on. I run W10, with:
Asus Maximus IX Hero mobo
Intel i7 7700K
GTX 1070
16GB ram
2 SSDs + 1 HDD
Corsair AX 860W PSU
I honnestly feel like it would be a pity for me to use anything else than this. I could upgrade RAM and run SLI if needed. So now to my questions:
- Since I am primarily a Windows user, I would like to run a virtual box (VB, or something similar, better if existing?) for my DLS. Are there any downsides, primarily in terms of speed, to running it through VB or will it work just as the AWS?
- Should I consider upgrading the PC? RAM is of course easy, but for SLI I would have to drain the system and remake some of the water pipes.
If anyone has some good insight and/or links on the matter of setting up a DLS using a VB I would be very grateful!
Cheers!
Kaplan
Hi Hwang,
Same problem here with Paperspace cloud P5000 instance. Kernel is dying after i run the learn.fit() method after 30 seconds. Did you come around with a sloution? Help me with this.
Thanks,
Subash
Hi Class!
Iām trying to submit an attempt for the dogs/cats redux. Problem is, my prediction only has 8,819 rows 
. Iām pretty sure Iāve parsed the data correctly. My redux data directory contains the three sub-folders I created (train,test, and valid) plus the two that were created when I trained my model (models and tmp). Train and valid each have two sub-folders (cats and dogs). The path variable has been set to the following:
PATH = "data/newdogscats/"
The output of os.listdir(PATH) is as follows:
['.ipynb_checkpoints', 'test', 'valid', 'models', 'train', 'tmp']
The image splits (number of image files) for three sub-folders are as follows:
Train: 8,090 (per class)
Valid: 4,410 (per class)
Test: 12,500
Iāve verified each of these by passing the os.listdir() output to the len() function. Below are the commands I am using to train my model:
arch=resnet34
data = ImageClassifierData.from_paths(PATH, test_name='test', tfms=tfms_from_model(arch, sz))
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.fit(0.05, 2)
and yet my learn.predict() method is only creating 8,819 predictions!? Does this have something to do with the discrepancy between the size of the test and train folders, or is there something else I might want to verify? Thanks in advance for your input.
Update: len(data.val_y) only output 8,819. So, Iām thinking it might an issue reading files from the test folder.
Second Update: Solved my own problem 
It had nothing to do with the commands I pasted. When using learn.predict() you need the is_test = True argument.
1 Like
Hi Reshama,
I am currently trying to configure AWS to start working on this course. Iām following the link of yours. I tried logging in with
ssh -i aws_fastai_gpu.pem ubuntu@54.175.101.64 -L8888:localhost:8888 .
However, the output I got was:
Warning: Identity file aws_fastai_gpu.pem not accessible: No such file or directory.
ssh: connect to host 54.175.101.64 port 22: Operation timed out
It turns out that there is no file called aws_fastai_gpu.pem in the .ssh directory.I am on Mac - I had to a create a .ssh folder within /Users/arunvs/ and the only contents of the directory after running sshgen are id_rsa, id_rsa.pub and known_hosts.
I am not sure how to resolve this issue about logging in. I am new to both AWS and Mac. Your help will be much appreciated.
Thanks,
Arun
Can someone please explain the ābinary_lossā in the last couple of cells of lesson 1?
def binary_loss(y, p):
return np.mean(-(y * np.log(p) + (1-y)*np.log(1-p)))
- Why is it the ābinaryā loss? Does that just mean that it is normalized between 0 and 1?
-
What is the ā
yā here? It is not theyfromlog_preds,y = learn.TTA(), because it crashes if I use that āyā -
I see that that when a list is made to call this function, the variable is called ā
actsā, what does this stand for? I realise that it isy, but I donāt understand. -
How would I get y from the confusion matrix? I got inf when I tried using
probs- so maybe what isprobsthenā¦
Also, one really dumb question, but I just want to verify I understand - what does āprecomputeā actually precompute when set to True?
Thanks!
Check your understanding of the lesson 1
Check your understanding of the lesson 2 >>>
(post original in portuguese at Deep Learning BrasĆlia - LiĆ§Ć£o 1)
Hi guys,
I did watch again the video of the lesson 1 (part 1) to get the whole image and I took notes of the vocabulary used by @jeremy.
Letās play ! OK ? 
Can you give a definition / a url / an explanation for all the followings terms and expressions ?
If yes, you are done with the first lesson !!! 
PS : you do not want to test yourself or you want to check your answers ? Go to the blog post āDeep Learning 2: Part 1 Lesson 1ā of @hiromi : " super travail !!! "
- course Fastai
- forum Fastai
- GPU
- CUDA
- NVIDIA
- Crestle / PaperSpace
- jupyter notebook
- Data Science
- SHIFT + ENTER in a jupyter notebook
- python 3
- wget
- exclamation mark in a cell (ex : !ls)
- bash command
- python variable into brackets
- training set
- validation set
- Fastai Machine Learning course : prerequesite or not ?
- image Classifier
- label
- keras
- plt.imread
- plt.imshow
- python 3.6 format string
- img.shape
- 3 dimensional array (rank 3 tensor)
- Red Green Blue (RGB) pixel values between 0 and 255
- kaggal competition
- pre-trained model
- resnet24
- ImageNet competition
- Convolucional Neural Network (CNN)
- accuracy
- train a model
- 3 lines of code
- epoch
- testing set
- learning rate
- loss function
- cross entropy loss
- validation and testing set accuracy
- Fastai library
- transfer learning
- pytorch
- tensorflow
- network architecture
- data augmentation
- validation set dependent variable val_y
- data.classes
- classes
- object data
- object learn
- the model
- prediction on validation set
- learn.predict()
- log of the predictions : log_preds
- get the predictions on validation set np.argmax(log_preds, axis=1)
- get probabilities on dogs : np.exp(log_preds[:,1])
- numpy
- top-down, the whole game
- code driven approach
- world class neural network
- stalelite images
- structured data
- NLP classifier
- recommendation system
- text generator
- create our own architecture from scratch
- donwload a pre-trained model and precompute
- alphago
- image classifier for fraude dectection
- machine learning
- Arthur Samuels, 1950s, ML father
- IBM mainframe
- play checkers
- traditional Machine Learning
- features engineering
- domaine experts and specialits
- algorithm (Deep Learning) :
** infinitely flexible function
** all-purpose parameters fitting
** fast and scalable - neural network, number of simple linear layers interspersed with a number of non linear layers
- universal approximation theorem
- Fit parameters, Gradient Descent (how good are they, find a minimum on loss function curve, local miminim)
- minimum time, GPU 10 time faster than a CPU
- hidden layer
- increase of number of parameters by layer is a problem but increase number of layers is teh solution
- DL = neural network with multiple hidden layers
- Google starts using DL in 2012
- Geoffrey Hinton, DL father
- Andrej Karpathy
- inBox by Gmail
- Skype Translator
- Semantic Style Transfer
- cancer detection
- true/false positive/negative
- CNN, Convolucional Neural Network
- convolucional
- find edges
- multiplication of pixels values by a kernel (filter)
- linear operation
- linear layer
- non linear layer
- sigmoid
- Relu
- element wise multiplication
- michael Neslon
- Stochastic Gradient Descent
- derivative
- small step
- learning rate
- combine convolution, non linearity, gradient descent
- picture of what each layer learns
- parameters of the kernels are learnt using gradient descent
- learn.fit()
- learning rate not too high, but not too low as well
- choosing a learning rate
- learn.lr_find()
- best improvement of the loss before it gets worse
- learn.shed.plot_lr()
- learn.sched.plot()
- mini batches
- traing loss
- validation loss
- validation accuracy
- overfitting : stop fitting your model
- tab to get list of function
- SHIFT + TAB (once : parameters, twice : documentation, 3 times : pops up a window with source code)
- 1 question mark : documentation
- 2 question mark : pops up source code
- H to get short codes in jupyter notebooks
- Stop your Crestle or PaperSpace machine !
- use the fastai forum !
12 Likes