I did AWS setup as told by Jeremy and initially used a t2.micro instance to get a hang of AWS.
I was able to successfully execute Lesson1 code in my local laptop itself. I was able to run vgg.test without any hassles. Later moved to a p2.xlarge instance. Tried to execute Dogs Vs Cats Redux and do a kaggle submission. Am unable to go past batches, preds = vgg.test(test_path, batch_size = 64).
It gives me the following error:
I took utils.py, vgg16.py, vgg16bn.py from github. For whatever reasons, copy link address and wget saved .py files as .html. So I did a cat > utils.py and pasted contents from git. I’d missed the line return test_batches in vgg.test method initially. I figured that out and corrected. But no use.
I initially ran 3 epochs, but since I wanted to get this up and running, I changed to 1 epoch and re-executed. Noticed below in ft0.h5 under results folder
But I don’t see any error or warning message in notebook for that cell execution. Am sure something is wrong.
Went through all possible discussions in Lesson1 and Lesson2 forum, but looks like none has faced this.
Would somebody be able to help me ? Dying to make my 1st kaggle submission Thanks in advance.
Some more additional observation,
All my test images are under /test/unknown/*.jpg
So I gave my testpath as below
When I execute vgg.test, I get 0 images found and the NoneType is not iterable error
Whereas when my testpath did not have unknown folder, it found 12500 images. Am totally confused now
@prettycold test_path should be defined as:
test_path = DATA_HOME_DIR + ‘/test/’
unknown is a keyword which keras library uses to indicate an unknown class.
But all the .jpgs should be inside unknown folder.
It would be better to ask questions in the appropriate Lesson threads. in your case it would be
Lesson2: discussion or Lesson1: discussion. It increases the chances that somebody would answer your question. Just a suggestion from me…
Many thanks @gnavink. I shall re-post it in Lesson 2 Discussion thread.
When I executed it initially, I used test_path = DATA_HOME_DIR + ‘/test/’ only but got the error as pasted in snapshot. Any suggestions there?
TypeError: ‘NoneType’ object is not iterable.
Its difficult to point out the error without seeing the context you are getting this error.
My guess is your test_path variable is not pointing to the correct directory at the time of execution of the corresponding cell.
It would be better to restart the kernel and clear the output.Then do the below:
- At the bash shell do for example:
ls -l <path + train/cats/*.jpg> | wc -l
repeat the same for all folders train, valid, test, sample ( sample/train, sample/test, sample/valid) folders respectively) and confirm all is well…
Put a print for all your paths to check whether they are pointing to the correct folders and validate whether its expected.
If you change something in the previous cell, ensure you run that cell…
It would be a lot better if you use the AWS instance as given by FastAI because all the necessary files should be present by default. It seems you have done manual changes ( for eg. cat, wget etc… to .py files) as described by you in your first post…
Generally people assume the infrastructure is the same as used in the class and they think what could be the problem in your code… You can really help yourself and others if you can adhere to this… would save you a lot of time and effort…
Please go through the post: http://wiki.fast.ai/index.php/How_to_ask_for_Help
hope it helps.
All the files should be in your instance if you had followed the below link:
No need to manually do wget/cat etc…
@gnavink - Many thanks. I restarted my kernel and printed all folder paths as you suggested, they seemed right. Good news is, vgg.test went through without errors. Thanks again.
As you said, probably somewhere the path was pointing to a wrong folder. I used to re-run my cells whenever I made a change but didn’t give a complete re-start as the output of folder path seemed right. Since I executed many things many times, something should have got messed up.
Also, I did not take a clone of git as I used to do watching jeremy’s videos and just follow whatever he did and I wanted to try things myself.
Yeah, in future, I shall be more specific with snapshots and code and infrastructure details, so that it’s easier for others to provide help.