@Billy, replace the underscore with variable ‘ndx’
e.g… pred,ndx,probs = learn.predict(...)
then probs[ndx] will give the probability of the prediction.
1 Like
Got problem when trying to acquire a larger dataset, Keyerror: 'Next' when using search_iamges, anyone knows why?
My effort: tried different max_images parameter, and when it is less than 400 it works fine, but when larger than 1000 there is an error.
Here is the error info:
KeyError Traceback (most recent call last)
<ipython-input-12-3ba183898746> in <module>()
6 downloaded_dest = (downloaded_path/o)
7 downloaded_dest.mkdir(exist_ok = True, parents = True)
----> 8 download_images(downloaded_dest, urls = search_images(f"{o} photo", max_images=1000))
9 resize_images(downloaded_path/o, max_size = 400, dest = resized_path/o)
10 path = resized_path
<ipython-input-2-0d0938f9c459> in search_images(term, max_images)
10 data = urljson(requestUrl,data=params)
11 urls.update(L(data['results']).itemgot('image'))
---> 12 requestUrl = url + data['next']
13 time.sleep(0.2)
14 return L(urls)[:max_images]
KeyError: 'next
Here is my code:
searches = 'female','male'
downloaded_path = Path('downloaded_female_or_not')
resized_path = Path("resized_female_or_not")
for o in searches:
downloaded_dest = (downloaded_path/o)
downloaded_dest.mkdir(exist_ok = True, parents = True)
download_images(downloaded_dest, urls = search_images(f"{o} photo", max_images=500))
resize_images(downloaded_path/o, max_size = 400, dest = resized_path/o)
path = resized_path
# get all imgs
searches = 'female','male'
downloaded_path = Path('downloaded_female_or_not')
resized_path = Path("resized_female_or_not")
for o in searches:
downloaded_dest = (downloaded_path/o)
downloaded_dest.mkdir(exist_ok = True, parents = True)
download_images(downloaded_dest, urls = search_images(f"{o} photo", max_images=1000))
resize_images(downloaded_path/o, max_size = 400, dest = resized_path/o)
path = resized_path
1 Like
When I tried using fast setup and run./setup-conda.sh, I got curl: option --no-progress-meter: is unknown err. I searched online but seems that no answer regarding this, anyone know how to solve it?
fastsetup % ./setup-conda.sh
Downloading installer...
curl: option --no-progress-meter: is unknown
curl: try 'curl --help' or 'curl --manual' for more information
Just remove that option from the script – sounds like you have an old version of curl.
Hi, I just finished my first homework and I have two questions:
- Where am I supposed to “share” it? I can’t seem to find the proper discussion.
- What’s the purpose of the “probs” output from learn.predict ? I assumed it was the probabilities of the input belonging to one class, but after some testing, they seem uncorrelated (e.g. with my classifier an image is classified as “class 1” but the highest probs is for “class 3”). Any advice?
Hi @nerusskyhigh, welcome to fastai!
-
This is the discussion post you are most likely looking for: Share your work here ✅
-
The article pasted below does a good job of explaining the output of
learn.predict, I have copy and pasted the relevant part here:
“a fully decoded prediction including reversing transforms from the dataloader, a decoded prediction usingdecodes, and the prediction from the model passed through the loss function’sactivation”
Which leads to some questions in regards to what your model is doing, is it a classification task? Can you please paste the output here?
3 Likes
Thank you!
For those who are facing the same problem:
I removed --no-progress-meter option
from curl -LO --no-progress-meter $DOWNLOAD in ./setup-conda.sh in fastsetup folder
and get curl -LO $DOWNLOAD instead.
And it worked!
3 Likes
Hi, first thanks for the links and the advice. I’ll post my work there when it is completed.
Here is a screenshot of the output. I wanted to try and distinguish between Magic, Pokémon, and Yugioh cards with a classifier. You can see that the game is correctly predicted but the biggest output is assigned to magic cards. Furthermore, I just notice that those can not be probabilities because they don’t add up to 1 but I guess the conversion is just a normalization that will preserve the order relation.
If you need it, here is the complete notebook.
2 Likes
Hi @nerusskyhigh, thanks for sharing the notebook and it’s a cool idea and classification task!
I was able to decipher what the issue is. If you were to run dls.vocab on your data loader – once it’s been instantiated – you can see that there are more than three classes. So while the three classes will not add up 1.0 as expected, if you were to run probs.sum() (after the cell you’ve pasted), you will see the probabilities adding up to 1.0, as expected!
I’ll leave it up to you to determine how to figure out how to rectify the path naming issue, but I hope that this helps and it was clear enough for you to understand your question in regards to the learn.predict.
2 Likes
The “dls.vocab” made the trick, I thought there wasn’t any real way yo check the class found other than using the verbose option (which is not completely clear). Once I noticed the other classes I looked for other folders inside of “cards/” and worked from there. I don’t really know where they came from, but I settle things once and for all by sanitizing the cards’ names. Thanks a gain for your time. If there is a proper way to thank you other than the heart let me know!
1 Like
What is the reason for resizing images to be 192 x 192?
item_tfms=[Resize(192, method='squish')]
My understanding is resnet18 is trained with images of a size 224 x 224. Would that size perform better?
Try it and see! Smaller images are faster, so see what the time vs performance trade-off looks like.
Try it and see!
Great idea! I gave it a shot to see if resnet18 performs better with 192 pixel images or 224 pixel images.
Sized to 192:
Sized to 224:
I’m not sure the results are meaningful because of the small dataset size for the birds vs. forest problem. So I might need to try with some other datasets.
1 Like
I’m having trouble running the search_images function, i did ran it couple of times before in the afternoon, but when I ran it again in the night, it has this http error 403. I’m thinking that there’s some sort of API limit for urlread(), is this accurate or could there be another limitation that I’m not aware of? Any help or pointers would be appreciated.
7 Likes
Got the same problem as well. no idea how to solve this
My workaround while waiting for an explanation is to use this library, the functions closely mimic what the search_images() does
1 Like
The output of the model tuning has two sets of runs:
What are the 2 groups of runs for?
Is the first run (only 1 epoch) more like a sanity check to see how well the loaded model (AWD_LSTM in this case) does with just one epoch?
hey everyone! just started the fast.ai course and got a bit confused. so i watched the video recently, there was presented a model that distinguishes between birds and forests. now when i start reading the first chapter of the book, they show the cell and notebook with another model, called “cats and dogs”. and the notebook itself is called “00_intro” in the book but i have not been able to find it anywhere.
the book then continues on the first cell:
To do so, just press Shift-Enter on your keyboard, or click the Play button on the toolbar. Then wait a few minutes while the following things happen:
A dataset called the Oxford-IIIT Pet Dataset that contains 7,349 images of cats and dogs from 37 breeds will be downloaded from the fast.ai datasets collection to the GPU server you are using, and will then be extracted.
but in the model from the notebook called “Is it a bird? creating a model from your own data” there is no Oxford Pet dataset. feel a bit confused
am i correct to believe that the content of the first lesson was updated in the most recent version of the course and i can disregard the out-dated information in the first chapter of the book?
1 Like
got exactly the same issue, HTTP Error 403: Forbidden raised.
However, when I changed the url from ‘duckduckgo.com’ to ‘google.com’, i got another type of error: HTTP Error 405: Method not allowed.
and when changed from ‘google.com’ to ‘yandex.com’ (russian-based search engine) it throws another error: AttributeError: 'NoneType' object has no attribute 'group'.
2 Likes
My understanding is that the materials are basically the same thing but with a bit of a variation.
I think this might be the notebook you are looking for but it might be better to stick to the 2022 stuff?
1 Like