Hi, Is there any resolution for this? I am getting the same error. Thanks.
1 Like
I hope this is the right place to ask for help. LMK if not!
I’m trying to run the code in the 1st lecture and getting this error when trying to get 1 pic from DuckDuckGo.
Code:
urls = search_images('bird', max_images=1)
urls[0]
Error:
HTTP403ForbiddenError: HTTP Error 403: Forbidden
4 Likes
Hello lesson 1 fails at cell 5 with:
HTTP403ForbiddenError: HTTP Error 403: Forbidden
3 Likes
Is the internet enabled on your kaggle notebook?
DDG seems to be requiring a referer header. I had success changing the cell that defines search_images to the following:
from fastcore.all import *
import time
import json
def search_images(term, max_images=200):
url = 'https://duckduckgo.com/'
res = urlread(url,data={'q':term})
searchObj = re.search(r'vqd=([\d-]+)\&', res)
requestUrl = url + 'i.js'
params = dict(l='us-en', o='json', q=term, vqd=searchObj.group(1), f=',,,', p='1', v7exp='a')
headers = dict( referer='https://duckduckgo.com/' )
urls,data = set(),{'next':1}
while len(urls)<max_images and 'next' in data:
res = urlread(requestUrl, data=params, headers=headers)
data = json.loads(res) if res else {}
urls.update(L(data['results']).itemgot('image'))
requestUrl = url + data['next']
time.sleep(0.2)
return L(urls)[:max_images]
25 Likes
or look at the suggestion above…
Tnx! This seems to work.
I’ve updated the code in the repo now and DDG seems to work. (I hadn’t seen the suggestion to add a referer header above – if it breaks again we’ll try that).
2 Likes
@jeremy Hi Jeremy, The output of the model tuning always has two sets of runs. What are the 2 groups of runs for?
Is the first run (only 1 epoch) more like a sanity check to see how well the loaded model (AWD_LSTM in this case) does with just one epoch?
I understand that the 2nd group of runs are the epochs that are specified in the fine_tune() function.
Screenshot for reference: Lesson 1 official topic - #277 by replica2915
Hi @jeremy – I had got this error yesterday when I ran your notebook on Kaggle. Can you please share what the issue was and the code change that fixed it? Thanks.
I had the same question as I could not find the 00 intro notebook anywhere.
fastai is open source. To answer future questions like this yourself, you could try…
- Go to… fast.ai · GitHub
- Search the organisation like this… org:fastai “def fine_tune”
- Review the code… fastai/schedule.py at e867fc08978eeeb2803ff2c8aa7e4c631347b8c9 · fastai/fastai · GitHub
def fine_tune(self:Learner, epochs, base_lr=2e-3, freeze_epochs=1, lr_mult=100,
pct_start=0.3, div=5.0, **kwargs):
"Fine tune with `Learner.freeze` for `freeze_epochs`, then with `Learner.unfreeze` for `epochs`, using discriminative LR."
self.freeze()
self.fit_one_cycle(freeze_epochs, slice(base_lr), pct_start=0.99, **kwargs)
base_lr /= 2
self.unfreeze()
self.fit_one_cycle(epochs, slice(base_lr/lr_mult, base_lr), pct_start=pct_start, div=div, **kwargs)
You can see it runs fit_one_cycle() twice, the first time frozen, defaulting 1-time, and the second time unfrozen, requiring you to specify the number-of-times.
Freeze locks all weights except the last layer, which you just replaced with your categories.
1 Like
[Side note: Jeremy requests he not be @notified unless something is wrong with the server - otherwise imagine the stream of notifications interupting him from the many people doing this popular course.]
I also was curious also what the code change was, so I went looking to answer this myself, using Github’s facility to show recent commits - from a repo, click Insights > Networks as shown below.
I had to check a few fastai repos, and then in “course22” I found two recent commits shown below boxed in red under the number 16…
Clicking on each of those dots will take you to their commits:
A second method is from the repo main page, click on “commits”
which today shows…
the first of which has significant lines related to…
from time import sleep
3 Likes
Thanks for the help!
Apologies – thanks for letting me know, I will not @notify Jeremy. I’ll check the updates as per your note here. Thanks again!
1 Like
Hey everyone, I have trained my model in fastai using the MIDOG 2021 dataset, and my object detection model is working great, I have a minor issue while inferencing the model on totally new two images which are aperio images of .scn format and having aperio imagescope created annotation in XML format.
The thing is I trained my model using the fastai input pipeline based on the tutorial notebook provided by the MIDOG challenge team on their website: Google Colab
The notebook is based on the object detection library of fastai by Christian marshal:
GitHub - ChristianMarzahl/ObjectDetection: Some experiments with object detection in PyTorch
The input pipeline is as follows it takes the bunch of .tiff images and the annotations in MS COCO .json format and then creates the imagedatabunch object of fastai.
Now how should I use my new data which consists of only one .scn image and the respective . XML file of the same, which has annotations in the ellipse box (top left corner and top bottom corner coordinates) to be used for inference, and testing of my trained model.
Any resource, code snippet, notebook, etc which might be able to help to do this would be really helpful.
Thanking you all in advance,
Harshit
Hey everyone. I just started the fast.ai today and I am very eager to continue with all the interesting material and with a level that is general enough for us in the IT.
While watching the Lesson 1 video, Tabular Data is discussed among other, things. However, I couldn’t find something relevant in the notebook of lesson 1. Same goes for collaborative filtering. Am I missing something? What is the case with those two subjects?
Thanks in advance
@ypanagis You can find tutorials for simple implementations of tabular training and collaborative filtering on FastAI’s docs website (as well as several other tutorials ranging from beginner level to advanced):
Tabular training tutorial | Collaborative filtering tutorial
I believe the lessons may not correspond to the notes you’re looking at exactly.
Tabular training deals with data that can be expressed in tabular format, basically excel/spreadsheets (think labeled columns, rows of data). The tutorial teaches you how to access and manipulate the data to be used in training a model.
Collaborative filtering is responsible for what you see in a recommendation system, like when netflix or youtube recommends you a show or a video, that’s utilizing collaborative filtering (and other bits and bobs). Collaborative filtering uses similarities between items (take shows, for example), and similarities between users, and uses that data to do stuff.
For example, if you really like show A, and show B is similar to show A, then the system might recommend you show B. Alternatively, if you and your friend have very similar interests, and your friend has watched show C, then the system might recommend you show C.
The collaborative filtering tutorial goes over loading a movie database, and using that database to extrapolate similarities, the backbone of any recommendation system.
1 Like