Ddg not replying?

Hello,

I’ve been working on rewriting lesson1 notebook to use the search_images_ddg method locally on my laptop. I’m getting timeout errors.

Note that I’ve tried the search_images_ddg code that is installed via conda and the code in fastbook/utils.py at b7f756b49d4eb0d3ce96c0c29be98f4f293cde9f · fastai/fastbook · GitHub, which is different.

I even can’t ping duckduckgo.com, but can ping google.com. If I disconnect my cell phone from my home wifi and use cell-service I can search duckduckgo, but if I connect it to the home wifi the searches time out.

Searches via a browser to ddg through my home wifi ip seem to be locked out.

It appears that ddg has blocked my home-wifi ip perhaps?

Has anyone else had similar issues in using ddg?

The bing image search seems to have charges associated with it. Does anyone have an image search process that remains free and doesn’t get blocked?

Thanks,

mark

3 Likes

Explored a bit about the diffs between the search_images_ddg you get from conda install -c fastai fastbook vs the one you get if you use the code at the github repo link above and have a possible explanation if anyone runs into this.

Looks like the conda install version puts you in an infinite loop hitting urljson every time through. The infinite loop happens, at least for me, because an exception is thrown and the except block simply passes and you end up at the top of the loop again with no additional item added to the urls iterable, which is the loop exit criteria. In general, except: pass can be a little dangerous :frowning:

Just a guess, but DDG probably blocks an ip if it sees too many hits in a short period of time from it, which is likely what happened to me.

So I’d recommend:

  • someone update the code that one gets from ’ conda install -c fastai fastbook’ to not have the infinite loop
  • in the meantime, use the code from the github repo link above instead.

And thanks so much for the great tools, great teachings and great community!

I’m seeing similar issue on both colab and kaggle

I’ve updated fastbook now to fix that problem.

I’ve tried using both the GitHub snippet version and the original documented function for search_images_ddg and am getting a timeout on my end.

Running this on VSCode that’s connected to a remote Jupyter server on Paperspace.

I found the solution in the new course22 code. Thanks! course22/00-is-it-a-bird-creating-a-model-from-your-own-data.ipynb at master · fastai/course22 · GitHub