Duckduckgo search not working

Use search_images_ddg from fastbook instead it still works.

2 Likes

Could you share how you were to secure the api key? I’ve searched all over that azure console and couldn’t find it

from duckduckgo_search import ddg_images
from fastcore.all import *
def search_images(term, max_images=200): return L(ddg_images(term, max_results=max_images)).itemgot(‘image’)
urls = search_images(‘bird photos’, max_images=1)

is the implementation like this?

I have recorded a loom for you explaining the steps.

1 Like

you don’t need an api key to use duck duck go, you may be trying to use bing search, finding and using an api key is described in the video tutorial as a “pain”, so not the recommended path.

# Check fastbook is installed
!pip install fastbook
# Import fastbook functions, in our case we'll be using: search_images_ddg
from fastbook import *
# create a function that will take an arbitrary search term and return a list of urls.
def search_images(term, max=30):
  print(f"Searching for '{term}'")
  # search_images_ddg comes from fastbook: https://github.com/fastai/fastbook/blob/master/utils.py#L45
  return search_images_ddg(term, max_images=max)

urls = search_images("cats eating ice cream")

10 Likes

with given function it returns

name ‘L’ is not defined

Do I need to install fastcore also?

Yes, since ddg was giving me the same issues, decided to go for bing search API.
The above video was really helpful, but would like to add that it ll only work once you have started the free trial (had to add my CC) but bing search api offers free 1K searches per month, so I m guessing it should be fine.

Also, for those who are looking to shift to bing search instead of ddg, you can refer to the fastai book (production).

I ll share the snipped of the code I used, in case it ll be helpful for others who want to shift to bing api. Don’t forget to import the necessary fastai /fastbook libraries.


key = os.environ.get('AZURE_SEARCH_KEY', 'your key here') //add key variable


bear_types = 'grizzly','black','teddy'
path = Path('bears')

from time import sleep

if not path.exists():
    path.mkdir()
    for o in bear_types:
        dest = (path/o)
        dest.mkdir(exist_ok=True)
        results = search_images_bing(key, f'{o} bear',  min_sz=128, max_images=150)
        download_images(dest, urls=results.attrgot('contentUrl'))
        sleep(10)  # Pause between searches to avoid over-loading server


1 Like

For anyone running into this, until the fix in the linked github issue is resolved:
The docs indicate that beginning May 12 2023 older duckduckgo versions will not work. I thought the update (-U) flag in !pip install -Uqq duckduckgo_search would pull the latest version but either it didn’t for some reason or a regression was introduced in a later version.
At any rate, explicitly asking for a version worked for me:

!pip install -Uqq duckduckgo_search==2.9.4

2 Likes

Looks like the fix has been committed. Release v3.0.1 · deedy5/duckduckgo_search · GitHub

2 Likes

I got {“code”:“DeploymentFailed”,“target”:“/subscriptions/03eef13f-3f60-4106-9c3c-ec622619d84c/resourceGroups/searchforfastaicourse/providers/Microsoft.Resources/deployments/Microsoft.BingSearch-20230518084223”,“message”:“At least one resource deployment operation failed. Please list deployment operations for details. Please see Deployment history - Azure Resource Manager | Microsoft Learn for usage details.”,“details”:[{“code”:“InsufficientQuota”,“message”:“Insufficient Quota”}]}
eve though I chose free tier

Yes! It has been fixed and everything works fine for me and hopefully for all you too. Thanks to all who participated and replied. Let’s go!!

I seem to get this error when doing !pip install -Uqq duckduckgo_search==2.9.4

Is there something I’m doing wrong?

what changes did you make to the code?

maybe remove one of the =
=2.9.4 instead of ==2.9.4 or even don’t specify version at all

Thanks a lot Iain - this is the only working solutions for me for now.

I just needed to change slightly the definition of the function:

def search_images(term, max_images=30):
    print(f"Searching for ‘{term}’")
     return search_images_ddg(term, max_images)

The function search_images_ddg expects max_n instead of max_images:
    def search_images_ddg(key,max_n=200)

2 Likes

Hey, I’ve just tried doing the same thing and I get the following error:

TypeError: search_images_ddg() got an unexpected keyword argument ‘max_n’

Although I see that in the definition of search_images_ddg I see it expects max_n. Very weird. Any ideas how to fix this?

OK it now seems to work with max_images instead of max_n for some reason, for anyone having the same issue.

@jsk123 , @rosh-eth and others who are running into this issue, I struggled with it as well and finally found that importing fastbook and using ‘search_images_ddg’ is what got everything to work in the end.

I’ve documented what I did in a kaggle notebook in the hope that it can help others get through this. I found this issue quite frustrating until I finally got it to work.

2 Likes

Thanks for your time putting this up. This is exactly what worked for me, and the only method that worked for me essentially. Cheers!