For the course_v4, 2nd ipynb-- production. I was not able to get the Azure key. My free bing search api account expired. I am not able to get the key. What can I do?
Hi paidion hope all is well
Create you own data sets, upload them, skip the Azure part in the notebook and proceed to this section.
dest = 'images/grizzly.jpg' download_url(ims, dest)
Also Look at how the dataset and images are used in https://github.com/fastai/fastbook/blob/master/04_mnist_basics.ipynb for the 3, 7 classifier.
cheers hope this helps
Thanks you for your suggestions.
I tried the bing image search api a couple months ago. It’s rather easy and straight forwards. I also tried the google download, it was working with fastai v.1. As I am thinking to use the data for longer term after the course. I also have the copyright issues.
I will try to wait and see. How other might help with other alternatives.
here’s my alternative:
(@jeremy as requested)
urls=Array.from(document.querySelectorAll("img.tile--img__img")).map(el=> el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('src')).map(src => decodeURIComponent(src.split("?u="))); window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));
urls=Array.from(document.querySelectorAll("a.iusc")).reduce((acc, el) => el.hasAttribute("m") ? acc.concat(JSON.parse(el.getAttribute("m")).murl) : acc, ) window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));
After the free time expired on my azure account, I signed up for the paid service. Just got to make sure to shut down services when you are done & regenerate keys. This made it simpler for me to continue to follow this course.
@joedockrill Thanks for that trick. Do you have any idea if Bing TOS is different to Google?
@joedockrill thanks for your solution!
question, is there a way to turn the solution to a pypi package that can just be installed with pip?
oh apologies, maybe i didn’t understand it. lemme read it again (the notebook).
i was thinking of having a similar function like the one provided by jeremy
and i saw sth like
in your notebook. cant we just pip install sth and use the duckduckgo_search() function directly?
Yeah I was just coming back to reply again. It’s early here and I was dealing with a whiney teenager at the same time. Yes, I don’t see why not. I’ll look into it soon.
again thanks @joedockrill for your current solution.
i’ve just tried it out, works like a charm.
for other users looking for pip package, there’s bing-image-downloader, that does the job in 2 lines of code.
pip install bing-image-downloader
from bing_image_downloader import downloader downloader.download(query_string, limit=100, output_dir='dataset', adult_filter_off=True, force_replace=False, timeout=60)
i did actually turn mine into a package like @jimmiemunyi asked.
!pip install -q jmd_imagescraper from pathlib import Path root = Path().cwd()/"images" from jmd_imagescraper.core import * duckduckgo_search(root, "Cats", "cute kittens", max_results=100) duckduckgo_search(root, "Dogs", "cute puppies", max_results=100)
from jmd_imagescraper.imagecleaner import * display_image_cleaner(root)
This is brilliant! Thank you for doing this for the community!