If you want to load images from your dataset on Kaggle, you can use the ImageDataLoaders.from_folder method from the fastai library. Just give it the path to your dataset and it’ll create a DataLoaders object for you. Here’s some example code:
from fastai.vision.all import *
path = Path(‘…/input/dataset’)
data = ImageDataLoaders.from_folder(path)
Just make sure your dataset is organized into separate folders for each category and you’re good to go!
Don’t delete that, that’s much more useful than what I was doing.
Thanks for it!
After I have the Data object, how do I pass it to the DataBlock?
I do have two folders with images, so I create two Data objects pointing to each folder and then pass them to DataBlock?
If you’ve already created a DataLoaders object using the ImageDataLoaders.from_folder method, you don’t need to worry about passing it to a DataBlock . The DataBlock API is just another way to create a DataLoaders object that gives you more control over how your data is processed.
I see I see
Thank you, I will give it a try using your method as it looks more clean.
After training my model, I got terrible results. It’s not getting the categorization accurately.
What’s the process of fixing this? I suppose I will learn in other lessons, as I just started lesson 2.
1.When you run a Colab notebook and download images using code in the notebook, the images are downloaded and stored in the cloud where the Colab notebook is running(same of Kaggle nbs). They are not stored on your local computer.
2. The verify_images function is a utility function provided by the fastai library to check if a set of image files can be opened. It does this by attempting to open each image file and returning a list of files that cannot be opened.
For more information about the function you can use these tricks:
?function-name:: Shows the definition and docstring for that function??function-name:: Shows the source code for that functiondoc(function-name):: Shows the definition, docstring and links to the documentation of the function3.Depends on how imbalanced the classes are in the dataset.This link might help
EDIT: a related answear
4.You can identify overfitting by looking at the traning loss and validation loss,if at some point the training loss is low and the validation loss is high that means that the model is memorizing the traning that and it dosent do well on unseen data (aka overfitting).
Thanks so much (Y)
try this:
from torchvision import models
resnet34 = models.resnet34(weights='ResNet34_Weights.DEFAULT')
before your code
Facing the following error after making the modificationr:
TypeError: ResNet.forward() got an unexpected keyword argument ‘pretrained’
Code:
from fastai.vision.all import *
from torchvision import models
resnet34 = models.resnet34(weights=‘ResNet34_Weights.DEFAULT’)
path = untar_data(URLs.PETS)/‘images’
def is_cat(x):
return x[0].isupper()
dls = ImageDataLoaders.from_name_func(
path, get_image_files(path), valid_pct=0.2,
label_func=is_cat, item_tfms=Resize(224))
learn = vision_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(2)
You can disable warnings with the following line of code.
import logging; logging.disable(logging.warning)
I see, so do just as @ForBo7 said and just disable warnings
I’m currently going through the is it a bird? notebook on Kaggle, but I ran into an error.
from duckduckgo_search import ddg_images
from fastcore.all import *
def search_images(term, max_images=30):
print(f"Searching for '{term}'")
return L(ddg_images(term, max_results=max_images)).itemgot('image')
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/tmp/ipykernel_17/1717929076.py in <module>
----> 1 from duckduckgo_search import ddg_images
2 from fastcore.all import *
3
4 def search_images(term, max_images=30):
5 print(f"Searching for '{term}'")
/opt/conda/lib/python3.7/site-packages/duckduckgo_search/__init__.py in <module>
8
9 # isort: off
---> 10 from .compat import (
11 ddg,
12 ddg_answers,
/opt/conda/lib/python3.7/site-packages/duckduckgo_search/compat.py in <module>
2 import warnings
3
----> 4 from .duckduckgo_search import DDGS
5
6 logger = logging.getLogger(__name__)
/opt/conda/lib/python3.7/site-packages/duckduckgo_search/duckduckgo_search.py in <module>
11
12 import requests
---> 13 from requests.exceptions import HTTPError, JSONDecodeError, Timeout
14 from requests.models import Response
15
ImportError: cannot import name 'JSONDecodeError' from 'requests.exceptions' (/opt/conda/lib/python3.7/site-packages/requests/exceptions.py)
It looks like an error from the duckduckgo_search package, but is it because the code is out of date? What else can I use instead? Is there a latest version of the code maybe?
you can try Google Images Search: Google-Images-Search · PyPI
I tried substituting googleimagesearch into the pip install and got a bunch of errors.
Then got this 'Nonetype" object is not subscriptable error when I add this code
gis = GoogleImagesSearch('XXXXXXXXXX', 'XXXXXX')
_search_params = {
'q': 'birds',
'num': 1,
}
urls = gis.search(_search_params)
urls[0]
What have I done wrong?
I’m running into the same issue as well
This is a “known issue”, at least there is a thread about this.
The following post solved it for me: Duckduckgo search not working - #18 by maitland
Thank you that one was solved. Got a new one though.
download_url(search_images_ddg(‘forest photos’, max_images=1)[0], ‘forest.jpg’, show_progress=False)
Image.open(‘forest.jpg’).to_thumb(256,256)
Missing: Training Diagnostics
When I run learn.fine_tune(3) in the notebooks or in iPython, I don’t get any tables returned with the training log. Can anyone suggest why this is happening and how I could fix it?
I’ve tried the following:
For all, there’s no output. Can anyone suggest why this is happening and how I could fix it?
TIA!
Solved!
Refer to this post: fastprogress progress bar not showing · Issue #13163 · microsoft/vscode-jupyter · GitHub
