Same File Path Error while Resizing images - lesson 1

[kashish18 (Kashish Mukheja)]

Yeah, I did that but still faced the same error

[wyquek (Qbiwan)]

That’s strange.
I got the same error when I tried to run this cell in Kaggle the second time too.

SameFileError: Path('bird_or_not/forest/43bf75a4-3020-4ad6-9f82-86e1895a93ee.jpg') 
and Path('bird_or_not/forest/43bf75a4-3020-4ad6-9f82-86e1895a93ee.jpg') 
are the same file

but once I delete the folders using shutil.rmtree(path) as shown below I could run the same cell many times

searches = 'forest','bird'
path = Path('bird_or_not')

import shutil
shutil.rmtree(path)

for o in searches:
    dest = (path/o)
    dest.mkdir(exist_ok=True, parents=True)
    download_images(dest, urls=search_images(f'{o} photo'))
    resize_images(path/o, max_size=400, dest=path/o)

[longm89 (Mai Tien Long)]

Hi,

I got the same error. What I did to solve the problem was to use 2 folders: one folder for downloading the images, and the other for containing the resized images:

searches = fruits_list
downloaded_path = Path("downloaded_fruits")
resized_path = Path("resized_fruits")
for o in searches:
    print(o)
    downloaded_dest = (downloaded_path/o)
    downloaded_dest.mkdir(exist_ok = True, parents = True)
    download_images(downloaded_dest, urls = search_images(f"{o} fruit photo"))
    resize_images(downloaded_path/o, max_size = 400, dest = resized_path/o)

Best,
Long.

[kashish18 (Kashish Mukheja)]

Thanks, this worked !! but I’m curious when running the notebook without changing anything (i.e., just executing the is_bird classifier that Jeremy created), I didn’t encounter this error. However, when tried with a new dataset, with almost no change in the code, the cell failed with the above error.

[kashish18 (Kashish Mukheja)]

Thanks for the answer, but that didn’t work in my case . I had to provide a different destination path for resized_images to proceed further.

[kurianbenoy]

@kashish18 can you check if this works for you

Lesson 1 official topic

Looking at your code it looks like the way you have used resize_images method looks kind of wrong to me, as you are resizing to same folder. In Jeremy’s notebook in Paddy competition, I have seen him resizing like the below code: trn_path = Path('sml') resize_images(path/'train_images', dest=trn_path, max_size=256, recurse=True) I have tweaked the code similarly and it’s running as expected when I tried running it as a python script. from fastcore.all import * from fastai.vision.all import * …

[kurianbenoy]

Yeah I also found the same hypothesis correct. The reason probably is because how resize_images function is written in fastai:

[kashish18 (Kashish Mukheja)]

Yes, that worked, Thanks! However, I was referring to this Notebook : Is it a bird? Creating a model from your own data | Kaggle , where the dest parameter in resize_images is same as the source path.

[Christophe (VERGNES)]

Hello, I had also the same issue. I investigate and found that in FastAi code, resize images ( fastai/utils.py at b3ec3e1ab032a34777565d1649f091110ff2fa8f · fastai/fastai · GitHub) call the function shutil.copy2 even when the file doesn’t need to be modified (which could happen if image is small enough and dest folder is same than source folder). And shutil.copy2 doesn’t accept source and destination file to be the exactly the same.
I’ve applied the same workaround than you : define a destination folder distinct from source path

Furthermore, I will try to propose a PR to fast AI code in order to fix this issue.

[Christophe (VERGNES)]

Hi everybody,
Here is the issue : SameFileError on resizeImages · Issue #3744 · fastai/fastai · GitHub

And the PR : fix same file error message when resizing image by cvergnes · Pull Request #3743 · fastai/fastai · GitHub

Hope it will help

[kashish18 (Kashish Mukheja)]

Awesome Thanks!

[ashish_patel]

Hi Long,

Thank you for helping out with the error. If possible, please share the link of your notebook, this will really help.

Regards,
Ashish

[ashish_patel]

Hi Kashish, I would like to go through your corrected code. If possible, please share the link of your note book. Thanks in advance.

[zsweng]

Hi Everyone,

Ran into the same issue as well and was stuck for two days. Thanks so much for all the helpful sharing, greatly appreciate it!

[kashish18 (Kashish Mukheja)]

Getting Started - Creating Categorical Image Predictor Models | fastpages Here’s a blog post I made for lesson 1, the section that I shared contains the corrected code

[ashish_patel]

Thank you Kashish. I have read the blog and thanks for explaining the snippets of code at the end. This is really helpful.

[bitcoindoctor (Joe Terry)]

Just three days … that’s all it took for this community to identify and resolve a frustrating error. Lesson 1 is awesome, but to really master it, you have to test the code, try something different and the moment you do, you get a very difficult error to resolve.

But, not for this community. Thanks everyone.

First it was @longm89 who discovers a workaround, then a PR is issued, then @kashish18 publishes a very clean clear blog post that explains everything with a fix before the PR is fixed.

Thanks Everyone !!

[vfross (Valentin F.)]

Another workaround, that worked for me and avoids having two folders uses a try-except statement. Imho this also save a little storage. Are there any disadvantages in following this approach?

import shutil
try:
      resize_images(path/o, max_size=400, dest=path/o)
except(shutil.SameFileError):
      print("SameFileError occured, and was ignored")

[jeremy (Jeremy Howard)]

@vfross

Are there any disadvantages in following this approach?

The disadvantage of that approach is that all images after the 1st conflicting name will be skipped.

[vfross (Valentin F.)]

Thanks! That makes sense