Thanks. These are the lines I get when I run path = untar_data(URLs.CAMVID)
and then print out the path
variable.
[PosixPath('/home/jupyter/.fastai/data/camvid/valid.txt'),
PosixPath('/home/jupyter/.fastai/data/camvid/labels'),
PosixPath('/home/jupyter/.fastai/data/camvid/images'),
PosixPath('/home/jupyter/.fastai/data/camvid/codes.txt')]
which I think is what we expect.
Now, after running untar_data
, I still don’t see the files in my folder. But this is likely because my data
folder is in the path tutorials/fastai/course-v3/nbs/dl1
. So the pictures are not downloading into my folder since the path inside URLs.CAMVID are different.
Thus far, I have assumed that this is supposed to be the case. Whenever I run the jupyter notebook instance (as advised in the fastai webpage), the ‘root’ folder I have access to is only tutorials
and not home
or jupyter
. So, I have assumed that (due to resource constraints?) that pictures downloaded by untar_data
are downloaded into some other folder home/jupyter/.fastai ...
in which I am not supposed to have access to.
Sorry I am not a data engineer so I have no idea how the structure is supposed to work.
Anyhow, just FYI, I tried running your Colab notebook. I ran into an OS error in the cell:
codes = np.loadtxt(path/'codes.txt', dtype=str); codes
The OS error is :
data/camvid-small/codes.txt not found.
But that’s because the data was originally downloaded to data/camvid
and not data/camvid-small
. That should mean that the path variable got modified to data/camvid-small
after untar_data
was called. I can’t see which line of code modified that though. Anyhow, once I manually changed the path to data/camvid/codes.txt
, it ran fine.
Regardless, I also just tried running my own jupyter notebook in GCP again. For some strange reason, everything works now. I don’t know if running your Colab notebook downloaded the pictures into my computer/whichever destination folder. But it works.
One guess as to what is causing all this is that sometimes the connection to the web server to download the images may not be stable, or that not all the data/files get downloaded, and that untar_data
does not check if all files are downloaded properly. The user needs to check it himself/herself. But that’s just my guess.
Finally, something that I did forget to mention, is that the ‘Subset classes’ block in the jupyter notebook was commented out in the first place, so I wasn’t meant to run it. I only uncommented it because I was experiencing OSError (similar to what I just experienced in your Colab notebook) where the codes.txt
file was not being located.
Thanks again for all of your help!