Hi
Since Kaggle does not allow to use GPU for custom modules, downgraded catsanddogs in kaggle with fastai 0.7.x is very slow. So I am trying to use latest fastai for same cats and dogs example. However, I get very pathetic accuracy. Since its top down approach, after spending couple of hours already., I really want to move on Part 2 tutorial. So can any one please let me know what is wrong in below code? I see, I do not have validation folder, but I have no idea what is that at the moment. I could peel that later, for now, want a running model. Kindly help.
Data preperation
from fastai.vision import *
from fastai.datasets import * # this is needed for untar_data
from fastai.metrics import * # for accuracy
from fastai.vision.data import *
from fastai import *
SOURCE = "../input/"
DEST = "../dogsandcats/"
TRAIN = DEST + 'train/'
TEST = DEST + 'test/'
# write to a temporary writeable folder
# you can disable this part if dataset not changed
import shutil
if os.path.exists(TEST):
shutil.rmtree(TEST)
if os.path.exists(TRAIN):
shutil.rmtree(TRAIN)
shutil.copytree(SOURCE + 'test', DEST + 'test')
shutil.copytree(SOURCE + 'train', DEST + 'train')
# end of what I just said about disabling copy paste
from subprocess import check_output
print(check_output(["ls", DEST]).decode("utf8"))
Data Labelling
fnames = get_files(DEST + 'train', recurse=True)
# print(fnames)
def get_labels(file_path): return 'dogs' if 'dog.' in str(file_path) else 'cats'
dataf = ImageDataBunch.from_name_func(DEST,fnames, label_func=get_labels, size=24, bs=64, valid_pct=0.90, ds_tfms=get_transforms(do_flip=False))
dataf.classes
Training
learn = create_cnn(dataf, models.resnet34, metrics=[accuracy], pretrained=True)
learn.fit(1)
Alternate failed approach
I also tried this, but that results in error as below.
path = untar_data(URLs.DOGS)
data = ImageDataBunch.from_folder(path)
learn = create_cnn(data, models.resnet18, metrics=accuracy)
learn.fit(1)
Update: I think I solved it as below but am newbie here. Please add any feedback for betterment.
Also of course, if its still wrong.
Alternate kinda worked approach:
I finally tried below tweaks, it runs with 94% but had to limit num_workers.
path = untar_data(URLs.DOGS)
data = ImageDataBunch.from_folder(path, ds_tfms=get_transforms(), size=112, num_workers=2)
learn = create_cnn(data, models.resnet18, metrics=accuracy, pretrained=True)
learn.fit(1)
