NotADirectoryError: [WinError 267] The directory name is invalid: data\\BTS\\valid\\features.npy'

My complete code is as below:

`import sys

sys.path.append(“C:\Users\Anurag\github_pro\fastai”)

from fastai.imports import *
from fastai.transforms import *
from fastai.conv_learner import *
from fastai.model import *
from fastai.dataset import *
from fastai.sgdr import *
from fastai.plots import *
PATH = “C:\Users\Anurag\github_pro\fastai\courses\dl1\data\BTS\”
sz=224
arch=resnet34
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.fit(0.01, 3)`

error coming up is as:

features.npy is generated in the valid folder.


What am I missing here?
Please help…
Thanks.

The from_paths expects that the train and valid folders will only contain subfolders which denote the labels. If you look at the source code (or the stack trace displayed) the only files that can be in those folders are the hidden .ipynb_checkpoints and .DS_Store files which the code ignores. Move the features.npy and labels.npy files from the folder.

1 Like

Thanks a lot.
It Worked!!

One more question, how is it testing the accuracy?
In test folder there is no folder like cats and dogs so from where it is comparing whether predicted result is right or not?

The accuracy metric you’re seeing during training is on the validation set (valid folder). Not from the test set. If the test set doesn’t have labels such as the case in the dogs/cats then all you will be able to get is the predictions. If you look at how from_paths is defined, you’ll see that since you aren’t specifying a test_name parameter in your code above, then the test set is not being used. Only the train and valid folders are being utilized since those parameters have default values of ‘train’ and ‘valid’. If you did want to make predictions on the test set, you’d need to specify the test_name parameter. If your test set also had labels (which in this case it doesn’t), you would also then specify test_with_labels=True.

def from_paths(cls, path, bs=64, tfms=(None,None), trn_name='train', val_name='valid', test_name=None, test_with_labels=False, num_workers=8):
1 Like

So, if I want to see the accuracy over testing data what should I do?

I tried setting test_name and test_with_labels true
and also put images of dogs and cats in different folders under test folder(tests).
But it was still showing the accuracy for validating set.

learn = ConvLearner.pretrained(arch, data, precompute=True,test_name=‘tests’, test_with_labels=True)
learn.fit(0.01, 3)`

There may be but I don’t know of anything that will directly calculate the accuracy over the test set directly but you can call the learn object’s predict_with_targs method with is_test=True, then calculate the accuracy yourself based on the returned prediction probabilities and the targets. I’ve provided some code which should do it for you.

log_preds,targs = learn.predict_with_targs(is_test=True)
correct = 0
preds = np.argmax(log_preds, axis=1)
for idx, pred in enumerate(preds):
    if pred == targs[idx]:
        correct += 1
accuracy = correct / len(preds)
len(preds), correct, accuracy
1 Like

NotADirectoryError Traceback (most recent call last)
in ()
148 transfer(os.path.join(file_path,file),image_path,size,aug_bool)
149 else:
–> 150 for file in os.listdir(file_path):
151 transfer(os.path.join(file_path,file),image_path,size,aug_bool)
152

NotADirectoryError: [WinError 267] The directory name is invalid: ‘C:\Users\Gayusowthariya\Anaconda3\lib\site-packages\ipykernel_launcher.py’

since my file path is in f folder,but why the error occurs in c folder.
kindly reply as soon as possible.