Strange, that happened to me also. I got a score 0.84 from Kaggle the first time I tried, but then I just repeated everything trying to isolate a possible mistake I might have made, and got a score of 0.91777 from Kaggle.
1 Like
interesting but I get the same 0.84 score no matter how many times I repeat. when you say tried to isolate a possible mistake, what do you mean? did you look at individual wrong predictions?
I thought I might have messed up something when I repeated the same steps three times using three test folders to produce three result files which I then combined manually (to avoid my Kernel dying). The second time I tried I also ended up changing one thing: I did not apply data = data.resize(int(sz*1.3), ‘tmp’). I left this step out mainly because I wanted to simplify things a little bit (no need to copy the weights file from “data/planet/tmp/83/models” to “data/planet/models”) and also because I didn’t really understand this step. How could it be beneficial to first resize the files to size 64x64 and then immediately resize them to 83x83?
I don’t know if this is what made the difference. This is just what I did and what my thoughts were when trying this.
I don’t think that made the difference. Jeremy said in the lecture that resize step is for speeding up the run time. I did not do that step and still get 0.85. I suspect it is due to my test files. Thanks anyways.
Hey Amrit, did you end up resolving the “ValueError: Length of values does not match length of index” error? Debugging the same one at the moment
Hey @fero it got fixed after the update mentioned in this thread, Kaggle Comp: Plant Seedlings Classification and then it worked fine. I am assuming you are using the latest fastai libraries?
Yes, using the latest libraries and notebook but still getting the “list index out of range” error when using the following code to export the predictions, and “Length of values does not match length of index” after removing the ‘for’ loop (lines 4 and 5):
tta = learn.TTA(is_test=True)
test_fnames = data.test_ds.fnames
for i in range(len(test_fnames)):
test_fnames[i] = test_fnames[i].split("/")[1].split(".")[0]
classes = np.array(data.classes, dtype=str)
res = [[" ".join(classes[np.where(p > 0.5)])] for pp in tta[0] for p in pp]
submission = pd.DataFrame(data=res)
submission.columns = ["tags"]
submission.insert(0, 'image_name', test_fnames)
submission.to_csv(PATH+"Planet_Sub.csv", index=False)I am following the lesson2 notebook for this as shown in lecture.
I want to understand the meaning of
from planet import f2
metrics=[f2]
from cell 5.
Is planet a package, as we’re importing f2?
What is f2?
I have seen the files on the contest but f2 isn’t there.
Hi all,
Like @layla.tadjpour I am running out of memory when running learn.predict(is_test=True) on the full test set
Is there a way of splitting the test data and running learn.predict on batches of images without splitting them into separate folders?
I’ve been trying to read through the library and figure it out, but my python is not there yet.
The best I get is:
predictions = []
for i in range(0,len(data.test_dl.dataset)-1):
p = learn.predict_array(data.test_ds[i])
predictions.append(p)
Which yields a NotImplementedError from T(a) in core.py
Alternatively, If I were to give up and go down the route of splitting it into separate folders, would I be creating another instance of ImageClassifierData, using ImageData, or the function set_data? (Or something else?)
EDIT:
I worked around the issue by reducing my batch size to 64. It would be nice to be able to set the batch size for training v. prediction independently. If anyone knows of a way to do this, I’m all ears!
f2 is a function located in the file planet.py.
You can find it in fastai/courses/dl1/planet.py
It is the metric used to assess performance in the competition.
see: https://www.kaggle.com/c/planet-understanding-the-amazon-from-space#evaluation
If you’re confused about the difference between a loss function and a metric (I was) see:
1 Like
Hi all,
Has anyone run the learning rate finder on kaggle planet dataset following the code in the lesson 2 notebook:
lrf=learn.lr_find()
learn.sched.plot()
It gives me the following error and I was wondering if anyone can suggest what i might be doing wrong?
ValueError: Expected array-like (array or non-string sequence), got
1 0 0 … 0 0 1
0 0 0 … 0 0 0
1 0 0 … 0 0 0
… ⋱ …
1 0 0 … 0 0 1
0 0 0 … 0 0 0
0 0 0 … 0 0 1
[torch.cuda.FloatTensor of size 64x17 (GPU 0)]
Hi everyone,
I am having a small problem interms of predicting on the test data.
- Firstly I downloaded the primary test jpeg dataset and the additional test jpeg dataset.
- I moved the files from the additional test data folder to the primary test folder.
Now when i run the model and try to implement TTF(isTest = True), It is raising an exception telling me img-20xx is not found. I dont understand what is happening.
Can someone please explain how how we are supposed to prep the test data?
I’m getting the same error … running on fastai AMI, latest pull of fastai repo
learn.lr_find()
ValueError: Expected array-like (array or non-string sequence), got
1 0 0 … 0 0 1
0 0 0 … 0 0 0
1 0 0 … 0 0 0
… ⋱ …
1 0 0 … 0 0 1
0 0 0 … 0 0 0
0 0 0 … 0 0 1
[torch.FloatTensor of size 64x17]
The problem appears to be in the f2 metric. I tweaked the function in planet.py slightly to convert the torch.Tensor object to numpy arrays
def f2(preds, targs, start=0.17, end=0.24, step=0.01):
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return max([fbeta_score(targs.numpy(), (preds.numpy()>th), 2, average='samples')
for th in np.arange(start,end,step)])
and that seemed to fix it (ie. output learning rate-loss plot looks identical to the notebook in the video lecture).
Ha yeah, I had run into a separate issue where Torch wasn’t accessing my GPU properly (that’s why our errors are slightly different [torch.FloatTensor of size 64x17] vs [torch.cuda.FloatTensor of size 64x17 (GPU 0)]).
If you are operating with GPU you first need to switch the tensor to cpu before converting to a np array. This seems to work ok
def f2(preds, targs, start=0.17, end=0.24, step=0.01):
with warnings.catch_warnings():
warnings.simplefilter("ignore")
return max([fbeta_score(targs.cpu().numpy(), (preds.cpu().numpy()>th), 2, average='samples')
for th in np.arange(start,end,step)])
1 Like
Thanks @Cam Somehow the original code started working again now. I am not entirely sure what happened over the last 2 weeks.
Hi Kevin,
Thanks for your sharing.
May I ask how long it took you to run all the models?
My 256 size model ran over one hour and the TTA was running until the kernel was dead.
I am not sure if I am doing it right.
@nithiroj I’ve had the same issue. I went into the ~/fastai/models.py line 43 and changed the y to call y.float() and same for line 54 to get it to work but not sure if that’s the right fix though. BinaryCrossEntrpopy expects a float tensor target and not a long tensor which is causing this error. But since this is a classification task it would make sense that our target is a one-hot encoded vector represented as a long tensor.