fredguth
(Fred Guth)
July 20, 2018, 10:06pm
1
I have a problem similar to Lesson 14 (Carvana Segmentation). I am following Carvana’s notebook but I need to use a test dataset.
Unfortunately, get_ds code has no comments and I can’t understand how to add a test dataset.
Here is what I tried:
datasets = ImageData.get_ds(MatchedFilesDataset, (trn_x,trn_y), (val_x,val_y), tfms, path=PATH, test=testFilenames)
md = ImageData(PATH, datasets, bs, num_workers=8, classes=None)
where:
['Validation_Balanced_256/ISIC_0012585.jpg',
'Validation_Balanced_256/ISIC_0022029.jpg',
'Validation_Balanced_256/ISIC_0022192.jpg']
Can someone help?
1 Like
charming
(Charm)
July 23, 2018, 1:55pm
2
i also have this problem , If you solve it, please help me
fredguth
(Fred Guth)
July 23, 2018, 4:58pm
3
I solved and forgot to mention. Create a tuple test=(testfilenames, testfilenames)
where testfilenames is a list of all files in test dataset (with path).
Then:
datasets = ImageData.get_ds(MatchedFilesDataset, (trn_x,trn_y), (val_x,val_y), tfms, path=PATH, test=test)
7 Likes
charming
(Charm)
July 24, 2018, 2:19am
4
Thanks! That’s very helpful. But I was confused by another question.
When submitted in the kaggle competition, how to convert the last prediction of the network to rle_mask.
fredguth
(Fred Guth)
July 24, 2018, 4:13am
5
No idea. I was just applying Lesson’s 14 implementation for the Carvana challenge to another segmentation problem.
charming
(Charm)
July 24, 2018, 6:23am
6
Thanks~
If you are interested, you can check out the latest split games in kaggle.
Hello guys,
I still have problems regarding read in test data. Can you guys help me out with the code below please?
TESTPATH = Path("../data/tgs/test")
test = (testname, TESTPATH) #testname is a list of the files in test folder.
tfms = tfms_from_model(resnet34, sz, crop_type=CropType.NO, tfm_y=TfmType.CLASS, aug_tfms=aug_tfms)
datasets = ImageData.get_ds(MatchedFilesDataset, (trn_x,trn_y), (val_x,val_y), tfms, test = test, path= PATH)
md = ImageData(PATH, datasets, bs, num_workers=16, classes=None)
denorm = md.trn_ds.denorm
Also, should I change the parameters in MatchedFilesDataset as well?
Many thanks!
charming
(Charm)
August 1, 2018, 2:44am
8
test = (TESTPATH/testname,TESTPATH/testname)
try this
Hi Charm,
Thanks a lot! I think it works.
But I got another problem here, should I change MatchedFilesDataset function as well? as this error comes out.
class MatchedFilesDataset(FilesDataset):
def __init__(self, fnames, y, transform, path):
self.y=y
assert(len(fnames)==len(y))
super().__init__(fnames, transform, path)
def get_y(self, i): return open_image(os.path.join(self.path, self.y[i]))
def get_c(self): return 0
import pandas as pd
testname = pd.read_csv("filename.csv")
testname = ''.join(testname['0'].values.tolist())
TESTPATH = Path("../data/tgs/test")
test = (TESTPATH/testname, TESTPATH/testname) #testname is a list of the files in test folder.
tfms = tfms_from_model(resnet34, sz, crop_type=CropType.NO, tfm_y=TfmType.CLASS, aug_tfms=aug_tfms)
datasets = ImageData.get_ds(MatchedFilesDataset, (trn_x,trn_y), (val_x,val_y), tfms, test = test, path= PATH)
md = ImageData(PATH, datasets, bs, num_workers=16, classes=None)
denorm = md.trn_ds.denorm
<ipython-input-21-055d0623841d> in __init__(self, fnames, y, transform, path)
2 def __init__(self, fnames, y, transform, path):
3 self.y=y
----> 4 assert(len(fnames)==len(y))
5 super().__init__(fnames, transform, path)
6 def get_y(self, i): return open_image(os.path.join(self.path, self.y[i]))
TypeError: object of type 'PosixPath' has no len()
charming
(Charm)
August 1, 2018, 7:20am
11
x_names = np.array([Path(TRAIN_DN)/o for o in masks_csv[‘img’]])
test_name = (x_names ,x_names )
The format should be the same as trn_x in the training set
charming
(Charm)
August 1, 2018, 1:25pm
13
congratulations, do you konw how to convert the last prediction of the network to rle_mask,when submitted in the kaggle competition?
I am not there yet, I need to finish IoU part first. I will let you know when I work it out.
By the way, do you have a problem running the code below?
x,y = next(iter(md.trn_dl))
It is so weird that it fails to run on colab, but it runs perfectly well on my laptop …
charming
(Charm)
August 2, 2018, 7:25am
15
I just runs on my laptop , it’s no problem.
No worries, false alarm, just a batch size problem. thanks for the help!
maw501
(Mark Worrall)
August 3, 2018, 3:44pm
17
@charming Thanks for this, I was stuck here too.
1 Like
tony_m
(Anthony)
August 14, 2018, 6:15am
18
Thank you! Now I’m also looking for how to convert to rle_mask.
1 Like