Hello, sir, I am stuck in this situation for two weeks.
I am doing the same made a set col
and name into train, valid, test but I am using is_valid
col which combines test and validation split into one e.g. valid and my test is empty.
My dataloader is below:
def get_chestxray8(path:PathOrStr, bs:int, img_sz:int, valid_only_bbx:bool=False, tfms:bool=False, convert_mode:str='RGB',
normalize:bool=True, norm_stats:Tuple[Floats, Floats]=imagenet_stats, processor:Optional[Callable]=None,
**kwargs:Any)->DataBunch:
'''
TODO
'''
path = Path(path)
df = pd.read_pickle(path/'full_ds_bbx.pkl')
df['is_valid'] = df.set!='Train'
if valid_only_bbx: df = df[(df.set=='Train')]
if processor is not None: df = processor(df)
lbl_dict = df[['file','label']].set_index('file')['label'].to_dict()
def bbox_label_func(fn:str)->list: return lbl_dict[Path(fn).name]
lbls = ['No finding', 'Atelectasis', 'Cardiomegaly', 'Consolidation', 'Infiltration',
'Lung Opacity', 'Mass', 'Pleural effusion', 'Pleural thickening', 'Pneumothorax', 'Pulmonary fibrosis']
src = (CustomObjectItemList.from_df(df, path / 'images', cols='file', convert_mode=convert_mode)
.split_from_df('is_valid')
.label_from_func(bbox_label_func, classes=lbls))
if tfms: src = src.transform(get_transforms(**kwargs), size=img_sz, tfm_y=True)
data = src.databunch(bs=bs, collate_fn=multiclass_bb_pad_collate)
if normalize: data = data.normalize(stats=norm_stats)
return data
and I get this databunch, where test
and valid
combines and get me only valid
.
I get my databunch like this:
ImageDataBunch;
Train: LabelList (12662 items)
x: CustomObjectItemList
Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512)
y: CustomObjectCategoryList
ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512)
Path: /home/ali/Desktop/CX Product/RpSalWeaklyDet/images;
Valid: LabelList (2129 items)
x: CustomObjectItemList
Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512),Image (3, 512, 512)
y: CustomObjectCategoryList
ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512),ImageBBox (512, 512)
Path: /home/ali/Desktop/CX Product/RpSalWeaklyDet/images;
Test: None
Please suggest how I would add test set too without combining test + valid.
Thanks