Hello Seniors/experts I need urgent help please help or tag someone
@Patrick5 @frozenbanana @muellerzr
I am working on medical imaging where I am diagnosing chest diseases using a chest-Xray. Metadata of images is stored in a CSV like the label, BoundingBox, test/train/valid set
I already split the data in train, valid, valid in CSV now the code I am using, loads only train and valid databunch uing fastai library. The test set is empty and I want to include that test set in my databunch to test the model accuracy. I am sharing my data loader.
def get_chestxray(path:PathOrStr, bs:int, img_sz:int, valid_only_bbx:bool=False, tfms:bool=False, convert_mode:str='RGB',
normalize:bool=True, norm_stats:Tuple[Floats, Floats]=imagenet_stats, processor:Optional[Callable]=None,
**kwargs:Any)->DataBunch:
'''
TODO
'''
path = Path(path)
df = pd.read_pickle(path/'full_ds_bbx.pkl')
df['is_valid'] = df.set!='Train'
if valid_only_bbx: df = df[(df.set=='Train')]
if processor is not None: df = processor(df)
lbl_dict = df[['file','label']].set_index('file')['label'].to_dict()
def bbox_label_func(fn:str)->list: return lbl_dict[Path(fn).name]
lbls = ['No finding', 'Atelectasis', 'Cardiomegaly', 'Consolidation',
'Infiltration', 'Lung Opacity', 'Mass', 'Pleural effusion',
'Pleural thickening', 'Pneumothorax', 'Pulmonary fibrosis']
src = (CustomObjectItemList.from_df(df, path / 'images', cols='file',
convert_mode=convert_mode).split_from_df('is_valid').label_from_func(bbox_label_func, classes=lbls))
if tfms: src = src.transform(get_transforms(**kwargs), size=img_sz, tfm_y=True)
data = src.databunch(bs=bs, collate_fn=multiclass_bb_pad_collate)
if normalize: data = data.normalize(stats=norm_stats)
return data
I am figuring out that why my data loader is not loading the test set.
I am doing changes but couldn’t find a code or I can’t understand how it works.
Please help how to make and add the code test snippet to include the test set in my databunch or give any example which loads data from a data frame. Thanks