I want to use shuffle=True for imagedatabunch in fastaiv1

Part 1 (2019) Advanced (Part 1 v3)
1 

batch_size = 32

do_flip = True
flip_vert = True 
max_rotate = 90 
max_zoom = 1.1 
max_lighting = 0.2
max_warp = 0.2
p_affine = 0.75 
p_lighting = 0.75 

tfms = get_transforms(do_flip=do_flip,
                      flip_vert=flip_vert,
                      max_rotate=max_rotate,
                      max_zoom=max_zoom,
                      max_lighting=max_lighting,
                      max_warp=max_warp,
                      p_affine=p_affine,
                      p_lighting=p_lighting)

train, valid = ObjectItemListSlide(train_images), ObjectItemListSlide(valid_images)
item_list = ItemLists(".", train, valid)
lls = item_list.label_from_func(lambda x: x.y, label_cls=SlideObjectCategoryList)
lls = lls.transform(tfms, tfm_y=True, size=patch_size)
data = lls.databunch(bs=batch_size, collate_fn=bb_pad_collate,num_workers=0,shuffle=True).normalize()

I want random shuffling of images during training , how can this be achieved, please all suggestions are welcome.
Note: This is fastaiv1 notebook code , so any kind of monkey patching of functions all welcome so that @patched code in the v2 and v3 could work for this scenario.
Thanking You,
Harshit

2

Hi @joshiharshit5077,

I think you will have to implement the get_idxs function to sample the data in a random manner. However, I know how to do so in fastai2 but not in fastai1.

There is a small section here which might be of interest to you to implement random sampling of your dataloader.

Hope this helps.

Thanks,
Vinayak

1 Like
3

hey @ElisonSherton thanks for your suggestion, but the first part seems blurry from my code’s POV could you please provide the link for your notebook, so I could see how you implemented the fastaiv2 library for midog dataset or for any dataset with MSCOCO format like annotations.
Your notebooks/codes would be really helpful for my research work, as I’m stuck on most of this training due to the transition from FASTAIv1 to v2 library.
Thanking you in advance,
Harshit Joshi

4

Hello Harshit, I’m curious as to why you prefer to user fastaiv1? Is it a lot of code to port over to fastaiv2 (which I’m guessing is just time and cost prohibitive?)

5

I don’t prefer to use fastai v1 , i really want my code to be shifted to fastaiv2 its just that the dataset I’m using is in MSCOCO format and that too using json file containing labels and I used the dataset from completion where they gave a sample notebook , and it is working fine , but when I try to use it in fastaiv2(from a notebook that uses fastaiv2 MSCOCO format same as min) it keeps on giving so many errors some errors are showing 0 results on google search, I have been searching days for transforming my code to fastai v2, please suggest some help.
My research has halted at this point because of this.
LINKS FOR CHALLENGE AND NOTEBOOK I USED:
https://imig.science/midog2021/download-dataset/
(refer to notebook at end of page)

Also if you have any suggestions of using any other pytorch library ,they are most welcome.
Please share among your peers.
Thanking you in advance,
Warm regards,
Harshit

6

notebook that im using for FASTAIv1 for MIDOD Challenge

This notebook contains somewhat that code , but it is edited at few places because I wanted to use fit.sgdr which is not available in v1 library.

7

Also, I tried running fastaiv2 notebook on Kaggle but keeps on throwing no module found error for fastai.core and fastai.callbacks, I tried all the method of installation of fastai .
If you guys have a working example of v2 notebook on kaggle, kindly share it.