Different plot from lr_find

tfms0 = get_transforms(do_flip=False, p_affine=0.00, p_lighting=0)
data = src.transform(tfms0, size=size, resize_method=ResizeMethod.SQUISH).databunch(bs=bs, num_workers=4).normalize(imagenet_stats)

Above is some of my codes. Though use the same augmentation, I still got different plot when lr_find done, can someone help me out of the problem? Thank in advance!

Hey @zzgong,

Would you mind specifying how you find them different?
To the best of my knowledge, since both Jeremy and you have not set a seed (a constant that will determine how pseudo-random numbers are generated), the shuffle in your dataloader will be different. And since lr_find is taking first images after this shuffle, it will not operate on the same images (at least in the same order) as Jeremy.

Same goes for a training that you would run with the same architecture, augmentation and LR than any other person. Randomness occurs at least in:

  1. the order of images that your model will be shown (assuming you left the shuffle=True to its default value in the dataloader)
  2. the data augmentations. For instance torchvision.transforms.transforms.RandomRotation(30) will rotate your image by an angle whose value is randomly picked between -30° and 30°.

Consequently, if you have about the same LR minimum and same overall shape of plot, it’s quite normal as per above. If not, it’s hard to tell without the entire code!

Here is a good explanation on pseudo-random numbers and seed: https://www.quora.com/What-is-seed-in-random-number-generation

I hope this helped, cheers!

Really appreciate your answer!
But i set random seed in my code before i train my model.

def seed_everything(seed=2019):
random.seed(seed)
os.environ[‘PYTHONHASHSEED’] = str(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
torch.backends.cudnn.deterministic = True
seed_everything()

And i also find there is no random seed or shuffle in Class ImageList where i feed my data into the model.

Hi @zzgong,

Good practice then! But my point is that, even if one of you two set a seed, if you both haven’t set the same seed value, this will not make a difference.

Are you positive that Jeremy also set his seed 2019? (If I recall well, Jeremy is more about not setting any seed)

Best regards,