Would this method be improved by shuffling the training or validation sets when creating the minibatches? (Or maybe using the previously demonstrated integer indexing to index random rows)? Or is it on the practitioner to decide whether that’s appropriate & do it as part of preprocessing?
1 Like
That’s the next steps 
2 Likes
Yes, sorry, I think found it https://docs.fast.ai/torch_core.html#Functions-to-deal-with-model-initialization
2 Likes
it’s more of a lazily evaluated sequence than a graph
The fastai library is written on top of PyTorch, for reasons outlined in these posts:
https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/
https://www.fast.ai/2018/10/02/fastai-ai/
We will be covering fastai for Swift for TensorFlow (S4TF) in the last 2 sessions of this course:
https://www.fast.ai/2019/03/06/fastai-swift/
2 Likes
This short chapter explains yield pretty well:
https://book.pythontips.com/en/latest/coroutines.html
13 Likes
That, and look in the layers module, bu we usually override PyTorch init when it’s faulty.
could someone explain coroutine in few sentences?.. Tried to read some related post before but could not understand it clearly.
it’s essentially a set of instructions for how to recreate some sort of iterable, correct?
Why do we always do zero_grad?
1 Like
This has been answered earlier in this thread., look for it
lazily executed instruction set
1 Like
I found a quick tutorial for yield: the-python-yield-keyword-explained
5 Likes
Are there times when you wouldn’t want to zero out the gradients or why do we have to do those two steps separately?
opt.step and opt.zero_grad are the two lines I’m curious about
2 Likes
otherwise it will be accumulated in future iterations.
Probably this one also could be helpful. Doesn’t cover all possible cases but probably can give some additional insights.
3 Likes
Rachel posted a link up there in the thread that explains the PyTorch’s team design choice, look for it, it’s really interesting.
2 Likes
Yes sorry, I didn’t formulate the question properly. I meant if it was paid any attention towards some integration with tf as well - not full integration, but some way to add value there as well. But I take it this wasn’t considered 
(I tried myself to integrate fast.ai to tf last month, but indeed didn’t find some natural integration points)
:), yes it will accumulate if we don’t clear it, but shouldn’t that be just a default, instead of setting it explicitly
1 Like
How does this ensure non-replacement? Is it because of yield - i.e. it “uses” the indices as it iterates along, and doesn’t reshuffle them every time?