Hey, I want to build my deep learning framework (like Chainer or pytorch) so that I need to know for this if there is enough knowledge gained from the two parts of the fast.ai course? if not, advise what else I need to know and where to get this knowledge.
I would say that you can take inspiration from some exciting frameworks to simplify your task. Some time ago I’ve written a post about custom training loop implementation. Probably it could give you some clues. I’ve started even writing a deep learning library but abandoned it for now because of some new tasks
But there are many much better examples (not talking about
fastai itself), like (1) ignite and (2) catalyst. The last one was inspired by
fastai I believe, but has a bit differently structured codebase that plays nice with PyTorch classes.
this is not exactly what I meant. I want to create not a library but a framework such as Chainer or PyTorch (of course, not so large-scale), so I asked. Will there be enough knowledge from parts 1 and 2 of fast.ai courses? or do i need something else?
Ah I see. Well, in this case, you need to dig deeper into the chain rule and autodiff methods. I think that Part 2 gives a nice overview of the topic so you should be able to start. (Probably you’ll need some additional sources as you go). Especially helpful could be the lectures on Swift (last two in the Part 2). They show how to build a “custom chainer” solutions.
From a practical point of view, you can also look into
spaCy library. They use custom diff framework called
thinc, probably it could give you some additional inspiration and examples.