I can’t find pretrained ResNext in vision.models. Has it been removed/moved somewhere else in Fastai V1?
You should find it models, e.g, models.resnet34, from the torch library.
I need “ResNext” not Resnet.
Ups, then maybe this can help you: Using Pretrained Models from Cadene repository
I am already using it with PyTorch, but having everything in Fastai would have been simpler. I don’t understand why was it removed in Fastai v1. It was there in v0.7! @jeremy Fastai v1 is already released right, or is it still under development?
They code snippet for fastai v1 is still in the subforum for the v3 course part 1 (which I guess is not open yet for the public): https://forums.fast.ai/t/lesson-5-advanced-discussion/30865/40
PS: I sent you the relevant code snippet.
@MicPie Thanks a lot for your help.
I have two questions though!
-
So this is how it’s going to be? It’s not going to be added to Fastai, and we have to use separate libraries? Any reasoning behind the same? Is it possible to merge Cadene’s repository into fastai.models?
-
I was going through the Fastai v1 documentation, and even as a regular Pytorch user, I am finding the code-structure super-confusing. Is it just my poor understanding skills, or are you facing the same problems too? The way the whole code base is divided into modules and sub-modules doesn’t seem as intuitive as say Pytorch’s structure. Would like to know the views of regular users of Fastai library. I was thinking of making a switch from Pytorch to Fastai, given the high level abstraction to many useful tools, but finding it very difficult to find my way through…
2 Likes
I find it a bit difficult too. But I remember Jeremy said he will expand the documentation more and provides more examples.
What helped me, is to open a pycharm IDE and using the same env of fastai, and when I want to debug something not so easy, or if I don’t know what are the arguments that should be passed, I will run the code on pycharm, and it is helpful more than Jupyter. I would set a breakpoint, and open the debug console of pycharm and investigate the code much more intuitively than what it was in Jupyter nb.
For some reason, Jupyter was not able sometimes to show the arguments that can be passed into a method. Pycharm doing much better job in this. Also Pycharm is very useful in navigating the definitions and declarations of the methods.
Other people who use remote servers use Vim or emacs… etc.
For me, I think I have to strengthen my OOP skills to understand fastai codebase better. How is your OOP skills? If it is good, then maybe I am wrongly thinking that I need more OOP skills to understand fastai code better!
I am interested to know others opinion in this too. How to understand fastai codebase better? Learn more OOP?
I remember Jeremy said too, more models are coming into fastai. In the meantime use the code that @MicPie provided you. I tried it and it works, and it is easy to use any ~40 additional models in Cadene.
2 Likes
Using a IDE (I use VS Code) and looking more into Python OOP is also my recipe combined with playing around with the functions/code.
Looking a little bit into PyTorch to understand why fastai is built the way it is built is also helpful (starting from the general concepts).
Also familiarization with the Python debugger is quite handy to track down errors.
Everything is step by step, as long as you make (small) progress and learn something on the way you should be doing fine.
1 Like
It seems even the great @radek is looking into enhancing his OOP skills in Python. I don’t know what is the reason though. And would be very interested to know if he is reading this.
But I suspect it is for enhancing his fastai skills too, since he is a Ruby developer. Which makes me even more convinced that to understand fastai really really well, you should be an OOP Ninja.
I haven’t read radek’s article yet, but seems very good.
You are right about Pytorch too. It helped a lot to take Udacity’s excellent Pytorch course after finishing fastai part1. The top-down approach of Jeremy has its own pros and cons.
Pros : you can do exciting things from the very beginning.
Cons: We will feel uncomfortable to find a lot of gaps in our understanding and we should actively seek to learn the basic stuff later.
Still I find top-down approach better for me. I could learn much faster and better than the other approaches. And it is much more exciting and rewarding.
But what intrigued me is that even @Chinmoy who is a regular Pytorch user find it difficult to understand fastai codebase. I cannot see but sharpening OOP skills is the way to go.
1 Like
I am not really sure I would agree - probably the best method to understand fastai code is to spend time with the library, changing things in code, reading it, trying things out, etc. But I honestly do not know as I have not attempted this - I am only taking a small bite at a time when I need to figure out how something works. Don’t think this approach does the library justice though and I hope I will be able to spend more time with it, looking at the code. There is one other aspect that relates to this - it seems that the best way to make sure you understand how something works is to rebuild it yourself (or some simplified version of it) so taking a chunk of the library and putting it in your own words might be the way to go as well and is something I have thought about doing.
Why do I care about OOP? Good things in my life seem to come through my ability to program. When I started to give fastai a proper go, I realized my ability to do ML / DL is mostly bounded by my ability to read and write code and being nearly a year and a half into this journey, I don’t think this has changed much if at all.
But maybe that is just rationalization 
I seem to be going through a phase where I am enjoying thinking about what makes code readable and easy to maintain and share with others and this is genuinely fun. There seems to be a richness of ideas out there that I have not suspected existed. Much of my interest in programming has been rekindled by ML related things, such as this awsome fastai style guide, so I think that is really cool and is yet another thing I am very grateful to the fastai community for (the list of things I have learned from hanging around these parts is so long… its hard for me to believe).
4 Likes
Dear radek.
Your detailed answer meant a lot for me. My real fastai journey has begun just few months ago. It was important to know, how to grow with fastai from a veteran like your goodself.
Very true. Usually I know more DL ideas than my ability translate them into code. The limiting factor seems is the ability on how fast and how well I can turn those ideas into code. Top kagglers seems always very good coders. And usually their ideas are freely discussed by the generous kaggle community. I dare to think that the leader board is a reflection on how is one’s ability to code.
I have been coding for nearly 15 years or more. And true that without actively seeking to enhance my coding ability, I haven’t seen much improvement.
By the way the first thing that I did when I started learning fastai is to revise the top previous posts in the forum. And boy, I learned a LOT from you and from the others. In particular your posts seemed important that I had to read all your posts from your forum profile
Also wanted to thank you for your generous code sharing of the kaggle competition base line solutions. I learned so much from your solutions. Words do not help me to express my appreciation that I feel in my heart.
Thank you very much Haider for your extremely kind words It might be that I am a veteran of the fastai way - have been following advice from the lectures for something between a year and a year and a half now - but overall I feel I am very new to a lot of things. Many of the things I share are quite literally something I read about very recently. Same goes for OOP - there is a lot of very basic things that I am only learning now as I write about them.
Very happy to be on this learning journey with you and the rest of this amazing community!
2 Likes
This is something I noticed in your old posts too, when you was extremely active in this forum. Truthfulness and humbleness in your writing about your learning journey.
With any coding project, always I need tips here and there to solve an issue, and go back and forth to stackoverflow to craft a few lines of code. When I finish it, looking at that part of code, I would say, no way I could do it. It is the very nature of any design that it is a pile of ideas carefully crafted into a single embodiment. When it is done, it is truly scary for the new designer. I appreciate your humble and truthful writings repeating this over and over, to encourage new fellows in the forum. This is something that I found too with the amazing teacher that we have.
One more thing I wanted to share my feelings about, and wanted to know your opinion. 2 years ago I spent few weeks learning tensorflow and worked on a simple project. And didn’t like it at all. However when I switched to keras and did another project, I felt in love. Life didn’t let me to continue with DL, and now I am back with fastai. I love fastai and @jeremy is truly an inspiring human being. This framework will have a big future. Partly because it is still very dynamic and continuously evolving in what are the best research outcome in DL, which is something amazing that I didn’t see in any other framework. And partly because the guy who built it is Jeremy.
However this rapidly evolving API which is a necessary criteria for any promising framework, sometimes it is its own difficulty to learn and to use. And top of that the very small documentation and little community that is behind it. I will stick with fastai for sure, because I believe in Jeremy and love his exceptional work, but good things need time.
So eventually, there are few things made me considering learning keras beside fastai. Most importantly because the open source projects built with keras is so large in number that it is not easy to ignore, and the community support is so huge. It is not just the quality of the framework what makes it important. These factors (documentation size, code examples and number of github projects and SO posts count) are playing an important factor in deciding for what language or framework to go with. Sometimes people even calculate how many posts in SO are written to decide to go with vue.js vs react framework for instance.
Take for example Martin’s 1st winning solution using Siamese network for the previous whale competition. It is truly a master piece written in keras that I cannot ignore. Sometimes we are lucky (and even the owners of the competition are lucky too, I think) that a guy like Martin, who is software engineer lead in Cisco, participated in the competition and released his winning code. I feel I have to be more flexible and use keras too. Just for the sake of a bigger exposure of choices in term of code examples for learning DL/ML.
I am currently reading the book of Francois on Keras and it is very well written. Downloaded Martin 's notebook and playing with his code. Considered porting his solution into fastai and pytorch, but that is beyond my time bandwidth. It is really big and a lot of technical ideas are coded there, and not so easy to repeat all his work in pytorch.
Fastai will be my goto framework always, but if I hit a road block and there are better solutions in keras compared to the new fastai and pytorch, may be I can learn a few more DL stuff from keras then.
So what do you think about learning both fastai and keras? What are you doing yourself? And what is your opinion about the other frameworks out there?
I don’t think I can speak about what in general would make sense to do (I don’t think myself competent to speak to that) but I can tell you about what I am doing. For me, its pytorch all the way. I tried to take a look at tensorflow, but didn’t find the experience too appealing. From what I remember about keras - have not used it for a very long time - I hated the level of indirection, that my data / parameters lived somewhere but I couldn’t access it directly from python.
I spent quite a bit of time using pytorch and I feel at home in this ecosystem. I can relate to your concerns regarding the fastai library but I think that despite it quickly evolving and maybe there not being there many examples of its usage out there yet, it fits well with what I want to do and how. I have so many plans and ideas involving this specific set of tools.
But this is all very personal. I don’t claim any superpowers and I can relate to many of the struggles you mention. Above all, my choice feels right for me and I have now gathered enough experience to feel very strongly about what I want or don’t want to do.
Whatever choice you feel is right, you should embrace it and head down that path and you should not feel sorry about that. The worse thing one can do is probably trying to do something some other person could tell them to do while they feel like as if they are banging their head against the wall taking this approach and not making progress.
It’s all very individual and situation dependent and I bet you yourself know the answer best!
That is on the framework bit - for learning, I don’t know any other community that would even come close to how supportive this community is, but I don’t think using any specific set of tools is how one becomes a member.
1 Like
Very well said dear Radek. I will follow my heart and will not be sorry about my decisions. Even if it turned out wrong someday, then at least I’ve taken the chance to try it out what felt right someday.
Currently, I will give keras a little chance to be my secondary framework beside my favorite fastai and pytorch framework. I think it is just one or two years and the right choice will be very clear in favor pytorch and fastai. Just like the python2 vs 3 debate few years ago.
Thanks for the detailed answers and wishing you all the best in your endeavor!
2 Likes
Might be there is no right or wrong here - just various things being good for us at different times and given our circumstances, is all 
Maybe it would still be the right call for me to stick to Pytorch if I feel this is the fastest way for me to make progress right now even if Pytorch should disappear of the face the Earth in 2 years time 
Or maybe that’s just the heretic in me talking
Anyhow, best of luck to you in your journey and be seeing you around!
1 Like