I really needed some help with this.
I want to research new DL architectures (c.f ResNet, DenseNet, etc).
What is the best way to do that in terms of, what software to use?
Which libraries allow rapid experimentation for quickly rejecting ideas that dont work etc.
My first order thinking is that there must be better alternernatives that using bare metal Pytorch and TF. Would really appreciate any pointers to know what people are using these days both in industries and academia for rapid experiments on DL architectures.
( i am not so much interested in NAS. My question is a lil bit different from that)
My personal advice, always use Pytorch or a high level library on top of Pytorch.
Regarding bare Pytorch vs high level library, it depends on which problems are you working. If you are working with an standard deep learning problem:
NLP -> HuggingFace, AllenNLP or ¿fastai?. Note that I’m not an expert in NLP related problems.
Image classification or segmentation -> fastai or timm pytorch-image-models. Using fastai to do research isn’t typical but it’s perfect to some problems.
Object detection / instantiation -> Detectron2, MMDetection and maybe icevision.
Tabular data -> fastai or pytorch lightning.
Finally, pytorch lightning is designed to do reproducible research in mind while taking care of engineering aspects. So, you can use it for any problem and it almost always a better choice than using bare Pytorch.