Fastai v2 roadmap

#1

fastai v2 now under development

During the in-person part 2 of course v3 (that will be released as a MOOC in June), we focused on building the library from scratch (starting with matrix multiplication, and up to training a state-of-the-art ImageNet model) which allowed us to have a second take at how we did things when we built v1. We built a better and more flexible way to have a training loop with callbacks, and an easier to use data block API. That lead to the development of v1.1 to integrate those changes.

We soon realized however that, even if we don’t want to change the high-level API from v1, it meant rewriting the core modules, and in turn all the other modules need to be adapted, so we’re on an almost complete rewrite. Calling it a v1.1 would be a bit of a stretch so we’re building v2, hopefully to be released in a few weeks. Below is the roadmap for what we’re doing, and the breaking changes you can except (mostly in the data block API or the way callbacks are written).

The most interesting part isn’t the features we’re adding but the way we settled in developing it. After the initial process of prototyping v1 entirely in notebooks, then the second part of the course that did much the same, we have greatly improved our scripts. The idea is that v2 will really be entirely developed in notebooks. The library will be created by an exporting script, and all the PRs, whether for bug fixes or new features will have to be done in the proper notebook.

Why? Well as you know notebooks are interactive. And we’re not just leaving the code cells: each function is followed by documentation (if more is needed after the 1-line docstring) and some unit tests. So if you’re fixing something, you can add a unit test that failed without your fix super easily and check the intended behavior isn’t broken by running the cell just below the code.

The script that builds the documentation hides the source code of the functions in the module, and shows the definition of the function with the docstring in its place, to give docs like you’re used to in v1. The unit tests that are of interest to users will be left, as they’re useful to understand what the function is supposed to do (adding a test link in v1 to all the docs was a feature a lot of people were excited about). We’re working hard to write tests that are as clear and informative as possible.

You can find examples of this process in the dev repo.

Important

There will be one last release of v1.0, which will then live in a separate branch for bug fixes. v1.0 will continue to be maintained, and there will be a separate v1.0 docs site. So you don’t have to migrate if you don’t want to.

Once we’re ready, the notebooks that build v2 will be exported on master, so master will be a bit unstable while we test everything is working properly and convert the course notebook to v2 (the v1 version will still be available). Once it seems to be working OK we’ll do a release as v2.

Roadmap to v2

Data block API supercharged

The data block API will be completely reworked to make it easier for users to plug in at every level they want. The high level API will look something like this (on the pets dataset):

class PetsData(DataBlock):
    types = Image,Category
    get_items  = get_image_files
    split_func = RandomSplitter()
    label_func = RegexLabeller(pat = r'/([^/]+)_\d+.jpg$')

But every step of the way will be more easily accessible. If you like writing your own Dataset in PyTorch for instance, you can just explain how to get your inputs and targets from the filenames in one transform:

class PetTfm(Transform):
    def __init__(self, items, vocab=None):
        super().__init__()
        self.items = items
        self.labeller = RegexLabeller(pat = r'/([^/]+)_\d+.jpg$')
        if vocab is None:
            vals = map(self.labeller, items[train_idx])
            self.vocab = uniqueify(vals, sort=True)
        else: self.vocab=vocab
        self.o2i = {w:i for i,w in enumerate(self.vocab)}
        
    def encodes(self, i):
        o = self.items[i]
        return resized_image(o), self.o2i[self.labeller(o)]

where resized_image is a function that opens and resizes the image (for batching). If you want to use all the visualization tools the library offers, you’ll just need to add to that transform a decodes (takes the output of encodes an returns what you want to display) and a show method (explain how to display the output of decodes).

    def decodes(self, o): return o[0],self.vocab[o[1]]
    def shows(self, o, ax=None): show_image(o[0], ax, title=o[1])

New callback system

One of the highlights of v2 will be the new callback system developed in the part 2 of the course. This will be a breaking change for anyone having custom callbacks (they’ll need to rewrite them with the new API) but shouldn’t impact the high-level API much. There’ll be more events and more flexibility, and it will be easier to use and understand. Stay tuned for tutorials showing how this will work if you haven’t followed the part 2 of the course.

Data augmentation

We won’t keep the current pipeline, instead allowing you to mix between:

  • transforms of single image byte tensors on the CPU with PIL
  • transforms of single image float tensors on the CPU with pytorch (i.e. similar to v1.0)
  • transforms of batches of images on the GPU once the resize is done.

We’ll be encouraging most data augmentation to be done on the GPU, resulting in quite a boost of performance.

Customizable Optimizer

On top of the PyTorch optimizers, we’ve developed an optimizer that is entirely customizable and built with blocks that you can mix and match. Implementing the new LAMB optimizer with it took just 8 lines of code.

def lamb_step(p, lr, mom, mom_damp, step, sqr_mom, sqr_damp, grad_avg, sqr_avg, eps, wd, **kwargs):
    debias1 = debias(mom,     mom_damp, step)
    debias2 = debias(sqr_mom, sqr_damp, step)
    r1 = p.data.pow(2).mean().sqrt()
    step = (grad_avg/debias1) / ((sqr_avg/debias2).sqrt()+eps) + wd*p.data
    r2 = step.pow(2).mean().sqrt()
    p.data.add_(-lr * min(r1/r2,10), step)
    return p
lamb_step._defaults = dict(eps=1e-6, wd=0.)

lamb = partial(StatefulOptimizer, steppers=lamb_step, 
               stats=[AverageGrad(dampening=True), AverageSqrGrad(), StepCount()])

NLP preprocessing

Tokenizing won’t be done with a preprocessor anymore, but with helper functions that will return something similar to your inputs, but tokenized (so files in a folder architecture like your input files, or a new dataframe, or a new csv file). There’ll be a slight gain of speed but mostly, the memory usage will be greatly reduced for files or csv, as we won’t load the entire dataset in memory anymore.

There will also be a SentencePieceTokenizer if you have installed SentencePiece (it won’t be a dependency of the library).

65 Likes

Fastai-v2 - read this before posting please! 😊
LAMB Optimizer
Starting new project - recommended way to include the latest FastAI from the course?
Why make a v2?
Fastai v2 code walk-thru 1
Part2 before Part1 again?
(Phuc Ng. Su) #2

Hi @sgugger,
firstly, really thanks for your help to the community.
I am not really clear when you say v2 will really be entirely developed in notebooks. It means you code all the modules by using jupyter notebook instead of an IDE like Pycharm?
I feel quite struggled while working with notebook for debugging, refactor, etc. Those software tasks are much easier to handle by an IDE.

5 Likes

(深度碎片) #3

I love Jeremy’s vim demo on fastai v1 and just get very comfortable at using vim+pdbpp to study every detail of v1. However, I totally agree notebook is much faster at iterate and refactor ideas. Any idea of nice way of exploring source in notebook like vim+pdbpp?

@jeremy @stas

0 Likes

(nirant) #4

Hey @sgugger- can you please link to the export and tests tooling we’d be using for this?

It would be very useful for people looking to contribute.

0 Likes

#5

Do you teach in the part 2 of the lecture about API which changes in v2? Or do I need to see the doc and learn it by myself?

0 Likes

#6

The tools will be packaged in a separate repo once we’re finished (we make them better as we progress on the development.

No the new version isn’t taught in part 2, only its development. There will be tutorials to learn the differences with v1.

3 Likes

(Etienne Tremblay) #7

I like the idea but something I am really worried about is being able to set breakpoints indie the library code while learning… While learning the code of fastai, I would often work on a personal project inside visual studio code using the fastai cloned from master as a pip library. Then while learning I would inevitably get an exception inside fastai library because I didn’t pass the right thing (wrong tensor shape or any other newbie error)… So I set breakpoints directly inside the fastai code to inspect variables, check tensor shapes etc. I feel like I learned a lot how everything worked this way.

Would this kind of interactive debugging still be possible from visual studio code? Even if I love Jupyter notebooks, debugging inside of it compared to modern IDE is really annoying…

Thank for the great work and can’t wait for part 2!!

3 Likes

(深度碎片) #8

I had the same worry before, but after trying a little, you will realize there is no need to worry about it, since all the notebooks will generate py files, and these py files are the real source codes for us to debug using vim, vscode, or pycharm. So in terms of debugging, it works just as before.

1 Like

(Etienne Tremblay) #9

Oh ok thanks makes sense!

0 Likes

(Harry Coultas Blum) #10

It can be same if there is a script running somewhere that compiles notebooks on changes. I’ve been working with notebooks as source code and it is tedious having to keep running compile scripts.

There are several ways to do this but I think one that supports multiple operating systems should be preferred. Maybe something like nodemon?

Install Node.js:

wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
nvm insall stable
nvm use stable

Convert files on .ipynb changes
npm i -g nodemon
nodemon -e ipynb --exec python notebook2script.py

0 Likes

(深度碎片) #11

Thanks! I have tried but when running nvm install stable report no nvm command.

0 Likes

(Harry Coultas Blum) #12

You have to restart you terminal like it says in the output :slight_smile:

0 Likes

(深度碎片) #13

Thanks! but still

0 Likes

(Harry Coultas Blum) #14

Try running this in your terminal (Again it says this in the output)

# Node Version Manager
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

Otherwise just get node.js installed somehow on your machine and continue from npm i -g nodemon

1 Like

(深度碎片) #15

thank you so much! you are right I didn’t follow the instruction exactly.
This is very convenient!

1 Like

(HOANG Bao Tin) #16

So the current s4tf lessons at fastai_docs/dev_swift follows v1 and will soon be changed with v2 as well?

0 Likes

#17

No, v2 is for PyTorch only. We’ll go back to Swift after the v2 release.

4 Likes

(Alexander) #18

Any chance the data blocks will improve support for composite, multi-input models? For example, tabular+nlp+image backbones? I managed to get those working in current version, but model export for those unfortunately not supported and efficient loading is tricky too.

4 Likes

(Jamali) #19

Can you tell us the release date of course?

0 Likes

(Etienne Tremblay) #20

I dont use emoticons often but for this: :heartpulse:

0 Likes