Documentation improvements

Thanks @stas
I have pushed a PR to explain how freeze_to work under the scenes here, and made a few edits from the kaggle kernel version trying to make words clearer. could you have a look? Thanks!

Hi @stas
Since I have done two PR by now and a little more confidence in the procedures, I have created a visual guide for myself and other beginners.

Could you have a look? Thanks!

1 Like

Since I have done two PR by now and a little more confidence in the procedures, I have created a visual guide for myself and other beginners.

Looks great, @Daniel! I linked to it from https://docs.fast.ai/gen_doc_main.html#step-4-submit-a-pr-with-your-changes - just please don’t change the url or if you do, send a PR that fixes it. Thanks.

The only recommendation I’d add is to add the commands in your console snapshots so that users could copy-n-paste them.

Also I expanded on the branch update section: https://docs.fast.ai/dev/git.html#how-to-keep-your-feature-branch-up-to-date

1 Like

I see Sylvain beat me to merging your commit, @Daniel! Your final edits were good. Thank you.

BTW, you don’t need to notify us in the forums when you make a PR, github sends everybody who is interested in watching PRs and Issues a notification (email or browser), so it will be seen and will be attended when the maintainers get a chance to do so.

2 Likes

Thanks @stas

The url won’t be changed, and I have updated codes for easy copy-n-paste to the guide, together with the relevant links you recommended.

1 Like

Hi @stas

When we do freeze , unfreeze , we do it to ‘layer groups’. We know ‘layer groups’ are groups of layers of a model. Different models may have different number of layer groups, some has 2, some has more.

The natural question to follow is, why do we use ‘layer groups’ instead of individual layers? Of course, it is much simpler to deal with a few layer groups instead of dozens or hundreds of layers. But how do the model designer choose which layers to group together and how many groups to have? What purpose does it serve besides convenience?

One small thing I want to check is whether learn.layer_groups comes with the Resnet model itself or it is a feature of fastai.

I digged a little into vision.models.resnet34 and found out the model has 4 types of ‘layers’ (rather look like learn.layer_groups ), but when looking into learn.model for its layer groups they are not quite the same. Also, learn has 3 layer groups, but Resnet34 has 4 so-called 'layer’s. Is there a relationship between Resnet34’s layer1 , layer2 , layer3 , layer4 with learn.layer_groups ? If so, what is it?

The paragraphs and links above are from my kaggle kernel, you can see details there.

Thanks!

Mostly convenience, and we empirically found it worked. As for where we put the split between the first and the second group, it’s empirical trials too.

No, there is no link. A Learner in vision usually has three layer groups that are lower part of the body, upper part of the body and head. The 4 layers you see in the resnet correspond to the the 4 different parts, each ending with the image size being divided by 2.

2 Likes

Thank you so much for your explanation! it is very helpful!
When you say ‘empirically found it worked’, do you indicate that although in lesson notebooks we only see two ways of freezing the model: normal mode (freeze up to the head, the last layer group), unfreeze mode (freeze no layers), there is actually a third way of freezing the model for training in your actual practice worth trying, that is middle mode (freeze up to the middle group, if there are three layer groups)?

The two different groups are intended for discriminative LRs (give a lower one to the first group), but you could certainly try unfreezing one group after the next.

1 Like

@sgugger Thanks a lot! This is nice to know the groups’ purposes.

@ashaw, here is another small doc gen improvement request.

We get quite a few PRs with users modifying the autogenerated html, since they don’t realize they are autogenerated.

In the fastai_docs when we autogenerate .py code we inject this header at the top of the file:

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: dev_nb/01_matmul.ipynb

So I was thinking perhaps it’d work to inject something similar in our html files? e.g. for docs/basic_train.html:

<!--


#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: docs_src/basic_train.ipynb
# instructions: https://docs.fast.ai/gen_doc_main.html


-->

I added the ample vertical whitespace so that hopefully it’ll stand out from the dense HTML once the user opens it in their editor. I’m not sure whether it can appear at the very top, or after the jekyll headers.

Thank you.

2 Likes

Is from fastai import * really necessary?

Hi everyone,

on import of docs.fast.ai, it says:

In order to do so, the module dependencies are carefully managed (see next section), with each exporting a carefully chosen set of symbols when using import * . In general, for interactive computing, you’ll want to import from both fastai , and from one of the applications , such as:

from fastai.vision import *

it seems to suggest we should do import for interactive computing in the following way

from fastai import *
from fastai.vision import *

However, if you experiment as I did it here on kaggle, you will notice that from fastai import * add nothing to from fastai.vision import *.

Therefore, I attempt to say that from fastai import * is unnecessary.

Am I missing something here? if so, please correct me. Thanks

Oh this is legacy behavior. It used to be necessary to do two imports, but nowadays it’s either

from fastai.basics import *

(just the core + training loop)
or

from fastai.{application} import *

If you want to adjust the docs, feel free to suggest a PR!

3 Likes

Thanks @sgugger

Here is my proposed change to the doc. Please have a look.


In order to do so, the module dependencies are carefully managed (see next section), with each exporting a carefully chosen set of symbols when using import * . In general, for interactive computing, to play around the core module and training loop you can do

from fastai.basics import *

If you want experiment with one of the applications such as vision, then you can do

from fastai.vision import *

index page: data link points to wrong page?

On this page https://docs.fast.ai/index.html#Dependencies, all the data links point to vision.data, but according to the context, they should point to links of basic_data. Do I understand the context correctly? Could anyone double check them for me? thanks!

Then, there are three modules directly on top of torch_core :

This takes care of the basics, then we regroup a model with some data in a Learner object to take care of training. More specifically:

  • callback (depends on data ) defines the basis of callbacks and the CallbackHandler . Those are functions that will be called every step of the way of the training loop and can allow us to customize what is happening there;

From data we can split on one of the four main applications , which each has their own module: vision , text collab , or tabular . Each of those submodules is built in the same way with:

@sgugger

Which modules do learn below refer to

https://docs.fast.ai/index.html#Dependencies in the last two blocks of text, we can see

  • learn (depends on callbacks ) defines helper functions to invoke the callbacks more easily.
  • optionally, a submodule named learn that will contain Learner specific to the application.

There are no modules named learn any more. My guess is the following. Could you verify them for me? @sgugger Thanks!

Yes this wasn’t properly updated when we changed data to basic_data.

1 Like

The first learn is now train, and in the second, the submodule is {application}.learner, that’s correct.
Thanks so much for proofreading and making this consistent with the current stage of the library!!!

1 Like

I am trying to work with the tabular module but found the documentation a bit incomplete (for beginners)… Now I am going to try and look for improvements! I think it is a great way to learn and help other beginners. (Although I am a bit afraid of making mistakes)

@Eva
My experience tells me the following

Try hard and ask for help and keep up, and you will see how friendly and supportive this place is, and your worry will be gone.

1 Like