Developer chat

[sgugger]

Had a few commits today and vision docs is almost finished, only vision.learner is missing for now.

[jeremy (Jeremy Howard)]

@sgugger that reminds me - not a big deal but you may want to rename vision_learner and text_learner in the module dependencies diagram too.

[stas (Stas Bekman)]

Indeed, it’s this one that causes problems:
c:\users\sylvain\anaconda3\envs\fastai1\lib\subprocess.py(997)_execute_child()
Which doesn’t help us since it’s not the file not found, it’s where the file wasn’t found in code

I suppose you are replying to Jeremy’s suggestion, could you please run the modified code I pasted which should tell us what is it trying to find?

I don’t know what to make of those strings you pasted - lacking context.

Thank you.

[stas (Stas Bekman)]

I think your idea is better, however I don’t think checking for signatures and and signing if unsigned will take longer since usually there will be more unmodified notebooks than modified notebooks

Please go ahead and proceed with your suggestion, @lesscomfortable, we can always measure/tweak it later if we discover that it’s becoming a hurdle. Thank you for looking into it.

[stas (Stas Bekman)]

And btw, we have a fledgling test suite already. Please see:

http://docs.fast.ai/developers.html#test-suite

And you may start adding tests to it as you create new features.

Or if you have some time on your hands, please, write tests for the existing ones.

Note that some of the v0 tests can be ported to v1. https://github.com/fastai/fastai/tree/master/tests
I ported a few subtests from tests/test_core.py of v0, but there a lot more there to port.

Thank you.

[stas (Stas Bekman)]

Some of you might appreciate this new addition:

Github commit emails with full diff of changes

I’m still doing some tweaks, but it’s mostly fantastic! I’m so happy to be able to get diffs by email! Thank you, @jeremy, for supporting this idea. You can subscribe or unsubscribe at any time on your own accord here: https://groups.google.com/forum/#!forum/fastai-diff Make sure you use a descent email client, surely don’t read it on google groups or gmail, you need to have a client that can properly render fixed fonts and not use a variable fonts which look horrible with diffs. But google d…

[sgugger]

@sgugger that reminds me - not a big deal but you may want to rename vision_learner and text_learner in the module dependencies diagram too.

It’s done.

I suppose you are replying to Jeremy’s suggestion, could you please run the modified code I pasted which should tell us what is it trying to find?

Here the output of the modified script

Path tools
Executing: tools\trust-origin-git-config
Traceback (most recent call last):
  File "tools\run-after-git-clone", line 31, in <module>
    run_script(path/"trust-origin-git-config")
  File "tools\run-after-git-clone", line 18, in run_script
    result = subprocess.run(cmd.split(), shell=False, check=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "C:\Users\Sylvain\Anaconda3\envs\fastai1\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\Sylvain\Anaconda3\envs\fastai1\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Users\Sylvain\Anaconda3\envs\fastai1\lib\subprocess.py", line 997, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

[stas (Stas Bekman)]

Thank you, @sgugger. It is all correct.

I think we probably need to add `python’ to the command, no? Try:

- cmd = f"{script}"
+ cmd = f"python {script}"

[sgugger]

Yup, that works fine.

[stas (Stas Bekman)]

yay, thank you for testing, @sgugger . This is now fixed in git.

[nok]

Looking great! Will there be some description on how to generate a doc website as well?

[jeremy (Jeremy Howard)]

The doc website is built with jekyll from docs/, so just follow the usual jekyll docs for that bit. To create the docs/ folder from the notebooks in docs_src/ follow the instructions here:

http://docs.fast.ai/gen_doc.gen_notebooks.html

Theres a tools/docs_src/sgen_notebooks.py script that let’s you do the important stuff from the command line.

[lesscomfortable (Francisco Ingham)]

Hey @stas, just pushed my suggestion. However, the original script is not working for me (I can see some notebooks in docs_src as not trusted). Do you know why that might be the case?

[stas (Stas Bekman)]

@jeremy, can you please put what you shared into some place in the documentation?

Plus, this should include instructions on how to add/rename documents - i.e. how to link them, etc.

In particular, the current incarnation of developers.md should be renamed to something like release.md, and we need placeholders for other dev docs. So perhaps we need something like:

• docs/dev/build.md (move the last part from CONTRIBUTING.md)
• docs/dev/test.md (how to write/run tests)
• docs/dev/docs.md (this discussion)
• docs/dev/release.md (currently developers.md)

Other naming is totally fine with me, this is just something to start with. I’m thinking more along the lines of mirroring the more or less standard make targets, which we may have down the road:

• make
• make test
• make docs
• make release

Thanks.

[stas (Stas Bekman)]

I will have a look shortly.

It works just fine on my side. And your change sped it up by x3 on my slow setup! excellent! we might add an additional speed up with timestamps down the road.

Are you sure it’s installed as a hook? re-run:

tools/trust-doc-nbs-install-hook

or run directly to test:

tools/trust-doc-nbs

update: I have just added the last_modified < last_checked file timestamp check, so it’s much faster now when only a few nbs have changed. please make sure to re-run tools/trust-doc-nbs-install-hook after git pull once.

[jeremy (Jeremy Howard)]

@jeremy, can you please put what you shared into some place in the documentation?

Plus, this should include instructions on how to add/rename documents - i.e. how to link them, etc.

Working on it:

http://docs.fast.ai/gen_doc.html

[stas (Stas Bekman)]

http://docs.fast.ai/gen_doc.html

Hmm, but now we no longer have .md to work with. there is no way to preserve the easy to edit format?

we have developers.md at the moment, we can’t have the same for doc gen and everything else?

Also most of the instructions that got moved from CONTRIBUTING aren’t belonging to doc-gen, should be in a different file.

[stas (Stas Bekman)]

heads up to a few renames under tools/:

git mv tools/trust-doc-nbs tools/trust-doc-nbs-install-hook
git mv tools/trust-doc-nbs-hook tools/trust-doc-nbs

but it should be transparent to you if you use tools/run-after-git-clone.

and another rename:

git mv tools/build tools/sync-nb-exports

so now it’s easier to know what we are trying to do.

[jeremy (Jeremy Howard)]

we have developers.md at the moment, we can’t have the same for doc gen and everything else?

We’re using our documentation framework to document our documentation framework.

[sgugger]

In case you missed it, fastai_pytorch is now merged on the fastai main repo. PRs should now be done there and only then. The old fastai library is in the old folder, the docs sources have moved in this repo.