the toc generator is quite a limping piece of software and required quite a lot of massaging to make it work (we didn’t write it) so I won’t be surprised that it just needs some more love.
Hahaha very true. I’ll fix this parenthesis error, and see if we can find a better one further down the line
1 Like
Guys, I found a potential solution for the current difficulties in collaborating on notebooks in GitHub.
Here is a tool called Notedown; basically it converts back and forth between Jupyter Notebook and Markdown. It also allows editing the markdown version as notebook in Jupyter Notebook. Basically what it does is that it puts code cells in markdown cell block, and then keep everything else as plain markdown. The only issue is that the Jupyter Notebook’s Hide Input extension stores the hide-input-cell information in the JSON, so it would be lost when converted to the markdown format.
Below is what I have in mind as a possible workflow:
-
Only source control the markdown. It will allow very straightforward diff and thus easy collaboration and code review in GitHub.
-
When updating the notebook, the users make whatever changes they want, convert the notebook back to markdown, then commit the changes to the repository.
-
The server then will rerun the markdown as notebook using
notedownutilities, add hide-input cell information to the JSON source file for all theshow_doccells, then build docs and the website.
Let me know what you guys think.
The problem with this, is (as many other solutions) that you need to execute the notebook to get back the source notebook. Executing the whole docs sources take almost half an hour on a standard GPU so it’s not really feasible to have it automated.
1 Like
I was thinking one more thing our API documentation is lacking is when any of those functions was added. Here are some situations where it’s important:
- the docs include just added functions that aren’t in the release yet. the users have no way of telling if they have access to it or have to wait till the next release (workaround: check CHANGES file)
- some people might be using an older version for a specific reason e.g. a regression in some parts of fastai for their specific need, or they just don’t want to keep syncing their code with the ever changing API. Now they can’t tell whether their version supports a function they are thinking of using (workaround: again, check CHANGES file, but we might miss some of those)
- if we want a deprecation cycle, we also need to be clear in which version it will be removed.
This is not a huge issue, but I thought I’d share. And I thought the docgen could easily automate this process and add [since 1.0.42] when it first generates the entry for a new function that hasn’t been documented yet. And to catch up it could just start with 1.0.43 for all already documented functions. Just an idea.
1 Like
I tried to fix the links. But unfortunately I just could fix a few. This is probably not enough for a pull request. 
I haven’t found out yet how the method show_doc() creates text, which is included into the docs then. And for some links I couldn’t find the class descriptions in the docs to which they refer. I’m not sure if the docs are not up to date, but I assume I simply just don’t know too much about the fastai library, yet. I just watched the second lesson of the course.
Oh, one more thing, that I noticed. When I tried to run the script for testing the docs locally, I got an error, which said, that a folder could not be find. Then I changed the path inside the script, which made the script run. However, the result was different compared to the results generated by the non-local test script. Thus, I wondered if the script is broken!? But I assume, I made something wrong (although I think I followed the instructions)!?
Thank you for wanting to contribute to the fastai docs, @bam098.
Any small fix counts - so even a single url fixed is a goodness.
docs.fast.ai is pretty up-to-date, but it doesn’t matter since you should be working with the git checkout for the latest version always.
If you’re comfortable with grep that should help find the correct anchors. or of course use your editor’s search across files feature if that’s easier.
Just remember not to fix .html files as they are autogenerated - just ignore those. ipynb are the source.
I’m not sure why you need show_doc, you can just ignore it. When you run the doc notebook show_doc generates the output for each function, but you can’t modify that output anyway, since it’s overwritten. So it’s changing all the other markdown cells that are created manually. And for that purpose you don’t need to run the notebook, just edit and save it. You could even use a text editor to edit it directly instead of jupyter, it’s just more difficult to do.
If you have some specific questions please ask - there is always a chance the guide to doing this can be improved, so please let us know if something wasn’t clear, hard to make sense of/follow, etc.
Oh, one more thing, that I noticed. When I tried to run the script for testing the docs locally, I got an error, which said, that a folder could not be find. Then I changed the path inside the script, which made the script run. However, the result was different compared to the results generated by the non-local test script. Thus, I wondered if the script is broken!? But I assume, I made something wrong (although I think I followed the instructions)!?
If you don’t share the exact commands you attempted to run and the exact errors you received it’s not possible to address this difficulty. Once you do then we can sort it out.
Always remember that when you communicate this-doesn’t-work kind of issue, always put yourself in the shoes of the other person, who doesn’t know neither what this is, nor how it failed.
1 Like
Hi,
I wanted to ask before making my way to submitting a PR if this is a good idea:
I had presented the fastai image aug using an example and ‘where this might be useful’-application basis.
Link to Kaggle Kernel, I think these would be helpful when going through the docs and understanding for example, how a dihedral transform might be useful.
Looking forward to the feedback.
Regards,
Sanyam
3 Likes
I think we can definitely add a link to this in the docs for vision.transform.
Thanks!
I’ll write a blogpost as well that is more detailed and descriptive and then submit a PR.
Quick question: I couldn’t understand the cases where jitter and random crop might be useful?
I’d like to give back to the fastai community to express my gratitude for all the amazing work you guys put in. I decided to start with adding to the docs, which appears to be a task within my reach. Core and torch_core both seem to be missing examples for some of the functions, perhaps I’ll start there since I’m already somewhat familiar with those tools.
3 Likes
Good stuff, your post was really inspiring!
2 Likes
A PR for a few functions in fastai core (docs) has been submitted. I started with some of the “simplest” functions to check whether or not you guys think the style is appropriate. Any feedback is appreciated!
It looks great!
@ashaw, would it be possible to adjust docgen to generate consistent font size for the test/source items on the right?
Currently they are included in h1, h2, etc. And you can see from the snapshot result in a weird large/small outcome:
Suggestion: those links shouldn’t be part of the header and use a consistent font. I tweaked it in a saved html and it appears to be fine. it should be the same for all headers.
Thank you.
1 Like
This is just a notebook problem right?
Definitely fixable, but kind of an ugly hack - adding inline css for only the notebook case.
I’ll play around to see if there’s a cleaner way
No, this is a snapshot from docs.fast.ai. Perhaps re-reading my description again would help - each h1, h2, … has a different font size, so if [test][source] are included in <hX>...</hX> they will be rendered in different sizes as it can be seen from the screenshot.
I proposed to move them out from <hX>...</hX> to keep their size consistent through the document. But perhaps there are other solutions.
Here it is again with circles around it so it’s easy to see the size discrepancy:
Ah, thank you for that validation, @ashaw - it looks like a problem in the old firefox I use - I re-checked in chrome and it does appear as your snapshot.
So it’s a non-issue. Thanks again.
1 Like