Clicking the link would list links to notebook cells (lines) from Deep Learning Notebooks fast.ai class demoing the API
Value would be to giving users example code snippets, project is a continuation of the Test Registry Project
User Interface: similar to Test Registry Project: [ANDREW TO ADD]
Structure of the json db (similar to test_api_db); [ANDREW TO CONSULT for link generation: what info do we need, name of notebook, a line number of can we reference a cell block so the link can point there?]
Side effect (potentially): In case we find many API usages that are not tested, this project could support the test coverage because one could suggest to just take the notebook example and make a little test of it
IMPORTANT: Non functional Requirement: No Maintenance
Creating a Registry of Notebooks & Cells which demo a certain API manually is not feasible. It is a huge ‘dumb work’ one time effort, which would need to be repeated with every new fast.ai class or update of notebooks and likely would require maintenance. Scarce Resourcing in that case is better used for other tasks
Considerations:
It is common to use a smart grep or discovery functionality like in the Test Registry Project to find usages of API. Such greps might deliver wrong hits (or too few hits). While the results might not be perfect they would still add considerable value to most fast.ai users, supposedly a grep we come up with would still be better & faster than what individuals do searching already now
We should consider a technical design to go through all python notebooks and use the API to search create a registry, see a draft below
Staffing:
Generating such a link list can take considerable time, the feature should be estimated & prioritized against other work of Resources (some of that helped with the Test Registry), that would need to help
DRAFT DESIGN: import Notebook / leverage ipython Notebook API to discover notebook API calls
Start with 2 loops:
Loop over all APIs from the fast.ai doc (Question: where do we find that list?)
Make a simple grep over all ipython notebooks json file in the directory to find potential hits for further inspection. We rather take too many than too few hits, see next step
@stas@ashaw I drafted here the idea to register all API usages from notebooks into the fast.ai doc.
It could be quite useful - but to me, it’s feasible only if we do it dynamically. But then, I think even a simple grep behind such a link can be useful. Or we rather find a way to leverage the ipython API or open a notebook and inspect the grep hits to be sure.
And I have to be careful proposing this because I could see with Test Registry that I can always put in less time than I would like to (and I have a small PR to do 1st for the Test API project and need to focus on part2 & swift :-)), i.o.w. we should also check who can help and how valuable compared to other projects this could be to enhance the doc. Making a little design, putting it on the shelf and maybe asking around a little, who can help, could also be a good 1st step.
Let me know your thoughts, feel free to update & correct or add to above outlines of course.
Crazy (?) thought: would it be helpful adding a 3rd links left [notebooks] [test] [source]
So basically you show a list of links into the all notebooks using this specific API. I would think we find these calls with some string search and then generate a link.
So the doc links back into all notebooks, because there you find many example usages from Jeremy?
is it feasible, easy to do ? Even generating a link pointing into a cell of a nb?
Its actually what I sometimes did manually when not understanding an API: make a string search on my local copy and then go to a notebook. If there was a direct link, would be cool.
So then docs could be tying together: notebooks, test scripts, source code and the core doc examples of course all in one.
In general that’s a good idea, @Benudek. I do use grep on nbs often.
However, there are several problems that I can think off hand:
course nbs are in a different repo, and it’s not tied to the codebase, so it’d be difficult to keep in sync.
doing string search will work at times and many times it’ll find totally unrelated things, so unless it’s manually curated as it is with this_tests it will probably create a lot of irrelevant results.
And if you do that feature, also include ./examples/, and well doc notebooks have lots of code samples too.
In any case, if you want to do that please start a new thread, let’s keep this thread focused on test registry. thank you.
I shared my thoughts on it here. Currently I foresee more problems with this than goodness it can provide, so I’m not participating in this one.
p.s. I moved the relevant posts from the other thread here, but not sure how to place them in correct chronological order, so it’s a bit confusing to a reader as they are out of order. So please feel free to edit/move things around if need be.
I agree with your points, would be nice but too hacky implementing and unstable
@ashaw thoughts? Maybe we just add a grep statement below the pytest lines with a comment, how to search through notebooks on the command line for the specific API.
To get a list of Notebooks using this API run:
git clone git://github.com/fastai/course-v3.git
cd course-v3/nbs
grep -lr “fit_one_cycle” *
If they run pytest they are anyways there and could generate a list of notebooks. I’d consider that a nice shortcut - albeit almost trivial. For non-developers it’s maybe not so trivial.
Mark this thread as closed / deprecated and take off from dev projects
