OK, I did some reorg/cleanup and tried to give more consistent/intuitive names, so the last incarnation is:
TestAPIRegistry and the flag is matching
--testapireg. So we are creating a Test API registry. Hope that is an intuitive change. Too bad
doctest is already used for something very different in the python world.
Note, I moved almost everything out of
doctest.py, and it’s starting to emerge into a normal class, so we might switch to that eventually, instead of using class methods. But it’s fine for now.
I implemented the check for valid fastai API. Your code already had everything that was needed, just needed to add a few exceptions and word them so that it’s quickly easy to see what needs to be done.
I added a test that tests test_this. Please check that all cases were covered.
For your convenience, in
conftest.py I left:
#individualtests = 0 so you can uncomment it to do quick tests, without needing to finish
I will go over your notes earlier and make a new summary, including new issues:
- Some tests will sometimes add the same entries more than once, see this example. So perhaps we should check for already existing entries in our registry? Surely, the test could be modified to hardcode
"fastai.basic_train.load_learner" but why put more onus on the developers when we can easily remove dups automatically.
I reviewed your previous comments and unless I missed something I think the only unadressed item is:
- I use inspect.currentframe while I seem to remember we want to just say currentframe and import inspect accordingly
but I’m not sure what you mean there.