Test Registry project

[Benudek (benedikt herudek)]

@stas

Please try to be consistent with your code - thank you!

will do, can you point out what matters so next time will do? Couldnt find changes in a quick look but I probably overlook sth

@stas

Also please, please, make sure that file doesn’t change if the test suite doesn’t change. i.e. it should always generate an identical output (sorted, et. al) if the inputs haven’t changed.

You mean sort_key=true in json.dump (think its there) or sth else?

As to do’s (looping @ashaw also and suggestion for action owner):

• Andrew: commit fastai/test_apid_db.json to the repo so Andrews code can use it (?)
• Andrew: add test_api_db.json into MANIFEST.in
• Andrew: make sure function derives class.method automatically

@stas

      function should be able to derive the class.method automatically.

• Benedikt: make PRs for test _this after agreement on details and Andrew committed change

@stas

      recommend testing live methods and not hardcoded classes:

      Instead of:

      ```
      def test_fit_one_cycle(learn):
         this_tests(Learner.fit_one_cycle)
      ```

      this:

      ```
      def test_fit_one_cycle(learn):
          this_tests(learn.fit_one_cycle)
      ```

      so the function should be able to derive the class.method automatically.

• Benedikt: test_this() should be able to receive multiple entries in one call, so it should be *args
• Benedikt: a bigger puzzle to figure out is that the only time the json file is written is when ‘pytest’ completes with 100% success. You already have the check to run it only for non-specific tests, this is the 2nd half of the setup.currently, if I run ‘make test’ and abort it we get {} in the json file - not what we want obviously as we have just lost all the data.