Querying attributes of Learners

fastai fastai dev
1

I’m just digging into the fastai library, and am also reasonably new to deep learning, though fairly seasoned as a ML practitioner.

From the docs it doesn’t look like it, but does a Learner (eg a ConvLearner) have a method for describing its configuration and the history of operations that have been performed on it? For example, it would seem very helpful to be able to query a Learner to find out that a particular model architecture was used, with these metrics, that it has been trained on such and such data using fit_one_cycle for N epochs, its layers are currently frozen or unfrozen. For monitoring, management, and repeatability, maintaining configuration attributes and history would seem useful. As it is, it’s common for practitioners while prototyping to repeatedly overwrite a variable (eg “learn”) with very different Learners, which can be problematic. Or is this not a problem or are there better approaches?

2

There isn’t anything like that at the moment. I’m not sure such an auditing path should be in a Learner itself - maybe some kind of external API would be better? I’m not sure - it’s not something I’ve spent enough time thinking about.

3

Yes, an external API might make better sense. Doesn’t seem like a major pain point for you, so probably not worth much attention right now.

4

Do you mean something like (mockup function names):

learn = ConvLearner(data, resnet34, metrics=[accuracy])
learn.fit(3)
state = inspect(learn)

Where config is a structure/dictionary with learner parameters? For example:

assert state.metrics_names == ['accuracy']
assert state.called_methods == [('fit', 3)]
5

Thanks, Ilia, yes, something like that would be helpful during interactive prototyping and tuning.