I’m not sure if this will be a topic in Part 2 but if not, I’m hoping it’ll give me the foundations to tackle it.
Basically instead of training your model on all of the data at once, you train it on one example or a small batch at a time, hopefully without the model forgetting its past training.
Here’s an example : suppose you run a mail app and the user clicks on the “report spam” button on a message. That message instantly gets trained on in the model as a spam message. Similarly, when the user browses through his spam folder and clicks on the “not spam” button on a message, that message is instantly sent to the model to be trained.
love this, great idea, would love to learn about this, the network incrementally learning as new interactions happen with the world, I’m all for this
btw, minute 42 aprox of lesson 3 of Part 1, Jeremy talks a bit about fine-tuning a model, somehow connected to this topic, would definitely be great to expand on this
If you’re trying to look up papers on this topic, I believe the term you’ll want to search for is “online learning”.
Yeah, I prefer to call it “incremental learning” since “online learning” might also imply learning from moocs or something. But when I search for papers or articles, i might search something like “online learning” AND incremental.
For time series data, the last data point is arguably the most important so adjusting how much online/incremental learning you do should be related to the confusion of the prediction for the last data point.
I think they were talking about this concept with the OpenAI TWiML talk last week as it relates to NLP; substituting “last word” for “last data point”.
Just a couple days ago, our ML group saw the term “online” learning, mostly for the first time. We had a debate about what it meant and what would be more appropriate. We thought “real-time” was an improvement and “streaming” was better. I think “incremental” is even better. Someone should take “online” out back and de-letter it.
Also, if I understand correctly, does reinforcement learning use incremental/online learning to train on new data coming in?
You can also use instance-based learning for this approach where you can add new instances and throw away old instances. The collection of instances are the actual model you compare against using some measure (usually angle or distance between nearest N instances).
More here https://en.wikipedia.org/wiki/Instance-based_learning