How to save and restore my training?

A whole set of training lasts from hours to days.
If I shutdown my AWS p2 instance, the next time I need to re-train from zero again. This is very expansive in both time and money.
Is there any way I could save and then restore from my last progress ?

You can use model.save_weights() when you’re finished training or you can use the keras training callback ModelCheckpoint and use the save_best_only option to automatically save the best set of weights:

https://keras.io/models/about-keras-models/
https://keras.io/callbacks/

Then when you want to start training again use the model.load_weights() to restart where you left off.

Use ModelCheckpoint in keras while training it will save your best model automatically. And while loading use model.load_weights(). Use hdf5 format and pickle format officially is not recommended.