I am attempting to reproduce parts of a dynamic climate model using machine learning in place of physics code.
I have both inputs and outputs for several climate model runs, and my intention is to use these to train a machine learning model, hopefully to the point where it can predict similar output values when given the same inputs.
The nature of the input/output (feature/label) datasets is that they have a 4-D structure per feature/label, with dimensions (time, elevation, lat, lon).
Initially, I am assuming that only adjacent lat/lon cells should influence the behavior of neighboring cells, but later we may want to incorporate algorithms that also take into account adjacent cells in the elevation dimension, as well as something to account for the temporal nature of the data (i.e. we have a time series for each lat/lon/elevation).
Can anyone comment as to a good first approach using the fastai library? My initial idea was to use one or more convolutional layers to account for the spatial aspect of the dataset then to use an LSTM layer to address the temporal aspect, but I have not yet worked out how this should be done using Keras/TF. Maybe there’s a better way to go about this using the fastai library?
I am a beginner with ML/DL, any ideas, even basics, are welcome. Thanks in advance for any suggestions, I appreciate your help.
There is some recent work in the fluid dynamics literature, https://arxiv.org/abs/1708.00588 (and citing articles thereon) but they use the knowledge of the governing equations to set their loss objective (this also allows them to use a simple deep feed-forward network) which also serves as a strong prior. Since climate models have all sorts of hackery built in other than whatever core equations they are using, this might be out of question.
A simple prediction approach might be pretty challenging. And a simple LSTM would probably fail. You would at least need an an encoder-decoder framework with attention, in the vein of a Language Models. Fast-ai has a language model framework (AWD-LSTM) which could be re-purposed.
Finally on a spherical surface, simple CNNs are formally incorrect since they lack the right symmetries, so you would need to use Spherical CNNs (https://arxiv.org/abs/1801.10130) which fast-ai (or Keras) does not support but they do have their code up on github.
There might be some value in posing the problem formally and breaking down the various complexities systematically, and trying to find the simplest problem you can solve easily through machine learning and going from there.
Hope I am not muddying the waters. Anyway, good luck!
My apologies for the delayed response, I’m just getting back into learning fastai and haven’t been here in several months. Thank you for the very helpful message, Kaushik!