Hi. Welcome to the forum and thanks for posting your question. I found your post because I also have a casual interest in stock market prediction.
First, (in case you have not realized it by now) fastai is built on PyTorch. Though Keras was used in much earlier versions, most forum users are focused on using fastai, PyTorch, and most recently Swift/Tensorflow. Therefore you may have better luck posting in a community that actively uses Keras and Tensorflow.
A couple of suggestions: there are several fastai lessons that address RNNs. Time series are not specifically addressed (the focus is on language). However, studying them would fill in the big picture about RNNs and clarify some of your confusions around designing an appropriate model.
Fortunately, there is also a thread specifically devoted to time series. There you are more likely to find help with specific questions.
HTH, and good luck with your project!