Hello together,
I found an interesting github repo, in which Ruben implemented many deep learning models.
If you may stuck on your project and/or looking for a way to solve your problem - you may find something similar in one of the files
Bests,
Benedikt