What nvidia Driver are you using ? Your question is too generic !
Its difficult to help you out holding information with you. I don’t see any problem with WGAN at all that could slow down your training process.
May be is a bad implementation of the network , or your data is bad distributed or even your learning rate is not good enough.Also my question were important and you may think not but in fact the Nvidia driver 390 had memory leak slowing down applications overtime.
Without any clue from your side is very difficult to debate anything to you because I don’t know nothing about your model and nothing about your implementation and data…
Man I am cool … my question is … do you really want some help ?