I have a Nvidia GTX 1060 wit 6 GB. I use my own dataset, so it is hard to compare but an epoch takes around 2…3 minutes.
The Windows task manager does not show GPU performance, so ask Google for something like a Nvidia performance tool or the like. You can then see if your GPU is under load.