@FourMoBro Interesting to compare the iniitial numbers with your results, given a lesser powered GPU, but appreciating the GPU isn’t everything. You’ve got me wondering about my HD IO. Here are the key run times.
OS: Ubuntu 18.04.1 LTS
RAM: 32GB
CPU: Intel i7-7700K
HD: Samsung SM961 Polaris M.2 NVMe
GPU: Titan XP, 2080 Ti. CUDA 9.2, Driver 410.
Benchmarks:
Training: resnet34
learn.fit_one_cycle(4): Total time: 01:23 (2080 ti)
learn.fit_one_cycle(4): Total time: 01:23 (2080 ti fp16)
learn.fit_one_cycle(4): Total time: 01:25 (Titan XP)
Training: resnet50
learn.fit_one_cycle(5): Total time: 03:21 (2080 ti)
learn.fit_one_cycle(5): Total time: 02:46 (2080 ti fp16)
learn.fit_one_cycle(5): Total time: 03:58 (Titan XP)
resnet50 after Unfreeze:
learn.fit_one_cycle(1, max_lr=slice(1e-6,1e-4)): Total time: 00:51 (2080 ti)
learn.fit_one_cycle(1, max_lr=slice(1e-6,1e-4)): Total time: 00:40 (2080 ti fp16)
learn.fit_one_cycle(1, max_lr=slice(1e-6,1e-4)): Total time: 01:02 (Titan XP)
No dual GPU improvement.