Use multiple GPUs in FastAI Course v3 - 2019?

ml22 · August 30, 2020, 3:03am

Hi all,

I’ve spent a number of months building a
workstation for machine learning. Some of the
posts I read talked about multiple GPUs.

So I bought and installed two GPUs in my motherboard.

nvidia-smi --list-gpus
GPU 0: GeForce GTX 1060 6GB (UUID: …)
GPU 1: GeForce GTX 1060 6GB (UUID: …)

I’m finally getting started on Lesson 1 of FastAI 2019

github.com

fastai/course-v3/blob/master/nbs/dl1/lesson1-pets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 1 - What's your pet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welcome to lesson 1! For those of you who are using a Jupyter Notebook for the first time, you can learn about this useful tool in a tutorial we prepared specially for you; click `File`->`Open` now and click `00_notebook_tutorial.ipynb`. \n",
    "\n",
    "In this lesson we will build our first image classifier from scratch, and see if we can achieve world-class results. Let's dive in!\n",
    "\n",
    "Every notebook starts with the following three lines; they ensure that any edits to libraries you make are reloaded here automatically, and also that any charts or images displayed are shown in this notebook."
   ]
  },

This file has been truncated. show original

The initial code with RESNET34 worked

But skipping over the RESNET34 code, and
using RESNET50 code, I got the error:

RuntimeError: CUDA out of memory.
Tried to allocate 2.00 MiB (GPU 0; 5.93 GiB total capacity;
4.57 GiB already allocated; 2.25 MiB free; 91.03 MiB cached)

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1058 G /usr/lib/xorg/Xorg 35MiB |
| 0 1671 G /usr/lib/xorg/Xorg 390MiB |
| 0 1901 G cinnamon 116MiB |
| 0 2467 C /usr/lib/libreoffice/program/soffice.bin 63MiB |
| 0 2543 G …AAAAAAAAAAAACAAAAAAAAAA= --shared-files 156MiB |
| 0 3750 C /home/oracle/anaconda3/bin/python 5193MiB |
±----------------------------------------------------------------------------+

The code didn’t use the second GPU at all.

I was doing some reading at:

…

All this webpages mentions is:
Order of GPUs
Not how to use multiple GPUs

…

and

How to use Multiple GPUs?

Part 1 (2018)

Posts from 2017 to early 2019

With that version, some found that
it might have been beneficial
to work with 2, but not 3 or 4 GPUs

Q1:
For FastAI Course v3 - 2019,
was a solution found to use two GPUs?

Q2:
For Course v4 - 2020 (Part 1) → fastai v2
was a multi GPU solution found?

If so, please send the links.

Thanks a lot

orendar · August 30, 2020, 1:15pm

Hey, using multiple GPUs usually refers to either running multiple experiments in parallel (where every experiment is running on 1 GPU) or running batches on multiple GPUs at the same time. However, your GPUs are old and don’t have much RAM and so you’re running out of GPU memory when trying to train RESNET50 - the 2nd GPU won’t be able to help you there since the entire model has to be placed on the same GPU when using fastai.

ml22 · August 31, 2020, 2:49pm

I found this thread from October 2018

Some people got multiple GPUs to work

Then had issues saving the model

How to use multiple gpus

ml22 · August 31, 2020, 2:50pm

Looks like fastai library v2 has
libraries to take advantage of multiple GPUs

fastgpu:
… If more than one GPU is available, multiple scripts are run in parallel, one per GPU.

fastgpu library:

ml22 · August 31, 2020, 3:07pm

There is also this documentation:

Distributed and parallel training

Although it doesn’t state the
version of the fastai library
it applies to

felixsmueller · October 7, 2021, 4:24pm

Just in case you want to use multiple GPUs for inference you can split up the work and then load the mode on dedicated GPUs as follows:
fastai_learner = load_learner(self.model_directory + ‘/’ + Train.TRAINED_MODEL_FILE_NAME, cpu=True) #cpu=True, to avoid that we load it to GPU 0
self.fastai_learner.model.cuda(gpu_id) #Load it do the GPU definde by gpu_id

sailngarbwm21 · May 25, 2022, 4:34am

HI,

I have tried following the instructions on this documentation page, however, for step two it states

Run configure_accelerate from the command line, however, it doesn’t say where that CLI comes from.
Is that a CUDA command?

Thank you in advance,

Jon