XGBoost with GPU support

Deep Learning
2

I just did this by following the “Build with GPU support” instructions on that page

On one test, an i7-5930k (6 core 3.5Gz) ran some code (small CV grid search) in 2 minutes using 12 threads, and the GPU completed in 1 minute.

By the way, I read that you can use multiple GPUs by building with NCCL, though I wasn’t able to get that to work yet. Here are the steps I followed to install XGBoost with GPU and NCCL in case anyone’s interested,

  1. Install NCCL

    • Follow Nvidia’s guide for 16.04. It doesn’t mention where to download one needed .deb file, so:
    • Download nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb from here, assuming you have 16.04
    • Complete the steps in the guide

  2. Follow the build instructions and add the USE_NCCL parameter

    • cmake .. -DUSE_CUDA=ON -DUSE_NCCL=ON

  3. In the code, supply the appropriate parameters as mentioned on this page and this issue

     params = {'tree_method':'gpu_hist', 'n_gpus': 2, 'eta': 0.01, ' subsample': 0.7,
               'colsample_bytree': 0.8, 'max_depth':3, 'min_child_weight':3,
               'objective': 'binary:logistic', 'seed':0} 

 cv_xgb = xgb.cv(params = params, dtrain = train_dmat,
                num_boost_round = 3000, nfold = 5, metrics = ['error'],
                early_stopping_rounds = 100)

If I set n_gpus to more than 1, I get this error:

Check failed: device_ordinals.size() == 1 (2 vs. 1) XGBoost must be compiled with NCCL to use more than one GPU

I did reinstall the python XGBoost tools from source after rebuilding XGBoost with NCCL turned on.

By the way, I found this guide/walkthrough to be a pretty good introduction to XG-Boost