XGBoost Parameter Documentation and Using GPUs:

Using XGBoost with GPU

For those of you using XGBoost in your Kaggle competitions. Here’s the documentation for some of the parameters you might consider tuning.

When using the GPU, makes sure to include the correct parameters

Also, note that if you are running XGBoost on computer with a GPU, you have to pass a parameter to take advantage of it.

param['gpu_id'] = 0
param['max_bin'] = 16
param['tree_method'] = 'gpu_hist'


tree_method Description
gpu_exact The standard XGBoost tree construction algorithm. Performs exact search for splits. Slower and uses considerably more memory than ‘gpu_hist’
gpu_hist Equivalent to the XGBoost fast histogram algorithm. Much faster and uses considerably less memory. NOTE: Cannot be used with labels larger in magnitude than 2^16 due to it’s histogram aggregation algorithm.

Notes on GPU usage:


Parameters Documentation:

The best advice on XGB optimization I’ve seen is the one @kcturgutlu just posted: