timlee
November 12, 2017, 10:12pm
1
Using XGBoost with GPU
For those of you using XGBoost in your Kaggle competitions. Here’s the documentation for some of the parameters you might consider tuning.
When using the GPU, makes sure to include the correct parameters
Also, note that if you are running XGBoost on computer with a GPU, you have to pass a parameter to take advantage of it.
param['gpu_id'] = 0
param['max_bin'] = 16
param['tree_method'] = 'gpu_hist'
Algorithms
tree_method
Description
gpu_exact
The standard XGBoost tree construction algorithm. Performs exact search for splits. Slower and uses considerably more memory than ‘gpu_hist’
gpu_hist
Equivalent to the XGBoost fast histogram algorithm. Much faster and uses considerably less memory. NOTE: Cannot be used with labels larger in magnitude than 2^16 due to it’s histogram aggregation algorithm.
Notes on GPU usage:
http://xgboost.readthedocs.io/en/latest/gpu/
Parameters Documentation:
XGBoost Parameters
==================
Before running XGboost, we must set three types of parameters: general parameters, booster parameters and task parameters.
- General parameters relates to which booster we are using to do boosting, commonly tree or linear model
- Booster parameters depends on which booster you have chosen
- Learning Task parameters that decides on the learning scenario, for example, regression tasks may use different parameters with ranking tasks.
- Command line parameters that relates to behavior of CLI version of xgboost.
Parameters in R Package
-----------------------
In R-package, you can use .(dot) to replace underscore in the parameters, for example, you can use max.depth as max_depth. The underscore parameters are also valid in R.
General Parameters
------------------
* booster [default=gbtree]
- which booster to use, can be gbtree, gblinear or dart. gbtree and dart use tree based model while gblinear uses linear function.
* silent [default=0]
- 0 means printing running messages, 1 means silent mode.
* nthread [default to maximum number of threads available if not set]
- number of parallel threads used to run xgboost
show original
jeremy
November 13, 2017, 2:50pm
2
The best advice on XGB optimization I’ve seen is the one @kcturgutlu just posted:
2 Likes