gpboost.train

gpboost.train(params, train_set, num_boost_round=100, gp_model=None, use_gp_model_for_validation=True, train_gp_model_cov_pars=True, valid_sets=None, valid_names=None, fobj=None, feval=None, init_model=None, feature_name='auto', categorical_feature='auto', early_stopping_rounds=None, evals_result=None, verbose_eval=True, learning_rates=None, keep_training_booster=False, callbacks=None)[source]

Training function.

Parameters:

params (dict) – Parameters for training. See https://github.com/fabsig/GPBoost/blob/master/docs/Main_parameters.rst#tuning-parameters–hyperparameters-for-the-tree-boosting-part
train_set (Dataset) – Data to be trained on.
num_boost_round (int, optional (default=100)) – Number of boosting iterations.
gp_model (GPModel or None, optional (default=None)) – GPModel object for the GPBoost algorithm
use_gp_model_for_validation (bool, optional (default=True)) – If True, the ‘gp_model’ (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If False, the ‘gp_model’ (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.
train_gp_model_cov_pars (bool, optional (default=True)) – If True, the covariance parameters of the ‘gp_model’ (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the ‘gp_model’ parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide values via the ‘init_cov_pars’ parameter when creating the ‘gp_model’
valid_sets (list of Datasets or None, optional (default=None)) – List of data to be evaluated on during training.
valid_names (list of strings or None, optional (default=None)) – Names of valid_sets.
fobj (callable or None, optional (default=None)) –
Customized objective function. Only for independent boosting. The GPBoost algorithm currently does not support this. Should accept two parameters: preds, train_data, and return (grad, hess).

predslist or numpy 1-D array
The predicted values.

train_dataDataset
The training dataset.

gradlist or numpy 1-D array
The value of the first order derivative (gradient) for each sample point.

hesslist or numpy 1-D array
The value of the second order derivative (Hessian) for each sample point.

For binary task, the preds is margin. For multi-class task, the preds is group by class_id first, then group by row_id. If you want to get i-th row preds in j-th class, the access way is score[j * num_data + i] and you should group grad and hess in this way as well.
feval (callable, list of callable functions or None, optional (default=None)) –
Customized evaluation function. Each evaluation function should accept two parameters: preds, train_data, and return (eval_name, eval_result, is_higher_better) or list of such tuples.

predslist or numpy 1-D array
The predicted values.

train_dataDataset
The training dataset.

eval_namestring
The name of evaluation function (without whitespaces).

eval_resultfloat
The eval result.

is_higher_betterbool
Is eval result higher better, e.g. AUC is is_higher_better.

For binary task, the preds is probability of positive class (or margin in case of specified fobj). For multi-class task, the preds is group by class_id first, then group by row_id. If you want to get i-th row preds in j-th class, the access way is preds[j * num_data + i]. To ignore the default metric corresponding to the used objective, set the metric parameter to the string "None" in params.
init_model (string, Booster or None, optional (default=None)) – Filename of GPBoost model or Booster instance used for continue training.
feature_name (list of strings or 'auto', optional (default="auto")) – Feature names. If ‘auto’ and data is pandas DataFrame, data columns names are used.
categorical_feature (list of strings or int, or 'auto', optional (default="auto")) – Categorical features. If list of int, interpreted as indices. If list of strings, interpreted as feature names (need to specify feature_name as well). If ‘auto’ and data is pandas DataFrame, pandas unordered categorical columns are used. All values in categorical features should be less than int32 max value (2147483647). Large values could be memory consuming. Consider using consecutive integers starting from zero. All negative values in categorical features will be treated as missing values. The output cannot be monotonically constrained with respect to a categorical feature.
early_stopping_rounds (int or None, optional (default=None)) – Activates early stopping. The model will train until the validation score stops improving. Validation score needs to improve at least every early_stopping_rounds round(s) to continue training. Requires at least one validation data and one metric. If there’s more than one, will check all of them. But the training data is ignored anyway. To check only the first metric, set the first_metric_only parameter to True in params. The index of iteration that has the best performance will be saved in the best_iteration field if early stopping logic is enabled by setting early_stopping_rounds.
evals_result (dict or None, optional (default=None)) –
This dictionary used to store all evaluation results of all the items in valid_sets. .. rubric:: Example

With a valid_sets = [valid_set, train_set], valid_names = [‘eval’, ‘train’] and a params = {‘metric’: ‘logloss’} returns {‘train’: {‘logloss’: [‘0.48253’, ‘0.35953’, …]}, ‘eval’: {‘logloss’: [‘0.480385’, ‘0.357756’, …]}}.
verbose_eval (bool or int, optional (default=True)) –
Requires at least one validation data. If True, the eval metric on the valid set is printed at each boosting stage. If int, the eval metric on the valid set is printed at every verbose_eval boosting stage. The last boosting stage or the boosting stage found by using early_stopping_rounds is also printed.

Example

With verbose_eval = 4 and at least one item in valid_sets, an evaluation metric is printed every 4 (instead of 1) boosting stages.
learning_rates (list, callable or None, optional (default=None)) – List of learning rates for each boosting round or a customized function that calculates learning_rate in terms of current number of round (e.g. yields learning rate decay).
keep_training_booster (bool, optional (default=False)) – Whether the returned Booster will be used to keep training. If False, the returned value will be converted into _InnerPredictor before returning. When your model is very large and cause the memory error, you can try to set this param to True to avoid the model conversion performed during the internal call of model_to_string. You can still use _InnerPredictor as init_model for future continue training.
callbacks (list of callables or None, optional (default=None)) – List of callback functions that are applied at each iteration. See Callbacks in Python API for more information.

Returns:

booster – The trained Booster model.

Return type:

Booster

Example

>>> gp_model = gpb.GPModel(group_data=group, likelihood="gaussian")
>>> data_train = gpb.Dataset(X, y)
>>> params = {'learning_rate': 0.01, 'max_depth': 3, 'num_leaves': 2**10, 'verbose': 0}
>>> bst = gpb.train(params=params, train_set=data_train,  gp_model=gp_model,
>>>                 num_boost_round=100)

Authors:: Authors of the LightGBM Python package Fabio Sigrist