AddGBoost: A gradient boosting-style algorithm based on strong learners

Moshe Sipper; Jason H. Moore

doi:10.1016/j.mlwa.2021.100243

AddGBoost: A gradient boosting-style algorithm based on strong learners

Moshe Sipper, Jason H. Moore

Department of Computer Science

Ben-Gurion University of the Negev

Research output: Contribution to journal › Article › peer-review

Abstract

We present AddGBoost, a gradient boosting-style algorithm, wherein the decision tree is replaced by a succession of (possibly) stronger learners, which are optimized via a state-of-the-art hyperparameter optimizer. Through experiments over 90 regression datasets we show that AddGBoost emerges as the top performer for 33% (with 2 stages) up to 42% (with 5 stages) of the datasets, when compared with seven well-known machine-learning algorithms: KernelRidge, LassoLars, SGDRegressor, LinearSVR, DecisionTreeRegressor, HistGradientBoostingRegressor, and LGBMRegressor.

Original language	American English
Article number	100243
Number of pages	4
Journal	Machine Learning with Applications
Volume	7
DOIs	https://doi.org/10.1016/j.mlwa.2021.100243
State	Published - Mar 2022

Access to Document

10.1016/j.mlwa.2021.100243

Cite this

@article{87c9e1b8d22a498c8f952e2be705064a,

title = "AddGBoost: A gradient boosting-style algorithm based on strong learners",

abstract = "We present AddGBoost, a gradient boosting-style algorithm, wherein the decision tree is replaced by a succession of (possibly) stronger learners, which are optimized via a state-of-the-art hyperparameter optimizer. Through experiments over 90 regression datasets we show that AddGBoost emerges as the top performer for 33% (with 2 stages) up to 42% (with 5 stages) of the datasets, when compared with seven well-known machine-learning algorithms: KernelRidge, LassoLars, SGDRegressor, LinearSVR, DecisionTreeRegressor, HistGradientBoostingRegressor, and LGBMRegressor.",

author = "Moshe Sipper and Moore, {Jason H.}",

year = "2022",

month = mar,

doi = "10.1016/j.mlwa.2021.100243",

language = "American English",

volume = "7",

}

TY - JOUR

T1 - AddGBoost: A gradient boosting-style algorithm based on strong learners

AU - Sipper, Moshe

AU - Moore, Jason H.

PY - 2022/3

Y1 - 2022/3

N2 - We present AddGBoost, a gradient boosting-style algorithm, wherein the decision tree is replaced by a succession of (possibly) stronger learners, which are optimized via a state-of-the-art hyperparameter optimizer. Through experiments over 90 regression datasets we show that AddGBoost emerges as the top performer for 33% (with 2 stages) up to 42% (with 5 stages) of the datasets, when compared with seven well-known machine-learning algorithms: KernelRidge, LassoLars, SGDRegressor, LinearSVR, DecisionTreeRegressor, HistGradientBoostingRegressor, and LGBMRegressor.

AB - We present AddGBoost, a gradient boosting-style algorithm, wherein the decision tree is replaced by a succession of (possibly) stronger learners, which are optimized via a state-of-the-art hyperparameter optimizer. Through experiments over 90 regression datasets we show that AddGBoost emerges as the top performer for 33% (with 2 stages) up to 42% (with 5 stages) of the datasets, when compared with seven well-known machine-learning algorithms: KernelRidge, LassoLars, SGDRegressor, LinearSVR, DecisionTreeRegressor, HistGradientBoostingRegressor, and LGBMRegressor.

U2 - 10.1016/j.mlwa.2021.100243

DO - 10.1016/j.mlwa.2021.100243

M3 - Article

SN - 2666-8270

VL - 7

JO - Machine Learning with Applications

JF - Machine Learning with Applications

M1 - 100243

ER -