Page 179 - Kỷ yếu hội thảo khoa học lần thứ 12 - Công nghệ thông tin và Ứng dụng trong các lĩnh vực (CITA 2023)

Tran Quy Nam and Phi Cong Huy 163

To find out the set of functions used in the model, the following normative objective
function minimization algorithm:

where

Where, l is a differentiable convex loss function used to measure the difference
between the predicted value and the actual value yi.
penalty for model complexity (e.g. function of a regression tree). The additional
normalization component smooth the learned final weights to avoid over-fitting.
Visually, the normative objective tends to choose a model that uses simple but highly
predictive functions.
The Gradient Tree Boosting algorithm is performed when the model is
continuously trained in the way of feature addition. Formally, if is the i-th
th
prediction value at the t loop, the algorithm will need to add the f t component to
reduce the objective function as follows:

The second order approximation is used to optimize faster than the objective function
in the algorithm implementation.

where và is the first and second

order gradients on the loss function. We can remove the constants to obtain a simpler
objective function as follows in step t.

Definition Ij = {i|q(xi) = j} is the set representing the composition of leaf j. We can
calculate the optimal weight of leaf j by:

Calculate the corresponding optimal value by:

ISBN: 978-604-80-8083-9 CITA 2023