Mastering Machine Learning: What’s New in XGBoost 2.0
XGBoost 2.0 marks a significant milestone in the evolution of this renowned machine learning library. This release introduces groundbreaking features, including multi-target trees with vector-leaf outputs, enhanced GPU support, and improved memory management. The default “hist” tree method now ensures more efficient model training, while the introduction of learning-to-rank capabilities and quantile regression further expands XGBoost’s versatility. With a focus on performance, flexibility, and usability, XGBoost 2.0 empowers data scientists and machine learning practitioners to achieve even greater insights and accuracy in their models.
Multi-Target Trees with Vector-Leaf Outputs
Version 2.0 introduces vector-leaf tree models for multi-target regression, multi-label classification, and multi-class classification.
Unlike the previous version, XGBoost can now build one tree for all targets.
This feature helps prevent overfitting, produces smaller models, and considers target correlations.
It’s still a work in progress.
New Device Parameter
A new “device” parameter replaces several GPU-related parameters.
Users can specify the device and its ordinal to run XGBoost on a specific device.
The old “gpu_hist” behavior is deprecated.
Hist as Default Tree Method
Starting from version 2.0, the “hist” tree method is the default.
This improves model training efficiency and consistency
GPU-Based Approx Tree Method
Version 2.0 introduces initial support for the “approx” tree method on GPU.
It can be used with the “device” and “tree_method” parameters.
Performance is still being optimized.
Optimizing Histogram Size on CPU
A new parameter, “max_cached_hist_node,” limits CPU cache size for histograms.
It prevents aggressive caching and reduces memory usage.
There’s also a memory usage reduction for “hist” and “approx” tree methods on distributed systems.
0 Comments