[mt] Feature importance variants.#11950
Merged
trivialfis merged 9 commits intodmlc:masterfrom Jan 22, 2026
Merged
Conversation
- Store sum hessian over all targets. - Store split gain. We might change it to per-target sum hessian, but I would like to see how the SHAP value works with vector leaf before doing that.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds multi-target tree support for additional feature importance variants (gain/cover and totals) by storing and exposing per-node statistics, plus expands test coverage across CPU/GPU and mixed multi-strategy training.
Changes:
- Persist and expose
loss_chg(gain) andsum_hess(cover) for multi-target trees, including JSON serialization. - Update CPU/GPU multi-target training paths and tree views to populate/read these statistics efficiently (optionally skipping stats for inference paths).
- Add/relocate tests for feature importance variants, column sampling, and mixed multi-strategy behavior on CPU and GPU.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/python/test_multi_target.py | Adds CPU test entrypoints for new multi-target test helpers. |
| tests/python/test_basic_models.py | Removes multi-tree slicing test (coverage moved to multi-target suite). |
| tests/python-gpu/test_gpu_multi_target.py | Adds GPU tests for mixed strategy and feature-importance comparison. |
| tests/cpp/tree/test_partitioner.h | Updates test helpers to new SetRoot/ExpandNode multi-target signatures. |
| tests/cpp/tree/test_multi_target_tree_model.cc | Updates signatures and adds serialization round-trip test for stats. |
| tests/cpp/tree/hist/test_evaluate_splits.cc | Ensures root sum_hess is set for multi-target hist evaluator tests. |
| tests/cpp/predictor/test_predictor.cc | Updates predictor test tree construction to include multi-target stats. |
| src/tree/updater_quantile_hist.cc | Computes and sets root sum_hess for multi-target quantile hist builder. |
| src/tree/updater_gpu_hist.cuh | Populates multi-target node stats on GPU hist path; avoids stats for partitioning. |
| src/tree/tree_view.h | Adds multi-target stat pointers and new constructor flag need_stat. |
| src/tree/tree_view.cc | Implements conditional stat loading for multi-target tree views. |
| src/tree/tree_model.cc | Updates multi-target ExpandNode signature plumbing. |
| src/tree/multi_target_tree_model.cc | Stores stats in MultiTargetTree, and saves/loads them in JSON. |
| src/tree/hist/evaluate_splits.h | Populates multi-target split stats (gain/cover) on CPU hist expand. |
| src/tree/gpu_hist/multi_evaluate_splits.cu | Tracks summed child hessians across all targets for GPU multi-evaluator. |
| src/tree/gpu_hist/expand_entry.cuh | Renames/repurposes hessian fields to summed left/right hessians. |
| src/predictor/gbtree_view.h | Passes need_stat through when building per-tree views. |
| src/gbm/gbtree.h | Enables gain/cover feature importance for multi-target trees using new stats. |
| python-package/xgboost/testing/multi_target.py | Adds new Python test helpers covering importance variants and strategies. |
| include/xgboost/tree_model.h | Extends multi-target SetRoot/ExpandNode APIs with stat parameters. |
| include/xgboost/multi_target_tree_model.h | Extends multi-target tree structure to store loss change and hessian sums. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
Author
|
We can have vector leaf for #11926 as well |
Member
Author
|
cc @rongou |
rongou
approved these changes
Jan 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
At the moment, I store the sum hessian for all targets. We might need to store a vector instead. I will leave it as future work if it comes up.
Ref #9043
todos: