Skip to content

[mt] Feature importance variants.#11950

Merged
trivialfis merged 9 commits intodmlc:masterfrom
trivialfis:mt-stat
Jan 22, 2026
Merged

[mt] Feature importance variants.#11950
trivialfis merged 9 commits intodmlc:masterfrom
trivialfis:mt-stat

Conversation

@trivialfis
Copy link
Copy Markdown
Member

@trivialfis trivialfis commented Jan 22, 2026

  • Gain and coverage.

At the moment, I store the sum hessian for all targets. We might need to store a vector instead. I will leave it as future work if it comes up.

Ref #9043

todos:

  • Test mixed tree types.

- Store sum hessian over all targets.
- Store split gain.

We might change it to per-target sum hessian, but I would like to see how the SHAP value
works with vector leaf before doing that.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds multi-target tree support for additional feature importance variants (gain/cover and totals) by storing and exposing per-node statistics, plus expands test coverage across CPU/GPU and mixed multi-strategy training.

Changes:

  • Persist and expose loss_chg (gain) and sum_hess (cover) for multi-target trees, including JSON serialization.
  • Update CPU/GPU multi-target training paths and tree views to populate/read these statistics efficiently (optionally skipping stats for inference paths).
  • Add/relocate tests for feature importance variants, column sampling, and mixed multi-strategy behavior on CPU and GPU.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/python/test_multi_target.py Adds CPU test entrypoints for new multi-target test helpers.
tests/python/test_basic_models.py Removes multi-tree slicing test (coverage moved to multi-target suite).
tests/python-gpu/test_gpu_multi_target.py Adds GPU tests for mixed strategy and feature-importance comparison.
tests/cpp/tree/test_partitioner.h Updates test helpers to new SetRoot/ExpandNode multi-target signatures.
tests/cpp/tree/test_multi_target_tree_model.cc Updates signatures and adds serialization round-trip test for stats.
tests/cpp/tree/hist/test_evaluate_splits.cc Ensures root sum_hess is set for multi-target hist evaluator tests.
tests/cpp/predictor/test_predictor.cc Updates predictor test tree construction to include multi-target stats.
src/tree/updater_quantile_hist.cc Computes and sets root sum_hess for multi-target quantile hist builder.
src/tree/updater_gpu_hist.cuh Populates multi-target node stats on GPU hist path; avoids stats for partitioning.
src/tree/tree_view.h Adds multi-target stat pointers and new constructor flag need_stat.
src/tree/tree_view.cc Implements conditional stat loading for multi-target tree views.
src/tree/tree_model.cc Updates multi-target ExpandNode signature plumbing.
src/tree/multi_target_tree_model.cc Stores stats in MultiTargetTree, and saves/loads them in JSON.
src/tree/hist/evaluate_splits.h Populates multi-target split stats (gain/cover) on CPU hist expand.
src/tree/gpu_hist/multi_evaluate_splits.cu Tracks summed child hessians across all targets for GPU multi-evaluator.
src/tree/gpu_hist/expand_entry.cuh Renames/repurposes hessian fields to summed left/right hessians.
src/predictor/gbtree_view.h Passes need_stat through when building per-tree views.
src/gbm/gbtree.h Enables gain/cover feature importance for multi-target trees using new stats.
python-package/xgboost/testing/multi_target.py Adds new Python test helpers covering importance variants and strategies.
include/xgboost/tree_model.h Extends multi-target SetRoot/ExpandNode APIs with stat parameters.
include/xgboost/multi_target_tree_model.h Extends multi-target tree structure to store loss change and hessian sums.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@trivialfis trivialfis marked this pull request as ready for review January 22, 2026 19:16
@trivialfis
Copy link
Copy Markdown
Member Author

We can have vector leaf for #11926 as well

@trivialfis
Copy link
Copy Markdown
Member Author

cc @rongou

@trivialfis trivialfis merged commit 8ba2d98 into dmlc:master Jan 22, 2026
74 checks passed
@trivialfis trivialfis deleted the mt-stat branch January 22, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants