[WIP] Pass objective args through constructors#12097
[WIP] Pass objective args through constructors#12097RAMitchell wants to merge 7 commits intodmlc:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates XGBoost’s objective construction path so objective configuration can be passed at construction time via Args, with learner/objective registrations updated accordingly and tests migrated to prefer constructor-time initialization.
Changes:
- Extend
ObjFunction::Create(...)and objective registry factories to acceptArgs, and pass them through from the learner when (re)constructing objectives. - Add/adjust objective constructors and registrations to initialize internal parameters from constructor args (while keeping
Configure(...)for reconfiguration / loaded instances). - Update C++ tests (and plugin example usage) to construct objectives with args instead of
Create(...); Configure(...).
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
include/xgboost/objective.h |
Update objective factory API + registry signature to accept Args. |
src/objective/objective.cc |
Pass Args through registry body when creating objectives. |
src/learner.cc |
Construct new objectives with args; only Configure(...) existing objectives. |
src/objective/regression_obj.cu |
Update regression objectives to accept args in constructors/registrations; adjust Tweedie load behavior. |
src/objective/quantile_obj.cu |
Add args constructor + update registry body for quantile objective. |
src/objective/multiclass_obj.cu |
Add args-aware constructor + update registrations for softmax/softprob. |
src/objective/lambdarank_obj.cc |
Add args ctor forwarding and pass args through registrations. |
src/objective/aft_obj.cu |
Add args ctor and pass args through registration. |
src/objective/hinge.cu |
Update registration body to match new factory signature. |
plugin/example/custom_obj.cc |
Update plugin objective registration to new factory signature. |
tests/cpp/objective/test_objective.cc |
Prefer constructor-time args initialization across objective registry tests. |
tests/cpp/objective/test_regression_obj.cc |
Update regression objective tests to pass args into Create. |
tests/cpp/objective/test_regression_obj_cpu.cc |
Update CUDA-vs-CPU objective create path to pass args. |
tests/cpp/objective/test_quantile_obj.cc |
Update quantile objective tests to pass args into Create. |
tests/cpp/objective/test_multiclass_obj.cc |
Update multiclass objective tests to pass args into Create and reformat. |
tests/cpp/objective/test_hinge.cc |
Update hinge objective test to pass args into Create and reformat. |
tests/cpp/objective/test_aft_obj.cc |
Update AFT objective test to use constructor-time args for configuration case; reformat. |
tests/cpp/predictor/test_shap.cc |
Construct objective with model args directly instead of calling Configure. |
tests/cpp/plugin/test_sycl_regression_obj.cc |
Update SYCL regression objective creation signature + minor formatting. |
tests/cpp/plugin/test_example_objective.cc |
Update example objective test to pass args into Create + header formatting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@trivialfis after playing around with objective configuration I think I'm starting to understand the learner configuration process and how we might tackle its fundamental difficulties. I see the main difficulty in that parameter updates are stored in learner as "deltas" but we don't keep a snapshot of the canonical learner config. If we keep at all times a config as a snapshot of truth and reconcile updates against it I think things will simplify. |
|
I think that's a rabbit hole once you start handling things like eta decay. Basically, you need to preserve an entire copy of the parameter structure in the learner as a mutable state, with internal dependencies. Then you need to compare the states to check whether changes have occurred in some components. In the end, you might end up something like what we are currently doing: each component handles its own parameter. |
|
I noticed that decay issue during this but I think it's still a significant simplification to go from deltas of configurations to whole configurations. |
This PR moves objective initialization toward constructor-time configuration.
ObjFunction::Create(...)now acceptsArgs, the learner passes configuration when constructing fresh objectives, and stateful objectives initialize from constructor arguments while still supportingConfigure(...)for existing loaded instances.As a follow-up within the same branch, the tests were updated to prefer constructor-time objective initialization instead of
Create(...); Configure(...)where deferred setup was not intentional. The remaining deferred-init cases are now mostly limited to LambdaRank and load/clone-style paths like intercept estimation, which makes the next cleanup step much clearer.Verification
cmake --build build-cpu-quantile -j35 --target testxgboost./build-cpu-quantile/testxgboost --gtest_filter='Objective.*:AFTObj.*:*Quantile*:*RegressionObj*:*MultiClass*:*Hinge*'./build-cpu-quantile/testxgboost --gtest_filter='Objective.CPU_vs_CUDA:SyclObjective.CPUvsSycl:Plugin.ExampleObjective:Predictor.ShapOutputCasesCPU:Predictor.ApproxContribsBasic:Predictor.ShapIterationRange'