Skip to content

Quantile test failures. #12139

@trivialfis

Description

@trivialfis

Tested on b2f15e6

Python:

tests/python-gpu/test_gpu_data_iterator.py:31: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _tests/python-gpu/test_gpu_data_iterator.py:48: in test_gpu_data_iterator
    run_data_iterator(
tests/python/test_data_iterator.py:203: in run_data_iterator
    _assert_cut_rank_error_within_tolerance(indptr_it, cuts_it, x_np, w_np)
>               np.testing.assert_array_less(
                    np.array([abs(expected_rank - sum_weight)]),
                    np.array([acceptable_error + 1e-12]),
                )
E               AssertionError: 
E               Arrays are not strictly ordered `x < y`
E               
E               Mismatched elements: 1 / 1 (100%)
E               Max absolute difference among violations: 0.14805494
E               Max relative difference among violations: 0.12515479
E                x: array([1.33103])
E                y: array([1.182975])
E               Falsifying example: test_gpu_data_iterator(
E                   n_samples_per_batch=3,
E                   n_features=3,
E                   n_batches=4,
E                   subsample=False,
E                   use_cupy=False,
E                   on_host=False,
E               )
E               
E               You can reproduce this example by temporarily adding @reproduce_failure('6.151.10', b'AEEDQQNBBAAAAA==') as a decorator on your test case

CPP with 2 GPUs:

[ RUN      ] MGPUQuantileTest.AllReduceBasic
[17:31:30] Insecure federated server listening on 0.0.0.0:0, world size 2
/home/jiamingy/ws/xgboost_dev/xgboost/tests/cpp/common/test_hist_util.h:219: Failure
Expected: (stats.max_normalized_error) <= (max_normalized_rank_error), actual: 22.349999999999941 vs 20
feature=6, cut=902, normalized_error=22.349999999999941, absolute_error=39.210526315789366, target_rank=1584.2105263157894, rank_lo=1544, rank_hi=1545, total_weight=2000, num_cuts=1140

/home/jiamingy/ws/xgboost_dev/xgboost/tests/cpp/common/test_hist_util.h:219: Failure
Expected: (stats.max_normalized_error) <= (max_normalized_rank_error), actual: 22.349999999999941 vs 20
feature=6, cut=902, normalized_error=22.349999999999941, absolute_error=39.210526315789366, target_rank=1584.2105263157894, rank_lo=1544, rank_hi=1545, total_weight=2000, num_cuts=1140

/home/jiamingy/ws/xgboost_dev/xgboost/tests/cpp/common/test_hist_util.h:219: Failure
Expected: (stats.max_normalized_error) <= (max_normalized_rank_error), actual: 20.498 vs 20
feature=61, cut=230, normalized_error=20.498000000000001, absolute_error=36.024604569420035, target_rank=405.97539543057997, rank_lo=442, rank_hi=443, total_weight=2000, num_cuts=1138

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions