fix: correct pvalue logic and add discretization step#54
Open
drbh wants to merge 1 commit into
Open
Conversation
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR supersedes #53 and ensures that the experimental output is identical to the current output with a relative accuracy of
<1e-6and absolute accuracy of<1e-12.Additionally this PR renames and improves the
tests/bench_expr.pyfile. The newtests/test_experimental_correctness.pyfile will run with the other test (prefixed withtest_) and compares the output between the reference and experimental outputs to ensure they are within a extremely small tolerancenote these changes slow down the overall computation by a small amount (mainly when the discretization step is needed) which can be avoid but providing a the raw integer data
======================================================== test session starts ======================================================== platform darwin -- Python 3.10.15, pytest-8.4.1, pluggy-1.6.0 benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /Users/drbh/Projects/pdex configfile: pyproject.toml plugins: benchmark-5.1.0 collected 9 items tests/test_experimental_correctness.py ......... [100%] ========================================================= warnings summary ========================================================== tests/test_experimental_correctness.py::test_correctness_comparison tests/test_experimental_correctness.py::test_detailed_correctness_metrics tests/test_experimental_correctness.py::test_edge_cases_numerical_stability tests/test_experimental_correctness.py::test_edge_cases_numerical_stability /Users/drbh/Projects/pdex/.venv/lib/python3.10/site-packages/anndata/_core/aligned_df.py:68: ImplicitModificationWarning: Transforming to str index. warnings.warn("Transforming to str index.", ImplicitModificationWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ------------------------------------------------------------------------------------------------------------ benchmark: 6 tests ------------------------------------------------------------------------------------------------------------ Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- test_benchmark_parameterized_datasets[True-500-100-10] 7.3009 (1.0) 8.3150 (1.0) 7.9079 (1.0) 0.4193 (1.0) 7.9405 (1.0) 0.6697 (1.0) 1;0 126.4562 (1.0) 5 1 test_benchmark_parameterized_datasets[True-1000-300-100] 81.2446 (11.13) 83.1028 (9.99) 82.1008 (10.38) 0.7036 (1.68) 82.1472 (10.35) 0.9652 (1.44) 2;0 12.1802 (0.10) 5 1 test_benchmark_parameterized_datasets[True-2000-500-50] 113.6737 (15.57) 124.0537 (14.92) 118.8101 (15.02) 4.5535 (10.86) 119.7459 (15.08) 8.1712 (12.20) 2;0 8.4168 (0.07) 5 1 test_benchmark_parameterized_datasets[False-500-100-10] 1,121.4055 (153.60) 1,163.2690 (139.90) 1,146.4066 (144.97) 15.3490 (36.61) 1,150.1841 (144.85) 13.6819 (20.43) 2;0 0.8723 (0.01) 5 1 test_benchmark_parameterized_datasets[False-2000-500-50] 3,598.1983 (492.84) 3,636.3652 (437.33) 3,620.5203 (457.84) 14.3509 (34.23) 3,620.4431 (455.95) 17.2623 (25.78) 2;0 0.2762 (0.00) 5 1 test_benchmark_parameterized_datasets[False-1000-300-100] 3,952.2674 (541.34) 4,037.7101 (485.59) 3,976.9548 (502.91) 34.6742 (82.70) 3,966.9077 (499.58) 30.1742 (45.06) 1;1 0.2514 (0.00) 5 1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Legend: Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. OPS: Operations Per Second, computed as 1 / Mean ============================================= 9 passed, 4 warnings in 71.21s (0:01:11) ==============================================