Commit 9c0da51
committed
test(dialect): strengthen compression_stats and count_tokens assertions
Adds coverage on top of the honest-stats test fix that already landed
in main:
- test_stats now also asserts original_tokens_est is strictly greater
than summary_tokens_est, which catches a class of regressions where
the token estimator flattens to a constant.
- test_count_tokens gains edge cases for the empty string and the
single-word input. Both exercise the max(1, ...) guard in
Dialect.count_tokens, so a future refactor that drops the guard
fails loudly instead of silently returning 0.1 parent b57c160 commit 9c0da51
1 file changed
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| 114 | + | |
114 | 115 | | |
115 | 116 | | |
| 117 | + | |
| 118 | + | |
116 | 119 | | |
| 120 | + | |
| 121 | + | |
117 | 122 | | |
118 | 123 | | |
119 | 124 | | |
| |||
0 commit comments