Conversation
maxjeblick
left a comment
There was a problem hiding this comment.
Thanks a lot for opening this PR!
The overall code looks good, thanks for adding it. There are several points that I'd ask to address before merging:
- We prefer to have a new press class (
ChunkKVPress) associated with your paper, as your method (selecting whole chunks) differs fromChunkPress's idea (compress each chunk independently). Both methods also have different default parameters (e.g.chunk_length). - After introducing a new class, you can add a new test (similar to
test_chunk_press). Please do not modify the existing test that usesExpectedAttentionPress. - Please add your class in the corresponding README section (available presses).
- Most style errors can be fixed my running
make format. You can test style locally bymake style. Feel free to add# noqa:flags if necessary; e.g. sometimes mypy has issues to resolve tensor attributes correctly. - Please don't forget to sign off your commit. See https://github.com/NVIDIA/kvpress/blob/main/CONTRIBUTING.md for additional information.
Feel free to reach out with any questions or comments!
|
Hi! Thanks for the review and suggestions! I've made the following changes as requested:
|
maxjeblick
left a comment
There was a problem hiding this comment.
Thanks a lot for the updates, code looks good!
Could you shorten your description of ChunkKVPress in the README (see also other descriptions)?
Also, there's a minor issue with style checks, it should be easy to fix.
maxjeblick
left a comment
There was a problem hiding this comment.
Thanks a lot for integrating ChunkKVPress, LGTM!
|
@Dominic789654
|
3f4332f to
4fb6d1b
Compare
Signed-off-by: Dominic789654 <[email protected]>
Signed-off-by: Dominic789654 <[email protected]>
Signed-off-by: Dominic789654 <[email protected]>
Signed-off-by: Dominic789654 <[email protected]>
Signed-off-by: Dominic789654 <[email protected]>
PR Description
Add ChunkKV paper implementation (https://arxiv.org/abs/2502.00299)
Benchmark Results
Results on Ruler (4096) dataset under different compression ratios (0.1, 0.25, 0.5):
New press checklist
mypress_press.pyin thepressesdirectoryMyPressin__init__.pyREADME.mdwith a 1 liner about my new press in the Available presses sectiondefault_presseslist intests/default_presses.py