Skip to content

Add configurable arXiv cross-list support#195

Merged
TideDra merged 3 commits into
TideDra:mainfrom
reoLantern:main
Mar 12, 2026
Merged

Add configurable arXiv cross-list support#195
TideDra merged 3 commits into
TideDra:mainfrom
reoLantern:main

Conversation

@reoLantern

Copy link
Copy Markdown
Contributor

Summary

This PR adds a config switch to control whether arXiv cross-listed papers should be included when retrieving papers from subscribed categories.

Previously, only papers with arxiv_announce_type == "new" were included. This caused some relevant papers to be missed, even when they appeared in the official arXiv email for a subscribed category.

Changes

  • add source.arxiv.include_cross_list to the base config
  • keep the default behavior unchanged with include_cross_list: false
  • update the arXiv retriever to include both new and cross entries when the switch is enabled
  • update the README example for CUSTOM_CONFIG to show where this option should be added

Example

Users can enable this in CUSTOM_CONFIG with:

source:
  arxiv:
    include_cross_list: true

Why

Some papers are not submitted primarily to a subscribed category, but are cross-listed there. These papers can still be highly relevant, so making cross-list inclusion configurable helps users choose between:

  • stricter category-only results
  • broader coverage including cross-listed papers

Copilot AI review requested due to automatic review settings March 11, 2026 16:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a configuration flag to optionally include arXiv cross-listed papers when retrieving papers for subscribed arXiv categories, preserving the existing default behavior.

Changes:

  • Add source.arxiv.include_cross_list to the base configuration (default false)
  • Update arXiv RSS filtering logic to optionally include both new and cross announce types
  • Update README CUSTOM_CONFIG example and guidance to mention the new option

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/zotero_arxiv_daily/retriever/arxiv_retriever.py Adds configurable filtering to include cross-listed (cross) arXiv RSS entries when enabled.
config/base.yaml Introduces the include_cross_list config key with a default of false.
README.md Documents where/how to set include_cross_list in CUSTOM_CONFIG.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/zotero_arxiv_daily/retriever/arxiv_retriever.py
Comment thread README.md Outdated
Comment on lines +30 to +35
allowed_announce_types = {"new", "cross"} if include_cross_list else {"new"}
all_paper_ids = [
i.id.removeprefix("oai:arXiv.org:")
for i in feed.entries
if i.get("arxiv_announce_type", "new") in allowed_announce_types
]

Copilot AI Mar 11, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new include_cross_list behavior isn’t covered by tests. There is an existing tests/retriever/test_arxiv_retriever.py that asserts only announce_type == "new" entries are included; please add a companion test that sets config.source.arxiv.include_cross_list = True and asserts announce_type == "cross" entries are included as well (the existing RSS fixture already contains cross entries).

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

reoLantern and others added 2 commits March 11, 2026 16:51
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@TideDra TideDra merged commit c2330b9 into TideDra:main Mar 12, 2026
1 check passed
@reoLantern

Copy link
Copy Markdown
Contributor Author

感谢!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants