Skip to content

CHANGELOG.md

Latest commit

 

History

History
265 lines (169 loc) · 14.5 KB

CHANGELOG.md

File metadata and controls

265 lines (169 loc) · 14.5 KB

Changelog

0.3.2 (2026-05-02)

Compare the full difference.

New

  • Add warning support to ReportCollector and renderer. a0ebb0c

  • Add id-token: write permission to publish-docs workflow. 65619e1

Other

Updates

0.3.1 (2026-03-21)

Compare the full difference.

Fixes

  • Fix Lower Python version requirement to 3.9 in pyproject.toml. dc9e97b

0.3.0 (2026-03-21)

Compare the full difference.

Other

  • Expand Python version matrix to include 3.9 and 3.10, update project status to Beta, and adjust supported Python classifiers. 4214619

0.2.0 (2026-03-21)

Compare the full difference.

Fixes

  • Fix linting errors. 711d308

  • Fix 8 code quality and latent bug issues identified in code roast. b85aa81

    • Strip ANSI from skip_reason in _extract_skip_reason (bug fix)
    • Deduplicate pytest_sessionfinish: shared setup, branch only on destination
    • Pass verbose as int instead of bool to render_report
    • Remove empty pytest_runtest_logreport module-level stub
    • Use sys.executable instead of hardcoded "python" in benchmark
    • Update benchmark model to claude-haiku-4-5-20251001
    • Replace manual counts loop with collections.Counter
    • Remove defensive len-check on skip longrepr tuple (trust the contract)

    co-authored-by: Claude Sonnet 4.6 noreply@anthropic.com

New

  • Add token benchmarking script and integrate Anthropic API. 04bd2b1

    • Introduce benchmark_tokens.py for comparing token counts between raw pytest output and LLM reports.
    • Use Anthropic API for token counting to validate a core success metric: ≥20% token reduction.
    • Add unit tests for benchmarking functionality, API integration, and output comparison.
    • Update uv.lock to include anthropic dependency.
    • Generate and restructure documentation using gen_doc_stubs.py.
    • Refactor existing files for improved organization: move docs structure to docs/reference/api.
    • Add zensical.toml for streamlined configuration of Markdown rendering and documentation features.

  • Add CLAUDE.md for development guidelines and lockfile for dependency management. 0c6725b

Other

  • Bump the github-actions group with 9 updates. 6b54c69

    Bumps the github-actions group with 9 updates:

    Package From To
    actions/checkout 4 6
    actions/download-artifact 4 8
    actions/setup-python 5 6
    astral-sh/setup-uv 5 7
    github/codeql-action 3 4
    docker/login-action 3 4
    docker/metadata-action 5 6
    docker/build-push-action 6 7
    actions/attest-build-provenance 2 4

    Updates actions/checkout from 4 to 6

    Updates actions/download-artifact from 4 to 8

    Updates actions/setup-python from 5 to 6

    Updates astral-sh/setup-uv from 5 to 7

    Updates github/codeql-action from 3 to 4

    Updates docker/login-action from 3 to 4

    Updates docker/metadata-action from 5 to 6

    Updates docker/build-push-action from 6 to 7

    Updates actions/attest-build-provenance from 2 to 4

    updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions

    signed-off-by: dependabot[bot] support@github.com

  • Handle missing outcome in renderer by falling back to uppercase string of the outcome. 210a2aa

  • Migrate pytest_llm_report to pytest_agent_digest, renaming all modules, classes, and references accordingly. Remove unfinished doc stubs and update package metadata (__init__.py) for new plugin. e2f95c8

  • Implement Ticket 7: add integration tests and skipped test handling. a776968

    • Add four new integration tests to validate --llm-report behavior under various scenarios: mixed outcomes order, empty sessions, tb=no, and combined term/file outputs.
    • Update collector.py to handle skipped tests during the setup phase by capturing their results for accurate Markdown reporting.

  • Implement Ticket 6: file output mode (--llm-report=file). 538ea3f

    • Save test results to a Markdown file (default path: test-results.md) and confirm output in stdout.
    • Support custom output paths via --llm-report-file and llm_report_file ini option.
    • New behavior overwrites files on subsequent runs instead of appending.
    • File output mode does not suppress pytest default output.
    • Add integration tests for all acceptance criteria.

  • Implement Ticket 5: terminal output mode (--llm-report=term). 3d914df

    • Unregister pytest's built-in terminalreporter when term mode is active (uses trylast=True on pytest_configure so the reporter is already registered before we remove it)
    • Render and print the Markdown report to stdout in pytest_sessionfinish
    • Add integration tests: Markdown in stdout, reporter suppressed, verbose passes section, no ANSI codes, both modes together

    co-authored-by: Claude Sonnet 4.6 noreply@anthropic.com

  • Implement Ticket 4: Markdown renderer. 5f22ad2

    Add pytest_llm_report/renderer.py with render_report() that converts a populated ReportCollector into a Markdown string. The output includes a summary line, a ## Failures section (failed + xpassed), a ## Skipped section, and an optional ## Passes section (verbose mode only). Traceback code blocks are suppressed when tb_style="no" or longrepr is absent.

    34 new unit tests in tests/test_renderer.py cover all acceptance criteria: no tests, all-passed (non-verbose + verbose), mixed outcomes, --tb=no, ANSI round-trip, no trailing whitespace, and single blank-line separation between sections.

    co-authored-by: Claude Sonnet 4.6 noreply@anthropic.com

  • Implement Ticket 3: test result collector. faf2e08

    • Add collector.py with TestResult dataclass, ReportCollector class, and strip_ansi helper covering all five outcome classifications
    • Wire ReportCollector into plugin via LLMReportPlugin class so pytest_runtest_logreport has access to session-scoped state
    • 29 new unit tests cover all outcome paths, ANSI stripping, counts property, and has_failures property

    co-authored-by: Claude Sonnet 4.6 noreply@anthropic.com

  • Set up basic pytest plugin structure with hook implementations and tests. 98e58f3

Updates

  • Delete .github/workflows/release-container.yaml. 7a682a8

  • Remove unused container release workflow, update Anthropic and Coverage dependencies, and refine documentation build and deployment steps. 126b0f7

  • Refactor _extract_longrepr and _extract_skip_reason to deduplicate logic for handling longrepr, moving shared functionality to _longrepr_reason. Remove unused pytest_sessionfinish export. 6abf199

  • Refactor render_report to avoid reusing generic variable names for section lines (e.g., lines), improving clarity and maintainability. 2af8595

  • Refactor pytest_sessionfinish to integrate directly into AgentDigestPlugin, removing redundant module-level hook implementation. Update related unit tests accordingly. d258540

  • Update charset-normalizer to version 3.4.6 in dependency lockfile. c52384e

0.1.0 (2026-03-14)

Other
⚡