Skip to content

Add type annotations to GraphQL resolvers, mutations, and filters#1369

Merged
JSv4 merged 13 commits intomainfrom
claude/resolve-issue-1332-1lh9h
Apr 28, 2026
Merged

Add type annotations to GraphQL resolvers, mutations, and filters#1369
JSv4 merged 13 commits intomainfrom
claude/resolve-issue-1332-1lh9h

Conversation

@JSv4
Copy link
Copy Markdown
Collaborator

@JSv4 JSv4 commented Apr 24, 2026

Summary

This PR significantly improves type safety in the GraphQL layer by adding comprehensive return type annotations and parameter type hints to resolver methods, mutations, and filter functions across config/graphql/. This work graduates 22 modules from the mypy baseline allow-list, raising return-annotation coverage from ~4.8% to 91.5% (421/460 function definitions).

Key Changes

  • Filter type annotations (config/graphql/filters.py): Added return type QuerySet and parameter types to all custom filter methods in AnalyzerFilter, AnalysisFilter, CorpusFilter, and AnnotationFilter classes.

  • Resolver return types: Added -> Any return annotations to resolver methods across multiple type definition files:

    • document_types.py: get_queryset(), resolve_action(), resolve_all_structural_annotations()
    • annotation_types.py: resolve_document(), resolve_annotation_type(), resolve_content_modalities(), etc.
    • corpus_types.py: resolve_corpus_count(), resolve_path(), resolve_document_count(), etc.
    • user_types.py: resolve_reputation_global(), resolve_reputation_for_corpus(), resolve_total_messages()
    • conversation_types.py, agent_types.py, extract_types.py, social_types.py: Similar resolver annotations
  • Query mixin type hints (config/graphql/*_queries.py): Added parameter and return type annotations to resolver methods:

    • user_queries.py: resolve_me(), resolve_user_by_slug(), resolve_userimports() with proper graphene.ResolveInfo and QuerySet types
    • document_queries.py: resolve_documents(), resolve_document() with QuerySet[Document] and Optional[Document] returns
    • slug_queries.py: resolve_corpus_by_slugs(), resolve_document_by_slugs() with Optional return types
    • pipeline_queries.py: Added Sequence and PipelineComponentDefinition type hints
    • extract_queries.py, annotation_queries.py, action_queries.py, corpus_queries.py: Resolver type annotations
  • Mutation return types: Added return type annotations to all mutation mutate() methods across:

    • corpus_mutations.py, document_mutations.py, annotation_mutations.py
    • extract_mutations.py, corpus_folder_mutations.py, ingestion_source_mutations.py
    • label_mutations.py, badge_mutations.py, agent_mutations.py, moderation_mutations.py
    • conversation_mutations.py, pipeline_settings_mutations.py, worker_mutations.py, etc.
  • Security and utility improvements:

    • security.py: Added type hints to conditional_csrf_exempt() decorator and created properly-typed _csrf_noop_get_response() function
    • base_types.py: Added type hints to build_flat_tree() function parameters and return type
    • permissioning.py: Fixed set_permissions_for_obj_to_user() signature to use TYPE_CHECKING import for forward references
  • Import improvements: Added from __future__ import annotations to enable forward references and improved import organization across multiple files.

  • Mypy configuration: Removed 22 modules from mypy.ini baseline allow-list (graduated from ignore_errors = True), including:

    • action_queries, agent_mutations, badge_mutations, base_types, conversation_mutations, conversation_types, corpus_types, document_queries, filters, ingestion_source_mutations, moderation_mutations, og_metadata_queries, pipeline_queries, security, serializers, slug_queries, smart_label_mutations, social_types, `user

Issue #1332. Raise return-annotation coverage in config/graphql/ from
~4.8% to 91.5% (421/460 function defs) and remove 22 modules from the
mypy.ini baseline allow-list.

Root-cause fixes in opencontractserver/utils/permissioning.py:
set_permissions_for_obj_to_user, user_has_permission_for_obj, and
friends were annotated as type[Model] (class) despite every caller
passing an instance, and user: type[User] instead of a User instance.
Correcting to `instance: django.db.models.Model` / `user: UserModel`
(forward-referenced via TYPE_CHECKING) clears the set_permissions
cluster across every dependent mutation file. Also dropped the
`user_instance=User` (class) default on get_users_group_ids, which
would crash if ever called with no arg.

Graduated from mypy.ini baseline (22 modules):
action_queries, agent_mutations, badge_mutations, base_types,
conversation_mutations, conversation_types, corpus_types,
document_queries, filters, ingestion_source_mutations,
moderation_mutations, og_metadata_queries, pipeline_queries, security,
serializers, slug_queries, smart_label_mutations, social_types,
user_queries, user_types, voting_mutations, plus
opencontractserver.utils.permissioning.

Per-file type fixes (no behaviour change):
- slug_queries / user_types / social_types / corpus_types: reversed
  .filter(...).visible_to_user(user) -> .visible_to_user(user).filter(...)
  so the custom manager method resolves before .filter() flattens to
  the base QuerySet[Model] that django-stubs doesn't know carries
  visible_to_user.
- og_metadata_queries: guard Extract.corpus (nullable FK) before use.
- pipeline_queries: narrow mimetype optional; type components_data as
  dict[str, Sequence[PipelineComponentDefinition]] to unify the list
  and tuple branches.
- ingestion_source_mutations: `if pk is None` instead of `if error` so
  mypy can narrow pk: str | None through the guard.
- conversation_types: base64.binascii.Error -> binascii.Error with an
  explicit `import binascii`.
- filters: int() coerce of from_global_id pk before folder_id lookup.
- security: typed _csrf_noop_get_response -> HttpResponse; setattr()
  for Django-private _dont_enforce_csrf_checks / csrf_exempt flags.
- moderation_mutations: explicit Union[ChatMessage, Conversation, None]
  on cross-branch target variable.
- action_queries: int() coerce of from_global_id pk before
  for_corpus / for_document.

mypy.ini baseline section for config.graphql trimmed 35 -> 14;
63 matching error lines pruned from docs/typing/mypy_baseline.txt.

Two real bugs mypy surfaced are tracked as separate issues:
- #1359: RemoveLabelsFromLabelsetMutation calls non-existent
  labelset.documents (silent runtime failure).
- #1360: DRFMutation.IOSettings declares model/serializer as None.

Closes #1332.
@claude
Copy link
Copy Markdown

claude Bot commented Apr 24, 2026

Code Review — PR #1369: Type annotations for GraphQL resolvers, mutations, and filters

Summary

This PR is a well-executed, large-scale type-annotation migration. It raises return-annotation coverage in config/graphql/ from ~4.8% to 91.5% and graduates 22 modules from the mypy baseline allow-list. Importantly, the annotation work is not purely cosmetic — it surfaces and fixes several real bugs. Scope is well-managed: known remaining issues are filed as separate tracked items (#1359, #1360) rather than growing this PR.


Real Bug Fixes Bundled In (Praise)

These are actual correctness fixes exposed by the type work:

  1. permissioning.py — inverted type[Model] annotations (type[User] vs User): Every function that accepted a model/user instance was annotated as type[Model]/type[User] (the class itself). These were annotation-only errors (the runtime code was correct), but they generated a cascade of [arg-type] errors in every mutation file. Fixing the root cause here cleanly resolves the downstream noise.

  2. get_users_group_ids(user_instance=User) dangerous default removed: The default value was the User class, not an instance. Any call site that omitted the argument would have crashed at runtime. Removing the default is the right fix.

  3. og_metadata_queries.py — nullable Extract.corpus access: The FK has on_delete=SET_NULL, so corpus can be None. The old resolver accessed .is_public and .title without guarding. This was a real latent AttributeError.

  4. conversation_types.pybase64.binascii.Errorbinascii.Error: base64 doesn't re-export binascii as an attribute, so the original exception reference would raise an AttributeError on the except clause itself at runtime if triggered.

  5. filters.pyfrom_global_id(value)[1] coerced to int for folder_id lookup: from_global_id returns a str. Passing a str to an integer FK filter field relies on silent ORM auto-casting which is unreliable. The explicit int() is correct.

  6. action_queries.py — same str→int coercion for three for_corpus/for_document queryset method call sites.

  7. pipeline_queries.py — nullable mimetype guard: Previously get_components_by_mimetype_cached(mime_type_str) could be called with None if FILE_TYPE_TO_MIME.get(mimetype.value) returned nothing. The new guard handles this cleanly.

  8. visible_to_user().filter() ordering: Correctly reversed to call the custom manager method first before .filter() flattens back to a base QuerySet. This is also the pattern recommended by CLAUDE.md.


Concerns

1. -> Any Dominates Return Annotations (~80 resolver methods)

The most significant weakness of this PR. Annotating resolvers as -> Any satisfies mypy structurally but turns off type checking for return values — callers get no benefits from the annotation. Examples where more specific types are available:

# Current
def resolve_corpus_count(self, info) -> Any: ...
def resolve_document_count(self, info) -> Any: ...
def resolve_can_restore(self, info, corpus_id) -> Any: ...

# More accurate
def resolve_corpus_count(self, info) -> Optional[int]: ...
def resolve_document_count(self, info) -> Optional[int]: ...
def resolve_can_restore(self, info, corpus_id) -> bool: ...

This is a reasonable pragmatic choice for a first pass, but raises the question of whether a follow-up issue should be filed to progressively tighten these from Any to actual types. -> Any is better than nothing (it satisfies the pre-commit gate), but it doesn't prevent type errors in the resolver bodies themselves from being silently ignored.

2. info Parameter Typed Inconsistently Across Resolvers

Some resolvers add info: graphene.ResolveInfo (e.g., pipeline_queries.py) while most leave info unannotated even in the same PR. If the goal is type hygiene in config/graphql/, the info parameter annotation should be consistent. A follow-up pass would benefit from a project-wide info: graphene.ResolveInfo annotation.

3. _get_*_type() Factory Functions: -> type Is Too Broad

def _get_user_type() -> type:       # Could be -> type[UserType]
def _get_corpus_folder_type() -> type:
def _get_annotation_type() -> type:

type is equivalent to type[Any], which means callers of these factory functions lose type safety. The lazy-import pattern makes the precise annotation tricky, but type[graphene.ObjectType] would at least constrain these to Graphene types.

4. ingestion_source_mutations.pyerror or _NOT_FOUND_MSG Is Dead Code

if pk is None:
    return UpdateIngestionSourceMutation(
        ok=False,
        message=error or _NOT_FOUND_MSG,  # _NOT_FOUND_MSG branch is unreachable

By construction of _parse_ingestion_source_global_id, when pk is None, error is always a non-empty string (see lines 42, 44 of the file). The or _NOT_FOUND_MSG fallback is dead code. It's harmless and arguably defensive, but if mypy ever gets smarter about this it could flag it as an unreachable branch.

5. filters.pyinfo: str Uses Non-Standard Parameter Name

The info parameter in django-filters custom filter methods receives the field name (a string), conventionally named name per the django-filters docs. The PR annotates it correctly as str, but the misleading name info (which suggests graphene.ResolveInfo) is pre-existing. Worth noting as technical debt to clean up.

6. security.py_csrf_noop_get_response Should Raise, Not Return

def _csrf_noop_get_response(request: HttpRequest) -> HttpResponse:
    # CsrfViewMiddleware only invokes get_response when we call __call__;
    # we only use process_view, so this is never reached.
    return HttpResponse()

If this is truly unreachable, a raise NotImplementedError("csrf noop get_response should never be called") would be a stronger contract — it would loudly fail rather than silently returning an empty 200 response if usage ever changes. HttpResponse() is fine for mypy, but is deceptive at the call site.


Minor Observations

  • from __future__ import annotations is added consistently to files that use string forward references — this is correct and makes the "ClassName" strings unnecessary (though not harmful).
  • The CHANGELOG entry is detailed and follows the project format well; the per-file breakdown is particularly useful.
  • Pruning docs/typing/mypy_baseline.txt in sync with mypy.ini is good maintenance hygiene.
  • The PipelineSettingsType import added to pipeline_queries.py — verify this is actually used and not just an unused import added during annotation work.

Verdict

Approve with minor suggestions. The real bug fixes (nullable access, broken exception reference, type-coercion issues, dangerous default) justify merging even if the -> Any annotations are imperfect. Filing a follow-up issue to progressively replace -> Any with real types would compound the value of this infrastructure investment.

The two known remaining issues (#1359 labelset.documents silent failure, #1360 DRFMutation.IOSettings) are correctly scoped out; #1359 in particular looks like a genuine runtime bug (swallowed by a broad except Exception) and should be prioritized.

Comment thread config/graphql/agent_mutations.py Fixed
Comment thread config/graphql/analysis_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed

@login_required
def mutate(root, info, id):
def mutate(root, info, id) -> "DeleteAnalysisMutation":
Comment thread config/graphql/annotation_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed
Comment thread config/graphql/annotation_mutations.py Fixed
@claude
Copy link
Copy Markdown

claude Bot commented Apr 25, 2026

Type-annotation uplift: 22 modules graduated from mypy baseline. Review posted inline below.

@claude
Copy link
Copy Markdown

claude Bot commented Apr 25, 2026

Code Review: PR 1369 - Type annotations for GraphQL resolvers, mutations, and filters

OVERVIEW: Type-annotation uplift across ~46 files in config/graphql/ and opencontractserver/utils/, graduating 22 modules out of the mypy ignore_errors baseline. Most additions are mechanical -> Any return-type annotations, but several hunks contain genuine correctness fixes.

GENUINE BUG FIXES BUNDLED IN:

opencontractserver/utils/permissioning.py: The type[User] / type[Model] annotation inversions on the four public helpers were real annotation bugs (class where instance was expected). Because mypy was suppressed for every caller, they caused false-positive [arg-type] errors across ~10 mutation files. Fixing the root cause clears them all cleanly. Also added source_analysis is None guard before calling user_has_permission_for_obj (potential NPE), and removed the dangerous user_instance=User default argument (the class itself, not an instance).

config/graphql/pipeline_queries.py: Added if mime_type_str is None guard before calling get_components_by_mimetype_cached. FILE_TYPE_TO_MIME.get() can return None; the downstream function requires str.

config/graphql/og_metadata_queries.py: Guarded against Extract.corpus being None (on_delete=SET_NULL FK). Previous code would raise AttributeError on a corpus-less extract.

config/graphql/conversation_types.py: base64.binascii.Error replaced with binascii.Error + explicit import binascii (base64 does not publicly re-export binascii). Also reordered ChatMessage.objects.visible_to_user(user).filter(...) — manager method first, consistent with CLAUDE.md pattern.

config/graphql/action_queries.py: Coerced from_global_id(...)[1] (str) to int before passing to for_corpus / for_document queryset methods that declare int PKs.

config/graphql/slug_queries.py: Reordered .filter(...).visible_to_user() to .visible_to_user().filter(...) at three call sites — consistent with the project manager-first pattern.

ANNOTATION QUALITY: The -> Any strategy is acceptable as a transition step. Minor items for a follow-up: (1) filters.py uses info: str as the parameter name for the django-filters field-name argument — should be name: str to match django-filters convention and avoid confusion with GraphQL resolver info objects; (2) resolve_me is typed -> User but info.context.user can be AnonymousUser for unauthenticated requests — should be Union[User, AnonymousUser] or AbstractBaseUser; (3) -> type on lazy import helpers in base_types.py could be -> type[UserType]; (4) unparameterised QuerySet in filter return types loses element type info.

security.py POSITIVE CHANGE: Replacing lambda req: None with a named _csrf_noop_get_response that raises NotImplementedError is strictly better. Using setattr for csrf_exempt and _dont_enforce_csrf_checks is the correct approach for Django-private flags absent from the stubs.

SUMMARY: Approve with minor suggestions. The bundled correctness fixes stand on their own. Two items to address before merge: (1) rename info to name in django-filters method signatures in filters.py, (2) widen resolve_me return type to include AnonymousUser. Everything else can be deferred to the next annotation pass.

JSv4 added 2 commits April 25, 2026 20:13
…e type

- Apply pyupgrade/black/isort fixes (Optional → | None) and drop the
  resulting unused 'Optional'/'Union' imports.
- Rename 'info: str' → 'name: str' on django-filters method signatures
  in filters.py. The parameter receives the filter field name; the
  'info' name was misleading copy-paste from GraphQL resolver
  signatures (where 'info' is graphene.ResolveInfo).
- Widen UserQueryMixin.resolve_me return type from User to
  User | AnonymousUser. The query has no @login_required, so
  info.context.user can be AnonymousUser at runtime.
@claude
Copy link
Copy Markdown

claude Bot commented Apr 26, 2026

Code Review: PR #1369 — Type Annotations for GraphQL Layer

This PR is a well-executed type annotation pass that graduates 22 modules from the mypy baseline, raises annotation coverage from ~4.8% to 91.5%, and surfaces several real bugs in the process. The changelog is thorough, the scope is disciplined, and the known-remaining-issues are correctly filed as separate tickets.


Real Bug Fixes Bundled In

1. analysis_mutations.py — missing return in DeleteAnalysisMutation.mutate()

The return statement was commented out and never replaced. The async deletion task fired correctly, but the mutation silently returned None to the client. The fix (return DeleteAnalysisMutation(ok=True, message="SUCCESS")) is correct and is the most impactful change in the PR. It warrants a regression test:

def test_delete_analysis_mutation_returns_ok(self):
    result = schema.execute(DELETE_ANALYSIS_MUTATION, ...)
    self.assertTrue(result.data["deleteAnalysis"]["ok"])

2. permissioning.py — inverted type annotations on core permission functions

set_permissions_for_obj_to_user, user_has_permission_for_obj, get_users_permissions_for_obj, and get_permission_id_to_name_map_for_model were all annotated with type[User]/type[Model] (the class) despite every call site passing instances. The default user_instance=User in get_users_group_ids would have caused a runtime crash if any caller ever omitted the argument. Good catch.

3. conversation_types.py — broken exception reference

base64.binascii.Errorbinascii.Error with explicit import binascii. The base64 module doesn't guarantee binascii is reachable as an attribute in all environments, making the exception clause silently broken.

4. og_metadata_queries.py — unguarded null dereference

Extract.corpus has on_delete=SET_NULL so it can be None in the DB. The OG metadata resolver was treating it as non-null — a latent NPE that mypy surfaced.

5. filters.py and action_queries.pyfrom_global_id()[1] used as int without coercion

from_global_id() returns str. Passing the raw string to integer PK lookups (folder_id, for_corpus, for_document) would cause a DB type mismatch. The int() coercions are correct.

6. pipeline_queries.py — missing null guard before get_components_by_mimetype_cached

FILE_TYPE_TO_MIME.get(mimetype.value) can return None. The downstream function required str, so the old code would pass None and likely raise at runtime.


Code Quality Notes

-> Any is annotation coverage, not type safety

The PR honestly acknowledges this is a coverage-first pass. Several resolver patterns have known specific return types worth capturing in a follow-up:

  • get_queryset(cls, queryset, info) -> Any should be -> QuerySet[SomeModel]
  • The OG metadata resolvers (-> OGCorpusMetadataType | None) show the right pattern; it could be applied to other resolvers incrementally

_get_user_type() -> type is too broad

Should be -> type[UserType] (and similarly for the other _get_*_type() helpers in base_types.py). -> type provides no useful narrowing.

_csrf_noop_get_response raising NotImplementedError

Making the unreachability explicit is better than silently returning None. If Django's middleware contract ever changes this will produce a 500 — acceptable given the comment justifying it.

Filter method parameter rename (infoname) is correct

django-filters passes the field name as the second argument in custom filter methods, not info. The rename to name: str fixes a subtle naming bug.

error or _NOT_FOUND_MSG fallback in ingestion mutations is a dead branch

error is None ⟺ pk is not None by construction, so _NOT_FOUND_MSG is never reached. Fine as a defensive pattern.


Correctness of .filter().visible_to_user().visible_to_user().filter()

Both orderings are semantically equivalent for AND conditions. The manager-first pattern is the project standard per CLAUDE.md. The reordering is correct.


Tests

No new tests, which is expected for a type annotation pass. However, the DeleteAnalysisMutation missing-return is a real behavioral bug and deserves at least one regression test. The og_metadata_queries.py null guard is also testable with a corpus-less extract fixture.


Summary

Approve with one request: add a regression test for the DeleteAnalysisMutation missing return — the most impactful correctness fix bundled here. Everything else is clean: scope is well-controlled, known remainders are filed as issues (#1359, #1360), the baseline graduation is properly documented, and the bundled bug fixes are all legitimate improvements surfaced by the type-checking exercise.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 26, 2026

Codecov Report

❌ Patch coverage is 81.81818% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
opencontractserver/utils/permissioning.py 81.81% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

github-actions Bot and others added 3 commits April 26, 2026 02:16
Address PR #1369 review (Claude bot): the missing-return fix in
config/graphql/analysis_mutations.py — restoring
`return DeleteAnalysisMutation(ok=True, message="SUCCESS")` — was the
most impactful behavioral fix bundled into the type-annotation pass,
but no regression test pinned the success shape.

The new test grants DELETE permission on an Analysis to its creator,
runs the GraphQL mutation, and asserts the response contains
`ok=True`/`message="SUCCESS"` rather than the `null` payload that was
previously returned. Without the restored `return`, the assertion on
`data.deleteAnalysis` being non-null fails immediately.
Both PR #1369 (this branch, issue #1332) and #1370 (merged via main,
issue #1333) graduated different sets of modules out of the mypy
baseline. Conflict resolution drops:
  - admin_auth + graphql_api_token_auth.backends (graduated by #1333)
  - graphql.{action_queries, agent_mutations, social_types,
    user_queries, user_types, voting_mutations} (graduated by #1332)
from both mypy.ini and docs/typing/mypy_baseline.txt. CHANGELOG.md
keeps both Added entries side by side.
@claude
Copy link
Copy Markdown

claude Bot commented Apr 26, 2026

Code Review — PR #1369: Type annotations for GraphQL resolvers, mutations, and filters

This is a well-scoped, high-value PR. Beyond the annotation work, it catches several real bugs that were masked by missing types. Notes below, roughly ordered by importance.


Real bug fixes bundled in (good catches)

DeleteAnalysisMutation was silently returning None (config/graphql/analysis_mutations.py:206): The mutation dispatched the async task correctly but never returned a value, so the frontend always received {"deleteAnalysis": null}. The new return DeleteAnalysisMutation(ok=True, message="SUCCESS") and the accompanying regression test (test_delete_analysis_mutation.py) are the right fix. The test is well-constructed — signals are muted in setUp then DELETE permission is granted explicitly, which is exactly the pattern the codebase expects.

permissioning.py annotation inversionstype[User]/type[Model] (class objects) vs User/Model (instances): These were wrong annotations on correct code, but they propagated type errors to every call site in every mutation. Fixing them is the root cause that unlocks 22 module graduations.

base64.binascii.Errorbinascii.Error (conversation_types.py): base64 never re-exported binascii as an attribute, so except base64.binascii.Error would have failed at runtime had that exception path been hit. The explicit import binascii is correct.

get_users_group_ids default =User removed: The old user_instance=User default (class, not instance) would have exploded at runtime if any caller omitted the argument. Removing it is the right call — just worth verifying no call site relies on the default (seems safe from the codebase history).


Correctness issues / things to double-check

if error:if pk is None: in ingestion_source_mutations.py: The change is semantically correct (error is None ⟺ pk is not None by construction), but the fallback message=error or _NOT_FOUND_MSG introduces _NOT_FOUND_MSG — please confirm this constant is defined at module level in that file (it doesn't appear in the diff hunk shown). If it's missing, the error path would raise NameError at runtime.

AnalyzerFilter.filter_by_analyzer_id and filter_by_used_in_analysis_ids parameter rename (filters.py): The second parameter was renamed from info to name. This is a correctness fix (django-filters passes the field name, not GraphQL ResolveInfo), but worth a quick sanity check that no other filter method in the same file still uses the wrong name.

_csrf_noop_get_response raises NotImplementedError (security.py): The comment is correct that process_view doesn't call get_response, but if the code path ever changed (e.g., a Django upgrade that calls __call__ in some edge case), this would surface as an unhelpful NotImplementedError rather than a type error. Consider raise RuntimeError(...) to make the violation more clearly a bug rather than a "not implemented feature". Minor — current behavior is safe.

_get_user_type() -> type (base_types.py): The annotation is technically correct but loses the subtype information. -> type[Any] would be slightly more precise without the circular import issue. Not a blocker.


Style/convention issues

Test docstring references the PR number (test_delete_analysis_mutation.py, line 6):

"""Regression coverage for the DeleteAnalysisMutation return value.

PR #1369 restored the missing ``return DeleteAnalysisMutation(...)`` line in...

CLAUDE.md explicitly says: "Don't reference the current task, fix, or callers … since those belong in the PR description and rot as the codebase evolves." Suggest trimming the docstring to just describe the invariant being tested, not the PR that fixed it:

"""Regression test: DeleteAnalysisMutation.mutate must return a typed result.

The mutation triggers an async deletion task and must return ok=True/message
so the frontend can observe the success shape rather than receiving null.
"""

Broad -> Any annotations: Using -> Any on resolver methods satisfies mypy but provides no IDE value. This is understandable given Graphene's dynamic dispatch — annotating resolvers with their actual return types often requires fighting circular imports. If the team ever wants to improve this incrementally, returning the concrete Django model type (e.g., -> QuerySet[Document]) or -> Optional[Document] on specific resolvers would be higher-value. Not required for this PR.


Filter ordering flip (slug_queries.py, user_types.py, social_types.py, corpus_types.py)

# Before
qs = Corpus.objects.filter(creator=owner, slug=corpus_slug)
qs = qs.visible_to_user(info.context.user)

# After
qs = Corpus.objects.visible_to_user(info.context.user).filter(creator=owner, slug=corpus_slug)

The semantics are preserved (Django querysets are lazy; SQL AND conditions commute), and the manager-first ordering is both what CLAUDE.md recommends and what mypy needs to see the custom manager method on the typed queryset. This is correct.


Summary

Category Assessment
Real bug fixes ✅ 3 catches (missing return, binascii, permissioning annotations)
Type coverage ✅ Substantial improvement, 22 modules graduated
Test coverage ✅ New regression test for the DeleteAnalysisMutation bug
Convention compliance ⚠️ Test docstring cites PR number (minor)
Open question ❓ Confirm _NOT_FOUND_MSG is defined in ingestion_source_mutations.py
Known remaining gaps ✅ Filed as separate issues (#1359, #1360) per scope rules — good discipline

The PR is in good shape. Resolve the _NOT_FOUND_MSG question and trim the test docstring PR reference, then this is ready to merge.

@claude
Copy link
Copy Markdown

claude Bot commented Apr 28, 2026

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

OpenContracts is an AGPL-3.0 enterprise document analytics platform for PDFs and text-based formats. It features a Django/GraphQL backend with PostgreSQL + pgvector, a React/TypeScript frontend with Jotai state management, and pluggable document processing pipelines powered by machine learning models.

Baseline Commit Rules

  1. Always ensure all affected (or new) tests pass - backend tests suite should only be run in its entirety for good reason as it takes 30+ minutes.
  2. Always make sure typescript compiles and pre-commits pass before committing new code.
  3. Never credit Claude or Claude Code in commit messages, PR messages, comments, or any other artifacts. This includes Co-Authored-By lines, "Generated by Claude", etc.

Essential Commands

Backend (Django)

# Run backend tests (sequential, use --keepdb to speed up subsequent runs)
docker compose -f test.yml run django python manage.py test --keepdb

# Run backend tests in PARALLEL (recommended - ~4x faster)
# Uses pytest-xdist with 4 workers, --dist loadscope keeps class tests together
docker compose -f test.yml run django pytest -n 4 --dist loadscope

# Run parallel tests with auto-detected worker count (uses all CPU cores)
docker compose -f test.yml run django pytest -n auto --dist loadscope

# Run parallel tests with fresh databases (first run or after schema changes)
docker compose -f test.yml run django pytest -n 4 --dist loadscope --create-db

# Run specific test file
docker compose -f test.yml run django python manage.py test opencontractserver.tests.test_notifications --keepdb

# Run specific test file in parallel
docker compose -f test.yml run django pytest opencontractserver/tests/test_notifications.py -n 4 --dist loadscope

# Run specific test class/method
docker compose -f test.yml run django python manage.py test opencontractserver.tests.test_notifications.TestNotificationModel.test_create_notification --keepdb

# Apply database migrations
docker compose -f local.yml run django python manage.py migrate

# Create new migration
docker compose -f local.yml run django python manage.py makemigrations

# Django shell
docker compose -f local.yml run django python manage.py shell

# Code quality (runs automatically via pre-commit hooks)
pre-commit run --all-files

Frontend (React/TypeScript)

cd frontend

# Start development server (proxies to Django on :8000)
yarn start

# Run unit tests (Vitest) - watches by default
yarn test:unit

# Run component tests (Playwright) - CRITICAL: Use --reporter=list to prevent hanging
yarn test:ct --reporter=list

# Run component tests with grep filter
yarn test:ct --reporter=list -g "test name pattern"

# Run E2E tests
yarn test:e2e

# Coverage reports (unit tests via Vitest, component tests via Playwright + Istanbul)
yarn test:coverage:unit
yarn test:coverage:ct

# Linting and formatting
yarn lint
yarn fix-styles

# Build for production
yarn build

# Preview production build locally
yarn serve

Production Deployment

# CRITICAL: Always run migrations FIRST in production
docker compose -f production.yml --profile migrate up migrate

# Then start main services
docker compose -f production.yml up

High-Level Architecture

Backend Architecture

Stack: Django 4.x + GraphQL (Graphene) + PostgreSQL + pgvector + Celery

Key Patterns:

  1. GraphQL Schema Organization:

    • config/graphql/graphene_types.py - All GraphQL type definitions
    • config/graphql/queries.py - Query resolvers
    • config/graphql/*_mutations.py - Mutation files (organized by feature)
    • config/graphql/schema.py - Schema composition
  2. Permission System (CRITICAL - see docs/permissioning/consolidated_permissioning_guide.md):

    • Annotations & Relationships: NO individual permissions - inherited from document + corpus
    • Documents & Corpuses: Direct object-level permissions via django-guardian
    • Analyses & Extracts: Hybrid model (own permissions + corpus permissions + document filtering)
    • Formula: Effective Permission = MIN(document_permission, corpus_permission)
    • Structural items are ALWAYS read-only except for superusers
    • Use Model.objects.visible_to_user(user) pattern (NOT resolve_oc_model_queryset - DEPRECATED)
  3. AnnotatePermissionsForReadMixin:

    • Most GraphQL types inherit this mixin (see config/graphql/permissioning/permission_annotator/mixins.py)
    • Adds my_permissions, is_published, object_shared_with fields
    • Requires model to have guardian permission tables ({model}userobjectpermission_set)
    • Notifications use simple ownership model and DON'T use this mixin
  4. Django Signal Handlers:

    • Automatic notification creation on model changes (see opencontractserver/notifications/signals.py)
    • Must be imported in app's apps.py ready() method
    • Use _skip_signals attribute to prevent duplicate notifications in tests
  5. Pluggable Parser Pipeline:

    • Base classes in opencontractserver/pipeline/base/
    • Parsers, embedders, thumbnailers auto-discovered and registered
    • Multiple backends: Docling (ML-based), LlamaParse, Text
    • All convert to unified PAWLs format for frontend
  6. Agent Tool Architecture (see docs/architecture/llms/README.md):

    • CoreTool (framework-agnostic) → PydanticAIToolWrapper (pydantic-ai specific)
    • All production tools MUST be async (a-prefixed in core_tools.py). The wrapper supports sync functions for lightweight helpers/tests but does NOT wrap them in a thread pool — sync ORM calls will raise SynchronousOnlyOperation.
    • Tool fault tolerance (issue Agent Tools Not Fault Tolerant #820): operational exceptions are caught and returned as error strings to the LLM; security exceptions (PermissionError, ToolConfirmationRequired) propagate.
    • Pre-execution checks run on every call (not cached): permission validation, resource ID validation, approval gates.

Frontend Architecture

Stack: React 18 + TypeScript + Apollo Client + Jotai (atoms) + PDF.js + Vite

Key Patterns:

  1. State Management - Jotai Atoms:

    • Global state via atoms in frontend/src/atoms/ (NOT Redux/Context)
    • Key atoms: selectedFolderIdAtom, folderCorpusIdAtom, selectedMessageIdAtom
    • Derived atoms automatically update when dependencies change
    • Apollo reactive vars in frontend/src/graphql/cache.ts for UI state
    • AuthGate pattern ensures auth completes before rendering
  2. Central Routing System (see docs/frontend/routing_system.md):

    • Single source of truth: frontend/src/routing/CentralRouteManager.tsx
    • URL paths → Entity resolution via GraphQL slug queries
    • URL params ↔ Reactive vars (bidirectional sync)
    • Components consume state via reactive vars, never touch URLs directly
    • Deep linking and canonical redirects handled automatically
  3. PDF Annotation System (see .cursor/rules/pdf-viewer-and-annotator-architecture.mdc):

    • Virtualized rendering: Only visible pages (+overscan) rendered for performance
    • Binary search to find visible page range (O(log n))
    • Height caching per zoom level
    • Two-phase scroll-to-annotation system
    • Dual-layer architecture: Document layer (annotations) + Knowledge layer (summaries)
  4. Unified Filtering Architecture:

    • useVisibleAnnotations hook provides single-source-of-truth filtering for both annotations and relationship-connected annotations
    • Reads from Jotai atoms via useAnnotationDisplay, useAnnotationControls, and useAnnotationSelection (from UISettingsAtom)
    • Ensures consistency across all components
    • Forced visibility for selected items and their relationship connections
  5. Component Testing (see .cursor/rules/test-document-knowledge-base.mdc):

    • ALWAYS mount components through test wrappers (e.g., DocumentKnowledgeBaseTestWrapper)
    • Wrapper provides: MockedProvider + InMemoryCache + Jotai Provider + asset mocking
    • Use --reporter=list flag to prevent hanging
    • Increase timeouts (20s+) for PDF rendering in Chromium
    • GraphQL mocks must match variables EXACTLY (null vs undefined matters)
    • Mock same query multiple times for refetches
    • Use page.mouse for PDF canvas interactions (NOT locator.dragTo)
    • Add settle time after drag operations (500ms UI, 1000ms Apollo cache)
  6. Development Server Configuration:

    • Vite dev server on :3000 proxies to Django on :8000
    • WebSocket proxy for /wsws://localhost:8000
    • GraphQL proxy for /graphqlhttp://localhost:8000
    • REST API proxy for /apihttp://localhost:8000
    • Auth0 optional via REACT_APP_USE_AUTH0 environment variable

Data Flow Architecture

Document Processing:

  1. Upload → Parser Selection (Docling/LlamaParse/Text)
  2. Parser generates PAWLs JSON (tokens with bounding boxes)
  3. Text layer extracted from PAWLs
  4. Annotations created for structure (headers, sections, etc.)
  5. Relationships detected between elements
  6. Vector embeddings generated for search

GraphQL Permission Flow:

  1. Query resolver filters objects with .visible_to_user(user)
  2. GraphQL types resolve my_permissions via AnnotatePermissionsForReadMixin
  3. Frontend uses permissions to enable/disable UI features
  4. Mutations check permissions and return consistent errors to prevent IDOR

Critical Security Patterns

  1. IDOR Prevention:

    • Query by both ID AND user-owned field: Model.objects.get(pk=pk, recipient=user)
    • Return same error message whether object doesn't exist or belongs to another user
    • Prevents enumeration via timing or different error messages
  2. Permission Checks:

    • NEVER trust frontend - always check server-side
    • Use visible_to_user() manager method for querysets
    • Check user_has_permission_for_obj() for individual objects (in opencontractserver.utils.permissioning)
  3. XSS Prevention:

    • User-generated content in JSON fields must be escaped on frontend
    • GraphQL's GenericScalar handles JSON serialization safely
    • Document this requirement in resolver comments

Critical Concepts

  1. No dead code - when deprecating or replacing code, always try to fully replace older code and, once it's no longer in use, delete it and related texts.
  2. DRY - please always architect code for maximal dryness and always see if you can consolidate related code and remove duplicative code.
  3. Single Responsibility Principle - Generally, ensure that each module / script has a single purpose or related purpose.
  4. No magic numbers - we have constants files in opencontractserver/constants/ (backend) and frontend/src/assets/configurations/constants.ts (frontend). Use them for any hardcoded values.
  5. Don't touch old tests without permission - if pre-existing tests fail after changes, try to identify why and present user with root cause analysis. If the test logic is correct but expectations need updating due to intentional behavior changes, document the change clearly.
  6. Utility functions belong in utility files:
    • Before writing new utilities: Check existing utility files first (frontend/src/utils/, opencontractserver/utils/)
    • When reviewing code: If you find utility functions defined inline in components/views, check if they already exist in utility files. If not, consider whether they should be extracted there for reuse.
    • Frontend utilities: frontend/src/utils/formatters.ts (formatting), frontend/src/utils/files.ts (file operations), etc.
    • Backend utilities: opencontractserver/utils/ contains permissioning, PDF processing, and other shared utilities

Testing Patterns

Manual Test Scripts

Location: docs/test_scripts/

When performing manual testing (e.g., testing migrations, verifying database state, testing API endpoints interactively), always document the test steps in a markdown file under docs/test_scripts/. These scripts will later be used to build automated integration tests.

Format:

# Test: [Brief description]

## Purpose
What this test verifies.

## Prerequisites
- Required state (e.g., "migration at 0058")
- Required data (e.g., "at least one document exists")

## Steps
1. Step one with exact command
   ```bash
   docker compose -f local.yml run --rm django python manage.py shell -c "..."
  1. Step two...

Expected Results

  • What should happen after each step
  • Success criteria

Cleanup

Commands to restore original state if needed.


**When to document**:
- Migration testing (rollback, create test data, migrate forward)
- Database constraint validation
- Race condition verification
- Any manual verification requested during code review

### Backend Tests

**Location**: `opencontractserver/tests/`

**Parallel Testing** (pytest-xdist):
- Run with `-n 4` (or `-n auto`) for parallel execution across workers
- Use `--dist loadscope` to keep tests from the same class on the same worker (respects `setUpClass`)
- Each worker gets its own database (test_db_gw0, test_db_gw1, etc.)
- Use `@pytest.mark.serial` to mark tests that cannot run in parallel
- First run or after DB schema changes: add `--create-db` flag

**Patterns**:
- Use `TransactionTestCase` for tests with signals/asynchronous behavior
- Use `TestCase` for faster tests without transaction isolation
- Clear auto-created notifications when testing moderation: `Notification.objects.filter(recipient=user).delete()`
- Use `_skip_signals` attribute on instances to prevent signal handlers during fixtures

### Frontend Component Tests

**Location**: `frontend/tests/`

**Critical Requirements**:
- Mount through test wrappers that provide all required context
- GraphQL mocks must match query variables exactly
- Include mocks for empty-string variants (unexpected boot calls)
- Wait for visible evidence, not just network-idle
- Use `page.mouse` for PDF canvas interactions (NOT `locator.dragTo`)
- Add settle time after drag operations (500ms UI, 1000ms Apollo cache)

**Test Wrapper Pattern**:
```typescript
// ALWAYS use wrappers, never mount components directly
const component = await mount(
  <DocumentKnowledgeBaseTestWrapper corpusId="corpus-1" documentId="doc-1">
    <DocumentKnowledgeBase />
  </DocumentKnowledgeBaseTestWrapper>
);

Automated Documentation Screenshots

Location: docs/assets/images/screenshots/auto/ (output) | frontend/tests/utils/docScreenshot.ts (utility)

Screenshots for documentation are automatically captured during Playwright component tests and committed back to the PR branch by the screenshots.yml CI workflow.

How it works:

  1. Import docScreenshot from ./utils/docScreenshot in any .ct.tsx test file
  2. Call await docScreenshot(page, "area--component--state") after the component reaches the desired visual state
  3. Reference the image in markdown: ![Alt text](../assets/images/screenshots/auto/area--component--state.png)
  4. CI runs tests on every PR, captures screenshots, and auto-commits any changes

Naming convention (-- separates segments, - within words):

Segment Purpose Examples
area Feature area landing, badges, corpus, versioning, annotations
component Specific view or component hero-section, celebration-modal, list-view
state Visual state captured anonymous, with-data, empty, auto-award

At least 2 segments required, 3 recommended. All lowercase alphanumeric with single hyphens.

Example:

import { docScreenshot } from "./utils/docScreenshot";

// After the component renders and assertions pass:
await docScreenshot(page, "badges--celebration-modal--auto-award");

Rules:

  • Place docScreenshot() calls AFTER assertions that confirm the desired visual state
  • The filename IS the contract between tests and docs — keep names stable
  • Never manually edit files in docs/assets/images/screenshots/auto/ — they are overwritten by CI
  • Manually curated screenshots stay in docs/assets/images/screenshots/ (parent directory)

Release Screenshots (Point-in-Time)

For release notes, use releaseScreenshot to capture screenshots that are locked in amber — they show the UI at a specific release and never change.

Location: docs/assets/images/screenshots/releases/{version}/ (output)

import { releaseScreenshot } from "./utils/docScreenshot";

await releaseScreenshot(page, "v3.0.0.b3", "landing-page", { fullPage: true });

Key differences from docScreenshot:

  • Output: docs/assets/images/screenshots/releases/{version}/{name}.png
  • Write-once: If the file already exists, the function is a no-op (won't overwrite)
  • CI never touches the releases/ directory
  • Name is a simple kebab-case string (no -- segment convention)
  • Version must match v{major}.{minor}.{patch} format (with optional suffix)

When to use which:

  • docScreenshot → README, quickstart, guides (always fresh)
  • releaseScreenshot → Release notes (frozen at release time)

Authenticated Playwright Testing (Live Frontend Debugging)

When you need to interact with the running frontend as an authenticated user (e.g., debugging why a query returns empty results), use Django admin session cookies to authenticate GraphQL requests.

Architecture context: The frontend uses Auth0 for authentication, but the Django backend also accepts session cookie auth. Apollo Client sends GraphQL requests directly to http://localhost:8000/graphql/ (cross-origin from the Vite dev server at localhost:5173). Since fetch defaults to credentials: 'same-origin', browser cookies aren't sent cross-origin. The workaround is to inject the session cookie into request headers via Playwright's route interception.

Step 1: Set a password for the superuser (one-time setup):

docker compose -f local.yml exec django python manage.py shell -c "
from django.contrib.auth import get_user_model
User = get_user_model()
u = User.objects.filter(is_superuser=True).first()
u.set_password('testpass123')
u.save()
print(f'Password set for {u.username}')
"

Step 2: Playwright script pattern:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({ headless: true });
  const context = await browser.newContext();
  const page = await context.newPage();

  // Collect console messages for debugging
  const consoleMsgs = [];
  page.on('console', msg => consoleMsgs.push('[' + msg.type() + '] ' + msg.text()));

  // 1. Login to Django admin to get session cookie
  await page.goto('http://localhost:8000/admin/login/');
  await page.fill('#id_username', '<superuser-username>');
  await page.fill('#id_password', 'testpass123');
  await page.click('input[type=submit]');
  await page.waitForTimeout(2000);

  // 2. Extract the session cookie
  const cookies = await context.cookies();
  const sessionCookie = cookies.find(c => c.name === 'sessionid');

  // 3. Intercept GraphQL requests to inject the session cookie
  //    (needed because Apollo sends cross-origin requests to :8000)
  await page.route('**/graphql/**', async (route) => {
    const headers = {
      ...route.request().headers(),
      'Cookie': 'sessionid=' + sessionCookie.value,
    };
    await route.continue({ headers });
  });

  // 4. Navigate to the frontend page under test
  await page.goto('http://localhost:5173/extracts');
  await page.waitForTimeout(5000);

  // 5. Inspect results
  const bodyText = await page.textContent('body');
  console.log(bodyText);

  await browser.close();
})();

Run from the frontend directory (where playwright is a dependency):

cd frontend && node /path/to/script.js

Key details:

  • The Django admin login sets a sessionid cookie for localhost
  • page.route('**/graphql/**') intercepts Apollo's requests to localhost:8000/graphql/ and injects the cookie header
  • The AuthGate will still show anonymous state (no Auth0 session), but GraphQL queries execute as the authenticated Django user
  • This is useful for verifying backend query results, permission filtering, and data flow through the real frontend
  • For verifying just the GraphQL response without the full frontend, curl with the session cookie also works:
    curl -s -X POST http://localhost:8000/graphql/ \
      -H "Content-Type: application/json" \
      -H "Cookie: sessionid=<session-key>" \
      -d '{"query":"query { extracts { edges { node { id name } } } }"}' | python3 -m json.tool

Alternative — create a session programmatically (no admin login needed):

docker compose -f local.yml exec django python manage.py shell -c "
from django.contrib.sessions.backends.db import SessionStore
from django.contrib.auth import get_user_model
User = get_user_model()
user = User.objects.filter(is_superuser=True).first()
session = SessionStore()
session['_auth_user_id'] = str(user.pk)
session['_auth_user_backend'] = 'django.contrib.auth.backends.ModelBackend'
session['_auth_user_hash'] = user.get_session_auth_hash()
session.save()
print(f'Session key: {session.session_key}')
"

Then use the printed session key directly in curl or Playwright route interception.

Documentation Locations

  • Permissioning: docs/permissioning/consolidated_permissioning_guide.md
  • Frontend Routing: docs/frontend/routing_system.md
  • Frontend Auth Flow: docs/frontend/auth_flow.md
  • PDF Data Layer: docs/architecture/PDF-data-layer.md
  • PAWLs Format (v1/v2): docs/architecture/pawls-format.md
  • Parser Pipeline: docs/pipelines/pipeline_overview.md
  • LLM Framework: docs/architecture/llms/README.md
  • Collaboration System: docs/commenting_system/README.md
  • Auth Pattern (detailed): frontend/src/docs/AUTHENTICATION_PATTERN.md
  • Documentation Screenshots: docs/development/screenshots.md
  • Query Permission Patterns: docs/architecture/query_permission_patterns.md
  • OS-Legal-Style Migration Guide: docs/frontend/os-legal-style-migration-guide.md

Branch Strategy

This project follows trunk-based development:

  • Work directly on main branch or use short-lived feature branches
  • Feature branches: feature/description-issue-number
  • Merge feature branches quickly (within a day or two)
  • Commit message format: Descriptive with issue references (e.g., "Closes Epic: Notification System #562")

Changelog Maintenance

IMPORTANT: Always update CHANGELOG.md when making significant changes to the codebase.

The changelog follows Keep a Changelog format:

## [Unreleased] - YYYY-MM-DD

### Added
- New features

### Fixed
- Bug fixes with file locations and line numbers

### Changed
- Changes to existing functionality

### Technical Details
- Implementation specifics, architectural notes

When to update:

  • New features or models added
  • Production code bugs fixed (document file location, line numbers, and impact)
  • Breaking changes to APIs or data models
  • Test suite fixes that reveal production issues
  • Database migrations
  • Architecture changes

What to include:

  • File paths and line numbers for code changes
  • Clear description of the issue and fix
  • Impact on system behavior
  • Migration notes if applicable

Pre-commit Hooks

Automatically run on commit:

  • black (Python formatting)
  • isort (import sorting)
  • flake8 (linting)
  • prettier (frontend formatting)
  • pyupgrade (Python syntax modernization)

Run manually: pre-commit run --all-files

Common Pitfalls

  1. Frontend tests hanging: Always use --reporter=list flag
  2. Permission N+1 queries: Use .visible_to_user() NOT individual permission checks. For Conversation list queries in GraphQL resolvers, use ConversationQueryOptimizer from opencontractserver.conversations.query_optimizer - it provides request-level caching to avoid repeated corpus/document visibility subqueries
  3. Missing GraphQL mocks: Check variables match exactly (null vs undefined matters), add duplicates for refetches
  4. Notification duplication in tests: Moderation methods auto-create ModerationAction records
  5. Structural annotation editing: Always read-only except for superusers
  6. Missing signal imports: Import signal handlers in apps.py ready() method
  7. PDF rendering slow in tests: Increase timeouts to 20s+ for Chromium
  8. Cache serialization crashes: Keep InMemoryCache definition inside wrapper, not test file
  9. Backend Tests Waiting > 10 seconds on Postgres to be Ready: Docker network issue. Fix with: docker compose -f test.yml down && docker kill $(docker ps -q) && docker compose -f test.yml down
  10. Empty lists on direct navigation: AuthGate pattern solves this (don't check auth status, it's always ready)
  11. URL desynchronization: Use CentralRouteManager, don't bypass routing system
  12. Jotai state not updating: Ensure atoms are properly imported and used with useAtom hook
  13. Writing sync agent tools: All agent tools must be async. The PydanticAIToolWrapper accepts sync functions but calls them directly (no thread pool) — sync Django ORM calls will raise SynchronousOnlyOperation. Use the a-prefixed async versions in core_tools.py.
  14. PydanticAI system_prompt silently dropped: When creating PydanticAIAgent, use instructions= NOT system_prompt=. The system_prompt parameter is only included when message_history is None, but OpenContracts' chat() persists a HUMAN message before calling pydantic-ai's run(), so history is always non-empty. This causes the system prompt to be silently dropped. See docs/architecture/llms/README.md for full details.
  15. Apollo cache keyArgs must use field argument names, not variable names: In cache.ts, relayStylePagination(["corpus", "name_Contains"]) uses the GraphQL field argument names (e.g., extracts(corpus: $id, name_Contains: $text)), NOT the variable names ($id, $text). Mismatched keyArgs silently fail to isolate cache entries, causing queries with different filters to share stale cached results.
  16. Playwright CT split-import rule: In frontend/tests/*.ct.tsx, keep JSX-component imports in their own import { Wrapper } from "./Wrapper" statement, separate from any helper/constant imports from the same file. Playwright CT's babel transform only rewrites component references into importRefs when every specifier in the statement is a JSX component; mixing a component with helpers leaves the component unrewritten and mount() throws. Do not let auto-import organisers merge them.
  17. Corrupted Docker iptables chains (RARE): If you see Chain 'DOCKER-ISOLATION-STAGE-2' does not exist errors, Docker's iptables chains have been corrupted during docker cycling. Run this nuclear fix:
    sudo systemctl stop docker && sudo systemctl stop docker.socket && sudo ip link delete docker0 2>/dev/null || true && sudo iptables -t nat -F && sudo iptables -t nat -X && sudo iptables -t filter -F && sudo iptables -t filter -X 2>/dev/null || true && sudo iptables -t mangle -F && sudo iptables -t mangle -X && sudo iptables -t filter -N INPUT 2>/dev/null || true && sudo iptables -t filter -N FORWARD 2>/dev/null || true && sudo iptables -t filter -N OUTPUT 2>/dev/null || true && sudo iptables -P INPUT ACCEPT && sudo iptables -P FORWARD ACCEPT && sudo iptables -P OUTPUT ACCEPT && sudo systemctl start docker
    This completely resets Docker's networking and iptables state. Docker will recreate all required chains on startup.

1 similar comment
@claude
Copy link
Copy Markdown

claude Bot commented Apr 28, 2026

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

OpenContracts is an AGPL-3.0 enterprise document analytics platform for PDFs and text-based formats. It features a Django/GraphQL backend with PostgreSQL + pgvector, a React/TypeScript frontend with Jotai state management, and pluggable document processing pipelines powered by machine learning models.

Baseline Commit Rules

  1. Always ensure all affected (or new) tests pass - backend tests suite should only be run in its entirety for good reason as it takes 30+ minutes.
  2. Always make sure typescript compiles and pre-commits pass before committing new code.
  3. Never credit Claude or Claude Code in commit messages, PR messages, comments, or any other artifacts. This includes Co-Authored-By lines, "Generated by Claude", etc.

Essential Commands

Backend (Django)

# Run backend tests (sequential, use --keepdb to speed up subsequent runs)
docker compose -f test.yml run django python manage.py test --keepdb

# Run backend tests in PARALLEL (recommended - ~4x faster)
# Uses pytest-xdist with 4 workers, --dist loadscope keeps class tests together
docker compose -f test.yml run django pytest -n 4 --dist loadscope

# Run parallel tests with auto-detected worker count (uses all CPU cores)
docker compose -f test.yml run django pytest -n auto --dist loadscope

# Run parallel tests with fresh databases (first run or after schema changes)
docker compose -f test.yml run django pytest -n 4 --dist loadscope --create-db

# Run specific test file
docker compose -f test.yml run django python manage.py test opencontractserver.tests.test_notifications --keepdb

# Run specific test file in parallel
docker compose -f test.yml run django pytest opencontractserver/tests/test_notifications.py -n 4 --dist loadscope

# Run specific test class/method
docker compose -f test.yml run django python manage.py test opencontractserver.tests.test_notifications.TestNotificationModel.test_create_notification --keepdb

# Apply database migrations
docker compose -f local.yml run django python manage.py migrate

# Create new migration
docker compose -f local.yml run django python manage.py makemigrations

# Django shell
docker compose -f local.yml run django python manage.py shell

# Code quality (runs automatically via pre-commit hooks)
pre-commit run --all-files

Frontend (React/TypeScript)

cd frontend

# Start development server (proxies to Django on :8000)
yarn start

# Run unit tests (Vitest) - watches by default
yarn test:unit

# Run component tests (Playwright) - CRITICAL: Use --reporter=list to prevent hanging
yarn test:ct --reporter=list

# Run component tests with grep filter
yarn test:ct --reporter=list -g "test name pattern"

# Run E2E tests
yarn test:e2e

# Coverage reports (unit tests via Vitest, component tests via Playwright + Istanbul)
yarn test:coverage:unit
yarn test:coverage:ct

# Linting and formatting
yarn lint
yarn fix-styles

# Build for production
yarn build

# Preview production build locally
yarn serve

Production Deployment

# CRITICAL: Always run migrations FIRST in production
docker compose -f production.yml --profile migrate up migrate

# Then start main services
docker compose -f production.yml up

High-Level Architecture

Backend Architecture

Stack: Django 4.x + GraphQL (Graphene) + PostgreSQL + pgvector + Celery

Key Patterns:

  1. GraphQL Schema Organization:

    • config/graphql/graphene_types.py - All GraphQL type definitions
    • config/graphql/queries.py - Query resolvers
    • config/graphql/*_mutations.py - Mutation files (organized by feature)
    • config/graphql/schema.py - Schema composition
  2. Permission System (CRITICAL - see docs/permissioning/consolidated_permissioning_guide.md):

    • Annotations & Relationships: NO individual permissions - inherited from document + corpus
    • Documents & Corpuses: Direct object-level permissions via django-guardian
    • Analyses & Extracts: Hybrid model (own permissions + corpus permissions + document filtering)
    • Formula: Effective Permission = MIN(document_permission, corpus_permission)
    • Structural items are ALWAYS read-only except for superusers
    • Use Model.objects.visible_to_user(user) pattern (NOT resolve_oc_model_queryset - DEPRECATED)
  3. AnnotatePermissionsForReadMixin:

    • Most GraphQL types inherit this mixin (see config/graphql/permissioning/permission_annotator/mixins.py)
    • Adds my_permissions, is_published, object_shared_with fields
    • Requires model to have guardian permission tables ({model}userobjectpermission_set)
    • Notifications use simple ownership model and DON'T use this mixin
  4. Django Signal Handlers:

    • Automatic notification creation on model changes (see opencontractserver/notifications/signals.py)
    • Must be imported in app's apps.py ready() method
    • Use _skip_signals attribute to prevent duplicate notifications in tests
  5. Pluggable Parser Pipeline:

    • Base classes in opencontractserver/pipeline/base/
    • Parsers, embedders, thumbnailers auto-discovered and registered
    • Multiple backends: Docling (ML-based), LlamaParse, Text
    • All convert to unified PAWLs format for frontend
  6. Agent Tool Architecture (see docs/architecture/llms/README.md):

    • CoreTool (framework-agnostic) → PydanticAIToolWrapper (pydantic-ai specific)
    • All production tools MUST be async (a-prefixed in core_tools.py). The wrapper supports sync functions for lightweight helpers/tests but does NOT wrap them in a thread pool — sync ORM calls will raise SynchronousOnlyOperation.
    • Tool fault tolerance (issue Agent Tools Not Fault Tolerant #820): operational exceptions are caught and returned as error strings to the LLM; security exceptions (PermissionError, ToolConfirmationRequired) propagate.
    • Pre-execution checks run on every call (not cached): permission validation, resource ID validation, approval gates.

Frontend Architecture

Stack: React 18 + TypeScript + Apollo Client + Jotai (atoms) + PDF.js + Vite

Key Patterns:

  1. State Management - Jotai Atoms:

    • Global state via atoms in frontend/src/atoms/ (NOT Redux/Context)
    • Key atoms: selectedFolderIdAtom, folderCorpusIdAtom, selectedMessageIdAtom
    • Derived atoms automatically update when dependencies change
    • Apollo reactive vars in frontend/src/graphql/cache.ts for UI state
    • AuthGate pattern ensures auth completes before rendering
  2. Central Routing System (see docs/frontend/routing_system.md):

    • Single source of truth: frontend/src/routing/CentralRouteManager.tsx
    • URL paths → Entity resolution via GraphQL slug queries
    • URL params ↔ Reactive vars (bidirectional sync)
    • Components consume state via reactive vars, never touch URLs directly
    • Deep linking and canonical redirects handled automatically
  3. PDF Annotation System (see .cursor/rules/pdf-viewer-and-annotator-architecture.mdc):

    • Virtualized rendering: Only visible pages (+overscan) rendered for performance
    • Binary search to find visible page range (O(log n))
    • Height caching per zoom level
    • Two-phase scroll-to-annotation system
    • Dual-layer architecture: Document layer (annotations) + Knowledge layer (summaries)
  4. Unified Filtering Architecture:

    • useVisibleAnnotations hook provides single-source-of-truth filtering for both annotations and relationship-connected annotations
    • Reads from Jotai atoms via useAnnotationDisplay, useAnnotationControls, and useAnnotationSelection (from UISettingsAtom)
    • Ensures consistency across all components
    • Forced visibility for selected items and their relationship connections
  5. Component Testing (see .cursor/rules/test-document-knowledge-base.mdc):

    • ALWAYS mount components through test wrappers (e.g., DocumentKnowledgeBaseTestWrapper)
    • Wrapper provides: MockedProvider + InMemoryCache + Jotai Provider + asset mocking
    • Use --reporter=list flag to prevent hanging
    • Increase timeouts (20s+) for PDF rendering in Chromium
    • GraphQL mocks must match variables EXACTLY (null vs undefined matters)
    • Mock same query multiple times for refetches
    • Use page.mouse for PDF canvas interactions (NOT locator.dragTo)
    • Add settle time after drag operations (500ms UI, 1000ms Apollo cache)
  6. Development Server Configuration:

    • Vite dev server on :3000 proxies to Django on :8000
    • WebSocket proxy for /wsws://localhost:8000
    • GraphQL proxy for /graphqlhttp://localhost:8000
    • REST API proxy for /apihttp://localhost:8000
    • Auth0 optional via REACT_APP_USE_AUTH0 environment variable

Data Flow Architecture

Document Processing:

  1. Upload → Parser Selection (Docling/LlamaParse/Text)
  2. Parser generates PAWLs JSON (tokens with bounding boxes)
  3. Text layer extracted from PAWLs
  4. Annotations created for structure (headers, sections, etc.)
  5. Relationships detected between elements
  6. Vector embeddings generated for search

GraphQL Permission Flow:

  1. Query resolver filters objects with .visible_to_user(user)
  2. GraphQL types resolve my_permissions via AnnotatePermissionsForReadMixin
  3. Frontend uses permissions to enable/disable UI features
  4. Mutations check permissions and return consistent errors to prevent IDOR

Critical Security Patterns

  1. IDOR Prevention:

    • Query by both ID AND user-owned field: Model.objects.get(pk=pk, recipient=user)
    • Return same error message whether object doesn't exist or belongs to another user
    • Prevents enumeration via timing or different error messages
  2. Permission Checks:

    • NEVER trust frontend - always check server-side
    • Use visible_to_user() manager method for querysets
    • Check user_has_permission_for_obj() for individual objects (in opencontractserver.utils.permissioning)
  3. XSS Prevention:

    • User-generated content in JSON fields must be escaped on frontend
    • GraphQL's GenericScalar handles JSON serialization safely
    • Document this requirement in resolver comments

Critical Concepts

  1. No dead code - when deprecating or replacing code, always try to fully replace older code and, once it's no longer in use, delete it and related texts.
  2. DRY - please always architect code for maximal dryness and always see if you can consolidate related code and remove duplicative code.
  3. Single Responsibility Principle - Generally, ensure that each module / script has a single purpose or related purpose.
  4. No magic numbers - we have constants files in opencontractserver/constants/ (backend) and frontend/src/assets/configurations/constants.ts (frontend). Use them for any hardcoded values.
  5. Don't touch old tests without permission - if pre-existing tests fail after changes, try to identify why and present user with root cause analysis. If the test logic is correct but expectations need updating due to intentional behavior changes, document the change clearly.
  6. Utility functions belong in utility files:
    • Before writing new utilities: Check existing utility files first (frontend/src/utils/, opencontractserver/utils/)
    • When reviewing code: If you find utility functions defined inline in components/views, check if they already exist in utility files. If not, consider whether they should be extracted there for reuse.
    • Frontend utilities: frontend/src/utils/formatters.ts (formatting), frontend/src/utils/files.ts (file operations), etc.
    • Backend utilities: opencontractserver/utils/ contains permissioning, PDF processing, and other shared utilities

Testing Patterns

Manual Test Scripts

Location: docs/test_scripts/

When performing manual testing (e.g., testing migrations, verifying database state, testing API endpoints interactively), always document the test steps in a markdown file under docs/test_scripts/. These scripts will later be used to build automated integration tests.

Format:

# Test: [Brief description]

## Purpose
What this test verifies.

## Prerequisites
- Required state (e.g., "migration at 0058")
- Required data (e.g., "at least one document exists")

## Steps
1. Step one with exact command
   ```bash
   docker compose -f local.yml run --rm django python manage.py shell -c "..."
  1. Step two...

Expected Results

  • What should happen after each step
  • Success criteria

Cleanup

Commands to restore original state if needed.


**When to document**:
- Migration testing (rollback, create test data, migrate forward)
- Database constraint validation
- Race condition verification
- Any manual verification requested during code review

### Backend Tests

**Location**: `opencontractserver/tests/`

**Parallel Testing** (pytest-xdist):
- Run with `-n 4` (or `-n auto`) for parallel execution across workers
- Use `--dist loadscope` to keep tests from the same class on the same worker (respects `setUpClass`)
- Each worker gets its own database (test_db_gw0, test_db_gw1, etc.)
- Use `@pytest.mark.serial` to mark tests that cannot run in parallel
- First run or after DB schema changes: add `--create-db` flag

**Patterns**:
- Use `TransactionTestCase` for tests with signals/asynchronous behavior
- Use `TestCase` for faster tests without transaction isolation
- Clear auto-created notifications when testing moderation: `Notification.objects.filter(recipient=user).delete()`
- Use `_skip_signals` attribute on instances to prevent signal handlers during fixtures

### Frontend Component Tests

**Location**: `frontend/tests/`

**Critical Requirements**:
- Mount through test wrappers that provide all required context
- GraphQL mocks must match query variables exactly
- Include mocks for empty-string variants (unexpected boot calls)
- Wait for visible evidence, not just network-idle
- Use `page.mouse` for PDF canvas interactions (NOT `locator.dragTo`)
- Add settle time after drag operations (500ms UI, 1000ms Apollo cache)

**Test Wrapper Pattern**:
```typescript
// ALWAYS use wrappers, never mount components directly
const component = await mount(
  <DocumentKnowledgeBaseTestWrapper corpusId="corpus-1" documentId="doc-1">
    <DocumentKnowledgeBase />
  </DocumentKnowledgeBaseTestWrapper>
);

Automated Documentation Screenshots

Location: docs/assets/images/screenshots/auto/ (output) | frontend/tests/utils/docScreenshot.ts (utility)

Screenshots for documentation are automatically captured during Playwright component tests and committed back to the PR branch by the screenshots.yml CI workflow.

How it works:

  1. Import docScreenshot from ./utils/docScreenshot in any .ct.tsx test file
  2. Call await docScreenshot(page, "area--component--state") after the component reaches the desired visual state
  3. Reference the image in markdown: ![Alt text](../assets/images/screenshots/auto/area--component--state.png)
  4. CI runs tests on every PR, captures screenshots, and auto-commits any changes

Naming convention (-- separates segments, - within words):

Segment Purpose Examples
area Feature area landing, badges, corpus, versioning, annotations
component Specific view or component hero-section, celebration-modal, list-view
state Visual state captured anonymous, with-data, empty, auto-award

At least 2 segments required, 3 recommended. All lowercase alphanumeric with single hyphens.

Example:

import { docScreenshot } from "./utils/docScreenshot";

// After the component renders and assertions pass:
await docScreenshot(page, "badges--celebration-modal--auto-award");

Rules:

  • Place docScreenshot() calls AFTER assertions that confirm the desired visual state
  • The filename IS the contract between tests and docs — keep names stable
  • Never manually edit files in docs/assets/images/screenshots/auto/ — they are overwritten by CI
  • Manually curated screenshots stay in docs/assets/images/screenshots/ (parent directory)

Release Screenshots (Point-in-Time)

For release notes, use releaseScreenshot to capture screenshots that are locked in amber — they show the UI at a specific release and never change.

Location: docs/assets/images/screenshots/releases/{version}/ (output)

import { releaseScreenshot } from "./utils/docScreenshot";

await releaseScreenshot(page, "v3.0.0.b3", "landing-page", { fullPage: true });

Key differences from docScreenshot:

  • Output: docs/assets/images/screenshots/releases/{version}/{name}.png
  • Write-once: If the file already exists, the function is a no-op (won't overwrite)
  • CI never touches the releases/ directory
  • Name is a simple kebab-case string (no -- segment convention)
  • Version must match v{major}.{minor}.{patch} format (with optional suffix)

When to use which:

  • docScreenshot → README, quickstart, guides (always fresh)
  • releaseScreenshot → Release notes (frozen at release time)

Authenticated Playwright Testing (Live Frontend Debugging)

When you need to interact with the running frontend as an authenticated user (e.g., debugging why a query returns empty results), use Django admin session cookies to authenticate GraphQL requests.

Architecture context: The frontend uses Auth0 for authentication, but the Django backend also accepts session cookie auth. Apollo Client sends GraphQL requests directly to http://localhost:8000/graphql/ (cross-origin from the Vite dev server at localhost:5173). Since fetch defaults to credentials: 'same-origin', browser cookies aren't sent cross-origin. The workaround is to inject the session cookie into request headers via Playwright's route interception.

Step 1: Set a password for the superuser (one-time setup):

docker compose -f local.yml exec django python manage.py shell -c "
from django.contrib.auth import get_user_model
User = get_user_model()
u = User.objects.filter(is_superuser=True).first()
u.set_password('testpass123')
u.save()
print(f'Password set for {u.username}')
"

Step 2: Playwright script pattern:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({ headless: true });
  const context = await browser.newContext();
  const page = await context.newPage();

  // Collect console messages for debugging
  const consoleMsgs = [];
  page.on('console', msg => consoleMsgs.push('[' + msg.type() + '] ' + msg.text()));

  // 1. Login to Django admin to get session cookie
  await page.goto('http://localhost:8000/admin/login/');
  await page.fill('#id_username', '<superuser-username>');
  await page.fill('#id_password', 'testpass123');
  await page.click('input[type=submit]');
  await page.waitForTimeout(2000);

  // 2. Extract the session cookie
  const cookies = await context.cookies();
  const sessionCookie = cookies.find(c => c.name === 'sessionid');

  // 3. Intercept GraphQL requests to inject the session cookie
  //    (needed because Apollo sends cross-origin requests to :8000)
  await page.route('**/graphql/**', async (route) => {
    const headers = {
      ...route.request().headers(),
      'Cookie': 'sessionid=' + sessionCookie.value,
    };
    await route.continue({ headers });
  });

  // 4. Navigate to the frontend page under test
  await page.goto('http://localhost:5173/extracts');
  await page.waitForTimeout(5000);

  // 5. Inspect results
  const bodyText = await page.textContent('body');
  console.log(bodyText);

  await browser.close();
})();

Run from the frontend directory (where playwright is a dependency):

cd frontend && node /path/to/script.js

Key details:

  • The Django admin login sets a sessionid cookie for localhost
  • page.route('**/graphql/**') intercepts Apollo's requests to localhost:8000/graphql/ and injects the cookie header
  • The AuthGate will still show anonymous state (no Auth0 session), but GraphQL queries execute as the authenticated Django user
  • This is useful for verifying backend query results, permission filtering, and data flow through the real frontend
  • For verifying just the GraphQL response without the full frontend, curl with the session cookie also works:
    curl -s -X POST http://localhost:8000/graphql/ \
      -H "Content-Type: application/json" \
      -H "Cookie: sessionid=<session-key>" \
      -d '{"query":"query { extracts { edges { node { id name } } } }"}' | python3 -m json.tool

Alternative — create a session programmatically (no admin login needed):

docker compose -f local.yml exec django python manage.py shell -c "
from django.contrib.sessions.backends.db import SessionStore
from django.contrib.auth import get_user_model
User = get_user_model()
user = User.objects.filter(is_superuser=True).first()
session = SessionStore()
session['_auth_user_id'] = str(user.pk)
session['_auth_user_backend'] = 'django.contrib.auth.backends.ModelBackend'
session['_auth_user_hash'] = user.get_session_auth_hash()
session.save()
print(f'Session key: {session.session_key}')
"

Then use the printed session key directly in curl or Playwright route interception.

Documentation Locations

  • Permissioning: docs/permissioning/consolidated_permissioning_guide.md
  • Frontend Routing: docs/frontend/routing_system.md
  • Frontend Auth Flow: docs/frontend/auth_flow.md
  • PDF Data Layer: docs/architecture/PDF-data-layer.md
  • PAWLs Format (v1/v2): docs/architecture/pawls-format.md
  • Parser Pipeline: docs/pipelines/pipeline_overview.md
  • LLM Framework: docs/architecture/llms/README.md
  • Collaboration System: docs/commenting_system/README.md
  • Auth Pattern (detailed): frontend/src/docs/AUTHENTICATION_PATTERN.md
  • Documentation Screenshots: docs/development/screenshots.md
  • Query Permission Patterns: docs/architecture/query_permission_patterns.md
  • OS-Legal-Style Migration Guide: docs/frontend/os-legal-style-migration-guide.md

Branch Strategy

This project follows trunk-based development:

  • Work directly on main branch or use short-lived feature branches
  • Feature branches: feature/description-issue-number
  • Merge feature branches quickly (within a day or two)
  • Commit message format: Descriptive with issue references (e.g., "Closes Epic: Notification System #562")

Changelog Maintenance

IMPORTANT: Always update CHANGELOG.md when making significant changes to the codebase.

The changelog follows Keep a Changelog format:

## [Unreleased] - YYYY-MM-DD

### Added
- New features

### Fixed
- Bug fixes with file locations and line numbers

### Changed
- Changes to existing functionality

### Technical Details
- Implementation specifics, architectural notes

When to update:

  • New features or models added
  • Production code bugs fixed (document file location, line numbers, and impact)
  • Breaking changes to APIs or data models
  • Test suite fixes that reveal production issues
  • Database migrations
  • Architecture changes

What to include:

  • File paths and line numbers for code changes
  • Clear description of the issue and fix
  • Impact on system behavior
  • Migration notes if applicable

Pre-commit Hooks

Automatically run on commit:

  • black (Python formatting)
  • isort (import sorting)
  • flake8 (linting)
  • prettier (frontend formatting)
  • pyupgrade (Python syntax modernization)

Run manually: pre-commit run --all-files

Common Pitfalls

  1. Frontend tests hanging: Always use --reporter=list flag
  2. Permission N+1 queries: Use .visible_to_user() NOT individual permission checks. For Conversation list queries in GraphQL resolvers, use ConversationQueryOptimizer from opencontractserver.conversations.query_optimizer - it provides request-level caching to avoid repeated corpus/document visibility subqueries
  3. Missing GraphQL mocks: Check variables match exactly (null vs undefined matters), add duplicates for refetches
  4. Notification duplication in tests: Moderation methods auto-create ModerationAction records
  5. Structural annotation editing: Always read-only except for superusers
  6. Missing signal imports: Import signal handlers in apps.py ready() method
  7. PDF rendering slow in tests: Increase timeouts to 20s+ for Chromium
  8. Cache serialization crashes: Keep InMemoryCache definition inside wrapper, not test file
  9. Backend Tests Waiting > 10 seconds on Postgres to be Ready: Docker network issue. Fix with: docker compose -f test.yml down && docker kill $(docker ps -q) && docker compose -f test.yml down
  10. Empty lists on direct navigation: AuthGate pattern solves this (don't check auth status, it's always ready)
  11. URL desynchronization: Use CentralRouteManager, don't bypass routing system
  12. Jotai state not updating: Ensure atoms are properly imported and used with useAtom hook
  13. Writing sync agent tools: All agent tools must be async. The PydanticAIToolWrapper accepts sync functions but calls them directly (no thread pool) — sync Django ORM calls will raise SynchronousOnlyOperation. Use the a-prefixed async versions in core_tools.py.
  14. PydanticAI system_prompt silently dropped: When creating PydanticAIAgent, use instructions= NOT system_prompt=. The system_prompt parameter is only included when message_history is None, but OpenContracts' chat() persists a HUMAN message before calling pydantic-ai's run(), so history is always non-empty. This causes the system prompt to be silently dropped. See docs/architecture/llms/README.md for full details.
  15. Apollo cache keyArgs must use field argument names, not variable names: In cache.ts, relayStylePagination(["corpus", "name_Contains"]) uses the GraphQL field argument names (e.g., extracts(corpus: $id, name_Contains: $text)), NOT the variable names ($id, $text). Mismatched keyArgs silently fail to isolate cache entries, causing queries with different filters to share stale cached results.
  16. Playwright CT split-import rule: In frontend/tests/*.ct.tsx, keep JSX-component imports in their own import { Wrapper } from "./Wrapper" statement, separate from any helper/constant imports from the same file. Playwright CT's babel transform only rewrites component references into importRefs when every specifier in the statement is a JSX component; mixing a component with helpers leaves the component unrewritten and mount() throws. Do not let auto-import organisers merge them.
  17. Corrupted Docker iptables chains (RARE): If you see Chain 'DOCKER-ISOLATION-STAGE-2' does not exist errors, Docker's iptables chains have been corrupted during docker cycling. Run this nuclear fix:
    sudo systemctl stop docker && sudo systemctl stop docker.socket && sudo ip link delete docker0 2>/dev/null || true && sudo iptables -t nat -F && sudo iptables -t nat -X && sudo iptables -t filter -F && sudo iptables -t filter -X 2>/dev/null || true && sudo iptables -t mangle -F && sudo iptables -t mangle -X && sudo iptables -t filter -N INPUT 2>/dev/null || true && sudo iptables -t filter -N FORWARD 2>/dev/null || true && sudo iptables -t filter -N OUTPUT 2>/dev/null || true && sudo iptables -P INPUT ACCEPT && sudo iptables -P FORWARD ACCEPT && sudo iptables -P OUTPUT ACCEPT && sudo systemctl start docker
    This completely resets Docker's networking and iptables state. Docker will recreate all required chains on startup.

@claude
Copy link
Copy Markdown

claude Bot commented Apr 28, 2026

Code Review

Overview

This PR is a solid, well-scoped typing initiative. It raises return-annotation coverage in config/graphql/ from ~4.8% to 91.5% and graduates 22 modules out of the mypy baseline allow-list. Importantly, it also bundles several real bug fixes discovered through annotation work — not just syntactic -> Any additions.


What Is Good

Real bug fixes included alongside annotations:

  • permissioning.py — The type[User] / type[Model] annotation inversions (class vs instance) were genuine documentation bugs that would have misled any reader or downstream tool. The fix to get_users_group_ids(user_instance=User) is important: the old default silently passed the User class as an instance; removing it forces callers to be explicit.
  • permissioning.py — The source_analysis is None guard before the recursive user_has_permission_for_obj call is a real potential null-dereference in the case of data inconsistency between created_by_analysis_id and created_by_analysis.
  • conversation_types.py — Using a proper import binascii is correct; base64.binascii is a private implementation detail.
  • slug_queries.py + user_types.py — Moving visible_to_user() before filter() aligns with the project convention (manager method first, narrowing filters second).
  • document_queries.py — The relay_id is None guard is strictly safer than the old from_global_id(None) path, which would silently produce an empty string and blow up on int("").

Good structural choices:

  • TYPE_CHECKING guard for the User forward-reference in permissioning.py avoids a circular import while still satisfying mypy.
  • from __future__ import annotations added consistently to files that use union syntax.
  • _csrf_noop_get_response is cleaner than lambda req: None; the NotImplementedError with a descriptive message documents the invariant well.
  • setattr(request, "_dont_enforce_csrf_checks", True) is the right way to silence mypy on Django-private attributes.

Issues to Address

1. Inconsistent missing-ID error strategy

Three resolvers raise Model.DoesNotExist() when the id kwarg is None, but resolve_assignment uses GraphQLError. DoesNotExist means "we looked and found nothing"; here the argument was never supplied. Prefer raise GraphQLError("<ModelName> id is required"). If the schema marks id as non-null, this branch is unreachable — assert relay_id is not None with a comment is cleaner.

Affected: resolve_document_relationship, resolve_userimport, resolve_userexport (which raise DoesNotExist), vs resolve_assignment (which raises GraphQLError).

2. -> Any on resolvers with knowable concrete types

This is acknowledged as a graduation step, and the tradeoff is reasonable for the batch size. But a handful of trivially-typeable resolvers are worth a follow-up: resolve_feedback_count (could be int), resolve_all_source_node_in_relationship (could be QuerySet[Relationship]), resolve_total_messages (could be int). Leaving these as Any means mypy will accept incorrect callers. Worth a follow-up issue.

3. _get_user_type() -> type and siblings (low priority)

type is equivalent to type[object] — callers learn nothing about what fields exist. The right annotation is type[UserType] etc., using a string literal or TYPE_CHECKING guard to avoid the circular import.

4. resolve_document_relationship return type should be Optional

The declared return type is -> DocumentRelationship (non-Optional). If DocumentRelationshipQueryOptimizer.get_relationship_by_id can return None, the annotation is wrong. Graphene also expects nullable fields to be typed as Optional to reflect what the client can receive.

5. id_to_children key type in base_types.py (minor)

dict[Any, list[Any]] is over-broad. Node IDs in this codebase are int | str; dict[int | str, list[int | str]] would let mypy catch unexpected key types at call sites.


No-ops / Style Notes

  • annotation_types.py and user_types.py intentionally omit from __future__ import annotations since they only use -> Any — this is correct.
  • get_queryset -> Any on DjangoObjectType subclasses is fine; graphene-django stubs do not express the return type usefully.
  • The comment block in mypy.ini explaining which 14 modules remain and why is a nice touch.

Summary

High-value, low-risk PR. The genuine bug fixes (permissioning annotation inversions, null guard on created_by_analysis, binascii import) are worth merging independently of the typing work, and the 22-module mypy graduation is substantial. Main asks before merge: standardise the missing-ID error pattern (issue 1) and widen the resolve_document_relationship return type to Optional (issue 4). Issues 2, 3, and 5 can be tracked as follow-ups.

github-actions Bot and others added 3 commits April 28, 2026 04:30
- Missing-id resolvers now raise GraphQLError consistently (was Model.DoesNotExist on three resolvers, GraphQLError on resolve_assignment)
- _get_user_type/corpus_folder_type/annotation_type return type[T] instead of bare type, with TYPE_CHECKING forward refs to avoid circular imports
- resolve_feedback_count: int, resolve_total_messages: int, resolve_all_source_node_in_relationship: QuerySet[Relationship]
- id_to_children: dict[int | str, list[int | str]] (was dict[Any, list[Any]])
…1332-1lh9h

# Conflicts:
#	config/graphql/analysis_mutations.py
#	config/graphql/annotation_types.py
#	config/graphql/base_types.py
#	config/graphql/corpus_queries.py
#	config/graphql/document_queries.py
#	config/graphql/filters.py
#	config/graphql/moderation_mutations.py
#	config/graphql/og_metadata_queries.py
#	config/graphql/pipeline_queries.py
#	config/graphql/security.py
#	config/graphql/slug_queries.py
#	config/graphql/user_queries.py
#	config/graphql/user_types.py
#	docs/typing/mypy_baseline.txt
#	mypy.ini
@claude
Copy link
Copy Markdown

claude Bot commented Apr 28, 2026

Code Review

Overview

This PR advances the mypy graduation effort for config/graphql/, moving 22 modules off the ignore_errors = True baseline and raising return-annotation coverage from ~4.8% to 91.5%. The approach is conservative and scope-disciplined: root-cause annotation bugs are fixed in permissioning.py, flow-narrowing tricks resolve the bulk of mypy errors in callers, and two known bugs surfaced by mypy are deferred to separate issues (#1359, #1360) rather than conflated here. The risk profile is low — almost all changes are annotations with no runtime effect.


Substantive Bug Fixes (worth calling out)

permissioning.py — inverted type annotations (real bug)

The existing annotations user: type[User] and instance: type[django.db.models.Model] were backwards — they described the class itself, not an instance. The code worked at runtime because Python doesn't enforce annotations, but mypy was flagging every call site. The fix (user: UserModel, instance: django.db.models.Model) is correct. The TYPE_CHECKING guard for the import avoids a circular import at runtime.

permissioning.pyget_users_group_ids default user_instance=User

Removing the default is correct. The old default was the User class object itself, which would have caused a AttributeError at runtime if any caller ever omitted the argument. Good catch.

permissioning.py — null guards for instance.document_id is None and instance.created_by_analysis/extract

These are real defensive improvements. Annotations without a parent document now return False (deny) rather than propagating a None into downstream logic, and the created_by_analysis / created_by_extract optional FK dereferences are guarded before use. Both are semantically correct: denying access to orphaned objects is safer than crashing.

analysis_mutations.pyreturn Nonereturn DeleteAnalysisMutation(ok=True, ...)

This is a real behaviour fix: the mutation was returning null to the frontend on success. The added regression test (test_delete_analysis_mutation.py) correctly pins the return shape.

og_metadata_queries.pyextract.corpus nullable guard

Correct. The FK uses on_delete=SET_NULL so corpus can be None in the DB. The old code would raise AttributeError at runtime for any extract whose corpus was deleted.

document_queries.pyresolve_document_relationship now raises GraphQLError instead of silently passing None to from_global_id

Good improvement. from_global_id(None) would produce unexpected behaviour; an explicit early error is cleaner.


Minor Issues

ingestion_source_mutations.pymessage=error or _NOT_FOUND_MSG

This is functionally safe given the invariant error is None ⟺ pk is not None, but the or fallback is a code smell: if the invariant ever breaks, it silently swallows the real error message. Consider instead:

if pk is None:
    return UpdateIngestionSourceMutation(ok=False, message=error if error else _NOT_FOUND_MSG, ...)

or just assert error is not None before using it — either makes the invariant explicit rather than implicit.

annotation_types.py — removing -> Any from inner CTE lambdas

The CTE helper functions (get_descendants, get_full_tree) now have no return annotation at all. That's strictly less informative than -> Any. For a library like django_cte the return type may be complex, but -> QuerySet or -> Any would be better than implicit -> None. Low priority, but worth noting.

corpus_queries.py_corpus_count_subqueries() -> tuple[Any, Any]

tuple[Any, Any] is accurate but opaque. The two elements are subquery Subquery objects — tuple[Subquery, Subquery] would be more informative. Not blocking, but easy to tighten.

filters.pyqueryset: QuerySet (unparameterised)

Most filter methods now declare queryset: QuerySet without a type parameter. QuerySet[Any] or the specific model type (e.g. QuerySet[Annotation]) would give mypy enough to narrow downstream. This is a minor delta from the existing state and probably acceptable at this stage of the typing effort.


Security Observations

security.pysetattr(request, "_dont_enforce_csrf_checks", True)

The comment correctly explains this is a Django-private flag. Using setattr is the right workaround for the stubs gap. No security regression: the flag is only set when HTTP_AUTHORIZATION is present (token-based auth), which is the pre-existing logic.

security.py_csrf_noop_get_response raising NotImplementedError

This is safe as described: CsrfViewMiddleware.process_view is the only method called in this code path. However, if the Django version ever changes this assumption, the NotImplementedError will surface quickly in tests, which is the desired failure mode.

permissioning.pydocument_id is Nonereturn False

This is a safe default (deny). Worth confirming that no existing test creates annotations without a document intentionally (e.g. corpus-level structural annotations). The change could introduce a regression if that case exists.


Test Coverage

  • test_delete_analysis_mutation.py is well-structured: it tests the return shape directly via graphene.test.Client, mutes signals in setUp to avoid noise, and grants permissions explicitly. One note: the test only covers the success path. A test for the error path (e.g. wrong owner, non-existent analysis) would complete coverage of the mutation's two code branches.

CLAUDE.md Compliance

  • visible_to_user(user).filter(...) ordering: the PR correctly reverses several call sites to use the manager-first pattern, matching CLAUDE.md's recommendation.
  • No dead code introduced.
  • Changelog updated per the Keep a Changelog format.
  • No magic numbers introduced.
  • Type annotations use TYPE_CHECKING guard to avoid circular imports — consistent with the project pattern.

Summary

Approve with minor suggestions. The substantive fixes (null guards, inverted annotations, return None mutation bug) are all correct and well-tested. The annotation-only changes follow project conventions and don't change runtime behaviour. The two deferred issues (#1359, #1360) are appropriately scoped out. The minor points above (ingestion source error propagation, unparameterised QuerySet, CTE inner function annotations) are worth addressing in follow-up but don't block merging.

@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, conversation_id, reason="") -> "LockThreadMutation":
def mutate(root, info, conversation_id, reason="") -> LockThreadMutation:
@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, conversation_id, reason="") -> "UnlockThreadMutation":
def mutate(root, info, conversation_id, reason="") -> UnlockThreadMutation:
@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, conversation_id, reason="") -> "PinThreadMutation":
def mutate(root, info, conversation_id, reason="") -> PinThreadMutation:
@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, conversation_id, reason="") -> "UnpinThreadMutation":
def mutate(root, info, conversation_id, reason="") -> UnpinThreadMutation:
@login_required
@graphql_ratelimit(rate="10/m")
def mutate(root, info, conversation_id, reason=None) -> "DeleteThreadMutation":
def mutate(root, info, conversation_id, reason=None) -> DeleteThreadMutation:
@login_required
@graphql_ratelimit(rate="10/m")
def mutate(root, info, conversation_id, reason=None) -> "RestoreThreadMutation":
def mutate(root, info, conversation_id, reason=None) -> RestoreThreadMutation:
@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, corpus_id, user_id, permissions) -> "AddModeratorMutation":
def mutate(root, info, corpus_id, user_id, permissions) -> AddModeratorMutation:
@login_required
@graphql_ratelimit(rate="20/m")
def mutate(root, info, corpus_id, user_id) -> "RemoveModeratorMutation":
def mutate(root, info, corpus_id, user_id) -> RemoveModeratorMutation:
def mutate(
root, info, action_id, reason=None
) -> "RollbackModerationActionMutation":
def mutate(root, info, action_id, reason=None) -> RollbackModerationActionMutation:
@JSv4 JSv4 merged commit 2ed8ab3 into main Apr 28, 2026
6 checks passed
@JSv4 JSv4 deleted the claude/resolve-issue-1332-1lh9h branch April 28, 2026 13:15
JSv4 added a commit that referenced this pull request Apr 29, 2026
…e type

- Apply pyupgrade/black/isort fixes (Optional → | None) and drop the
  resulting unused 'Optional'/'Union' imports.
- Rename 'info: str' → 'name: str' on django-filters method signatures
  in filters.py. The parameter receives the filter field name; the
  'info' name was misleading copy-paste from GraphQL resolver
  signatures (where 'info' is graphene.ResolveInfo).
- Widen UserQueryMixin.resolve_me return type from User to
  User | AnonymousUser. The query has no @login_required, so
  info.context.user can be AnonymousUser at runtime.
JSv4 added a commit that referenced this pull request Apr 29, 2026
Address PR #1369 review (Claude bot): the missing-return fix in
config/graphql/analysis_mutations.py — restoring
`return DeleteAnalysisMutation(ok=True, message="SUCCESS")` — was the
most impactful behavioral fix bundled into the type-annotation pass,
but no regression test pinned the success shape.

The new test grants DELETE permission on an Analysis to its creator,
runs the GraphQL mutation, and asserts the response contains
`ok=True`/`message="SUCCESS"` rather than the `null` payload that was
previously returned. Without the restored `return`, the assertion on
`data.deleteAnalysis` being non-null fails immediately.
JSv4 added a commit that referenced this pull request Apr 29, 2026
Both PR #1369 (this branch, issue #1332) and #1370 (merged via main,
issue #1333) graduated different sets of modules out of the mypy
baseline. Conflict resolution drops:
  - admin_auth + graphql_api_token_auth.backends (graduated by #1333)
  - graphql.{action_queries, agent_mutations, social_types,
    user_queries, user_types, voting_mutations} (graduated by #1332)
from both mypy.ini and docs/typing/mypy_baseline.txt. CHANGELOG.md
keeps both Added entries side by side.
JSv4 added a commit that referenced this pull request Apr 29, 2026
…1332-1lh9h

Add type annotations to GraphQL resolvers, mutations, and filters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants