False negatives with `dict.__(r)or__`

[adamtheturtle]

I have code equivalent to the following:

from requests.structures import CaseInsensitiveDict

CaseInsensitiveDict() | dict()

mypy does not error.

My environment has:

• mypy==1.5.1
• types-requests==2.31.0.2
• Python 3.11.5

At runtime I get:

TypeError: unsupported operand type(s) for |: 'CaseInsensitiveDict' and 'dict'

I would expect an [operator] error. For example, if I swap the CaseInsensitiveDict() for 1, I get:

error: Unsupported operand types for | ("int" and "dict[str, str]")  [operator]

I am happy to contribute, but I'm not sure what change is needed. I imagine a change is needed in https://github.com/python/typeshed/blob/main/stubs/requests/requests/structures.pyi.

[AlexWaygood]

In actual fact, the issue is our stubs for dict.__or__ and dict.__ror__:

typeshed/stdlib/builtins.pyi

Lines 1122 to 1129 in aa39b99

	@overload
	def __or__(self, __value: Mapping[_KT, _VT]) -> dict[_KT, _VT]: ...
	@overload
	def __or__(self, __value: Mapping[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...
	@overload
	def __ror__(self, __value: Mapping[_KT, _VT]) -> dict[_KT, _VT]: ...
	@overload
	def __ror__(self, __value: Mapping[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...

We currently claim that dict.__(r)or__ accepts any Mapping, but this isn't actually true. At runtime, the object passed to dict.__or__ has to be an instance of dict (and requests.structures.CaseInsensitiveDict is not a subclass of dict).

Ideally we'd make this change, but I think we tried it recently, and it caused an unfortunate number of false positives:

diff --git a/stdlib/builtins.pyi b/stdlib/builtins.pyi
index 71cccee16..cf4f857c5 100644
--- a/stdlib/builtins.pyi
+++ b/stdlib/builtins.pyi
@@ -1120,13 +1120,13 @@ class dict(MutableMapping[_KT, _VT], Generic[_KT, _VT]):
     if sys.version_info >= (3, 9):
         def __class_getitem__(cls, __item: Any) -> GenericAlias: ...
         @overload
-        def __or__(self, __value: Mapping[_KT, _VT]) -> dict[_KT, _VT]: ...
+        def __or__(self, __value: dict[_KT, _VT]) -> dict[_KT, _VT]: ...
         @overload
-        def __or__(self, __value: Mapping[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...
+        def __or__(self, __value: dict[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...
         @overload
-        def __ror__(self, __value: Mapping[_KT, _VT]) -> dict[_KT, _VT]: ...
+        def __ror__(self, __value: dict[_KT, _VT]) -> dict[_KT, _VT]: ...
         @overload
-        def __ror__(self, __value: Mapping[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...
+        def __ror__(self, __value: dict[_T1, _T2]) -> dict[_KT | _T1, _VT | _T2]: ...
         # dict.__ior__ should be kept roughly in line with MutableMapping.update()
         @overload  # type: ignore[misc]
         def __ior__(self, __value: SupportsKeysAndGetItem[_KT, _VT]) -> Self: ...

[AlexWaygood]

Okay, making the type hints more accurate here is actually less disruptive than I thought it would be! See #10679 (comment)

[adamtheturtle]

Thank you for picking this up so quickly.

[sobolevn]

This change broke dict | TypedDict:

from typing import TypedDict, Any, TypeVar, Mapping

class Foo(TypedDict):
    key: int

foo: Foo = {
    "key": 1
}

reveal_type({'key': 1} | foo)

Causes:

ex.py:10: error: No overload variant of "__or__" of "dict" matches argument type "Foo"  [operator]
ex.py:10: note: Possible overload variants:
ex.py:10: note:     def __or__(self, dict[str, int], /) -> dict[str, int]
ex.py:10: note:     def [_T1, _T2] __or__(self, dict[_T1, _T2], /) -> dict[str | _T1, int | _T2]

Because TypedDict is not compatible with dict[_T1, _T2], only with Mapping

See python/mypy#16249

[AlexWaygood]

Because TypedDict is not compatible with dict[_T1, _T2], only with Mapping

I can't reproduce that error? https://mypy-play.net/?mypy=latest&python=3.11&gist=e2e367d8d6ef21d84703ea8f0e31f57e

And this should be handled by _TypedDict.__ror__:

typeshed/stdlib/typing.pyi

Lines 914 to 917 in 95cee31

	@overload
	def __ror__(self, __value: typing_extensions.Self) -> typing_extensions.Self: ...
	@overload
	def __ror__(self, __value: dict[str, Any]) -> dict[str, object]: ...

[AlexWaygood]

@sobolevn I don't think this change caused python/mypy#16249 either. I can reproduce the error reported in that issue all the way back to mypy==0.902: https://mypy-play.net/?mypy=0.902&python=3.11&gist=8ddd4cfee34714e9cbdb773c7170443a

[sobolevn]

Yeap, we went with special-casing it 👍