dataclasses.replace does not type correctly

[LunarLanding]

Note: if you are reporting a wrong signature of a function or a class in the standard library, then the typeshed tracker is better suited for this report: https://github.com/python/typeshed/issues.

Describe the bug
Hi, I'm using dataclasses with frozen=True as immutable values.
However the very useful 'replace' function from the same module doesn't type correctly (see docs ).
This allows to create a new value by changing fields specified via keyword args.
To Reproduce
Create a dataclass, use the replace function with incorrect arguments.
Expected behavior
Typecheck fail.
Screenshots or Code

from dataclasses import dataclass,replace
@dataclass
class A:
    x : int
    y : int
a = A(0,1)
b = replace(a,x=1) #types correctly
c = replace(a,not_a_field=1) #should not typecheck

VS Code extension or command-line
VS Code extension.

[erictraut]

Thanks for the suggestion.

I consider this feature request (as opposed to a bug report), since the type checker is appropriately applying the type signature of the replace method as it's defined in the dataclasses.pyi stub. I've also confirmed that mypy doesn't provide any special-case handling for the dataclasses.replace method.

Let me think about this a bit more. I generally don't like adding special-case logic in the type checker for specific functions because that approach isn't scalable.

[LunarLanding]

If the default is following mypy, then I will try to bring it up there.

[erictraut]

After thinking about this more, I don't think it's appropriate to add this special-case logic in the type checker for a specific function. The case would be a bit stronger if this were a method on a dataclass, but it's a module-level function.

[danr]

I figured out a way to make a mixin for a correctly typed replace method on data classes. It reuses the type of the constructor of the class, accessed using Type[Self]. One minor wrinkle is that you get a type error if not supplying arguments for all fields without default values.

from dataclasses import dataclass, replace, fields
from typing import Type, Any
from typing_extensions import Self

class ReplaceMixin:
    @property
    def replace(self) -> Type[Self]:
        def replacer(*args: Any, **kws: Any) -> Self:
            for field, arg in zip(fields(self), args):
                kws[field.name] = arg
            return replace(self, **kws)
        return replacer # type: ignore

@dataclass
class A(ReplaceMixin):
    x: int = 0
    y: int = 0

a = A(1, 2)
print(a)
print(a.replace(x=10))
print(a.replace(100))
print(a.replace(y=20))

'''output:
A(x=1, y=2)
A(x=10, y=2)
A(x=100, y=2)
A(x=1, y=20)
'''

@dataclass
class B(ReplaceMixin):
    '''
    No default values
    '''
    x: int
    y: int

b = B(1, 2)
print(b)
print(b.replace(y=20)) # Type error: Argument missing for parameter "x"
                # (however it still does the correct thing at runtime:)

'''output:
B(x=1, y=2)
B(x=1, y=20)
'''

[0xjc]

Is there any good workaround for a type-safe dataclasses.replace? For a non-frozen dataclass, writing a.b = c type checks perfectly. But as soon as you switch to frozen dataclasses and start writing code like replace(a, b=c) instead, all type safety goes out the window. (We've encountered multiple bugs due to this in an otherwise mostly type-safe codebase.)

• The above comment is a clever way to go about it, but requiring defaults for all fields is a dealbreaker for me.
• PEP 692 looks like it could maybe be useful, but the PEP is exclusively for TypedDict and TypedDict seems incompatible with dataclasses.
• If there were some way to provide a dataclass type + a field name as a string literal, and get back the field type, then that would be a starting point for constructing a workaround. But I don't know of anything like that.

[jakkdl]

This has now been implemented in mypy, though with some restrictions: python/mypy#14849

[Badg]

I came acrosss this while looking for a way to do operations on dataclass fields while preserving typing (similar to this, but for a completely different application). FWIW, there are a few discussions / potential PEPs which would allow resolving this directly in typeshed, instead of needing to special-case stuff:

• Inner fields annotations would allow replace to be more specific in the **kwargs specification, directly referencing the field names (and potentially their corresponding types, though I'm not 100% sure how that would work based on the proposed "spelling" (also discussed here
• A general-purpose Map[] type might allow (somehow) the use of TypedDict, see here

See also:

• Unpacking compatibility with dataclass

[shiba-hiro]

I hope an official type definition will be provided for this, but in the meantime, I'd like to share my workaround for anyone facing the same issue.

from dataclasses import replace
from typing import Callable, Concatenate, Any

def typed_replace[T, **P](c: Callable[P, T]) -> Callable[Concatenate[T, P], T]:
    def _replace(base: T, *_args: P.args, **kwargs: P.kwargs) -> T:
        return replace(base, **kwargs) # type: ignore
    return _replace

placeholder: dict[str, Any] = {}
"""Used as a substitute for the required arguments for the class"""

Use Cases;

import dataclasses
import pydantic

from .typed_replace import typed_replace, placeholder

@dataclasses.dataclass(frozen=True)
class NativeDataclass:
    x: int
    y: str
    z: bytes


b1 = NativeDataclass(1, '2', b'3')
r1 = typed_replace(NativeDataclass)(b1, **placeholder, z=b"6")
print(b1)
print(r1)


# The `placeholder` is not required if all required arguments are provided
typed_replace(NativeDataclass)(b1, x=4, y="5", z=b"6")


# Pydantic dataclass is also accepted
@pydantic.dataclasses.dataclass(frozen=True)
class PydanticDataclass:
    x: int
    y: str
    z: bytes

b2 = PydanticDataclass(1, '2', b'3')
r2 = typed_replace(PydanticDataclass)(b2, **placeholder, z=b"6")
print(b2)
print(r2)

I know there are some problems;

• Use of # type ignore in the typed_replace function
• Unused *_args: P.args in the _replace function
• Accepts any Callable, not limited to dataclass instances
• Redundant placeholder

But it can achieve some goals;

• Static type checking for keyword argument types and names
• IDEs can autocomplete the fields

cf. If you are using Python < 3.12, the following code will work;

from dataclasses import replace
from typing import Any, Callable, Concatenate, ParamSpec, TypeVar

_P = ParamSpec("_P")
_R = TypeVar("_R")


def typed_replace(_c: Callable[_P, _R]) -> Callable[Concatenate[_R, _P], _R]:
    def _replace(base: _R, *_args: _P.args, **kwargs: _P.kwargs) -> _R:
        return replace(base, **kwargs)  # type: ignore
    return _replace

[davetapley]

Petition to reopen this @erictraut, since mypy now has it: 🙏🏻

#1047 (comment)