Document endpoint supporting both many=True and many=False

[CelestialGuru]

I have a viewset that currently supports creation of a single or multiple items at once. It looks something like this:

class FooViewSet(viewsets.ModelViewSet):
    def create(self, request, *args, **kwargs):
        if not isinstance(request.data, list):
            return super().create(request, *args, **kwargs)
        else:
            serializer = self.get_serializer(data=request.data, many=True)
            serializer.is_valid(raise_exception=True)
            self.perform_bulk_create(serializer)
            headers = self.get_success_headers(serializer.data)
            return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)

There are two ways this could be documented. Either by reusing the component schema with something like this:

content:
  application/json:
    schema:
      anyOf:
        - $ref: '#/components/schemas/Foo'
        - type: array
          items:
            $ref: '#/components/schemas/Foo'

schema.yaml reusing component schemas

openapi: 3.0.3
info:
  title: ''
  version: 0.0.0
paths:
  /api/foos/foo/:
    post:
      operationId: foo_foo_create
      description: ''
      requestBody:
        content:
          application/json:
            schema:
              anyOf:
                - $ref: '#/components/schemas/FooRequest'
                - type: array
                  items:
                    $ref: '#/components/schemas/FooRequest'
                  
        required: true
      responses:
        '201':
          content:
            application/json:
              schema:
                anyOf:
                  - $ref: '#/components/schemas/Foo'
                  - type: array
                    items:
                      $ref: '#/components/schemas/Foo'
          description: ''
components:
  schemas:
    Foo:
      type: object
      properties:
        id:
          type: integer
        some_field:
          type: integer
      required:
        - id
        - some_field
    FooRequest:
      type: object
      properties:
        some_field:
          type: integer
      required:
        - some_field

Or perhaps one could define multiple component schemas:

content:
  application/json:
    schema:
      anyOf:
        - $ref: '#/components/schemas/Foo'
        - $ref: '#/components/schemas/FooList'

schema.yaml with multiple component schemas

openapi: 3.0.3
info:
  title: ''
  version: 0.0.0
paths:
  /api/foos/foo/:
    post:
      operationId: foo_foo_create
      description: ''
      requestBody:
        content:
          application/json:
            schema:
              anyOf:
                - $ref: '#/components/schemas/FooRequest'
                - $ref: '#/components/schemas/FooRequestList'
                  
        required: true
      responses:
        '201':
          content:
            application/json:
              schema:
                anyOf:
                  - $ref: '#/components/schemas/Foo'
                  - $ref: '#/components/schemas/FooList'
          description: ''
components:
  schemas:
    Foo:
      type: object
      properties:
        id:
          type: integer
        some_field:
          type: integer
      required:
        - id
        - some_field
    FooList:
      type: array
      items:
        $ref: '#/components/schemas/Foo'
    FooRequest:
      type: object
      properties:
        some_field:
          type: integer
      required:
        - some_field
    FooRequestList:
      type: array
      items:
        $ref: '#/components/schemas/FooRequest'

I've tried the PolymorphicProxySerializer but that doesn't seem to work here.

This just generates an empty request:

@extend_schema(
    request=PolymorphicProxySerializer(
        "DifferentRequests",
        serializers=[
            FooSerializer,
            inline_serializer("ListSerializer", fields={"items": FooSerializer()}),
        ],
        resource_type_field_name=None,
    )
)

This just gives an error 'child' is a required argument:

@extend_schema(
    request=PolymorphicProxySerializer(
        "DifferentRequests",
        serializers=[
            FooSerializer,  # FooSerializer.Meta.list_serializer_class == FooListSerializer
            FooListSerializer,  # FooListSerializer isinstance of ListSerializer
        ],
        resource_type_field_name=None,
    )
)

This just fails:

@extend_schema(
    request=PolymorphicProxySerializer(
        "DifferentRequests",
        serializers=[
            FooSerializer,
            FooSerializer(many=True),  # extend_schema expecting type, not an instance
        ],
        resource_type_field_name=None,
    )
)

How can I have drf-spectacular generate the correct documentation for me in this case? I want to have an api that supports both single objects and list of objects.

[tfranzel]

Hi @CelestialGuru

This just generates an empty request:

No it does not. It's not what you want but it's certainly not empty.

This just gives an error 'child' is a required argument:

That is an assert from DRF and I'm pretty sure it's a usage mistake by you. If done "right" should also yield the error below.

This just fails:

Yep, that fails. Your feature request certainly makes sense. This fails because list handling and serializer resolution are two separate steps. You found a small gap in the API because it's only possible to create such a structure here and without PolymorphicProxySerializer that is simply not possible to even get there. But yes, one would assume this is certainly possible.

The fix shouldn't be super complicated. I'll have a look.

[tfranzel]

So this was bigger than expected.

• There was a bug in the many=True/False override handling of PolymorphicProxySerializer, which should be fixed now.
• Wrong usage patterns should now emit better warnings with instructions.
• Unhandled exception should not occur anymore.

Some more details for the new feature. By using many=False on PolymorphicProxySerializer your take over control regarding list handling. Spectacular will disable automatic list detection here. Using many=True on the sub-serializers also requires the discriminator to be disabled (resource_type_field_name=None). So you should be able to achieve your goal with

@extend_schema(
    request=PolymorphicProxySerializer(
        "DifferentRequests",
        serializers=[FooSerializer, FooSerializer(many=True)],
        resource_type_field_name=None,
        many=False,
    )
)

This might look a little bit awkward but that is a consequence of the implications behind this particular usage pattern. The other variations (when properly initialized), should now also work.

[CelestialGuru]

I made sure to pip install git+https://github.com/tfranzel/drf-spectacular.git and verified that cat /usr/local/lib/python3.10/site-packages/drf_spectacular/serializers.py matched the changes you made. However, when I try

Setup

class QuoteSerializer(serializers.ModelSerializer):
    class Meta:
        model = models.Quote
        fields = "__all__"

@extend_schema_view(
    create=extend_schema(  # Let's start with just manually specifying the documentation, shall we?
        request=PolymorphicProxySerializer(
            "DifferentRequests",
            serializers=[
                serializers.QuoteSerializer,  # With or without ()
                serializers.QuoteSerializer(many=True),
            ],
            resource_type_field_name=None,
            many=False,
        )
    )
)
class QuoteViewSet(viewsets.ModelViewSet):
    serializer_class = serializers.QuoteSerializer
    queryset = models.Quote.objects.all()

I get an exception:

AssertionError: internal assumption violated because we expected a basic serializer here and instead got a "QuoteSerializer(many=True)"

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/views.py", line 70, in get
    return self._get_schema_response(request)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/views.py", line 78, in _get_schema_response
    data=generator.get_schema(request=request, public=self.serve_public),
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/generators.py", line 262, in get_schema
    paths=self.parse(request, public),
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/generators.py", line 233, in parse
    operation = view.schema.get_operation(
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/utils.py", line 357, in get_operation
    return super().get_operation(path, path_regex, path_prefix, method, registry)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 81, in get_operation
    request_body = self._get_request_body()
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1168, in _get_request_body
    schema, request_body_required = self._get_request_for_media_type(request_serializer)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1199, in _get_request_for_media_type
    component = self.resolve_serializer(serializer, 'request')
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1450, in resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 855, in _map_serializer
    schema = serializer_extension.map_serializer(self, direction)
  File "/app/config/schema.py", line 27, in map_serializer
    resolved_sub_serializers = [auto_schema.resolve_serializer(sub, direction) for sub in sub_serializers]
  File "/app/config/schema.py", line 27, in <listcomp>
    resolved_sub_serializers = [auto_schema.resolve_serializer(sub, direction) for sub in sub_serializers]
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1437, in resolve_serializer
    assert_basic_serializer(serializer)
  File "/usr/local/lib/python3.10/site-packages/drf_spectacular/plumbing.py", line 132, in assert_basic_serializer
    assert is_basic_serializer(serializer), (
AssertionError: internal assumption violated because we expected a basic serializer here and instead got a "QuoteSerializer(many=True):
    id = IntegerField(label='ID', read_only=True)
    name = CharField(max_length=256)
    company = CharField(max_length=256)
    address_line_1 = CharField(max_length=256)
    address_line_2 = CharField(max_length=256, required=False)
    city = CharField(max_length=256)
    state = CharField(max_length=256)
    zip = CharField(max_length=256)
    email = EmailField(max_length=256)
    phone = CharField(max_length=256)
    user = PrimaryKeyRelatedField(queryset=User.objects.all())
    preferred_country_of_origin = PrimaryKeyRelatedField(allow_null=True, queryset=Country.objects.all(), required=False)
    internal_sales_rep = PrimaryKeyRelatedField(queryset=User.objects.all())
    external_sales_rep = PrimaryKeyRelatedField(queryset=User.objects.all())". This may be the result of another app doing some unexpected magic or an invalid internal call. Feel free to report this as a bug at https://github.com/tfranzel/drf-spectacular/issues

My model I am using isn't doing anything unexpected either.

models.py

class Quote(models.Model):
    name = models.CharField(max_length=256)
    company = models.CharField(max_length=256)
    user = models.ForeignKey(get_user_model(), related_name="quotes", on_delete=models.CASCADE)

    address_line_1 = models.CharField(max_length=256)
    address_line_2 = models.CharField(max_length=256, default="")
    city = models.CharField(max_length=256)
    state = models.CharField(max_length=256)
    zip = models.CharField(max_length=256)
    email = models.EmailField(max_length=256)
    phone = models.CharField(max_length=256)

    preferred_country_of_origin = models.ForeignKey(
        "configuration.Country",
        null=True,
        blank=True,
        on_delete=models.CASCADE,
    )

    internal_sales_rep = models.ForeignKey(
        get_user_model(),
        related_name="quotes_by_internal",
        on_delete=models.CASCADE,
    )
    external_sales_rep = models.ForeignKey(
        get_user_model(),
        related_name="quotes_by_external",
        on_delete=models.CASCADE,
    )

[tfranzel]

that is very weird. that error message is the behavior before. Also the added test-case is almost identical to your example. can you please double-check the commit you are at with:

✗ pip freeze | grep "drf-spec"
drf-spectacular @ git+https://github.com/tfranzel/drf-spectacular.git@6f5bd1688182829204bb61db6a9d7903fdaf0bbe

I strongly suspect you are looking in the wrong directory or rather your python is using a different location. you may have to pip uninstall drf-spectacular and then install it again with the link. I had issues there previously.

[tfranzel]

  File "/app/config/schema.py", line 27, in <listcomp>
    resolved_sub_serializers = [auto_schema.resolve_serializer(sub, direction) for sub in sub_serializers]

• is part of your code.

• the exception stacktrace does not contain serializers.py where the error was happening before. As a matter of fact that line looks like the RollupMixin from the blueprints.

• There might be a followup bug in the RollupMixin. Please confirm this is what you are doing. Also try to disable the extension in /app/config/schema.py just to make sure

[tfranzel]

2 more thoughts since this is most likely due to the usage of rollup.

1. The concept of rollup simply does not make sense anymore, if you have sub-serializers that are mixed lists/non-list (via this new feature). You cannot use both things at the same time. They are mutually exclusive.
2. Rollup is not officially supported, because it only serves special cases while having quite a few caveats. It is in the blueprint just for reference. The implementation is complicated and the trade-offs are not that easy to grasp.

[CelestialGuru]

Holy cow you're right. /app/config.schema.py was something I copied from https://drf-spectacular.readthedocs.io/en/latest/blueprints.html#polymorphic-models and have forgotten about. I suppose this worth having fixed as well? I've temporarily removed /app/config.schema.py and documentation works as I'd expect (at least for the view I'm examining atm)

It looks like in RollupMixin, where it goes resolved_sub_serializers = [auto_schema.resolve_serializer(sub, direction) for sub in sub_serializers] fails when sub is the serializer with many=True. auto_schema.resolve_serializer goes nuts on that one.

Looks like that it is openapi.py resolve_serializer() which checks is_serializer(obj) and not is_list_serializer(obj). Obviously we're dealing with list serializers in this example. If someone were to use this RollupMixin AND had a create-single-or-create-many request like mine, they'd run into this issue.

[tfranzel]

works as I'd expect

glad to hear.

fails when sub is the serializer with many=True.

Yes that is true. Probably should add a warning or something. Before this feature addition, the opportunity to break it simply wasn't there yet.

I suppose this worth having fixed as well?

As I said above (1.), I'm pretty certain the "rollup" is conceptually broken once you use this new feature. You can't elevate a field one step upwards through a wrapped list. It adds an additional layer of indirection which breaks the underlying idea. Not saying its not possible somehow, but the blueprint is for reference and not officially supported. I lack the time to deal with this.

If someone were to use this RollupMixin AND had a create-single-or-create-many request like mine, they'd run into this issue.

yes. you have to choose which feature is more important to you. The motivation behind the rollup was to fix a specific typescript code generation issue. I don't see much benefit outside of that and swagger-ui will not display this in a nice way anymore.

[CelestialGuru]

Solution works for me. Thanks tfranzel for fixing the polymorphic serializer. I agree that the issue with the rollup isn't really an issue since it's not officially supported. I'll be able to adapt the rollup to suite my needs elsewhere, but the original problem is solved. Thanks.