AssertionError in plumbing.ResolvedComponent with 'COMPONENT_SPLIT_REQUEST': True

[johnthagen]

Describe the bug

We ported medium sized DRF application from drf-yasg to drf-spectacular. Everything is working well and the schema generates, and it can be browsed using swagger-ui and redoc.

But when we set 'COMPONENT_SPLIT_REQUEST': True, the following AssertionError is thrown when loading swagger-ui or redoc:

Internal Server Error: /rest/openapi/
Traceback (most recent call last):
  File "venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "venv/lib/python3.9/site-packages/drf_spectacular/views.py", line 67, in get
    return self._get_schema_response(request)
  File "venv/lib/python3.9/site-packages/drf_spectacular/views.py", line 74, in _get_schema_response
    return Response(generator.get_schema(request=request, public=self.serve_public))
  File "venv/lib/python3.9/site-packages/drf_spectacular/generators.py", line 259, in get_schema
    paths=self.parse(request, public),
  File "venv/lib/python3.9/site-packages/drf_spectacular/generators.py", line 233, in parse
    operation = view.schema.get_operation(
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 76, in get_operation
    request_body = self._get_request_body()
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 1045, in _get_request_body
    schema, request_body_required = self._get_request_for_media_type(request_serializer)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 1076, in _get_request_for_media_type
    component = self.resolve_serializer(serializer, 'request')
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 1315, in resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 765, in _map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 840, in _map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 543, in _map_serializer_field
    return append_meta(self._unwrap_list_serializer(field, direction), meta)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 1148, in _unwrap_list_serializer
    self._unwrap_list_serializer(serializer.child, direction)
  File "venv/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 1145, in _unwrap_list_serializer
    return self.resolve_serializer(serializer, direction).ref
  File "venv/lib/python3.9/site-packages/drf_spectacular/plumbing.py", line 590, in ref
    assert self.__bool__()
AssertionError

To Reproduce

Python: 3.9.6

Django-related PyPI packages installed:

Django                               3.2.8
django-auth-ldap                     3.0.0
django-cleanup                       5.2.0
django-cors-headers                  3.10.0
django-crispy-forms                  1.13.0
django-dbbackup                      3.3.0
django-debug-toolbar                 3.2.2
django-debug-toolbar-request-history 0.1.3
django-downloadview                  2.1.1
django-enumfields                    2.1.1
django-filter                        21.1
django-guardian                      2.4.0
django-solo                          1.2.0
djangorestframework                  3.12.4
djangorestframework-guardian         0.3.0
djangorestframework-simplejwt        4.7.2
drf-extra-fields                     3.1.1
drf-spectacular                      0.20.1
drf-spectacular-sidecar              2021.10.11
drf-writable-nested                  0.6.3

Settings:

SPECTACULAR_SETTINGS = {
    ...
    "COMPONENT_SPLIT_REQUEST": True,
    ...
    "SWAGGER_UI_DIST": "SIDECAR",
    "SWAGGER_UI_FAVICON_HREF": "SIDECAR",
    "REDOC_DIST": "SIDECAR",
}

I will try to narrow down the error to a small reproducible example if possible. Any tips on where to start looking would be appreciated.

Expected behavior

Swagger-UI and Redoc pages to load properly and display extra split models.

[tfranzel]

🤦 I think I have a pretty good idea what goes wrong there.

_unwrap_list_serializer is a new addition, so its not as battle-tested as the rest. When you do SPLIT, it's possible to generate components without any fields for one direction. Since those get "garbage collected", there can be no ref to a non-existent component.

[ngnpope]

Just to cross-reference, this was a very similar traceback to the one I encountered for #542 - again this was COMPONENT_SPLIT_REQUEST set to True and assert self.__bool__() failing.

[johnthagen]

@tfranzel Should I try out master branch and see if I'm still seeing the issue?

[tfranzel]

@ngnpope yes same loose end, but in a slightly different context.

@johnthagen yes please test the master branch. this should do it once and for all. It fixes an old loose end plus the regression from 0.19.0

This issue here is that empty serializers are kind of awkward to model properly, which means the schema is not 100% correct. If there are no fields, we simplify and say there is no response, although in reality the response is always an empty list. not exactly the same. Those bugs were caused because I did not follow through that edge case for all variations.

[johnthagen]

@tfranzel I can confirm that after doing pip install git+https://github.com/tfranzel/drf-spectacular.git "COMPONENT_SPLIT_REQUEST": True, no longer throws the AssertionError. 🎉

I see a bunch of new _Request models as well, so the splitting seems to be working. Thanks!

Do you think #569 could also be included in the next release?

[tfranzel]

excellent @johnthagen!

i will have a look at #569 next.