Skip to content

fix: relax JSON normalizer type validation#3906

Open
rooperuu wants to merge 2 commits intodlt-hub:develfrom
rooperuu:fix/json-normalizer-validation
Open

fix: relax JSON normalizer type validation#3906
rooperuu wants to merge 2 commits intodlt-hub:develfrom
rooperuu:fix/json-normalizer-validation

Conversation

@rooperuu
Copy link
Copy Markdown
Contributor

Description

Support various operations that previously only worked on the dlt.common.normalizers.json.relational normalizer. This includes dlt.common.normalizers.json.relational_no_coercion or any derived custom normalizers such as:

from dlt.common.normalizers.json import relational

class DataItemNormalizer(relational.DataItemNormalizer):
    pass

This also fixes max_nesting configuration on these normalizer subtypes.

import os
import dlt

os.environ["SCHEMA__JSON_NORMALIZER"] = (
    '{"module": "dlt.common.normalizers.json.relational_no_coercion", "config": {"max_nesting": 3}}'
)

@dlt.source
def nothing():
    yield from []

source = nothing()
print(source.max_table_nesting)  # 3

Related Issues

@rooperuu rooperuu changed the title Fix/json normalizer validation fix: relax JSON normalizer type validation Apr 29, 2026
@Travior Travior self-assigned this Apr 29, 2026
@Travior Travior self-requested a review May 5, 2026 08:59
@Travior
Copy link
Copy Markdown
Contributor

Travior commented May 5, 2026

@rooperuu thanks a lot for posting your fix. There's a couple of things I would want to do differently when incorporating this into dlt. I forgot to ask: Are you willing (and have time 😁) to work on this then I can give you a proper review, otherwise I can take this over

@rooperuu
Copy link
Copy Markdown
Contributor Author

rooperuu commented May 5, 2026

@Travior Thanks for taking a look!

I think we need this for our use case, so I can spend some time on this, or whichever is easiest. Let me know what you would like to change and I can do changes accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data item normalizer type validation is overly strict

2 participants