Update generated config models

[ofek]

What does this PR do?

This upgrades our config model generation to use Pydantic v2

Motivation

• Improved performance and memory utilization
• Projects will increasingly depend only on the newest version so we should get ahead of that and adapt our code to the breaking changes to avoid dependency conflicts

Additional Notes

The major change is that now options are nullable and the ones that are not required will be None by default rather than the default value of its type. For example, an optional password field will be None rather than an empty string if unset.

A few other things that are noteworthy about the migration:

• Field optionality is determined differently and as a result declarations of fields that are not required now end in a = None
• The validate_default=True on the ConfigDict object is what instructs validation to always run even for fields that are unset (allowing us to set defaults), rather than the old always=True option on the decorators
• The frozen=True on the ConfigDict object is what instructs the model to have best effort immutability, rather than the old allow_mutation = False
• Validation now allows for passing around an arbitrary context object which we now use rather than setting a bogus key within the config dictionary to pass around state
• There is not yet a way to modify the final dictionary immediately before model instantiation so the finalize stage is now broken into 2 steps:
  • a validation runs on every final value to make it immutable
  • the final validation now has nothing to modify but just is able to validate the state of the created model. user-defined validator functions for this are now prefixed by check_ rather than finalize_

Warnings are emitted because of our transformation of mutable types into immutable types. This will be fixed in a subsequent PR.

Review checklist (to be filled by reviewers)

The changes exist only in the base package and in the developer package, the generated code itself is the reason for the large diff

[github-actions]

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

[codecov]

Codecov Report

Merging #15212 (dc5da2f) into master (b78614a) will decrease coverage by 0.28%.
The diff coverage is 99.42%.

Flag	Coverage Δ
active_directory	100.00% <ø> (+17.64%)	⬆️
aerospike	87.17% <ø> (+0.32%)	⬆️
airflow	90.00% <ø> (ø)
cassandra_nodetool	93.16% <ø> (ø)
ceph	91.07% <ø> (ø)
cilium	80.45% <ø> (+0.90%)	⬆️
clickhouse	95.49% <ø> (ø)
cockroachdb	91.52% <ø> (ø)
couch	95.43% <ø> (+0.48%)	⬆️
couchbase	84.28% <ø> (ø)
datadog_checks_base	89.20% <100.00%> (-0.08%)	⬇️
datadog_checks_dev	80.07% <98.59%> (-2.53%)	⬇️
datadog_checks_downloader	81.65% <ø> (ø)
disk	84.85% <ø> (-6.51%)	⬇️
dns_check	93.90% <ø> (ø)
dotnetclr	91.39% <ø> (+12.90%)	⬆️
elastic	93.12% <ø> (ø)
envoy	95.09% <ø> (+0.41%)	⬆️
etcd	95.56% <ø> (ø)
exchange_server	96.85% <ø> (+11.81%)	⬆️
gearmand	78.26% <ø> (+1.24%)	⬆️
gitlab	92.10% <ø> (+0.80%)	⬆️
gitlab_runner	91.94% <ø> (ø)
glusterfs	80.09% <ø> (+0.92%)	⬆️
haproxy	95.13% <ø> (+0.16%)	⬆️
harbor	80.04% <ø> (ø)
hdfs_datanode	89.74% <ø> (ø)
hdfs_namenode	86.72% <ø> (ø)
ibm_ace	91.79% <ø> (ø)
ibm_db2	95.30% <ø> (ø)
ibm_mq	91.13% <ø> (-0.14%)	⬇️
ibm_was	96.08% <ø> (ø)
iis	95.00% <ø> (+37.60%)	⬆️
kafka_consumer	93.24% <ø> (ø)
kong	87.56% <ø> (ø)
lighttpd	83.64% <ø> (ø)
linkerd	85.14% <ø> (+1.14%)	⬆️
mapreduce	81.35% <ø> (ø)
mesos_master	89.75% <ø> (ø)
mesos_slave	93.63% <ø> (ø)
mongo	96.55% <ø> (ø)
mysql	87.29% <ø> (ø)
network	63.19% <ø> (-29.71%)	⬇️
nginx	95.24% <ø> (+0.54%)	⬆️
oracle	89.78% <ø> (ø)
pdh_check	97.82% <ø> (ø)
pgbouncer	91.33% <ø> (ø)
proxysql	98.97% <ø> (ø)
rabbitmq	96.04% <ø> (ø)
redisdb	87.78% <ø> (ø)
rethinkdb	97.93% <ø> (ø)
riakcs	93.61% <ø> (ø)
sap_hana	91.64% <ø> (+0.26%)	⬆️
scylla	100.00% <ø> (ø)
snmp	64.32% <ø> (+0.02%)	⬆️
sonarqube	98.24% <ø> (ø)
spark	93.91% <ø> (+0.28%)	⬆️
sqlserver	86.73% <ø> (+0.08%)	⬆️
ssh_check	91.58% <ø> (ø)
tcp_check	90.23% <ø> (-2.70%)	⬇️
teamcity	88.23% <ø> (+3.17%)	⬆️
tls	92.18% <ø> (+0.82%)	⬆️
varnish	84.39% <ø> (+0.26%)	⬆️
vault	95.53% <ø> (+0.57%)	⬆️
vertica	98.50% <ø> (ø)
voltdb	96.85% <ø> (ø)
vsphere	95.35% <ø> (+0.06%)	⬆️
win32_event_log	86.40% <ø> (+0.27%)	⬆️
windows_performance_counters	98.36% <ø> (ø)
windows_service	98.00% <ø> (ø)
zk	82.18% <ø> (+1.33%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

[github-actions]

Test Results

     975 files       975 suites   6h 48m 32s ⏱️
  5 714 tests   5 622 ✔️      84 💤 4 ❌ 4 🔥
22 732 runs  18 884 ✔️ 3 840 💤 4 ❌ 4 🔥

For more details on these failures and errors, see this check.

Results for commit dc5da2f.

♻️ This comment has been updated with latest results.

[cswatt]

docs-approved

[FlorentClarret]

Great job!

[vivek-datadog]

LGTM, thank you Ofek for the updates 👍