So to sum up:
- We want to start using
contentMediaType to know more details about the type of the field. For binaries, we would like to have application/octet-stream. For other types we can also utilise the same
- In materializations, for historical reasons, we are handling
contentEncoding: base64 inconsistently: some connectors materialize to string, some to binary. We want to add a new feature flag (enabled by default) so that all connectors now materialize to binary. Existing tasks will have that feature flag set to off to retain backward compatibility.
Going forward that means materializations will materialize any field with either:
contentEncoding: base64
contentEncoding: base64 and contentMediaType: application/octet-stream
As BINARY columns in their destination
So to sum up:
contentMediaTypeto know more details about the type of the field. For binaries, we would like to haveapplication/octet-stream. For other types we can also utilise the samecontentEncoding: base64inconsistently: some connectors materialize to string, some to binary. We want to add a new feature flag (enabled by default) so that all connectors now materialize to binary. Existing tasks will have that feature flag set to off to retain backward compatibility.Going forward that means materializations will materialize any field with either:
contentEncoding: base64contentEncoding: base64andcontentMediaType: application/octet-streamAs BINARY columns in their destination