AES-GCM for RPC encryption is supported by SPARK-47172, however, now the feature doesn't correctly handle multiple messages per channel. In YARN application, the auth handshake succeeds but all post-auth RPC messages are dropped or corrupted, leaving the channel hung until YARN kills the container.
Spark version: 3.5.2 and upper
AES-GCM for RPC encryption is supported by SPARK-47172, however, now the feature doesn't correctly handle multiple messages per channel. In YARN application, the auth handshake succeeds but all post-auth RPC messages are dropped or corrupted, leaving the channel hung until YARN kills the container.
Spark version: 3.5.2 and upper