feat: enhance encrypt_fields to support nested structures

[nic-6443]

Description

Enhance the encrypt_fields mechanism in plugin.lua to support nested and complex field structures. The current implementation only handles flat keys and 2-level dotted paths (e.g., sasl.password).

What's changed

Core: Rewrite encrypt/decrypt traversal logic (apisix/plugin.lua)

Replace the flat/2-level if/elseif logic with a recursive process_encrypt_field() helper that supports:

1. Arbitrary depth dotted paths — e.g., auth.gcp.service_account_json traverses 3 levels deep
2. Array traversal at intermediate nodes — when an intermediate value is an array (e.g., instances), iterate each element and continue recursion
3. Leaf type dispatch:
   • string → encrypt/decrypt directly
   • array of strings → iterate and encrypt/decrypt each element
   • map of strings → iterate and encrypt/decrypt each value

Uses core.table.isarray() for reliable array vs map detection.

Fix: Decrypt before schema validation in plugin_checker

Reorder plugin_checker to run decrypt_conf before check_schema, matching the gateway's approach. This is necessary because plugins like ai-proxy perform content-level validation (e.g., parsing service_account_json as JSON) — without decrypting first, encrypted values fail these checks during config sync.

Plugin schema updates:

Plugin	encrypt_fields added
ai-proxy	auth.header, auth.query, auth.gcp.service_account_json
ai-proxy-multi	instances.auth.header, instances.auth.query, instances.auth.gcp.service_account_json
ai-rag	embeddings_provider.azure_openai.api_key, vector_search_provider.azure_ai_search.api_key

Backward compatibility

• Flat keys (e.g., password) continue to work unchanged
• Existing 2-level dotted paths (e.g., sasl.password, auth.password) continue to work
• Regression tests included for both cases

Tests

Added t/node/data_encrypt3.t with 7 test cases covering:

• Map of strings encryption (ai-proxy auth.header, auth.query)
• 3-level nested string (ai-proxy auth.gcp.service_account_json, ai-rag API keys)
• Array with nested encrypted fields (ai-proxy-multi instances[].auth.header)
• Nil/missing field handling
• Flat key regression (basic-auth password)
• 2-level dotted path regression (kafka-proxy sasl.password)
• Array of strings leaf encryption via process_encrypt_field

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Enhances APISIX data encryption support for plugin encrypt_fields by enabling recursive traversal of nested objects/arrays and ensuring decryption occurs before schema validation for content-level checks.

Changes:

• Add recursive process_encrypt_field() traversal for arbitrarily nested dotted paths and array/map leaf handling.
• Reorder plugin_checker to decrypt plugin configs before schema validation.
• Extend plugin schemas to declare additional encrypt_fields, and add a new test suite covering nested/enhanced cases.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
apisix/plugin.lua	Replaces flat/2-level encryption logic with recursive traversal; changes validation flow to decrypt before schema checks.
apisix/plugins/ai-proxy/schema.lua	Adds encrypt_fields for nested auth header/query/GCP JSON fields (including multi-instance paths).
apisix/plugins/ai-rag.lua	Adds encrypt_fields for nested Azure API keys.
t/node/data_encrypt3.t	Adds integration tests covering nested paths, maps, arrays, regressions, and a direct helper test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.