Skip to content

[FEATURE] Improve EncryptorImpl with Asynchronous Handling for Scalability#3919

Merged
dhrubo-os merged 46 commits intoopensearch-project:mainfrom
akolarkunnu:EncryptorRefactoring
Mar 12, 2026
Merged

[FEATURE] Improve EncryptorImpl with Asynchronous Handling for Scalability#3919
dhrubo-os merged 46 commits intoopensearch-project:mainfrom
akolarkunnu:EncryptorRefactoring

Conversation

@akolarkunnu
Copy link
Copy Markdown
Collaborator

@akolarkunnu akolarkunnu commented Jun 17, 2025

Description

Removed CountDownLatch completely and now it is purely based on action listeners.
Along with this, fixed the issue of duplicate master key generation. If key generation for a tenant is in progress and another request come for encryption/decryption with same tenant id, it will again try to generate another master key , because old request is in the process of creating key and not yet completed. So there is a chance of creating duplicate keys for single tenant.

Fix : Removed CountDownLatch completely and now it is purely based on action listeners.
Storing all tenants who are waiting for the key generation in the map tenantWaitingListenerMap. Whenever the key generation completes or error happened for a tenant, notify all requestors waiting for that tenant key generation.
Now both encrypt and decrypt APIs accepts list of Strings as parameter to encrypt/decrypt. Previously it was single String object.

Testing:
Added more test cases with multi threaded use cases.
Also manually tested single and multi tenant use cases :

  • Register the model and invoke predict API - This cover both encryption and decryption cycle.
  • Tested error scenarios by mocking errors. It's behaving as previously
  • Tested With single and multiple credentials
  • Tested Agents with MCP Connector

Commands used for testing(eg: Multi tenant):

Register the model:
curl -XPOST "http://localhost:9200/_plugins/_ml/models/_register"
-H 'Content-Type: application/json'
-H 'x-tenant-id: 1234567'
-d '{
"name": "My OpenAI model: gpt-5",
"function_name": "remote",
"description": "test model",
"connector": {
"name": "My openai connector: gpt-5",
"description": "The connector to openai chat model",
"version": 1,
"protocol": "http",
"parameters": {
"model": "gpt-5"
},
"credential": {
"openAI_key": "..."
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "url",
"headers": {
"Authorization": "Bearer ${credential.openAI_key}"
},
"request_body": "{ "model": "${parameters.model}", "messages": [{"role":"developer","content":"${parameters.system_prompt}"},${parameters._chat_history:-}{"role":"user","content":"${parameters.user_prompt}"}${parameters._interactions:-}], "user": "abdulmun", "reasoning_effort":"minimal"${parameters.tool_configs:-}}"
}
]
}
}'

Invoking PREDICT API:
curl -XPOST "http://localhost:9200/_plugins/_ml/models/3GTtrJwBgpGVOGjb9j0-/_predict"
-H 'Content-Type: application/json'
-H 'x-tenant-id: 1234567'
-d '{
"parameters": {
"user_prompt": "What is 2+2?",
"system_prompt": "You are a helpful assistant."
}
}'

Agents with MCP Connector:

Creating MCP Connector:
curl -XPOST "http://localhost:9200/_plugins/_ml/connectors/_create" -H 'Content-Type: application/json' -d'
{
"name": "My MCP Connector",
"description": "The connector to MCP server",
"version": 1,
"protocol": "mcp_streamable_http",
"credential":{
},
"parameters": {
"endpoint": "/mcp"
},
"url": "https://mcp.api.coingecko.com",
"headers": {
"Authorization": "Bearer ${credential.access}",
"Content-Type": "application/json"
}
}
'

Creating Agents with MCP connector and model created above
curl -XPOST "http://localhost:9200/_plugins/_ml/agents/_register" -H 'Content-Type: application/json' -d'
{
"name": "No cache Claude",
"type": "conversational",
"description": "Use this for Agentic Search",
"llm": {
"model_id": "0dBYr5wBiCznJrIKg6DB",
"parameters": {
"max_iteration": 15
}
},
"memory": {
"type": "conversation_index"
},
"parameters": {
"_llm_interface": "openai/v1/chat/completions",
"mcp_connectors":[
{
"mcp_connector_id": "z9BXr5wBiCznJrIKw6BE"
}
]
},
"tools": [
{
"type": "QueryPlanningTool"
},
{
"type": "ListIndexTool"
},
{
"type": "IndexMappingTool"
}
],
"app_type": "os_chat"
}
'
Also tested update model:
curl -X PUT "http://localhost:9200/_plugins/_ml/models/rxnetpwBy1_WPefbq2da"
-H "Content-Type: application/json" -d '{
"connector": {
"name": "My openai connector: gpt-5",
"description": "The connector to openai chat model",
"version": 1,
"protocol": "http",
"parameters": {
"model": "gpt-5"
},
"credential": {
"openAI_key": "openAI_key"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "url",
"headers": {
"Authorization": "Bearer ${credential.openAI_key}"
},
"request_body": "{ "model": "${parameters.model}", "messages": [{"role":"developer","content":"${parameters.system_prompt}"},${parameters._chat_history:-}{"role":"user","content":"${parameters.user_prompt}"}${parameters._interactions:-}], "user": "abdulmun", "reasoning_effort":"minimal"${parameters.tool_configs:-}}"
}
]
}
}'

Related Issues

Resolves #3510

Check List

  • New functionality includes testing.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Summary by CodeRabbit

  • Refactor

    • Encryption/decryption APIs converted from synchronous returns to asynchronous, listener-driven callbacks across engine and connector layers, with connector interactions updated to use the async callbacks.
  • New Features

    • Per-tenant master-key initialization now queues listeners so concurrent requests are served once keys are ready.
  • Bug Fixes

    • Improved async error propagation for encrypt/decrypt flows.
  • Tests

    • Test suite migrated to listener-based async patterns; added concurrency and error-path coverage.

✏️ Tip: You can customize this high-level summary in your review settings.

…ility

Removed the usage of ContDownLatch. Every requets will be submitted and returns the Future.
Added a list to track the ongoing master key generation. If any tenant id is in the list, then it's key generation is on going and it will wait until other thread completes the key genearion. Same time system will accept other requests, if key is already avaialble in the map that will procced otherwise key generation for new tenant will start in different thread. So, multiple tenants key generation can happen simulatneuosly.

Resolves opensearch-project#3510

Signed-off-by: Abdul Muneer Kolarkunnu <muneer.kolarkunnu@netapp.com>
@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 12:23 — with GitHub Actions Inactive
@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 12:23 — with GitHub Actions Inactive
@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 12:23 — with GitHub Actions Inactive
@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 12:23 — with GitHub Actions Inactive
@dhrubo-os
Copy link
Copy Markdown
Collaborator

Awesome! Thanks for raising the PR. This will be a great improvement. I'll start actively reviewing this PR from tomorrow.

Can you also please update your PR in details like how did you test for single tenancy and also for multi tenancy?

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 17, 2025

Codecov Report

❌ Patch coverage is 61.18421% with 177 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.20%. Comparing base (72b888d) to head (946dd49).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...n/java/org/opensearch/ml/model/MLModelManager.java 60.86% 30 Missing and 6 partials ⚠️
...search/ml/action/tasks/GetTaskTransportAction.java 0.00% 31 Missing ⚠️
.../opensearch/ml/engine/encryptor/EncryptorImpl.java 78.88% 14 Missing and 5 partials ⚠️
...engine/memory/RemoteAgenticConversationMemory.java 0.00% 17 Missing ⚠️
...tion/connector/TransportCreateConnectorAction.java 61.36% 15 Missing and 2 partials ⚠️
...ml/action/tasks/CancelBatchJobTransportAction.java 0.00% 15 Missing ⚠️
...nsearch/ml/common/connector/AbstractConnector.java 70.45% 11 Missing and 2 partials ⚠️
...nsearch/ml/engine/algorithms/agent/AgentUtils.java 62.50% 7 Missing and 2 partials ⚠️
...earch/ml/engine/algorithms/remote/RemoteModel.java 72.41% 6 Missing and 2 partials ⚠️
...c/main/java/org/opensearch/ml/engine/MLEngine.java 76.92% 3 Missing ⚠️
... and 3 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3919      +/-   ##
============================================
- Coverage     77.32%   77.20%   -0.12%     
- Complexity    11534    11537       +3     
============================================
  Files           947      947              
  Lines         51772    51868      +96     
  Branches       6274     6275       +1     
============================================
+ Hits          40031    40044      +13     
- Misses         9091     9172      +81     
- Partials       2650     2652       +2     
Flag Coverage Δ
ml-commons 77.20% <61.18%> (-0.12%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 21:46 — with GitHub Actions Inactive
@akolarkunnu akolarkunnu temporarily deployed to ml-commons-cicd-env-require-approval June 17, 2025 21:46 — with GitHub Actions Inactive
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval June 18, 2025 15:19 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval June 18, 2025 15:19 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval June 18, 2025 15:19 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval June 18, 2025 15:19 — with GitHub Actions Failure
@ylwu-amzn
Copy link
Copy Markdown
Collaborator

See IT failed in CI

REPRODUCE WITH: ./gradlew ':opensearch-ml-plugin:integTest' --tests 'org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model' -Dtests.seed=80D66F063F38F05C -Dtests.security.manager=false -Dtests.locale=en-SH -Dtests.timezone=Asia/Irkutsk -Druntime.java=21 -Djava.security.properties=/__w/ml-commons/ml-commons/distribution/src/config/fips_java.security

RestBedRockInferenceIT > test_bedrock_multimodal_model FAILED
    java.lang.AssertionError: Failing test case name: with_step_size, inference result: {"error":{"root_cause":[{"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys"}],"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys","caused_by":{"type":"cannot_unwrap_data_key_exception","reason":"Unable to decrypt any data keys","caused_by":{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"},"suppressed":[{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"}]}},"status":500.0}
        at __randomizedtesting.SeedInfo.seed([80D66F063F38F05C:3C5AF602511CAB6B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model(RestBedRockInferenceIT.java:137)

@zane-neo
Copy link
Copy Markdown
Collaborator

zane-neo commented Mar 6, 2026

@akolarkunnu I see the code coverage is low in this PR: https://app.codecov.io/gh/opensearch-project/ml-commons/pull/3919?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=opensearch-project, can you check and increase the code coverage?
Also the DCO is failing, can you fix this as well?

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

Persistent review updated to latest commit 044ea8f

@akolarkunnu
Copy link
Copy Markdown
Collaborator Author

See IT failed in CI

REPRODUCE WITH: ./gradlew ':opensearch-ml-plugin:integTest' --tests 'org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model' -Dtests.seed=80D66F063F38F05C -Dtests.security.manager=false -Dtests.locale=en-SH -Dtests.timezone=Asia/Irkutsk -Druntime.java=21 -Djava.security.properties=/__w/ml-commons/ml-commons/distribution/src/config/fips_java.security

RestBedRockInferenceIT > test_bedrock_multimodal_model FAILED
    java.lang.AssertionError: Failing test case name: with_step_size, inference result: {"error":{"root_cause":[{"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys"}],"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys","caused_by":{"type":"cannot_unwrap_data_key_exception","reason":"Unable to decrypt any data keys","caused_by":{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"},"suppressed":[{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"}]}},"status":500.0}
        at __randomizedtesting.SeedInfo.seed([80D66F063F38F05C:3C5AF602511CAB6B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model(RestBedRockInferenceIT.java:137)

@ylwu-amzn I saw it, Is it a random failure or consistent? I am not able to reproduce it because this test case has dependency with AWS ACEES KEY.
From my research, this can happen if we use different master keys for encryption and decryption. So does it because of the change d2c1468 ? Since if I understand correctly, every 5 minutes this cache will expire and regenerate new keys. So it can happen that encryption did with one key and expired it before decrypting that credential and regenerated new key. And decryption did with newly generated key.
But, if we see this failure only with this PR ,root cause will be different.

Is it possible to give me an environment where I can reproduce the issue and debug it?

@akolarkunnu
Copy link
Copy Markdown
Collaborator Author

https://app.codecov.io/gh/opensearch-project/ml-commons/pull/3919?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=opensearch-project

@zane-neo Looks like this is a false report. Eg: I can see GetTaskTransportAction.java has 0% coverage, but there is a test class GetTaskTransportActionTests.java . It applies to all other files which shows 0% coverage. And in the latest CI run I can see coverage task is passed.

@zane-neo
Copy link
Copy Markdown
Collaborator

zane-neo commented Mar 9, 2026

@akolarkunnu I reran the failed checks, and the DCO is still failing, can you rebase your code to fix this? Thanks.

@akolarkunnu
Copy link
Copy Markdown
Collaborator Author

@akolarkunnu I reran the failed checks, and the DCO is still failing, can you rebase your code to fix this? Thanks.

@zane-neo I tried to do "git rebase HEAD~42 --signoff", but it ended up with too many conflicts. @dhrubo-os was saying there is a way to correct DCO from maintainers side. Can you please check that. It is hard to do from my end, since it is showing a cycle of conflicts.

dhrubo-os
dhrubo-os previously approved these changes Mar 9, 2026
@dhrubo-os
Copy link
Copy Markdown
Collaborator

@zane-neo I solved the DCO issue in the PR.

@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit e2f3d9c

@akolarkunnu
Copy link
Copy Markdown
Collaborator Author

Merged the latest code. @dhrubo-os @zane-neo Can you please help me to figure out is there any real issue to fix related to RestBedRockInferenceIT > test_bedrock_multimodal_model failure

@ylwu-amzn
Copy link
Copy Markdown
Collaborator

See IT failed in CI

REPRODUCE WITH: ./gradlew ':opensearch-ml-plugin:integTest' --tests 'org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model' -Dtests.seed=80D66F063F38F05C -Dtests.security.manager=false -Dtests.locale=en-SH -Dtests.timezone=Asia/Irkutsk -Druntime.java=21 -Djava.security.properties=/__w/ml-commons/ml-commons/distribution/src/config/fips_java.security

RestBedRockInferenceIT > test_bedrock_multimodal_model FAILED
    java.lang.AssertionError: Failing test case name: with_step_size, inference result: {"error":{"root_cause":[{"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys"}],"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys","caused_by":{"type":"cannot_unwrap_data_key_exception","reason":"Unable to decrypt any data keys","caused_by":{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"},"suppressed":[{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"}]}},"status":500.0}
        at __randomizedtesting.SeedInfo.seed([80D66F063F38F05C:3C5AF602511CAB6B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model(RestBedRockInferenceIT.java:137)

@ylwu-amzn I saw it, Is it a random failure or consistent? I am not able to reproduce it because this test case has dependency with AWS ACEES KEY. From my research, this can happen if we use different master keys for encryption and decryption. So does it because of the change d2c1468 ? Since if I understand correctly, every 5 minutes this cache will expire and regenerate new keys. So it can happen that encryption did with one key and expired it before decrypting that credential and regenerated new key. And decryption did with newly generated key. But, if we see this failure only with this PR ,root cause will be different.

Is it possible to give me an environment where I can reproduce the issue and debug it?

@zane-neo , can you help find root cause of this failure?

@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit 5859d81

@zane-neo
Copy link
Copy Markdown
Collaborator

RestBedRockInferenceIT > test_bedrock_multimodal_model FAILED
java.lang.AssertionError: Failing test case name: with_step_size, inference result: {"error":{"root_cause":[{"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys"}],"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys","caused_by":{"type":"cannot_unwrap_data_key_exception","reason":"Unable to decrypt any data keys","caused_by":{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"},"suppressed":[{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"}]}},"status":500.0}
at __randomizedtesting.SeedInfo.seed([80D66F063F38F05C:3C5AF602511CAB6B]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model(RestBedRockInferenceIT.java:137)

I checked this and it's an issue introduced in this PR, I left comments in corresponding lines: https://github.com/opensearch-project/ml-commons/pull/3919/changes#r2916783932

@zane-neo
Copy link
Copy Markdown
Collaborator

zane-neo commented Mar 11, 2026

@zane-neo I tried to do "git rebase HEAD~42 --signoff", but it ended up with too many conflicts. @dhrubo-os was saying there is a way to correct DCO from maintainers side. Can you please check that. It is hard to do from my end, since it is showing a cycle of conflicts.

@dhrubo-os I see the DCO is still failing. @akolarkunnu, understand it's hard to rebase too many commits, maintainers can set the DCO to pass, it's fine. My suggestion when developing new features is always use rebase instead of merging upstream code, rebase has much more benefit than merging, and always add -s when committing code.

Resolves opensearch-project#3510

Signed-off-by: Abdul Muneer Kolarkunnu <muneer.kolarkunnu@netapp.com>
@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit 217d870

@akolarkunnu
Copy link
Copy Markdown
Collaborator Author

RestBedRockInferenceIT > test_bedrock_multimodal_model FAILED
java.lang.AssertionError: Failing test case name: with_step_size, inference result: {"error":{"root_cause":[{"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys"}],"type":"m_l_exception","reason":"com.amazonaws.encryptionsdk.exception.CannotUnwrapDataKeyException: Unable to decrypt any data keys","caused_by":{"type":"cannot_unwrap_data_key_exception","reason":"Unable to decrypt any data keys","caused_by":{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"},"suppressed":[{"type":"a_e_a_d_bad_tag_exception","reason":"Tag mismatch"}]}},"status":500.0}
at __randomizedtesting.SeedInfo.seed([80D66F063F38F05C:3C5AF602511CAB6B]:0)
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.opensearch.ml.rest.RestBedRockInferenceIT.test_bedrock_multimodal_model(RestBedRockInferenceIT.java:137)

I checked this and it's an issue introduced in this PR, I left comments in corresponding lines: https://github.com/opensearch-project/ml-commons/pull/3919/changes#r2916783932

Thanks for the details, Corrected it by reverting master key setting based on tenant id is null or not.

@dhrubo-os
Copy link
Copy Markdown
Collaborator

@zane-neo I tried to do "git rebase HEAD~42 --signoff", but it ended up with too many conflicts. @dhrubo-os was saying there is a way to correct DCO from maintainers side. Can you please check that. It is hard to do from my end, since it is showing a cycle of conflicts.

@dhrubo-os I see the DCO is still failing. @akolarkunnu, understand it's hard to rebase too many commits, maintainers can set the DCO to pass, it's fine. My suggestion when developing new features is always use rebase instead of merging upstream code, rebase has much more benefit than merging, and always add -s when committing code.

@zane-neo yeah every time @akolarkunnu is taking any updates from main or pushing any changes DCO started failing, so my plan is before merging the PR, I will set the DCO to pass from maintainer side, so we don't need to worry on DCO for now (specially for this PR). But agree with your instructions to @akolarkunnu !!

Resolves opensearch-project#3510

Signed-off-by: Abdul Muneer Kolarkunnu <muneer.kolarkunnu@netapp.com>
@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit 946dd49

@dbwiddis
Copy link
Copy Markdown
Member

Woohoo, this PR is merged!

Thanks all for the patience in pushing this through!

I hope to see fewer flaky tests on my downstream plugin :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Improve EncryptorImpl with Asynchronous Handling for Scalability

9 participants