-
Notifications
You must be signed in to change notification settings - Fork 122
platform_permission deletion fails with _artifact error when repositories are destroyed concurrently #1382
Description
platform_permission deletion fails with _artifact error when repositories are destroyed concurrently
Description
When a platform_permission resource and the repositories it targets are destroyed in the same terraform apply, the concurrent deletions produce a race condition on the Artifactory backend. The error is:
Error: Could not update permission <name> with identifier <name>_artifact
This happens because each time Artifactory deletes a repository, it automatically tries to update any permission referencing that repo (to remove it from the permission's targets). When many repo deletes and the permission delete itself all run in parallel, the backend-side PUT (update permission) and DELETE (remove permission) collide, resulting in a StorageException on the _artifact identifier.
The permission and repos are actually deleted from Artifactory, but Terraform receives error responses for some repo deletions, leaving orphaned entries in the state file that require manual terraform state rm.
Reproduction
Prerequisite: Artifactory < 7.125.0 (tested on 7.111.9). A readers group must exist.
export JFROG_URL="http://localhost:8082"
export JFROG_ACCESS_TOKEN="<token>"main.tf
terraform {
required_providers {
artifactory = { source = "jfrog/artifactory", version = "12.9.4" }
platform = { source = "jfrog/platform", version = "2.2.6" }
}
}
provider "artifactory" {}
variable "repo_count" {
type = number
default = 10
}
locals {
repo_keys = [for i in range(var.repo_count) : "repo-del-${i}-generic"]
}
resource "artifactory_local_generic_repository" "generic_repo" {
key = "repo-del-${count.index}-generic"
count = var.repo_count
}
resource "platform_permission" "shared_permission_target" {
count = var.repo_count > 0 ? 1 : 0
name = "repo-del-shared-permission-target"
artifact = {
actions = {
groups = [{ name = "readers", permissions = ["READ"] }]
}
targets = [
for repo in local.repo_keys : {
name = repo
include_patterns = ["**"]
}
]
}
}Note: The permission targets are built from
local.repo_keys(a computed list of strings) rather than referencingartifactory_local_generic_repository.generic_repo[*].keydirectly. This is intentional — it removes the implicit Terraform dependency between the permission and the repos, so Terraform destroys them in parallel. This simulates real-world configurations where the permission and repositories are managed in separate modules.
Steps
terraform init
# Create 500 repos + 1 permission
terraform apply -var repo_count=500 -auto-approve
# Tear down everything (no --target, no --destroy)
terraform apply -var repo_count=0 -auto-approveExpected
All 501 resources destroyed cleanly.
Actual
The permission and repos are destroyed concurrently. The permission deletion takes ~40s under this load. During that window, 9 out of 500 repo deletions fail with:
Error: Unable to Delete Resource
Error: Could not update permission repo-del-shared-permission-target
with identifier repo-del-shared-permission-target_artifact
After the run, all resources are actually gone from Artifactory, but terraform state list shows 9 orphaned entries.
Key Observation
At small scale (e.g. 20 repos), the permission deletes in under 1 second and completes before any repo deletes can race with it — so the error doesn't appear. At 500 repos, the permission delete takes ~40s, creating a wide window for the concurrent repo deletes to trigger the conflicting backend updates.
If the permission references repos via a direct resource attribute (e.g. artifactory_local_generic_repository.generic_repo[*].key), Terraform infers a dependency and serializes destruction (permission first, then repos), which also avoids the race. The issue only manifests when there's no Terraform-level dependency — which is common in modular configurations.
Workaround
Option 1: -parallelism=1
terraform apply -var repo_count=0 -parallelism=1 -auto-approveForces Terraform to delete resources one at a time, eliminating the race. Slower, but reliable on any Artifactory version.
Option 2: Upgrade Artifactory (permanent fix)
Upgrade to Artifactory 7.125.0+ and add to system.yaml:
shared:
extraJavaOpts: "-Dartifactory.enable.access.client.delete.repo.from.permission=true"This was validated — both Flow A (repo-only delete with --target) and Flow B (permission + repos together) pass cleanly at 500 repos on 7.125.0 with this flag.
Related
- JA-17196: Concurrent deletes of repositories failing when repositories share the same permission target
- RTDEV-58457: RT to use new gRPC API exposed by Access Client during repo delete
- RTDEV-62362: Calls to
PermissionResourceV2/DeleteResourcerandomly missing for delete repo call via Terraform
Environment
- Artifactory: 7.111.9 (fails), 7.125.0 (fixed with flag)
- jfrog/artifactory provider: 12.9.4
- jfrog/platform provider: 2.2.6
- Terraform: 1.x
- OS: macOS (reproduction), RHEL 8.10 (customer)