Terraform crashes on an invalid object definition (duplicate attribute)

[weakcamel]

Terraform Version

$ terraform version
Terraform v1.5.6
on darwin_amd64
+ provider registry.terraform.io/hashicorp/azurerm v3.71.0
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/hashicorp/vault v3.19.0

Terraform Configuration Files

variable "keys" {
  description = "Map of K/V keys to store in KV"
  type = map(object({
    operations = optional(list(string), [])
    type       = optional(string, "RSA")
    type       = optional(number)
  }))
  default = {}
}

Debug Output

2023-08-25T17:22:31.984+0100 [INFO]  Terraform version: 1.5.6
2023-08-25T17:22:31.985+0100 [DEBUG] using github.com/hashicorp/go-tfe v1.26.0
2023-08-25T17:22:31.985+0100 [DEBUG] using github.com/hashicorp/hcl/v2 v2.16.2
2023-08-25T17:22:31.985+0100 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.0
2023-08-25T17:22:31.985+0100 [DEBUG] using github.com/zclconf/go-cty v1.12.2
2023-08-25T17:22:31.985+0100 [INFO]  Go runtime version: go1.20.7
2023-08-25T17:22:31.985+0100 [INFO]  CLI args: []string{"terraform", "plan", "-out=p"}
2023-08-25T17:22:31.985+0100 [TRACE] Stdout is not a terminal
2023-08-25T17:22:31.985+0100 [TRACE] Stderr is not a terminal
2023-08-25T17:22:31.985+0100 [TRACE] Stdin is a terminal
2023-08-25T17:22:31.985+0100 [DEBUG] Attempting to open CLI config file: /Users/waldekm/.terraformrc
2023-08-25T17:22:31.985+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2023-08-25T17:22:31.986+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2023-08-25T17:22:31.986+0100 [DEBUG] ignoring non-existing provider search directory /Users/waldekm/.terraform.d/plugins
2023-08-25T17:22:31.986+0100 [DEBUG] ignoring non-existing provider search directory /Users/waldekm/Library/Application Support/io.terraform/plugins
2023-08-25T17:22:31.986+0100 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2023-08-25T17:22:31.988+0100 [INFO]  CLI command args: []string{"plan", "-out=p"}
2023-08-25T17:22:32.013+0100 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2023-08-25T17:22:32.013+0100 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2023-08-25T17:22:32.013+0100 [DEBUG] New state was assigned lineage "f601f2d3-731e-7175-91fd-556bffaf6c5c"
2023-08-25T17:22:32.013+0100 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2023-08-25T17:22:32.013+0100 [TRACE] Meta.Backend: instantiated backend of type <nil>
2023-08-25T17:22:32.015+0100 [TRACE] providercache.fillMetaCache: scanning directory .terraform/providers
2023-08-25T17:22:32.016+0100 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v3.71.0 for darwin_amd64 at .terraform/providers/registry.terraform.io/hashicorp/azurerm/3.71.0/darwin_amd64
2023-08-25T17:22:32.017+0100 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/random v3.5.1 for darwin_amd64 at .terraform/providers/registry.terraform.io/hashicorp/random/3.5.1/darwin_amd64
2023-08-25T17:22:32.017+0100 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/vault v3.19.0 for darwin_amd64 at .terraform/providers/registry.terraform.io/hashicorp/vault/3.19.0/darwin_amd64
2023-08-25T17:22:32.018+0100 [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/hashicorp/azurerm/3.71.0/darwin_amd64 as a candidate package for registry.terraform.io/hashicorp/azurerm 3.71.0
2023-08-25T17:22:32.018+0100 [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/hashicorp/random/3.5.1/darwin_amd64 as a candidate package for registry.terraform.io/hashicorp/random 3.5.1
2023-08-25T17:22:32.018+0100 [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/hashicorp/vault/3.19.0/darwin_amd64 as a candidate package for registry.terraform.io/hashicorp/vault 3.19.0
2023-08-25T17:22:32.952+0100 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/providers
2023-08-25T17:22:33.011+0100 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/providers
2023-08-25T17:22:33.142+0100 [DEBUG] checking for provisioner in "."
2023-08-25T17:22:33.151+0100 [DEBUG] checking for provisioner in "/usr/local/bin"
2023-08-25T17:22:33.151+0100 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2023-08-25T17:22:33.162+0100 [INFO]  backend/local: starting Plan operation
2023-08-25T17:22:33.164+0100 [TRACE] backend/local: requesting state manager for workspace "default"
2023-08-25T17:22:33.164+0100 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2023-08-25T17:22:33.164+0100 [TRACE] backend/local: requesting state lock for workspace "default"
2023-08-25T17:22:33.169+0100 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2023-08-25T17:22:33.175+0100 [TRACE] statemgr.Filesystem: existing snapshot has lineage "bc89622d-6c6c-bc3e-5377-d1a028bb5905" serial 7
2023-08-25T17:22:33.175+0100 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2023-08-25T17:22:33.175+0100 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2023-08-25T17:22:33.176+0100 [TRACE] backend/local: reading remote state for workspace "default"
2023-08-25T17:22:33.176+0100 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2023-08-25T17:22:33.179+0100 [TRACE] statemgr.Filesystem: read snapshot with lineage "bc89622d-6c6c-bc3e-5377-d1a028bb5905" serial 7
2023-08-25T17:22:33.179+0100 [TRACE] backend/local: populating backend.LocalRun for current working directory

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

runtime error: invalid memory address or nil pointer dereference
goroutine 16 [running]:
runtime/debug.Stack()
	/Users/runner/hostedtoolcache/go/1.20.7/x64/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
	/Users/runner/hostedtoolcache/go/1.20.7/x64/src/runtime/debug/stack.go:16 +0x19
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x153
panic({0x3048960, 0x4e09260})
	/Users/runner/hostedtoolcache/go/1.20.7/x64/src/runtime/panic.go:884 +0x213
github.com/hashicorp/hcl/v2/ext/typeexpr.getType({0x3a03020, 0xc000224c30}, 0x1, 0x1)
	/Users/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.16.2/ext/typeexpr/get_type.go:261 +0x3b86
github.com/hashicorp/hcl/v2/ext/typeexpr.getType({0x3a03020, 0xc000224d20}, 0x1, 0x1)
	/Users/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.16.2/ext/typeexpr/get_type.go:151 +0x1c1b
github.com/hashicorp/hcl/v2/ext/typeexpr.TypeConstraintWithDefaults(...)
	/Users/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.16.2/ext/typeexpr/public.go:42
github.com/hashicorp/terraform/internal/configs.decodeVariableType({0x3a03020?, 0xc000224d20})
	/Users/runner/work/terraform/terraform/internal/configs/named_values.go:255 +0x17b
github.com/hashicorp/terraform/internal/configs.decodeVariableBlock(0xc000d56dd0, 0x0)
	/Users/runner/work/terraform/terraform/internal/configs/named_values.go:109 +0x956
github.com/hashicorp/terraform/internal/configs.(*Parser).loadConfigFile(0xc000a7f3a0, {0xc000cc0720?, 0x4?}, 0x4?)
	/Users/runner/work/terraform/terraform/internal/configs/parser_config.go:121 +0x107c
github.com/hashicorp/terraform/internal/configs.(*Parser).LoadConfigFile(...)
	/Users/runner/work/terraform/terraform/internal/configs/parser_config.go:25
github.com/hashicorp/terraform/internal/configs.(*Parser).loadFiles(0x3409e56?, {0xc0001ead80?, 0x5, 0x15?}, 0x0)
	/Users/runner/work/terraform/terraform/internal/configs/parser_config_dir.go:80 +0x108
github.com/hashicorp/terraform/internal/configs.(*Parser).LoadConfigDir(0x314fee0?, {0xc00019bbf4, 0x2})
	/Users/runner/work/terraform/terraform/internal/configs/parser_config_dir.go:40 +0xb3
github.com/hashicorp/terraform/internal/configs/configload.(*Loader).moduleWalkerLoad(0xc0003bc410, 0xc000369d40)
	/Users/runner/work/terraform/terraform/internal/configs/configload/loader_load.go:98 +0x5e9
github.com/hashicorp/terraform/internal/configs/configload.(*Loader).makeModuleWalkerSnapshot.func1(0xc000369d40)
	/Users/runner/work/terraform/terraform/internal/configs/configload/loader_snapshot.go:137 +0x51
github.com/hashicorp/terraform/internal/configs.ModuleWalkerFunc.LoadModule(0x301f860?, 0xc000cd8a80?)
	/Users/runner/work/terraform/terraform/internal/configs/config_build.go:144 +0x1f
github.com/hashicorp/terraform/internal/configs.buildChildModules(0xc000a60460, {0x39e67a0, 0xc000729a40})
	/Users/runner/work/terraform/terraform/internal/configs/config_build.go:73 +0x4d5
github.com/hashicorp/terraform/internal/configs.BuildConfig(0xc000cd6f70, {0x39e67a0, 0xc000729a40})
	/Users/runner/work/terraform/terraform/internal/configs/config_build.go:28 +0x7c
github.com/hashicorp/terraform/internal/configs/configload.(*Loader).LoadConfigWithSnapshot(0xc0003bc410, {0x39d20d0, 0x1})
	/Users/runner/work/terraform/terraform/internal/configs/configload/loader_snapshot.go:34 +0xb9
github.com/hashicorp/terraform/internal/backend/local.(*Local).localRunDirect(0x34d452a?, 0xc00035d440, 0xc000c9bcb0, 0x34cf1dc?, {0x3a09f50, 0xc000a5dd00})
	/Users/runner/work/terraform/terraform/internal/backend/local/backend_local.go:138 +0x73
github.com/hashicorp/terraform/internal/backend/local.(*Local).localRun(0xc000380000, 0xc00035d440)
	/Users/runner/work/terraform/terraform/internal/backend/local/backend_local.go:101 +0x7ff
github.com/hashicorp/terraform/internal/backend/local.(*Local).opPlan(0xc000380000, {0x3a026b8, 0xc00098fcc0}, {0x3a026b8, 0xc00098fd10}, 0xc00035d440, 0xc00092d140)
	/Users/runner/work/terraform/terraform/internal/backend/local/backend_plan.go:80 +0x509
github.com/hashicorp/terraform/internal/backend/local.(*Local).Operation.func1()
	/Users/runner/work/terraform/terraform/internal/backend/local/backend.go:330 +0xc3
created by github.com/hashicorp/terraform/internal/backend/local.(*Local).Operation
	/Users/runner/work/terraform/terraform/internal/backend/local/backend.go:323 +0x44a

Expected Behavior

Code (config) is obviously invalid and happened due to my copy-paste error.

I would expect some kind of syntax error though, not a fatal crash.

Actual Behavior

Crash.

Steps to Reproduce

1. terraform init

Additional Context

No response

References

No response

[apparentlymart]

Hi @weakcamel! Sorry for this crash, and thanks for reporting it.

Thanks in particular for including the trace logs; the early parts of that are particularly useful in situations like these because it allows us to quickly see exactly which versions of important dependencies like HCL are compiled into the Terraform release you are using, which allowed me to quickly narrow down that crash to some code that's trying to report an error with the default value of one of the attributes:

https://github.com/hashicorp/hcl/blob/e54a1960efd6cdfe35ecb8cc098bed33cd6001a8/ext/typeexpr/get_type.go#L254-L267

I think what's happened here is that because you have a duplicate attribute name there's already a map entry for defaultValues[attrName] from the first iteration that had a default value, but this iteration didn't have a default value argument and so defaultExpr is nil here.

But the root problem is that it's missing a check near the top of the loop to see if that name was already declared: https://github.com/hashicorp/hcl/blob/e54a1960efd6cdfe35ecb8cc098bed33cd6001a8/ext/typeexpr/get_type.go#L181

I think it should be checking to make sure there isn't already an entry of the same name in atys before it tries to do anything else.

This is actually a HCL bug rather than a Terraform bug, but since HCL is mostly maintained by the Terraform team I think we can keep this tracked here for now until someone is ready to work on a fix, at which point we can decide whether it's worth opening an issue in the HCL repository too or if this is straightforward enough to go straight to a PR -- I suspect it is.

[liamcervante]

I've put together hashicorp/hcl#624 to address this.

[weakcamel]

@apparentlymart @liamcervante
Many thanks for the detailed explanation and the quick fix! Much appreciated.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.