global secondary index always recreated

This issue was originally opened by @umeat as hashicorp/terraform#17600. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.11.3

Terraform Configuration Files

resource "aws_dynamodb_table" "file-index-table" {
  name           = "${var.application}-file-index-${var.environment}"
  hash_key       = "StationId"
  range_key      = "QueryString"
  read_capacity  = 30
  write_capacity = 30

  attribute {
    name = "StationId"
    type = "S"
  }

  attribute {
    name = "QueryString"
    type = "S"
  }

  attribute {
    name = "EventId"
    type = "N"
  }

  attribute {
    name = "Position"
    type = "N"
  }

  global_secondary_index {
    name               = "EventIdIndex"
    hash_key           = "EventId"
    range_key          = "Position"
    write_capacity     = 20
    read_capacity      = 20
    projection_type    = "INCLUDE"
    non_key_attributes = ["StationId", "QueryString"]
  }

  tags {
    Name        = "File-Index-Table"
    Environment = "${var.environment}"
  }
}

Behavior

The global secondary index is destroyed and recreated when no changes have been made to it.

module.file-index.aws_dynamodb_table.file-index-table: Modifying... (ID: archive-file-index-dev)
  global_secondary_index.593012552.hash_key:             "" => "EventId"
  global_secondary_index.593012552.name:                 "" => "EventIdIndex"
  global_secondary_index.593012552.non_key_attributes.#: "0" => "2"
  global_secondary_index.593012552.non_key_attributes.0: "" => "StationId"
  global_secondary_index.593012552.non_key_attributes.1: "" => "QueryString"
  global_secondary_index.593012552.projection_type:      "" => "INCLUDE"
  global_secondary_index.593012552.range_key:            "" => "Position"
  global_secondary_index.593012552.read_capacity:        "" => "20"
  global_secondary_index.593012552.write_capacity:       "" => "20"
  global_secondary_index.611285021.hash_key:             "EventId" => ""
  global_secondary_index.611285021.name:                 "EventIdIndex" => ""
  global_secondary_index.611285021.non_key_attributes.#: "2" => "0"
  global_secondary_index.611285021.non_key_attributes.0: "QueryString" => ""
  global_secondary_index.611285021.non_key_attributes.1: "StationId" => ""
  global_secondary_index.611285021.projection_type:      "INCLUDE" => ""
  global_secondary_index.611285021.range_key:            "Position" => ""
  global_secondary_index.611285021.read_capacity:        "20" => "0"
  global_secondary_index.611285021.write_capacity:       "20" => "0"

[jmettes]

+1

[miesvanderrobot]

Seeing same behavior here on Terraform 0.10.7

[laurams]

+1

[pgieser]

Still seeing this behavior on 0.11.7

[ctippur]

Same here .. Observed this in Terraform v0.11.7.

[alekbarszczewski]

Same issue here - is someone planning to fix this?

[ctippur]

Yes. It is causing quite a few inconsistencies. Terraform times out creating indexes. :(

[alekbarszczewski]

Guys are you planning to fix this?

[ctippur]

Any updates on this please?

[stevebreault]

If you look at the output above, you'll notice that the order of the non_key_attributes is different in the second half of the block.

module.file-index.aws_dynamodb_table.file-index-table: Modifying... (ID: archive-file-index-dev)
global_secondary_index.593012552.hash_key:             "" => "EventId"
global_secondary_index.593012552.name:                 "" => "EventIdIndex"
global_secondary_index.593012552.non_key_attributes.#: "0" => "2"
global_secondary_index.593012552.non_key_attributes.0: "" => "StationId"          <-- 1
global_secondary_index.593012552.non_key_attributes.1: "" => "QueryString"        <-- 2
global_secondary_index.593012552.projection_type:      "" => "INCLUDE"
global_secondary_index.593012552.range_key:            "" => "Position"
global_secondary_index.593012552.read_capacity:        "" => "20"
global_secondary_index.593012552.write_capacity:       "" => "20"
global_secondary_index.611285021.hash_key:             "EventId" => ""
global_secondary_index.611285021.name:                 "EventIdIndex" => ""
global_secondary_index.611285021.non_key_attributes.#: "2" => "0"
global_secondary_index.611285021.non_key_attributes.0: "QueryString" => ""        <-- 2
global_secondary_index.611285021.non_key_attributes.1: "StationId" => ""          <-- 1
global_secondary_index.611285021.projection_type:      "INCLUDE" => ""
global_secondary_index.611285021.range_key:            "Position" => ""
global_secondary_index.611285021.read_capacity:        "20" => "0"
global_secondary_index.611285021.write_capacity:       "20" => "0"

If you change the order in your configuration file to match the order in the later part of the block, it will no longer try to recreate. That's what fixed it for me anyhow.

[davehowell]

@stevebreault nice spotting. Clearly the state file is storing sorted, and using that to compare against the unsorted list specified in the HIL.

I can add that it also does this for the aws_dynamodb_table_item resource that I am using to load a few initial values to a table. If you don't specify item argument with sorted attributes it thinks there is a change and updates to the exact same values. Very annoying when you have dynamodb streams or something similar triggered from the table.

The ordering of the attributes is not really important given all dynambodb values are named kv pairs, and the api responses give you json with named fields, so storing it sorted in the state file is fine, as long as the comparison is also done between sorted existing-state and sorted new-state.

[bam0382]

Related to #3807