Manage datashare consumer permissions to use datashares

[sworisbreathing]

Note: this PR builds off of (and thus includes the changes in) #18.

Adds a redshift_datashare_privilege resource which is used to grant usage permissions to consumer clusters and destination accounts. As with other datasharing related changes, this functionality only works on RA3 clusters.

For permissions in the same account:

resource "redshift_datashare_privilege" "within_account" {
  name = redshift_datashare.share.name # Required
  namespace = "d34dbe3f-d34d-b33f-d3ad-b33fd34db33f" # Required
}

For permissions across accounts:

resource "redshift_datashare_privilege" "within_account" {
  name = redshift_datashare.share.name # Required
  account = "123456789012" # Required
}

Notes on cross-account data sharing:

• cross-account data sharing is still in preview. Your cluster must be on the preview maintenance track, however there are 3 different preview track versions and I'm not sure which version enables cross-account data sharing.
• After the cross-account share is created, per the AWS docs, you have to authorize it in the AWS console. I've called this out in the documentation. If we wanted to automate it as part of the permission management, looks like it would involve making a call to redshift.AuthorizeDataShare, and would require some additional info to construct the data share arn needed for the request.
• I have written a test case for cross-account data sharing. I have not been able to run the test, however I believe what I've got is correct per the AWS docs.

Test cases are conditionally enabled on environment variables:

• REDSHIFT_DATASHARE_SUPPORTED - must be non-empty to enable either test
• REDSHIFT_DATASHARE_CONSUMER_NAMESPACE - must be a valid cluster namespace guid to enable the test for sharing within the same account
• REDSHIFT_DATASUARE_CONSUMER_ACCOUNT - must be a valid AWS account ID to enable the test for sharing across accounts.

The test case for data sharing within the same AWS account passes, and using a local build from this PR, I am able to successfully run a manual end-to-end test case of data sharing between two RA3 clusters in the same account.

[sworisbreathing]

here is the terraform template for the end-to-end test case. You will need to populate the variables accordingly.

# Variables for provider configuration
variable "producer_cluster_host" {
  type = string
}
variable "producer_cluster_port" {
  type = number
}
variable "producer_cluster_database" {
  type = string
}
variable "consumer_cluster_host" {
  type = string
}
variable "consumer_cluster_port" {
  type = number
}
variable "consumer_cluster_database" {
  type = string
}

terraform {
  required_providers {
    redshift = {
      # version info is omitted as this is a local build using provider development overrides
      # see https://www.terraform.io/docs/cli/config/config-file.html#development-overrides-for-provider-developers
      source  = "brainly/redshift"
    }
  }
}

# Set up provider aliases for producer and consumer cluster
provider "redshift" {
  alias = "producer"
  host = var.producer_cluster_host
  port = var.producer_cluster_port
  database = var.producer_cluster_database
}
provider "redshift" {
  alias = "consumer"
  host = var.consumer_cluster_host
  port = var.consumer_cluster_port
  database = var.consumer_cluster_database
}

# Variables for resource configuration
variable "internal_schemas" {
  type = list(string)
}

variable "datashare_name" {
  type = string
}

variable "datashare_db_name" {
  type = string
}

# Define all of the internal schemas on the producer cluster
resource "redshift_schema" "producer_internal_schemas" {
  provider = redshift.producer
  for_each = toset(var.internal_schemas)
  name = each.key
}

# Define the datashare on the producer cluster,
# and add all internal schemas to it
resource "redshift_datashare" "producer_share" {
  provider = redshift.producer
  name = var.datashare_name
  schemas = [for s in redshift_schema.producer_internal_schemas : s.name]
}

data "redshift_namespace" "consumer" {
  provider = redshift.consumer
}

# Ensure the consumer cluster has permission to access the data share.
resource "redshift_datashare_privilege" "consumer" {
  provider = redshift.producer
  share_name = redshift_datashare.producer_share.name
  namespace = data.redshift_namespace.consumer.id
}

# Create an external database from the datashare
resource "redshift_database" "consumer_share" {
  provider = redshift.consumer
  name = var.datashare_db_name
  datashare_source {
    share_name = redshift_datashare.producer_share.name
    namespace = redshift_datashare.producer_share.producer_namespace
  }
  depends_on = [
    redshift_datashare_privilege.consumer
  ]
}

# Create external schemas from the datashare on the consumer cluster
resource "redshift_schema" "consumer_share_schemas" {
  for_each = toset(redshift_datashare.producer_share.schemas)
  provider = redshift.consumer
  name = each.key
  external_schema {
    database_name = redshift_database.consumer_share.name
    redshift_source {
      schema = each.key
    }
  }
}