feat(custom-resources): make physical resource id optional (provider framework) (#4946)

In order to make it easier to get started and implement custom resources that do not require changes to physical resource IDs, the provider framework now allows `onEvent` to omit the `PhysicalResourceId` return value.

For `CREATE` operations, it will default to the `RequestId`. For `UPDATE` and `DELETE` it will return the current `PhysicalResourceId`.

Misc: in aws-custom-resource, use `fs.readFileSync(__dirname)` instead of `require` to load `sdk-api-metadata.json`, so that the typescript compiler won't yell that this file is not defined in  tsconfig.json.

Author: Elad Ben-Israel

packages/@aws-cdk/aws-cloudformation/lib/custom-resource.ts

@@ -78,25 +78,32 @@ export class CustomResourceProvider implements ICustomResourceProvider {
  */
 export interface CustomResourceProps {
   /**
-   * The provider which implements the custom resource
-   *
-   * @example invoke an AWS Lambda function when a lifecycle event occurs
-   *
-   *    CustomResourceProvider.fromLambda(myFunction)
-   *
-   * @example publish lifecycle events to an SNS topic
-   *
-   *    CustomResourceProvider.fromTopic(myTopic)
-   *
-   * @example use the custom resource provider framework
-   *
-   *    import cr = require('@aws-cdk/custom-resources');
-   *    const myProvider = new cr.Provider(...)
-   *
-   *    new cfn.CustomResource(this, 'myResource', {
-   *      provider: myProvider
-   *    });
-   *
+   * The provider which implements the custom resource.
+   *
+   * You can implement a provider by listening to raw AWS CloudFormation events
+   * through an SNS topic or an AWS Lambda function or use the CDK's custom
+   * [resource provider framework] which makes it easier to implement robust
+   * providers.
+   *
+   * [resource provider framework]: https://docs.aws.amazon.com/cdk/api/latest/docs/custom-resources-readme.html
+   *
+   * ```ts
+   * // use the provider framework from aws-cdk/custom-resources:
+   * provider: new custom_resources.Provider({
+   *   onEventHandler: myOnEventLambda,
+   *   isCompleteHandler: myIsCompleteLambda, // optional
+   * });
+   * ```
+   *
+   * ```ts
+   * // invoke an AWS Lambda function when a lifecycle event occurs:
+   * provider: CustomResourceProvider.fromLambda(myFunction)
+   * ```
+   *
+   * ```ts
+   * // publish lifecycle events to an SNS topic:
+   * provider: CustomResourceProvider.fromTopic(myTopic)
+   * ```
    */
   readonly provider: ICustomResourceProvider;
 

packages/@aws-cdk/custom-resources/README.md

@@ -43,6 +43,7 @@ and powerful custom resources and includes the following capabilities:
   deployments
 * Validates handler return values to help with correct handler implementation
 * Supports asynchronous handlers to enable long operations which can exceed the AWS Lambda timeout
+* Implements default behavior for physical resource IDs.
 
 The following code shows how the `Provider` construct is used in conjunction
 with `cfn.CustomResource` and a user-provided AWS Lambda function which
@@ -109,7 +110,7 @@ The return value from `onEvent` must be a JSON object with the following fields:
 
 |Field|Type|Required|Description
 |-----|----|--------|-----------
-|`PhysicalResourceId`|String|Yes|The allocated/assigned physical ID of the resource. Must be returned for all request types, including `Update` and `Delete`, even if the physical ID hasn't changed.
+|`PhysicalResourceId`|String|No|The allocated/assigned physical ID of the resource. If omitted for `Create` events, the event's `RequestId` will be used. For `Update`, the current physical ID will be used. If a different value is returned, CloudFormation will follow with a subsequent `Delete` for the previous ID (resource replacement). For `Delete`, it will always return the current physical resource ID, and if the user returns a different one, an error will occur.
 |`Data`|JSON|No|Resource attributes, which can later be retrieved through `Fn::GetAtt` on the custom resource object.
 
 [Custom Resource Provider Request]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-ref-requests.html#crpg-ref-request-fields
@@ -132,9 +133,10 @@ with the message "Operation timed out".
 If an error is thrown, the framework will submit a "FAILED" response to AWS
 CloudFormation.
 
-The input event to `isComplete` is similar to [`onEvent`](#handling-lifecycle-events-onevent), 
-with an additional guarantee that `PhysicalResourceId` is defines and contains the value returned 
-from `onEvent`.
+The input event to `isComplete` is similar to
+[`onEvent`](#handling-lifecycle-events-onevent), with an additional guarantee
+that `PhysicalResourceId` is defines and contains the value returned from
+`onEvent` or the described default. At any case, it is guaranteed to exist.
 
 The return value must be a JSON object with the following fields:
 
@@ -148,28 +150,25 @@ The return value must be a JSON object with the following fields:
 Every resource in CloudFormation has a physical resource ID. When a resource is
 created, the `PhysicalResourceId` returned from the `Create` operation is stored
 by AWS CloudFormation and assigned to the logical ID defined for this resource
-in the template.
+in the template. If a `Create` operation returns without a `PhysicalResourceId`,
+the framework will use `RequestId` as the default. This is sufficient for
+various cases such as "pseudo-resources" which only query data.
 
-When an `Update` operation occurs, if the returned `PhysicalResourceId` is
-different from the one currently stored (and passed in the event through the
-`PhysicalResourceId` field), AWS CloudFormation will treat this as a **resource
-replacement**, and it will issue a subsequent `Delete` operation for the old
-resource.
+For `Update` and `Delete` operations, the resource event will always include the
+current `PhysicalResourceId` of the resource.
+
+When an `Update` operation occurs, the default behavior is to return the current
+physical resource ID. if the `onEvent` returns a `PhysicalResourceId` which is
+different from the current one, AWS CloudFormation will treat this as a
+**resource replacement**, and it will issue a subsequent `Delete` operation for
+the old resource.
 
 As a rule of thumb, if your custom resource supports configuring a physical name
 (e.g. you can specify a `BucketName` when you define an `AWS::S3::Bucket`), you
 must return this name in `PhysicalResourceId` and make sure to handle
 replacement properly. The `S3File` example demonstrates this
 through the `objectKey` property.
 
-If your custom resource doesn't support configuring a physical name for the
-resource, it is safe to use `RequestId` as `PhysicalResourceId` in the `Create`
-operation, but you must return the same value in subsequent `Update` operations,
-or otherwise CloudFormation will think your resource is being replaced and will
-issue a `Delete` event. The `S3Assert` example demonstrates this by returning
-`event.PhysicalResourceId` if defined (in `Update` and `Delete`) and otherwise
-`event.RequestId` (for `Create`).
-
 ### Error Handling
 
 As mentioned above, if any of the user handlers fail (i.e. throws an exception)

packages/@aws-cdk/custom-resources/lib/aws-custom-resource/aws-custom-resource.ts

@@ -2,8 +2,12 @@ import { CustomResource, CustomResourceProvider } from '@aws-cdk/aws-cloudformat
 import iam = require('@aws-cdk/aws-iam');
 import lambda = require('@aws-cdk/aws-lambda');
 import cdk = require('@aws-cdk/core');
+import fs = require('fs');
 import path = require('path');
-import metadata = require('./sdk-api-metadata.json');
+
+// don't use "require" since the typescript compiler emits errors since this
+// file is not listed in tsconfig.json.
+const metadata = JSON.parse(fs.readFileSync(path.join(__dirname, 'sdk-api-metadata.json'), 'utf-8'));
 
 /**
  * AWS SDK service metadata.

packages/@aws-cdk/custom-resources/lib/provider-framework/runtime/framework.ts

@@ -134,23 +134,47 @@ function createResponseEvent(cfnRequest: AWSLambda.CloudFormationCustomResourceE
   //
   // validate that onEventResult always includes a PhysicalResourceId
 
-  if (!onEventResult || !onEventResult.PhysicalResourceId) {
-    throw new Error(`onEvent response must include a PhysicalResourceId for all request types`);
-  }
+  onEventResult = onEventResult || { };
+
+  // if physical ID is not returned, we have some defaults for you based
+  // on the request type.
+  const physicalResourceId = onEventResult.PhysicalResourceId || defaultPhysicalResourceId(cfnRequest);
 
   // if we are in DELETE and physical ID was changed, it's an error.
-  if (cfnRequest.RequestType === 'Delete' && onEventResult.PhysicalResourceId !== cfnRequest.PhysicalResourceId) {
+  if (cfnRequest.RequestType === 'Delete' && physicalResourceId !== cfnRequest.PhysicalResourceId) {
     throw new Error(`DELETE: cannot change the physical resource ID from "${cfnRequest.PhysicalResourceId}" to "${onEventResult.PhysicalResourceId}" during deletion`);
   }
 
   // if we are in UPDATE and physical ID was changed, it's a replacement (just log)
-  if (cfnRequest.RequestType === 'Update' && onEventResult.PhysicalResourceId !== cfnRequest.PhysicalResourceId) {
+  if (cfnRequest.RequestType === 'Update' && physicalResourceId !== cfnRequest.PhysicalResourceId) {
     log(`UPDATE: changing physical resource ID from "${cfnRequest.PhysicalResourceId}" to "${onEventResult.PhysicalResourceId}"`);
   }
 
   // merge request event and result event (result prevails).
   return {
     ...cfnRequest,
-    ...onEventResult
+    ...onEventResult,
+    PhysicalResourceId: physicalResourceId,
   };
+}
+
+/**
+ * Calculates the default physical resource ID based in case user handler did
+ * not return a PhysicalResourceId.
+ *
+ * For "CREATE", it uses the RequestId.
+ * For "UPDATE" and "DELETE" and returns the current PhysicalResourceId (the one provided in `event`).
+ */
+function defaultPhysicalResourceId(req: AWSLambda.CloudFormationCustomResourceEvent): string {
+  switch (req.RequestType) {
+    case 'Create':
+      return req.RequestId;
+
+    case 'Update':
+    case 'Delete':
+      return req.PhysicalResourceId;
+
+    default:
+      throw new Error(`Invalid "RequestType" in request "${JSON.stringify(req)}"`);
+  }
 }

packages/@aws-cdk/custom-resources/lib/provider-framework/types.d.ts

@@ -65,8 +65,11 @@ interface OnEventResponse {
    *   deleted, and CloudFormation will immediately send a `Delete` event with
    *   the old physical ID.
    * - For `Delete`, this must be the same value received in the event.
+   *
+   * @default - for "Create" requests, defaults to the event's RequestId, for
+   * "Update" and "Delete", defaults to the current `PhysicalResourceId`.
    */
-  readonly PhysicalResourceId: string;
+  readonly PhysicalResourceId?: string;
 
   /**
    * Resource attributes to return.