Custom expression decoding in function calls and hcldec specs

[apparentlymart]

These changes include some new capabilities for those building languages on top of HCL, but I'm going to start by talking about the main motivating use-cases that led me here, which all HCL functions that require some special handling for one or more of their arguments:

• convert(value, type) for generalized type conversion using a type expression instead of a value expression in the second argument, like convert([], set(string)). This can be useful when a value is being used as part of defining an API, like in Terraform Output Values, where we can be explicit about what result type we're intending and see an error if the value is not sufficiently close to that type to be converted successfully. (Terraform does already have functions like toset(...) which get partway there, but they don't allow explicitly specifying element/attribute types.)
• try(expr, expr...) for trying multiple expressions in sequence and taking the value of the first one that succeeds. This one is primarily useful for working with complex data structures of an unknown shape, for similar reasons as discussed in lang/funcs: add jsonpath function terraform#22460 but without introducing a whole new traversal syntax into the language.
• can(expr), which is related to try but allows using the success or failure of the given expression as a boolean value to make other decisions.

These functions are all defined in extension packages, so merging them will not cause any change in behavior to any existing HCL caller immediately, but will allow each application to selectively opt-in to these if desired.

---

The underlying mechanism here is taking some inspiration from what is possible with both the low-level HCL API and with gohcl, where it's possible to access the structural hcl.Expression directly and perform arbitrary analyses on it either instead of or prior to evaluating it.

That sort of technique was previously unavailable in any situation which deals in cty.Value results, because there was no way to opt in to that special decoding in those contexts. Using an extension mechanism built in to cty's "capsule types" mechanism, this introduces a new convention (whose public API is in ext/customdecode) of specifying a specially-annotated capsule type as a type constraint for an argument. This is approximately analogous to using custom named types and struct field tags in gohcl, but it's handled completely at runtime within cty's type system instead.

There are some higher-level helpers here aiming to see that for common use-cases calling applications won't need to work directly with that low-level convention and can instead just work with cty types or cty functions already defined here for convenient use.

Perhaps the most interesting building block, which is the foundation of both try(...) and can(...), is the customdecode.ExpressionClosureType capsule type: it allows any argument using it as a type constraint to capture both the physical expression and the EvalContext that was passed to evaluate it. That means a function using this mechanism can delay evaluation of the expression while still retaining all of the same variables and functions that were available to it at original evaluation.

By analogy to the gohcl features using special types and struct tags, this custom decoding only applies to "argument-like" contexts, which for our purposes here is defined as the following two locations:

• hcldec attribute specifications whose type constraints are suitably-annotated capsule types, likewise allowing an attribute expression to be treated as raw syntax rather than as a value. (This is the closest analog to the equivalent gohcl capabilities, which Terraform uses for its special arguments like depends_on, input variable type arguments, etc.)
• Arguments to function calls, where the corresponding parameter has a type constraint that is a suitably-annotated capsule type, allowing a function to treat one of its arguments as raw syntax rather than as a value. This is a new capability, but is conceptually similar to custom handling of attribute expressions.

Applications that have no need for these special capabilities can completely ignore them, by not importing any of the extension packages defined here. Although there are some small modifications to function call handling and hcldec decoding, those codepaths cannot be visited unless the calling program activates them through the featuers of these extension packages.

[mildwonkey]

LGTM! It took longer to peruse the linked issues to make sure I understood the use case than review the code itself :)