Skip to content

Commit becb48e

Browse files
davidfowlCopilot
andauthored
Resolve cross-compute-environment endpoint references for Foundry hosted agents (#17756)
* Resolve cross-compute-environment endpoint references to Foundry hosted agents Fixes #17749. When a resource deployed to App Service, Azure Container Apps, or Kubernetes used WithReference() to reference a Foundry hosted agent, publishing failed because the publisher resolved the endpoint against its own local endpoint map, which does not contain the agent (deployed to the Foundry project compute environment). Introduce a shared ComputeEnvironmentEndpointResolver that, when an endpoint's owning resource is deployed to a different compute environment than the current publisher, delegates resolution to that owning environment's GetEndpointPropertyExpression. The three compute-environment publishers now call it in both the EndpointReference and EndpointReferenceExpression branches. Azure Front Door and the Foundry hosted-agent resolver are refactored onto the same shared lookup. AzureCognitiveServicesProjectResource gets a GetEndpointPropertyExpression override because the agent address is already a full https URL; the default scheme://host composition would produce a malformed double-scheme value. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add branch tests for ComputeEnvironmentEndpointResolver and correct misleading comment Add direct unit tests covering each branch of TryGetCrossEnvironmentEndpointExpression: cross-environment delegation, same-environment deployment target, WithComputeEnvironment binding backstop, no-compute-environment, bound multi-target (no throw), and unbound multi-target (throws). Correct the comment on the fast-path loop which incorrectly claimed it never throws on multi-target resources. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove redundant fast-path loop from ComputeEnvironmentEndpointResolver The first foreach loop using GetDeploymentTargetAnnotation(current) was redundant with TryGetEffectiveComputeEnvironment + the ReferenceEquals backstop for all well-formed inputs, and did not provide multi-target throw-safety. Remove it and keep the simpler resolve-then- compare flow. All six branch tests continue to pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop params and move out parameter last in cross-env endpoint resolver Replace the params IComputeEnvironmentResource?[] with an explicit IReadOnlyList parameter placed before the out, so the out parameter comes last per convention. Kubernetes still passes two current environments (its environment plus OwningComputeEnvironment) via a collection expression; ACA and AppService pass a single-element list. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add EndpointReference overload for cross-env endpoint resolver Add an overload taking EndpointReference that uses ep.Property(EndpointProperty.Url) internally, so the three EndpointReference call sites (ACA, AppService, Kubernetes) pass the endpoint directly instead of repeating the .Property(Url) projection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use hosted-agent deployment name in cross-env Foundry agent URL Hosted-agent deployment creates the Foundry agent version using the wrapper AzureHostedAgentResource.Name (e.g. "agent-ha" for a target named "agent"). The published cross-environment endpoint path was built from the bare resource name, producing /agents/agent which does not match the deployed /agents/agent-ha. Resolve the hosted-agent deployment target and use its name when present, falling back to the resource name for non-hosted agents. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Auto-wire Azure AI User role for hosted-agent consumers When a compute resource references a Foundry hosted agent's node app via WithReference, automatically grant the consumer the Azure AI User role on the owning Foundry account and provision a managed identity, removing the two manual post-deploy `az role assignment create` steps. Introduces a public, experimental ReferenceRoleAssignmentAnnotation in Aspire.Hosting.Azure. A resource that "fronts" an Azure resource (without being an IAzureResource itself) carries this annotation; AzureResourcePreparer folds its (Target, Roles) into the same role-assignment path used for direct Azure references. Foundry's AsHostedAgent stamps the annotation on the agent's node app granting only the least-privilege Azure AI User role; account defaults and explicit WithRoleAssignments suppression remain owned by the preparer and are not reintroduced. GetAllRoleAssignments now dedupes roles per target to avoid colliding bicep role-assignment identifiers. Adds preparer end-to-end tests (suppression preserved, defaults preserved, dedup) and Foundry stamp tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Regenerate cross-env Foundry snapshots for RBAC auto-wiring The cross-compute-environment Foundry hosted-agent snapshots were committed before the RBAC auto-wiring change and never regenerated. With the consumer now receiving a managed identity (web-identity) plus AZURE_CLIENT_ID and AZURE_TOKEN_CREDENTIALS env vars, the generated bicep/json changed. CI failed because the verified baselines were stale; local runs masked it because Verify auto-accepts on developer machines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent e138509 commit becb48e

26 files changed

Lines changed: 1264 additions & 25 deletions

src/Aspire.Hosting.Azure.AppContainers/Aspire.Hosting.Azure.AppContainers.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
<ItemGroup>
1212
<Compile Include="$(SharedDir)AzurePortalUrls.cs" Link="AzurePortalUrls.cs" />
1313
<Compile Include="$(SharedDir)BicepFormattingHelpers.cs" LinkBase="Shared\BicepFormattingHelpers.cs" />
14+
<Compile Include="$(SharedDir)ComputeEnvironmentEndpointResolver.cs" LinkBase="Shared\ComputeEnvironmentEndpointResolver.cs" />
1415
<Compile Include="$(SharedDir)ContainerRegistryInfrastructure.cs" LinkBase="Shared\ContainerRegistryInfrastructure.cs" />
1516
<Compile Include="$(SharedDir)ResourceNameComparer.cs" LinkBase="Shared\ResourceNameComparer.cs" />
1617
</ItemGroup>

src/Aspire.Hosting.Azure.AppContainers/BaseContainerAppContext.cs

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,15 @@ BicepValue<string> GetHostValue(string? prefix = null, string? suffix = null)
222222

223223
if (value is EndpointReference ep)
224224
{
225+
// The referenced endpoint may belong to a resource deployed to a different compute
226+
// environment (for example a Foundry hosted agent). In that case delegate to the owning
227+
// compute environment instead of looking it up in this environment's local endpoint map.
228+
if (ComputeEnvironmentEndpointResolver.TryGetCrossEnvironmentEndpointExpression(
229+
ep, [_containerAppEnvironmentContext.Environment], out var crossExpr))
230+
{
231+
return ProcessValue(crossExpr, secretType, parent);
232+
}
233+
225234
var context = ep.Resource == resource
226235
? this
227236
: _containerAppEnvironmentContext.GetContainerAppContext(ep.Resource);
@@ -274,6 +283,12 @@ BicepValue<string> GetHostValue(string? prefix = null, string? suffix = null)
274283

275284
if (value is EndpointReferenceExpression epExpr)
276285
{
286+
if (ComputeEnvironmentEndpointResolver.TryGetCrossEnvironmentEndpointExpression(
287+
epExpr, [_containerAppEnvironmentContext.Environment], out var crossExpr))
288+
{
289+
return ProcessValue(crossExpr, secretType, parent);
290+
}
291+
277292
var context = epExpr.Endpoint.Resource == resource
278293
? this
279294
: _containerAppEnvironmentContext.GetContainerAppContext(epExpr.Endpoint.Resource);

src/Aspire.Hosting.Azure.AppService/Aspire.Hosting.Azure.AppService.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
<ItemGroup>
1414
<Compile Include="$(SharedDir)AzurePortalUrls.cs" Link="AzurePortalUrls.cs" />
1515
<Compile Include="$(SharedDir)BicepFormattingHelpers.cs" LinkBase="Shared\BicepFormattingHelpers.cs" />
16+
<Compile Include="$(SharedDir)ComputeEnvironmentEndpointResolver.cs" LinkBase="Shared\ComputeEnvironmentEndpointResolver.cs" />
1617
<Compile Include="$(SharedDir)ContainerRegistryInfrastructure.cs" LinkBase="Shared\ContainerRegistryInfrastructure.cs" />
1718
<Compile Include="$(SharedDir)DashboardConfigNames.cs" LinkBase="Shared\DashboardConfigNames.cs" />
1819
<Compile Include="$(SharedDir)KnownConfigNames.cs" LinkBase="Shared\KnownConfigNames.cs" />

src/Aspire.Hosting.Azure.AppService/AzureAppServiceWebsiteContext.cs

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,15 @@ private void ProcessEndpoints()
167167

168168
if (value is EndpointReference ep)
169169
{
170+
// The referenced endpoint may belong to a resource deployed to a different compute
171+
// environment (for example a Foundry hosted agent). In that case delegate to the owning
172+
// compute environment instead of looking it up in this environment's local endpoint map.
173+
if (ComputeEnvironmentEndpointResolver.TryGetCrossEnvironmentEndpointExpression(
174+
ep, [environmentContext.Environment], out var crossExpr))
175+
{
176+
return ProcessValue(crossExpr, secretType, parent, isSlot);
177+
}
178+
170179
var context = environmentContext.GetAppServiceContext(ep.Resource);
171180
return isSlot ?
172181
(GetEndpointValue(context._slotEndpointMapping[ep.EndpointName], EndpointProperty.Url), secretType) :
@@ -206,6 +215,12 @@ private void ProcessEndpoints()
206215

207216
if (value is EndpointReferenceExpression epExpr)
208217
{
218+
if (ComputeEnvironmentEndpointResolver.TryGetCrossEnvironmentEndpointExpression(
219+
epExpr, [environmentContext.Environment], out var crossExpr))
220+
{
221+
return ProcessValue(crossExpr, secretType, parent, isSlot);
222+
}
223+
209224
var context = environmentContext.GetAppServiceContext(epExpr.Endpoint.Resource);
210225
var mapping = isSlot ? context._slotEndpointMapping[epExpr.Endpoint.EndpointName] : context._endpointMapping[epExpr.Endpoint.EndpointName];
211226
var val = GetEndpointValue(mapping, epExpr.Property);

src/Aspire.Hosting.Azure.FrontDoor/Aspire.Hosting.Azure.FrontDoor.csproj

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@
1515
<PackageReference Include="Azure.Provisioning.Cdn" />
1616
</ItemGroup>
1717

18+
<ItemGroup>
19+
<Compile Include="$(SharedDir)ComputeEnvironmentEndpointResolver.cs" LinkBase="Shared\ComputeEnvironmentEndpointResolver.cs" />
20+
</ItemGroup>
21+
1822
<ItemGroup>
1923
<InternalsVisibleTo Include="Aspire.Hosting.Azure.Tests" />
2024
</ItemGroup>

src/Aspire.Hosting.Azure.FrontDoor/AzureFrontDoorExtensions.cs

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -201,16 +201,11 @@ public static IResourceBuilder<AzureFrontDoorResource> WithOrigin<T>(
201201

202202
private static IComputeEnvironmentResource GetEffectiveComputeEnvironment(IResource resource)
203203
{
204-
if (resource.GetComputeEnvironment() is { } computeEnvironment)
204+
if (ComputeEnvironmentEndpointResolver.TryGetEffectiveComputeEnvironment(resource, out var computeEnvironment))
205205
{
206206
return computeEnvironment;
207207
}
208208

209-
if (resource.GetDeploymentTargetAnnotation()?.ComputeEnvironment is { } deploymentComputeEnvironment)
210-
{
211-
return deploymentComputeEnvironment;
212-
}
213-
214209
throw new InvalidOperationException(
215210
$"Resource '{resource.Name}' does not have a compute environment. " +
216211
"Ensure a compute environment (e.g., Azure Container Apps, Azure App Service) is configured in the application model.");

src/Aspire.Hosting.Azure/AzureResourcePreparer.cs

Lines changed: 44 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,8 @@ private async Task BuildRoleAssignmentAnnotations(DistributedApplicationModel ap
133133
foreach (var resource in resourceSnapshot)
134134
{
135135
var prerequisiteResources = new HashSet<AzureBicepResource>();
136-
var azureReferences = await GetAzureReferences(resource, cancellationToken).ConfigureAwait(false);
136+
var directDependencies = await resource.GetResourceDependenciesAsync(executionContext, ResourceDependencyDiscoveryMode.DirectOnly, cancellationToken).ConfigureAwait(false);
137+
var azureReferences = new HashSet<IAzureResource>(directDependencies.OfType<IAzureResource>());
137138

138139
var azureReferencesWithRoleAssignments =
139140
(resource.TryGetAnnotationsOfType<RoleAssignmentAnnotation>(out var annotations)
@@ -184,6 +185,42 @@ private async Task BuildRoleAssignmentAnnotations(DistributedApplicationModel ap
184185
}
185186
}
186187

188+
// A direct dependency that is not itself an Azure resource can still "front" one
189+
// (e.g. a Foundry hosted agent's node app fronts its owning Foundry account). Such a
190+
// resource carries ReferenceRoleAssignmentAnnotation(s) declaring that any resource
191+
// referencing it should be granted roles on a transitive Azure target the normal
192+
// IAzureResource-only reference walk above cannot reach. Fold those implied targets
193+
// into the same role-assignment path so the consumer gets an identity + role bicep
194+
// exactly as it would for a direct Azure reference.
195+
foreach (var dependency in directDependencies)
196+
{
197+
if (!dependency.TryGetAnnotationsOfType<ReferenceRoleAssignmentAnnotation>(out var impliedRoleAssignments))
198+
{
199+
continue;
200+
}
201+
202+
foreach (var impliedRoleAssignment in impliedRoleAssignments)
203+
{
204+
var target = impliedRoleAssignment.Target;
205+
if (target.IsContainer() || target.IsEmulator())
206+
{
207+
continue;
208+
}
209+
210+
if (executionContext.IsRunMode)
211+
{
212+
AppendGlobalRoleAssignments(globalRoleAssignments, target, impliedRoleAssignment.Roles);
213+
}
214+
else
215+
{
216+
// In PublishMode, materialize as an explicit RoleAssignmentAnnotation so
217+
// GetAllRoleAssignments (which groups by target and unions roles) picks it
218+
// up alongside any roles the consumer already declares for the same target.
219+
resource.Annotations.Add(new RoleAssignmentAnnotation(target, impliedRoleAssignment.Roles));
220+
}
221+
}
222+
}
223+
187224
// in PublishMode with SupportsTargetedRoleAssignments, we need to create the identity and role assignment resources
188225
// if the resource references any Azure resources, or has role assignments to Azure resources
189226
if (executionContext.IsPublishMode)
@@ -250,7 +287,12 @@ private static Dictionary<AzureProvisioningResource, IEnumerable<RoleDefinition>
250287
{
251288
foreach (var g in roleAssignments.GroupBy(r => r.Target))
252289
{
253-
result[g.Key] = g.SelectMany(r => r.Roles);
290+
// Deduplicate roles per target. A target can accumulate multiple RoleAssignmentAnnotations
291+
// (e.g. an implied ReferenceRoleAssignmentAnnotation from two hosted agents on the same
292+
// Foundry account, plus a direct reference). Emitting the same RoleDefinition twice would
293+
// produce two RoleAssignment bicep resources with the same identifier ("{prefix}_{roleName}")
294+
// and fail bicep compilation. This mirrors the RunMode path, which unions into a HashSet.
295+
result[g.Key] = g.SelectMany(r => r.Roles).Distinct();
254296
}
255297
}
256298
return result;
@@ -359,19 +401,6 @@ private sealed class AddRoleAssignmentsContext(
359401
public DistributedApplicationExecutionContext ExecutionContext => executionContext;
360402
}
361403

362-
private async Task<HashSet<IAzureResource>> GetAzureReferences(IResource resource, CancellationToken cancellationToken)
363-
{
364-
var dependencies = await resource.GetResourceDependenciesAsync(executionContext, ResourceDependencyDiscoveryMode.DirectOnly, cancellationToken).ConfigureAwait(false);
365-
366-
HashSet<IAzureResource> azureReferences = [];
367-
foreach (var azureResource in dependencies.OfType<IAzureResource>())
368-
{
369-
azureReferences.Add(azureResource);
370-
}
371-
372-
return azureReferences;
373-
}
374-
375404
private static void AppendGlobalRoleAssignments(Dictionary<AzureProvisioningResource, HashSet<RoleDefinition>> globalRoleAssignments, AzureProvisioningResource azureResource, IEnumerable<RoleDefinition> newRoles)
376405
{
377406
if (!globalRoleAssignments.TryGetValue(azureResource, out var existingRoles))
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
// Licensed to the .NET Foundation under one or more agreements.
2+
// The .NET Foundation licenses this file to you under the MIT license.
3+
4+
using System.Diagnostics.CodeAnalysis;
5+
using Aspire.Hosting.ApplicationModel;
6+
7+
namespace Aspire.Hosting.Azure;
8+
9+
/// <summary>
10+
/// Declares that any compute resource referencing the annotated resource should be granted
11+
/// <see cref="Roles"/> on the Azure resource <see cref="Target"/>.
12+
/// </summary>
13+
/// <param name="target">The Azure resource that referencing resources should be granted roles on.</param>
14+
/// <param name="roles">The roles that referencing resources should be assigned on <paramref name="target"/>.</param>
15+
/// <remarks>
16+
/// <para>
17+
/// This annotation is applied to a resource that "fronts" an Azure resource without being an
18+
/// <see cref="IAzureResource"/> itself. For example, a Foundry hosted agent's node app is a plain
19+
/// compute resource, but invoking the agent requires the caller to hold a role on the owning
20+
/// Foundry account. The account is only a transitive dependency of a consumer, so
21+
/// <see cref="AzureResourcePreparer"/>'s normal reference walk — which only acts on direct
22+
/// <see cref="IAzureResource"/> dependencies — cannot reach it.
23+
/// </para>
24+
/// <para>
25+
/// When a compute resource takes a direct dependency on a resource carrying this annotation,
26+
/// <see cref="AzureResourcePreparer"/> folds <c>(Target, Roles)</c> into the same role-assignment
27+
/// path used for direct Azure references, so the consumer gets a managed identity and the
28+
/// corresponding role assignment on <see cref="Target"/> with no additional wiring.
29+
/// </para>
30+
/// </remarks>
31+
[Experimental("ASPIREAZURE003", UrlFormat = "https://aka.ms/aspire/diagnostics/{0}")]
32+
public sealed class ReferenceRoleAssignmentAnnotation(AzureProvisioningResource target, IReadOnlySet<RoleDefinition> roles) : IResourceAnnotation
33+
{
34+
/// <summary>
35+
/// Gets the Azure resource that resources referencing the annotated resource should be granted roles on.
36+
/// </summary>
37+
public AzureProvisioningResource Target { get; } = target;
38+
39+
/// <summary>
40+
/// Gets the set of roles that resources referencing the annotated resource should be assigned on <see cref="Target"/>.
41+
/// </summary>
42+
public IReadOnlySet<RoleDefinition> Roles { get; } = roles;
43+
}

src/Aspire.Hosting.Foundry/Aspire.Hosting.Foundry.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
<ItemGroup>
2424
<Compile Include="$(RepoRoot)src\Shared\AzureRoleAssignmentUtils.cs" />
2525
<Compile Include="$(RepoRoot)src\Shared\ContainerRegistryInfrastructure.cs" />
26+
<Compile Include="$(SharedDir)ComputeEnvironmentEndpointResolver.cs" LinkBase="Shared\ComputeEnvironmentEndpointResolver.cs" />
2627
<Compile Include="$(RepoRoot)src\Aspire.Hosting\Utils\FormattingHelpers.cs" Link="Utils\FormattingHelpers.cs" />
2728
<Compile Include="..\Shared\AzureCredentialHelper.cs" Link="AzureCredentialHelper.cs" />
2829
<Compile Remove="tools\**\*.cs" />

src/Aspire.Hosting.Foundry/HostedAgent/AzureHostedAgentResource.cs

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,10 @@ namespace Aspire.Hosting.Foundry;
2525
/// </summary>
2626
public class AzureHostedAgentResource : Resource, IResourceWithEnvironment
2727
{
28-
private const string AzureAIUserRoleDefinitionId = "53ca6127-db72-4b80-b1b0-d745d6d5456d";
28+
// The "Azure AI User" built-in role (data-plane access to Foundry agents/inference). Granted to
29+
// the agent's own instance identity below, and to consumers that reference the agent (see
30+
// HostedAgentResourceBuilderExtensions.GrantHostedAgentConsumerRoles).
31+
internal const string AzureAIUserRoleDefinitionId = "53ca6127-db72-4b80-b1b0-d745d6d5456d";
2932

3033
/// <summary>
3134
/// Creates a new instance of the <see cref="AzureHostedAgentResource"/> class.
@@ -429,8 +432,7 @@ internal static async Task<Dictionary<string, string>> GetResolvedEnvironmentVar
429432
throw CreateEndpointResolutionException(hostedAgent, resource, environmentVariableName, endpointReference, $"Endpoint '{endpoint.Name}' is internal. Foundry hosted agents can only reference externally exposed endpoints during publish.");
430433
}
431434

432-
var deploymentTarget = endpointReference.Resource.GetDeploymentTargetAnnotation();
433-
if (deploymentTarget?.ComputeEnvironment is not { } computeEnvironment)
435+
if (!ComputeEnvironmentEndpointResolver.TryGetEffectiveComputeEnvironment(endpointReference.Resource, out var computeEnvironment))
434436
{
435437
var reason = $"Resource '{endpointReference.Resource.Name}' does not have a compute environment deployment target.";
436438
throw CreateEndpointResolutionException(hostedAgent, resource, environmentVariableName, endpointReference, reason);

0 commit comments

Comments
 (0)