feat: support network interface configuration by ryan-mist · Pull Request #9027 · aws/karpenter-provider-aws

ryan-mist · 2026-03-23T09:02:50Z

Fixes #N/A

Description

adds support for configuring network interfaces on the nodeclass

spec:
  networkInterfaces:
        - networkCardIndex: int
          deviceIndex: int
          interfaceType: "interface" | "efa-only"

How was this change tested?

unit tests
manual testing

Manual Tests

Drift

# Apply NodeClass with no network interface configuration
ryanmist@c889f3b6ff52 efa % k apply -f nodeclass.yaml
ec2nodeclass.karpenter.k8s.aws/efa-1 created
ryanmist@c889f3b6ff52 efa % k get ec2nodeclass
NAME    READY   AGE
efa-1   True    25s
ryanmist@c889f3b6ff52 efa % k apply -f nodepool.yaml
nodepool.karpenter.sh/i4g-16xlarge-pool created

# Create Node; Launched with No EFA Configurations
ryanmist@c889f3b6ff52 efa % k apply -f pod.yaml
deployment.apps/non-efa-deployment configured
ryanmist@c889f3b6ff52 efa % aws ec2 describe-instances --instance-ids i-030f2ec9b6b7104fa  --query "Reservations[].Instances[].NetworkInterfaces[].InterfaceType"
[
    "interface"
]

# Change NodeClass Configuration 
ryanmist@c889f3b6ff52 efa % k apply -f nodeclass.yaml
ec2nodeclass.karpenter.k8s.aws/efa-1 configured

# Node is Drifted
    Last Transition Time:  2026-03-23T07:34:36Z
    Message:               NodeClassDrift
    Observed Generation:   1
    Reason:                NodeClassDrift
    Status:                True
    Type:                  Drifted

# New Node Launched with EFA
ryanmist@c889f3b6ff52 efa % aws ec2 describe-instances --instance-ids i-0d2a8ae49f5ab9137  --query "Reservations[].Instances[].NetworkInterfaces[].InterfaceType"
[
    "efa-only",
    "interface"
]

Static Provisioning

# Apply NodeClass And Static NodePool
# Multiple NodeClaims provisioned
NAME                      TYPE           CAPACITY   ZONE         NODE   READY     AGE
i4g-16xlarge-pool-dk5sb   i4g.16xlarge   spot       us-west-2d          Unknown   16s
i4g-16xlarge-pool-r5t6h   i4g.16xlarge   spot       us-west-2d          Unknown   30s
# With Correct Allocatable / Capacity 
Status:
  Allocatable:
    vpc.amazonaws.com/efa:      1

Dynamic Provisioning

# When pods does not request EFA - instance is still launched with NodeClass config
ryanmist@c889f3b6ff52 efa % aws ec2 describe-instances --instance-ids i-00bca795a6864fbf7  --query "Reservations[].Instances[].NetworkInterfaces[].InterfaceType"
[
    "interface",
    "efa-only",
    "efa-only",
    "efa-only",
    "efa-only"
]
Allocatable:
    vpc.amazonaws.com/efa:      4

# When pods does request EFA - instance is launched with NodeClass config
ryanmist@c889f3b6ff52 efa % k apply -f efa-pod.yaml
deployment.apps/efa-deployment configured
Resources:
  Requests:
    Cpu:                    1150m
    Memory:                 100Mi
    Pods:                   4
    vpc.amazonaws.com/efa:  1
Status:
  Allocatable:
    Cpu:                        191450m
    Ephemeral - Storage:        17Gi
    Memory:                     1937871Mi
    nvidia.com/gpu:             8
    Pods:                       149
    vpc.amazonaws.com/efa:      4

# When pod requests over EFA resource of NodeClass
ryanmist@c889f3b6ff52 efa % k apply -f efa-pod.yaml
deployment.apps/five-efa-deployment created

From Karpenter Logs
{"level":"ERROR","time":"2026-03-23T08:47:09.824Z","logger":"controller","caller":"scheduling/scheduler.go:257","message":"could not schedule pod","commit":"cfc53f3-dirty","controller":"provisioner","namespace":"","name":"","reconcileID":"38b08750-bb1b-4ed3-9929-d5687cfd95f2","Pod":{"name":"five-efa-deployment-769b5785f6-7nqrs","namespace":"default"},"error":"no instance type has enough resources, requirements=karpenter.k8s.aws/ec2nodeclass In [efa-1], karpenter.sh/nodepool In [efa-pool], kubernetes.io/os In [linux], resources={\"cpu\":\"1150m\",\"memory\":\"100Mi\",\"pods\":\"4\",\"vpc.amazonaws.com/efa\":\"5\"}"}

# When pod requests EFA w/ NodeClass with no Network Interface Configurations
ryanmist@c889f3b6ff52 efa % k apply -f efa-pod.yaml
deployment.apps/efa-deployment configured
ryanmist@c889f3b6ff52 efa % aws ec2 describe-instances --instance-ids i-035054426a7520613  --query "Reservations[].Instances[].NetworkInterfaces[].InterfaceType"
[
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa",
    "efa"
]

Does this change impact docs?

Yes, PR includes docs updates
Yes, issue opened: #
No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

nathangeology

Hope these comments help. Let me know if you have any questions.

github-actions · 2026-03-26T01:27:27Z

Preview deployment ready!

Preview URL: https://pr-9027.d18coufmbnnaag.amplifyapp.com

Built from commit 7979c6d7162c4fd851aa48ee350bc562071e8205

ryan-mist

/karpenter snapshot

github-actions · 2026-03-26T03:39:30Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-936b08d1884d1646c503ae66f451c7eb2a87d0c9.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-936b08d1884d1646c503ae66f451c7eb2a87d0c9" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-03-26T21:25:05Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-bbcdb60da8d671b2491de2e789ad84fd476a76ff.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-bbcdb60da8d671b2491de2e789ad84fd476a76ff" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-03-27T01:39:57Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-27fffb37a48dbadb9cbcc8e57c81bbd2e4da8b0e.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-27fffb37a48dbadb9cbcc8e57c81bbd2e4da8b0e" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-03-27T21:19:49Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-ff174646bcfc676c072f846dfe3bff5eb52945aa.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-ff174646bcfc676c072f846dfe3bff5eb52945aa" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-03-30T17:15:18Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-57b0329ec8c2dda0547dd43fdaf244f1aaa08eef.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-57b0329ec8c2dda0547dd43fdaf244f1aaa08eef" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

ryan-mist

/karpenter snapshot

ryan-mist

/karpenter snapshot

github-actions · 2026-03-31T09:14:55Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-74ef6f74363ff90bf2af58fac69f0c260c9faf0c.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-74ef6f74363ff90bf2af58fac69f0c260c9faf0c" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-03-31T22:16:30Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-930348c626cfa4137edbf96ec87c74e4bc72b95d.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-930348c626cfa4137edbf96ec87c74e4bc72b95d" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

ryan-mist

/karpenter snapshot

github-actions · 2026-04-01T06:49:09Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-7979c6d7162c4fd851aa48ee350bc562071e8205.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-7979c6d7162c4fd851aa48ee350bc562071e8205" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

jmdeal

LGTM 🚀

feat: support network interface configuration

ecb1e4f

ryan-mist force-pushed the network-interface-support branch from cfc53f3 to ecb1e4f Compare March 23, 2026 15:52

ryan-mist requested a review from a team as a code owner March 23, 2026 15:52

ryan-mist requested a review from azishabibi March 23, 2026 15:52

nathangeology reviewed Mar 24, 2026

View reviewed changes

validation fix

04bca56

jmdeal reviewed Mar 25, 2026

View reviewed changes

ryan-mist added 2 commits March 25, 2026 10:31

fixes based on comments

76c505b

efa count label triggers EFA provisioning

ced7850

Merge branch 'main' into network-interface-support

8a4ebc3

ryan-mist force-pushed the network-interface-support branch from de1c210 to 8a4ebc3 Compare March 26, 2026 01:33

e2e test

936b08d

ryan-mist commented Mar 26, 2026

View reviewed changes

docs

6eed7cb

ryan-mist commented Mar 26, 2026

View reviewed changes

jmdeal reviewed Mar 26, 2026

View reviewed changes

Comment thread pkg/apis/v1/ec2nodeclass.go

bump e2e test time

27fffb3

ryan-mist force-pushed the network-interface-support branch from bbcdb60 to 27fffb3 Compare March 27, 2026 01:33

ryan-mist commented Mar 27, 2026

View reviewed changes

ryan-mist added 2 commits March 27, 2026 13:37

Merge branch 'main' into network-interface-support

757845a

fix merging

ff17464

ryan-mist commented Mar 27, 2026

View reviewed changes

jamesmt-aws reviewed Mar 28, 2026

View reviewed changes

Comment thread pkg/providers/instancetype/compatibility/compatibility.go Outdated

jamesmt-aws reviewed Mar 28, 2026

View reviewed changes

Comment thread pkg/cloudprovider/suite_test.go Outdated

fix based on comments

57b0329

ryan-mist force-pushed the network-interface-support branch from b3e4a5e to 57b0329 Compare March 30, 2026 16:50

ryan-mist commented Mar 30, 2026

View reviewed changes

ryan-mist force-pushed the network-interface-support branch from 22cc9be to 097e3fe Compare March 31, 2026 08:48

remove efa count label

c31964e

ryan-mist force-pushed the network-interface-support branch from 097e3fe to c31964e Compare March 31, 2026 08:51

ryan-mist commented Mar 31, 2026

View reviewed changes

remove

74ef6f7

ryan-mist commented Mar 31, 2026

View reviewed changes

from ptr

c8c569d

jmdeal reviewed Mar 31, 2026

View reviewed changes

Comment thread pkg/providers/instance/instance.go Outdated

Comment thread pkg/providers/instancetype/types.go

Comment thread pkg/providers/launchtemplate/launchtemplate.go Outdated

ryan-mist added 2 commits March 31, 2026 14:35

fix comments

8cb0cec

combine CEL validation

930348c

ryan-mist commented Mar 31, 2026

View reviewed changes

Merge branch 'main' into network-interface-support

7979c6d

ryan-mist force-pushed the network-interface-support branch from efef3b7 to 7979c6d Compare April 1, 2026 06:20

ryan-mist commented Apr 1, 2026

View reviewed changes

jmdeal approved these changes Apr 1, 2026

View reviewed changes

ryan-mist enabled auto-merge (squash) April 1, 2026 18:34

ryan-mist disabled auto-merge April 1, 2026 18:34

ryan-mist enabled auto-merge (squash) April 1, 2026 18:34

ryan-mist disabled auto-merge April 1, 2026 18:34

ryan-mist merged commit 1de6d8c into aws:main Apr 1, 2026
35 of 38 checks passed

ryan-mist deleted the network-interface-support branch April 1, 2026 20:47

ryan-mist mentioned this pull request Apr 3, 2026

feat: Add EFA configuration to EC2NodeClass for static NodePools #8898

Closed

3 tasks

Conversation

ryan-mist commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Manual Tests

Uh oh!

nathangeology left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 26, 2026

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 26, 2026

Uh oh!

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

Uh oh!

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 30, 2026

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

ryan-mist left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

jmdeal left a comment

ryan-mist commented Mar 23, 2026 •

edited

Loading

github-actions bot commented Mar 26, 2026 •

edited

Loading