unable to escape NoStock error

[daimaxiaxie]

What happened:

I used a very simple NodePool. No CapacityType is specified, but the InstanceType is specified. I only used one VSwitch, it is in zone cn-shanghai-b.

spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: [ "amd64" ]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
            - ecs.c7a.4xlarge

However, the provider failed to start the instance because there was no stock in zone B.

{"level":"ERROR","time":"2024-12-25T11:48:21.756Z","logger":"controller","caller":"lifecycle/launch.go:61","message":"failed launching nodeclaim","commit":"935dd24","controller":"nodeclaim.lifecycle","controllerGroup":"karpenter.sh","controllerKind":"NodeClaim","NodeClaim":{"name":"default-qgc6n"},"namespace":"","name":"default-qgc6n","reconcileID":"2c1d3739-1235-4bc5-b75d-b3427cb2e0ae","error":"creating instance, insufficient capacity, requestId: 19691F1C-858F-54AA-80C8-05789EC1XXXX, failed to launch instance: errorCode=NoInstanceStock, errorMessage=The instanceTypes are out of usage"}

Then I added the log. Output InstanceType and CapacityType.

karpenter-provider-alibabacloud/pkg/providers/instance/instance.go

Lines 370 to 371 in a6c5b5f

	capacityType := p.getCapacityType(nodeClaim, instanceTypes)
	zonalVSwitchs, err := p.vSwitchProvider.ZonalVSwitchesForLaunch(ctx, nodeClass, instanceTypes, capacityType)

{"level":"DEBUG","time":"2024-12-25T11:48:21.069Z","logger":"controller","caller":"instance/instance.go:112","message":"launch InstanceType: ecs.c7a.4xlarge, type: spot","commit":"935dd24","controller":"nodeclaim.lifecycle","controllerGroup":"karpenter.sh","controllerKind":"NodeClaim","NodeClaim":{"name":"default-42cbv"},"namespace":"","name":"default-42cbv","reconcileID":"99f4d4f5-d385-4f88-a5b2-a8097258bf8b"}

The results show that it always tries only spot.

What you expected to happen:

OnDemand(postpaid) is available, and is expected to automatically fallback to OnDemand.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Karpenter-provider-alibabacloud version (use git describe --tags --dirty --always):
• AliCloud ACK version:
• Others:

[jwcesign]

In which region?

[daimaxiaxie]

In which region?

cn-shanghai

[jwcesign]

Thanks for reporting, we will fix it soon, the capacity information could be resolved: https://api.aliyun.com/api/Ecs/2014-05-26/DescribeAvailableResource?params={%22RegionId%22:%22cn-shanghai%22,%22InstanceChargeType%22:%22PostPaid%22,%22SpotStrategy%22:%22SpotAsPriceGo%22,%22DestinationResource%22:%22InstanceType%22}&tab=DEBUG

[jxs1211]

Maybe I can try to fix it.

[jwcesign]

@daimaxiaxie Is that ok?

Maybe, you are alreay fixing

[daimaxiaxie]

/assign @jxs1211

My idea is: added unavailableOfferings to InstanceTypeProvider.

[jxs1211]

Sorry, I don't get the point, if you don't set the CapacityTypeLabelKey, it should be setted as on-demand in getCapacityType, and if I understand you idea correctly we can record the unavailableOfferings through DescribeAvailableResource api before launchInstance?

[jwcesign]

Before set the offerings:

karpenter-provider-alibabacloud/pkg/providers/instancetype/instancetype.go

Line 382 in 0e75315

offerings = append(offerings, p.createOffering(zone.ID, karpv1.CapacityTypeSpot, spotPrice, offeringAvailable))

Call https://api.aliyun.com/api/Ecs/2014-05-26/DescribeAvailableResource?params={%22RegionId%22:%22cn-shanghai%22,%22InstanceChargeType%22:%22PostPaid%22,%22SpotStrategy%22:%22SpotAsPriceGo%22,%22DestinationResource%22:%22InstanceType%22}&tab=DEBUG to resolve the stock state

@jxs1211 @daimaxiaxie
If I understand correctly

[daimaxiaxie]

There are two ways:

1. DescribeAvailableResource before launch. unable to escape NoStock error #167 (comment)
2. Mark instancetype unavailable after launch error.

[jwcesign]

I prefer option 1

[jxs1211]

Before set the offerings:

karpenter-provider-alibabacloud/pkg/providers/instancetype/instancetype.go

Line 382 in 0e75315

offerings = append(offerings, p.createOffering(zone.ID, karpv1.CapacityTypeSpot, spotPrice, offeringAvailable))

Call https://api.aliyun.com/api/Ecs/2014-05-26/DescribeAvailableResource?params={%22RegionId%22:%22cn-shanghai%22,%22InstanceChargeType%22:%22PostPaid%22,%22SpotStrategy%22:%22SpotAsPriceGo%22,%22DestinationResource%22:%22InstanceType%22}&tab=DEBUG to resolve the stock state

@jxs1211 @daimaxiaxie If I understand correctly

How about the offerings meet AvailableZone is Available and "StatusCategory": "WithStock" can be added to result?

    "AvailableZone": [{
        "Status": "Available",
        "StatusCategory": "WithStock",
        "ZoneId": "cn-hangzhou-i",
        "AvailableResources": {
            "AvailableResource": [{
                "Type": "InstanceType",
                "SupportedResources": {
                    "SupportedResource": [{
                        "Status": "Available",
                        "StatusCategory": "ClosedWithStock",
                        "Value": "ecs.c5.2xlarge"
                    }]
                }
            }]
        },
        "RegionId": "cn-hangzhou"
    }

[jwcesign]

Check

karpenter-provider-alibabacloud/pkg/providers/instancetype/instancetype.go

Line 183 in 0e75315

if !p.instanceTypesOfferings[lo.FromPtr(i.InstanceTypeId)].Has(zoneID) || !vSwitchsZones.Has(zoneID) {

We may need another filed to record the spot instance state, and set the corresponding offering. For example, spotInstanceTypesOfferings

[jxs1211]

Check

karpenter-provider-alibabacloud/pkg/providers/instancetype/instancetype.go

Line 183 in 0e75315

if !p.instanceTypesOfferings[lo.FromPtr(i.InstanceTypeId)].Has(zoneID) || !vSwitchsZones.Has(zoneID) {

We may need nother filed to record the spot instance state, and set the corresponding offering. For example, spotInstanceTypesOfferings

If we can get the available status from the instanceInfo, why not use it as checker before appending to the result, instead of callling the api?

why do we need spotInstanceTypesOfferings,where need it, in the createOfferings or something?