Bandwidth plugin support

[jayanthvn]

What type of PR is this?
feature

Which issue does this PR fix:
#463 #1172 #1253

What does this PR do / Why do we need it:
Chaining was blocked because of this issue - containerd/cri#1544. Also if we just send "eth0" and sandbox ID then bandwidth plugin will fail here - https://github.com/containernetworking/plugins/blob/649e0181fe7b3a61e708f3e4249a798f57f25cc5/plugins/meta/bandwidth/main.go#L155-L161. But CNI plugin needs to send out all interfaces so that the chained plugin picks the correct interface based on the use case. Hence sending eth0 with sandbox and hostveth without sandbox would suffice for this to work.
Thanks to @anguslees for the suggestion.

This is just one part of the fix. Next is to have an approach to update 10-awsconflist file with -

        { 
          "type": "bandwidth",
          "capabilities": {"bandwidth": true}
        }

Option 1: introduce a new env variable which when set add the above lines to the file.
Option 2: Do something like this - https://github.com/anguslees/amazon-vpc-cni-k8s/blob/v6-cni/config/master/imds-cni.yaml#L61-L94

Option 2 won't need the init script to be updated for each plugin addition.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:
N/A

Testing done on this change:

Yes

iPerf containerd -

ingress/egress bandwidth 10M -

Client on 192.168.32.207:  Connecting to host iperf3-server, port 5201
Client on 192.168.32.207:  [  5] local 192.168.41.50 port 39424 connected to 10.100.187.145 port 5201
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
Client on 192.168.32.207:  [  5]   0.00-1.00   sec   178 MBytes  1.50 Gbits/sec    0    411 KBytes
Client on 192.168.32.207:  [  5]   1.00-2.00   sec  1.43 MBytes  12.0 Mbits/sec    0    471 KBytes
Client on 192.168.32.207:  [  5]   2.00-3.00   sec  1.12 MBytes  9.38 Mbits/sec    0    529 KBytes
Client on 192.168.32.207:  [  5]   3.00-4.00   sec  1.18 MBytes  9.90 Mbits/sec    0    587 KBytes
Client on 192.168.32.207:  [  5]   4.00-5.00   sec  1.30 MBytes  10.9 Mbits/sec    0    646 KBytes
Client on 192.168.32.207:  [  5]   5.00-6.00   sec  1.43 MBytes  12.0 Mbits/sec    0    704 KBytes
Client on 192.168.32.207:  [  5]   6.00-7.00   sec   764 KBytes  6.25 Mbits/sec    0    762 KBytes
Client on 192.168.32.207:  [  5]   7.00-8.00   sec  1.68 MBytes  14.1 Mbits/sec    0    820 KBytes
Client on 192.168.32.207:  [  5]   8.00-9.00   sec   891 KBytes  7.30 Mbits/sec    0    880 KBytes
Client on 192.168.32.207:  [  5]   9.00-10.00  sec  1.93 MBytes  16.2 Mbits/sec    0    938 KBytes
Client on 192.168.32.207:  - - - - - - - - - - - - - - - - - - - - - - - - -
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr
Client on 192.168.32.207:  [  5]   0.00-10.00  sec   190 MBytes   159 Mbits/sec    0             sender
Client on 192.168.32.207:  [  5]   0.00-10.85  sec   189 MBytes   146 Mbits/sec                  receiver
Client on 192.168.32.207:

ingress/egress bandwidth 10G

Client on 192.168.32.207:  Connecting to host iperf3-server, port 5201
Client on 192.168.32.207:  [  5] local 192.168.41.50 port 60250 connected to 10.100.242.98 port 5201
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
Client on 192.168.32.207:  [  5]   0.00-1.00   sec  1.59 GBytes  13.7 Gbits/sec   46   1.20 MBytes
Client on 192.168.32.207:  [  5]   1.00-2.00   sec  1.11 GBytes  9.58 Gbits/sec    0   1.76 MBytes
Client on 192.168.32.207:  [  5]   2.00-3.00   sec  1.11 GBytes  9.55 Gbits/sec    0   2.18 MBytes
Client on 192.168.32.207:  [  5]   3.00-4.00   sec  1.11 GBytes  9.57 Gbits/sec    0   2.54 MBytes
Client on 192.168.32.207:  [  5]   4.00-5.00   sec  1.11 GBytes  9.55 Gbits/sec    0   2.84 MBytes
Client on 192.168.32.207:  [  5]   5.00-6.00   sec  1.11 GBytes  9.57 Gbits/sec    0   3.00 MBytes
Client on 192.168.32.207:  [  5]   6.00-7.00   sec  1.11 GBytes  9.56 Gbits/sec    0   3.00 MBytes
Client on 192.168.32.207:  [  5]   7.00-8.00   sec  1.11 GBytes  9.56 Gbits/sec    0   3.00 MBytes
Client on 192.168.32.207:  [  5]   8.00-9.00   sec  1.11 GBytes  9.55 Gbits/sec    0   3.00 MBytes
Client on 192.168.32.207:  [  5]   9.00-10.00  sec  1.11 GBytes  9.57 Gbits/sec    0   3.00 MBytes
Client on 192.168.32.207:  - - - - - - - - - - - - - - - - - - - - - - - - -
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr
Client on 192.168.32.207:  [  5]   0.00-10.00  sec  11.6 GBytes  9.98 Gbits/sec   46             sender
Client on 192.168.32.207:  [  5]   0.00-10.04  sec  11.6 GBytes  9.93 Gbits/sec                  receiver

ingress/egress bandwidth - 1M

Client on 192.168.32.207:  Connecting to host iperf3-server, port 5201
Client on 192.168.32.207:  [  5] local 192.168.41.50 port 50562 connected to 10.100.229.126 port 5201
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
Client on 192.168.32.207:  [  5]   0.00-1.00   sec  20.3 MBytes   170 Mbits/sec    1    283 KBytes
Client on 192.168.32.207:  [  5]   1.00-2.00   sec   335 KBytes  2.75 Mbits/sec    0    290 KBytes
Client on 192.168.32.207:  [  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0    297 KBytes
Client on 192.168.32.207:  [  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0    303 KBytes
Client on 192.168.32.207:  [  5]   4.00-5.00   sec   382 KBytes  3.13 Mbits/sec    0    313 KBytes
Client on 192.168.32.207:  [  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0    334 KBytes
Client on 192.168.32.207:  [  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0    368 KBytes
Client on 192.168.32.207:  [  5]   7.00-8.00   sec   445 KBytes  3.65 Mbits/sec    0    417 KBytes
Client on 192.168.32.207:  [  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0    475 KBytes
Client on 192.168.32.207:  [  5]   9.00-10.00  sec   509 KBytes  4.17 Mbits/sec    0    533 KBytes
Client on 192.168.32.207:  - - - - - - - - - - - - - - - - - - - - - - - - -
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr
Client on 192.168.32.207:  [  5]   0.00-10.00  sec  21.9 MBytes  18.4 Mbits/sec    1             sender
Client on 192.168.32.207:  [  5]   0.00-14.59  sec  21.2 MBytes  12.2 Mbits/sec                  receiver

iperf docker -

ingress/egress bandwidth - 1M

Client on 192.168.32.207:  Connecting to host iperf3-server, port 5201
Client on 192.168.32.207:  [  5] local 192.168.37.101 port 46924 connected to 10.100.33.76 port 5201
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
Client on 192.168.32.207:  [  5]   0.00-1.00   sec  26.1 MBytes   219 Mbits/sec    2    266 KBytes
Client on 192.168.32.207:  [  5]   1.00-2.00   sec   318 KBytes  2.61 Mbits/sec    0    272 KBytes
Client on 192.168.32.207:  [  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0    277 KBytes
Client on 192.168.32.207:  [  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0    284 KBytes
Client on 192.168.32.207:  [  5]   4.00-5.00   sec   318 KBytes  2.61 Mbits/sec    0    296 KBytes
Client on 192.168.32.207:  [  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0    318 KBytes
Client on 192.168.32.207:  [  5]   6.00-7.00   sec   382 KBytes  3.13 Mbits/sec    0    352 KBytes
Client on 192.168.32.207:  [  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0    403 KBytes
Client on 192.168.32.207:  [  5]   8.00-9.00   sec   509 KBytes  4.17 Mbits/sec    0    461 KBytes
Client on 192.168.32.207:  [  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0    520 KBytes
Client on 192.168.32.207:  - - - - - - - - - - - - - - - - - - - - - - - - -
Client on 192.168.32.207:  [ ID] Interval           Transfer     Bitrate         Retr
Client on 192.168.32.207:  [  5]   0.00-10.00  sec  27.6 MBytes  23.2 Mbits/sec    2             sender
Client on 192.168.32.207:  [  5]   0.00-14.50  sec  27.1 MBytes  15.7 Mbits/sec                  receiver
Client on 192.168.32.207:
Client on 192.168.32.207:  iperf Done.

Automation added to e2e:

No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[M00nF1sh]

/lgtm