Prometheus metrics scraping from CNI metrics helper

[jayanthvn]

What type of PR is this?
Feature

Which issue does this PR fix:
Fixes #2071

What does this PR do / Why do we need it:
Reopening #2192 and some additional refactoring.

CNI metrics helper merges all the metrics from aws-nodes and aggregates the metrics to send for CW. With this PR the aggregated data is sent to Prometheus when "USE_PROMETHEUS" is set. As part of initial enhancement only Prometheus type GAUGE is streamed for prometheus. Future enhancements will be made for other types of metrics.

This will include -

                "awscni_eni_max":                   EnisMax,
		"awscni_ip_max":                    IpMax,
		"awscni_add_ip_req_count":          AddIPCnt,
		"awscni_del_ip_req_count":          DelIPCnt,
		"awscni_eni_allocated":             Enis,
		"awscni_total_ip_addresses":        TotalIPs,
		"awscni_assigned_ip_addresses":     AssignedIPs,
		"awscni_force_removed_enis":        ForceRemovedENIs,
		"awscni_force_removed_ips":         ForceRemovedIPs,
		"awscni_total_ipv4_prefixes":       TotalPrefixes,
		"awscni_no_available_ip_addresses": NoAvailableIPAddrs,

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:

Testing done on this change:

kube-system   cni-metrics-helper-5d4d84b8d4-6pxmn                    1/1     Running   0          3m3s   192.168.38.249   

curl http://192.168.38.249:61681/metrics
# HELP awscni_add_ip_req_count The number of add IP address requests
# TYPE awscni_add_ip_req_count gauge
awscni_add_ip_req_count 0
# HELP awscni_assigned_ip_addresses The number of IP addresses assigned to pods
# TYPE awscni_assigned_ip_addresses gauge
awscni_assigned_ip_addresses 14
# HELP awscni_eni_allocated The number of ENIs allocated
# TYPE awscni_eni_allocated gauge
awscni_eni_allocated 4
# HELP awscni_eni_max The maximum number of ENIs that can be attached to the instance, accounting for unmanaged ENIs
# TYPE awscni_eni_max gauge
awscni_eni_max 8
# HELP awscni_force_removed_enis The number of ENIs force removed while they had assigned pods
# TYPE awscni_force_removed_enis gauge
awscni_force_removed_enis 0
# HELP awscni_force_removed_ips The number of IPs force removed while they had assigned pods
# TYPE awscni_force_removed_ips gauge
awscni_force_removed_ips 0
# HELP awscni_ip_max The maximum number of IP addresses that can be allocated to the instance
# TYPE awscni_ip_max gauge
awscni_ip_max 112
# HELP awscni_no_available_ip_addresses The number of pod IP assignments that fail due to no available IP addresses
# TYPE awscni_no_available_ip_addresses gauge
awscni_no_available_ip_addresses 0
# HELP awscni_total_ip_addresses The total number of IP addresses
# TYPE awscni_total_ip_addresses gauge
awscni_total_ip_addresses 56
# HELP awscni_total_ipv4_prefixes The total number of IPv4 prefixes
# TYPE awscni_total_ipv4_prefixe

Automation added to e2e:

N/A

Will this PR introduce any new dependencies?:

No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jdn5126]

Change overall LGTM

[jayanthvn]

Let's wait on merge..I see an issue here.

[jayanthvn]

Issue with aggregation fixed..

[jdn5126]

Changes look good overall, just a few nits. Can you also rebase to latest master?