Add the VolumeAttributesClass feature

[rybas-dv]

We are deploying kubernetes on bare metal and using openebs lvm with ioLimits capability(RIOPS_PER_GB/WIOPS_PER_GB) and it works great thanks. Starting with k8s version 1.34, it is now possible to use VolumeAttributesClass to regulate iops/throughput. Are there plans to add a vac support to csidriver?

Environment:

• LVM Driver version - 1.8.0
• Kubernetes version - 1.31
• Cloud provider or hardware configuration - bare metal

[abhilashshetty04]

Hi @rybas-dv , We have thought about it. But not planned in immediate roadmap. It would be great to have it. Are you interested in contributing?

[rybas-dv]

I'll try to figure it out in the future. If I can, I will prepare a pull request.

[abhilashshetty04]

Hi @rybas-dv , If you are interested, You can start by raising an OEP (OpenEBS enhancement proposal). This helps community to understand the nature of the proposed change before the PR.

Template:
https://github.com/openebs/openebs/blob/develop/designs/oep-template.md

Example:
https://github.com/openebs/openebs/pull/4060/changes

[rybas-dv]

hi @abhilashshetty04 thanks for OEP.

I tried to implement vac support in local area. When i'm trying to implement this, I ran into the problem of getting the pod uid(for cgroup v2 path) on vac update (when reconcile updateVol and syncVol). I solved this by getting information from mount, but so far I don't like the solution. Do you have any idea how to do this without mount?

[abhilashshetty04]

@rybas-dv , Apologies Im not aware of the context. Can you please elaborate? You are trying to have a way of enforcing per volume QoS through VAC right?

[rybas-dv]

@abhilashshetty04 , You're right, I should have provided more details. I am trying to implements vac support for pvc in filesystem and block modes, to do this, I follow kep and spec. As far as I understand, according to these documents, to support vac, I need:

1. The CSI driver controller requires the RPC_MODIFY_VOLUME capability. Once it's declared, we can obtain the VAC from GetMutableParameters. Next, we need to convert the VAC parameters into a resource understood by the controller. I added a structure to LVMVolume.spec.io and implemented the necessary VAC conversion logic so that CreateVolume and ControllerModifyVolume can perform the necessary work. like so, there's no problem here:

apiVersion: local.openebs.io/v1alpha1
kind: LVMVolume
metadata:
  namespace: lvm-localpv
spec:
  capacity: "1000341504"
  io:
    readBPS: 7340032000
    readIOPS: 10
    writeBPS: 7340032000
    writeIOPS: 10
  ownerNodeID: kworker1
  shared: "no"
  thinProvision: "no"
  vgPattern: ^lvmg$
  volGroup: lvmg

2. The problem occurred on CSI driver node side. When using vac for the "first time" NodePublishVolume does all the work and calls lvm part by passing the podLVinfo parameters(with UID), which I use when searching for the pod directory in cgroupv2 to apply io.max it works. But when updating vac, NodePublishVolume is no longer called, and a different solution is needed. To avoid breaking the logic and creating a new watcher, informer, etc. separately for vac, I decided to integrate and use the existing one for lvmvolume and modified it a bit. Now, at updateVol (updating vac) and syncVol (applying vac parameters), I need to find where to apply vac; there is no podLVinfo, I find it by mounting. And then there is a hardcode logic for cgroupv2 - so, essentially, I need to find the path in /sys/fs/cgroup for the pod, and I do this using information from the mount, like so:

// Helper function which find pod cgroup directory and supports both systemd and cgroupfs paths.
func getPodCgroupPath(podUID string) ([]string, error) {
	kDriver := "/sys/fs/cgroup/kubepods.slice"
	cDriver := "/sys/fs/cgroup/kubepods"

	uidDash := podUID
	uidUnderscore := strings.ReplaceAll(podUID, "-", "_")

	candidates := []string{
		// systemd
		filepath.Join(kDriver, "kubepods-burstable.slice", fmt.Sprintf("kubepods-burstable-pod%s.slice", uidUnderscore)),
		filepath.Join(kDriver, "kubepods-besteffort.slice", fmt.Sprintf("kubepods-besteffort-pod%s.slice", uidUnderscore)),
		filepath.Join(kDriver, "kubepods-guaranteed.slice", fmt.Sprintf("kubepods-guaranteed-pod%s.slice", uidUnderscore)),
		filepath.Join(kDriver, fmt.Sprintf("kubepods-pod%s.slice", uidUnderscore)),

		// cgroupfs
		filepath.Join(cDriver, "burstable", fmt.Sprintf("pod%s", uidDash)),
		filepath.Join(cDriver, "besteffort", fmt.Sprintf("pod%s", uidDash)),
		filepath.Join(cDriver, fmt.Sprintf("pod%s", uidDash)),
	}
......................................................

if we decide that this suits us, then I will move on and if everything is fine, then I will start preparing pr.

[abhilashshetty04]

@rybas-dv, Thanks for clarifying. Should be okay to use volume controller when updating the VAC. Please note that addVol, deleteVol and updateVol just enqueues objects and syncVol is called by the worker thread for each enqueued volume.

Can you please raise an OEP before raising PRs as this is a new feature into LocalPV LVM.

[rybas-dv]

@abhilashshetty04
Initiated the OEP discussion
openebs/openebs#4145

[rybas-dv]

@abhilashshetty04 hi
Please consider the proposal to add a VAC:
#444

[Tulvar]

@abhilashshetty04 Are there any updates on this feature?

[devops-prog-bit]

Oh, this is a very useful feature, it was stopping us from using the operator in production