ERROR: Module rbd not found.

[nazarewk]

Is your feature request related to a problem? Please describe.
I want to use Rook's Ceph integration on k3os based Raspberry Pi 4 cluster.
Does not look like i can do that due to missing rbd kernel module

Additional context
I managed to get up to PV issuing, but getting errors with following errors:

MountVolume.MountDevice failed for volume "pvc-297d546b-6c64-462b-bb7a-7063e6fd0f02" : rpc error: code = Internal desc = rbd: map failed exit status 2, rbd output: modinfo: ERROR: Module rbd not found.
modprobe: FATAL: Module rbd not found.
rbd: failed to load rbd kernel module (1)
rbd: sysfs write failed
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (2) No such file or directory

[billimek]

I have the same issue.

It is possible to solve this by installing k3os via the overlay installation on top of a modern kernel with rbd already present. Ubuntu 20.04, for example. However, there appears to be an issue right now with running k3os via overlay so this approach isn’t ideal right now: https://github.com/rancher/k3s/issues/1706

[dweomer]

I’d be surprised if the rbd module wasn’t compiled. It’s currently the bionic/hwe kernel which means eoan and I think “modern”. It’s possible it isn’t on disk as it should be however. Load the configs module and zcat /proc/config.gz

[sgielen]

I have instructions here on compiling the module yourself if you know which exact build of the kernel you're running: sgielen/picl-k3os-image-generator#6 (comment)

Although to be honest, I found rook-ceph to be too heavy for my own arm64 kubernetes cluster. It restarted too often and was too unstable because of its high resource usage. YMMV.

I switched to OpenEBS, where version 1.9.0 wasn't fully stable on arm64 yet, but after openebs/openebs#3028 / mayadata-io/cstor#310 / mayadata-io/cstor#309 / openebs/openebs#3037 are addressed I'm quite confident release 1.11.0 will work very well on arm64 out of the box. So far, my version, with said patches applied, has been running very stable on my cluster, and it barely requires any kernel support, just the iSCSI driver & userland support which is usually present.

[dweomer]

For amd64 systems installed from ISO the module is there:

Given that RPI is overlay, its a BYO scenario. Glad to see that OpenEBS is filling the gap.

[nazarewk]

I'm actually not using overlay installation unless https://github.com/sgielen/picl-k3os-image-generator counts as an overlay.