Plugin won't load when installed as external plugin

[sebastian-de]

Overview of the Issue

I'd like to contribute to this plugin, but I'm stuck at the very beginning. I read CONTRIBUTING.md, did some changes and ran make dev. The resulting plugin gets found when running packer, but packer loads the internal plugin instead of the external one. This doesn't happen when I install packer-plugin-amazon so I assume this is a bug in packer-plugin-proxmox.
I also tried to install the plugin in the directory packer is run from, but packer still loads the internal plugin.

Reproduction Steps

• make dev
• packer validate . in a directory with a file using any proxmox-* builder (without required_plugins being set)

Plugin and Packer version

packer-plugin-proxmox built from main
Packer v1.8.4

Operating system and Environment details

Fedora 36, packer installed via go (I also tried packer from the Hashicorp Fedora repository, the reult is the same)

Log Fragments and crash.log files

For comparison I included the relevant log parts of both the proxmox and the amazon plugin. You can see that both plugins are found, but only the exernal amazon plugin gets loaded.

packer-plugin-proxmox

2022/10/29 19:38:47 [TRACE] discovering plugins in /home/user/.packer.d/plugins
2022/10/29 19:38:47 [DEBUG] Discovered plugin: proxmox = /home/user/.packer.d/plugins/packer-plugin-proxmox
2022/10/29 19:38:47 [INFO] found external [-packer-default-plugin-name- proxmox-clone proxmox-iso] builders from proxmox plugin
[...]
2022/10/29 19:38:47 [TRACE] Starting internal plugin packer-builder-proxmox-clone
2022/10/29 19:38:47 Starting plugin: /home/user/go/bin/packer []string{"/home/user/go/bin/packer", "plugin", "packer-builder-proxmox-clone"}

packer-plugin-amazon

2022/10/29 19:40:32 [TRACE] discovering plugins in /home/user/.packer.d/plugins
2022/10/29 19:40:32 [DEBUG] Discovered plugin: amazon = /home/user/.packer.d/plugins/packer-plugin-amazon
2022/10/29 19:40:32 [INFO] found external [chroot ebs ebssurrogate ebsvolume instance] builders from amazon plugin
2022/10/29 19:40:32 [INFO] found external [import] post-processors from amazon plugin
2022/10/29 19:40:32 found external [ami parameterstore secretsmanager] datasource from amazon plugin
[...]
2022/10/29 19:40:32 [TRACE] Starting external plugin /home/user/.packer.d/plugins/packer-plugin-amazon start builder ebs
2022/10/29 19:40:32 Starting plugin: /home/user/.packer.d/plugins/packer-plugin-amazon []string{"/home/user/.packer.d/plugins/packer-plugin-amazon", "start", "builder", "ebs"}

Full log for packer-plugin-proxmox

PACKER_LOG=1 packer validate debian-clone.pkr.hcl
2022/10/29 20:00:51 [INFO] Packer version: 1.8.4 [go1.18.7 linux amd64]
2022/10/29 20:00:51 Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 [TRACE] discovering plugins in /home/user/go/bin
2022/10/29 20:00:51 Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 [TRACE] discovering plugins in /home/user/.packer.d/plugins
2022/10/29 20:00:51 [DEBUG] Discovered plugin: proxmox = /home/user/.packer.d/plugins/packer-plugin-proxmox
2022/10/29 20:00:51 [INFO] found external [-packer-default-plugin-name- proxmox-clone proxmox-iso] builders from proxmox plugin
2022/10/29 20:00:51 [TRACE] discovering plugins in .
2022/10/29 20:00:51 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/10/29 20:00:51 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/user/.packerconfig
2022/10/29 20:00:51 [WARN] Config file doesn't exist: /home/user/.packerconfig
2022/10/29 20:00:51 Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 [INFO] Setting cache directory: /home/user/.cache/packer
2022/10/29 20:00:51 Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 [TRACE] Starting internal plugin packer-builder-proxmox-clone
2022/10/29 20:00:51 Starting plugin: /home/user/go/bin/packer []string{"/home/user/go/bin/packer", "plugin", "packer-builder-proxmox-clone"}
2022/10/29 20:00:51 Waiting for RPC address for: /home/user/go/bin/packer
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: [INFO] Packer version: 1.8.4 [go1.18.7 linux amd64]
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/user/.packerconfig
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: [WARN] Config file doesn't exist: /home/user/.packerconfig
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: [INFO] Setting cache directory: /home/user/.cache/packer
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: args: []string{"packer-builder-proxmox-clone"}
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 Received unix RPC address for /home/user/go/bin/packer: addr is /tmp/packer-plugin2217448539
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Plugin address: unix /tmp/packer-plugin2217448539
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Waiting for connection...
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: Serving a plugin connection...
2022/10/29 20:00:51 packer-builder-proxmox-clone plugin: CPU type not set, using default 'kvm64'
2022/10/29 20:00:51 [TRACE] Starting internal plugin packer-provisioner-shell
2022/10/29 20:00:51 Starting plugin: /home/user/go/bin/packer []string{"/home/user/go/bin/packer", "plugin", "packer-provisioner-shell"}
2022/10/29 20:00:51 Waiting for RPC address for: /home/user/go/bin/packer
2022/10/29 20:00:51 packer-provisioner-shell plugin: [INFO] Packer version: 1.8.4 [go1.18.7 linux amd64]
2022/10/29 20:00:51 packer-provisioner-shell plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/10/29 20:00:51 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/user/.packerconfig
2022/10/29 20:00:51 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /home/user/.packerconfig
2022/10/29 20:00:51 packer-provisioner-shell plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 packer-provisioner-shell plugin: [INFO] Setting cache directory: /home/user/.cache/packer
2022/10/29 20:00:51 packer-provisioner-shell plugin: Old default config directory found: /home/user/.packer.d
2022/10/29 20:00:51 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2022/10/29 20:00:51 packer-provisioner-shell plugin: Plugin address: unix /tmp/packer-plugin616124422
2022/10/29 20:00:51 packer-provisioner-shell plugin: Waiting for connection...
2022/10/29 20:00:51 Received unix RPC address for /home/user/go/bin/packer: addr is /tmp/packer-plugin616124422
2022/10/29 20:00:51 packer-provisioner-shell plugin: Serving a plugin connection...
The configuration is valid.
2022/10/29 20:00:51 [INFO] (telemetry) Finalizing.
2022/10/29 20:00:52 waiting for all plugin processes to complete...
2022/10/29 20:00:52 /home/user/go/bin/packer: plugin process exited
2022/10/29 20:00:52 /home/user/go/bin/packer: plugin process exited

[sebastian-de]

After a lot of trial and error I found out that packer starts the external plugin when the builders are called proxmox-proxmox-clone and proxmox-proxmox-clone in the template file.

With proxmox-clone and proxmox-iso the internal plugin is used.

But I can't figure out why this is the case.

[nywilken]

After a lot of trial and error I found out that packer starts the external plugin when the builders are called proxmox-proxmox-clone and proxmox-proxmox-clone in the template file.

With proxmox-clone and proxmox-iso the internal plugin is used.

But I can't figure out why this is the case.

@sebastian-de thank you for this detailed issue, for your discovery, and for your fix. I'll work on getting this released.

As for your questions, allow me to explain.

Packer uses the following naming convention for all external plugins packer-plugin-<name>, where name represents the top level provider name for the plugin; in this case proxmox. When Packer loads an external plugin it will extract the the provider name and use that as the prefix for each of the components (i.e builders, provisioners, post-processor). This is how Packer is able to distinguish proxmox-clone from vmware-clone in the template files. Running the plugin directly with the describe argument packer-plugin-proxmox describe will indicate to Packer what components it should register for the plugin. You can run the command directly to see the results your self. And as you found the components prior to your fix are called proxmox-iso and proxmox-clone so Packer thinks that the component names are actually proxmox-proxmox-iso and proxmox-proxmox-clone. As you found above.

For backwards compatibility with version prior to Packer v1.7.0 all external plugins are bundled into Packer using the hardcoded name, hence why the template for Proxmox does not fail as there is a plugin that provides poxmox-clone and proxmox-iso.

I hope this helps answer your question on why things are working the way they are.

[sebastian-de]

@nywilken thank you for your detailed explanation.
I didn't know about packer-plugin-proxmox describe, thanks for pointing that out. I guess I should give packer-plugin-sdk a closer look.