Attempting to turn on administrative action or F5 logs me out

[taylorjonl]

Explain what happens

I have a fresh install of Ubuntu 22.04 that I installed cockpit with this command:

sudo apt install -t jammy-backports cockpit

After install I am able to login and work in the UI for non-administrative tasks but when I click the button to enable administrative stuff, it pops up the window prompting for my password for a second but it goes away very quickly and logs me out. If I hit the F5 button on the browser it is the same thing. When looking at the logs it appears it closes the session because it sees an existing session, then forces me to login again.

Not sure if I did something wrong or what is going on.

Version of Cockpit

310.1-1~bpo22.04.1

Where is the problem in Cockpit?

Unknown or not applicable

Server operating system

Ubuntu

Server operating system version

22.04

What browsers are you using?

Edge

System log

10:56 AM: [systemd] : Started Session 35 of User jltaylor.
10:56 AM: [systemd-logind] : New session 35 of user jltaylor.
10:56 AM: [cockpit-session] : pam_unix(cockpit:session): session opened for user jltaylor(uid=1000) by (uid=0)
10:56 AM: [cockpit-session] : pam_ssh_add: Failed adding some keys
10:55 AM: [systemd-logind] : Removed session 34.
10:55 AM: [systemd] : session-34.scope: Consumed 4.076s CPU time.
10:55 AM: [systemd] : session-34.scope: Deactivated successfully.
10:55 AM: [systemd-logind] : Session 34 logged out. Waiting for processes to exit.
10:55 AM: [cockpit-session] : pam_unix(cockpit:session): session closed for user jltaylor
10:55 AM: [cockpit-ws] : cockpit.protocol.CockpitProtocolError: channel is already open
10:55 AM: [cockpit-ws] : raise CockpitProtocolError('channel is already open')
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/router.py", line 200, in channel_control_received
10:55 AM: [cockpit-ws] : self.channel_control_received(channel, command, message)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 98, in control_received
10:55 AM: [cockpit-ws] : self.control_received(data)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 88, in frame_received
10:55 AM: [cockpit-ws] : self.frame_received(data[start:end])
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 134, in consume_one_frame
10:55 AM: [cockpit-ws] : result = self.consume_one_frame(self.buffer)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 184, in data_received
10:55 AM: [cockpit-ws] : await self._communication_done
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/router.py", line 258, in communicate
10:55 AM: [cockpit-ws] : await router.communicate()
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/bridge.py", line 178, in run
10:55 AM: [cockpit-ws] : return future.result()
10:55 AM: [cockpit-ws] : File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
10:55 AM: [cockpit-ws] : return loop.run_until_complete(main)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
10:55 AM: [cockpit-ws] : asyncio.run(main, debug=debug)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/event.py", line 135, in run_async
10:55 AM: [cockpit-ws] : run_async(run(args), debug=args.debug)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/bridge.py", line 311, in main
10:55 AM: [cockpit-ws] : sys.exit(main())
10:55 AM: [cockpit-ws] : File "/usr/bin/cockpit-bridge", line 8, in <module>
10:55 AM: [cockpit-ws] : Traceback (most recent call last):
10:55 AM: [cockpit-ws] : OSError: [Errno 22] sd_bus_attach_event: Invalid argument
10:55 AM: [cockpit-ws] : raise OSError(-result, f"{func.__name__}: {os.strerror(-result)}")
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line 73, in errcheck
10:55 AM: [cockpit-ws] : setattr(cls, name, lambda *args: func(*args))
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line 194, in <lambda>
10:55 AM: [cockpit-ws] : self.bus.attach_event(None, 0)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/channels/dbus.py", line 248, in do_open
10:55 AM: [cockpit-ws] : self.do_open(message)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 119, in do_control
10:55 AM: [cockpit-ws] : self.do_control(command, message)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 141, in do_channel_control
10:55 AM: [cockpit-ws] : endpoint.do_channel_control(channel, command, message)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/router.py", line 219, in channel_control_received
10:55 AM: [cockpit-ws] : self.channel_control_received(channel, command, message)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 98, in control_received
10:55 AM: [cockpit-ws] : self.control_received(data)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 88, in frame_received
10:55 AM: [cockpit-ws] : self.frame_received(data[start:end])
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 134, in consume_one_frame
10:55 AM: [cockpit-ws] : result = self.consume_one_frame(self.buffer)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 184, in data_received
10:55 AM: [cockpit-ws] : self._protocol.data_received(data)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3/dist-packages/cockpit/transports.py", line 110, in _read_ready
10:55 AM: [cockpit-ws] : self._context.run(self._callback, *self._args)
10:55 AM: [cockpit-ws] : File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
10:55 AM: [cockpit-ws] : Traceback (most recent call last):
10:55 AM: [cockpit-ws] : handle: <Handle _Transport._read_ready()>
10:55 AM: [cockpit-ws] : asyncio-ERROR: Exception in callback _Transport._read_ready()
10:55 AM: [systemd] : Started Hostname Service.
10:55 AM: [dbus-daemon] : [system] Successfully activated service 'org.freedesktop.hostname1'
10:55 AM: [cockpit-tls] : cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received. 5Occurrences
10:55 AM: [systemd] : Starting Hostname Service...
10:55 AM: [dbus-daemon] : [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.114' (uid=1000 pid=58629 comm="/usr/bin/python3 /usr/bin/cockpit-bridge " label="unconfined")
10:55 AM: [cockpit-tls] : cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received. 15Occurrences
10:52 AM: [systemd] : Started Time & Date Service.
10:52 AM: [dbus-daemon] : [system] Successfully activated service 'org.freedesktop.timedate1'
10:52 AM: [systemd] : Starting Time & Date Service...
10:52 AM: [dbus-daemon] : [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.114' (uid=1000 pid=58629 comm="/usr/bin/python3 /usr/bin/cockpit-bridge " label="unconfined")
10:51 AM: [systemd] : systemd-timedated.service: Deactivated successfully.
10:51 AM: [systemd] : Started Time & Date Service.
10:51 AM: [dbus-daemon] : [system] Successfully activated service 'org.freedesktop.timedate1'
10:51 AM: [systemd] : Starting Time & Date Service...
10:51 AM: [dbus-daemon] : [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.114' (uid=1000 pid=58629 comm="/usr/bin/python3 /usr/bin/cockpit-bridge " label="unconfined")
10:50 AM: [systemd] : systemd-timedated.service: Deactivated successfully.
10:47 AM: [systemd] : systemd-hostnamed.service: Deactivated successfully.
10:47 AM: [PackageKit] : get-updates transaction /43_bbcbeeea from uid 1000 finished with success after 404ms
10:47 AM: [PackageKit] : resolve transaction /42_adaebaba from uid 1000 finished with success after 273ms
10:47 AM: [systemd] : Started Time & Date Service.
10:47 AM: [dbus-daemon] : [system] Successfully activated service 'org.freedesktop.timedate1'
10:47 AM: [systemd] : Starting Time & Date Service...
10:47 AM: [dbus-daemon] : [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.114' (uid=1000 pid=58629 comm="/usr/bin/python3 /usr/bin/cockpit-bridge " label="unconfined")
10:47 AM: [cockpit-tls] : cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received. 2Occurrences
10:47 AM: [systemd] : Started Hostname Service.
10:47 AM: [dbus-daemon] : [system] Successfully activated service 'org.freedesktop.hostname1'
10:47 AM: [systemd] : Starting Hostname Service...
10:47 AM: [dbus-daemon] : [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.114' (uid=1000 pid=58629 comm="/usr/bin/python3 /usr/bin/cockpit-bridge " label="unconfined")
10:47 AM: [cockpit-tls] : cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received. 6Occurrences
10:47 AM: [systemd] : Started Session 34 of User jltaylor.
10:47 AM: [systemd-logind] : New session 34 of user jltaylor.

[bobzer]

Same behavior for me on debian bookworm, fresh install and up to date

[bobzer]

I fix my problem using a real certificate, (not a self signed one)

[oOblik]

I am also experiencing this issue on fresh up to date install of Arch Linux (aarch64), cockpit 330-1

[darkyzhou]

Also experiencing this issue, on AOSC for LoongArch, cockpit 333 compiled from source

[computerquip]

I think the issue is something to do with the session created for domain users. For example, if I login via ssh, I can do stuff, use sudo, some things appear to be proper but if I were to do something like systemctl --user, I'd get Failed to connect to user scope bus via local transport: No medium found.

There appear to be logs in the code to show which path it takes when assigning bus but I'm struggling to decipher what needs to be done to get them to show. I've tried adding environment variables shown here system wide and get no additional logging.

[darkyzhou]

@computerquip In my investigation, do_open in src/cockpit/channels/dbus.py might be related to the cause.

In https://github.com/cockpit-project/cockpit/blob/4e640b636354105ff89fb6ddcde463956fe06186/src/cockpit/channels/dbus.py#L247C8-L251C22, the error sd_bus_attach_event: Invalid argument always comes with self.bus = Bus.default_user() branch being selected.

Interestingly, when I edit the codes so that self.bus.attach_event(None, 0) won't be called if a session bus is selected, the problem seems to be "fixed" and cockpit seems to run without issues.

[computerquip-work]

That's the thing though, self.bus = Bus.default_user() should work afaict but I think it's failing silently because the domain user isn't given a proper session. I would imagine removing self.bus.attach_event(None, 0) would kill the dbus event loop but I'm not sure where all it's used so I'm not sure what consequences might show up.

[blinkin]

Seems to be related to pam authentication. In my case i was able to fix this error by adding session include password-auth at the end of pam.d/cockpit file

# pam_selinux.so open should only be followed by sessions to be executed in the user context
session    required     pam_selinux.so open env_params
session    optional     pam_keyinit.so force revoke
session    optional     pam_ssh_add.so
session    include      password-auth
session    include      postlogin

[ksikka]

Same issue, here are my logs:

May 06 11:35:31 DESKTOP-KF26FAO sudo[23490]: pam_unix(sudo:session): session closed for user root
May 06 11:35:33 DESKTOP-KF26FAO systemd[1]: Started Cockpit session 23/20000/65109 (PID 20000/UID 65109).
May 06 11:35:33 DESKTOP-KF26FAO cockpit-session[23621]: pam_ssh_add: Identity added: /home/ksikka/.ssh/id_rsa (ksikka@DESKTOP-KF26FAO)
May 06 11:35:33 DESKTOP-KF26FAO cockpit-session[23621]: pam_ssh_add: Identity added: /home/ksikka/.ssh/id_ed25519 (karanssikka@gmail.com)
May 06 11:35:33 DESKTOP-KF26FAO cockpit-session[23621]: pam_unix(cockpit:session): session opened for user ksikka(uid=1000) by (uid=0)
May 06 11:35:33 DESKTOP-KF26FAO systemd-logind[212]: New session 24 of user ksikka.
May 06 11:35:33 DESKTOP-KF26FAO systemd[1]: Started Session 24 of User ksikka.
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname>
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Starting Hostname Service...
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Successfully activated service 'org.freedesktop.hostname1'
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Started Hostname Service.
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate>
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Starting Time & Date Service...
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Successfully activated service 'org.freedesktop.timedate1'
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Started Time & Date Service.
May 06 11:35:35 DESKTOP-KF26FAO PackageKit[17649]: resolve transaction /27_dbcebead from uid 1000 finished with success after 509ms
May 06 11:35:36 DESKTOP-KF26FAO PackageKit[17649]: get-updates transaction /28_decacccd from uid 1000 finished with success after 586ms
May 06 11:35:36 DESKTOP-KF26FAO PackageKit[17649]: get-update-detail transaction /30_aabcdbce from uid 1000 finished with success after 345ms
May 06 11:35:37 DESKTOP-KF26FAO /usr/bin/cockpit-bridge[23628]: Exception in callback _Transport._read_ready()
                                                                handle: <Handle _Transport._read_ready()>
                                                                Traceback (most recent call last):
                                                                  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
                                                                    self._context.run(self._callback, *self._args)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/transports.py", line 110, in _read_ready
                                                                    self._protocol.data_received(data)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 192, in data_received
                                                                    result = self.consume_one_frame(self.buffer)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 142, in consume_one_frame
                                                                    self.frame_received(data[start:end])
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 96, in frame_received
                                                                    self.control_received(data)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 106, in control_received
                                                                    self.channel_control_received(channel, command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/router.py", line 219, in channel_control_received
                                                                    endpoint.do_channel_control(channel, command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 155, in do_channel_control
                                                                    self.do_control(command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 133, in do_control
                                                                    self.do_open(message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channels/dbus.py", line 248, in do_open
                                                                    self.bus.attach_event(None, 0)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line >
                                                                    setattr(cls, name, lambda *args: func(*args))
                                                                  File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line >
                                                                    raise OSError(-result, f"{func.__name__}: {os.strerror(-result)}")
May 06 11:35:33 DESKTOP-KF26FAO systemd-logind[212]: New session 24 of user ksikka.
May 06 11:35:33 DESKTOP-KF26FAO systemd[1]: Started Session 24 of User ksikka.
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname>
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Starting Hostname Service...
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Successfully activated service 'org.freedesktop.hostname1'
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Started Hostname Service.
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate>
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Starting Time & Date Service...
May 06 11:35:34 DESKTOP-KF26FAO dbus-daemon[204]: [system] Successfully activated service 'org.freedesktop.timedate1'
May 06 11:35:34 DESKTOP-KF26FAO systemd[1]: Started Time & Date Service.
May 06 11:35:35 DESKTOP-KF26FAO PackageKit[17649]: resolve transaction /27_dbcebead from uid 1000 finished with success after 509ms
May 06 11:35:36 DESKTOP-KF26FAO PackageKit[17649]: get-updates transaction /28_decacccd from uid 1000 finished with success after 586ms
May 06 11:35:36 DESKTOP-KF26FAO PackageKit[17649]: get-update-detail transaction /30_aabcdbce from uid 1000 finished with success after 345ms
May 06 11:35:37 DESKTOP-KF26FAO /usr/bin/cockpit-bridge[23628]: Exception in callback _Transport._read_ready()
                                                                handle: <Handle _Transport._read_ready()>
                                                                Traceback (most recent call last):
                                                                  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
                                                                    self._context.run(self._callback, *self._args)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/transports.py", line 110, in _read_ready
                                                                    self._protocol.data_received(data)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 192, in data_received
                                                                    result = self.consume_one_frame(self.buffer)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 142, in consume_one_frame
                                                                    self.frame_received(data[start:end])
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 96, in frame_received
                                                                    self.control_received(data)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 106, in control_received
                                                                    self.channel_control_received(channel, command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/router.py", line 219, in channel_control_received
                                                                    endpoint.do_channel_control(channel, command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 155, in do_channel_control
                                                                    self.do_control(command, message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channel.py", line 133, in do_control
                                                                    self.do_open(message)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/channels/dbus.py", line 248, in do_open
                                                                    self.bus.attach_event(None, 0)
                                                                  File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line >
                                                                    setattr(cls, name, lambda *args: func(*args))
                                                                  File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/librarywrapper.py", line >
                                                                    raise OSError(-result, f"{func.__name__}: {os.strerror(-result)}")
                                                                OSError: [Errno 22] sd_bus_attach_event: Invalid argument
May 06 11:35:39 DESKTOP-KF26FAO sudo[23681]: pam_unix(sudo:auth): conversation failed
May 06 11:35:39 DESKTOP-KF26FAO sudo[23681]: pam_unix(sudo:auth): auth could not identify password for [ksikka]
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]: Traceback (most recent call last):
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/bin/cockpit-bridge", line 8, in <module>
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     sys.exit(main())
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/bridge.py", line 315, in main
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     run_async(run(args), debug=args.debug)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/_vendor/systemd_ctypes/event.py", line 135, in run_async
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     asyncio.run(main, debug=debug)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     return loop.run_until_complete(main)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     return future.result()
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/bridge.py", line 166, in run
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/bridge.py", line 166, in run
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     await router.communicate()
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/router.py", line 258, in communicate
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     await self._communication_done
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 192, in data_received
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     result = self.consume_one_frame(self.buffer)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 142, in consume_one_frame
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     self.frame_received(data[start:end])
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 96, in frame_received
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     self.control_received(data)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/protocol.py", line 106, in control_received
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     self.channel_control_received(channel, command, message)
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:   File "/usr/lib/python3/dist-packages/cockpit/router.py", line 200, in channel_control_received
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]:     raise CockpitProtocolError('channel is already open')
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23628]: cockpit.protocol.CockpitProtocolError: channel is already open
May 06 11:35:39 DESKTOP-KF26FAO cockpit-session[23621]: pam_unix(cockpit:session): session closed for user ksikka

[darkyzhou]

After investigation, I think I have found the cause:

1. When a regular user logs in, libsystemd.so fails to get the DBUS_SESSION_BUS_ADDRESS or XDG_RUNTIME_DIR, leading to sd_bus_default_user() returning NULL. The function is called by src/cockpit/_vendor/systemd_ctypes/bus.py when cockpit is opening a dbus channel.

   @staticmethod
   def default_user(attach_event: bool = True) -> 'Bus':
       if Bus._default_user_instance is None:
           Bus._default_user_instance = Bus()
           # !!! The return value of this call somehow is not checked for success!
           Bus._default_user(byref(Bus._default_user_instance))
           if attach_event:
               Bus._default_user_instance.attach_event(None, 0)
       return Bus._default_user_instance

2. This failure tanks the D-Bus connection for the user session, leading to sd_bus_attach_event() of libsystemd.so getting a NULL bus pointer and throwing an EINVAL.

I modified src/session/session.c to dump the environment from pam_getenvlist(pamh) right after the PAM session is set up. The output looks like this:

XDG_SESSION_CLASS=user
XDG_SESSION_TYPE=web
HOME=/home/darkyzhou
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
COCKPIT_LOGIN_MESSAGES_MEMFD=3

As you can see, no DBUS_SESSION_BUS_ADDRESS or XDG_RUNTIME_DIR. These are critical and should be set by pam_systemd.so. Although SSH logins for the same user work fine and get all the necessary session environment variables.

I also set #define DEBUG_SESSION 0 to #define DEBUG_SESSION 1 in session-utils.h and got following logs during logging in:

May 11 12:23:41 ryustation systemd[1]: Started cockpit-wsinstance-https@e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.service - Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
May 11 12:23:41 ryustation systemd[1]: Started cockpit-session@8-342935-63714.service - Cockpit session 8/342935/63714 (PID 342935/UID 63714).
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: writing auth challenge
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: finished auth request
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: reading authorization authorize message
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: initial authorize response rhost: ::ffff:10.10.10.10
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: reading kerberos auth from cockpit-ws
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: acquiring cockpit service credentials
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: gssapi: major_status: 00070000, minor_status: 96c73ab5
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: writing PAM init problem 9
May 11 12:23:41 ryustation cockpit-session[342942]: cockpit-session: writing init problem authentication-unavailable message Authentication service cannot retrieve authentication info
May 11 12:23:41 ryustation systemd[1]: cockpit-session@8-342935-63714.service: Deactivated successfully.
May 11 12:23:46 ryustation systemd[1]: Started cockpit-session@9-342935-63714.service - Cockpit session 9/342935/63714 (PID 342935/UID 63714).
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: writing auth challenge
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: finished auth request
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: reading authorization authorize message
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: initial authorize response rhost: ::ffff:10.10.10.10
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: basic authentication
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: authenticating
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: answered pam password prompt
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: checking access for darkyzhou
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: opening pam session for darkyzhou
May 11 12:23:47 ryustation cockpit-session[342969]: pam_unix(cockpit:session): session opened for user darkyzhou(uid=1000) by (uid=0)
May 11 12:23:47 ryustation cockpit-session[342969]: cockpit-session: user_has_valid_login_shell: exited with status 4700
May 11 12:23:47 ryustation sudo[342978]: darkyzhou : HOST=ryustation ; PWD=/ ; USER=root ; COMMAND=/usr/bin/cockpit-bridge --privileged
May 11 12:23:47 ryustation sudo[342978]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)

My pam configs are listed below:

$ cat cockpit.pam
#%PAM-1.0
auth      include   system-remote-login
account   include   system-remote-login
# List of users to deny access to Cockpit, by default root is included.
account   required  pam_listfile.so item=user sense=deny file=/etc/cockpit/disallowed-users onerr=succeed
password  include   system-remote-login
session   include   system-remote-login
session   include   password-auth

$ cat system-remote-login 
#%PAM-1.0
auth      include   system-login
account   include   system-login
password  include   system-login
session   include   system-login

$ cat system-login
#%PAM-1.0
auth       required   pam_shells.so
auth       requisite  pam_nologin.so
auth       include    system-auth
account    required   pam_access.so
account    required   pam_nologin.so
account    include    system-auth
password   include    system-auth
session    optional   pam_loginuid.so
session    include    system-auth
session    optional   pam_motd.so          motd=/etc/motd
session    optional   pam_mail.so          dir=/var/spool/mail standard quiet
session    required   pam_env.so
session    optional   pam_systemd.so

[computerquip-work]

I saw this but I never figured out why pam_systemd.so doesn't give the user a session number. There's a related SSSD issue I made (which in retrospect, may not be the correct place to file an issue here).

SSSD/sssd#7853

That issue links to several systemd issues and pull requests:
systemd/systemd#30846
systemd/systemd#34617

Ultimately, I was unable to determine the cause of all of this. In that SSSD issue, at least one person was able to get proper systemd seats for domain users without issues... but I don't know the difference between their environment and my own.

To be honest, it doesn't seem like a cockpit issue but also, I don't know where to file a bug anymore. If I file a bug on the systemd repository, it's likely to get ignored and/or closed as a duplicate and nothing gets solved for the next 5+ years. So... I sort of just gave up.