[RFC] Headless mode for Improving Authentication

[AndreyNikiforov]

Improve [Re-]authentication flow by replacing local user interface of the icloudpd with a remote option (thus, running icloudpd in a headless mode).

1. Context

Most common use case for icloudpd is to keep iCloud Photo collection synchronized to local storage/collection. Latency between adding the original to iCloud and the asset appearing in the local storage is the main effectiveness metric for this use case.

icloudpd runs as a process with command line interface (CLI) and terminal input/output. As part of the operation user input may be required, e.g. password and multi-factor authentication (MFA) token for iCloud.

While password is known by user ahead of time and can be provided as a CLI parameter, MFA token is generated during the authentication process and must be supplied by user in order for the authentication to complete and the image downloading to start.

Authentication to iCloud periodically expires (once in a couple of month? - Apple's decision), leaving icloudpd in a non-operating state until user re-authenticates.

2. Challenge

Many users run icloudpd on servers (e.g. NAS) without immediate UI into the running app (running as a service). To authenticate with password or MFA token into iCloud, user must 1) connect to the NAS (e.g. ssh or WebUI), 2) find running instance of the icloudpd, e.g. get docker container id) 3) connect to the terminal interface of the running instance of the icloudpd (e.g. docker attach) and supply password/MFA token. That process is cumbersome and painful.

Expired authentication ends with a request for password or, more often, with exception and program termination. Unless user employs external monitoring and alerting, expired authentication will be unnoticed and local storage will be unsynchronized with iCloud for a long time.

When icloudpd is expecting input from the user, all processing is stopped. That is a blocker for supporting more that one account per instance of the icloudpd - feature that may be valuable for many users who download images for more than one iCloud account in the household.

3. Solution Options

There are multiple options available to solve the challenges, each with its own set of trade offs. Some solutions may be combined.

3.1. Do Nothing

The pain of connecting to remote host may be not a pain at all (or tolerable) and dev efforts better be put on other features. Some pain may be alleviated with remote connection to docker daemon.

Re-authentication flow can be detected with regular manual inspection of the running icloudpd process.

3.2. Remote GUI

X Window System is an existing technology to separate services for user interface. Requires adding GUI code to the app. I am not familiar with that tech.

3.3. Chat Bots for Messaging Systems

Integrate icloudpd with messaging systems and provide simple chat-bot-like interaction, e.g. "icloudpd requires MFA token for your@appleid.address account. Please respond with 6 digits". Interfaces to Messaging Systems (MS) are implemented in related project.

It is unclear to me if two-way chat-bot-like interfaces are possible with APIs available for MS (feasibility). icloudpd will have to hold credentials for the MS account of the user (security implication).

3.4. WebUI

Integrate web server into icloudpd and accept password/MFA through web UI. Web interface can be used for communicating other information to the user - downloading progress, errors etc.

Web page can be "turned into" mobile app ("Add to Home Screen" option on iOS) - becomes an icon on the home screen. In that mode, icon can have badges (e.g. exclamation mark in red circle) and alert a user through iOS Notification System (even if the "app" is not in foreground). These capabilities improve user experience and allow users to react on re-authentication requests faster.

4. Evaluation

If my understanding of the problem/pain is correct, then with my knowledge of the Messaging System capabilities, Web UI seems to be a first choice.

I would love to hear opinions from others.

[boredazfcuk]

Regarding the 3.2. Remote GUI option, I've seen some containers use a VNC system to do this: https://github.com/linuxserver/docker-mediaelch is one I remember. Could be worth looking at how they've achieved it if that's a direction you want to take.

Regarding 3.4. WebUI I did spend some time trying to build a WebUI for my container, so that it could be used to configure all the options for my icloudpd.conf file. I started out with a Flask server and WTForms to collect the data. I got quite far with it, but I found it difficult to validate the details that were being entered were in the correct format for some stuff, like for the chown permissions I needed. I was also finding it tricky to have different sections appear based on values entered. If the person set the Notification Type to Telegram, then only Telegram options should be displayed. That all fell by the wayside in the end as I don't have as much spare time as I used to.

I think that the messaging system stuff would be a way forward, one poster did say on an issue of mine that there's a library dedicated to interfacing with different messaging systems. It would cut code down quite a bit if most of the work could be handed off to that. I can't remember the name of it at the moment though and can't find the issue. I may have another look when I have a bit more spare time. Just off out now.

[ADMiNZ]

Hello @AndreyNikiforov, @boredazfcuk I’m just using the system on a NAS and copying data from several accounts. Yes, there is such a minor problem. You need to re-authorize about once every month. Probably the easiest way would be a chatbot.

But the problem is that for me specifically, the task works at night - and even if I answer the bot in the morning - the code will most likely already be outdated.

So, I’ll probably only enter the code manually at a time convenient for me. Perhaps a chatbot with the ability to force a specific account to log in and then enter a code (from SMS or device) would simplify this. But I can’t say that re-authorization is a really complicated thing. And is it necessary to automate this process (which is not so complicated anyway)

[AndreyNikiforov]

introduced in v1.21.0