Check iCloud on schedule

[AndreyNikiforov]

Summary

Support scheduling periodic iCloud check

Context

Many/most use icloudpd to sync iCloud to their local storage on a schedule. Today one option is to run manually. The second option is to run external scheduler, like cron

iCloudPD can keep running and "watching" for iCloud changes simplifying setup.

Benefits:

• simple setup
• opportunities to better collection management -- we can do full scans periodically and incremental in between
• opportunities to improve performance and reduce traffic -- since it is know what was synchronized last time, there is no need for functionality like --until-found; since we continuing session there maybe less need to do re-authentication with Apple, possibly extending two-factor period
• long-term local monitoring and alerting (e.g. Prometheus) may be easier

Challenges:

• edge-case error handling may need to be included, e.g. local volume disappearing

[boredazfcuk]

Hi, Andrey.

I have a question about the operation of the --watch-with-interval parameter...

Does this maintain a connection to icloud.com, so it only performs the logon process the first time it is run, and subsequent download runs just re-use the same session until it expires... or is it just a loop which repeats the logon process each time it attempts a download?

I've coded my container so that it can now loop through albums, and have noticed I'm getting the "private db access disabled for this account" error quite a lot. I was wondering if it's the multiple logons which are causing the throttling. If --watch-with-interval re-uses the same session, I would be able to implement that feature for downloading and let users reduce their sync times. to hourly or something.

Thanks.

[AndreyNikiforov]

Does this maintain a connection to icloud.com, so it only performs the logon process the first time it is run, and subsequent download runs just re-use the same session until it expires... or is it just a loop which repeats the logon process each time it attempts a download?

When in watch mode, icloudpd logs in once and then reuses session for periodic meta data fetching & downloading. Actual IP connection may be cached by connection pooling in the libs that are used by icloudpd but I don't expect that fact to affect session timeouts (cached connection in the pools are to avoid unnecessary TCP & TLS round trips on connection opening).

I cannot say for sure that periodically checking Apple servers on existing session will extend that sessions forever. I am running it since first release and had errors due to internet connectivity loss only (never session expiration).

In a long run I am contemplating the idea to separate credential input (~=UI) from processing. Once possible implementation may be a webUI interface to enter credential and monitoring. If works, then watch mode becomes default and will not terminate even in the face of the credential & connectivity issues, but just interact with user through webUI (e.g. to get MFA).

I've coded my container so that it can now loop through albums, and have noticed I'm getting the "private db access disabled for this account" error quite a lot. I was wondering if it's the multiple logons which are causing the throttling. If --watch-with-interval re-uses the same session, I would be able to implement that feature for downloading and let users reduce their sync times. to hourly or something.

You probably have to experiment. I have only basic knowledge on Apple session expiration and try to come up with solutions that are not bound to current Apple expiration policy (as it may change or may not be universal across geo regions).