Downloading the "All Photos" album only downloads a single (old) photo

[DanielO]

Overview

Downloading the "All Photos" album only downloads a single (old) photo.

Steps to Reproduce

1. Run icloudpd -u me@mine.com -d /iCloudPhotos --threads-num 3 --log-level debug

Expected Behavior

All of my photos would be synced.

Actual Behavior

It downloads a single photo:

(icloudpd) [midget 12:04] ~ >icloudpd -u me@mine.com -d /iCloudPhotos --threads-num 3 --log-level debug
2025-02-15 12:05:14 DEBUG    Authenticating...
2025-02-15 12:05:17 DEBUG    Looking up all photos and videos from album All Photos...
2025-02-15 12:05:18 INFO     Downloading the first original photo or video to /iCloudPhotos ...
2025-02-15 12:05:18 DEBUG    /iCloudPhotos/2021/07/06/IMG_1264.HEIC already exists
2025-02-15 12:05:18 INFO     All photos have been downloaded

Context

version:1.26.1, commit sha:10af090, commit timestamp:Fri Jan 24 02:23:28 2025 ACDT

I can download other albums, both ones I have created and 'implicit' (like Live or Videos, etc) and they transfer OK.

Listing albums works, and I only see one library "PrimarySync".

[AndreyNikiforov]

How many photos are in your collections? There were reports that icloudpd downloads only part of them (half) in each run - not a drastic as only one in your case but seems similar.

What happens if you run icloudpd again? Does it get more assets on subsequent runs?

[DanielO]

There are 20000 in "all photos" and hundreds in others, one thing I just noticed is that when I request all photos it says "Downloading the first original photo or video", however when I grab from a specific album it gives the correct number (and downloads them all) eg:

(icloudpd) [midget 8:16] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos --threads-num 3 --log-level debug -a "Taiwan 2024 (work)"
2025-02-20 08:16:25 DEBUG    Authenticating...
2025-02-20 08:16:28 DEBUG    Looking up all photos and videos from album Taiwan 2024 (work)...
2025-02-20 08:16:29 INFO     Downloading 151 original photos and videos to /iCloudPhotos ...
2025-02-20 08:16:32 DEBUG    /iCloudPhotos/2024/07/18/IMG_2623.HEIC already exists
2025-02-20 08:16:32 DEBUG    /iCloudPhotos/2024/07/18/IMG_2622.HEIC already exists
...
2025-02-20 08:16:36 DEBUG    /iCloudPhotos/2024/07/04/IMG_2318.HEIC already exists
2025-02-20 08:16:36 DEBUG    /iCloudPhotos/2024/07/04/IMG_2318_HEVC.MOV already exists
2025-02-20 08:16:37 INFO     All photos have been downloaded

[AndreyNikiforov]

Do you have "Keep originals" flag set on your device? If not, I wonder if default value of --size original messes up with it somehow. You can try --size adjusted. See doc.

That theory does not explain why you see "Download first original photo or video". You can try --recent 5 and see if that changes the outcome.

[DanielO]

My phone is set to 'Optimise Storage'/
Unfortunately neither of those options affect things materially:

(icloudpd) [midget 19:06] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos --threads-num 3 --log-level debug -a "All Photos" --recent 5
2025-02-21 08:13:16 DEBUG    Authenticating...
2025-02-21 08:13:23 DEBUG    Looking up all photos and videos from album All Photos...
2025-02-21 08:13:23 INFO     Downloading 5 original photos and videos to /iCloudPhotos ...
2025-02-21 08:13:24 DEBUG    /iCloudPhotos/2021/07/06/IMG_1264.HEIC already exists
2025-02-21 08:13:24 INFO     All photos have been downloaded
(icloudpd) [midget 8:13] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos --threads-num 3 --log-level debug -a "All Photos" --size adjusted
2025-02-21 08:14:27 DEBUG    Authenticating...
2025-02-21 08:14:30 DEBUG    Looking up all photos and videos from album All Photos...
2025-02-21 08:14:30 INFO     Downloading the first adjusted photo or video to /iCloudPhotos ...
2025-02-21 08:14:31 DEBUG    /iCloudPhotos/2021/07/06/IMG_1264.HEIC already exists
2025-02-21 08:14:31 INFO     All photos have been downloaded

[boredazfcuk]

--threads-num was removed nearly five years ago : #180

Take that parameter out and see what happens.

Could be that it's being interpreted as something else, silently failing after the first download, starting the first thread and failing because the code to open parallel threads was removed... etc.

[DanielO]

Unfortunately that doesn't help either.
Is there a way to get more debugging?

[boredazfcuk]

(icloudpd) [midget 19:06] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos --threads-num 3 --log-level debug -a "All Photos" --recent 5
2025-02-21 08:13:16 DEBUG    Authenticating...
2025-02-21 08:13:23 DEBUG    Looking up all photos and videos from album All Photos...
2025-02-21 08:13:23 INFO     Downloading 5 original photos and videos to /iCloudPhotos ...
2025-02-21 08:13:24 DEBUG    /iCloudPhotos/2021/07/06/IMG_1264.HEIC already exists
2025-02-21 08:13:24 INFO     All photos have been downloaded

Get rid of -a "All Photos" too.

This is the default "meta album" that photos are downloaded from if an actual album name is not specified.

Strip it back to basics with: icloudpd -u me@mine.com -d /iCloudPhotos

Also try icloudpd -u me@mine.com -d /iCloudPhotos --only-print-filenames and see what it kicks out.

[DanielO]

I get no output:

(icloudpd) [midget 12:23] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos --only-print-filenames
(icloudpd) [midget 12:23] ~ >

Works OK for a named album, eg:

(icloudpd) [midget 12:23] ~ >icloudpd -u darius@dons.net.au -d /iCloudPhotos -a Live --only-print-filenames
/iCloudPhotos/2025/02/16/IMG_3063.HEIC
/iCloudPhotos/2025/02/16/IMG_3063_HEVC.MOV
^C
Aborted!

[boredazfcuk]

One thing I've noticed is that any files which are added to the Hidden album are no longer included in the 'All Photos' meta album. You've not got all your pictures added to this album? Or possibly other albums which may mean they are removed from the 'All Photos' album?

You can see them all in iCloud ok? They aren't all organised into folders? Downloading files located inside folders is not currently supported.

[DanielO]

I don't believe I have any photos hidden, they show up in iCloud and I don't have any folders.

Is there a way to get more debugging out of the program?

[AndreyNikiforov]

Is there a way to get more debugging out of the program?

Not without modifying source code I am afraid. If you see assets in iCloud.com web, but they are not "visible" by icloudpd, I would think checking response/payload from the apple servers is the next step.

[opswhisperer]

Just set this up today and getting the same result, one image downloaded. I am not specifying anything except the data directory. Tried with both the docker image and plain python client with the same result

icloudpd --username xxxxxxxxxx --password xxxxxxxxx --directory ./data
2025-03-06 10:34:49 DEBUG    Authenticating...
2025-03-06 10:34:53 DEBUG    Looking up all photos and videos from album All Photos...
2025-03-06 10:34:54 INFO     Downloading the first original photo or video to data ...
2025-03-06 10:34:54 DEBUG    Downloading data/2021/06/18/IMG_3893.HEIC...
2025-03-06 10:34:55 INFO     Downloaded data/2021/06/18/IMG_3893.HEIC
2025-03-06 10:34:56 INFO     All photos have been downloaded

[opswhisperer]

Turns out I had an old album called "All Photos" with one photo in it. I didn't create it, but when I deleted it everything works fine.

[boredazfcuk]

Turns out I had an old album called "All Photos" with one photo in it. I didn't create it, but when I deleted it everything works fine.

Amazing. That'll explain it then.