[RAW] RAW (.dng) files are not being downloaded

[ethereal-engineer]

Overview

I'm trying to exit the Apple iCloud Photos system, but I want all my data. Running iCloud Photos Downloader seems to download all images, but it doesn't even mention the DNG files. I checked that they're there on iCloud itself. Help! (Please)

Steps to Reproduce

1. Create a virtual environment python -m venv venv and source it source venv/bin/activate
2. Install icloudpd pip install icloudpd
3. Set it going sudo -u www-data ./icloudpd -d /********/Photos -u "******@*******" --set-exif-datetime
4. Make about fifteen thousand cups of coffee
5. Run it again just to verify that everything is there
6. DNGs are not there at all

Expected Behavior

ALL content would be downloaded from iCloud Photos to the destination

Actual Behavior

No DNG files have been downloaded from iCloud Photos. I hope there are no others missing!?

Context

Python 3.10.12
Linux ******* 5.15.0-78-generic #85-Ubuntu SMP Fri Jul 7 15:25:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Ext4 on LVM2

[ethereal-engineer]

It's worth noting that my capture method for RAW is using Halide set to RAW + HEIC, which results in two files with the same file name prefix, but differing extensions (HEIC, and dng).

[Ternoc]

Same for CR2 files.

It downloads only the JPG file.

[AndreyNikiforov]

Same for CR2 files.

It downloads only the JPG file.

How are you getting CR2 files into iCloud? How are they presented in icloud.com UI? How they are handled in icloud.com UI (e.g. downloads)?

[Ternoc]

I put CR2 files using the importer in the macos photos app, which automatically import photos (both JPG and CR2 files) from my SD card into the cloud.

On icloud.com ui, it is presented as RAW+JPEG, as shown here.

When I use the download button, It downloads only the jpg file. However, I can use more download option and then select Unmodified Originals to download both files

[smaugfm]

Is there any progress on this?

It seems that iCloud stores RAW version of the photos in resOriginalAltRes field. PyIcloud support was added here.

I see that this PR only addresses the situation when there is a single RAW file, but not RAW + JPEG. I.e. only resOriginalRes but not resOriginalRes + resOriginalAltRes. resOriginalAltRes is currently ignored by icloud-photos-downloader.

The solution might be to support resOriginalAlrRes with original_alt size option and then to allow for multiple sizes of the photo to be downloaded, as opposed to currently only one. For example: --size original, original_alt.

Also, there might be a problem in restoring original file extension of the resOriginalAltRes contents as it is not stored anywhere in the _master_record['fields']. The only thing that is stored is a filetype description, i.e. com.olympus.raw-image for Olympus ORF files or com.canon.cr2-raw-image for Canon CR2 files etc. So a hardcoded mapping might be needed, like one maintained in another similar iCloud photo sync tool: file-type.ts

In theory, I can draft a PR with support for multiple sizes and original_alt, but I am not very productive in Python and would need help in writing tests and finishing it.

[AndreyNikiforov]

Is there any progress on this?

It seems that iCloud stores RAW version of the photos in resOriginalAltRes field. PyIcloud support was added here.

I see that this PR only addresses the situation when there is a single RAW file, but not RAW + JPEG. I.e. only resOriginalRes but not resOriginalRes + resOriginalAltRes. resOriginalAltRes is currently ignored by icloud-photos-downloader.

The solution might be to support resOriginalAlrRes with original_alt size option and then to allow for multiple sizes of the photo to be downloaded, as opposed to currently only one. For example: --size original, original_alt.

Also, there might be a problem in restoring original file extension of the resOriginalAltRes contents as it is not stored anywhere in the _master_record['fields']. The only thing that is stored is a filetype description, i.e. com.olympus.raw-image for Olympus ORF files or com.canon.cr2-raw-image for Canon CR2 files etc. So a hardcoded mapping might be needed, like one maintained in another similar iCloud photo sync tool: file-type.ts

In theory, I can draft a PR with support for multiple sizes and original_alt, but I am not very productive in Python and would need help in writing tests and finishing it.

• reproducing responses from icloud for raw+jpeg would be better then compiling support from multiple other project with unclarified source of info; I'll see if I can get raw+jpeg from old cannon
• writing tests seems to be hard although vcr supposed to make it easier

[AndreyNikiforov]

1.19.0 can download raw from different manufacturers as well as handle raw+jpeg

[ethereal-engineer]

Well done 👍🏻 On 1 Jun 2024, at 4:29 PM, Andrey Nikiforov ***@***.***> wrote: 1.19.0 can download raw from different manufacturers as well as handle raw+jpeg —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>