Use --filter=blob:none again, optimize with git 2.49+

[sbidoul]

No description provided.

[sbidoul]

@sbejaoui I re-read the git docs and found out git clone has a new --revision option. So I did some tests, and I now have doubts about #5.

I'm testing with this script, with git-autoshare disabled.

#!/usr/bin/env python3
import subprocess


from pip_preserve_requirements._vcs_registry import vcs_registry

SOURCE_REPO = "https://github.com/OCA/mis-builder.git"
TARGET_REPO = "git@github.com:acsone/mis-builder.git"
SHA = "7dc9e2c80ecc4922064840ab6e235cf7d09e5173"
TAG = "test-perf"

# delete remote tag
subprocess.run(["git", "push", TARGET_REPO, f":{TAG}"])

vcs = vcs_registry("git")
vcs.place_tag_on_commit(
    source_repo=SOURCE_REPO,
    target_repo=TARGET_REPO,
    sha=SHA,
    tag=TAG,
)

With master (clone --bare --depth=1 + fetch sha), it downloads 2.19+3.8 MiB:

Details

To github.com:acsone/mis-builder.git
 - [deleted]         test-perf
Cloning into bare repository '/tmp/tmp7wvc5ozd'...
remote: Enumerating objects: 265, done.
remote: Counting objects: 100% (265/265), done.
remote: Compressing objects: 100% (219/219), done.
remote: Total 265 (delta 66), reused 125 (delta 32), pack-reused 0 (from 0)
Receiving objects: 100% (265/265), 2.19 MiB | 8.43 MiB/s, done.
Resolving deltas: 100% (66/66), done.
remote: Enumerating objects: 12741, done.
remote: Counting objects: 100% (12740/12740), done.
remote: Compressing objects: 100% (3170/3170), done.
remote: Total 12562 (delta 9209), reused 12416 (delta 9069), pack-reused 0 (from 0)
Receiving objects: 100% (12562/12562), 3.80 MiB | 4.26 MiB/s, done.
Resolving deltas: 100% (9209/9209), completed with 98 local objects.
From https://github.com/OCA/mis-builder
 * branch            7dc9e2c80ecc4922064840ab6e235cf7d09e5173 -> FETCH_HEAD
Enumerating objects: 54, done.
Counting objects: 100% (51/51), done.
Delta compression using up to 16 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (8/8), 2.48 KiB | 2.48 MiB/s, done.
Total 8 (delta 4), reused 6 (delta 2), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:acsone/mis-builder.git
 * [new tag]         test-perf -> test-perf
1.14user 0.36system 0:06.53elapsed 23%CPU (0avgtext+0avgdata 18432maxresident)k
0inputs+13784outputs (0major+17988minor)pagefaults     ####0swaps

With this PR and git < 2.49 (clone --bare --filter=blob:none), it downloads 1.77 + 1.13 MiB:

Details

To github.com:acsone/mis-builder.git
 - [deleted]         test-perf
Cloning into bare repository '/tmp/tmp6oo81cj_'...
remote: Enumerating objects: 12875, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 12875 (delta 18), reused 12 (delta 11), pack-reused 12837 (from 2)
Receiving objects: 100% (12875/12875), 1.77 MiB | 7.67 MiB/s, done.
Resolving deltas: 100% (7959/7959), done.
remote: Enumerating objects: 8247, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 8247 (delta 16), reused 10 (delta 10), pack-reused 8224 (from 2)
Receiving objects: 100% (8247/8247), 1.13 MiB | 6.71 MiB/s, done.
Resolving deltas: 100% (5097/5097), done.
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Enumerating objects: 20, done.
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (3/3), 757 bytes | 757.00 KiB/s, done.
Resolving deltas: 100% (2/2), done.
Counting objects: 100% (20/20), done.
Delta compression using up to 16 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 2.16 KiB | 2.17 MiB/s, done.
Total 8 (delta 4), reused 5 (delta 1), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:acsone/mis-builder.git
 * [new tag]           test-perf -> test-perf
0.79user 0.39system 0:05.46elapsed 21%CPU (0avgtext+0avgdata 18816maxresident)k
0inputs+7640outputs (0major+18427minor)pagefaults 0swaps

With this PR and git 2.49 (clone --bare --filter=blob:none --revision=sha), it downloads 1.13 MiB:

Details

To github.com:acsone/mis-builder.git
 - [deleted]         test-perf
Cloning into bare repository '/tmp/tmp27908xa7'...
remote: Enumerating objects: 8247, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 8247 (delta 16), reused 10 (delta 10), pack-reused 8224 (from 2)
Receiving objects: 100% (8247/8247), 1.13 MiB | 6.14 MiB/s, done.
Resolving deltas: 100% (5097/5097), done.
Enumerating objects: 20, done.
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (3/3), 757 bytes | 757.00 KiB/s, done.
Resolving deltas: 100% (2/2), done.
Counting objects: 100% (20/20), done.
Delta compression using up to 16 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 2.16 KiB | 2.17 MiB/s, done.
Total 8 (delta 4), reused 5 (delta 1), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:acsone/mis-builder.git
 * [new tag]         test-perf -> test-perf
0.46user 0.22system 0:04.77elapsed 14%CPU (0avgtext+0avgdata 18944maxresident)k
0inputs+3168outputs (0major+13178minor)pagefaults 0swaps

So with this simple test, it would seem #5 causes more data to be downloaded than before, and using git 2.49 and --revision with the previous algorithm does improve things.

What do you think?