Skip to content

feat: push speed optimizations#1189

Merged
jescalada merged 9 commits intofinos:mainfrom
jescalada:985-optimize-pullRemote-for-large-repos
Sep 19, 2025
Merged

feat: push speed optimizations#1189
jescalada merged 9 commits intofinos:mainfrom
jescalada:985-optimize-pullRemote-for-large-repos

Conversation

@jescalada
Copy link
Copy Markdown
Contributor

@jescalada jescalada commented Sep 7, 2025

Fixes #985. The profiling screenshots are in the issue discussion.

This PR reduces push times by around 50% by using the depth: 1 and singleBranch: true options in pullRemote when cloning. It also adds the stream: true option to the express-http-proxy configuration, which should help with larger pushes.

I've experimented with other optimizations to the chain actions but it seems that the majority of time is spent cloning the target repository (and therefore limited by download speeds).

I'd say storing the target repository so we only clone the entire repo once is the best option, the tradeoff being that we might need to store several GB of data for larger repos. Perhaps we could have a config option to toggle this so admins can do as they see fit?

@netlify
Copy link
Copy Markdown

netlify Bot commented Sep 7, 2025

Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit 3a60eba
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/68bd6ac32801ad0008e40877

@netlify
Copy link
Copy Markdown

netlify Bot commented Sep 7, 2025

Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit be57ca4
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/68cc08b09b832a0008c0d928

@jescalada jescalada changed the title feat: pullRemote optimizations feat: push speed optimizations Sep 7, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 13, 2025

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 83.72%. Comparing base (c29b56c) to head (be57ca4).
⚠️ Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
src/proxy/processors/push-action/pullRemote.ts 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1189      +/-   ##
==========================================
- Coverage   83.74%   83.72%   -0.03%     
==========================================
  Files          67       67              
  Lines        2892     2888       -4     
  Branches      366      366              
==========================================
- Hits         2422     2418       -4     
  Misses        410      410              
  Partials       60       60              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jescalada jescalada marked this pull request as ready for review September 13, 2025 06:36
@jescalada jescalada requested review from a team, coopernetes and kriswest September 13, 2025 06:36
Copy link
Copy Markdown
Contributor

@coopernetes coopernetes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one minor point on the singleBranch arg.

Comment thread src/proxy/processors/push-action/pullRemote.ts
@jescalada jescalada merged commit 50a1761 into finos:main Sep 19, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

optimize pullRemote for large repos

3 participants