Skip to content

fix: retry on ECONNRESET#1098

Merged
giovanni-guidini merged 7 commits intomainfrom
gio/retry-on-econnreset
Aug 10, 2023
Merged

fix: retry on ECONNRESET#1098
giovanni-guidini merged 7 commits intomainfrom
gio/retry-on-econnreset

Conversation

@giovanni-guidini
Copy link
Copy Markdown
Contributor

Retry the requests POST to codecov and PUT to storage in case of ECONNRESET. Actually it retries in the following cases:

  • errors.UndiciError with code 'ECONNRESET';
  • errors.ConnectTimeoutError;
  • errors.SockerError.

That is because I didn't actually know what error ECONNRESET is. I used UndiciError as the error base class because I noticed we use the request from undici, so I figured that if an error happens then it would probably be wrapped by the lib.

Fixes #537

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 2, 2023

Codecov Report

Merging #1098 (f7f7b94) into main (cba973c) will increase coverage by 0.08%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1098      +/-   ##
==========================================
+ Coverage   92.47%   92.55%   +0.08%     
==========================================
  Files          36       36              
  Lines        1342     1357      +15     
  Branches      272      273       +1     
==========================================
+ Hits         1241     1256      +15     
  Misses         69       69              
  Partials       32       32              
Flag Coverage Δ
aarch64 92.48% <100.00%> (+0.08%) ⬆️
aarch64-without-git 92.48% <100.00%> (+0.08%) ⬆️
alpine 92.48% <100.00%> (+0.08%) ⬆️
alpine-proxy 92.48% <100.00%> (+0.08%) ⬆️
alpine-without-git 92.48% <100.00%> (+0.08%) ⬆️
linux 92.48% <100.00%> (+0.08%) ⬆️
linux-without-git 92.48% <100.00%> (+0.08%) ⬆️
macos 92.48% <100.00%> (+0.08%) ⬆️
macos-without-git 92.48% <100.00%> (+0.08%) ⬆️
windows 91.08% <45.83%> (-0.73%) ⬇️
windows-without-git 91.08% <45.83%> (-0.73%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
src/helpers/util.ts 76.47% <100.00%> (+1.47%) ⬆️
src/helpers/web.ts 85.00% <100.00%> (+3.18%) ⬆️

Retry the requests POST to codecov and PUT to storage in case of ECONNRESET.
Actually it retries in the following cases:
* `errors.UndiciError` with code `'ECONNRESET'`;
* `errors.ConnectTimeoutError`;
* `errors.SockerError`.

That is because I didn't actually know what error ECONNRESET is.
I used `UndiciError` as the error base class because I noticed we use the `request` from `undici`,
so I figured that if an error happens then it would probably be wrapped by the lib.

Fixes #537
Increasing the retry delay and logging the error received when POSTing to Codecov.
Now we will retry after 1, 2, 4 and 8 seconds (so total wait of 15s + the request times)

I had to move the sleep function to another module because it's the only way I managed to mock it.
I tried several different mocking arrangements but that was the one that sticked.
It does have the benefit of making it reusable, so I guess it's not that bad.
Retry the requests POST to codecov and PUT to storage in case of ECONNRESET.
Actually it retries in the following cases:
* `errors.UndiciError` with code `'ECONNRESET'`;
* `errors.ConnectTimeoutError`;
* `errors.SockerError`.

That is because I didn't actually know what error ECONNRESET is.
I used `UndiciError` as the error base class because I noticed we use the `request` from `undici`,
so I figured that if an error happens then it would probably be wrapped by the lib.

Fixes #537
Increasing the retry delay and logging the error received when POSTing to Codecov.
Now we will retry after 1, 2, 4 and 8 seconds (so total wait of 15s + the request times)

I had to move the sleep function to another module because it's the only way I managed to mock it.
I tried several different mocking arrangements but that was the one that sticked.
It does have the benefit of making it reusable, so I guess it's not that bad.
@giovanni-guidini giovanni-guidini force-pushed the gio/retry-on-econnreset branch from 06cb5bc to 58b33de Compare August 8, 2023 17:14
import { IServiceParams, PostResults, UploaderArgs, UploaderEnvs } from '../../src/types.js'
import { createEmptyArgs } from '../test_helpers'

import * as util_module from '../../src/helpers/util'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use utilModule, this is JS world 😆

Copy link
Copy Markdown
Contributor

@thomasrockhu-codecov thomasrockhu-codecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending var change

@giovanni-guidini giovanni-guidini enabled auto-merge (squash) August 10, 2023 16:15
@giovanni-guidini giovanni-guidini merged commit acbf664 into main Aug 10, 2023
@giovanni-guidini giovanni-guidini deleted the gio/retry-on-econnreset branch August 10, 2023 16:32
} catch (error: unknown) {
if (
((error instanceof errors.UndiciError && error.code == 'ECONNRESET') ||
error instanceof errors.ConnectTimeoutError ||
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this doesn't catch ETIMEDOUT.

@webknjaz
Copy link
Copy Markdown
Contributor

@giovanni-guidini apparently, this doesn't cover all the cases. See: #1180.

thomasrockhu-codecov pushed a commit that referenced this pull request Oct 10, 2023
Retry the requests POST to codecov and PUT to storage in case of ECONNRESET.
Actually it retries in the following cases:
* `errors.UndiciError` with code `'ECONNRESET'`;
* `errors.ConnectTimeoutError`;
* `errors.SockerError`.

That is because I didn't actually know what error ECONNRESET is.
I used `UndiciError` as the error base class because I noticed we use the `request` from `undici`,
so I figured that if an error happens then it would probably be wrapped by the lib.
We will retry after 1, 2, 4 and 8 seconds (so total wait of 15s + the request times)

I had to move the sleep function to another module because it's the only way I managed to mock it.
I tried several different mocking arrangements but that was the one that sticked.
It does have the benefit of making it reusable, so I guess it's not that bad.

Fixes #537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The uploader should retry on ECONNRESET but it doesn't

3 participants