Skip to content
This repository was archived by the owner on Mar 3, 2026. It is now read-only.

fix: File#save iterable fixes#2293

Merged
danielbankhead merged 5 commits intomainfrom
file-save-iterable-fixes
Sep 6, 2023
Merged

fix: File#save iterable fixes#2293
danielbankhead merged 5 commits intomainfrom
file-save-iterable-fixes

Conversation

@danielbankhead
Copy link
Copy Markdown
Contributor

@danielbankhead danielbankhead commented Sep 6, 2023

Fixes #2278 🦕

@danielbankhead danielbankhead requested review from a team September 6, 2023 18:05
@product-auto-label product-auto-label Bot added size: s Pull request size is small. api: storage Issues related to the googleapis/nodejs-storage API. labels Sep 6, 2023
Comment thread src/file.ts
}
const returnValue = retry(
async (bail: (err: Error) => void) => {
if (data instanceof Readable) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this logic as for the vast majority it would simply slow down their uploads unnecessarily. Additionally, later versions of Node.js catch this for us.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what versions catch this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node 18.0.0:

Here's a simple reproduction for a REPL (anything before 18 the console.log isn't called):

const readable = stream.Readable({read() {this.push(null)}});
readable.read();
readable.readable // false
stream.pipeline(readable, stream.PassThrough, () => console.log('done'));

@product-auto-label product-auto-label Bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Sep 6, 2023
Comment thread test/file.ts
file.save(generator(), options, (err: Error) => {
assert.strictEqual(
err.message,
'The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of Object'
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just testing Node internals. Removed.

Comment thread src/file.ts
Comment on lines -3690 to -3695
// If data is not a valid PipelineSource, then pipeline will
// fail without destroying the writable stream. If data is a
// PipelineSource that yields invalid chunks (e.g. a stream in
// object mode or an iterable that does not yield Buffers or
// strings), then pipeline will destroy the writable stream.
if (!writable.destroyed) writable.destroy();
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This case doesn't happen - instead, Node throws immediately and this callback is never called.

@danielbankhead danielbankhead added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 6, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 6, 2023
@danielbankhead danielbankhead merged commit 87c3f41 into main Sep 6, 2023
@danielbankhead danielbankhead deleted the file-save-iterable-fixes branch September 6, 2023 19:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: storage Issues related to the googleapis/nodejs-storage API. size: m Pull request size is medium.

Projects

None yet

4 participants