Skip to content

Instanced buffer size#1354

Merged
adamhathcock merged 8 commits into
masterfrom
adam/instance-buffer-size
Jun 13, 2026
Merged

Instanced buffer size#1354
adamhathcock merged 8 commits into
masterfrom
adam/instance-buffer-size

Conversation

@adamhathcock

Copy link
Copy Markdown
Owner

Addresses #1351

This pull request introduces configurable buffer sizes for extraction and writing operations, allowing users to control the memory usage and performance of archive operations. The changes add a BufferSize property to both ExtractionOptions and WriterOptions, update the relevant APIs and extension methods to use these buffer sizes, and update documentation and usage examples accordingly.

Configurable Buffer Size for Extraction and Writing:

  • Added a BufferSize property to ExtractionOptions and WriterOptions (and their interfaces) to allow users to specify the buffer size used during stream copy operations. Default is set to Constants.BufferSize (81920 bytes). [1] [2] [3]
  • Updated IArchiveEntryExtensions, IReaderExtensions, and IAsyncReaderExtensions to use the buffer size from the provided options, falling back to the default if not specified. This includes both synchronous and asynchronous extraction methods. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
  • Refactored internal reader and writer implementations to propagate the buffer size from options, ensuring consistent usage throughout the codebase. [1] [2] [3]

Documentation and Usage Updates:

  • Updated documentation (API.md, USAGE.md) to show how to set the buffer size in ExtractionOptions and WriterOptions, and explained the effect of the buffer size on extraction and writing. [1] [2] [3] [4] [5]

Other Notable Changes:

  • Bumped the csharpier tool version in .config/dotnet-tools.json from 1.2.6 to 1.3.0.
  • Minor refactoring for clarity and maintainability, such as extracting helper methods for progress reporting and stream copying. [1] [2]

These changes give users more control over performance and resource usage during archive extraction and writing, and make the API more flexible and transparent regarding buffer management.

Copilot AI review requested due to automatic review settings June 12, 2026 13:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements configurable stream-copy buffer sizes during extraction and writing, exposing a new BufferSize option on ExtractionOptions and WriterOptions (and propagating it through readers/writers and docs) to help performance on slow/high-latency destinations (e.g., SMB network shares).

Changes:

  • Add BufferSize to extraction/writer options (and interfaces) with defaults based on Constants.BufferSize.
  • Thread buffer size through reader/writer copy paths (sync/async) and add tests validating propagation.
  • Update docs/examples and bump csharpier tool version.

Review findings (changes needed):

  • Stream.TransferTo in src/SharpCompress/Utility.cs changed signature from (destination, maxLength) to (destination, maxLength, bufferSize) without keeping the old overload or making the new parameter optional. If this is a public API (it appears to be, since tests call it cross-assembly), this is a breaking change for existing consumers. Consider restoring the old overload and delegating to the new one, and/or making the bufferSize parameter optional (while still keeping the old overload if binary compatibility matters).
  • tests/SharpCompress.Test/OptionsUsabilityTests.cs is now entirely wrapped in #if !LEGACY_DOTNET, which removes all existing tests in that file from legacy target runs—not just the newly added buffer-size tests. If only some new code requires the guard, consider scoping the preprocessor directive to only the new tests/helpers so prior coverage isn’t lost.
  • src/SharpCompress/Readers/IReaderExtensions.cs / IAsyncReaderExtensions.cs moved from reader.WriteEntryTo(fs) to manually opening the entry stream and copying. This bypasses AbstractReader.WriteEntryTo’s internal “already wrote this entry” guard/state (_wroteCurrentEntry) visible in the diff, which is a behavioral change. Consider adding a buffer-size-aware entry write path that preserves the reader’s write semantics/state tracking, rather than duplicating the copy logic in extensions.
  • docs/API.md states “If it is not set, SharpCompress falls back to Constants.BufferSize”, but WriterOptions.BufferSize is a non-nullable int with a default initializer—so it is always set. Consider rewording to “Defaults to Constants.BufferSize”.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/SharpCompress.Test/UtilityTests.cs Updates TransferTo tests for new signature.
tests/SharpCompress.Test/OptionsUsabilityTests.cs Adds buffer-size propagation tests; introduces #if !LEGACY_DOTNET guard for entire file.
src/SharpCompress/Writers/Zip/ZipWriterOptions.cs Adds BufferSize to zip writer options and copy constructors.
src/SharpCompress/Writers/Zip/ZipWriter.cs Uses WriterOptions.BufferSize for entry stream copy.
src/SharpCompress/Writers/WriterOptionsExtensions.cs Adds fluent WithBufferSize helper.
src/SharpCompress/Writers/WriterOptions.cs Adds BufferSize to common writer options.
src/SharpCompress/Writers/Tar/TarWriterOptions.cs Adds BufferSize to tar writer options and copy constructors.
src/SharpCompress/Writers/Tar/TarWriter.cs Uses configured buffer size when transferring entry data.
src/SharpCompress/Writers/Tar/TarWriter.Async.cs Async transfer uses configured buffer size.
src/SharpCompress/Writers/SevenZip/SevenZipWriterOptions.cs Adds BufferSize to 7z writer options and copy constructors.
src/SharpCompress/Writers/GZip/GZipWriterOptions.cs Adds BufferSize to gzip writer options and copy constructors.
src/SharpCompress/Writers/GZip/GZipWriter.cs Uses WriterOptions.BufferSize for entry stream copy.
src/SharpCompress/Utility.cs Changes TransferTo signature and applies buffer size to CopyTo.
src/SharpCompress/Utility.Async.cs Adds optional bufferSize parameter to TransferToAsync.
src/SharpCompress/Readers/IReaderExtensions.cs Copies entry streams using extraction buffer size (new helper methods).
src/SharpCompress/Readers/IAsyncReaderExtensions.cs Async copy path uses extraction buffer size (new helper methods).
src/SharpCompress/Readers/AbstractReader.cs Plumbs buffer size into internal copy and WriteEntryTo.
src/SharpCompress/Readers/AbstractReader.Async.cs Async copy now uses Options.BufferSize; refactors some internal methods.
src/SharpCompress/Common/Options/IWriterOptions.cs Adds BufferSize to writer options interface.
src/SharpCompress/Common/Options/IExtractionOptions.cs Adds BufferSize to extraction options interface.
src/SharpCompress/Common/ExtractionOptions.cs Adds BufferSize with default to extraction options.
src/SharpCompress/Common/Constants.cs Notes TODO about remaining non-extraction usages.
src/SharpCompress/Archives/IArchiveEntryExtensions.cs Adds buffer-size-aware internal overloads; uses extraction buffer size for file extraction.
docs/USAGE.md Updates extraction example to show BufferSize.
docs/API.md Updates writer/extraction examples and describes BufferSize.
.config/dotnet-tools.json Bumps csharpier tool version.

@adamhathcock adamhathcock changed the title nstance buffer size Instanced buffer size Jun 12, 2026
Copilot AI review requested due to automatic review settings June 12, 2026 14:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

@adamhathcock adamhathcock merged commit 668d599 into master Jun 13, 2026
6 checks passed
@adamhathcock adamhathcock deleted the adam/instance-buffer-size branch June 13, 2026 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants