Merged
Conversation
|
One of the tests failed for 14f9ff1. @admin check logs None, packit dashboard https://dashboard.packit.dev/jobs/srpm/575978 and external service dashboard https://copr.fedorainfracloud.org/coprs/build/10274458/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add chunked transfer-encoding support to HTTP client
Problem
The synchronous and asynchronous request paths in
glz::http_clientonly read response bodies based on theContent-Lengthheader. When a server usesTransfer-Encoding: chunked(common with proxies like Cloudflare, streaming APIs, and many HTTP/1.1 servers), theContent-Lengthheader is absent. The client returns a 200 status with headers parsed correctly but an emptyresponse_body.Reported in #2410.
Changes
include/glaze/net/http_client.hppSynchronous path (
perform_sync_request):Transfer-Encoding: chunkedheader during response parsing0chunk;key=value) by ignoring themprotocol_erroron malformed chunk sizesAsynchronous path (
parse_and_read_body):async_read_chunked_body/async_read_chunk_datamethods implementing chunked decoding via recursive async readsasync_consume_trailersmethod to skip trailer headers before building the response and returning the connection to the poolPortability fix:
strncasecmp(POSIX-only) withglz::strncasecmpin the sync path header parsing forContent-Length,Transfer-Encoding, andConnectionheadersTests (
tests/networking_tests/http_chunked_test/)New test suite with 43 tests covering:
get_async/post_asyncfutures, concurrent async requests to mixed endpoints, callback-based async with large bodystream_request_v2withon_datacallbacks for single/multi/large/empty chunked responsesConnection: close, connection reuse after chunked response, malformed chunk sizes, empty chunk size lines, server disconnect mid-chunk, trailer headers after terminal chunk, trailer + connection reuse, leading zeros in chunk size, multiple transfer encodings (gzip, chunked)Documentation (
docs/networking/max-response-body-size.md)Proposal for a future configurable maximum response body size to guard against unbounded memory growth from malicious servers. This applies equally to content-length and chunked paths and is tracked separately from this change.