Skip to content

HTTP Client Chunked Support#2414

Merged
stephenberry merged 4 commits intomainfrom
chunked_http
Mar 31, 2026
Merged

HTTP Client Chunked Support#2414
stephenberry merged 4 commits intomainfrom
chunked_http

Conversation

@stephenberry
Copy link
Copy Markdown
Owner

@stephenberry stephenberry commented Mar 30, 2026

Add chunked transfer-encoding support to HTTP client

Problem

The synchronous and asynchronous request paths in glz::http_client only read response bodies based on the Content-Length header. When a server uses Transfer-Encoding: chunked (common with proxies like Cloudflare, streaming APIs, and many HTTP/1.1 servers), the Content-Length header is absent. The client returns a 200 status with headers parsed correctly but an empty response_body.

Reported in #2410.

Changes

include/glaze/net/http_client.hpp

Synchronous path (perform_sync_request):

  • Detect Transfer-Encoding: chunked header during response parsing
  • Implement a chunked decoding loop: read hex chunk size, read chunk data, append to body, repeat until the terminal 0 chunk
  • Properly skip optional trailer headers after the terminal chunk (RFC 7230 §4.1)
  • Handle chunk extensions (;key=value) by ignoring them
  • Return protocol_error on malformed chunk sizes

Asynchronous path (parse_and_read_body):

  • Same header detection for chunked encoding
  • New async_read_chunked_body / async_read_chunk_data methods implementing chunked decoding via recursive async reads
  • New async_consume_trailers method to skip trailer headers before building the response and returning the connection to the pool

Portability fix:

  • Replace bare strncasecmp (POSIX-only) with glz::strncasecmp in the sync path header parsing for Content-Length, Transfer-Encoding, and Connection headers

Tests (tests/networking_tests/http_chunked_test/)

New test suite with 43 tests covering:

  • Sync path (14 tests): single chunk, multi-chunk, empty body, large body (100KB), binary data with null bytes, custom headers, many tiny chunks, varying chunk sizes, POST echo, sequential requests, interleaved chunked/content-length, concurrent requests
  • Async path (10 tests): mirrors of key sync tests via get_async/post_async futures, concurrent async requests to mixed endpoints, callback-based async with large body
  • Streaming path (4 tests): stream_request_v2 with on_data callbacks for single/multi/large/empty chunked responses
  • Raw socket edge cases (15 tests): hand-crafted HTTP responses testing basic chunked decoding, single-byte chunks, chunk extensions, uppercase/lowercase hex, empty body, large hex sizes, case-insensitive headers, Connection: close, connection reuse after chunked response, malformed chunk sizes, empty chunk size lines, server disconnect mid-chunk, trailer headers after terminal chunk, trailer + connection reuse, leading zeros in chunk size, multiple transfer encodings (gzip, chunked)

Documentation (docs/networking/max-response-body-size.md)

Proposal for a future configurable maximum response body size to guard against unbounded memory growth from malicious servers. This applies equally to content-length and chunked paths and is tracked separately from this change.

@stephenberry stephenberry linked an issue Mar 30, 2026 that may be closed by this pull request
@packit-as-a-service
Copy link
Copy Markdown

One of the tests failed for 14f9ff1. @admin check logs None, packit dashboard https://dashboard.packit.dev/jobs/srpm/575978 and external service dashboard https://copr.fedorainfracloud.org/coprs/build/10274458/

@stephenberry stephenberry merged commit 00a73d0 into main Mar 31, 2026
48 of 49 checks passed
@stephenberry stephenberry deleted the chunked_http branch March 31, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Glaze Http Client - Get Request: No data in response_body

1 participant