Skip to content

feat: add HTTP/2, HTTP/3, and QUIC protocol support#26776

Open
jupilhwang wants to merge 4 commits intovlang:masterfrom
jupilhwang:http2_http3
Open

feat: add HTTP/2, HTTP/3, and QUIC protocol support#26776
jupilhwang wants to merge 4 commits intovlang:masterfrom
jupilhwang:http2_http3

Conversation

@jupilhwang
Copy link
Copy Markdown

Implement complete HTTP/2 (RFC 7540/7541), HTTP/3 (RFC 9114/9204), and QUIC (RFC 9000/9001) protocol support for the V standard library.

HTTP/2 (net.http.v2):

  • HPACK header compression with Huffman coding and O(1) static table
  • All 10 frame types with padding, CONTINUATION flood protection (CVE-2024-27316)
  • Stream multiplexing, flow control (bidirectional), stream state machine
  • TLS (h2) and plain TCP (h2c) server modes with h2c upgrade mechanism
  • Connection pooling, CONNECT tunneling, GREASE, cookie compression
  • Request/response validation per RFC 7540 Section 8

HTTP/3 (net.http.v3):

  • QPACK header compression with ring buffer dynamic table and blocked stream queueing
  • 17 H3 error codes, control/encoder/decoder unidirectional streams
  • 2-phase GOAWAY graceful shutdown, background control stream reader
  • Alt-Svc discovery and caching, GREASE support
  • Request validation, header lowercase enforcement per RFC 9114

QUIC (net.quic):

  • ngtcp2 C bindings with TLS 1.3 crypto (AES-128-GCM, HKDF, header protection)
  • Connection migration with PATH_CHALLENGE/RESPONSE and NAT rebinding
  • 0-RTT session resumption with anti-replay cache and ticket extraction
  • CONNECTION_CLOSE frames, idle timeout monitoring
  • CID-based packet matching, flow control exposure

Integration (net.http):

  • Version negotiation with automatic HTTP/2/3 selection
  • ALPN get_alpn_selected() added to both mbedtls and OpenSSL backends
  • Alt-Svc header parsing and HTTP/3 endpoint discovery
  • 421 Misdirected Request handling

Security:

  • CONTINUATION flood protection, max header/body size limits
  • Connection count limits, forbidden cipher blacklist
  • Thread-safe flow control, pools, caches with sync.Mutex
  • Never-indexed HPACK encoding for sensitive headers
  • Single-allocation AEAD encryption (zero-copy)

Tests: 37 test files, all passing (19 HTTP/2 + 12 HTTP/3 + 5 QUIC + 1 Alt-Svc)

External dependencies: ngtcp2, ngtcp2_crypto_ossl, OpenSSL 3.x

Implement complete HTTP/2 (RFC 7540/7541), HTTP/3 (RFC 9114/9204),
and QUIC (RFC 9000/9001) protocol support for the V standard library.

HTTP/2 (net.http.v2):
- HPACK header compression with Huffman coding and O(1) static table
- All 10 frame types with padding, CONTINUATION flood protection (CVE-2024-27316)
- Stream multiplexing, flow control (bidirectional), stream state machine
- TLS (h2) and plain TCP (h2c) server modes with h2c upgrade mechanism
- Connection pooling, CONNECT tunneling, GREASE, cookie compression
- Request/response validation per RFC 7540 Section 8

HTTP/3 (net.http.v3):
- QPACK header compression with ring buffer dynamic table and blocked stream queueing
- 17 H3 error codes, control/encoder/decoder unidirectional streams
- 2-phase GOAWAY graceful shutdown, background control stream reader
- Alt-Svc discovery and caching, GREASE support
- Request validation, header lowercase enforcement per RFC 9114

QUIC (net.quic):
- ngtcp2 C bindings with TLS 1.3 crypto (AES-128-GCM, HKDF, header protection)
- Connection migration with PATH_CHALLENGE/RESPONSE and NAT rebinding
- 0-RTT session resumption with anti-replay cache and ticket extraction
- CONNECTION_CLOSE frames, idle timeout monitoring
- CID-based packet matching, flow control exposure

Integration (net.http):
- Version negotiation with automatic HTTP/2/3 selection
- ALPN get_alpn_selected() added to both mbedtls and OpenSSL backends
- Alt-Svc header parsing and HTTP/3 endpoint discovery
- 421 Misdirected Request handling

Security:
- CONTINUATION flood protection, max header/body size limits
- Connection count limits, forbidden cipher blacklist
- Thread-safe flow control, pools, caches with sync.Mutex
- Never-indexed HPACK encoding for sensitive headers
- Single-allocation AEAD encryption (zero-copy)

Tests: 37 test files, all passing (19 HTTP/2 + 12 HTTP/3 + 5 QUIC + 1 Alt-Svc)

External dependencies: ngtcp2, ngtcp2_crypto_ossl, OpenSSL 3.x
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ede3439c36

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread vlib/net/http/request_version.v Outdated
Comment thread vlib/net/http/request_version.v Outdated
Comment thread vlib/net/http/request_version.v Outdated
Comment thread vlib/net/http/v2/client.v Outdated
@changrui
Copy link
Copy Markdown
Contributor

changrui commented Mar 26, 2026

Maybe module net.http.http2, module net.http.http3 and module net.http.quic.

@JalonSolov
Copy link
Copy Markdown
Collaborator

Personally, I'd rather keep the single net.http, but the code negotiates with the other end of the connection as to the best style to use.

That way I don't write something that does import net.http.quic and then fails to work with a lot of sites. I also don't need 4 different imports to make a connection work.

@jupilhwang
Copy link
Copy Markdown
Author

I am currently modifying it so that http, http2, and http3 can all be used with a single net.http import.

… review fixes

QUIC FIN layer:
- Add NGTCP2_WRITE_STREAM_FLAG_FIN/MORE constants and flags parameter
  to conn_writev_stream
- Register recv_stream_data and stream_close ngtcp2 callbacks with
  FIN detection and overflow-safe event buffering
- Add send_fin(), send_with_fin(), send_with_flags() methods
- Add drain_stream_events() with error propagation on overflow
- Add ensure_stream(), stream_has_fin(), stream_exists() abstraction API
- Auto-create stream entries for FIN events on unknown streams

HTTP/3 FIN integration:
- Replace non-standard empty DATA frame end-marker with proper QUIC
  FIN signaling per RFC 9114 §4.1
- Client sends FIN after last frame via send_frame_with_fin()
- Server detects request completion via check_fin_completions() sweep
  after frame processing, handling separate-packet FIN and empty-body
  POST/PUT/PATCH
- Server coalesces response FIN with last data write
- Per-connection packet_mu mutex serializing QUIC state mutations
- Split process_packet_frames into ingest/decode/dispatch helpers

HTTP/1 hardening:
- Add max_request_body_size (10MB default) to Server struct matching
  HTTP/2 and HTTP/3 defaults
- Add parse_request_with_limit() checking Content-Length before allocation
- Strict Content-Length validation rejecting negative, non-numeric, and
  overflow values via validate_and_parse_content_length()
- Detect truncated request bodies (unexpected EOF)
- Backward-compatible Handler interface with ServerHandler adapter
@Jengro777
Copy link
Copy Markdown
Contributor

How's that going?

@jupilhwang
Copy link
Copy Markdown
Author

How's that going?

I already committed the fix for single import.

@Jengro777
Copy link
Copy Markdown
Contributor

@JalonSolov

@JalonSolov
Copy link
Copy Markdown
Collaborator

Look at the list of conflicts that need to be resolved. Can't merge until those are fixed.

@Jengro777
Copy link
Copy Markdown
Contributor

Look at the list of conflicts that need to be resolved. Can't merge until those are fixed.

Recently, a large number of submissions have indeed put some pressure on merging new PRs.

@medvednikov
Copy link
Copy Markdown
Member

  1. P0: net.http does not compile
    /tmp/v-pr-26776/vlib/net/http/request.v:314 calls req.ssl_do(nport, method, host_name,
    path) with 4 args, but ssl_do now requires 6: (port, method, host, path, data, header).
    This breaks any program importing net.http, plus request_test.v, alt_svc_test.v,
    examples, and CI vpm builds.
  2. P1: import net.http now requires QUIC/ngtcp2 native libraries
    /tmp/v-pr-26776/vlib/net/http/request_version.v:7 imports net.http.v3, which imports
    net.quic; /tmp/v-pr-26776/vlib/net/quic/ngtcp2.c.v:11 unconditionally links -lngtcp2,
    -lngtcp2_crypto_ossl, -lssl, -lcrypto.
    That makes HTTP/1.1-only users depend on external HTTP/3 native deps. This is a broad
    stdlib regression.
  3. P1: HTTP/3/QUIC receive path cannot deliver response/request bodies correctly
    /tmp/v-pr-26776/vlib/net/quic/quic_stubs.c:120 receives stream data but explicitly
    ignores data and datalen; /tmp/v-pr-26776/vlib/net/quic/connection_io.v:163 then returns
    stream.data, which is never populated from the peer.
    Separately, /tmp/v-pr-26776/vlib/net/http/v3/server_packet.v:51 decrypts raw UDP packets
    and parses them as HTTP/3 frames, but HTTP/3 frames live inside QUIC stream frames. This
    likely passes unit tests while failing real interop.
  4. P1: http.Server.handler is a public API break
    /tmp/v-pr-26776/vlib/net/http/server.v:23 changes Server.handler from the old Handler
    interface to ServerHandler. The old adapter exists in /tmp/v-pr-26776/vlib/net/http/
    server_adapters.v:20, but existing code using http.Server{ handler: MyHandler{} } no
    longer compiles unless manually wrapped.
  5. P2: request-line parsing regresses double-slash paths
    /tmp/v-pr-26776/vlib/net/http/request.v:981 changed request-target parsing from
    parse_request_uri to parse. For GET //another.html HTTP/1.1, this now treats
    another.html as the host instead of preserving path //another.html. The PR also removes
    the tests that covered this case.
  6. P2: header override logic uses the wrong header source
    /tmp/v-pr-26776/vlib/net/http/request.v:331 accepts a per-attempt header argument, but
    default suppression checks still read req.header. Redirect handling mutates local
    method, data, and header, so this can produce duplicate or stale Host, User-Agent, and
    Content-Length.
  7. P2: a new HTTP/2 test file is stale and does not compile
    /tmp/v-pr-26776/vlib/net/http/v2/integration_full_test.v:57 initializes
    ServerResponse{ headers: ... }, but the struct field is header. ./vnew -gc none vlib/
    net/http/v2/integration_full_test.v fails with four unknown field headers errors.
  8. P2: TLS verification/mTLS settings are not forwarded into the HTTP/2 path
    /tmp/v-pr-26776/vlib/net/http/request_version.v:64 builds a fresh v2 client from the
    address only. /tmp/v-pr-26776/vlib/net/http/v2/client.v:23 creates TLS with only ALPN.
    Existing http.Request settings like verify, cert, cert_key, and in-memory verification
    are ignored.

…ectness issues

- Guard net.http.v3/net.quic imports behind -d use_ngtcp2 compile flag
  so HTTP/1.1/2 users do not require ngtcp2 native libraries (P1)
- Forward TLS verify/cert/cert_key/validate settings to HTTP/2 client
  path, fixing silent certificate validation bypass (P2-security)
- Fix QUIC receive callback to capture stream data instead of
  discarding data/datalen parameters (P1)
- Fix double-slash path parsing regression using parse_request_uri
  instead of urllib.parse for request-targets (P2)
- Fix redirect 303 to switch method to GET and drop body (P2)
- Fix HTTP/2 integration_full_test.v: headers -> header field name (P2)
- Harden C/V struct interop: use i32 for QuicStreamEvents fields,
  add struct size assertion test
- Sanitize DebugHandler to not echo sensitive request data
…iew findings

Critical fixes (17 resolved):
- Header.add()/set() now return errors on overflow instead of
  silent header drops that could lose Content-Length/Authorization
- Remove unsafe mutation of immutable request during 303 redirects;
  use local effective_data variable instead of UB cast
- set_custom() iterates only populated header slots (cur_pos), not
  the full 50-element fixed array
- Server worker threads receive channel close signal on shutdown
  instead of leaking permanently
- HTTP/2 stream state violations now return PROTOCOL_ERROR per
  RFC 7540 §5.1, not silently ignored log messages
- HTTP/2 stream ID overflow check (>0x7FFFFFFF) prevents reuse
- Connection pool evicts stale/closed connections before returning
- QUIC RAND_bytes failure detected with arc4random fallback
- QUIC timestamps use monotonic clock (sys_mono_now) instead of
  wall clock that drifts with NTP/DST corrections (6 occurrences)
- Negative offset/length bounds check in QUIC stream data events

High fixes (12 resolved):
- Retry loops return explicit max-retries error, not misleading
  "unsupported scheme"
- Body boundary detection uses >= 0 (not > 0) for position check
- URL params properly encoded with query_escape in fetch()
- HTTP/2 unknown SETTINGS return Option (none) per RFC 7540 §6.5.2
- encode_optimized adds never-indexed check for sensitive headers
  (authorization, cookie) preventing intermediary indexing
- ConnectionPool.size() acquires mutex for thread safety
- Extension HTTP methods (PROPFIND, BREW) no longer rejected
- IPv6 address parsing supports bracket notation [::1]:port
- README Quick Start examples rewritten to match actual API
- Empty/println-only QUIC tests replaced with real assertions
- Certificate generation command fixed (separate key.pem/cert.pem)
@medvednikov
Copy link
Copy Markdown
Member

Thanks. Conflicts now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants