Initial report: stack overflow on windows with 0.15

[jbr]

Hi! I'm still trying to understand this, but the following code works with 0.14 and stack overflows on windows with 0.15:

[package]
name = "async-tungstenite-stream-overflow-windows"
version = "0.1.0"
edition = "2018"

[dependencies]
async-global-executor = "2.0.2"
async-net = "1.6.1"
async-tungstenite = "0.15.0"
futures-util = "0.3.17"

fn main() {
    async_global_executor::block_on(async {
        let socket = async_net::TcpStream::connect("localhost:8080").await.unwrap();
        let (mut client, _) = async_tungstenite::client_async("ws://localhost/some/route", socket).await.unwrap();
        use futures_util::{SinkExt, StreamExt};
        client.send(async_tungstenite::tungstenite::Message::text("hello")).await.unwrap();
        dbg!(&client.next().await);  // commenting this line out avoids the stack overflow
    });
}

This expects a websocket echo server of some sort running at :8080, but I believe as long as the server responds with a message, this will overflow the stack on 0.15. Expected output (and 0.14 output) is something like:

&client.next().await = Some(
    Ok(
        Text(
            "received your message: hello",
        ),
    ),
)

Instead, the output is

thread 'main' has overflowed its stack
error: process didn't exit successfully: `target\debug\tungstenite-stream-overflow-windows.exe` (exit code: 0xc00000fd, STATUS_STACK_OVERFLOW)

I don't know nearly anything about windows and have been trying to investigate this on a cloud instance, but someone with a local windows machine will likely have more success. I have no understanding of why this is fine on my local mac.

Let me know if there's any more information I can provide to help investigate this, and thanks for async tungstenite!

[sdroege]

Between 0.14 and 0.15 the only real change was the update to tungstenite 0.15, so that's where I would look for the underlying problem here.

I don't have a Windows machine set up here so can't really look into this, but if you can provide a backtrace of the stackoverlow that might give some thints.

[jbr]

I'll see if I can make a sync tungstenite repro and file over there if so

[sdroege]

Thanks, please link it here then. The problem might of course still be here and be caused by wrong usage of tungstenite :)

[najamelan]

Can confirm there is a stack overflow on linux as well. Glad to see I'm not the only one. I also think it's in tungstenite-rs.

[sdroege]

The testcase at the top does not fail with a stack overflow on Linux for me. Do you have another testcase, @najamelan?

[najamelan]

There is a test for ws_stream_tungstentite that fails with a stack overflow. I suspect the new input buffer they put in tungstenite but unfortunately ws_stream_tungstenite doesn't compile with 0.14. So I can't just bisect it. I will try to create a reproducable example without async-tungstenite and ws_stream to see if it can really be detected on tungstenite alone, because that would simplify things a lot.

In any case if you want to give it a shot you can checkout ws_stream_tungstenite, change deps to be 0.15 and run ci/test.bash. The test ping_pong_async_std fails with a SO.

The last few functions in the stack trace are:

#1  0x000055555562e75a in tungstenite::handshake::machine::HandshakeMachine<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>>::single_round<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>, http::response::Response<()>> (self=...)
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tungstenite-0.15.0/src/handshake/machine.rs:39
#2  0x000055555563fa0e in tungstenite::handshake::MidHandshake<tungstenite::handshake::client::ClientHandshake<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>>>::handshake<tungstenite::handshake::client::ClientHandshake<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>>> (self=...)
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/tungstenite-0.15.0/src/handshake/mod.rs:42
#3  0x000055555564754b in async_tungstenite::client_async_with_config::{generator#0}::{closure#0}<http::request::Request<()>, async_std::net::tcp::stream::TcpStream> (allow_std=...)
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/async-tungstenite-0.15.0/src/lib.rs:131
#4  0x00005555556390e0 in async_tungstenite::handshake::{impl#1}::poll<tungstenite::handshake::client::ClientHandshake<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>>, async_tungstenite::client_async_with_config::{generator#0}::{closure#0}, async_std::net::tcp::stream::TcpStream> (self=..., ctx=0x7ffff7b2fb98)
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/async-tungstenite-0.15.0/src/handshake.rs:138
#5  0x000055555563a746 in async_tungstenite::handshake::handshake::{generator#0}<tungstenite::handshake::client::ClientHandshake<async_tungstenite::compat::AllowStd<async_std::net::tcp::stream::TcpStream>>, async_tungstenite::client_async_with_config::{generator#0}::{closure#0}, async_std::net::tcp::stream::TcpStream> ()
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/async-tungstenite-0.15.0/src/handshake.rs:77
#6  0x00005555556576ed in core::future::from_generator::{impl#1}::poll<async_tungstenite::handshake::handshake::{generator#0}> (self=..., cx=0x7ffff7b2fb98) at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/future/mod.rs:80
#7  0x0000555555639ad5 in async_tungstenite::handshake::client_handshake::{generator#0}<async_tungstenite::client_async_with_config::{generator#0}::{closure#0}, async_std::net::tcp::stream::TcpStream> ()
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/async-tungstenite-0.15.0/src/handshake.rs:99
#8  0x00005555556573cd in core::future::from_generator::{impl#1}::poll<async_tungstenite::handshake::client_handshake::{generator#0}> (self=..., cx=0x7ffff7b2fb98) at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/future/mod.rs:80
#9  0x0000555555646f4b in async_tungstenite::client_async_with_config::{generator#0}<http::request::Request<()>, async_std::net::tcp::stream::TcpStream> ()
    at /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/async-tungstenite-0.15.0/src/lib.rs:133
#10 0x00005555556575dd in core::future::from_generator::{impl#1}::poll<async_tungstenite::client_async_with_config::{generator#0}> (self=..., cx=0x7ffff7b2fb98) at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/future/mod.rs:80

Unfortunately it's not straight forward to know which function uses to much stack space...

[sdroege]

That's quite a small stack trace. The buffer used by HandshakeState::Reading is about 4kB large and on the stack, everything else seems small enough. That's probably multiple times on the stack here as it's also part of the StartedHandshake from async-tungstenite and MidHandshake from tungstenite.

[najamelan]

@sdroege overflow has been found: snapview/tungstenite-rs#241

[sdroege]

Ah good, thanks for notifying. So that was basically the one I pointed out above.

[najamelan]

Ah good, thanks for notifying. So that was basically the one I pointed out above.

Yes, well spotted. tungstenite 0.16 has been released with the fix.

[sdroege]

Thanks for the notification, I'll get a new async-tungstenite 0.16 release with updated dependencies done in a bit.