upgrade rust and tokio + set max threads to num of cpus

[depombo]

No description provided.

[dfellis]

This doesn't actually fix the real issue of #354 which is that we're still creating a tokio task per array element, which explodes the AVM with an OOM on large arrays unnecessarily (also because we duplicate the working memory for each task right now, so we get array size to the array size amount of memory consumption)

[dfellis]

Is this the Rust version? Or why did you change it?

[depombo]

this is the Rust version

[dfellis]

Not according to: https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field

[dfellis]

It appears you can't specify the Rust compiler version in Cargo? emk/heroku-buildpack-rust#11 That explains why we've had such difficulties with the submitted projects compiling on our laptops.

[dfellis]

Tracing through several years of github issues leads me to: rust-lang/rustup#1172

We apparently need to add a rust-toolchain file that contains, as far as I can tell, only the version of rust that we want to use?

[dfellis]

I assumed this was the field you need to change?

[depombo]

Initially, I thought so too

[dfellis]

It looks like this upgrade also broke the HTTP client and server code :/

[depombo]

This doesn't actually fix the real issue of #354 which is that we're still creating a tokio task per array element, which explodes the AVM with an OOM on large arrays unnecessarily (also because we duplicate the working memory for each task right now, so we get array size to the array size amount of memory consumption)

I see. To some extent I did not understand the issue correctly. It should not spin one thread per array element, but also it should not spin on task per array element. Since tokio uses a work stealing mechanism this addresses the former, but not the latter.

[depombo]

It looks like this upgrade also broke the HTTP client and server code :/

Yeah, I think this is a bug between enable_all not actually turning on enable_io

[dfellis]

It looks like this upgrade also broke the HTTP client and server code :/

Yeah, I think this is a bug between enable_all not actually turning on enable_io

...that's crazy! What else would you mean by "enable_all"?

[depombo]

It looks like this upgrade also broke the HTTP client and server code :/

Yeah, I think this is a bug between enable_all not actually turning on enable_io

...that's crazy! What else would you mean by "enable_all"?

So it turns out that it was silently failing because the latest version of hyper and reqwst are still using tokio 0.2. hyper's master branch already made the change, but until their next release we can't upgrade. I'm going to abandon and make a new PR where I fix the actual issue of spinning up a tokio task per array element and set max threads without upgrading tokio