STD Cleanup and Test

[Meziu]

We have worked with most of the std while updating ctru-rs's modules. However, we now have to merge our changes into the rustlang tree, and this will require cleanup and a lot of testing. We must make sure to never have to make PR's to std ever again, since the effort for the contributor and the reviewer is great.

Since very basic/already tested multiple times, I'd skip testing of fundamental datatypes and of these modules:

• alloc
• any
• array
• ascii
• borrow
• boxed
• cell
• char
• clone
• cmp
• collections
• convert
• default
• error
• f32
• f64
• ffi
• fmt
• hint
• io
• iter
• marker
• mem
• num
• ops
• option
• os (it's empty for our target)
• panic
• prelude
• ptr
• rc
• result
• slice
• str
• string
• vec

TESTED AND NOW WORKING MODULES:

• hash
• future
• pin
• time
• env
• process
• sync
• thread

Modules we should test, though we have already worked with:

• fs
• path

Other than testing, cleaning up lints and other problems is needed.

@AzureMarker @ian-h-chamberlain
Please, a combined effort is the best and fastest way.

[AzureMarker]

I'm interested to see what kind of futures executor I can get running on the 3DS. Then again, there isn't much in the futures module that could be broken.

[AzureMarker]

Maybe there's a way to run the runtime std tests on-device?

[ian-h-chamberlain]

At one point I actually was able to build libc-test for the 3DS but ran into some issues when running it. I didn't spend much time to debug the issue, but based on that I believe it should be possible to build and run tests on the device (not as part of CI or anything, but perhaps with Citra there's something possible...)

I can start taking a look at cleaning up lints and warnings, I noticed that std does not seem to build using the default RUSTFLAGS in x.py, so I think there's some cleanup to do there, which should help make submitting upstream easier in the future.

[AzureMarker]

I noticed that std does not seem to build using the default RUSTFLAGS in x.py

x.py build -i library/std --stage 1 doesn't work for you?

[Meziu]

Maybe there's a way to run the runtime std tests on-device?

rust3ds/ctru-rs#25 Maybe we could do something with this, or Citra. Though I suggest on-hardware tests (as I also can't run Citra).

[Meziu]

I'm interested to see what kind of futures executor I can get running on the 3DS. Then again, there isn't much in the futures module that could be broken.

This was me before noticing HashMap are broken. Yes. A collection type (it's actually an issue with randoms).

[ian-h-chamberlain]

x.py build -i library/std --stage 1 doesn't work for you?

This was a while ago, so it might have been a combination of libc changes and my own setup, but IIRC there were some warnings which compiled (but warned) with -Zbuild-std but failed with x.py build library/std --target armv6k-nintendo-3ds because of -D warnings or something like that. I'll double check, right now I've been trying various x.py check, x.py clippy just to see what comes up. A bunch of test stuff doesn't pass the check due to the getrandom issue @Meziu mentioned, which looks like something we might be able to easily fix by adding some target_os = "horizon" in the getrandom crate.

[Meziu]

IIRC there were some warnings which compiled (but warned) with -Zbuild-std but failed with x.py build library/std --target armv6k-nintendo-3ds

I personally never built the std separately, and always built the stage 1 or 2 compiler.

A bunch of test stuff doesn't pass the check due to the getrandom issue @Meziu mentioned, which looks like something we might be able to easily fix by adding some target_os = "horizon" in the getrandom crate.

It's just a small change in sys/unix/rand.rs to use rand() instead of non-available syscalls.

[AzureMarker]

I haven't looked into it, but there's some randomness code in libctru used by ctru-rs which we might be able to use in the linker fix crate to supply a libc call?

[Meziu]

I haven't looked into it, but there's some randomness code in libctru used by ctru-rs which we might be able to use in the linker fix crate to supply a libc call?

Is there? I haven't found anything other than rand

[ian-h-chamberlain]

This API is the only thing I've found, but it includes this note at the top of the module:

This is normally not accessible to userland apps

So I'm not sure if we could use that.

BTW, I've found that searching with Google and site:libctru.devkitpro.org is a bit more effective than the builtin search on the generated documention

Edit: there are a few other APIs, sslcGenerateRandomData which is probably more for cryptographically secure randomness and I guess uses PS_GenerateRandomBytes internally. I guess the hosted docs haven't been updated to include these, since I can only find them in the headers on my system and not online, except for the wiki.

[Meziu]

We could implement a call to sslcGenerateRandomData into getrandom from Newlib, but that would require another libc declaration. We can make all the changes and once testing the std is over, we'll open a new PR to libc.

[Meziu]

Turns out time::Instant doesn't work because the impl uses libc::CLOCK_MONOTONIC, which is either wrong or nonexistent for our platform. I searched for a WHILE but didn't find anything about it. Resorted to Meziu/libc@8744910 in case you also can't find anything.