Skip to content

Rust fix compilation for no_std targets#7338

Closed
jul-sh wants to merge 12 commits into
google:masterfrom
jul-sh:master
Closed

Rust fix compilation for no_std targets#7338
jul-sh wants to merge 12 commits into
google:masterfrom
jul-sh:master

Conversation

@jul-sh

@jul-sh jul-sh commented Jun 6, 2022

Copy link
Copy Markdown

core2 is only used if the no_std feature is active. As of technocreatives/core2@4072d2a core2 includes std features by default. To achieve no_std compataibility this must be disabled. Fixes broken no_std builds on rustc 1.62.0-nightly.

core2 is only used if the no_std feature is active. As of technocreatives/core2@4072d2a core2 includes std features by default. To achieve no_std compataibility this must be disabled. Fixes broken no_std builds on rustc 1.62.0-nightly.
@dbaileychess

Copy link
Copy Markdown
Collaborator

@CasperN

@dbaileychess

Copy link
Copy Markdown
Collaborator

@CasperN Could you take a look?

@CasperN

CasperN commented Aug 12, 2022

Copy link
Copy Markdown
Collaborator

Hello, I'm having trouble reproducing a build error at head. I ran

 cargo +nightly-2022-06-06 test --features no_std

in flatbuffers/tests/rust_usage_test and it came out fine

@dbaileychess

Copy link
Copy Markdown
Collaborator

@jul-sh Any update to @CasperN comments?

@jul-sh

jul-sh commented Aug 13, 2022

Copy link
Copy Markdown
Author

@jul-sh Any update to @CasperN comments?

cargo test doesn't ordinarily test no_std compatibility in dependency of libraries. This is likely why it looks fine there. Ref https://blog.dbrgn.ch/2019/12/24/testing-for-no-std-compatibility/

At a conf rn, can look into this in more detail later.

@CasperN

CasperN commented Aug 13, 2022

Copy link
Copy Markdown
Collaborator

Thanks for the explanation. Whenever you're free, can you add the no_std smoke test to tests/rust_usage_test, RustTest.sh, and RustTest.bat to prevent regressions?

@dbaileychess

Copy link
Copy Markdown
Collaborator

@jul-sh Any update to this?

@jul-sh

jul-sh commented Aug 30, 2022

Copy link
Copy Markdown
Author

@jul-sh Any update to this?

Haven't looked into yet, but should be able to in mid September

@jul-sh

jul-sh commented Sep 8, 2022

Copy link
Copy Markdown
Author

This is blocked by bbqsrc/thiserror-core2#3 getting merged upstream, as flatbuffer depends on that crate. It needs (very minor) changes to ensure no_std compatibility. Unfortunately no news from the maintainer so far.

We could proceed with using a fork until bbqsrc/thiserror-core2#3 is merged. (we did so in https://github.com/project-oak/oak).

@jul-sh jul-sh changed the title Rust no_std fix: Disable core2 default features Rust fix compilation for no_std targets Sep 8, 2022
@jul-sh

jul-sh commented Sep 8, 2022

Copy link
Copy Markdown
Author

@CasperN Could you approve running the workflows? Attempted the CI integration. Would be surprised if it works right away (Windows runs especially might be tricky), but would be good to be able to test it.

@CasperN

CasperN commented Sep 8, 2022

Copy link
Copy Markdown
Collaborator

@jul-sh thanks for the fix and test. I approved the workflows, looks like CI found something.

We don't publish often but we cannot publish our library on crates.io unless all our dependencies are in crates.io too. So I'm happy to submit this (once CI passes) so long the thiserror upstream change gets merged soon-ish

@bbqsrc

bbqsrc commented Sep 10, 2022

Copy link
Copy Markdown

This crate doesn't follow the appropriate idiom for how no_std is used in production in Rust projects.

The pattern is to have a std feature, on by default. Here's some highly popular crates demonstrating this pattern:

In your code, you then just write #![cfg_attr(not(feature = "std"), no_std)] to enable no_std, and #[cfg(not(feature = "std"))] for all the no_std implementations.

This has the benefit that you can easily disable std for all crates that use this pattern with default-features = false, and then in your std feature, enable all the std features in dependencies, eg:

default = ["std"]
std = ["thiserror-core2/std"]

@dbaileychess

Copy link
Copy Markdown
Collaborator

@jul-sh update to this given @bbqsrc comment?

@danlapid

Copy link
Copy Markdown
Contributor

Seems to me like core2 and thiserror can be removed entirely.
They are only used in verifier.rs to derive Error for one of the error types even though the other error type is implementing format regularly
Less dependencies is always good imho and no_std will be as easy as adding the following code:

impl Error for InvalidFlatbuffer {
    fn source(&self) -> Option<&(dyn Error + 'static)> {
        match self {
            InvalidFlatbuffer::MissingRequiredField { .. } => None,
            InvalidFlatbuffer::InconsistentUnion { .. } => None,
            InvalidFlatbuffer::Utf8Error { error: source, .. } => Some(source),
            InvalidFlatbuffer::MissingNullTerminator { .. } => None,
            InvalidFlatbuffer::Unaligned { .. } => None,
            InvalidFlatbuffer::RangeOutOfBounds { .. } => None,
            InvalidFlatbuffer::SignedOffsetOutOfBounds { .. } => None,
            InvalidFlatbuffer::TooManyTables { .. } => None,
            InvalidFlatbuffer::ApparentSizeTooLarge { .. } => None,
            InvalidFlatbuffer::DepthLimitReached { .. } => None,
        }
    }
}

impl core::fmt::Display for InvalidFlatbuffer {
    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
        match self {
            InvalidFlatbuffer::MissingRequiredField {
                required,
                error_trace,
            } => {
                writeln!(f, "Missing required field `{}`.\n{}", required, error_trace)?;
            }
            InvalidFlatbuffer::InconsistentUnion {
                field,
                field_type,
                error_trace,
            } => {
                writeln!(f,"Union exactly one of union discriminant (`{}`) and value (`{}`) are present.\n{}", field_type, field, error_trace)?;
            }
            InvalidFlatbuffer::Utf8Error {
                error,
                range,
                error_trace,
            } => {
                writeln!(
                    f,
                    "Utf8 error for string in {:?}: {}\n{}",
                    range, error, error_trace
                )?;
            }
            InvalidFlatbuffer::MissingNullTerminator { range, error_trace } => {
                writeln!(
                    f,
                    "String in range [{}, {}) is missing its null terminator.\n{}",
                    range.start, range.end, error_trace
                )?;
            }
            InvalidFlatbuffer::Unaligned {
                position,
                unaligned_type,
                error_trace,
            } => {
                writeln!(
                    f,
                    "Type `{}` at position {} is unaligned.\n{}",
                    unaligned_type, position, error_trace
                )?;
            }
            InvalidFlatbuffer::RangeOutOfBounds { range, error_trace } => {
                writeln!(
                    f,
                    "Range [{}, {}) is out of bounds.\n{}",
                    range.start, range.end, error_trace
                )?;
            }
            InvalidFlatbuffer::SignedOffsetOutOfBounds {
                soffset,
                position,
                error_trace,
            } => {
                writeln!(
                    f,
                    "Signed offset at position {} has value {} which points out of bounds.\n{}",
                    position, soffset, error_trace
                )?;
            }
            InvalidFlatbuffer::TooManyTables {} => {
                writeln!(f, "Too many tables.")?;
            }
            InvalidFlatbuffer::ApparentSizeTooLarge {} => {
                writeln!(f, "Apparent size too large.")?;
            }
            InvalidFlatbuffer::DepthLimitReached {} => {
                writeln!(f, "Nested table depth limit reached.")?;
            }
        }
        Ok(())
    }
}

It's basically what Derive(Error) creates

@CasperN

CasperN commented Sep 27, 2022 via email

Copy link
Copy Markdown
Collaborator

@bbqsrc

bbqsrc commented Sep 27, 2022

Copy link
Copy Markdown

I'd note it's a separate issue from how the feature flags are specified.

And given the extremely small amount of code using core2, would highly recommend dropping it.

@tiziano88

Copy link
Copy Markdown
Member

Friendly ping on this issue, it's currently still not possible to use flatbuffers as no_std in practice.

@danlapid

Copy link
Copy Markdown
Contributor

Friendly ping on this issue, it's currently still not possible to use flatbuffers as no_std in practice.

#7553 was merged, you can try again, no_std should work now.

@dbaileychess

Copy link
Copy Markdown
Collaborator

@jul-sh There are merge conflicts now.

@CasperN

CasperN commented Oct 19, 2022

Copy link
Copy Markdown
Collaborator

This has been superseded with #7553

@CasperN CasperN closed this Oct 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants