Linux and mac compatibility by aimir · Pull Request #1029 · knurling-rs/defmt

aimir · 2026-01-21T08:42:43Z

Currently without correct locations in symbol tables on mac, but everything else should work as expected, with no impact on embedded systems

…one is not enough to make sure symbols are actually exported on mac

…either avoid it by prefixing with or strip this prefix explicitly

… on linux and mac, to offset addresses accordingly, since the linker script won't work on them (there's probably a more normal way to do this only once)

…s-calculations Return to no-std, use linker symbols to calculate the base address

remove two deps from defmt/Cargo.toml

On macOS, each defmt log statement created a unique section in the .defmt segment. With 200+ logs, this exceeded Mach-O's 255 section limit. Fix: Group logs by severity level instead of unique hash per log.

fix(macos): avoid 255 section-per-segment limit

…y type, instead of the decoding os

Linux and mac compatibility issue fixes

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

…ather than the decoding host

Linux and mac compatibility fixes, part 2

… LLVM behavior so I would prefer it for now

This reverts commit 57acf1b.

This reverts commit ba1f9ae.

Linux and mac compatibility fixes3

jonathanpallant · 2026-01-21T13:44:12Z

Nice - I'm excited to look through this in detail soon.

…the base address

…bol by its export name, make it macos-only

jonathanpallant · 2026-02-03T16:20:29Z

I had a trial run with this, but defmt-print cannot open Mach-O files, so this isn't usable on macOS.

How have you been testing it?

tomasbarton-stemcell · 2026-02-17T16:13:42Z

I tested this on Linux and it works well 🥳

Although I still needed to have a custom linker file that collects all interned strings into a single section:

Error: defmt version found, but no `.defmt` section - check your linker configuration

Ideally the elf parser could deal with it without the linker file, because the symbols are in the elf file when I print them out.

Alternatively modify the provided defmt.x so that it does INSERT AFTER ... (somehow, maybe behind a feature flag?) otherwise it cannot be used as by default it replaces the default linker file instead of extending it.

jonathanpallant · 2026-02-26T13:16:25Z

I think the best solution is to only put items into the .defmt section when running bare-metal (when cfg(target_os = "none")). Otherwise let linker just put them wherever. And then fix the decoder to be more flexible about where it looks for these symbols.

aimir · 2026-02-26T14:46:49Z

I had a trial run with this, but defmt-print cannot open Mach-O files, so this isn't usable on macOS.

How have you been testing it?

I just used the decoder directly, but let me go ahead and change this - added a commit to support Mach-O files in defmt-print 😁

jonathanpallant · 2026-03-03T14:47:38Z

@knoellle is going to take a look at this one, right after his Robocup competition.

knoellle · 2026-03-23T18:54:32Z

Thank you for your work so far.
I was able to craft a native Linux program and a linker script that worked but don't have a Mac to test with.

What is your plan with the linker scripts? As @tomasbarton-stemcell mentioned already, I think we should either include linker scripts for Linux/Mac or adjust the elf parser to work without linker scripts.
The defmt/build.rs already generates/modifies linker scripts depending enabled features. Using a different linker script as a base depending on the platform seems reasonable to me, though not needing one would be preferable I think.

mhatzl · 2026-03-24T15:22:30Z

Thanks for the PR!
Tested it on macOS and it worked without the need for a linker script.
The encoded defmt logs are output on stdout and piping it to defmt-print given the binary correctly decoded and displayed the logs.

As mentioned in the PR description, the location information currently does not work for macOS.
@aimir do you have an idea why or maybe even time to fix that?

aimir · 2026-04-05T12:53:10Z

Thank you for your work so far. I was able to craft a native Linux program and a linker script that worked but don't have a Mac to test with.

What is your plan with the linker scripts? As @tomasbarton-stemcell mentioned already, I think we should either include linker scripts for Linux/Mac or adjust the elf parser to work without linker scripts. The defmt/build.rs already generates/modifies linker scripts depending enabled features. Using a different linker script as a base depending on the platform seems reasonable to me, though not needing one would be preferable I think.

Mac doesn't have linker scripts, and like @mhatzl said it should work without the need for one.

It does mean that things like the default panic handler that's auto-configured by defmt/build.rs. I didn't find an elegant workaround yet, and I really want to merge this - is this a must for you?

aimir · 2026-04-05T12:56:37Z

Thanks for the PR! Tested it on macOS and it worked without the need for a linker script. The encoded defmt logs are output on stdout and piping it to defmt-print given the binary correctly decoded and displayed the logs.

As mentioned in the PR description, the location information currently does not work for macOS. @aimir do you have an idea why or maybe even time to fix that?

The issue is that by default mac location information is scattered across files, and the table parsing doesn't handle this well.

Running without split debug info (RUSTFLAGS="-C split-debuginfo=off) seems to resolve the issue - @mhatzl can you check if that works for you?

mhatzl · 2026-04-15T11:06:27Z

Hi @aimir,
thanks for your help!
Unfortunately I am not able to get correct location infos on macOS with split-debuginfo=off.

looking at the docs for split-debuginfo and the cargo profile docs, the default for macOS is packed except for Cargo profiles that have debug infos enabled it is unpacked.

Since the dev profile has debug infos enabled, split-debuginfo is set to unpacked by default when run via Cargo. Setting it to packed creates this dSym folder containing DWARF data, but the location info is still missing if this DWARF file is passed to defmt-print.

knoellle · 2026-04-20T12:47:21Z

It does mean that things like the default panic handler that's auto-configured by defmt/build.rs. I didn't find an elegant workaround yet, and I really want to merge this - is this a must for you?

I spent some time attempted making the default panic stuff work without linker scripts but didn't find a way without breaking user code. See #1050.

Again, I don't have a mac to test with, but if I understand correctly then attempting to use defmt::panic on macOS shouldn't successfully link or run (still not entirely sure how that works over there) unless the user defines a custom defmt panic handler.
defmt::panic calls _defmt_panic, which is weakly linked to __defmt_default_panic but can be overridden using the defmt::panic_handler macro.
Without the linker script and no custom implementation, _defmt_panic would be an undefined symbol.

What are your thoughts on requiring a custom panic handler for building for macOS specifically?

knoellle · 2026-04-20T13:01:12Z

I'm curious how you tested the linux compatibility.
I have been able to get it to work, but I haven't found a good way of passing the linker script to the linker. My workaround so far was setting RUSTFLAGS="-C link-arg=-T<path to my repo>/.cargo/defmt.x"

For embedded platforms, we use flip-link which does a bunch of magic to find linker scripts inside library search paths which other linkers don't do.
So far I have always had to pass an absolute path to my linker script to compile my linux binaries.

defmt build.rs generates the default linker script but doesn't know if a linker script is going to be used at all.
end user .cargo/config.toml shouldn't contain an absolute path to a file in the repo or worse to the one generated by defmt build.rs
Requiring a build.rs in the end user project to find either the defmt-generated linker script (somewhere in target/, maybe requires the defmt build.rs to somehow pass it's own OUT_DIR to the end user build.rs) or one provided by the user project and adding the rustflag that way seems very heavy handed.

How do you build your linux binaries?

Fix macos dsym mapping

aimir · 2026-05-06T09:41:22Z

@lakechurch suggested a fix for the symbol table collection that seems to work well - thanks!!

@mhatzl can you take a look now?

aimir · 2026-05-06T09:43:27Z

@knoellle what I ended up doing is using build.rs to find the relative path to the linker script, which is definitely not ideal

CI (knurling-rs#1029) failed across the whole host matrix with three small issues introduced by the macOS dSYM mapping work: - print/src/main.rs: `let mut dwarf_bytes = bytes.clone();` had `mut` only for the `#[cfg(target_os = "macos")]` reassignment, so non-macOS hosts hit `error: variable does not need to be mutable` under `-D warnings`. Fix by binding `dwarf_bytes` separately on the two cfgs: a non-mut `bytes.clone()` on non-macOS, and a block-expression that does the mutable reassignment and returns the result on macOS. - decoder/src/elf2table/mod.rs: `clippy::collapsible_if` flagged the nested `if name_str == "DEFMT_LOG_STATEMENT" { if table.raw_symbols()…` in the standard-ELF branch. Collapse with `&&`. - decoder/src/frame.rs: `clippy::cloned_ref_to_slice_refs` (Rust 1.93, not yet in the CI matrix but coming): `&[arg.clone()]` -> `std::slice::from_ref(arg)`. Also picks up `cargo fmt --all` whitespace fixes in the same areas. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Apple's linker strips memory addresses from `#[no_mangle]` statics that its visibility analysis treats as unreferenced, so the standard ELF DWARF→symtab correlation (`DW_AT_location` → address → symbol-table entry) collapses every defmt log to address 0 on macOS host builds. Result today on macOS: `Locations` is empty/wrong and ERROR-level log lines render without `file:line` (gen2's `runtime/observability/logging/output.rs:361` already plumbs the `Locations` map; it just has nothing to look up). This commit ports the design from upstream PR knurling-rs#1029 (lakechurch's `fix-macos-dsym-mapping`) onto our `gen-bc/defmt:main` line, adapted for the existing unified `.defmt.{severity}` section layout we already have. What changes: - macros/src/construct.rs: alongside every defmt log static, on macOS emit a sibling marker `DEFMT_LOC_MARKER_{hash}` whose source ident encodes a 64-bit SipHash of the log's `sym_name`. `#[used]` keeps the marker referenced; `#[link_section = "__DATA,__defmt_loc"]` parks it in its own data section. No emission on non-macOS. - decoder/src/elf2table/mod.rs: on Mach-O, build a `hash_to_index` side table keyed by hash of each entry's `raw_symbol` (with leading `_` stripped, since Mach-O exports carry it) and valued at the entry's table key. Walk DWARF, find variable DIEs whose `DW_AT_name` starts with `DEFMT_LOC_MARKER_`, parse the hex hash, and resolve to the table key via `hash_to_index`. The non-Mach-O path is untouched. Hardening: bail on hash collisions both when building `hash_to_index` (two symbols hashing to the same value) and when inserting into the result map (the actual location-collision bug we're guarding). - print/src/main.rs: on macOS with `split-debuginfo = "packed"`, DWARF lives in a sibling `<elf>.dSYM` bundle rather than embedded in the Mach-O. Open the first file under `<elf>.dSYM/Contents/Resources/DWARF/` and feed those bytes to `get_locations`. On other platforms the main ELF has DWARF embedded so we keep using `bytes` directly. Build: `cargo build` clean. Two pre-existing warnings on `gen-bc/main` (`unused_unsafe` in defmt/src/export/mod.rs and `unused defmt_section` in elf2table/mod.rs) are not introduced by this commit. Tests: `cargo test -p defmt-decoder --lib` — all 56 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

aimir and others added 29 commits October 20, 2025 17:23

Mac compatibility, part 1: macho support in object

5da272f

Mac compatibility, part 2: add explicit #[used] tags, as the trick al…

5733047

…one is not enough to make sure symbols are actually exported on mac

Mac compatibility, part 3: mac symbols are exported with a leading - …

4aed089

…either avoid it by prefixing with or strip this prefix explicitly

Mac compatibility, part 4: get sections from the .defmt segment

c0415da

Mac compatibility, part 5: calculate the base address for the section…

37e22ca

… on linux and mac, to offset addresses accordingly, since the linker script won't work on them (there's probably a more normal way to do this only once)

Return to no-std, use linker symbols to calculate the base address

44692fc

Merge pull request #1 from gen-bc/feature/remove-std-from-base-addres…

60c0faf

…s-calculations Return to no-std, use linker symbols to calculate the base address

remove two deps from defmt/Cargo.toml

fd4e0f3

Merge pull request #2 from gen-bc/linux-and-mac-compatibility-cleanup

e74bdc4

remove two deps from defmt/Cargo.toml

fix(macos): avoid 255 section-per-segment limit

87b4745

On macOS, each defmt log statement created a unique section in the .defmt segment. With 200+ logs, this exceeded Mach-O's 255 section limit. Fix: Group logs by severity level instead of unique hash per log.

Update macros/src/construct.rs

0138020

Merge pull request #3 from gen-bc/fix/macos-255-section-limit

06da809

fix(macos): avoid 255 section-per-segment limit

add a fallback for base address calculation logic

dca26e6

determine if we need to strip the leading _ based on the actual binar…

93eebd5

…y type, instead of the decoding os

leftover from some debugging session

62aa71c

make this comment more straightforward

fc26e8b

Merge pull request #6 from gen-bc/linux-and-mac-compatibility-fixes

dcb8988

Linux and mac compatibility issue fixes

Update decoder/src/elf2table/symbol.rs

0383da7

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

section finding logic should be toggled based on the actual binary, r…

d24c7c3

…ather than the decoding host

Merge pull request #7 from gen-bc/linux-and-mac-compatibility-fixes2

267737b

Linux and mac compatibility fixes, part 2

move this comment to the right place

492fea1

avoid getting the section twice

3b4131b

can I revert to this now?

57acf1b

this is fix is more lines, but I think it's less dependent on obscure…

2dd95bd

… LLVM behavior so I would prefer it for now

Update the parsing logic accordingly

067b94c

debug prints

ba1f9ae

Revert "can I revert to this now?"

311a12d

This reverts commit 57acf1b.

Revert "debug prints"

8eb25bf

This reverts commit ba1f9ae.

Merge pull request #8 from gen-bc/linux-and-mac-compatibility-fixes3

34b395a

Linux and mac compatibility fixes3

aimir added 4 commits January 27, 2026 10:28

Merge branch 'main' into linux-and-mac-compatibility-branched

81410f3

Fix tests - anything touching absolute addresses should now subtract …

4238ec7

…the base address

The "used" attribute causes Windows linker to try and resolve the sym…

9601766

…bol by its export name, make it macos-only

Merge branch 'main' into linux-and-mac-compatibility-branched

85ac935

jonathanpallant reviewed Feb 3, 2026

View reviewed changes

Comment thread macros/src/construct.rs

jonathanpallant reviewed Feb 3, 2026

View reviewed changes

Comment thread defmt/src/export/mod.rs

move print to use object crate, so it can parse general object files

d378fde

Merge branch 'main' into linux-and-mac-compatibility-branched

77d6e3c

Merge branch 'main' into linux-and-mac-compatibility-branched

f1dd0fc

resolve macOS dSYM location mapping via hash lookup

e9b82a6

knoellle mentioned this pull request Apr 20, 2026

Refactor avoid-default-panic to not rely on the linker script #1050

Open

Merge pull request #11 from lakechurch/fix-macos-dsym-mapping

785a6fb

Fix macos dsym mapping

aimir mentioned this pull request May 6, 2026

macOS: hash-based DWARF location mapping for defmt logs gen-bc/defmt#12

Open

5 tasks

Merge branch 'main' into linux-and-mac-compatibility-branched

4bbe27f

Uh oh!

Conversation

aimir commented Jan 21, 2026

Uh oh!

jonathanpallant commented Jan 21, 2026

Uh oh!

Uh oh!

Uh oh!

jonathanpallant commented Feb 3, 2026

Uh oh!

tomasbarton-stemcell commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanpallant commented Feb 26, 2026

Uh oh!

aimir commented Feb 26, 2026

Uh oh!

jonathanpallant commented Mar 3, 2026

Uh oh!

knoellle commented Mar 23, 2026

Uh oh!

mhatzl commented Mar 24, 2026

Uh oh!

aimir commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aimir commented Apr 5, 2026

Uh oh!

mhatzl commented Apr 15, 2026

Uh oh!

knoellle commented Apr 20, 2026

Uh oh!

knoellle commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aimir commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aimir commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

tomasbarton-stemcell commented Feb 17, 2026 •

edited

Loading

aimir commented Apr 5, 2026 •

edited

Loading

knoellle commented Apr 20, 2026 •

edited

Loading

aimir commented May 6, 2026 •

edited

Loading