Crash on panic!

[Meziu]

From testing the network example, i noticed that panic!, unwraps and anything that calls a panic in the Rust runtime crashes the program as a translation - error data abort in Luma3DS. This is unintended behaviour, and my previous work on the std should have fixed it. It came back only now, possibly because of changes in the std code. @AzureMarker, if you want to help, this is pretty much the priority right now.

[Meziu]

@AzureMarker just for info, are you able to run GDB with the Luma3DS stub? I'm trying in every way to run the hello-world-3ds example with gdb but I can't get it to work anymore... If you do, please tell me your exact procedure 😄

[AzureMarker]

I did notice the panic data abort thing in debug builds, but release builds don't seem to have the problem. The release build behavior is that the error message doesn't stay up and the program immediately exits. Is the message supposed to stay visible, perhaps until a button is pressed?

GDB:

1. Run homebrew launcher
2. Open Luma3DS menu with L + Down + Select
3. Debugger options -> Enable debugger + Force-debug next application at launch
4. Close Luma3DS menu
5. Run application either through SD card or 3dslink
6. Open gdb with gdb -tui <elf file path>. Example: gdb -tui target/armv6k-nintendo-3ds/debug/n3ds-controller.elf
7. Run command target remote <3DS ip address>:4003 to connect to the 3DS gdb.
8. Continue debugging like normal.

The GDB stub is pretty slow and sometimes/eventually crashes for me. It crashes sooner when I try to use CLion's remote debug feature, so I've just been using command line for more stability.

[AzureMarker]

Right when it calls panic_impl it gets a segfault:

[Meziu]

I did notice the panic data abort thing in debug builds, but release builds don't seem to have the problem. The release build behavior is that the error message doesn't stay up and the program immediately exits. Is the message supposed to stay visible, perhaps until a button is pressed?

Sadly we can't put ctru functions in the std, so buttons aren't a possibility. Maybe just a std::thread::sleep(Duration::from_secs(5)) to read the message or something.

[Meziu]

6. Open gdb with `gdb -tui <elf file path>`. Example: `gdb -tui target/armv6k-nintendo-3ds/debug/n3ds-controller.elf`

Wait, is that a standard gdb or just an alias? arm-none-eabi-gdb doesn't support tui afaik.

Edit: wow, gdb actually supports the arch. Didn't know.

[AzureMarker]

Yeah, just normal gdb works since it's using a remote server for the heavy lifting.

Also, set substitute-path is handy for debugging libctru, since you probably didn't compile it locally:
https://sourceware.org/gdb/onlinedocs/gdb/Source-Path.html#set-substitute_002dpath

[Meziu]

#0  0x0012012c in newRow ()
    at /home/fincs/pacman-packages/libctru/src/libctru-2.0.1/libctru/source/console.c:627
#1  0x0011f488 in consolePrintChar (c=10)
    at /home/fincs/pacman-packages/libctru/src/libctru-2.0.1/libctru/source/console.c:743
#2  consolePrintChar (c=10)
    at /home/fincs/pacman-packages/libctru/src/libctru-2.0.1/libctru/source/console.c:700
#3  0x0011f58c in con_write (r=<optimized out>, fd=<optimized out>, len=<optimized out>, ptr=<optimized out>)
    at /home/fincs/pacman-packages/libctru/src/libctru-2.0.1/libctru/source/console.c:476
#4  con_write (r=<optimized out>, fd=<optimized out>, ptr=<optimized out>, len=<optimized out>)
    at /home/fincs/pacman-packages/libctru/src/libctru-2.0.1/libctru/source/console.c:206
#5  0x0013a1e0 in <std::io::Write::write_fmt::Adapter<T> as core::fmt::Write>::write_str ()
#6  0x00135a54 in std::backtrace_rs::print::BacktraceFrameFmt::print_raw_with_column ()
#7  0x0013c518 in std::sys_common::backtrace::_print_fmt::_$u7b$$u7b$closure$u7d$$u7d$::h11c5d2ac13daa081 ()
#8  0x00137e68 in std::backtrace_rs::backtrace::libunwind::trace::trace_fn ()
#9  0x001271f0 in __gnu_Unwind_Backtrace ()
#10 0x001006d4 in ___Unwind_Backtrace ()
#11 0x00131880 in <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt ()
#12 0x0014834c in core::fmt::write ()
#13 0x00158520 in std::io::Write::write_fmt ()
#14 0x00162a54 in std::panicking::default_hook::{{closure}} ()
#15 0x0016267c in std::panicking::default_hook ()
#16 0x00163020 in std::panicking::rust_panic_with_hook ()

By looking at this backtrace, the obvious guess is that the Console gets dropped after the panic, yet it is used correctly before that line...

[AzureMarker]

If I force the Console to not get dropped via std::mem::forget, it panics with an unknown instruction error:

[Meziu]

619                     u16 *dst = &currentConsole->frameBuffer[(currentConsole->windowX * 8 * 240) + (239 - (currentConsole->windowY * 8))];
(gdb) p currentConsole->cursorY
$7 = 29
(gdb) p *currentConsole
$8 = {font = {gfx = 0x0, asciiOffset = 0, numChars = 0}, frameBuffer = 0x30000000, cursorX = 28, 
  cursorY = 29, prevCursorX = 17, prevCursorY = 1, consoleWidth = 0, consoleHeight = 1908700, windowX = 41, 
  windowY = 1907224, windowWidth = 1907224, windowHeight = 30, tabSize = 3, fg = 7, bg = 0, flags = 0, 
  PrintChar = 0x0, consoleInitialised = 40}

Alright, looks like the windowY variable goes flying through the roof and the dst calculation goes bad. Those pointers aren't valid and thus the sigsegv.

[AzureMarker]

The current implementation of intrinsics::abort is to invoke an invalid instruction, on most platforms.

The current console pointer points to the box in Console, which might have gotten deallocated by that point.

[AzureMarker]

The backtrace from the abort:

[Meziu]

It's weird though, this all seems like it should happen in a release version too. If the problem is really the order of drop and print, then it should print either no message, or all of it, and it should be the same in either debug or release.
This I cannot understand

[AzureMarker]

Is there a difference between if it unwinds or aborts in debug vs release? I'm trying to step through the debug build's panic to see if it's triggering a second panic. Otherwise, maybe we need to customize the abort code for the 3DS to call some libctru function instead of executing an undefined instruction?

[Meziu]

I don't think there is a difference. It may even have to do with the threading speed and stuff. Directly using ctru functions is a no go. Maybe the console object doesn't get corrupted in release and thus survives enough to not sigsegv?

Thread 1 hit Hardware watchpoint 2: currentConsole->windowY

Old value = 0
New value = 1907192
0x00128ec0 in _malloc_r ()
(gdb) bt
#0  0x00128ec0 in _malloc_r ()
#1  0x0017d8ec in alloc::alloc::alloc (layout=...)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/alloc/src/alloc.rs:87
#2  0x0017da18 in alloc::alloc::Global::alloc_impl (self=0x1c11bc, layout=..., zeroed=false)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/alloc/src/alloc.rs:169
#3  0x0017db28 in alloc::alloc::{impl#1}::allocate (self=0x1c11bc, layout=...)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/alloc/src/alloc.rs:229
#4  0x0017dbec in alloc::alloc::exchange_malloc (size=12, align=4)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/alloc/src/alloc.rs:318
#5  0x0017e710 in std::thread::local::os::Key<core::cell::Cell<usize>>::try_initialize<core::cell::Cell<usize>> (self=0x1d1fdc <std::panicking::panic_count::LOCAL_PANIC_COUNT::__getit::__KEY>, 
    init=0x1780ec <std::panicking::panic_count::LOCAL_PANIC_COUNT::__init>)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/thread/local.rs:739
#6  0x0017e5b0 in std::thread::local::os::Key<core::cell::Cell<usize>>::get<core::cell::Cell<usize>> (
    self=0x1d1fdc <std::panicking::panic_count::LOCAL_PANIC_COUNT::__getit::__KEY>, 
    init=0x1780ec <std::panicking::panic_count::LOCAL_PANIC_COUNT::__init>)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/thread/local.rs:721
#7  0x00178124 in std::panicking::panic_count::LOCAL_PANIC_COUNT::__getit ()
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/thread/local.rs:324
#8  0x0017895c in std::thread::local::LocalKey<core::cell::Cell<usize>>::try_with<core::cell::Cell<usize>, std::panicking::panic_count::decrease::{closure#0}, usize> (self=0x1c0414, f=...)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/thread/local.rs:411
#9  0x00178498 in std::thread::local::LocalKey<core::cell::Cell<usize>>::with<core::cell::Cell<usize>, std::panicking::panic_count::decrease::{closure#0}, usize> (self=0x1c0414, f=...)
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/thread/local.rs:388
#10 0x00177cc8 in std::panicking::panic_count::decrease ()
    at /home/andrea/Desktop/ProgrammingStuff/Rust/rust-3ds/rust-horizon/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/src/rust/library/std/src/panicking.rs:276
#11 0x00189de0 in std::panicking::try::cleanup (payload=0x800ccc8)

The trigger for the corruption is the generation of a thread key requested by the panic. Maybe this is the issue...

[Meziu]

    // We consider unwinding to be rare, so mark this function as cold. However,
    // do not mark it no-inline -- that decision is best to leave to the
    // optimizer (in most cases this function is not inlined even as a normal,
    // non-cold function, though, as of the writing of this comment).
    #[cold]
    unsafe fn cleanup(payload: *mut u8) -> Box<dyn Any + Send + 'static> {
        // SAFETY: The whole unsafe block hinges on a correct implementation of
        // the panic handler `__rust_panic_cleanup`. As such we can only
        // assume it returns the correct thing for `Box::from_raw` to work
        // without undefined behavior.
        let obj = unsafe { Box::from_raw(__rust_panic_cleanup(payload)) };
        panic_count::decrease();
        obj
    }

Alright, i read "decision left to the optimizer"...