Signals

[rennergade]

We've talked about eventually adding signal handling functionality in the past, and it seems like its necessary for pgbench to work, and possibly the rest of postgres, which seems heavily dependent on signals for communication.

I think its time to take a whack at this. Max is going to attempt to implement the syscalls for sigaction(), sigprocmask, and kill(), while Jonathan is going to attempt to have nacl call the registered function.

Will set up a meeting next week for us to go over this and put together a small design document.

[rennergade]

Max has implemented kill() and sigaction() and has passed that off to @jesings who's going to work on the signal handler. Max now is going to work on implementing sigprocmask().

[jesings]

Currently I am able to receive a signal while the receiving thread is in untrusted code and then dispatch to a signal handler in untrusted code. In order to be able to return from that signal handler we need to do some return address hijacking on the stack to some trampoline stub that will restore the registers from their previous state.
Additionally we need to figure out what we do when we receive a signal when the thread is in trusted code--we can't immediately process it because what if we're i.e. in the middle of modifying the vmmap. We should discuss how to implement this.

[jesings]

I am looking to implement this by adding an untrusted function on the springboard whose return address is placed onto the stack right below the stack frame for the handler.
The NaCl signal handler currently does the following relevant things:
It starts a new stack frame below the red zone of the stack frame that received the signal
By creating a new stack pointer address for untrusted
And then subtracts more space from that stack pointer address
In order to have enough room to store this struct which contains, among other things, all of the registers
It then populates the registers and other stuff into that stack
The stack frame for the signal handling function starts below this one
So when we return from the signal handler, these contents will still be on the stack
And so we should return from the signal handler to some function that can actually populate these into our registers

[rennergade]

Great progress, thank you! I think this makes sense as a way to handle returns.

@moyix doe this make sense to you? any concerns or advice?

[jesings]

This register restoration technique actually won't work safely either in cagespace or in trusted without clobbering a register, which would be bad.

The issue is that when we return from the cagespace signal handler, we would execute some code that would need to restore registers, and after that code was finished we would need to return execution back to the point in cagespace that we were right before the signal was received. This code could either be in cagespace or trusted, and this problem would hold either way.

We need to restore every single register before finally restoring the instruction pointer, as once we restore the instruction pointer we start executing the pre-signal cagespace code again.
Once we have restored every register except the instruction pointer, we are left in a situation where we cannot clobber any of these registers as we need them to be exactly as they were before cagespace received the signal.
x86 does not allow you to change the instruction pointer without either using a fixed address jmp (which obviously doesn't work for our case), a different register for an indirect jmp, or using a ret to pop a return address from the stack and put it into %rip.
Using a register for an indirect jump clearly doesn't work as that would require clobbering it.
Returns from the stack are fraught because at this point the stack pointer points to cagespace memory which in order to safely use as a return address needs to have its top 32 bits masked off by the value in %r15, which is handled in nacl (in the naclret pseudo-instruction) by using the %r11 register to hold the address, masking the top 32 bits of that register off, and then using it as the target for an indrect jump--this clobbers %r11 so is no good, and in fact any such masking scheme requires the clobbering of a register.

Therefore, accomplishing this without for example adding a new reserved register in nacl-gcc seems like it may not be possible in a satisfactory way.

[moyix]

Couldn't you do something like:

1. restore all registers except RIP
2. push r11
3. read saved RIP into r11
4. mask r11
5. write masked r11 back to stack
6. pop r11
7. ret

Or is the concern that since the stack is in untrusted memory someone could modify the saved RIP in between masking it and the ret?

[moyix]

I think the other part I don't understand is – if the normal naclret approach clobbers R11, why isn't it a problem for them?

[jesings]

@moyix Yes, the modification of the stack memory in between saving RIP and masking it is indeed the concern.
I have that implemented already but I think the TOCTTOU attack really breaks NaCl's security model.
The normal naclret is fine because r11 is a caller saved register so there is no problem if it gets clobbered in the return statement.
However for signal handlers, we need to restore both caller and callee saved registers so that won't work.

[rennergade]

@moyix we're going to investigate if we can reserve another register and its effects. Do you think this is a good idea? We're struggling to come up with any other options.