softint overhead: splx .vs. interrupt

Given how many splx calls the kernel does, what's less overhead?

Checking for pending softint in splx or just blasting COP_0_STATUS and letting an interrupt happen?

An interrupts is several hundred instruction. Testing for a softint is a few but it also makes splx a nested function and calling softint_process would be another dozen or instructions. Guessing, I'd say there's a few hundred splx between each softintr so that maybe the interrupt is cheaper. Probably should measure that...

kernel crash dump use ELF core file format.

NetBSD currently dumps its kernel crashes in a NetBSD specific format. This requires gdb to know how to read them (and be linked against libkbm).

Instead, NetBSD should dump kernel core dumps as a ELF core dump (just like any other application). User pages would not be dumped. There would be a note section for each LWP allowing automatic viewing of the LWP's register using the normal gdb thread command (encoding the thread id might get tricky for system LWPs or processes with multiple LWPs).

ELF phdrs would be written so that gdb can automagically map kernel virtual addresses to physical addresses (including direct mapped pages for PMAP_MAP_POOLPAGE support). This may require a lot of phdrs but that's true even for normal ELF core files.

cored

The coredump code in the kernel is already writes the core dump in a single pass over
a file descriptor. This was done so that coredumps could be written directly to a pipe or socket as well as a file. Currently, though, it just writes to a file.

This project would add support for sending a message to a crash daemon which would use a config file to determine what happens next. For instance it could decide to write the crashdump to a pipeline of |gzip -v | ssh ... or maybe directly to a socket used for a http put. This would enabled embedded NetBSD to save coredumps remotely automagically.

AF_GDB / GDB socket

I hate ptrace. But before killing it I need something to replace it. I could use a procfs style interface with ioctls but that's a complex interface.

Instead I want to use the GDB protocol over an AF_GDB socket. It would use socket(AF_GDB, SOCK_STREAM, pid) to connect to the process to be debugged.

To do a run command in gdb, gdb will fork, connect to its child, request a message for "exec", and continue the child. When the child execs the program gdb will grab control of the new process and proceed from there.

As part of this effort, connecting to pid 0 would enable gdb access to the kernel (mostly as a tools to examine memory and threads; no breakpoints would be allowed).

Credentials? We need no stinking credentials!

NetBSD adopted a kauth similar to the one used in MacOSX. But as time has gone on, I've become less enthralled with it. Maybe we need a radical rethought on how authorization is done in the kernel. Part of the problem I have with it is the massive amount of hardcoding of authorization requests which then queries the credentials to see if the request should be allowed. It's amazing inflexible and continuously bloats as new requests are added.

Since it's always been done this way, we keep on doing it this way. Just maybe, though, we shouldn't. Instead of testing credentials, we should be testing capabilities. When our credentials change, we recalculate our capabilities. And we keep that capability set in a opaque object which we can quickly query. The credentials will be ignored by rest of the kernel (even things like the ownership of a newly created file could be gotten from capabilities if we want to extend things that far).

And how we construct our capabilities from our credentials (the security model) could be done using a simple BPF-like language. Imagine that it could say that members of group _sshd could bind to port 22, or that uid foo is allowed to chroot.

And if you attach credentials to a fd, you can do even more crazy stuff. Like specifying that certain operations are allowed on this fd. That fd can then be inherited or passed to a new process which normally would not have those capabilities but the fd retains them so the unprived process can do a small amount of prived actions.

And this is just scratching at the surface... and maybe it's time to turn security on its head.

lwp handoff between CPUs

Normally, if a CPU decides to run a thread and that thread has previously run on a different CPU, it just grabs and executes it. If you have CPU-dependent state (lazy FP context), this can be a bad idea since it requires extra IPIs and synchronization.

Instead, I propose a standard IPI to request the that LWP be divorced from its last CPU. And until the divorce is complete, the lwp can not be married to a new cpu.

Killing ticks

NetBSD uses implements its clock via a clock interrupt that fires at a (mostly) constant rate, which was the way BSD 4.2 did it over 25 years ago. However, that implementation is showing its age due to the overhead of the clock timers, the coarseness of them, and their power unfriendliness.

This will require a change to the way callouts and other kernel timeouts are done. Instead of passing the number of ticks for the callout to fire, a hard deadline in the future represented in the opaque type of clock_deadline_t will be passed instead. This deadline is based on the monotonic clock kept by the kernel. There will be functions to convert timespecs and nanosecond unitized delays into clock_deadline_t. Other uses of ticks will need to be converted to use callouts such as quantum scheduling or sleep intervals.

The fallout of this is that the system now knows when the next timer needs to fire, which hopefully is more than a few ticks ahead in the future. The system can now pass that knowledge to the MD clock code so it can avoid scheduling premature clock timers. So whenever the deadline changes (either to a sooner or later value), the timeout code will call void cpu_set_deadline(clock_deadline_t). The MD code can use clock_deadline_to_ns(clock_deadline_t) to find when that deadline is and do the appropriate thing.

Additionally, hardclock needs to take an additional uint64_t argument which is the number of nanoseconds since the last call to hardclock.