Are there any examples of that being successfully done? Even djb's software, intentionally written to be minimalist and as secure as possible, has had exploitable overflows (both qmail and djbdns have suffered from this). Every Linux and BSD distribution (even OpenBSD) has suffered buffer overflows so severe that arbitrary internet users could get remote root access. Etc.
> Even djb's software, intentionally written to be minimalist and as secure as possible, has had exploitable overflows (both qmail and djbdns have suffered from this).
Which of these are you describing as a buffer overflow?
That's one approach, yeah. I used to think it was a likely one, but I now think three others are more likely:
1. A language that is low-level and safe but also gives you enough interesting & new to build some buzz/interest, rather than "just" safety. Rust is a candidate here, perhaps.
2. Static analyzers in C advance to the point where a subset of C large enough to be useful can be routinely checked for common types of errors. And it then becomes socially expected that at least core OS stuff will be written in that "checkable" subset of C, treating "unable to prove safety" warnings as errors, or at the very least as suspicious.
3. Mitigate it at the OS level with finer-grained access controls. Utilities like strings(1) or objdump(1) are the easy case here: they do not need to actually have permissions other than "read a file" and "print to output". Even in the worst case, arbitrary code execution in objdump(1) should not be able to delete your home directory, join a botnet, or email your ssh key somewhere, because objdump(1) does not need those permissions. FreeBSD's libcapsicum looks promising, in the sense that it is actually being implemented in the base system, rather than just being yet another ACL proposal going nowhere (Solaris/Illumos also has an actually-shipped privileges system, but I don't know how extensively the base install itself uses it).
1. "Just" safety is hardly peanuts given the status quo. In fact, "just" safety would be much more practical. It'd be much easier to port all the existing code to a C dialect (like Cyclone) than rewrite from scratch in something like Rust.
2. I find compiler instrumentation (think AddressSanitizer and Mudflap) to be more promising than static analysis. Much of the latter is still stuck in the lint era and give out too much noise. That said, tools like Coverity have come a long way and I know a lot of FOSS projects use them frequently. I personally haven't.
3. Capsicum is quite promising, indeed. I like that it extends the existing file descriptor metaphor and offers sandboxing based on namespaces instead of system calls (unlike seccomp), as opposed to the crufty POSIX 1003.1e capabilities which are underdeveloped and still limited to executable processes, AFAIK. That said, we shouldn't just rely on sandboxing, jailing and capability-based security. We need to fix underlying application bugs, as well (the applications that implement the capabilities and sandboxing themselves, particularly so!)