I understand device drivers cannot be easily tested (unless we write accurate hardware simulators, which can be done with a lot of effort), and the same happens with time-critical stuff (that could be solved with even more hardware emulation) but this kind of stuff (checking if a known exploit fails) could and should be tested in automated fashion.
Not everything can be tested reliably and automatically, but what can, should.
Oh, you're talking about regression testing. While you have a point, I'd like to point out a recent vulnerability [1] that would likely fail many a test for two reasons:
- The bug is not concrete. It's not entirely in the kernel, and it's not entirely in userspace.
- The developers have a poor understanding of the bug. The current "fix" only mitigates the problem. There are system configurations where it can still be exploited. There are other issues [2] that arise from large address space management that are waiting to be fixed because of this.
But I agree that regression testing for the whole kernel tree should probably be implemented. (for the various subsystems, many developers develop their own test suites)
I understand device drivers cannot be easily tested (unless we write accurate hardware simulators, which can be done with a lot of effort), and the same happens with time-critical stuff (that could be solved with even more hardware emulation) but this kind of stuff (checking if a known exploit fails) could and should be tested in automated fashion.
Not everything can be tested reliably and automatically, but what can, should.