Weekly Fedora kernel bug statistics – May 11th 2012

  15 16 17 rawhide  
Open: 311 583 76 141 (1111)
Opened since 2012-05-04 3 37 14 0 (54)
Closed since 2012-05-04 3 43 12 1 (59)
Changed since 2012-05-04 10 61 15 4 (90)

F16 bug count is out of control at this point. We’re still managing to close them faster than they’re coming in, but the sheer number of bugs still open is overwhelming. There still exist a lot of older bugs that may have been fixed but the user hasn’t updated the bug. As we’re coming across these bugs that have been in needinfo state for a while we’re closing them out if we have a reasonable expectation that the bug is fixed.

There’s also still a huge number of duplicate bugs, where it’s not immediately obvious that they are duplicates. The end frame in the call chain may be the same, but how we got there differs for example. Or we see multiple instances of a particular trace, from various hardware. (A good example of this case being the numerous irqpoll bugs open for which we still have no real explanation. Likewise softlockup reports).

A curious observation: We’ve had a slight uptick lately in the number of bugs that seem to be caused by bad memory or other hardware problems. Sometimes these stick out like a sore thumb (a single bit flips changing an address from ffffffff81c8bca0 to ffefffff81c8bca0 for example). Other times, they aren’t as obvious.
I have been wondering though if we see more of these in Fedora just due to the extra debug options we leave enabled even in the production builds (like linked list debug, which is pretty much free).

Something that has been interesting though when we’ve found a user with a bad ram report, is searching for other bugs that user filed. Suddenly “weird” oopses make a lot more sense.