Why We Close Bugs

It seems that my recent post on requesting retest on NEW bugs showed a divide in the community. There are certain community members who agree that old bugs should be pinged, while others think that it is an annoyance rather than a help. And I agree it is a fine line that needs to be walked. There are pros and cons, and I think we as a community need to determine where to draw the line. Closing bugs as INCOMPLETE hurts, but it helps too.

Cons of closing bugs:

  • It can be annoying. Getting thousands of bugspam emails at once is pretty annoying (I know, I get them too). And if your bug has slipped into a query it can be annoying to be asked if you’ve reproduced a bug in a newer version of Firefox when you already have. It happens.
  • It can appear that Mozilla does not care about users and their Bugs. When a user files a bug in 2006, and it never gets touched until a comment is placed on it in 2011 asking the user to test again in the most recent Firefox version, reporters can understandably feel unappreciated.
  • Real bugs can be closed in the process. In closing bugs, we run a slight chance of accidentally closing a bug that is an edge case and has not had any dupes filed.

Pros of closing bugs:

  • It assists with finding bugs that will actually be fixed. If a developer has a list of 1000 bugs covering multiple versions of Firefox, it is a bit hard to find a bug with enough information to fix. However, if Triage has gone through that list, weeded out the ones that do not have active reporters anymore and closed them, that list could potentially go down to 300 bugs. All 300 with recent comments from their reporters or other users who have reproduced the bug, giving recent relevant information. A list of 300 up-to-date bugs is much better than 1000 of varying quality.
  • It enlists the help of reporters to reproduce bugs. There are simply too many bugs for the Triage team and QA team to reproduce all of them at once. By asking reporters to reproduce, we can help enlist our community more.
  • We have nothing to lose. These bugs are sitting there stagnate anyway. Nothing will change if we don’t go after them. If we do comment on the bugs, the chance that we will get valuable information is significantly higher than if they just sat. We have nothing to lose, and plenty to gain.

Now, there are some ways we can improve:

  • Write better follow up comments. Possibly, something like David Eaves suggested in his blog post. Prettified emails. Perhaps Bugzilla could be modified to send an mmail saying “Since you reported this bug, two new versions of Firefox have been released, with many changes in x y z. You can read the release notes of these updates at mozilla.com. The Mozilla community would appreciate it if you would take the time to retest your bug XXXXXX with the latest version using a fresh profile. And update your bug accordingly.” Or something like. I’m not sure how feasible this is, but we definitely need to make our bug replies more friendly.
  • Be better at following up ourselves. I mean come on, we ask the reporter to follow up on their bug, then we as Mozilla let the bug sit, again. It is disheartening to a reporter to be like “They asked me to follow up, maybe they are finally going to fix the darn thing.” Then it sits, again, for years because as a community we dropped the ball. I think that is what is more of a discouragement to our community of bug reporters. That they report a bug that never gets fixed. They spent time that they didn’t have too creating a bugzilla account, reporting a bug, retesting a bug (perhaps multiple times) then it sits. We really really need to pick up the game here. And start to focus on our community a little more.
Advertisements
  1. That Bugzilla ‘retest bug’ mail sounds like a great idea to distribute triage!

    • Huns Warst
    • February 6th, 2011

    It breaks my heart even more when I find a bug to a yet-not-fixed problem with a bitrotten patch attached.

    • Mook
    • February 6th, 2011

    It is incorrect that “we have nothing to lose.” We would, in fact, be losing the goodwill of the bug reporters and gaining (more) of a reputation as not fixing old bugs. Perhaps, in the grand scheme of things, that loss is (perceived to be) less of an impact than the loss of triage time; that means somebody made a judgement call and decided that was better choice. But that in no way means we lost nothing.

    Note that I mentioned nothing about developer time – in my experience, developers rarely go back to the bug list and just browse. Things only get fixed in the course of working towards some other goal. Sometimes that goal can best be accomplished by looking through bug lists, but that’s uncommon, since hacking on new things is much, much more interesting.

    • I agree that we could lose some good will, which is why it is important that we choose the date ranges wisely, and after a bug is retested, developers get on it.
      I know bug fixing is not glamorous work and that developers normally do not go through and just fix bugs out of a list. However, I think that is something that should be changed.
      Having hundreds of minor little bugs that we simply ignore is what annoys the community. (Bug 78414 is a classic, that went how many years without anyone even trying to work on it?). Maybe taking a quarter and having a team just focus on cleaning up old bugs before adding more projects would be a great focus for development in the future. Not just regressions either. Just honest to goodness bugs.

    • Nuss
    • February 6th, 2011

    I think this sounds great. I’ve filed a few bugs of which none have been fixed (if you don’t count those marked as duplicates). I really don’t have a problem with this since I understand that there are tons of other bugs that have higher priority.

    Anyhow, an idea: what if you get a message (either by email or in Bugzilla) after you file a bug that says “Thank you for filing bug X. In the last 24 hours Y bugs have been filed in [the selected component] and Z bugs in total have been filed in Bugzilla today. As of now [the selected component] have A open bugs and B bugs are open in total. Please understand that possible silence from the community does not mean your contribution is not appreciated”, etc.

    By telling bug reporters this right away, I think a lot of the frustration would be avoided from the beginning.

    • That sounds like a good idea Nuss. I think that telling the community that Bugzilla only has so many people maintaining it would go a long way towards helping improve community relations.

    • cuz84d
    • February 6th, 2011

    Very well thought out, I like the ideas too. I think some times bugs sit because the developers have other plans, and other bugs sit because they had no followup. I would think filing a bug against bugzilla to make some sort of auto reply that you can edit or something would be helpful.

  2. Great post Tyler!
    I’m still thinking that PROS of closing INCOMPLETE bugs are greater than CONS. Plus, your posts and the related user comments show many great ideas about how to mitigate the CONS.

  3. Thank you for summarizing the arguments so far. I’m a lot less angry now than I was after reading your previous post!

    I’m CON, largely for the same reasons as Mook. Auto-closing means asking bug reporters to do a lot of work just to clean up lists that developers might not even be looking at.

    If you limit auto-closing to components where a triager has committed to really cleaning up the remaining bugs, and a developer has committed to fixing a good subset of them, it might be worthwhile.

    I think you also need a triager to look through the list of bugs about to be auto-closed, just to make sure none of the bugs in the list look like legitimate edge cases or security bugs. I have seen several security bugs get auto-closed 😦

    • I’m glad I could make you a little less angry 😉 It’s rare for me to do that, so yay! 🙂

      Yes, I agree, you basically have to admit to reporters “we messed up, we let these bugs get away, can you pick up the slack”. I’m by no means saying this is a perfect solution. Unfortunately, in Bugzilla, you have alot of choosing between the least of Two Evils. Bugzilla is being used in ways that it was never designed or meant for (hint for an up-coming post), and as such, it has serious serious flaws. We shouldn’t be having this discussion in the first place if we had the perfect solution. Unfortunately, we don’t, and we are stuck basically two options.
      1. Closing bugs, possibly losing edge cases and pissing reporters off, but cleaning up lists to make Bugzilla better for developers, Traigers, etc.
      2. Leaving the bugs open, possibly letting them drift off and never be touched (at least closing them we do get plenty of reporters coming back say “Yes this still happens” and we can get more info. But if we leave them open, we know we aren’t irritating people and we know the edge cases are still there.

      One thing that really makes me less hesitant about closing bugs is, it’s completely reversible. In fact, you can search closed bugs, even VERIFIED INCOMPLETE can be searched and reopened quickly. And removing a CLOSEME is really fast. If this was an irreversible process, I’d be the first to say “be more cautious”, but it’s not. Getting any reply on these bugs is better than none. That’s why I close.

      And yes, I do agree, we need developers committed to going through the bug lists that have at least been retested. However, I’ve yet to see a developer in any component with that kind of dedication to QA and squashing bugs. This really frustrates me, but there is nothing I can do to change that. I can’t code, not at the level required for most bugs. Believe me, if I could, I would be fixing them.

    • guanxi
    • February 9th, 2011

    “We have nothing to lose”

    I appreciate the concern about the community, but I’m still reading that volunteers’ time is worth “nothing”. Perhaps that’s not what you meant, but that’s the implication.

    I think the discussion overlooks a fundamental of economics and management, opportunity cost. Any time you consume a non-infinite resource (person-hours, in this case) for a task, you lose the benefit of completing another task. The solution is to spend the resources where they will provide the most benefit.

    Volunteers time is limited, and it’s hard to believe that the best use of it is triaging the bugs least wanted by developers. How about if developers listed the bugs they were most interested in, and people triaged those?

    I understand the urge to clean up Bugzilla; nobody likes chaos. But I run an IT organization and our ticket db is a mess too. We live with it because we have bigger and better ways to help people.

    Good luck!

    • Seeing as I am fully a volunteer, I think it’s funny how I’m accused of treating volunteers time as worth nothing. I’m trying to maximize the impact of the time spent by triagers, who are entirely volunteer. Nobody knows limited time like I do, working Mozilla in between school and two jobs.

      IF developers would commit to fixing certain lists of bugs, then there would be a chance for what you suggest. However they don’t. Triagers are left to do a thankless job that developers criticize and hinder change.

      Cleaning up Bugzilla is not a wasted task. Going through, closing 100 bugs, and getting 1 reply out of those lists is way better than leaving those bugs to rot for years and never getting them touched. That’s what people don’t understand, these bugs that get closed, are dying anyway! Nobody will touch them if we don’t. There are simply too many bugs to try to work through them all right now.

        • guanxi
        • February 9th, 2011

        Interesting. Years ago I triaged bugs, but I decided my time wasn’t being used efficiently. At the time, I suggested to a couple of people that devs simply post the issues/bugs they planned to work on or needed help with, but nobody responded.

        I respect that you feel differently and I didn’t say the time was a waste, but I feel that there are other places in the world to donate time where it will have more impact. It wouldn’t take much effort by Mozilla to prioritize triaging and make it more worthwhile, and I doubt I’m the only person who feels this way.

        Regards,
        guanxi

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: