Links, Lots of Links

I wanted give a bit of input to an idea I have started to hear tossed around which I find quite intriguing. Shutting down BMO for a week or so, and devoting ALL Mozilla’s resources (devs, QA, the community, everyone) to cleaning up BMO. With 5868 UNCO bugs, and 7191 NEW bugs in Firefox alone, we obviously have a bit of need for cleaning up. Then throw in Toolkit (UNCO bugs, NEW Bugs), Thunderbird (UNCO bugs, NEW bugs), Core (UNCO bugs, NEW bugs), and so many other components and products, and really, we do have a mess on our hands.

The solution isn’t as simple as it sounds. If we just shut BMO down now, we will be using resources ineffectively, and without a long-term solution we will be right back at square 1 within a year or two. I whole-heartedly approve of shutting BMO down for a week or so, but I also know we need a multi-pronged approach. If you read my series on why we need to radically change triage, you can read most of my points there, and they are ones I have been saying repeatedly for the past year. I’ll quickly summarize them here though.

  • Create a separate product for UNCO bugs and triage. There is a new Input Tool in the works right now, and it will be using this idea in a more constrained form. I have had some discussions with the people working on it, and hopefully we can get this UNCO product rolled out to cover all bugs submitted for Firefox/Toolkit/Thunderbird/etc. I do see promise of progress here, so this is good news. Once we get this fully implemented, I feel all UNCO and abandoned NEW bugs should be moved into the product.
  • Create a way for Triagers to tell how far a bug is in the triage process, using multi-state flags. This idea was proposed at the Summit 2010 by tmyoung. Unfortunately, a lot of devs didn’t want to have their bugs cluttered up with more flags (which is understandable). By implementing the flags only in the UNCO product, they will serve triagers, and once a bug is moved out of the product, they will be no longer be needed and removed. Hopefully this way we can make both sides happy.
  • We desperately need a way to mark Support and Extension bugs as such. Having a RESOLVED>Support status and a way to automagically open a support issue for the bug will go far to help users feel that we care about them and their issues. The same goes for extensions.
  • We need Bug Triage Guidelines. Something along the lines of what I threw onto the wiki a few months ago (obviously a lot more fleshed out). But something the community, especially new members, can reference.
  • I think that if we implement a lot of the ideas that David Eaves has proposed, we can go far in improving both triage, and BMO in general. I know a few of his suggestions have already been put into production, but I don’t see any of them that would harm the community, and I think they should be thought about again.
  • I’m not sure how practical this is, but perhaps we can create some sort of bot to automatically ping a triager when a bug has gone a certain amount of time without a comment, and to ping reporters when they have not replied to their bug after a certain amount of time. This would eliminate much of the time that is spent “cleaning up”, freeing more time for “real” triage.
  • Hire several official Traigers. To do several things (most of which I’ve already discussed, so I give it in brief here). Organize the community, help new community members, communicate with Devs, do full-time triage work, be advocates for the community in Mozilla.
  • Having a yearly intense cleaning like Mike proposed would also be great, because even with the best of efforts, some bugs will slip through the cracks. The first one I think we should set a week aside for, but subsequent BMO cleaning could probably only be done in 3-4 days.

So I know this is a fairly short summary, but these are by far my most important changes to BMO, and the Triage process that I think need to be made. Some progress is happening, so I look forward to seeing what is done in the near future.

Advertisements
  1. On adding a bot to ping triagers, if we create a search for bugs that have not had a changed to a given field or status in “x months,” then we can use Bugzilla whining feature to notify specific triagers; it will send that list of bugs to whomever is on the list. No new application or tool would need to be written.

    • Dao
    • September 7th, 2011

    “Once we get this fully implemented, I feel all UNCO and abandoned NEW bugs should be moved into the product.”

    Over my dead body!

    Seriously, I actually review unconfirmed/old bugs in Firefox::Theme and Toolkit::Themes from time to time. Obviously I’m not going to do that if they are buried under thousands of non-theme bugs.

    • How about if we just Move Firefox/General bugs over for now Dao? I would be happy with the compromise, as it would take care of the most troublesome area for triage.

        • Dao
        • September 7th, 2011

        That would save my existing bugs. But if the policy would be that all new bugs be filed in a catch-all bucket, I’ll still have to stop triaging future theme bugs.

      • Dao, nothing is stopping you from going into the UNCO bug product and working on theme bugs. Also, this won’t be for “Every” bug. I wrote in my previous blog posts that this would be a default for people new to BMO (like the guided bug reporting form), and reporters can always change the settings in their preferences. The theory of having the UNCO product and official triagers is that the triage will be done within a few days of the bug being reported, and so Theme bugs that were valid would then be marked NEW and moved into Firefox>Theme.

        • Dao
        • September 7th, 2011

        The thing is, even people new to bugzilla sometimes file useful bugs. Incidentally, those are also the people who tend to file bugs in the right product and component. Getting them triaged “within a few days” doesn’t help me, as I currently get notified about them as they are filed. You can’t beat that. This process wouldn’t help triagers either, it would just be more work for them (work they currently don’t need to do because I do it).

      • Well Dao, I would be interested in how you feel the process should be improved? because the fact is, it is broken, and while you do work on Theme bugs, so many other bugs aren’t ever getting touched, we need to do something, and that might mean making major changes.

  2. These multi-state “triage” flags seem tro me to be redundant with part of the new “per version” flags, which are also multi-state but remain useful throughout the life of the bug:
    — = wasn’t tested with this version
    unaffected = WFM on this version
    affected = confirmed on this version
    wontfix = won’t be fixed on this branch (usu. too late for l10n)
    fixed = fixed on this branch

    IMHO the early states of these flags relate to triaging, the late stages to fixing and even to backporting the trunk fix to aurora, beta, etc.

    • Often a Triager needs to go into more detail as to the next step needed in triage, which is what the Triage Flags are for.

  3. As I think I commented on one of the much earlier posts, taking the UNCO bugs out of developers’ hands has negatives as well as positives.

    I know the triage done by developers is inconsistent or non-existent (depending on which developers and components), but it happens and sometimes it leads to bugs getting diagnosed and fixed within a matter of hours of them getting filed. If all those bugs have to go through a separate triage product first, then even if the official or unofficial triagers can move them on quickly, it’s still going to be a hurdle that isn’t there now.

    I guess, like with other big changes, there’s a trade off of making things slower and more painful in the short term in order to improve things in the long term. I just wonder if there’s a way of getting the long term gain without negative bits…

    • I agree, there is some triage that devs do (dao, unfortunately you are the exception, and not the norm). But we need to figure out a way to make this better for the community as a whole, and to do that we need to have discussions about what we should change.

    • Dao
    • September 7th, 2011

    Tyler Downer :
    Well Dao, I would be interested in how you feel the process should be improved? because the fact is, it is broken, and while you do work on Theme bugs, so many other bugs aren’t ever getting touched, we need to do something, and that might mean making major changes.

    What’s broken is mostly Firefox::General. Other than that, it appears that you’re trying to fix what isn’t broken. What’s needed is a solution specifically for Firefox::General. The UNCONFIRMED product isn’t that solution.

    Triage-specific flags can be hidden by default and opt-in for triagers, or you could use a tool that queries bugzilla and keeps triage meta data for bugs without bugzilla actually knowing about it.

    • Well, Firefox and Toolkit >Theme have 313 bugs that are UNCO or NEW and haven’t been changed in 90+ days. 90 days is June 9th… Firefox 5 and 6 shipped without these bugs being changed.

      Tabbed Browsing has 528 bugs.

      Bookmarks and History has 904.

      Location Bar has 392.

      Toolkit Download manager has 462.

      Addons manager has 417.

      This chart for Toolkit, the same thing for Firefox, and core. I think we do have a problem. Yes, certain component owners do triage their bugs, but the sad thing is, most don’t, or they do inconsistently. We need a way to fix this, and unfortunately, it isn’t as simple as you make it sound.

        • Dao
        • September 7th, 2011

        I’ve looked at every single bug filed in Themes and Tabbed Browser during the Firefox 5 and 6 dev cycles. In the past I’ve also looked at all legacy bugs in Themes and Tabbed Browser. Sure, I didn’t touch all of them. Nevertheless, I’m confident that we do not have a QA problem in these components. If you think your numbers speak a different language, then maybe there’s a problem with your interpretation of these numbers.

      • Dao, if they haven’t been worked on, they are sitting dormant. If they are valid bugs, they should be marked as NEW, if they are not, they should be RESOLVED in some way. I fail to see why you feel a bug that was “looked at” but still is UNCO (For example, Bug 357201, filed in 2006, only touched by me and Wayne moving it around, no Dev input). If it isn’t a valid bug, close it, if it is, mark it NEW. at least once a bug is NEW, Triage doesn’t have to worry about it. I know how Bugs work Dao, I haven’t spent 3 years on BMO without learning that basic. But what is needed is to change how we do things.

      • Having just looked at a few of the earlier bugs from that 313… A couple could probably be “INCOMPLETE” because they’re waiting for information that’s probably not ever going to be provided. More of them though, I would say, don’t look they need “triage” as such. What they need is QA (testcases, testing on a particular obscure setup, etc), or a decision on whether they are actually bugs (“This doesn’t work the way I think it should, please change it”).

        A bunch of them were actually filed by developers, or people who I know have editbugs, so they would be skipping the new component anyway (as I understand it).

        Attacking that list might reduce the number of UNCO and probably resolve a few, but I suspect (at least with the older ones), you’d just move to having a huge pile of NEW bugs that would never get worked on – is that an improvement?

      • I agree we need to clean those lists up, which is why we need a BMO cleanup week. I’m not set on moving all old bugs to an UNCO product, but we need something to help with incoming bugs. Because what we aren’t ding now isn’t helping.

    • I think something along those lines could be useful, perhaps a link before the bug is submitted “Before you try this, please do safe mode, new profile, etc”

      • the guided form supports per-product hints which are displayed on the second step. the currently blurb for firefox is “If you are new to Firefox or Bugzilla, please consider checking Firefox Help instead of creating a bug.”. it’s easy to change this, and i’m open to suggestions.

    • Emily
    • November 2nd, 2011

    HONEY, you are so AMAZING! I love you!

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: