Answering some Questions, and Responding to Comments, Part 1

It has been an interesting week. I published a blog post describing in detail some of my concerns with Triage on August 27th. I intended to simply spark a discussion about some of the finer details of the Triage issues, and explain a few of the reasons why I did and didn’t leave. It was written by a triager for triagers, and as such, it used terms that probably weren’t understood by many outside of the triage and Mozilla community. I would recommend that anyone who wants to a good grounding in how bugzilla.mozilla.org (BMO) works, and what UNCONFIRMED Bugs, Triage, etc. are, read the links at the end of this post. And if anyone needs clarification on the technical details of the What and How of BMO and Triage, I’d be more than happy to write a post on that.

I have gotten quite a few questions and comments regarding triage however, so I wanted to divide them up into a couple main themes that I noticed, and discuss them that way. I will be repeating myself from time to time to hammer home points that are important.

Some Questions and (hopefully) Answers:

  • Should we focus on bringing bugs to zero?

Matthias Versen: “In the beginning i thought that we should try to manage every incoming bug report but i realized later that this isn’t possible. It’s not possible because mozilla is a huge project with millions of users and the target to get all UNCO Firefox bug reports to zero will never happen.”

Gavin Sharp: “I don’t think that focusing on the number of UNCO bugs is useful. We should focus on outcomes, not on driving a number to 0 for the sake of it.”

So, we have the question, what should the goal of triage be? To have zero UNCO bugs? To drive every bug to resolved within a week?

I feel the goal of triage is to filter out the real bugs from the duplicates, invalid and spam bugs in an efficient, professional and through manner. The number of bugs shouldn’t be our emphasis, at least not the main one. We definitely don’t want to repeat the 13,000 bug low that we once had. As I have said, the number of bugs doesn’t matter as much to me as how fast we respond. We have Rapid Release; maybe we should have Rapid Response for triage. I don’t mean closing bugs quickly. I mean not letting a bug languish for 6 months, or even years without a comment.

It would actually be quite an interesting metric to see what the average response time to an UNCO bug is now. Then, as it seems that at least a few of my ideas are being implemented into Triage, to re-measure after they have begun to take effect. While a Rapid Response won’t fix every issue in Triage, by keeping our reporters happy, we will be able to keep a dedicated community willing to contribute again in the future.

What would help our Rapid Response strategy most will be a separate UNCO Bug Product, making bugs that need to be triaged easier to find. Along with that, we need some sort of triage flag, so then we can tell what stage of the triage process a bug is in. Organization, so the community can focus on areas of BMO that haven’t been triaged thoroughly yet. Finally, we really badly need several people to do triage full-time. Community Volunteers who are working in their spare time simply can’t always get to every bug. The community does an awesome job (pretty much the only job, as there are no Official Triagers), but, it isn’t quite enough.

  • We need Increased Communication between Triager and Developer

Steve Fink: “I just feel like there’s a conversation that needs to be happening between triagers and developers, and it isn’t, for the most part.” (I also recommend you read the rest of his comment)

There is, and it isn’t happening. We need to turn triage from Triagers vs. Developers to Triagers & Developers vs. Bugs. Triage and Development have the same end-goal, to make a product better, and to fix bugs. In theory, they make a perfect team. Triage takes incoming bugs, filters out the junk, the spam, the distractions, and hands the workable bugs over to devs, who then fix them. In reality, there is very little communication between the triage community and the developer community. I’m sure a lot of this problem stems from the lack of communication within the triage community itself. I would love to have developers come to the triage group and say “I have X amount of bugs that have to do with printing. Here is the list, would you mind working it a bit.” All this without having to organize a bugday. Having a quick reaction time to changes and trends in the Rapid Mozilla is essential.

We also need to learn from developers how they want their bugs triaged. Certain developers may not like certain methods of triage (“don’t use the closeme tag” for example). That is fine, so long as the triage team is aware. Once a good inter-group communication method is figured out, it would be nice to sit down with the various module owners, and find ways that triage can benefit them, and what they feel are dos and don’ts.

But communication needs to be two way. Steve also said “As a developer, I have an instinctive fear of getting sucked into triage because I know it has an insatiable appetite for time. I’d bet many devs would be willing to help with the overall triage process. But if there’s a hint that it means we have to jump on every vague bug report that a triager forwards to us and work on it until it’s good and dead, we’ll run away and hide under the nearest potted plant.”

Triagers often do need to ping a developer on a bug to get input on if it is a real bug or not. Many developers are good about replying when a long-time triager pings them on a bug, but for new beginners, it can be hard to figure out whom to ping, and once you do, to get a response. Having some sort of an agreement with even just a few developers who will agree to help out the triage group when it is needed would be nice.

Intermission

I want to try to split this into several chunks that are easy to digest and discuss. So, I’ll be coming out with a few posts in the next couple of days. Let’s try to keep any discussion relevant to the points I have in each post, it will make my life a bit easier when trying to follow-up. I will get to the all in time, don’t worry.

For Further Reading

The Current Mozilla Wiki Page on Triage

quality.mozilla.org

Advertisements
  1. About whom to ping, the wiki.mozilla.org “Modules” page and its subpages ought to be more discoverable; but they lack one thing, which is “QA” or something, for a list of people who don’t necessarily know how to code but do know how to identify bugs for a specific Product::Component. As a triager specializing in SeaMonkey, I’ve started to Cc wsmwk for Mail & News bugs or whimboo for add-on manager bugs, but how did I learn about them? Mostly by observing what they did on which bugs, and, in wsmwk’s case, by him pinging me when he wanted a SeaMonkey opinion. Not the best “discoverability”.

    • Dao
    • September 1st, 2011

    “What would help our Rapid Response strategy most will be a separate UNCO Bug Product, making bugs that need to be triaged easier to find.”

    How would it make them easier to find?

    “As a triager specializing in SeaMonkey, I’ve started to Cc wsmwk for Mail & News bugs or whimboo for add-on manager bugs, but how did I learn about them?”

    You shouldn’t have to learn about them. The QA contact associated with the component should take care of this.

    • By separating out the Triage from the development, triagers will know that every bug in the UNCO product needs to be triaged, and devs will know every bug in other products is ready to be worked on. Right now, UNCO and NEW are pretty much the same thing, by separating the UNCO out, it at least eases the work on triage, so they will have their own area to work.

      And Firefox General, where most bugs sit forever, doesn’t have a QA contact.

        • Dao
        • September 1st, 2011

        People who can file NEW bugs now would continue to do so when there would be an UNCONFIRMED bugs product. So I still don’t see how this would improve anything. Or are you proposing that we restrict the rights of those users? We could do that today, without an UNCONFIRMED bugs product.

      • Yes. And no, I don’t want to restrict them. I don’t see any benefit in that. I do love the idea that Mike Kaply has, of closing Bugzilla down for a week, and clearing out all the old and abandoned bugs that are on BMO, UNCO, NEW and otherwise. By having an UNCO product, Triagers will basically be able to go through UNCO bugs, have triage flags on the bugs (which devs don’t like apparently. Having a separate product will give us the best of both worlds), give a more simplified interface to first timers, and allow triagers to set up their own rules.

  2. On the idea of a separate Triage product, note that right now, a number of UNCO bugs that get filed in the correct component directly get tackled by a developer. That can only happen in the right components where developers are listening, though. In a separate product, you’d be guaranteed that nobody except (self-)designated triagers will look at those bugs.

    • Yeah, but, the vast majority of bugs go into Firefox General, which doesn’t have ANYBODY working on it (ie. no real owner). Which, again, brings us back to the point that we need several full-timers doing this.

  3. What Component? SeaMonkey::General, which is where most of the bugs I triage start up in? I Cc them _before_ moving a bug to either MailNews_Core or Toolkit, when I’m in doubt whether a particular bug is SeaMonkey-only or not.

  4. Well, don’t Firefox triagers watch The Firefox::General component (or the general@firefox.bugs email address) in Bugzilla? I’ve been watching general@seamonkey.bugs since long before Bugzilla invented “component watching”, because that’s where most of the bugs I triage start in.

  5. [Off Topic Mode ON]

    Irony: since you announced your department from the Mozilla community a lot of interest popped up about your work in Mozilla community. I think you haven’t departed yet.. :p

    [Off Topic Mode OFF]

    • Well, anything I can do to bring improvements to the Mozilla community I will do. I have no bad will towards Mozilla. Just frustration at a lack of change.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: