Why we need to Radically Change Bugzilla, Part 1

Since April 7th, 1998, bugzilla.mozilla.org has served Mozilla. Over 657,000 Bugs have been filed in the multiple products and components covered in the vast database. Over these 13+ years, Bugzilla’s users have been divided into two groups.

  1. End-users: Those people who have found what they think is a bug in a Mozilla product. These may or may not be legitimate bugs in the product and the likelihood that the original reporter will ever reply on their bug is unfortunately low. However, many very real bugs are reported by general end-users, this is a very very important function of BMO.
  2. Developers: Mozilla employees, contributors, etc. Those who actually are working on the Code of Mozilla products. They submit patches, they review changes, etc. They use Bugzilla for both reporting bugs, and for reviewing new changes and features to a product.

These two groups are both essential to the Mozilla ecosystem. Without end-users reporting bugs, software quality would tank, because lets face it, there is simply no possibility that Mozilla’s QA team can catch every bug. The more eyes on the software, the better. Without developers, well, there would be very little change going on in Mozilla.

These two groups use BMO in very different ways. That is why we have the UNCO and NEW bug resolutions. Developers and established community members can file a bug as NEW without having to go through the confirmation process. End-users submitting a first-time bug need to have the bug double-checked and confirmed by another person before it changes state. Developers usually know right where they want their bug to be, end-users often need help navigating Bugzilla’s complex interface. Triagers exist to help end-users, we hardly need to do anything to help developers.

While there have been attempts to improve Bugzilla, the main problem is, Bugzilla is broken. Yes, it is busted, broke, shot, however you want to say it. Now, before you start hyperventilating until your pocket-protector breaks, let me clarify. Bugzilla is broken for end-users. For developers it is an extremely powerful tool. A painfully complicated tool from time to time, but it is a wonderful tool that will continue to serve Mozilla for many years. The recent upgrade to Bugzilla 4.0 makes it much easier to use and quite a bit better. However, for end-users it is a broken tool that only serves to confuse, intimidate and create bad feelings ( I won’t give specific example to protect the innocent, but I’m sure you’ve all seen this).

I think that the best overall solution to this problem is two Bugzillas. One to be dedicated to Developers, the other for End-Users. Yes, this is a very very complicated solution. However, I don’t believe it is possible to “fix” bugzilla in its current state. It is being used for things it wasn’t ever meant to be used for and right now is juggling being an end-user support site and a developer bug database. This brings quite a few benefits to end-users, as well as triagers, developers and QA. While it may seem like too large of a solution to the problem, at this point I think we are standing between major changes, or seeing BMO become unusable and resulting decrease in software quality. I’ll flesh this idea out more over the next week, so please bear with me and tolerate my insanity. I think we still have a few more years of use out of bugzilla, but we do need to seriously change how we handle bugs.

Advertisements
    • Steuard
    • May 16th, 2011

    This seems like a valid set of issues to raise, and I hope some resolution can be found. Let me just say, though, that my own evolution from “end user” to “knowledgeable supporter” and eventually to “(very minor) developer” was very much Bugzilla inspired. I filed my first bugs, interacted with developers, learned to write good test cases, watched the development process, and eventually contributed some patches, and the transitions all happened gradually within the Bugzilla interface. I felt like a part of the “real work” right from the beginning. So whatever improvements you’re considering, I hope that they’ll have a “user upgrade path” at least as natural as what I had there.

    • Yes and Bugzilla is where most people begin their journey with Mozilla. I want to try to preserve that connection with the core, the “real mozilla”, yet still improve the process for everyone involved.

  1. I’ve said in a number of places that my web developer friends have me file bugs for them because they don’t want anything to do with Bugzilla, but expect that if they report their problems–which are almost always genuine bugs in Firefox, and when they’re not, they’re bugs in the relevant standards–anywhere else they’ll get ignored. There’s value-add in the conversation I have with them before filing the bug, I usually get a whittled-down reproducible test case out of it that they might not have been able to produce by themselves, but still, these are knowledgeable people who could be empowered to work directly with FF developers (other than myself 😉 if we put a little thought into it.

    Which is to say that I suppport making Bugzilla better for people who aren’t FF developers, and I hope this class of people is considered when we do.

  2. Hey Tyler. There’s already Hendrix, which is one part of the solution, and sumo, which is another. I think part of the problem is that most users are forced to use the old Guided Form for entering bugs, and thus they haven’t been able to benefit from the work we’ve been doing upstream to make bug filing better. However, all this is going to change soon with a new guided form that I think should make things much better.

    Ultimately, what’s needed is to find the balance where we collect enough information to make a bug useful while not blocking that useful information.

    End users are a part of the user base that we do care about, though, and plans are underway to make upstream Bugzilla better for them. How quickly that gets done depends on how many people pitch in for the actual coding of the solution, though, as Bugzilla development is almost entirely volunteer-based.

    -Max

    • Hendrix is not a solution to this issue unfortunately. While useful for gathering info about trends and public perception, there is simply not enough quality of data collected through Hendrix. Submissions range from “Firefox sucks” to rants about random topics. There is useful information, but filtering it out and then sending it off to BMO is simply a daunting task (coming from a guy who just changed ~2000 bugs versions by hand lol). The Guided Form will help, but that is only putting a band-aid on the issue, we are using BMO for two purposes.

      The whole reason I want to redesign BMO is to help end-users feel more appreciated. Nothing is worse than submitting a bug and never getting a reply. Giving users prompt replies, showing them they matter and making it easier for developers is the goal here.

      • Hey Tyler. Thanks for the response about Hendrix. What about the rest of my comment though?

        I have a lot of thoughts about how to effectively manage public bug reporting, if you want to talk about it in person or on IRC some time.

        -Max

      • Oh, reading again, I see you did address most of my comment. Sorry there. 🙂 I fully agree with you that prompt replies and attention to real problems is extremely important.

        -Max

    • Robert O’Callahan
    • May 17th, 2011

    I don’t see how two Bugzillas would preserve the communication we have between users/Web devs and Mozilla developers, or the process Steuard mentioned where people progress from users to developers.

    A non-Bugzilla interface for end users that eventually generates a bug with the end user CCed on it might work.

    • Roc, I wasn’t thinking of a full-featured Bugzilla database. My terminology last night have been slightly confusing. I’ll try to clarify on another post.

    • Ed
    • May 17th, 2011

    You may find this to be of interest 🙂
    https://bugzilla.mozilla.org/show_bug.cgi?id=657317

    • Yep, my Auto detect version bug is related to this whole redesign of the guided form

    • Stephanie Daugherty
    • May 17th, 2011

    Many end users are going to do the wrong thing. They will assume that a crash can’t possibly be their setup, and file a bug on it without any meaningful information.

    Getting these bugs out of Bugzilla and over to SUMO would help the S/N ratio – we aren’t going to stop them without raising the barrier to entry on bugzilla, and as we know, as soon as we do that, we start losing out on “good” bugs. I’ll shamelessly plug bug 444302 as a step in the right direction – give triagers a tool to get bogus bugs over to someone that can help them quickly, and the overhead of dealing with them will drop dramatically. There shouldn’t be any questions on a report that consists only of “it crashed”, they should just be redirected to the right place.

    Hendrix and SUMO don’t really help the Bugzilla situation right now because they exist in a vacuum from each other. They need to be tightly, if not seamlessly integrated so that they complement rather than overlap with one another.

    Developers, triagers, and support teams should share a unified toolset that helps them address the overlap, and use these tools in tandem to sort through the masses of data, and deliver results. All the compartmentalizing doesn’t help things at all.

    As an aside, SUMO is a monumental effort for Mozilla, but it has the feel of something that was designed from the ground up by a marketing department. From a user experience standpoint, that’s probably desirable, but from a standpoint of being able to help users, harvest data, and generally make Firefox better, it’s got a way to go. There’s been so much trying to reinvent the wheel there that I think people have forgotten that there’s a reason why wheels are supposed to be round. 🙂

    • Stephanie, Bug 444302 is actually something we discussed at the Summit last year. I was very strongly feeling that we need to have better integration between SUMO and Bugzilla, and possibly AMO as well. The ability to move bugs back and forth would dramatically make bug reporting much easier and make alot of the Triage process easier. Thank you for summarizing alot of my feelings.

  3. “Bugzilla is broken for end-users. For developers it is an extremely powerful tool.”

    I think you’re implying that Bugzilla isn’t broken for developers, and I strongly disagree with that implication. 😉 The thing that makes change difficult here is that it’s not _unworkably_ broken. It does just enough things just well enough that it’s hard to move to a new tool.

    I think your idea’s worth exploring, though. Perhaps a webapp purpose-built for serving non-technical users — not developers — and only using Bugzilla as a storage database (somewhere between just dumping an issue into Bugzilla after triage, and using a bug’s comments/whiteboard/etc to invisibly present a radically different UX.)

    • Yes, bugzilla is broken for developers, however, developers have stronger motivation for working around the quirks of BMO. End-users have no reason except their sense of contributing to a good cause. If they sense that they are not appreciated for their work, they go “screw it” and leave, and rightly so. So for developers, the brokenness isn’t worth quitting over (usually). For end users, it’s way too high of a barrier.

      >Perhaps a webapp purpose-built for serving non-technical users — not developers — and only using Bugzilla as a storage database (somewhere between just dumping an issue into >Bugzilla after triage, and using a bug’s comments/whiteboard/etc to invisibly present a radically different UX.)

      Yes, you are thinking along the same wavelength as me I believe 😉

  1. August 27th, 2011
  2. September 7th, 2011

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: