Archive for January, 2011

Short Hiatus from Blogging

Well Saturday night I had a most unfortunate accident. A friend’s dog that I was watching decided to bite me on both wrists. This has curtailed my ability to type for extended periods of time and made me lose feeling in my left thumb. So I’m going to have to delay my planned posts on Triage, continuing from the previous few. However, I would greatly appreciate any reader feedback and suggestions. Maybe you’ll have a few ideas that I can throw into a blog post 😉 I hope to be back in a week or two, depending on how my appointment with the hand specialist goes.

Advertisements

Extension Bugs

One common source of bugs that are resolved as “INVALID” on Bugzilla are bugs caused by third-party extensions (Google Toolbar for example). Currently in Bugzilla, most Triagers will move a bug that has been determined to be caused by an extension  to the “Extension Compatibility” component of Firefox, close the bug as RESOLVED>INVALID, and also mark in the whiteboard the offending extension (If known). Sometimes these bugs will morph into blacklist bugs, and there are cases where addons caused legitimate bugs in Firefox that needed to be addressed by Mozilla Developers (see Bug 327859 for a bug in Firefox that Adblock Plus exposed).

However, most extension bugs are bugs in the extensions themselves, not Firefox. There is little our developers can do to fix the issues. It is up to the extension developers themselves to fix the issues. Normally after closing a bug, the triager will ask the reporter to report the issue directly to the developer of the extension. This does rather unfairly place a burden on the original reporter. The majority gladly goes and reports the issue. However, extension reporters can be hard to find and get a hold of. AMO doesn’t necessarily do a great job of this. (Comments aren’t the place to report bugs, and not all developers have the resources or ability to have a bug tracker).

In trying to reach a solution for this, we run into scalability issues. There are over 6,000 addons for Firefox, and each of these has the potential to have bugs. I’ve considered having a specific Extension Product created (similar to the Pugins component on BMO). This approach works for Plugins because there are only 20-30 on BMO. There is no way we could list all 6,000 addons and keep them updated. So I’ve thought of two ways to approach the issue, one much more simple than the other, but not as efficient.

  1. Create an Extension Product in BMO with one component. Move all extension bugs into that product, and mark in the whiteboard and/or Summary what the offending extension is. Then, hope extension developers check for their extension in the list of bugs. This has the benefit of being very simple to implement, but the Extension product then runs the risk of becoming a bug graveyard, similar to the current Firefox:General Component.
  2. This one is more complex and would require more work on the part of Bugzilla and especially AMO developers. However, we already offer plenty of resources to Addon developers on AMO. Why can’t we offer a very simple bug tracker for them as well? Logged on users would be able to go to a bug’s page on AMO, and report a bug on the AMO bug tracker. Then addon developers would be able to see all reported bugs, and mark them as FIXED when they are fixed in the addon. In a way I envision this being similar to Get Satisfaction. Not just a comment section on a blog, but not a full bug tracker either. In addition, web developers that already have a bug tracker and don’t want to use the simple AMO one, can disable it and redirect users to their own bug tracker.
    The nice thing about this is we can direct users to the bug tracker, it would be the same for all extension on AMO, and users would not feel that the burden of reporting a bug is entirely left up to them. We could still resolve bugs on BMO as INVALID, but we can then put a “See Also” link to the appropriate AMO bug when it gets filed. This would make duplicate tracking much easier and users wouldn’t feel like their problem is being ignored.

I am not sure how feasible the second option is, but I don’t see it being too difficult. I plan to try to discuss this with a few AMO guys when I get the opportunity to. Comments are always welcome, maybe there is another option I haven’t thought of or ways to improve the above two.

My Goals for 2011

Well, I’ve decided that I should try to put my goals into words, so then I can go back to them at the end of the year, and see how I’ve done. I always will try to set goals, then modify them halfway through. Maybe if I post them for the Internets to see, I shall be held accountable 😉

  1. Get the number of UNCO Bugs in Firefox down to the 2,000 range and steady. This doesn’t mean closing them all as INCO. I had to do that to get the numbers down to a humanly reachable level. But now that they are almost there (I consider ~4,000 as humanly manageable) I instead to go through and change quantity for quality. Hopefully this will not backfire on me, but I do intend to clean BMO up as much as I can, thus improving it for triagers and developers alike.
  2. Improve the Triage process. I am going to try to brainstorm with other triagers as much as possible, and then maybe this year get some of the ideas that we discussed at the Summit into reality.
  3. Improve Triage relationship with Developers. I would like to see someday a developer or module peer to come to the triage community and be able to say “I have this query that I would like help in going through”. And the triage community would do it. I like the idea of bug days that we used to have, but those ended up turning into just “old bug cleanup days”, rather boring and uninspiring while important non the less. If we can get developers and triagers working together, then I think we can make Bugzilla work for us, not against us.
  4. And other bits and pieces. But these 3 main goals will be enough to keep me busy for a while at least 😉

UNCO Bugs and Useful Information: First Take

Recently, I have been focusing on cleaning up old, forgotten and outdated bugs on Bugzilla. Currently we have almost 8000 bugs that are UNCO in the Firefox Product (an extremely handy table) (there are roughly 1100+ bugs waiting until Jan 30th to be closed as INCO). This number is down from a total of 12,000+ in July at the Mozilla Summit (Not a very good graph, but gets the point across). As far as I can tell this was an all-time high in Firefox, which began around the middle of 2009, when UNCO bugs really took off. Over the next few months I’m going to try to analyze why this massive jump in UNCO bugs happened, what Mozilla can do to help prevent it, problems with the current method of CLOSEME etc. Tonight I’m going to give some first impressions of this 6+ month personal project, as well as ways to help improve the triage of the 6-7 thousand bugs left.

My Process

My usual process in cleaning up bugs in the past few months has been three main steps:

  1. Run a query of bugs I want to target. This started with bugs that had been filed against Firefox 1, 1.5, and 2.0. These versions of Firefox have been End of Life for quite a while now. I encouraged all reporters of these bugs to retest their bug against the most current version of Firefox at the time I ran the query, update their plugins, create a fresh profile, then report the results. The vast majority of bug reporters never got back, which is sad, but something for another blog post. Of those that did (I’d say roughly <10%), the majority didn’t see the issue anymore with the current Firefox (Kudos to our developers, some of the greatest out there). Those few (~1%) that still saw the issue I then went back (yes I was getting hundreds of emails a day for weeks after running a query) and updated their bug with correct version info and asked for anymore information that was needed. I also tried to place it in a better component than Firefox:General (the location most my queries ran through).
  2. When I went through and marked the bugs as CLOSEME in the whiteboard, I typically picked a date 3-4 weeks into the future. I like to give reporters plenty of time to come back. In fact today I received a bug mail from a bug I had marked as CLOSEME in Nov 2010, and closed as INCO in December. The reporter came back with more information. When the date rolled around, I’d typically wait even a few days after the date, then I’d run through and close all bugs that hadn’t received a reply as RESOLVED INCOMPLETE.
  3. After the mass INCO, I’d usually receive another round of emails. Reporters who missed the first email, people whose bugs had slipped though (unfortunately it happens, my bugzilla query skills are by no means ninja), and people who instead of commenting mark their bug as RESOLVED FIXED, without a patch being checked in.

So that pretty much sums up my methods for closing those forgotten bugs in BMO. Now, some issues I’ve run into…

Whiteboard UNCO Bugs

In the process of marking bugs as CLOSEME and INCOMPLETE, I ran into an issue with the current process of running mass queries and marking bugs as CLOSEME. This issue is mainly the fault of a bug in Bugzilla itself, which has sat inactive since 2006. It is that when you go to change multiple bugs in a query at once and make a whiteboard change, it erases the whiteboard, instead of appending the new info. I’m hoping this bug gets fixed sometime soon (birthday present Bugzilla developers? ;). In the meantime, I think this has exposed a bit of a flaw in our process for dealing with bugs.

Currently, we have the keyword list for bugs, which has a large list that we can choose from to mark bugs (checkin-needed being one I use from time to time myself). However, there are many de-facto “keywords” that are used in the whiteboard that aren’t in the keyword list. DUPEME, CLOSEME, etc. Various groups and development leads within Mozilla also use the whiteboard for their own uses ([needs review] [sg:dos] [see comment X], etc.).

Unfortunately, while running through the bug queries of bugs that qualified for closing (I usually make it so the minimum days before I close a bug is roughly 300), I ran into several bugs, in the Firefox:General component that had not been touched in well over 300 days that had this comments in the whiteboard. Due to this Bugzilla bug of erasing whiteboard entries, and me not thinking about it before running queries, these bugs lost the previous whiteboard entries. So, to you guys that lost bugs due to my queries, I do sincerely apologize. Several people have kindly brought this to my attention, and I’m going to be excluding bugs that have content in the whiteboard from future queries.

Closing Real Bugs

While running these queries I’ve often tried to exercise caution that I don’t accidentally close a real legitimate bug, hence my long waiting period before actually closing. I fear that doing mass closings could leave rarely run into bugs in Firefox in the wild without being fixed. However, my reasoning behind this is this: These bugs are not getting any attention right now as they are. The majority of them (and this has been backed up for the past 6 months in bug replies) have been fixed already in new versions of Firefox. Closing as INCO is completely reversible so the bugs are not lost. You can still run queries against them. Closing the bug actually increases the chances that the reporter will come back and look at it, giving us more much needed information. And having a backlog of 12,000 bugs is hurting Firefox development more than helping. I feel that the pro vastly outweigh the cons.

Now, if you have a bug that was closed but you are still seeing it using the current Firefox release or a nightly, please, reopen it, remove the CLOSEME, and mark it against the version you tested. This doc might help you. Just because your bug was closed is nothing against you, nor does it mean we don’t think it is valuable. It just means that more information is needed, and it wasn’t provided.

Ways to Improve the Process

It’s too soon for me to really have too many ideas on how to “fix” bug closure. It has always been painful, and I think it will always be. There may be some ways that we can improve it though.

  • Fix the bugs: On my wish list for fixing, Bug 330509 andBug 577561. These would help future bug closures much easier, making whiteboard dataloss harder, and marking newly filed Firefox bugs with the correct version number, helping prevent 6,000+ Unconfirmed Unspecified version bugs.
  • Take the whiteboard out of the equation in closing bugs: Using the whiteboard for CLOSEME is really a hack that has turned into accepted practice. I don’t know how we could fix this, but possibly having a checkbox that we can use to mark bugs as Inactive after a certain number of days. These bugs wouldn’t be INCOMPLETE per say, but they would be excluded from queries unless you select to search Inactive bugs, and as soon as a comment is made, the Inactive field would be cleared automatically.
  • Make it so end user’s cannot close as FIXED: The majority of our UNCO bugs are reported by end-users. And let’s face it, Bugzilla is not friendly towards end-users. Even long-time developers still fight with Bugzilla from time to time. These reporters see a “RESOLVED FIXED” status, think “My bug disappeared when this update came through”, and naturally think “I should mark this FIXED”. They are only trying to be helpful, we don’t give them much else of a chance. We don’t let just anyone mark a bug as NEW, why should we let them mark it as FIXED? Maybe we can change Bugzilla permissions so anyone who does not have CANCONFIRM can’t close a bug a FIXED. I’ve filed Bug 624135 to address this.

I’m sure I’m going to have more ideas and more issues as the months go on, and I plan on trying to expand my ideas further. This is kind of just a first take, I’m just throwing ideas out there. Please, let me know if you have any ideas on ways to improve triage. Let’s brainstorm.

Firefox 3.5 EOL?

Has Firefox 3.5 been End Of Lifed? I think it’s about time, seeing as it’s been almost a year since we released Firefox 3.6, we are getting ready to release Firefox 4, and the download page for 3.5 says it isn’t receiving security updates, even though it is. It’s alot of work to maintain 3 versions of Firefox.