Bug tracking moving to Github Issues, effective now

[denise]

This entry will be stickied in dw_dev for at least a few weeks; please point people at it if they haven't seen it!

Apologies for the delay on writing up guidance on this, folks; we were wrangling out the last of the details. (Or rather, we were wrangling enough of the details for me to make this post; I'm sure there are multiple remaining details unwrangled, which we'll figure out as we go.)

Starting now, and thanks to Bugzilla's sad demise, we will be starting a trial period of using Github Issues to keep track of our bugs. It's possible that it won't work out for us, but right now the downsides are outweighed by the very strong advantage of having everything in one place and the heavy integration.

We will not be importing the entire Bugzilla database, for two reasons: one, putting together something to port over all the data is effort that would be best spent elsewhere, and two, it's been a long time since we've been through the list to cull things that are no longer features that are no longer needed and bugs that are no longer manifesting. We decided that starting fresh with a blank slate and only entering things as they come up will be a good way of making sure the list is only full of relevant things.

This post will be about two things: how to log bugs and how to find bugs to work on. There will be a future post on how a particular item (issue) will move through the workflow with you; we have a few last little things to clear up there first, but I wanted to get this posted as soon as possible so people could start work again.

Of note:

* Please only use this process for code-related bugs. We're not yet 100% certain what the process for docs bugs will be. For now, if you spot a FAQ that needs to be changed or needs to be written, report it with a comment on this entry in dw-docs.

* DO NOT use this process for security bugs. If you think you've found a security bug and want to report it, email webmaster@dreamwidth.org (private support category) and we'll take it from there.

* Part of this transition involves declaring total bug amnesty. If you had a bug assigned to you in Bugzilla, and decide that you don't want it anymore (or if you've forgotten about it completely), you're off the hook. (Not that you couldn't have been off the hook at any time anyway, but I know how hard it is to admit defeat sometimes.)

* On the other hand, if you had a bug assigned to you in Bugzilla, and you're still working on it or were almost ready to submit a pull request, we still want it! Open a new issue in Github for it and carry on.

* Just because we're not making a concerted effort to migrate every open bug from the old Bugzilla database doesn't mean that we don't want bugs to be re-reported. If you logged a bug in Bugzilla (or remember a bug from Bugzilla), and the bug is still happening and still annoying you, please open an issue for it. I will be trying to go through the last year or so of still-open bugs from Bugzilla to find "bugs that are still bugs" and re-create them, but I can't guarantee how long it will take me and I'd rather the bugs get logged sooner than I can commit for-sure to doing it, especially so that we have a nice collection of stuff for people to pick from if they just want to pick something up and hack.

That having been said:

Logging bugs

The process for logging a bug/issue:

1. Go to the dw-free repository's Issues. (You can get there by going to the main dw-free repository, then looking on the right-hand sidebar for the "Issues" link.)

2. Do a quick skim over the existing issues to see if yours is a duplicate. (Right now, finding things is harder than it will be: adding more tags to help significantly with categorization and search is my next major task, but again, I wanted to get this posted as quickly as possible. In the meantime, it's okay if we get a few dupes.)

3. Open an issue if you don't see one already existing for what you want to report. To open an issue, use the green button in the upper right hand corner, before the list of open issues, labelled "New Issue".

4. For "Title", put something both concise and descriptive: what's the one-sentence summary of the task? When in doubt, make the first word an active verb ('remove', 'update', 'fix', etc) and make the sentence itself imperative: "add missing link on update page", "correct display issue in Celerity on community pages", "change text on crosspost tab of Manage Settings to reflect new workflow", yadda.

5. For the larger "Leave a comment" box, describe the issue as thoroughly as you can: steps to reproduce, details of what needs to be changed, what it's doing that it shouldn't do, what it should do that it isn't doing, what would be necessary for the bug to be considered 'fixed', etc. This might be a few sentences, or it might be a few paragraphs. Whatever it takes to give enough detail that someone coming along to work on it can understand the bug and be reasonably confident in tackling it.

6. When you're done, hit 'submit new issue'.

7. (Optional) Go to Open Issues, find the issue you just created, change the milestone (on the right-hand side) to 'Unclaimed', and pick labels for the bug. Each bug will get at least three labels: one 'effort', one 'is', and one 'severity'. More about what those mean in a minute. (Later on there will be four, one for code area as well, but see above re: me not getting to those yet.) If you aren't certain as to what applies, just make your best guess. (Honestly, half the time I'm guessing on effort myself.) Even if you don't pick labels for the bug, it will be very, very helpful if you set the milestone to 'unclaimed'. Nevermind. You can't do this unless you're a member of the organization.

Finding bugs to work on

1. Go to the dw-free repository's Issues. You'll see all the current open issues. This view is a bit of a muddle, with claimed/unclaimed issues all lumped together and pull requests lumped in with the issues.

2. Fortunately, there are labels and milestones! We've done our best to come up with a collection of labels and milestones that will be useful for people to search with, and we're going to keep refining them. (We may, for instance, convert the unclaimed/in progress/pull request milestones into labels instead, freeing up milestones for "which major ongoing project, if any, does this belong to", if it turns out to work better that way.)

3. For now, though: to see all unclaimed bugs, go to milestones (top left 'tab', next to the 'browse issues' tab) and select Unclaimed. Once you've selected the "Unclaimed" milestone, use the labels on the left hand side of the results page to further refine what you want to browse. You can go for "unclaimed issues that are labeled lower-effort", or "unclaimed issues that are bugs", and you can combine labels by clicking on them to get unclaimed issues that are both bugs and major severity.

3a. This use of milestones is why it's very important that newly opened issues be placed in the 'Unclaimed' milestone -- they won't turn up for people searching for them until they are. If you're not sure that the issue is valid or you want me to look at it and accept or reject it first, that's fine, but if you're opening a straightforward issue (or see somebody else opening a straightforward issue), please set the milestone to 'unclaimed' as soon as possible.

3b. This use of labels is why it would also be very helpful for people to label issues as they get reported. Even if you didn't open the issue, if you spot an unlabelled issue, please do add labels to it. (Again, if you're off on the estimate of severity or effort required, that's fine -- it can always be changed later.) Labeling things quickly will get to be more of a priority as we get more items in the issue list and once I've created labels for "area this touches on" (icons, importer, S2 backend, yadda), but again, every little bit that people can chip in on will help. Nevermind. You can't do this unless you're a member of the organization.

4. Once you've found something you want to work on, load the issue. Leave a comment saying "I'll claim this" (so that people can know at a glance that it's spoken for without having to look at the milestones, or in case you forget to change the milestone and so I know to change the milestone).

5. Look at the milestone box on the right-hand side of the issue, where the milestone is listed as 'Unclaimed'. Change it to 'In Progress'. (Once you click on the 'In Progress' line, it will change automatically; you don't need to explicitly save your changes.) Nevermind. You can't do this unless you're a member of the organization.

6. Go through the normal process of creating a branch, making your changes, testing your changes, committing your changes to your branch, opening up a pull request, etc. In your commit message for when you commit your changes, include the sentence "Fixes issue #XXX", where #XXX is the issue number. (You can find the issue number after the title of the issue on the issue detail page, on the right-hand side of the issue on the "all issues" page, or -- if all else fails -- in the URL of the issue. For instance, "Update in-repository documentation to point to GHI instead of Bugzilla", an issue I opened this morning, is issue #663: this is listed after the title on the issue detail page, and at the end of the URL for the issue detail page.)

7. Once you have opened the pull request, set the milestone on both your pull request and the originating issue to 'Pull Requests'. (If you want to be exceptionally helpful and make life easier for fu, leave a comment in the pull request with a link to the originating issue, and leave a comment in the originating issue with a link to the pull request!) Nevermind. You can't do this unless you're a member of the organization. It will still be helpful if you comment with links, though.

There you have it! It will probably take a little bit of getting used to (I know it took me a bit to figure it all out) but -- having gone through the process a few times in the course of figuring things out -- it really is very straightforward once you start doing it. The biggest gotcha, I think, is going to be remembering to set the milestone for all the unclaimed bugs. (That's part of the reason why we're considering using labels for that functionality instead of milestones; there are benefits and detriments to both. Fu and I will decide in a week or two once we see how a small scale test works out.)

Please take this chance to log all the bugs you're still working on, and all the bugs you can think of as still affecting you, over the next few days! Once that's been done, I'll start going through the various "upkeep" tasks (new themes, new embed sources, etc) and add those, then start working through the dw_suggestions posts tagged "bugzilla: migrated" to see which ones should be moved over to GHI.

(ninetydegrees, I know you have spreadsheets for themes that were in Bugzilla and not yet patched; if you want to add those, that would be awesome, but if you don't have the time/energy, I will get to them when I fill in the various "is: upkeep" things.) (Also, I will explain the "is: upkeep" and how that differs from other "is: foo" tags in a few paragraphs!)

Again, I'm sorry for my delay in writing up the guidance for What We're Doing With This -- things took a little more discussion. Thank you all for being so willing to roll with things and try out new workflows.

If you run into questions as you work, just holler.

Appendix: What the labels mean

This is a summary of what the labels mean, and how we plan on using them. This is a work in progress, and will -- once it's definitive -- be added to the wiki and to the Contributing.md document that appears when you open an issue or a pull request.

Last updated 1 April 2014.

1. effort: For estimating the amount of time and effort it will take someone to achieve a satisfactory fix for the issue. Effort is an estimate, and is categorized relative to other issues, not as an absolute -- for instance, it might take Person A two hours for an effort: lower and twenty hours for an effort: average, and take Person B ten minutes for an effort: lower and two hours for an effort: average, but bugs in the same category of effort should be reasonably similar in how long they take you, with the wiggle room that comes from only having three buckets.
   
   • effort: (1) lower: The most straightforward issues, usually requiring small changes only and limited to one or two files.
   
   
   • effort: (2) average: Issues that require a bit more work (usually because they change multiple files, require heavier testing, or require additional research) but can still be done without sweeping changes.
   
   
   • effort: (3) higher: The biggest bugs: adding large new features or fixing very gnarly bugs that require extra planning, careful testing, and multiple areas of work.



2. from: For cases where we want to keep track of how an issue was reported, either for filtering in or filtering out. Not every issue will have a from: tag; that's okay.
   
   • from: suggestions: For bug reports and feature requests that come in through dw_suggestions.
   
   
   • from: support: For bug reports and feature requests that come in through Support. (Not for code changes that will make the process of doing support easier for volunteers; that will be handled with the topic tags.)



3. is: Describing the type of issue.
   
   • is: bug: Reporting a problem with an existing functionality either a) not working as designed or b) working as designed, but as designed is stupid. Does not request new functionality.
   
   
   • is: feature: Requests new functionality to an existing area or workflow of the site, or adds an entirely new area or workflow to the site.
   
   
   • is: pull request: To be used on the 'issue' that's automatically opened when a pull request is opened so they can be filtered around.
   
   
   • is: upkeep: Maintenance-type tasks where the parent task is ongoing and will never be "complete", but each individual task can be considered done once the pull request is closed. Examples: adding new themes, adding new embed sources, adding new mood themes, etc.



4. severity: The impact of the issue on the userbase or on the business.
   
   • severity: (1) minor: Only affecting a few users, or, affecting a larger number of users but is only a minor nuisance, or, extends functionality where the existing functionality works just fine as-is but having this would make it more shiny.
   
   
   • severity: (2) major: Affects a larger number of users, or, affects a smaller number of users but has a direct impact on the business, or, is time-sensitive in some way and should really be done sooner rather than later.
   
   
   • severity: (3) critical: Affects most or all users, or, is very obviously broken and needs to be fixed right now, or, is having a seriously detrimental effect on the business, or, breaks core functionality of the site. Or something's on fire.



5. topic: Will be added shortly; for sorting issues into what code area or functionality it deals with.