denise: Image: Me, facing away from camera, on top of the Castel Sant'Angelo in Rome (Default)
Denise ([staff profile] denise) wrote in [site community profile] dw_dev2014-04-01 11:26 am
Entry tags:

Bug tracking moving to Github Issues, effective now

This entry will be stickied in [site community profile] dw_dev for at least a few weeks; please point people at it if they haven't seen it!

Apologies for the delay on writing up guidance on this, folks; we were wrangling out the last of the details. (Or rather, we were wrangling enough of the details for me to make this post; I'm sure there are multiple remaining details unwrangled, which we'll figure out as we go.)

Starting now, and thanks to Bugzilla's sad demise, we will be starting a trial period of using Github Issues to keep track of our bugs. It's possible that it won't work out for us, but right now the downsides are outweighed by the very strong advantage of having everything in one place and the heavy integration.

We will not be importing the entire Bugzilla database, for two reasons: one, putting together something to port over all the data is effort that would be best spent elsewhere, and two, it's been a long time since we've been through the list to cull things that are no longer features that are no longer needed and bugs that are no longer manifesting. We decided that starting fresh with a blank slate and only entering things as they come up will be a good way of making sure the list is only full of relevant things.

This post will be about two things: how to log bugs and how to find bugs to work on. There will be a future post on how a particular item (issue) will move through the workflow with you; we have a few last little things to clear up there first, but I wanted to get this posted as soon as possible so people could start work again.

Of note:



* Please only use this process for code-related bugs. We're not yet 100% certain what the process for docs bugs will be. For now, if you spot a FAQ that needs to be changed or needs to be written, report it with a comment on this entry in dw-docs.

* DO NOT use this process for security bugs. If you think you've found a security bug and want to report it, email webmaster@dreamwidth.org (private support category) and we'll take it from there.

* Part of this transition involves declaring total bug amnesty. If you had a bug assigned to you in Bugzilla, and decide that you don't want it anymore (or if you've forgotten about it completely), you're off the hook. (Not that you couldn't have been off the hook at any time anyway, but I know how hard it is to admit defeat sometimes.)

* On the other hand, if you had a bug assigned to you in Bugzilla, and you're still working on it or were almost ready to submit a pull request, we still want it! Open a new issue in Github for it and carry on.

* Just because we're not making a concerted effort to migrate every open bug from the old Bugzilla database doesn't mean that we don't want bugs to be re-reported. If you logged a bug in Bugzilla (or remember a bug from Bugzilla), and the bug is still happening and still annoying you, please open an issue for it. I will be trying to go through the last year or so of still-open bugs from Bugzilla to find "bugs that are still bugs" and re-create them, but I can't guarantee how long it will take me and I'd rather the bugs get logged sooner than I can commit for-sure to doing it, especially so that we have a nice collection of stuff for people to pick from if they just want to pick something up and hack.

That having been said:

Logging bugs



The process for logging a bug/issue:

1. Go to the dw-free repository's Issues. (You can get there by going to the main dw-free repository, then looking on the right-hand sidebar for the "Issues" link.)

2. Do a quick skim over the existing issues to see if yours is a duplicate. (Right now, finding things is harder than it will be: adding more tags to help significantly with categorization and search is my next major task, but again, I wanted to get this posted as quickly as possible. In the meantime, it's okay if we get a few dupes.)

3. Open an issue if you don't see one already existing for what you want to report. To open an issue, use the green button in the upper right hand corner, before the list of open issues, labelled "New Issue".

4. For "Title", put something both concise and descriptive: what's the one-sentence summary of the task? When in doubt, make the first word an active verb ('remove', 'update', 'fix', etc) and make the sentence itself imperative: "add missing link on update page", "correct display issue in Celerity on community pages", "change text on crosspost tab of Manage Settings to reflect new workflow", yadda.

5. For the larger "Leave a comment" box, describe the issue as thoroughly as you can: steps to reproduce, details of what needs to be changed, what it's doing that it shouldn't do, what it should do that it isn't doing, what would be necessary for the bug to be considered 'fixed', etc. This might be a few sentences, or it might be a few paragraphs. Whatever it takes to give enough detail that someone coming along to work on it can understand the bug and be reasonably confident in tackling it.

6. When you're done, hit 'submit new issue'.

7. (Optional) Go to Open Issues, find the issue you just created, change the milestone (on the right-hand side) to 'Unclaimed', and pick labels for the bug. Each bug will get at least three labels: one 'effort', one 'is', and one 'severity'. More about what those mean in a minute. (Later on there will be four, one for code area as well, but see above re: me not getting to those yet.) If you aren't certain as to what applies, just make your best guess. (Honestly, half the time I'm guessing on effort myself.) Even if you don't pick labels for the bug, it will be very, very helpful if you set the milestone to 'unclaimed'. Nevermind. You can't do this unless you're a member of the organization.

Finding bugs to work on



1. Go to the dw-free repository's Issues. You'll see all the current open issues. This view is a bit of a muddle, with claimed/unclaimed issues all lumped together and pull requests lumped in with the issues.

2. Fortunately, there are labels and milestones! We've done our best to come up with a collection of labels and milestones that will be useful for people to search with, and we're going to keep refining them. (We may, for instance, convert the unclaimed/in progress/pull request milestones into labels instead, freeing up milestones for "which major ongoing project, if any, does this belong to", if it turns out to work better that way.)

3. For now, though: to see all unclaimed bugs, go to milestones (top left 'tab', next to the 'browse issues' tab) and select Unclaimed. Once you've selected the "Unclaimed" milestone, use the labels on the left hand side of the results page to further refine what you want to browse. You can go for "unclaimed issues that are labeled lower-effort", or "unclaimed issues that are bugs", and you can combine labels by clicking on them to get unclaimed issues that are both bugs and major severity.

3a. This use of milestones is why it's very important that newly opened issues be placed in the 'Unclaimed' milestone -- they won't turn up for people searching for them until they are. If you're not sure that the issue is valid or you want me to look at it and accept or reject it first, that's fine, but if you're opening a straightforward issue (or see somebody else opening a straightforward issue), please set the milestone to 'unclaimed' as soon as possible.

3b. This use of labels is why it would also be very helpful for people to label issues as they get reported. Even if you didn't open the issue, if you spot an unlabelled issue, please do add labels to it. (Again, if you're off on the estimate of severity or effort required, that's fine -- it can always be changed later.) Labeling things quickly will get to be more of a priority as we get more items in the issue list and once I've created labels for "area this touches on" (icons, importer, S2 backend, yadda), but again, every little bit that people can chip in on will help.
Nevermind. You can't do this unless you're a member of the organization.

4. Once you've found something you want to work on, load the issue. Leave a comment saying "I'll claim this" (so that people can know at a glance that it's spoken for without having to look at the milestones, or in case you forget to change the milestone and so I know to change the milestone).

5. Look at the milestone box on the right-hand side of the issue, where the milestone is listed as 'Unclaimed'. Change it to 'In Progress'. (Once you click on the 'In Progress' line, it will change automatically; you don't need to explicitly save your changes.) Nevermind. You can't do this unless you're a member of the organization.

6. Go through the normal process of creating a branch, making your changes, testing your changes, committing your changes to your branch, opening up a pull request, etc. In your commit message for when you commit your changes, include the sentence "Fixes issue #XXX", where #XXX is the issue number. (You can find the issue number after the title of the issue on the issue detail page, on the right-hand side of the issue on the "all issues" page, or -- if all else fails -- in the URL of the issue. For instance, "Update in-repository documentation to point to GHI instead of Bugzilla", an issue I opened this morning, is issue #663: this is listed after the title on the issue detail page, and at the end of the URL for the issue detail page.)

7. Once you have opened the pull request, set the milestone on both your pull request and the originating issue to 'Pull Requests'. (If you want to be exceptionally helpful and make life easier for [personal profile] fu, leave a comment in the pull request with a link to the originating issue, and leave a comment in the originating issue with a link to the pull request!) Nevermind. You can't do this unless you're a member of the organization. It will still be helpful if you comment with links, though.

There you have it! It will probably take a little bit of getting used to (I know it took me a bit to figure it all out) but -- having gone through the process a few times in the course of figuring things out -- it really is very straightforward once you start doing it. The biggest gotcha, I think, is going to be remembering to set the milestone for all the unclaimed bugs. (That's part of the reason why we're considering using labels for that functionality instead of milestones; there are benefits and detriments to both. Fu and I will decide in a week or two once we see how a small scale test works out.)

Please take this chance to log all the bugs you're still working on, and all the bugs you can think of as still affecting you, over the next few days! Once that's been done, I'll start going through the various "upkeep" tasks (new themes, new embed sources, etc) and add those, then start working through the [site community profile] dw_suggestions posts tagged "bugzilla: migrated" to see which ones should be moved over to GHI.

([personal profile] ninetydegrees, I know you have spreadsheets for themes that were in Bugzilla and not yet patched; if you want to add those, that would be awesome, but if you don't have the time/energy, I will get to them when I fill in the various "is: upkeep" things.) (Also, I will explain the "is: upkeep" and how that differs from other "is: foo" tags in a few paragraphs!)

Again, I'm sorry for my delay in writing up the guidance for What We're Doing With This -- things took a little more discussion. Thank you all for being so willing to roll with things and try out new workflows.

If you run into questions as you work, just holler.

Appendix: What the labels mean



This is a summary of what the labels mean, and how we plan on using them. This is a work in progress, and will -- once it's definitive -- be added to the wiki and to the Contributing.md document that appears when you open an issue or a pull request.

Last updated 1 April 2014.

  1. effort: For estimating the amount of time and effort it will take someone to achieve a satisfactory fix for the issue. Effort is an estimate, and is categorized relative to other issues, not as an absolute -- for instance, it might take Person A two hours for an effort: lower and twenty hours for an effort: average, and take Person B ten minutes for an effort: lower and two hours for an effort: average, but bugs in the same category of effort should be reasonably similar in how long they take you, with the wiggle room that comes from only having three buckets.
    • effort: (1) lower: The most straightforward issues, usually requiring small changes only and limited to one or two files.

    • effort: (2) average: Issues that require a bit more work (usually because they change multiple files, require heavier testing, or require additional research) but can still be done without sweeping changes.

    • effort: (3) higher: The biggest bugs: adding large new features or fixing very gnarly bugs that require extra planning, careful testing, and multiple areas of work.


  2. from: For cases where we want to keep track of how an issue was reported, either for filtering in or filtering out. Not every issue will have a from: tag; that's okay.
    • from: suggestions: For bug reports and feature requests that come in through [site community profile] dw_suggestions.

    • from: support: For bug reports and feature requests that come in through Support. (Not for code changes that will make the process of doing support easier for volunteers; that will be handled with the topic tags.)


  3. is: Describing the type of issue.
    • is: bug: Reporting a problem with an existing functionality either a) not working as designed or b) working as designed, but as designed is stupid. Does not request new functionality.

    • is: feature: Requests new functionality to an existing area or workflow of the site, or adds an entirely new area or workflow to the site.

    • is: pull request: To be used on the 'issue' that's automatically opened when a pull request is opened so they can be filtered around.

    • is: upkeep: Maintenance-type tasks where the parent task is ongoing and will never be "complete", but each individual task can be considered done once the pull request is closed. Examples: adding new themes, adding new embed sources, adding new mood themes, etc.


  4. severity: The impact of the issue on the userbase or on the business.
    • severity: (1) minor: Only affecting a few users, or, affecting a larger number of users but is only a minor nuisance, or, extends functionality where the existing functionality works just fine as-is but having this would make it more shiny.

    • severity: (2) major: Affects a larger number of users, or, affects a smaller number of users but has a direct impact on the business, or, is time-sensitive in some way and should really be done sooner rather than later.

    • severity: (3) critical: Affects most or all users, or, is very obviously broken and needs to be fixed right now, or, is having a seriously detrimental effect on the business, or, breaks core functionality of the site. Or something's on fire.


  5. topic: Will be added shortly; for sorting issues into what code area or functionality it deals with.

kaberett: A sleeping koalasheep (Avatar: the Last Airbender), with the dreamwidth logo above. (dreamkoalasheep)

[personal profile] kaberett 2014-04-01 03:52 pm (UTC)(link)
Is this good to go into the wiki (poss with tweaks)? If so, I'll get on that this weekend [or possibly before, how is it only Tuesday].

oh whoops I should actually say thank you, too :-)
Edited 2014-04-01 16:15 (UTC)
kaberett: Overlaid Mars & Venus symbols, with Swiss Army knife tools at other positions around the central circle. (Default)

[personal profile] kaberett 2014-04-02 11:02 am (UTC)(link)
Sure thing.
fu: Close-up of Fu, bringing a scoop of water to her mouth (Default)

[personal profile] fu 2014-04-01 04:02 pm (UTC)(link)
Hmm, for issues with no milestones, if you go to https://github.com/dreamwidth/dw-free/issues, and then the cog beside "no milestone selected", you can view all issues with no milestone.

If we move over to using milestones for projects (which I like, because that would take the place of meta bugs), we could make something that labels incoming issues with "untriaged", or similar.
fu: Close-up of Fu, bringing a scoop of water to her mouth (Default)

[personal profile] fu 2014-04-01 04:12 pm (UTC)(link)
For the pull requests, we can filter by type:pr or type:issue, so we can skip that step!

ETA: but no interface. Another pair of auto-generated labels if we're already doing auto-labels?
Edited 2014-04-01 16:28 (UTC)
sophie: A cartoon-like representation of a girl standing on a hill, with brown hair, blue eyes, a flowery top, and blue skirt. ☀ (Default)

[personal profile] sophie 2014-04-01 04:18 pm (UTC)(link)
I'll get to work on updating the code tour generator to work with GitHub!
fu: Close-up of Fu, bringing a scoop of water to her mouth (Default)

[personal profile] fu 2014-04-01 04:28 pm (UTC)(link)

Awesome, thank you!

ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:10 pm (UTC)(link)
What about issues that are both for free and non-free. Should that be duplicated or should we keep only one issue in free?
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:11 pm (UTC)(link)
P.S. Also I think the list of styles and themes to convert is pretty short so I should be able to do it. I'll let you know if the UI makes it too hard for me, though.
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 03:19 pm (UTC)(link)
All done with Dreamscapes submissions, D!
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:16 pm (UTC)(link)
Ok thanks.
Second question: how do you change the labels? I don't see anything I can click on to do that.
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:26 pm (UTC)(link)
Ok then I think there are privs at play here because there's no gear for me and the labels header isn't a link. I'm viewing the page in GH's default style and am logged in.
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:36 pm (UTC)(link)
maybe only members of GH/dreamwidth can do that.
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 02:17 pm (UTC)(link)
Re: BTW

Ok, will edit.
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-04-02 12:34 pm (UTC)(link)
Third related question: should we still link to the support request when filing 'from-support' bugs. The interface being more public and open I don't wanna presume. :)

[personal profile] swaldman 2014-04-02 08:22 pm (UTC)(link)
I like the idea of starting from scratch - there were a lot of VERY old bugs. Will somebody be going through and capturing the RFEs that were in Bugzilla? It would be a shame to lose those IMHO, especially those that came from Suggestions. (We may *want* to lose some of the older ones, but that should be deliberate, IYSWIM).

The process stuff seems unclear reading it, but I'm sure that it'll make sense if/when I need to use it (I say if/when, since the chances of having time to work on bugs in the next few years seem slim!)

[personal profile] swaldman 2014-04-02 08:37 pm (UTC)(link)
Sorry! Request For Enhancement. Feature requests.
hotlevel4: (Default)

[personal profile] hotlevel4 2014-04-09 01:42 pm (UTC)(link)
Just a note that apparently in #6 under "Finding Bugs to Work On", the line needs to be "Fixes #XXX" rather than "Fixes issue #XXX".

And links don't manually need to be added to the pull request, as by mentioning the issue you are fixing somewhere in your commit message, it will automatically link the two. ([personal profile] fu can correct me if this is wrong!)
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-05-14 11:55 am (UTC)(link)
Where do site-copy issues go? I noticed event.journal_new_comment.reply.mycomment was missing some words but I don't know if I should file an issue for this. Should I GH it or is there a specific comm for it?
ninetydegrees: Drawing: a girl's face, with a yellow and green stripe over one eye (Default)

[personal profile] ninetydegrees 2014-05-14 12:02 pm (UTC)(link)
Ok thanks! I wasn't sure it covered both.