Bugzilla Update

[mark]

Hi all,

First, I'm sorry for the delay in posting this and, in advance, quite sorry for the problem.

Bugzilla has died. In a rather messy and, unfortunately, irrecoverable way. The long story short is that, for historical reasons, it still ran on my personal server. I had a human-error incident with said server the other night and, alas, it was the kind of incident that ended with me accidentally deleting the server and all of its data.

Further unfortunately, I thought it was actually backed up on EBS (one of Amazon's storage systems). It wasn't. (Or well, more accurately, the root volume was -- so I still have /etc and such. But no actual /var/lib/mysql or /home.) I contacted Amazon to ask for help and they heroically tried, but it was to no avail.

In good news, we actually have substantially all of the data collected across our collected email accounts. We can recover things we want to keep, so very little is actually gone.

What's next? Well, we're going to take this time to take a step back and think about how we do bug/task/feature/etc management and how to fit that into the GitHub Issues system. Since we already use GitHub, there would be some advantages to having a centralized system that fits in with the rest of our workflow. (If you've got experience with GitHub Issues and have some advice, please let us know!)

More details will be coming sometime in the next week or two.

Comments

[liv]

Ugh, that really sucks. Thank you for being honest about it, and I'm glad you could recreate most of the data from emails. I liked Bugzilla, but it's kind of old and creaky, so most of the reason I like it is because it reminds me of early days LJ, which isn't a good reason.

It seems me like centralizing everything in GitHub makes sense. Does GitHub allow you to restrict who can report and edit bugs? Because I know the whole ethos of GitHub is to be very open and collaborative, but I would worry about having the bug-reporting system either deliberately trolled, or polluted by people who aren't aware of how our dev structure works, eg people who are frustrated with the Support system may try to use GitHub as a shortcut to get their pet issue worked on.

[denise]

No, it doesn't, but then again neither did Bugzilla (anyone with an account, which anyone could make, could create a bug). Very few people ever did go straight to the bug tracker, and if they do and it's something that shouldn't be there, it could always be closed!

[liv]

I think we kind of got away with Bugzilla being obscure. Anyone theoretically could create an account, but they'd have to know where to find the Bugzilla, which wasn't linked from anywhere prominent, and they'd have to go through the process of account creation. GitHub is really really well known, and it's the first place people would think of looking and lots of people have accounts there already. I don't know if this is ever actually a problem for GitHub based projects; I hope you're right and we won't have a big problem with spam, trollish or plain clueless bug reports.

[denise]

Well, under that theory, we would have been getting reports in Github all along, then, and we haven't. I think 95% of DW users have no idea what Github is and the 5% who do are generally either already familiar with dw_dev and our dev process, or are clued in enough to read the instructions.

[alexbayleaf]

Ugh! What a bummer. I know only too well that horrible sinking feeling as you realise how screwed you are.

Bugzilla is definitely a bit old and creaky, and I'm not a huge fan, but it is pretty powerful. Github's issue tracker is... not powerful. I have a feeling denise would be frustrated by it. It's only got one sort of way to categorise things (tags), and it's hard to do complex searches, and it doesn't really have states other than open/closed. On the other hand, its discussion features are not bad and conversation actually tends to happen there, and the integration with pull requests is nice -- so it's quite good for that side of the process.

There are some third-party issue trackers that can integrate with Github in certain ways. Growstuff uses Pivotal Tracker which would NOT suit you (it's quite specific to a certain agile development methodology with regular iterations, and it's terrible for searching for tickets and stuff), but this shows you the sort of integration that's possible.

If you're interested in looking for good task trackers with github integration, one place to start might be to look at Github's list of supported service hooks. You can find them at https://github.com/blah/blah/settings/hooks (replace blah/blah with the name of your repo). As for which ones are reasonable candidates as task trackers, I don't know, but some of them might be!

Best of luck with it.

[denise]

I have a feeling denise would be frustrated by it.

It's like you know me or something ;)

I am willing to give it a try, though!

[alexbayleaf]

Yeah it's definitely worth trying if only to help you realise what you really want/need. We did that for a couple of months with Growstuff before we got frustrated, but it did at least help us figure out where the friction points were, which in turn helped us choose PT.

[ninetydegrees]

Thanks for the update.
If other trackers are being considered, could accessibility be also taken into account? GH doesn't really care about some aspects of it (based on fb I got from them) and I had to make a special stylesheet to be able to read anything on the site. I don't know how it fares for other volunteers with different needs than mine though.

[denise]

'Accessibility' in that context is so broad as to be unhelpful! What are your particular needs? (You can email me privately if you don't want to get into it in a public comm, denise@dreamwidth.org is the address that's working at the moment.)

[ninetydegrees]

I always need a minimum font size and high contrast and, if I need to understand something or follow a discussion, I need some 'clean space'. GH has very low contrast and doesn't adapt well bigger/minimum font sizes (their answer to that was 'use the zoom function instead'). Their UI doesn't work for me at all: I find the site very cluttered and clunky and generally hard to browse. I go there as rarely as possible.

[denise]

Thanks for the info!

[ninetydegrees]

I'd like to add that I don't mind making a stylesheet so a site works for me but GH's CSS structure is so ridiculously intricate and obscure it makes it almost impossible.

[deborah]

github uses icon fonts without any kind of alternative background text, which is first of all a UX ugh (they are not particularly representational of anything, or in any way obvious), but relevant to this conversation makes getting to some of their page sections nigh-impossible with a screen reader. Also, not all of their JavaScript actions can be accessed without a mouse. I make the site work, but it's nontrivial. We could ask some of our screen reader testers if they would be willing to navigate to the bug site and find a bug that they had filed (in a test sense).

(although to be fair jira has been getting worse about accessibility as well, and I mostly like jira. I haven't used RT in while so I don't know what it's accessibility is like, but on the bright side if we were to try it out and add accessibility problems than we could (a) submit a patch or (b) send me down to the café where Jesse hangs out and I could poke him until he fixes it.)

[havocthecat]

Eep! Good luck!

(I keep this comm on my reading list because I'd like to come play some day, when I'm not working and schooling full time.)

[jewelfox]

I'm aware of the strong practicality considerations and network effects behind choosing Github as a base of operations, but this might be worth considering: http://techcrunch.com/2014/03/15/julie-ann-horvath-describes-sexism-and-intimidation-behind-her-github-exit/

[mark]

I've read the article; thank you for sharing! It's definitely something we've been thinking about and will keep an eye on to see how things go.

[beatrice_otter]

So, once you guys have figured out what the next bug tracker thingy will be, how will you rebuild what you can of the bugs that were in Bugzilla? Will you even try, or just start from scratch? Obviously, you can't do all of them, and for "features we want to implement but haven't yet" you can just go through http://dw-suggestions.dreamwidth.org/tag/bugzilla:+migrated. But what about, say, the guts of the system that were marked for modernization that hadn't happened yet?

I have no actual need to know this, I'm just curious.

[denise]

As Mark says in the entry, we have most of the data. I'd say 'all' of the data, but I'm not 100% certain, although it's looking pretty likely.

We aren't going to put the whole bug database into the new tracker automatically, because of a combination of the data being there but not easily parsed and thus needing human intervention, and the advantages of a fresh start in not having to individually evaluate older items for "is this still applicable and is this bug still happening", but we have it.

[beatrice_otter]

Ah! Thank you.

[sophie]

Would you like me to set something up so that people can forward their Bugzilla email to a certain automated address, which would automatically put it into a format that could be usable? (It could even put it into a new Bugzilla database, if we want.) It's the sort of thing I could code while other things are looked into.

[edit: It could also automatically put the info into GitHub, say.]

[denise]

We have the data, thanks! And we aren't going to be automatically importing the whole bug db.

[sophie]

Okay, awesome. :)

[erika]

I'm sorry to hear this! /o\ I hope you guys can get back up and running on an appropriate venue for your bug swatting.

[sophie]

Bummer; I only just learned about this.

How did the accident occur, out of curiosity? I doubt I'd be able to help recover anything (you've probably already tried everything I'd suggest), but I am curious whether it was the infamous rm -rf / var/tmp/* sort of thing (which, of course, most system administrators will do at least once in their lifetime).