Workflow overview

[fu]

After a bunch of discussion, we've decided to move from Mercurial to Git. Mercurial and Git are different version control systems. If you're not sure what that is, they basically manage code and the history of how the code has changed.

One big reason for using Git is Github. Github is a service that hosts code, and provides web-based tools to manage commits and reviews. This will make it a lot easier to work collaboratively, and simplify the process of submitting patches. The changeover may be a bit of a pain, but we will work to make it as easy as possible!

We've also decided to simplify our development environment. Instead of having the $LJHOME directory contain a bunch of repositories that need to be synced and copied around -- no one has ever understood this setup without a lot of explaining and re-explaining -- we will just have one set of directories. This means we can get rid of cvsreport.pl, which is the script that does the syncing between the $LJHOME directory and the repositories within it.

Nothing will change yet, but I thought (and denise agreed) it might be a good idea to post a very basic overview of how the process will be changing.

I'm still banging on the workflow; right now I'm just working with a couple of the more experienced devs so we can get a general idea of what needs to be done. We'll invite more people for a test run once to iron out the more fiddly bits once we have a list of steps to try.I expect we'll have to adjust again after people have tried it out for real.

Don't worry! Nothing is going to happen without a lot more in the way of information and tutorials and explanations, but here is a very high-level overview.

This is the general idea we have in mind for devs who work on a Dreamhack and prefer GUI-based tools:

1. Edit your code.
2. Save (commit) your changes to your local copy of the code. No one else will see your changes at this point
3. Push the changes to your repository on your Dreamhack. The effects of the code changes will be public on your Dreamhack, but no one else can see the actual code (yet)
4. Test your changes. Does it do what you want? Are you sure it's not doing anything you don't want (like break related fnuctionality)? Does it look okay? Does it behave okay?
5. Publish your changes to Github, where other people can finally see the code!

Devs who prefer working purely with the commandline have a slightly simpler workflow, but a more complex series of commands ;)

That wasn't the first idea we had. We tossed around an alternate workflow which would let you update your Dreamhack without touching the commandline at all:

1. Edit your code.
2. Save (commit) your changes to your local copy of the code. No one else will see your changes at this point
3. Publish your changes to Github, where other people can see the code. This triggers a script (that we'd write) which would automatically put this code onto your Dreamhack
4. Test your changes.

Pros: it makes the learning curve that much less steep. It means you might be able to get away without touching the commandline in most cases.

Cons: it's clever, maybe too clever. It would encourage publishing code without testing that it even works, and that could bring us to a world of pain. We'd probably have to increase what that automatic deployment involves, until it grows unwieldy because we wouldn't be able to abstract away everything forever.

So somewhat regretfully, we've shelved it. It was an interesting idea though!

In case you were curious, here's how the old workflow looks like, assuming you had copied over the code to your own computer so you could use a GUI text editor:

I hope the difference between the old and the new is pretty clear! More details of exactly what this will entail will come soon.

Comments

[allen]

So I have a couple of questions:

First, I don't see any steps for 'update your Dreamhack to the latest version of the code'. That should really be step one, right? And assuming that the patch you're working on takes enough time that some patches get accepted, you should really pull down those changes while you're working to make sure you don't get out of sync, or at least do it before the final push, right?

Second, what's the plan for working on multiple bugs? So if I start a large project, and then want to take a break and fix a smaller bug or two, how does that work? And if I submit the first smaller patch, then start working on a second one, and get feedback on the first one, how do we manage that?

I'm sure that there are good answers to these questions--I just figure that the sooner we get them out there, the better. :)

(And disclaimer: my work habits aren't necessarily typical, I develop on my own VM rather than on the Dreamhack server, etc....)

[fu]

Re: updating your 'hack

You're absolutely right. We'll have that in the detailed steps, but I think it would make the diagram too crowded.


Re: multiple bugs

Branches! Not in the diagram, but... branches.

More specifically, branch for each feature (question #2), do a rebase before the final push in order to update to the latest (question #1).


(I suspect your work habits are a lot like mine *g*)

[randomling]

I am excite! I like Git.

That is all. :D

[exor674]

Rebases don't work that well if you've already pushed somewhere, though.

[fu]

Hmmm, any suggestions for what we can do then, to be able to make sure our feature branch is at a recent version of the code, before submitting it for review?

[exor674]

Merge merge merge merge merge?

This *may* also be an issue ( and I still want to test what happens when I rebase something I have an open pull request for ) once they've opened a pull request. If we train people "rebase" and their patch gets sent back for changes and they rebase and github doesn't like having a pull request rebased out from under it...

It's generally not safe to edit history on things you've already put somewhere public, as far as I can tell.

[fu]

Ahaha what a coincidence. So do I :D

[fu]

Merge is /o\ like I can see the case for merging the feature branch into the develop branch, but merging the develop branch into the feature branch seems a bit less clean. (versus detaching your changes and attaching them again from to a more recent point?)

I think we definitely need to test what happens if you rebase something that's been submitted for a pull request!

[fu]

Yeah, it's not safe to edit history, and I'd never do it on the master or develop branches, but the feature branches are almost, I dunno, private to one person.

Even if we talk about collaboration, the rebasing will only be done at a point when all other development on that branch should have ceased as it's being set up for a formal submission, does that make sense?

[exor674]

Not necessarily. A feature that depends on a second feature may have to have either a lot of internal rebasing or internal merging. A really long-running feature needing to use the newest code because I sent in a fix for a routing bug that fey was hitting, etc.

[cesy]

I regularly rebase pull requests - you just have to force when pushing to your branch.

[deborah]

this is what I would like to make sure works and relatively simply. My main workflow is submit a patch (or in this case a pull request), start working on another one, and then usually get comments that I have to change something on the bug I have already submitted.

issue tracking

[deborah]

how would we deal with issue tracking? As far as I can tell the github issue tracker is so incredibly oversimplified that we would lose major functionality (I used had all the time to search through the bugs, and I don't think you can tag in the issue tracker). Bugzilla has a github plug-in, right?

Re: issue tracking

[fu]

We're not planning on moving issue tracking away from Bugzilla. There's *some* form of integration between BZ and GH, but I'm not sure yet to what level.

[brainwane]

Good luck. We (MediaWiki) just switched from Subversion to Git and we're still picking things out of our hair... your crowd will probably have a better time of it because you'll be more mindful and because you're already using a DVCS.

[fu]

We'll make sure to write it down (it's one of the things that's almost hidden as a consideration that turns out to be really important!)

Just to give a general idea: all development in separate branches, so you can at any time switch off the branch you're on and switch to the branch of the reviewed bug, commit your changes, and then publish these changes.

[ninetydegrees]

Nothing is going to happen without a lot more in the way of information and tutorials and explanations, but here is a very high-level overview.

Looking forward to these because this s too vague to be helpful for now tbh (I don't think I'm doing the old workflow either so I find it a bit hard to see how different or not it will be).

[fu]

Do you have a link to the workflow you do, btw? I'd love to see it!

[ninetydegrees]

-- Update DH
-- Create patch
-- Edit repo code
-- Refresh to generate actual patch
-- Sync from repo to live
-- Update database
-- Test
-- Rinse and repeat if needed
-- Save patch to hard drive
-- Upload patch to Zilla.

I don't like having a big stack of patches in my queue so I generally pop them out and remove them very quickly. If I need to make modifications I'll import the patch again and push it to the top. Also, if I'm working with styles or CSS, I may do most of the coding and testing here on DW first. Depends on what I need to do.

It seems to me the drawing shows someone working with the live code instead of the repo one and there is generally no 'Your code' part for me I think, which is why I don't really understand the change in workflow (I get I'd no longer have to generate and upload patches but the rest doesn't really make sense to me).

[fu]

Thanks. Good luck to you as well. Looks like some loving attention was paid to the docs -- I hope it works out!

[fu]

One major change we're going with is that we're removing the need to sync from the repo to live -- so the distinction will be moot at some point. It's one of those things that has really tripped up a lot of people!

(Thanks for breaking down your workflow like that <3)