Release notes

[mark]

We did our first release from git tonight. It went well, I will chalk that up to all of the hard work from folks like fu and exor674 (and others!) to make sure that everything was in place. fu put together a nice "things we have to do" in dw_staff, so that was very helpful.

Anyway, I wanted to document some things that are important to know for the period between releases. This probably won't apply to most of you, but just in case...

So, we are following the document about a successful git branching model. This is what git flow is based on and it's a really efficient model, so we're using it for doing our features and in this case, our release.

The release process was, in essence:

• Create a release-1.11.0 branch
• Make final tweaks in this branch to get things ready for release
• Push this branch live
• Merge this branch back in to develop and master
• Add a release tag with the version number on master

This leaves our repository in this state:

• develop is still open, you can keep committing things to it (no code freeze, yay!)
• master has stable tags, so you can always roll back to tags here (easy production rolls!)
• release-1.11.0 is open for doing fixes on the live release (easy fixing production, yay!)

The thing that is important to know, if you are going to be committing fixes that you intend to go live, is that you need to make your changes on the release-x.xx.x branch -- not on develop. Any code committed to develop will, because of its nature, have to wait until the next code push -- weeks away.

If you have stuff you need to get live sooner (fixes to things broken in the last code push or stability/important fixes), you need to adjust your workflow slightly. It's pretty easy though, and goes something like this:

git checkout -b release-1.11.0 dreamwidth/release-1.11.0

Now make your fixes, as per normal, and submit a pull request. As long as your current branch is the release branch, then you will be able to submit a pull request on the appropriate branch, and we can merge it in and push it out to the site quite easily.

Comments

[mark]

Hi! I'm up and online now and would be happy to help with this. Is this resolved or are there still problems?

[ninetydegrees]

No I still get the !rejected error when updating repos and the code you gave Yvi didn't work. At this point I think I don't know what to do except rebuild my whole hack but I've been trying to make this work all afternoon so I'm currently playing a little Glitch to go back my zen state. /rambling.

[mark]

Do you mind if I look directly at your repositories on the hack server? I want to see what's up with them, and will come back and document what/if anything I did.

[ninetydegrees]

Oh please go ahead! Thank you!

[mark]

Okay, I'm starting in ~/dw...

The first thing I've noticed is that "git status" shows that this is on the master branch. It should be on develop for doing stuff, so I am switching it with "git checkout develop".

Once I did that, "git status" says that this branch is ahead of dreamwidth/develop. That happens when there are commits locally that aren't on the remote branch.

This is the reverse of the problem yvi had, actually. I took some screenshots to show:

official branch graph:



ninetyd branch graph:



These are almost identical, but the difference is where "develop" is pointing. On your network graph, it's pointing at the same commit as master -- and on Dreamwidth, the develop branch is one back.

It looks like you have merged master into your develop -- and that's where it breaks, since that commit on master will never appear on develop on Dreamwidth's end. You can't pull or update or merge back since you have this extra commit. (Well, you could try to submit a pull request to merge this commit to Dreamwidth's develop, but we would reject it.)

So! How to fix? The reverse of what I gave to yvi:

git checkout develop
git fetch dreamwidth
git reset --hard dreamwidth/develop

This says to switch to the develop branch, then fetch all changesets from GitHub for Dreamwidth, then "blow away whatever I have, overwrite it with Dreamwidth's".

Finally, you will need to push this up to GitHub. Since you will be asking GitHub to go backwards, you have to emphatically push:

git push --force origin develop

You will have to do the same for dw-nonfree, but it will be the exact same steps since we use the same branch names.

Let me know how it goes, or if anything goes wary! I'll be around.

[ninetydegrees]

Yeah I saw it wasn't right: the network graph is really useful for that but the stupid merge is what happens when I run dwu. Otherwise, I had the same setup at the beginning that I have now that I ran the commands you gave me. I don't know why it does that. I'm gonna run dwu again.

Edit: ran it again and it keeps happening. I get that frakking merge thing I pasted above.

[mark]

I don't know what "dwu" is ... is this an update script? It sounds like it's broken, then. Can you link me?

[ninetydegrees]

Oh sorry I pasted it above too. Let me find it... Yeah right here: http://dw-dev.dreamwidth.org/124621.html?thread=1024717#cmt1024717.

Edit: it's from http://wiki.dwscoalition.org/wiki/index.php/Dev_Maintenance#dwu_-_Updating_the_repos. I've never had any problems with it before.

[mark]

Thanks.

That script is not switching to the develop branch first, so it's not going to do what the author expects. If you do a "git pull dreamwidth develop:develop" while on master, it will update master with the commits from develop. And since, right now, the master has different commits, it generates the merge commits that are being a pain.

You can only do a git pull if you're on the right branch, so it needs to be preceded with a git checkout, like this:

git checkout develop
git pull --ff dreamwidth develop

That first makes sure you're on the develop branch, then it pulls it down. I added the --ff because we only want to allow fast-forwarded merges -- merge commits are going to cause problems here, so we don't ever want them.

I updated the dwu script on the wiki page, so that in the future people shouldn't run into this.

Presently though, since dwu pushes up to GitHub, you have pushed the merge commits that dwu made and so your GitHub is in a bad state. The best way to recover, IMO, is just to wipe things out and then start fresh.

This is the process I was advocating a bit ago, that will reset everything. Then you should be able to do the dwu and it will work...

git checkout develop
git fetch dreamwidth
git reset --hard dreamwidth/develop
git push --force origin develop

git checkout master
git fetch dreamwidth
git reset --hard dreamwidth/master
git push --force origin master

Do this in both dw-free and dw-nonfree directories. I will watch my email!

[ninetydegrees]

For the record, I was always in develop (I check before I do anything) but I'll try this.

[mark]

Okay.

git pull dreamwidth develop:develop
git pull dreamwidth master:master

That will merge "develop" into whatever branch you're on. Then it will merge "master" into whatever branch you're on. It doesn't matter where you are -- that particular set of steps will screw up your repositories and then immediately push it up to GitHub, causing all sorts of trouble.

The only reason this ever worked is because we branched develop and then never touched master -- so it worked if someone was always in develop (like you) -- until yesterday, when I updated master. (And the only reason it worked to begin with is because master was an exact subset of develop, down to the ordering and everything.)

When yvi had a problem, it was because she was in master, so it pulled develop over that way.

TMYK, I suppose.

[ninetydegrees]

Okayyyyyyyyyy. That makes sense. Sorry I'm so slow! ;)

[ninetydegrees]

Could you check that my repos are correct before I do anything else please?

[mark]

Sure. GitHub looks fine, and I just checked the repos on your hack. That looks fine too. If you update dwu to the new version, you should be good to go!

Thanks for putting up with all of that! I hope it'll be smooth sailing from here on out. :)

[ninetydegrees]

OMG no error message! No merge! *mwah* Thank you so much. I can work now! Yay! Thanks again.

[mark]

SUPER DELAYED YAAAAAAAY. Glad it works! :D

[ninetydegrees]

Ok I'm not sure everything's alright today. my non-free seems to match dw's non-free but I'm not sure my free does. Both graphs look different for develop and I get "Your branch is ahead of 'dreamwidth/develop' by 18 commits." and "Your branch is ahead of 'dreamwidth/master" by 5 commits.' when I update. :/

Edit: I could reset things so they match thanks to the code you've give me above and dwu didn't give me any warnings this time but I still don't understand why this happens and I have to keep doing that. I'm not doing anything. I haven't created any branch or done any coding. I just open my software and pull updates with dwu.

[mark]

I see you're on the release-1.11.0 branch right now. Are you doing anything with the release branch? I had assumed you would only be touching develop.

[ninetydegrees]

I did that only after I'd 'fixed' my free repo. I hadn't done anything before that. Hadn't even loaded PuTTY (terminal I use on Windows) since yesterday. I just ran it, entered my password, switched to LJHOME, checked which branches I was on (develop) then ran the dwu script. It fetched some changes (I can't remember what it said but I didn't expect things to go wrong then so I didn't pay too much attention :/). Updated the database. Ran dwu again and that's where I got the 'ahead' messages.

This is my dwu script:

#!/bin/bash
stop-apache

cd $LJHOME

# pull changes from dreamwidth
git checkout develop
git pull --ff dreamwidth develop
git checkout master
git pull --ff dreamwidth master

# push them to Github forks
git push origin develop
git push origin master

# change to dw-nonfree
cd $LJHOME/ext/dw-nonfree

# pull changes from dw-nonfree
git checkout develop
git pull --ff dreamwidth develop
git checkout master
git pull --ff dreamwidth master

# push them to Github forks
git push origin develop
git push origin master

start-apache

This is my dwdb script:

#!/bin/bash
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb && \
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb && \
$LJHOME/bin/upgrading/texttool.pl load

[mark]

Your master and develop branches look just fine.

Your release branch has one commit on it -- that you've opened a pull request for. That also looks fine, assuming that you intend to make this commit on the release branch (i.e., high priority bug fixes we want to release before the next official coce push).

So -- assuming that's true, everything looks just fine... where are you seeing the comments about being 'commits ahead'?

[ninetydegrees]

Sorry I was unclear: that was before I did all that.

I did all the steps I mentioned in my previous comment. When I saw the 'ahead' message and my graph didn't match the one for dreamwidth/dw-free I did this:


git checkout develop
git fetch dreamwidth
git reset --hard dreamwidth/develop
git push --force origin develop

git checkout master
git fetch dreamwidth
git reset --hard dreamwidth/master
git push --force origin master

Checked my branches, my graph, ran dwu to make sure everything was ok and it was then. That's when I created the release branch to fix a bug in Librarian's Dream. So everything is fine now but it wasn't before. And I don't understand why because I hadn't done anything on my end. So now it feels my repos are going to keep 'breaking' every time I fetch updates.

[mark]

The repos were broken because of the old version of dwu. Now that you have the new version, you should be fine and nothing else should break. :)

[ninetydegrees]

No, I had the new version. Have had it since yesterday.

And it was only the free repo this time, not the nonfree one.

[mark]

I can't diagnose it post-reset, unfortunately. If it gets into a broken state again, just pause and ping me, and I'll investigate. I need to be able to see what it did, unfortunately.

It would also be helpful, at that time, to see a history of the last few things you typed in the console. Between that and looking at the repositories, I should be able to divine what did what.

[ninetydegrees]

Ok. I've expanded the number of lines I can scrollback to in case this happens again.