Release notes

[mark]

We did our first release from git tonight. It went well, I will chalk that up to all of the hard work from folks like fu and exor674 (and others!) to make sure that everything was in place. fu put together a nice "things we have to do" in dw_staff, so that was very helpful.

Anyway, I wanted to document some things that are important to know for the period between releases. This probably won't apply to most of you, but just in case...

So, we are following the document about a successful git branching model. This is what git flow is based on and it's a really efficient model, so we're using it for doing our features and in this case, our release.

The release process was, in essence:

• Create a release-1.11.0 branch
• Make final tweaks in this branch to get things ready for release
• Push this branch live
• Merge this branch back in to develop and master
• Add a release tag with the version number on master

This leaves our repository in this state:

• develop is still open, you can keep committing things to it (no code freeze, yay!)
• master has stable tags, so you can always roll back to tags here (easy production rolls!)
• release-1.11.0 is open for doing fixes on the live release (easy fixing production, yay!)

The thing that is important to know, if you are going to be committing fixes that you intend to go live, is that you need to make your changes on the release-x.xx.x branch -- not on develop. Any code committed to develop will, because of its nature, have to wait until the next code push -- weeks away.

If you have stuff you need to get live sooner (fixes to things broken in the last code push or stability/important fixes), you need to adjust your workflow slightly. It's pretty easy though, and goes something like this:

git checkout -b release-1.11.0 dreamwidth/release-1.11.0

Now make your fixes, as per normal, and submit a pull request. As long as your current branch is the release branch, then you will be able to submit a pull request on the appropriate branch, and we can merge it in and push it out to the site quite easily.

Comments

[ninetydegrees]

I believe running dwu before you finalize things and merge release can lead to Bad things: I had master being merged into develop (it's sort of asking you if you want to do that except it's not formulated like that so it's confusing). I simply deleted and rebuilt both repos but I didn't expect that because I'm a noob.

Edit: ok, it's doing it again. Is that normal? Because I get errors when I run dwu after that.

Edit2: and I can no longer access my hack. :/

[fu]

OH hmm, what are the exact steps that dwu made happen?

(And weird about no longer being able to access your hack! Are you getting any kind of error message when you try t oconnect?)

[ninetydegrees]

#!/bin/bash
stop-apache

cd $LJHOME

# pull changes from dreamwidth
git pull dreamwidth develop:develop
git pull dreamwidth master:master

# push them to Github forks
git push origin develop
git push origin master

# change to dw-nonfree
cd $LJHOME/ext/dw-nonfree

# pull changes from dw-nonfree
git pull dreamwidth develop:develop
git pull dreamwidth master:master

# push them to Github forks
git push origin develop
git push origin master

start-apache


---------------

500 Internal Server Error

Edit: I can't see the access log right now because I'm deleting dw-old and it's taking ages.

Also, the second time, when I rebuilt everything I got an error message when running checkconfig. I think it was 'couldn't get the master database handle' or something like that.

[fu]

Hmm.

What does it say in the apache logs?

Strike that. It looks like you haven't copied over the config file, so see if you can stop the deletion (ctrl+c) and try to retrieve the files from ext/local

[ninetydegrees]

Oh I see what I've done. Sorry about that.

Still getting '! [rejected] develop -> develop (non-fast-forward)' when running dwu. Is that normal?

[fu]

Hmm, let me run through those directions and get back to you.

ETA:

Actually, question, can you run those commands individually, and let me know where it fails? I mean in dw-free or dw-nonfree or something?

[ninetydegrees]

Sure!

Went with dw-free and this is where I got the failure message:

git pull dreamwidth develop:develop

[fu]

Hmmm. Okay, that *is* weird. I've usually only seen the nonfastforward error when pushing, not pulling.


Ummm.

This is with a clean repository? That is, you said that you cloned and started from scratch?

[ninetydegrees]

Yeah but I had the issue with the config files. I'll try again.

[ninetydegrees]

Ok I tried again and got the same errors so I'm trying again to make sure I didn't miss any step.

Question: when I do cp $LJHOME-old/etc/config* local/etc/ it copies only one file: config.pl. Local and private aren't there, right? I've got them in dw-old in dw-old/ext/local/etc/ but not in dw-old/etc/.

Edit: ok I see local is dw-nonfree. Makes perfect sense. Don't see private, though.

[fu]

Yup,the ones that are yours are in dw-old/ext/local/etc (the ones in dw-free/etc dw-nonfree/etc are sample ones to get you started).

I am officially stumped right now! Can you show me the exact commands you're using (starting from cloning ,etc)?

[ninetydegrees]

I don't see where private comes from, though. I looked at the repos on GH and I can't see it.

This is what I've done so far:

dh-ninetydegrees@newhack:~/bin$ cd ~/
dh-ninetydegrees@newhack:~$ git clone https://ninetyd@github.com/ninetyd/dw-free.git $LJHOME
Cloning into '/dreamhack/home/8174-ninetydegrees/dw'...
remote: Counting objects: 53805, done.
remote: Compressing objects: 100% (16319/16319), done.
remote: Total 53805 (delta 36803), reused 53680 (delta 36695)
Receiving objects: 100% (53805/53805), 53.48 MiB | 21.76 MiB/s, done.
Resolving deltas: 100% (36803/36803), done.
dh-ninetydegrees@newhack:~$ cd $LJHOME
dh-ninetydegrees@newhack:~/dw$ git remote add dreamwidth https://github.com/dreamwidth/dw-free
dh-ninetydegrees@newhack:~/dw$ git fetch dreamwidth
From https://github.com/dreamwidth/dw-free
* [new branch] develop -> dreamwidth/develop
* [new branch] master -> dreamwidth/master
* [new branch] release-1.11.0 -> dreamwidth/release-1.11.0
dh-ninetydegrees@newhack:~/dw$ git branch --set-upstream develop dreamwidth/develop
Branch develop set up to track remote branch develop from dreamwidth.
dh-ninetydegrees@newhack:~/dw$ git branch --set-upstream master dreamwidth/master
Branch master set up to track remote branch master from dreamwidth.
dh-ninetydegrees@newhack:~/dw$ cd $LJHOME/ext
dh-ninetydegrees@newhack:~/dw/ext$ mkdir -p local/etc
dh-ninetydegrees@newhack:~/dw/ext$ echo "highest" > local/.dir_scope
dh-ninetydegrees@newhack:~/dw/ext$ cp $LJHOME-old/etc/config* local/etc/
dh-ninetydegrees@newhack:~/dw/ext$ cd $LJHOME/ext
dh-ninetydegrees@newhack:~/dw/ext$ git clone https://ninetyd@github.com/ninetyd/dw-nonfree.git
Cloning into 'dw-nonfree'...
remote: Counting objects: 3876, done.
remote: Compressing objects: 100% (1635/1635), done.
remote: Total 3876 (delta 1841), reused 3859 (delta 1824)
Receiving objects: 100% (3876/3876), 4.26 MiB, done.
Resolving deltas: 100% (1841/1841), done.
dh-ninetydegrees@newhack:~/dw/ext$ cd dw-nonfree
dh-ninetydegrees@newhack:~/dw/ext/dw-nonfree$ git remote add dreamwidth https://github.com/dreamwidth/dw-nonfree
dh-ninetydegrees@newhack:~/dw/ext/dw-nonfree$ git fetch dreamwidth
From https://github.com/dreamwidth/dw-nonfree
* [new branch] develop -> dreamwidth/develop
* [new branch] master -> dreamwidth/master
* [new branch] release-1.11.0 -> dreamwidth/release-1.11.0
dh-ninetydegrees@newhack:~/dw/ext/dw-nonfree$ git branch --set-upstream develop dreamwidth/develop
Branch develop set up to track remote branch develop from dreamwidth.
dh-ninetydegrees@newhack:~/dw/ext/dw-nonfree$ git branch --set-upstream master dreamwidth/master
Branch master set up to track remote branch master from dreamwidth.
dh-ninetydegrees@newhack:~/dw/ext/dw-nonfree$ cd $LJHOME
dh-ninetydegrees@newhack:~/dw$ bin/checkconfig.pl
[Checking Timezone...]
[Checking for Perl Modules....]
[Checking LJ Environment...]
Use of uninitialized value in concatenation (.) or string at /dreamhack/home/8174-ninetydegrees/dw/cgi-bin/DBI/Role.pm line 245.
Use of uninitialized value in concatenation (.) or string at /dreamhack/home/8174-ninetydegrees/dw/cgi-bin/DBI/Role.pm line 245.

Problem:
* Couldn't get master database handle.
dh-ninetydegrees@newhack:~/dw$

[yvi]

I had it when pulling, too. Not sure if my thread on this helps: http://dw-dev-training.dreamwidth.org/39778.html

[ninetydegrees]

I'm on develop on both repos. Wasn't I supposed to be there then?

I'll definitely try the code Mark gave you because this is driving me crazy.

[ninetydegrees]

Well, this didn't work. I'm still getting the !rejected error. *sigh* I have no idea what's going on but I'll just go do something else because I can't deal with it anymore.

Thanks a lot for trying to help, Yvi!

[mark]

Hi! I'm up and online now and would be happy to help with this. Is this resolved or are there still problems?

[ninetydegrees]

No I still get the !rejected error when updating repos and the code you gave Yvi didn't work. At this point I think I don't know what to do except rebuild my whole hack but I've been trying to make this work all afternoon so I'm currently playing a little Glitch to go back my zen state. /rambling.

[mark]

Do you mind if I look directly at your repositories on the hack server? I want to see what's up with them, and will come back and document what/if anything I did.

[ninetydegrees]

Oh please go ahead! Thank you!

[mark]

Okay, I'm starting in ~/dw...

The first thing I've noticed is that "git status" shows that this is on the master branch. It should be on develop for doing stuff, so I am switching it with "git checkout develop".

Once I did that, "git status" says that this branch is ahead of dreamwidth/develop. That happens when there are commits locally that aren't on the remote branch.

This is the reverse of the problem yvi had, actually. I took some screenshots to show:

official branch graph:



ninetyd branch graph:



These are almost identical, but the difference is where "develop" is pointing. On your network graph, it's pointing at the same commit as master -- and on Dreamwidth, the develop branch is one back.

It looks like you have merged master into your develop -- and that's where it breaks, since that commit on master will never appear on develop on Dreamwidth's end. You can't pull or update or merge back since you have this extra commit. (Well, you could try to submit a pull request to merge this commit to Dreamwidth's develop, but we would reject it.)

So! How to fix? The reverse of what I gave to yvi:

git checkout develop
git fetch dreamwidth
git reset --hard dreamwidth/develop

This says to switch to the develop branch, then fetch all changesets from GitHub for Dreamwidth, then "blow away whatever I have, overwrite it with Dreamwidth's".

Finally, you will need to push this up to GitHub. Since you will be asking GitHub to go backwards, you have to emphatically push:

git push --force origin develop

You will have to do the same for dw-nonfree, but it will be the exact same steps since we use the same branch names.

Let me know how it goes, or if anything goes wary! I'll be around.

[ninetydegrees]

Yeah I saw it wasn't right: the network graph is really useful for that but the stupid merge is what happens when I run dwu. Otherwise, I had the same setup at the beginning that I have now that I ran the commands you gave me. I don't know why it does that. I'm gonna run dwu again.

Edit: ran it again and it keeps happening. I get that frakking merge thing I pasted above.

[mark]

I don't know what "dwu" is ... is this an update script? It sounds like it's broken, then. Can you link me?

[ninetydegrees]

Oh sorry I pasted it above too. Let me find it... Yeah right here: http://dw-dev.dreamwidth.org/124621.html?thread=1024717#cmt1024717.

Edit: it's from http://wiki.dwscoalition.org/wiki/index.php/Dev_Maintenance#dwu_-_Updating_the_repos. I've never had any problems with it before.

[mark]

Thanks.

That script is not switching to the develop branch first, so it's not going to do what the author expects. If you do a "git pull dreamwidth develop:develop" while on master, it will update master with the commits from develop. And since, right now, the master has different commits, it generates the merge commits that are being a pain.

You can only do a git pull if you're on the right branch, so it needs to be preceded with a git checkout, like this:

git checkout develop
git pull --ff dreamwidth develop

That first makes sure you're on the develop branch, then it pulls it down. I added the --ff because we only want to allow fast-forwarded merges -- merge commits are going to cause problems here, so we don't ever want them.

I updated the dwu script on the wiki page, so that in the future people shouldn't run into this.

Presently though, since dwu pushes up to GitHub, you have pushed the merge commits that dwu made and so your GitHub is in a bad state. The best way to recover, IMO, is just to wipe things out and then start fresh.

This is the process I was advocating a bit ago, that will reset everything. Then you should be able to do the dwu and it will work...

git checkout develop
git fetch dreamwidth
git reset --hard dreamwidth/develop
git push --force origin develop

git checkout master
git fetch dreamwidth
git reset --hard dreamwidth/master
git push --force origin master

Do this in both dw-free and dw-nonfree directories. I will watch my email!