Dreamhack reinstallation dilemma

[sophie]

As most people here will know, I'm the maintainer of the Dreamhacks service which allows you to have a Dreamwidth development environment quickly and easily.

Some people will also know that lately the reinstallation script hasn't been working for some time. It's been broken ever since the move to git and GitHub, and I recently put a check in there so that anybody trying to use it won't accidentally end up with an unuseable 'hack. (By the way, if anybody does have an unuseable 'hack because of this, please feel free to let me know and I'll gladly reinstall you by hand!)

Although it should be fairly easy to get the reinstaller script working again, I've been running into some issues that I'm not entirely sure how to address. If you have a Dreamhack account, please read inside the cut as this could quite possibly affect you.

---

The point of the reinstaller script back in the Mercurial days was so that if you end up with a 'hack that you didn't know how to fix, it would be possible to easily get back to a working state by reinstalling your Dreamhack account as if you were a completely new user, without you having to worry about bothering anybody. You'd lose your work in the process unless you had backed it up, but you'd be pretty much guaranteed a working 'hack and being able to get back to somewhere you knew. A lot of people used it and I've heard from a lot of people that it was really helpful.

It worked because the previous setup procedure for Dreamhacks meant getting a completely fresh copy of the code straight from the official repository, so unless any bugs had been committed into the official codebase, it would be guaranteed to work as expected.

However, this isn't the case any more - the setup process for Dreamhacks now involves forking the official repository and setting up the Dreamhack to pull from that repository instead. As a result, if you've already pushed the commits that broke your Dreamhack to your own repository, a reinstall wouldn't help because it'd just pull the broken commits straight back down. It may not even fix a broken repository, depending on the situation; all in all, there's no guarantee any more that the reinstall process would get you to a known state any more. It would all depend on what you had pushed to the repo before you reinstalled.

I have an idea for how this might be fixable, but it would require a fair amount of code on my part and I want to be sure that it's worth it, so I thought I'd come to you guys for ideas!

My own idea involves something I've been planning for a while anyway - the ability to link Dreamhack SSH accounts with GitHub via OAuth. It would take some work, but with this in place it would be possible to have a reinstallation process that could (after making VERY clear what it would do) delete your dw-free/dw-nonfree repositories on GitHub and re-fork them automatically, either as part of the process or (more likely) as a separate script which you could run beforehand. The reinstallation could then proceed using these newly re-forked repositories, and the end result would be almost exactly the same as what you would have had under the old script - code that comes straight from the horse's mouth, as it were, and an almost-sure guarantee that you would get a working 'hack out of it, bar any bugs in the official codebase.

It's probable that something like this would be best implemented as two scripts - the reinstallation script as it is right now (or at least as it would be if it were working) that uses your current GitHub repositories, and a script you could run beforehand if you wanted to to do the deleting/re-forking. I think this would be the best way, personally, because the main reason to have the reinstall process is to make it quick, easy, and not require many spoons. While having two commands to run does take some more spoons, I believe that it's worth it to stop people from accidentally wiping out all their work (since people wouldn't normally expect a reinstallation on the Dreamhack machine to affect GitHub).

However, these are just my ideas. It's quite possible that people here have better ideas, and that's why I'm here. Do you have any ideas about how the reinstallation script should work? Is the process I've outlined too easy to accidentally activate, too difficult to work with, or something else? Things like that. The Dreamhack machine is your resource and I want to make it as easy as I can to hack on the code while not getting in your way. How can I do that in this case?

Comments

[cdybedahl]

I'm not at all familiar with the Dreamhack environments, so I may be way off here. But it feels to me that deleting and re-forking repositories is the wrong solution, if nothing else so because it throws away history. If I understand correctly what you're trying to do, it should be able to do it by playing around with branches.

First, a repo can have more than one remote. When you clone from a remote repo (like, for example, one on Github), git automatically creates an "origin" remote pointing back to where you cloned from. But that's just a convenience, there's nothing magic about it. You can add new ones just fine. With forked repos on Github, it's often useful to add one pointing back a the repo one forked from. In this case, that might be something like...

git remote add ur_source https://github.com/dreamwidth/dw-free.git

You can then fetch from that, base branches off of it, and if I understand the problem correctly that should do fine as the pristine starting point.

I think (but I haven't actually tested) that a reset process might go something like this, assuming ur_source is set already and the messed-up work is done on the local master branch:

1) Fetch from ur_source.
2) Rename the current master branch (git branch -m) to something predictable and auto-generated, just to keep history.
3) Create a new master branch based on remotes/ur_source/master.
4) Set the new master to track the old remotes/origin/master.
5) Push back to Github with --mirror to force it to look like the local repo.
6) Proceed working on the new master branch.

One possible drawback is that any local branches made from the messed-up master would still be based on that, and have the problems intact. I'm not sure if that would be a problem or not. Nor am I sure if all this works at all, I just think it should. It shouldn't take that long to give it a manual try, if you think it looks like it might be useful.

[sophie]

Thanks! You're right, deleting and re-forking will throw away history. Let me explain a little bit about what's happening here, though, to make it a bit clearer as to why I think this is beneficial in this case.

Firstly, a bit of background: A lot of our developers are fairly new to git, and in fact to programming in general. One of the aims of the Dreamhack box is to provide a playground to encourage messing around with the code without fear that anything you do will mess up your Dreamhack beyond repair, because if you're not sure whether you might mess things up, it can really get in the way of experimentation. Yes, people can always ask me for help (and I'm always open to helping people!), but it's a fact of human nature that you generally don't want to bother other people with problems that you consider to have been your own fault, especially if it's something that sounds like it should be major, like reinstalling your account or unbreaking your Dreamhack.

When it worked, the reinstaller script could be triggered by running a single command. Within a minute, the reinstaller would warn you what it was going to do. Within another minute, it logged you off the system, killed all your processes, deleted your user account and everything associated with it, and then set you back up again as if you were a completely new user. In total, it took about 5-10 minutes, and you would be back up and running again, being guaranteed a working Dreamhack without having to talk to a soul. That sort of guarantee is invaluable in a "playground" environment such as this. (I'd call it a sandbox, but it isn't a sandbox in the true technical sense of the word, hence "playground" instead.)

OTOH, while you're almost certain to be able to get back to a working state with git in the way you describe, there are several reasons why that might not be the best method. From a technical perspective, the local git repository may be corrupted in such a fashion that it just doesn't work as expected. From a psychological perspective, because git never throws away commits, you'll still know that your old "baggage" is still in there somewhere, which might throw you off. You would also need to make sure your old branches were deleted, otherwise you might accidentally refer to them in a future command and end up with something wholly unlike what you expected.

Deleting and re-forking on GitHub, then reinstalling the local user account, is by far the easiest way to deal with these problems. (Why not just clone the re-forked repository back to the user account? Because it's not just repository problems that can affect your Dreamhack. Each Dreamhack has their own Apache configuration file which can be edited as need be, and it's possible - though unlikely - that if you were editing the config, or removed the file totally, you could break your Dreamhack that way. While it would be easy enough to fix - and in fact I may just make a script to make it easy to replace your Apache config file if need be - it's another thing that can go wrong.)

All told, it's about three things - a guarantee that it's going to work as you expect it to, a psychological guarantee that you're not going to have anything left that could mess it up, and the knowledge that you don't have to tell anybody if you don't have the spoons for human interaction; you can just reinstall and continue hacking.

Hopefully that helps clear up why I think doing this is the best way in this case!

[liv]

Can I just say, I really appreciate how the spec discussion for this takes into account psychological issues for newbies!

[aedifica]

Yesyesyes! Me too.

[cdybedahl]

Sounds like solid reasons, yes. And thanks for explaining rather than just agreeing that I was way off :-)