dw_dev | Entries tagged with github

Hi all!

As of today, I've made a few fairly broad changes to the way our repositories are set up. You will need to likely make some changes to your development workflow. But! They should be simple.

First, the changes:

The dw-free repository has been renamed to dreamwidth -- because,
The dw-nonfree repository is no more, it has been merged into the dw-free under the path ext/dw-nonfree/.
Finally, the master branch has been renamed to main.

These two changes mean that no more do we have to track and coordinate changes across two repositories, they can be done atomically in one place. Since we don't really support or test running Dreamwidth on its own (without the branding) this will also allow us to simplify development by ultimately collapsing the code, too.

To update your checkout of Dreamwidth, you will need to do two separate things. First, go to Github and to your existing dw-free repository, go to the settings and:

Rename the repository to dreamwidth
Go to branches and rename the master branch to main

Once that's done, you need to update your local checkout like this:

 cd $LJHOME

 # WARNING: Only do this if you have no code here that you care about!
 rm -rf ext/dw-nonfree

 # Update the repository root (for your "origin" remote, **set your username here**)
 git remote set-url origin git@github.com:YOUR_GITHUB_USERNAME/dreamwidth.git

 # Update the repository root (for your "upstream" remote, please change as needed)
 git remote set-url upstream git@github.com:dreamwidth/dreamwidth.git

 # Fetch only to get the new main branch
 git fetch

 # Switch over to it
 git checkout main

 # Get rid of local master
 git branch -d master

 # Redirect HEAD
 git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main

That's it; you should be good to go now. I'll work on updating the Dreamhack scripts shortly, but wanted to get these instructions out. Please let me know if you have any questions/comments/issues.

pauamma

(aka, the thing that lets us do changelog posts and a few other things, like - I believe - the autoclaiming thing)

https://developer.github.com/changes/2018-04-25-github-services-deprecation/

What should replace it for us? Or should there be a bug for it?

h/t: the warning at the bottom of recent changelog entries.

swaldman

Nice flowchart here.

It's not new, but I haven't seen it here before :-)

Forgot to say -- if you forget to claim a bug formally, but you submit a pull request that refers to that bug using the "#123" format in a commit, then that will also automatically assign the bug to you.

kaberett

I've synthesised discussions about and practical use of GHI into a single wiki page. Please shout if you want anything added or changed!

I've also gone through and replaced as many references to Bugzilla with references to GHI as (1) makes sense and (2) I am immediately able. There's about 20 pages still on my hit-list, not all of which I have the knowledge to deal with - if you feel like pitching in, please take one and let me know when it's done (or, if you don't have wiki access, let me know what the appropriate edits would be).

( List of pages outstanding )

We discovered that you have to be a committer on a repository in order to claim a bug for yourself on Github. If you're not one, you have to ask someone to assign the bug to you.

This didn't really seem feasible, so we've fixed it. You can now claim an issue by leaving a comment with the words "claim", "claimed", or "claiming".

Case doesn't matter -- you can use capital letters if you want. You can also have other words be part of the comment, so you don't need to memorize a specific format. "I'm claiming this" will work just as well as "Claimed!".

Claiming will only work if the current issue is not yet claimed -- this will avoid the problem of accidentally grabbing the bug from someone else during a long discussion about claiming something else in a different context.

You *do* need to be part of the Dreamwidth contributors team on Github before assigning issues will work so we've gone and added everyone who's been active in the past year (if you got an email from Github welcoming you the Dreamwidth contributors team that's what that's about!). If you're a new contributor, you'll be automatically added when you give your CLA to

denise.

For existing contributors, do double-check if you're on any of the Dreamwidth teams. If it shows you a list of teams you're on, you're good! If we missed you somehow and tells you that you're not on any teams, let us know and we'll add you ASAP.

We have two main branches in the Dreamwidth repositories: master and develop.

master was intended to be used but... never really got any love.

So here's what I propose: remove all the instructions in our documentation which mention master, and just have 'develop`.

Since develop is where all work goes, nothing in the dev workflow will officially change. But occasionally we have accidents where master gets merged into developinstead of the other way around, or someone finds themselves on master when they should be on develop instead -- and removing any mention of master from our instructions should prevent that from ever happening again.

As for what will happen to master... we never actually used it, and at this point ingrained habits for develop + all setups old and new assume develop, so I suggest we continue not using master the way we currently do.

kareila

Hi all! During Saturday's dev chat, one of the discussions that came up was the current status of our work on improving the process of using Github Issues as our bug tracker.

Here is where we are right now:

It is now possible for anyone to use inline text tags to categorize new issues.

fu has put up a rough draft of the instructions for this process, along with a list of the labels that are currently recognized. We are also planning to allow issues to be "claimed" in a similar manner, although the format for doing that hasn't been finalized yet.

Another part of Github Issues that we've been experimenting with is the use of milestones. If you look at the milestones on dw-free you can see, for example, the status of open bugs on certain major projects like Foundation and mobile styles. I think this could be really useful, especially in light of recent comments saying that we haven't been doing a good job of communicating the progress of our major development projects within the development community.

There are also other milestones currently displayed on that page such as "unclaimed", "curated", and "in progress" - a lively discussion ensued as to whether these would be better used as tags instead of milestones, and I'd like to continue that discussion below. The main issue with milestones on Github in general is that a bug cannot be assigned to more than one milestone. From a workflow perspective, we would expect to see a progression of untriaged -> unassigned -> claimed/in progress -> pull request -> closed, and the fact that an issue cannot exist in more than one of those states at a time makes it a good candidate for the use of milestones - but could conflict with using milestones for the purpose of tracking large projects.

There was also a great deal of confusion as to what "curated" was supposed to mean in this context.

mark said what he desired was a list of three or so "top priority" unclaimed bugs, so that he could easily decide what to spend his limited time on. Again, this may be a better candidate for a tag than a milestone, and maybe with a label that is easier to interpret.

The last issue mentioned is that we still haven't come up with tags for indicating which area of the code is involved with a bug (e.g. styles, notifications, etc.), which is something we relied on pretty heavily with our previous Bugzilla setup in order to find new things to work on.

Please continue the discussion in comments!

denise

1) People who do not have write access to the repo can't change labels or milestones, so ignore all the bits about changing labels and milestones in yesterday's entry. Somebody with commit access will have to add the labels and change the milestones when an issue is submitted and when somebody decides to start working on an issue.

2) We did discover that it's possible to assign issues to people once we add them to a read-only team membership of the repo. I've added people who have open pull requests and assigned the issues they're working on to them.

denise

This entry will be stickied in

dw_dev for at least a few weeks; please point people at it if they haven't seen it!

Apologies for the delay on writing up guidance on this, folks; we were wrangling out the last of the details. (Or rather, we were wrangling enough of the details for me to make this post; I'm sure there are multiple remaining details unwrangled, which we'll figure out as we go.)

Starting now, and thanks to Bugzilla's sad demise, we will be starting a trial period of using Github Issues to keep track of our bugs. It's possible that it won't work out for us, but right now the downsides are outweighed by the very strong advantage of having everything in one place and the heavy integration.

We will not be importing the entire Bugzilla database, for two reasons: one, putting together something to port over all the data is effort that would be best spent elsewhere, and two, it's been a long time since we've been through the list to cull things that are no longer features that are no longer needed and bugs that are no longer manifesting. We decided that starting fresh with a blank slate and only entering things as they come up will be a good way of making sure the list is only full of relevant things.

This post will be about two things: how to log bugs and how to find bugs to work on. There will be a future post on how a particular item (issue) will move through the workflow with you; we have a few last little things to clear up there first, but I wanted to get this posted as soon as possible so people could start work again.

Of note:

* Please only use this process for code-related bugs. We're not yet 100% certain what the process for docs bugs will be. For now, if you spot a FAQ that needs to be changed or needs to be written, report it with a comment on this entry in dw-docs.

* DO NOT use this process for security bugs. If you think you've found a security bug and want to report it, email webmaster@dreamwidth.org (private support category) and we'll take it from there.

* Part of this transition involves declaring total bug amnesty. If you had a bug assigned to you in Bugzilla, and decide that you don't want it anymore (or if you've forgotten about it completely), you're off the hook. (Not that you couldn't have been off the hook at any time anyway, but I know how hard it is to admit defeat sometimes.)

* On the other hand, if you had a bug assigned to you in Bugzilla, and you're still working on it or were almost ready to submit a pull request, we still want it! Open a new issue in Github for it and carry on.

* Just because we're not making a concerted effort to migrate every open bug from the old Bugzilla database doesn't mean that we don't want bugs to be re-reported. If you logged a bug in Bugzilla (or remember a bug from Bugzilla), and the bug is still happening and still annoying you, please open an issue for it. I will be trying to go through the last year or so of still-open bugs from Bugzilla to find "bugs that are still bugs" and re-create them, but I can't guarantee how long it will take me and I'd rather the bugs get logged sooner than I can commit for-sure to doing it, especially so that we have a nice collection of stuff for people to pick from if they just want to pick something up and hack.

That having been said:

( Logging bugs )

( Finding bugs to work on )

There you have it! It will probably take a little bit of getting used to (I know it took me a bit to figure it all out) but -- having gone through the process a few times in the course of figuring things out -- it really is very straightforward once you start doing it. The biggest gotcha, I think, is going to be remembering to set the milestone for all the unclaimed bugs. (That's part of the reason why we're considering using labels for that functionality instead of milestones; there are benefits and detriments to both. Fu and I will decide in a week or two once we see how a small scale test works out.)

Please take this chance to log all the bugs you're still working on, and all the bugs you can think of as still affecting you, over the next few days! Once that's been done, I'll start going through the various "upkeep" tasks (new themes, new embed sources, etc) and add those, then start working through the

dw_suggestions posts tagged "bugzilla: migrated" to see which ones should be moved over to GHI.

(

ninetydegrees, I know you have spreadsheets for themes that were in Bugzilla and not yet patched; if you want to add those, that would be awesome, but if you don't have the time/energy, I will get to them when I fill in the various "is: upkeep" things.) (Also, I will explain the "is: upkeep" and how that differs from other "is: foo" tags in a few paragraphs!)

Again, I'm sorry for my delay in writing up the guidance for What We're Doing With This -- things took a little more discussion. Thank you all for being so willing to roll with things and try out new workflows.

If you run into questions as you work, just holler.

( Appendix: What the labels mean: a work in progress )

swaldman

~/dw (develop)$ git status
# On branch develop
# Your branch is ahead of 'upstream/develop' by 540 commits.

Did we ever figure out what causes this? I have a theory, but I don't really understand...

(Theory: the dwu script does "git pull --ff-only upstream develop" straight from the remote repo into the local develop branch. This doesn't update the local copy of "remotes/upstream/develop" that was created by a "git fetch" sometime in the past. This is what the current branch is being compared to. I have no idea whether this makes sense, and if so how to deal with it...)

Current Mood: annoyed

mark

I've created release-1.17 branches in both repos in preparation for tomorrow's code push. If you have anything you really want to land in tomorrow's code push, please let me or fu know and we'll see if we can get it in before then.

This frees the develop branch for continuing development. Anything that lands there will be slated for release in the NEXT code push (which will hopefully be next week, on account of next week being OSCON, and our typical enjoyment of Conference Driven Development).

foxfirefey

So, sometimes Bugzilla can be slow to reference/search, or sometimes you want to break up a bigger bug into a set of tinier bugs without cluttering up Bugzilla or making a bunch of comments to your bug in Bugzilla that you then have to manually scan for information.

Your personal GitHub fork can have its own simple bug tracker! Log in, go to your fork of dw-free or dw-nonfree, and find the settings in the button bar. Go check the box next to issues. Voila! You now have a very simple, fast issue tracker you can use in a way that is most convenient to you. The link to the issues interface will be a button in the toolbar. Here's mine so far:

https://github.com/foxfirefey/dw-free/issues

You can note that I'm putting the Bugzilla bug number in the header of each issue, just so I know what bug each issue belongs to. I have also made myself a milestone for the next code push (tomorrow!), so I can filter down to which issues I want to have finished by then.

You can also check the wiki box to have your own personal wiki notebook for taking your research notes that aren't relevant to anyone else by checking the wiki box on that settings page.

I hope this is useful for you! I'm thinking it will be very useful for me.

There are two ways to leave per-line comments when reviewing a pull request:

opening up each commit in a separate tab and going through them one-by-one (commits are listed in both the Discussion and the Commits tabs)
going to the Files Changed tab, and seeing the entire final result

I used to open up each commit individually, but:

if the person is forced to rebase (either to get rid of extra commits, or to clean up their history), the discussion is lost
under the discussions tab, there's no context, just the comment, which makes it possible to know that there's activity going on, but very difficult to keep track of exact context
it's possible that something you reviewed in one commit was fixed later on in the commit series, invalidating your comment
in the email notification, the "view it on github" link shows you the commit, not the associated pull request (and there's no easy way to find which pull request a commit is on, given just the commit)

I'd been ignoring the Files Changed tab for some reason, but I've now found that they actually don't have any the disadvantages of the per-commit view:

if the person is forced to rebase, old discussion is preserved in the pull request. If the rebase altered the diff, you'll get an "comment made on outdated diff"; if the diff is the same after a rebase, it shows up like it did before
under the discussions tab, it shows the context of the diff you're commenting on, along with your comment, which makes it very easy to keep track of the exact reason a comment was made (especially on short comments hehe)
you see the final version of the code that's being submitted
in the email notification, "view it on github" links to the pull request where all the discussion is taking place

All this is to say that if you're poking around and want to comment on something you see in someone else's pull request (and we do encourage code review in this manner!), use the Files Changed tab. It's much more useful.

Some examples:

actual pull request where you can see how easy it is to tell that a review comment has been handled (and what still needs work, or what doesn't need to be changed after all)
test pull request where I committed a change, commented using both the commit/files changed view, rebased+force pushed the rebase (which blew away the comment on the commit, but retained the comment on files changed), and then made the diff outdated -- which collapsed the discussion but still kept it visible

foxfirefey

I just had to figure out how to make a branch I was developing be based on the "develop" branch instead of the "master" branch. The command to do this ended up being, as far as I can tell:

git rebase --onto develop master Bug3426/editotherstyles

The advice that finally made sense came from this guide and was explained thusly:

git rebase --onto topicC topicA topicB

Where topicC is the newbase, topicA is the oldbase, and topicB is the reference for what HEAD of topicC will become.

Current Mood: accomplished

I had some weird behavior on my computer (running Mac OSX), because we had an image file named Sinisteria.jpeg, and another named sinisteria.jpeg, so I committed a fix.

Depending on your OS, you may need to run an extra step next time you upgrade. You'll need this for Mac OSX for sure, I can't remember how Windows and Linux deal with case in filenames.

Pull in updates as usual. This will delete Sinisteria.jpeg (uppercase)
$ git pull Updating ecc1d71..7cb6ab7 Fast-forward htdocs/img/styles/paperme/Sinisteria.jpeg | Bin 786 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 htdocs/img/styles/paperme/Sinisteria.jpeg

Then check if you accidentally deleted sinisteria.jpeg (lowercase)
$ git status # On branch develop # Changes not staged for commit: # (use "git add/rm ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # deleted: htdocs/img/styles/paperme/sinisteria.jpeg #

If you didn't see the "deleted: ..." bit, you're fine. No need to do anything more.

If you do see the "deleted:..." bit, then do this:
git checkout -- htdocs/img/styles/paperme/sinisteria.jpeg

Now check again to make sure that worked:
$ git status # On branch develop nothing to commit (working directory clean)

Done!

foxfirefey

So, as far as I can tell from my researching over the past few days, the only real way to get a pull request to review it is to:

* Manually add the submitter's dw-* repo as a remote in the repo on your hack (or Github I suppose, but there is no GUI advantage here).
* Manually pull the branch in question

Is there a better way to do this? Am I missing something? This seems really, well, annoying, with manually crafting URLs and whatnot, and not very user friendly. (I'm trying to make some documents on reviewing pull requests for people who are not

fu, since we need to try and spread out that work a little.)

foxfirefey

They've been being hit by DDOS attacks--so if you're trying to do something with our Github and it isn't responding, know that this might be the cause!

foxfirefey

For the folks who like digging into things like this: codeq (warning: Javascript generated blog page, ew)

denise

We still want to buy you books, and since this week O'Reilly is having a sale on resources about Git (our new version control system), we'll even buy you more than one of them if you spot stuff that you think will help :)

Browse the list of eligible titles (or the wider O'Reilly catalog) and if you'd like a copy of the book or the videos, let me know: we'll buy you that plus another title. (Maybe a book on Perl, or Javascript, or jQuery, or UI design, or or or...) I'm trying to figure out a way with O'Reilly that will make it easier for us to pay for ebooks that are then delivered to your O'Reilly account, but in the meantime, we can either do it where you buy it through your O'Reilly account so you can have future access to it and we'll write you a reimbursement check, or I buy it through my O'Reilly account with the DW card and send the file to you. Or you can get the hard copy, of course, if you don't want an ebook version; it's up to you.

Either way: let us buy you stuff! People have been really hesitant to take us up on this, but I promise you, we really want to, and if you meet the criteria (5 patches in last 12 months, at least one in the past two months), we mean you even if you think you haven't done 'enough'.

Profile

Dreamwidth Open Source Development

May 2025

S	M	T	W	T	F	S
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

Syndicate

Page Summary

mark - GitHub repository rename and updates
pauamma - GitHub Services going away
swaldman - "How to fix a mess in git"
fu - Claiming a bug on Github, another way
kaberett - Github Issues: now on the wiki!
fu - Claiming a bug on Github
fu - Retiring master
kareila - Categorization of Issues on Github
denise - addenda to yesterday's entry
denise - Bug tracking moving to Github Issues, effective now
swaldman - Your branch is ahead of 'upstream/develop' by 540 commits.
mark - Release branch 1.17 created
foxfirefey - Your own personal DW bug tracker on your GitHub fork
fu - quick tip when reviewing pull requests
foxfirefey - Git tip: moving where a branch is based
fu - Case conflict: Sinisteria.jpeg
foxfirefey - Best way to review pull requests
foxfirefey - Github might be experiencing intermittant downtime
foxfirefey - Something that might be fun to play with our Git repository with
denise - O'Reilly offering a sale on Git books

Active Entries

1: Question thread #141
2: Question thread #140
3: Question thread #139
4: Question thread #138
5: Code tour for 2023-11-22 to 2024-11-30
6: Question thread #137
7: Question thread #135
8: Current Code Contributors: November 2024
9: Patching Mood Themes
10: Question thread #134

Style Credit

Style: Lettuce for Crossroads by branchandroot
Resources: P r o l o g u e

Expand Cut Tags

No cut tags

Page generated Jun. 7th, 2025 12:22 am

Dreamwidth Development

Entries tagged with github

GitHub repository rename and updates

GitHub Services going away

"How to fix a mess in git"

Claiming a bug on Github, another way

Github Issues: now on the wiki!

Claiming a bug on Github

Retiring master

Categorization of Issues on Github

addenda to yesterday's entry

Bug tracking moving to Github Issues, effective now

Of note:

Your branch is ahead of 'upstream/develop' by 540 commits.

Release branch 1.17 created

Your own personal DW bug tracker on your GitHub fork

quick tip when reviewing pull requests

Git tip: moving where a branch is based

Case conflict: Sinisteria.jpeg

Best way to review pull requests

Github might be experiencing intermittant downtime

Something that might be fun to play with our Git repository with

O'Reilly offering a sale on Git books

Profile

May 2025

Syndicate

Most Popular Tags

Page Summary

Active Entries

Style Credit

Expand Cut Tags