Code Tour, January 03-January 21, 2010

[azurelunatic]

In the spirit of teamwork, the rather scary code tour from the end of December to the present day, the one that everyone had been procrastinating on, got split up into a few parts.

There were 21 bugs completed during this slice of January. Lots of backend stuff, little user interface tweaks, with some new themes as well. There has been a code push since the last code tour, so some of these may well be live; keep checking dw_maintenance for the next scheduled one.

Somehow, even though I upgraded from a (nearly) 5-quart pot to a 7.5-quart one, I still had trouble fitting all the curry in the pot. Isn't that just how it goes?


Bug 3354: sanitize GET URLs in directory.bml
Category: Misc Backend
Patch by: kareila
Description: kareila describes: "I'm working on bugs related to the display of directorysearch results and becoming increasingly annoyed with the URLs of the results: 1) they rewrite themselves to include the .bml suffix 2) they include long strings of arguments with null values." Translated: 1) BML is the page-building language that bradfitz wrote to make it easy for him to build webpages, back in 1997, before LiveJournal. In 2011, it is being replaced with more modern and intuitive tools. 2) Say you're searching for something, and you leave some fields blank. You might expect the search to ignore those fields. But the search grabs up those things, and includes them, with whatever thing means "don't know, don't care". This makes the URL of the page you're headed to much, much longer than it actually needed to be.

Bug 3369: All workers need to be executable
Category: Site Administration
Patch by: fu
Description: There are these little programs called "workers" that do things to make the site work, like making sure comment notification e-mails get sent out, making sure birthday reminders get sent, and so forth. Occasionally a worker stops running. This is bad. Making sure that all workers will be running when they need to be is good.

Bug 3357: generate 'warning' email when check/money order order is checked out
Category: General/Unknown
Patch by: fu
Description: So denise and her wife moved. Logistically speaking, this means that users now send checks/money orders/cookies/small children entertaining packages to a post office box. Wouldn't it be nice if, whenever someone checked out an order that was to be paid with a check or money order, the system would send out an e-mail alerting them to go check the mailbox in a couple days? Why yes, yes it would!

Bug 3371: ViewEntryStyle/ViewJournalStyle settings causing errors on /manage/settings when logged out
Category: Misc Backend
Patch by: fu
Description: Did you know that logged-out users (people who do have accounts but aren't logged in, and people who don't have accounts at all) can use http://www.dreamwidth.org/manage/settings/ to set a cookie so they can see site-scheme colors other than red/dilute red/grey/white? This page was getting errors when logged-out users were using it, though. Fixed.

Bug 3375: errors in TT conversion of htdocs/legal/dmca.bml
Category: Frontend, details matter
Patch by: fu
Description: In the long and arduous process of converting the back ends of site pages from BML to the more modern Template Toolkit, some stuff sometimes goes awry. This time, it was the DMCA (like YMCA, but less fun to sing) page, with a bad variable in the email address, an outdated address, and oh, why the hell do we have a fax number: this is 2011!

Bug 3377: Min-width on site-skinned comments causes excessive side-scrolling when font is enlarged
Category: Misc UI/Frontend, side-scrolling is not your friend
Patch by: fu
Description: Consider long comment threads. This one is more of a problem for visual users, unless your screen reader tells you about soft line breaks. If the space set aside for the comment is too narrow, the comment will break up so every word is on its own line, which can become amazingly tedious to read. If the space set aside for the comment is too wide, if your font size is large and your screen is not wide enough, there will be side-scrolling. I think it's safe to say that most people hate side-scrolling. So Fu tweaked the minimum comment width to something that should be readable for most, including people with large font sizes.

Bug 3170: Clean up constant namespaces in DW::Template
Category: Routing/TT
Patch by: fu
Description: You know when you say "Bring me the thing!" expecting whoever's with you to know which thing you meant, and bring it to you, but they bring you the other thing instead? Computers can have the same problem, more or less, which can be helped by making absolutely sure that when someone says "thing", there's only one thing it could be referring to across the whole site.

Bug 2761: Skittlish Dreams: 5 new themes by sarken
Category: Style System
Patch by: sarken
Description: New themes: Academy, Desert Cream, In the Bag, Like Sunshine, Snow Cherries: http://dreamscapes.dreamwidth.org/36389.html

Bug 3372: Improve sorting of themes in /layers
Category: Misc UI/Frontend
Patch by: fu
Description: Unless you're the sort of person who does things with styles a lot, you may not have ever visited http://www.dreamwidth.org/customize/advanced/layers.bml -- but if you do, the things there are now sorted neatly, in the following order: themes with names in alphabetical order; themes with the name "Auto-generated Customizations"; themes without names (below themes with names) in layer id order.

Bug 3380: style=* links on control strip broken when control strip is added using JS
Category: Misc UI/Frontend
Patch by: fu
Description: The optional control strip (that can be put on the top of your journal or shown when viewing someone else's) has those handy links, so you can quickly view someone else's eye-searing style in your own comforting style, or a light style. Some journal styles don't get along well with the control strip; it has that "pasted on yey" feeling, with bonus javascript doing the pasting. Sadly, when the control strip was pastede on yey, the style links were wrong and didn't work as they should.

Bug 3383: Reply pages need to follow entry preferences
Category: Misc UI/Frontend
Patch by: foxfirefey
Description: You know how you can set your display preferences so you can see individual entries in your own style or the light style? It would be helpful if that applied to the reply mode, too, wouldn't it?

Bug 3388: Renaming tags when the new name already exists error message wrong
Category: Misc UI/Frontend
Patch by: foxfirefey
Description: Say you've realized that you've got too many tags for your taste, and you'd like to clean up. You've got the tag 'butter chicken', and you'd like for it to be named 'curry' instead. So you try renaming. Lo, an error! "The tag name 'butter chicken' is already in use", it says, among other things. And then you take a gander at your other tags, and realize that oho, you've already got a 'curry' tag, that was what the poor confused tag thing should have said to you. Time to go merge all your 'butter chicken' entries into the realm of 'curry', then...

Bug 3363: refactor paging_bar logic in talkread.bml
Category: Misc UI/Frontend
Patch by: kareila
Description: Coding in a rapidly changing environment can be an exercise of mono no aware: in this case crafting code with care and beauty, to know that its season is not long, and it will soon be replaced by something else. You can only hope that some of the changes you have made will be of use to the next developer, and that your code will lighten the day of someone using the page while it is live. This particular bit improves the stuff behind comments pages and makes things look a little better.

Bug 3373: $LJ::T_HAS_ALL_CAPS makes all users read only
Category: Misc Backend
Patch by: fu
Description: The capability of any given journal to do any given thing is controlled via a journal property called a "cap". (Inherited from our code-parent. Hey, it was there, it was short, it was hard to misspell. Removing opportunities for developers to misspell makes things that much easier for everyone.) It's easier to test stuff on developers' testing environments by just turning all the user-capabilities on. This is where $LJ::T_HAS_ALL_CAPS comes in. (It has nothing to do with places like capslock_dreamwidth.) However, there's another cap floating around out there: readonly. Three guesses what that does to a journal, and the first two don't count. Unsurprisingly, it's harder to test some things on journals that have suddenly become read-only. This is a quick and dirty fix that nonetheless gets the job done.

Bug 3407: suspended entry message has incorrect link back to journal
Category: Misc UI/Frontend
Patch by: kareila
Description: The Terms of Service team has the capability to suspend single entries, instead of the whole journal, if ever there's an entry that's enough of a problem according to the Terms of Service that it just needs to go away, but the whole journal doesn't need to go. The message on a suspended entry is meant to say: "Suspended: This entry has been suspended. You can visit the journal here." The "here" as the link text aside (and a bug's filed to fix that too), the link was mangled and didn't actually go to the journal. Woops.

Bug 3370: xposter needs to check if entry still exists before trying to xpost it
Category: Crossposter, making things make sense
Patch by: fu
Description: Sometimes you've got an entry, you tell it to crosspost, then you for some reason delete the entry. If the crossposter is slow for any reason, the crossposter may not have crossposted that deleted entry yet. And, bless its little mechanical heart, it will try. Or it would, until it was fixed. Now you should get a message telling you what happened.

Bug 3237: Expose create_url in the TT plugin
Category: Routing/TT
Patch by: exor674
Description: More backend cleanup, of the sort that users may not ever have to know about. This one results in a "dw.create_url" method for Template Toolkit. But while this sort of thing does not directly affect users, developers working on other, user-facing, things, will be affected by these changes, and it will make it that much happier and easier for them to actually go forth and code, without having to untangle the obscure stuff.

Bug 3171: Routing should output text/plain errors for unknown ( and non-HTML ) formats
Category: Routing/TT
Patch by: fu
Description: Every now and then, something causes an error. When the format of the error page is unknown, the error should be plain text, not HTML or anything else fancy.

Bug 3414: Digest authorization popup uses "lj" as the realm
Category: Misc UI/Frontend, we're not LJ
Patch by: fu
Description: It would be a bit jarring to be asked to log in, to "lj", when you're trying to log in to Dreamwidth, wouldn't it? A little more search-and-convert.

Bug 3348: Logging in from the control strip causes warnings in the logs
Category: Misc Backend
Patch by: fu
Description: The error here makes a certain amount of sense: on developer servers, if your browser still has a cookie from the last time you logged in, when you're logging in freshly (and you logged out on the control strip, and you're logging back in from the control strip), there may be a bit of back-end freak-out: WTF, that cookie you have, very not fresh, that batch expired long ago! *fling* But this isn't a very critical sort of problem, so in this case the warnings have been made more user-friendly, so developers won't see them and wonder WTF was going on.

Bug 2322: Update 'checkfriends' protocol mode
Category: APIs: RPC/XML/Flat
Patch by: fu
Description: People who were/are paid users on LJ and use clients may recognize the client feature where the little pencil in your system tray blinks to tell you that someone on your friends page has updated. The thing that made that possible on LJ's end was a little thing called 'checkfriends'. "Friends" is now obsolete for us, so 'checkfriends' has been updated to become 'checkforupdates', which looks for updates on your reading page. This may not be that exciting now, but just wait until someone builds a client that takes advantage of it. When that happens, no more refreshing to see if someone posted!

[Edit: That was fast. There's a Perl client that takes advantage of 'checkforupdates' now. See comments.]

Comments

[cesy]

Yay for all the back-end fixes and code cleanup. I'm looking forward to seeing all the new stuff it enables.

[msilverstar]

This is a fabulous code tour, accurate and amusing at the same time. Kudos!

[azurelunatic]

Thank you!

[cheyinka]

Is this a good place to mention that I've updated jlj.pl to take advantage of checkforupdates? I mean, even somebody who really doesn't like posting with jlj might still like the convenience of
% perl jlj.pl -p dreamwidth -c
:D

[azurelunatic]

Ooo! Oo! Oo!

[cheyinka]

http://www.endersgame.net/~cheyan/code/updatestojlj.html

[azurelunatic]

Awesome!

I've added it to Compatible clients as well.

[cheyinka]

Thanks!

[sophie]

BTW, about the edit (and looking at it, the description too), the new command is 'checkforupdates', not 'checkupdates'. :)

[azurelunatic]

Thanks! Fixed.

[sophie]

I couldn't help but notice this line:

2.12.2 caused it to always send &ver=1 when posting an entry.

If you do that, you probably want to check that the entry text you're sending is valid UTF-8 before sending it. (From what I gather, one thing about jlj is that you basically have to trust that whatever editor the user used saved it in UTF-8.) &ver=1 says to the server that you can and will speak UTF-8 to it, including for the entry text. I'm not quite sure what would happen if you sent high characters that aren't UTF-8.

(Apologies if you already do this!)

no apologies needed, thank you for pointing that out

[cheyinka]

I don't actually do that, because I'm not sure how, honestly, and am also not sure even whether there's a way to let the user specify that &ver=1 shouldn't (or should, I suppose?) be sent besides altering the options available in .livejournal.rc.

I suppose I should either learn how or at least verify that sending that option and also sending non-UTF-8 characters won't break things unfixably!

Re: no apologies needed, thank you for pointing that out

[sophie]

You want to use the Encode module, which is a core part of Perl since 5.8. The best way of doing it would be to use it like this:

open(my $fh, "<:encoding(utf-8)", $myfile);
# read from $fh here
close($fh);

That will read from $myfile as if it's a UTF-8 file. If the input file is not, in actual fact, UTF-8, then it will spew errors along the lines of 'utf8 "\xA1" does not map to Unicode', and your input text will have a stringified "\xA1" (or whatever the character was) in the text where the invalid character appeared. In this way, your scalar will be guaranteed to be valid UTF-8.

There are other things you can do with Encode; I've used it a lot. If you need help on how to do anything with it, let me know - character encodings are something of a specialty for me.

[edit: Fixing bad text.]

Re: no apologies needed, thank you for pointing that out

[cheyinka]

What I think I want to do is make a new option in the .rc where the user can either have those substitutions made without further input, have the entry opened for editing again so the substitutions can be corrected, or have nothing done (and ver=1 not sent).

Re: no apologies needed, thank you for pointing that out

[sophie]

Another option you could do - and probably a better one - is to allow the user to specify their preferred character encoding in the .rc. Then, you run that value through Encode::resolve_alias() to make sure it exists and is supported by Encode, and use the returned value in the open() call instead of "utf-8".

That'll cause Perl to read the file as if in that character encoding and convert it on-the-fly to UTF-8 internally. You'll then have UTF-8 data that you can send to the server. :)

Re: no apologies needed, thank you for pointing that out

[cheyinka]

Hooray! It seems to work. How should I credit you in jlj.pl / on the site?

Re: no apologies needed, thank you for pointing that out

[sophie]

I'm glad it works. :)

Referring to me by my Dreamwidth username is fine. :) (for the site, you probably want to copy the stuff from the text box on the profile)

[silverflight8]

there may be a bit of back-end freak-out: WTF, that cookie you have, very not fresh, that batch expired long ago! *fling*

:D :D Sentences like this make it totally worth it to read code-tours for just the humour.

[denise]

That is one of my very favorite parts about code tours. *g*