Twitter / Dreamwidth Interaction

[random]

So, like many people, I have both a Twitter and DW account, and I've been trying to figure out how to integrate them intelligently, since I find the automated Twitter daily posts incredibly annoying in other people's journal.

So, one thing that I was playing with was seeing if DW's styling code would allow me to embed the Twitter embedded feeds And the answer seems to be clearly that no, you can't embed it - I tried both putting it in Transmogrified as custom text, and putting it in a custom S1 style as a hard-code print command right after the sidebar was drawn, and both had the scripts stripped. That seems to be the case whether you use the Flash object or the JavaScript HTML embed version. (As an aside, since people load other people's pages in their own styles sometimes, I think it's the right call for the JS - I'm a little more surprised about the Flash, but I assume that it's to prevent embedded advertising by adver-bots.)

But I'm wondering if the S2 modules could give an option of actually creating a Twitter module, that would simply take as a username as an option (presumably in Text / Options in the "Customize Styles"), and then display the badge controlling the code that's used, so there's no risk of problems with XSS or adver-bots? Would there be interest from the development team in this? If so, is there anyone who is interested in doing this, since I've got way more experience in PHP and Java development than Perl? If there's interest in the feature but no one wants to jump on it, would someone who has been working on the DW codebase be willing to give me some pointers and do a code review for me?

Comments

[random]

My thought was to display your own tweets in the sidebar of your page, so that people visiting your page or reading your posts would see them instantly. (There's an example of what I'm thinking about in the right-hand sidebar of my friend Chris' blog, which uses the Wordpress plugin that exists for it.)

It would be interesting, though in the context of the intermediate role that LJ/DW have played between Facebook-style social media and traditional blogs, if it could also display your friends' tweets on your reading page, and I hadn't really thought about it. However, the difficulty there is that (like LJ/DW) quite a few people lock their tweets, and so you want to simultaneously be able to show all tweets to the logged-in user whosee reading page it is, but only show public tweets to other users. Twitter uses oauth, so (especially if you're already looking at oauth for cross-posting) this probably isn't impossible, though.

On the "I've opened my big mouth" principle, I may have to put together a preliminary design spec for this, figuring out what technologies are needed to string together to make it work in various use cases.

[random]

And while I'm thinking through use-cases, to give complete Twitter integration equivalent to the Wordpress plugin and other similar blog tools, the final option that it would probably want to have in a perfect world is to also have an option in the cross-poster to post to Twitter; the way most blogging software handles this is to provide something like " New on Dreamwidth: {AS MUCH OF THE SUBJECT AS FITS} http://bit.ly/shorturl ".

While I wouldn't use this personally, I know lots of people who do, which probably means that if I'm trying to help think through this feature set, I should at least see what's involved in it. Which means I also need to look at the cross-poster code, since I have no idea how modular it is. I think I need to stop thinking now.

[matgb]

I use Twitterfeed to send my posts to Twitter, nice and easy. I find it more reliable than Twitter Tools, which is the WP plugin, although I'm told that's stopped not working in a recent update.

[random]

OK, if there's already a good working tool to do it from DW, then I'd say document it somewhere, and no reason to re-invent the wheel.

[random]

Continuing to reply to myself, so that I don't forget stuff and I have it all in one place.

Some things in this are made fairly easy - Net::Twitter has been around for quite a while in CPAN, and Net::Twitter::OAuth is under fairly active development right now to add oauth authentication to the Net::Twitter package, so there's a bunch of tools that make this not a matter of re-inventing the wheel.

However, while until about a year ago it was possible to query the Twitter API using the form http://twitter.com/status/friends_timeline/username.xml to get a given username's unprotected friends entries (exactly what we're looking for), they disabled that feature to reduce server load during the period when Twitter was having a lot of server outages. And while one of the last posts in this thread is about a year ago, with one of the developers saying they were looking for how to bring it back, it's still not possible to do that without authentication, which makes the whole project impossible. In theory, I suppose, we could have it show whichever tweets could be read by any logged-in DW user who had logged in to oauth THEMSELVES and had an oauth token stored attached to their profile - the Twitter REST API does allow that - but there's no code for doing that in Net::Twitter, and it's kludgey at best anyways.

(Also, if we were going to use oauth to show protected entries on the logged-in user's reading page, it leaves the somewhat tricky question of how/when they get redirected to the oauth login page. Presumably the first time they go to their reading page after enabling the module, but that still feels unelegant. So all things consider, I'm not at all sure the reading page part of this is at all viable right now, which is a pity, because I really liked the idea.)

On the upside, using the Twitter API to just show the user's own tweets on their own pages is fairly simple, since user_timeline still doesn't require any authentication at all, dodging that whole can of worms. To do that, all we'd need is to store the Twitter username in a preferences table somewhere (I haven't looked at DW's database schema yet), use the user_timeline method to get the most recent tweets, and then generate the properly formatted sidebar block.

Another thing we'd probably need to take into account is that Twitter has IP-based rate limiting for queries. By default, it's limited to 100 queries per hour - but they have a process where you can ask for whitelisting up to 20,000 queries per hour. Even with that kind of a limit, though, as Dreamwidth grows, we're going to need to be aware of it, and in any case we want to reduce unnecessary load on both DW and Twitter. Some kind of caching (maybe store for 5 or 10 minutes, maybe even a bit longer) is probably essential, and it's probably good long-term practice (even if it adds a little load) to make sure that whenever we go to refresh the cached feed, we use the rate_limit_status call first to make sure that we still have queries that can be used, and extend the cache until the hour limit ends if we're out of queries, rather than querying for the results, triggering the limit, and getting an error back (the time at which the number of queries allowed resets is in the response to the rate_limit_status call).

And now I'm going to walk away from my computer. :)