Twitter / Dreamwidth Interaction

[random]

So, like many people, I have both a Twitter and DW account, and I've been trying to figure out how to integrate them intelligently, since I find the automated Twitter daily posts incredibly annoying in other people's journal.

So, one thing that I was playing with was seeing if DW's styling code would allow me to embed the Twitter embedded feeds And the answer seems to be clearly that no, you can't embed it - I tried both putting it in Transmogrified as custom text, and putting it in a custom S1 style as a hard-code print command right after the sidebar was drawn, and both had the scripts stripped. That seems to be the case whether you use the Flash object or the JavaScript HTML embed version. (As an aside, since people load other people's pages in their own styles sometimes, I think it's the right call for the JS - I'm a little more surprised about the Flash, but I assume that it's to prevent embedded advertising by adver-bots.)

But I'm wondering if the S2 modules could give an option of actually creating a Twitter module, that would simply take as a username as an option (presumably in Text / Options in the "Customize Styles"), and then display the badge controlling the code that's used, so there's no risk of problems with XSS or adver-bots? Would there be interest from the development team in this? If so, is there anyone who is interested in doing this, since I've got way more experience in PHP and Java development than Perl? If there's interest in the feature but no one wants to jump on it, would someone who has been working on the DW codebase be willing to give me some pointers and do a code review for me?

Comments

[liv]

I would dearly love to be able to put a Twitter module in S2! A friend wrote a hack for me to do it on LJ, which involves (I think) using php to dynamically generate a custom CSS file with Twitter updates in an "before" element. But that's clearly not the right way to do it, it just gets round the restrictions that LJ and DW have on including any kind of scripting in journal layouts.

[unixronin]

Honestly, what I'd really prefer (and implemented in my style on LJ) is an option to simply never ever see Twitter posts, PERIOD. Call me a curmudgeon if you will, but if I wanted to read people's usually-almost-or-completely-context-free "tweets", I'd go join Twitter. I don't quite understand why the Twitter crowd don't get this.

[random]

Well, my goal in this is kind of to make Twitter as unobtrusive as possible, while still easy to follow if you want to - and to put the tweets in slightly more context. But it also has the side effect of making it really easy to avoid seeing anyone's Twitter posts then - read their comment pages in your own style, which is easy to set, and they won't show up at all. And even if they're there, they're not in with the LJ/DW entries.

[matgb]

Simple. Add a line of CSS to your Custom CSS:
.loudtwitter { display:none!important}

Job done. I've stopped displaying my tweets, but at some point I'll set up a custom filter for them as some people like them to be there.

Some like it a lot, some dislike it a lot, and people like to do what they like with their blog. If you don't like their blog, then not reading it is easy.

[unixronin]

That does part of the job, but leaves the page cluttered with empty posts. I did that to start with, but ended up modified my LJ style to detect posts containing twitter content and skip displaying them altogether.

Possibly I find it more annoying than others do. Or maybe I'm just a perfectionist about things. But then, frankly I don't "get" Twitter in the first place.

Help a fellow curmudgeon?

[the_lighthouse]

I know you posted this like two years ago, but I was searching google for some way to block twitter feeds from showing up in DW and it brought me to this thread. You mentioned that you modified your style to detect posts containing twitter content and not display them, I was wondering if you could share with me how you go about doing that? I can't stand Twitter feeds! If I wanted to see them, I'd subscribe to Twitter. Any help would be appreciated.

Re: Help a fellow curmudgeon?

[cesy]

Matgb has a post on how to do this for people using LoudTwitter.

Edit: Ah, Matgb explained that further up, sorry.

Re: Help a fellow curmudgeon?

[unixronin]

Well, it's specific to the style, and I did it on LiveJournal not on Dreamwidth. But the key snippet of code was this:

function FriendsPage::print_body ()
{
    foreach var Entry e ($.entries) {
      if (($e.text->contains("Automatically shipped by <a href=") and $e.text->contains(">LoudTwitter</a>"))
           or $e.text->contains(">Daily Tweets</a>")) {
            # skip tweets
        } else {
           .....

Re: Help a fellow curmudgeon?

[the_lighthouse]

Thanks so much, at least it gives me something to try/starting point!

[mark]

Short message, in car--

Yes an S2 module sounds appropriate for this sort of thing, and I have nothing against the idea. Would have to see exactly what this module does. Shows you your friends tweets I assume?

[random]

My thought was to display your own tweets in the sidebar of your page, so that people visiting your page or reading your posts would see them instantly. (There's an example of what I'm thinking about in the right-hand sidebar of my friend Chris' blog, which uses the Wordpress plugin that exists for it.)

It would be interesting, though in the context of the intermediate role that LJ/DW have played between Facebook-style social media and traditional blogs, if it could also display your friends' tweets on your reading page, and I hadn't really thought about it. However, the difficulty there is that (like LJ/DW) quite a few people lock their tweets, and so you want to simultaneously be able to show all tweets to the logged-in user whosee reading page it is, but only show public tweets to other users. Twitter uses oauth, so (especially if you're already looking at oauth for cross-posting) this probably isn't impossible, though.

On the "I've opened my big mouth" principle, I may have to put together a preliminary design spec for this, figuring out what technologies are needed to string together to make it work in various use cases.

[random]

And while I'm thinking through use-cases, to give complete Twitter integration equivalent to the Wordpress plugin and other similar blog tools, the final option that it would probably want to have in a perfect world is to also have an option in the cross-poster to post to Twitter; the way most blogging software handles this is to provide something like " New on Dreamwidth: {AS MUCH OF THE SUBJECT AS FITS} http://bit.ly/shorturl ".

While I wouldn't use this personally, I know lots of people who do, which probably means that if I'm trying to help think through this feature set, I should at least see what's involved in it. Which means I also need to look at the cross-poster code, since I have no idea how modular it is. I think I need to stop thinking now.

[matgb]

I use Twitterfeed to send my posts to Twitter, nice and easy. I find it more reliable than Twitter Tools, which is the WP plugin, although I'm told that's stopped not working in a recent update.

[random]

OK, if there's already a good working tool to do it from DW, then I'd say document it somewhere, and no reason to re-invent the wheel.

[random]

Continuing to reply to myself, so that I don't forget stuff and I have it all in one place.

Some things in this are made fairly easy - Net::Twitter has been around for quite a while in CPAN, and Net::Twitter::OAuth is under fairly active development right now to add oauth authentication to the Net::Twitter package, so there's a bunch of tools that make this not a matter of re-inventing the wheel.

However, while until about a year ago it was possible to query the Twitter API using the form http://twitter.com/status/friends_timeline/username.xml to get a given username's unprotected friends entries (exactly what we're looking for), they disabled that feature to reduce server load during the period when Twitter was having a lot of server outages. And while one of the last posts in this thread is about a year ago, with one of the developers saying they were looking for how to bring it back, it's still not possible to do that without authentication, which makes the whole project impossible. In theory, I suppose, we could have it show whichever tweets could be read by any logged-in DW user who had logged in to oauth THEMSELVES and had an oauth token stored attached to their profile - the Twitter REST API does allow that - but there's no code for doing that in Net::Twitter, and it's kludgey at best anyways.

(Also, if we were going to use oauth to show protected entries on the logged-in user's reading page, it leaves the somewhat tricky question of how/when they get redirected to the oauth login page. Presumably the first time they go to their reading page after enabling the module, but that still feels unelegant. So all things consider, I'm not at all sure the reading page part of this is at all viable right now, which is a pity, because I really liked the idea.)

On the upside, using the Twitter API to just show the user's own tweets on their own pages is fairly simple, since user_timeline still doesn't require any authentication at all, dodging that whole can of worms. To do that, all we'd need is to store the Twitter username in a preferences table somewhere (I haven't looked at DW's database schema yet), use the user_timeline method to get the most recent tweets, and then generate the properly formatted sidebar block.

Another thing we'd probably need to take into account is that Twitter has IP-based rate limiting for queries. By default, it's limited to 100 queries per hour - but they have a process where you can ask for whitelisting up to 20,000 queries per hour. Even with that kind of a limit, though, as Dreamwidth grows, we're going to need to be aware of it, and in any case we want to reduce unnecessary load on both DW and Twitter. Some kind of caching (maybe store for 5 or 10 minutes, maybe even a bit longer) is probably essential, and it's probably good long-term practice (even if it adds a little load) to make sure that whenever we go to refresh the cached feed, we use the rate_limit_status call first to make sure that we still have queries that can be used, and extend the cache until the hour limit ends if we're out of queries, rather than querying for the results, triggering the limit, and getting an error back (the time at which the number of queries allowed resets is in the response to the rate_limit_status call).

And now I'm going to walk away from my computer. :)

[mundens]

Assuming what you mean by this is something that could sit in the sidebar and display ur latest twats, so u don't have to post them as entries in the journal?

If so, I'd agree that would be a great idea! Other blogs (wordpress, 4x) already have such widgets available.

Not sure I can help, I'm a web-app developer for a big corp, so we don't use hacky things like Perl *grin*, but I'm pretty good on the underlying tech, CSS, XML etc.

[random]

Yeah, that's exactly what I'm thinking. If nothing else, I'll post a lightweight XP-ish use-case design spec in this community if I end up putting one together, and that's all about the underlying tech, rather than language-specific implementation - and I'd love to have people look at the first spec, so that problems can be caught before lots of code is written.

[msilverstar]

unobtrusive twitter would be great. If you're fetching the tweets anyway, be nice to have a dynamic expand when the cursor pauses over the link...

[ree]

It sounds like you're looking for a more thorough integration than TwitStamp provides, but I figured I'd throw it out there anyway. The nice thing about TwitStamp is that it works pretty much anywhere, including DW.

[matgb]

The ideal way of getting this isn't to grab stuff just from Twitter. It's to have a way of including RSS feeds (with options) as sidebar modules.

I'd like to have my most recent Tweets and my most recent posts on other blogs displaying (in seperate modules)—if we can figure out a way of having an S2 layer that parses feeds well that'd be really cool and would provide you with your solution as well as being more flexible.

But yeah, no chance of getting a flash or JS embed to work here, and I think that's a policy that should stay.

[unixronin]

The ideal way of getting this isn't to grab stuff just from Twitter. It's to have a way of including RSS feeds (with options) as sidebar modules.

+1. That would be a very nice ability to have. Some kind of an RSS-tracker object that could even appear multiple times in the sidebar, tracking multiple feeds.

[random]

OK, I really like this too, for a bunch of reasons.

But for it to work with Twitter properly, though, there'd have to be at least a few configurable options, which could make the feature rather complicated to use. The big problem is that in general, an RSS sidebar module should display both the title and the description, and possibly the post time, along with a link to the full story.

While Twitter does make RSS feeds available, so that people can follow individual Twitter accounts in an RSS reader, they'd look really wacky if you did them that way - there's an example of a Twitter RSS feed here. The title and the description are the same (format username: Tweet), and the link goes to the status update page, which just shows the same Tweet again. Drawing a Tweet the same way you'd want to draw a normal RSS feed would make for a really ugly Twitter update.

I think you'd probably also want for any RSS module to have options for the maximum description length to display, and for the number of entries to show, and I'm not sure how that would fit into the UI, although I suspect someone could come up with something clever.

That being said, it might be easier (and easier to maintain) to subclass an RSS module for a Twitter module, rather than developing one using Net::Twitter. All you'd have to do is change the display code to only show the description code, a set number of them, and link at the top to the user's main Twitter page to make a subclass work.

Including RSS feeds as sidebar modules

[ursamajor]

I would love this implementation, but wonder about potential copyright violations if one decides to embed content that isn't theirs. One way I could see to alleviate this might be to only allow feeds you've already "claimed" as yours in your profile - like how you can already include your twitter and delicious and other such social networking info in specific fields there. Similarly, for nonSN websites such as a personaltop level domain, you could include it as a module as long as you'd already claimed it in your profile.

Re: Including RSS feeds as sidebar modules

[matgb]

There's no greater copyright issue than someone embedding a youtube or copying an article. RSS feeds are there to promote content and get people to visit, if someone uses a feed they don't own, they're promoting that site.

The BBC actively encourages people to embed their feeds.

I'd have a problem claiming/proving all my sites are mine, I'd put my author feed up for the blogs I write for, they're owned by others. I'd probably even want to combine some of them using something like Pipes, and there's no way that could be claimed in any way; some of my sites would need to be piped or fed through Feedburner first anyway.

[miss_s_b]

Yeah, I'd still like to see this, four years on.

[natf]

Oooo let me know if this gets done!

Hello from the future.

[gushi]

I came across this on a search while trying to get back in to writing long-form, and was pretty prolific back in the LJ days.

My goal is pretty specific: Many of my friends use twitter. I'd like to, upon posting a new DW entry, update a twitter with a link to the post. The subject, and perhaps some key as to the length. If the post is so short as to fit into a tweet, then just print it as a tweet as well, but I want the DW blog to be the "primary source".

Effectively, twitter would be what RSS was back when people respected open standards.

Yes, this might only work for "public" posts. Maybe friends posts where I can post the subject only.

Now, as I'm also working on my own perl-based client (from the command line) I can just wrap that functionality there, and do it just-for-me. But I'd like to know what others have done for this.