Banyan Speak (Permitting cross-posting of threads) - GSOC

[trob]

As someone who came to DW from the LJ community (and who's still more active over there than over here), one of the biggest barriers I find to participation is that while DW provides very good cross-posting, there's no way to cross-post comment threads. So in order to avoid discussions fragmenting across the two services it's currently necessary to tell people from one service to go to the other to comment. While this isn't always a barrier to participation (with OpenID implemented), it does provide an extra step, and the "friction" involved in that interaction seems to decrease the amount of participation.

Since my primary use of either service is focused around social interaction, the lost comments from people just not feeling up to logging into another site (even if they don't have to register an account or anything) mean lost chances for interaction, and that's a shame.

Which brings me to my project.

Basically I propose to construct a framework for synchronizing comment threads across multiple instances of DW (with the hope of making the code changes light/modular enough to let them be applied back to the LJ codebase with little or no modification). I've got a working implementation of this in WordPress at the moment, but there would need to be some changes for it to work for DW.

Initially I'll be using Disqus for synchronization (using their account system to handle the 'canonical' central version of a thread, and then synchronizing to and from that for all accounts/posts/services that need it).

What I'm aiming for is something that's almost UI-invisible. There will be a couple of user-facing changes.

• In account settings there will be a section for users to enable the system and save their Disqus credentials. This will be a simple username and password update which will save a token. (Sort of like OAuth, but not exactly. It's a Disqus-specific credential system, but the users don't have to deal with it.)


• When writing a new post the user will be given a new option in the meta box (probably in-column with the "comments"/"comment screening" drop-downs) which allows them to either create a new thread for the post (the default) or pick an existing thread (from a previous post that they've made. Existing threads can be from any service that the user has connected to Disqus, so it allows bridging across multiple journals, and potentially multiple instances of DW (or in the future, possibly LJ).

The rest of the code changes will be on the back-end.

• There will need to be a comparison between the way DW stores comments in the database and the way Disqus does. Since they're both threaded systems they should be relatively similar, so I anticipate little needed in the way of modification here. Possibly an extra column or two in the DW system.


• Helper functions to make calls to the Disqus API will be needed.


• A function which automatically synchronizes comments/threads between the two services whenever comments are viewed on DW, and one that posts them to both services whenever new comments are posted.

In the long-run, there may be some structural changes as well. I'm a bit uncomfortable using a third-party system such as Disqus to handle synchronization. They're a good bunch of guys, but they're outside the control of anyone working on a DW deployment. I'd like to begin looking at alternative systems such as building a synchronization API into the DW codebase directly, or implementing something like the Salmon Protocol.

To sum things up: the project I want to work on adds to the existing cross-posting systems by allowing the cross-posting of threads as well as posts. This means that no matter where a reader is seeing a post, they see the same conversation attached to it.

Comments

[cesy]

This sounds like a really cool idea in principle, though I'm slightly uncomfortable about the Disqus side of it. How do they handle security (e.g. if the post is locked differently on both sides), freezing threads, screening comments, etc.? Or would this only be an option for public posts?

[trob]

That's actually one of my big concerns, too. Not because Disqus doesn't handle this stuff well, but because it creates a potential point of conflict.

So, basically, you can set up a Disqus account to display no threads on Disqus itself. This, in turn, allows you to do the permissions thing entirely on the client/display side. However, this does lead to potentially weird conflicts where (say) a post is public on LJ and private on DW, or your list of permitted viewers isn't synchronized between the two services and someone who can't see it on one can see it on the other.

Essentially, this would attach the thread to a post, and the thread would only be visible to people who could see that post, but because you're actually posting across multiple services it's possible that people will be able to see the post in one place when they shouldn't unless you're scrupulous about keeping your permissions tightly policed.

So, your worry is pretty much spot-on. Because this system doesn't implement any security on the synchronization, if you're screening in one place and not in the other then the screening will only take place in one of the locations. (Actually, screening in general is probably going to be hard to handle... I'll have to think on that one.)

Additionally, this is a pure back-end code, so there's no real guarantee that anyone will adopt it. And while I may live in hope, I actually think it would be hard to convince LJ to push it into their system. After all, they benefit pretty significantly from the fact that some people won't jump services because it would mean losing easy access to certain conversations.

[zvi]

Having the comments controlled by Disqus, especially invisibly, makes me extremely uncomfortable.

Also, would this affect the DW comment front-end? I've heard complaints about Disqus' accessibility, and we don't want to introduce their problems into our system.

[trob]

The idea is never to touch the Disqus front-end and to use it purely as a synchronization service. Which is to say, no, there shouldn't be any impact on the DW front-end at all.

Which doesn't serve to solve the concerns with letting someone else's back-end touch your threads (no matter who's used as the controlling, 'canonical' version of those threads). I'd definitely be open to suggestions, and there is a reason I'm looking at Salmon, but at the moment I can't seem to come up with something that would be broadly workable that doesn't use some third-party service...

[denise]

I am really uncomfortable about bringing in a third party to do something (commenting) that we already do well and that people are used to us doing. I am *particularly* not okay with Disqus, which has usability and accessibility nightmares and doesn't degrade gracefully.

And I see that you comment elsewhere saying this, but there is less than zero chance of getting LJ to accept this. Why remove accessibility/functionality from DW for something that won't even have the desired end result?

[trob]

Fair concern, and fair question.

In order. I'm curious as to whether your concerns with Disqus are offset at all by it never being user-facing? At the moment I'm looking at it as a convenient and relatively robust back-end. Because the accessibility stuff is a big problem. That doesn't really solve all the concerns with using them, but I want something that does what DW doesn't (currently) do is handle synchronization of threads, which is something that Disqus handles explicitly.

As for the second question, there's always the faint glimmer of hope, but that's not the main reason. The main reason is that there end up being a number of useful (I think, anyway) uses purely within a single DW instance. For instance, if you cross-post something to multiple communities, you can share threads between all of them. This isn't, of course, always desirable behavior, but sometimes it would be. Additionally, it will allow for this sort of connection across potential future instances of DW (the software).

I don't know if that's a good response, but that's what's in my head, at least.

[foxfirefey]

I'm a bit uncomfortable integrating a proprietary third-party system like Disqus into the actual Dreamwidth codebase itself, but I think it might be possible to make a third-party script that does this based off of exporting comments (and maybe some new XML-RPC functions to make them), and that would sidestep a lot of my more pressing questions. (Such as, with Automattic buying up Intense Debate, where does that leave Disqus in the realm of making enough money to pay back investor funding?)

I feel like I should note I don't feel the same way about all proprietary third-party systems--I just want to know that Disqus has a solid future first and won't do awkward things as it flails to survive.

[trob]

Yeah, I share those same concerns. In fact, they consume a lot of my brainspace thinking about the project in the long term.

And, yeah, the coupling between Disqus (or whatever gets used in the long run) and DW would need to be as loos as possible. You don't want core site functionality to be dependent upon some other service which might or might not survive, or (maybe worse?) might or might not change its policies in ways which make using it undesirable.

I think I've got a decent set or architecture to handle that. I should work up a swim-lane diagram or something...