DW authentication

[ari_linn]

Hello,

Sorry if my question is offtopic in this community, but it doesn't seem like your dev question thread is very active, so I'll take my chances and ask here. I'm trying to programmatically download DW posts from my journal and have troubles accessing friends-only and private posts. Suppose I have the following request:

GET https://ari-linn.dreamwidth.org/545896.html?format=light&expand_all=1 HTTP/1.1
Host: ari-linn.dreamwidth.org

It results in an error page: "You need to be logged in to see this content."

I thought about logging in programmatically to DW and grabbing cookies to pass with the request, but my attempts are getting me nowhere. I can successfully get challenge and compute response from this:

POST http://www.dreamwidth.org/interface/flat HTTP/1.1
Host: www.dreamwidth.org

mode=getchallenge&ver=1

But I can't login using the challenge-response pair. The following request is invariably giving me an error page: "There was an error processing your request: This account name was not found." Why it can't recognize the user=ari_linn parameter is honestly beyond me.

POST https://www.dreamwidth.org/login?ret=1 HTTP/1.1
Host: www.dreamwidth.org

chal=c0:some:very:complicated:challenge:here&response=md5(challenge_plus_md5(password))&user=ari_linn

Is there any way I can programmatically login to DW to access private posts?

Comments

[mark]

Your best bet is to take an existing client and modify it to do what you want. The other thread you commented on had a link to jbackup.pl client, I believe, which is a good starting point for a simple client that gets all comments/posts.

The /login page is designed for browsers, not programmatic access. You shouldn't be accessing it from code. You should use the XML-RPC method sessiongenerate to create an ljsession cookie which you then pass in through your requests to the comment export page (which is designed for bots to access).

But yeah, I'd definitely recommend taking an existing client and making it work. This protocol is over a decade old and doesn't conform to any modern expectations of what a protocol should be. In other words it's still pretty janky and the easiest path to success is probably just copying someone else's code.

Code link: https://github.com/dreamwidth/dw-free/blob/develop/src/jbackup/jbackup.pl