ari_linn: (warrior - normal)
Ари Линн ([personal profile] ari_linn) wrote in [site community profile] dw_dev2016-12-24 02:51 pm
Entry tags:

DW authentication


Sorry if my question is offtopic in this community, but it doesn't seem like your dev question thread is very active, so I'll take my chances and ask here. I'm trying to programmatically download DW posts from my journal and have troubles accessing friends-only and private posts. Suppose I have the following request:


It results in an error page: "You need to be logged in to see this content."

I thought about logging in programmatically to DW and grabbing cookies to pass with the request, but my attempts are getting me nowhere. I can successfully get challenge and compute response from this:



But I can't login using the challenge-response pair. The following request is invariably giving me an error page: "There was an error processing your request: This account name was not found." Why it can't recognize the user=ari_linn parameter is honestly beyond me.



Is there any way I can programmatically login to DW to access private posts?
mark: A photo of Mark kneeling on top of the Taal Volcano in the Philippines. It was a long hike. (Default)

[staff profile] mark 2016-12-24 08:13 pm (UTC)(link)
Your best bet is to take an existing client and modify it to do what you want. The other thread you commented on had a link to client, I believe, which is a good starting point for a simple client that gets all comments/posts.

The /login page is designed for browsers, not programmatic access. You shouldn't be accessing it from code. You should use the XML-RPC method sessiongenerate to create an ljsession cookie which you then pass in through your requests to the comment export page (which is designed for bots to access).

But yeah, I'd definitely recommend taking an existing client and making it work. This protocol is over a decade old and doesn't conform to any modern expectations of what a protocol should be. In other words it's still pretty janky and the easiest path to success is probably just copying someone else's code.

Code link:
mark: A photo of Mark kneeling on top of the Taal Volcano in the Philippines. It was a long hike. (Default)

[staff profile] mark 2016-12-28 11:29 pm (UTC)(link)
So I think the issue is that you imported your journal a long time ago (looks like 4 years ago, originally) and I think there was some sort of error at the time. ~3,000 comments got imported without an entry attachment so they're floating (jitemid=0).

I'd suggest that we clear out the imported content. This means I would take an admin action to purge the imported content (only imported content!) and then you re-initiate the import. This will get fresh copies of the data which should work. (These days the importer is pretty solid.)

What do you think?
Edited 2016-12-28 23:30 (UTC)
mark: A photo of Mark kneeling on top of the Taal Volcano in the Philippines. It was a long hike. (Default)

[staff profile] mark 2017-01-01 10:46 pm (UTC)(link)
For better or worse there's quite a bit more that goes into it. At any rate, I've taken care of it -- the "0" comments are gone now.

Also, we've now got pretty effective ability to detect and delete imported duplicates. If you want me to do so, I can. (Too be clear, this is just the lower numbered ones without comments.)