Question thread #43

[pauamma]

It's time for another question thread!

The rules:

- You may ask any dev-related question you have in a comment. (It doesn't even need to be about Dreamwidth, although if it involves a language/library/framework/database Dreamwidth doesn't use, you will probably get answers pointing that out and suggesting a better place to ask.)
- You may also answer any question, using the guidelines given in To Answer, Or Not To Answer and in this comment thread.

Comments

XML-RPC API Error

[zetasyanthis]

I'm attempting to use the python-lj library to access DW's XML-RPC API. It works on LJ, and kind of on DW, but I get the following error sometimes. Can someone point me to where I might find details on what would cause that error?

Fault 406: "Client error: Client is making repeated requests. Perhaps it's broken?"

Re: XML-RPC API Error

[denise]

I'd lay money on the answer being a bug somewhere -- we don't have many people using the API (because it's so old and out of date), so things like backwards compatability issues with newer versions of Perl or Perl modules we use don't tend to get caught very quickly -- but try making a top-level dw-dev post and see if anybody has any suggestions?

(We know the API is in a dreadful state; we've been working on a replacement that was actually architected in this century...)

Re: XML-RPC API Error

[zetasyanthis]

Understood. I just wanted to nab something that'd work for now. Building a little archival / posting tool, one module at a time, and this API was on my list since I use the site. :) [Said project is still suuuuper early and not remotely ready for use. XD]

Re: XML-RPC API Error

[denise]

Good luck! If you do find a bug, let us know and we'll try to fix it :)

Re: XML-RPC API Error

[zetasyanthis]

I've take a *loooooong* time responding because I had to shelf this project for a while, but it looks like a bug in pulling comment data. Possibly something inherited from LJ judging by threads from about 2009. When you pull the data for them, you get two bytes rather than an actual useful data-stream.

I'm not sure of the exact API call that's being used, but it's in update_journal_comments in python-lj. The Python code is available here if it helps: https://github.com/zetasyanthis/python-lj

Re: XML-RPC API Error

[zetasyanthis]

And I was hitting the retry issue since I was re-running my script several times trying to figure out what was going on. Looks like it's *really* *really* aggressive on rate-limiting. You only get ~2 pulls an hour.

Re: XML-RPC API Error

[azurelunatic]

That suggests it's hitting some duplicate entry prevention thing, like something with the same exact content being posted within some timeout window that I'm a little vague on.


*dives*

dw-free/cgi-bin/LJ/Protocol.pm

https://github.com/dreamwidth/dw-free/blob/346829d666d389c9e12eae7d1fa45134e0d42875/cgi-bin/LJ/Protocol.pm#L2320

        my $now = time();
        # broken client loop prevention
        if ($req->{'lastsync'}) {
            my $pname = "rl_syncitems_getevents_loop";
            # format is:  time/date/time/date/time/date/... so split
            # it into a hash, then delete pairs that are older than an hour
            my %reqs = split( m!/!, $u->prop( $pname ) );
            foreach (grep { $_ < $now - 60*60 } keys %reqs) { delete $reqs{$_}; }
            my $count = grep { $_ eq $date } values %reqs;
            $reqs{$now} = $date;
            if ($count >= 2) {
                # 2 prior, plus this one = 3 repeated requests for same synctime.
                # their client is busted.  (doesn't understand syncitems semantics)
                return fail($err,406);
            }

I hope this is helpful? I don't understand it fully, but that's where it's defined.

Re: XML-RPC API Error

[zetasyanthis]

Oh, that's *very* helpful! Now I can track down what would cause that...