Question thread #118

[pauamma]

It's time for another question thread!

The rules:

- You may ask any dev-related question you have in a comment. (It doesn't even need to be about Dreamwidth, although if it involves a language/library/framework/database Dreamwidth doesn't use, you will probably get answers pointing that out and suggesting a better place to ask.)
- You may also answer any question, using the guidelines given in To Answer, Or Not To Answer and in this comment thread.

Comments

Re: well howdy

[denise]

hmmmmmmm. Yeah, we're way out past the edges of my knowledge here (I'm just working from what we've seen before in the importer).

You can tell if an entry is backdated by editing it: if you're using the old update page, it'll be a checkbox near the date item labeled "date out of order" or "don't show on entry pages" (I forget if we renamed it there or not), and if you're using the new entry page it's in the Display Date panel and labeled "don't show on reading pages".

If it's always getting stuck on comments, I wonder if the script you're using just doesn't have good comment mapping error handling. Have you taken a look at jbackup.pl that Pau linked to above? Is there anything like that in the script you're using? I've never looked at it, so I don't know how well it handles weirdnesses. I know you've probably hit the point of "I've put this much effort into it, I'm going to make this work goddammit," but just to rule out issues with the script itself: if you try running jbackup.pl does it get stuck in the same place?

Re: well howdy

[solarbird]

If it's always getting stuck on comments, I wonder if the script you're using just doesn't have good comment mapping error handling.

That's not it. We're expecting C- items mixed in with L- items; this is the one big pull of all item metadata.

Also, neither side is getting stuck on a single actual item, the comment identifiers keep changing, as per the debug output I provided in another reply. It's just that they have huge swaths of repeated lastsyncs.

I'm kinda thinking this would fine at this point if we didn't have the shutdown for repeated pulls of the same lastsync.

Re: well howdy

[denise]

Have you been able to try jbackup.pl to see if it gets stuck the same way?

Re: well howdy

[solarbird]

I haven't - I got busy finishing another project that went well past anticipated work. I'll give it a go tomorrow (Friday).

Re: well howdy

[denise]

I don't necessarily expect it to work but it will help rule out issues with the script!

Re: well howdy

[solarbird]

Okay. I haven't been eager to dive in because I know literally nothing about perl. But I found the needed dependencies (via Debian, which was easier than trying to do it in MacOS lol) and it ran to completion and after running again specifying XML (vs. default which I guess means raw which gave me some chonky binary file) I got a 60MB file which on casual inspection seems like it's probably complete.

Which tells me mostly I think that the database on your end is fine as far as your own tools are concerned (which isn't really surprising, I mean, of course it is).

Sadly, from what I've been able to find, there are not tools to import Dreamwidth XML exports to Wordpress - I'm not the first person to try this, I'm just the first person who hasn't bailed and copied everything over by hand. (At least, that's what I was finding in searches before I started trying to adapt the Livejournal importer to Dreamwidth.) As far as I've been able to find, few people have tried to make this work and so far none have succeeded.

I wonder if there's something different in Wordpress's PHP implementation of the XMLRPC libraries? But now I'm just in "whelp I dunno" territory and am guessing.

Re: well howdy

[denise]

That ... is really weird and interesting and I was not expecting that result! Okay, yeah, at this point it could be because the PHP XML/RPC libraries may have bitrotted a bit because it's not a very popular format anymore or Wordpress is using a bad version or something. We're way out past the far, far edge of my knowledge too, but my suggestion would be a) check what version of the library Wordpress is using and see if there's a newer one (and that the newer one doesn't break Wordpress, heh), or b) step the module backwards a few versions and keep trying after each regression to see if an older version of it works. Basically, playing around with the versions and seeing if any of them fix the problem. But I don't know PHP or Wordpress, so this is just me guessing too. (We've had a few instances where we have to pin to a particular version of a Perl module for a while because a newer one breaks something, etc.)