API wishlist?

[momijizukamori]

So my workplace gives us a chance twice a year to work on anything we want for a few days, work-related or otherwise, and I thought I'd use some of my time to actually get the new API into a state where we can make it live. With that in mind! What endpoints would you like to see implemented? We're making this Swagger/OpenAPI-compliant, so everything takes JSON as arguments and return JSON formatted data. Stuff already on the list to do or partially done:

/api/v1/users/{username} - GET, shows info about the user (specific info TBD; subset of profile, probably?)
/api/v1/users/{username}/icons - GET, lists icons
/api/v1/users/{username}/icons/{iconid} - GET, returns icon data
/api/v1/users/{username}/journals - GET, returns list of journals with write access.
-- for now, returns only user's primary journal
/api/v1/journals/{username}/accesslists - GET, list of access lists for journal
/api/v1/journals/{username}/tags - GET, list of tags for journal
/api/v1/journals/{username}/xpostaccounts - GET list of xpost accounts
/api/v1/moods - GET list of moods
/api/v1/commentsettings - GET list of allowed comment settings
/api/v1/journals/{username}/entries - POST new entry, GET list of recent entries
/api/v1/journals/{username}/entries/{id} - POST edit entry, GET specific entry
/api/v1/journals/{username}/files - POST new file
/api/v1/spec - GET, returns a description of the API

(Mark and D, as usual, get the final say - some stuff people want may end up being security/privacy risks in non-obvious ways)

Comments

[pauamma]

Most obvious (to me) things missing are lack of anything for logging in or out, commenting, or retrieving a user's reading page entries.

[momijizukamori]

Ah, yeah, I forgot to put login/logout there because it's likely to be a slightly different beast - what I've got written already works with regular auth cookies, but obviously that's not the best option for an API :)

[siderea]

*starts hyperventilating*

As it happens now's kinda bad for me – this is interrupting my procrastination of other things. What's your timeline for comments being useful? Can you give us a deadline for getting back to you?

[momijizukamori]

My big push is going to be end of this week/this weekend, but I doubt I'll get EVERYTHING done so - whenever you've got time?

[ewx]

This is excellent news.

My suggestion is: 'list of posts and comments that have been created, modified or deleted since <time>', where <time> is something that can be used to reliably avoid gaps; it might be a sequence number rather than an actual time. Could be multiple queries for different kinds of object, with the caveat that nobody wants to poll every entry in a journal to see if there are changes to comments.

Obviously what I'm interested in here is incremental journal backups l-)

[momijizukamori]

Hm, I'll look into it! This may end up being better-handled in other ways, though - I know we have at least partial code for 'back up my whole journal' already done somewhere, and I'm guessing it may use DB flags like the importer does, which aren't exposed outside of the server backend.

[brooksmoses]

FWIW, I'd want that for a "show me which of my friends' posts have new comments" functionality. Which goes with what siderea said above about enabling functionality that DW doesn't have.

[kaisa]

I once (years ago) considered writing a mobile app as a learning project for school to give me notifications of new posts, but realized that the api really should support a couple of functions first.

- Number of new entries on reading page since time x
- Number of new entries on subscription filter y since time x
- List of subscription filters for my account

Those would allow displaying the number of new posts on mobile's front screen easily and setting up which number should be displayed on the mobile app.

Using push notifications for that would be extra nice, but I know nothing about them, being a noob. :)

[momijizukamori]

I will add those to the list! I'm not sure push notifs would be possible under this framework, because that requires the server to make outbound contact all the time, instead of waiting on requests, and I could see that getting expensive quickly, but the others are definitely possible.

[ironymaiden]

Get inbox updates.
Track.

AKA as a user I'd like to get real push notifications for tracked items and be able to view new comments.

[siderea]

I'd like to see access to the inbox, too. It would be great to be able to clients that backup and manage the inbox.

Also, I'm concerned that if the API manages to cause clients to happen, and people start DWing from the phones or desktop apps or whatever, and those don't support the inbox, it will become even more impossible to direct-message people, because those users will never see anything about it, unless they have turned on email notification (which apparently is not on by default?)

I don't just want the API to support the inbox, I kinda want inbox support to be mandatory for DW posting/reading clients. Because I'm afraid that otherwise clients will turn the inbox into dead.letter.

[kaisa]

I completely forgot about inbox, because I don't use it. But yes, I think it would be good to have a call for number of new items in my inbox, because then that could also be set as a notification on a mobile device.

[momijizukamori]

Those options are definitely possible! Though as I mentioned in another comment, I think push notifs may be too computationally expensive - that's a call for Mark to make

[green_knight]

My wishlist item is 'documentation', especially if you're changing major aspects of how the API works. (I looked into implementing the reading list/access list split for an open-source client a couple of years ago, and had to bow out because there wasn't enough API documentation support. Now that I'm a better developer, I'd like to get back to this - OpenLJ works great with DW otherwise.)

[momijizukamori]

Oh, for sure! Part of the nice thing about OpenAPI + the way I've started implementing it is that it's partially self-documenting - the files that generate the docs are the same ones that generate the actual code. A human still has to add the descriptive text that the server doesn't need, but they can't actually get totally unsync'd, and will always at least have paths/methods/what it takes/what it gives back. The current XML-RPC API and the docs that go with it, uh, leave a lot to be desired.

[metahacker]

This is a good starting point--a very data-driven API. Thanks for doing this! I also get some days like that, and should figure out if this could count.

A second level down might also be nice:
given (auth, user, tag), get a list of entries
given (auth, user, accesslist), get a list of entries.
given (auth, user, entry), get previous or next entries.
Auth here is the identity to query using, since the results of those matter based on who you are.

Some more queries I find myself doing manually that might be nice to have a simpler way to do:

1. Translating between date and entries, in both directions. (It's one-to-many, of course.) Even cooler would be "get entries between START and END", but that's fancier.
2. Something regarding polls. Probably the ability to map entry to poll ID, then querying by poll ID to get poll results/status. (I put a poll in almost every entry, but I am...not typical.)

[andrewducker]

/api/v1/journals/{username}/subscriptions - GET, list of subscription lists for journal (and POST to update them).

[momijizukamori]

Definitely doable!

[mdlbear]

(here via siderea)

I may have missed it, but I don't see anything here for retrieving, posting, or editing comments.

Can't wait to get my hands on this. Also, thanks for pointing me at Swagger/OpenAPI -- that's really useful.

[momijizukamori]

You didn't want! The list wasn't intended to be complete, just 'what's happening first' - about half of it already has working code in dev.

[gatheringrivers]

Not sure this was covered, and my geek-speek is kinda impaired lately with crap sleep.

My suggestion:
Force the API to recognize security levels (filters) on posts. As in, if there is no implementation of filter levels, the only posts the script/program can see are public ones.

Reason:
There was some hullabaloo a LONG time ago about one of the various third party LJ apps (I don't remember which one, it was at least a decade ago that I heard about this) being able to *completely* bypass post filters/security levels, allowing anyone using that particular app to see *any* post on a journal, regardless of whether the user was included on a filter for that post or not.

[green_knight]

Yikes. (That must have been before my time, so over a decade ago, but that sounds like a spectacular failure.)

[casimirian]

Woah, so many comments.

This is great news, since some code I used in the past no longer seems to work for dw-free.

Thank you for taking this initiative. I wish you the best of luck!

Where will you make the code available?

[momijizukamori]

So the code will be part of the core DW repositories - If you mean more documentation stuff, you can see a sample of what gets generated here: http://www.momiji.hack.dreamwidth.net/api/v1/spec - which can be fed into something to generate a nice webpage view of it all pretty easily

[annathecrow]

Ohhh, that looks good. I do have some wishlist points:

some specific params for GET /api/v1/journals/{username}/entries:
- number of entries to fetch
- offset (so it's possible to get entries in batches)
- from & to timestamps (to get entries from a specific timeframe)
- fetch posts by tag

/api/v1/comments/{post_id} - GET comments on a post, with these params:
- number of comments to fetch
- offset
- from & to timestamps


Generally, I'm hoping for more "frontend" type of endpoints that would make the public data available for automatization/processing - i.e. backups, import/export. The old LJ API is geared toward content-generating apps, I'd love something more akin to Tumblr's API.

[siderea]

/api/v1/comments/{post_id} - GET comments on a post, with these params:
- number of comments to fetch
- offset
- from & to timestamps

Oooh, also a param for fetching just unscreened comments.

/api/v1/journals/{username}/entries/{id}

[darael]

I see that you've decided to use POST for entry-editing (presumably by way of wholesale replacement). Have you considered allowing PATCH, which is intended for the purpose? I realise you'd have to either figure out how to apply application/json-patch+json to some kind of JSON representation of an entry, or break consistency and use text/x-patch as input, but it might be worth it for small edits on large entries.

Re: /api/v1/journals/{username}/entries/{id}

[momijizukamori]

Yeah, the idea with POST was basically to mirror the way on-site edit functionality works (with the bonus of being able to co-opt a bunch of the existing pathways for doing that). I hadn't considered PATCH, though since writing this initial spec we've swapped to OpenAPI3.0 which is a lot more flexible about content-type.