API wishlist?

[momijizukamori]

So my workplace gives us a chance twice a year to work on anything we want for a few days, work-related or otherwise, and I thought I'd use some of my time to actually get the new API into a state where we can make it live. With that in mind! What endpoints would you like to see implemented? We're making this Swagger/OpenAPI-compliant, so everything takes JSON as arguments and return JSON formatted data. Stuff already on the list to do or partially done:

/api/v1/users/{username} - GET, shows info about the user (specific info TBD; subset of profile, probably?)
/api/v1/users/{username}/icons - GET, lists icons
/api/v1/users/{username}/icons/{iconid} - GET, returns icon data
/api/v1/users/{username}/journals - GET, returns list of journals with write access.
-- for now, returns only user's primary journal
/api/v1/journals/{username}/accesslists - GET, list of access lists for journal
/api/v1/journals/{username}/tags - GET, list of tags for journal
/api/v1/journals/{username}/xpostaccounts - GET list of xpost accounts
/api/v1/moods - GET list of moods
/api/v1/commentsettings - GET list of allowed comment settings
/api/v1/journals/{username}/entries - POST new entry, GET list of recent entries
/api/v1/journals/{username}/entries/{id} - POST edit entry, GET specific entry
/api/v1/journals/{username}/files - POST new file
/api/v1/spec - GET, returns a description of the API

(Mark and D, as usual, get the final say - some stuff people want may end up being security/privacy risks in non-obvious ways)

Comments

[pauamma]

Most obvious (to me) things missing are lack of anything for logging in or out, commenting, or retrieving a user's reading page entries.

[siderea]

*starts hyperventilating*

As it happens now's kinda bad for me – this is interrupting my procrastination of other things. What's your timeline for comments being useful? Can you give us a deadline for getting back to you?

[ewx]

This is excellent news.

My suggestion is: 'list of posts and comments that have been created, modified or deleted since <time>', where <time> is something that can be used to reliably avoid gaps; it might be a sequence number rather than an actual time. Could be multiple queries for different kinds of object, with the caveat that nobody wants to poll every entry in a journal to see if there are changes to comments.

Obviously what I'm interested in here is incremental journal backups l-)

[kaisa]

I once (years ago) considered writing a mobile app as a learning project for school to give me notifications of new posts, but realized that the api really should support a couple of functions first.

- Number of new entries on reading page since time x
- Number of new entries on subscription filter y since time x
- List of subscription filters for my account

Those would allow displaying the number of new posts on mobile's front screen easily and setting up which number should be displayed on the mobile app.

Using push notifications for that would be extra nice, but I know nothing about them, being a noob. :)

[momijizukamori]

My big push is going to be end of this week/this weekend, but I doubt I'll get EVERYTHING done so - whenever you've got time?

[momijizukamori]

Ah, yeah, I forgot to put login/logout there because it's likely to be a slightly different beast - what I've got written already works with regular auth cookies, but obviously that's not the best option for an API :)

[momijizukamori]

I will add those to the list! I'm not sure push notifs would be possible under this framework, because that requires the server to make outbound contact all the time, instead of waiting on requests, and I could see that getting expensive quickly, but the others are definitely possible.

[momijizukamori]

Hm, I'll look into it! This may end up being better-handled in other ways, though - I know we have at least partial code for 'back up my whole journal' already done somewhere, and I'm guessing it may use DB flags like the importer does, which aren't exposed outside of the server backend.

[ironymaiden]

Get inbox updates.
Track.

AKA as a user I'd like to get real push notifications for tracked items and be able to view new comments.

Access to the accesslist membership lists?

[siderea]

Okay, I'm back.

You have:

/api/v1/journals/{username}/accesslists - GET, list of access lists for journal

How about allowing manipulation of accesslists by API? In parallel with your other URLs, we could have:

/api/v1/journals/{username}/accesslist/{accesslistid} - GET, list of members of access list; POST, save list of members of access list.

But honestly, I would feel better, as a developer, being able to add or subtract individual users from an accesslist, rather than having to overwrite the whole thing every time I made an edit. The latter seems like a recipe for really awful bugs.

Use cases: GET: being able to retreive a list of members means I can back up my access lists, something that is important to me, and actually super hard. I've also kicked around an idea of combining such a feature with OpenID, such that I could run a website where if users authenticate there with their DW OpenID, they can have access to things based on my access lists – basically API access to my accesslist memberships lets me use DW's ACLs elsewhere. So I could, for instance, set up sound or PDF or video sharing just for people on my access lists. POST: remote/client management of membership.

Need to think through how this will work with comms

[siderea]

We should probably think through how all this will work with communities, to make sure nothing is incompatible with them, later on when the API is extended to them.

I mean everything looks fine to me at the moment, but it's 2:21am local time.

[siderea]

/api/v1/journals/{username}/entries/defaults - GET, returns the key-value pairs of all the default settings for a journal's entries (i.e. security, commenting, screening, ?).

Use case: so posting clients can correctly apprise the user of what the default behavior will be when they hit the Post button.

[siderea]

I'd kind of like to be able to do the sorts of navigational things via the API one can do via the standard web interface: return a batch of entries all of the same:
• tag
• security level (including custom levels)
• year
• month
• day
in chronological or reverse chronological order.

(If it would be possible to intersections of any of {tag, security, date}, that would be grand.)

Usecase: that would be a step in the direction of being able to have an entry-management tool, that allows bulk changes to batches of entries, like "set all entries older than X to No Comments" and "set all entries tagged 'my sex life' to Private".

[siderea]

/api/v1/journals/{username}/files - POST new file

And can we have GET list of files?

(A way to back up files!!)

[siderea]

I'd like to see access to the inbox, too. It would be great to be able to clients that backup and manage the inbox.

Also, I'm concerned that if the API manages to cause clients to happen, and people start DWing from the phones or desktop apps or whatever, and those don't support the inbox, it will become even more impossible to direct-message people, because those users will never see anything about it, unless they have turned on email notification (which apparently is not on by default?)

I don't just want the API to support the inbox, I kinda want inbox support to be mandatory for DW posting/reading clients. Because I'm afraid that otherwise clients will turn the inbox into dead.letter.

[siderea]

I want to acknowledge I'm dumping a lot out here, and want to apologize if I'm being overwhelming. I was taking that "wishlist" thing at its word.

As a general principle, I'm personally far more interested in the power of third-party apps (mobile, desktop, or even server) to provide functionality to DW that it doesn't have, rather than to provide alternative platforms for access that do what the website (or a subset of the website) already does. I think there's a tendency for APIs for web-based services to be designed to focus on the core functionality of the web user interface; but by opening up the non-core functionality, an API allows third parties to solve problems for the service and provide novel features.

Backup utilities is a classic example, and what with the Great Russian Migration from LJ earlier this year, I think it's on a lot of people's minds. There was no reasonable way to back up one's images from LJ, because the API didn't cover it.

Similarly, providing API access to images allows hypothetical client developers to do things like make phone clients that interoperate with cameras, to make it super easy to post photos to DW. That would probably be a popular feature – it could even make DW more competitive as a platform – but not so certain a thing that DW would want to invest in developing such a client. An API allows third parties to take a stab at it, and DW to benefit by it if they do.

[kaisa]

I completely forgot about inbox, because I don't use it. But yes, I think it would be good to have a call for number of new items in my inbox, because then that could also be set as a notification on a mobile device.

[green_knight]

My wishlist item is 'documentation', especially if you're changing major aspects of how the API works. (I looked into implementing the reading list/access list split for an open-source client a couple of years ago, and had to bow out because there wasn't enough API documentation support. Now that I'm a better developer, I'd like to get back to this - OpenLJ works great with DW otherwise.)

[brooksmoses]

FWIW, I'd want that for a "show me which of my friends' posts have new comments" functionality. Which goes with what siderea said above about enabling functionality that DW doesn't have.

[momijizukamori]

Oh, for sure! Part of the nice thing about OpenAPI + the way I've started implementing it is that it's partially self-documenting - the files that generate the docs are the same ones that generate the actual code. A human still has to add the descriptive text that the server doesn't need, but they can't actually get totally unsync'd, and will always at least have paths/methods/what it takes/what it gives back. The current XML-RPC API and the docs that go with it, uh, leave a lot to be desired.

[metahacker]

This is a good starting point--a very data-driven API. Thanks for doing this! I also get some days like that, and should figure out if this could count.

A second level down might also be nice:
given (auth, user, tag), get a list of entries
given (auth, user, accesslist), get a list of entries.
given (auth, user, entry), get previous or next entries.
Auth here is the identity to query using, since the results of those matter based on who you are.

Some more queries I find myself doing manually that might be nice to have a simpler way to do:

1. Translating between date and entries, in both directions. (It's one-to-many, of course.) Even cooler would be "get entries between START and END", but that's fancier.
2. Something regarding polls. Probably the ability to map entry to poll ID, then querying by poll ID to get poll results/status. (I put a poll in almost every entry, but I am...not typical.)

[andrewducker]

/api/v1/journals/{username}/subscriptions - GET, list of subscription lists for journal (and POST to update them).

[mdlbear]

(here via siderea)

I may have missed it, but I don't see anything here for retrieving, posting, or editing comments.

Can't wait to get my hands on this. Also, thanks for pointing me at Swagger/OpenAPI -- that's really useful.

[gatheringrivers]

Not sure this was covered, and my geek-speek is kinda impaired lately with crap sleep.

My suggestion:
Force the API to recognize security levels (filters) on posts. As in, if there is no implementation of filter levels, the only posts the script/program can see are public ones.

Reason:
There was some hullabaloo a LONG time ago about one of the various third party LJ apps (I don't remember which one, it was at least a decade ago that I heard about this) being able to *completely* bypass post filters/security levels, allowing anyone using that particular app to see *any* post on a journal, regardless of whether the user was included on a filter for that post or not.

[green_knight]

Most excellent! If you want someone to run an editorial eye over the documentation afterwards, please ping me; I have some free time.