API things

[momijizukamori]

I kept meaning to write this up but kaberett's volunteer weekend was a good extra nudge, heh.


what is live
- To generate API keys (and delete them), go here - this can be found via Manage Accounts -> Mobile -> Advanced Options (which is at the bottom of the mobile tab)
- The JSON-formatted spec is here and doesn't require an API key to access - we don't have a more human-friendly version yet, though any client that can parse OpenAPI v3.0 specs can take that - Postman is the one I've tried.
- There are not many routes yet - we wanted to get the backend solid before adding stuff that had potential privacy/security risks if something in the backend was wrong
- To use the API key, you need to add an Authorization header to your HTTP request with the value 'Bearer [api key]'.



what is in progress
- I have a pile of routes mostly-written, in this branch. They probably need some light editing, as we moved from OpenAPI v2.0 to v3.0 after I wrote these, and we also hadn't implemented API keys yet. They also need LOTS AND LOTS of testing, oh god please, I'm so bad at QA.
- I wrote a quick'n'dirty primer to adding moar routes here on the wiki. Happy to answer additional questions on it!



what really needs to be done
- TESTING TESTING TESTING
- image-upload-by-API is near the top of the list of 'most wanted things I haven't written yet'
- I want to add a nice UI for the spec a la redoc but I'm not totally sold on that as a look and could use design input and inspiration
- We really need some intro/'API for beginners' text to go on said doc page, I am probably not the right person to write this because I Know Too Much
- It would be great to have a priv system for API keys (so that you could make a key that can read and not post, or vice versa). There are stubs in the API key files for this, but it needs UI and DB stuff as well as... filling out the stubs.
- 404s under /api should really return a JSON 404 message and not... the entire HTML output of the 404 page.

Comments

Re: And API question/feedback

[momijizukamori]

Oh, excellent points, I'll add that to my to-do list! I really need to sit down and set up reference handling for the spec file parser, so that we can have reusable definitions for stuff like entries, users, icons, etc - at the moment every endpoint spec has to have that information in it, which leads to a lot of inconsistency.

Re: And API question/feedback

[nataraj]

I also miss some basic info, that can be fetched about journal. Like journal title. It would be very handy for testing...

So I can do

if ($client->user("nataraj")->journals->{'nataraj'}->title) eq "Nataraj's cool journal")

and check that WebService::Dreamwidth::Journal objects hash have been properly created by journals method.

Re: And API question/feedback

[nataraj]

Some more notes about consistency:

/users/{username}/icons/ returns full list of "icons" objects, but you can also access single one via /users/{username}/icons/{picid}

/journals/{username}/tags returns full list of "tags" objects, but you can't get certain "tag" object by /journals/{username}/tags/{tagid} (Though I would like to have this API-call while creating tags method for entry)

/users/{username}/journals just lists journal names.

I am not sure what is the REST API tradition, but I would expect that I can guess what I am going to get, by looking at API URL:

Like: /users/{username}/icons/ will return lists of IDs, /users/{username}/icons/1 will return list icon object with ID==1, /users/{username}/icons/* will return full list of icon objects. Or something like that I am not fixed on exact implementation, and do not know what do current tradition of REST APIs prescribes here... Key point is to see what will be returned in each case

Re: And API question/feedback

[nataraj]

What I should get via journals/{username}/accesslists? I am trying to do

curl -H "Authorization: Bearer UJxroElRIqdhgRVYS7y4h6DrFGPP5Wcz" http://www.nataraj.hack.dreamwidth.net/api/v1/journals/nataraj/accesslists

this auth key belongs to nataraj user. It has mutual access and subscription with nataraj2 user. But I still get empty list. How should it work? What should I do to get non-empty list?

Re: And API question/feedback

[momijizukamori]

Ah, the implementation on that one is broken I think, because I started writing it and then realized I needed to figure out the bitmask arithmetic that backs access filters. For the future, you can set up access filters here: https://www.dreamwidth.org/manage/circle/editfilters and that response should return information about them (which you need if you want to set one as an entry filter). 'accesslist' is maybe not the best wording - we call them access filters on the editing page for them, and 'access groups' on the newer entry creation page.

Re: And API question/feedback

[nataraj]

Most of the objects returned by API has all "coordinates" that is needed to locate that object.

Like icon has "username" and "picid", and this is all you need, to fetch this object again.

Meanwhile "tag" object does not have "journalname" attribute, so you do not have all you need to fetch this object again.

Re: And API question/feedback

[nataraj]

would be also nice to have journal_name in the entry object

Re: And API question/feedback

[momijizukamori]

Good point on both of those, will make a note of it!

Re: And API question/feedback

[(Anonymous)]

would be also nice to have journal_name in the enrty object

Re: And API question/feedback

[nataraj]

Pagination:

I have 3 entries in my test journal: "Entry 1", "Entry 2" and "Entry 3"

curl -H "Authorization: Bearer UJxroElRIqdhgRVYS7y4h6DrFGPP5Wcz" http://www.nataraj.hack.dreamwidth.net/api/v1/journals/nataraj/entries

Fetches them in backward order: "Entry 3", "Entry 2", "Entry 1"

curl -H "Authorization: Bearer UJxroElRIqdhgRVYS7y4h6DrFGPP5Wcz" http://www.nataraj.hack.dreamwidth.net/api/v1/journals/nataraj/entries?count=1

Returns "Entry 3"

curl -H "Authorization: Bearer UJxroElRIqdhgRVYS7y4h6DrFGPP5Wcz" http://www.nataraj.hack.dreamwidth.net/api/v1/journals/nataraj/entries?count=1\&offset=0

Also returns "Entry 3"

But if I change offset to 1

curl -H "Authorization: Bearer UJxroElRIqdhgRVYS7y4h6DrFGPP5Wcz" http://www.nataraj.hack.dreamwidth.net/api/v1/journals/nataraj/entries?count=1\&offset=1

I am getting "Entry 1" and I guess, this is totally wrong

Re: And API question/feedback

[nataraj]

And speaking about pagination, I guess this offset story is not the best...

If I am fetching some long community, and meanwhile somebody added a new entry, I will get double entry. And if somebody delete entry from the top, I will miss one entry

I guess that all paginated search results should also have some metainfo telling how to fetch next page of the same query...

For DW it may be "next_id": you get the first page, it says "next_id=xxx" in the result data, you query same query with "&start_from=xxx" and getting the next page. Changing any record (except one with id xxx) will not change search result sanity. If looks like more stable...

I also have seen systems that has "search tocken", each search request have been cached, and you can paginate through it using tocken, you've got from the first request. But this was heavy commercial systems, do not think it is DW's case.

Re: And API question/feedback

[momijizukamori]

Yeah, this is one of the things that I know other people have come up with solutions for, but I need to sit down and research them some.

And you're right, that return order behavior does seem like a bug.