From 8e75c5ed67e8acce5448a8f37858f011d167d56e Mon Sep 17 00:00:00 2001 From: Kegsay Date: Thu, 19 Feb 2015 14:17:31 +0000 Subject: [PATCH] Add in passive application services --- specification/40_application_service_api.rst | 444 ++++++++++++++++++- 1 file changed, 441 insertions(+), 3 deletions(-) diff --git a/specification/40_application_service_api.rst b/specification/40_application_service_api.rst index 687f477e..fdaf5e7c 100644 --- a/specification/40_application_service_api.rst +++ b/specification/40_application_service_api.rst @@ -18,10 +18,448 @@ Client-Server Services Passive Application Services ---------------------------- +"Passive" application services can only observe events from a given home server. +They cannot prevent events from being sent, nor can they modify the event being +sent. -.. TODO-spec - API that extends the client-server API to allow events to be - received with better-than-client permissions. +In order to observe events from a home server, the application service needs +to register itself and tell the home server what kinds of events it wants +to receive. + +Application Service -> Home Server API +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This contains home server APIs which are used by the application service. + +Registration API +++++++++++++++++ + +This API registers the application service with its host homeserver to offer its +services. + +Inputs: + - Credentials (e.g. some kind of string token) + - Namespace[users] + - Namespace[room aliases] + - URL base to receive inbound comms +Output: + - The credentials the HS will use to query the AS with in return. (e.g. some + kind of string token) +Side effects: + - The HS will start delivering events to the URL base specified if this 200s. +API called when: + - The application service wants to register with a brand new home server. +Notes: + - An application service can state whether they should be the only ones who + can manage a specified namespace. This is referred to as an "exclusive" + namespace. An exclusive namespace prevents humans and other application + services from creating/deleting entities in that namespace. Typically, + exclusive namespaces are used when the rooms represent real rooms on + another service (e.g. IRC). Non-exclusive namespaces are used when the + application service is merely augmenting the room itself (e.g. providing + logging or searching facilities). + - Namespaces are represented by POSIX extended regular expressions in JSON. + They look like:: + + users: [ + { + "exclusive": true, + "regex": "@irc\.freenode\.net/.*" + } + ] + +:: + + POST /register + + Request format + { + url: "https://my.application.service.com/matrix/", + as_token: "some_AS_token", + namespaces: { + users: [ + { + "exclusive": true, + "regex": "@irc\.freenode\.net/.*" + } + ], + aliases: [ + { + "exclusive": true, + "regex": "#irc\.freenode\.net/.*" + } + ], + rooms: [ + { + "exclusive": true, + "regex": "!irc\.freenode\.net/.*" + } + ] + } + } + + + Returns: + 200 : Registration accepted. + 400 : Namespaces do not conform to regex + 401 : Credentials need to be supplied. + 403 : AS credentials rejected. + + + 200 OK response format + + { + hs_token: "string" + } + +Unregister API +++++++++++++++ +This API unregisters a previously registered AS from the home server. + +Inputs: + - AS token +Output: + - None. +Side effects: + - The HS will stop delivering events to the URL base specified for this AS if + this 200s. +API called when: + - The application service wants to stop receiving all events from the HS. + +:: + + POST /unregister + + Request format + { + as_token: "string" + } + + +Home Server -> Application Service API +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This contains application service APIs which are used by the home server. All +application services MUST implement these APIs. + +User Query +++++++++++ + +This API is called by the HS to query the existence of a user on the Application +Service's namespace. + +Inputs: + - User ID + - HS Credentials +Output: + - Whether the user exists. +Side effects: + - User is created on the HS by the AS via CS APIs during the processing of this request. +API called when: + - HS receives an event for an unknown user ID in the AS's namespace, e.g. an + invite event to a room. +Notes: + - When the AS receives this request, if the user exists, it must create the user via + the CS API. + - It can also set arbitrary information about the user (e.g. display name, join rooms, etc) + using the CS API. + - When this setup is complete, the AS should respond to the HS request. This means the AS + blocks the HS until the user is created. + - This is deemed more flexible than alternative methods (e.g. returning a JSON blob with the + user's display name and get the HS to provision the user). +Retry notes: + - The home server cannot respond to the client's request until the response to + this API is obtained from the AS. + - Recommended that home servers try a few times then time out, returning a + 408 Request Timeout to the client. + +:: + + GET /users/$user_id?access_token=$hs_token + + Returns: + 200 : User is recognised. + 404 : User not found. + 401 : Credentials need to be supplied. + 403 : HS credentials rejected. + + + 200 OK response format + + {} + +Room Alias Query +++++++++++++++++ +This API is called by the HS to query the existence of a room alias on the +Application Service's namespace. + +Inputs: + - Room alias + - HS Credentials +Output: + - Whether the room exists. +Side effects: + - Room is created on the HS by the AS via CS APIs during the processing of + this request. +API called when: + - HS receives an event to join a room alias in the AS's namespace. +Notes: + - When the AS receives this request, if the room exists, it must create the room via + the CS API. + - It can also set arbitrary information about the room (e.g. name, topic, etc) + using the CS API. + - It can send messages as other users in order to populate scrollback. + - When this setup is complete, the AS should respond to the HS request. This means the AS + blocks the HS until the room is created and configured. + - This is deemed more flexible than alternative methods (e.g. returning an initial sync + style JSON blob and get the HS to provision the room). It also means that the AS knows + the room ID -> alias mapping. +Retry notes: + - The home server cannot respond to the client's request until the response to + this API is obtained from the AS. + - Recommended that home servers try a few times then time out, returning a + 408 Request Timeout to the client. + +:: + + GET /rooms/$room_alias?access_token=$hs_token + + Returns: + 200 : Room is recognised. + 404 : Room not found. + 401 : Credentials need to be supplied. + 403 : HS credentials rejected. + + + 200 OK response format + + {} + +Pushing ++++++++ +This API is called by the HS when the HS wants to push an event (or batch of +events) to the AS. + +Inputs: + - HS Credentials + - Event(s) to give to the AS + - HS-generated transaction ID +Output: + - None. + +Data flows: + +:: + + Typical + HS ---> AS : Home server sends events with transaction ID T. + <--- : AS sends back 200 OK. + + AS ACK Lost + HS ---> AS : Home server sends events with transaction ID T. + <-/- : AS 200 OK is lost. + HS ---> AS : Home server retries with the same transaction ID of T. + <--- : AS sends back 200 OK. If the AS had processed these events + already, it can NO-OP this request (and it knows if it is the same + events based on the transacton ID). + + +Retry notes: + - If the HS fails to pass on the events to the AS, it must retry the request. + - Since ASes by definition cannot alter the traffic being passed to it (unlike + say, a Policy Server), these requests can be done in parallel to general HS + processing; the HS doesn't need to block whilst doing this. + - Home servers should use exponential backoff as their retry algorithm. + - Home servers MUST NOT alter (e.g. add more) events they were going to + send within that transaction ID on retries, as the AS may have already + processed the events. + +Ordering notes: + - The events sent to the AS should be linearised, as they are from the event + stream. + - The home server will need to maintain a queue of transactions to send to + the AS. + +:: + + PUT /transactions/$transaction_id?access_token=$hs_token + + Request format + { + events: [ + ... + ] + } + +Client-Server v2 API Extensions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Passive application services can utilise a more powerful version of the +client-server API by identifying itself as an application service to the +home server. + +Identity assertion +++++++++++++++++++ +The client-server API infers the user ID from the ``access_token`` provided in +every request. It would be an annoying amount of book-keeping to maintain tokens +for every virtual user. It would be preferable if the application service could +use the CS API with its own ``as_token`` instead, and specify the virtual user +they wish to be acting on behalf of. For real users, this would require +additional permissions granting the AS permission to masquerade as a matrix user. + +Inputs: + - Application service token (``access_token``) + + Either: + - User ID in the AS namespace to act as. + Or: + - OAuth2 token of real user (which may end up being an access token) +Notes: + - This will apply on all aspects of the CS API, except for Account Management. + - The ``as_token`` is inserted into ``access_token`` which is usually where the + client token is. This is done on purpose to allow application services to + reuse client SDKs. + +:: + + /path?access_token=$token&user_id=$userid + + Query Parameters: + access_token: The application service token + user_id: The desired user ID to act as. + + /path?access_token=$token&user_token=$token + + Query Parameters: + access_token: The application service token + user_token: The token granted to the AS by the real user + +Timestamp massaging ++++++++++++++++++++ +The application service may want to inject events at a certain time (reflecting +the time on the network they are tracking e.g. irc, xmpp). Application services +need to be able to adjust the ``origin_server_ts`` value to do this. + +Inputs: + - Application service token (``as_token``) + - Desired timestamp +Notes: + - This will only apply when sending events. + +:: + + /path?access_token=$token&ts=$timestamp + + Query Parameters added to the send event APIs only: + access_token: The application service token + ts: The desired timestamp + +Server admin style permissions +++++++++++++++++++++++++++++++ +The home server needs to give the application service *full control* over its +namespace, both for users and for room aliases. This means that the AS should +be able to create/edit/delete any room alias in its namespace, as well as +create/delete any user in its namespace. No additional API changes need to be +made in order for control of room aliases to be granted to the AS. Creation of +users needs API changes in order to: + +- Work around captchas. +- Have a 'passwordless' user. + +This involves bypassing the registration flows entirely. This is achieved by +including the AS token on a ``/register`` request, along with a login type of +``m.login.application_service`` to set the desired user ID without a password. + +:: + + /register?access_token=$as_token + + Content: + { + type: "m.login.application_service", + user: "" + } + +Application services which attempt to create users or aliases *outside* of +their defined namespaces will receive an error code ``M_EXCLUSIVE``. Similarly, +normal users who attempt to create users or alises *inside* an application +service-defined namespace will receive the same ``M_EXCLUSIVE`` error code, +but only if the application service has defined the namespace as ``exclusive``. + +ID conventions +~~~~~~~~~~~~~~ +.. NOTE:: + - Giving HSes the freedom to namespace still feels like the Right Thing here. + - Exposing a public API provides the consistency which was the main complaint + against namespacing. + - This may have knock-on effects for the AS registration API. E.g. why don't + we let ASes specify the *URI* regex they want? + +This concerns the well-defined conventions for mapping 3P network IDs to matrix +IDs, which we expect clients to be able to do by themselves. + +User IDs +++++++++ +Matrix users may wish to directly contact a virtual user, e.g. to send an email. +The URI format is a well-structured way to represent a number of different ID +types, including: + +- MSISDNs (``tel``) +- Email addresses (``mailto``) +- IRC nicks (``irc`` - https://tools.ietf.org/html/draft-butcher-irc-url-04) +- XMPP (xep-0032) +- SIP URIs (RFC 3261) + +As a result, virtual user IDs SHOULD relate to their URI counterpart. This +mapping from URI to user ID can be expressed in a number of ways: + +- Expose a C-S API on the HS which takes URIs and responds with user IDs. +- Munge the URI with the user ID. + +Exposing an API would allow HSes to internally map user IDs however they like, +at the cost of an extra round trip (of which the response can be cached). +Munging the URI would allow clients to apply the mapping locally, but would force +user X on service Y to always map to the same munged user ID. Considering the +exposed API could just be applying this munging, there is more flexibility if +an API is exposed. + +:: + + GET /_matrix/app/v1/user?uri=$url_encoded_uri + + Returns 200 OK: + { + user_id: + } + +Room Aliases +++++++++++++ +We may want to expose some 3P network rooms so Matrix users can join them directly, +e.g. IRC rooms. We don't want to expose every 3P network room though, e.g. mailto, +tel. Rooms which are publicly accessible (e.g. IRC rooms) can be exposed as an alias by +the application service. Private rooms (e.g. sending an email to someone) should not +be exposed in this way, but should instead operate using normal invite/join semantics. +Therefore, the ID conventions discussed below are only valid for public rooms which +expose room aliases. + +Matrix users may wish to join XMPP rooms (e.g. using XEP-0045) or IRC rooms. In both +cases, these rooms can be expressed as URIs. For consistency, these "room" URIs +SHOULD be mapped in the same way as "user" URIs. + +:: + + GET /_matrix/app/v1/alias?uri=$url_encoded_uri + + Returns 200 OK: + { + alias: + } + +Event fields +++++++++++++ +We recommend that any gatewayed events should include an ``external_url`` field +in their content to provide a way for Matrix clients to link into the 'native' +client from which the event originated. For instance, this could contain the +message-ID for emails/nntp posts, or a link to a blog comment when gatewaying +blog comment traffic in & out of matrix Active Application Services ----------------------------