diff --git a/drafts/as-http-api.rst b/drafts/as-http-api.rst index 64b0e5d2..2c8f8e37 100644 --- a/drafts/as-http-api.rst +++ b/drafts/as-http-api.rst @@ -1,574 +1,574 @@ -.. TODO - Sometimes application services need to create rooms (e.g. when lazy loading - from room aliases). Created rooms need to have a user that created them, so - federation works (as it relies on an entry existing in m.room.member). We - should be able to add metadata to m.room.member to state that this user is an - application service, a virtual user, etc. - - -Application Services HTTP API -============================= - -.. contents:: Table of Contents - -.. sectnum:: - -Application Service -> Home Server ----------------------------------- -This contains home server APIs which are used by the application service. - -Registration API ``[Draft]`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This API registers the application service with its host homeserver to offer its -services. - -Inputs: - - Credentials (e.g. some kind of string token) - - Namespace[users] - - Namespace[room aliases] - - URL base to receive inbound comms -Output: - - The credentials the HS will use to query the AS with in return. (e.g. some - kind of string token) -Side effects: - - The HS will start delivering events to the URL base specified if this 200s. -API called when: - - The application service wants to register with a brand new home server. -Notes: - - An application service can state whether they should be the only ones who - can manage a specified namespace. This is referred to as an "exclusive" - namespace. An exclusive namespace prevents humans and other application - services from creating/deleting entities in that namespace. Typically, - exclusive namespaces are used when the rooms represent real rooms on - another service (e.g. IRC). Non-exclusive namespaces are used when the - application service is merely augmenting the room itself (e.g. providing - logging or searching facilities). - - Namespaces are represented by POSIX extended regular expressions in JSON. - They look like:: - - users: [ - { - "exclusive": true, - "regex": "@irc\.freenode\.net/.*" - } - ] - -:: - - POST /register - - Request format - { - url: "https://my.application.service.com/matrix/", - as_token: "some_AS_token", - namespaces: { - users: [ - { - "exclusive": true, - "regex": "@irc\.freenode\.net/.*" - } - ], - aliases: [ - { - "exclusive": true, - "regex": "#irc\.freenode\.net/.*" - } - ], - rooms: [ - { - "exclusive": true, - "regex": "!irc\.freenode\.net/.*" - } - ] - } - } - - - Returns: - 200 : Registration accepted. - 400 : Namespaces do not conform to regex - 401 : Credentials need to be supplied. - 403 : AS credentials rejected. - - - 200 OK response format - - { - hs_token: "string" - } - -Unregister API ``[Draft]`` -~~~~~~~~~~~~~~~~~~~~~~~~~~ -This API unregisters a previously registered AS from the home server. - -Inputs: - - AS token -Output: - - None. -Side effects: - - The HS will stop delivering events to the URL base specified for this AS if - this 200s. -API called when: - - The application service wants to stop receiving all events from the HS. - -:: - - POST /unregister - - Request format - { - as_token: "string" - } - - -Home Server -> Application Service ----------------------------------- -This contains application service APIs which are used by the home server. - -User Query ``[Draft]`` -~~~~~~~~~~~~~~~~~~~~~~ - -This API is called by the HS to query the existence of a user on the Application -Service's namespace. - -Inputs: - - User ID - - HS Credentials -Output: - - Whether the user exists. -Side effects: - - User is created on the HS by the AS via CS APIs during the processing of this request. -API called when: - - HS receives an event for an unknown user ID in the AS's namespace, e.g. an - invite event to a room. -Notes: - - When the AS receives this request, if the user exists, it must create the user via - the CS API. - - It can also set arbitrary information about the user (e.g. display name, join rooms, etc) - using the CS API. - - When this setup is complete, the AS should respond to the HS request. This means the AS - blocks the HS until the user is created. - - This is deemed more flexible than alternative methods (e.g. returning a JSON blob with the - user's display name and get the HS to provision the user). -Retry notes: - - The home server cannot respond to the client's request until the response to - this API is obtained from the AS. - - Recommended that home servers try a few times then time out, returning a - 408 Request Timeout to the client. - -:: - - GET /users/$user_id?access_token=$hs_token - - Returns: - 200 : User is recognised. - 404 : User not found. - 401 : Credentials need to be supplied. - 403 : HS credentials rejected. - - - 200 OK response format - - {} - -Room Alias Query ``[Draft]`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This API is called by the HS to query the existence of a room alias on the -Application Service's namespace. - -Inputs: - - Room alias - - HS Credentials -Output: - - Whether the room exists. -Side effects: - - Room is created on the HS by the AS via CS APIs during the processing of - this request. -API called when: - - HS receives an event to join a room alias in the AS's namespace. -Notes: - - When the AS receives this request, if the room exists, it must create the room via - the CS API. - - It can also set arbitrary information about the room (e.g. name, topic, etc) - using the CS API. - - It can send messages as other users in order to populate scrollback. - - When this setup is complete, the AS should respond to the HS request. This means the AS - blocks the HS until the room is created and configured. - - This is deemed more flexible than alternative methods (e.g. returning an initial sync - style JSON blob and get the HS to provision the room). It also means that the AS knows - the room ID -> alias mapping. -Retry notes: - - The home server cannot respond to the client's request until the response to - this API is obtained from the AS. - - Recommended that home servers try a few times then time out, returning a - 408 Request Timeout to the client. - -:: - - GET /rooms/$room_alias?access_token=$hs_token - - Returns: - 200 : Room is recognised. - 404 : Room not found. - 401 : Credentials need to be supplied. - 403 : HS credentials rejected. - - - 200 OK response format - - {} - -Pushing ``[Draft]`` -~~~~~~~~~~~~~~~~~~~ -This API is called by the HS when the HS wants to push an event (or batch of -events) to the AS. - -Inputs: - - HS Credentials - - Event(s) to give to the AS - - HS-generated transaction ID -Output: - - None. - -Data flows: - -:: - - Typical - HS ---> AS : Home server sends events with transaction ID T. - <--- : AS sends back 200 OK. - - AS ACK Lost - HS ---> AS : Home server sends events with transaction ID T. - <-/- : AS 200 OK is lost. - HS ---> AS : Home server retries with the same transaction ID of T. - <--- : AS sends back 200 OK. If the AS had processed these events - already, it can NO-OP this request (and it knows if it is the same - events based on the transacton ID). - - -Retry notes: - - If the HS fails to pass on the events to the AS, it must retry the request. - - Since ASes by definition cannot alter the traffic being passed to it (unlike - say, a Policy Server), these requests can be done in parallel to general HS - processing; the HS doesn't need to block whilst doing this. - - Home servers should use exponential backoff as their retry algorithm. - - Home servers MUST NOT alter (e.g. add more) events they were going to - send within that transaction ID on retries, as the AS may have already - processed the events. - -Ordering notes: - - The events sent to the AS should be linearised, as they are from the event - stream. - - The home server will need to maintain a queue of transactions to send to - the AS. - -:: - - PUT /transactions/$transaction_id?access_token=$hs_token - - Request format - { - events: [ - ... - ] - } - -Client-Server v2 API Extensions -------------------------------- - -Identity assertion -~~~~~~~~~~~~~~~~~~ -The client-server API infers the user ID from the ``access_token`` provided in -every request. It would be an annoying amount of book-keeping to maintain tokens -for every virtual user. It would be preferable if the application service could -use the CS API with its own ``as_token`` instead, and specify the virtual user -they wish to be acting on behalf of. For real users, this would require -additional permissions granting the AS permission to masquerade as a matrix user. - -Inputs: - - Application service token (``access_token``) - - Either: - - User ID in the AS namespace to act as. - Or: - - OAuth2 token of real user (which may end up being an access token) -Notes: - - This will apply on all aspects of the CS API, except for Account Management. - - The ``as_token`` is inserted into ``access_token`` which is usually where the - client token is. This is done on purpose to allow application services to - reuse client SDKs. - -:: - - /path?access_token=$token&user_id=$userid - - Query Parameters: - access_token: The application service token - user_id: The desired user ID to act as. - - /path?access_token=$token&user_token=$token - - Query Parameters: - access_token: The application service token - user_token: The token granted to the AS by the real user - -Timestamp massaging -~~~~~~~~~~~~~~~~~~~ -The application service may want to inject events at a certain time (reflecting -the time on the network they are tracking e.g. irc, xmpp). Application services -need to be able to adjust the ``origin_server_ts`` value to do this. - -Inputs: - - Application service token (``as_token``) - - Desired timestamp -Notes: - - This will only apply when sending events. - -:: - - /path?access_token=$token&ts=$timestamp - - Query Parameters added to the send event APIs only: - access_token: The application service token - ts: The desired timestamp - -Server admin style permissions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The home server needs to give the application service *full control* over its -namespace, both for users and for room aliases. This means that the AS should -be able to create/edit/delete any room alias in its namespace, as well as -create/delete any user in its namespace. No additional API changes need to be -made in order for control of room aliases to be granted to the AS. Creation of -users needs API changes in order to: - -- Work around captchas. -- Have a 'passwordless' user. - -This involves bypassing the registration flows entirely. This is achieved by -including the AS token on a ``/register`` request, along with a login type of -``m.login.application_service`` to set the desired user ID without a password. - -:: - - /register?access_token=$as_token - - Content: - { - type: "m.login.application_service", - user: "" - } - -Application services which attempt to create users or aliases *outside* of -their defined namespaces will receive an error code ``M_EXCLUSIVE``. Similarly, -normal users who attempt to create users or alises *inside* an application -service-defined namespace will receive the same ``M_EXCLUSIVE`` error code. - -ID conventions ``[Draft]`` --------------------------- -.. NOTE:: - - Giving HSes the freedom to namespace still feels like the Right Thing here. - - Exposing a public API provides the consistency which was the main complaint - against namespacing. - - This may have knock-on effects for the AS registration API. E.g. why don't - we let ASes specify the *URI* regex they want? - -This concerns the well-defined conventions for mapping 3P network IDs to matrix -IDs, which we expect clients to be able to do by themselves. - -User IDs -~~~~~~~~ -Matrix users may wish to directly contact a virtual user, e.g. to send an email. -The URI format is a well-structured way to represent a number of different ID -types, including: - -- MSISDNs (``tel``) -- Email addresses (``mailto``) -- IRC nicks (``irc`` - https://tools.ietf.org/html/draft-butcher-irc-url-04) -- XMPP (xep-0032) -- SIP URIs (RFC 3261) - -As a result, virtual user IDs SHOULD relate to their URI counterpart. This -mapping from URI to user ID can be expressed in a number of ways: - -- Expose a C-S API on the HS which takes URIs and responds with user IDs. -- Munge the URI with the user ID. - -Exposing an API would allow HSes to internally map user IDs however they like, -at the cost of an extra round trip (of which the response can be cached). -Munging the URI would allow clients to apply the mapping locally, but would force -user X on service Y to always map to the same munged user ID. Considering the -exposed API could just be applying this munging, there is more flexibility if -an API is exposed. - -:: - - GET /_matrix/app/v1/user?uri=$url_encoded_uri - - Returns 200 OK: - { - user_id: - } - -Room Aliases -~~~~~~~~~~~~ -We may want to expose some 3P network rooms so Matrix users can join them directly, -e.g. IRC rooms. We don't want to expose every 3P network room though, e.g. mailto, -tel. Rooms which are publicly accessible (e.g. IRC rooms) can be exposed as an alias by -the application service. Private rooms (e.g. sending an email to someone) should not -be exposed in this way, but should instead operate using normal invite/join semantics. -Therefore, the ID conventions discussed below are only valid for public rooms which -expose room aliases. - -Matrix users may wish to join XMPP rooms (e.g. using XEP-0045) or IRC rooms. In both -cases, these rooms can be expressed as URIs. For consistency, these "room" URIs -SHOULD be mapped in the same way as "user" URIs. - -:: - - GET /_matrix/app/v1/alias?uri=$url_encoded_uri - - Returns 200 OK: - { - alias: - } - -Event fields -~~~~~~~~~~~~ -We recommend that any gatewayed events should include an `external_url` field in -their content to provide a way for Matrix clients to link into the 'native' -client from which the event originated. For instance, this could contain the -message-ID for emails/nntp posts, or a link to a blog comment when gatewaying -blog comment traffic in & out of matrix - - -Examples --------- -.. NOTE:: - - User/Alias namespaces are subject to change depending on ID conventions. - -IRC -~~~ -Pre-conditions: - - Server admin stores the AS token "T_a" on the home server. - - Home server has a token "T_h". - - Home server has the domain "hsdomain.com" - -1. Application service registration - -:: - - AS -> HS: Registers itself with the home server - POST /register - { - url: "https://someapp.com/matrix", - as_token: "T_a", - namespaces: { - users: [ - { - "exclusive": true, - "regex": "@irc\.freenode\.net/.*" - } - ], - aliases: [ - { - "exclusive": true, - "regex": "#irc\.freenode\.net/.*" - } - ] - } - } - - Returns 200 OK: - { - hs_token: "T_h" - } - -2. IRC user "Bob" says "hello?" on "#matrix" at timestamp 1421416883133: - -:: - - - AS stores message as potential scrollback. - - Nothing happens as no Matrix users are in the room. - -3. Matrix user "@alice:hsdomain.com" wants to join "#matrix": - -:: - - User -> HS: Request to join "#irc.freenode.net/#matrix:hsdomain.com" - - HS -> AS: Room Query "#irc.freenode.net/#matrix:hsdomain.com" - GET /rooms/%23irc.freenode.net%2F%23matrix%3Ahsdomain.com?access_token=T_h - [Starts blocking] - AS -> HS: Creates room. Gets room ID "!aasaasasa:hsdomain.com". - AS -> HS: Sets room name to "#matrix". - AS -> HS: Sends message as ""@irc.freenode.net/Bob:hsdomain.com" - PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message - ?access_token=T_a - &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com - &ts=1421416883133 - { - body: "hello?" - msgtype: "m.text" - } - HS -> AS: User Query "@irc.freenode.net/Bob:hsdomain.com" - GET /users/%40irc.freenode.net%2FBob%3Ahsdomain.com?access_token=T_h - [Starts blocking] - AS -> HS: Creates user using CS API extension. - POST /register?access_token=T_a - { - type: "m.login.application_service", - user: "irc.freenode.net/Bob" - } - AS -> HS: Set user display name to "Bob". - [Finishes blocking] - [Finished blocking] - - - HS sends room information back to client. - -4. @alice:hsdomain.com says "hi!" in this room: - -:: - - User -> HS: Send message "hi!" in room !aasaasasa:hsdomain.com - - - HS sends message. - - HS sees the room ID is in the AS namespace and pushes it to the AS. - - HS -> AS: Push event - PUT /transactions/1?access_token=T_h - { - events: [ - { - content: { - body: "hi!", - msgtype: "m.text" - }, - origin_server_ts: , - user_id: "@alice:hsdomain.com", - room_id: "!aasaasasa:hsdomain.com", - type: "m.room.message" - } - ] - } - - - AS passes this through to IRC. - - -5. IRC user "Bob" says "what's up?" on "#matrix" at timestamp 1421418084816: - -:: - - IRC -> AS: "what's up?" - AS -> HS: Send message via CS API extension - PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message - ?access_token=T_a - &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com - &ts=1421418084816 - { - body: "what's up?" - msgtype: "m.text" - } - - - HS modifies the user_id and origin_server_ts on the event and sends it. +.. TODO + Sometimes application services need to create rooms (e.g. when lazy loading + from room aliases). Created rooms need to have a user that created them, so + federation works (as it relies on an entry existing in m.room.member). We + should be able to add metadata to m.room.member to state that this user is an + application service, a virtual user, etc. + + +Application Services HTTP API +============================= + +.. contents:: Table of Contents + +.. sectnum:: + +Application Service -> Home Server +---------------------------------- +This contains home server APIs which are used by the application service. + +Registration API ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This API registers the application service with its host homeserver to offer its +services. + +Inputs: + - Credentials (e.g. some kind of string token) + - Namespace[users] + - Namespace[room aliases] + - URL base to receive inbound comms +Output: + - The credentials the HS will use to query the AS with in return. (e.g. some + kind of string token) +Side effects: + - The HS will start delivering events to the URL base specified if this 200s. +API called when: + - The application service wants to register with a brand new home server. +Notes: + - An application service can state whether they should be the only ones who + can manage a specified namespace. This is referred to as an "exclusive" + namespace. An exclusive namespace prevents humans and other application + services from creating/deleting entities in that namespace. Typically, + exclusive namespaces are used when the rooms represent real rooms on + another service (e.g. IRC). Non-exclusive namespaces are used when the + application service is merely augmenting the room itself (e.g. providing + logging or searching facilities). + - Namespaces are represented by POSIX extended regular expressions in JSON. + They look like:: + + users: [ + { + "exclusive": true, + "regex": "@irc\.freenode\.net/.*" + } + ] + +:: + + POST /register + + Request format + { + url: "https://my.application.service.com/matrix/", + as_token: "some_AS_token", + namespaces: { + users: [ + { + "exclusive": true, + "regex": "@irc\.freenode\.net/.*" + } + ], + aliases: [ + { + "exclusive": true, + "regex": "#irc\.freenode\.net/.*" + } + ], + rooms: [ + { + "exclusive": true, + "regex": "!irc\.freenode\.net/.*" + } + ] + } + } + + + Returns: + 200 : Registration accepted. + 400 : Namespaces do not conform to regex + 401 : Credentials need to be supplied. + 403 : AS credentials rejected. + + + 200 OK response format + + { + hs_token: "string" + } + +Unregister API ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ +This API unregisters a previously registered AS from the home server. + +Inputs: + - AS token +Output: + - None. +Side effects: + - The HS will stop delivering events to the URL base specified for this AS if + this 200s. +API called when: + - The application service wants to stop receiving all events from the HS. + +:: + + POST /unregister + + Request format + { + as_token: "string" + } + + +Home Server -> Application Service +---------------------------------- +This contains application service APIs which are used by the home server. + +User Query ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~ + +This API is called by the HS to query the existence of a user on the Application +Service's namespace. + +Inputs: + - User ID + - HS Credentials +Output: + - Whether the user exists. +Side effects: + - User is created on the HS by the AS via CS APIs during the processing of this request. +API called when: + - HS receives an event for an unknown user ID in the AS's namespace, e.g. an + invite event to a room. +Notes: + - When the AS receives this request, if the user exists, it must create the user via + the CS API. + - It can also set arbitrary information about the user (e.g. display name, join rooms, etc) + using the CS API. + - When this setup is complete, the AS should respond to the HS request. This means the AS + blocks the HS until the user is created. + - This is deemed more flexible than alternative methods (e.g. returning a JSON blob with the + user's display name and get the HS to provision the user). +Retry notes: + - The home server cannot respond to the client's request until the response to + this API is obtained from the AS. + - Recommended that home servers try a few times then time out, returning a + 408 Request Timeout to the client. + +:: + + GET /users/$user_id?access_token=$hs_token + + Returns: + 200 : User is recognised. + 404 : User not found. + 401 : Credentials need to be supplied. + 403 : HS credentials rejected. + + + 200 OK response format + + {} + +Room Alias Query ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This API is called by the HS to query the existence of a room alias on the +Application Service's namespace. + +Inputs: + - Room alias + - HS Credentials +Output: + - Whether the room exists. +Side effects: + - Room is created on the HS by the AS via CS APIs during the processing of + this request. +API called when: + - HS receives an event to join a room alias in the AS's namespace. +Notes: + - When the AS receives this request, if the room exists, it must create the room via + the CS API. + - It can also set arbitrary information about the room (e.g. name, topic, etc) + using the CS API. + - It can send messages as other users in order to populate scrollback. + - When this setup is complete, the AS should respond to the HS request. This means the AS + blocks the HS until the room is created and configured. + - This is deemed more flexible than alternative methods (e.g. returning an initial sync + style JSON blob and get the HS to provision the room). It also means that the AS knows + the room ID -> alias mapping. +Retry notes: + - The home server cannot respond to the client's request until the response to + this API is obtained from the AS. + - Recommended that home servers try a few times then time out, returning a + 408 Request Timeout to the client. + +:: + + GET /rooms/$room_alias?access_token=$hs_token + + Returns: + 200 : Room is recognised. + 404 : Room not found. + 401 : Credentials need to be supplied. + 403 : HS credentials rejected. + + + 200 OK response format + + {} + +Pushing ``[Draft]`` +~~~~~~~~~~~~~~~~~~~ +This API is called by the HS when the HS wants to push an event (or batch of +events) to the AS. + +Inputs: + - HS Credentials + - Event(s) to give to the AS + - HS-generated transaction ID +Output: + - None. + +Data flows: + +:: + + Typical + HS ---> AS : Home server sends events with transaction ID T. + <--- : AS sends back 200 OK. + + AS ACK Lost + HS ---> AS : Home server sends events with transaction ID T. + <-/- : AS 200 OK is lost. + HS ---> AS : Home server retries with the same transaction ID of T. + <--- : AS sends back 200 OK. If the AS had processed these events + already, it can NO-OP this request (and it knows if it is the same + events based on the transacton ID). + + +Retry notes: + - If the HS fails to pass on the events to the AS, it must retry the request. + - Since ASes by definition cannot alter the traffic being passed to it (unlike + say, a Policy Server), these requests can be done in parallel to general HS + processing; the HS doesn't need to block whilst doing this. + - Home servers should use exponential backoff as their retry algorithm. + - Home servers MUST NOT alter (e.g. add more) events they were going to + send within that transaction ID on retries, as the AS may have already + processed the events. + +Ordering notes: + - The events sent to the AS should be linearised, as they are from the event + stream. + - The home server will need to maintain a queue of transactions to send to + the AS. + +:: + + PUT /transactions/$transaction_id?access_token=$hs_token + + Request format + { + events: [ + ... + ] + } + +Client-Server v2 API Extensions +------------------------------- + +Identity assertion +~~~~~~~~~~~~~~~~~~ +The client-server API infers the user ID from the ``access_token`` provided in +every request. It would be an annoying amount of book-keeping to maintain tokens +for every virtual user. It would be preferable if the application service could +use the CS API with its own ``as_token`` instead, and specify the virtual user +they wish to be acting on behalf of. For real users, this would require +additional permissions granting the AS permission to masquerade as a matrix user. + +Inputs: + - Application service token (``access_token``) + + Either: + - User ID in the AS namespace to act as. + Or: + - OAuth2 token of real user (which may end up being an access token) +Notes: + - This will apply on all aspects of the CS API, except for Account Management. + - The ``as_token`` is inserted into ``access_token`` which is usually where the + client token is. This is done on purpose to allow application services to + reuse client SDKs. + +:: + + /path?access_token=$token&user_id=$userid + + Query Parameters: + access_token: The application service token + user_id: The desired user ID to act as. + + /path?access_token=$token&user_token=$token + + Query Parameters: + access_token: The application service token + user_token: The token granted to the AS by the real user + +Timestamp massaging +~~~~~~~~~~~~~~~~~~~ +The application service may want to inject events at a certain time (reflecting +the time on the network they are tracking e.g. irc, xmpp). Application services +need to be able to adjust the ``origin_server_ts`` value to do this. + +Inputs: + - Application service token (``as_token``) + - Desired timestamp +Notes: + - This will only apply when sending events. + +:: + + /path?access_token=$token&ts=$timestamp + + Query Parameters added to the send event APIs only: + access_token: The application service token + ts: The desired timestamp + +Server admin style permissions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The home server needs to give the application service *full control* over its +namespace, both for users and for room aliases. This means that the AS should +be able to create/edit/delete any room alias in its namespace, as well as +create/delete any user in its namespace. No additional API changes need to be +made in order for control of room aliases to be granted to the AS. Creation of +users needs API changes in order to: + +- Work around captchas. +- Have a 'passwordless' user. + +This involves bypassing the registration flows entirely. This is achieved by +including the AS token on a ``/register`` request, along with a login type of +``m.login.application_service`` to set the desired user ID without a password. + +:: + + /register?access_token=$as_token + + Content: + { + type: "m.login.application_service", + user: "" + } + +Application services which attempt to create users or aliases *outside* of +their defined namespaces will receive an error code ``M_EXCLUSIVE``. Similarly, +normal users who attempt to create users or alises *inside* an application +service-defined namespace will receive the same ``M_EXCLUSIVE`` error code. + +ID conventions ``[Draft]`` +-------------------------- +.. NOTE:: + - Giving HSes the freedom to namespace still feels like the Right Thing here. + - Exposing a public API provides the consistency which was the main complaint + against namespacing. + - This may have knock-on effects for the AS registration API. E.g. why don't + we let ASes specify the *URI* regex they want? + +This concerns the well-defined conventions for mapping 3P network IDs to matrix +IDs, which we expect clients to be able to do by themselves. + +User IDs +~~~~~~~~ +Matrix users may wish to directly contact a virtual user, e.g. to send an email. +The URI format is a well-structured way to represent a number of different ID +types, including: + +- MSISDNs (``tel``) +- Email addresses (``mailto``) +- IRC nicks (``irc`` - https://tools.ietf.org/html/draft-butcher-irc-url-04) +- XMPP (xep-0032) +- SIP URIs (RFC 3261) + +As a result, virtual user IDs SHOULD relate to their URI counterpart. This +mapping from URI to user ID can be expressed in a number of ways: + +- Expose a C-S API on the HS which takes URIs and responds with user IDs. +- Munge the URI with the user ID. + +Exposing an API would allow HSes to internally map user IDs however they like, +at the cost of an extra round trip (of which the response can be cached). +Munging the URI would allow clients to apply the mapping locally, but would force +user X on service Y to always map to the same munged user ID. Considering the +exposed API could just be applying this munging, there is more flexibility if +an API is exposed. + +:: + + GET /_matrix/app/v1/user?uri=$url_encoded_uri + + Returns 200 OK: + { + user_id: + } + +Room Aliases +~~~~~~~~~~~~ +We may want to expose some 3P network rooms so Matrix users can join them directly, +e.g. IRC rooms. We don't want to expose every 3P network room though, e.g. mailto, +tel. Rooms which are publicly accessible (e.g. IRC rooms) can be exposed as an alias by +the application service. Private rooms (e.g. sending an email to someone) should not +be exposed in this way, but should instead operate using normal invite/join semantics. +Therefore, the ID conventions discussed below are only valid for public rooms which +expose room aliases. + +Matrix users may wish to join XMPP rooms (e.g. using XEP-0045) or IRC rooms. In both +cases, these rooms can be expressed as URIs. For consistency, these "room" URIs +SHOULD be mapped in the same way as "user" URIs. + +:: + + GET /_matrix/app/v1/alias?uri=$url_encoded_uri + + Returns 200 OK: + { + alias: + } + +Event fields +~~~~~~~~~~~~ +We recommend that any gatewayed events should include an ``external_url`` field +in their content to provide a way for Matrix clients to link into the 'native' +client from which the event originated. For instance, this could contain the +message-ID for emails/nntp posts, or a link to a blog comment when gatewaying +blog comment traffic in & out of matrix + + +Examples +-------- +.. NOTE:: + - User/Alias namespaces are subject to change depending on ID conventions. + +IRC +~~~ +Pre-conditions: + - Server admin stores the AS token "T_a" on the home server. + - Home server has a token "T_h". + - Home server has the domain "hsdomain.com" + +1. Application service registration + +:: + + AS -> HS: Registers itself with the home server + POST /register + { + url: "https://someapp.com/matrix", + as_token: "T_a", + namespaces: { + users: [ + { + "exclusive": true, + "regex": "@irc\.freenode\.net/.*" + } + ], + aliases: [ + { + "exclusive": true, + "regex": "#irc\.freenode\.net/.*" + } + ] + } + } + + Returns 200 OK: + { + hs_token: "T_h" + } + +2. IRC user "Bob" says "hello?" on "#matrix" at timestamp 1421416883133: + +:: + + - AS stores message as potential scrollback. + - Nothing happens as no Matrix users are in the room. + +3. Matrix user "@alice:hsdomain.com" wants to join "#matrix": + +:: + + User -> HS: Request to join "#irc.freenode.net/#matrix:hsdomain.com" + + HS -> AS: Room Query "#irc.freenode.net/#matrix:hsdomain.com" + GET /rooms/%23irc.freenode.net%2F%23matrix%3Ahsdomain.com?access_token=T_h + [Starts blocking] + AS -> HS: Creates room. Gets room ID "!aasaasasa:hsdomain.com". + AS -> HS: Sets room name to "#matrix". + AS -> HS: Sends message as ""@irc.freenode.net/Bob:hsdomain.com" + PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message + ?access_token=T_a + &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com + &ts=1421416883133 + { + body: "hello?" + msgtype: "m.text" + } + HS -> AS: User Query "@irc.freenode.net/Bob:hsdomain.com" + GET /users/%40irc.freenode.net%2FBob%3Ahsdomain.com?access_token=T_h + [Starts blocking] + AS -> HS: Creates user using CS API extension. + POST /register?access_token=T_a + { + type: "m.login.application_service", + user: "irc.freenode.net/Bob" + } + AS -> HS: Set user display name to "Bob". + [Finishes blocking] + [Finished blocking] + + - HS sends room information back to client. + +4. @alice:hsdomain.com says "hi!" in this room: + +:: + + User -> HS: Send message "hi!" in room !aasaasasa:hsdomain.com + + - HS sends message. + - HS sees the room ID is in the AS namespace and pushes it to the AS. + + HS -> AS: Push event + PUT /transactions/1?access_token=T_h + { + events: [ + { + content: { + body: "hi!", + msgtype: "m.text" + }, + origin_server_ts: , + user_id: "@alice:hsdomain.com", + room_id: "!aasaasasa:hsdomain.com", + type: "m.room.message" + } + ] + } + + - AS passes this through to IRC. + + +5. IRC user "Bob" says "what's up?" on "#matrix" at timestamp 1421418084816: + +:: + + IRC -> AS: "what's up?" + AS -> HS: Send message via CS API extension + PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message + ?access_token=T_a + &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com + &ts=1421418084816 + { + body: "what's up?" + msgtype: "m.text" + } + + - HS modifies the user_id and origin_server_ts on the event and sends it. diff --git a/drafts/definitions.rst b/drafts/definitions.rst index 4db39222..cfb56857 100644 --- a/drafts/definitions.rst +++ b/drafts/definitions.rst @@ -5,7 +5,7 @@ Definitions # *Event* -- A JSON object that represents a piece of information to be distributed to the the room. The object includes a payload and metadata, -including a `type` used to indicate what the payload is for and how to process +including a ``type`` used to indicate what the payload is for and how to process them. It also includes one or more references to previous events. # *Event graph* -- Events and their references to previous events form a @@ -13,7 +13,7 @@ directed acyclic graph. All events must be a descendant of the first event in a room, except for a few special circumstances. # *State event* -- A state event is an event that has a non-null string valued -`state_key` field. It may also include a `prev_state` key referencing exactly +`state_key` field. It may also include a ``prev_state`` key referencing exactly one state event with the same type and state key, in the same event graph. # *State tree* -- A state tree is a tree formed by a collection of state events diff --git a/drafts/erik-model.rst b/drafts/erik-model.rst index c21c558d..b2f9908d 100644 --- a/drafts/erik-model.rst +++ b/drafts/erik-model.rst @@ -1,4 +1,7 @@ -This is a standalone description of the data architecture of Synapse. There is a lot of overlap with the currennt specification, so it has been split out here for posterity. Hopefully all the important bits have been merged into the relevant places in the main spec. +This is a standalone description of the data architecture of Synapse. There is a +lot of overlap with the current specification, so it has been split out here for +posterity. Hopefully all the important bits have been merged into the relevant +places in the main spec. Model diff --git a/drafts/general_api.rst b/drafts/general_api.rst index 2b42b8b0..ea24c135 100644 --- a/drafts/general_api.rst +++ b/drafts/general_api.rst @@ -555,7 +555,7 @@ signature. Requesting the "raw" federation event will have to return these keys. Account Management API ``[Draft]`` ---------------------------------- -The registration and login APIs in v2 do not support specifying device IDs. In v2, +The registration and login APIs in v1 do not support specifying device IDs. In v2, this will become *mandatory* when sending your initial request. Access tokens will be scoped per device, so using the same device ID twice when logging in will clobber the old access token. @@ -810,6 +810,11 @@ Notes: Presence API ``[Draft]`` ------------------------ + +.. FIXME + this seems to be ignoring activity timers entirely, which were present on + the planning etherpad and are present in the actual HTTP API. Needs attention. + The goals of presence are to: - Let other users know if someone is "online". diff --git a/drafts/media_repository.rst b/drafts/media_repository.rst deleted file mode 100644 index 879e1b85..00000000 --- a/drafts/media_repository.rst +++ /dev/null @@ -1,77 +0,0 @@ -Media Repository -================ - -File uploading and downloading. - -HTTP API --------- - -Uploads are POSTed to a resource which returns a token which is used to GET -the download. Uploads are POSTed to the sender's local homeserver, but are -downloaded from the recipient's local homeserver, which must thus first transfer -the content from the origin homeserver using the same API (unless the origin -and destination homeservers are the same). The upload/download API is:: - - => POST /_matrix/media/v1/upload HTTP/1.1 - Content-Type: - - - - <= HTTP/1.1 200 OK - Content-Type: application/json - - { "content-uri": "mxc:///" } - - => GET /_matrix/media/v1/download// HTTP/1.1 - - <= HTTP/1.1 200 OK - Content-Type: - Content-Disposition: attachment;filename= - - - -Clients can get thumbnails by supplying a desired width and height and -thumbnailing method:: - - => GET /_matrix/media/v1/thumbnail/ - /?width=&height=&method= HTTP/1.1 - - <= HTTP/1.1 200 OK - Content-Type: image/jpeg or image/png - - - -The thumbnail methods are "crop" and "scale". "scale" trys to return an -image where either the width or the height is smaller than the requested -size. The client should then scale and letterbox the image if it needs to -fit within a given rectangle. "crop" trys to return an image where the -width and height are close to the requested size and the aspect matches -the requested size. The client should scale the image if it needs to fit -within a given rectangle. - -Homeservers may generate thumbnails for content uploaded to remote -homeservers themselves or may rely on the remote homeserver to thumbnail -the content. Homeservers may return thumbnails of a different size to that -requested. However homeservers should provide exact matches where reasonable. -Homeservers must never upscale images. - -Security --------- - -Clients may try to upload very large files. Homeservers should not store files -that are too large and should not serve them to clients. - -Clients may try to upload very large images. Homeservers should not attempt to -generate thumbnails for images that are too large. - -Remote homeservers may host very large files or images. Homeserver should not -proxy or thumbnail large files or images from remote homeservers. - -Clients may try to upload a large number of files. Homeservers should limit the -number and total size of media that can be uploaded by clients. - -Clients may try to access a large number of remote files through a homeserver. -Homeservers should restrict the number and size of remote files that it caches. - -Clients or remote homeservers may try to upload malicious files targeting -vulnerabilities in either the homeserver thumbnailing or the client decoders. diff --git a/drafts/object_model.rst b/drafts/object_model.rst index 1ae60b2c..0341d6a5 100644 --- a/drafts/object_model.rst +++ b/drafts/object_model.rst @@ -1,6 +1,5 @@ - - - +..TODO + What are the start & end tokens doing here?! :: diff --git a/scripts/gendoc.py b/scripts/gendoc.py index 1b008685..92024035 100755 --- a/scripts/gendoc.py +++ b/scripts/gendoc.py @@ -101,6 +101,7 @@ def prepare_env(): pass def cleanup_env(): + #pass shutil.rmtree("./tmp") def main(): diff --git a/specification/00_basis.rst b/specification/00_basis.rst index e97d3406..5faacc0d 100644 --- a/specification/00_basis.rst +++ b/specification/00_basis.rst @@ -1,8 +1,8 @@ Matrix Specification ==================== -.. NOTE:: - The git version of this document is ``$GIT_VERSION`` +Version: ``$GIT_VERSION`` +------------------------------------------- Table of Contents ================= @@ -13,11 +13,14 @@ Table of Contents Introduction ============ -Matrix is a new set of open APIs for open-federated Instant Messaging and VoIP -functionality, designed to create and support a new global real-time -communication ecosystem on the internet. This specification is the ongoing -result of standardising the APIs used by the various components of the Matrix -ecosystem to communicate with one another. +Matrix is a set of open APIs for open-federated Instant Messaging (IM), Voice +over IP (VoIP) and Internet of Things (IoT) communication, designed to create +and support a new global real-time communication ecosystem. The intention is to +provide an open decentralised pubsub layer for the internet for securely +persisting and publishing/subscribing JSON objects. + +This specification is the ongoing result of standardising the APIs used by the +various components of the Matrix ecosystem to communicate with one another. .. WARNING:: The Matrix specification is still evolving: the APIs are not yet frozen @@ -97,8 +100,36 @@ Overview Architecture ------------ -Clients transmit data to other clients through home servers (HSes). Clients do -not communicate with each other directly. +Matrix defines APIs for synchronising extensible JSON objects known as +``events`` between compatible clients, servers and services. Clients are +typically messaging/VoIP applications or IoT devices/hubs and communicate by +synchronising communication history with their ``homeserver`` using the +``Client-Server API``. Each homeserver stores the communication history and +account information for all of its clients, and shares data with the wider +Matrix ecosystem by synchronising communication history with other homeservers +and their clients. + +Clients typically communicate with each other by emitting events in the +context of a virtual ``room``. Room data is replicated across *all of the +homeservers* whose users are participating in a given room. As such, *no +single homeserver has control or ownership over a given room*. Homeservers +model communication history as a partially ordered graph of events known as +the room's ``event graph``, which is synchronised with eventual consistency +between the participating servers using the ``Server-Server API``. This process +of synchronising shared conversation history between homeservers run by +different parties is called ``Federation``. Matrix optimises for the the +Availability and Partitioned properties of CAP theorem at +the expense of Consistency. + +For example, for client A to send a message to client B, client A performs an +HTTP PUT of the required JSON event on its homeserver (HS) using the +client-server API. A's HS appends this event to its copy of the room's event +graph, signing the message in the context of the graph for integrity. A's HS +then replicates the message to B's HS by performing an HTTP PUT using the +server-server API. B's HS authenticates the request, validates the event's +signature, authorises the event's contents and then adds it to its copy of the +room's event graph. Client B then receives the message from his homeserver via +a long-lived GET request. :: @@ -107,21 +138,23 @@ not communicate with each other directly. { Matrix client A } { Matrix client B } ^ | ^ | - | events | | events | + | events | Client-Server API | events | | V | V +------------------+ +------------------+ | |---------( HTTPS )--------->| | | Home Server | | Home Server | | |<--------( HTTPS )----------| | - +------------------+ Federation +------------------+ + +------------------+ Server-Server API +------------------+ + History Synchronisation + (Federation) + + +Users +~~~~~ -A "Client" typically represents a human using a web application or mobile app. -Clients use the "Client-to-Server" (C-S) API to communicate with their home -server, which stores their profile data and their record of the conversations -in which they participate. Each client is associated with a user account (and -may optionally support multiple user accounts). A user account is represented -by a unique "User ID". This ID is namespaced to the home server which allocated -the account and looks like:: +Each client is associated with a user account, which is identified in Matrix +using a unique "User ID". This ID is namespaced to the home server which +allocated the account and has the form:: @localpart:domain @@ -131,19 +164,14 @@ this user. They are case-insensitive. .. TODO-spec - Need to specify precise grammar for Matrix IDs -A "Home Server" is a server which provides C-S APIs and has the ability to -federate with other HSes. It is typically responsible for multiple clients. -"Federation" is the term used to describe the sharing of data between two or -more home servers. - Events ~~~~~~ -Data in Matrix is encapsulated in an "event". An event is an action within the -system. Typically each action (e.g. sending a message) correlates with exactly -one event. Each event has a ``type`` which is used to differentiate different -kinds of data. ``type`` values MUST be uniquely globally namespaced following -Java's `package naming conventions +All data exchanged over Matrix is expressed as an "event". Typically each client +action (e.g. sending a message) correlates with exactly one event. Each event +has a ``type`` which is used to differentiate different kinds of data. ``type`` +values MUST be uniquely globally namespaced following Java's `package naming +conventions `, e.g. ``com.example.myapp.event``. The special top-level namespace ``m.`` is reserved for events defined in the Matrix specification - for instance ``m.room.message`` @@ -153,36 +181,39 @@ of a "Room". Event Graphs ~~~~~~~~~~~~ -Each event has a list of zero or more `parent` events. These relations form -directed acyclic graphs of events called `event graphs`. Every event graph has a single root event, and each event graph forms the -basis of the history of a matrix room. - -Event graphs give a partial ordering of events, i.e. given two events one may -be considered to have come before the other if one is an ancestor of the other. -Since two events may be on separate branches, not all events can be compared in -this manner. - -Every event has a metadata `depth` field that is a positive integer that is -strictly greater than the depths of any of its parents. The root event should -have a depth of 1. - -[Note: if one event is before another, then it must have a strictly smaller -depth] +Events exchanged in the context of a room are stored in a directed acyclic graph +(DAG) called an ``event graph``. The partial ordering of this graph gives the +chronological ordering of events within the room. Each event in the graph has a +list of zero or more ``parent`` events, which refer to any preceeding events +which have no chronological successor from the perspective of the homeserver +which created the event. + +Typically an event has a single parent: the most recent message in the room at +the point it was sent. However, homeservers may legitimately race with each +other when sending messages, resulting in a single event having multiple +successors. The next event added to the graph thus will have multiple parents. +Every event graph has a single root event with no parent. + +To order and ease chronological comparison between the events within the graph, +homeservers maintain a ``depth`` metadata field on each event. An event's +``depth`` is a positive integer that is strictly greater than the depths of any +of its parents. The root event should have a depth of 1. Thus if one event is +before another, then it must have a strictly smaller depth. Room structure ~~~~~~~~~~~~~~ -A room is a conceptual place where users can send and receive events. -Events are sent to a room, and all participants in -that room with sufficient access will receive the event. Rooms are uniquely -identified internally via a "Room ID", which look like:: +A room is a conceptual place where users can send and receive events. Events are +sent to a room, and all participants in that room with sufficient access will +receive the event. Rooms are uniquely identified internally via "Room IDs", +which have the form:: !opaque_id:domain There is exactly one room ID for each room. Whilst the room ID does contain a domain, it is simply for globally namespacing room IDs. The room does NOT reside on the domain specified. Room IDs are not meant to be human readable. -They ARE case-sensitive. +They are case-sensitive. The following conceptual diagram shows an ``m.room.message`` event being sent to the room ``!qporfwt:matrix.org``:: @@ -190,6 +221,7 @@ the room ``!qporfwt:matrix.org``:: { @alice:matrix.org } { @bob:domain.com } | ^ | | + [HTTP POST] [HTTP GET] Room ID: !qporfwt:matrix.org Room ID: !qporfwt:matrix.org Event type: m.room.message Event type: m.room.message Content: { JSON object } Content: { JSON object } @@ -200,7 +232,7 @@ the room ``!qporfwt:matrix.org``:: | matrix.org | | domain.com | +------------------+ +------------------+ | ^ - | | + | [HTTP PUT] | | Room ID: !qporfwt:matrix.org | | Event type: m.room.message | | Content: { JSON object } | @@ -222,7 +254,7 @@ the room ``!qporfwt:matrix.org``:: | Content: { JSON object } | |...................................| -Federation maintains shared data structures per-room between multiple home +Federation maintains *shared data structures* per-room between multiple home servers. The data is split into ``message events`` and ``state events``. ``Message events`` describe transient 'once-off' activity in a room such as an @@ -252,7 +284,7 @@ participating in a room. Room Aliases ++++++++++++ -Each room can also have multiple "Room Aliases", which looks like:: +Each room can also have multiple "Room Aliases", which look like:: #room_alias:domain @@ -272,7 +304,7 @@ that are in the room that can be used to join via. :: - GET + HTTP GET #matrix:domain.com !aaabaa:matrix.org | ^ | | @@ -285,7 +317,7 @@ that are in the room that can be used to join via. |________________________________| Identity -++++++++ +~~~~~~~~ Users in Matrix are identified via their matrix user ID (MXID). However, existing 3rd party ID namespaces can also be used in order to identify Matrix @@ -306,47 +338,39 @@ the Matrix ecosystem. However, without one clients will not be able to look up user IDs using 3PIDs. Presence -++++++++ - -Each user has the concept of presence information. This encodes the -"availability" of that user, suitable for display on other user's clients. This -is transmitted as an ``m.presence`` event and is one of the few events which -are sent *outside the context of a room*. The basic piece of presence -information is represented by the ``presence`` key, which is an enum of one of -the following: - - - ``online`` : The default state when the user is connected to an event - stream. - - ``unavailable`` : The user is not reachable at this time. - - ``offline`` : The user is not connected to an event stream. - - ``free_for_chat`` : The user is generally willing to receive messages - moreso than default. - - ``hidden`` : Behaves as offline, but allows the user to see the client - state anyway and generally interact with client features. (Not yet - implemented in synapse). - -.. TODO-spec - This seems like a very constrained list of states - surely presence states - should be extensible, with us providing a baseline, and possibly a scale of - availability? For instance, do-not-disturb is missing here, as well as a - distinction between 'away' and 'busy'. +~~~~~~~~ -This basic ``presence`` field applies to the user as a whole, regardless of how -many client devices they have connected. The presence state is pushed by the homeserver to all connected clients for a user to ensure a consistent experience for the user. +Each user has the concept of presence information. This encodes: -.. TODO-spec - We need per-device presence in order to handle push notification semantics and similar. + * Whether the user is currently online + * How recently the user was last active (as seen by the server) + * Whether a given client considers the user to be currently idle + * Arbitrary information about the user's current status (e.g. "in a meeting"). + +This information is collated from both per-device (online; idle; last_active) and +per-user (status) data, aggregated by the user's homeserver and transmitted as +an ``m.presence`` event. This is one of the few events which are sent *outside +the context of a room*. Presence events are sent to all users who subscribe to +this user's presence through a presence list or by sharing membership of a room. -In addition, the server maintains a timestamp of the last time it saw a -pro-active event from the user; either sending a message to a room, or changing -presence state from a lower to a higher level of availability (thus: changing -state from ``unavailable`` to ``online`` counts as a proactive event, whereas in -the other direction it will not). This timestamp is presented via a key called -``last_active_ago``, which gives the relative number of milliseconds since the -message is generated/emitted that the user was last seen active. +.. TODO + How do we let users hide their presence information? -Presence List -~~~~~~~~~~~~~ +.. TODO + The last_active specifics should be moved to the detailed presence event section + +Last activity is tracked by the server maintaining a timestamp of the last time +it saw a pro-active event from the user. Any event which could be triggered by a +human using the application is considered pro-active (e.g. sending an event to a +room). An example of a non-proactive client activity would be a client setting +'idle' presence status, or polling for events. This timestamp is presented via a +key called ``last_active_ago``, which gives the relative number of milliseconds +since the message is generated/emitted that the user was last seen active. + +N.B. in v1 API, status/online/idle state are muxed into a single 'presence' field on the m.presence event. + +Presence Lists +~~~~~~~~~~~~~~ Each user's home server stores a "presence list". This stores a list of user IDs whose presence the user wants to follow. @@ -355,38 +379,31 @@ To be added to this list, the user being added must be invited by the list owner and accept the invitation. Once accepted, both user's HSes track the subscription. -Presence and Permissions -~~~~~~~~~~~~~~~~~~~~~~~~ - -For a viewing user to be allowed to see the presence information of a target -user, either: - - The target user has allowed the viewing user to add them to their presence - list, or - - The two users share at least one room in common - -In the latter case, this allows for clients to display some minimal sense of -presence information in a user list for a room. +Profiles +~~~~~~~~ +Users may publish arbitrary key/value data associated with their account - such +as a human readable ``display name``, a profile photo URL, contact information +(email address, phone nubers, website URLs etc). -Profiles -++++++++ +In Client-Server API v2, profile data is typed using namespaced keys for +interoperability, much like events - e.g. ``m.profile.display_name``. -.. TODO-spec - - Metadata extensibility +.. TODO + Actually specify the different types of data - e.g. what format are display + names allowed to be? -Internally within Matrix users are referred to by their user ID, which is -typically a compact unique identifier. Profiles grant users the ability to see -human-readable names for other users that are in some way meaningful to them. -Additionally, profiles can publish additional information, such as the user's -age or location. +Private User Data +~~~~~~~~~~~~~~~~~ -A Profile consists of a display name, an avatar picture, and a set of other -metadata fields that the user may wish to publish (email address, phone -numbers, website URLs, etc...). This specification puts no requirements on the -display name other than it being a valid unicode string. Avatar images are not -stored directly; instead the home server stores an ``http``-scheme URL from which clients may fetch the image. +Users may also store arbitrary private key/value data in their account - such as +client preferences, or server configuration settings which lack any other +dedicated API. The API is symmetrical to managing Profile data. +.. TODO + Would it really be overengineered to use the same API for both profile & + private user data, but with different ACLs? API Standards ------------- @@ -424,8 +441,9 @@ response". This is a JSON object which looks like:: The ``error`` string will be a human-readable error message, usually a sentence explaining what went wrong. The ``errcode`` string will be a unique string which can be used to handle an error message e.g. ``M_FORBIDDEN``. These error -codes should have their namespace first in ALL CAPS, followed by a single _. -For example, if there was a custom namespace ``com.mydomain.here``, and a +codes should have their namespace first in ALL CAPS, followed by a single _ to +ease seperating the namespace from the error code.. For example, if there was a +custom namespace ``com.mydomain.here``, and a ``FORBIDDEN`` code, the error code should look like ``COM.MYDOMAIN.HERE_FORBIDDEN``. There may be additional keys depending on the error, but the keys ``error`` and ``errcode`` MUST always be present. @@ -502,81 +520,3 @@ In contrast, these are invalid requests:: "key": "This is a put but it is missing a txnId." } -Glossary --------- - -Backfilling: - The process of synchronising historic state from one home server to another, - to backfill the event storage so that scrollback can be presented to the - client(s). Not to be confused with pagination. - -Context: - A single human-level entity of interest (currently, a chat room) - -EDU (Ephemeral Data Unit): - A message that relates directly to a given pair of home servers that are - exchanging it. EDUs are short-lived messages that related only to one single - pair of servers; they are not persisted for a long time and are not forwarded - on to other servers. Because of this, they have no internal ID nor previous - EDUs reference chain. - -Event: - A record of activity that records a single thing that happened on to a context - (currently, a chat room). These are the "chat messages" that Synapse makes - available. - -PDU (Persistent Data Unit): - A message that relates to a single context, irrespective of the server that - is communicating it. PDUs either encode a single Event, or a single State - change. A PDU is referred to by its PDU ID; the pair of its origin server - and local reference from that server. - -PDU ID: - The pair of PDU Origin and PDU Reference, that together globally uniquely - refers to a specific PDU. - -PDU Origin: - The name of the origin server that generated a given PDU. This may not be the - server from which it has been received, due to the way they are copied around - from server to server. The origin always records the original server that - created it. - -PDU Reference: - A local ID used to refer to a specific PDU from a given origin server. These - references are opaque at the protocol level, but may optionally have some - structured meaning within a given origin server or implementation. - -Presence: - The concept of whether a user is currently online, how available they declare - they are, and so on. See also: doc/model/presence - -Profile: - A set of metadata about a user, such as a display name, provided for the - benefit of other users. See also: doc/model/profiles - -Room ID: - An opaque string (of as-yet undecided format) that identifies a particular - room and used in PDUs referring to it. - -Room Alias: - A human-readable string of the form #name:some.domain that users can use as a - pointer to identify a room; a Directory Server will map this to its Room ID - -State: - A set of metadata maintained about a Context, which is replicated among the - servers in addition to the history of Events. - -User ID: - A string of the form @localpart:domain.name that identifies a user for - wire-protocol purposes. The localpart is meaningless outside of a particular - home server. This takes a human-readable form that end-users can use directly - if they so wish, avoiding the 3PIDs. - -Transaction: - A message which relates to the communication between a given pair of servers. - A transaction contains possibly-empty lists of PDUs and EDUs. - -.. TODO - This glossary contradicts the terms used above - especially on State Events v. "State" - and Non-State Events v. "Events". We need better consistent names. - diff --git a/specification/20_client_server_api.rst b/specification/10_client_server_api.rst similarity index 80% rename from specification/20_client_server_api.rst rename to specification/10_client_server_api.rst index af071d6f..5129af16 100644 --- a/specification/20_client_server_api.rst +++ b/specification/10_client_server_api.rst @@ -1,377 +1,436 @@ Client-Server API v1 ==================== -This outlines version 1 of the client-server API. - -Registration and Login ----------------------- - -Clients must register with a home server in order to use Matrix. After -registering, the client will be given an access token which must be used in ALL -requests to that home server as a query parameter 'access_token'. - -If the client has already registered, they need to be able to login to their -account. The home server may provide many different ways of logging in, such as -user/password auth, login via a social network (OAuth2), login by confirming a -token sent to their email address, etc. This specification does not define how -home servers should authorise their users who want to login to their existing -accounts, but instead defines the standard interface which implementations -should follow so that ANY client can login to ANY home server. Clients login -using the |login|_ API. Clients register using the |register|_ API. -Registration follows the same general procedure as login, but the path requests -are sent to and the details contained in them are different. - -In both registration and login cases, the process takes the form of one or more -stages, where at each stage the client submits a set of data for a given stage -type and awaits a response from the server, which will either be a final -success or a request to perform an additional stage. This exchange continues -until the final success. - -In order to determine up-front what the server's requirements are, the client -can request from the server a complete description of all of its acceptable -flows of the registration or login process. It can then inspect the list of -returned flows looking for one for which it believes it can complete all of the -required stages, and perform it. As each home server may have different ways of -logging in, the client needs to know how they should login. All distinct login -stages MUST have a corresponding ``type``. A ``type`` is a namespaced string -which details the mechanism for logging in. - -A client may be able to login via multiple valid login flows, and should choose -a single flow when logging in. A flow is a series of login stages. The home -server MUST respond with all the valid login flows when requested by a simple -``GET`` request directly to the ``/login`` or ``/register`` paths:: - - { - "flows": [ - { - "type": "", - "stages": [ "", "" ] - }, - { - "type": "", - "stages": [ "", "" ] - }, - { - "type": "" - } - ] - } +Overview +-------- -The client can now select which flow it wishes to use, and begin making -``POST`` requests to the ``/login`` or ``/register`` paths with JSON body -content containing the name of the stage as the ``type`` key, along with -whatever additional parameters are required for that login or registration type -(see below). After the flow is completed, the client's fully-qualified user -ID and a new access token MUST be returned:: +The client-server API provides a simple lightweight API to let clients send +messages, control rooms and synchronise conversation history. It is designed to +support both lightweight clients which store no state and lazy-load data from +the server as required - as well as heavyweight clients which maintain a full +local peristent copy of server state. + +This describes v1 of the Client-Server API as featured in the original September +2014 launch of Matrix. Version 2 is currently in development (as of Jan-March +2015) as an incremental but backwards-incompatible refinement of Version 1 and +will be released shortly. + +Pagination +---------- + +Querying large datasets in Matrix always uses the same pagination API pattern to +to give clients a consistent way of selecting subsets of a potentially changing +dataset. Requests pass in ``from``, ``to`` and ``limit`` parameters which describe +where to read from the stream. ``from`` and ``to`` are opaque textual 'stream +tokens' which describe positions in the dataset. The response returns new +``start`` and ``end`` stream token values which can then be passed to subsequent +requests to continue pagination. + +Pagination Request Query Parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Query parameters: + from: + $streamtoken - The opaque token to start streaming from. + to: + $streamtoken - The opaque token to end streaming at. Typically, + clients will not know the item of data to end at, so this will usually be + omitted. + limit: + integer - An integer representing the maximum number of items to + return. + +'START' and 'END' are placeholder values used in these examples to describe the +start and end of the dataset respectively. + +Unless specified, the default pagination parameters are from=START, to=END, +without a limit set. This allows you to hit an API like +/events without any query parameters to get everything. + +For example, the event stream has events E1 -> E15. The client wants the last 5 +events and doesn't know any previous events:: + + S E + |-E1-E2-E3-E4-E5-E6-E7-E8-E9-E10-E11-E12-E13-E14-E15-| + | | | + | _____| | + |__________________ | ___________________| + | | | + GET /events?to=START&limit=5&from=END + Returns: + E15,E14,E13,E12,E11 + + +Another example: a public room list has rooms R1 -> R17. The client is showing 5 +rooms at a time on screen, and is on page 2. They want to +now show page 3 (rooms R11 -> 15):: + + S E + | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | stream token + |-R1-R2-R3-R4-R5-R6-R7-R8-R9-R10-R11-R12-R13-R14-R15-R16-R17| room + |____________| |________________| + | | + Currently | + viewing | + | + GET /rooms/list?from=9&to=END&limit=5 + Returns: R11,R12,R13,R14,R15 + +Note that tokens are treated in an *exclusive*, not inclusive, manner. The end +token from the intial request was '9' which corresponded to R10. When the 2nd +request was made, R10 did not appear again, even though from=9 was specified. If +you know the token, you already have the data. + +Pagination Response +~~~~~~~~~~~~~~~~~~~ + +Responses to pagination requests MUST follow the format:: { - "user_id": "@user:matrix.org", - "access_token": "abcdef0123456789" + "chunk": [ ... , Responses , ... ], + "start" : $streamtoken, + "end" : $streamtoken } -The ``user_id`` key is particularly useful if the home server wishes to support -localpart entry of usernames (e.g. "user" rather than "@user:matrix.org"), as -the client may not be able to determine its ``user_id`` in this case. +Where $streamtoken is an opaque token which can be used in another query to +get the next set of results. The "start" and "end" keys can only be omitted if +the complete dataset is provided in "chunk". -If the flow has multiple stages to it, the home server may wish to create a -session to store context between requests. If a home server responds with a -``session`` key to a request, clients MUST submit it in subsequent requests -until the flow is completed:: +Events +------ - { - "session": "" - } +Overview +~~~~~~~~ -This specification defines the following login types: - - ``m.login.password`` - - ``m.login.oauth2`` - - ``m.login.email.code`` - - ``m.login.email.url`` - - ``m.login.email.identity`` +The model of conversation history exposed by the client-server API can be +considered as a list of events. The server 'linearises' the +eventually-consistent event graph of events into an 'event stream' at any given +point in time:: + + [E0]->[E1]->[E2]->[E3]->[E4]->[E5]->[E6]->[E7]->[E8]->[E9] + +Clients can add to the stream by POSTing message or state events, and can read +from the stream via the |initialSync|_, |/rooms//initialSync|_, `Event +Stream`_ and |/rooms//messages|_ APIs. + +For reading events, the intended flow of operation is to call +$PREFIX/initialSync, which returns all of the state and the last N events in the +event stream for each room, including ``start`` and ``end`` values describing the +pagination of each room's event stream. For instance, +$PREFIX/initialSync?limit=5 might return the events for a room in the +rooms[0].messages.chunk[] array, with tokens describing the start and end of the +range in rooms[0].messages.start as '1-2-3' and rooms[0].messages.end as +'a-b-c'. + +You can visualise the range of events being returned as:: + + [E0]->[E1]->[E2]->[E3]->[E4]->[E5]->[E6]->[E7]->[E8]->[E9] + ^ ^ + | | + start: '1-2-3' end: 'a-b-c' + +Now, to receive future events in realtime on the eventstream, you simply GET +$PREFIX/events with a ``from`` parameter of 'a-b-c': in other words passing in the +``end`` token returned by initialsync. The request blocks until new events are +available or until your specified timeout elapses, and then returns a +new paginatable chunk of events alongside new start and end parameters:: + + [E0]->[E1]->[E2]->[E3]->[E4]->[E5]->[E6]->[E7]->[E8]->[E9]->[E10] + ^ ^ + | | + | end: 'x-y-z' + start: 'a-b-c' + +To resume polling the events stream, you pass in the new ``end`` token as the +``from`` parameter of $PREFIX/events and poll again. + +Similarly, to paginate events backwards in order to lazy-load in previous +history from the room, you simply GET $PREFIX/rooms//messages +specifying the ``from`` token to paginate backwards from and a limit of the number +of messages to retrieve. For instance, calling this API with a ``from`` parameter +of '1-2-3' and a limit of 5 would return: + + [E0]->[E1]->[E2]->[E3]->[E4]->[E5]->[E6]->[E7]->[E8]->[E9]->[E10] + ^ ^ + | | + start: 'u-v-w' end: '1-2-3' + +To continue paginating backwards, one calls the /messages API again, supplying +the new ``start`` value as the ``from`` parameter. -Password-based -~~~~~~~~~~~~~~ -:Type: - ``m.login.password`` -:Description: - Login is supported via a username and password. -To respond to this type, reply with:: +Receiving live updates on a client +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - { - "type": "m.login.password", - "user": "", - "password": "" - } +Clients receive new events by long-polling the home server via the +$PREFIX/events API, specifying a timeout in milliseconds in the timeout +parameter. This will hold open the HTTP connection for a short period of time +waiting for new events, returning early if an event occurs. This is called the +`Event Stream`_. All events which are visible to the client will appear in the +event stream. When the request returns, an ``end`` token is included in the +response. This token can be used in the next request to continue where the +last request left off. -The home server MUST respond with either new credentials, the next stage of the -login process, or a standard error response. +All events must be deduplicated based on their event ID. -Captcha-based -~~~~~~~~~~~~~ -:Type: - ``m.login.recaptcha`` -:Description: - Login is supported by responding to a captcha (in the case of the Synapse - implementation, Google's Recaptcha library is used). +.. TODO + is deduplication actually a hard requirement in CS v2? -To respond to this type, reply with:: +.. TODO-spec + Do we ever return multiple events in a single request? + Don't we get lots of request setup RTT latency if we only do one event per request? + Do we ever support streaming requests? Why not websockets? - { - "type": "m.login.recaptcha", - "challenge": "", - "response": "" - } +When the client first logs in, they will need to initially synchronise with +their home server. This is achieved via the |initialSync|_ API. This API also +returns an ``end`` token which can be used with the event stream. See the 'Room Sync' section below. -.. NOTE:: - In Synapse, the Recaptcha parameters can be obtained in Javascript by calling: - Recaptcha.get_challenge(); - Recaptcha.get_response(); +Events in a room +~~~~~~~~~~~~~~~~ -The home server MUST respond with either new credentials, the next stage of the -login process, or a standard error response. +Room events are split into two categories: -OAuth2-based -~~~~~~~~~~~~ -:Type: - ``m.login.oauth2`` -:Description: - Login is supported via OAuth2 URLs. This login consists of multiple requests. +:Message events: + These are events which describe transient "once-off" activity in a room: + typically communication such as sending an instant messaage or setting up a + VoIP call. These used to be called 'non-state' events. -To respond to this type, reply with:: +:State Events: + These are events which update the metadata state of the room (e.g. room topic, + room membership etc). State is keyed by a tuple of event ``type`` and a + ``state_key``. State in the room with the same key-tuple will be overwritten. - { - "type": "m.login.oauth2", - "user": "" - } +This specification outlines several events, all with the event type prefix +``m.``. However, applications may wish to add their own type of event, and this +can be achieved using the REST API detailed in the following sections. If new +events are added, the event ``type`` key SHOULD follow the Java package naming +convention, e.g. ``com.example.myapp.event``. This ensures event types are +suitably namespaced for each application and reduces the risk of clashes. -The server MUST respond with:: +State events +~~~~~~~~~~~~ - { - "uri": - } +State events can be sent by ``PUT`` ing to +|/rooms//state//|_. These events will be +overwritten if ````, ```` and ```` all match. +If the state event has no ``state_key``, it can be omitted from the path. These +requests **cannot use transaction IDs** like other ``PUT`` paths because they +cannot be differentiated from the ``state_key``. Furthermore, ``POST`` is +unsupported on state paths. Valid requests look like:: -The home server acts as a 'confidential' client for the purposes of OAuth2. If -the uri is a ``sevice selection URI``, it MUST point to a webpage which prompts -the user to choose which service to authorize with. On selection of a service, -this MUST link through to an ``Authorization Request URI``. If there is only 1 -service which the home server accepts when logging in, this indirection can be -skipped and the "uri" key can be the ``Authorization Request URI``. + PUT /rooms/!roomid:domain/state/m.example.event + { "key" : "without a state key" } -The client then visits the ``Authorization Request URI``, which then shows the -OAuth2 Allow/Deny prompt. Hitting 'Allow' returns the ``redirect URI`` with the -auth code. Home servers can choose any path for the ``redirect URI``. The -client should visit the ``redirect URI``, which will then finish the OAuth2 -login process, granting the home server an access token for the chosen service. -When the home server gets this access token, it verifies that the cilent has -authorised with the 3rd party, and can now complete the login. The OAuth2 -``redirect URI`` (with auth code) MUST respond with either new credentials, the -next stage of the login process, or a standard error response. + PUT /rooms/!roomid:domain/state/m.another.example.event/foo + { "key" : "with 'foo' as the state key" } -For example, if a home server accepts OAuth2 from Google, it would return the -Authorization Request URI for Google:: +In contrast, these requests are invalid:: - { - "uri": "https://accounts.google.com/o/oauth2/auth?response_type=code& - client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos" - } + POST /rooms/!roomid:domain/state/m.example.event/ + { "key" : "cannot use POST here" } -The client then visits this URI and authorizes the home server. The client then -visits the REDIRECT_URI with the auth code= query parameter which returns:: + PUT /rooms/!roomid:domain/state/m.another.example.event/foo/11 + { "key" : "txnIds are not supported" } - { - "user_id": "@user:matrix.org", - "access_token": "0123456789abcdef" - } +Care should be taken to avoid setting the wrong ``state key``:: -Email-based (code) -~~~~~~~~~~~~~~~~~~ -:Type: - ``m.login.email.code`` -:Description: - Login is supported by typing in a code which is sent in an email. This login - consists of multiple requests. + PUT /rooms/!roomid:domain/state/m.another.example.event/11 + { "key" : "with '11' as the state key, but was probably intended to be a txnId" } -To respond to this type, reply with:: +The ``state_key`` is often used to store state about individual users, by using +the user ID as the ``state_key`` value. For example:: - { - "type": "m.login.email.code", - "user": "", - "email": "" - } + PUT /rooms/!roomid:domain/state/m.favorite.animal.event/%40my_user%3Adomain.com + { "animal" : "cat", "reason": "fluffy" } -After validating the email address, the home server MUST send an email -containing an authentication code and return:: +In some cases, there may be no need for a ``state_key``, so it can be omitted:: - { - "type": "m.login.email.code", - "session": "" - } + PUT /rooms/!roomid:domain/state/m.room.bgd.color + { "color": "red", "hex": "#ff0000" } -The second request in this login stage involves sending this authentication -code:: +See `Room Events`_ for the ``m.`` event specification. - { - "type": "m.login.email.code", - "session": "", - "code": "" - } +Message events +~~~~~~~~~~~~~~ -The home server MUST respond to this with either new credentials, the next -stage of the login process, or a standard error response. +Message events can be sent by sending a request to +|/rooms//send/|_. These requests *can* use transaction +IDs and ``PUT``/``POST`` methods. Message events allow access to historical +events and pagination, making it best suited for sending messages. For +example:: -Email-based (url) -~~~~~~~~~~~~~~~~~ -:Type: - ``m.login.email.url`` -:Description: - Login is supported by clicking on a URL in an email. This login consists of - multiple requests. + POST /rooms/!roomid:domain/send/m.custom.example.message + { "text": "Hello world!" } -To respond to this type, reply with:: + PUT /rooms/!roomid:domain/send/m.custom.example.message/11 + { "text": "Goodbye world!" } - { - "type": "m.login.email.url", - "user": "", - "email": "" - } +See `Room Events`_ for the ``m.`` event specification. -After validating the email address, the home server MUST send an email -containing an authentication URL and return:: +Syncing rooms +~~~~~~~~~~~~~ - { - "type": "m.login.email.url", - "session": "" - } +.. NOTE:: + This section is a work in progress. -The email contains a URL which must be clicked. After it has been clicked, the -client should perform another request:: +When a client logs in, they may have a list of rooms which they have already +joined. These rooms may also have a list of events associated with them. The +purpose of 'syncing' is to present the current room and event information in a +convenient, compact manner. The events returned are not limited to room events; +presence events will also be returned. A single syncing API is provided: - { - "type": "m.login.email.url", - "session": "" - } + - |initialSync|_ : A global sync which will present room and event information + for all rooms the user has joined. -The home server MUST respond to this with either new credentials, the next -stage of the login process, or a standard error response. +.. TODO-spec room-scoped initial sync + - |/rooms//initialSync|_ : A sync scoped to a single room. Presents + room and event information for this room only. + - Room-scoped initial sync is Very Tricky because typically people would + want to sync the room then listen for any new content from that point + onwards. The event stream cannot do this for a single room currently. + As a result, commenting room-scoped initial sync at this time. -A common client implementation will be to periodically poll until the link is -clicked. If the link has not been visited yet, a standard error response with -an errcode of ``M_LOGIN_EMAIL_URL_NOT_YET`` should be returned. +The |initialSync|_ API contains the following keys: +``presence`` + Description: + Contains a list of presence information for users the client is interested + in. + Format: + A JSON array of ``m.presence`` events. -Email-based (identity server) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -:Type: - ``m.login.email.identity`` -:Description: - Login is supported by authorising an email address with an identity server. +``end`` + Description: + Contains an event stream token which can be used with the `Event Stream`_. + Format: + A string containing the event stream token. -Prior to submitting this, the client should authenticate with an identity -server. After authenticating, the session information should be submitted to -the home server. +``rooms`` + Description: + Contains a list of room information for all rooms the client has joined, + and limited room information on rooms the client has been invited to. + Format: + A JSON array containing Room Information JSON objects. -To respond to this type, reply with:: +Room Information: + Description: + Contains all state events for the room, along with a limited amount of + the most recent events, configured via the ``limit`` query + parameter. Also contains additional keys with room metadata, such as the + ``room_id`` and the client's ``membership`` to the room. + Format: + A JSON object with the following keys: + ``room_id`` + A string containing the ID of the room being described. + ``membership`` + A string representing the client's membership status in this room. + ``messages`` + An event stream JSON object containing a ``chunk`` of recent + events (both state events and non-state events), along with an ``end`` token. + ``state`` + A JSON array containing all the current state events for this room. - { - "type": "m.login.email.identity", - "threepidCreds": [ - { - "sid": "", - "clientSecret": "", - "idServer": "" - } - ] - } +Getting events for a room +~~~~~~~~~~~~~~~~~~~~~~~~~ +There are several APIs provided to ``GET`` events for a room: +``/rooms//state//`` + Description: + Get the state event identified. + Response format: + A JSON object representing the state event **content**. + Example: + ``/rooms/!room:domain.com/state/m.room.name`` returns ``{ "name": "Room name" }`` -N-Factor Authentication -~~~~~~~~~~~~~~~~~~~~~~~ -Multiple login stages can be combined to create N-factor authentication during -login. +|/rooms//state|_ + Description: + Get all state events for a room. + Response format: + ``[ { state event }, { state event }, ... ]`` + Example: + TODO-doc -This can be achieved by responding with the ``next`` login type on completion -of a previous login stage:: +|/rooms//members|_ + Description: + Get all ``m.room.member`` state events. + Response format: + ``{ "start": "", "end": "", "chunk": [ { m.room.member event }, ... ] }`` + Example: + TODO-doc - { - "next": "" - } +|/rooms//messages|_ + Description: + Get all events from the room's timeline. This API supports + pagination using ``from`` and ``to`` query parameters, coupled with the + ``start`` and ``end`` tokens from an |initialSync|_ API. + + Response format: + ``{ "start": "", "end": "" }`` + Example: + TODO-doc -If a home server implements N-factor authentication, it MUST respond with all -``stages`` when initially queried for their login requirements:: +|/rooms//initialSync|_ + Description: + Get all relevant events for a room. This includes state events, paginated + non-state events and presence events. + Response format: + `` { TODO-doc } `` + Example: + TODO-doc - { - "type": "<1st login type>", - "stages": [ <1st login type>, <2nd login type>, ... , ] - } +Redactions +~~~~~~~~~~ +Since events are extensible it is possible for malicious users and/or servers +to add keys that are, for example offensive or illegal. Since some events +cannot be simply deleted, e.g. membership events, we instead 'redact' events. +This involves removing all keys from an event that are not required by the +protocol. This stripped down event is thereafter returned anytime a client or +remote server requests it. -This can be represented conceptually as:: +Events that have been redacted include a ``redacted_because`` key whose value +is the event that caused it to be redacted, which may include a reason. - _______________________ - | Login Stage 1 | - | type: "" | - | ___________________ | - | |_Request_1_________| | <-- Returns "session" key which is used throughout. - | ___________________ | - | |_Request_2_________| | <-- Returns a "next" value of "login type2" - |_______________________| - | - | - _________V_____________ - | Login Stage 2 | - | type: "" | - | ___________________ | - | |_Request_1_________| | - | ___________________ | - | |_Request_2_________| | - | ___________________ | - | |_Request_3_________| | <-- Returns a "next" value of "login type3" - |_______________________| - | - | - _________V_____________ - | Login Stage 3 | - | type: "" | - | ___________________ | - | |_Request_1_________| | <-- Returns user credentials - |_______________________| +Redacting an event cannot be undone, allowing server owners to delete the +offending content from the databases. -Fallback -~~~~~~~~ -Clients cannot be expected to be able to know how to process every single login -type. If a client determines it does not know how to handle a given login type, -it should request a login fallback page:: +.. TODO + Currently, only room admins can redact events by sending a ``m.room.redaction`` + event, but server admins also need to be able to redact events by a similar + mechanism. - GET matrix/client/api/v1/login/fallback +Upon receipt of a redaction event, the server should strip off any keys not in +the following list: -This MUST return an HTML page which can perform the entire login process. + - ``event_id`` + - ``type`` + - ``room_id`` + - ``user_id`` + - ``state_key`` + - ``prev_state`` + - ``content`` -Events ------- +The content object should also be stripped of all keys, unless it is one of +one of the following event types: -Receiving live updates on a client -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + - ``m.room.member`` allows key ``membership`` + - ``m.room.create`` allows key ``creator`` + - ``m.room.join_rules`` allows key ``join_rule`` + - ``m.room.power_levels`` allows keys that are user ids or ``default`` + - ``m.room.add_state_level`` allows key ``level`` + - ``m.room.send_event_level`` allows key ``level`` + - ``m.room.ops_levels`` allows keys ``kick_level``, ``ban_level`` + and ``redact_level`` + - ``m.room.aliases`` allows key ``aliases`` -Clients can receive new events by long-polling the home server. This will hold -open the HTTP connection for a short period of time waiting for new events, -returning early if an event occurs. This is called the `Event Stream`_. All -events which are visible to the client will appear in the event stream. When -the request returns, an ``end`` token is included in the response. This token -can be used in the next request to continue where the client left off. +.. TODO + Need to update m.room.power_levels to reflect new power levels formatting -All events must be deduplicated based on their event ID (TODO: is this actually a -hard requirement in CS v2?) +The redaction event should be added under the key ``redacted_because``. -.. TODO-spec - How do we filter the event stream? - Do we ever return multiple events in a single request? Don't we get lots of request - setup RTT latency if we only do one event per request? Do we ever support streaming - requests? Why not websockets? -When the client first logs in, they will need to initially synchronise with -their home server. This is achieved via the |initialSync|_ API. This API also -returns an ``end`` token which can be used with the event stream. +When a client receives a redaction event it should change the redacted event +in the same way a server does. Rooms @@ -687,247 +746,351 @@ member's state, by making a request to "membership": "ban" } -Events in a room -~~~~~~~~~~~~~~~~ -Room events can be split into two categories: -:State Events: - These are events which replace events that came before it, depending on a set - of unique keys. These keys are the event ``type`` and a ``state_key``. - Events with the same set of keys will be overwritten. Typically, state events - are used to store state, hence their name. - -:Non-state events: - These are events which cannot be overwritten after sending. The list of - events continues to grow as more events are sent. As this list grows, it - becomes necessary to provide a mechanism for navigating this list. Pagination - APIs are used to view the list of historical non-state events. Typically, - non-state events are used to send messages. +Registration and Login +---------------------- -This specification outlines several events, all with the event type prefix -``m.``. However, applications may wish to add their own type of event, and this -can be achieved using the REST API detailed in the following sections. If new -events are added, the event ``type`` key SHOULD follow the Java package naming -convention, e.g. ``com.example.myapp.event``. This ensures event types are -suitably namespaced for each application and reduces the risk of clashes. +Clients must register with a home server in order to use Matrix. After +registering, the client will be given an access token which must be used in ALL +requests to that home server as a query parameter 'access_token'. -State events +If the client has already registered, they need to be able to login to their +account. The home server may provide many different ways of logging in, such as +user/password auth, login via a social network (OAuth2), login by confirming a +token sent to their email address, etc. This specification does not define how +home servers should authorise their users who want to login to their existing +accounts, but instead defines the standard interface which implementations +should follow so that ANY client can login to ANY home server. Clients login +using the |login|_ API. Clients register using the |register|_ API. +Registration follows the same general procedure as login, but the path requests +are sent to and the details contained in them are different. + +In both registration and login cases, the process takes the form of one or more +stages, where at each stage the client submits a set of data for a given stage +type and awaits a response from the server, which will either be a final +success or a request to perform an additional stage. This exchange continues +until the final success. + +In order to determine up-front what the server's requirements are, the client +can request from the server a complete description of all of its acceptable +flows of the registration or login process. It can then inspect the list of +returned flows looking for one for which it believes it can complete all of the +required stages, and perform it. As each home server may have different ways of +logging in, the client needs to know how they should login. All distinct login +stages MUST have a corresponding ``type``. A ``type`` is a namespaced string +which details the mechanism for logging in. + +A client may be able to login via multiple valid login flows, and should choose +a single flow when logging in. A flow is a series of login stages. The home +server MUST respond with all the valid login flows when requested by a simple +``GET`` request directly to the ``/login`` or ``/register`` paths:: + + { + "flows": [ + { + "type": "", + "stages": [ "", "" ] + }, + { + "type": "", + "stages": [ "", "" ] + }, + { + "type": "" + } + ] + } + +The client can now select which flow it wishes to use, and begin making +``POST`` requests to the ``/login`` or ``/register`` paths with JSON body +content containing the name of the stage as the ``type`` key, along with +whatever additional parameters are required for that login or registration type +(see below). After the flow is completed, the client's fully-qualified user +ID and a new access token MUST be returned:: + + { + "user_id": "@user:matrix.org", + "access_token": "abcdef0123456789" + } + +The ``user_id`` key is particularly useful if the home server wishes to support +localpart entry of usernames (e.g. "user" rather than "@user:matrix.org"), as +the client may not be able to determine its ``user_id`` in this case. + +If the flow has multiple stages to it, the home server may wish to create a +session to store context between requests. If a home server responds with a +``session`` key to a request, clients MUST submit it in subsequent requests +until the flow is completed:: + + { + "session": "" + } + +This specification defines the following login types: + - ``m.login.password`` + - ``m.login.oauth2`` + - ``m.login.email.code`` + - ``m.login.email.url`` + - ``m.login.email.identity`` + +Password-based +~~~~~~~~~~~~~~ +:Type: + ``m.login.password`` +:Description: + Login is supported via a username and password. + +To respond to this type, reply with:: + + { + "type": "m.login.password", + "user": "", + "password": "" + } + +The home server MUST respond with either new credentials, the next stage of the +login process, or a standard error response. + +Captcha-based +~~~~~~~~~~~~~ +:Type: + ``m.login.recaptcha`` +:Description: + Login is supported by responding to a captcha (in the case of the Synapse + implementation, Google's Recaptcha library is used). + +To respond to this type, reply with:: + + { + "type": "m.login.recaptcha", + "challenge": "", + "response": "" + } + +.. NOTE:: + In Synapse, the Recaptcha parameters can be obtained in Javascript by calling: + Recaptcha.get_challenge(); + Recaptcha.get_response(); + +The home server MUST respond with either new credentials, the next stage of the +login process, or a standard error response. + +OAuth2-based ~~~~~~~~~~~~ -State events can be sent by ``PUT`` ing to -|/rooms//state//|_. These events will be -overwritten if ````, ```` and ```` all match. -If the state event has no ``state_key``, it can be omitted from the path. These -requests **cannot use transaction IDs** like other ``PUT`` paths because they -cannot be differentiated from the ``state_key``. Furthermore, ``POST`` is -unsupported on state paths. Valid requests look like:: +:Type: + ``m.login.oauth2`` +:Description: + Login is supported via OAuth2 URLs. This login consists of multiple requests. - PUT /rooms/!roomid:domain/state/m.example.event - { "key" : "without a state key" } +To respond to this type, reply with:: - PUT /rooms/!roomid:domain/state/m.another.example.event/foo - { "key" : "with 'foo' as the state key" } + { + "type": "m.login.oauth2", + "user": "" + } -In contrast, these requests are invalid:: +The server MUST respond with:: - POST /rooms/!roomid:domain/state/m.example.event/ - { "key" : "cannot use POST here" } + { + "uri": + } - PUT /rooms/!roomid:domain/state/m.another.example.event/foo/11 - { "key" : "txnIds are not supported" } +The home server acts as a 'confidential' client for the purposes of OAuth2. If +the uri is a ``sevice selection URI``, it MUST point to a webpage which prompts +the user to choose which service to authorize with. On selection of a service, +this MUST link through to an ``Authorization Request URI``. If there is only 1 +service which the home server accepts when logging in, this indirection can be +skipped and the "uri" key can be the ``Authorization Request URI``. -Care should be taken to avoid setting the wrong ``state key``:: +The client then visits the ``Authorization Request URI``, which then shows the +OAuth2 Allow/Deny prompt. Hitting 'Allow' returns the ``redirect URI`` with the +auth code. Home servers can choose any path for the ``redirect URI``. The +client should visit the ``redirect URI``, which will then finish the OAuth2 +login process, granting the home server an access token for the chosen service. +When the home server gets this access token, it verifies that the cilent has +authorised with the 3rd party, and can now complete the login. The OAuth2 +``redirect URI`` (with auth code) MUST respond with either new credentials, the +next stage of the login process, or a standard error response. - PUT /rooms/!roomid:domain/state/m.another.example.event/11 - { "key" : "with '11' as the state key, but was probably intended to be a txnId" } +For example, if a home server accepts OAuth2 from Google, it would return the +Authorization Request URI for Google:: -The ``state_key`` is often used to store state about individual users, by using -the user ID as the ``state_key`` value. For example:: + { + "uri": "https://accounts.google.com/o/oauth2/auth?response_type=code& + client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos" + } - PUT /rooms/!roomid:domain/state/m.favorite.animal.event/%40my_user%3Adomain.com - { "animal" : "cat", "reason": "fluffy" } +The client then visits this URI and authorizes the home server. The client then +visits the REDIRECT_URI with the auth code= query parameter which returns:: -In some cases, there may be no need for a ``state_key``, so it can be omitted:: + { + "user_id": "@user:matrix.org", + "access_token": "0123456789abcdef" + } - PUT /rooms/!roomid:domain/state/m.room.bgd.color - { "color": "red", "hex": "#ff0000" } +Email-based (code) +~~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.code`` +:Description: + Login is supported by typing in a code which is sent in an email. This login + consists of multiple requests. -See `Room Events`_ for the ``m.`` event specification. +To respond to this type, reply with:: -Non-state events -~~~~~~~~~~~~~~~~ -Non-state events can be sent by sending a request to -|/rooms//send/|_. These requests *can* use transaction -IDs and ``PUT``/``POST`` methods. Non-state events allow access to historical -events and pagination, making it best suited for sending messages. For -example:: + { + "type": "m.login.email.code", + "user": "", + "email": "" + } + +After validating the email address, the home server MUST send an email +containing an authentication code and return:: + + { + "type": "m.login.email.code", + "session": "" + } - POST /rooms/!roomid:domain/send/m.custom.example.message - { "text": "Hello world!" } +The second request in this login stage involves sending this authentication +code:: - PUT /rooms/!roomid:domain/send/m.custom.example.message/11 - { "text": "Goodbye world!" } + { + "type": "m.login.email.code", + "session": "", + "code": "" + } -See `Room Events`_ for the ``m.`` event specification. +The home server MUST respond to this with either new credentials, the next +stage of the login process, or a standard error response. -Syncing rooms -~~~~~~~~~~~~~ -.. NOTE:: - This section is a work in progress. +Email-based (url) +~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.url`` +:Description: + Login is supported by clicking on a URL in an email. This login consists of + multiple requests. -When a client logs in, they may have a list of rooms which they have already -joined. These rooms may also have a list of events associated with them. The -purpose of 'syncing' is to present the current room and event information in a -convenient, compact manner. The events returned are not limited to room events; -presence events will also be returned. A single syncing API is provided: +To respond to this type, reply with:: - - |initialSync|_ : A global sync which will present room and event information - for all rooms the user has joined. + { + "type": "m.login.email.url", + "user": "", + "email": "" + } -.. TODO-spec room-scoped initial sync - - |/rooms//initialSync|_ : A sync scoped to a single room. Presents - room and event information for this room only. - - Room-scoped initial sync is Very Tricky because typically people would - want to sync the room then listen for any new content from that point - onwards. The event stream cannot do this for a single room currently. - As a result, commenting room-scoped initial sync at this time. +After validating the email address, the home server MUST send an email +containing an authentication URL and return:: -The |initialSync|_ API contains the following keys: + { + "type": "m.login.email.url", + "session": "" + } -``presence`` - Description: - Contains a list of presence information for users the client is interested - in. - Format: - A JSON array of ``m.presence`` events. +The email contains a URL which must be clicked. After it has been clicked, the +client should perform another request:: -``end`` - Description: - Contains an event stream token which can be used with the `Event Stream`_. - Format: - A string containing the event stream token. + { + "type": "m.login.email.url", + "session": "" + } -``rooms`` - Description: - Contains a list of room information for all rooms the client has joined, - and limited room information on rooms the client has been invited to. - Format: - A JSON array containing Room Information JSON objects. +The home server MUST respond to this with either new credentials, the next +stage of the login process, or a standard error response. -Room Information: - Description: - Contains all state events for the room, along with a limited amount of - the most recent non-state events, configured via the ``limit`` query - parameter. Also contains additional keys with room metadata, such as the - ``room_id`` and the client's ``membership`` to the room. - Format: - A JSON object with the following keys: - ``room_id`` - A string containing the ID of the room being described. - ``membership`` - A string representing the client's membership status in this room. - ``messages`` - An event stream JSON object containing a ``chunk`` of recent non-state - events, along with an ``end`` token. *NB: The name of this key will be - changed in a later version.* - ``state`` - A JSON array containing all the current state events for this room. +A common client implementation will be to periodically poll until the link is +clicked. If the link has not been visited yet, a standard error response with +an errcode of ``M_LOGIN_EMAIL_URL_NOT_YET`` should be returned. -Getting events for a room -~~~~~~~~~~~~~~~~~~~~~~~~~ -There are several APIs provided to ``GET`` events for a room: -``/rooms//state//`` - Description: - Get the state event identified. - Response format: - A JSON object representing the state event **content**. - Example: - ``/rooms/!room:domain.com/state/m.room.name`` returns ``{ "name": "Room name" }`` +Email-based (identity server) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.identity`` +:Description: + Login is supported by authorising an email address with an identity server. -|/rooms//state|_ - Description: - Get all state events for a room. - Response format: - ``[ { state event }, { state event }, ... ]`` - Example: - TODO-doc +Prior to submitting this, the client should authenticate with an identity +server. After authenticating, the session information should be submitted to +the home server. +To respond to this type, reply with:: -|/rooms//members|_ - Description: - Get all ``m.room.member`` state events. - Response format: - ``{ "start": "", "end": "", "chunk": [ { m.room.member event }, ... ] }`` - Example: - TODO-doc + { + "type": "m.login.email.identity", + "threepidCreds": [ + { + "sid": "", + "clientSecret": "", + "idServer": "" + } + ] + } -|/rooms//messages|_ - Description: - Get all ``m.room.message`` and ``m.room.member`` events. This API supports - pagination using ``from`` and ``to`` query parameters, coupled with the - ``start`` and ``end`` tokens from an |initialSync|_ API. - Response format: - ``{ "start": "", "end": "" }`` - Example: - TODO-doc -|/rooms//initialSync|_ - Description: - Get all relevant events for a room. This includes state events, paginated - non-state events and presence events. - Response format: - `` { TODO-doc } `` - Example: - TODO-doc -Redactions -~~~~~~~~~~ -Since events are extensible it is possible for malicious users and/or servers -to add keys that are, for example offensive or illegal. Since some events -cannot be simply deleted, e.g. membership events, we instead 'redact' events. -This involves removing all keys from an event that are not required by the -protocol. This stripped down event is thereafter returned anytime a client or -remote server requests it. +N-Factor Authentication +~~~~~~~~~~~~~~~~~~~~~~~ +Multiple login stages can be combined to create N-factor authentication during +login. -Events that have been redacted include a ``redacted_because`` key whose value -is the event that caused it to be redacted, which may include a reason. +This can be achieved by responding with the ``next`` login type on completion +of a previous login stage:: -Redacting an event cannot be undone, allowing server owners to delete the -offending content from the databases. + { + "next": "" + } -Currently, only room admins can redact events by sending a ``m.room.redaction`` -event, but server admins also need to be able to redact events by a similar -mechanism. +If a home server implements N-factor authentication, it MUST respond with all +``stages`` when initially queried for their login requirements:: -Upon receipt of a redaction event, the server should strip off any keys not in -the following list: + { + "type": "<1st login type>", + "stages": [ <1st login type>, <2nd login type>, ... , ] + } - - ``event_id`` - - ``type`` - - ``room_id`` - - ``user_id`` - - ``state_key`` - - ``prev_state`` - - ``content`` +This can be represented conceptually as:: -The content object should also be stripped of all keys, unless it is one of -one of the following event types: + _______________________ + | Login Stage 1 | + | type: "" | + | ___________________ | + | |_Request_1_________| | <-- Returns "session" key which is used throughout. + | ___________________ | + | |_Request_2_________| | <-- Returns a "next" value of "login type2" + |_______________________| + | + | + _________V_____________ + | Login Stage 2 | + | type: "" | + | ___________________ | + | |_Request_1_________| | + | ___________________ | + | |_Request_2_________| | + | ___________________ | + | |_Request_3_________| | <-- Returns a "next" value of "login type3" + |_______________________| + | + | + _________V_____________ + | Login Stage 3 | + | type: "" | + | ___________________ | + | |_Request_1_________| | <-- Returns user credentials + |_______________________| - - ``m.room.member`` allows key ``membership`` - - ``m.room.create`` allows key ``creator`` - - ``m.room.join_rules`` allows key ``join_rule`` - - ``m.room.power_levels`` allows keys that are user ids or ``default`` - - ``m.room.add_state_level`` allows key ``level`` - - ``m.room.send_event_level`` allows key ``level`` - - ``m.room.ops_levels`` allows keys ``kick_level``, ``ban_level`` - and ``redact_level`` - - ``m.room.aliases`` allows key ``aliases`` +Fallback +~~~~~~~~ +Clients cannot be expected to be able to know how to process every single login +type. If a client determines it does not know how to handle a given login type, +it should request a login fallback page:: -The redaction event should be added under the key ``redacted_because``. + GET matrix/client/api/v1/login/fallback +This MUST return an HTML page which can perform the entire login process. -When a client receives a redaction event it should change the redacted event -in the same way a server does. Presence ~~~~~~~~ @@ -1050,36 +1213,6 @@ have to wait in milliseconds before they can try again. homeserver come up with their own idea, causing totally unpredictable performance over federated rooms? -End-to-End Encryption -~~~~~~~~~~~~~~~~~~~~~ - -.. TODO-doc - - Why is this needed. - - Overview of process - - Implementation - -Content repository ------------------- -.. NOTE:: - This section is a work in progress. - -.. TODO-spec - - path to upload - - format for thumbnail paths, mention what it is protecting against. - - content size limit and associated M_ERROR. - - -Address book repository ------------------------ -.. NOTE:: - This section is a work in progress. - -.. TODO-spec - - format: POST(?) wodges of json, some possible processing, then return wodges of json on GET. - - processing may remove dupes, merge contacts, pepper with extra info (e.g. matrix-ability of - contacts), etc. - - Standard json format for contacts? Piggy back off vcards? - .. Links through the external API docs are below .. ============================================= diff --git a/specification/10_events.rst b/specification/20_events.rst similarity index 88% rename from specification/10_events.rst rename to specification/20_events.rst index 20dccb6b..57b4c925 100644 --- a/specification/10_events.rst +++ b/specification/20_events.rst @@ -250,7 +250,7 @@ prefixed with ``m.`` Summary: A message. Type: - Non-state event + Message event JSON format: ``{ "msgtype": "string" }`` Example: @@ -266,8 +266,9 @@ prefixed with ``m.`` ``m.room.message.feedback`` Summary: A receipt for a message. + N.B. not implemented in Synapse, and superceded in v2 CS API by the 'relates_to' event field. Type: - Non-state event + Message event JSON format: ``{ "type": "enum [ delivered|read ]", "target_event_id": "string" }`` Example: @@ -283,7 +284,7 @@ prefixed with ``m.`` Summary: Indicates a previous event has been redacted. Type: - Non-state event + Message event JSON format: ``{ "reason": "string" }`` Description: @@ -292,7 +293,7 @@ prefixed with ``m.`` admins to remove offensive or illegal content that may have been attached to any event. This cannot be undone, allowing server owners to physically delete the offending data. There is also a concept of a moderator hiding a - non-state event, which can be undone, but cannot be applied to state + message event, which can be undone, but cannot be applied to state events. The event that has been redacted is specified in the ``redacts`` event level key. @@ -447,6 +448,61 @@ outlined below: .. TODO-spec Make the definitions "inherit" from FileInfo where necessary... +Presence Events +~~~~~~~~~~~~~~~ + +``m.presence`` + Summary: + Informs you of a user's presence state changes. + + Type: + Presence event + + JSON format:: + + { + "displayname": "utf-8 string", + "avatar_url": "url", + "presence": "enum [ online|unavailable|offline|free_for_chat|hidden ]", + "last_active_ago": "milliseconds" + } + + Example:: + + { + "displayname": "Matthew", + "avatar_url": "mxc://domain/id", + "presence": "online", + "last_active_ago": 10000 + } + + Description: + Each user has the concept of presence information. This encodes the + "availability" of that user, suitable for display on other user's clients. + This is transmitted as an ``m.presence`` event and is one of the few events + which are sent *outside the context of a room*. The basic piece of presence + information is represented by the ``presence`` key, which is an enum of one + of the following: + + - ``online`` : The default state when the user is connected to an event + stream. + - ``unavailable`` : The user is not reachable at this time. + - ``offline`` : The user is not connected to an event stream. + - ``free_for_chat`` : The user is generally willing to receive messages + moreso than default. + - ``hidden`` : Behaves as offline, but allows the user to see the client + state anyway and generally interact with client features. (Not yet + implemented in synapse). + + In addition, the server maintains a timestamp of the last time it saw a + pro-active event from the user; either sending a message to a room, or + changing presence state from a lower to a higher level of availability + (thus: changing state from ``unavailable`` to ``online`` counts as a + proactive event, whereas in the other direction it will not). This timestamp + is presented via a key called ``last_active_ago``, which gives the relative + number of milliseconds since the message is generated/emitted that the user + was last seen active. + Events on Change of Profile Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -457,7 +513,7 @@ values. This change is conveyed using two separate mechanisms: - a ``m.room.member`` event is sent to every room the user is a member of, to update the ``displayname`` and ``avatar_url``. - - a presence status update is sent, again containing the new values of the + - a ``m.presence`` presence status update is sent, again containing the new values of the ``displayname`` and ``avatar_url`` keys, in addition to the required ``presence`` key containing the current presence state of the user. @@ -496,8 +552,17 @@ This event is sent by the caller when they wish to establish a call. Optional keys: None. - Example: - ``{ "version" : 0, "call_id": "12345", "offer": { "type" : "offer", "sdp" : "v=0\r\no=- 6584580628695956864 2 IN IP4 127.0.0.1[...]" } }`` + + Example:: + + { + "version" : 0, + "call_id": "12345", + "offer": { + "type" : "offer", + "sdp" : "v=0\r\no=- 6584580628695956864 2 IN IP4 127.0.0.1[...]" + } + } ``Offer Object`` Required keys: diff --git a/specification/40_application_service_api.rst b/specification/25_application_service_api.rst similarity index 100% rename from specification/40_application_service_api.rst rename to specification/25_application_service_api.rst diff --git a/specification/11_event_signing.rst b/specification/31_event_signing.rst similarity index 100% rename from specification/11_event_signing.rst rename to specification/31_event_signing.rst diff --git a/drafts/push_overview.rst b/specification/42_push_overview.rst similarity index 99% rename from drafts/push_overview.rst rename to specification/42_push_overview.rst index c84fc8d2..972a8eea 100644 --- a/drafts/push_overview.rst +++ b/specification/42_push_overview.rst @@ -1,6 +1,9 @@ Push Notifications ================== +Overview +-------- + :: +--------------------+ +-------------------+ @@ -72,3 +75,4 @@ Push Gateway For information on the client-server API for setting pushers and push rules, see the Client Server API section. For more information on the format of HTTP notifications, see the HTTP Notification Protocol section. + diff --git a/drafts/push_csapi.rst b/specification/43_push_cs_api.rst similarity index 99% rename from drafts/push_csapi.rst rename to specification/43_push_cs_api.rst index 6b91736e..981147d3 100644 --- a/drafts/push_csapi.rst +++ b/specification/43_push_cs_api.rst @@ -1,8 +1,6 @@ -Push Notifications -================== +Pushers HTTP API +---------------- -Pushers -------- To receive any notification pokes at all, it is necessary to configure a 'pusher' on the Home Server that you wish to receive notifications from. There is a single API endpoint for this:: @@ -240,8 +238,8 @@ Actions that have no parameters are represented as a string. Otherwise, they are represented as a dictionary with a key equal to their name and other keys as their parameters, eg. { "set_tweak": "sound", "value": "default" } -Push Rule Actions: Tweaks -------------------------- +Push Rules: Actions: Tweaks +--------------------------- The 'set_tweak' key action is used to add an entry to the 'tweaks' dictionary that is sent in the notification poke. The following tweaks are defined: @@ -262,8 +260,8 @@ notification light on a mobile device. If a kind of tweak that a client understands is not specified in an action, the client may choose a sensible behaviour for the tweak. -Push Rules: Conditions: ------------------------ +Push Rules: Conditions +---------------------- Override, Underride and Default rules have a list of 'conditions'. All conditions must hold true for an event in order for a rule to be applied to an event. A rule with no conditions always matches. Matrix specifies the following @@ -416,3 +414,5 @@ Rules can be enabled or disabled with a PUT operation to the 'enabled' component beneath the rule's URI with a content of 'true' or 'false':: curl -X PUT -H "Content-Type: application/json" -d 'false' "http://localhost:8008/_matrix/client/api/v1/pushrules/global/sender/%40spambot%3Amatrix.org/enabled?access_token=123456" + + diff --git a/drafts/push_pgwapi.rst b/specification/44_push_push_gw_api.rst similarity index 98% rename from drafts/push_pgwapi.rst rename to specification/44_push_push_gw_api.rst index d3da9ab2..b182503b 100644 --- a/drafts/push_pgwapi.rst +++ b/specification/44_push_push_gw_api.rst @@ -1,5 +1,6 @@ -Push Notifications: HTTP Notification Protocol -============================================== +HTTP Notification Protocol +-------------------------- + This describes the format used by "http" pushers to send notifications of events. @@ -140,3 +141,4 @@ gateway). However, Matrix strongly recommends: * That APNS push gateways do not attempt to wait for errors from the APNS gateway before returning and instead to store failures and return 'rejected' responses next time that pushkey is used. + diff --git a/drafts/typing_notifications.rst b/specification/45_typing_notifications.rst similarity index 99% rename from drafts/typing_notifications.rst rename to specification/45_typing_notifications.rst index 048eba98..4c7d9b72 100644 --- a/drafts/typing_notifications.rst +++ b/specification/45_typing_notifications.rst @@ -55,3 +55,4 @@ originating HSes to ensure they eventually send "stop" notifications. ((This will eventually need addressing, as part of the wider typing/presence timer addition work)) +