From 556e3f8a7176a88bf4477bef6a96d4a9adbf5ad0 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Thu, 9 Oct 2014 20:30:40 +0200 Subject: [PATCH] move matrix-generic documentation from synapse/docs into new matrix-doc project --- drafts/definitions.rst | 53 + drafts/human-id-rules.rst | 79 + drafts/state_resolution.rst | 51 + meta/documentation_style.rst | 43 + specification/spec-additions.rst | 89 + specification/specification-NOTHAVE.rst | 30 + specification/specification.rst | 2739 +++++++++++++++++++++++ tutorials/client-server-howto.rst | 637 ++++++ 8 files changed, 3721 insertions(+) create mode 100644 drafts/definitions.rst create mode 100644 drafts/human-id-rules.rst create mode 100644 drafts/state_resolution.rst create mode 100644 meta/documentation_style.rst create mode 100644 specification/spec-additions.rst create mode 100644 specification/specification-NOTHAVE.rst create mode 100644 specification/specification.rst create mode 100644 tutorials/client-server-howto.rst diff --git a/drafts/definitions.rst b/drafts/definitions.rst new file mode 100644 index 00000000..b0f95ae9 --- /dev/null +++ b/drafts/definitions.rst @@ -0,0 +1,53 @@ +Definitions +=========== + +# *Event* -- A JSON object that represents a piece of information to be +distributed to the the room. The object includes a payload and metadata, +including a `type` used to indicate what the payload is for and how to process +them. It also includes one or more references to previous events. + +# *Event graph* -- Events and their references to previous events form a +directed acyclic graph. All events must be a descendant of the first event in a +room, except for a few special circumstances. + +# *State event* -- A state event is an event that has a non-null string valued +`state_key` field. It may also include a `prev_state` key referencing exactly +one state event with the same type and state key, in the same event graph. + +# *State tree* -- A state tree is a tree formed by a collection of state events +that have the same type and state key (all in the same event graph. + +# *State resolution algorithm* -- An algorithm that takes a state tree as input +and selects a single leaf node. + +# *Current state event* -- The leaf node of a given state tree that has been +selected by the state resolution algorithm. + +# *Room state* / *state dictionary* / *current state* -- A mapping of the pair +(event type, state key) to the current state event for that pair. + +# *Room* -- An event graph and its associated state dictionary. An event is in +the room if it is part of the event graph. + +# *Topological ordering* -- The partial ordering that can be extracted from the +event graph due to it being a DAG. + +(The state definitions are purposely slightly ill-defined, since if we allow +deleting events we might end up with multiple state trees for a given event +type and state key pair.) + +Federation specific +------------------- +# *(Persistent data unit) PDU* -- An encoding of an event for distribution of +the server to server protocol. + +# *(Ephemeral data unit) EDU* -- A piece of information that is sent between +servers and doesn't encode an event. + +Client specific +--------------- +# *Child events* -- Events that reference a single event in the same room +independently of the event graph. + +# *Collapsed events* -- Events that have all child events that reference it +included in the JSON object. diff --git a/drafts/human-id-rules.rst b/drafts/human-id-rules.rst new file mode 100644 index 00000000..3a1ff398 --- /dev/null +++ b/drafts/human-id-rules.rst @@ -0,0 +1,79 @@ +This document outlines the format for human-readable IDs within matrix. + +Overview +-------- +UTF-8 is quickly becoming the standard character encoding set on the web. As +such, Matrix requires that all strings MUST be encoded as UTF-8. However, +using Unicode as the character set for human-readable IDs is troublesome. There +are many different characters which appear identical to each other, but would +identify different users. In addition, there are non-printable characters which +cannot be rendered by the end-user. This opens up a security vulnerability with +phishing/spoofing of IDs, commonly known as a homograph attack. + +Web browers encountered this problem when International Domain Names were +introduced. A variety of checks were put in place in order to protect users. If +an address failed the check, the raw punycode would be displayed to disambiguate +the address. Similar checks are performed by home servers in Matrix. However, +Matrix does not use punycode representations, and so does not show raw punycode +on a failed check. Instead, home servers must outright reject these misleading +IDs. + +Types of human-readable IDs +--------------------------- +There are two main human-readable IDs in question: + +- Room aliases +- User IDs + +Room aliases look like ``#localpart:domain``. These aliases point to opaque +non human-readable room IDs. These pointers can change, so there is already an +issue present with the same ID pointing to a different destination at a later +date. + +User IDs look like ``@localpart:domain``. These represent actual end-users, and +unlike room aliases, there is no layer of indirection. This presents a much +greater concern with homograph attacks. + +Checks +------ +- Similar to web browsers. +- blacklisted chars (e.g. non-printable characters) +- mix of language sets from 'preferred' language not allowed. +- Language sets from CLDR dataset. +- Treated in segments (localpart, domain) +- Additional restrictions for ease of processing IDs. + - Room alias localparts MUST NOT have ``#`` or ``:``. + - User ID localparts MUST NOT have ``@`` or ``:``. + +Rejecting +--------- +- Home servers MUST reject room aliases which do not pass the check, both on + GETs and PUTs. +- Home servers MUST reject user ID localparts which do not pass the check, both + on creation and on events. +- Any home server whose domain does not pass this check, MUST use their punycode + domain name instead of the IDN, to prevent other home servers rejecting you. +- Error code is ``M_FAILED_HUMAN_ID_CHECK``. (generic enough for both failing + due to homograph attacks, and failing due to including ``:`` s, etc) +- Error message MAY go into further information about which characters were + rejected and why. +- Error message SHOULD contain a ``failed_keys`` key which contains an array + of strings which represent the keys which failed the check e.g:: + + failed_keys: [ user_id, room_alias ] + +Other considerations +-------------------- +- Basic security: Informational key on the event attached by HS to say "unsafe + ID". Problem: clients can just ignore it, and since it will appear only very + rarely, easy to forget when implementing clients. +- Moderate security: Requires client handshake. Forces clients to implement + a check, else they cannot communicate with the misleading ID. However, this is + extra overhead in both client implementations and round-trips. +- High security: Outright rejection of the ID at the point of creation / + receiving event. Point of creation rejection is preferable to avoid the ID + entering the system in the first place. However, malicious HSes can just allow + the ID. Hence, other home servers must reject them if they see them in events. + Client never sees the problem ID, provided the HS is correctly implemented. +- High security decided; client doesn't need to worry about it, no additional + protocol complexity aside from rejection of an event. \ No newline at end of file diff --git a/drafts/state_resolution.rst b/drafts/state_resolution.rst new file mode 100644 index 00000000..fec290dd --- /dev/null +++ b/drafts/state_resolution.rst @@ -0,0 +1,51 @@ +State Resolution +================ +This section describes why we need state resolution and how it works. + + +Motivation +----------- +We want to be able to associate some shared state with rooms, e.g. a room name +or members list. This is done by having a current state dictionary that maps +from the pair event type and state key to an event. + +However, since the servers involved in the room are distributed we need to be +able to handle the case when two (or more) servers try and update the state at +the same time. This is done via the state resolution algorithm. + + +State Tree +------------ +State events contain a reference to the state it is trying to replace. These +relations form a tree where the current state is one of the leaf nodes. + +Note that state events are events, and so are part of the PDU graph. Thus we +can be sure that (modulo the internet being particularly broken) we will see +all state events eventually. + + +Algorithm requirements +---------------------- +We want the algorithm to have the following properties: +- Since we aren't guaranteed what order we receive state events in, except that + we see parents before children, the state resolution algorithm must not depend + on the order and must always come to the same result. +- If we receive a state event whose parent is the current state, then the + algorithm will select it. +- The algorithm does not depend on internal state, ensuring all servers should + come to the same decision. + +These three properties mean it is enough to keep track of the current state and +compare it with any new proposed state, rather than having to keep track of all +the leafs of the tree and recomputing across the entire state tree. + + +Current Implementation +---------------------- +The current implementation works as follows: Upon receipt of a newly proposed +state change we first find the common ancestor. Then we take the maximum +across each branch of the users' power levels, if one is higher then it is +selected as the current state. Otherwise, we check if one chain is longer than +the other, if so we choose that one. If that also fails, then we concatenate +all the pdu ids and take a SHA1 hash and compare them to select a common +ancestor. diff --git a/meta/documentation_style.rst b/meta/documentation_style.rst new file mode 100644 index 00000000..c365d09d --- /dev/null +++ b/meta/documentation_style.rst @@ -0,0 +1,43 @@ +=================== +Documentation Style +=================== + +A brief single sentence to describe what this file contains; in this case a +description of the style to write documentation in. + + +Sections +======== + +Each section should be separated from the others by two blank lines. Headings +should be underlined using a row of equals signs (===). Paragraphs should be +separated by a single blank line, and wrap to no further than 80 columns. + +[[TODO(username): if you want to leave some unanswered questions, notes for +further consideration, or other kinds of comment, use a TODO section. Make sure +to notate it with your name so we know who to ask about it!]] + +Subsections +----------- + +If required, subsections can use a row of dashes to underline their header. A +single blank line between subsections of a single section. + + +Bullet Lists +============ + + * Bullet lists can use asterisks with a single space either side. + + * Another blank line between list elements. + + +Definition Lists +================ + +Terms: + Start in the first column, ending with a colon + +Definitions: + Take a two space indent, following immediately from the term without a blank + line before it, but having a blank line afterwards. diff --git a/specification/spec-additions.rst b/specification/spec-additions.rst new file mode 100644 index 00000000..8e3ffa56 --- /dev/null +++ b/specification/spec-additions.rst @@ -0,0 +1,89 @@ +In C-S API > Registration/Login: + +Captcha-based +~~~~~~~~~~~~~ +:Type: + ``m.login.recaptcha`` +:Description: + Login is supported by responding to a captcha, in this case Google's + Recaptcha. + +To respond to this type, reply with:: + + { + "type": "m.login.recaptcha", + "challenge": "", + "response": "" + } + +The Recaptcha parameters can be obtained in Javascript by calling:: + + Recaptcha.get_challenge(); + Recaptcha.get_response(); + +The home server MUST respond with either new credentials, the next stage of the +login process, or a standard error response. + + + + +In Events: + +Common event fields +------------------- +All events MUST have the following fields: + +``event_id`` + Type: + String. + Description: + Represents the globally unique ID for this event. + +``type`` + Type: + String. + Description: + Contains the event type, e.g. ``m.room.message`` + +``content`` + Type: + JSON Object. + Description: + Contains the content of the event. When interacting with the REST API, this is the HTTP body. + +``room_id`` + Type: + String. + Description: + Contains the ID of the room associated with this event. + +``user_id`` + Type: + String. + Description: + Contains the fully-qualified ID of the user who *sent* this event. + +State events have the additional fields: + +``state_key`` + Type: + String. + Description: + Contains the state key for this state event. If there is no state key for this state event, this + will be an empty string. The presence of ``state_key`` makes this event a state event. + +``required_power_level`` + Type: + Integer. + Description: + Contains the minimum power level a user must have before they can update this event. + +``prev_content`` + Type: + JSON Object. + Description: + Optional. Contains the previous ``content`` for this event. If there is no previous content, this + key will be missing. + +.. TODO-spec + How do "age" and "ts" fit in to all this? Which do we expose? diff --git a/specification/specification-NOTHAVE.rst b/specification/specification-NOTHAVE.rst new file mode 100644 index 00000000..369594f6 --- /dev/null +++ b/specification/specification-NOTHAVE.rst @@ -0,0 +1,30 @@ +Matrix Specification NOTHAVEs +============================= + +This document contains sections of the main specification that have been +temporarily removed, because they specify intentions or aspirations that have +in no way yet been implemented. Rather than outright-deleting them, they have +been moved here so as to stand as an initial version for such time as they +become extant. + + +Presence +======== + +Idle Time +--------- +As well as the basic ``presence`` field, the presence information can also show +a sense of an "idle timer". This should be maintained individually by the +user's clients, and the home server can take the highest reported time as that +to report. When a user is offline, the home server can still report when the +user was last seen online. + +Device Type +----------- + +Client devices that may limit the user experience somewhat (such as "mobile" +devices with limited ability to type on a real keyboard or read large amounts of +text) should report this to the home server, as this is also useful information +to report as "presence" if the user cannot be expected to provide a good typed +response to messages. + diff --git a/specification/specification.rst b/specification/specification.rst new file mode 100644 index 00000000..9f7c86f2 --- /dev/null +++ b/specification/specification.rst @@ -0,0 +1,2739 @@ +Matrix Specification +==================== + +WARNING +======= + +.. WARNING:: + The Matrix specification is still very much evolving: the API is not yet frozen + and this document is in places incomplete, stale, and may contain security + issues. Needless to say, we have made every effort to highlight the problem + areas that we're aware of. + + We're publishing it at this point because it's complete enough to be more than + useful and provide a canonical reference to how Matrix is evolving. Our end + goal is to mirror WHATWG's `Living Standard `_ + approach except right now Matrix is more in the process of being born than actually being + living! + +.. contents:: Table of Contents +.. sectnum:: + +Matrix is a new set of open APIs for open-federated Instant Messaging and VoIP +functionality, designed to create and support a new global real-time +communication ecosystem on the internet. This specification is the ongoing +result of standardising the APIs used by the various components of the Matrix +ecosystem to communicate with one another. + +The principles that Matrix attempts to follow are: + +- Pragmatic Web-friendly APIs (i.e. JSON over REST) +- Keep It Simple & Stupid + + + provide a simple architecture with minimal third-party dependencies. + +- Fully open: + + + Fully open federation - anyone should be able to participate in the global + Matrix network + + Fully open standard - publicly documented standard with no IP or patent + licensing encumbrances + + Fully open source reference implementation - liberally-licensed example + implementations with no IP or patent licensing encumbrances + +- Empowering the end-user + + + The user should be able to choose the server and clients they use + + The user should be control how private their communication is + + The user should know precisely where their data is stored + +- Fully decentralised - no single points of control over conversations or the + network as a whole +- Learning from history to avoid repeating it + + + Trying to take the best aspects of XMPP, SIP, IRC, SMTP, IMAP and NNTP + whilst trying to avoid their failings + +The functionality that Matrix provides includes: + +- Creation and management of fully distributed chat rooms with no + single points of control or failure +- Eventually-consistent cryptographically secure synchronisation of room + state across a global open network of federated servers and services +- Sending and receiving extensible messages in a room with (optional) + end-to-end encryption +- Extensible user management (inviting, joining, leaving, kicking, banning) + mediated by a power-level based user privilege system. +- Extensible room state management (room naming, aliasing, topics, bans) +- Extensible user profile management (avatars, displaynames, etc) +- Managing user accounts (registration, login, logout) +- Use of 3rd Party IDs (3PIDs) such as email addresses, phone numbers, + Facebook accounts to authenticate, identify and discover users on Matrix. +- Trusted federation of Identity servers for: + + + Publishing user public keys for PKI + + Mapping of 3PIDs to Matrix IDs + +The end goal of Matrix is to be a ubiquitous messaging layer for synchronising +arbitrary data between sets of people, devices and services - be that for +instant messages, VoIP call setups, or any other objects that need to be +reliably and persistently pushed from A to B in an interoperable and federated +manner. + +Basis +===== + +Architecture +------------ + +Clients transmit data to other clients through home servers (HSes). Clients do +not communicate with each other directly. + +:: + + How data flows between clients + ============================== + + { Matrix client A } { Matrix client B } + ^ | ^ | + | events | | events | + | V | V + +------------------+ +------------------+ + | |---------( HTTP )---------->| | + | Home Server | | Home Server | + | |<--------( HTTP )-----------| | + +------------------+ Federation +------------------+ + +A "Client" typically represents a human using a web application or mobile app. +Clients use the "Client-to-Server" (C-S) API to communicate with their home +server, which stores their profile data and their record of the conversations +in which they participate. Each client is associated with a user account (and +may optionally support multiple user accounts). A user account is represented +by a unique "User ID". This ID is namespaced to the home server which allocated +the account and looks like:: + + @localpart:domain + +The ``localpart`` of a user ID may be a user name, or an opaque ID identifying +this user. They are case-insensitive. + +.. TODO-spec + - Need to specify precise grammar for Matrix IDs + +A "Home Server" is a server which provides C-S APIs and has the ability to +federate with other HSes. It is typically responsible for multiple clients. +"Federation" is the term used to describe the sharing of data between two or +more home servers. + +Data in Matrix is encapsulated in an "event". An event is an action within the +system. Typically each action (e.g. sending a message) correlates with exactly +one event. Each event has a ``type`` which is used to differentiate different +kinds of data. ``type`` values MUST be uniquely globally namespaced following +Java's `package naming conventions +`, e.g. +``com.example.myapp.event``. The special top-level namespace ``m.`` is reserved +for events defined in the Matrix specification. Events are usually sent in the +context of a "Room". + +Room structure +~~~~~~~~~~~~~~ + +A room is a conceptual place where users can send and receive events. Rooms can +be created, joined and left. Events are sent to a room, and all participants in +that room with sufficient access will receive the event. Rooms are uniquely +identified internally via a "Room ID", which look like:: + + !opaque_id:domain + +There is exactly one room ID for each room. Whilst the room ID does contain a +domain, it is simply for globally namespacing room IDs. The room does NOT +reside on the domain specified. Room IDs are not meant to be human readable. +They ARE case-sensitive. + +The following diagram shows an ``m.room.message`` event being sent in the room +``!qporfwt:matrix.org``:: + + { @alice:matrix.org } { @bob:domain.com } + | ^ + | | + Room ID: !qporfwt:matrix.org Room ID: !qporfwt:matrix.org + Event type: m.room.message Event type: m.room.message + Content: { JSON object } Content: { JSON object } + | | + V | + +------------------+ +------------------+ + | Home Server | | Home Server | + | matrix.org |<-------Federation------->| domain.com | + +------------------+ +------------------+ + | ................................. | + |______| Shared State |_______| + | Room ID: !qporfwt:matrix.org | + | Servers: matrix.org, domain.com | + | Members: | + | - @alice:matrix.org | + | - @bob:domain.com | + |.................................| + +Federation maintains shared state between multiple home servers, such that when +an event is sent to a room, the home server knows where to forward the event on +to, and how to process the event. State is scoped to a single room, and +federation ensures that all home servers have the information they need, even +if that means the home server has to request more information from another home +server before processing the event. + +Room Aliases +~~~~~~~~~~~~ + +Each room can also have multiple "Room Aliases", which looks like:: + + #room_alias:domain + + .. TODO + - Need to specify precise grammar for Room Aliases + +A room alias "points" to a room ID and is the human-readable label by which +rooms are publicised and discovered. The room ID the alias is pointing to can +be obtained by visiting the domain specified. They are case-insensitive. Note +that the mapping from a room alias to a room ID is not fixed, and may change +over time to point to a different room ID. For this reason, Clients SHOULD +resolve the room alias to a room ID once and then use that ID on subsequent +requests. + +When resolving a room alias the server will also respond with a list of servers +that are in the room that can be used to join via. + +:: + + GET + #matrix:domain.com !aaabaa:matrix.org + | ^ + | | + _______V____________________|____ + | domain.com | + | Mappings: | + | #matrix >> !aaabaa:matrix.org | + | #golf >> !wfeiofh:sport.com | + | #bike >> !4rguxf:matrix.org | + |________________________________| + +Identity +~~~~~~~~ + +Users in Matrix are identified via their user ID. However, existing ID +namespaces can also be used in order to identify Matrix users. A Matrix +"Identity" describes both the user ID and any other existing IDs from third +party namespaces *linked* to their account. + +Matrix users can *link* third-party IDs (3PIDs) such as email addresses, social +network accounts and phone numbers to their user ID. Linking 3PIDs creates a +mapping from a 3PID to a user ID. This mapping can then be used by other Matrix +users in order to discover other users, according to a strict set of privacy +permissions. + +In order to ensure that the mapping from 3PID to user ID is genuine, a globally +federated cluster of trusted "Identity Servers" (IS) are used to perform +authentication of the 3PID. Identity servers are also used to preserve the +mapping indefinitely, by replicating the mappings across multiple ISes. + +Usage of an IS is not required in order for a client application to be part of +the Matrix ecosystem. However, without one clients will not be able to look up +user IDs using 3PIDs. + +Presence +~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +Each user has the concept of presence information. This encodes the +"availability" of that user, suitable for display on other user's clients. This +is transmitted as an ``m.presence`` event and is one of the few events which +are sent *outside the context of a room*. The basic piece of presence +information is represented by the ``presence`` key, which is an enum of one of +the following: + + - ``online`` : The default state when the user is connected to an event + stream. + - ``unavailable`` : The user is not reachable at this time. + - ``offline`` : The user is not connected to an event stream. + - ``free_for_chat`` : The user is generally willing to receive messages + moreso than default. + - ``hidden`` : Behaves as offline, but allows the user to see the client + state anyway and generally interact with client features. (Not yet + implemented in synapse). + +This basic ``presence`` field applies to the user as a whole, regardless of how +many client devices they have connected. The home server should synchronise +this status choice among multiple devices to ensure the user gets a consistent +experience. + +In addition, the server maintains a timestamp of the last time it saw an active +action from the user; either sending a message to a room, or changing presence +state from a lower to a higher level of availability (thus: changing state from +``unavailable`` to ``online`` will count as an action for being active, whereas +in the other direction will not). This timestamp is presented via a key called +``last_active_ago``, which gives the relative number of miliseconds since the +message is generated/emitted, that the user was last seen active. + +Home servers can also use the user's choice of presence state as a signal for +how to handle new private one-to-one chat message requests. For example, it +might decide: + + - ``free_for_chat`` : accept anything + - ``online`` : accept from anyone in my addres book list + - ``busy`` : accept from anyone in this "important people" group in my + address book list + +Presence List ++++++++++++++ +Each user's home server stores a "presence list" for that user. This stores a +list of other user IDs the user has chosen to add to it. To be added to this +list, the user being added must receive permission from the list owner. Once +granted, both user's HS(es) store this information. Since such subscriptions +are likely to be bidirectional, HSes may wish to automatically accept requests +when a reverse subscription already exists. + +As a convenience, presence lists should support the ability to collect users +into groups, which could allow things like inviting the entire group to a new +("ad-hoc") chat room, or easy interaction with the profile information ACL +implementation of the HS. + +Presence and Permissions +++++++++++++++++++++++++ +For a viewing user to be allowed to see the presence information of a target +user, either: + + - The target user has allowed the viewing user to add them to their presence + list, or + - The two users share at least one room in common + +In the latter case, this allows for clients to display some minimal sense of +presence information in a user list for a room. + +Profiles +~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +.. TODO-spec + - Metadata extensibility + +Internally within Matrix users are referred to by their user ID, which is +typically a compact unique identifier. Profiles grant users the ability to see +human-readable names for other users that are in some way meaningful to them. +Additionally, profiles can publish additional information, such as the user's +age or location. + +A Profile consists of a display name, an avatar picture, and a set of other +metadata fields that the user may wish to publish (email address, phone +numbers, website URLs, etc...). This specification puts no requirements on the +display name other than it being a valid unicode string. Avatar images are not +stored directly; instead the home server stores an ``http``-scheme URL where +clients may fetch it from. + +API Standards +------------- + +The mandatory baseline for communication in Matrix is exchanging JSON objects +over RESTful HTTP APIs. HTTPS is mandated as the baseline for server-server +(federation) communication. HTTPS is recommended for client-server +communication, although HTTP may be supported as a fallback to support basic +HTTP clients. More efficient optional transports for client-server +communication will in future be supported as optional extensions - e.g. a +packed binary encoding over stream-cipher encrypted TCP socket for +low-bandwidth/low-roundtrip mobile usage. + +.. TODO + We need to specify capability negotiation for extensible transports + +For the default HTTP transport, all API calls use a Content-Type of +``application/json``. In addition, all strings MUST be encoded as UTF-8. + +Clients are authenticated using opaque ``access_token`` strings (see +`Registration and Login`_ for details), passed as a query string parameter on +all requests. + +.. TODO + Need to specify any HMAC or access_token lifetime/ratcheting tricks + +Any errors which occur on the Matrix API level MUST return a "standard error +response". This is a JSON object which looks like:: + + { + "errcode": "", + "error": "" + } + +The ``error`` string will be a human-readable error message, usually a sentence +explaining what went wrong. The ``errcode`` string will be a unique string +which can be used to handle an error message e.g. ``M_FORBIDDEN``. These error +codes should have their namespace first in ALL CAPS, followed by a single _. +For example, if there was a custom namespace ``com.mydomain.here``, and a +``FORBIDDEN`` code, the error code should look like +``COM.MYDOMAIN.HERE_FORBIDDEN``. There may be additional keys depending on the +error, but the keys ``error`` and ``errcode`` MUST always be present. + +Some standard error codes are below: + +:``M_FORBIDDEN``: + Forbidden access, e.g. joining a room without permission, failed login. + +:``M_UNKNOWN_TOKEN``: + The access token specified was not recognised. + +:``M_BAD_JSON``: + Request contained valid JSON, but it was malformed in some way, e.g. missing + required keys, invalid values for keys. + +:``M_NOT_JSON``: + Request did not contain valid JSON. + +:``M_NOT_FOUND``: + No resource was found for this request. + +:``M_LIMIT_EXCEEDED``: + Too many requests have been sent in a short period of time. Wait a while then + try again. + +Some requests have unique error codes: + +:``M_USER_IN_USE``: + Encountered when trying to register a user ID which has been taken. + +:``M_ROOM_IN_USE``: + Encountered when trying to create a room which has been taken. + +:``M_BAD_PAGINATION``: + Encountered when specifying bad pagination query parameters. + +:``M_LOGIN_EMAIL_URL_NOT_YET``: + Encountered when polling for an email link which has not been clicked yet. + +The C-S API typically uses ``HTTP POST`` to submit requests. This means these +requests are not idempotent. The C-S API also allows ``HTTP PUT`` to make +requests idempotent. In order to use a ``PUT``, paths should be suffixed with +``/{txnId}``. ``{txnId}`` is a unique client-generated transaction ID which +identifies the request, and is scoped to a given Client (identified by that +client's ``access_token``). Crucially, it **only** serves to identify new +requests from retransmits. After the request has finished, the ``{txnId}`` +value should be changed (how is not specified; a monotonically increasing +integer is recommended). It is preferable to use ``HTTP PUT`` to make sure +requests to send messages do not get sent more than once should clients need to +retransmit requests. + +Valid requests look like:: + + POST /some/path/here?access_token=secret + { + "key": "This is a post." + } + + PUT /some/path/here/11?access_token=secret + { + "key": "This is a put with a txnId of 11." + } + +In contrast, these are invalid requests:: + + POST /some/path/here/11?access_token=secret + { + "key": "This is a post, but it has a txnId." + } + + PUT /some/path/here?access_token=secret + { + "key": "This is a put but it is missing a txnId." + } + +Glossary +-------- +.. NOTE:: + This section is a work in progress. + +Backfilling: + The process of synchronising historic state from one home server to another, + to backfill the event storage so that scrollback can be presented to the + client(s). Not to be confused with pagination. + +Context: + A single human-level entity of interest (currently, a chat room) + +EDU (Ephemeral Data Unit): + A message that relates directly to a given pair of home servers that are + exchanging it. EDUs are short-lived messages that related only to one single + pair of servers; they are not persisted for a long time and are not forwarded + on to other servers. Because of this, they have no internal ID nor previous + EDUs reference chain. + +Event: + A record of activity that records a single thing that happened on to a context + (currently, a chat room). These are the "chat messages" that Synapse makes + available. + +PDU (Persistent Data Unit): + A message that relates to a single context, irrespective of the server that + is communicating it. PDUs either encode a single Event, or a single State + change. A PDU is referred to by its PDU ID; the pair of its origin server + and local reference from that server. + +PDU ID: + The pair of PDU Origin and PDU Reference, that together globally uniquely + refers to a specific PDU. + +PDU Origin: + The name of the origin server that generated a given PDU. This may not be the + server from which it has been received, due to the way they are copied around + from server to server. The origin always records the original server that + created it. + +PDU Reference: + A local ID used to refer to a specific PDU from a given origin server. These + references are opaque at the protocol level, but may optionally have some + structured meaning within a given origin server or implementation. + +Presence: + The concept of whether a user is currently online, how available they declare + they are, and so on. See also: doc/model/presence + +Profile: + A set of metadata about a user, such as a display name, provided for the + benefit of other users. See also: doc/model/profiles + +Room ID: + An opaque string (of as-yet undecided format) that identifies a particular + room and used in PDUs referring to it. + +Room Alias: + A human-readable string of the form #name:some.domain that users can use as a + pointer to identify a room; a Directory Server will map this to its Room ID + +State: + A set of metadata maintained about a Context, which is replicated among the + servers in addition to the history of Events. + +User ID: + A string of the form @localpart:domain.name that identifies a user for + wire-protocol purposes. The localpart is meaningless outside of a particular + home server. This takes a human-readable form that end-users can use directly + if they so wish, avoiding the 3PIDs. + +Transaction: + A message which relates to the communication between a given pair of servers. + A transaction contains possibly-empty lists of PDUs and EDUs. + +.. TODO + This glossary contradicts the terms used above - especially on State Events v. "State" + and Non-State Events v. "Events". We need better consistent names. + +Events +====== + +Receiving live updates on a client +---------------------------------- + +Clients can receive new events by long-polling the home server. This will hold +open the HTTP connection for a short period of time waiting for new events, +returning early if an event occurs. This is called the `Event Stream`_. All +events which are visible to the client will appear in the event stream. When +the request returns, an ``end`` token is included in the response. This token +can be used in the next request to continue where the client left off. + +.. TODO-spec + How do we filter the event stream? + Do we ever return multiple events in a single request? Don't we get lots of request + setup RTT latency if we only do one event per request? Do we ever support streaming + requests? Why not websockets? + +When the client first logs in, they will need to initially synchronise with +their home server. This is achieved via the |initialSync|_ API. This API also +returns an ``end`` token which can be used with the event stream. + +Room Events +----------- +.. NOTE:: + This section is a work in progress. + +This specification outlines several standard event types, all of which are +prefixed with ``m.`` + +``m.room.name`` + Summary: + Set the human-readable name for the room. + Type: + State event + JSON format: + ``{ "name" : "string" }`` + Example: + ``{ "name" : "My Room" }`` + Description: + A room has an opaque room ID which is not human-friendly to read. A room + alias is human-friendly, but not all rooms have room aliases. The room name + is a human-friendly string designed to be displayed to the end-user. The + room name is not *unique*, as multiple rooms can have the same room name + set. The room name can also be set when creating a room using |createRoom|_ + with the ``name`` key. + +``m.room.topic`` + Summary: + Set a topic for the room. + Type: + State event + JSON format: + ``{ "topic" : "string" }`` + Example: + ``{ "topic" : "Welcome to the real world." }`` + Description: + A topic is a short message detailing what is currently being discussed in + the room. It can also be used as a way to display extra information about + the room, which may not be suitable for the room name. The room topic can + also be set when creating a room using |createRoom|_ with the ``topic`` + key. + +``m.room.member`` + Summary: + The current membership state of a user in the room. + Type: + State event + JSON format: + ``{ "membership" : "enum[ invite|join|leave|ban ]" }`` + Example: + ``{ "membership" : "join" }`` + Description: + Adjusts the membership state for a user in a room. It is preferable to use + the membership APIs (``/rooms//invite`` etc) when performing + membership actions rather than adjusting the state directly as there are a + restricted set of valid transformations. For example, user A cannot force + user B to join a room, and trying to force this state change directly will + fail. See the `Rooms`_ section for how to use the membership APIs. + +``m.room.create`` + Summary: + The first event in the room. + Type: + State event + JSON format: + ``{ "creator": "string"}`` + Example: + ``{ "creator": "@user:example.com" }`` + Description: + This is the first event in a room and cannot be changed. It acts as the + root of all other events. + +``m.room.join_rules`` + Summary: + Descripes how/if people are allowed to join. + Type: + State event + JSON format: + ``{ "join_rule": "enum [ public|knock|invite|private ]" }`` + Example: + ``{ "join_rule": "public" }`` + Description: + TODO-doc : Use docs/models/rooms.rst + +``m.room.power_levels`` + Summary: + Defines the power levels of users in the room. + Type: + State event + JSON format: + ``{ "": , ..., "default": }`` + Example: + ``{ "@user:example.com": 5, "@user2:example.com": 10, "default": 0 }`` + Description: + If a user is in the list, then they have the associated power level. + Otherwise they have the default level. If not ``default`` key is supplied, + it is assumed to be 0. + +``m.room.add_state_level`` + Summary: + Defines the minimum power level a user needs to add state. + Type: + State event + JSON format: + ``{ "level": }`` + Example: + ``{ "level": 5 }`` + Description: + To add a new piece of state to the room a user must have the given power + level. This does not apply to updating current state, which is goverened + by the ``required_power_level`` event key. + +``m.room.send_event_level`` + Summary: + Defines the minimum power level a user needs to send an event. + Type: + State event + JSON format: + ``{ "level": }`` + Example: + ``{ "level": 0 }`` + Description: + To send a new event into the room a user must have at least this power + level. This allows ops to make the room read only by increasing this level, + or muting individual users by lowering their power level below this + threshold. + +``m.room.ops_levels`` + Summary: + Defines the minimum power levels that a user must have before they can + kick and/or ban other users. + Type: + State event + JSON format: + ``{ "ban_level": , "kick_level": , "redact_level": }`` + Example: + ``{ "ban_level": 5, "kick_level": 5 }`` + Description: + This defines who can ban and/or kick people in the room. Most of the time + ``ban_level`` will be greater than or equal to ``kick_level`` since + banning is more severe than kicking. + +``m.room.aliases`` + Summary: + These state events are used to inform the room about what room aliases it + has. + Type: + State event + JSON format: + ``{ "aliases": ["string", ...] }`` + Example: + ``{ "aliases": ["#foo:example.com"] }`` + Description: + This event is sent by a homeserver directly to inform of changes to the + list of aliases it knows about for that room. As a special-case, the + ``state_key`` of the event is the homeserver which owns the room alias. + For example, an event might look like:: + + { + "type": "m.room.aliases", + "event_id": "012345678ab", + "room_id": "!xAbCdEfG:example.com", + "state_key": "example.com", + "content": { + "aliases": ["#foo:example.com"] + } + } + + The event contains the full list of aliases now stored by the home server + that emitted it; additions or deletions are not explicitly mentioned as + being such. The entire set of known aliases for the room is then the union + of the individual lists declared by all such keys, one from each home + server holding at least one alias. + + Clients `should` check the validity of any room alias given in this list + before presenting it to the user as trusted fact. The lists given by this + event should be considered simply as advice on which aliases might exist, + for which the client can perform the lookup to confirm whether it receives + the correct room ID. + +``m.room.message`` + Summary: + A message. + Type: + Non-state event + JSON format: + ``{ "msgtype": "string" }`` + Example: + ``{ "msgtype": "m.text", "body": "Testing" }`` + Description: + This event is used when sending messages in a room. Messages are not + limited to be text. The ``msgtype`` key outlines the type of message, e.g. + text, audio, image, video, etc. Whilst not required, the ``body`` key + SHOULD be used with every kind of ``msgtype`` as a fallback mechanism when + a client cannot render the message. For more information on the types of + messages which can be sent, see `m.room.message msgtypes`_. + +``m.room.message.feedback`` + Summary: + A receipt for a message. + Type: + Non-state event + JSON format: + ``{ "type": "enum [ delivered|read ]", "target_event_id": "string" }`` + Example: + ``{ "type": "delivered", "target_event_id": "e3b2icys" }`` + Description: + Feedback events are events sent to acknowledge a message in some way. There + are two supported acknowledgements: ``delivered`` (sent when the event has + been received) and ``read`` (sent when the event has been observed by the + end-user). The ``target_event_id`` should reference the ``m.room.message`` + event being acknowledged. + +``m.room.redaction`` + Summary: + Indicates a previous event has been redacted. + Type: + Non-state event + JSON format: + ``{ "reason": "string" }`` + Description: + Events can be redacted by either room or server admins. Redacting an event + means that all keys not required by the protocol are stripped off, allowing + admins to remove offensive or illegal content that may have been attached + to any event. This cannot be undone, allowing server owners to physically + delete the offending data. There is also a concept of a moderator hiding a + non-state event, which can be undone, but cannot be applied to state + events. + The event that has been redacted is specified in the ``redacts`` event + level key. + +m.room.message msgtypes +~~~~~~~~~~~~~~~~~~~~~~~ + +.. TODO-spec + How a client should handle unknown message types. + +Each ``m.room.message`` MUST have a ``msgtype`` key which identifies the type +of message being sent. Each type has their own required and optional keys, as +outlined below: + +``m.text`` + Required keys: + - ``body`` : "string" - The body of the message. + Optional keys: + None. + Example: + ``{ "msgtype": "m.text", "body": "I am a fish" }`` + +``m.emote`` + Required keys: + - ``body`` : "string" - The emote action to perform. + Optional keys: + None. + Example: + ``{ "msgtype": "m.emote", "body": "tries to come up with a witty explanation" }`` + +``m.image`` + Required keys: + - ``url`` : "string" - The URL to the image. + Optional keys: + - ``info`` : "string" - info : JSON object (ImageInfo) - The image info for + image referred to in ``url``. + - ``thumbnail_url`` : "string" - The URL to the thumbnail. + - ``thumbnail_info`` : JSON object (ImageInfo) - The image info for the + image referred to in ``thumbnail_url``. + - ``body`` : "string" - The alt text of the image, or some kind of content + description for accessibility e.g. "image attachment". + + ImageInfo: + Information about an image:: + + { + "size" : integer (size of image in bytes), + "w" : integer (width of image in pixels), + "h" : integer (height of image in pixels), + "mimetype" : "string (e.g. image/jpeg)", + } + +``m.audio`` + Required keys: + - ``url`` : "string" - The URL to the audio. + Optional keys: + - ``info`` : JSON object (AudioInfo) - The audio info for the audio + referred to in ``url``. + - ``body`` : "string" - A description of the audio e.g. "Bee Gees - Stayin' + Alive", or some kind of content description for accessibility e.g. + "audio attachment". + AudioInfo: + Information about a piece of audio:: + + { + "mimetype" : "string (e.g. audio/aac)", + "size" : integer (size of audio in bytes), + "duration" : integer (duration of audio in milliseconds), + } + +``m.video`` + Required keys: + - ``url`` : "string" - The URL to the video. + Optional keys: + - ``info`` : JSON object (VideoInfo) - The video info for the video + referred to in ``url``. + - ``body`` : "string" - A description of the video e.g. "Gangnam style", or + some kind of content description for accessibility e.g. "video + attachment". + + VideoInfo: + Information about a video:: + + { + "mimetype" : "string (e.g. video/mp4)", + "size" : integer (size of video in bytes), + "duration" : integer (duration of video in milliseconds), + "w" : integer (width of video in pixels), + "h" : integer (height of video in pixels), + "thumbnail_url" : "string (URL to image)", + "thumbanil_info" : JSON object (ImageInfo) + } + +``m.location`` + Required keys: + - ``geo_uri`` : "string" - The geo URI representing the location. + Optional keys: + - ``thumbnail_url`` : "string" - The URL to a thumnail of the location + being represented. + - ``thumbnail_info`` : JSON object (ImageInfo) - The image info for the + image referred to in ``thumbnail_url``. + - ``body`` : "string" - A description of the location e.g. "Big Ben, + London, UK", or some kind of content description for accessibility e.g. + "location attachment". + +The following keys can be attached to any ``m.room.message``: + + Optional keys: + - ``sender_ts`` : integer - A timestamp (ms resolution) representing the + wall-clock time when the message was sent from the client. + +Events on Change of Profile Information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Because the profile displayname and avatar information are likely to be used in +many places of a client's display, changes to these fields cause an automatic +propagation event to occur, informing likely-interested parties of the new +values. This change is conveyed using two separate mechanisms: + + - a ``m.room.member`` event is sent to every room the user is a member of, + to update the ``displayname`` and ``avatar_url``. + - a presence status update is sent, again containing the new values of the + ``displayname`` and ``avatar_url`` keys, in addition to the required + ``presence`` key containing the current presence state of the user. + +Both of these should be done automatically by the home server when a user +successfully changes their displayname or avatar URL fields. + +Additionally, when home servers emit room membership events for their own +users, they should include the displayname and avatar URL fields in these +events so that clients already have these details to hand, and do not have to +perform extra roundtrips to query it. + +Voice over IP +------------- +Matrix can also be used to set up VoIP calls. This is part of the core +specification, although is still in a very early stage. Voice (and video) over +Matrix is based on the WebRTC standards. + +Call events are sent to a room, like any other event. This means that clients +must only send call events to rooms with exactly two participants as currently +the WebRTC standard is based around two-party communication. + +Events +~~~~~~ +``m.call.invite`` +This event is sent by the caller when they wish to establish a call. + + Required keys: + - ``call_id`` : "string" - A unique identifier for the call + - ``offer`` : "offer object" - The session description + - ``version`` : "integer" - The version of the VoIP specification this + message adheres to. This specification is version 0. + - ``lifetime`` : "integer" - The time in milliseconds that the invite is + valid for. Once the invite age exceeds this value, clients should discard + it. They should also no longer show the call as awaiting an answer in the + UI. + + Optional keys: + None. + Example: + ``{ "version" : 0, "call_id": "12345", "offer": { "type" : "offer", "sdp" : "v=0\r\no=- 6584580628695956864 2 IN IP4 127.0.0.1[...]" } }`` + +``Offer Object`` + Required keys: + - ``type`` : "string" - The type of session description, in this case + 'offer' + - ``sdp`` : "string" - The SDP text of the session description + +``m.call.candidates`` +This event is sent by callers after sending an invite and by the callee after +answering. Its purpose is to give the other party additional ICE candidates to +try using to communicate. + + Required keys: + - ``call_id`` : "string" - The ID of the call this event relates to + - ``version`` : "integer" - The version of the VoIP specification this + messages adheres to. his specification is version 0. + - ``candidates`` : "array of candidate objects" - Array of object + describing the candidates. + +``Candidate Object`` + + Required Keys: + - ``sdpMid`` : "string" - The SDP media type this candidate is intended + for. + - ``sdpMLineIndex`` : "integer" - The index of the SDP 'm' line this + candidate is intended for + - ``candidate`` : "string" - The SDP 'a' line of the candidate + +``m.call.answer`` + + Required keys: + - ``call_id`` : "string" - The ID of the call this event relates to + - ``version`` : "integer" - The version of the VoIP specification this + messages + - ``answer`` : "answer object" - Object giving the SDK answer + +``Answer Object`` + + Required keys: + - ``type`` : "string" - The type of session description. 'answer' in this + case. + - ``sdp`` : "string" - The SDP text of the session description + +``m.call.hangup`` +Sent by either party to signal their termination of the call. This can be sent +either once the call has has been established or before to abort the call. + + Required keys: + - ``call_id`` : "string" - The ID of the call this event relates to + - ``version`` : "integer" - The version of the VoIP specification this + messages + +Message Exchange +~~~~~~~~~~~~~~~~ +A call is set up with messages exchanged as follows: + +:: + + Caller Callee + m.call.invite -----------> + m.call.candidate --------> + [more candidates events] + User answers call + <------ m.call.answer + [...] + <------ m.call.hangup + +Or a rejected call: + +:: + + Caller Callee + m.call.invite -----------> + m.call.candidate --------> + [more candidates events] + User rejects call + <------- m.call.hangup + +Calls are negotiated according to the WebRTC specification. + + +Glare +~~~~~ +This specification aims to address the problem of two users calling each other +at roughly the same time and their invites crossing on the wire. It is a far +better experience for the users if their calls are connected if it is clear +that their intention is to set up a call with one another. + +In Matrix, calls are to rooms rather than users (even if those rooms may only +contain one other user) so we consider calls which are to the same room. + +The rules for dealing with such a situation are as follows: + + - If an invite to a room is received whilst the client is preparing to send an + invite to the same room, the client should cancel its outgoing call and + instead automatically accept the incoming call on behalf of the user. + - If an invite to a room is received after the client has sent an invite to + the same room and is waiting for a response, the client should perform a + lexicographical comparison of the call IDs of the two calls and use the + lesser of the two calls, aborting the greater. If the incoming call is the + lesser, the client should accept this call on behalf of the user. + +The call setup should appear seamless to the user as if they had simply placed +a call and the other party had accepted. Thusly, any media stream that had been +setup for use on a call should be transferred and used for the call that +replaces it. + +Client-Server API +================= + +Registration and Login +---------------------- + +Clients must register with a home server in order to use Matrix. After +registering, the client will be given an access token which must be used in ALL +requests to that home server as a query parameter 'access_token'. + +If the client has already registered, they need to be able to login to their +account. The home server may provide many different ways of logging in, such as +user/password auth, login via a social network (OAuth2), login by confirming a +token sent to their email address, etc. This specification does not define how +home servers should authorise their users who want to login to their existing +accounts, but instead defines the standard interface which implementations +should follow so that ANY client can login to ANY home server. Clients login +using the |login|_ API. Clients register using the |register|_ API. +Registration follows the same general procedure as login, but the path requests +are sent to and the details contained in them are different. + +In both registration and login cases, the process takes the form of one or more +stages, where at each stage the client submits a set of data for a given stage +type and awaits a response from the server, which will either be a final +success or a request to perform an additional stage. This exchange continues +until the final success. + +In order to determine up-front what the server's requirements are, the client +can request from the server a complete description of all of its acceptable +flows of the registration or login process. It can then inspect the list of +returned flows looking for one for which it believes it can complete all of the +required stages, and perform it. As each home server may have different ways of +logging in, the client needs to know how they should login. All distinct login +stages MUST have a corresponding ``type``. A ``type`` is a namespaced string +which details the mechanism for logging in. + +A client may be able to login via multiple valid login flows, and should choose +a single flow when logging in. A flow is a series of login stages. The home +server MUST respond with all the valid login flows when requested by a simple +``GET`` request directly to the ``/login`` or ``/register`` paths:: + + { + "flows": [ + { + "type": "", + "stages": [ "", "" ] + }, + { + "type": "", + "stages": [ "", "" ] + }, + { + "type": "" + } + ] + } + +The client can now select which flow it wishes to use, and begin making +``POST`` requests to the ``/login`` or ``/register`` paths with JSON body +content containing the name of the stage as the ``type`` key, along with +whatever additional parameters are required for that login or registration type +(see below). After the flow is completed, the client's fully-qualified user +ID and a new access token MUST be returned:: + + { + "user_id": "@user:matrix.org", + "access_token": "abcdef0123456789" + } + +The ``user_id`` key is particularly useful if the home server wishes to support +localpart entry of usernames (e.g. "user" rather than "@user:matrix.org"), as +the client may not be able to determine its ``user_id`` in this case. + +If the flow has multiple stages to it, the home server may wish to create a +session to store context between requests. If a home server responds with a +``session`` key to a request, clients MUST submit it in subsequent requests +until the flow is completed:: + + { + "session": "" + } + +This specification defines the following login types: + - ``m.login.password`` + - ``m.login.oauth2`` + - ``m.login.email.code`` + - ``m.login.email.url`` + - ``m.login.email.identity`` + +Password-based +~~~~~~~~~~~~~~ +:Type: + ``m.login.password`` +:Description: + Login is supported via a username and password. + +To respond to this type, reply with:: + + { + "type": "m.login.password", + "user": "", + "password": "" + } + +The home server MUST respond with either new credentials, the next stage of the +login process, or a standard error response. + +OAuth2-based +~~~~~~~~~~~~ +:Type: + ``m.login.oauth2`` +:Description: + Login is supported via OAuth2 URLs. This login consists of multiple requests. + +To respond to this type, reply with:: + + { + "type": "m.login.oauth2", + "user": "" + } + +The server MUST respond with:: + + { + "uri": + } + +The home server acts as a 'confidential' client for the purposes of OAuth2. If +the uri is a ``sevice selection URI``, it MUST point to a webpage which prompts +the user to choose which service to authorize with. On selection of a service, +this MUST link through to an ``Authorization Request URI``. If there is only 1 +service which the home server accepts when logging in, this indirection can be +skipped and the "uri" key can be the ``Authorization Request URI``. + +The client then visits the ``Authorization Request URI``, which then shows the +OAuth2 Allow/Deny prompt. Hitting 'Allow' returns the ``redirect URI`` with the +auth code. Home servers can choose any path for the ``redirect URI``. The +client should visit the ``redirect URI``, which will then finish the OAuth2 +login process, granting the home server an access token for the chosen service. +When the home server gets this access token, it verifies that the cilent has +authorised with the 3rd party, and can now complete the login. The OAuth2 +``redirect URI`` (with auth code) MUST respond with either new credentials, the +next stage of the login process, or a standard error response. + +For example, if a home server accepts OAuth2 from Google, it would return the +Authorization Request URI for Google:: + + { + "uri": "https://accounts.google.com/o/oauth2/auth?response_type=code& + client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos" + } + +The client then visits this URI and authorizes the home server. The client then +visits the REDIRECT_URI with the auth code= query parameter which returns:: + + { + "user_id": "@user:matrix.org", + "access_token": "0123456789abcdef" + } + +Email-based (code) +~~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.code`` +:Description: + Login is supported by typing in a code which is sent in an email. This login + consists of multiple requests. + +To respond to this type, reply with:: + + { + "type": "m.login.email.code", + "user": "", + "email": "" + } + +After validating the email address, the home server MUST send an email +containing an authentication code and return:: + + { + "type": "m.login.email.code", + "session": "" + } + +The second request in this login stage involves sending this authentication +code:: + + { + "type": "m.login.email.code", + "session": "", + "code": "" + } + +The home server MUST respond to this with either new credentials, the next +stage of the login process, or a standard error response. + +Email-based (url) +~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.url`` +:Description: + Login is supported by clicking on a URL in an email. This login consists of + multiple requests. + +To respond to this type, reply with:: + + { + "type": "m.login.email.url", + "user": "", + "email": "" + } + +After validating the email address, the home server MUST send an email +containing an authentication URL and return:: + + { + "type": "m.login.email.url", + "session": "" + } + +The email contains a URL which must be clicked. After it has been clicked, the +client should perform another request:: + + { + "type": "m.login.email.url", + "session": "" + } + +The home server MUST respond to this with either new credentials, the next +stage of the login process, or a standard error response. + +A common client implementation will be to periodically poll until the link is +clicked. If the link has not been visited yet, a standard error response with +an errcode of ``M_LOGIN_EMAIL_URL_NOT_YET`` should be returned. + + +Email-based (identity server) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +:Type: + ``m.login.email.identity`` +:Description: + Login is supported by authorising an email address with an identity server. + +Prior to submitting this, the client should authenticate with an identity +server. After authenticating, the session information should be submitted to +the home server. + +To respond to this type, reply with:: + + { + "type": "m.login.email.identity", + "threepidCreds": [ + { + "sid": "", + "clientSecret": "", + "idServer": "" + } + ] + } + + + +N-Factor Authentication +~~~~~~~~~~~~~~~~~~~~~~~ +Multiple login stages can be combined to create N-factor authentication during +login. + +This can be achieved by responding with the ``next`` login type on completion +of a previous login stage:: + + { + "next": "" + } + +If a home server implements N-factor authentication, it MUST respond with all +``stages`` when initially queried for their login requirements:: + + { + "type": "<1st login type>", + "stages": [ <1st login type>, <2nd login type>, ... , ] + } + +This can be represented conceptually as:: + + _______________________ + | Login Stage 1 | + | type: "" | + | ___________________ | + | |_Request_1_________| | <-- Returns "session" key which is used throughout. + | ___________________ | + | |_Request_2_________| | <-- Returns a "next" value of "login type2" + |_______________________| + | + | + _________V_____________ + | Login Stage 2 | + | type: "" | + | ___________________ | + | |_Request_1_________| | + | ___________________ | + | |_Request_2_________| | + | ___________________ | + | |_Request_3_________| | <-- Returns a "next" value of "login type3" + |_______________________| + | + | + _________V_____________ + | Login Stage 3 | + | type: "" | + | ___________________ | + | |_Request_1_________| | <-- Returns user credentials + |_______________________| + +Fallback +~~~~~~~~ +Clients cannot be expected to be able to know how to process every single login +type. If a client determines it does not know how to handle a given login type, +it should request a login fallback page:: + + GET matrix/client/api/v1/login/fallback + +This MUST return an HTML page which can perform the entire login process. + + +Rooms +----- + +Creation +~~~~~~~~ +To create a room, a client has to use the |createRoom|_ API. There are various +options which can be set when creating a room: + +``visibility`` + Type: + String + Optional: + Yes + Value: + Either ``public`` or ``private``. + Description: + A ``public`` visibility indicates that the room will be shown in the public + room list. A ``private`` visibility will hide the room from the public room + list. Rooms default to ``private`` visibility if this key is not included. + +``room_alias_name`` + Type: + String + Optional: + Yes + Value: + The room alias localpart. + Description: + If this is included, a room alias will be created and mapped to the newly + created room. The alias will belong on the same home server which created + the room, e.g. ``!qadnasoi:domain.com >>> #room_alias_name:domain.com`` + +``name`` + Type: + String + Optional: + Yes + Value: + The ``name`` value for the ``m.room.name`` state event. + Description: + If this is included, an ``m.room.name`` event will be sent into the room to + indicate the name of the room. See `Room Events`_ for more information on + ``m.room.name``. + +``topic`` + Type: + String + Optional: + Yes + Value: + The ``topic`` value for the ``m.room.topic`` state event. + Description: + If this is included, an ``m.room.topic`` event will be sent into the room + to indicate the topic for the room. See `Room Events`_ for more information + on ``m.room.topic``. + +``invite`` + Type: + List + Optional: + Yes + Value: + A list of user ids to invite. + Description: + This will tell the server to invite everyone in the list to the newly + created room. + +Example:: + + { + "visibility": "public", + "room_alias_name": "thepub", + "name": "The Grand Duke Pub", + "topic": "All about happy hour" + } + +The home server will create a ``m.room.create`` event when the room is created, +which serves as the root of the PDU graph for this room. This event also has a +``creator`` key which contains the user ID of the room creator. It will also +generate several other events in order to manage permissions in this room. This +includes: + + - ``m.room.power_levels`` : Sets the power levels of users. + - ``m.room.join_rules`` : Whether the room is "invite-only" or not. + - ``m.room.add_state_level``: The power level required in order to add new + state to the room (as opposed to updating exisiting state) + - ``m.room.send_event_level`` : The power level required in order to send a + message in this room. + - ``m.room.ops_level`` : The power level required in order to kick or ban a + user from the room or redact an event in the room. + +See `Room Events`_ for more information on these events. + +Room aliases +~~~~~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +Room aliases can be created by sending a ``PUT /directory/room/``:: + + { + "room_id": + } + +They can be deleted by sending a ``DELETE /directory/room/`` with +no content. Only some privileged users may be able to delete room aliases, e.g. +server admins, the creator of the room alias, etc. This specification does not +outline the privilege level required for deleting room aliases. + +As room aliases are scoped to a particular home server domain name, it is +likely that a home server will reject attempts to maintain aliases on other +domain names. This specification does not provide a way for home servers to +send update requests to other servers. + +Rooms store a *partial* list of room aliases via the ``m.room.aliases`` state +event. This alias list is partial because it cannot guarantee that the alias +list is in any way accurate or up-to-date, as room aliases can point to +different room IDs over time. Crucially, the aliases in this event are +**purely informational** and SHOULD NOT be treated as accurate. They SHOULD +be checked before they are used or shared with another user. If a room +appears to have a room alias of ``#alias:example.com``, this SHOULD be checked +to make sure that the room's ID matches the ``room_id`` returned from the +request. + +Room aliases can be checked in the same way they are resolved; by sending a +``GET /directory/room/``:: + + { + "room_id": , + "servers": [ , , ] + } + +Home servers can respond to resolve requests for aliases on other domains than +their own by using the federation API to ask other domain name home servers. + + +Permissions +~~~~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +Permissions for rooms are done via the concept of power levels - to do any +action in a room a user must have a suitable power level. Power levels are +stored as state events in a given room. + +Power levels for users are defined in ``m.room.power_levels``, where both a +default and specific users' power levels can be set:: + + { + "": , + "": , + "default": 0 + } + +By default all users have a power level of 0, other than the room creator whose +power level defaults to 100. Users can grant other users increased power levels +up to their own power level. For example, user A with a power level of 50 could +increase the power level of user B to a maximum of level 50. Power levels for +users are tracked per-room even if the user is not present in the room. + +State events may contain a ``required_power_level`` key, which indicates the +minimum power a user must have before they can update that state key. The only +exception to this is when a user leaves a room, which revokes the user's right +to update state events in that room. + +To perform certain actions there are additional power level requirements +defined in the following state events: + +- ``m.room.send_event_level`` defines the minimum ``level`` for sending + non-state events. Defaults to 50. +- ``m.room.add_state_level`` defines the minimum ``level`` for adding new + state, rather than updating existing state. Defaults to 50. +- ``m.room.ops_level`` defines the minimum ``ban_level`` and ``kick_level`` to + ban and kick other users respectively. This defaults to a kick and ban levels + of 50 each. + + +Joining rooms +~~~~~~~~~~~~~ +.. TODO-doc What does the home server have to do to join a user to a room? + - See SPEC-30. + +Users need to join a room in order to send and receive events in that room. A +user can join a room by making a request to |/join/|_ with:: + + {} + +Alternatively, a user can make a request to |/rooms//join|_ with the +same request content. This is only provided for symmetry with the other +membership APIs: ``/rooms//invite`` and ``/rooms//leave``. If +a room alias was specified, it will be automatically resolved to a room ID, +which will then be joined. The room ID that was joined will be returned in +response:: + + { + "room_id": "!roomid:domain" + } + +The membership state for the joining user can also be modified directly to be +``join`` by sending the following request to +``/rooms//state/m.room.member/``:: + + { + "membership": "join" + } + +See the `Room events`_ section for more information on ``m.room.member``. + +After the user has joined a room, they will receive subsequent events in that +room. This room will now appear as an entry in the |initialSync|_ API. + +Some rooms enforce that a user is *invited* to a room before they can join that +room. Other rooms will allow anyone to join the room even if they have not +received an invite. + +Inviting users +~~~~~~~~~~~~~~ +.. TODO-doc Invite-join dance + - Outline invite join dance. What is it? Why is it required? How does it work? + - What does the home server have to do? + +The purpose of inviting users to a room is to notify them that the room exists +so they can choose to become a member of that room. Some rooms require that all +users who join a room are previously invited to it (an "invite-only" room). +Whether a given room is an "invite-only" room is determined by the room config +key ``m.room.join_rules``. It can have one of the following values: + +``public`` + This room is free for anyone to join without an invite. + +``invite`` + This room can only be joined if you were invited. + +Only users who have a membership state of ``join`` in a room can invite new +users to said room. The person being invited must not be in the ``join`` state +in the room. The fully-qualified user ID must be specified when inviting a +user, as the user may reside on a different home server. To invite a user, send +the following request to |/rooms//invite|_, which will manage the +entire invitation process:: + + { + "user_id": "" + } + +Alternatively, the membership state for this user in this room can be modified +directly by sending the following request to +``/rooms//state/m.room.member/``:: + + { + "membership": "invite" + } + +See the `Room events`_ section for more information on ``m.room.member``. + +Leaving rooms +~~~~~~~~~~~~~ +.. TODO-spec - HS deleting rooms they are no longer a part of. Not implemented. + - This is actually Very Tricky. If all clients a HS is serving leave a room, + the HS will no longer get any new events for that room, because the servers + who get the events are determined on the *membership list*. There should + probably be a way for a HS to lurk on a room even if there are 0 of their + members in the room. + - Grace period before deletion? + - Under what conditions should a room NOT be purged? + + +A user can leave a room to stop receiving events for that room. A user must +have joined the room before they are eligible to leave the room. If the room is +an "invite-only" room, they will need to be re-invited before they can re-join +the room. To leave a room, a request should be made to +|/rooms//leave|_ with:: + + {} + +Alternatively, the membership state for this user in this room can be modified +directly by sending the following request to +``/rooms//state/m.room.member/``:: + + { + "membership": "leave" + } + +See the `Room events`_ section for more information on ``m.room.member``. + +Once a user has left a room, that room will no longer appear on the +|initialSync|_ API. + +If all members in a room leave, that room becomes eligible for deletion. + +Banning users in a room +~~~~~~~~~~~~~~~~~~~~~~~ +A user may decide to ban another user in a room. 'Banning' forces the target +user to leave the room and prevents them from re-joining the room. A banned +user will not be treated as a joined user, and so will not be able to send or +receive events in the room. In order to ban someone, the user performing the +ban MUST have the required power level. To ban a user, a request should be made +to |/rooms//ban|_ with:: + + { + "user_id": "" + } + +Banning a user adjusts the banned member's membership state to ``ban`` and +adjusts the power level of this event to a level higher than the banned person. +Like with other membership changes, a user can directly adjust the target +member's state, by making a request to +``/rooms//state/m.room.member/``:: + + { + "membership": "ban" + } + +Events in a room +~~~~~~~~~~~~~~~~ +Room events can be split into two categories: + +:State Events: + These are events which replace events that came before it, depending on a set + of unique keys. These keys are the event ``type`` and a ``state_key``. + Events with the same set of keys will be overwritten. Typically, state events + are used to store state, hence their name. + +:Non-state events: + These are events which cannot be overwritten after sending. The list of + events continues to grow as more events are sent. As this list grows, it + becomes necessary to provide a mechanism for navigating this list. Pagination + APIs are used to view the list of historical non-state events. Typically, + non-state events are used to send messages. + +This specification outlines several events, all with the event type prefix +``m.``. However, applications may wish to add their own type of event, and this +can be achieved using the REST API detailed in the following sections. If new +events are added, the event ``type`` key SHOULD follow the Java package naming +convention, e.g. ``com.example.myapp.event``. This ensures event types are +suitably namespaced for each application and reduces the risk of clashes. + +State events +~~~~~~~~~~~~ +State events can be sent by ``PUT`` ing to +|/rooms//state//|_. These events will be +overwritten if ````, ```` and ```` all match. +If the state event has no ``state_key``, it can be omitted from the path. These +requests **cannot use transaction IDs** like other ``PUT`` paths because they +cannot be differentiated from the ``state_key``. Furthermore, ``POST`` is +unsupported on state paths. Valid requests look like:: + + PUT /rooms/!roomid:domain/state/m.example.event + { "key" : "without a state key" } + + PUT /rooms/!roomid:domain/state/m.another.example.event/foo + { "key" : "with 'foo' as the state key" } + +In contrast, these requests are invalid:: + + POST /rooms/!roomid:domain/state/m.example.event/ + { "key" : "cannot use POST here" } + + PUT /rooms/!roomid:domain/state/m.another.example.event/foo/11 + { "key" : "txnIds are not supported" } + +Care should be taken to avoid setting the wrong ``state key``:: + + PUT /rooms/!roomid:domain/state/m.another.example.event/11 + { "key" : "with '11' as the state key, but was probably intended to be a txnId" } + +The ``state_key`` is often used to store state about individual users, by using +the user ID as the ``state_key`` value. For example:: + + PUT /rooms/!roomid:domain/state/m.favorite.animal.event/%40my_user%3Adomain.com + { "animal" : "cat", "reason": "fluffy" } + +In some cases, there may be no need for a ``state_key``, so it can be omitted:: + + PUT /rooms/!roomid:domain/state/m.room.bgd.color + { "color": "red", "hex": "#ff0000" } + +See `Room Events`_ for the ``m.`` event specification. + +Non-state events +~~~~~~~~~~~~~~~~ +Non-state events can be sent by sending a request to +|/rooms//send/|_. These requests *can* use transaction +IDs and ``PUT``/``POST`` methods. Non-state events allow access to historical +events and pagination, making it best suited for sending messages. For +example:: + + POST /rooms/!roomid:domain/send/m.custom.example.message + { "text": "Hello world!" } + + PUT /rooms/!roomid:domain/send/m.custom.example.message/11 + { "text": "Goodbye world!" } + +See `Room Events`_ for the ``m.`` event specification. + +Syncing rooms +~~~~~~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +When a client logs in, they may have a list of rooms which they have already +joined. These rooms may also have a list of events associated with them. The +purpose of 'syncing' is to present the current room and event information in a +convenient, compact manner. The events returned are not limited to room events; +presence events will also be returned. A single syncing API is provided: + + - |initialSync|_ : A global sync which will present room and event information + for all rooms the user has joined. + +.. TODO-spec room-scoped initial sync + - |/rooms//initialSync|_ : A sync scoped to a single room. Presents + room and event information for this room only. + - Room-scoped initial sync is Very Tricky because typically people would + want to sync the room then listen for any new content from that point + onwards. The event stream cannot do this for a single room currently. + As a result, commenting room-scoped initial sync at this time. + +The |initialSync|_ API contains the following keys: + +``presence`` + Description: + Contains a list of presence information for users the client is interested + in. + Format: + A JSON array of ``m.presence`` events. + +``end`` + Description: + Contains an event stream token which can be used with the `Event Stream`_. + Format: + A string containing the event stream token. + +``rooms`` + Description: + Contains a list of room information for all rooms the client has joined, + and limited room information on rooms the client has been invited to. + Format: + A JSON array containing Room Information JSON objects. + +Room Information: + Description: + Contains all state events for the room, along with a limited amount of + the most recent non-state events, configured via the ``limit`` query + parameter. Also contains additional keys with room metadata, such as the + ``room_id`` and the client's ``membership`` to the room. + Format: + A JSON object with the following keys: + ``room_id`` + A string containing the ID of the room being described. + ``membership`` + A string representing the client's membership status in this room. + ``messages`` + An event stream JSON object containing a ``chunk`` of recent non-state + events, along with an ``end`` token. *NB: The name of this key will be + changed in a later version.* + ``state`` + A JSON array containing all the current state events for this room. + +Getting events for a room +~~~~~~~~~~~~~~~~~~~~~~~~~ +There are several APIs provided to ``GET`` events for a room: + +``/rooms//state//`` + Description: + Get the state event identified. + Response format: + A JSON object representing the state event **content**. + Example: + ``/rooms/!room:domain.com/state/m.room.name`` returns ``{ "name": "Room name" }`` + +|/rooms//state|_ + Description: + Get all state events for a room. + Response format: + ``[ { state event }, { state event }, ... ]`` + Example: + TODO-doc + + +|/rooms//members|_ + Description: + Get all ``m.room.member`` state events. + Response format: + ``{ "start": "", "end": "", "chunk": [ { m.room.member event }, ... ] }`` + Example: + TODO-doc + +|/rooms//messages|_ + Description: + Get all ``m.room.message`` and ``m.room.member`` events. This API supports + pagination using ``from`` and ``to`` query parameters, coupled with the + ``start`` and ``end`` tokens from an |initialSync|_ API. + Response format: + ``{ "start": "", "end": "" }`` + Example: + TODO-doc + +|/rooms//initialSync|_ + Description: + Get all relevant events for a room. This includes state events, paginated + non-state events and presence events. + Response format: + `` { TODO-doc } `` + Example: + TODO-doc + +Redactions +~~~~~~~~~~ +Since events are extensible it is possible for malicious users and/or servers +to add keys that are, for example offensive or illegal. Since some events +cannot be simply deleted, e.g. membership events, we instead 'redact' events. +This involves removing all keys from an event that are not required by the +protocol. This stripped down event is thereafter returned anytime a client or +remote server requests it. + +Events that have been redacted include a ``redacted_because`` key whose value +is the event that caused it to be redacted, which may include a reason. + +Redacting an event cannot be undone, allowing server owners to delete the +offending content from the databases. + +Currently, only room admins can redact events by sending a ``m.room.redaction`` +event, but server admins also need to be able to redact events by a similar +mechanism. + +Upon receipt of a redaction event, the server should strip off any keys not in +the following list: + + - ``event_id`` + - ``type`` + - ``room_id`` + - ``user_id`` + - ``state_key`` + - ``prev_state`` + - ``content`` + +The content object should also be stripped of all keys, unless it is one of +one of the following event types: + + - ``m.room.member`` allows key ``membership`` + - ``m.room.create`` allows key ``creator`` + - ``m.room.join_rules`` allows key ``join_rule`` + - ``m.room.power_levels`` allows keys that are user ids or ``default`` + - ``m.room.add_state_level`` allows key ``level`` + - ``m.room.send_event_level`` allows key ``level`` + - ``m.room.ops_levels`` allows keys ``kick_level``, ``ban_level`` + and ``redact_level`` + - ``m.room.aliases`` allows key ``aliases`` + +The redaction event should be added under the key ``redacted_because``. + + +When a client receives a redaction event it should change the redacted event +in the same way a server does. + +Presence +~~~~~~~~ +The client API for presence is on the following set of REST calls. + +Fetching basic status:: + + GET $PREFIX/presence/:user_id/status + + Returned content: JSON object containing the following keys: + presence: "offline"|"unavailable"|"online"|"free_for_chat" + status_msg: (optional) string of freeform text + last_active_ago: miliseconds since the last activity by the user + +Setting basic status:: + + PUT $PREFIX/presence/:user_id/status + + Content: JSON object containing the following keys: + presence and status_msg: as above + +When setting the status, the activity time is updated to reflect that activity; +the client does not need to specify the ``last_active_ago`` field. + +Fetching the presence list:: + + GET $PREFIX/presence/list + + Returned content: JSON array containing objects; each object containing the + following keys: + user_id: observed user ID + presence: "offline"|"unavailable"|"online"|"free_for_chat" + status_msg: (optional) string of freeform text + last_active_ago: miliseconds since the last activity by the user + +Maintaining the presence list:: + + POST $PREFIX/presence/list + + Content: JSON object containing either or both of the following keys: + invite: JSON array of strings giving user IDs to send invites to + drop: JSON array of strings giving user IDs to remove from the list + +.. TODO-spec + - Define how users receive presence invites, and how they accept/decline them + +Profiles +~~~~~~~~ +The client API for profile management consists of the following REST calls. + +Fetching a user account displayname:: + + GET $PREFIX/profile/:user_id/displayname + + Returned content: JSON object containing the following keys: + displayname: string of freeform text + +This call may be used to fetch the user's own displayname or to query the name +of other users; either locally or on remote systems hosted on other home +servers. + +Setting a new displayname:: + + PUT $PREFIX/profile/:user_id/displayname + + Content: JSON object containing the following keys: + displayname: string of freeform text + +Fetching a user account avatar URL:: + + GET $PREFIX/profile/:user_id/avatar_url + + Returned content: JSON object containing the following keys: + avatar_url: string containing an http-scheme URL + +As with displayname, this call may be used to fetch either the user's own, or +other users' avatar URL. + +Setting a new avatar URL:: + + PUT $PREFIX/profile/:user_id/avatar_url + + Content: JSON object containing the following keys: + avatar_url: string containing an http-scheme URL + +Fetching combined account profile information:: + + GET $PREFIX/profile/:user_id + + Returned content: JSON object containing the following keys: + displayname: string of freeform text + avatar_url: string containing an http-scheme URL + +At the current time, this API simply returns the displayname and avatar URL +information, though it is intended to return more fields about the user's +profile once they are defined. Client implementations should take care not to +expect that these are the only two keys returned as future versions of this +specification may yield more keys here. + +Security +-------- + +Rate limiting +~~~~~~~~~~~~~ +Home servers SHOULD implement rate limiting to reduce the risk of being +overloaded. If a request is refused due to rate limiting, it should return a +standard error response of the form:: + + { + "errcode": "M_LIMIT_EXCEEDED", + "error": "string", + "retry_after_ms": integer (optional) + } + +The ``retry_after_ms`` key SHOULD be included to tell the client how long they +have to wait in milliseconds before they can try again. + +.. TODO-spec + - Surely we should recommend an algorithm for the rate limiting, rather than letting every + homeserver come up with their own idea, causing totally unpredictable performance over + federated rooms? + +End-to-End Encryption +~~~~~~~~~~~~~~~~~~~~~ + +.. TODO-doc + - Why is this needed. + - Overview of process + - Implementation + +Content repository +------------------ +.. NOTE:: + This section is a work in progress. + +.. TODO-spec + - path to upload + - format for thumbnail paths, mention what it is protecting against. + - content size limit and associated M_ERROR. + + +Address book repository +----------------------- +.. NOTE:: + This section is a work in progress. + +.. TODO-spec + - format: POST(?) wodges of json, some possible processing, then return wodges of json on GET. + - processing may remove dupes, merge contacts, pepper with extra info (e.g. matrix-ability of + contacts), etc. + - Standard json format for contacts? Piggy back off vcards? + +Federation API +=============== + +Federation is the term used to describe how to communicate between Matrix home +servers. Federation is a mechanism by which two home servers can exchange +Matrix event messages, both as a real-time push of current events, and as a +historic fetching mechanism to synchronise past history for clients to view. It +uses HTTPS connections between each pair of servers involved as the underlying +transport. Messages are exchanged between servers in real-time by active +pushing from each server's HTTP client into the server of the other. Queries to +fetch historic data for the purpose of back-filling scrollback buffers and the +like can also be performed. Currently routing of messages between homeservers +is full mesh (like email) - however, fan-out refinements to this design are +currently under consideration. + +There are three main kinds of communication that occur between home servers: + +:Queries: + These are single request/response interactions between a given pair of + servers, initiated by one side sending an HTTPS GET request to obtain some + information, and responded by the other. They are not persisted and contain + no long-term significant history. They simply request a snapshot state at + the instant the query is made. + +:Ephemeral Data Units (EDUs): + These are notifications of events that are pushed from one home server to + another. They are not persisted and contain no long-term significant + history, nor does the receiving home server have to reply to them. + +:Persisted Data Units (PDUs): + These are notifications of events that are broadcast from one home server to + any others that are interested in the same "context" (namely, a Room ID). + They are persisted to long-term storage and form the record of history for + that context. + +EDUs and PDUs are further wrapped in an envelope called a Transaction, which is +transferred from the origin to the destination home server using an HTTP PUT +request. + + +Transactions +------------ +.. WARNING:: + This section may be misleading or inaccurate. + +The transfer of EDUs and PDUs between home servers is performed by an exchange +of Transaction messages, which are encoded as JSON objects, passed over an HTTP +PUT request. A Transaction is meaningful only to the pair of home servers that +exchanged it; they are not globally-meaningful. + +Each transaction has: + - An opaque transaction ID. + - A timestamp (UNIX epoch time in milliseconds) generated by its origin + server. + - An origin and destination server name. + - A list of "previous IDs". + - A list of PDUs and EDUs - the actual message payload that the Transaction + carries. + +``origin`` + Type: + String + Description: + DNS name of homeserver making this transaction. + +``ts`` + Type: + Integer + Description: + Timestamp in milliseconds on originating homeserver when this transaction + started. + +``previous_ids`` + Type: + List of strings + Description: + List of transactions that were sent immediately prior to this transaction. + +``pdus`` + Type: + List of Objects. + Description: + List of updates contained in this transaction. + +:: + + { + "transaction_id":"916d630ea616342b42e98a3be0b74113", + "ts":1404835423000, + "origin":"red", + "destination":"blue", + "prev_ids":["e1da392e61898be4d2009b9fecce5325"], + "pdus":[...], + "edus":[...] + } + +The ``prev_ids`` field contains a list of previous transaction IDs that the +``origin`` server has sent to this ``destination``. Its purpose is to act as a +sequence checking mechanism - the destination server can check whether it has +successfully received that Transaction, or ask for a retransmission if not. + +The ``pdus`` field of a transaction is a list, containing zero or more PDUs.[*] +Each PDU is itself a JSON object containing a number of keys, the exact details +of which will vary depending on the type of PDU. Similarly, the ``edus`` field +is another list containing the EDUs. This key may be entirely absent if there +are no EDUs to transfer. + +(* Normally the PDU list will be non-empty, but the server should cope with +receiving an "empty" transaction.) + +PDUs and EDUs +------------- +.. WARNING:: + This section may be misleading or inaccurate. + +All PDUs have: + - An ID + - A context + - A declaration of their type + - A list of other PDU IDs that have been seen recently on that context + (regardless of which origin sent them) + +``context`` + Type: + String + Description: + Event context identifier + +``origin`` + Type: + String + Description: + DNS name of homeserver that created this PDU. + +``pdu_id`` + Type: + String + Description: + Unique identifier for PDU within the context for the originating homeserver + +``ts`` + Type: + Integer + Description: + Timestamp in milliseconds on originating homeserver when this PDU was + created. + +``pdu_type`` + Type: + String + Description: + PDU event type. + +``prev_pdus`` + Type: + List of pairs of strings + Description: + The originating homeserver and PDU ids of the most recent PDUs the + homeserver was aware of for this context when it made this PDU. + +``depth`` + Type: + Integer + Description: + The maximum depth of the previous PDUs plus one. + + +.. TODO-spec paul + - Update this structure so that 'pdu_id' is a two-element [origin,ref] pair + like the prev_pdus are + +For state updates: + +``is_state`` + Type: + Boolean + Description: + True if this PDU is updating state. + +``state_key`` + Type: + String + Description: + Optional key identifying the updated state within the context. + +``power_level`` + Type: + Integer + Description: + The asserted power level of the user performing the update. + +``required_power_level`` + Type: + Integer + Description: + The required power level needed to replace this update. + +``prev_state_id`` + Type: + String + Description: + PDU event type. + +``prev_state_origin`` + Type: + String + Description: + The PDU id of the update this replaces. + +``user_id`` + Type: + String + Description: + The user updating the state. + +:: + + { + "pdu_id":"a4ecee13e2accdadf56c1025af232176", + "context":"#example.green", + "origin":"green", + "ts":1404838188000, + "pdu_type":"m.text", + "prev_pdus":[["blue","99d16afbc857975916f1d73e49e52b65"]], + "content":... + "is_state":false + } + +In contrast to Transactions, it is important to note that the ``prev_pdus`` +field of a PDU refers to PDUs that any origin server has sent, rather than +previous IDs that this ``origin`` has sent. This list may refer to other PDUs +sent by the same origin as the current one, or other origins. + +Because of the distributed nature of participants in a Matrix conversation, it +is impossible to establish a globally-consistent total ordering on the events. +However, by annotating each outbound PDU at its origin with IDs of other PDUs +it has received, a partial ordering can be constructed allowing causality +relationships to be preserved. A client can then display these messages to the +end-user in some order consistent with their content and ensure that no message +that is semantically in reply of an earlier one is ever displayed before it. + +PDUs fall into two main categories: those that deliver Events, and those that +synchronise State. For PDUs that relate to State synchronisation, additional +keys exist to support this: + +:: + + {..., + "is_state":true, + "state_key":TODO-doc + "power_level":TODO-doc + "prev_state_id":TODO-doc + "prev_state_origin":TODO-doc} + +EDUs, by comparison to PDUs, do not have an ID, a context, or a list of +"previous" IDs. The only mandatory fields for these are the type, origin and +destination home server names, and the actual nested content. + +:: + + {"edu_type":"m.presence", + "origin":"blue", + "destination":"orange", + "content":...} + + +Protocol URLs +------------- +.. WARNING:: + This section may be misleading or inaccurate. + +All these URLs are namespaced within a prefix of:: + + /_matrix/federation/v1/... + +For active pushing of messages representing live activity "as it happens":: + + PUT .../send/:transaction_id/ + Body: JSON encoding of a single Transaction + Response: TODO-doc + +The transaction_id path argument will override any ID given in the JSON body. +The destination name will be set to that of the receiving server itself. Each +embedded PDU in the transaction body will be processed. + + +To fetch a particular PDU:: + + GET .../pdu/:origin/:pdu_id/ + Response: JSON encoding of a single Transaction containing one PDU + +Retrieves a given PDU from the server. The response will contain a single new +Transaction, inside which will be the requested PDU. + + +To fetch all the state of a given context:: + + GET .../state/:context/ + Response: JSON encoding of a single Transaction containing multiple PDUs + +Retrieves a snapshot of the entire current state of the given context. The +response will contain a single Transaction, inside which will be a list of PDUs +that encode the state. + +To backfill events on a given context:: + + GET .../backfill/:context/ + Query args: v, limit + Response: JSON encoding of a single Transaction containing multiple PDUs + +Retrieves a sliding-window history of previous PDUs that occurred on the given +context. Starting from the PDU ID(s) given in the "v" argument, the PDUs that +preceeded it are retrieved, up to a total number given by the "limit" argument. +These are then returned in a new Transaction containing all of the PDUs. + + +To stream events all the events:: + + GET .../pull/ + Query args: origin, v + Response: JSON encoding of a single Transaction consisting of multiple PDUs + +Retrieves all of the transactions later than any version given by the "v" +arguments. + + +To make a query:: + + GET .../query/:query_type + Query args: as specified by the individual query types + Response: JSON encoding of a response object + +Performs a single query request on the receiving home server. The Query Type +part of the path specifies the kind of query being made, and its query +arguments have a meaning specific to that kind of query. The response is a +JSON-encoded object whose meaning also depends on the kind of query. + +Backfilling +----------- +.. NOTE:: + This section is a work in progress. + +.. TODO-doc + - What it is, when is it used, how is it done + +SRV Records +----------- +.. NOTE:: + This section is a work in progress. + +.. TODO-doc + - Why it is needed + +State Conflict Resolution +------------------------- +.. NOTE:: + This section is a work in progress. + +.. TODO-doc + - How do conflicts arise (diagrams?) + - How are they resolved (incl tie breaks) + - How does this work with deleting current state + +Presence +-------- +The server API for presence is based entirely on exchange of the following +EDUs. There are no PDUs or Federation Queries involved. + +Performing a presence update and poll subscription request:: + + EDU type: m.presence + + Content keys: + push: (optional): list of push operations. + Each should be an object with the following keys: + user_id: string containing a User ID + presence: "offline"|"unavailable"|"online"|"free_for_chat" + status_msg: (optional) string of freeform text + last_active_ago: miliseconds since the last activity by the user + + poll: (optional): list of strings giving User IDs + + unpoll: (optional): list of strings giving User IDs + +The presence of this combined message is two-fold: it informs the recipient +server of the current status of one or more users on the sending server (by the +``push`` key), and it maintains the list of users on the recipient server that +the sending server is interested in receiving updates for, by adding (by the +``poll`` key) or removing them (by the ``unpoll`` key). The ``poll`` and +``unpoll`` lists apply *changes* to the implied list of users; any existing IDs +that the server sent as ``poll`` operations in a previous message are not +removed until explicitly requested by a later ``unpoll``. + +On receipt of a message containing a non-empty ``poll`` list, the receiving +server should immediately send the sending server a presence update EDU of its +own, containing in a ``push`` list the current state of every user that was in +the orginal EDU's ``poll`` list. + +Sending a presence invite:: + + EDU type: m.presence_invite + + Content keys: + observed_user: string giving the User ID of the user whose presence is + requested (i.e. the recipient of the invite) + observer_user: string giving the User ID of the user who is requesting to + observe the presence (i.e. the sender of the invite) + +Accepting a presence invite:: + + EDU type: m.presence_accept + + Content keys - as for m.presence_invite + +Rejecting a presence invite:: + + EDU type: m.presence_deny + + Content keys - as for m.presence_invite + +.. TODO-doc + - Explain the timing-based roundtrip reduction mechanism for presence + messages + - Explain the zero-byte presence inference logic + See also: docs/client-server/model/presence + +Profiles +-------- +The server API for profiles is based entirely on the following Federation +Queries. There are no additional EDU or PDU types involved, other than the +implicit ``m.presence`` and ``m.room.member`` events (see section below). + +Querying profile information:: + + Query type: profile + + Arguments: + user_id: the ID of the user whose profile to return + field: (optional) string giving a field name + + Returns: JSON object containing the following keys: + displayname: string of freeform text + avatar_url: string containing an http-scheme URL + +If the query contains the optional ``field`` key, it should give the name of a +result field. If such is present, then the result should contain only a field +of that name, with no others present. If not, the result should contain as much +of the user's profile as the home server has available and can make public. + +Server-Server Authentication +---------------------------- + +.. TODO-doc + - Why is this needed. + - High level overview of process. + - Transaction/PDU signing + - How does this work with redactions? (eg hashing required keys only) + + + +Threat Model +------------ + +Denial of Service +~~~~~~~~~~~~~~~~~ + +The attacker could attempt to prevent delivery of messages to or from the +victim in order to: + +* Disrupt service or marketing campaign of a commercial competitor. +* Censor a discussion or censor a participant in a discussion. +* Perform general vandalism. + +Threat: Resource Exhaustion ++++++++++++++++++++++++++++ + +An attacker could cause the victims server to exhaust a particular resource +(e.g. open TCP connections, CPU, memory, disk storage) + +Threat: Unrecoverable Consistency Violations +++++++++++++++++++++++++++++++++++++++++++++ + +An attacker could send messages which created an unrecoverable "split-brain" +state in the cluster such that the victim's servers could no longer dervive a +consistent view of the chatroom state. + +Threat: Bad History ++++++++++++++++++++ + +An attacker could convince the victim to accept invalid messages which the +victim would then include in their view of the chatroom history. Other servers +in the chatroom would reject the invalid messages and potentially reject the +victims messages as well since they depended on the invalid messages. + +.. TODO-spec + Track trustworthiness of HS or users based on if they try to pretend they + haven't seen recent events, and fake a splitbrain... --M + +Threat: Block Network Traffic ++++++++++++++++++++++++++++++ + +An attacker could try to firewall traffic between the victim's server and some +or all of the other servers in the chatroom. + +Threat: High Volume of Messages ++++++++++++++++++++++++++++++++ + +An attacker could send large volumes of messages to a chatroom with the victim +making the chatroom unusable. + +Threat: Banning users without necessary authorisation ++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +An attacker could attempt to ban a user from a chatroom with the necessary +authorisation. + +Spoofing +~~~~~~~~ + +An attacker could try to send a message claiming to be from the victim without +the victim having sent the message in order to: + +* Impersonate the victim while performing illict activity. +* Obtain privileges of the victim. + +Threat: Altering Message Contents ++++++++++++++++++++++++++++++++++ + +An attacker could try to alter the contents of an existing message from the +victim. + +Threat: Fake Message "origin" Field ++++++++++++++++++++++++++++++++++++ + +An attacker could try to send a new message purporting to be from the victim +with a phony "origin" field. + +Spamming +~~~~~~~~ + +The attacker could try to send a high volume of solicicted or unsolicted +messages to the victim in order to: + +* Find victims for scams. +* Market unwanted products. + +Threat: Unsoliticted Messages ++++++++++++++++++++++++++++++ + +An attacker could try to send messages to victims who do not wish to receive +them. + +Threat: Abusive Messages +++++++++++++++++++++++++ + +An attacker could send abusive or threatening messages to the victim + +Spying +~~~~~~ + +The attacker could try to access message contents or metadata for messages sent +by the victim or to the victim that were not intended to reach the attacker in +order to: + +* Gain sensitive personal or commercial information. +* Impersonate the victim using credentials contained in the messages. + (e.g. password reset messages) +* Discover who the victim was talking to and when. + +Threat: Disclosure during Transmission +++++++++++++++++++++++++++++++++++++++ + +An attacker could try to expose the message contents or metadata during +transmission between the servers. + +Threat: Disclosure to Servers Outside Chatroom +++++++++++++++++++++++++++++++++++++++++++++++ + +An attacker could try to convince servers within a chatroom to send messages to +a server it controls that was not authorised to be within the chatroom. + +Threat: Disclosure to Servers Within Chatroom +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An attacker could take control of a server within a chatroom to expose message +contents or metadata for messages in that room. + + +Identity Servers +================ +.. NOTE:: + This section is a work in progress. + +.. TODO-doc Dave + - 3PIDs and identity server, functions + +Lawful Interception +------------------- + +Key Escrow Servers +~~~~~~~~~~~~~~~~~~ + +Policy Servers +============== +.. NOTE:: + This section is a work in progress. + +.. TODO-spec + We should mention them in the Architecture section at least: how they fit + into the picture. + +Enforcing policies +------------------ + + + +.. Links through the external API docs are below +.. ============================================= + +.. |createRoom| replace:: ``/createRoom`` +.. _createRoom: /docs/api/client-server/#!/-rooms/create_room + +.. |initialSync| replace:: ``/initialSync`` +.. _initialSync: /docs/api/client-server/#!/-events/initial_sync + +.. |/rooms//initialSync| replace:: ``/rooms//initialSync`` +.. _/rooms//initialSync: /docs/api/client-server/#!/-rooms/get_room_sync_data + +.. |login| replace:: ``/login`` +.. _login: /docs/api/client-server/#!/-login + +.. |register| replace:: ``/register`` +.. _register: /docs/api/client-server/#!/-registration + +.. |/rooms//messages| replace:: ``/rooms//messages`` +.. _/rooms//messages: /docs/api/client-server/#!/-rooms/get_messages + +.. |/rooms//members| replace:: ``/rooms//members`` +.. _/rooms//members: /docs/api/client-server/#!/-rooms/get_members + +.. |/rooms//state| replace:: ``/rooms//state`` +.. _/rooms//state: /docs/api/client-server/#!/-rooms/get_state_events + +.. |/rooms//send/| replace:: ``/rooms//send/`` +.. _/rooms//send/: /docs/api/client-server/#!/-rooms/send_non_state_event + +.. |/rooms//state//| replace:: ``/rooms//state//`` +.. _/rooms//state//: /docs/api/client-server/#!/-rooms/send_state_event + +.. |/rooms//invite| replace:: ``/rooms//invite`` +.. _/rooms//invite: /docs/api/client-server/#!/-rooms/invite + +.. |/rooms//join| replace:: ``/rooms//join`` +.. _/rooms//join: /docs/api/client-server/#!/-rooms/join_room + +.. |/rooms//leave| replace:: ``/rooms//leave`` +.. _/rooms//leave: /docs/api/client-server/#!/-rooms/leave + +.. |/rooms//ban| replace:: ``/rooms//ban`` +.. _/rooms//ban: /docs/api/client-server/#!/-rooms/ban + +.. |/join/| replace:: ``/join/`` +.. _/join/: /docs/api/client-server/#!/-rooms/join + +.. _`Event Stream`: /docs/api/client-server/#!/-events/get_event_stream + diff --git a/tutorials/client-server-howto.rst b/tutorials/client-server-howto.rst new file mode 100644 index 00000000..37346224 --- /dev/null +++ b/tutorials/client-server-howto.rst @@ -0,0 +1,637 @@ +.. TODO kegan + Room config (specifically: message history, + public rooms). /register seems super simplistic compared to /login, maybe it + would be better if /register used the same technique as /login? /register should + be "user" not "user_id". + + +How to use the client-server API +================================ + +This guide focuses on how the client-server APIs *provided by the reference +home server* can be used. Since this is specific to a home server +implementation, there may be variations in relation to registering/logging in +which are not covered in extensive detail in this guide. + +If you haven't already, get a home server up and running on +``http://localhost:8008``. + + +Accounts +======== +Before you can send and receive messages, you must **register** for an account. +If you already have an account, you must **login** into it. + +`Try out the fiddle`__ + +.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/register_login + +Registration +------------ +The aim of registration is to get a user ID and access token which you will need +when accessing other APIs:: + + curl -XPOST -d '{"user":"example", "password":"wordpass", "type":"m.login.password"}' "http://localhost:8008/_matrix/client/api/v1/register" + + { + "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.AqdSzFmFYrLrTmteXc", + "home_server": "localhost", + "user_id": "@example:localhost" + } + +NB: If a ``user`` is not specified, one will be randomly generated for you. +If you do not specify a ``password``, you will be unable to login to the account +if you forget the ``access_token``. + +Implementation note: The matrix specification does not enforce how users +register with a server. It just specifies the URL path and absolute minimum +keys. The reference home server uses a username/password to authenticate user, +but other home servers may use different methods. This is why you need to +specify the ``type`` of method. + +Login +----- +The aim when logging in is to get an access token for your existing user ID:: + + curl -XGET "http://localhost:8008/_matrix/client/api/v1/login" + + { + "flows": [ + { + "type": "m.login.password" + } + ] + } + + curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/login" + + { + "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd", + "home_server": "localhost", + "user_id": "@example:localhost" + } + +Implementation note: Different home servers may implement different methods for +logging in to an existing account. In order to check that you know how to login +to this home server, you must perform a ``GET`` first and make sure you +recognise the login type. If you do not know how to login, you can +``GET /login/fallback`` which will return a basic webpage which you can use to +login. The reference home server implementation support username/password login, +but other home servers may support different login methods (e.g. OAuth2). + + +Communicating +============= + +In order to communicate with another user, you must **create a room** with that +user and **send a message** to that room. + +`Try out the fiddle`__ + +.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/create_room_send_msg + +Creating a room +--------------- +If you want to send a message to someone, you have to be in a room with them. To +create a room:: + + curl -XPOST -d '{"room_alias_name":"tutorial"}' "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token=YOUR_ACCESS_TOKEN" + + { + "room_alias": "#tutorial:localhost", + "room_id": "!CvcvRuDYDzTOzfKKgh:localhost" + } + +The "room alias" is a human-readable string which can be shared with other users +so they can join a room, rather than the room ID which is a randomly generated +string. You can have multiple room aliases per room. + +.. TODO(kegan) + How to add/remove aliases from an existing room. + + +Sending messages +---------------- +You can now send messages to this room:: + + curl -XPOST -d '{"msgtype":"m.text", "body":"hello"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/send/m.room.message?access_token=YOUR_ACCESS_TOKEN" + + { + "event_id": "YUwRidLecu" + } + +The event ID returned is a unique ID which identifies this message. + +NB: There are no limitations to the types of messages which can be exchanged. +The only requirement is that ``"msgtype"`` is specified. The Matrix +specification outlines the following standard types: ``m.text``, ``m.image``, +``m.audio``, ``m.video``, ``m.location``, ``m.emote``. See the specification for +more information on these types. + +Users and rooms +=============== + +Each room can be configured to allow or disallow certain rules. In particular, +these rules may specify if you require an **invitation** from someone already in +the room in order to **join the room**. In addition, you may also be able to +join a room **via a room alias** if one was set up. + +`Try out the fiddle`__ + +.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/room_memberships + +Inviting a user to a room +------------------------- +You can directly invite a user to a room like so:: + + curl -XPOST -d '{"user_id":"@myfriend:localhost"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/invite?access_token=YOUR_ACCESS_TOKEN" + +This informs ``@myfriend:localhost`` of the room ID +``!CvcvRuDYDzTOzfKKgh:localhost`` and allows them to join the room. + +Joining a room via an invite +---------------------------- +If you receive an invite, you can join the room:: + + curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/join?access_token=YOUR_ACCESS_TOKEN" + +NB: Only the person invited (``@myfriend:localhost``) can change the membership +state to ``"join"``. Repeatedly joining a room does nothing. + +Joining a room via an alias +--------------------------- +Alternatively, if you know the room alias for this room and the room config +allows it, you can directly join a room via the alias:: + + curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/join/%23tutorial%3Alocalhost?access_token=YOUR_ACCESS_TOKEN" + + { + "room_id": "!CvcvRuDYDzTOzfKKgh:localhost" + } + +You will need to use the room ID when sending messages, not the room alias. + +NB: If the room is configured to be an invite-only room, you will still require +an invite in order to join the room even though you know the room alias. As a +result, it is more common to see a room alias in relation to a public room, +which do not require invitations. + +Getting events +============== +An event is some interesting piece of data that a client may be interested in. +It can be a message in a room, a room invite, etc. There are many different ways +of getting events, depending on what the client already knows. + +`Try out the fiddle`__ + +.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/event_stream + +Getting all state +----------------- +If the client doesn't know any information on the rooms the user is +invited/joined on, they can get all the user's state for all rooms:: + + curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=YOUR_ACCESS_TOKEN" + + { + "end": "s39_18_0", + "presence": [ + { + "content": { + "last_active_ago": 1061436, + "user_id": "@example:localhost" + }, + "type": "m.presence" + } + ], + "rooms": [ + { + "membership": "join", + "messages": { + "chunk": [ + { + "content": { + "@example:localhost": 10, + "default": 0 + }, + "event_id": "wAumPSTsWF", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.power_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "join_rule": "public" + }, + "event_id": "jrLVqKHKiI", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.join_rules", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 10 + }, + "event_id": "WpmTgsNWUZ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.add_state_level", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 0 + }, + "event_id": "qUMBJyKsTQ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.send_event_level", + "user_id": "@example:localhost" + }, + { + "content": { + "ban_level": 5, + "kick_level": 5 + }, + "event_id": "YAaDmKvoUW", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.ops_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join" + }, + "event_id": "RJbPMtCutf", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409665586730, + "type": "m.room.member", + "user_id": "@example:localhost" + }, + { + "content": { + "body": "hello", + "hsob_ts": 1409665660439, + "msgtype": "m.text" + }, + "event_id": "YUwRidLecu", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "ts": 1409665660439, + "type": "m.room.message", + "user_id": "@example:localhost" + }, + { + "content": { + "membership": "invite" + }, + "event_id": "YjNuBKnPsb", + "membership": "invite", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@myfriend:localhost", + "ts": 1409666426819, + "type": "m.room.member", + "user_id": "@example:localhost" + }, + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join", + "prev": "join" + }, + "event_id": "KWwdDjNZnm", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409666551582, + "type": "m.room.member", + "user_id": "@example:localhost" + }, + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join" + }, + "event_id": "JFLVteSvQc", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409666587265, + "type": "m.room.member", + "user_id": "@example:localhost" + } + ], + "end": "s39_18_0", + "start": "t1-11_18_0" + }, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state": [ + { + "content": { + "creator": "@example:localhost" + }, + "event_id": "dMUoqVTZca", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.create", + "user_id": "@example:localhost" + }, + { + "content": { + "@example:localhost": 10, + "default": 0 + }, + "event_id": "wAumPSTsWF", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.power_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "join_rule": "public" + }, + "event_id": "jrLVqKHKiI", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.join_rules", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 10 + }, + "event_id": "WpmTgsNWUZ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.add_state_level", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 0 + }, + "event_id": "qUMBJyKsTQ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.send_event_level", + "user_id": "@example:localhost" + }, + { + "content": { + "ban_level": 5, + "kick_level": 5 + }, + "event_id": "YAaDmKvoUW", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.ops_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "membership": "invite" + }, + "event_id": "YjNuBKnPsb", + "membership": "invite", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@myfriend:localhost", + "ts": 1409666426819, + "type": "m.room.member", + "user_id": "@example:localhost" + }, + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join" + }, + "event_id": "JFLVteSvQc", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409666587265, + "type": "m.room.member", + "user_id": "@example:localhost" + } + ] + } + ] + } + +This returns all the room information the user is invited/joined on, as well as +all of the presences relevant for these rooms. This can be a LOT of data. You +may just want the most recent event for each room. This can be achieved by +applying query parameters to ``limit`` this request:: + + curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?limit=1&access_token=YOUR_ACCESS_TOKEN" + + { + "end": "s39_18_0", + "presence": [ + { + "content": { + "last_active_ago": 1279484, + "user_id": "@example:localhost" + }, + "type": "m.presence" + } + ], + "rooms": [ + { + "membership": "join", + "messages": { + "chunk": [ + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join" + }, + "event_id": "JFLVteSvQc", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409666587265, + "type": "m.room.member", + "user_id": "@example:localhost" + } + ], + "end": "s39_18_0", + "start": "t10-30_18_0" + }, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state": [ + { + "content": { + "creator": "@example:localhost" + }, + "event_id": "dMUoqVTZca", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.create", + "user_id": "@example:localhost" + }, + { + "content": { + "@example:localhost": 10, + "default": 0 + }, + "event_id": "wAumPSTsWF", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.power_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "join_rule": "public" + }, + "event_id": "jrLVqKHKiI", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.join_rules", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 10 + }, + "event_id": "WpmTgsNWUZ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.add_state_level", + "user_id": "@example:localhost" + }, + { + "content": { + "level": 0 + }, + "event_id": "qUMBJyKsTQ", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.send_event_level", + "user_id": "@example:localhost" + }, + { + "content": { + "ban_level": 5, + "kick_level": 5 + }, + "event_id": "YAaDmKvoUW", + "required_power_level": 10, + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "", + "ts": 1409665585188, + "type": "m.room.ops_levels", + "user_id": "@example:localhost" + }, + { + "content": { + "membership": "invite" + }, + "event_id": "YjNuBKnPsb", + "membership": "invite", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@myfriend:localhost", + "ts": 1409666426819, + "type": "m.room.member", + "user_id": "@example:localhost" + }, + { + "content": { + "avatar_url": null, + "displayname": null, + "membership": "join" + }, + "event_id": "JFLVteSvQc", + "membership": "join", + "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", + "state_key": "@example:localhost", + "ts": 1409666587265, + "type": "m.room.member", + "user_id": "@example:localhost" + } + ] + } + ] + } + +Getting live state +------------------ +Once you know which rooms the client has previously interacted with, you need to +listen for incoming events. This can be done like so:: + + curl -XGET "http://localhost:8008/_matrix/client/api/v1/events?access_token=YOUR_ACCESS_TOKEN" + + { + "chunk": [], + "end": "s39_18_0", + "start": "s39_18_0" + } + +This will block waiting for an incoming event, timing out after several seconds. +Even if there are no new events (as in the example above), there will be some +pagination stream response keys. The client should make subsequent requests +using the value of the ``"end"`` key (in this case ``s39_18_0``) as the ``from`` +query parameter e.g. ``http://localhost:8008/_matrix/client/api/v1/events?access +_token=YOUR_ACCESS_TOKEN&from=s39_18_0``. This value should be stored so when the +client reopens your app after a period of inactivity, you can resume from where +you got up to in the event stream. If it has been a long period of inactivity, +there may be LOTS of events waiting for the user. In this case, you may wish to +get all state instead and then resume getting live state from a newer end token. + +NB: The timeout can be changed by adding a ``timeout`` query parameter, which is +in milliseconds. A timeout of 0 will not block. + + +Example application +------------------- +The following example demonstrates registration and login, live event streaming, +creating and joining rooms, sending messages, getting member lists and getting +historical messages for a room. This covers most functionality of a messaging +application. + +`Try out the fiddle`__ + +.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/example_app