You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
580 lines
20 KiB
ReStructuredText
580 lines
20 KiB
ReStructuredText
Federation API
|
|
===============
|
|
|
|
Matrix home servers use the Federation APIs to communicate with each other.
|
|
Home servers use these APIs to push messages to each other in real-time, to
|
|
request historic messages from each other, and to query profile and presence
|
|
information about users on each other's servers.
|
|
|
|
The API's are implemented using HTTPS GETs and PUTs between each of the
|
|
servers. These HTTPS requests are strongly authenticated using public key
|
|
signatures at the TLS transport layer and using public key signatures in
|
|
HTTP Authorization headers at the HTTP layer.
|
|
|
|
There are three main kinds of communication that occur between home servers:
|
|
|
|
Persisted Data Units (PDUs):
|
|
These events are broadcast from one home server to any others that have
|
|
joined the same "context" (namely, a Room ID). They are persisted in
|
|
long-term storage and record the history of messages and state for a
|
|
context.
|
|
|
|
Like email, it is the responsibility of the originating server of a PDU
|
|
to deliver that event to its recepient servers. However PDUs are signed
|
|
using the originating server's public key so that it is possible to
|
|
deliver them through third-party servers.
|
|
|
|
Ephemeral Data Units (EDUs):
|
|
These events are pushed between pairs of home servers. They are not
|
|
persisted and are not part of the history of a "context", nor does the
|
|
receiving home server have to reply to them.
|
|
|
|
Queries:
|
|
These are single request/response interactions between a given pair of
|
|
servers, initiated by one side sending an HTTPS GET request to obtain some
|
|
information, and responded by the other. They are not persisted and contain
|
|
no long-term significant history. They simply request a snapshot state at
|
|
the instant the query is made.
|
|
|
|
|
|
EDUs and PDUs are further wrapped in an envelope called a Transaction, which is
|
|
transferred from the origin to the destination home server using an HTTPS PUT
|
|
request.
|
|
|
|
|
|
Transactions
|
|
------------
|
|
.. WARNING::
|
|
This section may be misleading or inaccurate.
|
|
|
|
The transfer of EDUs and PDUs between home servers is performed by an exchange
|
|
of Transaction messages, which are encoded as JSON objects, passed over an HTTP
|
|
PUT request. A Transaction is meaningful only to the pair of home servers that
|
|
exchanged it; they are not globally-meaningful.
|
|
|
|
Each transaction has:
|
|
- An opaque transaction ID.
|
|
- A timestamp (UNIX epoch time in milliseconds) generated by its origin
|
|
server.
|
|
- An origin and destination server name.
|
|
- A list of "previous IDs".
|
|
- A list of PDUs and EDUs - the actual message payload that the Transaction
|
|
carries.
|
|
|
|
Transaction Fields
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
==================== =================== ======================================
|
|
Key Type Description
|
|
==================== =================== ======================================
|
|
``origin`` String DNS name of homeserver making this
|
|
transaction.
|
|
``origin_server_ts`` Integer Timestamp in milliseconds on
|
|
originating homeserver when this
|
|
transaction started.
|
|
``previous_ids`` List of Strings List of transactions that were sent
|
|
immediately prior to this transaction.
|
|
``pdus`` List of Objects List of persistent updates to rooms.
|
|
``edus`` List of Objects List of ephemeral messages.
|
|
==================== =================== ======================================
|
|
|
|
::
|
|
|
|
{
|
|
"transaction_id":"916d630ea616342b42e98a3be0b74113",
|
|
"ts":1404835423000,
|
|
"origin":"red",
|
|
"prev_ids":["e1da392e61898be4d2009b9fecce5325"],
|
|
"pdus":[...],
|
|
"edus":[...]
|
|
}
|
|
|
|
The ``prev_ids`` field contains a list of previous transaction IDs that the
|
|
``origin`` server has sent to this ``destination``. Its purpose is to act as a
|
|
sequence checking mechanism - the destination server can check whether it has
|
|
successfully received that Transaction, or ask for a re-transmission if not.
|
|
|
|
The ``pdus`` field of a transaction is a list, containing zero or more PDUs.[*]
|
|
Each PDU is itself a JSON object containing a number of keys, the exact details
|
|
of which will vary depending on the type of PDU. Similarly, the ``edus`` field
|
|
is another list containing the EDUs. This key may be entirely absent if there
|
|
are no EDUs to transfer.
|
|
|
|
(* Normally the PDU list will be non-empty, but the server should cope with
|
|
receiving an "empty" transaction.)
|
|
|
|
PDUs
|
|
----
|
|
|
|
All PDUs have:
|
|
|
|
- An ID
|
|
- A context
|
|
- A declaration of their type
|
|
- A list of other PDU IDs that have been seen recently on that context
|
|
(regardless of which origin sent them)
|
|
|
|
|
|
Required PDU Fields
|
|
~~~~~~~~~~~~~~~~~~~
|
|
|
|
==================== ================== =======================================
|
|
Key Type Description
|
|
==================== ================== =======================================
|
|
``context`` String Event context identifier
|
|
``user_id`` String The ID of the user sending the PDU
|
|
``origin`` String DNS name of homeserver that created
|
|
this PDU
|
|
``pdu_id`` String Unique identifier for PDU on the
|
|
originating homeserver
|
|
``origin_server_ts`` Integer Timestamp in milliseconds on origin
|
|
homeserver when this PDU was created.
|
|
``pdu_type`` String PDU event type
|
|
``content`` Object The content of the PDU.
|
|
``prev_pdus`` List of (String, The originating homeserver, PDU ids and
|
|
String, Object) hashes of the most recent PDUs the
|
|
Triplets homeserver was aware of for the context
|
|
when it made this PDU
|
|
``depth`` Integer The maximum depth of the previous PDUs
|
|
plus one
|
|
``is_state`` Boolean True if this PDU is updating room state
|
|
==================== ================== =======================================
|
|
|
|
::
|
|
|
|
{
|
|
"context":"#example:green.example.com",
|
|
"origin":"green.example.com",
|
|
"pdu_id":"a4ecee13e2accdadf56c1025af232176",
|
|
"origin_server_ts":1404838188000,
|
|
"pdu_type":"m.room.message",
|
|
"prev_pdus":[
|
|
["blue.example.com","99d16afbc8",
|
|
{"sha256":"abase64encodedsha256hashshouldbe43byteslong"}]
|
|
],
|
|
"hashes":{"sha256":"thishashcoversallfieldsincasethisisredacted"},
|
|
"signatures":{
|
|
"green.example.com":{
|
|
"ed25519:key_version:":"these86bytesofbase64signaturecoveressentialfieldsincludinghashessocancheckredactedpdus"
|
|
}
|
|
},
|
|
"is_state":false,
|
|
"content": {...}
|
|
}
|
|
|
|
In contrast to Transactions, it is important to note that the ``prev_pdus``
|
|
field of a PDU refers to PDUs that any origin server has sent, rather than
|
|
previous IDs that this ``origin`` has sent. This list may refer to other PDUs
|
|
sent by the same origin as the current one, or other origins.
|
|
|
|
Because of the distributed nature of participants in a Matrix conversation, it
|
|
is impossible to establish a globally-consistent total ordering on the events.
|
|
However, by annotating each outbound PDU at its origin with IDs of other PDUs
|
|
it has received, a partial ordering can be constructed allowing causality
|
|
relationships to be preserved. A client can then display these messages to the
|
|
end-user in some order consistent with their content and ensure that no message
|
|
that is semantically in reply of an earlier one is ever displayed before it.
|
|
|
|
State Update PDU Fields
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
PDUs fall into two main categories: those that deliver Events, and those that
|
|
synchronise State. For PDUs that relate to State synchronisation, additional
|
|
keys exist to support this:
|
|
|
|
======================== ============ =========================================
|
|
Key Type Description
|
|
======================== ============ =========================================
|
|
``state_key`` String Combined with the ``pdu_type`` this
|
|
identifies the which part of the room
|
|
state is updated
|
|
``required_power_level`` Integer The required power level needed to
|
|
replace this update.
|
|
``prev_state_id`` String The homeserver of the update this
|
|
replaces
|
|
``prev_state_origin`` String The PDU id of the update this replaces.
|
|
``user_id`` String The user updating the state.
|
|
======================== ============ =========================================
|
|
|
|
::
|
|
|
|
{...,
|
|
"is_state":true,
|
|
"state_key":TODO-doc
|
|
"required_power_level":TODO-doc
|
|
"prev_state_id":TODO-doc
|
|
"prev_state_origin":TODO-doc
|
|
}
|
|
|
|
|
|
EDUs
|
|
----
|
|
|
|
EDUs, by comparison to PDUs, do not have an ID, a context, or a list of
|
|
"previous" IDs. The only mandatory fields for these are the type, origin and
|
|
destination home server names, and the actual nested content.
|
|
|
|
======================== ============ =========================================
|
|
Key Type Description
|
|
======================== ============ =========================================
|
|
``edu_type`` String The type of the ephemeral message.
|
|
``content`` Object Content of the ephemeral message.
|
|
======================== ============ =========================================
|
|
|
|
::
|
|
|
|
{"edu_type":"m.presence",
|
|
"origin":"blue",
|
|
"destination":"orange",
|
|
"content":...}
|
|
|
|
|
|
Protocol URLs
|
|
-------------
|
|
.. WARNING::
|
|
This section may be misleading or inaccurate.
|
|
|
|
All these URLs are name-spaced within a prefix of::
|
|
|
|
/_matrix/federation/v1/...
|
|
|
|
For active pushing of messages representing live activity "as it happens"::
|
|
|
|
PUT .../send/:transaction_id/
|
|
Body: JSON encoding of a single Transaction
|
|
Response: TODO-doc
|
|
|
|
The transaction_id path argument will override any ID given in the JSON body.
|
|
The destination name will be set to that of the receiving server itself. Each
|
|
embedded PDU in the transaction body will be processed.
|
|
|
|
|
|
To fetch a particular PDU::
|
|
|
|
GET .../pdu/:origin/:pdu_id/
|
|
Response: JSON encoding of a single Transaction containing one PDU
|
|
|
|
Retrieves a given PDU from the server. The response will contain a single new
|
|
Transaction, inside which will be the requested PDU.
|
|
|
|
|
|
To fetch all the state of a given context::
|
|
|
|
GET .../state/:context/
|
|
Response: JSON encoding of a single Transaction containing multiple PDUs
|
|
|
|
Retrieves a snapshot of the entire current state of the given context. The
|
|
response will contain a single Transaction, inside which will be a list of PDUs
|
|
that encode the state.
|
|
|
|
To backfill events on a given context::
|
|
|
|
GET .../backfill/:context/
|
|
Query args: v, limit
|
|
Response: JSON encoding of a single Transaction containing multiple PDUs
|
|
|
|
Retrieves a sliding-window history of previous PDUs that occurred on the given
|
|
context. Starting from the PDU ID(s) given in the "v" argument, the PDUs that
|
|
preceeded it are retrieved, up to a total number given by the "limit" argument.
|
|
These are then returned in a new Transaction containing all of the PDUs.
|
|
|
|
|
|
To stream events all the events::
|
|
|
|
GET .../pull/
|
|
Query args: origin, v
|
|
Response: JSON encoding of a single Transaction consisting of multiple PDUs
|
|
|
|
Retrieves all of the transactions later than any version given by the "v"
|
|
arguments.
|
|
|
|
|
|
To make a query::
|
|
|
|
GET .../query/:query_type
|
|
Query args: as specified by the individual query types
|
|
Response: JSON encoding of a response object
|
|
|
|
Performs a single query request on the receiving home server. The Query Type
|
|
part of the path specifies the kind of query being made, and its query
|
|
arguments have a meaning specific to that kind of query. The response is a
|
|
JSON-encoded object whose meaning also depends on the kind of query.
|
|
|
|
Backfilling
|
|
-----------
|
|
.. NOTE::
|
|
This section is a work in progress.
|
|
|
|
.. TODO-doc
|
|
- What it is, when is it used, how is it done
|
|
|
|
SRV Records
|
|
-----------
|
|
.. NOTE::
|
|
This section is a work in progress.
|
|
|
|
.. TODO-doc
|
|
- Why it is needed
|
|
|
|
State Conflict Resolution
|
|
-------------------------
|
|
.. NOTE::
|
|
This section is a work in progress.
|
|
|
|
.. TODO-doc
|
|
- How do conflicts arise (diagrams?)
|
|
- How are they resolved (incl tie breaks)
|
|
- How does this work with deleting current state
|
|
|
|
Presence
|
|
--------
|
|
The server API for presence is based entirely on exchange of the following
|
|
EDUs. There are no PDUs or Federation Queries involved.
|
|
|
|
Performing a presence update and poll subscription request::
|
|
|
|
EDU type: m.presence
|
|
|
|
Content keys:
|
|
push: (optional): list of push operations.
|
|
Each should be an object with the following keys:
|
|
user_id: string containing a User ID
|
|
presence: "offline"|"unavailable"|"online"|"free_for_chat"
|
|
status_msg: (optional) string of freeform text
|
|
last_active_ago: miliseconds since the last activity by the user
|
|
|
|
poll: (optional): list of strings giving User IDs
|
|
|
|
unpoll: (optional): list of strings giving User IDs
|
|
|
|
The presence of this combined message is two-fold: it informs the recipient
|
|
server of the current status of one or more users on the sending server (by the
|
|
``push`` key), and it maintains the list of users on the recipient server that
|
|
the sending server is interested in receiving updates for, by adding (by the
|
|
``poll`` key) or removing them (by the ``unpoll`` key). The ``poll`` and
|
|
``unpoll`` lists apply *changes* to the implied list of users; any existing IDs
|
|
that the server sent as ``poll`` operations in a previous message are not
|
|
removed until explicitly requested by a later ``unpoll``.
|
|
|
|
On receipt of a message containing a non-empty ``poll`` list, the receiving
|
|
server should immediately send the sending server a presence update EDU of its
|
|
own, containing in a ``push`` list the current state of every user that was in
|
|
the orginal EDU's ``poll`` list.
|
|
|
|
Sending a presence invite::
|
|
|
|
EDU type: m.presence_invite
|
|
|
|
Content keys:
|
|
observed_user: string giving the User ID of the user whose presence is
|
|
requested (i.e. the recipient of the invite)
|
|
observer_user: string giving the User ID of the user who is requesting to
|
|
observe the presence (i.e. the sender of the invite)
|
|
|
|
Accepting a presence invite::
|
|
|
|
EDU type: m.presence_accept
|
|
|
|
Content keys - as for m.presence_invite
|
|
|
|
Rejecting a presence invite::
|
|
|
|
EDU type: m.presence_deny
|
|
|
|
Content keys - as for m.presence_invite
|
|
|
|
.. TODO-doc
|
|
- Explain the timing-based roundtrip reduction mechanism for presence
|
|
messages
|
|
- Explain the zero-byte presence inference logic
|
|
See also: docs/client-server/model/presence
|
|
|
|
Profiles
|
|
--------
|
|
The server API for profiles is based entirely on the following Federation
|
|
Queries. There are no additional EDU or PDU types involved, other than the
|
|
implicit ``m.presence`` and ``m.room.member`` events (see section below).
|
|
|
|
Querying profile information::
|
|
|
|
Query type: profile
|
|
|
|
Arguments:
|
|
user_id: the ID of the user whose profile to return
|
|
field: (optional) string giving a field name
|
|
|
|
Returns: JSON object containing the following keys:
|
|
displayname: string of freeform text
|
|
avatar_url: string containing an http-scheme URL
|
|
|
|
If the query contains the optional ``field`` key, it should give the name of a
|
|
result field. If such is present, then the result should contain only a field
|
|
of that name, with no others present. If not, the result should contain as much
|
|
of the user's profile as the home server has available and can make public.
|
|
|
|
Server-Server Authentication
|
|
----------------------------
|
|
|
|
.. TODO-doc
|
|
- Why is this needed.
|
|
- High level overview of process.
|
|
- Transaction/PDU signing
|
|
- How does this work with redactions? (eg hashing required keys only)
|
|
|
|
|
|
|
|
Threat Model
|
|
------------
|
|
|
|
Denial of Service
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
The attacker could attempt to prevent delivery of messages to or from the
|
|
victim in order to:
|
|
|
|
* Disrupt service or marketing campaign of a commercial competitor.
|
|
* Censor a discussion or censor a participant in a discussion.
|
|
* Perform general vandalism.
|
|
|
|
Threat: Resource Exhaustion
|
|
+++++++++++++++++++++++++++
|
|
|
|
An attacker could cause the victims server to exhaust a particular resource
|
|
(e.g. open TCP connections, CPU, memory, disk storage)
|
|
|
|
Threat: Unrecoverable Consistency Violations
|
|
++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could send messages which created an unrecoverable "split-brain"
|
|
state in the cluster such that the victim's servers could no longer dervive a
|
|
consistent view of the chatroom state.
|
|
|
|
Threat: Bad History
|
|
+++++++++++++++++++
|
|
|
|
An attacker could convince the victim to accept invalid messages which the
|
|
victim would then include in their view of the chatroom history. Other servers
|
|
in the chatroom would reject the invalid messages and potentially reject the
|
|
victims messages as well since they depended on the invalid messages.
|
|
|
|
.. TODO-spec
|
|
Track trustworthiness of HS or users based on if they try to pretend they
|
|
haven't seen recent events, and fake a splitbrain... --M
|
|
|
|
Threat: Block Network Traffic
|
|
+++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to firewall traffic between the victim's server and some
|
|
or all of the other servers in the chatroom.
|
|
|
|
Threat: High Volume of Messages
|
|
+++++++++++++++++++++++++++++++
|
|
|
|
An attacker could send large volumes of messages to a chatroom with the victim
|
|
making the chatroom unusable.
|
|
|
|
Threat: Banning users without necessary authorisation
|
|
+++++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could attempt to ban a user from a chatroom with the necessary
|
|
authorisation.
|
|
|
|
Spoofing
|
|
~~~~~~~~
|
|
|
|
An attacker could try to send a message claiming to be from the victim without
|
|
the victim having sent the message in order to:
|
|
|
|
* Impersonate the victim while performing illict activity.
|
|
* Obtain privileges of the victim.
|
|
|
|
Threat: Altering Message Contents
|
|
+++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to alter the contents of an existing message from the
|
|
victim.
|
|
|
|
Threat: Fake Message "origin" Field
|
|
+++++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to send a new message purporting to be from the victim
|
|
with a phony "origin" field.
|
|
|
|
Spamming
|
|
~~~~~~~~
|
|
|
|
The attacker could try to send a high volume of solicicted or unsolicted
|
|
messages to the victim in order to:
|
|
|
|
* Find victims for scams.
|
|
* Market unwanted products.
|
|
|
|
Threat: Unsoliticted Messages
|
|
+++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to send messages to victims who do not wish to receive
|
|
them.
|
|
|
|
Threat: Abusive Messages
|
|
++++++++++++++++++++++++
|
|
|
|
An attacker could send abusive or threatening messages to the victim
|
|
|
|
Spying
|
|
~~~~~~
|
|
|
|
The attacker could try to access message contents or metadata for messages sent
|
|
by the victim or to the victim that were not intended to reach the attacker in
|
|
order to:
|
|
|
|
* Gain sensitive personal or commercial information.
|
|
* Impersonate the victim using credentials contained in the messages.
|
|
(e.g. password reset messages)
|
|
* Discover who the victim was talking to and when.
|
|
|
|
Threat: Disclosure during Transmission
|
|
++++++++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to expose the message contents or metadata during
|
|
transmission between the servers.
|
|
|
|
Threat: Disclosure to Servers Outside Chatroom
|
|
++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
An attacker could try to convince servers within a chatroom to send messages to
|
|
a server it controls that was not authorised to be within the chatroom.
|
|
|
|
Threat: Disclosure to Servers Within Chatroom
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
An attacker could take control of a server within a chatroom to expose message
|
|
contents or metadata for messages in that room.
|
|
|
|
|
|
Identity Servers
|
|
================
|
|
.. NOTE::
|
|
This section is a work in progress.
|
|
|
|
.. TODO-doc Dave
|
|
- 3PIDs and identity server, functions
|
|
|
|
Lawful Interception
|
|
-------------------
|
|
|
|
Key Escrow Servers
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
Policy Servers
|
|
==============
|
|
.. NOTE::
|
|
This section is a work in progress.
|
|
|
|
.. TODO-spec
|
|
We should mention them in the Architecture section at least: how they fit
|
|
into the picture.
|
|
|
|
Enforcing policies
|
|
------------------
|
|
|