* try to make 00_basis slightly more accessible and less terse
* move CS API before Events, as it's meaningless to dive into specific events if you haven't been introduced to how the CS API works
* Try to improve the beginning of the CS API by merging back in some of kegan's original v1 content (which has subsequently been deleted somewhere along the line?)
* document m.presence event consistently
* move typing from drafts to spec
* move push stuff from drafts to spec
...and lots of other stuff
move push & typing from drafts into main spec.
This is a standalone description of the data architecture of Synapse. There is a lot of overlap with the currennt specification, so it has been split out here for posterity. Hopefully all the important bits have been merged into the relevant places in the main spec.
This is a standalone description of the data architecture of Synapse. There is a
lot of overlap with the current specification, so it has been split out here for
posterity. Hopefully all the important bits have been merged into the relevant
@ -200,7 +229,7 @@ the room ``!qporfwt:matrix.org``::
| matrix.org | | domain.com |
| matrix.org | | domain.com |
+------------------+ +------------------+
+------------------+ +------------------+
| ^
| ^
| |
| [HTTP POST] |
| Room ID: !qporfwt:matrix.org |
| Room ID: !qporfwt:matrix.org |
| Event type: m.room.message |
| Event type: m.room.message |
| Content: { JSON object } |
| Content: { JSON object } |
@ -222,7 +251,7 @@ the room ``!qporfwt:matrix.org``::
| Content: { JSON object } |
| Content: { JSON object } |
|...................................|
|...................................|
Federation maintains shared data structures per-room between multiple home
Federation maintains *shared data structures* per-room between multiple home
servers. The data is split into ``message events`` and ``state events``.
servers. The data is split into ``message events`` and ``state events``.
``Message events`` describe transient 'once-off' activity in a room such as an
``Message events`` describe transient 'once-off' activity in a room such as an
@ -252,7 +281,7 @@ participating in a room.
Room Aliases
Room Aliases
++++++++++++
++++++++++++
Each room can also have multiple "Room Aliases", which looks like::
Each room can also have multiple "Room Aliases", which look like::
#room_alias:domain
#room_alias:domain
@ -272,7 +301,7 @@ that are in the room that can be used to join via.
::
::
GET
HTTP GET
#matrix:domain.com !aaabaa:matrix.org
#matrix:domain.com !aaabaa:matrix.org
| ^
| ^
| |
| |
@ -285,7 +314,7 @@ that are in the room that can be used to join via.
|________________________________|
|________________________________|
Identity
Identity
++++++++
~~~~~~~~
Users in Matrix are identified via their matrix user ID (MXID). However,
Users in Matrix are identified via their matrix user ID (MXID). However,
existing 3rd party ID namespaces can also be used in order to identify Matrix
existing 3rd party ID namespaces can also be used in order to identify Matrix
@ -306,47 +335,37 @@ the Matrix ecosystem. However, without one clients will not be able to look up
user IDs using 3PIDs.
user IDs using 3PIDs.
Presence
Presence
++++++++
~~~~~~~~
Each user has the concept of presence information. This encodes the
"availability" of that user, suitable for display on other user's clients. This
is transmitted as an ``m.presence`` event and is one of the few events which
are sent *outside the context of a room*. The basic piece of presence
information is represented by the ``presence`` key, which is an enum of one of
the following:
- ``online`` : The default state when the user is connected to an event
stream.
- ``unavailable`` : The user is not reachable at this time.
- ``offline`` : The user is not connected to an event stream.
- ``free_for_chat`` : The user is generally willing to receive messages
moreso than default.
- ``hidden`` : Behaves as offline, but allows the user to see the client
state anyway and generally interact with client features. (Not yet
implemented in synapse).
.. TODO-spec
This seems like a very constrained list of states - surely presence states
should be extensible, with us providing a baseline, and possibly a scale of
availability? For instance, do-not-disturb is missing here, as well as a
distinction between 'away' and 'busy'.
This basic ``presence`` field applies to the user as a whole, regardless of how
Each user has the concept of presence information. This encodes:
many client devices they have connected. The presence state is pushed by the homeserver to all connected clients for a user to ensure a consistent experience for the user.
.. TODO-spec
* Whether the user is currently online
We need per-device presence in order to handle push notification semantics and similar.
* How recently the user was last active (as seen by the server)
* Whether a given client considers the user to be currently idle
* Arbitrary information about the user's current status (e.g. "in a meeting").
In addition, the server maintains a timestamp of the last time it saw a
This information is collated from both per-device (online; idle; last_active) and
pro-active event from the user; either sending a message to a room, or changing
per-user (status) data, aggregated by the user's homeserver and transmitted as
presence state from a lower to a higher level of availability (thus: changing
an ``m.presence`` event. This is one of the few events which are sent *outside
state from ``unavailable`` to ``online`` counts as a proactive event, whereas in
the context of a room*. Presence events are sent to all users who subscribe to
the other direction it will not). This timestamp is presented via a key called
this user's presence through a presence list or by sharing membership of a room.
``last_active_ago``, which gives the relative number of milliseconds since the
message is generated/emitted that the user was last seen active.
Presence List
.. TODO
~~~~~~~~~~~~~
How do we let users hide their presence information?
.. TODO
The last_active specifics should be moved to the detailed presence event section
Last activity is tracked by the server maintaining a timestamp of the last time
it saw a pro-active event from the user; either sending a message to a room,
coming online or back from idle, etc. This timestamp is presented via a key
called ``last_active_ago``, which gives the relative number of milliseconds
since the message is generated/emitted that the user was last seen active.
N.B. in v1 API, status/online/idle state are muxed into a single 'presence' field on the m.presence event.
Presence Lists
~~~~~~~~~~~~~~
Each user's home server stores a "presence list". This stores a list of user IDs
Each user's home server stores a "presence list". This stores a list of user IDs
whose presence the user wants to follow.
whose presence the user wants to follow.
@ -355,38 +374,31 @@ To be added to this list, the user being added must be invited by the list owner
and accept the invitation. Once accepted, both user's HSes track the
and accept the invitation. Once accepted, both user's HSes track the
subscription.
subscription.
Presence and Permissions
~~~~~~~~~~~~~~~~~~~~~~~~
For a viewing user to be allowed to see the presence information of a target
Profiles
user, either:
~~~~~~~~
- The target user has allowed the viewing user to add them to their presence
list, or
- The two users share at least one room in common
In the latter case, this allows for clients to display some minimal sense of
presence information in a user list for a room.
Users may publish arbitrary key/value data associated with their account - such
as a human readable ``display name``, a profile photo URL, contact information
(email address, phone nubers, website URLs etc).
Profiles
Profile data is typed using namespaced keys for interoperability, much like
++++++++
events - e.g. ``m.profile.display_name``.
.. TODO-spec
..TODO
- Metadata extensibility
Actually specify the different types of data - e.g. what format are display
names allowed to be?
Internally within Matrix users are referred to by their user ID, which is
Private User Data
typically a compact unique identifier. Profiles grant users the ability to see
~~~~~~~~~~~~~~~~~
human-readable names for other users that are in some way meaningful to them.
Additionally, profiles can publish additional information, such as the user's
age or location.
A Profile consists of a display name, an avatar picture, and a set of other
Users may also store arbitrary private key/value data in their account - such as
metadata fields that the user may wish to publish (email address, phone
client preferences, or server configuration settings which lack any other
numbers, website URLs, etc...). This specification puts no requirements on the
dedicated API. The API is symmetrical to managing Profile data.
display name other than it being a valid unicode string. Avatar images are not
stored directly; instead the home server stores an ``http``-scheme URL from which clients may fetch the image.
..TODO
Would it really be overengineered to use the same API for both profile &
private user data, but with different ACLs?
API Standards
API Standards
-------------
-------------
@ -430,6 +442,9 @@ For example, if there was a custom namespace ``com.mydomain.here``, and a
``COM.MYDOMAIN.HERE_FORBIDDEN``. There may be additional keys depending on the
``COM.MYDOMAIN.HERE_FORBIDDEN``. There may be additional keys depending on the
error, but the keys ``error`` and ``errcode`` MUST always be present.
error, but the keys ``error`` and ``errcode`` MUST always be present.
..TODO
Why the weird mix of underscore and dots?
Some standard error codes are below:
Some standard error codes are below:
:``M_FORBIDDEN``:
:``M_FORBIDDEN``:
@ -502,81 +517,3 @@ In contrast, these are invalid requests::
"key": "This is a put but it is missing a txnId."
"key": "This is a put but it is missing a txnId."
}
}
Glossary
--------
Backfilling:
The process of synchronising historic state from one home server to another,
to backfill the event storage so that scrollback can be presented to the
client(s). Not to be confused with pagination.
Context:
A single human-level entity of interest (currently, a chat room)
EDU (Ephemeral Data Unit):
A message that relates directly to a given pair of home servers that are
exchanging it. EDUs are short-lived messages that related only to one single
pair of servers; they are not persisted for a long time and are not forwarded
on to other servers. Because of this, they have no internal ID nor previous
EDUs reference chain.
Event:
A record of activity that records a single thing that happened on to a context
(currently, a chat room). These are the "chat messages" that Synapse makes
available.
PDU (Persistent Data Unit):
A message that relates to a single context, irrespective of the server that
is communicating it. PDUs either encode a single Event, or a single State
change. A PDU is referred to by its PDU ID; the pair of its origin server
and local reference from that server.
PDU ID:
The pair of PDU Origin and PDU Reference, that together globally uniquely
refers to a specific PDU.
PDU Origin:
The name of the origin server that generated a given PDU. This may not be the
server from which it has been received, due to the way they are copied around
from server to server. The origin always records the original server that
created it.
PDU Reference:
A local ID used to refer to a specific PDU from a given origin server. These
references are opaque at the protocol level, but may optionally have some
structured meaning within a given origin server or implementation.
Presence:
The concept of whether a user is currently online, how available they declare
they are, and so on. See also: doc/model/presence
Profile:
A set of metadata about a user, such as a display name, provided for the
benefit of other users. See also: doc/model/profiles
Room ID:
An opaque string (of as-yet undecided format) that identifies a particular
room and used in PDUs referring to it.
Room Alias:
A human-readable string of the form #name:some.domain that users can use as a
pointer to identify a room; a Directory Server will map this to its Room ID
State:
A set of metadata maintained about a Context, which is replicated among the
servers in addition to the history of Events.
User ID:
A string of the form @localpart:domain.name that identifies a user for
wire-protocol purposes. The localpart is meaningless outside of a particular
home server. This takes a human-readable form that end-users can use directly
if they so wish, avoiding the 3PIDs.
Transaction:
A message which relates to the communication between a given pair of servers.
A transaction contains possibly-empty lists of PDUs and EDUs.
.. TODO
This glossary contradicts the terms used above - especially on State Events v. "State"
and Non-State Events v. "Events". We need better consistent names.
Matthew Hodgson
10 years ago