archive
/
matrix-spec
mirror of https://github.com/matrix-org/matrix-spec
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove proposals from this repository

Browse Source
pull/977/head
Richard van der Hoff 2 years ago
parent ca466b5a57
commit 95edf9b494
128 changed files with 0 additions and 15477 deletions
Show Stats Download Patch File Download Diff File

110
proposals/0000-proposal-template.md
Unescape Escape View File

@ -1,110 +0,0 @@
# MSC0000: Template for new MSCs
*Note: Text written in italics represents notes about the section or proposal process. This document
serves as an example of what a proposal could look like (in this case, a proposal to have a template)
and should be used where possible.*
*In this first section, be sure to cover your problem and a broad overview of the solution. Covering
related details, such as the expected impact, can also be a good idea. The example in this document
says that we're missing a template and that things are confusing and goes on to say the solution is
a template. There's no major expected impact in this proposal, so it doesn't list one. If your proposal
was more invasive (such as proposing a change to how servers discover each other) then that would be
a good thing to list here.*
*If you're having troubles coming up with a description, a good question to ask is "how
does this proposal improve Matrix?" - the answer could reveal a small impact, and that is okay.*
There can never be enough templates in the world, and MSCs shouldn't be any different. The level
of detail expected of proposals can be unclear - this is what this example proposal (which doubles
as a template itself) aims to resolve.
## Proposal
*Here is where you'll reinforce your position from the introduction in more detail, as well as cover
the technical points of your proposal. Including rationale for your proposed solution and detailing
why parts are important helps reviewers understand the problem at hand. Not including enough detail
can result in people guessing, leading to confusing arguments in the comments section. The example
here covers why templates are important again, giving a stronger argument as to why we should have
a template. Afterwards, it goes on to cover the specifics of what the template could look like.*
Having a default template that everyone can use is important. Without a template, proposals would be
all over the place and the minimum amount of detail may be left out. Introducing a template to the
proposal process helps ensure that some amount of consistency is present across multiple proposals,
even if each author decides to abandon the template.
The default template should be a markdown document because the MSC process requires authors to write
a proposal in markdown. Using other formats wouldn't make much sense because that would prevent authors
from copy/pasting the template.
The template should have the following sections:
* **Introduction** - This should cover the primary problem and broad description of the solution.
* **Proposal** - The gory details of the proposal.
* **Potential issues** - This is where problems with the proposal would be listed, such as changes
that are not backwards compatible.
* **Alternatives** - This section lists alternative solutions to the same
problem which have been considered and dismsissed.
* **Security considerations** - Discussion of what steps were taken to avoid security issues in the
future and any potential risks in the proposal.
Furthermore, the template should not be required to be followed. However it is strongly recommended to
maintain some sense of consistency between proposals.
## Potential issues
*Not all proposals are perfect. Sometimes there's a known disadvantage to implementing the proposal,
and they should be documented here. There should be some explanation for why the disadvantage is
acceptable, however - just like in this example.*
Someone is going to have to spend the time to figure out what the template should actually have in it.
It could be a document with just a few headers or a supplementary document to the process explanation,
however more detail should be included. A template that actually proposes something should be considered
because it not only gives an opportunity to show what a basic proposal looks like, it also means that
explanations for each section can be described. Spending the time to work out the content of the template
is beneficial and not considered a significant problem because it will lead to a document that everyone
can follow.
## Alternatives
*This is where alternative solutions could be listed. There's almost always another way to do things
and this section gives you the opportunity to highlight why those ways are not as desirable. The
argument made in this example is that all of the text provided by the template could be integrated
into the proposals introduction, although with some risk of losing clarity.*
Instead of adding a template to the repository, the assistance it provides could be integrated into
the proposal process itself. There is an argument to be had that the proposal process should be as
descriptive as possible, although having even more detail in the proposals introduction could lead to
some confusion or lack of understanding. Not to mention if the document is too large then potential
authors could be scared off as the process suddenly looks a lot more complicated than it is. For those
reasons, this proposal does not consider integrating the template in the proposals introduction a good
idea.
## Security considerations
*Some proposals may have some security aspect to them that was addressed in the proposed solution. This
section is a great place to outline some of the security-sensitive components of your proposal, such as
why a particular approach was (or wasn't) taken. The example here is a bit of a stretch and unlikely to
actually be worthwhile of including in a proposal, but it is generally a good idea to list these kinds
of concerns where possible.*
By having a template available, people would know what the desired detail for a proposal is. This is not
considered a risk because it is important that people understand the proposal process from start to end.
## Unstable prefix
*If a proposal is implemented before it is included in the spec, then implementers must ensure that the
implementation is compatible with the final version that lands in the spec. This generally means that
experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary.
For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section
should be used to document things such as what endpoints and names are being used while the feature is
in development, the name of the unstable feature flag to use to detect support for the feature, or what
migration steps are needed to switch to newer versions of the proposal.*
## Dependencies
This MSC builds on MSCxxxx, MSCyyyy and MSCzzzz (which at the time of writing have not yet been accepted
into the spec).

562
proposals/1219-storing-megolm-keys-serverside.md
Unescape Escape View File

@ -1,562 +0,0 @@
Storing megolm keys serverside
==============================
Background
----------
A user who uses end-to-end encryption will usually have many inbound group session
keys. Users who log into new devices and want to read old messages will need a
convenient way to transfer the session keys from one device to another. While
users can currently export their keys from one device and import them to
another, this is involves several steps and may be cumbersome for many users.
Users can also share keys from one device to another, but this has several
limitations, such as the fact that key shares only share one key at a time, and
require another logged-in device to be active.
To help resolve this, we *optionally* let clients store an encrypted copy of
their megolm inbound session keys on the homeserver. Clients can keep the
backup up to date, so that users will always have the keys needed to decrypt
their conversations. The backup could be used not just for new logins, but
also to support clients with limited local storage for keys (clients can store
old keys to the backup, and remove their local copy, retrieving the key from
the backup when needed).
To recover keys from the backup, a user will need to enter a recovery key to
decrypt the backup. The backup will be encrypted using public key
cryptography, so that any of a user's devices can back up keys without needing
the user to enter the recovery key until they need to read from the backup.
See also:
* https://github.com/matrix-org/matrix-doc/issues/1219
* https://github.com/vector-im/riot-web/issues/3661
* https://github.com/vector-im/riot-web/issues/5675
* https://docs.google.com/document/d/1MOoIA9qEKIhUQ3UmKZG-loqA8e0BzgWKKlKRUGMynVc/edit#
(old version of proposal)
Proposal
--------
This proposal creates new APIs to allow clients to back up room decryption keys
on the server. Room decryption keys are encrypted (using public key crypto)
before being sent to the server along with some unencrypted metadata to allow
the server to manage the backups. If a key for a new megolm session is
uploaded, it is added to the current backup. If a key is uploaded for a megolm
session is that is already present in the backup, the server will use the
metadata to determine which version of the key is "better". The way in which
the server determines which key is "better" is described in the [Storing
Keys](#storing-keys) section. The user is given a private recovery key in
order to recover the keys from the backup in the future.
Clients can create new key backups (sometimes also referred to in the API as
backup versions) to replace the current backup. Aside from the initial backup
creation, a client might start a new a backup when, for example, a user loses a
device and wants to ensure that that device does not get any new decryption
keys. In this case, the client will then create a new backup using a new key
that the device does not have access to.
Once one client has created a backup, other clients can fetch the public part
of the recovery key from the server and add keys to the backup, if they trust
that the backup was not created by a malicious device.
### Possible UX for interactive clients
This section gives an example of how a client might handle key backups. Clients
may behave differently.
On receipt of encryption keys (1st time):
1. client checks if there is an existing backup: `GET /room_keys/version`
1. if not, ask if the user wants to back up keys
1. if yes:
1. generate new curve25519 key pair, which will be the recovery key
2. create new backup: `POST /room_keys/version`
3. display private key for user to save (see below for the
[format of the recovery key](#recovery-key))
2. if no, exit and remember decision (user can change their mind later)
3. while prompting, continue to poll `GET /room_keys/versions`, as
another device may have created a backup. If so, go to 1.2.
2. if yes, either get the public part of the recovery key and check that it
is signed by the master cross-signing key, or prompt user to enter the
private part of the recovery key (which can derive the public part).
1. User can also decide to create a new backup, in which case, go to 1.1.
2. send key to backup: `PUT /room_keys/keys/${roomId}/${sessionId}?version=$v`
3. continue backing up keys as we receive them (may receive a
`M_WRONG_ROOM_KEYS_VERSION` error if a new backup has been created:
see below)
On `M_WRONG_ROOM_KEYS_VERSION` error when trying to `PUT` keys:
1. get the current version
2. notify the user that there is a new backup, and display relevant information
3. confirm with user that they want to use the backup (user may want use the
backup, to stop backing up keys, or to create a new backup)
4. ensure the public part of the recovery key is signed by the user's master
key, or prompt the user to enter the private part of the recovery key
On receipt of undecryptable message:
1. ask user if they want to restore backup (ask whether to get individual key,
room keys, or all keys). (This can be done in the same place as asking if
the user wants to request keys from other devices.)
2. if yes, prompt for private key, and get keys: `GET /room_keys/keys`
Users can also set up, disable, or rotate backups, or restore from backup via user
settings.
### Recovery key
The recovery key can be saved by the user directly, stored encrypted on the
server (using the method proposed in
[MSC1946](https://github.com/matrix-org/matrix-doc/issues/1946)), or both. If
the key is saved directly by the user, then the code is constructed as follows:
1. The 256-bit curve25519 private key is prepended by the bytes `0x8B` and
`0x01`
2. All the bytes in the string above, including the two header bytes, are XORed together to form a parity
byte. This parity byte is appended to the byte string.
3. The byte string is encoded using base58, using the same mapping as is used
for Bitcoin addresses.
This 58-character string is presented to the user to save. Implementations may
add whitespace to the recovery key; adding a space every 4th character is
recommended.
When reading in a recovery key, clients must disregard whitespace. Clients
must base58-decode the code, ensure that the first two bytes of the decoded
string are `0x8B` and `0x01`, ensure that XOR-ing all the bytes together
results in 0, and ensure that the total length of the decoded string
is 35 bytes. Clients must then remove the first two bytes and the last byte,
and use the resulting string as the private key to decrypt backups.
#### Encoding the recovery key for server-side storage via MSC1946
If MSC1946 is used to store the key on the server, it must be stored using the
`account_data` type `m.megolm_backup.v1`.
As a special case, if the recovery key is the same as the curve25519 key used
for storing the key, then the contents of the `m.megolm_backup.v1`
`account_data` for that key will be an object with a `passthrough` property
whose value is `true`. For example, if `m.megolm_backup.v1` is set to:
```json
{
"encrypted": {
"key_id": {
"passthrough": true
}
}
}
```
means that the recovery key for the backup is the same as the private key for
the key with ID `key_id`. (This is mostly intended to provide a migration path
for for backups that were created using an earlier draft that stored the
recovery information in the `auth_data`.)
### API
#### Backup versions
##### `POST /room_keys/version`
Create a new backup version.
Body parameters:
- `algorithm` (string): Required. The algorithm used for storing backups.
Currently, only `m.megolm_backup.v1.curve25519-aes-sha2` is defined.
- `auth_data` (object): Required. algorithm-dependent data. For
`m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of
this property](#auth_data-backup-versions).
Example:
```javascript
{
"algorithm": "m.megolm_backup.v1.curve25519-aes-sha2",
"auth_data": {
"public_key": "abcdefg",
"signatures": {
"something": {
"ed25519:something": "hijklmnop"
}
}
}
}
```
On success, returns a JSON object with keys:
- `version` (string): the backup version
##### `GET /room_keys/version/{version}`
Get information about the given version, or the current version if `/{version}`
is omitted.
On success, returns a JSON object with keys:
- `algorithm` (string): Required. Same as in the body parameters for `POST
/room_keys/version`.
- `auth_data` (object): Required. Same as in the body parameters for
`POST /room_keys/version`.
- `version` (string): Required. The backup version.
- `etag` (string): Required. The etag value which is an opaque string
representing stored keys in the backup. Clients can compare it with the
`etag` value they received in the response of their last key storage request.
If not equal, another client has pushed new keys to the backup.
- `count` (number): Required. The number of keys stored in the backup.
Error codes:
- `M_NOT_FOUND`: No backup version has been created. (with HTTP status code 404)
##### `PUT /room_keys/version/{version}`
Update information about the given version. Only `auth_data` can be updated.
Body parameters:
- `algorithm` (string): Required. Must be the same as in the body parameters for `GET
/room_keys/version`.
- `auth_data` (object): Required. algorithm-dependent data. For
`m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of
this property](#auth_data-backup-versions).
- `version` (string): Optional. The backup version. If present, must be the same as the path parameter.
Example:
```javascript
{
"algorithm": "m.megolm_backup.v1.curve25519-aes-sha2",
"auth_data": {
"public_key": "abcdefg",
"signatures": {
"something": {
"ed25519:something": "hijklmnop"
"ed25519:anotherthing": "abcdef"
}
}
},
"version": "42"
}
```
On success, returns the empty JSON object.
Error codes:
- `M_NOT_FOUND`: This backup version was not found. (with HTTP status code 404)
#### Storing keys
##### `PUT /room_keys/keys/${roomId}/${sessionId}?version=$v`
Store the key for the given session in the given room, using the given backup
version.
If the server already has a backup in the backup version for the given session
and room, then it will keep the "better" one. To determine which one is
"better", keys are compared:
- first by the `is_verified` flag (`true` is better than `false`),
- then, if `is_verified` is equal, by the `first_message_index` (a lower number is better),
- and finally, is `is_verified` and `first_message_index` are equal, by
`forwarded_count` (a lower number is better).
If neither key is better than the other (that is, if all three fields are
equal), then the server should keep the existing key.
Body parameters:
- `first_message_index` (integer): Required. The index of the first message
in the session that the key can decrypt.
- `forwarded_count` (integer): Required. The number of times this key has been
forwarded.
- `is_verified` (boolean): Required. Whether the device backing up the key has
verified the device that the key is from.
- `session_data` (object): Required. Algorithm-dependent data. For
`m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of
this property](#auth_data-backup-versions).
On success, returns a JSON object with keys:
- `etag` (string): Required. The new etag value representing stored keys. See
`GET /room_keys/version/{version}` for more details.
- `count` (number): Required. The new count of keys stored in the backup.
Error codes:
- `M_WRONG_ROOM_KEYS_VERSION`: the version specified does not match the current
backup version (with HTTP status code 403). The current backup version will
be included in the `current_version` field of the HTTP result.
Example:
`PUT /room_keys/keys/!room_id:example.com/sessionid?version=1`
```javascript
{
"first_message_index": 1,
"forwarded_count": 0,
"is_verified": true,
"session_data": {
"ephemeral": "base64+ephemeral+key",
"ciphertext": "base64+ciphertext+of+JSON+data",
"mac": "base64+mac+of+ciphertext"
}
}
```
Result:
```javascript
{
"etag": "abcdefghi",
"count": 10
}
```
##### `PUT /room_keys/keys/${roomId}?version=$v`
Store several keys for the given room, using the given backup version.
Behaves the same way as if the keys were added individually using `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
Body parameters:
- `sessions` (object): an object where the keys are the session IDs, and the
values are objects of the same form as the body in `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
Returns the same as `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
Example:
`PUT /room_keys/keys/!room_id:example.com?version=1`
```javascript
{
"sessions": {
"sessionid": {
"first_message_index": 1,
"forwarded_count": 0,
"is_verified": true,
"session_data": {
"ephemeral": "base64+ephemeral+key",
"ciphertext": "base64+ciphertext+of+JSON+data",
"mac": "base64+mac+of+ciphertext"
}
}
}
}
```
Result:
```javascript
{
"etag": "abcdefghi",
"count": 10
}
```
##### `PUT /room_keys/keys?version=$v`
Store several keys, using the given backup version.
Behaves the same way as if the keys were added individually using `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
Body parameters:
- `rooms` (object): an object where the keys are the room IDs, and the values
are objects of the same form as the body in `PUT
/room_keys/keys/${roomId}/?version=$v`.
Returns the same as `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`
Example:
`PUT /room_keys/keys/!room_id:example.com?version=1`
```javascript
{
"rooms": {
"!room_id:example.com": {
"sessions": {
"sessionid": {
"first_message_index": 1,
"forwarded_count": 0,
"is_verified": true,
"session_data": {
"ephemeral": "base64+ephemeral+key",
"ciphertext": "base64+ciphertext+of+JSON+data",
"mac": "base64+mac+of+ciphertext"
}
}
}
}
}
}
```
Result:
```javascript
{
"etag": "abcdefghi",
"count": 10
}
```
#### Retrieving keys
When retrieving keys, the `version` parameter is optional, and defaults to
retrieving keys from the latest backup version.
##### `GET /room_keys/keys/${roomId}/${sessionId}?version=$v`
Retrieve the key for the given session in the given room from the backup.
On success, returns a JSON object in the same form as the request body of `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
Error codes:
- M_NOT_FOUND: The session is not present in the backup, or the requested
backup version does not exist. (with HTTP status code 404)
##### `GET /room_keys/keys/${roomId}?version=$v`
Retrieve the all the keys for the given room from the backup.
On success, returns a JSON object in the same form as the request body of `PUT
/room_keys/keys/${roomId}?version=$v`.
If the backup version exists but no keys are found, then this endpoint returns
a successful response with body:
```
{
"sessions": {}
}
```
Error codes:
- `M_NOT_FOUND`: The requested backup version does not exist. (with HTTP status code 404)
##### `GET /room_keys/keys?version=$v`
Retrieve all the keys from the backup.
On success, returns a JSON object in the same form as the request body of `PUT
/room_keys/keys?version=$v`.
If the backup version exists but no keys are found, then this endpoint returns
a successful response with body:
```
{
"rooms": {}
}
```
Error codes:
- `M_NOT_FOUND`: The requested backup version does not exist. (with HTTP status code 404)
#### Deleting keys
##### `DELETE /room_keys/keys/${roomId}/${sessionId}?version=$v`
##### `DELETE /room_keys/keys/${roomId}?version=$v`
##### `DELETE /room_keys/keys/?version=$v`
Deletes keys from the backup.
Returns the same as `PUT
/room_keys/keys/${roomId}/${sessionId}?version=$v`.
#### `m.megolm_backup.v1.curve25519-aes-sha2` definitions
##### `auth_data` for backup versions
The `auth_data` property for the backup versions endpoints for
`m.megolm_backup.v1.curve25519-aes-sha2` is a [signed
json](https://matrix.org/docs/spec/appendices#signing-json) object with the
following keys:
- `public_key` (string): the curve25519 public key used to encrypt the backups
- `signatures` (object): signatures of the `auth_data`.
The `auth_data` should be signed by the user's [master cross-signing
key](https://github.com/matrix-org/matrix-doc/pull/1756), and may also be
signed by the user's device key. This allows clients to ensure that the public
key is valid, and prevents an attacker from being able to change the backup to
use a public key that they have the private key for.
##### `session_data` for key backups
The `session_data` field in the backups is constructed as follows:
1. Encode the session key to be backed up as a JSON object with the properties:
- `algorithm` (string): `m.megolm.v1.aes-sha2`
- `sender_key` (string): base64-encoded device curve25519 key
- `sender_claimed_keys` (object): object containing the identity keys for the
sending device
- `forwarding_curve25519_key_chain` (array): zero or more curve25519 keys
for devices who forwarded the session key
- `session_key` (string): base64-encoded (unpadded) session key in
[session-sharing
format](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/megolm.md#session-sharing-format)
2. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral
key and the backup's public key to generate a shared secret. The public
half of the ephemeral key, encoded using base64, becomes the `ephemeral`
property of the `session_data`.
3. Using the shared secret, generate 80 bytes by performing an HKDF using
SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string
as the info. The first 32 bytes are used as the AES key, the next 32 bytes
are used as the MAC key, and the last 16 bytes are used as the AES
initialization vector.
4. Stringify the JSON object, and encrypt it using AES-CBC-256 with PKCS#7
padding. This encrypted data, encoded using base64, becomes the
`ciphertext` property of the `session_data`.
5. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256
using the MAC key generated above. The first 8 bytes of the resulting MAC
are base64-encoded, and become the `mac` property of the `session_data`.
(The key HKDF, AES, and HMAC steps are the same as what are used for encryption
in olm and megolm.)
Security Considerations
-----------------------
An attacker who gains access to a user's account can delete or corrupt their
key backup. This proposal does not attempt to protect against that.
An attacker who gains access to a user's account can create a new backup
version using a key that they control. For this reason, clients SHOULD confirm
with users before sending keys to a new backup version or verify that it was
created by a trusted device by checking the signature. Alternatively, if the
signature cannot be verified, the backup can be validated by prompting the user
to enter the recovery key, and confirming that the backup's public key
corresponds to the recovery key.
Other Issues
------------
Since many clients will receive encryption keys at around the same time, they
will all want to back up their copies of the keys at around the same time,
which may increase load on the server if this happens in a big room. (TODO:
how much of an issue is this?) For this reason, clients should offset their
backup requests randomly.
Conclusion
----------
This proposal allows users to securely and conveniently back up and restore
their decryption keys so that users logging into a new device can decrypt old
messages.

592
proposals/1442-state-resolution.md
Unescape Escape View File

@ -1,592 +0,0 @@
- **Author**: Erik Johnston
- **Created**: 2018-07-20
- **Updated**:
- #1693: Clarify how to handle rejected events ─ Erik Johnston, 2018-10-30
# State Resolution: Reloaded
Thoughts on the next iteration of the state resolution algorithm that aims to
mitigate currently known attacks
# Background
The state of a room at an event is a mapping from key to event, which is built
up and updated by sending state events into the room. All the information about
the room is encoded in the state, from metadata like the name and topic to
membership of the room to security policies like bans and join rules.
It is therefore important that─wherever possible─the view of the state of the
room is consistent across all servers. If different servers have different
views of the state then it can lead to the room bifurcating, due to differing
ideas on who is in the room, who is allowed to talk, etc.
The difficulty comes when the room DAG forks and then merges again (which can
happen naturally if two servers send events at the same time or when a network
partition is resolved). The state after the merge has to be resolved from the
state of the two branches: the algorithm to resolve this is called the _state
resolution algorithm_.
Since the result of state resolution must be consistent across servers, the
information that the algorithm can use is strictly limited to the information
that will always be available to all servers (including future servers that may
not even be in the room at that point) at any point in time where the
resolution needs to be calculated. In particular, this has the consequence that
the algorithm cannot use information from the room DAG, since servers are not
required to store events for any length of time.
**As such, the state resolution algorithm is effectively a pure function from
sets of state to a single resolved set of state.**
The final important property for state resolution is that it should not allow
malicious servers to avoid moderation action by forking and merging the room
DAG. For example, if a server gets banned and then forks the room before the
ban, any merge back should always ensure that the ban is still in the state.
# Current Algorithm
The current state resolution is known to have some undesirable properties,
which can be summarized into two separate cases:
1. Moderation evasion ─ where an attacker can avoid e.g. bans by forking and
joining the room DAG in particular ways.
2. State resets ─ where a server (often innocently) sends an event that points
to disparate parts of the graph, causing state resolution to pick old state
rather than later versions.
These have the following causes:
1. Conflicting state must pass auth checks to be eligible to be picked, but the
algorithm does not consider previous (superseded) state changes in a fork.
For example, where Alice gives Bob power and then Bob gives Charlie power on
one branch of a conflict, when the latter power level event is authed
against the original power level (where Bob didn't have power), it fails.
1. The algorithm relies on the deprecated and untrustable depth parameter to
try and ensure that the "most recent" state is picked. Without having a copy
of the complete room DAG the algorithm doesn't know that e.g. one topic
event came strictly after another in the DAG. For efficiency and storage
reasons servers are not required (or expected) to store the whole room DAG.
1. The algorithm always accepts events where there are no conflicting
alternatives in other forks. This means that if an admin changed the join
rules to `private`, then new joins on forks based on parts of the DAG which
predate that change would always be accepted without being authed against
the join_rules event.
# Desirable Properties
As well as the important properties listed in the "Background" section, there
are also some other properties that would significantly improve the experience
of end users, though not strictly essential. These include:
* Banning and changing power levels should "do the right thing", i.e. end
users shouldn't have to take extra steps to make the state resolution
produce the "right" results.
* Minimise occurrences of "state resets". Servers will sometimes point to
disparate parts of the room DAG (due to a variety of reasons), which ideally
should not result in changes in the state.
* Be efficient; state resolution can happen a lot on some large rooms. Ideally
it would also support efficiently working on "state deltas" - i.e. the
ability to calculate state resolution incrementally from snapshots rather
than having to consider the full state of each fork each time a conflict is
resolved
# Ideas for New Algorithm
## Auth Chain
The _auth events_ of a given event is the set of events which justify why a
given event is allowed to be sent into a room (e.g. an m.room.create, an
m.room.power_levels and the sender's m.room.membership). The _auth chain_ of an
event is its auth events and their auth events, recursively. The auth chains of
a set of events in a given room form a DAG.
"Auth events" are events that can appear as auth events of an event. These
include power levels, membership etc.[^1]
Servers in a room are required to have the full auth chain for all events that
they have seen, and so the auth chain is available to be used by state
resolution algorithms.
## Unconflicted State
The current algorithm defines the notion of "unconflicted state" to be all
entries that for each set of state either has the same event or no entry. All
unconflicted state entries are included in the resolved state. This is
problematic due to the fact that any new entries introduced on forks always
appear in the resolved state, regardless of if they would pass the checks
applied to conflicted state.
The new algorithm could redefine "unconflicted state" to be all entries which
both exist and are the same in every state set (as opposed to previously where
the entry didn't need to exist in every state set).
## Replacing Depth
Since depth of an event cannot be reliably calculated without possessing the
full DAG, and cannot be trusted when provided by other servers, it can not be
used in future versions of state resolution. A potential alternative, however,
is to use "origin_server_ts". While it cannot be relied on to be accurate─an
attacker can set it to arbitrary values─it has the advantage over depth that
end users can clearly see when a server is using incorrect values. (Note that
server clocks don't need to be particularly accurate for the ordering to still
be more useful than other arbitrary orderings).
It can also be assumed that in most cases the origin_server_ts for a given
benign server will be mostly consistent. For example, if a server sends a join
and then a leave in the vast majority of cases the leave would have a greater
origin_server_ts.
This makes "origin_server_ts" a good candidate to be used as a last resort to
order events if necessary, where otherwise a different arbitrary ordering would
be used. However, it's important that there is some other mechanism to ensure
that malicious servers can't abuse origin_server_ts to ensure their state
always gets picked during resolution (In the proposal below we use the auth DAG
ordering to override users who set state with malicious origin_server_ts.)
## Ordering and Authing
Roughly, the current algorithm tries to ensure that moderation evasion doesn't
happen by ordering conflicted events by depth and (re)authing them
sequentially. The exact implementation has several issues, but the idea of
ensuring that state events from forks still need to pass auth subject to e.g.
bans and power level changes is a powerful one, as it reduces the utility of
maliciously forking.
For that to work we need to ensure that there is a suitable ordering that puts
e.g. bans before events sent in other forks. (However events can point to old
parts of the DAG, for a variety of reasons, and ideally in that case the
resolved state would closely match the recent state).
Similarly care should be taken when multiple changes to e.g. power levels happen
in a fork. If Alice gives Bob power (A), then Bob gives Charlie power (B) and
then Charlie, say, changes the ban level (C). If you try and resolve two state
sets one of which has A and the other has C, C will not pass auth unless B is
also taken into account. This case can be handled if we also consider the
difference in auth chains between the two sets, which in the previous example
would include B.
(This is also the root cause of the "Hotel California" issue, where left users
get spontaneously rejoined to rooms. This happens when a user has a sequence of
memberships changes of the form: leave (A), join (B) and then another leave (C).
In the current algorithm a resolution of A and C would pick A, and a resolution
of A and B would then pick B, i.e. the join. This means that a suitably forked
graph can reset the state to B. This is fixed if when resolving A and C we also
consider B, since its in the auth chain of C.)
## Power Level Ordering
Actions that malicious servers would try and evade are actions that require
greater power levels to perform, for example banning, reducing power level,
etc. We define "power events" as events that have the potential to remove the
ability of another user to do something.[^2] (Note that they are a subset of
auth events.)
In all these cases it is desirable for those privileged actions to take
precedence over events in other forks. This can be achieved by first
considering "power events", and requiring the remaining events to pass auth
based on them.
## Mainline
An issue caused by servers not storing the full room DAG is that one can't tell
how two arbitrary events are ordered. The auth chain gives a partial ordering
to certain events, though far from complete; however, all events do contain a
reference to the current power levels in their auth events. As such if two
state events reference two different power levels events, and one power levels'
auth chain references the other, then there is a strong likelihood that the
event referencing the latter power level came after the other event.
A "mainline" is a list of power levels events created if you pick a particular
power levels event (usually the current resolved power level) and recursively
follow each power level referenced in auth_events back to the first power level
event.
The mainline can then be used to induce an ordering on events by looking at
where the power level referenced in their auth_events is in the mainline (or
recursively following the chain of power level events back until one is found
that appears in the mainline). This effectively partitions the room into
epochs, where a new epoch is started whenever a new power level is sent.
If this mainline ordering is combined with ordering by origin_server_ts, then
it gives an ordering that is correct for servers that don't lie about the time,
while giving a mechanism that can be used to deal if a server lied (by room
admins starting a new epoch).
The natural course of action for a room admin to take when noticing a
user/server is misbehaving is to ban them from the room, rather than changing
the power levels. It would therefore be useful if banning a user or server
started a new epoch as well. This would require being able to create a mainline
that includes power level events and bans[^3], which would suggest that power
level and ban events would need to point to the latest ban event as well. (This
would be significantly easier if we maintained a list of bans in a single
event, however there is concern that would limit the number of possible bans in
a room.)
# Proposed Algorithm
First we define:
* **"State sets"** are the sets of state that the resolution algorithm tries
to resolve, i.e. the inputs to the algorithm.
* **"Power events"** are events that have the potential to remove the ability
of another user to do something. These are power levels, join rules, bans
and kicks.
* The **"unconflicted state map"** is the state where the value of each key
exists and is the same in every state set. The **"conflicted state map"** is
everything else. (Note that this is subtly different to the definition used
in the existing algorithm, which considered the merge of a present event
with an absent event to be unconflicted rather than conflicted)
* The "**auth difference"** is calculated by first calculating the full auth
chain for each state set and taking every event that doesn't appear in every
auth chain.
* The **"full conflicted set"** is the union of the conflicted state map and
auth difference.
* The **"reverse topological power ordering"**[^4] of a set of events is an
ordering of the given events, plus any events in their auth chains that
appear in the auth difference, topologically ordered by their auth chains
with ties broken such that x < y if:
1. x's sender has a greater power level than y (calculated by looking at
their respective auth events, or if
2. x's origin_server_ts is less than y's, or if
3. x's event_id is lexicographically less than y's
This is also known as a lexicographical topological sort (i.e. this is the
unique topological ordering such that for an entry x all entries after it
must either have x in their auth chain or be greater than x as defined
above). This can be implemented using Kahn's algorithm.
* The **"mainline ordering"** based on a power level event P of a set of
events is calculated as follows:
1. Generate the list of power levels starting at P and recursively take the
power level from its auth events. This list is called the mainline,
ordered such that P is last.
1. We say the "closest mainline event" of an event is the first power level
event encountered in mainline when iteratively descending through the
power level events in the auth events.
1. Order the set of events such that x < y if:
1. The closest mainline event of x appears strictly before the closest
of y in the mainline list, or if
1. x's origin_server_ts is less than y's, or if
1. x's event_id lexicographically sorts before y's
* The **"iterative auth checks"** algorithm is where given a sorted list of
events, the auth check algorithm is applied to each event in turn. The state
events used to auth are built up from previous events that passed the auth
checks, starting from a base set of state. If a required auth key doesn't
exist in the state, then the one in the event's auth_events is used if the
auth event is not rejected. (See _Variations_ and _Attack Vectors_ below).
The algorithm proceeds as follows:
1. Take all power events and any events in their auth chains that appear in the
_full_ _conflicted set_ and order them by the _reverse topological power
ordering._
1. Apply the _iterative auth checks_ algorithm based on the unconflicted state
map to get a partial set of resolved state.
1. Take all remaining events that weren't picked in step 1 and order them by
the _mainline ordering_ based on the power level in the partially resolved
state.
1. Apply the _iterative auth checks algorithm_ based on the partial resolved
state.
1. Update the result with the _unconflicted state_ to get the final resolved
state[^5]. (_Note_: this is different from the current algorithm, which
considered different event types at distinct stages)
An example python implementation can be found on github
[here](https://github.com/matrix-org/matrix-test-state-resolution-ideas).
Note that this works best if we also change which events to include as an
event's auth_events. See the "Auth Events" section below.
## Discussion
Essentially, the algorithm works by producing a sorted list of all conflicted
events (and differences in auth chains), and applies the auth checks one by
one, building up the state as it goes. The list is produced in two parts: first
the power events and auth dependencies are ordered by power level of the
senders and resolved, then the remaining events are ordered using the
"mainline" of the resolved power levels and then resolved to produce the final
resolved state.
(This is equivalent to linearizing the full conflicted set of events and
reapplying the usual state updates and auth rules.)
### Variations
There are multiple options for what to use as the base state for _iterative
auth checks_ algorithm; while it needs to be some variation of auth events and
unconflicted events, it is unclear exactly which combination is best (and least
manipulatable by malicious servers).
Care has to be taken if we want to ensure that old auth events that appear in
the _auth chain difference_ can't supersede unconflicted state entries.
Due to auth chain differences being added to the resolved states during
_iterative auth checks_, we therefore need to re-apply the unconflicted state
at the end to ensure that they appear in the final resolved state. This feels
like an odd fudge that shouldn't be necessary, and may point to a flaw in the
proposed algorithm.
### State Resets
The proposed algorithm still has some potentially unexpected behaviour.
One example of this is when Alice sets a topic and then gets banned. If an event
gets created (potentially much later) that points to both before and after the
topic and ban then the proposed algorithm will resolve and apply the ban before
resolving the topic, causing the topic to be denied and dropped from the
resolved state. This will result in no topic being set in the resolved state.
### Auth Events
The algorithm relies heavily on the ordering induced by the auth chain DAG.
There are two types of auth events (not necessarily distinct):
* Those that give authorization to do something
* Those that revoke authorization to do something.
For example, invites/joins are in the former category, leaves/kicks/bans are in
the latter and power levels are both.
Assuming[^6] revocations always point to (i.e., have in their auth chain) the
authorization event that they are revoking, and authorization events point to
revocations that they are superseding, then the algorithm will ensure that the
authorization events are applied in order (so generally the "latest"
authorization state would win).
This helps ensure that e.g. an invite cannot be reused after a leave/kick,
since the leave (revocation) would have the invite in their auth chain.
This idea also relies on revocations replacing the state that granted
authorization to do an action (and vice versa). For example, in the current
model bans (basically) revoke the ability for a particular user from being able
to join. If the user later gets unbanned and then rejoins, the join would point
to the join rules as the authorization that lets them join, but would not
(necessarily) point to the unban. This has the effect that if a state resolution
happened between the new join and the ban, the unban would not be included in
the resolution and so the join would be rejected.
The changes to the current model that would be required to make the above
assumptions true would be, for example:
1. By default permissions are closed.
1. Bans would need to be a list in either the join rules event or a separate
event type which all membership events pointed to.
1. Bans would only revoke the ability to join, not automatically remove users
from the room.
1. Change the defaults of join_rules to be closed by default
### Efficiency and Delta State Resolution
The current (unoptimised) implementation of the algorithm is 10x slower than
the current algorithm, based on a single, large test case. While hopefully some
optimisations can be made, the ability to [incrementally calculate state
resolution via deltas](https://github.com/matrix-org/synapse/pull/3122) will
also mitigate some of the slow down.
Another aspect that should be considered is the amount of data that is required
to perform the resolution. The current algorithm only requires the events for
the conflicted set, plus the events from the unconflicted set needed to auth
them. The proposed algorithm also requires the events in the auth chain
difference (calculating the auth chain difference may also require more data to
calculate).
Delta state resolution is where if you have, say, two state sets and their
resolution, then you can use that result to work out the new resolution where
there has been a small change to the state sets. For the proposed algorithm, if
the following properties hold true then the result can be found by simply
applying steps 3 and 4 to the state deltas. The properties are:
1. The delta contains no power events
1. The origin_server_ts of all events in state delta are strictly greater than
those in the previous state sets
1. Any event that has been removed must not have been used to auth subsequent
events (e.g. if we replaced a member event and that user had also set a
topic)
These properties will likely hold true for most state updates that happen in a
room, allowing servers to use this more efficient algorithm the majority of the
time.
### Full DAG
It's worth noting that if the algorithm had access to the full room DAG that it
would really only help by ensuring that the ordering in "reverse topological
ordering" and "mainline ordering" respected the ordering induced by the DAG.
This would help, e.g., ensure the latest topic was always picked rather than
rely on origin_server_ts and mainline. As well as obviate the need to maintain
a separate auth chain, and the difficulties that entails (like having to
reapply the unconflicted state at the end).
### Rejected Events
Events that have been rejected due to failing auth based on the state at the
event (rather than based on their auth chain) are handled as usual by the
algorithm, unless otherwise specified.
Note that no events rejected due to failure to auth against their auth chain
should appear in the process, as they should not appear in state (the algorithm
only uses events that appear in either the state sets or in the auth chain of
the events in the state sets).
This helps ensure that different servers' view of state is more likely to
converge, since rejection state of an event may be different. This can happen if
a third server gives an incorrect version of the state when a server joins a
room via it (either due to being faulty or malicious). Convergence of state is a
desirable property as it ensures that all users in the room have a (mostly)
consistent view of the state of the room. If the view of the state on different
servers diverges it can lead to bifurcation of the room due to e.g. servers
disagreeing on who is in the room.
Intuitively, using rejected events feels dangerous, however:
1. Servers cannot arbitrarily make up state, since they still need to pass the
auth checks based on the event's auth chain (e.g. they can't grant themselves
power levels if they didn't have them before).
2. For a previously rejected event to pass auth there must be a set of state
that allows said event. A malicious server could therefore produce a
fork where it claims the state is that particular set of state, duplicate the
rejected event to point to that fork, and send the event. The
duplicated event would then pass the auth checks. Ignoring rejected events
would therefore not eliminate any potential attack vectors.
Rejected auth events are deliberately excluded from use in the iterative auth
checks, as auth events aren't re-authed (although non-auth events are) during
the iterative auth checks.
### Attack Vectors
The main potential attack vector that needs to be considered is in the
_iterative auth checks_ algorithm, and whether an attacker could make use of
the fact that it's based on the unconflicted state and/or auth events of the
event.
# Appendix
The following are some worked examples to illustrate some of the mechanisms in
the algorithm. In each we're interested in what happens to the topic.
## Example 1 - Mainline
The following is an example room DAG, where time flows down the page. We shall
work through resolving the state at both _Message 2_ and _Message 3_.
![alt_text](images/state-res.png)
(Note that green circles are events sent by Alice, blue circles sent by Bob and
black arrows point to previous events. The red arrows are the mainline computed
during resolution.)
First we resolve the state at _Message 2_. The conflicted event types are the
power levels and topics, and since the auth chains are the same for both state
sets the auth difference is the empty set.
Step 1: The _full conflicted set_ are the events _P2, P3, Topic 2 _and _Topic
3_, of which _P2_ and _P3_ are the only power events. Since Alice (the room
creator) has a greater power level than Bob (and neither _P2 _and _P3_ appear
in each other's auth chain), the reverse topological ordering is: [_P2, P3_].
Step 2: Now we apply the auth rules iteratively, _P2_ trivially passes based on
the unconflicted state, but _P3_ does not pass since after _P2_ Bob no longer
has sufficient power to set state. This results in the power levels resolving
to _P2_.
Step 3: Now we work out the mainline based on P2, which is coloured in red on
the diagram. We use the mainline to order _Topic 2_ and _Topic 3_. _Topic 2_
points to_ P1_, while the closest mainline to _Topic 3_ is also _P1_. We then
order based on the _origin_server_ts_ of the two events, let's assume that
gives us: [_Topic 2_, _Topic 3_].
Step 4: Iteratively applying the auth rules results in _Topic 2_ being allowed,
but _Topic 3 _being denied (since Bob doesn't have power to set state anymore),
so the topic is resolved to _Topic 2_.
This gives the resolved state at _Message 2_ to be _P2 _and _Topic 2_. (This is
actually the same result as the existing algorithm gives)
Now let's look at the state at _Message 3_.
Step 1: The full conflicted set are simple: _Topic 2_ and _Topic 4_. There are
no conflicted power events.
Step 2: N/A
Step 3: _Topic 2_ points to _P1_ in the mainline, and _Topic 4_ points to _P2_
in its auth events. Since _P2_ comes after _P1_ in the mainline, this gives an
ordering of [_Topic 2, Topic 4_].
Step 4: Iteratively applying the auth rules results in both topics passing the
auth checks, and so the last topic, _Topic 4_, is chosen.
This gives the resolved state at _Message 3_ to be _Topic 4_.
## Example 2 - Rejected Events
The following is an example room DAG, where time flows down the page. The event
`D` is initially rejected by the server (due to not passing auth against the
state), but does pass auth against its auth chain.
![state-res-rejected.png](images/state-res-rejected.png)
(Note that the blue lines are the power levels pointed to in the event's auth
events)
At `F` we first resolve the power levels, which results in `E`. When we then go
to resolve the topics against the partially resolved state, Bob has ops and so
the resolved state includes the topic change `D`, even though it was initially
rejected.
## Notes
[^1]: In the current room protocol these are: the create event, power levels,
membership, join rules and third party invites. See the
[spec](https://github.com/matrix-org/matrix-doc/blob/7cb918407dc8c505c67c750578c63b43042c8425/specification/server_server_api.rst#41pdu-fields).
[^2]: In the current protocol these are: power levels, kicks, bans and join
rules.
[^3]: Future room versions may have a concept of server ban event that works
like existing bans, which would also be included
[^4]: The topology being considered here is the auth chain DAG, rather than the
room DAG, so this ordering is only applicable to events which appear in the
auth chain DAG.
[^5]: We do this so that, if we receive events with misleading auth_events, this
ensures that the unconflicted state at least is correct.
[^6]: This isn't true in the current protocol

45
proposals/1466-soft-logout.md
Unescape Escape View File

@ -1,45 +0,0 @@
# Soft Remote Logout Proposal
## Motivation
Currently when a user logs out of riot, the app will destroy things like
encryption keys. If the logout was done by the user in local app then it will
have first prompted the user to export the encryption keys. However, when the
app is logged out remotely (i.e. it received a 401 from the server) there is no
opportunity to ask the user to backup the keys, resulting in those keys being
lost.
While this behaviour is useful in many circumstances, e.g. remote logging out a
stolen/lost device, it also means that the server shouldn't automatically
log out devices, to avoid users losing encryption keys.
## Proposal
A new parameter is added to the JSON body of 401 responses, called
`soft_logout`. This is a boolean flag (defaulting to `false`) that signals to
the client whether it should keep local data and simply prompt to reauth (when
`true`) or to destroy the current data like it does today (when `false`).
The major disadvantage with this approach is that once the client is logged
out, the user can no longer remotely cause the client to destroy the local
data. However, this is not substantially different from today where the app has
to be opened to receive remote logout requests (via 401), as it allows
attackers to get at encryption keys even after remote logout if they simply
avoid opening the app.
### Example Client UX
When handling a soft logout the client could show a "re-login" dialogue, rather
than back to the default logged out screen. This new dialogue would then have
several options including logging out, exporting keys and logging out fully.
## Alternatives
One alternative is to force the user to enter a password for backing up keys
when they enter the app, and then have the app keep secure backups of they
keys. This then means its safer to not delete the secure backups when the app
is logged out remotely.

370
proposals/1501-room-version-upgrades.md
Unescape Escape View File

@ -1,370 +0,0 @@
# Room version upgrades
## Background
[MSC1425](https://github.com/matrix-org/matrix-doc/issues/1425) introduces a
mechanism for associating a "version" with a room, which allows us to introduce
changes to the mechanics of rooms.
Assuming that a given change is successful, the next challenge is to introduce
it to existing rooms. This proposal introduces a mechanism for doing so.
## Proposal
In short: room upgrades are implemented by creating a new room, shutting down
the old one, and linking between the two.
The mechanics of this are as follows. When Alice upgrades a room, her client
hits a new C-S api:
```
POST /_matrix/client/r0/rooms/{roomId}/upgrade
```
```json
{
"new_version": "2"
}
```
Response:
```json
{
"replacement_room": "!QtykxKocfsgujksjgd:matrix.org"
}
```
When this is called, the server:
* Checks that Alice has permissions to send `m.room.tombstone` state events.
* Creates a replacement room, with an `m.room.create` with a `predecessor` field
which links to the last known event in the old room:
```json
{
"sender": "@alice:somewhere.com",
"type": "m.room.create",
"state_key": "",
"room_id": "!QtykxKocfsgujksjgd:matrix.org",
"content": {
"room_version": "2",
"predecessor": {
"room_id": "!cURbaf:matrix.org",
"event_id": "$1235135aksjgdkg:matrix.org"
}
}
}
```
* Replicates PL/privacy/topic/etc events to the new room.
* Moves any local aliases to the new room.
* Sends an `m.room.tombstone` state event in the old room to tell participants
that it is dead:
```json
{
"sender": "@alice:somewhere.com",
"type": "m.room.tombstone",
"state_key": "",
"room_id": "!cURbaf:matrix.org",
"content": {
"body": "This room has been replaced",
"replacement_room": "!QtykxKocfsgujksjgd:matrix.org"
}
}
```
The `body` of the tombstone event is defined by the server (for now, at
least).
* Assuming Alice has the powers to do so, sets the power levels in the old
room to stop people speaking. In practice, this means setting
`events_default` and `invite` to the greater of `50` and `users_default+1`.
Bob's client understands the `m.room.tombstone` event, and:
* Hides the old room in the room list (the room continues to be accessible
via the old room id (permalinks, backlinks from the new room, etc).
* Displays, at the very bottom of the timeline of the old room: "This room
has been upgraded. Click here to follow the conversation to the new room".
The link is simply a permalink to the new room. When Bob opens it, he will
get joined to the new room.
[Note that if Bob is on a version of synapse which doesn't understand room
versions, following the permalink will take him to a room view which churns
for a while and eventually fails. Synapse 0.33.3 should at least give a
sensible error code.]
If it turns out that the replacement room also has a tombstone event, the
client may automatically keep following the chain until it reaches a room
that isn't dead.
* Optional extension: if the user is in both rooms, then the "N unread
messages" banner when scrolled up in the old room could be made to track
messages in the new room (so in practice the user would only ever see the
hiatus between the versions if they scrolled all the way to the beginning
of the new room or the end of the old one.)
Bob's client also understands the `predecessor` field in the `m.room.create`, and:
* At the top of scrollback in the new room, displays: "This room is a
continuation of a previous room. Click here to see the old conversation."
The link is a permalink to the old room.
* Optional extensions might include things like extending room search to
work across two rooms.
### Client changes needed
* Ability for an op to view the current room version and upgrade it (by
hitting `/upgrade`).
* ~~Also the ability for an op to see what versions the servers in the
current room supports (nb via a cap API) and so how many users will get
locked out~~ (This is descoped for now.)
* Display `m.room.tombstone`s as a sticky message at the bottom of the old
room (perhaps replacing the message composer input) as “This room has been
replaced. Please click here to continue” or similar.
* When the user clicks the link, the client attempts to join the new room if
we are not already a member, and then switches to a view on the new room.
The client should supply the name of the server which sent the tombstone
event as the `server_name` for the `/join` request. (That being the most
likely server to have an up-to-date copy of the room - this is essentially
a workaround for [vector-im/riot-web#2925](https://github.com/vector-im/riot-web/issues/2925).)
* If the client sees a pair of rooms with a tombstone correctly joined to the
new room, it should hide the old one from the RoomList.
* If one backpaginates the new room to its creation, we should show the
`m.room.create` as “This room is a continuation of a previous room; click here
to view” (or similar).
* Search is extended to search old rooms (could be done clientside for now).
Future eye-candy:
* When one is viewing the old room:
* Rather than showing a sticky tombstone at the bottom, one should probably
have the “10 unread messages” section in the status bar which refers to
the current version of the room.
* We should probably also show the membership list of the current room. (Perhaps?)
## Future extensions
### Invite-only rooms
Invite-only rooms are not dealt with explicitly here; they are made tricky by
the fact that users in the old room won't be able to join the new room without
an invite.
For now, we are assuming that the main reasons for carrying out a room-version
upgrade (ie, security problems due to state resets and the like) do not apply
as strongly to invite-only rooms, and we have descoped them for now.
In future, we will most likely deal with them as follows:
* For local users, we need to first create an invite for each user in the
room. This is easy, if a bit high-overhead.
* For remote users:
* Alice's server could send invites; however this is likely to give an
unsatisfactory UX due to servers being offline, maybe not supporting the
new room version, and then spamming Bob with mysterious invites.
* We could change the auth rules to treat a membership of the old room as
equivalent to an invite to the new room. However, this is likely to
reintroduce the problems we are trying to solve by replacing the room in
the first place.
* We could create a new type of membership which acts like an invite for
the purposes of allowing users into invite-only rooms, but doesn't need
to be sent to remote servers.
### Parting users from the old room
It's not obvious if users should stay members of the old room indefinitely
(until they manually leave). Perhaps they should automatically leave after a
respectful period? What if the user leaves the *new* room?
For now, we'll assume that they stay members until they manually leave. We can
see how that feels in practice.
## Potential issues
* What about clients that don't understand tombstones?
* I think they'll just show you two separate rooms (both with the same
name), and you won't be able to talk in one of them. It's not great but it
will probably do.
* It's a shame that scrollback in the new room will show a load of joins
before you get to the link to the old room.
## Dismissed solutions
### Variations on this proposal
#### Have servers auto-join their users on upgrade
In order to make the upgrade more seamless, it might be good for servers to
automatically join any users that were in the old room to the new room.
In short, when Bob's server receives a tombstone event, it would attempt to
auto-join Bob to the new room, and updates any aliases it may have to point to
the new room.
It's worth noting that the join may not be successful: for example, because
Bob's server is too old, or because Bob has been blocked from joining the new
room, or because joins are generally flaky. In this case Bob might attempt a
rejoin via his client.
#### Have servers merge old and new rooms together
Instead of expecting clients to interpret tombstone events, servers could merge
/sync results together for the two rooms and present both as the old room in
/sync results - and then map from the old room ID back to the new one for API
calls.
(Note that doing this the other way around (mapping both rooms into the *new*
room ID) doesn't really work without massively confusing clients which have
cached data about the old room.)
This sounds pretty confusing though: would the server do this forever
for all users on the server?
#### have clients merge old and new rooms
At the top of scrollback for the new room, shows some sort of "messages from a
previous version of this room" divider, above which it shows messages from the
old room <sup name="a10">[1](#f10)</sup>.
This gets quite messy quite quickly. For instance, it will be confusing to see
any ongoing traffic in the old room (people whose servers didn't get the memo
about the tombstone; people leaving the old room; etc). Ultimately it's all
client polish though, so we can consider this for the future.
<b name="f10">1</b> It's probably worth noting that this alone is a somewhat
fundamental reworking of a bunch of complicated stuff in matrix-react-sdk (and
presumably likewise in the mobile clients). [↩](#a10)
### Counter-proposal: upgrade existing rooms in-place
The general idea here is to divide the room DAG into v1 and v2 parts. Servers
which do not support v2 will not see the v2 part of the DAG. This might look
like the below:
![dag](1501-split-dag.png)
In this model, room version is tracked as a state event:
```json
{
"type": "m.room.version",
"content": {
"version": 2
}
}
```
This event can only ever be sent by the **original room creator** - i.e., the
sender of the `m.room.create event` - even if the `m.room.power_levels` say
otherwise <sup name="a1">[1](#f1)</sup> <sup name="a2">[2](#f2)</sup>.
The DAG is now divided into parts where the state of m.room.version is 2, and
those where it is missing (and therefore implicitly 1).
When sending out federation transactions, v2 events are listed under a new
versioned_pdus key in the /send request:
```
PUT /_matrix/federation/v1/send/991079979
{
"origin_server_ts": 1404835423000,
"origin": "matrix.org",
"pdus": [],
"versioned_pdus": {
2: [...]
},
"edus": [...]
}
```
Old servers which do not support v2 will therefore not receive any v2 parts of
the DAG, and transactions which only contain updates to the v2 part of the DAG
will be treated as no-ops. Servers should ignore entries under versioned_pdus
which correspond to versions they do not know about.
As a special case, `m.room_version events` themselves are treated as having the
version previously in force in the room, unless that was v1, in which case,
they are treated as being v2. This means that the upgrade from v1 to v2, *and*
the upgrade from v2 to v3, are included under `versioned_pdus[2]`, but the
upgrade from v3 to v4 is included under `versioned_pdus[3]`. If a server
receives an `m.room_version` event with a version it does not understand, it
must refuse to allow any other events into the upgraded part of the DAG (and
probably *should* refuse to allow any events into the DAG at all)
<sup name="a1">[3](#f3)</sup>. The reasons for this are as follows:
* We want to ensure that v1-only servers do not receive the room_version
event, since they don't know how to auth it correctly, and more importantly
we want to ensure that they can't get hold of it to construct bits of DAG
which refer to it, and would have to follow the v2 rules.
* However, putting subsequent upgrades in a place where older servers will see
it means that we can do more graceful upgrades in future by telling their
clients that the room seems to have been upgraded.
As normal, we require joining servers to declare their support for room
versions, and reject any which do not include the version of the room given by
the current room state.
In theory this is all that is required to ensure that v1-only servers never see
any v2 parts of the DAG (events are only sent via /send, and a v1 server should
have no way to reference v2 parts of the DAG); however we may want to consider
adding a "versions" parameter to API calls like /event and /state which allow a
server to specify an event_id to fill from, in case a (non-synapse, presumably)
implementation gets an event id from a /context request or similar and tries to
fill from it.
The experience for clients could be improved by awareness of the m.room.version
event and querying a capabilities API (e.g. /versions?) to determine whether a
room has been upgraded newer than the server can support. If so, a client
could say: “Your server must be upgraded to continue to participate in this
room” (and manually prevent the user from trying to speak).
#### Problems
* This leaves users on old v1-only servers in their own view of the room,
potentially continuing to speak but only seeing a subset of events (those
from their own and other v1 servers).
In the worst case, they might ask questions which those on v2 servers can see but have no way of replying to.
One way to mitigate this would be to ensure that the last event sent in the
v1 part of the DAG is an m.room.power_levels to make the room read-only to
casual users; then restore the situation in the v2 DAG.
* There's nothing to stop "inquisitive" users on v1 servers sending
m.room.version events, which is likely to completely split-brain the room.
* This doesn't help us do things like change the format of the room id.
<b name="f1">1</b> This avoids the version event itself being vulnerable to state
resets. [↩](#a1)
<b name="f2">2</b> This is therefore a change to the event authorisation rules
which we need to introduce with room version 2. [↩](#a2)
<b name="f3">3</b> It's worth noting this does give the room creator a
kill-switch for the room, even if they've subsequently been de-opped. Such is
life. [↩](#a3)

BIN
proposals/1501-split-dag.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 21 KiB

297
proposals/1543-qr_code_key_verification.md
Unescape Escape View File

@ -1,297 +0,0 @@
Bi-directional Key verification using QR codes
==============================================
Problem/Background
------------------
Key verification is essential in ensuring that end-to-end encrypted messages
cannot be read by unauthorized parties. Traditionally, key verification is
done by comparing long strings. To save users from the tedium of reading out
long strings, some systems allow one party to verify the other party by
scanning a QR code; by doing this twice, both parties can verify each other.
In this proposal, we present a method for both parties to verify each other by
only scanning one QR code.
Proposal
--------
When Alice and Bob meet in person to verify keys, Alice will scan a QR code
generated by Bob's device. The QR code will encode both Bob's key as well as what Bob
thinks Alice's key is. When Alice scans the QR code, she will ensure that the
keys match what is expected, in which case, she relays this information to Bob,
who can then tell his device that the keys match.
### Example flow
1. Alice and Bob meet in person, and want to verify each other's keys.
2. Alice requests a key verification through her device by sending an
`m.key.verification.request` message (see
[MSC2241](https://github.com/matrix-org/matrix-doc/pull/2241)), with
`m.qr_code.show.v1`, `m.qr_code.scan.v1`, and `m.reciprocate.v1` listed in
`methods`, and Bob responds with a `m.key.verification.ready` message.
3. Alice's client displays a QR code that Bob is able to scan, and an option to
scan Bob's QR code.
4. Bob's client prompts Bob to verify Alice's key. The prompt includes a QR
code that Alice can scan (if the `m.key.verification.request` message listed
`m.qr_code.scan.v1`), and an option to scan Alice's QR code (if the
`m.key.verification.request` message listed `m.qr_code.show.v1`). The QR
code encodes:
- Bob's master cross-signing public key,
- what Bob thinks Alice's master cross-signing public key is,
- a random shared secret.
5. Alice scans Bob's QR code.
6. Alice's device ensures that:
- Bob's key encoded in the QR code matches the key that she already has for
Bob, and
- Alice's cross-signing key matches the cross-signing key encoded in the QR
code.
If any of these checks fail, Alice's device displays an error message
indicating that the code is incorrect, and sends a
`m.key.verification.cancel` message to Bob's device.
Otherwise, at this point:
- Alice's device has now verified Bob's key, and
- Alice's device knows that Bob has the correct key for her.
Thus for Bob to verify Alice's key, Alice needs to tell Bob that he has the
right key.
7. Alice's device displays a message saying that all is well. This message
tells Alice that she has the right key for Bob, and tells Bob that he has
the right key for Alice.
8. Alice's device sends a `m.key.verification.start` message with `method` set
to `m.reciprocate.v1` to Bob (see below). The message includes the shared
secret from the QR code. This signals to Bob's device that Alice has
scanned Bob's QR code.
This message is merely a signal for Bob's device to proceed to the next
step, and is not used for verification purposes.
9. Upon receipt of the `m.key.verification.start` message, Bob's device ensures
that the shared secret matches.
If the shared secret does not match, it should display an error message
indicating that an attack was attempted. (This does not affect Alice's
verification of Bob's keys.)
If the shared secret does match, it asks Bob to confirm that Alice
has scanned the QR code.
10. Bob sees Alice's device confirm that the key matches, and presses the button
on his device to indicate that Alice's key is verified.
Bob's verification of Alice's key hinges on Alice telling Bob the result of
her scan. Since the QR code includes what Bob thinks Alice's key is,
Alice's device can check whether Bob has the right key for her. Alice has
no motivation to lie about the result, as getting Bob to trust an incorrect
key would only affect communications between herself and Bob. Thus Alice
telling Bob that the code was scanned successfully is sufficient for Bob to
trust Alice's key, under the assumption that this communication is done
over a trusted medium (such as in-person).
11. Both devices send an `m.key.verification.done` message.
This flow allows Alice to verify Bob's key, and Bob to verify Alice's key.
Alice verifies Bob's key because she can trust the QR code that Bob displays
for her, as this is done over a trusted medium. Bob verifies Alice's key
because Alice can trust the QR code that Bob displays, and Bob can trust Alice
to tell him the result of the verification.
#### Self-verification
QR codes can also be used by a user to verify their own devices. These examples
shows Alice verifying two devices, one of them (Osborne2) having cross-signing
already set up, and the other one (Dynabook) having just logged in.
In the first example, Osborne2 scans Dynabook:
1. Alice logs into her new Dynabook and wants other users to be able to trust
it via cross-signing, and to trust other devices via cross-signing.
2. Dynabook retrieves Alice's public cross-signing key from the server, and
displays a QR code that encodes:
- Dynabook's device key,
- what it thinks Alice's master key is, and
- a random shared secret.
Note that in this case, the QR code does not include Alice's master key in a
`key_<key_id>` parameter, since Dynabook does not know whether it is trusted
or not.
3. Osborne2 scans the QR code displayed by Dynabook. At this point, Osborne2
knows Dynabook's device key and can sign it with the self-signing key and
upload the signature, and can trust Dynabook for sending secrets via SSSS.
It also knows that Dynabook has the correct cross-signing key.
4. Osborne2 tells Alice that the scan was successful, and sends the
`reciprocate` message containing the shared secret.
5. Upon receipt of the `reciprocate` message, Dynabook (after checking the
shared secret) confirms with Alice that she successfully scanned the QR
code.
6. Alice confirms.
7. Dynabook now knows that it can trust Alice's cross-signing keys that it
fetched from the server.
In the second example, Dynabook scans Osborne2:
1. Alice logs into her new Dynabook and wants other users to be able to trust
it via cross-signing, and to trust other devices via cross-signing.
2. Osborne2 notices that Dynabook is a new device. Osborne2 fetches Dynabook's
identity key and displays a QR code that encodes:
- what it thinks Dynabook's key is,
- Alice's master key, and
- a random shared secret.
3. Dynabook scans the QR code shown by Osborne2. At this point, Dynabook knows
Alice's cross-signing key, and so it can trust it to sign other devices. It
also knows that Osborne2 as the correct key for it.
4. Dynabook tells Alice that the scan is successful, and sends the
`reciprocate` message containing the shared secret.
5. Upon receipt of the `reciprocate` message, Osborne2 (after checking the
shared secret) confirms with Alice that she successfully scanned the QR
code.
6. Alice confirms.
7. Osborne2 now knows that it has the correct device key for Dynabook, and can
sign it with the self-signing key and upload the signature. Osborne2 can
also trust Dynabook for sending secrets via SSSS.
### Verification methods
This proposal defines three verification methods that can be used in
`m.key.verification.request` messages (see
[MSC2241](https://github.com/matrix-org/matrix-doc/pull/2241)).
- `m.qr_code.show.v2`: means that the sender of the
`m.key.verification.request` message can show a QR code that the recipient
can scan. If the recipient can scan the QR code, it should allow the user to
do so. This method is never sent as part of a `m.key.verification.start`
message.
- `m.qr_code.scan.v2`: means that the sender of the
`m.key.verification.request` message can scan a QR code displayed by the
recipient. If the recipient can display a QR code, it should allow the user
to display it so that the sender can scan it. This method is never sent as
part of a `m.key.verification.start` message.
- `m.reciprocate.v1`: means that the sender can participate in a reciprocal
verification, either as initiator or responder, as described in the [Message
types](#message-types) section below.
### QR code format
The QR codes to be displayed and scanned using this format will encode binary
strings in the general form:
- the ASCII string "MATRIX"
- one byte indicating the QR code version (must be `0x02`)
- one byte indicating the QR code verification mode. May be one of the
following values:
- `0x00` verifying another user with cross-signing
- `0x01` self-verifying in which the current device does trust the master key
- `0x02` self-verifying in which the current device does not yet trust the
master key
- the event ID or `transaction_id` of the associated verification
request event, encoded as:
- two bytes in network byte order (big-endian) indicating the length in
bytes of the ID as a UTF-8 string
- the ID as a UTF-8 string
- the first key, as 32 bytes. The key to use depends on the mode field:
- if `0x00` or `0x01`, then the current user's own master cross-signing public key
- if `0x02`, then the current device's device key
- the second key, as 32 bytes. The key to use depends on the mode field:
- if `0x00`, then what the device thinks the other user's master
cross-signing key is
- if `0x01`, then what the device thinks the other device's device key is
- if `0x02`, then what the device thinks the user's master cross-signing key
is
- a random shared secret, as a byte string. It is suggested to use a secret
that is about 8 bytes long. Note: as we do not share the length of the
secret, and it is not a fixed size, clients will just use the remainder of
binary string as the shared secret.
For example, if Alice displays a QR code encoding the following binary string:
```
"MATRIX" |ver|mode| len | event ID
4D 41 54 52 49 58 02 00 00 2D 21 41 42 43 44 ...
| user's cross-signing key | other user's cross-signing key | shared secret
00 01 02 03 04 05 06 07 ... 10 11 12 13 14 15 16 17 ... 20 21 22 23 24 25 26 27
```
this indicates that Alice is verifying another user (say Bob), in response to
the request from event "$ABCD...", her cross-signing key is
`0001020304050607...` (which is "AAECAwQFBg..." in base64), she thinks that
Bob's cross-signing key is `1011121314151617...` (which is "EBESExQVFh..." in
base64), and the shared secret is `2021222324252627` (which is "ICEiIyQlJic" in
base64).
### Message types
#### `m.key.verification.start`
Alice's device tells Bob's device that the QR code has been scanned.
message contents:
- `method`: `m.reciprocate.v1`
- `m.relates_to`: as per [key verification framework](https://github.com/matrix-org/matrix-doc/pull/2241)
- `secret`: the shared secret from the QR code, encoded using unpadded base64
Example:
```json
{
"method": "m.reciprocate.v1",
"m.relates_to": {
"rel_type": "m.reference",
"event_id": "$event_id_of_verification_request"
},
"secret": "shared+secret"
}
```
Note that this message could be sent by either the sender or the recipient of
the `m.key.verification.request` message, depending on which user scanned the
QR code.
### Cancellation
In addition to the cancellation codes specified in [the spec for
`m.key.verification.cancel`](https://matrix.org/docs/spec/client_server/r0.5.0#m-key-verification-cancel),
the following cancellation codes may be used:
- `m.qr_code.invalid`: The QR code is invalid (e.g. it is not a URL of the
required form)
The verification can also be cancelled with the error codes:
- `m.key_mismatch`: if the QR code has keys that do not match the expected
value
- `m.user_mismatch`: if the QR code is for a different user from what was expected
Tradeoffs/Alternatives
----------------------
Other methods of verifying keys, which do not require scanning QR codes, are
needed for devices that are unable to scan QR codes. One such method is
[MSC1267](https://github.com/matrix-org/matrix-doc/issues/1267). Since the key
verification framework allows for multiple methods to be supported, clients can
allow users to use different methods depending on their capability.
Rather than embedding the keys in the QR codes directly, the two clients could
perform an exchange similar to
[MSC1267](https://github.com/matrix-org/matrix-doc/issues/1267), and encoding
the Short Authentication String code in the QR code. However, this means that
the clients must exchange several messages before they can verify each other,
which would delay showing the QR codes. This proposal is also simpler to
implement.
This proposal does not support the case of asynchronous verification, such as
printing a QR code on a business card for others to scan. That may be address
in a separate MSC.
Security Considerations
-----------------------
The security of verifying Alice's key depends on Bob not hitting the "Verified"
button (step 10 in the example flow) until after Alice's device indicates
success or failure. Users have a tendency to click on buttons without reading
what the screen says, but this is partially mitigated by the fact that it is
unlikely that Bob will be interacting with the device while Alice is scanning
and Alice's device will display the verification results immediately upon
scanning. Also, Bob's device will not display the button until it receives the
`m.key.verification.start` message that contains the shared secret from the QR
code, which means that an attacker would need to be physically present while
Alice and Bob verify. This issue can also be addressed by allowing Bob to
easily undo the verification if Alice's device displays an error.

130
proposals/1659-event-id-as-hashes.md
Unescape Escape View File

@ -1,130 +0,0 @@
# Changing Event IDs to be Hashes
## Motivation
Having event IDs separate from the hashes leads to issues when a server receives
multiple events with the same event ID but different reference hashes. While
APIs could be changed to better support dealing with this situation, it is
easier and nicer to simply drop the idea of a separate event ID entirely, and
instead use the reference hash of an event as its ID.
## Identifier Format
Currently hashes in our event format include the hash name, allowing servers to
choose which hash functions to use. The idea here was to allow a gradual change
between hash functions without the need to globally coordinate shifting from one
hash function to another.
However now that room versions exist, changing hash functions can be achieved by
bumping the room version. Using this method would allow using a simple string as
the event ID rather than a full structure, significantly easing their usage.
One side effect of this would be that there would be no indication about which
hash function was actually used, and it would need to be inferred from the room
version. To aid debuggability it may be worth encoding the hash function into
the ID format.
**Conclusion:** Don't encode the hash function, since the hash will depend on
the version specific redaction algorithm anyway.
The proposal is therefore that the event IDs are a sha256 hash, encoded using
[unpadded
Base64](https://matrix.org/docs/spec/appendices.html#unpadded-base64), and
prefixed with `$` (to aid distinguishing different types of identifiers). For
example, an event ID might be: `$CD66HAED5npg6074c6pDtLKalHjVfYb2q4Q3LZgrW6o`.
The hash is calculated in the same way as previous event reference hashes were,
which is:
1. Redact the event
2. Remove `signatures` field from the event
3. Serialize the event to canonical JSON
4. Compute the hash of the JSON bytes
Event IDs will no longer be included as part of the event, and so must be
calculated by servers receiving the event.
## Changes to Event Formats
As well as changing the format of event IDs, we also change the format of the
`auth_events` and `prev_events` keys in events to simply be lists of event IDs
(rather than being lists of tuples).
A full event would therefore look something like (note that this is just an
illustrative example, and that the hashes are not correct):
```json
{
"auth_events": [
"$5hdALbO+xIhzcLTxCkspx5uqry9wO8322h/OI9ApnHE",
"$Ga0DBIICBsWIZbN292ATv8fTHIGGimwjb++w+zcHLRo",
"$zc4ip/DpPI9FZVLM1wN9RLqN19vuVBURmIqAohZ1HXg",
],
"content": {
"body": "Here is the message content",
"msgtype": "m.message"
},
"depth": 6,
"hashes": {
"sha256": "M6/LmcMMJKc1AZnNHsuzmf0PfwladVGK2Xbz+sUTN9k"
},
"origin": "localhost:8800",
"origin_server_ts": 1548094046693,
"prev_events": [
"$MoOzCuB/sacqHAvgBNOLICiGLZqGT4zB16MSFOuiO0s",
],
"room_id": "!eBrhCHJWOgqrOizwwW:localhost:8800",
"sender": "@anon-20190121_180719-33:localhost:8800",
"signatures": {
"localhost:8800": {
"ed25519:a_iIHH": "N7hwZjvHyH6r811ebZ4wwLzofKhJuIAtrQzaD3NZbf4WQNijXl5Z2BNB047aWIQCS1JyFOQKPVom4et0q9UOAA"
}
},
"type": "m.room.message"
}
```
## Changes to existing APIs
All APIs that accept event IDs must accept event IDs in the new format.
For S2S API, whenever a server needs to parse an event from a request or
response they must either already know the room version *or* be told the room
version in the request/response. There are separate MSCs to update APIs where
necessary.
For C2S API, the only change clients will see is that the event IDs have changed
format. Clients should already be treating event IDs as opaque strings, so no
changes should be required. Servers must add the `event_id` when sending the
event to clients, however.
Note that the `auth_events` and `prev_events` fields aren't sent to clients, and
so the changes proposed above won't affect clients.
## Protocol Changes
The `auth_events` and `prev_events` fields on an event need to be changed from a
list of tuples to a list of strings, i.e. remove the old event ID and simply
have the list of hashes.
The auth rules also need to change:
- The event no longer needs to be signed by the domain of the event ID (but
still needs to be signed by the sender’s domain)
- We currently allow redactions if the domain of the redaction event ID
matches the domain of the event ID it is redacting; which allows self
redaction. This check is removed and redaction events are always accepted.
Instead, the redaction event only takes effect and is sent down to clients
if/when the original event is received, and the domain of the events'
senders match. (While this is clearly suboptimal, it is the only practical
suggestion)
## Room Version
There will be a new room version v3 that is the same as v2 except uses the new
event format proposed above. v3 will be marked as 'stable' as defined in [MSC1804](https://github.com/matrix-org/matrix-doc/blob/travis/msc/room-version-client-advertising/proposals/1804-advertising-capable-room-versions.md)

39
proposals/1704-matrix.to-permalinks.md
Unescape Escape View File

@ -1,39 +0,0 @@
# matrix.to permalink navigation
Currently Matrix uses matrix.to URIs to reference rooms and other entities in a
permanent manner. With just a room ID, users can't get into rooms if their server
is not already aware of the room. This makes permalinks to rooms or events difficult
as the user won't actually be able to join. A matrix.to link generated using a
room's alias is not a permanent link due to aliases being transferable.
In lieu of an improved way to reference entities permanently in Matrix, a new parameter
is to be added to matrix.to URIs to assist clients and servers receiving permanent links
in joining the room.
For reference, existing permalinks look like this:
```
https://matrix.to/#/!somewhere:example.org
https://matrix.to/#/!somewhere:example.org/$something:example.org
```
By adding a new parameter to the end, receivers can more easily join the room:
```
https://matrix.to/#/!somewhere:example.org?via=example-1.org&via=example-2.org
https://matrix.to/#/!somewhere:example.org/$something:example.org?via=example-1.org&via=example-2.org
```
Clients can pass the servers directly to `/join` in the form of `server_name`
parameters.
When generating the permalinks, clients should pick servers that have a reasonably
high chance of being in the room in the distant future. The current recommendation
is to pick up to 3 unique servers where the first one is that of the user with the
highest power level in the room, provided that power level is 50 or higher. The other
2 servers should be the most popular servers in the room based on the number of joined
users. This same heuristic should apply to the first server if no user meets the power
level requirements. Servers blocked by server ACLs should not be picked because they
are unlikely to continue being residents of the room. Similarly, IP addresses should
not be picked because they cannot be redirected to another location like domain names
can, making them a higher risk option.

203
proposals/1708-well-known-for-federation.md
Unescape Escape View File

@ -1,203 +0,0 @@
# MSC1708: .well-known support for server name resolution
Currently, mapping from a server name to a hostname for federation is done via
`SRV` records. However,
[MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711) proposes
requiring valid X.509 certificates on the federation endpoint. It will then be
necessary for the homeserver to present a certificate which is valid for the
server name. This presents difficulties for hosted server offerings: BigCorp
may want to delegate responsibility for running its Matrix homeserver to an
outside supplier, but it may be difficult for that supplier to obtain a TLS
certificate for `bigcorp.com` (and BigCorp may be reluctant to let them have
one).
This MSC proposes to solve this problem by augmenting the current `SRV` record
with a `.well-known` lookup.
## Proposal
For reference, the current [specification for resolving server
names](https://github.com/matrix-org/matrix-doc/blob/6dab4b28f80f5beeb1d4f475ddc624cf9e7ad085/specification/server_server_api.rst#21resolving-server-names)
is as follows:
1. If the hostname is an IP literal, then that IP address should be used,
together with the given port number, or 8448 if no port is given.
2. Otherwise, if the port is present, then an IP address is discovered by
looking up an AAAA or A record for the hostname, and the specified port is
used.
3. If the hostname is not an IP literal and no port is given, the server is
discovered by first looking up a `_matrix._tcp` SRV record for the
hostname, which may give a hostname (to be looked up using AAAA or A queries)
and port.
4. Finally, the server is discovered by looking up an AAAA or A record on the
hostname, and taking the default fallback port number of 8448.
We insert the following between Steps 3 and 4.
If the SRV record does not exist, the requesting server should make a `GET`
request to `https://<server_name>/.well-known/matrix/server`, with normal X.509
certificate validation, and following 30x redirects (being careful to avoid
redirect loops). If the request does not return a 200, continue to step 4,
otherwise:
The response must be valid JSON which follows the structure documented
below. Otherwise, continue to the next step in the discovery process. It is
NOT necessary for the response to have a `Content-Type` of `application/json`.
If the response is valid, the `m.server` property is parsed as
`<delegated_server_name>[:<delegated_port>]`, and processed as follows:
* If `<delegated_server_name>` is an IP literal, then that IP address should be
used, together with `<delegated_port>`, or 8448 if no port is given. The
server should present a valid TLS certificate for `<delegated_server_name>`.
* If `<delegated_server_name>` is not an IP literal, and `<delegated_port>` is
present, then an IP address is discovered by looking up an AAAA or A record
for `<delegated_server_name>`, and the specified port is used. The server
should present a valid TLS certificate for `<delegated_server_name>`.
(In other words, the federation connection is made to
`https://<delegated_server_name>:<delegated_port>`).
* If the hostname is not an IP literal and no port is given, a second SRV
record is looked up; this time for `_matrix._tcp.<delegated_server_name>`,
which may give yet another hostname (to be looked up using A/AAAA queries)
and port. The server must present a TLS cert for the
`<delegated_server_name>` from the .well-known.
* If no SRV record is found, the server is discovered by looking up an AAAA
or A record on `<delegated_server_name>`, and taking the default fallback
port number of 8448.
(In other words, the federation connection is made to
`https://<delegated_server_name>:8448`).
### Structure of the `.well-known` response
The contents of the `.well-known` response should be structured as shown:
```json
{
"m.server": "<server>[:<port>]"
}
```
If the response cannot be parsed as JSON, or lacks a valid `m.server` property,
the request is considered to have failed, and no fallback to port 8448 takes
place.
The formal grammar for the `m.server` property is the same as that of a [server
name](https://matrix.org/docs/spec/appendices.html#server-name): it is a
hostname or IP address, followed by an optional port.
### Caching
Servers should not look up the `.well-known` file for every request, as this
would impose an unacceptable overhead on both sides. Instead, the results of
the `.well-known` request should be cached according to the HTTP response
headers, as per [RFC7234](https://tools.ietf.org/html/rfc7234). If the response
does not include an explicit expiry time, the requesting server should use a
sensible default: 24 hours is suggested.
Because there is no way to request a revalidation, it is also recommended that
requesting servers cap the expiry time. 48 hours is suggested.
A failure to retrieve the `.well-known` file should also be cached, though care
must be taken that a single 500 error or connection failure should not break
federation for an extended period. A short cache time of about an hour might be
appropriate; alternatively, servers might use an exponential backoff.
## Problems
It will take a while for `.well-known` to be supported across the ecosystem;
until it is, it will be difficult to deploy homeservers which rely on it for
their routing: if Alice is using a current homeserver implementation, and Bob
deploys a new implementation which relies on `.well-known` for routing, then
Alice will be unable to send messages to Bob. (This is the same problem we have with
[SNI](https://github.com/matrix-org/synapse/issues/1491#issuecomment-415153428).)
The main defence against this seems to be to release support for `.well-known`
as soon as possible, to maximise uptake in the ecosystem. It is likely that, as
we approach Matrix 1.0, there will be sufficient other new features (such as
new Room versions) that upgrading will be necessary anyway.
## Security considerations
The `.well-known` file potentially broadens the attack surface for an attacker
wishing to intercept federation traffic to a particular server.
## Dismissed alternatives
For future reference, here are the alternative solutions which have been
considered and dismissed.
### Look up the `.well-known` file before the SRV record
We could make the request for `.well-known` before looking up the `SRV`
record. On the one hand this is maybe marginally simpler (and avoids the
overhead of having to make *two* `SRV` lookups in the case that a `.well-known`
is found. It might also open a future path for using `.well-known` for
information other than delegation.
Ultimately we decided to include the initial `SRV` lookup so that deployments
have a mechanism to avoid the `.well-known` overhead in the common case that it
is not required.
### Subdomain hack
As well as accepting TLS certs for `example.com`, we could also accept them for
`delegated--matrix.example.com`. This would allow `example.com` to delegate its
matrix hosting by (a) setting up the SRV record at `_matrix._tcp.example.com`
and (b) setting up a CNAME at `delegated--matrix.example.com`. The latter would
enable the delegatee to obtain an acceptable TLS certificate.
This was certainly an interesting idea, but we dismissed it for the following
reasons:
* There's a security trap for anybody who lets people sign up for subdomains
(which is certainly not an uncommon business model): if you can register for
delegated--matrix.example.com, you get to intercept all the matrix traffic
for example.com.
* Generally it feels quite unintuitive and violates the principle of least
surprise.
* The fact that we can't find any prior art for this sets off alarm bells too.
### Rely on DNS/DNSSEC
If we could trust SRV records, we would be able to accept TLS certs for the
*target* of the SRV record, which avoids this whole problem.
Such trust could come from assuming that plain DNS is "good enough". However,
DNS cache poisoning attacks are a real thing, and the fact that the designers
of TLS chose to implement a server-name check specifically to deal with this
case suggests we would be foolish to make this assumption.
The alternative is to rely on DNSSEC to provide security for SRV records. The
problem here is simply that DNSSEC is not that widely deployed currently. A
number of large organisations are actively avoiding enabling it on their
domains, so requiring DNSSEC would be a direct impediment to the uptake of
Matrix. Furthermore, if we required DNSSEC-authenticated SRV records for
domains doing delegation, we would end up with a significant number of
homeservers unable to talk to such domains, because their local DNS
infrastructure may not implement DNSSEC.
Finally, if we're expecting servers to present the cert for the *target* of the
SRV record, then we'll have to change the Host and SNI fields, and that will
break backwards compat everywhere (and it's hard to see how to mitigate that).
### Stick with perspectives
The final option is to double-down on the Perspectives approach, ie to skip
[MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711). MSC1711
discusses the reasons we do not believe this to be a viable option.
## Conclusion
This proposal adds a new mechanism, alongside the existing `SRV` record lookup
for finding the server responsible for a particular matrix server_name, which
will allow greater flexibility in deploying homeservers.

230
proposals/1711-x509-for-federation.md
Unescape Escape View File

@ -1,230 +0,0 @@
# MSC1711: X.509 certificate verification for federation connections
TLS connections for server-to-server communication currently rely on an
approach borrowed from the [Perspectives
project](https://web.archive.org/web/20170702024706/https://perspectives-project.org/)
to provide certificate verification, rather than the more normal model using
certificates signed by trusted Certificate Authorities. This document sets out
the reasons that this has not been a success, and suggests that we should
instead revert to the CA model.
## Background: the failure of the Perspectives approach
The Perspectives approach replaces the conventional hierarchy of trust provided
by the Certificate Authority model with a large number of "notary" servers
distributed around the world. The intention is that the notary servers
regularly monitor remote servers and observe the certificates they present;
when making a connection to a new site, a client can correlate the certificate
it presents with that seen by the notary servers. In theory this makes it very
hard to mount a Man-in-the-Middle (MitM) attack, because it would require
intercepting traffic between the target server and a large number of the notary
servers.
It is notable that the Perspectives project itself appears to have largely been
abandoned: its website has largely been repurposed, the [Firefox
extension](https://addons.mozilla.org/en-GB/firefox/addon/perspectives/) does
not work with modern versions of Firefox, the [mailing
list](https://groups.google.com/forum/#!forum/perspectives-dev) is inactive,
and several of the (ten) published notary servers are no longer functional. The
reasons for this are not entirely clear, though clearly it never gained
widespread adoption.
When Matrix was originally designed in 2014, the Perspectives project was
heavily active, and avoiding dependencies on the relatively centralised
Certificate Authorities was attractive, in accordance with Matrix's design as a
decentralised protocol. However, this has not been a success in practice.
Matrix was unable to make use of the existing notary servers (largely because
we wanted to extend the protocol to include signing keys): the intention was
that, as the Matrix ecosystem grew, public Matrix servers would act as notary
servers. However, in practice we have ended up in a situation where almost <sup
id="a1">[1](#f1)</sup> every Matrix homeserver either uses `matrix.org` as the
sole notary, or does no certificate verification at all. Far from avoiding the
centralisation of the Certificate Authorities, the entire protocol is therefore
dependent on a single point of control at `matrix.org` - and because
`matrix.org` only monitors from a single location, the protection against MitM
attacks is weak.
It is also clear that the Perspectives approach is poorly-understood. It is a
common error for homeservers to be deployed behind reverse-proxies which make
the Perspectives-based approach unreliable. The CA model, for all its flaws, is
at least commonly used, which makes it easier for administrators to deploy
(secure) homeservers, and allows server implementations to leverage existing
libraries.
## Proposal
We propose that Matrix homeservers should be required to present valid TLS
certificates, signed by a known Certificate Authority, on their federation
port.
In order to ease transition and give administrators time to switch to a signed
certificate, we will continue to follow the current, perspectives-based
approach for servers whose TLS certificates fail validation.
However, this fallback will be strictly time-limited, and Matrix S2S spec r0
will not accept self-signed certificates, nor will it include the
`tls_fingerprints` property of the
[`/_matrix/key/v2`](https://github.com/matrix-org/matrix-doc/blob/6dab4b28f80f5beeb1d4f475ddc624cf9e7ad085/specification/server_server_api.rst#23retrieving-server-keys)
endpoints. Synapse 1.0 will not accept self-signed certificates by default.
The `matrix.org` team will proactively attempt to reach out to homeserver
administrators who do not update their certificates in the coming weeks.
The process of determining which CAs are trusted to sign certificates would be
implementation-specific, though it should almost certainly make use of existing
operating-system support for maintaining such lists. It might also be useful if
administrators could override this list, for the purpose of setting up a
private federation using their own CA.
It would also be useful for administrators to be able to disable the
certificate checks for a whitelist of domains/netmasks. This would be useful
for testing, or for networks that provide server verification themselves,
such as like `.onion` domains on Tor or `fc00::/8` IPs on cjdns.
### Interaction with SRV records
With the use of `SRV` records, it is possible for the hostname of a homeserver
to be quite different from the matrix domain it is hosting. For example, if
there were an SRV record at `_matrix._tcp.matrix.org` which pointed to
`server.example.com`, then any federation requests for `matrix.org` would be
routed to `server.example.com`. The question arises as to which certificate
`server.example.com` should present.
In short: the server should present a certificate for the matrix domain
(`matrix.org` in the above example). This ensures that traffic cannot be
intercepted by a MitM who can control the DNS response for the `SRV` record
(perhaps via cache-poisoning or falsifying DNS responses).
This will be in line with the current
[requirements](https://github.com/matrix-org/matrix-doc/blob/6dab4b28f80f5beeb1d4f475ddc624cf9e7ad085/specification/server_server_api.rst#21resolving-server-names)
in the Federation API specification for the `Host`, and by implication, the TLS
Server Name Indication <sup id="a2">[2](#f2)</sup>. It is also consistent with
the recommendations of
[RFC6125](https://tools.ietf.org/html/rfc6125#section-6.2.1) and the
conventions established by the XMPP protocol (per [RFC6120](https://tools.ietf.org/html/rfc6120#section-13.7.2.1).
### Extensions
HTTP-Based Public Key Pinning (HPKP) and
[Certificate transparency](https://www.certificate-transparency.org) are
both HTTP extensions which attempt to work around some of the deficiencies in
the CA model, by making it more obvious if a CA has issued a certificate
incorrectly.
HPKP has not been particularly successful, and is
[deprecated](https://developers.google.com/web/updates/2018/04/chrome-67-deps-rems#deprecate_http-based_public_key_pinning)
in Google Chrome as of April 2018. Certificate transparency, however, is seeing
widespread adoption from Certificate Authories and HTTP clients.
This proposal sees both technologies as optional techniques which could be
provided by homeserver implementations. We encourage but do not mandate the use
of Certificate Transparency.
### Related work
The Perspectives approach is also currently used for exchanging the keys that
are used by homeservers to sign Matrix events and federation requests (the
"signing keys"). Problems similar to those covered here also apply to that
mechanism. This is discussed at [#1685](thttps://github.com/matrix-org/matrix-doc/issues/1685).
## Alternatives
There are well-known problems with the CA model, including a number of
widely-published incidents in which CAs have issued certificates
incorrectly. It is therefore important to consider alternatives to the CA
model.
### Improving support for the Perspectives model
In principle, we could double-down on the Perspectives approach, and make an effort
to get servers other than `matrix.org` used as notary servers. However, there
remain significant problems with such an approach:
* Perspectives remain complex to configure correctly. Ideally, administrators
need to make conscious choices about which notaries to trust, which is hard
to do, especially for newcomers to the ecosystem. (In practice, people use
the out-of-the-box configuration, which is why everyone just uses
`matrix.org` today).
* A *correct* implementation of Perspectives really needs to take into account
more than the latest state seen by the notary servers: some level of history
should be taken into account too.
Essentially, whilst we still believe the Perspectives approach has some merit,
we believe it needs further research before it can be relied upon. We believe
that the resources of the Matrix ecosystem are better spent elsewhere.
### DANE
DNS-Based Authentication of Named Entities (DANE) can be used as an alternative
to the CA model. (It is arguably more appropriately used *together* with the CA
model.)
It is not obvious to the author of this proposal that DANE provides any
material advantages over the CA model. In particular it replaces the
centralised trust of the CAs with the centralised trust of the DNS registries.
## Potential issues
Beyond the problems already discussed with the CA model, requiring signed
certificates comes with a number of downsides.
### More difficult setup
Configuring a working, federating homeserver is a process fraught with
pitfalls. This proposal adds the requirement to obtain a signed certificate to
that process. Even with modern intiatives such as Let's Encrypt, this is
another procedure requiring manual intervention across several moving parts.
On the other hand: obtaining an SSL certificate should be a familiar process to
anybody capable of hosting a production homeserver (indeed, they should
probably already have one for the client port). This change also opens the
possibility of putting the federation port behind a reverse-proxy without the
need for additional configuration. Hopefully making the certificate usage more
conventional will offset the overhead of setting up a certificate.
Furthermore, homeserver implementations could provide an implementation of the
ACME protocol and integration with Let's Encrypt, to make it easier for
administrators to get started. (This would of course be
implementation-specific, and administrators who wanted to keep control of the
certificate creation process would be free to do so).
### Inferior support for IP literals
Whilst it is possible to obtain an SSL cert which is valid for a literal IP
address, this typically requires purchase of a premium certificate; in
particular, Let's Encrypt will not issue certificates for IP literals. This may
make it impractical to run a homeserver which uses an IP literal, rather than a
DNS name, as its `server_name`.
It has long been the view of the `matrix.org` administrators that IP literals
are only really suitable for internal testing. Those who wish to use them for
that purpose could either disable certificate checks inside their network, or
use their own CA to issue certificates.
### Inferior support for hidden services (`.onion` addresses)
It is currently possible to correctly route traffic to a homeserver on a
`.onion` domain, provided any remote servers which may need to reach that
server are configured to route to such addresses via the Tor network. However,
it can be difficult to get a certificate for a `.onion` domain (again, Let's
Encrypt do not support them).
The reasons for requiring a signed certificate (or indeed, for using TLS at
all) are weakened when traffic is routed via the Tor network. Administrators
using the Tor network could disable certificate checks for `.onion` addresses.
## Conclusion
We believe that requiring homeservers to present an X.509 certificate signed by
a recognised Certificate Authority will improve security, reduce
centralisation, and eliminate some common deployment pitfalls.
<a id="f1"/>[1] It's *possible* to set up homeservers to use servers other than
`matrix.org` as notaries, but only a minority are actually set up this
way. [↩](#a1)
<a id="f2"/>[2] I've not been able to find an authoritative source on this, but
most reverse-proxies will reject requests where the SNI and Host headers do not
match. [↩](#a2)

167
proposals/1717-key_verification.md
Unescape Escape View File

@ -1,167 +0,0 @@
# Key verification mechanisms
Key verification is an essential part of ensuring that end-to-end encrypted
messages are secure. Matrix may support multiple verification methods that
require sending events; in fact, two such methods (such as [MSC
1267](https://github.com/matrix-org/matrix-doc/issues/1267) and [MSC
1543](https://github.com/matrix-org/matrix-doc/issues/1543)) have already been
proposed.
This proposal tries to present a common framework for verification methods to
use, and presents a way to request key verification.
## Proposal
Each key verification method is identified by a name. Verification method
names defined in the Matrix spec will begin with `m.`, and verification method
names that are not defined in the Matrix spec must be namespaced following the
Java package naming convention.
If Alice wants to verify keys with Bob, Alice's device may send `to_device`
events to Bob's devices with the `type` set to `m.key.verification.request`, as
described below. The `m.key.verification.request` messages should all have the
same `transaction_id`, and are considered to be a single request. Thus, for
example, if Bob rejects the request on one device, then the entire request
should be considered as rejected across all of his devices. Similarly, if Bob
accepts the request on one device, that device is now in charge of completing
the key verification, and Bob's other devices no longer need to be involved.
The `m.key.verification.request` event lists the verification methods that
Alice's device supports, and upon receipt of this message, Bob's client should
prompt him to verify keys with Alice using one of the applicable methods. In
order to avoid displaying stale key verification prompts, if Bob does not
interact with the prompt, it should be automatically hidden 10 minutes after
the message is sent (according to the `timestamp` field), or 2 minutes after
the client receives the message, whichever comes first. The prompt should also
be hidden if an appropriate `m.key.verification.cancel` message is received.
If Bob chooses to reject the key verification request, Bob's client should send
a `m.key.verification.cancel` message to Alice's device. This indicates to
Alice that Bob does not wish to verify keys with her. In this case, Alice's
device should send an `m.key.verification.cancel` message to all of Bob's
devices to notify them that the request has been rejected.
If one of Bob's clients does not understand any of the methods offered, it
should display a message to Bob saying so. However, it should not send a
`m.key.verification.cancel` message to Alice's device unless Bob chooses to
reject the verification request, as Bob may have another device that is capable
of verifying using one of the given methods.
To initiate a key verification process, Bob's device sends a `to_device` event
to one of Alice's devices with the `type` set to `m.key.verification.start`.
This may either be done in response to an `m.key.verification.request` message,
or can be done independently. If it is done in response to an
`m.key.verification.request` message, it should use the same `transaction_id`
as the `m.key.verification.request` message. If Alice's device receives an
`m.key.verification.start` message in response to an
`m.key.verification.request` message, it should send an
`m.key.verification.cancel` message to Bob's other devices that it had
originally sent an `m.key.verification.request` to, in order to cancel the key
verification request.
Verification methods can define other events required to complete the
verification. Event types for verification methods defined in the Matrix spec
should be in the `m.key.verification` namespace. Event types that are not
defined in the Matrix spec must be namespaced following the Java package naming
convention.
Alice's or Bob's devices can cancel a key verification process or a key
verification request by sending a `to_device` event with `type` set to
`m.key.verification.cancel`.
### Event Definitions
#### `m.key.verification.request`
Requests a key verification.
Properties:
- `from_device` (string): Required. The device ID of the device requesting
verification.
- `transaction_id` (string): Required. An identifier for the verification
request. Must be unique with respect to the pair of devices.
- `methods` ([string]): Required. The verification methods supported by the
sender.
- `timestamp` (integer): Required. The time when the request was made. If the
timestamp is in the future (by more than 5 minutes, to allow for clock skew),
or more than 10 minutes in the past, then the message must be ignored.
#### `m.key.verification.start`
Begins a key verification process.
Properties:
- `method` (string): Required. The verification method to use.
- `from_device` (string): Required. The device ID of the device starting the
verification process.
- `transaction_id` (string): Required. An identifier for the verification
process. If this message is sent in response to an
`m.key.verification.request` event, then it must use the same
`transaction_id` as the one given in the `m.key.verification.request`.
- `next_method` (string): Optional. If the selected verification method only
verifies one user's key, then this property can be used to indicate the
method to use to verify the other user's key, which will be started
immediately after after the current key verification is complete.
Key verification methods can define additional properties to be included.
#### `m.key.verification.cancel`
Cancels a key verification process or a key verification request. Upon
receiving an `m.key.verification.cancel` message, the receiving device must
cancel the verification or the request. If it is a verification process that
is cancelled, or a verification request initiated by the recipient of the
cancellation message, the device should inform the user of the reason.
Properties:
- `transaction_id` (string): the identifier for the request or key verification
to cancel.
- `code` (string): machine-readable reason for cancelling. Possible reasons
are:
- `m.user`: the user cancelled the verification.
- `m.timeout`: the verification process has timed out. Different verification
methods may define their own timeouts.
- `m.unknown_transaction`: the device does not know about the given transaction
ID.
- `m.unknown_method`: the device does not know how to handle the given method.
This can be sent in response to an `m.key.verification.start` message, or
can be sent in response to other verification method-specific messages.
- `m.unexpected_message`: the device received an unexpected message. For
example, a message for a verification method may have been received when it
was not expected.
- `m.key_mismatch`: the key was not verified.
- `m.user_mismatch`: the expected user did not match the user verified.
- `m.invalid_message`: an invalid message was received.
- `m.accepted`: when an `m.key.verification.request` is accepted by one
device, an `m.key.verification.cancel` message with `code` set to
`m.accepted` is sent to the other devices
- `reason` (string): human-readable reason for cancelling. This should only be
used if the receiving client does not understand the code given in the `code`
property.
Verification methods may define their own additional cancellation codes.
Cancellation codes defined in the Matrix spec will begin with `m.`; other
cancellation codes must be namespaced following the Java package naming
convention.
## Tradeoffs
Rather than broadcasting verification requests to Bob's devices, Alice could
simply send an `m.key.verification.start` request to a single device. However,
this would require Alice to choose the right device to send to, which may be
hard for Alice to do if, for example, Bob has many devices, or if his devices
have similar names.
## Security considerations
An attacker could try to spam a user with verification requests. Clients
should take care that such requests do not interfere with a user's use of the
client.
## Conclusion
This proposal presents common event definitions for use by key verification
methods and defines a way for users to request key verification.

59
proposals/1719-olm_unwedging.md
Unescape Escape View File

@ -1,59 +0,0 @@
# Olm unwedging
Olm sessions sometimes get out of sync, resulting in undecryptable messages.
This can happen for several reasons. For example, if a user restores their
client state from a backup, the client will be using an old ratchet state
([riot-web#3822](https://github.com/vector-im/riot-web/issues/3822)). Or a
client might expire a one-time key that another client is trying to use
([riot-web#3309](https://github.com/vector-im/riot-web/issues/3309)). This
proposal documents a method for devices to create a new session to replace the
broken session.
## Proposal
When a device receives an olm-encrypted message that it cannot decrypt, it
should assume that the olm session has become corrupted and create a new olm
session to replace it. It should then send a dummy message, using that
session, to the other party in order to inform them of the new session. To
send a dummy message, clients may send an event with type `m.dummy`, and with
empty contents.
In order to avoid creating too many extra sessions, a client should rate-limit
the number of new sessions it creates per device that it receives a message
from; the client should not create a new session with another device if it has
already created one for that given device in the past 1 hour.
Clients may wish to take steps to mitigate the loss of the undecryptable
messages. For example, megolm sessions that were sent using the old session
would have been lost, so the client can send
[`m.room_key_request`](https://matrix.org/docs/spec/client_server/r0.6.1.html#m-room-key-request)
messages to re-request any megolm sessions that it is unable to decrypt.
The spec currently says, "If a client has multiple sessions established with
another device, it should use the session from which it last received a
message." (the last paragraph of the [`m.olm.v1.curve25519-aes-sha2`
section](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-olm-v1-curve25519-aes-sha2)).
When comparing the time of the last received message for each session, the
client should only consider messages that were successfully decrypted,
and for sessions that have never received a message, it should use the creation
time of the session. The spec will be changed to read:
> If a client has multiple sessions established with another device, it should
> use the session from which it last received and successfully decrypted a
> message. For these purposes, a session that has not received any messages
> should use its creation time as the time that it last received a message.
## Tradeoffs
## Potential issues
## Security considerations
An attacker could use this to create a new session on a device that they are
able to read. However, this would require the attacker to have compromised the
device's keys.
## Conclusion
This proposal outlines how wedged olm sessions can be replaced by a new
session.

32
proposals/1721-rename-cas-to-sso.md
Unescape Escape View File

@ -1,32 +0,0 @@
# MSC1721: Rename `m.login.cas` to `m.login.sso`
The Matrix Client-Server spec includes a [section on client login using Central
Authentication Service
(CAS)](https://matrix.org/docs/spec/client_server/r0.4.0.html#cas-based-client-login).
The spec currently fails to mention it, but this process is triggered when [`GET
/login`](https://matrix.org/docs/spec/client_server/r0.4.0.html#get-matrix-client-r0-login)
returns a flow type of `m.login.cas`.
Nothing in this flow is specific to CAS - it is equally applicable for other
web-based single-sign-on processes, such as SAML.
Accordingly, we should rename `cas` to `sso`.
## Proposal
1. `m.login.sso` should be defined as a valid login type for return from `GET
/login`. (We should probably mention `m.login.cas` in the spec while we are
there.)
2. When a client wishes to use the SSO login type, it should redirect to
`/_matrix/client/r0/login/sso/redirect` (instead of
`/_matrix/client/r0/login/cas/redirect`).
3. Servers should treat `/_matrix/client/r0/login/sso/redirect` identically to
`/_matrix/client/r0/login/cas/redirect`: they should issue a redirect to
their configured single-sign-on system.
4. Servers which support `m.login.sso` should make sure they update their [login
fallback page](https://matrix.org/docs/spec/client_server/r0.4.0.html#login-fallback)
to understand the new login type.

158
proposals/1730-cs-api-in-login-response.md
Unescape Escape View File

@ -1,158 +0,0 @@
# MSC1730: Mechanism for redirecting to an alternative server during login
## Background/requirements
This is a proposal for a mechanism for handling the following situation.
A large, loosely-coupled organisation wants its members to be able to
communicate with one another via Matrix. The organisation consists of several
departments which are cooperative but prefer to host their own infrastructure.
The organisation has an existing single-sign-on system which covers the entire
organisation, and which they would like their members to use when
authenticating to the Matrix system.
## Proposal
The response to `POST /_matrix/client/r0/login` currently includes the fields
`user_id`, `access_token`, `device_id`, and the deprecated `home_server`.
We will add to this the the field `well_known`, which has the same format as
the [`/.well-known/matrix/client`
object](https://matrix.org/docs/spec/client_server/r0.4.0.html#get-well-known-matrix-client).
Servers MAY add this field to the login response if they wish to redirect
clients to an alternative homeserver after login. Clients SHOULD use the
provided `well_known` object to reconfigure themselves, optionally validating the
URLs within.
Note: a server that redirects all clients to different servers must nonetheless
consider clients making requests other than `/login`: for example, some clients
may fail to support redirection. It is acceptable in such a case to return a
401 response to all non-`/login` requests if the service does not wish to
support such clients.
## Application
Let's imagine for this description that our organisation is the University of
Canadialand, which is divided into departments including Engineering, History,
Physics, and so on.
Central University IT currently host a SAML2-based single-sign-on system, which
asks users to select their department, and then defers to the departmental
authentication system to authenticate them. Note that the users do not have a
globally-unique identifier.
University IT now sets up a Matrix Homeserver instance, which they host at
`https://matrix.ac.cdl`. They run a publicity campaign encouraging university
members to use the service by configuring off-the-shelf Matrix clients to use
the homeserver at `https://matrix.ac.cdl`. They may also release customised
clients configured to use that URL by default.
However, the departments actually want to host their own homeservers; these
might be at `https://matrix.eng.ac.cdl`, `https://matrix.hist.ac.cdl`, etc. The
central IT homeserver therefore redirects clients to the departmental
homeserver after login.
A complete login flow is as shown in the following sequence diagram:
![Sequence diagram](images/1730-seq-diagram.1.svg)
Note that this flow is complicated by the out-of-band SAML2 authentication. We
envisage that a similar technique could also be used for a standard
username/password authentication, however.
## Rejected solutions
Alternative solutions might include:
### Have all users on one homeserver
In many situations, it might be more appropriate to have a single homeserver,
so users' MXids would look like `@user:ac.cdl` instead of
`@user:eng.ac.cdl`.
However, there are circumstances where separate homeservers are required:
* the departments may be only very loosely related
* the departments may have privacy concerns
* the dpeartments may be geographically distributed with slow or unreliable
links to the central system
* load-balancing may be essential.
### Tell users the C-S API for their home homeserver
We could tell Engineering users to configure their clients with
`https://matrix.eng.ac.cdl`, History users to use `https://matrix.hist.ac.cdl`,
etc.
The problems with this are:
* Each department must issue its own documentation and publicity advising how
to configure a Matrix client
* It becomes impractical to distribute preconfigured clients.
### Proxy all C-S endpoints
It would be possible for the the central homeserver to proxy all C-S
interaction, as well as `/login`, directing requests to the right server for
the user.
This is unsatisfactory due to the additional latency imposed, the load on the
central homeserver, and the fact that it makes the central server a single
point of failure for the entire system.
### Require clients to perform a .well-known lookup after login
We could require clients to do a .well-known lookup on the domain of their MXID
once they have discovered it from the `/login` response.
This has the following problems:
* In most cases this `.well-known` lookup will be entirely redundant. It adds
latency and overhead, and complicates client implementations.
* It complicates deployment, since each department has to host a `.well-known`
file at their root domain.
### Add an alternative redirection mechanism in the login flow
We could specify that the `/login` response could contain a `redirect` field
property instead of the existing `user_id`/`access_token`/`device_id`
properties. The `redirect` property would give the C-S API of the target
HS. The client would then repeat its `/login` request, and use the specified
endpoint for all future C-S interaction.
This approach would complicate client implementations.
### Modify the single-sign-on flow
It would be possible to modify the single-sign-on flow to allow an alternative
homeserver to be specified for the final `m.login.token`-based call to
`/login` (and subsequent C-S API calls).
This is discussed in more detail in
[MSC1731](https://github.com/matrix-org/matrix-doc/blob/rav/proposals/homeserver_in_sso_login/proposals/1731-redirect-in-sso-login.md).
It has the disadvantage of limiting the solution to SSO logins. The solution
presented in this proposal also extends to password-based logins.
### Use a 3pid login flow
It has been suggested that we could use a login flow based on third-party
identifiers.
In the current ecosystem, to do a 3pid login, clients must still be configured
to send their `/login` request to a particular homeserver, which will then take
them through an authentication process. We are therefore still left with the
problem that we need to switch homeservers between login and initial sync.
An alternative would be for clients to somehow know that they should go through
the single-sign-on process *before* choosing a homeserver, and for the
output of the single-sign-on process to indicate the homeserver to use. This
would require either substantially customised Matrix clients, or substantial
modifications to the login flow in Matrix, possibly involving authenticating
against an identity server. The latter is something which could be considered,
but the scope of the changes required make it impractical in the short/medium
term.

119
proposals/1753-capabilities.md
Unescape Escape View File

@ -1,119 +0,0 @@
# MSC1753: client-server capabilities API
A mechanism is needed for clients to interrogate servers to establish whether
particular operations can be performed.
For example, users may not be able to change their password if a server is
configured to authenticate against a separate system, in which case it is
nonsensical to offer the user such an option.
## Proposal
### `GET /_matrix/client/r0/capabilities`
We will add a new endpoint to the client-server API: `GET
/_matrix/client/r0/capabilities`. The endpoint will be authenticated as normal
via an access token.
The server should reply with a list of supported features, as shown:
```json
{
"capabilities": {
"m.capability_one": {}
}
}
```
The keys of the `capabilities` object are capability identifiers. As with
other identifiers in the Matrix protocol, the `m.` prefix is reserved for
definition in the Matrix specification; other values can be used within an
organisation following the Java package naming conventions.
The values of the `capabilities` object will depend on the capability
identifier, though in general the empty object will suffice.
### Initial capability identifiers
As a starting point, a single capability identifier is proposed:
`m.change_password`, which should be considered supported if it is possible to
change the user's password via the `POST /_matrix/client/r0/account/password`
API.
The value of the `capabilities` object in the response should contain a single
boolean flag, `enabled`, to indicate whether a password change is possible. If
the capability is not listed, the client should assume that password changes
are possible.
### Fallback behaviour
Clients will need to be aware of servers which do not support the new endpoint,
and fall back to their current behaviour if they receive a 404 response.
### Suitable applications
In general, capabilities advertised via this endpoint should depend in some way
on the state of the user or server - in other words, they will be inherently
"optional" features in the API.
This endpoint should *not* be used to advertise support for experimental or
unstable features, which is better done via `/client/versions` (see
[MSC1497](https://github.com/matrix-org/matrix-doc/issues/1497)).
Examples of features which might reasonably be advertised here include:
* Whether the server supports user presence.
* Whether the server supports other optional features. The following could be
made optional via this mechanism:
* Room directory
* URL previews
* Policy restricitions, such as:
* Whether certain types of content are permitted on this server.
* The number of rooms you are allowed in.
* Configured ratelimits.
Features which might be better advertised elsewhere include:
* Support for e2e key backups
([MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219)) - list in
`/client/versions`.
* Support for lazy-loading of room members - list in `/client/versions`.
* Media size limits - list in `/media/r0/config`, because the media server may
be a separate process.
* Optional transports/encodings for the CS API - probably better handled via
HTTP headers etc.
* Variations in room state resolution - this is implied via the room version
(which is in the `m.room.create` event).
## Tradeoffs
One alternative would be to provide specific ways of establishing support for
each operation: for example, a client might send an `GET
/_matrix/client/r0/account/password` request to see if the user can change
their password. The concern with this approach is that this could require a
large number of requests to establish which entries should appear on a menu or
dialog box.
Another alternative is to provide a generic query mechanism where the client
can query for specific capabilities it is interested in. However, this adds
complication and makes it harder to discover capability identifiers.
## Potential issues
None yet identified.
## Security considerations
None yet identified.
## Conclusion
We propose adding a new endpoint to the Client-Server API, which will allow
clients to query for supported operations so that they can decide whether to
expose them in their user-interface.

567
proposals/1756-cross-signing.md
Unescape Escape View File

@ -1,567 +0,0 @@
# Cross-signing devices with device signing keys
## Background
If a user has multiple devices, each device will have a different key for
end-to-end encryption. Other users who want to communicate securely with this
user must then verify each key on each of their own devices. If Alice has *n*
devices, and Bob has *m* devices, then for Alice to be able to communicate with
Bob on any of their devices, this involves *n×m* key verifications.
One way to address this is for each user to use a device signing key to sign
all of their devices. Thus another user who wishes to verify their identity
only needs to verify the device signing key and can use the signatures created
by the device signing key to verify their devices.
[MSC1680](https://github.com/matrix-org/matrix-doc/issues/1680) presents a
different solution to the problem. A comparison between this proposal and
MSC1680 is presented below.
## Proposal
Each user has three key pairs:
- a *master* cross-signing key pair that is used to identify themselves and to
sign their other cross-signing keys,
- a *self-signing* key pair that is used to sign their own devices, and
- a *user-signing* key pair that is used to sign other users' master keys.
When one user (e.g. Alice) verifies another user's (Bob's) identity, Alice will
sign Bob's master key with her user-signing key. (This will mean that
verification methods will need to be modified to pass along the public part of
Bob's master key.) Alice's device will trust Bob's device if:
- Alice's device is using a master key that has signed her user-signing key,
- Alice's user-signing key has signed Bob's master key,
- Bob's master key has signed Bob's self-signing key, and
- Bob's self-signing key has signed Bob's device key.
### Key security
A user's master key could allow an attacker to impersonate that user to other
users, or other users to that user. Thus clients must ensure that the private
part of the master key is treated securely. If clients do not have a secure
means of storing the master key (such as a secret storage system provided by
the operating system), then clients must not store the private part. If a user
changes their master key, clients of users that they communicate with must
notify their users about the change.
A user's user-signing and self-signing keys are intended to be easily
replaceable if they are compromised by re-issuing a new key signed by the
user's master key and possibly by re-verifying devices or users. However,
doing so relies on the user being able to notice when their keys have been
compromised, and it involves extra work for the user, and so although clients
do not have to treat the private parts as sensitively as the master key,
clients should still make efforts to store the private part securely, or not
store it at all. Clients will need to balance the security of the keys with
the usability of signing users and devices when performing key verification.
The private halves of a user's cross-signing keys may be stored encrypted on the
server so that they may be retrieved by new devices, or shared between devices
using [MSC1946](https://github.com/matrix-org/matrix-doc/pull/1946). When
handled in this way, the keys must be base64-encoded, and use the names
`m.cross_signing.master`, `m.cross_signing.self_signing`, and
`m.cross_signing.user_signing` for the master, self-signing, and user-signing
keys, respectively.
### Signature distribution
Currently, users will only be allowed to see
* signatures made by their own master, self-signing or user-signing keys,
* signatures made by their own devices about their own master key,
* signatures made by other users' self-signing keys about their own respective
devices,
* signatures made by other users' master keys about their respective
self-signing key, or
* signatures made by other users' devices about their respective master keys
(these signatures are used for [migrating from device
verifications](#migrating-from-device-verifications)).
This is done in order to preserve the privacy of social connections. Future
proposals may define mechanisms for distributing signatures to other users in
order to allow for other web-of-trust use cases.
### Migrating from device verifications
Users who have verified individual devices may wish to migrate these
verifications to use cross-signing instead. In order to aid with this,
signatures of a user's master key, made by their own devices, may be uploaded
to the server. If another user's client sees that that a given user's master key has a valid
signature from a device that was previously verified, then the client may
choose to trust and sign the master key. The client should take precautions to
ensure that a stolen device cannot be used to cause it to trust a malicious
master key. For example, a client could prompt the user before signing the
master key, or it could only do this migration on the first master key that it
sees from a user.
### API description
#### Uploading signing keys
Public keys for the cross-signing keys are uploaded to the servers using
`/keys/device_signing/upload`. This endpoint requires [UI
Auth](https://matrix.org/docs/spec/client_server/r0.4.0.html#user-interactive-authentication-api).
`POST /keys/device_signing/upload`
``` json
{
"master_key": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+self+master+key",
}
},
"self_signing_key": {
"user_id": "@alice:example.com",
"usage": ["self_signing"],
"keys": {
"ed25519:base64+self+signing+public+key": "base64+self+signing+public+key",
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
},
"user_signing_key": {
"user_id": "@alice:example.com",
"keys": {
"ed25519:base64+device+signing+public+key": "base64+device+signing+public+key",
},
"usage": ["user_signing"],
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
}
}
```
Cross-signing keys are JSON objects with the following properties:
* `user_id` (string): The user who owns the key
* `usage` ([string]): Allowed uses for the key. Must contain `"master"` for
master keys, `"self_signing"` for self-signing keys, and `"user_signing"`
for user-signing keys.
* `keys` ({string: string}): an object that must have one entry, whose name is
"`ed25519:`" followed by the unpadded base64 encoding of the public key, and
whose value is the unpadded base64 encoding of the public key.
* `signatures` ({string: {string: string}}): signatures of the key. A
self-signing or user-signing key must be signed by the master key. A master
key may be signed by a device.
In order to ensure that there will be no collisions in the `signatures`
property, the server must respond with an `M_FORBIDDEN` error if any of
the uploaded public keys match an existing device ID for the user. Similarly,
if a user attempts to log in specifying a device ID matching one of the signing
keys, the server must respond with an `M_FORBIDDEN` error.
If a self-signing or user-signing key is uploaded, it must be signed by the
master key that is included in the request, or the current master key if no
master key is included. If the signature from the master key is incorrect, the
server should respond with an error code of `M_INVALID_SIGNATURE`.
After uploading cross-signing keys, they will be included under the
`/keys/query` endpoint under the `master_keys`, `self_signing_keys` and
`user_signing_keys` properties. The `user_signing_keys` property will only be
included when a user requests their own keys.
`POST /keys/query`
``` json
{
"device_keys": {
"@alice:example.com": []
},
"token": "string"
}
```
response:
``` json
{
"failures": {},
"device_keys": {
"@alice:example.com": {
// ...
}
},
"master_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+master+public+key"
}
}
},
"self_signing_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["self_signing"],
"keys": {
"ed25519:base64+self+signing+public+key": "base64+self+signing+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
}
}
}
```
Similarly, the federation endpoints `POST /user/keys/query` and `GET
/user/devices/{userId}` will include the master and self-signing keys. (It
will not include the user-signing key because it is not intended to be visible
to other users.)
`POST /user/keys/query`
``` json
{
"device_keys": {
"@alice:example.com": []
}
}
```
response:
``` json
{
"device_keys": {
"@alice:example.com": {
// ...
}
},
"master_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+master+public+key"
}
}
},
"self_signing_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["self_signing"],
"keys": {
"ed25519:base64+self+signing+public+key": "base64+self+signing+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
}
}
}
```
`GET /user/devices/%40alice%3Aexample.com`
response:
``` json
{
"user_id": "@alice:example.com",
"stream_id": 5,
"devices": [
// ...
],
"master_key": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+master+public+key"
}
},
"self_signing_key": {
"user_id": "@alice:example.com",
"usage": ["self_signing"],
"keys": {
"ed25519:base64+self+signing+public+key": "base64+self+signing+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
}
}
```
In addition, Alice's homeserver will send a `m.signing_key_update` EDU to
servers that have users who share encrypted rooms with Alice. The `content` of
that EDU has the following properties:
* `user_id` (string): Required. The user ID who owns the signing key
* `master_key` (object): The master key, as above.
* `self_signing_key` (object): The self-signing key, as above.
After uploading self-signing and user-signing keys, the user will show up in
the `changed` property of the `device_lists` field of the sync result of any
others users who share an encrypted room with that user.
#### Uploading signatures
Signatures of device keys can be uploaded using `/keys/signatures/upload`.
For example, Alice signs one of her devices (HIJKLMN) (using her self-signing
key), her own master key (using her HIJKLMN device), Bob's master key (using
her user-signing key).
`POST /keys/signatures/upload`
``` json
{
"@alice:example.com": {
"HIJKLMN": {
"user_id": "@alice:example.com",
"device_id": "HIJKLMN",
"algorithms": [
"m.olm.curve25519-aes-sha256",
"m.megolm.v1.aes-sha"
],
"keys": {
"curve25519:HIJKLMN": "base64+curve25519+key",
"ed25519:HIJKLMN": "base64+ed25519+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+self+signing+public+key": "base64+signature+of+HIJKLMN"
}
}
},
"base64+master+public+key": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+master+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:HIJKLMN": "base64+signature+of+master+key"
}
}
}
},
"@bob:example.com": {
"bobs+base64+self+signing+public+key": {
"user_id": "@bob:example.com",
"keys": {
"ed25519:bobs+base64+master+public+key": "bobs+base64+master+public+key"
},
"usage": ["master"],
"signatures": {
"@alice:example.com": {
"ed25519:base64+user+signing+public+key": "base64+signature+of+bobs+master+key"
}
}
}
}
}
```
response:
``` json
{
"failures": {}
}
```
The response contains a `failures` property, which is a map of user ID to
device ID to failure reason, if any of the uploaded keys failed. The
homeserver should verify that the signatures on the uploaded keys are valid.
If a signature is not valid, the homeserver should set the corresponding entry
in `failures` to a JSON object with the `errcode` property set to
`M_INVALID_SIGNATURE`.
After Alice uploads a signature for her own devices or master key, her
signature will be included in the results of the `/keys/query` request when
*anyone* requests her keys. However, signatures made for other users' keys,
made by her user-signing key, will not be included.
`POST /keys/query`
``` json
{
"device_keys": {
"@alice:example.com": []
},
"token": "string"
}
```
response:
``` json
{
"failures": {},
"device_keys": {
"@alice:example.com": {
"HIJKLMN": {
"user_id": "@alice:example.com",
"device_id": "HIJKLMN",
"algorithms": [
"m.olm.v1.curve25519-aes-sha256",
"m.megolm.v1.aes-sha"
],
"keys": {
"curve25519:HIJKLMN": "base64+curve25519+key",
"ed25519:HIJKLMN": "base64+ed25519+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:HIJKLMN": "base64+self+signature",
"ed25519:base64+self+signing+public+key": "base64+signature+of+HIJKLMN"
}
},
"unsigned": {
"device_display_name": "Alice's Osborne 2"
}
}
}
},
"master_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["master"],
"keys": {
"ed25519:base64+master+public+key": "base64+master+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:HIJKLMN": "base64+signature+of+master+key"
}
}
}
},
"self_signing_keys": {
"@alice:example.com": {
"user_id": "@alice:example.com",
"usage": ["self_signing"],
"keys": {
"ed25519:base64+self+signing+public+key": "base64+self+signing+public+key"
},
"signatures": {
"@alice:example.com": {
"ed25519:base64+master+public+key": "base64+signature"
}
}
}
}
}
```
Similarly, the federation endpoints `POST /user/keys/query` and `GET
/user/devices/{userId}` will include the new signatures for her own devices or
master key, but not signatures made by her user-signing key.
In addition, when Alice uploads signatures for her own device, Alice's server
will send an `m.device_list_update` EDU to servers that have users who share
encrypted rooms with Alice, updating her device to include her new signature.
And when a signature of a master key is uploaded, Alice's server will send an
`m.signing_key_update` EDU, updating her master key to include her new
signature.
After Alice uploads a signature for Bob's user-signing key, her signature will
be included in the results of the `/keys/query` request when Alice requests
Bob's key, but will not be included when anyone else requests Bob's key:
`GET /keys/query`
``` json
{
"failures": {},
"device_keys": {
"@bob:example.com": {
// ...
}
},
"master_keys": {
"@bob:example.com": {
"user_id": "@bob:example.com",
"keys": {
"ed25519:bobs+base64+master+public+key": "bobs+base64+master+public+key"
},
"usage": ["master"],
"signatures": {
"@alice:example.com": {
"ed25519:base64+user+signing+public+key": "base64+signature+of+bobs+master+key"
}
}
}
}
}
```
## Comparison with MSC1680
MSC1680 suffers from the fact that the attestation graph may be arbitrarily
complex and may become ambiguous how the graph should be interpreted. In
particular, it is not obvious exactly how revocations should be interpreted --
should they be interpreted as only revoking the signature created previously by
the device making the revocation, or should it be interpreted as a statement
that the device should not be trusted at all? As well, a revocation may split
the attestation graph, causing devices that were previously trusted to possibly
become untrusted. Logging out a device may also split the attestation graph.
Moreover, it may not be clear to a user what device verifications would be
needed to reattach the parts of the graph.
One way to solve this is by registering a "virtual device", which is used to
sign other devices. This solution would be similar to this proposal. However,
real devices would still form an integral part of the attestation graph. For
example, if Alice's Osborne 2 verifies Bob's Dynabook, the attestation graph might
look like:
![](images/1756-graph1.dot.png)
If Bob replaces his Dynabook without re-verifying with Alice, this will split
the graph and Alice will not be able to verify Bob's other devices. In
contrast, in this proposal, Alice and Bob sign each other's master keys
with their user-signing keys, and the attestation graph would look like:
![](images/1756-graph2.dot.png)
In this case, Bob's Dynabook can be replaced without breaking the graph.
With normal cross-signing, it is not clear how to recover from a stolen device.
For example, if Mallory steals one of Alice's devices and revokes Alice's other
devices, it is unclear how Alice can rebuild the attestation graph with her
devices, as there may be stale attestations and revocations lingering around.
(This also relates to the question of whether a revocation should only revoke
the signature created previously by the device making the attestation, or
whether it should be a statement that the device should not be trusted at all.)
In contrast, with this proposal, if a device is stolen, then only the
keys for which the device had access to the private keys must be re-issued,
along with any associated signatures. When the new keys are distributed, the
old keys and their signatures will no longer be part of the attestation graph.
## Security considerations
This proposal relies on servers to communicate when cross-signing keys are
deleted and replaced. An attacker who is able to both steal a user's device
and control their homeserver could prevent that device from being marked as
untrusted.
An attacker may be able to upload a large number of signatures in a DoS attack
against clients or servers, similar to the [attack against the SKS keyserver
network](https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f).
Since clients are only sent a subset of signatures, and the attestation graph
is limited, a DoS attack is less likely to be successful in this case.
## Conclusion
This proposal presents an alternative cross-signing mechanism to MSC1680,
allowing users to trust another user's devices without needing to verify each
one individually.

14
proposals/1759-rooms-v2.md
Unescape Escape View File

@ -1,14 +0,0 @@
# MSC 1759 - Rooms V2
This MSC proposes creating a new room version to allow servers and users to opt
in to using the new state resolution algorithm.
## Proposal
The new room version is called "2". The only difference between v2 and v1 is
that v2 rooms use the v2 state resolution algorithm, as defined in MSC 1442 and
MSC 1693.
It is not proposed that servers change the default room version used when
creating new rooms.

428
proposals/1772-groups-as-rooms.md
Unescape Escape View File

@ -1,428 +0,0 @@
# Proposal for Matrix "spaces" (formerly known as "groups as rooms (take 2)")
This MSC, and related proposals, supersede
[MSC1215](https://github.com/matrix-org/matrix-doc/issues/1215).
## Background and objectives
Collecting rooms together into groups is useful for a number of
purposes. Examples include:
* Allowing users to discover different rooms related to a particular topic:
for example "official matrix.org rooms".
* Allowing administrators to manage permissions across a number of rooms: for
example "a new employee has joined my company and needs access to all of our
rooms".
* Letting users classify their rooms: for example, separating "work" from
"personal" rooms.
We refer to such collections of rooms as "spaces".
Synapse and Element-Web currently implement an unspecced "groups" API (referred
to as "`/r0/groups`" in this document) which attempts to provide this
functionality (see
[MSC971](https://github.com/matrix-org/matrix-doc/issues/971)). However,
this is a complex API which has various problems (see
[appendix](#appendix-problems-with-the-r0groups-api)).
This proposal suggests a new approach where spaces are themselves represented
by rooms, rather than a custom first-class entity. This requires minimal
server changes.
The existing `/r0/groups` API would be deprecated in Synapse and remain
unspecified.
## Proposal
Each space is represented by its own room, known as a "space-room". The rooms
within the space are determined by state events within the space-room.
Space-rooms are distinguished from regular messaging rooms by the presence of
a `'type': 'm.space'` property in the content of the `m.room.create` event.
The value of the `type` property uses the Standardised Identifier Grammar from
[MSC2758](https://github.com/matrix-org/matrix-doc/pull/2758). This allows clients to offer slightly customised user experience
depending on the purpose of the room. Currently, no server-side behaviour is
expected to depend on this property. A `type` property on the `m.room.create`
event is used to ensure that a room cannot change between being a space-room
and a non-space room. For more information, see the "Rejected Alternatives"
section below. Additionally, no client behaviour is recommended for handling
unknown room types given the potential for legacy data: clients are free to
make their own decisions about hiding unknown room types from users, though
should note that a future conversation-like type (for example) might be
introduced and could be considered "unknown" by older versions of their client.
As with regular rooms, public spaces are expected to have an alias, for example
`#foo:matrix.org`, which can be used to refer to the space.
Space-rooms may have `m.room.name`, `m.room.avatar` and `m.room.topic` state
events in the same way as a normal room.
Normal messages within a space-room are discouraged (but not blocked by the
server): user interfaces are not expected to have a way to enter or display
such messages. Space-rooms should be created with a power level for
`events_default` of 100, to prevent the rooms accidentally/maliciously
clogging up with messages from random members of the space.
### Membership of spaces
Users can be members of spaces (represented by `m.room.member` state events as
normal). The existing [`m.room.history_visibility`
mechanism](https://matrix.org/docs/spec/client_server/r0.6.1#room-history-visibility)
controls whether membership of the space is required to view the room list,
membership list, etc. "Public" or "community" spaces would be set to
`world_readable` to allow clients to see the directory of rooms within the
space by peeking into the space-room (thus avoiding the need to add
`m.room.member` events to the event graph within the room).
Join rules, invites and 3PID invites work as for a normal room. In order for
clients to distinguish space invites from room invites, all invites must now
include the `m.room.create` event in their `invite_state` and `knock_state`.
### Relationship between rooms and spaces
The intention is that rooms and spaces form a hierarchy, which clients can use
to structure the user's room list into a tree view. The parent/child
relationship can be expressed in one of two ways:
1. The admins of a space can advertise rooms and subspaces for their space by
setting `m.space.child` state events. The `state_key` is the ID of a child
room or space, and the content must contain a `via` key which gives a list
of candidate servers that can be used to join the room. Something like:
```jsonc
// a child room
{
"type": "m.space.child",
"state_key": "!abcd:example.com",
"content": {
"via": ["example.com", "test.org"]
}
}
// a child room with an ordering.
{
"type": "m.space.child",
"state_key": "!efgh:example.com",
"content": {
"via": ["example.com"],
"order": "abcd"
}
}
// no longer a child room
{
"type": "m.space.child",
"state_key": "!jklm:example.com",
"content": {}
}
```
Children where `via` is not present or invalid (not an array) are ignored.
The `order` key is a string which is used to provide a default ordering of
siblings in the room list. (Rooms are sorted based on a lexicographic
ordering of the Unicode codepoints of the characters in `order` values.
Rooms with no `order` come last with no effective `order`. When the `order`
(or lack thereof) is the same, the rooms are sorted in ascending numeric
order of the `origin_server_ts` of their `m.room.create` events, or ascending
lexicographic order of their `room_id`s in case of equal
`origin_server_ts`. `order`s which are not strings, or do not consist
solely of ascii characters in the range `\x20` (space) to `\x7E` (`~`), or
consist of more than 50 characters, are forbidden and the field should be
ignored if received.)
2. Separately, rooms can claim parents via the `m.space.parent` state
event.
Similar to `m.space.child`, the `state_key` is the ID of the parent space,
and the content must contain a `via` key which gives a list of candidate
servers that can be used to join the parent.
```jsonc
{
"type": "m.space.parent",
"state_key": "!space:example.com",
"content": {
"via": ["example.com"],
"canonical": true
}
}
```
Parents where `via` is not present or invalid (not an array) are ignored.
`canonical` determines whether this is the main parent for the space. When
a user joins a room with a canonical parent, clients may switch to view the
room in the context of that space, peeking into it in order to find other
rooms and group them together. In practice, well behaved rooms should only
have one `canonical` parent, but given this is not enforced: if multiple
are present the client should select the one with the lowest room ID, as
determined via a lexicographic ordering of the Unicode code-points.
To avoid abuse where a room admin falsely claims that a room is part of a
space that it should not be, clients could ignore such `m.space.parent`
events unless either (a) there is a corresponding `m.space.child` event in
the claimed parent, or (b) the sender of the `m.space.parent` event has a
sufficient power-level to send such an `m.space.child` event in the
parent. (It is not necessarily required that that user currently be a
member of the parent room - only the `m.room.power_levels` event is
inspected.) [Checking the power-level rather than requiring an *actual*
`m.space.child` event in the parent allows for "secret" rooms (see below).]
Where the parent space also claims a parent, clients can recursively peek
into the grandparent space, and so on.
This structure means that rooms can end up appearing multiple times in the
room list hierarchy, given they can be children of multiple different spaces
(or have multiple parents in different spaces).
In a typical hierarchy, we expect *both* parent->child and child->parent
relationships to exist, so that the space can be discovered from the room, and
vice versa. Occasions when the relationship only exists in one direction
include:
* User-curated lists of rooms: in this case the space will not be listed as a
parent of the room.
* "Secret" rooms: rooms where the admin does not want the room to be
advertised as part of a given space, but *does* want the room to form part
of the hierarchy of that space for those in the know.
Cycles in the parent->child and child->parent relationships are *not*
permitted, but clients (and servers) should be aware that they may be
encountered, and MUST spot and break cycles rather than infinitely looping.
### Suggested children
Space admins can mark particular children of a space as "suggested". This
mainly serves as a hint to clients that that they can be displayed differently
(for example by showing them eagerly in the room list), though future
server-side interfaces (such as the summary API proposed in
[MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) might also
make use of it.
A suggested child is identified by a `"suggested": true` property in the
`m.space.child` event:
```jsonc
{
"type": "m.space.child",
"state_key": "!abcd:example.com",
"content": {
"via": ["example.com", "test.org"],
"suggested": true
}
}
```
A child which is missing the `suggested` property is treated identically to a
child with `"suggested": false`. A suggested child may be a room or a subspace.
### Extended "room invite state"
The specification is currently vague about what room state should be available
to users that have been invited to a room, though the Federation API spec does
recommend that the `invite_room_state` sent over federation via [PUT
`/_matrix/federation/v2/invite`](https://matrix.org/docs/spec/server_server/r0.1.4#put-matrix-federation-v2-invite-roomid-eventid)
should include "the join rules, canonical alias, avatar, and name of the room".
This MSC proposes adding `m.room.create` to that list, so that the recipient of
an invite can distinguish invites to spaces from other invites.
## Future extensions
The following sections are not blocking parts of this proposal, but are
included as a useful reference for how we imagine it will be extended in future.
### Auto-joined children
We could add an `auto_join` flag to `m.space.child` events to allow a space
admin to list the sub-spaces and rooms in that space which should be
automatically joined by members of that space.
This would be distinct from a force-join: the user could subsequently part any
auto-joined room if they desire.
Joining would be performed by the client. This could possibly be sped up by
using a summary API (such as that proposed in
[MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) to get a summary
of the spacetree to be joined, and then using a batch join API to join
whichever subset of it makes most sense for the client's UX.
Obviously auto-joining can be a DoS vector, and we consider it to be antisocial
for a space to try to autojoin its members to more than 100 children (in total).
Clients could display the auto-joined children in the room list whenever the
space appears in the list - thus helping users discover other rooms in a space
even if they're not joined to that space. For instance, if you join
`#matrix:matrix.org`, your client could show that room in the context of its
parent space, with that space's auto-joined children shown alongside it as
siblings.
### Restricting access to the spaces membership list
In the existing `/r0/groups` API, the group server has total control over the
visibility of group membership, as seen by a given querying user. In other
words, arbitrary users can see entirely different views of a group at the
server's discretion.
Whilst this is very powerful for mapping arbitrary organisational structures
into Matrix, it may be overengineered. Instead, the common case is (we believe)
a space where some users are publicly visible as members, and others are not.
One way of achieving this would be to create a separate space for the
private members - e.g. have `#foo:matrix.org` and `#foo-private:matrix.org`.
`#foo-private:matrix.org` is set up with `m.room.history_visibility` to not to
allow peeking; you have to be joined to see the members.
It's worth noting that any member of a space can currently see who else is a
member of that space, which might pose privacy problems for sensitive spaces.
While the server itself will inevitably track the space membership in state
events, a future MSC could restrict the membership from being sent to clients.
This is essentially the same as
[matrix-doc#1653](https://github.com/matrix-org/matrix-doc/issues/1653).
### Flair
("Flair" is a term we use to describe a small badge which appears next to a
user's displayname to advertise their membership of a space.)
The flair image for a group is given by the room avatar. (In future it might
preferable to use hand-crafted small resolution images: see
[matrix-doc#1778](https://github.com/matrix-org/matrix-doc/issues/1778).
One way this might be implemented is:
* User publishes the spaces they wish to announce on their profile
([MSC1769](https://github.com/matrix-org/matrix-doc/issues/1769)
as an `m.flair` state event: it lists the spaces which they are advertising.
* When a client wants to know the current flair for a set of users (i.e.
those which it is currently displaying in the timeline), it peeks the
profile rooms of those users. (Ideally there would be an API to support
peeking multiple rooms at once to facilitate this.)
* The client must check that the user is *actually* a member of the advertised
spaces. Nominally it can do this by peeking the membership list of the
space; however for efficiency we could expose a dedicated Client-Server API
to do this check (and both servers and clients can cache the results fairly
aggressively.)
## Related MSCs
* [MSC2946](https://github.com/matrix-org/matrix-doc/issues/2946): Spaces
Summary API.
* [MSC2962](https://github.com/matrix-org/matrix-doc/issues/2962): Managing
power levels via Spaces.
* [MSC3083](https://github.com/matrix-org/matrix-doc/issues/3083): Restricting
room membership based on space membership.
* [MSC2753](https://github.com/matrix-org/matrix-doc/issues/2753) for
effective peeking over the C/S API.
* [MSC2444](https://github.com/matrix-org/matrix-doc/issues/2444) (or similar)
for effective peeking over Federation.
## Security considerations
None at present.
## Potential issues
* If the membership of a space would be large (for example: an organisation of
several thousand people), this membership has to be copied entirely into the
room, rather than querying/searching incrementally.
* If the membership list is based on an external service such as LDAP, it is
hard to keep the space membership in sync with the LDAP directory. In
practice, it might be possible to do so via a nightly "synchronisation" job
which searches the LDAP directory, or via "AD auditing".
* No allowance is made for exposing different 'views' of the membership list to
different querying users. (It may be possible to simulate this behaviour
using smaller spaces).
* The requirement that `m.space.parent` links be ignored unless the sender has a
high PL in the parent room could lead to surprising effects where a parent
link suddenly ceases to take effect because a user loses their PL in the
parent room. This is mitigated in the general case by honouring the parent
link when there is a corresponding `m.space.child` event, however it remains
a problem for "secret" rooms.
* The `via` servers listed in the `m.space.child` and `m.space.parent` events
could get out of date, and will need to be updated from time to time. This
remains an unsolved problem.
## Rejected alternatives
### Use a separate state event for type of room
[MSC1840](https://github.com/matrix-org/matrix-doc/pull/1840) proposes the use
of a separate `m.room.type` state event to distinguish different room types.
This implies that rooms can dynamically switch between being a Space, and
being a regular non-Space room. That is not a usecase we consider useful, and
allowing it would impose significant complexity on client and server
implementations. Specifically, client and server implementations who store
spaces separately from rooms would have to support migrating back and forth
between them and dealing with the ambiguities of `room_id`s no longer pointing
to valid spaces, etc.
### Use a different sigil/twigil for spaces
Groups used + as a sigil to differentiate them from rooms (e.g. +matrix:matrix.org).
We considered doing similar for Spaces, e.g. a #+ twigil or reuse the + sigil,
but concluded that the resulting complexity and exoticism is not worth it.
This means that clients such as matrix.to have to peek into rooms to find out their
`type` before being able to display an appropriate UI, and users will not know
whether #matrix:matrix.org is a room or a space without using a client (e.g. if
reading an advert). It also means that if the client UI requires a space alias the
client will need to validate the entered data serverside.
## Unstable prefix
The following mapping will be used for identifiers in this MSC during
development:
Proposed final identifier | Purpose | Development identifier
------------------------------- | ------- | ----
`type` | property in `m.room.create` | `org.matrix.msc1772.type`
`m.space` | value of `type` in `m.room.create` | `org.matrix.msc1772.space`
`m.space.child` | event type | `org.matrix.msc1772.space.child`
`m.space.parent` | event type | `org.matrix.msc1772.space.parent`
## History
* This replaces [MSC1215](https://docs.google.com/document/d/1ZnAuA_zti-K2-RnheXII1F1-oyVziT4tJffdw1-SHrE).
* Other thoughts that led into this are [documented](https://docs.google.com/document/d/1hljmD-ytdCRL37t-D_LvGDA3a0_2MwowSPIiZRxcabs).
## Appendix: problems with the `/r0/groups` API
The existing `/r0/groups` API, as proposed in
[MSC971](https://github.com/matrix-org/matrix-doc/issues/971), has various
problems, including:
* It is a large API surface to implement, maintain and spec - particularly for
all the different clients out there.
* Much of the API overlaps significantly with mechanisms we already have for
managing rooms:
* Tracking membership identity
* Tracking membership hierarchy
* Inviting/kicking/banning user
* Tracking key/value metadata
* There are membership management features which could benefit rooms which
would also benefit groups and vice versa (e.g. "auditorium mode")
* The current implementations on Riot Web/iOS/Android all suffer bugs and
issues which have been solved previously for rooms.
* no local-echo of invites
* failures to set group avatars
* ability to specify multiple admins
* It doesn't support pushing updates to clients (particularly for flair
membership): https://github.com/vector-im/riot-web/issues/5235
* It doesn't support third-party invites.
* Groups could benefit from other features which already exist today for rooms
* e.g. Room Directories
* Groups are centralised, rather than being replicated across all
participating servers.

464
proposals/1779-open-governance.md
Unescape Escape View File

@ -1,464 +0,0 @@
# Proposal for Open Governance of Matrix.org
----
**For updated information on the Foundation, please see https://matrix.org/foundation/**.
This MSC exists for historical reference.
----
This whole document is the proposed constitution proposal for Matrix.org, and
will form the basis of the first full Articles of Association (AoA) for [The
Matrix.org Foundation
C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit
legal entity incorporated to act as the neutral guardian of the Matrix
decentralised communication standard on behalf of the whole Matrix community.
See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/
for more context.
This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318).
**This MSC is now formalised in the official Rules of the Matrix.org Foundation,
maintained at https://docs.google.com/document/d/1MhqsuIUxPc7Vf_y8D250mKZlLeQS6E39DPY6Azpc2NY**
## Introduction
Historically the core team of Matrix has been paid to work on it by the same
employer (currently New Vector; the startup incorporated to hire the core
team in Aug 2017). Whilst convenient in initially getting Matrix built, we
recognise that this could create a potential conflict of interest between the
core team’s responsibilities to neutrally support the wider Matrix.org ecosystem
versus the need for New Vector to be able to support the team, and it has always
been the plan to set up a completely neutral custodian for the standard once it
had reached sufficient maturity.
This proposal seeks to establish a new open governance process for Matrix.org,
such that once the specification has finally been ‘born’ and reached an initial
‘r0’ release across all APIs, control of Matrix.org can be decoupled from New
Vector and better support contributions from the whole ecosystem.
The concepts here are somewhat inspired by [Rust’s Governance
Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust-governance.md);
a highly regarded solution to a similar problem: an ambitious
open-source project which has been too many years in the making, incubated at
first by a single company (Mozilla Corporation), which also enjoys a very
enthusiastic community!
## Overview
Governance of the project is split into two teams: the Spec Core Team and the
Guardians of the Foundation. In brief:
The Spec Core Team are the technical experts who curate and edit the Matrix
Specification from day to day, and so steer the evolution of the protocol by
having final review over which Matrix Spec Changes (MSCs) are merged into the
core spec.
The Guardians are the legal directors of the non-profit Foundation, and are
responsible for ensuring that the Foundation (and by extension the Spec Core
Team) keeps on mission and neutrally protects the development of Matrix.
Guardians are typically independent of the commercial Matrix ecosystem and may
even not be members of today’s Matrix community, but are deeply aligned with the
mission of the project. Guardians are selected to be respected and trusted by
the wider community to uphold the guiding principles of the Foundation and keep
the other Guardians honest.
In other words; the Spec Core Team builds the spec, and the Guardians provide an
independent backstop to ensure the spec evolves in line with the Foundation's
mission.
## Guiding Principles
The guiding principles define the core philosophy of the project, and will be a
formal part of the final Articles of Association of the Matrix.org Foundation.
### Matrix Manifesto
We believe:
* People should have full control over their own communication.
* People should not be locked into centralised communication silos, but instead
be free to pick who they choose to host their communication without limiting
who they can reach.
* The ability to converse securely and privately is a basic human right.
* Communication should be available to everyone as a free and open,
unencumbered, standard and global network.
### Mission
The Matrix.org Foundation exists to act as a neutral custodian for Matrix and to
nurture it as efficiently as possible as a single unfragmented standard, for the
greater benefit of the whole ecosystem, not benefiting or privileging any single
player or subset of players.
For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix
protocol. This includes (non-exhaustively):
* End-users of Matrix clients.
* Matrix client developers and testers.
* Spec developers.
* Server admins.
* Matrix packagers & maintainers.
* Companies building products or services on Matrix.
* Bridge developers.
* Bot developers.
* Widget developers.
* Server developers.
* Matrix room and community moderators.
* End-users who are using Matrix indirectly via bridges.
* External systems which are bridged into Matrix.
* Anyone using Matrix for data communications.
"Greater benefit" is defined as maximising:
* the number of Matrix-native end-users reachable on the open Matrix network.
* the number of regular users on the Matrix network (e.g. 30-day retained federated users).
* the number of online servers in the open federation.
* the number of developers building on Matrix.
* the number of independent implementations which use Matrix.
* the number of bridged end-users reachable on the open Matrix network.
* the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam).
* the ability for users to discover content on their terms (empowering them to select what to see and what not to see).
* the quality and utility of the Matrix spec (as defined by ease and ability
with which a developer can implement spec-compliant clients, servers, bots,
bridges, and other integrations without needing to refer to any other
external material).
N.B. that we consider success to be the growth of the open federated network
rather than closed deployments. For example, if WhatsApp adopted Matrix it
wouldn’t be a complete win unless they openly federated with the rest of the
Matrix network.
### Values
As Matrix evolves, it's critical that the Spec Core Team and Guardians are
aligned on the overall philosophy of the project, particularly in more
subjective areas. The values we follow are:
* Supporting the whole long-term ecosystem rather than individual stakeholder gain.
* Openness rather than proprietary lock-in.
* Interoperability rather than fragmentation.
* Cross-platform rather than platform-specific.
* Collaboration rather than competition.
* Accessibility rather than elitism.
* Transparency rather than stealth.
* Empathy rather than contrariness.
* Pragmatism rather than perfection.
* Proof rather than conjecture.
Patent encumbered IP is strictly prohibited from being added to the standard.
Making the specification rely on non-standard/unspecified behaviour of other
systems or actors (such as SaaS services, even open-sourced, not governed by a
standard protocol) shall not be accepted, either.
## The Spec Core Team
The contents and direction of the Matrix Spec is governed by the Spec Core Team;
a set of experts from across the whole Matrix community, representing all
aspects of the Matrix ecosystem. The Spec Core Team acts as a subcommittee of
the Foundation.
Members of the Spec Core Team pledge to act as a neutral custodian for Matrix on
behalf of the whole ecosystem and uphold the Guiding Principles of the project
as outlined above. In particular, they agree to drive the adoption of Matrix as
a single global federation, an open standard unencumbered from any proprietary
IP or software patents, minimising fragmentation (whilst encouraging
experimentation), evolving rapidly, and prioritising the long-term success and
growth of the overall network over individual commercial concerns.
Spec Core Team members need to have significant proven domain experience/skill
and have had clear dedication and commitment to the project and community for >6
months. (In future, once we have subteams a la Rust, folks need to have proven
themselves there first).
Members need to demonstrate ability to work constructively with the rest of the
team; we want participation in the Spec Core Team to be an efficient, pleasant and
productive place, even in the face of inevitable disagreement. We do not want a
toxic culture of bullying or competitive infighting. Folks need to be able to
compromise; we are not building a culture of folks pushing their personal
agendas at the expense of the overall project.
The team should be particularly vigilant against 'trojan horse' additions to the
spec - features which only benefit particular players, or are designed to
somehow cripple or fragment the open protocol and ecosystem in favour of
competitive advantage. Commercial players are of course free to build
proprietary implementations, or use custom event types, or even custom API
extensions (e.g. more efficient network transports) - but implementations must
fall back to interoperating correctly with the rest of the ecosystem.
### Spec Core Team logistics
The Spec Core Team itself will be made up of roughly 8 members + 1 project lead.
Roughly half the members are expected to be from the historical core team
(similar to Rust). The team must have 5 members to be able to function, with
the aim of generally having between 7 and 9 members.
In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API;
IS API; Crypto), but as a starting point we are beginning with a single core
team in the interests of not over-engineering it and scaling up elastically.
Spec Core Team members need to be able to commit to at least 1 hour a week of
availability to work on the spec and (where relevant) reference implementations.
Members must arrange their own funding for their time.
Responsibilities include:
* Reviewing Matrix Spec Change proposals and Spec PRs.
* Contributing to and reviewing reference implementations of Matrix Spec Change
proposals.
* Shepherding Matrix Spec Changes on behalf of authors where needed.
* Triaging Matrix Spec issues.
* Coordinating reference implementations.
* Ensuring the code of conduct for +matrix:matrix.org community rooms is
maintained and applied.
If members are absent (uncontactable) for more than 8 weeks without prior
agreement, they will be assumed to have left the project.
Spec Core Team members can resign whenever they want, but must notify the rest
of the team and the Guardians on doing so.
New additions to the team must be approved by all current members of the team.
Membership has to be formally proposed by someone already on the Spec Core Team.
Members can be removed from the team if 75% of the current members approves and
agrees they are no longer following the goals and guiding principles of the
project. (The 75% is measured of the whole team, including the member in
question).
Guardians act as a safety net, and can appoint or remove Spec Core Team members
(requiring approval by 75% of the current Guardians) if the Spec Core Team is
unable to function or is failing to align with the Foundation's mission.
It's suggested that one of the Spec Core Team members should also be a Guardian,
to facilitate information exchange between the Guardians and the Spec Core Team,
and to represent the technical angle of the project to the other Guardians.
The project lead role acts to coordinate the team and to help steer the team to
consensus in the event of failing to get agreement on a Matrix Spec Change.
Every 12 months, a vote of confidence is held in the project lead, requiring the
approval of 75% of the current Spec Core Team members for the lead to be
renewed. There is no maximum term for the project lead. The lead may be
removed by the core team at any point (requiring 75% approval of current
members), and may resign the role at any point (notifying the team and the
Guardians). The lead automatically resigns the role if they resign from the
Spec Core Team. Resignation automatically triggers selection of a new lead, who
must be selected from the existing Spec Core Team with 75% approval from current
members within 14 days.
It is vital that the core spec team has strong domain expertise covering all
different domains of the spec (e.g. we don't want to end up with a core spec
team where nobody has strong experience in cryptography)
The initial Spec Core Team (and their domain areas) is:
* Matthew Hodgson (Lead, Guardian)
* Erik Johnston (Servers)
* Richard van der Hoff (Servers, Cryptography)
* David Baker (Clients, IS API, Push API, Media)
* Hubert Chathi (Cryptography, General)
* Andrew Morgan (Servers, AS API, Spec Process)
* Travis Ralston (Bots and Bridges & AS API, Media, acting with Dimension hat on)
* Alexey Rusakov (Clients on behalf of Community)
* TBD
MSCs require approval by 75% of the current members of the Spec Core Team to
proceed to Final Comment Period (see https://matrix.org/docs/spec/proposals for
the rest of the MSC process).
Even though a threshold of only 75% is required for approval, the Spec Core Team
is expected to seek consensus on MSCs.
The above governance process for the Spec Core Team is considered as part of the
spec and is updated using the Matrix Spec Change process. However, changes to
the governance process also require approval by 75% of the current Guardians
(acting as a formal decision of the Foundation's Directors), in order to ensure
changes are aligned with the Foundation's mission. For avoidance of doubt, Spec
Core Team votes and Guardians' votes are distinct and a person having both hats
has to vote independently on both forums with the respective hat on.
Spec Core Team decisions (e.g. appointing/removing members and lead)
should be published openly and transparently for the public.
## The Guardians
*This section will be used as the basis for the legal responsibilities of
Directors in the Articles of Association of the Foundation.*
The Guardians form the legal Board of Directors of The Matrix.org Foundation CIC
(Community Interest Company). They are responsible for ensuring the Foundation
is following its guiding principles, and provide a safety mechanism if the
structure of the Spec Core Team runs into trouble.
In practice, this means that:
* Guardians are responsible for ensuring the Spec Core Team continues to
function, and have the power to appoint/dismiss members of the spec core team
(with the agreement of 75% of the Guardians) to address issues with the Spec
Core Team.
* Guardians must keep each other honest, providing a ‘checks and balances’.
mechanism between each other to ensure that all Guardians and the Spec Core
Team act in the best interests of the protocol and ecosystem.
* Guardians may dismiss members of the Spec Core Team who are in serious
breach of the guiding principles.
* Guardians may appoint members of the Spec Core Team to break deadlocks in the
unanimous consent requirement for the Spec Core Team when appointing new
members.
* Guardians may also override deadlocks when appointing a Spec Core Team leader
(with approval of 75% of the current Guardians).
* Guardians must approve changes to the above Guiding Principles (with approval
of 75% of the current Guardians)
* Guardians are responsible for approving use of the Foundation's assets
(e.g. redistributing donations).
* In future, Guardians may also be responsible for ensuring staff are hired by
the Foundation to support administrative functions and other roles required
to facilitate the Foundation's mission.
* As well as the Spec Core Team committee, they may also oversee committees for
other areas such as marketing Matrix.org, registering custom event types,
or "Made for Matrix" certification.
* Guardians are responsible for choosing if, when and how staff are located by
the Foundation to fill administrative and other functions required to
facilitate the Foundations' mission.
* Guardians are responsible for choosing if and when additional committees are
formed, and to oversee those committees.
* Guardians are not required to be involved on a day-to-day basis, however
those not taking a hands on approach are required to monitor to ensure a
suitable balance is kept by those that do.
Guardians are chosen typically to be independent of the commercial Matrix
ecosystem (and especially independent from New Vector), and may even not be
members of today’s Matrix community. However, they should be deeply aligned with
the mission of the project, and respected and trusted by the wider community to
uphold the guiding principles of the Foundation and keep the other Guardians
honest.
Guardians are responsible for maintaining and updating the Guiding Principles
and Articles of Association of the Foundation if/when necessary. Changes to the
Guiding Principles require approval from 75% of the current Guardians and are
passed as a 'special resolution' of the board.
New Guardians may be appointed with approval from 75% of the current Guardians.
Guardians may resign at any time, with notification to the board.
Guardians may be removed due to serious breach of the guiding principles with
approval by 75% of the other current Guardians, or if absent from 3 consecutive
board meetings, or if they are legally disqualified from acting as a Director.
We aim to recruit roughly 5 Guardians. The initial Guardians are:
* Matthew Hodgson (CEO/CTO, New Vector)
* Amandine Le Pape (COO, New Vector)
* TBA (agreed, needs paperwork)
* TBD
* TBD
The intention is for Matthew & Amandine (the original founders of Matrix) to
form a minority of the Guardians, in order to ensure the neutrality of the
Foundation relative to Matthew & Amandine’s day jobs at New Vector.
Guardians must arrange their own funding for their time.
Guardian decisions (e.g. appointing/removing guardians; changes to the spec core
team; etc) should be published openly and transparently for the public.
## The Code Core Team (aka The Core Team)
The "Core Team" (or the "Code Core Team", to disambiguate from the Spec Core
Team) is a loose term that describes the set of people with access to commit
code to the public https://github.com/matrix-org repositories, who are either
working on matrix.org's reference implementations or the spec itself. Commit
access is decided by those responsible for the projects in question, much like
any other open source project. Anyone is eligible for commit access if they
have proved themselves a valuable long-term contributor, uphold the guiding
principles and mission of the project and have proved themselves able to
collaborate constructively with the existing core team. Active participation in
the core team is also signified by membership of the +matrix:matrix.org Matrix
community.
Responsibilities include:
* Helping ensure the quality of the projects' code repositories.
* Ensuring all commits are reviewed.
* Ensuring that all projects follow the Matrix spec.
* Helping architect the implementations.
* Contributing code to the implementations.
* Fostering contributions and engaging with contributors constructively in a
way that fosters a healthy and happy community.
* Following the Guiding Principles and promoting them within the community.
Code Core Team members must arrange their own funding for their time.
## Functions of the Foundation
* Independent legal entity which acts as neutral custodian of Matrix.
* Gathers donations.
* Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector:
* Owns the matrix.org domain and branding.
* Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org).
By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it.
* Owns the IP of the website.
* Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc).
* Responsible for finding someone to run the Matrix.org homeserver (currently New Vector).
* Publishes the spec.
* Responsible for tools and documentation that support the spec.
* Responsible for ensuring reference implementations and test suite exists for the spec.
* Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects).
* Manages any future IANA-style allocations for Matrix, such as:
* mx:// URI scheme.
* TCP port 8448.
* .well-known URIs
* Ensures that Matrix promotion is happening (e.g. ensuring that meetups &
events & community activity is supported).
In future:
* Contracts entities to work on Matrix if such contracts help the Foundation to
fulfil its mission and obey the Guiding Principles (e.g. redistributing
donations back to fund development of reference implementations or compliance
kits).
* Manages a "Made for Matrix" certification process? (to confirm that products
are actually compatible with Matrix).
## Timings
The Foundation was incorporated in October 2018 as a UK limited by guarantee
private company, using generic non-profit articles of association combined with
a high-level mission lock aligned with the above:
> 4. The objects of the Foundation are for the benefit of the community as a whole
> to:
> 4.1.1 empower users to control their communication data and have freedom over
> their communications infrastructure by creating, maintaining and promoting
> Matrix as an openly standardised secure decentralised communication protocol and
> network, open to all, and available to the public for no charge;
> 4.1.2 build and develop an appropriate governance model for Matrix through the
> Foundation, in order to drive the adoption of Matrix as a single global
> federation, an open standard unencumbered from any proprietary intellectual
> property and/or software patents, minimising fragmentation (whilst encouraging
> experimentation), maximising speed of development, and prioritising the long-
> term success and growth of the overall network over the commercial concerns of
> an individual person or persons.
The foundation was then converted into a Community Interest Company, formalising
its non-profit status under the approval of the independent [Community Interest
Companies Regulator](https://www.gov.uk/government/organisations/office-of-the-regulator-of-community-interest-companies),
which took effect Jan 2019.
We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and
finalise the Foundation's articles of association shortly afterwards based on the
contents of this MSC once passed FCP.
This will coincide with the formal asset transfer of Matrix.org's assets from
New Vector Ltd, and the appointment of the remaining Guardians.

78
proposals/1794-federation-v2-invites.md
Unescape Escape View File

@ -1,78 +0,0 @@
# MSC 1794 - Federation v2 Invite API
This proposal adds a new `/invite` API to federation that supports different
room versions.
## Motivation
It is planned for future room versions to be able to change the format of events
in various ways. To support this, all servers must know the room version
whenever they need to parse an event. Currently the `/invite` API does not
include the room version, so the target server would be unable to parse the event included in the payload.
## Proposal
Add a new version of the invite API under the prefix `/_matrix/federation/v2`,
which has a payload of:
```
PUT /_matrix/federation/v2/invite/<room_id>/<event_id>
{
"room_version": <room_version>,
"event": { ... },
"invite_room_state": [ ... ]
}
```
The differences between this and `v1` are:
1. The payload in `v1` is the event, while in `v2` the event is instead placed
under an `"event"` key. This is for consistency with other APIs, and to allow
extra data to be added to the request payload separately from the event.
2. A required field called `"room_version"` is added that specifies the room
version.
3. The `"invite_room_state"` is moved from the `unsigned` section of the event
to a top level key. The key `invite_room_state` being in the `event.unsigned`
was a hack due to point 1. above.
The response is identical to `v1`, except that:
1. If the receiving server does not support the given room version the
appropriate incompatible-room-version error is returned, in the same way
as e.g. for `/make_join` APIs.
2. The response payload is no longer in the format of `[200, { ... }]`, and is
instead simply the `{ ... }` portion. This fixes a historical accident to
bring the invite API into line with the rest of the federation API.
If a call to `v2` `/invite` results in an unrecognised request exception **AND**
the room version is `1` or `2` then the sending server should retry the request
with the `v1` API.
## Alternatives
### Reusing V1 API
One alternative is to add a `room_version` query string parameter to the `v1`
`/invite` API in a similar way as for the `/make_join` APIs. However, older
servers would ignore the query string parameter while processing an incoming
`/invite` request, resulting in the server attempting to parse the event in the
old `v1` format. This would likely result in either a `400` or `500` response,
which the sending server could interpret as the receiving server not supporting
the room version.
This method, however, is fragile and could easily mask legitimate `400` and
`500` errors that are not due to not supporting the room version.
### Using V1 API for V1 room versions
Instead of all servers attempting to use the new API and falling back if the API
is not found, servers could instead always use the current API for V1 and V2
rooms.
However, this would not allow us to deprecate the `v1` API.

50
proposals/1802-standardised-federation-response-format.md
Unescape Escape View File

@ -1,50 +0,0 @@
# Remove the '200' value from some federation responses
Some responses formats in the federation API specifications use the form `[200,
res]` in which `res` is the JSON object containing the actual response for the
affected endpoints. This was due to a mishap while building synapse's federation
features, and has been left as is because fixing it would induce backward
incompatibility.
This proposal proposes a backward compatible alternative
## Proposal
Add a new version of the following endpoints under the prefix
`/_matrix/federation/v2`:
* `PUT /_matrix/federation/v2/send_join/{roomId}/{eventId}`
* `PUT /_matrix/federation/v2/send_leave/{roomId}/{eventId}`
Which are the exact same endpoints as their equivalents under the `v1` prefix,
except for the response format, which changes from:
```
[
200,
res
]
```
To:
```
res
```
Where `res` is the JSON object containing the response to a request directed at
one of the affected endpoints.
This proposal doesn't address the `PUT
/_matrix/federation/v1/invite/{roomId}/{eventId}` endpoint since
[MSC1794](https://github.com/matrix-org/matrix-doc/pull/1794) already takes care
of it.
If a call to any of the `v2` endpoints described in this proposal results in an
unrecognised request exception (i.e. in a response with a 400 or a 404 status
code), then the sending server should retry the request with the `v1` API.
## Alternative solutions
An alternative solution would be to make the change in the `v1` federation API,
but would break backward compatibility, thus would be harder to manage.

50
proposals/1804-advertising-capable-room-versions.md
Unescape Escape View File

@ -1,50 +0,0 @@
# Proposal for advertising capable room versions to clients
Currently clients need to guess at which room versions the server supports, if any. This is particularly
difficult to do as it generally requires knowing the state of the ecosystem and what versions are
available and how keen users are to upgrade their servers and clients. The impossible judgement call
for when to encourage people to upgrade shouldn't be impossible, or even a judgement call.
## Proposal
Building off of [MSC1753](https://github.com/matrix-org/matrix-doc/pull/1753) (capabilities API) and
the [recommendations laid out for room versions](https://github.com/matrix-org/matrix-doc/pull/1773/files#diff-1436075794bb304492ca6953a6692cd0R463),
this proposal suggests a `m.room_versions` capability be introduced like the following:
```json
{
"capabilities": {
"m.room_versions": {
"default": "1",
"available": {
"1": "stable",
"2": "stable",
"state-res-v2-test": "unstable",
"event-ids-as-hashes": "unstable",
"3": "future-scifi-label"
}
}
}
}
```
Clients are encouraged to make use of this capability to determine if the server supports a given
version, and at what level of stability. Anything not flagged explicitly as `stable` should be treated
as `unstable` (ie: `future-scifi-label` is the same as `unstable`).
The default version is the version that the server is using to create new rooms with. Clients can
make the assumption that the default version is a stable version.
Clients should encourage people with sufficient permissions to perform an upgrade to upgrade their
rooms to the `default` room version when the room is using an `unstable` version.
## Potential issues
Changes aren't pushed to the client, which means clients may want to poll this endpoint on some
heuristic instead. For example, clients may want to poll the endpoint weekly or when the user relaunches
the client. Clients may also wish to provide users a way to upgrade without considering the capabilities
of the server, expecting that the server may not support the user-provided version - the intention
being such a feature would be used by eager room administrators which do not want to relaunch their
client, for example.

21
proposals/1812-federation-make-membership.md
Unescape Escape View File

@ -1,21 +0,0 @@
# MSC 1813 - Federation Make Membership Room Version
This proposal adds a new `room_version` field to the responses of `/make_leave`
and `/make_join` APIs.
## Motivation
It is planned for future room versions to be able to change the format of events
in various ways. To support this, all servers must know the room version
whenever they need to parse an event. Currently the `/make_*` APIs do not
include the room version in the response, so the requesting server is unable to
parse the event included in the response body.
## Proposal
Add a new `room_version` field to the response body of the `v1` `/make_join` and
`/make_leave` APIs, which describes the room version.
For backwards compatibility servers must correctly handle responses that do not
include the new field. In which case the room version is assumed to be one of
either `1` or `2`.

55
proposals/1819-remove-presence-lists.md
Unescape Escape View File

@ -1,55 +0,0 @@
# Remove references to presence lists
[Presence](https://matrix.org/docs/spec/client_server/r0.4.0.html#id107) lists
allow a given user the ability to subscribe to other users and be alerted to
their current presence status.
While spec'd, no established client has implemented support and the only server
side implementation raises privacy concerns.
The proposal is to simply remove references to presence lists with a view to
revisiting the same ideas in the future.
This MSC addresses
[#1810](https://github.com/matrix-org/matrix-doc/issues/1810)
## Proposal
Presence lists seem like a good fit for ['MSC1769: Extensible profiles as
rooms'](https://github.com/matrix-org/matrix-doc/pull/1769) proposal, meaning
that the current design will most likely be superseded.
Additionally, no major client has implemented the behaviour to date and the
only server implementation of presence lists (Synapse) auto-accepts invites
leading to privacy concerns in the wild.
With this in mind the most pragmatic solution is to remove presence lists ahead
of the r0 release.
Specifically:-
CS API: Remove
* [POST
/_matrix/client/r0/presence/list/{userId}](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-presence-list-userid)
* [GET
/_matrix/client/r0/presence/list/{userId}](https://matrix.org/docs/spec/client_server/r0.4.0.html#get-matrix-client-r0-presence-list-userid)
SS API: Remove
* [m.presence_invite](https://github.com/matrix-org/matrix-doc/blob/8b65da1cf6fce5f657a2a46b5c6c8bcc24d32ae3/api/server-server/definitions/event-schemas/m.presence_invite.yaml)
* [m.presence_accept](https://github.com/matrix-org/matrix-doc/blob/8b65da1cf6fce5f657a2a46b5c6c8bcc24d32ae3/api/server-server/definitions/event-schemas/m.presence_accept.yaml)
* [m.presence_deny](https://github.com/matrix-org/matrix-doc/blob/8b65da1cf6fce5f657a2a46b5c6c8bcc24d32ae3/api/server-server/definitions/event-schemas/m.presence_deny.yaml)
## Tradeoffs
Ideally this proposal would also come with an alternative design for this
functionality. Out of pragmatism the proposal only covers removal of what is
there today.
## Conclusions
This is a common sense attempt to remove unused portions of the spec ahead of
an r0 release. It does not suggest that the ability to subscribe to the
presence of others is undesirable and assumes that this behaviour will return
again in some form.

25
proposals/1831-srv-after-wellknown.md
Unescape Escape View File

@ -1,25 +0,0 @@
# Proposal to do SRV lookups after .well-known to discover homeservers
Currently there is a logistical error proposed by [MSC1708](https://github.com/matrix-org/matrix-doc/pull/1708)
which results in some homeservers unable to migrate to the new functionality
proposed by [MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711). This
can happen if the delegated homeserver cannot obtain a valid TLS certificate for
the domain, and an SRV record is used for backwards compatibility reasons.
Specifically, in order to be compatible with requests from both Synapse 0.34 and 1.0,
servers can have both a SRV and a .well-known file, with Synapse presenting a certificate
corresponding to the target of the .well-known. Synapse 0.34 is then happy because it
will follow the SRV (and won't care about the incorrect certificate); Synapse 1.0 is
happy because it will follow the .well-known (and will see the correct cert).
## Proposal
We change the order of operations to perform a .well-known lookup before falling
back to resolving the SRV record. This allows for domains to delegate to other
hostnames and maintains backwards compatibility with older homeservers.
## Tradeoffs
More HTTP hits will be made due to the .well-known lookup being first. This is
somewhat mitigated by servers caching the responses appropriately, and using
connection pools where possible.

14
proposals/1866-invite-unsupported-version-error-code.md
Unescape Escape View File

@ -1,14 +0,0 @@
# MSC 1866 - Unsupported Room Version Error Code for Invites
It is currently unspecified what error code should be relayed to clients when
they attempt to invite a user on a remote server that does not support the room
version.
The proposal is to reuse the `M_UNSUPPORTED_ROOM_VERSION` error code that is
currently returned by the create room API.
Strictly, the error returned by the create room API would mean the local server
didn't support the room version, while for the invite API it would mean the
remote server didn't. However, there is sufficient overlap that it makes sense
to reuse the same error code and rely on the context to differentiate the two
cases.

166
proposals/1884-replace-slashes-in-event_ids.md
Unescape Escape View File

@ -1,166 +0,0 @@
# MSC1884: Proposal to replace slashes in event IDs
[MSC1659](https://github.com/matrix-org/matrix-doc/pull/1659) mandated that,
starting in version 3 rooms, event IDs must be calculated as a base64-encoding
of a hash. This implies that event IDs may contain any character in the
standard Base64 alphabet, which notably includes the slash character, `/`.
Event IDs are often embedded in URI paths, and since the slash character is
used as a separator in URI paths, this presents a problem. The immediate
solution is to ensure that event IDs are URL-encoded, so that `/` is instead
represented as `%2F`. However, this is not entirely satisfactory for a number
of reasons:
* The act of escaping and unescaping slash characters when manually calling
the API during devops work becomes an constant and annoying chore which
is entirely avoidable. Whenever using tools like `curl` and `grep` or
manipulating SQL, developers will have to constantly keep in mind whether
they are dealing with escaped or unescaped IDs, and manually convert between
the two as needed. This will only get worse with further keys-as-IDs
landing with MSC1228.
* There exist a number of client (and possibly server) implementations which
do not currently URL-encode such parameters; these are therefore broken by
such event IDs and must be updated. Furthermore, all future client
implementers must remember to do the encoding correctly.
* Even if client implementations do remember to URL-encode their parameters,
they may not do it correctly: many URL-encoding implementations may be
intended to encode parameters in the query-string (which can of course
contain literal slashes) rather than the path component.
* Some proxy software may treat `%2F` specially: for instance, Apache, when
configured as a reverse-proxy, will reject requests for a path containing
`%2F` unless it is also configured with `nocanon`. Again this means that
existing setups will be broken by this change, and it is a trap for new
users of the software.
* Cosmetically, URL-escaping base64 in otherwise-constant-length IDs results
in variable length IDs, making it harder to visually scan lists of IDs and
manipulate them in columnar form when doing devops work.
* Those developing against the CS API might reasonably expect us to use
URL-safe identifiers in URLs where available, rather than deliberately
choosing non-URL-safe IDs, which could be seen as developer-unfriendly.
## Proposal
This MSC proposes that we should introduce a new room version, in which event
IDs are encoded using the [URL-safe Base64
encoding](https://tools.ietf.org/html/rfc4648#section-5) (which uses `-` and
`_` as the 62nd and 63rd characters instead of `+` and `/`).
We will then aim to use URL-safe Base64 encoding across Matrix in future,
such that typical CS API developers should be able to safely assume
that for all common cases (including upcoming MSC1228 identifiers) they should
use URL-safe Base64 when decoding base64 strings.
The exception would be for E2EE data (device keys and signatures etc) which
currently use normal Base64 with no easy mechanism to migrate to a new encoding.
Given E2EE development is rare and requires expert skills, it seems acceptable
to expect E2EE developers to be able to use the right encoding without tripping
up significantly.
Similarly, the S2S API could continue to use standard base64-encoded hashes and
signatures in the places it does today, given they are only exposed to S2S API
developers who are necessarily expert and should be able to correctly pick the
right encoding.
## Counterarguments
1. Inconsistency. Base64 encoding is used heavily elsewhere in the matrix
protocol and in all cases the standard encoding is used (though with some
variation as to the inclusion of padding characters). Further, SHA256 hashes
are used in a number of places and are universally included with standard,
unpadded Base64.
Changing event IDs alone would therefore leave us with a confusing mix of
encodings.
However, the current uses of standard Base64 encodings are not exposed to
common CS API developers, and so whilst this might be slightly confusing
for the minority of expert homeserver developers, the confusion does not
exist today for client developers (except those implementing E2EE).
Therefore it seems safe to standardise on URL-safe Base64 for identifiers
exposed to the client developers, who form by far the majority of the
Matrix ecosystem today, and expect as simple an API as possible.
A potential extension would be to change *all* Base64 encodings to be
URL-safe. This would address the inconsistency. However, it feels like a
large job which would span the entire matrix ecosystem (far larger than
updating clients to URL-encode their URL prarameters), and again the
situation would be confusing while the transition was in progress.
2. Incompleteness. Event IDs are certainly not the only identifier which can
contain slashes - Room aliases, Room IDs, Group IDs, User IDs [1], and state
keys can all contain slashes, as well as a number of identifiers whose
grammars are currently underspecified (eg transaction ids, event types,
device IDs). (Indeed, there was nothing preventing Event IDs from containing
slashes before room v3 - it just happened that Synapse used an algorithm
which didn't generate them).
All of these other identifiers can appear in URLs in either or both the
client-server or server-server APIs, and all have the potential to cause
misbehaviour if software does not correctly URL-encode them.
It can be argued that it is better for software to fail 50% of the time [2]
so that it can be fixed than it is to fail only on edge-cases or, worse,
when deliberately provoked by a malicious or "curious" actor.
Of course, an alternative is to modify the grammars of all of these
identifiers to forbid slashes.
The counter-counterargument to this is that it is of course best practice
for implementations is to URL-escape any IDs used in URLs, and human-selected
IDs such as Room aliases, Group IDs, Matrix user IDs etc apply an adequate
forcing function already to remind developers to do this. However,
it doesn't follow that we should then also deliberately pick URL-unsafe
encodings for machine-selected IDs - the argument that it is better for software
to fail 50% of the time to force a fix is irrelevant when the possibility
exists for the software to fail 0% of the time in the first place by picking
an identifier format which cannot fail.
[1] Discussion remains open as to whether allowing slashes in User IDs was a
good idea.
[2] 48% of random 32-byte sequences will contain a slash when Base64-encoded.
## Alternatives
An alternative would be to modify all REST endpoints to use query or body
parameters instead of path parameters. This would of course be a significant
and incompatible change, but it would also bring the benefit of solving a
common problem where forgetting to use `nocanon` in a reverse-proxy
configuration [breaks
federation](https://github.com/matrix-org/synapse/issues/3294) (though other
solutions to that are also possible).
## Conclusion
There are two main questions here:
1. Whether it's worth forcing CS API developers to juggle escaping of
machine-selected IDs during manual use of the API in order to remind them
to escape all variables in their URIs correctly when writing code.
2. Whether it's a significant problem for E2EE & SS API developers to have to
handle strings which are a mix of standard Base64 and URL-safe Base64
encodings.
Both of these are a subjective judgement call.
Given we wish the CS API particularly to be as easy as possible for manual
use, it feels that we should find another way to encourage developers to
escape variables in their URLs in general - e.g. by recommending that
developers test their clients against a 'torture room' full of exotic IDs and
data, or by improving warnings in the spec... rather than (ab)using
machine-selected IDs as a reminder.
Meanwhile, given we have many more people manually invoking the CS API than
developing on the SS or E2EE APIs, and we wish to make the CS API particularly
easy for developers to manually invoke, it feels we should not prioritise
consistency of encodings for SS/E2EE developers over the usability of the CS
API.
Therefore, on balance, it seems plausible that changing the format of event IDs
does solve sufficient problems to make it desirable.

109
proposals/1915-unbind-identity-server-param.md
Unescape Escape View File

@ -1,109 +0,0 @@
# MSC 1915 - Add unbind 3PID APIs
Note that this is a simplified version of MSC1194.
## Motivation
Currently we do not have a reasonable route for a user to unbind/remove a 3PID
from their account, particularly when deactivating their account. Users have an
expectation to be able to do this, and thus we should have an API to provide it.
This is meant as a simple extension to the current APIs, and so this explicitly
does not try and solve any existing usability concerns.
## API Changes
### Client-Server 3PID Delete API
Add an `id_server` param to `POST /_matrix/client/r0/account/3pid/delete` API,
which matches the 3PID creation APIs.
The new `id_server` parameter is optional and if missing the server will attempt
to unbind from the identity server used when originally binding the 3pid (if
known by the homeserver).
The 200 response is a JSON object with an `id_server_unbind_result` field whose
value is either `success` or `no-support`, where the latter indicates that the
identity server (IS) does not support unbinding 3PIDs directly. If the identity
server returns an error then that should be returned to the client. If the homeserver
is unable to determine an `id_server` to use, it should return `no-support` for
the `id_server_unbind_result`.
Example:
```
POST /_matrix/client/r0/account/3pid/delete HTTP/1.1
{
"medium": "email",
"address": "foobar@example.com",
"id_server": "https://matrix.org
}
HTTP/1.1 200 OK
{
"id_server_unbind_result": "success"
}
```
### Client-Server Deactivate account API
Add an `id_server` param to `POST /_matrix/client/r0/account/deactivate` API,
with the same semantics as above. This is used to unbind any bound threepids
from the given identity server.
### Identity Server 3PID Unbind API
Add `POST /_matrix/identity/api/v1/3pid/unbind` with `mxid` and `threepid` fields.
The `mxid` is the user's `user_id` and `threepid` is a dict with the usual
`medium` and `address` fields.
If the server returns a 400, 404 or 501 HTTP error code then the homeserver
should assume that the identity server doesn't support the `/3pid/unbind` API, unless
it returns a specific matrix error response (i.e. the body is a JSON object with
`error` and `errcode` fields).
The identity server should authenticate the request in one of two ways:
1. The request is signed by the homeserver which controls the `user_id`.
2. The request includes the `sid` and `client_secret` params (as per `/bind`),
which proves ownership of the given 3PID.
Example:
```
POST /_matrix/identity/api/v1/3pid/unbind HTTP/1.1
{
"mxid": "@foobar:example.com",
"threepid": {
"medium": "email",
"address": "foobar@example.com"
}
}
HTTP/1.1 200 OK
{}
```
# Trade-offs
A homeserver can unbind any 3PID associated with one of its users, and
specifically does not require a re-validation of control of the 3PID. This means
that users have to trust that their homeserver will not arbitrarily remove valid
3PIDs, however users must already trust their homeserver to a large extent. The
flip side is that this provides a mechanism for homeservers and users to remove
3PIDs directed at their user IDs that they no longer (or never did) have control
over.
Removing a 3PID does not require user interactive auth (UIA), which opens a
potential attack whereby a logged in device can remove all associated 3PIDs and
then log out all devices. If the user has forgotten their password they would no
longer be able to reset their password via a 3PID (e.g. email), resulting in
losing access to their account. However, given that clients and servers have
implemented these APIs in the wild this is considered a sufficient edge case
that adding UIA is unlikely to be worthwhile.

45
proposals/1930-tombstone-notifications.md
Unescape Escape View File

@ -1,45 +0,0 @@
# Proposal to add a default push rule for m.room.tombstone events
Currently users are unaware of when a room becomes upgraded, leaving them potentially in the old room
without knowing until they visit the room again. By having a notification for when the room is upgraded,
users are able to ensure they are able to stay relevant in rooms by joining the upgraded room.
## Proposal
A new default override rule is to be added which is similar to `@room` notifications:
```json
{
"rule_id": ".m.rule.tombstone",
"default": true,
"enabled": true,
"conditions": [
{
"kind": "event_match",
"key": "type",
"pattern": "m.room.tombstone"
},
{
"kind": "event_match",
"key": "state_key",
"pattern": ""
}
],
"actions": [
"notify",
{
"set_tweak": "highlight",
"value": true
}
]
}
```
## Tradeoffs
Clients could calculate this on their own and show some sort of "room upgraded" notification instead,
however by doing it this way it means that all clients would need to be aware of room upgrades. Having
a default push rule means that clients get this notification for free. Clients which want a more diverse
UX can still do so by ignoring this push rule locally.

293
proposals/1946-secure_server-side_storage.md
Unescape Escape View File

@ -1,293 +0,0 @@
# Secure Secret Storage and Sharing
Some features may require clients to store encrypted data on the server so that
it can be shared securely between clients. Clients may also wish to securely
send such data directly to each other. For example, key backups
([MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219)) can store the
decryption key for the backups on the server, or cross-signing
([MSC1756](https://github.com/matrix-org/matrix-doc/pull/1756)) can store the
signing keys. This proposal presents a standardized way of storing such data.
## Changes
- [MSC2472](https://github.com/matrix-org/matrix-doc/pull/2472) changed the
encryption algorithm used from an asymmetric algorithm (Curve25519) to a
symmetric algorithm (AES).
## Proposal
Secrets are data that clients need to use and that are sent through or stored
on the server, but should not be visible to server operators. Secrets are
plain strings -- if clients need to use more complicated data, they must be
encoded as a string, such as by encoding as JSON.
### Storage
If secret data is stored on the server, it must be encrypted in order to
prevent homeserver administrators from being able to read it. A user can have
multiple keys used for encrypting data. This allows the user to selectively
decrypt data on clients. For example, the user could have one key that can
decrypt everything, and another key that can only decrypt their user-signing
key for cross-signing.
Key descriptions and secret data are both stored in the user's account_data.
#### Key storage
Each key has an ID, and the description of the key is stored in the user's
account_data using the event type `m.secret_storage.key.[key ID]`. The contents
of the account data for the key will include an `algorithm` property, which
indicates the encryption algorithm used, as well as a `name` property, which is
a human-readable name. Other properties depend on the encryption algorithm,
and are described below.
Example:
A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg`
```json
{
"name": "Some key",
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
// ... other properties according to algorithm
}
```
A key can be marked as the "default" key by setting the user's account_data
with event type `m.secret_storage.default_key` to an object that has the ID of
the key as its `key` property. The default key will be used to encrypt all
secrets that the user would expect to be available on all their clients.
Unless the user specifies otherwise, clients will try to use the default key to
decrypt secrets.
#### Secret storage
Encrypted data is stored in the user's account_data using the event type
defined by the feature that uses the data. For example, decryption keys for
key backups could be stored under the type `m.megolm_backup.v1`,
or the self-signing key for cross-signing could be stored under the type
`m.cross_signing.self_signing`.
The account_data will have an `encrypted` property that is a map from key ID
to an object. The algorithm from the `m.secret_storage.key.[key ID]` data for
the given key defines how the other properties are interpreted, though it's
expected that most encryption schemes would have `ciphertext` and `mac`
properties, where the `ciphertext` property is the unpadded base64-encoded
ciphertext, and the `mac` is used to ensure the integrity of the data.
Example:
Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`:
`org.example.some.secret`:
```json
{
"encrypted": {
"key_id_1": {
"ciphertext": "base64+encoded+encrypted+data",
"mac": "base64+encoded+mac",
// ... other properties according to algorithm property in
// m.secret_storage.key.key_id_1
},
"key_id_2": {
// ...
}
}
}
```
and the key descriptions for the keys would be:
`m.secret_storage.key.key_id_1`:
```json
{
"name": "Some key",
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
// ... other properties according to algorithm
}
```
`m.secret_storage.key.key_id_2`:
```json
{
"name": "Some other key",
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
// ... other properties according to algorithm
}
```
#### Encryption algorithms
##### `m.secret_storage.v1.aes-hmac-sha2`
Secrets are encrypted using AES-CTR-256 and MACed using HMAC-SHA-256. The data
is encrypted and MACed as follows:
1. Given the secret storage key, generate 64 bytes by performing an HKDF with
SHA-256 as the hash, a salt of 32 bytes of 0, and with the secret name as
the info. The first 32 bytes are used as the AES key, and the next 32 bytes
are used as the MAC key
2. Generate 16 random bytes, set bit 63 to 0 (in order to work around
differences in AES-CTR implementations), and use this as the AES
initialization vector. This becomes the `iv` property, encoded using base64.
3. Encrypt the data using AES-CTR-256 using the AES key generated above. This
encrypted data, encoded using base64, becomes the `ciphertext` property.
4. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256
using the MAC key generated above. The resulting MAC is base64-encoded and
becomes the `mac` property.
(We use AES-CTR to match file encryption and key exports.)
For the purposes of allowing clients to check whether a user has correctly
entered the key, clients should:
1. encrypt and MAC a message consisting of 32 bytes of 0 as described above,
using the empty string as the info parameter to the HKDF in step 1.
2. store the `iv` and `mac` in the `m.secret_storage.key.[key ID]`
account-data.
For example, the `m.secret_storage.key.key_id` for a key using this algorithm
could look like:
```json
{
"name": "m.default",
"algorithm": "m.secret_storage.v1.aes-hmac-sha2",
"iv": "random+data",
"mac": "mac+of+encrypted+zeros"
}
```
and data encrypted using this algorithm could look like this:
```json
{
"encrypted": {
"key_id": {
"iv": "16+bytes+base64",
"ciphertext": "base64+encoded+encrypted+data",
"mac": "base64+encoded+mac"
}
}
}
```
###### Keys
When a user is given a raw key for `m.secret_storage.v1.aes-hmac-sha2`,
it will be encoded as follows (this is the same as what is proposed in MSC1703):
* prepend the two bytes 0x8b and 0x01 to the key
* compute a parity byte by XORing all bytes of the resulting string, and append
the parity byte to the string
* base58-encode the resulting byte string with the alphabet
'123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'.
* format the resulting ASCII string into groups of 4 characters separated by
spaces.
When decoding a raw key, the process should be reversed, with the exception
that whitespace is insignificant in the user's ASCII input.
###### Passphrase
A user may wish to use a chosen passphrase rather than a randomly generated
key. In this case, information on how to generate the key from a passphrase
will be stored in the `passphrase` property of the `m.secret_storage.key.[key
ID]` account-data:
```json
{
"passphrase": {
"algorithm": "m.pbkdf2",
"salt": "MmMsAlty",
"iterations": 100000,
"bits": 256
},
...
}
```
**`m.pbkdf2`**
The key is generated using PBKDF2 using the salt given in the `salt` parameter,
and the number of iterations given in the `iterations` parameter. The key size
that is generated is given by the `bits` parameter, or 256 bits if no `bits`
parameter is given.
### Sharing
Rather than (or in addition to) storing secrets on the server encrypted by a
shared key, devices can send secrets to each other, encrypted using olm.
To request a secret, a client sends a `m.secret.request` device event with `action`
set to `request` to other devices, and `name` set to the name of the secret
that it wishes to retrieve. A device that wishes to share the secret will
reply with a `m.secret.send` event, encrypted using olm. When the original
client obtains the secret, it sends a `m.secret.request` event with `action`
set to `request_cancellation` to all devices other than the one that it received the
secret from. Clients should ignore `m.secret.send` events received from
devices that it did not send an `m.secret.request` event to.
Clients MUST ensure that they only share secrets with other devices that are
allowed to see them. For example, clients SHOULD only share secrets with
the user’s own devices that are verified and MAY prompt the user to confirm sharing the
secret.
If a feature allows secrets to be stored or shared, then for consistency it
SHOULD use the same name for both the account_data event type and the `name` in
the `m.secret.request`.
#### Event definitions
##### `m.secret.request`
Sent by a client to request a secret from another device. It is sent as an
unencrypted to-device event.
- `name`: (string) Required if `action` is `request`. The name of the secret
that is being requested.
- `action`: (enum) Required. One of ["request", "request_cancellation"].
- `requesting_device_id`: (string) Required. ID of the device requesting the
secret.
- `request_id`: (string) Required. A random string uniquely identifying the
request for a secret. If the secret is requested multiple times, it should be
reused. It should also reused in order to cancel a request.
##### `m.secret.send`
Sent by a client to share a secret with another device, in response to an
`m.secret.request` event. It MUST be encrypted as an `m.room.encrypted` event,
then sent as a to-device event.
- `request_id`: (string) Required. The ID of the request that this a response to.
- `secret`: (string) Required. The contents of the secret.
## Tradeoffs
Currently, only a public/private key mechanism is defined. It may be useful to
also define a secret key mechanism.
## Potential issues
Keeping all the data and keys in account data means that it may clutter up
`/sync` requests. However, clients can filter out the data that they are not interested
in. One possibility for addressing this would be to add a flag to the account
data to indicate whether it should come down the `/sync` or not.
## Security considerations
By storing information encrypted on the server, this allows the server operator
to read the information if they manage to get hold of the decryption keys.
In particular, if the key is based on a passphrase and the passphrase can be
guessed, then the secrets could be compromised. In order to help protect the
secrets, clients should provide feedback to the user when their chosen
passphrase is considered weak, and may also wish to prevent the user from
reusing their login password.
## Conclusion
This proposal presents a common way for bits of encrypted data to be stored on
a user's homeserver for use by various features.

77
proposals/1954-remove-prev_event-from-essential-keys-list.md
Unescape Escape View File

@ -1,77 +0,0 @@
# Remove prev_content from the essential keys list
Matrix supports the concept of event redaction. The ability to redact rather
than delete is necessary because some events e.g. membership events are
essential to the protocol and _cannot_ be deleted. Therefore we do not delete
events outright and instead redact them. This involves removing all keys from
an event that are not required by the protocol. The stripped down event is
thereafter returned anytime a client or remote server requests it.
## Proposal
[The redaction algorithm](https://matrix.org/docs/spec/client_server/r0.4.0.html#redactions)
defines which keys must be retained through a redaction. Currently it lists
```prev_content``` as a key to retain, though in practice there is no need to
do so at the protocol level.
The proposal is simply to remove ```prev_content``` from the essential keys
list.
Note: the inclusion of ```prev_content``` in the essential keys list was
unintentional and should be considered a spec bug. Synapse (and other server
implementations) have not implemented the bug and already omit
```prev_content``` from redacted events.
## Tradeoffs
When sending events over federation the events are [hashed and
signed](https://matrix.org/docs/spec/server_server/r0.1.0#adding-hashes-and-signatures-to-outgoing-events),
this involves operating not only on the original event but also the redacted
form of the event. The redacted hash and redacted signed event are necessary if
the event is ever redacted in future. As a result, any change of the essential
keys list must be managed carefully. If disparate servers implement different
versions of the redaction algorithm (for a given event) attempts to send the
event over federation will fail.
We _could_ manage this change via room versioning and create a new room
version that implements this MSC. However, because the federation already
omits the ```prev_content``` key by convention, implementing this MSC only in
the new room version would mean that the entire existing federation would not
be spec compliant.
As a result it seems pragmatic to have the spec reflect reality, acknowledge
that the spec and federation have deviated and instead update the spec
retrospectively to describe the de-facto redaction algorithm.
## Potential issues
It is theoretically possible that a closed federation could exist whose servers
do follow the spec as is. This MSC would render those servers non-compliant with
the spec. On balance this seems unlikely and in the worst case those
implementors could add the change to a subsequent room version, eventually
reaching spec consistency as older room versions are deprecated.
Another scenario is that a client may redact events according to the spec as is
and persist prev_content through the redaction, thereby diverting from that on
the server(s). Client authors will have to update their code to drop
```prev_content``` - however, given that prev_content should not be used in
important calculations and/or visualisations, this ought to be a relatively
non-invasive change.
## Security considerations
A further reason to support removal of ```prev_content``` is the case where a
malicious user adds illegal or abusive content into a state event and then
overwrites that state event. The content would then be preserved through the
redaction.
Additionally, there are plenty of reasons to have security concerns over a
precedent that the federation can deviate from the spec.
## Conclusions
Removing ```prev_content``` is pragmatic response to the current situation. It
aligns the federation and the spec, and does so in a way that removes
unnecessary overhead.

189
proposals/1957-integrations-discovery.md
Unescape Escape View File

@ -1,189 +0,0 @@
# MSC1957: Integration manager discovery
**Note**: this proposal is part of a larger "Integrations API" which has not yet been defined.
See [MSC1956](https://github.com/matrix-org/matrix-doc/pull/1956) for details.
**Note**: this proposal makes use of the existing Widget API proposed by
[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236).
Users should have the freedom to choose which integration manager they want to use in their client, while
also accepting suggestions from their homeserver and client. Clients need to know where to find the different
integration managers and how to contact them.
## Proposal
A single logged in user may be influenced by zero or more integration managers at any given time. Managers
are sourced from the client's own configuration, homeserver discovery information, and the user's personal
account data in the form of widgets. Clients should support users using more than one integration manager
at a given time, although the rules for how this can be handled are defined later in this proposal.
#### Client-configured integration managers
This is left as an implementation detail. In the case of Riot, this is likely to be part of the existing
`config.json` options, although likely modified to support multiple managers instead of one.
#### Homeserver-configured integration managers
The integration managers suggested by a homeserver are done through the existing
[.well-known](https://matrix.org/docs/spec/client_server/r0.4.0.html#get-well-known-matrix-client) discovery
mechanism. The added optional fields, which should not affect a client's ability to log a user in, are:
```json
{
"m.integrations": {
"managers": [
{
"api_url": "https://integrations.example.org",
"ui_url": "https://integrations.example.org/ui"
},
{
"api_url": "https://bots.example.org"
}
]
}
}
```
As shown, the homeserver is able to suggest multiple integration managers through this method. Each manager
must have an `api_url` which must be an `http` or `https` URL. The `ui_url` is optional and if not provided
is the same as the `api_url`. Like the `api_url`, the `ui_url` must be `http` or `https` if supplied.
The `ui_url` is ultimately treated the same as a widget, except that the `data` object from the widget is not
present and must not be templated here. Variables like `$matrix_display_name` are able to function, however.
Integration managers should never use the `$matrix_user_id` as authoritative and instead seek other ways to
determine the user ID. This is covered by other proposals.
The `api_url` is the URL clients will use when *not* embedding the integration manager, and instead showing
its own purpose-built interface.
Clients should query the `.well-known` information for the homeserver periodically to update the integration
manager settings for that homeserver. The client is not expected to validate or use any other information
contained in the response. Current recommendations are to query the configuration when the client starts up
and every 8 hours after that. Clients can additionally refresh the configuration whenever they feel is
necessary (such as every time the user opens the integration manager).
#### User-configured integration managers
Users can specify integration managers in the form of account widgets. The `type` is to be `m.integration_manager`
and the content would look something similar to:
```json
{
"url": "https://integrations.example.org/ui?displayName=$matrix_display_name",
"data": {
"api_url": "https://integrations.example.org"
}
}
```
The `api_url` in the `data` object is required and has the same meaning as the homeserver-defined `api_url`.
The `url` of the widget is analogous to the `ui_url` from the homeserver configuration above, however normal
widget rules apply here.
The user is able to have multiple integration managers through use of multiple widgets.
The query string shown in the example is to demonstrate that integration managers are widgets and can
make use of the template options provided to widgets.
#### Display order of integration managers
Clients which have support for integration managers should display at least 1 manager, but should
display multiple via something like tabs. Clients must prefer to display the user's configured
integration managers over any defaults, and if only displaying one manager must pick the first
manager after sorting the `state_key`s of the applicable widgets in lexicographical order. Clients
can additionally display default managers if they so wish, and should preserve the order defined in
the various defaults. If the user has no configured integration managers, the client must prefer
to display one or more of the managers suggested by the homeserver over the managers recommended
by the client.
The client can optionally support a way to entirely disable integration manager support, even if the
user and homeserver have managers defined.
The rationale for having the client prefer to use the user's integration managers first is so that
the user can tailor their experience within Matrix if desired. Similarly, a homeserver may wish to
subject all of their users to the same common integration manager as would be common in some organizations.
The client's own preference is a last ditch effort to have an integration manager available to the
user so they don't get left out.
#### Displaying integration managers
Clients simply open the `ui_url` (or equivalent) in an `iframe` or similar. In the current ecosystem,
integration managers would receive a `scalar_token` to identify the user - this is no longer the case
and instead integration managers must seek other avenues for determining the user ID. Other proposals
cover how to do this in the context of the integrations API.
Integration managers shown in this way must be treated like widgets, regardless of source. In practice
this means exposing the Widget API to the manager and applying necessary scoping to keep the manager
as an account widget rather than a room widget.
#### Discovering a manager by only the domain name
Clients may wish to ask users for a single canonical domain name so they can find the manager to add
to the user's account transparently. This differs from the .well-known discovery which allows homeservers
to recommend their own integration manager: the homeserver is not recommending a default here. The
user has instead opted to pick an integration manager (identified only by domain name) and the client
is expected to resolve that to a set of URLs it can use for the manager.
Similar to the .well-known discovery done by servers (and clients during login), clients which have an
integrations domain (eg: "example.org") make a regular HTTPS request to
`https://example.org/.well-known/matrix/integrations` which returns an object which looks like the
following:
```json
{
"m.integrations_widget": {
"url": "https://integrations.example.org/ui?displayName=$matrix_display_name",
"data": {
"api_url": "https://integrations.example.org"
}
}
}
```
The response should be parsed as JSON. If the endpoint returns an error or is missing the `m.integrations_widget`
property, the client should assume there is no integrations manager running on that domain. The
`m.integrations_widget` is an object which has the exact same format as the account widget for
an integration manager, described above. The client should wrap the object verbatim into the appropriate
account data location.
Because the .well-known file would be accessed by web browsers, among other platforms, the server
should be using appropriate CORS headers for the request. The recommended headers are the same as those
which are already recommended for homeserver discovery in the Client-Server API.
*Note*: this could reuse the client-server mechanic for discovery and just omit the homeserver information
however that conflates many concerns together on the one endpoint. A new endpoint is instead proposed
to keep the concerns isolated.
The query string shown in the example is to demonstrate that integration managers are widgets and can
make use of the template options provided to widgets.
## Tradeoffs
We could limit the user (and by extension, the homeserver and client) to exactly 1 integration manager
and not worry about tabs or other concepts, however this restricts a user's access to integrations.
In a scenario where the user wants to use widgets from Service A and bots from Service B, they'd
end up switching accounts or clients to gain access to either service, or potentially just give up
and walk away from the problem. Instead of having the user switch between clients, we might as well
support this use case, even if it is moderately rare.
We could also define the integration managers in a custom account data event rather than defining them
as a widget. Doing so just adds clutter to the account data and risks duplicating code in clients as
using widgets gets us URL templating for free (see the section earlier on in this proposal about account
widgets for more information: "User-configured integration managers").
## Future extensions
Some things which may be desirable in the future are:
* Avatars for the different managers
* Human-readable names for the different managers
* Supporting `ui_url`s targeting specific clients for a more consistent design
## Security considerations
When displaying integration managers, clients should not trust that the input is sanitary. Per the
proposal above, an integration manager is only permitted to be served from HTTP(S) URIs. A given
integration manager can still have malicious intent however, and clients should ensure any sandboxing
on the manager is appropriate such that it can communicate with the client, but cannot perform
unauthorized actions. Other URI schemes are just as dangerous and could potentially be allowed by
this proposal - use cases are less defined and desirable for schemes like `file://` and are excluded
by this proposal. They can be added in a future proposal if a use case arises.

192
proposals/1960-integrations-openid.md
Unescape Escape View File

@ -1,192 +0,0 @@
# MSC1960: OpenID Connect information exchange for widgets
Widgets are currently left with no options to verify the user's ID, making it hard for
personalized and authenticated widgets to exist. The spec says the `$matrix_user_id`
template variable cannot be relied upon due to how easy it is to faslify, which is true.
This MSC aims to solve the problem with verifiably accurate OpenID Connect credentials.
As of writing, the best resource to learn more about the widgets spec is the following
spec PR: https://github.com/matrix-org/matrix-doc/pull/2764
## Proposal
Typically widgets which need to accurately verify the user's identity will also have a
backend service of some kind. This backend service likely already uses the integration
manager authentication APIs introduced by [MSC1961](https://github.com/matrix-org/matrix-doc/pull/1961).
Through using the same concepts from MSC1961, the widget can verify the user's identity
by requesting a fresh OpenID Connect credential object to pass along to its backend, like
the integration manager which might be running it.
The protocol sequence defined here is based upon the previous discussion in the Element Web
issue tracker: https://github.com/vector-im/element-web/issues/7153
It is proposed that after the capabilities negotiation, the widget can ask the client for
an OpenID Connect credential object so it can pass it along to its backend for validation.
The request SHOULD result in the user being prompted to confirm that the widget can have
their information. Because of this user interaction, it's not always possible for the user
to complete the approval within the 10 second suggested timeout by the widget spec. As
such, the initial request by the widget can have one of three states:
1. The client indicates that the user is being prompted (to be followed up on).
2. The client sends over credentials for the widget to verify.
3. The client indicates the request was blocked/denied.
The initial request from the widget looks as follows:
```json
{
"api": "fromWidget",
"action": "get_openid",
"requestId": "AAABBB",
"widgetId": "CCCDDD",
"data": {}
}
```
Which then receives a response which has a `state` field alongside potentially the credentials
to be verified. Matching the order of possible responses above, here are examples:
```json
{
"api": "fromWidget",
"action": "get_openid",
"requestId": "AAABBB",
"widgetId": "CCCDDD",
"data": {},
"response": {
"state": "request"
}
}
```
```json
{
"api": "fromWidget",
"action": "get_openid",
"requestId": "AAABBB",
"widgetId": "CCCDDD",
"data": {},
"response": {
"state": "allowed",
"access_token": "s3cr3t",
"token_type": "Bearer",
"matrix_server_name": "example.org",
"expires_in": 3600
}
}
```
```json
{
"api": "fromWidget",
"action": "get_openid",
"requestId": "AAABBB",
"widgetId": "CCCDDD",
"data": {},
"response": {
"state": "blocked"
}
}
```
The credential information is directly copied from the `/_matrix/client/r0/user/:userId/openid/request_token`
response.
In the case of `state: "request"`, the user is being asked to approve the widget's attempt to
verify their identity. To ensure that future requests are quicker, clients are encouraged to
include a "remember this widget" option to make use of the immediate `state: "allowed"` or
`state: "blocked"` responses above.
There is no timeout associated with the user making their selection. Once a user does make
a selection (allow or deny the request), the client sends a `toWidget` request to indicate the
result, using a very similar structure to the above immediate responses:
```json
{
"api": "toWidget",
"action": "openid_credentials",
"requestId": "EEEFFF",
"widgetId": "CCCDDD",
"data": {
"state": "allowed",
"original_request_id": "AAABBB",
"access_token": "s3cr3t",
"token_type": "Bearer",
"matrix_server_name": "example.org",
"expires_in": 3600
}
}
```
```json
{
"api": "toWidget",
"action": "openid_credentials",
"requestId": "EEEFFF",
"widgetId": "CCCDDD",
"data": {
"state": "blocked",
"original_request_id": "AAABBB"
}
}
```
`original_request_id` is the `requestId` of the `get_openid` request which started the prompt,
for the widget's reference.
The widget acknowledges receipt of the credentials with an empty `response` object.
A typical sequence diagram for this flow is as follows:
```
+-------+ +---------+ +---------+
| User | | Client | | Widget |
+-------+ +---------+ +---------+
| | |
| | Capabilities negotiation |
| |----------------------------------------->|
| | |
| | Capabilities negotiation |
| |<-----------------------------------------|
| | |
| | fromWidget get_openid request |
| |<-----------------------------------------|
| | |
| | ack with state "request" |
| |----------------------------------------->|
| | |
| Ask if the widget can have information | |
|<--------------------------------------------| |
| | |
| Approve | |
|-------------------------------------------->| |
| | |
| | toWidget openid_credentials request |
| |----------------------------------------->|
| | |
| | acknowledge request (empty response) |
| |<-----------------------------------------|
```
Prior to this proposal, widgets could use an undocumented `scalar_token` parameter if the client chose to
send it to the widget. Clients typically chose to send it if the widget's URL matched a whitelist for URLs
the client trusts. With the widget specification as written, widgets cannot rely on this behaviour.
Widgets may wish to look into cookies and other storage techniques to avoid continuously requesting
credentials. Widgets should also look into [MSC1961](https://github.com/matrix-org/matrix-doc/pull/1961)
for information on how to properly verify the OpenID Connect credentials it will be receiving. The
widget is ultimately responsible for how it deals with the credentials, though the author recommends
handing it off to an integration manager's `/register` endpoint to acquire a single token string
instead.
An implementation of this proposal's early draft is here: https://github.com/matrix-org/matrix-react-sdk/pull/2781
## Security considerations
The user is explicitly kept in the loop to avoid automatic and silent harvesting of private information.
Clients must ask the user for permission to send OpenID Connect information to a widget, but may optionally allow
the user to always allow/deny the widget access. Clients are encouraged to notify the user when future
requests are automatically handled due to the user's prior selection (eg: an unobtrusive popup saying
"hey, your sticker picker asked for your information. [Block future requests]").

73
proposals/1961-integrations-auth.md
Unescape Escape View File

@ -1,73 +0,0 @@
# MSC1961: Integration manager authentication
A set of common APIs needs to be defined for clients to be able to interact with an integration
manager. This proposal covers the authentication portion of that API.
**Note**: this proposal is part of a larger "Integrations API" which has not yet been defined.
See [MSC1956](https://github.com/matrix-org/matrix-doc/pull/1956) for details.
## Proposal
All specified APIs (except `/register`) will take an `Authorization` header with a `Bearer` token returned
from a call to `/register`. This token is used to authorize the request and to identify who is making the
request. The token may also be specified as the `access_token` query string parameter, similar to the
Client-Server API.
#### POST `/_matrix/integrations/v1/account/register`
Exchanges an OpenID object for a token which can be used to authorize future requests to the manager.
Request body is an OpenID object as returned by `/_matrix/client/r0/user/:userId/openid/request_token`.
Response is:
```json
{
"token": "OpaqueString"
}
```
The token should consist of URL-safe base64 characters. Integration managers should be careful to validate
the OpenID object by ensuring the `/_matrix/federation/v1/openid/userinfo` response has a `sub` which belongs
to the `matrix_server_name` provided in the original OpenID object.
Applications which register for a token are responsible for tracking which integration manager they are for.
This can usually be done by tracking the hostname of the integration manager and matching a token with it.
#### GET `/_matrix/integrations/v1/account`
Gets information about the token's owner, such as the user ID for which it belongs.
Besides a token, no other information is required for the request.
Response is:
```json
{
"user_id": "@alice:example.org"
}
```
The `user_id` is the user ID which was represented in the OpenID object provided to `/register`. Integration
managers may be interested in also supplying information about the user's credit balance for paid integrations
here. Preferably, custom information is stored under a namespaced key like so:
```json
{
"user_id": "@alice:example.org",
"org.example.paid.integrations": {
"credit": 20000
}
}
```
#### POST `/_matrix/integrations/v1/account/logout`
Logs the token out, rendering it useless for future requests.
Request body is an empty object. Response body is also an empty object if successful.
## Security considerations
Clients should be sure to call `/logout` where possible when the user is logging out or no longer needs access
to a given manager. Clients should additionally be cautious about which managers they register for tokens with,
as some integration managers may be untrusted.

39
proposals/1983-leave-reasons.md
Unescape Escape View File

@ -1,39 +0,0 @@
# Proposal to add reasons for leaving a room
This proposal intends to solve [#1295](https://github.com/matrix-org/matrix-doc/issues/1295). Currently
people can be kicked from a room for a human-readable reason, but when voluntarily leaving the `reason`
is not considered.
## Proposal
Like when kicking someone, `m.room.member` events with `membership: leave` can have a string `reason`.
The reason is a human-readable string of text which clients should display to users of the room, providing
users with the ability to leave a room for a reason of their choosing. The field is optional.
Having such a field gives Matrix better compatibility with other networks such as IRC which use the leave
reason to communicate connection problems ("ping timeout", etc). Although Matrix doesn't have a need for a
persistent connection to remain in the room, having a reason can improve the bridge experience.
An example of a partial leave event with reason would be:
```json
{
"type": "m.room.member",
"sender": "@travis:t2l.io",
"state_key": "@travis:t2l.io",
"content": {
"membership": "leave",
"reason": "I'm in too many rooms and never talk here"
}
}
```
## Potential issues
There is a chance that quality of communication may deteriorate as a result of adding this field. Arguments
in rooms may result in the leave reason having personal attacks against other room members or rude remarks.
Moderators would be expected to handle these situations for these cases, including redacting the state event
if needed.
This also opens up an avenue for spam by having a very large reason for leaving. Clients are encouraged to
trim/hide the reason at a reasonable length.

54
proposals/2002-rooms-v4.md
Unescape Escape View File

@ -1,54 +0,0 @@
# MSC 2002 - Rooms V4
This MSC proposes creating a new room version named v4 to allow servers to switch
event ID grammar to that proposed in MSC1884.
## Proposal
The new room version is called "4". The only difference between v4 and v3 is
that v4 rooms use the grammar for defining event IDs as proposed in MSC1884 -
expressing event IDs as url-safe base64 rather than plain base64 for the
convenience of client implementors.
It is not proposed that servers change the default room version used when
creating new rooms, and it is not proposed that servers recommend upgrading
existing rooms to v4.
## Rationale and Context
We would like to default to creating rooms with a reasonably secure room
algorithm in upcoming Matrix 1.0. We do not want to default to creating v3
rooms due to the inconvenience the v3 event ID grammar might cause, so instead
we are proposing favouring v4.
Ideally we would also include other room algorithm changes in v4 (e.g.
honouring server signing key validity periods, as per
https://github.com/matrix-org/synapse/issues/4364), but as spec &
implementation work is still ongoing there, we are proposing using v4 as a
room version which can be supported in the wild before Matrix 1.0 and then
used as the initial default for creating rooms. The expectation is for the
versions of the spec which coincide with 1.0 to also support v5 rooms, but in
practice v5 will not be marked as default until it has sufficient adoption on
the public network.
The expectation is never to recommend upgrading existing
rooms to v4, but instead v5 (once ready), to avoid overburdening room admins
with upgrade notifications.
To conclude, the proposed plan is:
1. Define room v4 as MSC1884
2. Introduce servers with v4 support into the public network as rapidly as possible
3. Wait for enough servers to speak v4. Meanwhile:
1. Define an MSC for honouring server signing key validity periods
2. Implement this MSC
3. Define room v5 as this MSC
4. Release Matrix 1.0, defining room v4 as the new default for creating rooms,
but also shipping support for room v5 for the first time.
5. Wait for enough servers to speak v5 rooms.
6. Define room v5 as the new default for creating rooms.
7. Define room versions prior to v5 as unsafe, thus prompting users to upgrade their
rooms to v5.
The reason we don't wait for v5 to be widespread before releasing 1.0 is to avoid
delaying the 1.0 yet further. It is good enough for 1.0 to support v5 without it
also being the default for creating rooms.

77
proposals/2010-spoilers.md
Unescape Escape View File

@ -1,77 +0,0 @@
# MSC 2010: Proposal to add client-side spoilers
Sometimes, while you want to put text into a spoiler to not have people accidentally read things that they don't want to see.
For example, when discussing a new movie or a TV series, not everyone might have watched it yet.
In such cases it would make sense to add a spoiler so that only those who have seen the movie or
don't mind spoilers read the content.
Another example would be e.g. in mental health communities where certain people have certain
triggers. People could put talking about abuse or the like into a spoiler, to not accidentally
trigger anyone just reading along the conversation.
Furthermore this is helpful for bridging to other networks that already have a spoiler feature.
To render the spoiler the content is hidden and then revealed once interacted somehow
(e.g. a click / hover).
## Proposal
This proposal is about adding a new attribute to the `formatted_body` of messages with type
`m.room.message` and message types which support the `org.matrix.custom.html` format.
It adds a new attribute, `data-mx-spoiler`, to the `<span>` tag. If the attribute is present the
contents of the span tag should be rendered as a spoiler. Optionally, you can specify a reason for
the spoiler by setting the attribute string. It could be rendered, for example, similar to this:
![Spoiler rendering idea](images/2010-spoiler-example.gif)
The plaintext fallback supported by the `body` is optional. A recommendation for clients is included
below.
To preserve the semantics of a spoiler in the plaintext fallback it is recommended to upload the contents of the spoiler
as a text file and then link this: `[Spoiler](mxc://someserver/somefile)` and
`[Spoiler for reason](mxc://someserver/somefile)` respectively.
### Example
Without reason:
```
{
"msgtype": "m.text",
"format": "org.matrix.custom.html",
"body": "Hello there, the movie was [spoiler](mxc://someserver/somefile)",
"formatted_body": "Hello there, the movie was <span data-mx-spoiler>awesome</span>"
}
```
With reason:
```
{
"msgtype": "m.text",
"format": "org.matrix.custom.html",
"body": "Hey [Spoiler for movie](mxc://someserver/somefile)",
"formatted_body": "Hey <span data-mx-spoiler="movie">the movie was awesome</span>"
}
```
## Tradeoffs
Instead of making this an attribute, an entirely new tag could be introduced (e.g. `<mx-spoiler>`),
however that wouldn't be HTML-compliant.
Instead of limiting the proposed `data-mx-spoiler` attribute only to the `<span>`-tag it could be
added to all tags, however it might make implementations for clients more complicated.
Alternatively the [details](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) tag could
be used. This, however, is a block element, and the spoilers are span elements. Furthermore
semantically there is a slight difference: with the details tag you hide something for a person
as it uses up a lot of screen space, while with a spoiler you hide something as a person might not
want to see it.
## Potential issues
Depending on context it might make sense to put other events, such as `m.image`, into spoilers,
too. This MSC doesn't address that at all. Using
`<span data-mx-spoiler><img src="mxc://server/media"></span>` seems rather sub-optimal for that.
This MSC doesn't take HTML block elements into account.
Clients would have to come up with a way to input spoilers. This could be done, for example,
by adding a custom markdown tag (like discord does), so that you do `Text ||spoiler||`, however
that doesn't take a spoiler reason into account.
## Security considerations
The spoiler reason needs to be properly escaped when rendered.

30
proposals/2033-whoami-device-id.md
Unescape Escape View File

@ -1,30 +0,0 @@
# Proposal to include device IDs in `/account/whoami`
There are some use cases (namely using
[Pantalaimon with bots](https://github.com/matrix-org/pantalaimon/issues/14))
which could benefit from knowing the `device_id` associated with a token.
## Proposal
The `/account/whoami` endpoint receives an additional response field for the `device_id`
associated with the access token. The field is optional because appservice users may not
have a real device associated with them. Non-appservice users should always have a device
associated with them.
Access tokens are already associated with at most 1 device, and devices are associated with
exactly 1 access token. Because of this, we do not need to worry about multiple devices
causing problems. For more information, see
https://matrix.org/docs/spec/client_server/r0.4.0.html#relationship-between-access-tokens-and-devices
*Note*: Pantalaimon would likely require a `device_id` be returned and error requests
otherwise. This should be considered expected behaviour by Pantalaimon in the MSC author's
opinion.
## Tradeoffs
We could introduce a `/device/whoami` endpoint, however that is a less superior option. Most
calls to `/device/whoami` would additionally need to call `/account/whoami` to determine the
user ID of the account. We had might as well bundle the two pieces of information into the
same request.

42
proposals/2076-enforce-validity-periods.md
Unescape Escape View File

@ -1,42 +0,0 @@
# MSC2076: Enforce key-validity periods when validating event signatures
## Background
The [Federation API
specification](https://matrix.org/docs/spec/server_server/r0.1.1.html#validating-hashes-and-signatures-on-received-events)
specifies that events should be validated via the signature verification
algorithm, but does not specify how the keys for that check should be obtained
and validated.
In practice, the implementation has been as follows. The receiving server
first requests a copy of the key via the [`GET /_matrix/key/v2/server/`
API](https://matrix.org/docs/spec/server_server/r0.1.1.html#get-matrix-key-v2-server-keyid)
directly from the server which created the signature, or via the [`POST
/_matrix/key/v2/query` API](https://matrix.org/docs/spec/server_server/r0.1.1.html#post-matrix-key-v2-query)
from a trusted key server. Once such a key is obtained, it is then cached
forever. No check is made on the `valid_until_ts` field, and
`minimum_valid_until_ts` is set to zero for calls to `POST
/_matrix/key/v2/query`.
This is highly unsatisfactory, as it means that, should a key be compromised,
then an attacker can spoof arbitrary events claiming to be from the compromised
server forever, since there is no revocation mechanism.
## Proposal
This MSC proposes to enforce the `valid_until_ts` property when validating
event signatures. In particular, the server must ensure that it has a copy of
the key with a `valid_until_ts` at least as large as the `origin_server_ts` of
the event being validated. If it does not have such a copy, it must try to
obtain one via the `GET /_matrix/key/v2/server/` or `POST
/_matrix/key/v2/query` APIs. For the latter, it must set
`minimum_valid_until_ts` to prompt the notary server to attempt to refresh the
key if appropriate.
Since this changes the rules used to validate events, it will be introduced
with a new room version. This will reduce the risk of divergence between
servers in a room due to some servers accepting events which others reject.
This MSC also proposes that the current situation - where `valid_until_ts` is
ignored - be formalised for the existing room versions v1-v4, rather than be
left as implementation-specific behaviour.

19
proposals/2077-rooms-v5.md
Unescape Escape View File

@ -1,19 +0,0 @@
# MSC2077 - Room version 5
This MSC proposes creating room version 5, which will enforce the signing key
`valid_until_ts` timestamps proposed in
[MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076).
## Proposal
The new room version is called `5`. The only difference between v5 and v4 is
that v5 rooms enforce the `valid_until_ts` timestamp on signing keys as
proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076).
It is not yet proposed to change the default room version to v5. Version 5 will
be considered a "stable" version.
## Notes
See also [MSC2002](./2002-rooms-v4.md), which proposed room v4 but also
mentioned that a v5 was anticipated and gave some context for this change.

154
proposals/2078-homeserver-password-resets.md
Unescape Escape View File

@ -1,154 +0,0 @@
# MSC2078 - Sending Third-Party Request Tokens via the Homeserver
This MSC proposes removing the current requirement of the identity server to
send third-party request tokens, and allows homeservers to implement the
functionality instead. These request tokens are used to verify the identity of
the request author as an owner of the third-party ID (3PID). This can be used
for binding a 3PID to an account, or for resetting passwords via email or SMS.
The latter is what this proposal mainly focuses on, but be aware that it allows
for any task that requires requesting a token for a 3PID to be taken on by the
homeserver instead of the identity server.
The intention is to put less trust in the identity server, which is currently
one of the most centralised components of Matrix. As it stands, an attacker in
control of a identity server can reset a user's password if the identity server
is considered trusted by that homeserver, and the user has registered at least
one 3PID. This is due to the identity server handling the job of confirming the
user's control of that identity.
The MSC seeks to clarify that homeservers can take on the responsibility of
sending password reset tokens themselves, and a new response field that will
aid homeservers in doing so.
# Background
Currently when a client requests a 3PID token, it makes a call to one of the
`/requestToken` endpoints on the homeserver. For instance, during password
resets, a token is requested from either
[/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken)
or
[/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken),
depending on the medium of the 3PID. These requests are supplied all the
necessary details as well as an `id_server` field containing the domain address
of a identity server trusted by the homeserver.
In order to facilitate these requests, the homeserver will simply proxy them to
the identity server. The IS will send out a token via email or sms, the user
will click a link or enter the token into their client, and either the client
or the user's browser will make a request **directly to the identity server**
with the token for verification. The IS then informs the homeserver that
verification was successful. At this point you can likely see that there is
potential for abuse here, so instead Homeservers should be given the option to
stop proxying the request to the identity server, and instead just send and
validate the token themselves.
## Proposal
The homeserver should be allowed to either proxy `/requestToken` requests or
handle them itself. Specifically, this means that the homeserver can both send
password reset tokens (via email or SMS), as well as accept requests on an
arbitrary endpoint (with the same parameters as
[/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken))
to verify that token.
One additional complication that in the case of SMS, just a code is sent to a
person's phone. This is then given to the client, but the client may not know
where to send the code now, as it doesn't know whether the homeserver or
identity server generated it.
In order to combat this problem, the field `submit_url` MUST be added in the
response from all of the variants of `/requestToken` in the Client-Server API,
if and only if the verification message contains a code the user is expected to
enter into the client (for instance in the case of a short code through SMS).
This URL is simply where the client should submit this token. The endpoint
should accept the same parameters as
[/_matrix/identity/api/v1/validate/{3pid_type}/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken)
in the Identity Service API. The only recommendation to homeserver developers
for this endpoint's path is to not be exactly the same as that of the identity
server, in order to prevent clashes between setups running both an identity
server and homeserver on the same domain. If `submit_url` is omitted, the
client MUST continue the same behaviour from before, which is to send the token
to the identity server directly. This is intended for backwards compatibility
with older servers.
If the client receives a response to `/requestToken` with `submit_url`, it MUST
accept a token from user input, then make a POST request to the content of
`submit_url` with the `sid`, `client_secret` and user-entered token.
`submit_url` can lead to anywhere the homeserver deems necessary for
verification. To be clear the content of `id_server` does not matter here, the
client should just submit a POST request to the value of `submit_url`. Additionally
data MUST be submitted as a JSON body.
An example exchange from the client's perspective is shown below:
```
POST https://homeserver.tld/_matrix/client/r0/account/password/email/requestToken
{
"client_secret": "monkeys_are_AWESOME",
"email": "alice@homeserver.tld",
"send_attempt": 1,
"id_server": "id.example.com"
}
```
If the server responds with a `submit_url` field, it means the client should
collect a token from the user and then submit it to the provided URL.
```
{
"sid": "123abc",
"submit_url": "https://homeserver.tld/_homeserver/password_reset/msisdn/submitToken"
}
```
Since a `submit_url` was provided, the client will now collect a token from the
user, say "123456", and then submit that as a POST request to the
`"submit_url"`.
```
POST https://homeserver.tld/_homeserver/password_reset/msisdn/submitToken
{
"sid": "123abc",
"client_secret": "monkeys_are_AWESOME",
"token": "123456"
}
```
The client will then receive an appropriate response:
```
{
"success": true
}
```
If the client did not receive a `submit_url` field, they should instead assume
that verification will be completed out of band (e.g. the user clicks a link in
their email and makes the submitToken request with their web browser).
## Dismissed Alternatives
Consideration was taken not to make `id_server` an optional field. Let's
assume for a moment that it was optional. Now, a client could send a request to
`/requestToken` omitting the `id_server` field. The homeserver however has
opted to continue proxying `/requestToken` to the identity server, even though
it knows this is potentially insecure. The homeserver now has no idea which
identity server to proxy the request to, and must return a failure to the
client. The client could then make another request with an `id_server`, but
we've now made two requests that ended up in the same outcome, instead of one,
in hopes of saving a very small amount of bandwidth by omitting the field
originally.
At some point we should look into removing the `id_server` field altogether and
removing any email/SMS message sending from the identity server. This would
drastically reduce the amount of trust needed in the identity server and its
required ability. This is, however, a good first step.
## Tradeoffs
If homeservers choose to not proxy the request, they will need to implement the
ability to send emails and/or SMS messages. This is left as a detail for the
homeserver implementation.

454
proposals/2134-identity-hash-lookup.md
Unescape Escape View File

@ -1,454 +0,0 @@
# MSC2134: Identity Hash Lookups
[Issue #2130](https://github.com/matrix-org/matrix-doc/issues/2130) has been
created in response to a security issue brought up by an independent party.
To summarise the issue, when a user wants to ask an identity server which of
its contacts have registered a Matrix account, it performs a lookup against
an identity server. The client currently sends all of its contact details in
the form of plain-text addresses, meaning that the identity server can
identify and record every third-party ID (3PID) of the user's contacts. This
allows the identity server to collect email addresses and phone numbers that
have a high probability of being connected to a real person. This data could
then be used for marketing, political campaigns, etc.
However, if these email addresses and phone numbers are hashed before they are
sent to the identity server, the server would have a more difficult time of
being able to recover the original addresses. This prevents contact
information of non-Matrix users being exposed to the lookup service.
Yet, hashing is not perfect. While reversing a hash is not possible, it is
possible to build a [rainbow
table](https://en.wikipedia.org/wiki/Rainbow_table), which maps known email
addresses and phone numbers to their hash equivalents. When the identity
server receives a hash, it is then be able to look it up in its rainbow table
and find the corresponding 3PID. To prevent this, one would use a hashing
algorithm such as [bcrypt](https://en.wikipedia.org/wiki/Bcrypt) with many
rounds, making the construction of a large rainbow table an infeasibly
expensive process. Unfortunately, this is impractical for our use case, as it
would require clients to also perform many, many rounds of hashing, linearly
dependent on the size of their address book, which would likely result in
lower-end mobile phones becoming overwhelmed. We are then forced to use a
fast hashing algorithm, at the cost of making rainbow tables easy to build.
The rainbow table attack is not perfect, because one does need to know email
addresses and phone numbers to build it. While there are only so many
possible phone numbers, and thus it is relatively inexpensive to generate the
hash value for each one, the address space of email addresses is much, much
wider. If your email address does not use a common mail server, is decently long
or is not publicly known to attackers, it is unlikely that it would be
included in a rainbow table.
Thus the approach of hashing, while adding complexity to implementation and
resource consumption of the client and identity server, does provide added
difficulty for the identity server to carry out contact detail harvesting,
which should be considered worthwhile.
## Proposal
This proposal suggests making changes to the Identity Service API's lookup
endpoints, consolidating them into a single `/lookup` endpoint. The endpoint
is to be on a `v2` path, to avoid confusion with the original `v1` `/lookup`.
The `/api` part is also dropped in order to preserve consistency across other
endpoints:
- `/_matrix/identity/v2/lookup`
A second endpoint is added for clients to request information about the form
the server expects hashes in.
- `/_matrix/identity/v2/hash_details`
The following back-and-forth occurs between the client and server.
Let's say the client wants to check the following 3PIDs:
```
alice@example.com
bob@example.com
carl@example.com
+1 234 567 8910
denny@example.com
```
The client will hash each 3PID as a concatenation of the medium and address,
separated by a space and a pepper, also separated by a space, appended to the
end. Note that phone numbers should be formatted as defined by
https://matrix.org/docs/spec/appendices#pstn-phone-numbers, before being
hashed). Note that "pepper" in this proposal simply refers to a public,
opaque string that is used to produce different hash results between identity
servers. Its value is not secret.
First the client must append the medium (plus a space) to the address:
```
"alice@example.com" -> "alice@example.com email"
"bob@example.com" -> "bob@example.com email"
"carl@example.com" -> "carl@example.com email"
"+1 234 567 8910" -> "12345678910 msisdn"
"denny@example.com" -> "denny@example.com email"
```
Hashes must be peppered in order to reduce both the information an identity
server gains during the process, and attacks the client can perform. [0]
In order for clients to know the pepper and hashing algorithm they should use,
identity servers must make the information available on the `/hash_details`
endpoint:
```
GET /_matrix/identity/v2/hash_details
{
"lookup_pepper": "matrixrocks",
"algorithms": ["sha256"]
}
```
The name `lookup_pepper` was chosen in order to account for pepper values
being returned for other endpoints in the future. The contents of
`lookup_pepper` MUST match the regular expression `[a-zA-Z0-9]+`, whether
hashing is being performed or not. When no hashing is occurring, a valid
pepper value of at least length 1 is still required.
If hashing, the client appends the pepper to the end of the 3PID string,
after a space.
```
"alice@example.com email" -> "alice@example.com email matrixrocks"
"bob@example.com email" -> "bob@example.com email matrixrocks"
"carl@example.com email" -> "carl@example.com email matrixrocks"
"12345678910 msdisn" -> "12345678910 msisdn matrixrocks"
"denny@example.com email" -> "denny@example.com email matrixrocks"
```
Clients can cache the result of this endpoint, but should re-request it
during an error on `/lookup`, to handle identity servers which may rotate
their pepper values frequently. Clients MUST choose one of the given
`algorithms` values to hash the 3PID during lookup.
Clients and identity servers MUST support SHA-256 as defined by [RFC
4634](https://tools.ietf.org/html/rfc4634), identified by the value
`"sha256"` in the `algorithms` array. SHA-256 was chosen as it is currently
used throughout the Matrix spec, as well as its properties of being quick to
hash.
There are certain situations when an identity server cannot be expected to
compare hashed 3PID values; for example, when a server is connected to a
backend provider such as LDAP, it is not efficient for the identity server to
pull all of the addresses and hash them upon lookup. For this case, identity
servers can also support receiving plain-text 3PID addresses from clients. To
agree upon this, the value `"none"` can be added to the `"algorithms"` array
of `GET /hash_details`. The client can then choose to send plain-text values
by setting the `"algorithm"` value in `POST /lookup` to `"none"`.
No hashing nor peppering will be performed if the client and server decide on
`"none"`, and 3PIDs will be sent in plain-text, similar to the v1 `/lookup`
API. When this occurs, it is STRONGLY RECOMMENDED for the client to prompt
the user before continuing.
When performing a lookup, the pepper and hashing algorithm the client used
must be part of the request body (even when using the `"none"` algorithm
value). If they do not match what the server has on file (which may be the
case if the pepper was changed right after the client's request for it), then
the server must inform the client that they need to query the hash details
again, as opposed to just returning an empty response, which clients would
assume to mean that no contacts are registered on that identity server.
If the algorithm is not supported by the server, the server should return a `400
M_INVALID_PARAM`. If the pepper does not match the server's, the server should
return a new error code, `400 M_INVALID_PEPPER`. A new error code is not
defined for an unsupported algorithm as that is considered a client bug.
The `M_INVALID_PEPPER` error response contains the correct `algorithm` and
`lookup_pepper` fields. This is to prevent the client from needing to query
`/hash_details` again, thus saving a request. `M_INVALID_PARAM` does not
include these fields. An example response to an incorrect pepper would be:
```
{
"error": "Incorrect value for lookup_pepper",
"errcode": "M_INVALID_PEPPER",
"algorithm": "sha256",
"lookup_pepper": "matrixrocks"
}
```
Now comes time for the lookup. We'll first cover an example of the client
choosing the `"sha256"` algorithm. Note that the resulting hash digest MUST
be encoded in URL-safe unpadded base64 (similar to [room version 4's event
IDs](https://matrix.org/docs/spec/rooms/v4#event-ids)). Once hashing has been
performed, the client sends each hash in an array.
```
"alice@example.com email matrixrocks" -> "4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc"
"bob@example.com email matrixrocks" -> "LJwSazmv46n0hlMlsb_iYxI0_HXEqy_yj6Jm636cdT8"
"carl@example.com email matrixrocks" -> "jDh2YLwYJg3vg9pEn3kaaXAP9jx-LlcotoH51Zgb9MA"
"12345678910 msisdn matrixrocks" -> "S11EvvwnUWBDZtI4MTRKgVuiRx76Z9HnkbyRlWkBqJs"
"denny@example.com email matrixrocks" -> "2tZto1arl2fUYtF6tQPJND69il3xke9OBlgFgnUt2ww"
POST /_matrix/identity/v2/lookup
{
"addresses": [
"4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc",
"LJwSazmv46n0hlMlsb_iYxI0_HXEqy_yj6Jm636cdT8",
"jDh2YLwYJg3vg9pEn3kaaXAP9jx-LlcotoH51Zgb9MA",
"S11EvvwnUWBDZtI4MTRKgVuiRx76Z9HnkbyRlWkBqJs",
"2tZto1arl2fUYtF6tQPJND69il3xke9OBlgFgnUt2ww"
],
"algorithm": "sha256",
"pepper": "matrixrocks"
}
```
The identity server, upon receiving these hashes, can simply compare against
the hashes of the 3PIDs it stores. The server then responds with the Matrix
IDs of those that match:
```
{
"mappings": {
"4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc": "@alice:example.com",
"S11EvvwnUWBDZtI4MTRKgVuiRx76Z9HnkbyRlWkBqJs": "@fred:example.com"
}
}
```
The client can now display which 3PIDs link to which Matrix IDs.
For the case of the identity server sending, and the client choosing,
`"none"` as the algorithm, we would do the following.
The client would first make `GET` a request to `/hash_details`, perhaps
receiving the response:
```
{
"lookup_pepper": "matrixrocks",
"algorithms": ["none", "sha256"]
}
```
The client decides that it would like to use `"none"`, and thus ignores the
lookup pepper, as no hashing will occur. Appending a space and the 3PID
medium to each address is still necessary:
```
"alice@example.com" -> "alice@example.com email"
"bob@example.com" -> "bob@example.com email"
"carl@example.com" -> "carl@example.com email"
"+1 234 567 8910" -> "12345678910 msisdn"
"denny@example.com" -> "denny@example.com email"
```
The client then sends these off to the identity server in a `POST` request to
`/lookup`:
```
POST /_matrix/identity/v2/lookup
{
"addresses": [
"alice@example.com email",
"bob@example.com email",
"carl@example.com email",
"12345678910 msisdn",
"denny@example.com email"
],
"algorithm": "none",
"pepper": "matrixrocks"
}
```
Note that even though we haven't used the `lookup_pepper` value, we still
include the same one sent to us by the identity server in `/hash_details`.
The identity server should still return `400 M_INVALID_PEPPER` if the pepper
is incorrect. This simplifies things and can help ensure the client is
requesting `/hash_details` properly before each lookup request.
Finally, the identity server will check its database for the Matrix user IDs
it has that correspond to these 3PID addresses, and returns them:
```
{
"mappings": {
"alice@example.com email": "@alice:example.com",
"12345678910 msisdn": "@fred:example.com"
}
}
```
No parameter changes will be made to
[/bind](https://matrix.org/docs/spec/identity_service/r0.2.1#post-matrix-identity-api-v1-3pid-bind)
as part of this proposal.
## Fallback considerations
`v1` versions of these endpoints may be disabled at the discretion of the
implementation, and should return a `403 M_FORBIDDEN` error if so.
If an identity server is too old and a HTTP 400 or 404 is received when
accessing the `v2` endpoint, clients should fallback to the `v1` endpoint
instead. However, clients should be aware that plain-text 3PIDs are required
for the `v1` endpoints, and are strongly encouraged to warn the user of this.
## Tradeoffs
* There is a small cost incurred by performing hashes before requests, but this
is outweighed by the privacy implications of sending plain-text addresses.
## Security Considerations
Hashes are still reversible with a rainbow table, but the provided pepper,
which can be rotated by identity servers at will, should help mitigate this.
Phone numbers (with their relatively short possible address space of 12
numbers), short email addresses at popular domains, and addresses of both
types that have been leaked in database dumps are more susceptible to hash
reversal.
Mediums and peppers are appended to the address as to prevent a common prefix
for each plain-text string, which prevents attackers from pre-computing the
internal state of the hash function.
## Other considered solutions
Bloom filters are an alternative method of providing private contact discovery.
However, they do not scale well due to requiring clients to download a large
filter that needs updating every time a new bind is made.
Further considered solutions are explored in
https://signal.org/blog/contact-discovery/. Signal's eventual solution of
using Software Guard Extensions (detailed in
https://signal.org/blog/private-contact-discovery/) is considered impractical
for a federated network, as it requires specialised hardware.
k-anonymity was considered as an alternative approach, in which the identity
server would never receive a full hash of a 3PID that it did not already know
about. Discussion and a walk-through of what a client/identity-server
interaction would look like are documented [in this Github
comment](https://github.com/matrix-org/matrix-doc/pull/2134#discussion_r298691748).
While this solution seems like a win for privacy, its actual benefits are a
lot more nuanced. Let's explore them by performing a threat-model analysis:
We consider three attackers:
1. A malicious third party trying to discover the identity server mappings
in the homeserver.
The malicious third party scenario can only be protected against by rate
limiting lookups, given otherwise it looks identical to legitimate traffic.
1. An attacker who has stolen an IS db
In theory the 3PIDs could be stored hashed with a static salt to protect
a stolen DB. This has been descoped from this MSC, and is largely an
orthogonal problem.
1. A compromised or malicious identity server, who may be trying to
determine the contents of a user's addressbook (including non-Matrix users)
Our approaches for protecting against a malicious identity server are:
* We resign ourselves to the IS knowing the 3PIDs at point of bind, as
otherwise it can't validate them.
* To protect the 3PIDs of non-Matrix users:
1. We could hash the uploaded 3PIDs with a static pepper; however, a
malicious IS could pre-generate a rainbow table to reverse these hashes.
1. We could hash the uploaded 3PIDs with a slowly rotating pepper; a
malicious IS could generate a rainbow table in retrospect to reverse these
hashes (but wouldn't be able to reuse the table)
1. We could send partial hashes of the uploaded 3PIDs (with full salted
hashes to disambiguate the 3PIDs), have the IS respond with anonymised
partial results, to allow the IS to avoid reversing the 3PIDs (a
k-anonymity approach). However, the IS could still claim to have mappings
for all 3PIDs, and so receive all the salted hashes, and be able to
reverse them via rainbow tables for that salt.
So, in terms of computational complexity for the attacker, respectively:
1. The attacker has to generate a rainbow table over all possible IDs once,
which can then be reused for subsequent attacks.
1. The attacker has to generate a rainbow table over all possible IDs for a
given lookup timeframe, which cannot be reused for subsequent attacks.
1. The attacker has to generate multiple but partial rainbow tables, one
per group of 3PIDs that share similar hash prefixes, which cannot then be
reused for any other attack.
For making life hardest for an attacker, option 3 (k-anon) wins. However, it
also makes things harder for the client and server:
* The client has to calculate new salted hashes for all 3PIDs every time it
uploads.
* The server has to calculate new salted hashes for all partially-matching
3PIDs hashes as it looks them up.
It's worth noting that one could always just go and load up a malicious IS DB
with a huge pre-image set of mappings and thus see what uploaded 3PIDs match,
no matter what algorithm is used.
For k-anon this would put the most computational onus on the server (as it
would effectively be creating a partial rainbow table for every lookup), but
this is probably not infeasible - so we've gone and added a lot of complexity
and computational cost for not much benefit, given the system can still be
trivially attacked.
Finally, as more and more users come onto Matrix, their contact lists will
get more and more exposed anyway given the IS server has to be able to
identity Matrix-enabled 3PIDs to perform the lookup.
Thus the conclusion is that while k-anon is harder to attack, it's unclear
that this is actually enough of an obstacle to meaningfully stop a malicious
IS. Therefore we should KISS and go for a simple hash lookup with a rotating
pepper (which is not much harder than a static pepper, especially if our
initial implementation doesn't bother rotating the pepper). Rather than
trying to make the k-anon approach work, we'd be better off spending that
time figuring out how to store 3pids as hashes in the DB (and in 3pid
bindings etc), or how to decentralise ISes in general. It's also worth noting
that a malicious server may fail to rotate the pepper, making the rotation
logic of questionable benefit.
A radical model was also considered where the first portion of the
k-anonyminity scheme was done with an identity server, and the second would
be done with various homeservers who originally reported the 3PID to the
identity server. While interesting and more decentralised, some attacks are
still possible if the identity server is running an evil homeserver which it
can direct the client to send its hashes to. Discussion on this matter has
taken place in the MSC-specific room [starting at this
message](https://matrix.to/#/!LlraCeVuFgMaxvRySN:amorgan.xyz/$4wzTSsspbLVa6Lx5cBq6toh6P3TY3YnoxALZuO8n9gk?via=amorgan.xyz&via=matrix.org&via=matrix.vgorcum.com).
Tangentially, identity servers would ideally just never receive plain-text
addresses, just storing and receiving hash values instead. However, it is
necessary for the identity server to have plain-text addresses during a
[bind](https://matrix.org/docs/spec/identity_service/r0.2.1#post-matrix-identity-api-v1-3pid-bind)
call, in order to send a verification email or sms message. It is not
feasible to defer this job to a homeserver, as the identity server cannot
trust that the homeserver has actually performed verification. Thus it may
not be possible to prevent plain-text 3PIDs of registered Matrix users from
being sent to the identity server at least once. Yet, it is possible that with
a few changes to other Identity Service endpoints, as described in [this
review
comment](https://github.com/matrix-org/matrix-doc/pull/2134#discussion_r309617900),
identity servers could refrain from storing any plaintext 3PIDs at rest. This
however, is a topic for a future MSC.
## Conclusion
This proposal outlines a simple method to stop bulk collection of user's
contact lists and their social graphs without any disastrous side effects. All
functionality which depends on the lookup service should continue to function
unhindered by the use of hashes.
## Footnotes
[0] Clients would have to generate a full rainbow table specific to the set
pepper to obtain all registered MXIDs, while the server would have to
generate a full rainbow table with the specific pepper to get the plaintext
3pids for non-matrix users.

348
proposals/2140-terms-of-service-2.md
Unescape Escape View File

@ -1,348 +0,0 @@
# MSC2140: Terms of Service API for Identity Servers and Integration Managers
*Note*: This MSC was added to in [MSC2264](https://github.com/matrix-org/matrix-doc/pull/2264)
[MSC1692](https://github.com/matrix-org/matrix-doc/issues/1692) introduces a
method for homeservers to require that users read and agree to certain
documents before being permitted to use the service. This proposal introduces a
corresponding method that can be used with Identity Servers and Integration
Managers.
Requirements for this proposal are:
* ISes and IMs should be able to give multiple documents a user must agree to
abide by
* Each document should be versioned
* ISes and IMs must, for each request that they handle, know that the user
making the request has agreed to their data being used. This need not be
absolute proof (we will always have to trust that the client actually
showed the document to the user) but it must be reasonably demonstrable that
the user has given informed consent for the client to use that service.
* ISes and IMs must be able to prevent users from using the service if they
have not provided agreement.
* A user should only have to agree to each version of each document once for
their Matrix ID, ie. having agreed to a set of terms in one client, they
should not have to agree to them again when using a different client.
* Documents should be de-duplicated between services. If two or more services
are hosted by the same organisation, the organisation should have the
option to give their users a single document that encompasses both services
(bearing in mind that the user must be able to opt-out of components of a
service whilst still being able to use the service without that component).
Identity Servers do not currently require any kind of user login to access the
service and so are unable to track what users have agreed to what terms in the
way that Homeservers do.
## Proposal
Throuhgout this proposal, $prefix will be used to refer to the prefix of the
API in question, ie. `/_matrix/identity/v2` for the IS API and
`/_matrix/integrations/v1` for the IM API.
Note the removal of the `/api` prefix and migration to v2 in the IS API
following convention from
[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134).
This proposal introduces:
* A v2 API prefix, with authentication, for the Identity Service
* The `$prefix/terms` endpoint
* The `m.accepted_terms` section in account data
* `POST /_matrix/client/r0/account/3pid/unbind` endpoints on the client/server
API
This proposal removes:
* The `bind_email` and `bind_msisdn` on the Homeserver `/register` endpoint
This proposal relies on both Integration Managers and Identity Servers being
able to identify users by their MXID and store the fact that a given MXID has
indicated that they accept the terms given. Integration Managers already
identify users in this way by authenticating them using the OpenID endpoint on
the Homeserver. This proposal introduces the same mechanism to Identity Servers
and adds authentication across the Identity Service API.
### IS API Authentication
All current endpoints within `/_matrix/identity/api/v1/` will be duplicated
into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of
lookups. Authentication is still expected on MSC2134's proposed endpoints.
Support for `application/x-form-www-urlencoded` parameters in requests will
be dropped from all endpoints.
Any request to any endpoint within `/_matrix/identity/v2`, with the exception
of:
* `/_matrix/identity/v2`
* `/_matrix/identity/v2/pubkey/*`
* The new `$prefix/account/register` endpoint
* The new `GET /_matrix/identity/v2/terms`
* `$prefix/account/logout`
...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401.
This indicates that the user must authenticate with OpenID and supply a valid
`access_token`.
Clients authenticate either via an `Authorization` header with a `Bearer` token
or an `access_token` query parameter.
The existing endpoints under `/_matrix/identity/api/v1/` continue to be
unauthenticated but will be deprecated. ISes may support the old v1 API for as
long as they wish. Once ISes remove support for the old APIs, those endpoints
must return HTTP Status 404. Clients must update to use the v2 API as soon as
possible.
OpenID authentication in the IS API will work the same as in the Integration Manager
API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961).
When clients supply an identity server to the Homeserver in order for the
Homeserver to make calls to the IS on its behalf, it must also supply its
access token for the Identity Server alongside in the `id_access_token` key of
the same JSON object. That is, in the main request object for `requestToken`
and `/_matrix/client/r0/rooms/{roomId}/invite` requests and in the
`threepidCreds` object when supplying 3PID credentials (eg. in the
`m.email.identity` UI auth stage). The server must also relay
`M_TERMS_NOT_SIGNED` errors back to the client. Exceptions to this are any
requests where the only IS operation the Homeserver may perform is unbinding,
ie. `/_matrix/client/r0/account/deactivate` and
`/_matrix/client/r0/account/3pid/delete`, in which case the unbind will be
authenticated by a signed request from the Homeserver.
### HS Register API
The `bind_email` and `bind_msisdn` options to `/_matrix/client/r0/register` in
the client/server API will be removed. Due to the fact that
`/_matrix/identity/v2/3pid/bind` requires authentication, it will no longer be
possible for the Homeserver to bind 3PIDs as part of the registration process.
### IS Register API
The following new APIs will be introduced to support OpenID auth as per
[MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961):
* `/_matrix/identity/v2/account/register`
* `/_matrix/identity/v2/account`
* `/_matrix/identity/v2/account/logout`
Note again the removal of the `/api` prefix and migration to v2 following
convention from
[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134).
### Terms API
New API endpoints will be introduced:
#### `GET $prefix/terms`:
This returns a set of documents that the user must agree to abide by in order
to use the service. Its response is similar to the structure used in the
`m.terms` UI auth flow of the Client/Server API:
```json
{
"policies": {
"terms_of_service": {
"version": "2.0",
"en": {
"name": "Terms of Service",
"url": "https://example.org/somewhere/terms-2.0-en.html"
},
"fr": {
"name": "Conditions d'utilisation",
"url": "https://example.org/somewhere/terms-2.0-fr.html"
}
},
"privacy_policy": {
"version": "1.2",
"en": {
"name": "Privacy Policy",
"url": "https://example.org/somewhere/privacy-1.2-en.html"
},
"fr": {
"name": "Politique de confidentialité",
"url": "https://example.org/somewhere/privacy-1.2-fr.html"
}
}
}
}
```
Each document (ie. key/value pair in the 'policies' object) MUST be
uniquely identified by its URL. It is therefore strongly recommended
that the URL contains the version number of the document. The name
and version keys, however, are used only to provide a human-readable
description of the document to the user.
This endpoint does *not* require authentication.
#### `POST $prefix/terms`:
Requests to this endpoint have a single key, `user_accepts` whose value is
a list of URLs (given by the `url` field in the GET response) of documents that
the user has agreed to:
```json
{
"user_accepts": ["https://example.org/somewhere/terms-2.0-en.html"]
}
```
This endpoint requires authentication.
The clients MUST include the correct URL for the language of the document that
was presented to the user and they agreed to. Servers should accept agreement
of any one language of each document as sufficient, regardless of what language
a client is operating in: users should not have to re-consent to documents if
they change their client to a different language.
The server responds with an empty JSON object. The server must not assume that
the client will agree to all documents in a single request.
### Accepted Terms Account Data
This proposal also defines the `m.accepted_terms` section in User Account
Data in the client/server API that clients SHOULD use to track what sets of
terms the user has consented to. This has an array of URLs under the 'accepted'
key to which the user has agreed to.
An `m.accepted_terms` section therefore resembles the following:
```json
{
"accepted": [
"https://example.org/somewhere/terms-1.2-en.html",
"https://example.org/somewhere/privacy-1.2-en.html"
]
}
```
Whenever a client submits a `POST $prefix/terms` request to an IS or IM or
completes an `m.terms` flow on the HS (or as soon as possible afterwards, ie.
after registration is complete), it SHOULD update this account data section
adding any the URLs of any additional documents that the user agreed to to this
list.
### Terms Acceptance in the API
Before any requests are made to an Identity Server or Integration Manager,
the client must use the `GET $prefix/terms` endpoint to fetch the set of
documents that the user must agree to in order to use the service.
It then cross-references this set of documents against the `m.accepted_terms`
account data and presents to the user any documents that they have not already
agreed to, along with UI for them to indicate their agreement. If there are no
such documents (ie. if the `policies` dict is empty or the user has already
agreed to all documents) the client proceeds to perform the OpenID
registration. If there are new terms documents, the client prompts the user for
agreement, then once the user has indicated their agreement, it adds these URLs
to `m.accepted_terms` account data and then proceeds with OpenID
authentication, getting a token from the Homeserver and submitting this to the
service using the `register` endpoint.
Having done this, if the user agreed to any new documents, it performs a `POST
$prefix/terms` request to signal to the server the set of documents that the
user has agreed to.
Any request to any endpoint in the IM API, and the `/_matrix/identity/v2/`
namespace of the IS API, with the exception of `/_matrix/identity/v2` itself,
may return:
* `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that
the user must authenticate with OpenID and supply a valid `access_token`.
* `M_TERMS_NOT_SIGNED` errcode with HTTP status code 403. This indicates
that the user must agree to (new) terms in order to use or continue to
use the service.
The `/_matrix/identity/v2/3pid/unbind` endpoint must not return either of these
errors if the request has a valid signature from a Homeserver, and is being authenticated as such.
In summary, the process for using a service that has not previously been used
in the current login session is:
* `GET $prefix/terms`
* Compare result with `m.accepted_terms` account data, get set of documents
pending agreement.
* If non-empty, show this set of documents to the user and wait for the user
to indicate their agreement.
* Add the newly agreed documents to `m.accepted_terms`.
* On success, or if there were no documents pending agreement, get an OpenID
token from the Homeserver and submit this token to the `register` endpoint.
Store the resulting access token.
* If the set of documents pending agreement was non-empty, Perform a
`POST $prefix/terms` request to the service with these documents.
### `POST /_matrix/client/r0/account/3pid/unbind`
A client uses this client/server API endpoint to request that the Homeserver
removes the given 3PID from the given Identity Server, or all Identity Servers.
Takes the same parameters as
`POST /_matrix/client/r0/account/3pid/delete`, ie. `id_server`, `medium`,
`address`. Similar to the other unbind endpoints, this endpoint does not
require an `id_access_token` because the homeserver can only unbind.
Returns the same as `POST /_matrix/client/r0/account/3pid/delete`.
Clients may add IS bindings for 3PIDs that already exist on the user's
Homeserver account by using the `POST /_matrix/client/r0/account/3pid`
to re-add the 3PID.
### Unstable feature flag for transition
In order to allow client implementations to determine if the homeserver they are developed
against supports `id_access_token`, an unstable feature flag of `m.id_access_token`
is to be added to `/versions`. When the flag is `false` or not present, clients must assume
that the homeserver does not support being given `id_access_token` and may receive an error
for doing so. Clients are expected to use the supported specification versions the homeserver
advertises instead of the feature flag's presence once this proposal is included in a release
of the specification.
## Tradeoffs
The Identity Service API previously did not require authentication, and OpenID
is reasonably complex, adding a significant burden to both clients and servers.
A custom HTTP header was also considered that could be added to assert that the
client agrees to a particular set of terms. We decided against this in favour
of re-using existing primitives that already exist in the Matrix ecosystem.
Custom HTTP headers are not used anywhere else within Matrix. This also gives a
very simple and natural way for ISes to enforce that users may only bind 3PIDs
to their own MXIDs.
This introduces a different way of accepting terms from the client/server API
which uses User-Interactive Authentication. In the client/server API, the use
of UI auth allows terms acceptance to be integrated into the registration flow
in a simple and backwards-compatible way. Another option here would be to use
UI Auth on the register endpoint. This would also not allow users to register
before accepting the terms. However, this would then make the OpenID
registration process different and non-standard.
The `m.accepted_terms` section contains only URLs of the documents that
have been agreed to. This loses information like the name and version of
the document, but:
* It would be up to the clients to copy this information correctly into
account data.
* Having just the URLs makes it much easier for clients to make a list
of URLs and find documents not already agreed to.
## Potential issues
This change deprecates all v1 endpoints and so will require clients to update
to continue working.
## Security considerations
Requiring authentication on the IS API means it will no longer be possible to
use it anonymously.
It is assumed that once servers publish a given version of a document at a
given URL, the contents of that URL will not change. This could be mitigated by
identifying documents based on a hash of their contents rather than their URLs.
Agreement to terms in the client/server API makes this assumption, so this
proposal aims to be consistent.
## Conclusion
This proposal adds an error response to all endpoints on the API and a custom
HTTP header on all requests that is used to signal agreement to a set of terms
and conditions. The use of the header is only necessary if the server has no
other means of tracking acceptance of terms per-user. The IS API is not
authenticated so ISes will have no choice but to use the header. The IM API is
authenticated so IMs may either use the header or store acceptance per-user.
A separate endpoint is specified with a GET request for retrieving the set
of terms required and a POST to indicate that the user consents to those
terms.

17
proposals/2174-move-redacts-key.md
Unescape Escape View File

@ -1,17 +0,0 @@
# MSC2174: move the `redacts` property to `content`
[`m.room.redaction`](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-redaction)
events currently have an *event-level* property `redacts` which gives the event
ID of the event being redacted.
The presence of this field at the event level, rather than under the `content`
key, is anomalous. This MSC proposes that, in a future room version, the
`redacts` property be moved under the `content` key.
For backwards-compatibility with *older* clients, servers should add a `redacts`
property to the top level of `m.room.redaction` events in *newer* room versions
when serving such events over the Client-Server API.
For improved compatibility with *newer* clients, servers should add a `redacts`
property to the `content` of `m.room.redaction` events in *older* room versions
when serving such events over the Client-Server API.

17
proposals/2175-remove-creator-field.md
Unescape Escape View File

@ -1,17 +0,0 @@
# MSC2175: Remove the `creator` field from `m.room.create` events
[`m.room.create`](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-create)
events have a mandatory `creator` property giving the ID of the user who
created the room. This field is redundant as it is always identical to the
`sender` of the create event.
This MSC proposes that, in a future room version, this field should be removed
and that the `sender` field be used instead.
Note that `creator` is mentioned in the [auth
rules](https://matrix.org/docs/spec/rooms/v1#authorization-rules). It can
safely be removed.
`creator` is also mentioned as a key to be preserved during [Event
redaction](https://matrix.org/docs/spec/client_server/r0.5.0#redactions). It
should be removed from that list.

97
proposals/2176-update-redaction-rules.md
Unescape Escape View File

@ -1,97 +0,0 @@
# MSC2176: Update the redaction rules
The current [redaction
algorithm](https://matrix.org/docs/spec/client_server/r0.5.0#redactions) is now
somewhat dated. This MSC proposes a number of changes to the rules which will
improve the security and reliability of the Matrix protocol.
## Proposal
The following changes will require a new room version, since changes to the
redaction algorithm also change the way that [event
hashes](https://matrix.org/docs/spec/server_server/r0.1.2#calculating-the-reference-hash-for-an-event)
(and hence event IDs) are calculated.
The following *event* keys are to be *removed* from the list of those to be
preserved by a redaction:
* `membership`
* `prev_state`
(Note this refers to the *event-level* `membership` property, rather than the
similarly-named sub-property under the `content` key.)
Rationale: neither of the above properties have defined meanings any more in the Matrix
protocol, so there is no reason for them to be special-cased in this way.
The following are to be added to the list of subkeys of the content property
which are preserved:
* `m.room.create` preserves *all* content. Rationale: the values in a
`create` event are deliberately intended to last the lifetime of the room,
and if values are redacted, there is no way to add correct settings
afterwards. It therefore seems non-sensical to allow redaction of a `create`
event.
* `m.room.redaction` should allow the `redacts` key (assuming
[MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174) is merged).
Rationale: currently, redacting a redaction can lead to inconsistent results
among homservers, depending on whether they receive the `m.room.redaction`
result before or after it is redacted (and therefore may or may not redact
the original event).
* `m.room.power_levels` should allow (in addition to the keys already listed
in the spec):
* the `invite` key. Rationale: this is required to authenticate
`m.room.member` events with the `invite` membership. Currently, redacting
a `power_levels` event will mean that such events cannot be authenticated,
potentially leading to a split-brain room.
## Other properties considered for preservation
Currently it is *not* proposed to add these to the list of properties which are
proposed for a redaction:
* The `notifications` key of `m.room.power_levels`. Unlike the other
properties in `power_levels`, `notifications` does not play a part in
authorising the events in the room graph. Once the `power_levels` are
replaced, historical values of the `notifications` property are
irrelevant. There is therefore no need for it to be protected from
redactions.
* The `algorithm` key of `m.room.encryption`. Again, historical values of
`m.room.encryption` have no effect, and servers do not use the value of the
property to authenticate events.
The effect of redacting an `m.room.encryption` event is much the same as that
of sending a new `m.room.encryption` event with no `algorithm` key. It's
unlikely to be what was intended, but adding rules to the redaction
algorithm will not help this.
### Background to things not included in the proposal
The approach taken here has been to minimise the list of properties preserved
by redaction; in general, the list is limited to those which are required by
servers to authenticate events in the room. One reason for this is to simplify
the implementation of servers and clients, but a more important philosophical
reason is as follows.
Changing the redaction algorithm requires changes to both servers and clients,
so changes are difficult and will happen rarely. Adding additional keys now
sets an awkward precedent.
It is likely that in the future more properties will be defined which might be
convenient to preserve under redaction. One of the two scenarios would then
happen:
* We would be forced to issue yet more updates to the redaction algorithm,
with a new room versions and mandatory updates to all servers and clients, or:
* We would end up with an awkward asymmetry between properties which were
preserved under this MSC, and those which were introduced later so were not
preserved.
In short, I consider it important for the elegance of the Matrix protocol that
we do not add unnecessary properties to the list of those to be preserved by
redaction.

50
proposals/2181-user-deactivated-errcode.md
Unescape Escape View File

@ -1,50 +0,0 @@
# Add an Error Code for Signaling a Deactivated User
Currently, when a user attempts to log in, they will receive an `M_FORBIDDEN`
error code if their password is incorrect. However, if the user's account is
deactivated, they will also receive an `M_FORBIDDEN`, leaving clients in a
state where they are unable to inform the user that the reason they cannot
log in is that their account has been deactivated. This leads to confusion
and password resetting which ultimately results in frustration.
## Proposal
This proposal asks to create a new error code, `M_USER_DEACTIVATED`, that may
be returned whenever an action is attempted that requires an activited user,
but the authenticating user is deactivated. The HTTP code to return alongside
is `403`.
An example of this could be returning `M_USER_DEACTIVATED` on `/login`, when
an identifier of a deactivated user is sent to the homeserver. Whether the
password has to be correct depends on whether the Homeserver implementation
removes login information on deactivation. This is an implementation detail.
It should be noted that this proposal is not requiring implementations to
return `M_USER_DEACTIVATED` on any endpoints when a request from a
deactivated user appears. Instead it is simply defining the new error code,
recommends that it should be used in situations as described above, and that
the client should understand the meaning of it when it is received.
## Tradeoffs
The alternative is to continue returning an `M_FORBIDDEN`, but send back a
different error message. This is undesirable as clients are supposed to treat
the message as an opaque string, and should not be performing any
pattern-matching on it.
## Potential issues
None
## Security considerations
While the existence of a user was already public knowledge (one can check if
the User ID is available through
[/_matrix/client/r0/register/available](https://matrix.org/docs/spec/client_server/r0.5.0#get-matrix-client-r0-register-available),
this proposal would allow any user to be able to detect if a registered
account has been deactivated, depending on the homeserver's implementation.
## Conclusion
Adding `M_USER_DEACTIVATED` would better inform clients about the state of a
user's account, and lead to less confusion when they cannot log in.

45
proposals/2184-allow-html-details.md
Unescape Escape View File

@ -1,45 +0,0 @@
# Allow the HTML `<details>` tag in messages
Currently, there's no available method for bot developers - among others - to provide larger informative
messages in a room without disrupting the conversation for all other users. This often causes bots
to appear needlessly "spammy".
## Proposal
This proposal suggests adding the existing HTML tags for `<details>` and `<summary>` to the list of
allowed tags in formatted Matrix messages. Which would allow for larger messages to be shown simply
as smaller - and less intrusive - summaries for the users who are not interested in their full contents.
## Tradeoffs
An alternative method to provide a summary/details split could possibly be done through [MSC1767],
with the details and summaries being specified through repeated bodies with added metadata. This could
then also allow clients better autonomy in deciding what to display - or how to structure the information.
However, allowing the use of the `<details>` and `<summary>` tags would still offer richer formatting
capabilities even in such messages, especially as more than one detail/summary block could be included
in a single message.
## Potential issues
Allowing more HTML tags in formatted messages could cause more work for client developers, as they would
have to fit a larger and more diverse corpus of input into their designs and user experience.
However, these are both well documented - and implemented - HTML tags, so there is plenty of prior work
available to take example from in how to incorporate them.
Additionally, as the addition of these tags will make it possible to fit even more information into
a single message without worry of overflowing the room, any client that doesn't render the formatting
of the body might end up with a lessened user experience - from either an under- or overflow of information.
The onus on ensuring that the unformatted body is a reasonable representation of the message has always
been on the user or bot writing the formatted message though, so providing an improved ability for
formatting should not negatively affect the experience for any clients that simply render unformatted
text.
## Security considerations
Allowing more HTML tags in client rendering could lead to a wider attack surface for DOM-based exploits.
However, these tags are very simple in both function and design, so any possible attack surface they
would offer would be minimal at best.
[MSC1767]: https://github.com/matrix-org/matrix-doc/blob/matthew/msc1767/proposals/1767-extensible-events.md

155
proposals/2197-search_filter_in_federation_publicrooms.md
Unescape Escape View File

@ -1,155 +0,0 @@
# MSC2197 – Search Filtering in Public Room Directory over Federation
This MSC proposes introducing the `POST` method to the `/publicRooms` Federation
API endpoint, including a `filter` argument which allows server-side filtering
of rooms.
We are motivated by the opportunity to make searching the public Room Directory
more efficient over Federation.
## Motivation
Although the Client-Server API includes the filtering capability in
`/publicRooms`, the Federation API currently does not.
This leads to a situation that is wasteful of effort and network traffic for
both homeservers; searching a remote server involves first downloading its
entire room list and then filtering afterwards.
## Proposal
Having a filtered `/publicRooms` API endpoint means that irrelevant or
uninteresting rooms can be excluded from a room directory query response.
In turn, this means that these room directory query responses can be generated
more quickly and then, due to their smaller size, transmitted over the network
more quickly.
These benefits have been utilised in the Client-Server API, which implements
search filtering using the `filter` JSON body parameter in the `POST` method on
the `/publicRooms` endpoint.
Ignoring the `server` parameter in the Client-Server API, the following specific
differences are noticed between the Client-Server and Federation API's
`/publicRooms` endpoints:
* the Federation API endpoint only accepts the `GET` method whereas the
Client-Server API accepts the `POST` method as well.
* the Federation API accepts `third_party_instance_id` and
`include_all_networks` parameters through the `GET` method, whereas the
Client-Server API only features these in the `POST` method.
This MSC proposes to introduce support for the `POST` method in the Federation
API's `/publicRooms` endpoint, with all but one of the parameters from that of
the Client-Server API.
The parameter that is intentionally omitted is the `server` query parameter, as
it does not make sense to include it – the requesting homeserver could make a
direct request instead of requesting that a request be relayed.
The parameters which are copied, however, shall have the same semantics as
they do in the Client-Server API.
In the interest of clarity, the proposed parameter set is listed below, along
with a repetition of the definitions of used substructures. The response format
has been omitted as it is the same as that of the current Client-Server and
Federation APIs, which do not differ in this respect.
### `POST /_matrix/federation/v1/publicRooms`
#### Query Parameters
There are no query parameters. Notably, we intentionally do not inherit the
`server` query parameter from the Client-Server API.
#### JSON Body Parameters
* `limit` (`integer`): Limit the number of search results returned.
* `since` (`string`): A pagination token from a previous request, allowing
clients to get the next (or previous) batch of rooms. The direction of
pagination is specified solely by which token is supplied, rather than via an
explicit flag.
* `filter` (`Filter`): Filter to apply to the results.
* `include_all_networks` (`boolean`): Whether or not to include all known
networks/protocols from application services on the homeserver.
Defaults to false.
* `third_party_instance_id` (`boolean`): The specific third party
network/protocol to request from the homeserver.
Can only be used if `include_all_networks` is false.
### `Filter` Parameters
* `generic_search_term` (`string`): A string to search for in the room metadata,
e.g. name, topic, canonical alias etc. (Optional).
## Tradeoffs
An alternative approach might be for implementations to carry on as they are but
also cache (and potentially index) remote homeservers' room directories.
This would not require a spec change.
However, this would be unsatisfactory because it would lead to outdated room
directory results and/or caches that provide no benefit (as room directory
searches are generally infrequent enough that a cache would be outdated before
being reused, on small – if not most – homeservers).
## Potential issues
### Backwards Compatibility
After this proposal is implemented, outdated homeservers will still exist which
do not support the room filtering functionality specified in this MSC. In this
case, homeservers will have to fall-back to downloading the entire room
directory and performing the filtering themselves, as currently happens.
This is not considered a problem since it will not lead to a situation that is
any worse than the current one, and it is expected that large homeservers
– which cause the most work with the current search implementations –
would be quick to upgrade to support this feature once it is available.
As the `POST` method was not previously accepted on the `/publicRooms` endpoint
over federation, then requesting servers should fall back to the old behaviour,
if one of the following errors is encountered:
- an HTTP `400` response with an `M_UNRECOGNIZED` standard error response
`errcode` (this is what would be typically expected in this situation)
- a `404` (Not Found) HTTP error response
- a `405` (Method Not Allowed) HTTP error response
## Security considerations
There are no known security considerations.
## Privacy considerations
At present, remote homeservers do not learn about what a user has searched for.
However, under this proposal, in the context of using the Federation API to
forward on queries from the Client-Server API, a client's homeserver would end
up sharing the client's search terms with a remote homeserver, which may not be
operated by the same party or even trusted. For example, users' search terms
could be logged.
The privacy implications of this proposal are not overly major, as the data
that's being shared is [\[1\]][1]:
- only covered by GDPR if:
- the search terms contain personal data, or
- the user's homeserver IP address or domain name is uniquely identifying
(because it's a single-person homeserver, perhaps)
- likely to be *expected* to be shared with the remote homeserver
[1]: https://github.com/matrix-org/matrix-doc/pull/2197#issuecomment-517641751
For the sake of clarity, clients are strongly encouraged to display a warning
that a remote search will take the user's data outside the jurisdiction of their
own homeserver, before using the `server` parameter of the Client-Server API
`/publicRooms`, as it can be assumed that this will lead to the server invoking
the Federation API's `/publicRooms` – on the specified remote server – with the
user's search terms.
## Conclusion
By allowing homeservers to pass on search filters, we enable remote homeservers'
room directories to be efficiently searched, because, realistically speaking,
only the remote homeserver is in a position to be able to perform search
efficiently, by taking advantage of indexing and other such optimisations.

77
proposals/2209-auth-rules-other-keys-in-m.room.power.levels.md
Unescape Escape View File

@ -1,77 +0,0 @@
# Update auth rules to check notifications key in m.room.power_levels
## Introduction
The key `notifications` was added to the `m.room.power_levels` event after the
finalisation of the auth rules specified in room version 1. This leads to the
fact, that this dictionary is not subject to the same validation as other
dictionaries in the event, such as `users` or `events`. This especially means
that Alice is able to alter any entry within the dictionary including ones,
that are above her own power level, which is inconsistent with the behaviour
for the other two dictionaries.
[m.room.power_levels](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-power-levels)
[room version 1](https://matrix.org/docs/spec/rooms/v1)
## Proposal
The auth rules regarding `m.room.power_levels` have been established in room
version 1. The general idea here was that creators of a new `m.room.power_levels`
event are only able to make changes to something that is equal to or lower than
their own power level.
So, assuming a room with Alice (50), Bob (100), `m.room.power_levels` at 50,
`m.room.name` at 75 and `m.room.topic` at 25 would mean the following:
* Alice CAN alter `m.room.topic` to any power level up to her own, in this case 50
* Alice is NOT able to alter `m.room.name` since the current value is higher than
her own (75 > 50)
* Alice is NOT able to alter the power level of Bob, since his current value is
higher than her own (100 > 50)
* Alice is free to set the level for anything that has not been defined such as
`org.alice.message` up to a maximum of 50
Later on the key `notifications` was added to the `m.room.power_levels` event.
It contains a mapping of notification keys to a power level, that is required
for a user to trigger the specific notification. The most famous notification
type is the `@room` notification.
Going back to the original example because this key was added later on, the auth
rules make no mention of it, which enables the following behaviour. *It is assumed
that `@room` is at 75*
* Alice can add any key to the dictionary and set the value to any value she wants,
including ones higher than her own.
* Alice can alter the value for `@room` to any value she wants, including ones that
are higher than her own, even though her own value is lower.
The proposed solution is to alter the auth rules and add the `notifications` dictionary
to the same rules that handle `events` and `users`.
So the rule [10.d](https://matrix.org/docs/spec/rooms/v1.html) of the auth rules in
room version 1 would be updated in a new room version to:
> For each entry being added, changed or removed in events, users __and notifications__
>keys:
## Tradeoffs
The proposed solution would be a breaking change with current room versions and
the alternative would be to leave the `notifications` key without any checks.
## Security considerations
This is likely to improve security because it prevents malicious users that were
only given the right to emit `m.room.power_levels` so that they could alter a very
specific key, such as `invite`, from altering the rules established for triggering
notifications.
## Conclusion
The current spec conform behaviour regarding `notifications` is inconsistent with
behaviour shown by the other dictionaries and room administrators are very likely
expecting the `notifications` to work the same as them. The required change is minimal
is and also in line with the general spirit of the auth rules regarding the
`m.room.power_levels` event. A new room version is, however, required. This can be
done with other pending changes.

112
proposals/2229-rebind-existing-3pid.md
Unescape Escape View File

@ -1,112 +0,0 @@
# Allowing 3PID Owners to Rebind
## Note: This MSC has been made obsolete by MSC2290.
MSC2290 provides two separate API endpoints, one for adding a 3PID to the
homeserver, and another for binding to an identity server. These new
endpoints will allow the homeserver to enforce rules on emails that already
exist on the homeserver, only when modifying homeserver email, while only
needing to forward requests when binding to an identity server. This removes
the problem MSC2229 is trying to solve, and it is thus made obsolete.
---
```
3PID
noun
A "third-party identifier" such as an email address or phone number, that
can be tied to your Matrix ID in order for your contacts outside of
Matrix to find you, typically with the help of an identity server.
Identity server
noun
A queryable server that holds mappings between 3PIDs and Matrix IDs.
Bind
verb
Create a mapping between a 3PID and a Matrix ID. Useful for people to
find a user based on their existing third-party contact information.
```
As part of the on-going privacy work, Matrix client applications are
attempting to make the concept of an identity server clearer to the user, as
well as allowing a user to interact with multiple identity servers while
logged in. In facilitating this, Matrix clients should be able to allow
logged-in users the ability to pick an identity server, see what 3PIDs they
currently have bound to their Matrix ID, and bind/unbind addresses as they
desire.
When implementing this functionality, a technicality in the spec was found
to prevent certain abilities for a user. A user could not add a 3PID to their
homeserver before binding it to an identity server. It also prevents users
from binding the same 3PID to multiple identity servers. The line "The
homeserver must check that the given email address is **not** already
associated with an account on this homeserver." appears under the [POST
/_matrix/client/r0/account/3pid/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-email-requesttoken)
endpoint description. The same goes for the [equivalent msisdn (phone)
endpoint](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-msisdn-requesttoken).
When a user adds an email to their account on their homeserver, they can
choose to bind that email to an identity server at the same time. This is
specified through a `bind` boolean. If the user first adds the 3PID with
`bind: false`, then decides they want to bind that 3PID to an identity server
to make themselves discoverable by it, by making another request with `bind:
true`, the homeserver will reject the second request, because this 3PID is
already tied to the user's account.
Similarly, when a user initially sends their 3PID with `bind: true` through a
homeserver to identity server A, the homeserver keeps a record and attaches
the address to the local account. If the user then switches to identity
server B to try and do the same, the homeserver will reject the second
request as this address has already been bound.
## Proposal
This proposal calls for allowing 3PID owners to rebind their 3PIDs using the
[`POST
/_matrix/client/r0/account/3pid/email/requestToken`](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-email-requesttoken)
and [`POST
/_matrix/client/r0/account/3pid/msisdn/requestToken`](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-msisdn-requesttoken)
endpoints by extending the definition of what homeservers should check before
rejecting a bind.
Homeservers should reject the binding of a 3PID if it has already been bound,
**unless** the requesting user is the one who originally bound that 3PID. If
so, then they should be able to bind it again and again if they so choose.
In doing so, users would be able to rebind their 3PIDs, even if the
homeserver has already been made aware of it.
## Tradeoffs
Identity servers will still let 3PIDs be rebound to another Matrix ID, while
a single homeserver won't let a 3PID transition between two users. If one
thinks about typical internet services however, you aren't allowed to simply
take an email address from another account even if you have control of it, so
this shouldn't be too unintuitive.
## Potential issues
Newer clients will expect homeservers to allow them to switch between
identity servers and bind/rebind emails as they please. If dealing with an
older homeserver, clients will receive an `HTTP 400 M_THREEPID_IN_USE`.
Clients should be prepared to understand that this may just mean they are
dealing with an old homeserver, versus the 3PID already being bound on this
homeserver by another user.
## Security considerations
None.
## Conclusion
By lifting the restriction of not allowing a user to bind a 3PID multiple
times, we allow the basic ability of publishing a 3PID after associating it
with an account, as well as allow users to interact with multiple identity
servers on the same account with the same 3PIDs. This not only allows the
user to play around and gain a better understanding of the purpose of an
identity server, but it is also one step towards further decentralisation in
the identity server space.

71
proposals/2230-identity-server-account-data.md
Unescape Escape View File

@ -1,71 +0,0 @@
# MSC2230: Store Identity Server in Account Data
The URL of the Identity Server to use is currently specified at registration and
login time and then used for the lifetime of a login session. If users wish to
specify a custom one, they must do so each time they log in on every client.
Once they have chosen an Identity Server to advertise their 3PIDs on, it would
be normal that they would wish to continue using this Identity Server for all
Identity requests in their account across all clients. This proposal aims to
make this easier.
## Proposal
The base URL of the Identity Server is to be stored in user account data. It
shall be stored in the same format as in a .well-known file under the event type
`m.identity_server` and shall comprise a single key, `base_url` which is the
base URL of the ID Server to use (that is, the part before `/_matrix`, including
`https://`).
Upon registration or login, a client SHOULD refrain from performing any requests
to the Identity Server until the account data has been fetched from the server.
Once it has the account data, it SHOULD check for the presence of the
`m.identity_server` key. If present, the `base_url` in this key SHOULD be used
as the Identity Server base URL for the duration of the login session. If this
key is not present, the client SHOULD use whatever value it would have used prior
to this MSC. It should not update the account data in this situation.
Client SHOULD listen for changes in the `m.identity_server` account data value
and update the URL that they use for ID Server requests accordingly.
Clients can offer a way for users to change the ID server being used. If they
do, the client MUST update the value of `m.identity_server` accordingly.
The `m.identity_server` may be present with a `base_url` of `null`. In this case,
clients MUST treat this as no ID Server URL being set and not perform ID
Server requests, disabling any functionality that requires such requests.
Conversely, if a user wishes to disable ID Server functionality, the client
shall action this by setting the `base_url` of the `m.identity_server`
account data entry to `null`.
### Transition Period
Clients will continue to use whatever IS URLs they currently use until the
user sets one explicitly, at which point it will be written to account data
and all clients will start using this value.
## Tradeoffs
There are a number of ways to transition to this new scheme. Clients could
populate the account data with their current ID Server URL as soon as
possible, and immediately use any new value seen on account data. This
would a much faster migration but any users with clients using different
ID Servers would suddenly find all their clients using the ID Server of
whichever client they updated first.
## Potential issues
Users will no longer be able to have different clients configured with
different ID Servers.
## Security considerations
An attacker would be able to force all a user clients to use a given ID Server
if they gained control of any of a user's logins.
## Conclusion
This makes the ID server an account setting which means it persists between
logins. The intention would be to phase out clients ever asking for an ID
Server URL at registration or login: this will be much easier for users to
understand whilst still retaining the flexibility for those who want it.

14
proposals/2240-rooms-v6.md
Unescape Escape View File

@ -1,14 +0,0 @@
# MSC2240: Room Version 6
A new room version, `6`, is proposed using [room version 5](https://matrix.org/docs/spec/rooms/v5.html) as a base
and incorporating the following MSCs:
* [MSC2209](https://github.com/matrix-org/matrix-doc/pull/2209) - Including notifications in power level auth rules.
* [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432) - Alias event authorisation and redaction.
* [MSC2540](https://github.com/matrix-org/matrix-doc/pull/2540) - Integers in canonical JSON compliance.
Though other MSCs are capable of being included in this version, they do not have sufficient implementation to be
considered stable enough for v6 rooms. A future room version may still include them.
Room version 6 upon being added to the specification shall be considered stable. No other room versions are affected
by this MSC.

314
proposals/2241-e2e-verification-in-dms.md
Unescape Escape View File

@ -1,314 +0,0 @@
# Key verification in DMs
Currently, key verification is done using `to_device` messages. However, since
`to_device` messages are not part of a timeline, there is no user-visible
record of the key verification.
As well, the current key verification framework does not provide any feedback
when interacting with clients that do not support it; if a client does not
support the key verification framework, there is no way for users to discover
this other than waiting for a while and noticing that nothing is happening.
This proposal will solve both problems.
## Proposal
The current [key verification
framework](https://matrix.org/docs/spec/client_server/r0.5.0#key-verification-framework)
will be replaced by a new framework that uses room messages rather than
`to_device` messages. Key verification messages will be sent in a [Direct
Messaging](https://matrix.org/docs/spec/client_server/r0.5.0#id185) room. If
there is no Direct Messaging room between the two users involved, the client
that initiates the key verification will create one.
In this proposal, we use "Alice" to denote the user who initiates the key
verification, and "Bob" to denote the other user involved in the key
verification.
### General framework
#### Requesting a key verification
To request a key verification, Alice will send an `m.room.message` event with the
following properties in its contents:
- `body`: a fallback message to alert users that their client does not support
the key verification framework, and that they should use a different method
to verify keys. For example, "Alice is requesting to verify keys with you.
However, your client does not support this method, so you will need to use
the legacy method of key verification."
Clients that do support the key verification framework should hide the body
and instead present the user with an interface to accept or reject the key
verification.
The event may also contain `format` and `formatted_body` properties as
described in the [m.room.message
msgtypes](https://matrix.org/docs/spec/client_server/r0.5.0#m-room-message-msgtypes)
section of the spec. Clients that support the key verification should
similarly hide these from the user.
- `msgtype`: `m.key.verification.request`
- `methods`: the verification methods supported by Alice's client
- `to`: Bob's Matrix ID. Users should only respond to verification requests if
they are named in this field. Users who are not named in this field and who
did not send this event should ignore all other events that have a
`m.reference` relationship with this event.
- `from_device`: Alice's device ID. This is required since some verification
methods may use the device IDs as part of the verification process.
Key verifications will be identified by the event ID of the key verification
request event.
Clients should ignore verification requests that have been accepted or
cancelled, or if they do not belong to the sending or target users.
The way that clients display this event can depend on which user and device the
client belongs to, and what state the verification is in. For example:
- If the verification has been completed (there is an `m.key.verification.done`
or `m.key.verification.cancel` event), the client can indicate that the
verification was successful or had an error.
- If the verification has been accepted (there is an `m.key.verification.start`
event) but has not been completed, the two devices involved can indicate that
the verification is in progress and can use this event as a place in the
room's timeline to display progress of the key verification and to interact
with the user as necessary. Other devices can indicate that the verification
is in progress on other devices.
- If the verification has not been accepted, clients for the target user can
indicate that a verification has been requested and allow the user to accept
the verification on that device. The sending client can indicate that it is
waiting for the request to be accepted, and the sending user's other clients
can indicate the that a request was initiated on a different device.
Clients may choose to display or not to display events of any other type that
reference the original request event; but it must not have any effect on the
verification itself.
#### Accepting a key verification
To accept a key verification, Bob will send an `m.key.verification.ready` event
with the following properties in its contents:
- `m.relates_to`: an object with the properties:
- `rel_type`: `m.reference`
- `event_id`: the event ID of the key verification request that is being
accepted
- `methods`: an array of verification methods that the device supports
- `from_device`: Bob's device ID. This is required since some verification
methods may use the device IDs as part of the verification process.
(Note: the form of the `m.relates_to` property is based on the current state of
[MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674), but is
independent from it since this MSC does not rely on any aggregations features.)
Clients should ignore `m.key.verification.ready` events that correspond to
verification requests that they did not send.
After this, either Alice or Bob may start the verification by sending an
`m.key.verification.start` event with the following properties in its contents:
- `m.relates_to`: an object with the properties:
- `rel_type`: `m.reference`
- `event_id`: the event ID of the key verification request that is being
started
- `method`: the key verification method that is being used. This should be a
method that both Alice's and Bob's devices support.
- `from_device`: The user's device ID.
If both Alice and Bob send an `m.key.verification.start` message, and they both
specify the same verification method, then the event sent by the user whose
user ID is the smallest is used, and the other event is ignored. If they both
send an `m.key.verification.start` message and the method is different, then
the verification should be cancelled with a `code` of `m.unexpected_message`.
After the `m.key.verification.start` event is sent, the devices may exchange
messages (if any) according to the verification method in use.
#### Rejecting a key verification
To reject a key verification, Alice or Bob will send an
`m.key.verification.cancel` event with the following properties in its
contents:
- `m.relates_to`: an object with the properties:
- `rel_type`: `m.reference`
- `event_id`: the event ID of the key verification that is being cancelled
- `reason`: A human readable description of the `code`. The client should only
rely on this string if it does not understand the `code`.
- `code`: The error code for why the process/request was cancelled by the
user. The contents are the same as the `code` property of the currently
defined [`m.key.verification.cancel` to-device
event](https://matrix.org/docs/spec/client_server/r0.5.0#m-key-verification-cancel),
or as defined for specific key verification methods.
This message may be sent at any point in the key verification process. Any
subsequent key verification messages relating to the same request are ignored.
However, this does not undo any verifications that have already been done.
#### Concluding a key verification
When the other user's key is verified and no more messages are expected, each
party will send an `m.key.verification.done` event with the following
properties in its contents:
- `m.relates_to`: an object with the properties:
- `rel_type`: `m.reference`
- `event_id`: the event ID of the key verification that is being concluded
This provides a record within the room of the result of the verification.
Any subsequent key verification messages relating to the same request are
ignored.
Although a client may have successfully completed its side of the verification,
it may wait until receiving an `m.key.verification.done` (or
`m.key.verification.cancel`) event from the other device before informing the
user that the verification was successful or unsuccessful.
#### Other events
Key verification methods may define their own event types, or extensions to the
above event types. All events sent as part of a key verification process
should have an `m.relates_to` property as defined for
`m.key.verification.accept` or `m.key.verification.cancel` events.
Clients should ignore events with an `m.relates_to` that have a `rel_type` of
`m.reference` that refer to a verification where it is neither the requester
nor the accepter.
Clients should not redact or edit verification messages. A client may ignore
redactions or edits of key verification messages, or may cancel the
verification with a `code` of `m.unexpected_message` when it receives a
redaction or edit.
### SAS verification
The messages used in SAS verification are the same as those currently defined,
except that instead of the `transaction_id` property, an `m.relates_to`
property, as defined above, is used instead.
If the key verification messages are encrypted, the hash commitment sent in the
`m.key.verification.accept` message MUST be based on the decrypted
`m.key.verification.start` message contents, and include the `m.relates_to`
field, even if the decrypted message contents do not include that field. For
example, if Alice sends a message to start the SAS verification:
```json
{
"content": {
"algorithm": "m.megolm.v1.aes-sha2",
"ciphertext": "ABCDEFG...",
"device_id": "Dynabook",
"sender_key": "alice+sender+key",
"session_id": "session+id",
"m.relates_to": {
"rel_type": "m.reference",
"event_id": "$verification_request_event"
}
},
"event_id": "$event_id",
"origin_server_ts": 1234567890,
"sender": "@alice:example.org",
"type": "m.room.encrypted",
"room_id": "!room_id:example.org"
}
```
which, when decrypted, yields:
```json
{
"room_id": "!room_id:example.org",
"type": "m.key.verification.start",
"content": {
"from_device": "Dynabook",
"hashes": [
"sha256"
],
"key_agreement_protocols": [
"curve25519"
],
"message_authentication_codes": [
"hkdf-hmac-sha256"
],
"method": "m.sas.v1",
"short_authentication_string": [
"decimal",
"emoji"
]
}
}
```
then the hash commitment will be based on the message contents:
```json
{
"from_device": "Dynabook",
"hashes": [
"sha256"
],
"key_agreement_protocols": [
"curve25519"
],
"message_authentication_codes": [
"hkdf-hmac-sha256"
],
"method": "m.sas.v1",
"short_authentication_string": [
"decimal",
"emoji"
],
"m.relates_to": {
"rel_type": "m.reference",
"event_id": "$verification_request_event"
}
}
```
## Alternatives
Messages sent by the verification methods, after the initial key verification
request message, could be sent as to-device messages. The
`m.key.verification.ready`, `m.key.verification.cancel`, and
`m.key.verification.done` messages must be still be sent in the room, as the
`m.key.verification.ready` notifies the sender's other devices that the request
has been acknowledged, and the `m.key.verification.cancel` and
`m.key.verification.done` provide a record of the status of the key
verification.
However, it seems more natural to have all messages sent via the same
mechanism.
## Potential issues
If a user wants to verify their own device, this will require the creation of a
Direct Messaging room with themselves. Instead, clients may use the current
`to_device` messages for verifying the user's other devices.
Direct Messaging rooms could have end-to-end encryption enabled, and some
clients can be configured to only send decryption keys to verified devices.
Key verification messages should be granted an exception to this (so that
decryption keys are sent to all of the target user's devices), or should be
sent unencrypted, so that unverified devices will be able to be verified.
Users might have multiple Direct Messaging rooms with other users. In this
case, clients could need to prompt the user to select the room in which they
want to perform the verification, or could select a room.
## Security considerations
Key verification is subject to the room's visibility settings, and may be
visible to other users in the room. However, key verification does not rely on
secrecy, so this will no affect the security of the key verification. This may
reveal to others in the room that Alice and Bob know each other, but this is
already revealed by the fact that they share a Direct Messaging room.
This framework allows users to see what key verifications they have performed
in the past. However, since key verification messages are not secured, this
should not be considered as authoritative.
## Conclusion
By using room messages to perform key verification rather than `to_device`
messages, the user experience of key verification can be improved.

78
proposals/2244-mass-redactions.md
Unescape Escape View File

@ -1,78 +0,0 @@
# Mass redactions
Matrix, like any platform with public chat rooms, has spammers. Currently,
redacting spam essentially requires spamming redaction events in a 1:1 ratio,
which is not optimal<sup>[1](images/2244-redaction-spam.png)</sup>. Most
clients do not even have any mass redaction tools, likely in part due to the
lack of a mass redaction API. A mass redaction API on the other hand has not
been implemented as it would require sending lots of events at once. However,
this problem could be solved by allowing a single redaction event to redact
many events instead of sending many redaction events.
## Proposal
This proposal builds upon [MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174)
and suggests making the `redacts` field in the content of `m.room.redaction`
events an array of event ID strings instead of a single event ID string.
It would be easiest to do this before MSC2174 is written into the spec, as then
only one migration would be needed: from an event-level redacts string to a
content-level redacts array.
### Backwards compatibility
There is no easy way to stay fully compatible with *older* clients, so the
proposed solution is to not support them. In order to not break old clients
completely, servers should still add a `redacts` string containing one of the
redacted event IDs to the top level of `m.room.redaction` events in *newer*
room versions when serving such events over the Client-Server API.
Like MSC2174, for improved compatibility with *newer* clients, servers should
add a `redacts` array to the `content` of `m.room.redaction` events in *older*
room versions when serving such events over the Client-Server API.
### Number of redactions
Room v4+ event IDs are 44 bytes long, which means the federation event size
limit would cap a single redaction event at a bit less than 1500 targets.
Redactions are not intrinsically heavy, so a separate limit should not be
necessary.
Due to the possible large number of redaction targets per redaction event,
servers should omit the list of redaction targets from the `unsigned` ->
`redacted_because` field of redacted events. If clients want to get the list
of targets of a redaction event in `redacted_because`, they should read the
`event_id` field of the `redacted_because` event and use the
`/rooms/{roomId}/event/{eventId}` endpoint to fetch the content.
### Client behavior
Clients shall apply existing `m.room.redaction` target behavior over an array
of event ID strings.
### Server behavior (auth rules)
The target events of an `m.room.redaction` shall no longer be considered when
authorizing an `m.room.redaction` event. Any other existing rules remain
unchanged.
After a server accepts an `m.room.redaction` using the modified auth rules, it
evaluates individually whether each target can be redacted under the existing
room v5 auth rules. Servers MUST NOT include failing and unknown entries to
clients.
> Servers do not know whether redaction targets are authorized at the time they
receive the `m.room.redaction` unless they are in possession of the target
event. Implementations retain entries in the original list which were not
shared with clients to later evaluate the target's redaction status.
When the implementation receives a belated target from an earlier
`m.room.redaction`, it evaluates at that point whether the redaction is
authorized.
> Servers should not send belated target events to clients if their redaction
was found to be in effect, as clients were not made aware of the redaction.
That fact is also used to simply ignore unauthorized targets and send the
events to clients normally.
## Tradeoffs
## Potential issues
## Security considerations
Server implementations should ensure that large redaction events do not become
a DoS vector, e.g. by processing redactions in the background.

56
proposals/2263-homeserver-pw-resets.md
Unescape Escape View File

@ -1,56 +0,0 @@
# MSC2263: Give homeservers the ability to handle their own 3PID registrations/password resets
In order to better protect the privacy of a user, Matrix is wanting to shift to
a model where identity servers have less control over the affairs of the homeserver.
Identity servers are currently used to reset the passwords of users on a given homeserver
as an identity verification technique, however there is no reason why the homeserver
itself can't handle the verification. This proposal allows for a homeserver to verify
the identity of users itself, without the use of an identity server.
## Proposal
The `id_server` parameter is to become optional on the following endpoints:
* `/_matrix/client/:version/account/3pid/:medium/requestToken`
* `/_matrix/client/:version/register/:medium/requestToken`
* `/_matrix/client/:version/account/password/:medium/requestToken`
The `id_server` parameter is additionally deprecated with intention of being removed
in a future specification release on the `/register/:medium` and `/account/password/:medium`
endpoints. Once appropriate adoption has been achieved, the specification can safely
remove the parameter as supported. The reason for this deprecation is to completely
remove the identity server's ability to be involved in password resets/registration.
Users wishing to bind their 3rd party identifiers can do so after registration, and
clients can automate this if they so desire.
Note that `bind_email` and `bind_msisdn` on `/register` have already been removed
by [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140).
As per [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140), an `id_access_token`
is required only if an `id_server` is supplied.
Although not specified as required in the specification currently, the `id_server`
as part of User-Interactive Authentication is also optional if this proposal is accepted.
When the client requests a token without an `id_server`, it should not specify an
`id_server` in UIA.
Homeservers can reuse HTTP 400 `M_SERVER_NOT_TRUSTED` as an error code on the `/requestToken`
endpoints listed above if they do not trust the identity server the user is supplying.
In order to allow client implementations to determine if the homeserver they are developed
against supports `id_server` being optional, an unstable feature flag of `m.require_identity_server`
is to be added to `/versions`. When the flag is `true` or not present, clients must assume
that the homeserver requires an `id_server` (ie: it has not yet considered it optional).
If this proposal is accepted, clients are expected to use the supported specification versions
the homeserver advertises instead of the feature flag's presence.
## Tradeoffs
Homeservers may have to set up MSISDN/email support to their implementations. This is believed
to be of minimal risk compared to allowing the identity server to continue being involved
with password reset/registration.
## Security considerations
The identity server was previously involved with affairs only the homeserver cares about.
This is no longer the case.

101
proposals/2265-email-lowercase.md
Unescape Escape View File

@ -1,101 +0,0 @@
# Proposal for mandating case folding when processing e-mail addresses
[RFC822](https://tools.ietf.org/html/rfc822#section-3.4.7) mandates that
localparts in e-mail addresses must be processed with the original case
preserved. [The Matrix spec](https://matrix.org/docs/spec/appendices#pid-types)
doesn't mandate anything about processing e-mail addresses, other than the fact
that the domain part must be converted to lowercase, as domain names are case
insensitive.
On the other hand, most major e-mail providers nowadays process the localparts
of e-mail addresses as case insensitive. Therefore, most users expect localparts
to be treated case insensitively, and get confused when it's not. Some users,
for example, get confused over the fact that registering a 3PID association for
`john.doe@example.com` doesn't mean that the association is valid for
`John.Doe@example.com`, and don't expect to have to remember the exact
case they used to initially register the association (and sometimes get locked
out of their account because of that). So far we've seen that confusion occur
and lead to troubles of various degrees over several deployments of Synapse and
Sydent.
## Proposal
This proposal suggests changing the specification of the e-mail 3PID type in
[the Matrix spec appendices](https://matrix.org/docs/spec/appendices#pid-types)
to mandate that, before any processing, e-mail addresses must go through a full
case folding as described under "Caseless Matching" in
[chapter 5 of the unicode standard](https://www.unicode.org/versions/Unicode13.0.0/ch05.pdf#G21790), on top of
having their domain lowercased.
This means that `Strauß@Example.com` must be considered as being the same e-mail
address as `strauss@example.com`.
## Other considered solutions
A first look at this issue concluded that there was no need to add such a
mention to the spec, and that it can be considered an implementation detail.
However, [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) changes
this: because hashing functions are case sensitive, we need both clients and
identity servers to follow the same policy regarding case sensitivity.
An initial version of this proposal proposed to mandate lowercasing e-mail
addresses instead of case folding them, however it was pointed out that this
solution might not be the best and most future-proof one.
Unicode normalisation was also looked at but judged unnecessary.
## Tradeoffs
Implementing this MSC in identity servers and homeservers might require the
databases of existing instances to be updated in a large part to case fold the
email addresses of existing associations, in order to avoid conflicts. However,
most of this update can usually be done by a background job running at startup,
so the UX improvement outweighs this trouble.
## Potential issues
### Conflicts with existing associations
Some users might already have two different accounts associated with the same
e-mail address but with different cases. This appears to happen in a small
number of cases, however, and can be dealt with by the identity server's or the
homeserver's maintainer.
For example, with Sydent, the process of dealing with such cases could look
like:
1. list all MXIDs associated with a variant of the email address, and the
timestamp of that association
2. delete all associations except for the most recent one [0]
3. inform the user of the deletion by sending them an email notice to the email
address
### Storing and querying
Most database engines don't support case folding, therefore querying all
e-mail addresses matching a case folded e-mail address might not be trivial,
e.g. an identity server querying all associations for `strauss@example.com` when
processing a `/lookup` request would be expected to also get associations for
`Strauß@Example.com`.
To address this issue, implementation maintainers are strongly encouraged to
make e-mail addresses go through a full case folding before storing them.
### Implementing case folding
The need for case folding in services on the Internet doesn't seem to be very
large currently (probably due to its young age), therefore there seem to be only
a few third-party implementation libraries out there. However, both
[Go](https://godoc.org/golang.org/x/text/cases#Fold), [Python
2](https://docs.python.org/2/library/stringprep.html#stringprep.map_table_b3)
and [Python 3](https://docs.python.org/3/library/stdtypes.html#str.casefold)
support it natively, and [a third-party JavaScript
implementation](https://github.com/ar-nelson/foldcase) exists which, although
young, seems to be working.
## Footnotes
[0]: This is specific to Sydent because of a bug it has where v1 lookups are
already processed case insensitively, which means it will return the most recent
association for any case of the given email address, therefore keeping only this
association won't change the result of v1 lookups.

24
proposals/2284-optional-identity-server-discovery.md
Unescape Escape View File

@ -1,24 +0,0 @@
# MSC2284: Making the identity server optional during discovery
Currently the specification requires that clients `FAIL_ERROR` (hard failure - do not continue)
when the `.well-known` file for `m.identity_server` points to somewhere invalid or is invalid
itself. This can cause problems for clients if they either don't need an identity server to
function (and are forced to validate it anyways) or the client ends up having to disable all
their login UX because the identity server is misconfigured/down.
This proposal aims to change that by allowing clients to make a conscious decision to continue
with the invalid identity server configuration, provided the homeserver configuration is valid.
## Proposal
Instead of `FAIL_ERROR` for an invalid `m.identity_server` schema/server, clients are to move
to the `FAIL_PROMPT` (inform the user, ask for input if applicable) state. Clients can decide
to show a warning that the identity server is unavailable and allow the user to continue with
the invalid (or client's default) configuration.
## Tradeoffs
Clients can end up being configured with an invalid or inoperable identity server. This is
considered a feature by this proposal to allow for less intelligent clients to have their
identity server disabled. Intelligent clients could interpret the lack of identity server
as the homeserver/user asking that identity server functionality be disabled in the client.

223
proposals/2290-separate-threepid-bind-hs.md
Unescape Escape View File

@ -1,223 +0,0 @@
# Separate Endpoints for Binding Threepids
*Note: This MSC obsoletes
[MSC2229](https://github.com/matrix-org/matrix-doc/pull/2229), which dealt
with changing the rules of the `bind` flag on [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid).
That endpoint is being deprecated as part of this MSC, thus MSC2229 is no
longer relevant.*
On the Client Server API there is currently a single endpoint for binding a
threepid (an email or a phone number): [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid).
Depending on whether the `bind` flag is `true` or `false`, the threepid will
be bound to either a user's account on the homeserver, or both the homeserver
and an identity server. Note that we use the term `add` when talking about
adding a threepid to a homeserver, and `bind` when binding a threepid to an
identity server. This terminology will be used throughout the rest of this
proposal.
Typically, when using the `/account/3pid` endpoint, the identity server
handles the verification -- either by sending an email to an email address,
or a SMS message to a phone number. Once completed, the homeserver will check
with the identity server that verification had indeed happened, and if so,
the threepid would be either added to the homeserver, or added to the
homeserver and bound to the identity server simultaneously.
Now, consider the fact that the identity server used in this process is
provided by the user, using the endpoint's `id_server` parameter. If the user were
to supply a malicious identity server that would immediately answer "yes" to
any threepid validation, then the user could add any threepid to their
account on the homeserver (which is likely not something homeserver admins want).
To be clear, this is not a long-standing security issue. It is not a problem
in any released version of Synapse, as Synapse keeps a list of "trusted
identity servers" that acts a whitelist for what identity servers a user can
specify.
The concept of this whitelist is being removed in this MSC however, as part
of lessening the reliance of homeservers on identity servers. This cannot be
done while the homeserver is still trusting an identity server for validation
of threepids. If the endpoints are split, the homeserver will handle the
validation of threepids being added to user accounts, and identity servers
will validate threepids being bound to themselves.
## Proposal
To solve this problem, two new endpoints will be added to the Client Server
API: `POST /account/3pid/bind` and `POST /account/3pid/add`. Binding to an
identity server will require standard authentication, whereas adding a 3pid
to a user account will require [User-Interactive
Authentication](https://matrix.org/docs/spec/client_server/r0.5.0#user-interactive-authentication-api).
The latter is to prevent someone from adding a 3pid (which can be used to
reset passwords) to someone who's left their account open on a public
computer, without needing their password to do so.
Both endpoints will be rate-limited. The request parameters of `POST
/account/3pid/bind` are the same as [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid),
minus the `bind` flag, and the contents of `three_pid_creds` have been
brought to the top level of the request body. The request parameters of `POST
/account/3pid/add` will simply consist of a JSON body containing
`client_secret` and `sid`.
The homeserver should prevent a threepid being added to a user's account if
it's already part of another user's account. However, the homeserver should not
check for existing threepids when binding to an identity server. Identity
servers do not enforce this requirement and neither should the proxying
homeserver.
An example of binding a threepid to an identity server with this new endpoint
is as follows:
First the client must request the threepid be validated by its chosen
identity server.
```
POST https://identity.server/_matrix/identity/v2/validate/email/requestToken
{
"client_secret": "don'tT3ll",
"email": "bob@example.com",
"send_attempt": 1
}
```
The identity server must send an email to the specified address, including a
link to a URL on the identity server which will accept the validation session
ID, the given client_secret, and a randomly-generated token.
Once an email has been sent, the user clicks the link in the email, which
notifies the identity server that the email has been verified.
Next, the client completes the bind by calling the new endpoint on the
homeserver:
```
POST https://home.server/_matrix/client/r0/account/3pid/bind
{
"id_server": "example.org",
"id_access_token": "abc123_OpaqueString",
"sid": "abc123987",
"client_secret": "don'tT3ll"
}
```
The homeserver will then make a bind request to the specified identity server
on behalf of the user. The homeserver will record if the bind was successful
and notify the user. The homeserver will remember this bind and the identity
server it occurred on so that it can perform an unbind later if the user
requests it or their account is deactivated.
The threepid has now been bound on the user's requested identity server
without causing that threepid to be used for password resets or any other
homeserver-related functions.
For completeness, here is an example of adding a threepid to the homeserver
only, using the `/account/3pid/add` endpoint:
The homeserver is validating the threepid in this instance, so the client
must use the `/requestToken` endpoint of the homeserver:
```
POST https://home.server/_matrix/client/r0/account/3pid/email/requestToken
{
"client_secret": "don'tT3ll",
"email": "bob@example.com",
"send_attempt": 1,
}
```
Here the homeserver must send an email to the specified address, including a
link to a URL on the homeserver which will accept the validation session ID,
the given client_secret, and a randomly-generated token.
Once an email has been sent, the user clicks the link in the email, which
notifies the homeserver that the threepid has been verified.
The client then sends a request to the endpoint on the homeserver to add
the threepid to a user's account.
```
POST https://home.server/_matrix/client/r0/account/3pid/add
{
"sid": "abc123987",
"client_secret": "don'tT3ll"
}
```
The homeserver checks the threepid validation session referred to by the
given ID and client_secret was validated, and if so adds the threepid to the
user's account.
To achieve the above flows, some changes need to be made to existing
endpoints. The [POST
/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid)
endpoint is deprecated as the two new endpoints replace its functionality.
The `bind` parameter is to be removed, with the endpoint functioning as if
`bind` was `false`. Allowing an endpoint to add a threepid to both the
identity server and homeserver at the same time requires one to trust the
other, which is the exact behaviour we're trying to eliminate. Doing this
also helps backward compatibility, as explained in [Backwards
compatibility](#backwards-compatibility).
Either the homeserver itself or a service that the homeserver delegates to
should be handling the sending of validation messages, not a user-provided
server. Any mention of the homeserver being able to proxy to an identity
server in the below endpoint descriptions:
* [POST /account/3pid/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-email-requesttoken)
* [POST /account/3pid/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-msisdn-requesttoken)
* [POST /register/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-email-requesttoken)
* [POST /register/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-msisdn-requesttoken)
* [POST /account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-email-requesttoken)
* [POST /account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-msisdn-requesttoken)
As well as the text "It is imperative that the homeserver keep a list of
trusted Identity Servers and only proxies to those that it trusts." is to be
removed from all parts of the spec, as the homeserver should no longer need
to trust any identity servers.
## Tradeoffs
One may question why clients don't just contact an identity server directly
to bind a threepid, bypassing the implications of binding through a
homeserver. While this will work, binds should still occur through a
homeserver such that the homeserver can keep track of which binds were made,
which is important when a user wishes to deactivate their account (and remove
all of their bindings made on different identity servers).
A verification could occur on an identity server, which could then tell the
homeserver that a validation happened, but then there are security
considerations about how to authenticate an identity server in that instance
(and prevent people pretending to be identity servers and telling homeservers
about hundreds of fake threepid additions to a user's account).
## Backwards compatibility
A new flag will be added to `/versions`' unstable_features section,
`m.separate_add_and_bind`. If this flag is present and set to `true`, then
clients should use the new API endpoints to carry out threepid adds and
binds. If this flag is not present or set to `false`, clients should use
`/account/3pid`, being aware that they can only bind threepids to the
homeserver, not the identity server.
Old matrix clients will continue to use the `/account/3pid` endpoint. This
MSC removes the `bind` parameter and forces `/account/3pid` calls to act as
if `bind` was set to `false`. Old clients will still be able to add 3pids to
the homeserver, but not bind to the identity server. New homeservers must
ignore any `id_server` information passed to this endpoint.
## Security considerations
Reducing the homeserver's trust in identity servers should be a boost to
security and improve decentralisation in the Matrix ecosystem to boot.
Some endpoints of the Client Server API allow a user to provide an
`id_server` parameter. Caution should be taken for homeserver developers to
stop using these user-provided identity servers for any sensitive tasks where
possible, such as password reset or account registration, if it removes the
concept of a trusted identity server list.

769
proposals/2312-matrix-uri.md
Unescape Escape View File

@ -1,769 +0,0 @@
# URI scheme for Matrix
This is a proposal of a URI scheme to identify Matrix resources in a wide
range of applications (web, desktop, or mobile) both throughout Matrix software
and (especially) outside it. It supersedes
[MSC455](https://github.com/matrix-org/matrix-doc/issues/455) in order
to continue the discussion in the modern GFM style.
While Matrix has its own resource naming system that allows it to identify
resources without resolving them, there is a common need to provide URIs
to Matrix resources (e.g., rooms, users, PDUs) that could be transferred
outside of Matrix and then resolved in a uniform way - matching URLs
in World Wide Web.
Specific use cases include:
1. Representation: as a Matrix user I want to refer to Matrix entities
in the same way as for web pages, so that others could unambiguously identify
the resource, regardless of the context or used medium to identify it to them
(within or outside Matrix, e.g., in a web page or an email message).
1. Inbound integration: as an author of Matrix software, I want to have a way
to invoke my software from the operating environment to resolve a Matrix URI
passed from another program. This is a case of, e.g.,
opening a Matrix client by clicking on a link from an email message.
1. Outbound integration: as an author of Matrix software, I want to have a way
to export identifiers of Matrix resources to non-Matrix environment
so that they could be resolved in another time-place in a uniform way.
An example of this case is the "Share via…" action in a mobile Matrix client.
Matrix identifiers as defined by the current specification have a form distinct
enough from other identifiers to mostly fulfil the representation use case.
Since they are not URIs, they can not cover the two integration use cases.
https://matrix.to somehow compensates for this; however:
* it requires a web browser to run JavaScript code that resolves identifiers
(basically limiting first-class support to browser-based clients), and
* it relies on matrix.to as an intermediary that provides that JavaScript code.
To cover the use cases above, the following scheme is proposed for Matrix URIs
(`[]` enclose optional parts, `{}` enclose variables):
```text
matrix:[//{authority}/]{type}/{id without sigil}[/{type}/{id without sigil}...][?{query}][#{fragment}]
```
with `{type}` defining the resource type (such as `r`, `u` or `roomid` - see
the "Path" section in the proposal) and `{query}` containing additional hints
or request details on the Matrix entity (see "Query" in the proposal).
`{authority}` and `{fragment}` parts are reserved for future use; this proposal
does not define them and implementations SHOULD ignore them for now.
This MSC does not introduce new Matrix entities, nor API endpoints -
it merely defines a mapping between URIs with the scheme name `matrix:`
and Matrix identifiers, as well as operations on them. The MSC should be
sufficient to produce an implementation that would convert Matrix URIs to
a series of [CS API](https://matrix.org/docs/spec/client_server/r0.6.1) calls,
entirely on the client side. It is recognised, however, that most of
the URI processing logic can and should (eventually) be on the server side
in order to facilitate adoption of Matrix URIs; further MSCs are needed
to define details for that, as well as to extend the mapping to more resources
(including those without equivalent Matrix identifiers, such as room state or
user profile data).
The Matrix identifier (or identifiers) can be reconstructed from
`{id without sigil}` by prepending a sigil character corresponding to `{type}`.
To support a hierarchy of Matrix resources, more `/{type}/{id without sigil}`
pairs can be appended, identifying resources within other resources.
As of now, there's only one such case, with exactly one additional pair -
pointing to an event in a room.
Examples:
* Room `#someroom:example.org`:
`matrix:r/someroom:example.org`
* User `@me:example.org`:
`matrix:u/me:example.org`
* Event in a room:
`matrix:r/someroom:example.org/e/Arbitrary_Event_Id`
* [A commit like this](https://github.com/her001/steamlug.org/commit/2bd69441e1cf21f626e699f0957193f45a1d560f)
could make use of a Matrix URI in the form of
`<a href="{Matrix URI}">{Matrix identifier}</a>`.
## Proposal
### Definitions
Further text uses the following terms:
- Matrix identifier - one of identifiers defined by the current
[Matrix Specification](https://matrix.org/docs/spec/appendices.html#identifier-grammar),
- Matrix URI - a uniform resource identifier proposed hereby, following
the RFC-compliant URI format.
- MUST/SHOULD/MAY etc. follow the conventions of
[RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
### Requirements
The following considerations drive the requirements for Matrix URIs:
1. Follow existing standards and practices.
1. Endorse the principle of the least surprise.
1. Humans first, machines second.
1. Cover as many entities as practical.
1. URIs are expected to be extremely portable and stable;
you cannot rewrite them once they are released to the world.
1. Ease of implementation, allowing reuse of existing codes.
The following requirements resulted from these drivers:
1. Matrix URI MUST comply with
[RFC 3986](https://tools.ietf.org/html/rfc3986) and
[RFC 7595](https://tools.ietf.org/html/rfc7595).
1. By definition, Matrix URI MUST unambiguously identify a resource
in a Matrix network, across servers and types of resources.
This means, in particular, that two Matrix identifiers distinct by
[Matrix Specification](https://matrix.org/docs/spec/appendices.html#identifier-grammar)
MUST NOT have Matrix URIs that are equal in
[RFC 3986](https://tools.ietf.org/html/rfc3986) sense
(but two distinct Matrix URIs MAY map to the same Matrix identifier).
1. References to the following entities MUST be supported:
1. User IDs (`@user:example.org`)
1. Room IDs (`!roomid:example.org`)
1. Room aliases (`#roomalias:example.org`)
1. Event IDs (`$arbitrary_eventid_with_or_without_serverpart`)
1. The mapping MUST take into account that some identifiers
(e.g. aliases) can have non-ASCII characters - reusing
[RFC 3987](https://tools.ietf.org/html/rfc3987) is RECOMMENDED,
but an alternative encoding can be used if there are reasons for that.
1. The mapping between Matrix identifiers and Matrix URIs MUST
be extensible (without invalidating previous URIs) to:
1. new classes of identifiers (there MUST be a meta-rule to produce
a new mapping for IDs following the `&somethingnew:example.org`
pattern assumed for Matrix identifiers);
1. new ways to navigate to and interact with objects in Matrix
(e.g., we might eventually want to have a mapping for
room-specific user profiles).
1. The mapping MUST support decentralised as well as centralised IDs.
This basically means that the URI scheme MUST have provisions
for mapping of identifiers with `:<serverpart>` but it MUST NOT require
`:<serverpart>` to be there.
1. Matrix URI SHOULD allow encoding of action requests such as joining a room.
1. Matrix URI SHOULD have a human-readable, if not necessarily
human-friendly, representation - to allow visual sanity-checks.
In particular, characters escaping/encoding should be reduced
to bare minimum in that representation. As food for thought, see
[Wikipedia: Clean URL, aka SEF URL](https://en.wikipedia.org/wiki/Clean_URL) and
[a use case from RFC 3986](https://tools.ietf.org/html/rfc3986#section-1.2.1).
1. It SHOULD be easy to parse Matrix URI in popular programming
languages: e.g., one should be able to use `parseUri()`
to dissect a Matrix URI into components in JavaScript.
1. The mapping SHOULD be consistent across different classes of
Matrix identifiers.
1. The mapping SHOULD support linking to unfederated servers/networks
(see also
[matrix-doc#2309](https://github.com/matrix-org/matrix-doc/issues/2309)
that calls for such linking).
The syntax and mapping discussed below meet all these requirements except
the last one that will be addressed separately.
Further extensions MUST NOT reduce the supported set of requirements.
### Syntax and high-level processing
The proposed generic Matrix URI syntax is a subset of the generic
URI syntax
[defined by RFC 3986](https://tools.ietf.org/html/rfc3986#section-3):
```text
MatrixURI = "matrix:" hier-part [ "?" query ] [ "#" fragment ]
hier-part = [ "//" authority "/" ] path
```
As mentioned above, this MSC assumes client-side URI processing
(i.e. mapping to Matrix identifiers and CS API requests).
However, even when URI processing is shifted to the server side
the client will still have to parse the URI at least to remove
the authority and fragment parts (if either exists)
before sending the request to the server (more on that below).
#### Scheme name
The proposed scheme name is `matrix`.
[RFC 7595](https://tools.ietf.org/html/rfc7595) states:
if there’s one-to-one correspondence between a service name and
a scheme name then the scheme name should be the same as
the service name.
Other considered options were `mx` and `web+matrix`;
[comments to MSC455](https://github.com/matrix-org/matrix-doc/issues/455)
mention two scheme names proposed and one more has been mentioned
in `#matrix-core:matrix.org`.
The scheme name is a definitive indication of a Matrix URI and MUST NOT
be omitted. As can be seen below, Matrix URI rely heavily on [relative
references](https://tools.ietf.org/html/rfc3986#section-4.2) and
omitting the scheme name makes them indistinguishable from a local path
that might have nothing to do with Matrix. Clients MUST NOT try to
parse pieces like `r/MyRoom:example.org` as Matrix URIs; instead,
users should be encouraged to use Matrix identifiers for in-text references
(`#MyRoom:example.org`) and client applications SHOULD turn them into
hyperlinks to Matrix URIs.
#### Authority
Basing on
[the definition in RFC 3986](https://tools.ietf.org/html/rfc3986#section-3.2),
this MSC restricts the authority part to never have a userinfo component,
partially to prevent confusion concerned with the `@` character that has its
own meaning in Matrix, but also because this component has historically been
a popular target of abuse.
```text
authority = host [ ":" port ]
```
Further definition of syntax or semantics for the authority part is left for
future MSCs. Clients MUST parse the authority part as per RFC 3986 (i.e.
the presence of an authority part MUST NOT break URI parsing) but SHOULD NOT
use data from the authority part other than for experiments or research.
The authority part may eventually be used to indicate access to a Matrix
resource (such as a room or a user) specifically through a given entity.
See "Ideas for further evolution".
#### Path
This MSC restricts
[the very wide definition of path in RFC 3986](https://tools.ietf.org/html/rfc3986#section-3.3),
to a simple pattern that allows to easily reconstruct a Matrix identifier or
a chain of identifiers and also to locate a certain sub-resource in the scope
of a given Matrix entity:
```text
path = entity-descriptor ["/" entity-descriptor]
entity-descriptor = nonid-segment / type-qualifier id-without-sigil
nonid-segment = segment-nz ; as defined in RFC 3986, see also below
type-qualifier = segment-nz "/" ; as defined in RFC 3986, see also below
id-without-sigil = string ; as defined in Matrix identifier spec, see below
```
The path component consists of 1 or more descriptors separated by a slash
(`/`) character. This is a generic pattern intended for reusing in future
extensions.
This MSC only proposes mappings along `type-qualifier id-without-sigil` syntax;
`nonid-segment` is unused and reserved for future use.
For the sake of integrity future `nonid-segment` extensions must follow
[the ABNF for `segment-nz` as defined in RFC 3986](https://tools.ietf.org/html/rfc3986#appendix-A).
This MSC defines the following `type` specifiers: `u` (user id, sigil `@`),
`r` (room alias, sigil `#`), `roomid` (room id, sigil `!`), and
`e` (event id, sigil `$`). This MSC does not define a type specifier for sigil `+`
([groups](https://github.com/matrix-org/matrix-doc/issues/1513) aka communities
or, in the more recent incarnation,
[spaces](https://github.com/matrix-org/matrix-doc/pull/1772)); a separate MSC
can introduce the specifier, along with the parsing/construction logic and
relevant CS API invocations, following the framework of this proposal.
The following type specifiers proposed in earlier editions of this MSC and
already in use in several implementations, are deprecated: `user`, `room`, and
`event`. Client applications MAY parse these specifiers as if they were
`u`, `r`, and `e` respectively; they MUST NOT emit URIs with the deprecated
specifiers. The rationale behind the switch is laid out in "Alternatives".
As of this MSC, `u`, `r`, and `roomid` can only be at the top
level. The type `e` (event) can only be used on the 2nd level and only under
`r` or `roomid`; this is driven by the current shape of Client-Server API
that does not provide a non-deprecated way to retrieve an event without knowing
the room (see [MSC2695](https://github.com/matrix-org/matrix-doc/pull/2695) and
[MSC2779](https://github.com/matrix-org/matrix-doc/issues/2779) that may
change this).
Further MSCs may introduce navigation to more top-level as well as
non-top-level objects; see "Ideas for further evolution" to get inspired. These
new proposals SHOULD follow the generic grammar laid out above, adding new
`type` and `nonid-segment` specifiers and/or allowing them in other levels,
rather than introduce a new grammar. It is recommended to only use abbreviated
single-letter specifiers if they are expected to be user visible and convenient
for type-in; if a URI for a given resource type is usually generated
(e.g. because the corresponding identifier is not human-friendly), it's
RECOMMENDED to use full (though short) words to avoid ambiguity and confusion.
`id-without-sigil` is defined as the `string` part of Matrix
[Common identifier format](https://matrix.org/docs/spec/appendices#common-identifier-format)
with percent-encoded characters that are NEITHER unreserved, sub-delimiters, `:` nor `@`,
[as per RFC 3986 rule for pchar](https://tools.ietf.org/html/rfc3986#appendix-A).
This notably exempts `:` from percent-encoding but includes `/`.
See the rationale behind dropping sigils and the respective up/downsides in
"Discussion points and tradeoffs" as well as "Alternatives" below.
#### Query
Matrix URI can optionally have
[the query part](https://tools.ietf.org/html/rfc3986#section-3.4).
This MSC defines the general form for the query and two "standard" query items;
further MSCs may add to this as long as RFC 3986 is followed.
```text
query = query-element *( "&" query-item )
query-item = action / routing / custom-query-item
action = "action=" ( "join" / "chat" )
routing = "via=” authority
custom-query-item = custom-item-name "=" custom-item-value
custom-item-name = 1*unreserved ; reverse-DNS name; see below
custom-item-value = ; see below
```
The `action` query item is used in contexts where, on top of identifying
the Matrix entity, a certain action is requested on it. This proposal
describes two possible actions:
* `action=join` is only valid in a URI resolving to a Matrix room;
applications MUST ignore it if found in other contexts and MUST NOT generate
it for other Matrix resources. This action means that a client application
SHOULD attempt to join the room specified by the URI path using the standard
CS API means.
* `action=chat` is only valid in a URI resolving to a Matrix user;
applications MUST ignore it if found in other contexts and MUST NOT generate
it for other Matrix resources. This action means that a client application
SHOULD open a direct chat window with the user specified by the URI path;
clients supporting
[canonical direct chats](https://github.com/matrix-org/matrix-doc/pull/2199)
SHOULD open the canonical direct chat.
For both actions, where applicable, client applications SHOULD ask for user
confirmation or at least notify the user before joining or creating a new room.
Conversely, no additional confirmation/notification is necessary when
the action leads to opening a room the user is already a member of.
It is worth reiterating on the (blurry) distinction between URIs with `action`
and those without:
- a URI with no `action` simply _identifies_ the resource; if the context
implies an operation, it is usually focused on the retrieval of the resource,
in line with RFC 3986 (see also the next paragraph);
- a URI with `action` in the query means that a client application should (but
is not obliged to) perform that action, with precautions as described above.
In some cases a client application may have no meaningful way to immediately
perform the default operation suggested by this MSC (see below); e.g.,
the client may be unable to display a room before joining it, while the URI
doesn't have `action=join`. In these cases client applications are free to do
what's best for user experience (e.g., suggest joining the room), even if that
means performing an action on a URI with no `action` in the query.
The routing query (`via=`) indicates servers that are likely involved in
the room (see also
[the feature of matrix.to](https://matrix.org/docs/spec/appendices#routing)).
In the meantime, it is proposed that this routing query be used not only with
room ids in a public federation but also when a URI refers to a resource in
a non-public Matrix network (see the question about closed federations in
"Discussion points and tradeoffs"). Note that `authority` in the definition
above is only a part of the _query parameter_ grammar; it is not proposed here
to generate or interpret the _authority part_ of the URI.
Clients MAY introduce and recognise custom query items, according to
the following rules:
- the name of a custom item MUST follow the reverse-DNS (aka "Java package")
naming convention, as per
[MSC2758](https://github.com/matrix-org/matrix-doc/pull/2758) - e.g.,
a custom action item for Element clients would be named `io.element.action`,
for Quaternion - `com.github.quaternion.action`, etc.
- the value of the item can be any content but its representation in the URI
MUST follow the general RFC requirements for the query part; on top of that,
if the raw value contains `&` it MUST be percent-encoded.
- clients SHOULD respect standard query items over their own ones; e.g.,
if a URI contains both `action` and the custom client action, the standard
action should be respected as much as possible. Client authors SHOULD strive
for consistent experience across their and 3rd party clients, anticipating
that the same user may happen to have both their client and a 3rd party one.
Client authors are strongly encouraged to standardise custom query elements
that gain adoption by submitting an MSC defining them in a way compatible
across the client ecosystem.
### Recommended implementation
#### URI parsing algorithm
The reference algorithm of parsing a Matrix URI follows. Note that, although
clients are encouraged to use lower-case strings in their URIs, all string
comparisons are case-INsensitive.
1. Parse the URI into main components (`scheme name`, `authority`, `path`,
`query`, and `fragment`), decoding special or international characters
as directed by [RFC 3986](https://tools.ietf.org/html/rfc3986) and
(for IRIs) [RFC 3987](https://tools.ietf.org/html/rfc3987). Authors are
strongly RECOMMENDED that they find an existing implementation of that step
for their language and SDK, rather than implement it from scratch based
on RFCs.
1. Check that `scheme name` is exactly `matrix`, case-insensitive. If
the scheme name doesn't match, exit parsing: this is not a Matrix URI.
1. Split the `path` into segments separated by `/` character; several
subsequent `/` characters delimit empty segments, as advised by RFC 3986.
1. Check that the URI contains either 2 or 4 segments; if it's not the case,
fail parsing; the Matrix URI is invalid.
1. To construct the top-level (primary) Matrix identifier:
a. Pick the leftmost segment of `path` until `/` (path segment) and match
it against the following list to produce `sigil-1`:
- `u` (or, optionally, `user` - see "Path") -> `@`
- `r` (or, optionally, `room`) -> `#`
- `roomid` -> `!`
- any other string, including an empty one -> fail parsing:
the Matrix URI is invalid.
b. Pick the next (2nd) leftmost path segment:
- if the segment is empty, fail parsing;
- otherwise, percent-decode the segment (unless the initial URI parse
has already done that) and make `mxid-1` by prepending `sigil-1`.
1. If `sigil-1` is `!` or `#` and the URI path has exactly 4 segments,
it may be possible to construct the 2nd-level Matrix identifier to
point to an event inside the room identified by `mxid-1`:
a. Pick the next (3rd) path segment:
- if the segment is exactly `e` (or, optionally, `event`), proceed;
- otherwise, including the case of an empty segment (trailing `/`, e.g.),
fail parsing.
b. Pick the next (4th) leftmost path segment:
- if the segment is empty, fail parsing;
- otherwise, percent-decode the segment (unless the initial URI parse
has already done that) and make `mxid-2` by prepending `$`.
1. Split the `query` into items separated by `&` character; several subsequent
`&` characters delimit empty items, ignored by this algorithm.
a. If `query` contains one or more items starting with `via=`: for each item, treat
the rest of the item as a percent-encoded homeserver name to be used in
[routing](https://matrix.org/docs/spec/appendices#routing).
b. If `query` contains one or more items starting with `action=`: treat
_the last_ such item as an instruction, as this proposal defines in [query](#query).
Clients MUST implement proper percent-decoding of the identifiers; there's no
liberty similar to that of matrix.to.
#### Operations on Matrix URIs
The main purpose of a Matrix URI is accessing the resource specified by the
identifier. This MSC defines the "default" operation
([in the sense of RFC 7595](https://tools.ietf.org/html/rfc7595#section-3.4))
that a client application SHOULD perform when the user activates
(e.g. clicks on) a URI; further MSCs may introduce additional operations
enabled either by passing an `action` value in the query part, or by other
means.
The classes of URIs and corresponding default operations (along with relevant
CS API calls) are collected below. The table assumes that the operations are
performed on behalf (using the access token) of the user `@me:example.org`:
| URI class/example | Interactive operation | Non-interactive operation / Involved CS API |
| ----------------- | --------------------- | --------------------------------------------- |
| User Id (no `action` in URI):<br/>`matrix:u/her:example.org` | _Outside the room context_: show user profile<br/>_Inside the room context:_ mention the user in the current room (client-local operation) | No default non-interactive operation<br/>`GET /profile/@her:example.org/display_name`<br/>`GET /profile/@her:example.org/avatar_url` |
| User Id (`action=chat`):<br/>`matrix:u/her:example.org?action=chat` | 1. Confirm with the local user if needed (see "Query")<br/>2. Open the room as defined in the next column | If [canonical direct chats](https://github.com/matrix-org/matrix-doc/pull/2199) are supported: `GET /_matrix/client/r0/user/@me:example.org/dm?involves=@her:example.org`<br/>Without canonical direct chats:<br/>1. `GET /user/@me:example.org/account_data/m.direct`<br/>2. Find the room id for `@her:example.org` in the event content<br/>3. if found, return this room id; if not, `POST /createRoom` with `"is_direct": true` and return id of the created room |
| Room (no `action` in URI):<br/>`matrix:roomid/rid:example.org`<br/>`matrix:r/us:example.org` | Attempt to "open" (usually: display the timeline at the latest or last remembered position) the room | No default non-interactive operation<br/>API: Find the respective room in the local `/sync` cache or<br/>`GET /rooms/!rid:example.org/...`<br/> |
| Room (`action=join`):<br/>`matrix:roomid/rid:example.org?action=join&via=example2.org`<br/>`matrix:r/us:example.org?action=join` | 1. Confirm with the local user if needed (see "Query")<br/>2. Attempt to join the room | `POST /join/!rid:example.org?server_name=example2.org`<br/>`POST /join/#us:example.org` |
| Event:<br/>`matrix:r/us:example.org/e/lol823y4bcp3qo4`<br/>`matrix:roomid/rid:example.org/event/lol823y4bcp3qo4?via=example2.org` | 1. For room aliases, resolve an alias to a room id (see the next column)<br/>2. Attempt to retrieve (see the next column) and display the event;<br/>3. If the event could not be retrieved due to access denial and the current user is not a member of the room, the client MAY offer the user to join the room and try to open the event again | Non-interactive operation: return event or event content, depending on context<br/>API: find the event in the local `/sync` cache or<br/>`GET /directory/room/%23us:example.org` (to resolve alias to id)<br/>`GET /rooms/!rid:example.org/event/lol823y4bcp3qo4?server_name=example2.org`<br/> |
#### URI construction algorithm
The following algorithm assumes a Matrix identifier that follows
the high-level grammar described in the specification. Clients MUST ensure
compliance of identifiers passed to this algorithm.
For room and user identifiers (including room aliases):
1. Remove the sigil character from the identifier and match it against
the following list to produce `prefix-1`:
- `@` -> `u/`
- `#` -> `r/`
- `!` -> `roomid/`
2. Build the Matrix URI as a concatenation of:
- literal `matrix:`;
- `prefix-1`;
- the remainder of identifier (`id without sigil`), percent-encoded as per
[RFC 3986](https://tools.ietf.org/html/rfc3986).
For event identifiers (assuming they need the room context, see
[MSC2695](https://github.com/matrix-org/matrix-doc/pull/2695) and
[MSC2779](https://github.com/matrix-org/matrix-doc/issues/2779) that
may change this):
1. Take the event's room id or canonical alias and build a Matrix URI for them
as described above.
2. Append to the result of previous step:
- literal `e/`;
- the event id after removing the sigil (`$`) and percent-encoding.
Clients MUST implement proper percent-encoding of the identifiers; there's no
liberty similar to that of matrix.to.
## Discussion and non-normative statements
### Ideas for further evolution
This MSC is obviously just the first step, keeping the door open for
extensions. Here are a few ideas:
* Add new actions; e.g. leaving a room (`action=leave`).
* Add specifying a segment of the room timeline (`from=$evtid1&to=$evtid2`).
* Unlock bare event ids (`matrix:e/$event_id`) - subject to change in
other areas of the specification.
* Bring tangible semantics to the authority part. The main purpose of
the authority part,
[as per RFC 3986](https://tools.ietf.org/html/rfc3986#section-3.2),
is to identify the entity governing the namespace for the rest of the URI.
The current MSC rules out the userinfo component but leaves it to a separate
MSC to define semantics of the remaining`host[:port]` piece.
Importantly, future MSCs are advised against using the authority part for
_routing over federation_ (the case for `via=` query items), as it would be
against the spirit of RFC 3986. The authority part can be used in cases when
a given Matrix entity is only available from certain servers (the case of
closed federations or non-federating servers).
While being a part of the original proposal in an attempt to address
[the respective case](https://github.com/matrix-org/matrix-doc/issues/2309),
the definition of the authority semantics has been dropped as a result of
[the subsequent discussion](https://github.com/matrix-org/matrix-doc/pull/2312#discussion_r348960282).
A further MSC may approach the same case (and/or others) and define the
meaning of the authority part (either on the client- or even on
the server-side - provided that using Matrix URIs on the server-side brings
some other value along the way). This might not necessarily be actual DNS
hostnames even - one (quite far-fetched for now) idea to entertain might be
introducing some decentralised system of "network names" in order to equalise
"public" and "non-public" federations.
Along the same lines, if providing any part of user credentials via
the authority part is found to be of considerable value in some case,
a separate MSC could both reinstate it in the grammar and define how
to construct, parse, and use it - provided that the same MSC addresses
the security concerns associated with such URIs.
* One could conceive a URI mapping of avatars in the form of
`matrix:u/uid:matrix.org/avatar/room:matrix.org`
(a user’s avatar for a given room).
* As described in "Alternatives", a synonymous system can be introduced that
uses Matrix identifiers with sigils by adding another path prefix (e.g.,
`matrix:id/%23matrix:matrix.org`). However, such MSC would have to address
the concerns of possible confusion arising from having two similar but
distinct notations.
* Interoperability of Matrix URIs with
[Linked Data](https://en.wikipedia.org/wiki/Linked_data).
### Past discussion points and tradeoffs
The below documents the discussion and outcomes in various prior forums;
further discussion should happen in GitHub comments.
1. _Why no double-slashes in a typical URI?_
Because `//` is used to mark the beginning of an authority
part. RFC 3986 explicitly forbids to start the path component with
`//` if the URI doesn't have an authority component. In other words,
`//` implies a centre of authority, and the (public) Matrix
federation is not supposed to have one; hence no `//` in most URIs.
1. ~~_Why do type specifiers use singular rather than plural
as is common in RESTful APIs?_~~
This is no more relevant with single-letter type specifiers. The answer
below is provided for history only.
Unlike in actual RESTful APIs, this MSC does not see `rooms/` or
`users/` as collections to browse. The type specifier completes
the id specification in the URI, defining a very specific and
easy to parse syntax for that. Future MSCs may certainly add
collection URIs, but it is recommended to use more distinct naming
for such collections. In particular, `rooms/` is ambiguous, as
different sets of rooms are available to any user at any time
(e.g., all rooms known to the user; or all routable rooms; or
public rooms known to the user's homeserver).
1. _Should we advise using the query part for collections then?_
Not in this MSC but that can be considered in the future.
1. _Why can't event URIs use the fragment part for the event ID?_
Because fragment is a part processed exclusively by the client
in order to navigate within a larger document, and room cannot
be considered a "document". Each event can be retrieved from the server
individually, so each event can be viewed as a self-contained document.
When/if URI processing is shifted to the server-side, servers are not even
going to receive fragments (as per RFC 3986), which is why usage of
fragments to remove the need for percent-encoding in other identifiers
would lead to URIs that cannot be resolved on servers. Effectively, all
clients would have to implement full URI processing with no chance
to offload that to the server. For that reason fragments, if/when ever
employed in Matrix, only should be used to pinpoint a position within events
and for similar strictly client-side operations.
1. _How does this MSC work with closed federations?_ ~~If you need to
communicate a URI to the bigger world where you cannot expect
the consumer to know in advance which federation they should use -
supply any server of the closed federation in the authority part.
Users inside the closed federation can omit the authority part if
they know the URI is not going to be used outside this federation.
Clients can facilitate that by having an option to always add or omit
the authority part in generated URIs for a given user account.~~
As of now, use `via=` in order to point to a homeserver in the closed
federation. The authority part may eventually be used for that (or for some
other case - see the previous section).
### Alternatives
#### Using full words for all types
During its draft state, this MSC was proposing type specifiers using full words
(`user`, `room`, `event` etc.), arguing that abbreviations can be introduced
separately as synonyms. Full words have several shortcomings pointed out in
discussions across the whole period of preparation, namely:
- The singular vs. plural choice (see also "Past discussion points")
- Using English words raises a question about eventual support of localised
URI variants (`matrix:benutzer/...`, `matrix:usuario/...` etc.) catering to
international audience, that would add complication to the Matrix technology.
- Abbreviated forms are popularised by Reddit and make URIs shorter which is
crucial for the outbound integration case (see the introduction).
Meanwhile, using `u`/`r`/`e` for users, rooms and events has the following
advantages:
1. there's a strong Reddit legacy, with users across the world quite familiar
with the abbreviated forms (and `r/` coincidentally standing for sub-Reddits
links to which have basically the same place in the Reddit ecosystem as
Matrix room aliases have in the Matrix ecosystem);
2. matrix.to links to users and room aliases are heavily used throughout Matrix,
specifically in end-user-facing contexts (see also use cases in the
introductory section of this MSC);
3. the singular vs. plural (`room` or `rooms`?) confusion is avoided;
4. it's shorter, which is crucial for typing the URI in an external medium.
The rationale behind not abbreviating `roomid/` is a better distinction between
room aliases and room ids; also, since room ids are almost never typed in
manually, the advantages (3) and (4) above don't hold.
For these reasons, it was decided in the end to use the single-letter style
for types most used in the outbound integration case. It's still possible to
reinstate full words as synonyms some time down the road, with the caveat that
a canonicalisation service from homeservers may be needed to avoid having
to enable synonyms at each client individually.
#### URNs
The discussion in
[MSC455](https://github.com/matrix-org/matrix-doc/issues/455)
mentions an option to standardise URNs rather than URLs/URIs,
with the list of resolvers being user-specific. While a URN namespace
such as `urn:matrix:`, along with a URN scheme, might be deemed useful
once we shift to (even) more decentralised structure of the network,
`urn:` URIs must be managed entities (see
[RFC 8141](https://tools.ietf.org/html/rfc8141)) which is not always
the case in Matrix (consider room aliases, e.g.).
With that said, a URN-styled (`matrix:room:example.org:roomalias`)
option was considered. However, Matrix already uses colon (`:`) as
a delimiter of id parts and, as can be seen above, reversing the parts
to meet the URN's hierarchical order would look confusing for Matrix
users (as in example above - is `room` a part of the identifier or
the type signifier?).
#### "Full REST"
Yet another alternative considered was to go "full REST" and structure
URLs in a more traditional way with serverparts coming first, followed
by type grouping (sic - not specifiers), and then by localparts,
i.e. `matrix://example.org/rooms/roomalias`. This is even more difficult
to comprehend for a Matrix user than the previous alternative and besides it
conflates the notion of an authority server with that of a namespace
discriminator: clients would not connect to `example.org` to resolve the alias
above, they would still connect to their own homeserver.
#### Minimal syntax
One early proposal was to simply prepend `matrix:` to a Matrix identifier
(without encoding it), assuming that it will only be processed on the client
side. The massive downside of this option is that such strings are not actual
URIs even though they look like ones: most URI parsers won't handle them
correctly. As laid out in the beginning of this proposal, Matrix URIs are
not striving to preempt Matrix identifiers; instead of trying to produce
an equally readable string, one should just use identifiers where they work.
Why Matrix identifiers look the way they look is way out of the MSC scope
to discuss here.
#### Minimal syntax based on the path component and percent-encoding
A simple modification of the previous option is much more viable:
proper percent-encoding of the Matrix identifier allows to use it as
a URI path part. A single identifier packed in a URI could look like
`matrix:/encoded_id_with_sigil`; an event-in-a-room URI would be something
like `matrix:/roomid_or_alias/$event_id` (NB: RFC 3986 doesn't require `$`
to be encoded). This is considerably more concise and encoding is only
needed for `#`.
Quite unfortunately, `#` is one of the two sigils in Matrix most relevant
to integration cases. The other one is `@`; it doesn't need encoding except
in the authority part - which is why the form above uses a leading `/` that
puts the identifier in the path part instead of what parsers treat as
the authority part. `#` has to be encoded wherever it appears, making a URI
for Matrix HQ, the first chat room many new users join, look like
`matrix:/%23matrix:matrix.org`. Beyond first-time usage, this generally impacts
[the "napkin" case](https://tools.ietf.org/html/rfc3986#section-1.2.1) from
RFC 3986 that the Requirements section of this MSC mentions. Until we have
applications generally recognising Matrix identifiers in the same way e-mail
addresses are recognised without prefixing `mailto:`, we should live with
the fact that people will have to produce Matrix URIs by hand in various
instances, from pen-and-paper to other instant messengers.
Putting the whole id to the URI fragment (`matrix:#id_with_sigil` or,
following on the `matrix.to` tradition, `matrix:#/id_with_sigil` for
readability) allows using `#` without encoding on many URI parsers. It is
still not fully RFC-compliant and rules out using URIs by homeservers
(see also "Past discussion points" on using fragments to address events).
Regardless of the placement (the fragment or the path), one more consideration
is that the character space for sigils is extremely limited and
Matrix identifiers are generally less expressive than full-blown URI paths.
Not that Matrix showed a tendency to produce many classes of objects that would
warrant a dedicated sigil but that cannot be ruled out. Rather than rely
on the institute of sigils, this proposal gives an alternative more
extensible syntax that can be used for more advanced cases - as a uniform way
to represent arbitrary sub-objects (with or without Matrix identifier) such as
user profiles, or a notifications feed for the room - and also, if ever needed,
as an escape hatch to a bigger namespace if we hit shortage of sigils.
The current proposal is also flexible enough to incorporate the minimal
syntax of this option as an alternative to its own notation - e.g., a further
MSC could enable `matrix:id/%23matrix:matrix.org` as a synonym for
`matrix:room/matrix:matrix.org`.
## Potential issues
Despite the limited functionality of URIs as proposed in this MSC,
Matrix authors are advised to use tools that would process URIs just
like an HTTP(S) URI instead of making home-baked parsers/emitters.
Even with that in mind, not all tools normalise and sanitise all cases
in a fully RFC-compliant way. This MSC tries to keep the required
transformations to the minimum and will likely not bring much grief even
with naive implementations; however, as functionality of Matrix URI grows,
the number of corner cases will increase.
## Security/privacy considerations
This MSC mostly builds on RFC 3986 but tries to reduce the scope
as much as possible. Notably, it avoids introducing complex traversable
structures and further restricts the URI grammar to the necessary subset.
In particular, dot path segments (`.` and `..`), while potentially useful
when URIs become richer, would come too much ahead of time for now. Care
is taken to not make essential parts of the URI omittable to avoid
even accidental misrepresentation of a local resource for a remote one
in Matrix and vice versa.
As mentioned in the authority part section, the MSC intentionally doesn't
support conveying any kind of user information in URIs.
The MSC strives to not be prescriptive in treating URIs except the `action`
query parameter. Actions without user confirmation may lead to unintended
leaks of certain metadata and/or changes in the account state with respect
to Matrix. To reiterate, clients SHOULD ask for a user consent if/when they
can unless applying the action doesn't lead to sending persistent (message
or state) events on user's behalf.
## Conclusion
A dedicated URI scheme is well overdue for Matrix. Many other networks
already have got one for themselves, benefiting both in terms of
branding (compare `matrix:r/weruletheworld:example.org` vs.
`#weruletheworld:example.org` from the standpoint of someone who
hasn't been to Matrix) and interoperability (`matrix.to` requires
opening a browser while clicking a `tg:` link dumped to the terminal
application will open the correct application for Telegram without
user intervention or can even offer to install one, if needed).
The proposed syntax makes conversion between Matrix URIs
and Matrix identifiers as easy as a bunch of string comparisons or
regular expressions; so even though client-side processing of URIs
might not be optimal longer-term, it's a very simple and quick way
that allows plenty of experimentation early on.

199
proposals/2313-moderation-policy-rooms.md
Unescape Escape View File

@ -1,199 +0,0 @@
# MSC2313: Moderation policies as rooms (ban lists)
Matrix is an open network and anyone can participate in it. As a result, a
very wide range of content exists, and it is important to empower users to be
able to select which content they wish to see, and which they wish to block. By
extension, room moderators and server admins should also be able to select
which content they do not wish to host in their rooms and servers.
The protocol's position in this solution should be one of neutrality: it
should not be deciding what content is undesirable for any particular entity and
should instead be empowering those entities to make their own decisions. This
proposal introduces "moderation policy rooms" as a basic mechanism to help users
manage this process, by providing a way of modelling sets of servers, rooms and users
which can then be used to make filtering decisions. This proposal makes no
attempt at interpreting the model and actually making those decisions however.
To reaffirm: where this proposal says that some content is undesirable it does not intend to
bias the reader into what that could entail. Undesirability is purely in the hands of the
entity perceiving the content. For example, someone who believes birthday cake is undesirable
is perfectly valid in taking that position and is encouraged by this proposal to set up or
use a policy room which prevents birthday cake from coming across their field of view.
## Proposal
Moderation policy lists, also known as ban lists in this proposal, are stored as room state events,
allowing for structures and concepts to be reused without defining a new room version. This
proposal does not make any restrictions on how the rooms are configured, just that the state
events described here are represented in a room. For example, a room which is invite only is
just as valid as a room that is not: the important details are specific state events and not
the accessibility, retention, or other aspects of the room.
Ban lists are stored as `m.policy.rule.<kind>` state events, with state keys being arbitrary IDs
assigned by the sender. The `<kind>` is currently one of `user`, `room`, and `server`. Three
fields are defined in `content`:
* `entity` (`string`) - **Required.** The entity/entities the recommendation applies to. Simple globs are supported
for defining entities (`*` and `?` as wildcards, just like `m.room.server_acl`).
* `recommendation` (`enum`) - **Required.** The action subscribed entities should take against
the described entity. Currently only `m.ban` is defined (see below).
* `reason` (`string`) - **Required.** The human-readable description for the recommendation.
Invalid or missing fields are to be treated as though the rule doesn't exist. This is to
allow for rules to be deleted while state events cannot be deleted in Matrix.
An example set of minimal state events for banning `@alice:example.org`, `!matrix:example.org`,
`evil.example.org`, and `*.evil.example.org` would be:
```json
[
{
"type": "m.policy.rule.user",
"state_key": "rule_1",
"content": {
"entity": "@alice:example.org",
"recommendation": "m.ban",
"reason": "undesirable behaviour"
}
},
{
"type": "m.policy.rule.room",
"state_key": "rule_2",
"content": {
"entity": "!matrix:example.org",
"recommendation": "m.ban",
"reason": "undesirable content"
}
},
{
"type": "m.policy.rule.server",
"state_key": "rule_3",
"content": {
"entity": "evil.example.org",
"recommendation": "m.ban",
"reason": "undesirable engagement"
}
},
{
"type": "m.policy.rule.server",
"state_key": "rule_4",
"content": {
"entity": "*.evil.example.org",
"recommendation": "m.ban",
"reason": "undesirable engagement"
}
}
]
```
When the entity is a room, it can be a room alias or ID - the subscriber is responsible for
resolving it to a room ID (if it wants to).
Non-standard recommendations are permitted using the Java package naming convention for
namespacing. A `recommendation` is just a recommendation: how implementations apply the rules
is left as a concern for those implementations. The only standard `recommendation` currently
defined is `m.ban`: The `entity` should be banned from participation where possible.
The enforcement mechanism of `m.ban` is deliberately left as an implementation detail to avoid the
protocol imposing its opinion on how the lists are interpreted. However, a suggestion for
a simple implementation is as follows:
* Is a user...
* Applied to a user: The user should be added to the subscriber's ignore list.
* Applied to a room: The user should be banned from the room (either on sight or immediately).
* Applied to a server: The user should not be allowed to send invites to users on the server.
* Is a room...
* Applied to a user: The user should leave the room and not join it
([MSC2270](https://github.com/matrix-org/matrix-doc/pull/2270)-style ignore).
* Applied to a room: No-op because a room cannot ban itself.
* Applied to a server: The server should prevent users from joining the room and from receiving
invites to it (similar to the `shutdown_room` API in Synapse).
* Is a server...
* Applied to a user: The user should not receive events or invites from the server.
* Applied to a room: The server is added as a denied server in the ACLs.
* Applied to a server: The subscriber should avoid federating with the server as much as
possible by blocking invites from the server and not sending traffic unless strictly
required (no outbound invites).
Other entities could be represented by this recommendation as well, however as per the
introduction to this proposal they are strictly out of scope. An example would be an integration
manager which doesn't want to offer integrations to banned entities - this is an implementation
detail for the integration manager to solve.
A new event type is introduced here instead of reusing existing events (membership, ACLs, etc)
because the implication of a recommendation/rule is less clear when using the more narrow-scoped
events. For example, a philosophical question arises over what a `membership` of `ban` means to a server
subscribed to the list. More questions get raised when the `membership` of a user isn't `ban`,
but is `join` or similar instead - if the subscriber was mirroring events, it would be inclined
to try and sync membership lists, which this proposal attempts to avoid by using a more generic
and neutral event type.
How subscriptions to ban lists are handled is also left as an implementation
detail (to avoid unnecessarily blocking progress on this MSC). The subscriber
could be anything that speaks Matrix, therefore this proposal makes no attempt
to describe how this should work for everything. Some ideas for how this could
be implemented include joining the ban list room to watch for updates and
applying them automatically, however there is no requirement that the
subscriber needs to join the room: they could peek or poll at an interval
instead, which is just as valid.
To ease sharing of these ban list rooms, a system very similar to [MSC1951's sharable URLs](
https://github.com/matrix-org/matrix-doc/pull/1951/files#diff-4ee6ed0ee1f2df73efac5fa9a9835642R50-R70)
is defined. There are two ways to share the ban list: a link to the room as one would when
sharing any reference to any other room ("please add `#bans:example.org` to your block list"),
or by using plain `http` or `https` URLs. Just like in MSC1951, the URL when approached with
a `Accept: application/json` header or has `.json` appended to the end of the URL should return
a `room_uri` which is a reference to the ban list room. Currently, the reference would be a
`matrix.to` URI, however in the future this could be a `mx://` or similar URL. When not approached
with the intent of JSON, the service could return a user-friendly page describing what is included
in the ban list.
## Future considerations
This proposal notably does not define specific behaviour for AI or machine learning applications.
Implementations are currently able to apply AI/ML to their systems if they see fit (for example,
spam detection or undesirable content being uploaded), however no specification is proposed
here to make the interaction standardized.
This proposal additionally does not describe how a server could subscribe to a ban list: this
is left for the server to figure out (possibly by using a utility user account?) potentially
with the support of other proposals, such as [MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777).
Further work on reputation systems could enhance ban lists by adding additional metrics to
assert their validity. This proposal assumes social trust ("don't use it if you
don't trust the creator") over verifiable/provable trust - a future proposal can easily add
such systems to these ban lists.
This proposal intentionally does not handle how a server could assist the user in preventing
undesirable content or subscribing to ban lists. Users already have some tools at their disposal,
such as being able to ignore other users, and are encouraged to make use of those first. Other
proposals are encouraged to specify what additional tools might look like to better handle
ban lists.
Media (uploaded content) is not handled by this proposal either. This is a concern left for
other proposals like [MSC2278](https://github.com/matrix-org/matrix-doc/pull/2278) and
[MSC701](https://github.com/matrix-org/matrix-doc/issues/701).
## Security considerations
Using this solution one can build a social system of shared blacklists, which
may create a divide within established communities if not carefully deployed.
This may well not be a suitable answer for all communities.
Depending on how the implementations handle subscriptions, user IDs may be linked to ban
lists and therefore expose the views of that user. Using the example from the introduction,
if a user who does not like birthday cake were to join the ban list room for blocking
birthday cake, that user's preference would be exposed to any other observers of that ban
list. Proposals like [MSC1228](https://github.com/matrix-org/matrix-doc/issues/1228) and
[MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777) could help solve this.
## Implementation notes
This proposal is partially implemented by [mjolnir](https://github.com/matrix-org/mjolnir)
using the `org.matrix.mjolnir.*` namespace until this becomes stable. This results in
the following mappings:
* `m.policy.rule.user` => `org.matrix.mjolnir.rule.user`
* `m.policy.rule.room` => `org.matrix.mjolnir.rule.room`
* `m.policy.rule.server` => `org.matrix.mjolnir.rule.server`
* `m.ban` => `org.matrix.mjolnir.ban`

35
proposals/2320-identity-versions.md
Unescape Escape View File

@ -1,35 +0,0 @@
# Versions information for identity servers
The client-server API currently specifies a `/versions` endpoint that allows
clients to know what version of that API are implemented by the server.
Identity servers could benefit from that endpoint as both homeservers and
clients interact with them, and therefore could know which features they can
expect a given identity server to implement by looking at the versions of the
API it claims to support.
## Proposal
This proposal adds the following endpoint to the identity server API.
### `GET /_matrix/identity/versions`
This endpoint serves information about the versions of the identity server API
this identity server supports. Its response uses the following format:
```json
{
"versions": [
"r0.1.0",
"r0.2.0",
"r0.2.1",
]
}
```
## Alternative solutions
Another solution which was considered was using the status check endpoint ([`GET
/_matrix/api/v1`](https://matrix.org/docs/spec/identity_service/r0.2.0#get-matrix-identity-api-v1))
to serve this information. This solution was discarded because it's using a
versioned endpoint, which doesn't make sense to advertise the supported versions
of the API to use.

124
proposals/2324-when-to-ship.md
Unescape Escape View File

@ -1,124 +0,0 @@
# MSC2324: Facilitating early releases of software dependent on spec
*Note*: This is a process change MSC, not a change to the spec itself.
There's currently an unanswered question by the spec process: when is it
safe to start using stable endpoints or to present a feature as "stable"?
Historically this question would receive very different answers depending
on who you asked, so in an effort to come up with a concise answer the
following process change is proposed.
## Proposal
The new process, from start to finish, is proposed as:
1. Have an idea for a feature.
2. Optionally: implement the feature using unstable endpoints, vendor prefixes,
and unstable feature flags as appropriate.
* When using unstable endpoints, they MUST include a vendor prefix. For
example: `/_matrix/client/unstable/com.example/login`. Vendor prefixes
throughout this proposal always use the Java package naming convention.
* Unstable endpoints **do not** inherit from stable (`/r0`) APIs. Previously,
one could access the entirety of the Matrix API through `/unstable` however
this is generally considered a bad practice. Therefore an implementation
can no longer assume that because its feature-specific endpoint exists that
any other endpoint will exist in the same unstable namespace.
* If the client needs to be sure the server supports the feature, an unstable
feature flag that MUST be vendor prefixed is to be used. This kind of flag
shows up in the `unstable_features` field of `/versions` as, for example,
`com.example.new_login`.
* You can ship the feature at *any* time, so long as you are able to accept
the technical debt that results from needing to provide adequate backwards
and forwards compatibility for the vendor prefixed implementation. The
implementation MUST support the flag disappearing and be generally safe for
users. Note that implementations early in the MSC review process may also be
required to provide backwards compatibility with earlier editions of the
proposal.
* If you don't want to support the technical debt (or if it's impossible to
provide adequate backwards/forwards compatibility - e.g. a user authentication
change which can't be safely rolled back), do not implement the feature and
wait for Step 7.
* If at any point the idea changes, the feature flag should also change so
that implementations can adapt as needed.
3. In parallel, or ahead of implementation, open an MSC and solicit review.
4. Before a FCP (Final Comment Period) can be called, the Spec Core Team will
require that evidence to prove the MSC works be presented. A typical example
of this is an implementation of the MSC (which does not necessarily need to have been shipped anywhere).
5. FCP is gone through, and assuming nothing is flagged the MSC lands.
6. A spec PR is written to incorporate the changes into Matrix.
7. A spec release happens.
8. Implementations switch to using stable prefixes (e.g.: `/r0`) if the server
supports the specification version released. If the server doesn't advertise
the specification version, but does have the feature flag, unstable prefixes
should still be used.
9. A transition period of about 2 months starts immediately after the spec release, before
implementations start to loudly encourage other implementations to switch to stable
endpoints. For example, the Synapse team should start asking the Riot team to
support the stable endpoints (as per Step 8) 2 months after the spec release if they
haven't already.
It's worth repeating that this process generally only applies if the implementation
wants to ship the feature ahead of the spec being available. By doing so, it takes
on the risk that the spec/MSC may change and it must adapt. If the implementation
is unable to take on that risk, or simply doesn't mind waiting, it should go through
the spec process without shipping an unstable implementation.
To help MSCs get incorporated by implementations as stable features, the spec core
team plans to release the specification more often. How often is undefined and is
largely a case-by-case basis.
To reiterate:
* Implementations MUST NOT use stable endpoints before the MSC is in the spec. This
includes NOT using stable endpoints post-FCP.
* Implementations CAN ship features that are exposed by default to users before an
MSC has been merged to the spec, provided they follow the process above.
* Implementations SHOULD be wary of the technical debt they are incurring by moving
faster than the spec.
To clarify:
* The vendor prefix is chosen by the developer of the feature, using the Java package
naming convention. For example, `org.matrix` is the foundation's vendor prefix.
* The vendor prefixes, unstable feature flags, and unstable endpoints should be included
in the MSC so other developers can benefit. The MSC MUST still say what the stable
endpoints are to look like.
### Specific examples outside of the client-server API
There are some instances where a change might be made outside of the client-server API,
which is where much of this proposal is targeted. The general spirit of the process
should be followed where possible, if implementations decide to work ahead of spec releases.
#### Room versions
When a new room version is needed, implementations MUST use vendor-prefixed versions
before using the namespace reserved for Matrix (see https://spec.matrix.org/unstable/rooms/).
A room version is considered released once it is listed as an "available room version" in
the spec. Often a new room version is accompanied with a server-server API release, but
doesn't have to be.
#### Server-server / Identity / Push / Appservice API
These APIs don't yet have a `/versions` endpoint or similar. Typically behaviour changes in
these APIs are introduced with backwards compatibility in mind (try X and if that fails fall
back to Y) and therefore don't always need a flag to indicate support. If a flag were to
be required, an `unstable_features` or similar array would need to be exposed somewhere.
#### Changes to request/response parameters
Parameters being added to request/response bodies and query strings MUST be vendor-prefixed
per the proposed process. For example, a new JSON field might be `{"org.matrix.example": true}`
with the proposal being for `example` being added. A query string parameter would be prefixed
the same way: `?org.matrix.example=true`.
If the MSC is simply adding fields to already-versioned endpoints, it does not need to put
the whole endpoint into the `/unstable` namespace provided the new parameters are prefixed
appropriately.
#### .well-known and other APIs that can't be versioned
Best effort is appreciated. Typically these endpoints will be receiving minor behavioural
changes or new fields. New fields should be appropriately prefixed, and behaviour changes
should be rolled out cautiously by implementations (waiting until after FCP has concluded
is probably best to ensure there's no major problems with the new behaviour).

29
proposals/2334-default-room-version-v5.md
Unescape Escape View File

@ -1,29 +0,0 @@
# [MSC2334](https://github.com/matrix-org/matrix-doc/pull/2334) - Change default room version to v5
This MSC proposes changing the recommended default room version from v4 to v5.
This implements steps 5 and 6 of the plan from
[MSC2002](https://github.com/matrix-org/matrix-doc/issues/2002).
Room version 5 enforces the `valid_until_ts` timestamp on signing keys as
proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076).
## Proposal
When [MSC2077](https://github.com/matrix-org/matrix-doc/issues/2077) proposed
room version 5, the default version continued to be v4 to allow old servers to
continue to chat in newly created rooms. Server-Server API r0.1.2 (which
contains the details on room version 5) and Synapse 1.0.0 (with support for room
version 5) were both released more than 4 months ago which has given people
plenty of time to update. Room version 5 should be the default room version so
that newly created rooms enforce key-validity periods.
## Potential issues
Servers which do not support room version 5 will not be able to participate in
newly created rooms on other servers. Hopefully this will encourage them to
update their server.
## Security considerations
Room version 5 fixes known security vulnerabilities but that doesn't do much
good if newly created rooms aren't using room version 5 by default.

80
proposals/2366-key-verification-accept.md
Unescape Escape View File

@ -1,80 +0,0 @@
# Key verification flow additions: `m.key.verification.ready` and `m.key.verification.done`
The current key verification framework is asymmetrical in that the user who
requests the verification is unable to select the key verification method.
This makes it harder for more experienced users who wish to guide less
experienced users through the verification process, especially if they are not
verifying in-person, but are using a trusted but remote channel of verification
(such as telephone or video conference).
As an example, let us say that Alice is an experienced Matrix user and is
introducing Bob to the wonders of federated communications. Alice wants to
verify keys with Bob, so she clicks on the "Verify" button in her client on
Bob's profile (which sends a `m.key.verification.request` message to Bob).
Bob's device receives the verification request and prompts Bob to accept the
verification request. At this point, under the current framework, Bob is
responsible for choosing the verification method to use. However, with this
proposal, Bob would be able to just accept the verification request without
choosing a method, and allow Alice to choose the verification method.
In addition, the current key verification framework does not have a method for
clients to signal to the other side that a key verification was successful.
Some clients may wish to wait until the other side has either confirmed a
successful verification or indicated an error before displaying the result of
the verification, in order to give the two users a consistent view of the
verification as a whole.
## Proposal
Two new event types are added to the [key verification
framework](https://matrix.org/docs/spec/client_server/r0.6.1#key-verification-framework)
when verifying in to-device messages. The new event
types are already described in [MSC2241 (Key verification in
DMs)](https://github.com/matrix-org/matrix-doc/pull/2241). This proposal adds
them to verifications in to-device messages.
The first event type is `m.key.verification.ready`, which must be sent by the
target of the `m.key.verification.request` message, upon receipt of the
`m.key.verification.request` event. It has the fields:
- `from_device`: the ID of the device that sent the `m.key.verification.ready`
message
- `methods`: an array of verification methods that the device supports
It also has the usual `transaction_id` or `m.relates_to` fields for key
verification events, depending on whether it is sent as a to-device event
or an in-room event.
After the `m.key.verification.ready` event is sent, either party can send an
`m.key.verification.start` event to begin the verification. If both parties
send an `m.key.verification.start` event, and they both specify the same
verification method, then the event sent by the user whose user ID is the
lexicographically smallest is used, and the other `m.key.verification.start` event is ignored.
In the case of a single user verifying two of their devices, the device ID is
compared instead. If both parties send an `m.key.verification.start` event,
but they specify different verification methods, the verification should be
cancelled with a `code` of `m.unexpected_message`.
With to-device messages, previously the sender of the
`m.key.verification.request` message would send an `m.key.verification.cancel`
message to the recipient's other devices when it received an
`m.key.verification.start` event. With this new event, the sender of the
`m.key.verification.request` message should send an `m.key.verification.cancel`
message when it receives an `m.key.verification.ready` or
`m.key.verification.start` message, whichever comes first.
The `m.key.verification.ready` event is required for verifications in both DMs
and in to-device messages to accept verifications requested using an
`m.key.verification.request` event.
The second event type is `m.key.verification.done`, which has no fields other
than the usual `transaction_id` or `m.relates_to` field. This indicates that
the device has successfully completed its side of the verification.
## Potential issues
Clients that follow the Client-Server 0.6.0 spec may not expect an
`m.key.verification.ready` message in response to `m.key.verification.request`.
However to our knowledge, no clients implement `m.key.verification.request` in
this way yet -- to our knowledge, all clients that implement verification
implement this proposal.

81
proposals/2367-membership-reasons.md
Unescape Escape View File

@ -1,81 +0,0 @@
# Allowing Reasons in all Membership Events
Currently users can specify a reason for kicking or banning users in a room
that both the target and other users in a room can see. This is useful to
explain *why* an action without having to send separate messages.
The proposal extends this to all events, including invites, invite rejections
and leaves for similar reasons.
## Proposal
Allow `reason` field to be specified for all of the following APIs:
```
POST /_matrix/client/r0/rooms/{roomId}/invite
POST /_matrix/client/r0/rooms/{roomId}/leave
POST /_matrix/client/r0/rooms/{roomId}/kick
POST /_matrix/client/r0/rooms/{roomId}/ban
POST /_matrix/client/r0/rooms/{roomId}/unban
POST /_matrix/client/r0/rooms/{roomId}/join
POST /_matrix/client/r0/join/{roomIdOrAlias}
PUT /_matrix/client/r0/rooms/{roomId}/state/m.room.member/{userID}
```
If specified the `reason` field will be added to the generated membership
event's content.
*Note: `/state/m.room.member` API currently allows this as clients can specify
arbitrary content already*
Clients may choose to display the reason for membership events in a room,
though may not do so if e.g. they have collapsed a set of membership changes.
Clients should not display an invite reason by default to the invitee as this
allows a classic abuse and harassment vector. However, clients may wish to show
invite reasons from known¹ senders, or have a button that allows viewing any
invite reason.
## Use Cases
Some basic use cases and examples are given below.
### Kick/ban
Kicking and banning already allow specifying a reason to allow giving a reason
for the moderation action (e.g. "Banned for spamming").
### Leaving a Room
A user may wish to leave a room e.g. because the room is no longer relevant
to them, allowing them to specify a reason means they can easily step out of
the room quietly without having to send a message to explain their actions.
### Invite
This can be useful to give some context for an invite, e.g. "Alice invites Bob
to get some feedback on the membership reasons MSC".
### Rejecting an Invite
If Alice has invited Bob (and many others) to a room to discuss going to a
concert then Bob may wish to simply reject the invite if he can't make it.
Adding a "will be out of town" reason to the rejection helps Alice to know why
her invite was rejected.
### Joining room
Adding a reason for joining could be used e.g. by automated bots to say why
they're joining. For example a bridge bot may join a room when asked to bridge
the room to an external network, in which case they may have a reason such as
"BridgeBot joined to bridge the room to RemoteNetwork at the request of Alice".
## Potential Issues
The main issue here is ensuring that the invite reason cannot be used as an
abuse vector, however if clients follow the recommendations above this concern
should be mitigated.
---
¹ This is left up to implementations to decide, if they wish to do so.

100
proposals/2399-reporting-no-key-sent.md
Unescape Escape View File

@ -1,100 +0,0 @@
# Reporting that decryption keys are withheld
When an encrypted message is sent in a room, the megolm key might not be
sent to all devices present in the room. Sometimes this may be inadvertent (for
example, if the sending device is not aware of some devices that have joined),
but some times, this may be purposeful. For example, the sender may have
blacklisted certain devices or users, or may be choosing to not send the megolm
key to devices that they have not verified yet.
Currently, when this happens, there is no feedback given to the affected
devices; devices that have not received keys do not know why they did not
receive the key, and so cannot inform the user as to whether it is expected
that the message cannot be decrypted. To address this, this proposal defines a
message that senders can (optionally) send to devices indicating that they
purposely did not send a megolm key.
A similar issue happens with keyshare requests; devices are not informed when
other devices decide not to send back keys, and so do not know whether to
expect to receive a key in response to the request.
## Proposal
Devices that purposely do not send megolm keys to a device may instead send an
`m.room_key.withheld` event as a to-device message to the device to indicate
that it should not expect to receive keys for the message. This message may
also be sent in reply to a `m.room_key_request`. The `m.room_key.withheld` event has
the properties:
- `room_id`: Required if `code` is not `m.no_olm`. The ID of the room that the
session belongs to.
- `algorithm`: Required. The encryption algorithm that the key is for.
- `session_id`: Required if `code` is not `m.no_olm`. The ID of the session.
- `sender_key`: Required. The unpadded base64-encoded device curve25519 key of the session creator.
- `code`: Required. A machine-readable code for why the megolm key was not sent.
Possible values are:
- `m.blacklisted`: the user/device was blacklisted
- `m.unverified`: the user/devices is unverified
- `m.unauthorised`: the user/device is not allowed have the key. For
example, this would usually be sent in response to a key request if the
user was not in the room when the message was sent
- `m.unavailable`: sent in reply to a key request if the device that the key
is requested from does not have the requested key
- `m.no_olm`: an olm session could not be established. This may happen, for
example, if the sender was unable to obtain a one-time key from the
recipient.
- `reason`: A human-readable reason for why the key was not sent. The
receiving client should only use this string if it does not understand the
`code`.
An `m.room_key.withheld` event should only be sent once per session per target; the
recipient of the event should assume that the event applies to all messages in
that session. If a sender unblocks a recipient, it may re-use the existing
session for which the recipient was previously informed that it was blocked, in
which case, the recipient of the `m.room_key.withheld` message should assume
that the event applies to all messages in the session prior to the index of the
first key that it has received for that session.
A `code` of `m.no_olm` is used to indicate that the sender is unable to
establish an olm session with the recipient. When this happens, multiple
sessions will be affected. In order to avoid filling the recipient's device
mailbox, the sender should only send one `m.room_key.withheld` message with no
`room_id` nor `session_id` set. If the sender retries and fails to create an
olm session again in the future, it should not send another
`m.room_key.withheld` message with a `code` of `m.no_olm`, unless another olm
session was previously established successfully. In response to receiving an
`m.room_key.withheld` message with a `code` of `m.no_olm`, the recipient may
start an olm session with the sender and send an `m.dummy` message to notify
the sender of the new olm session. The recipient may assume that this
`m.room_key.withheld` message applies to all encrypted room messages sent
before it receives the message.
### Interaction with key sharing
If Alice withholds a megolm session from Bob for some messages in a room, and
then later on decides to allow Bob to decrypt later messages, she can send Bob
the megolm session, ratcheted up to the point at which she allows Bob to
decrypt the messages. If Bob logs into a new device and uses key sharing to
obtain the decryption keys, the new device will be sent the megolm sessions
that have been ratcheted up. Bob's old device can include the reason that the
session was initially not shared by including a `withheld` property in the
`m.forwarded_room_key` message that is an object with the `code` and `reason`
properties from the `m.room_key.withheld` message.
## Potential issues
This does not handle all possible reasons why a device may not have received
megolm keys.
There is currently no way of indicating that part of a session was withheld in
key backups.
## Security considerations
A user might not want to notify another user of the reason why it was not sent
the keys. Sending `m.room_key.withheld` is optional.
## Unstable prefix
While in development, clients will send events of type
`org.matrix.room_key.withheld`.

680
proposals/2403-knock.md
Unescape Escape View File

@ -1,680 +0,0 @@
# MSC2403: Add "knock" feature
Many people are in invite-only rooms. Sometimes, someone wants to join such a
room and can't, as they aren't invited. This proposal adds a feature for a
user to indicate that they want to join a room.
# Proposal
This proposal implements the reserved "knock" membership type for the
`m.room.member` state event. This state event indicates that when a user
knocks on a room, they are asking for permission to join. Like all membership
events, it contains an optional "reason" parameter to specify the reason you
want to join. Like other membership types, the parameters "displayname" and
"avatar_url" are optional. This membership can be sent by users who aren't
currently in said room. An example for the membership would look like the
following:
```json
{
"membership": "knock",
"displayname": "Alice",
"avatar_url": "mxc://example.org/avatar",
"reason": "I want to join this room as I really love foxes!"
}
```
After a knock in a room, a member of the room can invite the knocker, or they
can decide to reject it instead.
## Client-Server API
A new endpoint is introduced in the Client-Server API: `POST
/_matrix/client/r0/knock/{roomIdOrAlias}`. This allows the client to state
their intent to knock on a room.
Additionally, extensions to the `GET /_matrix/client/r0/sync` endpoint are
introduced. These allow a client to receive information about the status of
their knock attempt.
### `POST /_matrix/client/r0/knock/{roomIdOrAlias}`
Or the knocking equivalent of
[`POST
/_matrix/client/r0/join/{roomIdOrAlias}`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-join-roomidoralias).
The path parameter (`roomIdOrAlias`) is either the room ID or the alias of
the room you want to knock on. Additionally, several `server_name` parameters
can be specified via the query parameters. The post body accepts an optional
string parameter, `reason`, which is the reason you want to join the room. A
request could look as follows:
```json
POST /_matrix/client/r0/knock/%23foxes%3Amatrix.org?server_name=matrix.org&server_name=elsewhere.ca HTTP/1.1
Content-Type: application/json
{
"reason": "I want to join this room as I really love foxes!"
}
```
This endpoint requires authentication and can be rate limited.
#### Responses:
##### Status code 200:
The user knocked successfully. The room ID of the knocked on room is returned. Example
reply:
```json
{
"room_id": "!ZclcEpFTORTjmWIrqH:matrix.org"
}
```
##### Status code 403:
The user wasn't allowed to knock (e.g. they are banned). Example error reply:
```json
{
"errcode": "M_FORBIDDEN",
"error": "The user isn't allowed to knock in this room."
}
```
##### Status code 404:
The room was not found. Example error reply:
```json
{
"errcode": "M_NOT_FOUND",
"error": "Unknown room."
}
```
### Extensions to `GET /_matrix/client/r0/sync`
In [the response to
`/sync`](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-sync)
is a `rooms` field. This is a dictionary which currently contains keys
`join`, `invite` and `leave`, which each provide information to the client on
various membership states regarding the user.
It is proposed to add a fourth possible key to `rooms`, called `knock`. Its
value is a mapping from room ID to room information. The room information is
a mapping from a key `knock_state` to another mapping with key `events` being
a list of `StrippedStateEvent`. `StrippedStateEvent`s are defined as state
events that only contain the `sender`, `type`, `state_key` and `content`
keys.
Note that while `join` and `leave` keys in `/sync` use `state`, we use
`knock_state` here. This mirrors `invite`s use of `invite_state`.
These stripped state events contain information about the room, most notably
the room's name and avatar. A client will need this information to show a
nice representation of pending knocked rooms. The recommended events to
include are the join rules, canonical alias, avatar, name and encryption
state of the room, rather than all room state. This behaviour matches the
information sent to remote homeservers when remote users are invited to a
room.
This prevents unneeded state from the room leaking out, and also speeds
things up (think not sending over hundreds of membership events from big
rooms).
Also note that like `invite_state`, state events from `knock_state` are
purely for giving the user some information about the current state of the
room that they have knocked on. If the user was previously in the room, the
state events in `knock_state` are not intended to overwrite any historical
state. This applies storage of state on both the homeserver and the client.
The following is an example of knock state coming down `/sync`.
Request:
```
GET /_matrix/client/r0/sync HTTP/1.1
Content-Type: application/json
```
Response:
```json
{
...
"rooms": {
"knock": {
"!abcdefghijklmo:example.com": {
"knock_state": {
"events": [
{
"content": {
"join_rule": "knock"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.join_rules"
},
{
"content": {
"name": "Some cool room"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.name"
},
{
"content": {
"url": "mxc://example.com/xyz54321"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.avatar"
},
{
"content": {
"avatar_url": "mxc://example.org/abc1234",
"displayname": "Knocking User",
"membership": "knock"
},
"sender": "@knocking_user:example.org",
"state_key": "@knocking_user:example.org",
"type": "m.room.member",
}
]
}
}
}
},
...
}
```
### Changes regarding the Public Rooms Directory
A problem arises for discovery of knockable rooms. Ideally one wouldn't have
to send their colleagues a room ID for a room that they need to knock on. One
of these methods for room discovery is the [public rooms
directory](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-publicrooms),
which allows us to explore a list of rooms we may be able to join.
The spec does not prevent us from adding rooms with 'knock' join_rules to the
public rooms directory. However, a user attempting
to join a room in the directory will not know whether to directly attempt a
join, or to knock first. The current content of a `PublicRoomsChunk` does not
contain this information:
```json
{
"avatar_url": "mxc://bleecker.street/CHEDDARandBRIE",
"guest_can_join": false,
"name": "CHEESE",
"num_joined_members": 37,
"room_id": "!ol19s:bleecker.street",
"topic": "Tasty tasty cheese",
"world_readable": true
}
```
Therefore this proposal adds `join_rule` as a new, optional field to a
`PublicRoomsChunk`. The `join_rule` of knockable rooms will be `knock`,
thus giving clients the information they need to attempt entry of a
room when a client selects it. It also allows clients to display
knockable rooms differently than publicly joinable ones.
For backwards compatibility with old servers, the default value of
`join_rule` is `public`.
### Push Rules
To help knocks be noticed earlier, it would be nice to send a push
notification to those in the room who can act on a knock when it
comes in, rather than everyone in the room. This would require a
push rule to fire only when that user's power level is high enough to
accept or reject a knock.
With the current push rules implementation it is possible to place a
condition on the sender's power level, but unfortunately the same does
not exist for event recipients.
This MSC thus does not propose any changes to push rules at this time,
but acknowledges that it would be useful for a future MSC to address when
the underlying push rules architecture can support it.
## Server-Server API
Similarly to [join](https://matrix.org/docs/spec/server_server/r0.1.4#joining-rooms)
and [leave](https://matrix.org/docs/spec/server_server/r0.1.4#leaving-rooms-rejecting-invites)
over federation, a ping-pong game with two new endpoints is introduced: `make_knock`
and `send_knock`. Both endpoints must be protected via server ACLs.
### `GET /_matrix/federation/v1/make_knock/{roomId}/{userId}`
Asks the receiving server to return information that the sending server will
need to prepare a knock event.
Request format:
| Parameter | Type | Description |
|-----------|------|-------------|
| Path parameters:
| roomId | string | Required. The room ID that should receive the knock.
| userId | string | Required. The user ID the knock event will be for.
| Query Parameters:
| ver | [string] | Required. The room versions the sending server has support for.
Note that `GET /_matrix/federation/v1/make_join/{roomId}/{userId}` does not make `ver`
a required query parameter for backwards compatibility reasons. We have no such restrictions.
Response Format:
| Parameter | Type | Description |
|-----------|------|-------------|
| room_version | string | The version of the room where the server is trying to knock.
| event | Event Template | An unsigned template event. May differ between room versions.
#### Responses
##### Status code 200:
Returns a template to be used to knock on rooms. May depend on room version.
```json
{
"room_version": "2",
"event": {
"type": "m.room.member",
"room_id": "!somewhere:example.org",
"content": {
"membership": "knock"
},
"state_key": "@someone:example.org",
"origin": "example.org",
"origin_server_ts": 1549041175876,
"sender": "@someone:example.org"
}
}
```
##### Status code 400:
This request was invalid, e.g. bad JSON. Example reply:
```json
{
"errcode": "M_INCOMPATIBLE_ROOM_VERSION",
"error": "Your homeserver does not support the features required to join this room",
"room_version": "3"
}
```
##### Status code 403:
This request is forbidden, e.g. the user is banned from the room. Example reply:
```json
{
"errcode": "M_FORBIDDEN",
"error": "You are not allowed to knock on this room"
}
```
##### Status code 404:
The room is unknown to the remote server. Example reply:
```json
{
"errcode": "M_NOT_FOUND",
"error": "Unknown room"
}
```
### `PUT /_matrix/federation/v1/send_knock/{roomId}/{eventId}`
Submits a signed knock event to the resident homeserver for it to accept into
the room's graph. Note that event format may differ between room versions.
Note that in the past all `send_*` federation endpoints were updated to `/v2`
to remove a redundant HTTP error code from the return body. While we don't
have the same redundancy here, we start off at `/v1` for this new endpoint
as per
[MSC2844](https://github.com/matrix-org/matrix-doc/pull/2844).
Request format:
| Parameter | Type | Description |
|-----------|------|-------------|
| Path parameters:
| roomId | string | Required. The room ID that should receive the knock.
| eventId | string | Required. The event ID for the knock event.
The JSON body is expected to be the full event.
Response Format:
| Parameter | Type | Description |
|-----------|------|-------------|
| `knock_room_state` | [StrippedStateEvent] | Required. State events providing public room metadata
A request could look as follows:
```json
PUT /_matrix/federation/v1/send_knock/%21abc123%3Amatrix.org/%24abc123%3Aexample.org HTTP/1.1
Content-Type: application/json
{
"sender": "@someone:example.org",
"origin": "matrix.org",
"origin_server_ts": 1234567890,
"type": "m.room.member",
"state_key": "@someone:example.org",
"content": {
"membership": "knock",
"displayname": "Alice",
"avatar_url": "mxc://example.org/avatar",
"reason": "I want to join this room as I really love foxes!"
}
}
```
#### Response:
##### Status code 200:
The event was successfully accepted into the graph by the homeserver that
received the knock. It must then send this knock into the room, before
responding to the knocking homeserver, indicating the knock succeeded.
The response contains `StrippedStateEvent`s with room metadata (room name,
avatar ...) that the knocking homeserver can pass to the client. The event
types that can be sent here should match those of the `/sync` extensions
mentioned above.
This is loosely based on the
[federated invite](https://matrix.org/docs/spec/server_server/r0.1.4#put-matrix-federation-v2-invite-roomid-eventid)
request content.
```json
{
"knock_room_state": [
{
"content": {
"join_rule": "knock"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.join_rules"
},
{
"content": {
"name": "Some cool room"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.name"
},
{
"content": {
"url": "mxc://example.com/xyz54321"
},
"sender": "@room_admin:example.com",
"state_key": "",
"type": "m.room.avatar"
},
{
"content": {
"avatar_url": "mxc://example.org/abc1234",
"displayname": "Knocking User",
"membership": "knock"
},
"sender": "@knocking_user:example.org",
"state_key": "@knocking_user:example.org",
"type": "m.room.member",
}
]
}
```
##### Status code 403:
This request is forbidden, e.g. the user is banned from the room. Example reply:
```json
{
"errcode": "M_FORBIDDEN",
"error": "You are not allowed to knock on this room"
}
```
##### Status code 404:
The room is unknown to the remote server. Example reply:
```json
{
"errcode": "M_NOT_FOUND",
"error": "Unknown room"
}
```
## Restrictions
There are restrictions to being able to set this membership, as well as
accepting or denying the knock. A formal description of the changes to the auth rules is given below;
first we summarise the semantics of the proposed changes.
### Current membership
Only users without a current membership or with their current membership
set to "knock" or "leave" can knock on a room. This means that a user that
is banned, is invited or is currently in the room cannot knock on it.
### Join Rules
This proposal makes use of the existing "knock" join rule. The value of
`join_rule` in the content of the `m.room.join_rules` state event for a room
must be set to "knock" for a knock to succeed. This means that existing rooms
will need to opt into allowing knocks in their rooms. Other than allowing
knocks, a join rule of "knock" is functionally equivalent to that of
"invite", except that it additionally allows external users to change their
membership to "knock" under certain conditions.
### Auth rules
Each room version defines the auth rules which should be applied in that room version.
This MSC proposes a new room version with the following changes to the [auth
rules from room version 6](https://matrix.org/docs/spec/rooms/v6#authorization-rules-for-events):
* Under "5. If type is `m.room.member`", insert the following after "e. If membership is `ban`":
```
f. If `membership` is `knock`:
i. If the `join_rule` is anything other than `knock`, reject.
ii. If `sender` does not match `state_key`, reject.
iii. If the `sender`'s membership is not `ban`, `invite` or `join`, allow.
iv. Otherwise, reject.
```
Note that:
- Both the `sender` and `state_key` fields are set to the user ID of the knocking
user. This is different to an `invite` membership event, where the `sender` is the inviter and
the `state_key` is the invitee.
- f.ii is justified as one user should not be able to knock on behalf of
another user.
- f.iii is justified as knocks should not be allowed if the knocking user
has been banned from the room, is invited to the room or if they are already
in the room.
- Knocks are not restricted by power level like invites are. The `join_rules`
are already used to enforce whether someone can or cannot knock. However,
power level rules do apply when approving or denying the knock, as discussed
in the Membership Changes section below.
Additionally, note that redactions of knock events are not a concern, as
`membership` keys are excluded from being redacted as defined by all current
room versions.
## Membership changes
Once someone has sent a `knock` membership into the room, the membership for
that user can be transitioned to the following possible states:
- `invite`: In this case, the knock was accepted by someone inside the room
and they are inviting the knocker into the room.
- `leave`: In this case, similar to how kicks are handled, the knock has
been rejected. Alternatively, the knocking user has rescinded their knock.
- `ban`: In this case, the knock was rejected and the user has been prevented
from sending further knocks.
Let's talk about each one of these in more detail.
### Membership change to `invite`
The knock has been accepted by someone in the room.
The user who is accepting the knock must have the power level to perform
invites. The accepting user's homeserver will then send an invite - over federation if
necessary - to the knocking user. The knocking user may then join the room as
if they had been invited normally.
To accept a knock, the client should call [`POST
/_matrix/client/r0/rooms/{roomId}/invite`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-invite)
with the user ID of the knocking user in the JSON body.
If the knocking user is on another homeserver, then the homeserver of the
accepting user will call [`PUT
/_matrix/federation/v2/invite/{roomId}/{eventId}`](https://matrix.org/docs/spec/server_server/r0.1.4#put-matrix-federation-v2-invite-roomid-eventid)
on the knocking homeserver to inform it that the knock has been accepted.
The knocking homeserver should assume an invite to a room it has knocked on means
that its knock has been accepted, even if the invite was not explicitly
related to the knock attempt.
Note that client or homeserver implementations are free to automatically
accept this invite given they're aware that it's the result of a previous
knock. In case of failing to auto-accept an invite on the homeserver, it's
recommended for homeservers to pass the invite down to the client so that it
may try at a later point (or reject the potentially broken invite) at the user's
discretion.
### Membership change to `leave` via rejecting a knock
The knock has been rejected by someone in the room.
To reject a knock, the rejecting user's client must call [`POST
/_matrix/client/r0/rooms/{roomId}/kick`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-kick)
with the user ID of the knocking user in the JSON body. Rejecting a knock
over federation has a slight catch, though.
When the knocking user is on another homeserver, the homeserver of the
rejecting user needs to send the `leave` event over federation to the
knocking homeserver. However, this is a bit tricky as it is currently very
difficult to have events from a room propagate over federation when the
receiving homeserver is not in the room. This is due to the remote homeserver
being unable to verify that the event being sent is actually from a
homeserver in the room - and that the homeserver in the room had the required
power level to send it. This is a problem that currently affects other,
similar operations, such as disinviting or unbanning a federated user. In
both cases, they won't be notified as their homeserver is not in the room.
While we could easily send the leave event as part of a generic
transaction to the remote homeserver, that homeserver would have no way to
validate the `prev_events` and `auth_events` that the event references. We
could send those events over as well, but those will also reference other
events that require validation and so on.
A simple thing we could easily do here is to trust the leave event implicitly
if it is sent by the homeserver we originally knocked through. We know this
homeserver is (or at least was) in the room, so they're probably telling the
truth. This is almost an edge case though, as while you'll knock through one
homeserver in the room, there's no guarantee that the admin that denies your
knock will be on the same homeserver you knocked through. Perhaps the
homeserver you knocked through could listen for this and then send the event
back to you - but what if it goes offline in the meantime?
As such, informing remote homeservers about the rejection of knocks over
federation is de-scoped for now, and left to a future MSC which can solve
this class of problem in a suitable way. Rejections should still work for the
homeservers that are in the room, as they can validate the leave event for
they have access to the events it references.
### Membership change to `leave` via rescinding a knock
The knocking user has rescinded their knock.
To rescind a knock, the knocking user's client must call [`POST
/_matrix/client/r0/rooms/{roomId}/leave`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-leave).
To rescind a knock over federation, the knocking homeserver must complete
a [`make_leave`, `send_leave` dance](
https://matrix.org/docs/spec/server_server/r0.1.4#leaving-rooms-rejecting-invites)
with a homeserver in the room.
### Membership change to `ban`
The knock has been rejected by someone in the room and the user has been
banned, and is unable to send further knocks.
This one is fairly straightforward. Someone with the appropriate power levels
in the room bans the user. This will have the same effect as rejecting the
knock, and in addition prevent any further knocks by this user from being
allowed into the room.
If the user is unbanned, they will be able to send a new knock which could be
accepted.
To ban the user, the client should call [`POST
/_matrix/client/r0/rooms/{roomId}/ban`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-ban) with the user ID of the knocking user in the JSON body.
Informing the knocking user about the update is the same as rejecting the
knock.
# Potential issues
This new feature would allow users to send events into rooms that they don't
partake in. That is why this proposal enables the `knock` join rule, in
order to allow room admins to opt in to this behaviour.
# Alternatives
The possibility of initiating a knock by sending EDUs between users instead of sending
a membership state event into a room has been raised. This is an ongoing discussion
occurring at https://github.com/matrix-org/matrix-doc/pull/2403/files#r573180627.
# Client UX recommendations
After a knock is received in a room, it is expected to be displayed in the
timeline, similar to other membership changes. Clients can optionally add a way
for users of a room to review all current knocks.
Please also note the recommendations for clients in the "Security considerations"
section below.
# Security considerations
Clients must take care when implementing this feature in order to prevent
simple abuse vectors that can be accomplished by individual users. For
instance, when a knock occurs, clients are advised to hide the reason until
the user interacts with the client in some way (e.g. clicking on a "show
reason" button). The user should reveal the reason only if they choose to.
It is recommended to not display the reason by default as else this would
essentially allow outsiders to send messages into the room.
It is still theoretically possible for a homeserver admin to create many users
with different user IDs or display names, all spelling out an abusive
message, and then having each of them knock in order.
Clients should also do their best to prevent impersonation attacks. Similar to
joins, users can set any displayname or avatar URL they'd like when knocking on
a room. Clients SHOULD display further information to help identify the user,
such as User ID, encryption verification status, rooms you share with the user,
etc. Care should be taken to balance the importance of preventing attacks while
avoiding overloading the user with too much information or raising false
positives.
Another abuse vector is allowed by the ability for users to rescind knocks.
This is to help users in case they knocked on a room accidentally, or simply
no longer want to join a room they've knocked on. While this is a useful
feature, it also allows users to spam a room by knocking and rescinding their
knocks over and over. Particularly note-worthy is that this will generate
state events that homeservers in the room will need to process. And while
join/leave state changes will do the same in a public room, the act of
knocking is much lighter than the act of joining a room.
In both cases, room admins should employ typical abuse mitigation tools, such
as user bans and server ACLs. Clients are encouraged to make employing these
tools easy even if the offensive user or server is not present in the room.
# Unstable prefix
An unstable feature flag is not required due to this proposal's requirement
of a new room version. Clients can check for a room version that includes
knocking via the Client-Server API's [capabilities
endpoint](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-capabilities).
Experimental implementation should use `xyz.amorgan.knock` as a room version identifier.
The new endpoints should contain an unstable prefix during experimental
implementation. The unstable counterpart for each endpoint is:
C-S knock:
* `POST /_matrix/client/knock/{roomIdOrAlias}`
* `POST /_matrix/client/unstable/xyz.amorgan.knock/knock/{roomIdOrAlias}`
S-S make_knock:
* `GET /_matrix/federation/v1/make_knock/{roomId}/{userId}`
* `GET /_matrix/federation/unstable/xyz.amorgan.knock/make_knock/{roomId}/{userId}`
S-S send_knock:
* `PUT /_matrix/federation/v1/send_knock/{roomId}/{eventId}`
* `PUT /_matrix/federation/unstable/xyz.amorgan.knock/send_knock/{roomId}/{eventId}`
Finally, an unstable prefix is added to the key that comes down `/sync`,
the join rule for rooms and the `content.membership` key of the member
event sent into rooms when a user has knocked successfully. Instead of
`knock`, experimental implementations should use `xyz.amorgan.knock`.

45
proposals/2414-optional-content-reporting-reason.md
Unescape Escape View File

@ -1,45 +0,0 @@
# MSC2414: Make `reason` and `score` optional for reporting content
## Proposal
On the [report content endpoint](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-report-eventid)
we remove the `required` flag for both the `reason` and `score` parameters, as
well as the "may be blank" clause in the description of `reason`.
## Rationale
### `reason` Parameter
Currently, the spec says that the `reason` parameter on the content reporting
endpoint is required, but also says that the string "may be blank." This
seems to be a contradiction.
Note that the kicking and banning endpoints already have optional `reason`
parameters. The other membership endpoints mentioned in
[#2367][membership-endpoints] will also add optional `reason` parameters,
so it would be more more consistent with the rest of the spec to make this
optional as well.
### `score` Parameter
The spec also requires the `score` parameter, but its usefulness is limited.
Offensiveness is difficult to measure and is likely not going to be applied
consistently across several rooms. Because of this ambiguity, it seems, many
clients [simply hard-code the integer value][hard-code].
To make this useful, for example, room administrators would need a way to map more
specific values to the integer range and perhaps even instruct the client to
display those mappings to the user. That may be possible to do in a closed
client/homeserver implementation, but not generally across the Matrix protocol.
Making `score` optional would enable this feature to be used in specific contexts
while not forcing clients to support the ambiguity it brings.
## Backwards Compatibility
Since servers currently expect these fields to be sent by all clients, making
them optional is a breaking change. Clients should check the spec versions
the homeserver supports to detect this change.
[membership-endpoints]: https://github.com/matrix-org/matrix-doc/pull/2367
[hard-code]: https://github.com/matrix-org/matrix-react-sdk/pull/3290/files#diff-551ca16d6a8ffb96888b337b5246402dR66

39
proposals/2422-allow-color-attribute-on-font-tag.md
Unescape Escape View File

@ -1,39 +0,0 @@
# MSC2422: Allow `color` as attribute for `<font>` in messages
Currently the spec recommends that you to use `data-mx-color` instead of the standard
`color` html attribute for the `<font>` tag. This is probably done to make it
consistent with `<span>`, where you may not want to allow a generic style tag for.
On the other hand the /rainbow command on almost every client just uses the
`color` attribute of the `<font>` tag. While some clients support
`data-mx-color` (i.e. Riot Web), most clients don't. Most clients support
rendering `color` however.
It would probably be for the best to allow or even prefer `color` on the
`<font>` tag.
## Proposal
Add the `color` attribute to the allowed attributes of `<font>` in section
13.2.1.7. No changes to the allowable values from the HTML spec are made here.
## Potential issues
- We now have a redundant attribute in the spec. While it matches what the
clients currently do, it may be better to fix each client instead.
- Clients may not sanitize the color attribute and will let other color values
through, increasing compatibility issues again.
- Clients may never support the data-mx-* attributes now.
- Old messages could loose their color
- This proposal doesn't touch span at all, maybe it should?
## Alternatives
- fix the clients
-> This currently seems not feasible. Multiple clients started using color first (i.e. RiotX, Gomuks) and if it isn't spelled out explicitly in the spec, this will probably continue.
- remove the `data-mx-color` and `data-mx-bg-color` attributes entirely, leaving us just with `color` for `<font>`
-> This would break old messages and can be done independently of this proposal at a later date, if it is deemed useful.
- Add a section to tell the clients to prefer `color` over `mx-data-color`
-> I don't really know, why mx-data-* was chosen, but I assume there was a reason, so I don't want to change that.
- Spec an entirely different format for messages (that would probably not make this proposal obsolete)
-> This wouldn't fix the issue, where some client may choose to remove the color tag, since it is discouraged in the spec. Migration would probably also take a while, so this proposal is a quick solution, that doesn't prevent other solutions at a later date.

252
proposals/2432-revised-alias-publishing.md
Unescape Escape View File

@ -1,252 +0,0 @@
# MSC2432: Updated semantics for publishing room aliases
This MSC offers an alternative to [MSC2260](https://github.com/matrix-org/matrix-doc/issues/2260).
## Background
The [`m.room.aliases`](https://matrix.org/docs/spec/client_server/r0.6.0#m-room-aliases)
state event exists to list the available aliases for a given room. This serves
two purposes:
* It allows existing members of the room to discover alternative aliases,
which may be useful for them to pass this knowledge on to others trying to
join.
* Secondarily, it helps to educate users about how Matrix works by
illustrating multiple aliases per room and giving a perception of the size
of the network.
However, it has problems:
* Any user in the entire ecosystem can create aliases for rooms, which are
then unilaterally added to `m.room.aliases`, and room admins are unable to
remove them. This is an abuse
vector (https://github.com/matrix-org/matrix-doc/issues/625).
* For various reasons, the `m.room.aliases` event tends to get out of sync
with the actual aliases (https://github.com/matrix-org/matrix-doc/issues/2262).
## Proposal
We propose that that room moderators should be able to manually curate a list
of "official" aliases for their room, instead of matrix servers automatically
publishing lists of all room aliases into the room state. No particular
guarantees are offered that this alias list is entirely accurate: it becomes
room moderators' responsibility to keep it so.
Meanwhile, the aliases that map to a given room on a given server become
the ultimate responsibility of the administrators of that server. We give them
tools to inspect the alias list and clean it up when necessary, in addition to
the current tools which allow restriction of who can create aliases in the
first place.
A detailed list of proposed modifications to the Matrix spec follows:
* `m.room.aliases` loses any special meaning within the spec. In particular:
* Clients should no longer format it specially in room timelines.
* Clients and servers should no longer consider `m.room.aliases` when
[calculating the display name for a
room](https://matrix.org/docs/spec/client_server/r0.6.0#calculating-the-display-name-for-a-room).
(Note: servers follow the room display-name algorithm when calculating
room names for certain types of push notification.)
* A future room version will remove the special [authorization
rules](https://matrix.org/docs/spec/rooms/v1#authorization-rules) and
[redaction rules](https://matrix.org/docs/spec/client_server/r0.6.0#redactions).
* [`m.room.canonical_alias`](https://matrix.org/docs/spec/client_server/r0.6.0#m-room-canonical-alias)
is extended to include a new `alt_aliases` property. This, if present,
should be a list of alternative aliases for the room. An example event might
look like:
```json
{
"content": {
"alias": "#somewhere:localhost",
"alt_aliases": [
"#somewhere:overthere.com",
"#somewhereelse:example.com"
]
},
"room_id": "!jEsUZKDJdhlrceRyVU:example.org",
"state_key": "",
"type": "m.room.canonical_alias"
}
```
It is valid for `alt_aliases` to be non-empty even if `alias` is absent or
empty. This means that no alias has been picked out as the 'main' alias.
(Note: although the spec currently claims that `alias` is mandatory, Synapse
generates `m.room.canonical_alias` events with no `alias` property when the
main alias is deleted. This change would legitimise that behaviour.)
(For clarity: it is not proposed that the `alt_aliases` be considered when
calculating the displayname for a room.)
* [`PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-rooms-roomid-state-eventtype-statekey)
is extended to recommend that servers validate any *new* aliases added to
`m.room.canonical_alias` by checking that it is a valid alias according to
the [syntax](https://matrix.org/docs/spec/appendices#room-aliases), and by
looking up the alias and and that it corresponds to the expected room ID.
(Note: Synapse currently implements this check on the main alias, though
this is unspecced.)
The following error codes are specified:
* HTTP 400, with `errcode: M_INVALID_PARAMETER` if an attempt is made to add
an entry which is not a well-formed alias (examples: too long, doesn't
start with `#`, doesn't contain a `:`).
* HTTP 400, with `errcode: M_BAD_ALIAS` if an added alias does not point at
the given room (either because the alias doesn't exist, because it can't
be resolved due to an unreachable server, or because the alias points at a
different room).
* Currently, [`PUT /_matrix/client/r0/directory/room/{roomAlias}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-directory-room-roomalias)
attempts to send updated `m.room.aliases` events on the caller's
behalf. (This is implemented in Synapse but does not appear to be explicitly
specced.) This functionality should be removed.
* Currently, [`DELETE /_matrix/client/r0/directory/room/{roomAlias}`](https://matrix.org/docs/spec/client_server/r0.6.0#delete-matrix-client-r0-directory-room-roomalias),
attempts to send updated `m.room.aliases` and/or `m.room.canonical_alias`
events on the caller's behalf, removing any aliases which have been
deleted. (Again, this is implemented in Synapse but does not appear to be
explicitly specced.) The `m.room.aliases` functionality should be removed,
and the `m.room.canonical_alias` functionality should be extended to cover
`alt_aliases`.
The behaviour if the calling user has permission to delete the alias but
does not have permission to send `m.room.canonical_alias` events in the room
(for example, by virtue of being a "server administrator", or by being the
user that created the alias) is implementation-defined. It is *recommended*
that in this case, the alias is deleted anyway, and a successful response is
returned to the client.
* A new api endpoint, `GET /_matrix/client/r0/rooms/{roomId}/aliases` is
added, which returns the list of aliases currently defined on the local
server for the given room. The response looks like:
```json
{
"aliases": [
"#somewhere:example.com",
"#somewhereelse:example.com",
"#special_alias:example.com"
]
}
```
This API can be called by any current member of the room (calls from other
users result in `M_FORBIDDEN`). For rooms with `history_visibility` set to
`world_readable`, it can also be called by users outside the room.
Servers might also choose to allow access to other users such as server
administrators.
* [`GET /_matrix/client/r0/publicRooms`](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-publicrooms)
(and the `POST` variant) no longer return `aliases` as part of `PublicRoomsChunk`.
Clients do not appear to make use of this field, and `canonical_alias` is maintained
to provide similar information.
Various APIs are currently subject to implementation-defined access
restrictions. No change to the specification is introduced in this regard
(implementations will continue to be free to impose their own
restrictions). Nevertheless as part of this MSC it is useful to consider some
proposed changes to Synapse's implementation:
* No change: `PUT /_matrix/client/r0/directory/room/{roomAlias}`: Synapse
only allows access to current members of the room, and also exposes some
configuration options which allow restriction of which users are allowed to
create aliases in general.
* `DELETE /_matrix/client/r0/directory/room/{roomAlias}`: in this case,
currently Synapse restricts its use to the user that created the alias, and
server admins.
It is proposed to extend this to local users who have a power-level
sufficient to send an `m.room.canonical_alias` event in the room that the
alias currently points to.
* [`PUT /_matrix/client/r0/directory/list/room/{roomId}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-directory-list-room-roomid)
and the corresponding unspecced `DELETE` api (both of which set the
visibility of a room in the public directory): currently Synapse restricts
their use to server admins and local users who have a PL sufficient to send
an `m.room.aliases` event in the room (ignoring the special auth
rules). This will be changed to check against the PL required to send an
`m.room.canonical_alias` event.
It is envisaged that Matrix clients will then change their "Room Settings" user
interface to display the aliases from `m.room.canonical_alias` instead of those
in `m.room.aliases`, as well as giving moderators the ability to update that
list. Clients might also wish to use the new `GET
/_matrix/client/r0/rooms/{roomId}/aliases` endpoint to obtain and display the
currently-available local aliases, though given that this list may be subject
to abuse, it should probably not be shown by default.
### Future work
This work isn't considered part of this MSC, but rather a potential extension
for the future.
* It may be useful to be able to query remote servers for their alias
list. This could be done by extending `GET
/_matrix/client/r0/rooms/{roomId}/aliases` to take a `server_name`
parameter, and defining an API in the server_server spec which will expose
the requested information, subject to the calling homeserver having at least
one user with a right to see it.
* Similarly, room moderators may wish to be able to delete aliases on a remote
server for their room. We could envisage a federation API which allows such
a request to be made, subject to the calling homeserver having at least one
moderator in the room.
## Potential issues
The biggest problem with this proposal is that existing clients, which rely on
`m.room.aliases` in one way or another, will lose functionality. In particular,
they may not know about aliases that exist, or they may look at outdated
`m.room.aliases` events that list aliases that no longer exist. However, since
`m.room.aliases` is best-effort anyway, these are both problems that exist to
some extent today.
## Alternatives
We considered continuing to use `m.room.aliases` to advertise room aliases
instead of `m.room.canonical_alias`, but the significant changes in semantics
made that seem inappropriate.
We also considered using separate state events for each advertised alias,
rather than listing them all in one event. This might increase the number of
aliases which can be advertised, and help to reduce races when editing the
list. However, the 64KB limit of an event still allows room for hundreds of
aliases of any sane length, and we don't expect the list to be changing
frequently enough for races to be a practical concern. Ultimately the added
complexity seemed redundant.
A previous suggestion was
[MSC2260](https://github.com/matrix-org/matrix-doc/issues/2260), which proposed
keeping `m.room.aliases` largely as-is, but giving room moderators tools to
control who can send them via room power-levels. We dismissed it for the
reasons set out at
https://github.com/matrix-org/matrix-doc/pull/2260#issuecomment-584207073.
## Security considerations
None currently identified.
## Unstable prefix
While this feature is in development, the following names will be in use:
| Proposed final name | Name while in development |
| --- | --- |
| `GET /_matrix/client/r0/rooms/{roomId}/aliases` | `GET /_matrix/client/unstable/org.matrix.msc2432/rooms/{roomId}/aliases` |
Servers will indicate support for the new endpoint via a non-empty value for feature flag
`org.matrix.msc2432` in `unstable_features` in the response to `GET
/_matrix/client/versions`.

56
proposals/2451-remove-query_auth-federation-endpoint.md
Unescape Escape View File

@ -1,56 +0,0 @@
# MSC2451: Remove the `query_auth` federation endpoint
This API was added without sufficient thought nor testing. The endpoint isn't
used in any known implementations, and we do not believe it to be necessary
for federation to work. The only known implementation (in Synapse) was not fully
fleshed out and is broken.
For background, the idea behind this endpoint was that two homeservers would be
able to share state events with the hope of filling in missing state from one
of homeservers allowing state resolution to complete. This was to protect
against a joining server not providing the full (or providing stale) state.
In addition to the ideas above not coming to fruition, it is unclear whether the
current design of this endpoint would be sufficient. If this state negotiation
feature is needed in the future it should be redesigned from scratch via the MSC
proposal process.
## Proposal
Remove the following endpoint:
* [POST `/_matrix/federation/v1/query_auth/{roomId}/{eventId}`](https://matrix.org/docs/spec/server_server/r0.1.3#post-matrix-federation-v1-query-auth-roomid-eventid)
## Potential issues
Removing this endpoint impacts backwards compatibility, in practice removing
this endpoint should have minimal impact as it was an unused error path in
Synapse. The federation client code to call this endpoint was removed in Synapse
v1.5.0rc1.
There is no evidence of other homeserver implementations having implemented this
endpoint.
### History
This endpoint (and the federation client code) to call it was initially
added in Synapse v0.7.0 (see [#43](https://github.com/matrix-org/synapse/pull/43)).
The federation client code was heavily modified for v1.0.0rc1 (see
[#5227](https://github.com/matrix-org/synapse/pull/5227/)),
The federation client code to call this endpoint was removed in v1.5.0rc1 of
Synapse (see [#6214](https://github.com/matrix-org/synapse/pull/6214). After
that point this endpoint is not called).
During removal it was noted that the code to call this endpoint was already
unreachable. It seems that this code was never reachable and was meant for an
error situation which was never built out (see `git log -S NOT_ANCESTOR`, the
error condition is never assigned).
## Alternatives
The endpoint could be deprecated and removed in a future version of the specification.
## Security considerations
None.

230
proposals/2454-ui-interactive-auth-for-sso.md
Unescape Escape View File

@ -1,230 +0,0 @@
# User-Interactive Authentication for SSO-backed homeserver
Certain endpoints, such as `DELETE /_matrix/client/r0/devices/{deviceId}` and
`POST /_matrix/client/r0/account/3pid/add`, require the user to reconfirm their
identity, as a guard against a leaked access token being used to take over an
entire account.
On a normal homeserver, this is done by prompting the user to enter their
password. However, on a homeserver where users authenticate via a single-sign-on
system, the user doesn't have a password registered with the homeserver. Instead
we need to delegate that check to the SSO system.
At the protocol level, this means adding support for SSO to the
[user-interactive authentication API](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api).
In theory, once SSO is added as a possible flow for authentication, any clients
that already implement the [fallback process for unknown authentication types](https://matrix.org/docs/spec/client_server/r0.6.0#fallback)
will work fine without modification. It is unknown whether this is widely
supported among clients.
## Proposal
An [additional authentication type](https://matrix.org/docs/spec/client_server/r0.6.0#authentication-types)
of `m.login.sso` is added to the user-interactive authentication specification.
There are no additional parameters as part of this authentication type. As per
the user-interactive authentication specification, the only parameter included in
the `auth` dictionary should be the session ID from the homeserver, e.g.:
```json
{
"auth": {
"session": "<session ID>"
}
}
```
### Detailed fallback authentication flow:
The following is a re-iteration of the [fallback authentication flow](https://matrix.org/docs/spec/client_server/r0.6.0#fallback),
but with details filled in for the proposed new authentication type.
When choosing this authentication flow, the following should occur:
1. If the client wants to complete authentication using SSO, it opens a browser
window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`
with session set to the UI-Auth session id (from the "auth" dict).
The homeserver returns a page which asks for the user's confirmation before
proceeding. See the security considerations section below for why this is
necessary. For example, the page could say words to the effect of:
> A client is trying to remove a device/add an email address/take over your
> account. To confirm this action, **re-authenticate with single sign-on**.
> If you did not expect this, your account may be compromised!
2. The link, once the user clicks on it, goes to the SSO provider's page.
3. The SSO provider validates the user, and redirects the browser back to the
homeserver.
4. The homeserver validates the response from the SSO provider, updates the
user-interactive auth session to show that the SSO has completed, and
[serves the fallback auth completion page as specced](https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
5. The client resubmits its original request, with its original session id,
which now should complete.
Note that the post-SSO URL on the homeserver is left up to the homeserver
implementation rather than forming part of the specification, choices might be
limited by the chosen SSO implementation, for example:
* SAML2 servers typically only support one URL per service provider, so in
practice it will need to be the same as that already used for the login flow
(for synapse, it's `/_matrix/saml2/authn_response`) - and the server needs to
be able to figure out if it's doing SSO for a login attempt or an SSO
attempt.
* CAS doesn't have the same restriction.
### Example flow:
A more complete example is provided below in which a user attempts to delete
a device and is pushed into the user interactive authentication process with
SSO being the only possible flow.
0. Client submits the request, which the server says requires SSO:
```
POST /_matrix/client/r0/delete_devices HTTP/1.1
Content-Type: application/json
Authorization: Bearer xyzzy
{
"devices": ["FSVVTZRRAA"]
}
HTTP/1.1 401 Unauthorized
Content-Type: application/json
{
"flows": [
{
"stages": [
"m.login.sso"
]
}
],
"params": {},
"session": "dTKfsLHSAJeAhqfxUsvrIVJd"
}
```
1. Client opens a browser window for the fallback endpoint:
```
GET /_matrix/client/r0/auth/m.login.sso/fallback/web
?session=dTKfsLHSAJeAhqfxUsvrIVJd HTTP/1.1
HTTP/1.1 200 OK
<body>
A client is trying to remove a device from your account. To confirm this
action, <a href="https://sso_provider/validate?SAMLRequest=...">re-authenticate with single sign-on</a>.
If you did not expect this, your account may be compromised!
</body>
```
2. The user clicks the confirmation link which goes to the SSO provider's site:
```
GET https://sso_provider/validate?SAMLRequest=<etc> HTTP/1.1
<SAML/CAS or other SSO data>
```
3. The SSO provider validates the user and ends up redirecting the browser back
to the homeserver. The example below shows a 302 for simplicity, this might
vary based on SSO implementation.
```
HTTP/1.1 302 Moved
Location: https://homeserver/_matrix/saml2/authn_response?
SAMLResponse=<etc>
```
4. The browser sends the SSO response to the homeserver, which validates it and
shows the fallback auth completion page:
```
GET /_matrix/saml2/authn_response?SAMLResponse=<etc>
HTTP/1.1 200 OK
<script>
if (window.onAuthDone) {
window.onAuthDone();
} else if (window.opener && window.opener.postMessage) {
window.opener.postMessage("authDone", "*");
}
</script>
<p>Thank you.</p>
<p>You may now close this window and return to the application.</p>
```
5. The client closes the browser popup if it is still open, and resubmits its
original request, which now succeeds:
```
POST /_matrix/client/r0/delete_devices HTTP/1.1
Content-Type: application/json
Authorization: Bearer xyzzy
{
"auth": {
"session": "dTKfsLHSAJeAhqfxUsvrIVJd"
}
}
HTTP/1.1 200 OK
Content-Type: application/json
{}
```
## Alternatives
An alternative client flow where the fallback auth ends up redirecting to a
given URI, instead of doing JavaScript `postMessage` foo could be considered.
This is probably an orthogonal change to the fallback auth though.
## Security considerations
### Why we need user to confirm before the SSO flow
Recall that the thing we are trying to guard against here is a single leaked
access-token being used to take over an entire account. So let's assume the
attacker has got hold of an access token for your account. What happens if the
confirmation step is skipped?
The attacker, who has your access token, starts a UI Authentication session to
add their email address to your account.
They then sends you a link "hey, check out this cool video!"; the link leads (via
an innocent-looking URL shortener or some other phishing technique) to
`/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`, with the ID of
the session that he just created.
Since there is no confirmation step, the server redirects directly to the SSO
provider.
It's common for SSO providers to redirect straight back to the app if you've
recently authenticated with them; even in the best case, the SSO provider shows
an innocent message along the lines of "Confirm that you want to sign in to
\<your Matrix homeserver>".
After redirecting back to the homeserver, the SSO is completed and the
attacker's session is validated. They are now able to make their malicious
change to your account.
This problem can be mitigated by clearly telling the user what is about to happen.
### Reusing User Interactive Authentication sessions
The security of this relies on User Interactive Authentication sessions only
being used for the same request as they were initiated for. This security is not
only a concern for the proposed SSO authentication type. It is not believed
that this is currently enforced in implementations.
## Unstable prefix
A vendor prefix of `org.matrix.login.sso` is proposed (instead of `m.login.sso`)
until this is part of the specification.

56
proposals/2457-password-modification-invalidating-devices.md
Unescape Escape View File

@ -1,56 +0,0 @@
# Invalidating devices during password modification
There are multiple use cases for why a user might want to modify their password:
* Adopting a password manager (to use a unique password or more secure password).
* Password rotation.
* Re-secure a compromised account.
* ... probably tons of others ...
These can be summarized into two groups:
1. "My account has been compromised and I need to re-secure it."
2. "I just want to change my password."
The [current Matrix specification](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password)
does not provide a way to differentiate between these use cases. It currently
specifies behavior that fits well into the first use-case above: that the
sessions except the current session should be revoked.
It is reasonable for a client to want to specify this behavior to offer two
different workflows:
1. Modify a password and log all other devices out (for use when an account has
been compromised).
2. Modify a password and do not touch any session data (for use in a
non-malicious situations).
Alternately a client may default to whichever workflow is best for their users.
## Proposal
An optional field is added to the JSON body of the [password reset endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password)
called `logout_devices`. This is a boolean flag (defaulting to `true`) that
signals to whether other devices and sessions should be invalidated after
modifying the password.
## Potential issues
The specification states:
> The homeserver SHOULD NOT revoke the access token provided in the request,
> however all other access tokens for the user should be revoked if the request
> succeeds.
Defaulting `logout_devices` to `true` should be backwards compatible.
## Alternatives
A new endpoint could be provided in a future version of the specification that
supports an additional field (as described above).
## Security considerations
By defaulting to invalidating devices and sessions the security considerations
of this endpoint should remain intact. A client will need to be modified to
choose to keep other devices active.

88
proposals/2472-symmetric-ssss.md
Unescape Escape View File

@ -1,88 +0,0 @@
# Symmetric SSSS
[MSC1946 (Secure Secret Storage and
Sharing)](https://github.com/matrix-org/matrix-doc/pull/1946) proposed a way of
storing encrypted secrets on the server. In the proposal, secrets were
encrypted using a Curve25519 key, which was chosen to allow easier migration
from key backups that we created before the backup key was stored using it.
However this does not provide any guarantees that data stored using the
proposal came from a trusted source. To remedy this, we propose to change the
encryption to use AES with a MAC to ensure that only someone who knows the key
is able to store data.
## Proposal
* The `m.secret_storage.v1.curve25519-aes-sha2` method proposed in MSC1946 is
removed.
* A new method, `m.secret_storage.v1.aes-hmac-sha2`, is added. With this
method, the Secret Storage key may be any size (though 256 bits is
recommended), and data is encrypted as follows:
1. Given the secret storage key, generate 64 bytes by performing an HKDF with
SHA-256 as the hash, a salt of 32 bytes of 0, and with the secret name as
the info. The first 32 bytes are used as the AES key, and the next 32 bytes
are used as the MAC key
2. Generate 16 random bytes, set bit 63 to 0 (in order to work around
differences in AES-CTR implementations), and use this as the AES
initialization vector. This becomes the `iv` property, encoded using base64.
3. Encrypt the data using AES-CTR-256 using the AES key generated above. This
encrypted data, encoded using base64, becomes the `ciphertext` property.
4. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256
using the MAC key generated above. The resulting MAC is base64-encoded and
becomes the `mac` property.
(We use AES-CTR to match file encryption and key exports.)
If the key Secret Storage key is generated from a passphrase, information
about how to generate the key is stored in the `passphrase` property of the
key's account-data in a similar manner to what was done with the original
`m.secret_storage.v1.curve25519-aes-sha2` method, except that there is an
optional `bits` parameter that defaults to 256, and indicates the number of
bits that should be generated from PBKDF2 (in other words, the size of the
key).
* For the purposes of allowing clients to check whether a user has correctly
entered the key, clients should:
1. encrypt and MAC a message consisting of 32 bytes of 0 as described above,
using the empty string as the info parameter to the HKDF in step 1.
2. store the `iv` and `mac` in the `m.secret_storage.key.[key ID]`
account-data.
* The `passthrough` property specified in the "Encoding the recovery key for
server-side storage via MSC1946" section of MSC1219 is removed. The primary
purpose of that property was to allow easy migration of pre-MSC1946 backups,
so that users could reuse the backup recovery key as the Secret Storage key
without needing to re-enter the recovery key. However, since we are now
using a symmetric encryption algorithm, the client needs to know the key that
is used to encrypt, so the purpose of the field cannot be fulfilled.
* Signing the Secret Storage key with the user's master cross-signing key is no
longer required. The key is trusted on the basis of the user entering the
key/passphrase.
## Potential issues
Users who have data stored using the old encryption algorithm will need their
data migrated. Clients that support the old algorithm but not the new
algorithm will not be able to use the migrated secrets until they are updated
with the new algorithms. This should not be a major problem because the only
clients that are known to have implemented the old algorithm are Riot
Web/Android/iOS, and they have been upgraded to implement the new algorithm.
## Alternatives
Rather than switching to a symmetric encryption algorithm, we could stay with
an asymmetric encryption algorithm, and add on a method to authenticate the
data. However, it is much safer to use well-known cryptographic methods rather
than trying to invent something new. Since the main reason for using an
asymmetric scheme was to ease migration from older key backups without
requiring the user to re-enter the key, but this is no longer possible due to
the need to authenticate the data using the Secret Storage key, there is no
reason to stay with an asymmetric algorithm. It is also better to use
cryptographic methods already used in Matrix where possible, rather than
introducing something new.

14
proposals/2526-add-delete-backup.md
Unescape Escape View File

@ -1,14 +0,0 @@
# MSC2526: Add ability to delete key backups
[MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219) defined a
mechanism for key backups. However, it inadvertently omitted the endpoint to
delete an entire key backup. This proposal adds the endpoint.
## Proposal
An endpoint is added, `DELETE /room_keys/version/{version}`, that deletes a
backup version. Both the information about the key backup, as well as all keys
associated with the backup should be deleted. If the specified version was
previously deleted, the endpoint succeeds, returning an HTTP code of 200. If
the specified version never existed, the endpoint returns an HTTP code of 404
with a Matrix `errcode` of `M_NOT_FOUND`.

65
proposals/2540-stricter-event-validation.md
Unescape Escape View File

@ -1,65 +0,0 @@
# MSC2540: Stricter event validation: JSON compliance
## Background
There has been [prior discussions](https://github.com/matrix-org/matrix-doc/issues/1646)
about validating events more strictly. This MSC proposes fixing a small piece of
this: JSON compliance.
The [Canonical JSON](https://matrix.org/docs/spec/appendices#canonical-json)
specification requires that numbers that are serialized in JSON are integers in
the inclusive range of `[-(2^53) + 1, (2^53) - 1]`, which matches the requirements of
[section 6 of RFC 7159](https://tools.ietf.org/html/rfc7159). Note that it is
not explicit, but all floats are invalid.
It is worth mentioning that there are common extensions to JSON which produce
invalid JSON according to the Matrix specification; some programming languages
even support these by default. One common additional feature is handling
"special" float values: `Infinity`, `-Infinity`, and `NaN`.
## Proposal
In a future room version, homeserver implementations are to strictly enforce
the JSON compliance of the Canonical JSON specification for events.
Non-compliant events should be treated like any other malformed event,
for example by rejecting the request with an HTTP 400 error with `M_BAD_JSON`,
or by discarding the event.
The rationale for doing this in a future room version is to avoid a split brain
room -- where some federated servers believe an event is valid and others reject
it as invalid. Rooms will be able to opt into this behavior as part of a room
version upgrade.
Homeserver implementations are not to strictly enforce this JSON compliance in
[room versions 1, 2, 3, 4, and 5](https://matrix.org/docs/spec/#complete-list-of-room-versions).
The rationale is essentially the same as why a future room version is necessary:
this ensures that all federated servers treat the same events as valid.
## Potential issues
Homeserver implementations might include JSON parsers which are stricter than
others. It may not be worthwhile or reasonable to loosen those restrictions for
stable room versions.
## Alternatives
It could be argued that this MSC is unnecessary since it does not add any new
requirements for handling of JSON data. Unfortunately starting to enforce these
requirements in current rooms could cause federation to break as homeservers
will disagree on whether events are valid.
## Security considerations
N/A
## Unstable prefix
A room version of `org.matrix.strict_canonicaljson` until a future room version
is available. This room version will use
[room version 5](https://matrix.org/docs/spec/rooms/v5) as base and include the
above modifications.

20
proposals/2557-spoiler-clarifications.md
Unescape Escape View File

@ -1,20 +0,0 @@
# MSC2557: Clarifications on spoilers
Spoiler messages are described in [MSC2010](https://github.com/matrix-org/matrix-doc/pull/2010)
though the MSC is unclear if the fallback is required to be sent by clients.
## Proposal
The fallback for spoiler messages is optional, though recommended to be sent by clients. Clients
should make reasonable efforts to represent the spoiler in the `body` field of a message.
The recommended fallback format is unchanged.
Additionally, this proposal opens up spoilers to any HTML-supporting message types. Currently
this includes `m.text` (already included by MSC2010), `m.notice`, and `m.emote`.
## Potential issues
Clients could inadvertently spoil parts of a message by not representing the spoiler correctly
in the `body` of the message. The author believes this would quickly show up as a bug report
on the client due to the nature of spoilers.

12
proposals/2582-remove-mimetype-from-encrypted-file.md
Unescape Escape View File

@ -1,12 +0,0 @@
# Remove `mimetype` from `EncryptedFile` object
## Proposal
The example in the spec currently lists `mimetype` in the [examples for `EncryptedFile`](https://matrix.org/docs/spec/client_server/r0.6.1#extensions-to-m-message-msgtypes) but not in
the object definition. As that is duplicate information of the `info` block of file events, the
mimetype should just be removed altogether.
## Potential issues
Some clients might depend on this. However, as of August 2021, all known clients have
been confirmed to not use this.

39
proposals/2604-login-fallback-device-info.md
Unescape Escape View File

@ -1,39 +0,0 @@
# Parameters for Login Fallback
The [login fallback](https://matrix.org/docs/spec/client_server/r0.6.1#login-fallback)
API can be used by clients to support logins that they do not recognize. It is
expected to be loaded in a web view and calls a JavaScript function
(`window.onLogin`) when the login process is complete.
Since the login fallback page does the full login process there is no
opportunity for the application to provide a device ID (to re-authenticate
an expired session in the [case of soft-logout](https://matrix.org/docs/spec/client_server/r0.6.1#soft-logout))
or an [initial device display name](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-login)
(in the case of an initial login). This causes a few issues:
* It can make it difficult for a user to manage their sessions (as additional
sessions get created for each soft-logout).
* Cross-signing information gets reset when a new device ID is returned from the
login process. This results in users needing to re-validate their device.
## Proposal
The login fallback page will accept query parameters for non-credential related
parameters of the login endpoint. These will be forwarded by the login fallback
API to the login API throughout the login process. Currently the following
parameters should be accepted:
* `device_id`
* `initial_device_display_name`
## Potential issues
There are no backwards compatibility concerns: if a client provides the query
parameters to a homeserver that does not check for them than the current
behavior will occur.
## Security considerations
None.

35
proposals/2610-remove-oauth2-auth-type.md
Unescape Escape View File

@ -1,35 +0,0 @@
# MSC2610: Remove `m.login.oauth2` User-Interactive Authentication type from the specification
The client-server API specification
[defines](https://matrix.org/docs/spec/client_server/r0.6.1#authentication-types)
a number of "authentication types" for use with the User-Interactive
Authentication protocol.
Of these, `m.login.oauth2` is underspecified and of no
real use. This MSC proposes removing them.
## Proposal
The definition of
[OAuth2-based](https://matrix.org/docs/spec/client_server/r0.6.1#oauth2-based)
authentication is incomplete. [OAuth2](https://oauth.net/2/) is best considered
as a framework for implementing authentication protocols rather than a protocol
in its own right, and this section says nothing about the grant types, flows
and scopes which a compliant implementation should understand.
A better candidate for OAuth2-based authentication of matrix clients is via
[OpenID Connect](https://openid.net/connect/), but this has already been
implemented in Matrix via the `m.login.sso` authentication type.
The `m.login.oauth2` section is therefore unimplementable in its current form,
and redundant. It should be removed from the specification to reduce confusion.
## Alternatives
It would be possible to extend the definition so that it is complete: as
mentioned above, a likely implementation would be based on OpenID
Connect. Matrix clients could then follow the standardised OpenID Connect flow
rather than the matrix-specific `m.login.sso` flow. However, this would require
significant design work, and development in both clients and servers, which
currently feels hard to justify when a working solution exists via
`m.login.sso`.

40
proposals/2611-remove-login-auth-type.md
Unescape Escape View File

@ -1,40 +0,0 @@
# MSC2611: Remove `m.login.token` User-Interactive Authentication type from the specification
The client-server API specification
[defines](https://matrix.org/docs/spec/client_server/r0.6.1#authentication-types)
a number of "authentication types" for use with the User-Interactive
Authentication protocol.
Of these, `m.login.token` is unused and confusing. This MSC proposes removing it.
## Proposal
The definition of
[token-based](https://matrix.org/docs/spec/client_server/r0.6.1#token-based)
authentication is unclear about how this authentication type should be used. It
suggests "via some external service, such as email or SMS", but in practice
those validation mechanisms have their own token-submission mechanisms (for
example, the
`submit_url` field of the responses from
[`/account/password/email/requestToken`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-account-password-email-requesttoken)
and
[`/account/password/msisdn/requestToken`](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-account-password-msisdn-requesttoken)
respectively). Additionally, the specification requires that "the server must
encode the user ID in the token", which seems at odds with any token which can
be sent to a user over SMS.
Additional confusion stems from the presence of an `m.login.token` [login
type](https://matrix.org/docs/spec/client_server/r0.6.1#login), which is used
quite differently: it forms part of the single-sign-on login flow. For clarity:
this proposal does not suggest making any changes to the `m.login.token` login
type.
In practice, we are not aware of any implementations of the `m.login.token`
authentication type, and the inconsistency adds unnecessary confusion to the
specification.
## Potential Issues
It's possible that somebody has found a use for this mechanism. However, that
would necessarily entail some custom development of clients and servers, so is
not materially affected by the removal from the specification.

55
proposals/2630-sas-check-public-keys.md
Unescape Escape View File

@ -1,55 +0,0 @@
# MSC2630: Checking public keys in SAS verification
The current SAS protocol does not ensure that the two users correctly received
each other's public keys. An attacker could send Alice and Bob public keys
that he has created and, if the attacker is lucky, could obtain the same shared
secret with both Alice and Bob, so that when they verify the SAS string, will
believe that the exchange was secure.
To mitigate against this, Alice and Bob can use the two public keys in the
generation of the SAS string by including it in the info parameter of the HKDF.
Thus if an attacker sends them different public keys, the info parameters will
be different, and so the key generated by the HKDF will be different.
Thanks to [David Wong](https://twitter.com/cryptodavidw) for identifying the
issue, disclosing responsibly, and for helping to design the fix.
## Proposal
A new `key_agreement_protocol`, `curve25519-hkdf-sha256` is introduced, and
will be mandatory for clients to support when performing SAS verification. It
is the same as `curve25519` except that the info parameter for the HKDF is the
concatenation of:
* The string `MATRIX_KEY_VERIFICATION_SAS|`.
* The Matrix ID of the user who sent the `m.key.verification.start` message,
followed by `|`.
* The Device ID of the device which sent the `m.key.verification.start`
message, followed by `|`.
* The public key from the `m.key.verification.key` message sent by the device
which sent the `m.key.verification.start` message, followed by `|`.
* The Matrix ID of the user who sent the `m.key.verification.accept` message,
followed by `|`.
* The Device ID of the device which sent the `m.key.verification.accept`
message, followed by `|`.
* The public key from the `m.key.verification.key` message sent by the device
which sent the `m.key.verification.accept` message, followed by `|`.
* The `transaction_id` being used.
The differences from `curve25519` are the addition of the public keys, and the
addition of `|` as delimiter between the fields.
The `key_agreement_protocol` `curve25519` is deprecated and may be removed in
the future. It will no longer be mandatory for clients to support, and new
implementations are discouraged from implementing it.
## Implementation
This has been implemented in:
- Riot Web 1.6.3 (matrix-js-sdk 6.2.0)
- Riot Android 0.9.12 (matrix-android-sdk 0.9.35)
- RiotX 0.21
- Riot iOS 0.11.5 (matrix-ios-sdk 0.16.5)
- matrix-weechat and pantalaimon (matrix-nio 0.12.0)
- famedlysdk

44
proposals/2663-errors-nonexistent-push-rules.md
Unescape Escape View File

@ -1,44 +0,0 @@
# MSC2663: Errors for dealing with non-existent push rules
This MSC proposes that homeservers respond with a HTTP 404 ('Not Found') status
code and an `errcode` of `M_NOT_FOUND` when a client attempts to read or write
the `enabled` status or `actions` of a non-existent push rule.
## Background
The current revision of the Client-Server specification does not make clear what
a homeserver implementation is meant to do when getting or setting the `enabled`
or `actions` properties of a supposed push rule that does not exist.
## Motivation
This change is considered minor and the proposed error code is deemed
unsurprising as it matches the remainder of the specification.
## Proposal
The following endpoints of the Client-Server specification are affected:
- `GET /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}`
- `DELETE /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}`
- `PUT /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}`
- `GET /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}/enabled`
- `PUT /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}/enabled`
- `GET /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}/actions`
- `PUT /_matrix/client/r0/pushrules/{scope}/{kind}/{ruleId}/actions`
The affected endpoints will have the following response status code added:
**Status code 404:**
The push rule does not exist.
**Example**
```json
{
"errcode": "M_NOT_FOUND"
}
```
This error response is to be returned when the push rule represented by
`{scope}/{kind}/{ruleId}` does not exist.

250
proposals/2674-event-relationships.md
Unescape Escape View File

@ -1,250 +0,0 @@
# MSC2674: Event relationships
It's common to want to send events in Matrix which relate to existing events -
for instance, reactions, edits and even replies/threads.
This proposal is one in a series of proposals that defines a mechanism for
events to relate to each other. Together, these proposals replace
[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849).
* This proposal defines a standard shape for indicating events which relate to
other events.
* [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines APIs to
let the server calculate the aggregations on behalf of the client, and so
bundle the related events with the original event where appropriate.
* [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676) defines how
users can edit messages using this mechanism.
* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how
users can annotate events, such as reacting to events with emoji, using this
mechanism.
* [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267) defines how events
can make a reference to other events.
* [MSC3389](https://github.com/matrix-org/matrix-doc/pull/3389) defines changes to
the redaction algorithm, to preserve the type and target id of a relation.
## Proposal
This proposal introduces the concept of relations, which can be used to
associate new information with an existing event.
A relationship is an object with a field `rel_type`, which is a string describing the type of relation,
and a field `event_id`, which is a string that represents the event_id of the target event of this relation.
The target event must exist in the same room as the relating event is sent.
Both of those fields are required. An event is said to contain a relationship if its `content` contains
a relationship with all the required fields under the `m.relates_to` key. If any of these conditions is not met,
clients and servers should treat the event as if it does not contain a relationship.
Servers should reject events not meeting these conditions with an HTTP 400 error when
they are received via the client-server API.
Here's a (partial) example of an event relating to another event:
```json
{
"content": {
"m.relates_to": {
"rel_type": "m.replace",
"event_id": "$abc:server.tld"
}
}
}
```
All the information about the relationship lives under the `m.relates_to` key.
If it helps, you can think of relations as a "subject verb object" triple,
where the subject is the relation event itself; the verb is the `rel_type`
field of the `m.relates_to` and the object is the `event_id` field.
We consciously do not support multiple different relations within a single event,
in order to keep the API simple. This means that if event A relates to event B
in two different ways you would send two events to describe the two relations,
rather than bundling them into a single event. Another MSC,
like [MSC 3051](https://github.com/matrix-org/matrix-doc/pull/3051),
can propose a change to add support for multiple relations if it turns out that
this would facilitate certain use cases.
Relations do not yet replace the
[reply mechanism currently defined in the spec](https://matrix.org/docs/spec/client_server/r0.6.1#rich-replies).
### Relation types
Any values for `rel_type` should abide the
[general guidelines for identifiers](https://github.com/matrix-org/matrix-doc/pull/3171).
The `rel_type` property determines how an event relates to another and can be used
by clients to determine how and in what context a relation should be displayed.
[MSC 2675](https://github.com/matrix-org/matrix-doc/pull/2675) proposes to also interpret the `rel_type` server-side.
It is left up to the discretion of other MSCs building on this one whether they introduce
`rel_type`s that are specific to their use case or that can serve a broad range
of use cases. MSCs may define additional properties on the relation object for a given `rel_type`.
Currently, a few `rel_type`s are already proposed. Here's a non-exhaustive list:
- `m.replace` in [MSC 2676](https://github.com/matrix-org/matrix-doc/pull/2676).
- `m.annotation` in [MSC 2677](https://github.com/matrix-org/matrix-doc/pull/2677).
- `m.reference` in [MSC 3267](https://github.com/matrix-org/matrix-doc/pull/3267).
- `m.thread` in [MSC 3440](https://github.com/matrix-org/matrix-doc/pull/3440).
### Sending relations
Related events are normal Matrix events, and can be sent by the normal `/send`
API.
The server should postprocess relations if needed before sending them into a
room, as defined by the relationship type. For example, a relationship type
might only allow a user to send one related message to a given event.
### Receiving relations
Relations are received like other non-state events, with `/sync`,
`/messages` and `/context`, as normal discrete Matrix events. As explained
in the limitations, clients may be unaware of some relations using just these endpoints.
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines ways in
which the server may aid clients in processing relations by aggregating the
events.
### Redactions
Events with a relation may be redacted like any other event.
[MSC3389](https://github.com/matrix-org/matrix-doc/pull/3389) proposes that
the redaction algorithm should preserve the type and target id of a relation.
However, event relationships can still be used in existing room versions, but
the user experience may be worse if redactions are performed.
## Potential issues
### Federation considerations
We have a problem with resynchronising relations after a gap in federation:
We have no way of knowing that an edit happened in the gap to one of the events
we already have. So, we'll show inconsistent data until we backfill the gap.
* We could write this off as a limitation.
* Or we could make *ALL* relations a DAG, so we can spot holes at the next
relation, and go walk the DAG to pull in the missing relations? Then, the
next relation for an event could pull in any of the missing relations.
Socially this probably doesn't work as reactions will likely drop off over
time, so by the time your server comes back there won't be any more reactions
pulling the missing ones in.
* Could we also ask the server, after a gap, to provide all the relations which
happened during the gap to events whose root preceded the gap.
* "Give me all relations which happened between this set of
forward-extremities when I lost sync, and the point i've rejoined the DAG,
for events which preceded the gap"?
* Would be hard to auth all the relations which this api coughed up.
* We could auth them based only the auth events of the relation, except we
lose the context of the nearby DAG which we'd have if it was a normal
backfilled event.
* As a result it would be easier for a server to retrospectively lie about
events on behalf of its users.
* This probably isn't the end of the world, plus it's more likely to be
consistent than if we leave a gap.
* i.e. it's better to consistent with a small chance of being maliciously
wrong, than inconsistent with a guaranteed chance of being innocently
wrong.
* We'd need to worry about pagination.
* This is probably the best solution, but can also be added as a v2.
* In practice this seems to not be an issue, which is worth complicating
the s-s API over. Clients very rarely jump over the federation gap to an edit.
In most cases they scroll up, which backfills the server and we have all the
edits, when we reach the event before the gap.
## Limitations
Based solely on this MSC, relations are only received as discrete events in
the timeline, so clients may only have an incomplete image of all the relations
with an event if they do not fill gaps (syncs with a since token that have
`limited: true` set in the sync response for a room) in the timeline.
In practice, this has proven not to be too big of a problem, as reactions
(as proposed in [MSC 2677](https://github.com/matrix-org/matrix-doc/pull/2677))
tend to be posted close after the target event in the timeline.
A more complete solution to this has been deferred to
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675).
## Tradeoffs
### Event shape
Shape of
```json
"content": {
"m.relates_to": {
"m.reference": {
"event_id": "$another:event.com"
}
}
}
```
versus
```json
"content": {
"m.relates_to": {
"rel_type": "m.reference",
"event_id": "$another:event.com"
}
}
```
The reasons to go with `rel_type` is:
* This format is now in use in the wider matrix ecosystem without a prefix,
in spite of the original MSC 1849 not being merged. This situation is not ideal
but we still don't want to break compatibility with several clients.
* We don't need the extra indirection to let multiple relations apply to a given pair of
events, as that should be expressed as separate relation events.
* If we want 'adverbs' to apply to 'verbs' in the subject-verb-object triples which
relations form, then we apply it as mixins to the relation data itself rather than trying
to construct subject-verb-verb-object sentences.
* We decided to not adopt the format used by `m.in_reply_to` as it allows for multiple relations
and is hence overly flexible. Also, the relation type of `m.in_reply_to` is also overly specific
judged by the guidelines for `rel_type`s laid out in this MSC. Having replies use the same
format as relations is postponed to a later MSC, but it would likely involve replies
adopting the relation format with a more broadly useful `rel_type` (possibly the `m.reference`
type proposed in [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267)),
rather than relations adopting the replies format.
## Historical context
pik's MSC441 has:
Define the JSON schema for the aggregation event, so the server can work out
which fields should be aggregated.
```json
"type": "m.room._aggregation.emoticon",
"content": {
"emoticon": "::smile::",
"msgtype": "?",
"target_id": "$another:event.com"
}
```
These would then be aggregated, based on target_id, and returned as annotations on
the source event in an `aggregation_data` field:
```json
"content": {
...
"aggregation_data": {
"m.room._aggregation.emoticon": {
"aggregation_data": [
{
"emoticon": "::smile::",
"event_id": "$14796538949JTYis:pik-test",
"sender": "@pik:pik-test"
}
],
"latest_event_id": "$14796538949JTYis:pik-test"
}
}
}
```

320
proposals/2675-aggregations-server.md
Unescape Escape View File

@ -1,320 +0,0 @@
# MSC2675: Serverside aggregations of message relationships
It's common to want to send events in Matrix which relate to existing events -
for instance, reactions, edits and even replies/threads.
Clients typically need to track the related events alongside the original
event they relate to, in order to correctly display them. For instance,
reaction events need to be aggregated together by summing and be shown next to
the event they react to; edits need to be aggregated together by replacing the
original event and subsequent edits, etc.
It is possible to treat relations as normal events and aggregate them
clientside, but to do so comprehensively could be very resource intensive, as
the client would need to spider all possible events in a room to find
relationships and maintain a correct view.
Instead, this proposal seeks to solve this problem by defining APIs to let the
server calculate the aggregations on behalf of the client, and so bundle the
aggregated data with the original event where appropriate. It also proposes an
API to let clients paginate through all relations of an event.
This proposal is one in a series of proposals that defines a mechanism for
events to relate to each other. Together, these proposals replace
[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849).
* [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674) defines a
standard shape for indicating events which relate to other events.
* This proposal defines APIs to let the server calculate the aggregations on
behalf of the client, and so bundle the aggregated data with the original
event where appropriate.
* [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676) defines how
users can edit messages using this mechanism.
* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how
users can annotate events, such as reacting to events with emoji, using this
mechanism.
## Proposal
### Aggregations
Relation events can be aggregated per relation type by the server.
The format of the aggregated value (hereafter called "aggregation")
depends on the relation type.
Some relation types might group the aggregations by the `key` property
in the relation and aggregate to an array, while
others might aggregate to a single object or any other value really.
Here are some non-normative examples of what aggregations can look like:
Example aggregation for [`m.thread`](https://github.com/matrix-org/matrix-doc/pull/3440) (which
aggregates all relations into a single object):
```
{
"latest_event": {
"content": { ... },
...
},
"count": 7,
"current_user_participated": true
}
```
Example aggregation for [`m.annotation`](https://github.com/matrix-org/matrix-doc/pull/2677) (which
aggregates relations into a list of objects, grouped by `key`).
```
[
{
"key": "👍",
"origin_server_ts": 1562763768320,
"count": 3
},
{
"key": "👎",
"origin_server_ts": 1562763768320,
"count": 2
}
]
```
#### Bundling
Other than during non-gappy incremental syncs, timeline events that have other
events relate to them should include the aggregation of those related events
in the `m.relations` property of their unsigned data. This process is
referred to as "bundling", and the aggregated relations included via
this mechanism are called "bundled aggregations".
By sending a summary of the relations, bundling
avoids us having to always send lots of individual relation events
to the client.
Aggregations are never bundled into state events. This is a current
implementation detail that could be revisited later,
rather than a specific design decision.
The following client-server APIs should bundle aggregations
with events they return:
- `GET /rooms/{roomId}/messages`
- `GET /rooms/{roomId}/context/{eventId}`
- `GET /rooms/{roomId}/event/{eventId}`
- `GET /sync`, only for room sections in the response where `limited` field
is `true`; this amounts to all rooms in the response if
the `since` request parameter was not passed, also known as an initial sync.
- `GET /relations`, as proposed in this MSC.
Deprecated APIs like `/initialSync` and `/events/{eventId}` are *not* required
to bundle aggregations.
The bundled aggregations are grouped according to their relation type.
The format of `m.relations` (here with *non-normative* examples of
the [`m.replace`](https://github.com/matrix-org/matrix-doc/pull/2676) and
[`m.annotation`](https://github.com/matrix-org/matrix-doc/pull/2677) relation
types) is as follows:
```json
{
"event_id": "abc",
"unsigned": {
"m.relations": {
"m.annotation": {
"key": "👍",
"origin_server_ts": 1562763768320,
"count": 3
},
"m.replace": {
"event_id": "$edit_event_id",
"origin_server_ts": 1562763768320,
"sender": "@alice:localhost"
},
}
}
}
```
#### Client-side aggregation
Bundled aggregations on an event give a snapshot of what relations were known
at the time the event was received. When relations are received through `/sync`,
clients should locally aggregate (as they might have done already before
supporting this MSC) the relation on top of any bundled aggregation the server
might have sent along previously with the target event, to get an up to date
view of the aggregations for the target event. The aggregation algorithm is the
same as the one described here for the server.
### Querying relations
A single event can have lots of associated relations, and we do not want to
overload the client by, for example, including them all bundled with the
related-to event. Instead, we also provide a new `/relations` API in
order to paginate over the relations, which behaves in a similar way to
`/messages`, except using `next_batch` and `prev_batch` names
(in line with `/sync` API). Tokens from `/sync` or `/messages` can be
passed to `/relations` to only get relating events from a section of
the timeline.
The `/relations` API returns the discrete relation events
associated with an event that the server is aware of
in standard topological order. Note that events may be missing,
see [limitations](#servers-might-not-be-aware-of-all-relations-of-an-event).
You can optionally filter by a given relation type and the event type of the
relating event:
```
GET /_matrix/client/v1/rooms/{roomID}/relations/{event_id}[/{rel_type}[/{event_type}]][?from=token][&to=token][&limit=amount]
```
```
{
"chunk": [
{
"type": "m.reaction",
"sender": "...",
"content": {
"m.relates_to": {
"rel_type": "m.annotation",
...
}
}
}
],
"prev_batch": "some_token",
"next_batch": "some_token",
}
```
The endpoint does not have any trailing slashes. It requires authentication
and is not rate-limited.
The `from` and `limit` query parameters are used for pagination, and work
just like described for the `/messages` endpoint.
Note that [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676)
adds the related-to event in `original_event` property of the response.
This way the full history (e.g. also the first, original event) of the event
is obtained without further requests. See that MSC for further details.
### End to end encryption
Since the server has to be able to aggregate relation events, structural
information about relations must be visible to the server, and so the
`m.relates_to` field must be included in the plaintext.
A future MSC may define a method for encrypting certain parts of the
`m.relates_to` field that may contain sensitive information.
### Redactions
Redacted relations should not be taken into consideration in
bundled aggregations, nor should they be returned from `/relations`.
Requesting `/relations` on a redacted event should
still return any existing relation events.
This is in line with other APIs like `/context` and `/messages`.
### Local echo
For the best possible user experience, clients should also include unsent
relations into the client-side aggregation. When adding a relation to the send
queue, clients should locally aggregate it into the relations of the target
event, ideally regardless of the target event having received an `event_id`
already or still being pending. If the client gives up on sending the relation
for some reason, the relation should be de-aggregated from the relations of
the target event. If the client offers the user a possibility of manually
retrying to send the relation, it should be re-aggregated when the user does so.
De-aggregating a relation refers to rerunning the aggregation for a given
target event while not considering the de-aggregated event any more.
Upon receiving the remote echo for any relations, a client is likely to remove
the pending event from the send queue. Here, it should also de-aggregate the
pending event from the target event's relations, and re-aggregate the received
remote event from `/sync` to make sure the client-side aggregation happens with
the same event data as on the server.
When adding a redaction for a relation to the send queue, the relation
referred to should be de-aggregated from the relations of the target of the
relation. Similar to a relation, when the sending of the redaction fails or
is cancelled, the relation should be aggregated again.
If the target event is still pending and hasn't received its `event_id` yet,
clients can locally relate relation events to their target by using
`transaction_id` like they already do for detecting remote echos when sending
events.
## Edge cases
### How do you handle ignored users?
* Information about relations sent from ignored users must never be sent to
the client, either in aggregations or discrete relation events.
This is to let you block someone from harassing you with emoji reactions
(or using edits as a side-channel to harass you). Therefore, it is possible
that different users will see different aggregations (a different last edit,
or a different reaction count) on an event.
## Limitations
### Relations can be missed while not being in the room
Relation events behave no different from other events in terms of room history visibility,
which means that some relations might not be visible to a user while they are not invited
or have not joined the room. This can cause a user to see an incomplete edit history or reaction count
based on discrete relation events upon (re)joining a room.
Ideally the server would not include these events in aggregations, as it would mean breaking
the room history visibility rules, but this MSC defers addressing this limitation and
specifying the exact server behaviour to [MSC3570](https://github.com/matrix-org/matrix-doc/pull/3570).
### Servers might not be aware of all relations of an event
The response of `/relations` might be incomplete because the homeserver
potentially doesn't have the full DAG of the room. The federation API doesn't
have an equivalent of the `/relations` API, so has no way but to fetch the
full DAG over federation to assure itself that it is aware of all relations.
[MSC2836](https://github.com/matrix-org/matrix-doc/pull/2836) provided a proposal for following relationships over federation in the "Querying relationships over federation" section via a `/_matrix/federation/v1/event_relationships` API
### Event type based aggregation and filtering won't work well in encrypted rooms
The `/relations` endpoint allows filtering by event type,
which for encrypted rooms will be `m.room.encrypted`, rendering this filtering
less useful for encrypted rooms. Aggregation algorithms that take the type of
the relating events they aggregate into account will suffer from the same
limitation.
## Future extensions
### Handling limited (gappy) syncs
For the special case of a gappy incremental sync, many relations (particularly
reactions) may have occurred during the gap. It would be inefficient to send
each one individually to the client, but it would also be inefficient to send
all possible bundled aggregations to the client.
The server could tell the client the event IDs of events which
predate the gap which received relations during the gap. This means that the
client could invalidate its copy of those events (if any) and then requery them
(including their bundled relations) from the server if/when needed,
for example using an extension of the `/event` API for batch requests.
The server could do this with a new `stale_events` field of each room object
in the sync response. The `stale_events` field would list all the event IDs
prior to the gap which had updated relations during the gap. The event IDs
would be grouped by relation type,
and paginated as per the normal Matrix pagination model.
This was originally part of this MSC but left out to limit the scope
to what is implemented at the time of writing.
## Prefix
While this MSC is not considered stable, the endpoints become:
- `GET /_matrix/client/unstable/rooms/{roomID}/relations/{eventID}[/{relationType}[/{eventType}]]`
None of the newly introduced identifiers should use a prefix though, as this MSC
tries to document relation support already being used in
the wider matrix ecosystem.

20
proposals/2689-fix-e2ee-for-guests.md
Unescape Escape View File

@ -1,20 +0,0 @@
# MSC2689: Allow guests to operate in encrypted rooms
[#751](https://github.com/matrix-org/matrix-doc/pull/751) granted guest users access to several endpoints in order to allow them to use E2EE.
I found that guests are able to join encrypted rooms and read messages from other members. But when the
guest wants to send an event into the room the client receives a "guest access not allowed" error
for the `/rooms/{room_id}/members` endpoint. I assume the client tries to read the list of room members
to prepare the encryption of the event for the present members. Tests with a patched Synapse showed that
allowing guests to use this endpoint results in a normal behaviour and enables guests to communicate in
encrypted rooms.
## Proposal
Allow guests to use the `GET /_matrix/client/r0/rooms/{room_id}/members` endpoint to enable them to
operate properly in encrypted rooms.
## Alternatives
The list of room members could also be read from the sync. However that would not work with Lazy Loading.

26
proposals/2713-remove-deprecated-identity-endpoints.md
Unescape Escape View File

@ -1,26 +0,0 @@
# MSC2713: Remove deprecated Identity Service endpoints
Implementations will have had plenty of time to adopt the new v2 API for Identity Servers, so
we should clean out the old endpoints.
All deprecated endpoints in the r0.3.0 Identity Service API specification are to be removed.
For completeness, this includes:
* `GET /_matrix/identity/api/v1`
* `GET /_matrix/identity/api/v1/pubkey/{keyId}`
* `GET /_matrix/identity/api/v1/pubkey/isvalid`
* `GET /_matrix/identity/api/v1/pubkey/ephemeral/isvalid`
* `GET /_matrix/identity/api/v1/lookup`
* `POST /_matrix/identity/api/v1/bulk_lookup`
* `POST /_matrix/identity/api/v1/validate/email/requestToken`
* `POST /_matrix/identity/api/v1/validate/email/submitToken`
* `GET /_matrix/identity/api/v1/validate/email/submitToken`
* `POST /_matrix/identity/api/v1/validate/msisdn/requestToken`
* `POST /_matrix/identity/api/v1/validate/msisdn/submitToken`
* `GET /_matrix/identity/api/v1/validate/msisdn/submitToken`
* `GET /_matrix/identity/api/v1/3pid/getValidated3pid`
* `POST /_matrix/identity/api/v1/3pid/bind`
* `POST /_matrix/identity/api/v1/3pid/unbind`
* `POST /_matrix/identity/api/v1/store-invite`
* `POST /_matrix/identity/api/v1/sign-ed25519`

97
proposals/2732-olm-fallback-keys.md
Unescape Escape View File

@ -1,97 +0,0 @@
# MSC2732: Olm fallback keys
Olm uses a set of one-time keys when initializing a session between two
devices: Alice uploads one-time keys to her homeserver, and Bob claims one of
them to perform a Diffie-Hellman to generate a shared key. As implied by the
name, a one-time key is only to be used once. However, if all of Alice's
one-time keys are claimed, Bob will not be able to create a session with Alice.
This can be addressed by Alice uploading a fallback key that is used in place
of a one-time key when no one-time keys are available.
## Proposal
A new request parameter, `fallback_keys`, is added to the body of the
[`/keys/upload` client-server API](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-keys-upload), which is in the same format as the
`one_time_keys` parameter with the exception that there must be at most one key
per key algorithm. If the user had previously uploaded a fallback key for a
given algorithm, it is replaced -- the server will only keep one fallback key
per algorithm for each user.
When uploading fallback keys for algorithms whose key format is a signed JSON
object, client should include a property named `fallback` with a value of
`true`.
Example:
`POST /keys/upload`
```json
{
"fallback_keys": {
"signed_curve25519:AAAAAA": {
"key": "base64+public+key",
"fallback": true,
"signatures": {
"@alice:example.org": {
"ed25519:DEVICEID": "base64+signature"
}
}
}
}
}
```
When Bob calls `/keys/claim` to claim one of Alice's one-time keys, but Alice
has no one-time keys left, the homeserver will return the fallback key instead,
if Alice had previously uploaded one. Unlike with one-time keys, fallback keys
are not deleted when they are returned by `/keys/claim`. However, the server
marks that they have been used.
A new response parameter, `device_unused_fallback_key_types`, is added to
`/sync`. This is an array listing the key algorithms for which the server has
an unused fallback key for the device. If the client wants the server to have a
fallback key for a given key algorithm, but that algorithm is not listed in
`device_unused_fallback_key_types`, the client will upload a new key as above.
The `device_unused_fallback_key_types` parameter must be present if the server
supports fallback keys. Clients can thus treat this field as an indication
that the server supports fallback keys, and so only upload fallback keys to
servers that support them.
Example:
`GET /sync`
Response:
```jsonc
{
// other fields...
"device_unused_fallback_key_types": ["signed_curve25519"]
}
```
## Security considerations
Using a fallback key rather than a one-time key has security implications. An
attacker can replay a message that was originally sent with a fallback key, and
the receiving client will accept it as a new message if the fallback key is
still active. Also, an attacker that compromises a client may be able to
retrieve the private part of the fallback key to decrypt past messages if the
client has still retained the private part of the fallback key.
For this reason, clients should not store the private part of the fallback key
indefinitely. For example, client should only store at most two fallback keys:
the current fallback key (that it has not yet received any messages for) and
the previous fallback key, and should remove the previous fallback key once it
is reasonably certain that it has received all the messages that use it (for
example, one hour after receiving the first message that used it).
For addressing replay attacks, clients can also keep track of inbound sessions
to detect replays.
## Unstable prefix
The `fallback_keys` request parameter and the `device_unused_fallback_key_types`
response parameter will be prefixed by `org.matrix.msc2732.`.

56
proposals/2758-textual-id-grammar.md
Unescape Escape View File

@ -1,56 +0,0 @@
# MSC2758: Common grammar for textual identifiers
The matrix specification uses textual identifiers for a wide range of
concepts. Examples include "event types" and "room versions".
In the past, these identifiers have often lacked a formal grammar, leaving
servers and clients to make assumptions about questions such as which
characters are permitted, minimum and maximum lengths, etc.
This proposal suggests a common grammar which can be used as a basis for
*future* identifier types, to reduce the work involved in future specification
work.
No attempt is made here to bring existing identifiers into line; however
examples of identifiers which might have benefitted from such a grammar in the
past include:
* [`capabilities`](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-capabilities)
identifiers.
* authentication types for the [User-Interactive Authentication mechanism](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api).
* login types for [`/_matrix/client/r0/login`](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-login).
* event types
* [`m.room.message` `msgtypes`](https://matrix.org/docs/spec/client_server/r0.6.0#m-room-message-msgtypes)
* `app_id` for [`POST /_matrix/client/r0/pushers/set`](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-pushers-set).
* `rule_ids`, `actions` and `tweaks` for [push rules](https://matrix.org/docs/spec/client_server/r0.6.0#push-rules).
* [E2E messaging algorithm names](https://matrix.org/docs/spec/client_server/r0.6.0#messaging-algorithm-names).
## Proposal
We define a "common namespaced identifier grammar". This can then be referenced
by other parts of the grammar, in much the same way as [Unpadded
Base64](https://matrix.org/docs/spec/appendices#unpadded-base64) is defined
today.
The grammar is defined as follows:
* An identifier may not be less than one character or more than 255 characters
in length.
* Identifiers must start with one of the characters `[a-z]`, and be entirely
composed of the characters `[a-z]`, `[0-9]`, `-`, `_` and `.`.
* Identifiers starting with the characters `m.` are reserved for use by the
formal matrix specification.
* Implementations wishing to implement unspecified identifiers should follow
the Java Package Naming convention of starting with a reversed domain
name (with a dot after the domain name part). For example, for the
organisation `example.com`, a valid identifier would be
`com.example.identifier`.
This grammar is intended for use entirely by internal identifiers, and *not*
for user-visible strings.
### Rationale
* Avoiding non-ascii characters sidesteps any issues with homoglyphs or
alternative encodings of the same characters.
* Avoiding upper-case character sidesteps any concerns over case-sensitivity.

51
proposals/2765-widget-avatars.md
Unescape Escape View File

@ -1,51 +0,0 @@
# MSC2765: Widget avatars
Currently widgets have a name and title associated with them, though no opportunity for avatars
for a favicon-like experience. This proposal introduces such a concept.
## Proposal
A new optional parameter named `avatar_url` is added to the widget definition. This parameter is
an MXC URI to an image clients can use to associate with the widget, likely alongside the `name`
and/or `title`.
Widget avatars SHOULD be legible at small sizes, such as 20x20. The MXC URI in the `avatar_url`
should be the source material to allow clients to use the `/thumbnail` API to get a size for their
use case.
Rendering avatars is optional for clients, much like how clients are not required to use the `name`
or `title` of a widget.
An example widget would be:
```json
{
"creatorUserId": "@alice:example.org",
"data": {
"custom_key": "This is a custom key",
"title": "This is a witty description for the widget"
},
"id": "20200827_WidgetExample",
"name": "My Cool Widget",
"type": "m.custom",
"url": "https://example.org/my/widget.html?roomId=$matrix_room_id",
"waitForIframeLoad": true,
"avatar_url": "mxc://example.org/abc123"
}
```
## Alternatives
We could define a whole structured system for different thumbnail sizes, though we have a thumbnail
API which can be used to request whatever size is needed by the client.
## Security considerations
Widget avatars could be non-images. Clients should use the thumbnail API to encourage error responses
from the server when a widget avatar is a non-image.
## Unstable prefix
While this MSC is not in a released version of the specification, clients should use an alternative
event type for widgets or use `org.matrix.msc2765.avatar_url` when using `m.widget` or `m.widgets`
as an event type.

54
proposals/2774-widget-id.md
Unescape Escape View File

@ -1,54 +0,0 @@
# MSC2774: Giving widgets their ID so they can communicate
Under the [current specification](https://github.com/matrix-org/matrix-doc/pull/2764), widgets are
able to communicate with their client host, however doing so is a bit difficult if they don't already
know their widget ID. Some widgets will be able to get their ID from another source like an
integration manager, however this is not the case for all widgets.
[MSC2762](https://github.com/matrix-org/matrix-doc/pull/2762) has a fair amount of background
information on widgets, as does the current specification linked above.
## Proposal
Currently widgets can expect a `?widgetId` query parameter sent to them in clients like Element,
however this has some issues and as such is not proposed to exist any further. One of the issues
is that the widget must retain the query string, which isn't entirely possible for some frontends
(they would instead prefer to use the fragment). It's also a privacy risk in that by being sent
through the query string, the server can be made aware of the widget ID. The widget ID doesn't
normally contain any useful information (it's an opaque string), however it's not required for
the server to function under typical usage.
The proposal is to introduce a `$matrix_widget_id` template variable for the URL, similar to the
existing `$matrix_room_id` variable. Widgets can then have their widget ID wherever they want in
the widget URL, making it easier on them to find and use.
This carries the same risks as a room ID being passed to the widget: the client can easily lie about
it, however there's no real risk involved in doing so because it's used for communication only. So
long as the client is able to identify which widget is talking to it, it doesn't really matter. It's
more of a problem if the client uses a widget ID from another widget as that could confuse the
client, however this is believed to be a bug. Clients should also be performing origin checks to
ensure the widget is talking from a sane origin and not somewhere else, like another tab or browser.
## Potential issues
This is not backwards compatible with the history of widgets so far, so clients and widgets will
need to be modified to support it. Clients which currently use the `?widgetId` parameter are
encouraged to continue supporting the parameter until sufficient adoption is reached.
## Alternatives
As mentioned, a query parameter could be used, though this has the issues previously covered. Another
solution might be to allow a single widget API action which has no widget ID solely for the purpose
of finding the widget ID, however clients are unlikely to be able to differentiate between two widgets
if this were the case.
Another solution would be to let the widget discover its widget ID by harvesting it out of the first
widget API request it sees. This can't always be relied upon (some flows require the widget to
speak first), and as the widget API becomes more capable it could become a security risk. A malicious
browser extension could spam the widget with fake requests to try and convince it to talk to it
instead of the client, thus redirecting some information to the wrong place.
## Unstable prefix
Implementations should use `$org.matrix.msc2774_widget_id` as a variable until this lands in a
released version of the specification.

135
proposals/2778-appservice-login.md
Unescape Escape View File

@ -1,135 +0,0 @@
# MSC2778: Providing authentication method for appservice users
Appservices within Matrix are increasingly attempting to support End-to-End Encryption. As such, they
need a way to generate devices for their users so that they can participate in E2E rooms. In order to
do so, this proposal suggests implementing an appservice extension to the
[`POST /login` endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-login).
Appservice users do not usually need to log in as they do not need their own access token, and do not
traditionally need a "device". However, E2E encryption demands that at least one user in a room has a
Matrix device which means bridge users need to be able to generate a device on demand. In the past,
bridge developers have used the bridge bot's device for all bridge users in the room, but this causes
problems should the bridge wish to only join ghosts to a room (e.g. for DMs).
Another advantage this provides is that an appservice can now be used to generate access tokens for
any user in its namespace without having to set a password for that user, which may be useful where
maintaining password(s) in the configuration is undesirable.
## Proposal
A new `type` is to be added to `POST /login`: `m.login.application_service`
The `/login` endpoint may now take an `access_token` in the same way that other
authenticated endpoints do. No additional parameters should be specified in the request body.
Example request
```json
{
"type": "m.login.application_service",
"identifier": {
"type": "m.id.user",
"user": "_bridge_alice"
}
}
```
Note: Implementations MUST use the `identifier.type`=`m.id.user` method of specifying the
localpart. The deprecated top-level `user` field **cannot** use this login flow type. This
is deliberate so as to coax developers into using the new identifier format when implementing
new flows.
The response body should be unchanged from the existing `/login` specification.
If one of the following conditions are true:
- The access token is not provided
- The access token does not correspond to an appservice
- Or the user has not previously been registered
Then the servers MUST reject with HTTP 403, with an `errcode` of `"M_FORBIDDEN"`.
If the access token DOES correspond to an appservice but the user is not inside its namespace,
then the `errcode` must be `"M_EXCLUSIVE"`.
Homeservers should ignore the `access_token` parameter if a type other than
`m.login.application_service` has been provided.
Appservices creating **new** users can still use the `/register` endpoint to generate an `access_token` / `device_id`
but for existing users, the `/login` endpoint can be used instead.
## Potential issues
This proposal means that there will be more calls to make when setting up a appservice user, when
using encryption. While this could be done during the registration step, this would prohibit creating
new devices should the appservice intentionally or inadvertently have lost the client-side device data.
## Alternatives
### 1. Include the token in the `/login` request body
One minor tweak to the current proposal could be to include the token as part of the auth data, rather than
being part of the header/params to the request. An argument could be made for either, but since the specification
expects the appservice to pass the token this way in all requests, including `/register`, it seems wise to keep
it that way.
### 2. Use implementation specific "shared secret" authentication
Some community members have used homeserver implementation details such as a "shared secret" authentication method to
log into the accounts without having to use the /login process at all. Synapse provides such a function,
but also means the appservice can now authenticate as any user on the homeserver. This is undesirable from a
security standpoint.
### 3. Keep using `/register` solely
A third option could be to create a new endpoint that simply creates a new device for an appservice user on demand.
Given the rest of the matrix eco-system does this with /login, and /login is already extensible with `type`, it would
create more work for all parties involved for little benefit.
Finally, `POST /register` does already return a `device_id` and `access_token` so appservices
could store this information rather than calling `POST /login` at all. This does however present a few problems:
- Quite a few appservices which only support unencrypted messaging do not use/store the `device_id`/`access_token` from a register call.
In the event that an appservice eventually gains the ability to support encryption, they would be unable to fetch a new `device_id`/
`access_token` for any existing users (as `/register` would fail for an existing user).
- If user tokens were lost or exposed, there is no way to programmatically create new access tokens for these users.
- Finally, if a user was registered externally and the appservice would like to masquerade as it, it would be unable to fetch
an access token for that user.
While `POST /register` does work, it is impactical as the sole method of fetching an access token.
## Security considerations
Appservices could use this new functionality to generate devices for any userId that are within its namespace e.g. setting the
user namespace regex to `@.*:example.com` would allow appservice to control anyone on the homeserver. While this sounds scary, in practice
this is not a problem because:
- Appservice namespaces are maintained by the homeserver admin. If the namespace were to change, then it's reasonable
to assume that the server admin is aware. There is no defense mechanism to stop a malicious server admin from creating new
devices for a given user's account as they could also do so by simply modifying the database.
- While an appservice *could* try to masquerade as a user maliciously without the server admin expecting it, it would still
be bound by the restrictions of the namespace. Server admins are expected to be aware of the implications of adding new
appservices to their server so the burden of responsibility lies with the server admin.
- Appservices already can /sync as any user using the `as_token` and send any messages as any user in the namespace, the only
difference is that without a dedicated access token they are unable to receive device messages. While in theory this
does make them unable to see encrypted messages, this is not designed to be a security mechanism.
In conclusion this MSC only automates the creation of new devices for users inside an AS namespace, which is something
a server admin could already do. Appservices should always be treated with care and so with these facts in mind the MSC should
be considered secure.
## Unstable prefix
Implementations should use `uk.half-shot.msc2778.login.application_service` for `type` given in the
`POST /login` until this lands in a released version of the specification.
## Implementations
The proposal has been implemented by a homeserver, a bridge SDK and two bridges:
- [synapse](https://github.com/matrix-org/synapse/pull/8320)
- [mautrix-python](https://github.com/tulir/mautrix-python/commit/12d7c48ca7c15fd3ff61608369af1cf69e289aeb)
- [mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp/commit/ead8a869c84d07fadc7cfcf3d522452c99faaa36)
- [matrix-appservice-bridge](https://github.com/matrix-org/matrix-appservice-bridge/pull/231/files#diff-5e93f1b51d50a44fcf0ca46ea1793c1cR851-R864)

29
proposals/2788-v6-default-version.md
Unescape Escape View File

@ -1,29 +0,0 @@
# MSC2788: Room version 6 as a default
Enough time has passed to allow the public federation to upgrade their servers to support room
version 6. This proposal aims to make v6 the default room version.
## Proposal
The specification adopts v6 as the suggested default room version, making no changes to the stability
of any room versions. As of writing, v5 is currently the suggested room version.
Room version 6 has the following specification: https://matrix.org/docs/spec/rooms/v6
## Potential issues
Servers will be encouraged to update their config/internal defaults to use v6 instead of v5. This
is considered a good problem to have.
## Alternatives
None relevant.
## Security considerations
None relevant.
## Unstable prefix
None relevant - servers can already choose a different default room version legally. This MSC
just formalizes v6 as the default.

189
proposals/2801-untrusted-event-data.md
Unescape Escape View File

@ -1,189 +0,0 @@
# MSC2801: Make it explicit that event bodies are untrusted data
As the Matrix Specification stands today, it is easy for client developers to
assume that all events will be structured as documented. For example, the
`displayname` key of an [`m.room.member`
event](https://matrix.org/docs/spec/client_server/r0.6.1#m-room-member) is
specified to be "`string` or `null`"; and the `info` key of an [`m.image`
message event](https://matrix.org/docs/spec/client_server/r0.6.1#m-image) must
be a Javascript object.
In reality, these are not safe assumptions. This MSC proposes that the
specification be updated to make it clear that developers should treat all
event data as untrusted.
## Reasons why events may not match the specification
Firstly, let's examine the reasons why such malformed events can exist. Some of
these reasons may appear to have trivial solutions; these are discussed below.
1. Most obviously, Synapse as it currently stands does very little validation
of event bodies sent over the Client-Server API. There is nothing stopping
any user sending a malformed `m.room.message` event with a simple `curl`
command or Element-Web's `/devtools`.
Any event sent in this way will be returned to the clients of all the other
users in the room.
2. Events may be encrypted. Any client implementing E2EE must be prepared to
deal with any encrypted content, since by definition a server cannot
validate it.
3. In order to allow the Matrix protocol to be extensible, server
implementations must tolerate unknown event types, and allow them to be
passed between clients. It is obvious that a pair of custom clients
implementing a `com.example.special.event` event type cannot rely on a
standard server implementation to do any validation for them.
However, this problem extends to event types and keys which have been added
to the specification. For example, the [`m.room.encrypted` event
type](https://matrix.org/docs/spec/client_server/r0.6.1#m-room-encrypted)
was added in Client-Server API r0.4.0. It therefore follows that a server
implementing CS-API r0.3.0 would have no way to validate an
`m.room.encrypted` event, so if a client is connected to such a server, it
could receive malformed events.
4. To extend from point 3, the problem is not even resolved by upgrading the
server. There may now be rooms which contain historical events which would
no longer be accepted, but these will still be served by the server.
This problem also applies to non-room data such as account data. For
example, Client-Server API r0.6.0 added the [`m.identity_server` account
data event
type](https://matrix.org/docs/spec/client_server/r0.6.1#m-identity-server).
It is possible, if unlikely, that a client could have uploaded an
`m.identity_server` event before the administrator upgraded the server.
5. Event redaction removes certain keys from an event. This is a bit of a
trivial case, though it is worth noting that the rules for event redaction
vary between room versions, so it's possible to see a variety of "partial"
events.
6. All the cases above can occur without federation. Federation adds
additional complexities due to the structure of Matrix rooms. In
particular, a server implementation cannot simply ignore any malformed
events since such events may either be critical to the structure of the
room (for example, they may be `m.room.membership` events), or at the very
least ignoring them would leave "holes" in the event graph which would
prevent correct back-pagination.
## Ideas for mitigating the problem
The problems above appear to have some easy solutions. Let's describe some of
those ideas, before considering the fundamental problem that none of them can
solve.
### Validate all events when they are submitted
In this idea, we would require all server implementations to strictly validate
any data which is sent over the Client-Server API, to ensure that it complied
with the specified formats.
This evidently solves problem 1 above, in that it would prevent local users from
creating malformed events of any event types that the server supports; however,
it would do nothing to address any of the other problems.
### Validate events at the point they are served to a client
We could require that server implementations validate any data that they are
about to serve to a client; we might recommend that malformed data be stripped
out of the response, or redacted, or similar.
It is worth mentioning that this would be tricky to implement efficiently on
the server side, but it at least helps address most of the problems above, such
as historical data, or malformed events received over federation.
### Have servers re-validate data on upgrade
Similar to the previous idea, but rather than validating data each time it is
served to a client, any stored data could be re-validated to check that it
complies with new validation requirements.
This could be more efficient in the case that changes to validation rules are
rare, but it could still be a huge amount of data processing on a large server.
### Create new room versions which require events to be well-formed
As outlined above, one of the big problems in this area is how we deal with
events sent over federation; in particular, if subsets of the servers in a room
have different ideas as to which events are "valid", then their concepts of the
room state can begin to drift, and the room can eventually become
"split-brained". This makes it hard to simply say, for example,
"`m.room.member` events with a non-string `displayname` are invalid and should
not form part of the room state": we have a risk that some servers will accept
the event, and some will not.
One approach to solving this is via [room versions](https://spec.matrix.org/unstable/rooms/).
By specifying that a change of rules only applies for a future room version,
we can eliminate this potential disagreement.
The process of changing a room from one version to another is intrusive, not
least because it requires that all servers in a room support the new room
version (or risk being locked out). For that reason, it is extremely
undesirable that any new feature require a new room version: whenever possible,
it should be possible to use new features in existing rooms. It therefore
follows that we cannot rely on room versions to provide validation of event
data.
### Create a single new room version which applies all event validation rules
This idea is included for completeness, though it is unclear how it would work
in practice.
It has been suggested that we create a new room version which explicitly states
that events which fail the current event schema, whatever that is at that
moment in time, should be rejected.
Let's imagine that in future, the `m.room.member` event schema is extended to
include an optional `location` key, which, if given, must be a string. The
implication of this idea is that servers should reject any `m.room.member`
event whose `location` is not a string. We now have a problem: any servers in
the room which are updated to the latest spec will reject such malformed
events, but any other servers yet to be upgraded will allow it. So is that user
in the room or not?
Even if all the servers in the room can be upgraded at once, what about any
`m.room.member` events which were sent before the rule change?
## The fundamental problem
The ideas above all mitigate the problems discussed earlier to a greater or
lesser extent, and may indeed be worth doing on their own merits. However, none
of them can address the problem of outdated server implementations.
For example, consider the case of a new key being added to an event body, say
`m.relates_to`. Now, we may have decided as above that all new specced keys
must be validated by the server, so `vN+1` of Synapse dutifully implements such
validation and refuses to accept events with a malformed `m.relates_to`.
The problem comes for users whose server is still Synapse `vN`. It knows
nothing of `m.relates_to`, so accepts and passes it through even if
malformed. The only potential solution is for clients seeking to implement
`m.relates_to` to refuse to talk to servers which do not declare support for
it.
However, this is an entirely client-side feature: it is illogical to require
that servers must be upgraded before it can be used. Consider that the hosted
element-web at `https://app.element.io` is upgraded to support the new feature;
in this scenario, that would lock out any user whose homeserver had not yet
been upgraded. This is not an acceptable user experience.
In short, we are left with the reality that clients must still handle the
unvalidated data.
## Conclusions
Short of closely coupling server and client versions - which violates the
fundamental ethos of the Matrix project - there is nothing that can completely
prevent clients from having to handle untrusted data. In addition, encrypted
events eliminate any possibility of server-side validation.
With that in mind, the advantages of the ideas above are diminished. If clients
must handle untrusted data in some circumstances, why not in all? "You can
trust the content of this data structure, provided you have checked that the
server knows how to validate it, in which case you need to treat it as
untrusted" is not a useful message for a client developer.
It may be possible to assert that specific, known cases can be treated as
trusted data, but these should be called out as specific cases. The default
message should be that clients must treat all event data as untrusted.

211
proposals/2844-global-versioning.md
Unescape Escape View File

@ -1,211 +0,0 @@
# MSC2844: Using a global version number for the entire specification
Currently we have 4 kinds of versions, all of which have slightly different use cases and semantics
which apply:
1. The individual API spec document versions, tracked as revisions (`r0.6.1`, for example).
2. Individual endpoint versioning underneath an API spec document version (`/v1/`, `/v2/`, etc). Note
that the client-server API currently ties the major version of its spec document version to the
endpoint, thus making most endpoints under it `/r0/` (currently).
3. Room versions which define a set of behaviour and algorithms on a per-room basis. These are well
defined in the spec and are not covered here: https://spec.matrix.org/unstable/rooms/
4. An overarching "Matrix" version, largely for marketing purposes. So far we've only cut Matrix 1.0
back when we finalized the initial versions of the spec documents, but have not cut another one
since.
This current system is slightly confusing, and has some drawbacks for being able to compile builds of
the spec documents (published on matrix.org) and generally try and communicate what supported versions
an implementation might have. For example, Synapse currently supports 4 different APIs, all of which
have their own versions, and all of which would need to be considered and compared when validating
another implementation of Matrix such as a client or push gateway. Instead, Synapse could say it
supports "Matrix 1.1", making compatibility much easier to determine - this is what this proposal aims
to define.
## Proposal
Instead of having per-API versions (`r0.6.1`, etc), we have a version that spans the entire specification.
This version represents versioning for the index (which has quite a bit of unversioned specification on
it currently), the APIs, room versions, and the appendices (which are also currently unversioned but
contain specification). Room versions are a bit more nuanced though, and are covered later in this MSC.
The version which covers the entire specification and all its parts is called the "Matrix version", and
is a promotion of the previously marketing-only version number assigned to the spec. The first version
after this proposal is expected to be Matrix 1.1, though the spec core team will make that decision.
v1.0 would be left in the marketing era and recorded for posterity (though still retains no significant
meaning).
Doing this has the benefits previously alluded to:
* Implementations of Matrix can now easily compare their supported versions using a single identifier
without having to (potentially) indicate which API they built support for.
* Publishing the specification is less likely to contain broken or outdated links due to API versions
not matching up properly. This is currently an issue where if we want to release a new version of
the server-server specification then we must also either rebuild or manually fix the blob of HTML
known as the client-server API to account for the new version - we often forget this step, sometimes
because it's just too difficult.
* Explaining to people what version Matrix or any of the documents is at becomes incredibly simplified.
No longer will we have to explain most of what the introduction to this proposal covers to every new
person who asks.
### Full Matrix version grammar
The Matrix versioning scheme takes heavy inspiration from semantic versioning, though intentionally does
not follow it for reasons described throughout this proposal. Primarily, the argument against semantic
versioning is held in the alternatives section below.
Given a version number `MAJOR.MINOR`, incremement the:
* `MAJOR` version when a substantial change is made to the core of the protocol. This is reserved for
interpretation by the Spec Core Team, though is intended to be for extremely invasive changes such
as switching away from JSON, introducing a number of features where a `MINOR` version increase just
doesn't feel good enough, or changes to the signing algorithms.
* `MINOR` version when a feature is introduced, or a backwards incompatible change has been managed
through the specification. Later on, this proposal explains what it means to manage a breaking change.
When present in the protocol itself, the Matrix version will always be prefixed with `v`. For example,
`v1.1`.
Additional information can be supplied in the version number by appending a dash (`-`) to the end of the
version and including any relevant information. This is typically used to denote alpha, beta, unstable,
or other similar off-cycle release builds. This MSC does not propose a scheme for RCs or pre-releases,
though the Spec Core Team may wish to do so. This can also be used to represent patch builds for the
documentation itself, such as correcting spelling mistakes. An example would be `v1.1-patch.20210109`.
See the section on brewing Matrix versions for information on how the unstable version is decided.
`MINOR` versions have a backwards compatibility scheme described later in this proposal. `MAJOR`
versions are expected to have zero backwards compatibility guarantees to them.
For clarity, `v1.2` will probably work with `v1.1`, though implementations should be wary if they
depend on a version. As mentioned, the backwards compatibility scheme section goes into more detail on
this.
Most notably, this MSC does not propose including a patch version at all. The specifics of what would
be included in a patch version (spelling changes, release process bug fixes, etc) do not impact any
implementations of Matrix and thus are not needing of a patch version.
### Structure changes and changelogs
The API documents remain mostly unchanged. We'll still have a client-server API, server-server API, etc,
but won't have versions associated with those particular documents. This also means they would lose their
individual changelogs in favour of a more general changelog. An exception to this rule is room versions,
which are covered later in this proposal.
### Endpoint versioning
Under this MSC, all HTTP endpoints in the specification are to be per-endpoint versioned. This is already
the case for all APIs except the Client-Server API, and so this section deals specifically with that API.
The deprecation of endpoints is handled later in this proposal.
Under this proposal, all endpoints in the client-server API get assigned `v3` as their per-endpoint
version as a starting point. This is primarily done to avoid confusion with the ancient client-server API
versions which had `v1` and called the `rN` system "v2". Though many of the endpoints available today
are not present in those older API editions, it is still proposed that they start at `v3` to avoid
confusion with long-standing implementations.
Servers would advertise support for the new Matrix version by appending it to the array in `/versions`.
If the sever also supports an older `rN` version, it would include those too.
For example: `["v1.1", "r0.6.1"]`.
### Room versions
*Author's note*: Having many things with the root word "version" can be confusing, so for this section
"room versions" are called "room editions" and the Matrix version refers to what this proposal is
introducing. This MSC does not propose renaming "room versions" - that is another MSC's problem.
Room editions are a bit special in that they have their own versioning scheme as servers and, potentially,
clients need to be aware of how to process the room. As such, a room edition's versioning scheme is not
altered by this proposal, however the publishing of the (in)stability of a given edition is now covered by the
newly proposed Matrix version.
Whenever a room edition transitions from stable to unstable, or unstable to stable, or is introduced
then it would get counted as a feature for a `MINOR` release of Matrix. We don't currently have a plan
to remove any room editions, so they are not covered as a potential process for this MSC.
### Deprecation approach
Previous to this proposal the deprecation approach was largely undocumented - this MSC aims to codify
a standardized approach.
An MSC is required to transition something from stable (the default) to deprecated. Once something has
been deprecated for suitably long enough (usually 1 version), it is eligible for removal from the
specification with another MSC. Today's process is the same, though not defined explicitly.
The present system for deprecation also allows implementations to skip implementation of deprecated
endpoints. This proposal does not permit such behaviour: for an implementation to remain compliant
with the specification, it must implement all endpoints (including deprecated ones) in the version(s)
it wishes to target.
As an example, if `/test` were introduced in v1.1, deprecated in v1.2, and removed in v1.3 then an
implementation can support v1.1, v1.2, and v1.3 by implementing `/test` as it was defined in v1.2 (minus
the deprecation flag). If the implementation wanted to support just v1.2 and v1.3, then it still must
implement `/test`. If the implementation only wanted to support v1.3, then it *should not* implement
`/test` at all because it was removed.
Generally deprecation is paired with replacement or breaking changes. For example, if `/v3/sync` were
to be modified such that it needed to be bumped to `v4`, the MSC which does so would deprecate `/v3/sync`
in favour of its proposed `/v4/sync`. Because endpoints are versioned on a per-endpoint basis, `/v4/sync`
will still work with a server that supports `/v3/profile` (for example) - the version number doesn't mean
an implementation can only use v4 endpoints.
## Potential issues
None appear to be relevant to be discussed on their own - they are discussed in their respective
sections above when raised.
## Alternatives
There are some strong opinions that we should use proper semantic versioning for the specification
instead of the inspired system proposed here. So, why shouldn't we use semantic versioning?
1. It's meant for software and library compatibility, not specifications. Though it could theoretically
be used as a specification version, the benefits of doing so are not immediately clear. The scheme
proposed here is simple enough where rudimentary comparisons are still possible between versions,
and existing semantic versioning libraries can still be made to work. Further, the specification's
version number should not be relied upon by a library for its versioning scheme - libraries,
applications, etc should have their own versioning scheme so they may work independently of the
spec's release schedule.
2. It has potential for causing very high major version numbers. Though largely an aesthetic concern,
it can be hard to market Matrix v45 (or even Matrix v4) to potential ecosystem adopters due to
the apparent unstable-ness of the specification. Similarly, the major version is used for advertising
purposes which could be confusing or overly noisy to say there's a major version every few
releases. By instead staying in the 1.x series for a long period of time, the specification appears
stable and easy to work with, attracting potential adopters and making that 2.0 release feel all
that more special.
3. The semantic versioning spec is not followed in practice. Most uses of semantic versioning are
actually off-spec adaptations which are largely compatible with the ideals of the system. This, however,
puts Matrix in a difficult spot as it would want to say we follow semantic versioning, but can't
because there's no relevant specification document to link to. Even if there was, it would appear
as though we were encouraging the idea of forking a specification as a specification ourselves,
which may be confusing if not sending the wrong message entirely. Though the system proposed here
is a reinvention of semantic versioning to a degree, this proposed system is different from how
semantic versioning works in so many ways it is not entirely comparable.
4. The benefit of saying we use a well-popularized versioning system is not a strong enough argument
to be considered here.
This MSC is also inherently incompatible with semantic versioning due to its approach to deprecation.
Instead of encouraging breaking changes (removal of endpoints) be major version changes, this MSC
says that happens at the minor version change level. As mentioned in the relevant section, this is
not foreseen to be an issue for Matrix given its a system already used by the protocol and is common
enough to at least be moderately familiar - the arguments for using semantic versioning in this respect
do not hold up, per above.
## Security considerations
None relevant - if we need to make a security release for Matrix then we simply make a release and
advertise accordingly.
## Unstable prefix
The author does not recommend that this MSC be implemented prior to it landing due to the complexity
involved as well as the behavioural changes not being possible to implement. However, if an implementation
wishes to try anyways, it should use `org.matrix.msc2844` in the `unstable_features` of `/versions`
and use `/_matrix/client/unstable/org.matrix.msc2844` in place of `/_matrix/client/r0`.
This MSC is largely proven as possible through an in-development build of the specification which uses
an alternative toolchain for rendering the specification: https://adoring-einstein-5ea514.netlify.app/
(see the 'releases' dropdown in the top right; link may not be available or even the same as described
here due to development changes - sorry).

247
proposals/2858-Multiple-SSO-Identity-Providers.md
Unescape Escape View File

@ -1,247 +0,0 @@
# MSC2858: Multiple SSO Identity Providers
Matrix already has generic SSO support, but it does not yield the best user experience especially for
instances which wish to offer multiple identity providers (IdPs). This MSC provides a simple and fully
backwards compatible way to extend the current spec which would allow clients to give users options
like `Continue with Google` and `Continue with Github` side-by-side.
Currently, Matrix supports `m.login.sso`, `m.login.token` and `/login/sso/redirect` for clients to
pass their user to the configured Identity provider and for them to come back with something which
is exchangeable for a Matrix access token. This flow offers no insight to the user as to what
Identity providers are available: clients can offer only a very generic `Sign in with SSO`
button. With the currently possible solutions and workarounds the experience is far from great
and users have to blindly click `Sign in with SSO` without any clue as to what's hiding on the other
side of the door. Some users will definitely not be familiar with `SSO` but will be with the concept of
"Continue with Google" or similar.
## Proposal
We extend the [login
flow](https://matrix.org/docs/spec/client_server/r0.6.1#login) to allow clients
to choose an SSO Identity provider before control is handed over to the
server. The following sequence diagram illustrates the proposed, updated, login flow:
<!-- source for the following is in images/2858-seq-diagram.txt -->
![Sequence diagram](https://user-images.githubusercontent.com/2403652/104897523-61fb4b00-5970-11eb-88f7-9fc0956b33a2.png)
### Extensions to login flow discovery
The response to [`GET /_matrix/client/r0/login`](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-login)
is extended to **optionally** include an `identity_providers` property for
flows whose type `m.login.sso`. This would look like this:
```json
{
"flows": [
{
"type": "m.login.sso",
"identity_providers": [
{
"id": "google",
"name": "Google",
"icon": "mxc://...",
"brand": "google"
},
{
"id": "github",
"name": "Github",
"icon": "mxc://...",
"brand": "github"
}
]
},
{
"type": "m.login.token"
}
]
}
```
The value of the `identity_providers` property is a list, each entry consisting
of an object with the following fields:
* The `id` field is **required**. It is an opaque string chosen by the
homeserver implementation, and uniquely identifies the identity provider on
that server. Clients should not infer any semantic meaning from the
identifier. The identifier should be between 1 and 255 characters in length,
and should consist of the characters matching unreserved URI characters as
defined in [RFC3986](http://www.ietf.org/rfc/rfc3986.txt):
```
ALPHA DIGIT "-" / "." / "_" / "~"
```
* The `name` field is **required**. It should be a human readable string
intended for printing by the client. No explicit length limit or grammar is
specified.
* The `icon` field is **optional**. It should point to an icon representing
the IdP. If present then it must be an MXC URI to an image resource.
* The `brand` field is **optional**. It allows the client to style the login
button to suit a particular brand. It should be a string using the following
grammar:
* Must be at least one character and no more than 255 characters in length.
* Must start with one of the characters `[a-z]`, and be entirely composed
of the characters `[a-z]`, `[0-9]`, `-`, `_` and `.`.
To reduce confusion over which identifier should be used for each brand
(for example: should "Sign in with Microsoft" be `microsoft` or
`azure`?), it is proposed to maintain a registry of identifiers outside
the core specification document, avoiding the need for a full MSC to add
entries to the list. An initial list of proposed identifiers is given below.
[Rationale: this grammar is based on the
[MSC2758](https://github.com/matrix-org/matrix-doc/pull/2758), removing the
requirements for a namespaced hierarchy. In
[discussion](https://github.com/matrix-org/matrix-doc/pull/2858#discussion_r565506802),
it was agreed that a separate registry was seen as important for a
lightweight process by which implementations can agree on identifiers. The
registry makes the namespacing of MSC2758 redundant; the namespacing system
was also somewhat confusing.]
Server implementations are free to add additional brands, though they should
be mindful of clients which do not recognise any given brand.
Clients are free to implement any set of brands they wish, including all or
any of the brands listed in the registry, but are expected to apply a
sensible unbranded fallback for any brand they do not recognise/support.
Where `icon` and `brand` are both present, it is recommended that clients
which support the `brand` give precedence to `brand` over `icon`.
### Extend the `/login/sso/redirect` endpoint
A new endpoint is added to support redirecting directly to one of the IdPs:
`GET /_matrix/client/r0/login/sso/redirect/{idp_id}`
This would behave identically to the existing endpoint without the last argument
except would allow the server to forward the user directly to the correct IdP.
For the case of backwards compatibility the existing endpoint is to remain,
and if the server supports multiple SSO IdPs it should offer the user a page
which lets them choose between the available IdP options as a fallback.
If the `idp_id` is unrecognised, the server should display some sort of error
page to the user. (A protocol whereby an error can be returned to the original
client could be a matter for a future improvement, but is out of scope for now.)
### Notes on user-interactive auth
No change is proposed to the SSO flow for User-Interactive Authentication.
For a reauthentication operation, the server implementation is free to choose
any suitable IdP to authenticate the user. (Often, this will simply be
the IdP that the user logged in with.)
### Proposed initial identifiers for the `brand` identifier
The following identifiers are proposed for the initial content of the `brand`
identifier registry. The descriptions are guidelines to help server
administrators pick a suitable brand identifier, and to help client authors
style buttons in their clients.
* Identifier: `apple`
Description: Suitable for "Sign in with Apple": see
https://developer.apple.com/design/human-interface-guidelines/sign-in-with-apple/overview/buttons/.
* Identifier: `facebook`
Description: "Continue with Facebook": see
https://developers.facebook.com/docs/facebook-login/web/login-button/.
* Identifier: `github`
Description: Logos available at https://github.com/logos.
* Identifier: `gitlab`
Description: Logos available at https://about.gitlab.com/press/press-kit/.
* Identifier: `google`
Description: Suitable for "Google Sign-In": see
https://developers.google.com/identity/branding-guidelines.
* Identifier: `twitter`
Description: Suitable for "Log in with Twitter": see
https://developer.twitter.com/en/docs/authentication/guides/log-in-with-twitter#tab1.
When considering a new identifier for private use, administrators should pick
some sensible name following the advice of [RFC6648 sec
3](https://tools.ietf.org/html/rfc6648#section-3).
## Alternatives
An alternative to the whole approach would be to allow `m.login.sso.$idp` but this forces
treating an opaque identifier as hierarchical and offers worse backwards compatibility.
An alternative to the proposed backwards compatibility plan where the server offers a
fallback page which fills the gap and lets the user choose which SSO IdP they need is
for the server to deterministically always pick one, maybe the first option and let
old clients only auth via that one but that means potentially locking users out of their
accounts.
[MSC2964](https://github.com/matrix-org/matrix-doc/pull/2964) proposes
replacing much of Matrix's authentication mechanism with OAuth2.0. If that is
adopted, then the Matrix client would not be able to specify an authentication
mechanism; rather it is left up to the server to host pages allowing the user
to choose their authentication mechanism.
### Styling information as an alternative to `brand`
The `brand` field is intended to allow clients to style "login" buttons according
to the identity provider in question. For example, a mobile application might
show:
![login buttons](images/2858-login.png)
Some identity providers have very specific rules about how such buttons should
be presented, so a fine level of control is important.
An alternative way to achieve this would be for the server to give full details
about the styling: icon, font colour, border colour, background colour,
etc. However, this soon becomes unscalable. For example, it might be desirable
to offer each logo at a range of resolutions to suit different screen sizes.
Likewise, some brands need different styling depending on the background
colour, so a complete second set of colours must be specified to account for
dark or light themes.
## Potential issues
* New Identity Providers added by server administators will be unbranded until
clients adopt support for the new brand.
## Security considerations
This could potentially aid phishing attacks by bad homeservers, where if the app says
`Continue with Google` and then they are taken to a page which is styled to look like
the Google login page they might be a tiny bit more susceptible to being phished as opposed
as to when they click a more generic `Sign in with SSO` button, but this attack was possible
anyhow using a different vector of a controlled Element/client instance which modifies
the text.
## Unstable prefix
Whilst in development use `org.matrix.msc2858.identity_providers` for the flow
discovery and
`/_matrix/client/unstable/org.matrix.msc2858/login/sso/redirect/{idp_id}` for
the new endpoints.
When identity providers are listed under the experimental
`org.matrix.msc2858.identity_providers` field of the response to `/login`,
(instead of `identity_providers`), different values for the `brand` field are
used. In particular the following were defined:
* `org.matrix.gitlab` (now `gitlab`).
* `org.matrix.github` (now `github`).
* `org.matrix.apple` (now `apple`).
* `org.matrix.google` (now `google`).
* `org.matrix.facebook` (now `facebook`).
* `org.matrix.twitter` (now `twitter`).

62
proposals/2874-single-ssss.md
Unescape Escape View File

@ -1,62 +0,0 @@
# MSC2874: Single SSSS
[Secure Secret Storage and
Sharing](https://github.com/matrix-org/matrix-doc/pull/1946) (SSSS) was
designed to allow the user to create multiple keys that would be able to
decrypt different subsets of the secrets. However, the vast majority of users
do not need this feature.
This proposal defines how clients should behave if they only wish to support a
single key, by defining which key clients should use if multiple keys are
present. It also makes the `name` field in the `m.secret_storage.key.*` events
optional, as this field was mainly added to allow a user to select between
different keys.
## Proposal
If a client wants to present a simplified interface to users by not supporting
multiple SSSS keys, then the client should use the default key (the key listed
in the `m.secret_storage.default_key` account data event.) If there is no
default key the client may behave as if there is no SSSS key at all. When such
a client creates an SSSS key, it must mark that key as being the default key.
The `name` field in the `m.secret_storage.key.*` account data events is
optional, rather than required. If a client wishes to display multiple keys to
a user and a given key does not have a `name` field, the client may use a
default name as the key's name, such as "Unnamed key", or "Default key" if the
key is marked as default.
For example, when a client creates a key with ID `abcdefg`, it will create an
`m.secret_storage.key.abcdefg` account data event to store information about
the key. It will then mark it as the default key by setting the
`m.secret_storage.default_key` account data to `{"key": "abcdefg"}`. When
another client logs in after this, it will see that the default key has been
set, and will know to use that key as the SSSS key.
## Potential issues
If secrets are encrypted using a key that is not marked as default, a client
might not decrypt the secrets, even if it would otherwise be able to.
## Alternatives
Rather than solely relying on the key marked as default, a client could guess
at what key to use. For example, it could look at the secrets that it needs,
see what keys they are encrypted with, and if there is only one common key,
then it could use that. (This is what Element currently does.) Or if there
are multiple keys, it could use some sort of heuristic to pick a key. However,
this approach can be error-prone, and it is better to rely on an explicit
marking.
## Security considerations
None
## Unstable prefix
An unstable prefix is not needed for a behaviour change in choosing the key to
use as there are no event/endpoint changes.
Some clients already omit the `name` field (notably, matrix-js-sdk
unintentionally does this -- mea culpa), and this does not seem to be causing
issues, so an unstable prefix seems unnecessary for this.

163
proposals/2918-refreshtokens.md
Unescape Escape View File

@ -1,163 +0,0 @@
# MSC2918: Refresh tokens
In Matrix, requests to the Client-Server API are currently authenticated using non-expiring, revocable access tokens.
An access token might leak for various reasons, including:
- leaking from the server database (and its backups)
- intercepting it with a man-in-the-middle attack
- leaking from the client storage (and its backups)
In the OAuth 2.0 world, this vector of attack is partly mitigated by having expiring access tokens with short lifetimes and rotating refresh tokens to renew them.
This MSC adds support for expiring access tokens and introduces refresh tokens to renew them.
A more [detailed rationale](#detailed-rationale) of what kind of attacks it mitigates lives at the end of this document.
## Proposal
Homeservers can choose to have access tokens expire after a short amount of time, forcing the client to renew them with a refresh token.
A refresh token is issued on login and rotates on each usage.
It allows homeservers to opt for signed and non-revocable access tokens (JWTs, Macaroon, etc.) for performance reasons if their expiration is short enough (less than 5 minutes).
It is heavily recommended for clients to support refreshing tokens for additional security.
They can advertise their support by adding a `"refresh_token": true` field in the request body on the `/login` and `/register` APIs.
Handling of clients that do *not* support refreshing access tokens is up to individual homeserver deployments.
For example, server administrators may choose to support such clients for backwards-compatibility, or to expire access tokens anyway for improved security at the cost of inferior user experience in legacy clients.
If a client uses an access token that has expired, the server will respond with an `M_UNKNOWN_TOKEN` error, preferably with the `soft_logout` parameter set to `true` to improve the user experience in legacy clients.
Thus, if a client receives an `M_UNKNOWN_TOKEN` error, and it has a refresh token available, it should no longer assume that it has been logged out, and instead attempt to refresh the token.
If the client was in fact logged out, then the server will respond with an `M_UNKNOWN_TOKEN` error to the token refresh request, possibly with the `soft_logout` parameter set.
### Login API changes
The login API returns two additional fields:
- `expires_in_ms`: The lifetime in milliseconds of the access token.
- `refresh_token`: The refresh token, which can be used to obtain new access tokens.
This also applies to logins done by application services.
Both fields are optional.
If `expires_in_ms` is missing, the client can assume the access token won't expire.
If `refresh_token` is missing but `expires_in_ms` is present, the client can assume the access token will expire but it won't have a way to refresh the access token without re-logging in.
Clients advertise their support for refreshing tokens by setting the `refresh_token` field to `true` in the request body.
### Account registration API changes
Unless `inhibit_login` is `true`, the account registration API returns two additional fields:
- `expires_in_ms`: The lifetime in milliseconds of the access token.
- `refresh_token`: The refresh token, which can be used to obtain new access tokens.
This also applies to registrations done by application services.
As in the login API, both field are optional.
Clients advertise their support for refreshing tokens by setting the `refresh_token` field to `true` in the request body.
### Token refresh API
This API lets the client refresh the access token.
A new refresh token is also issued.
The existing refresh token remains valid until the new access token (or refresh token) is used, at which point it is revoked.
This allows for the request to get lost in flight.
The Matrix server can revoke the old access token right away, but does not have to since its lifetime is short enough that it will expire anyway soon after.
`POST /_matrix/client/r0/refresh`
```json
{
"refresh_token": "aaaabbbbccccdddd"
}
```
response:
```json
{
"access_token": "xxxxyyyyzzz",
"expires_in_ms": 60000,
"refresh_token": "eeeeffffgggghhhh"
}
```
If the `refresh_token` is missing from the response, the client can assume the refresh token has not changed and use the same token in subsequent token refresh API requests.
The `refresh_token` parameter can be invalid for two reasons:
- if it does not exist
- if it was already used once
In both cases, the server must reply with a `401` HTTP status code and an `M_UNKNOWN_TOKEN` error code.
This new use case of the `M_UNKNOWN_TOKEN` error code must be reflected in the spec.
As with other endpoints, the server can include an extra `soft_logout` parameter in the response to signify the client it should do a soft logout.
This new API should be rate-limited and does not require authentication since only the `refresh_token` parameter is needed.
Identity assertion via the `user_id` query parameter as defined by the Application Service API specification is disabled on this endpoint.
### Device handling
The current spec states that "Matrix servers should record which device each access token is assigned to".
This must be updated to reflect that devices are bound to a session, which are created during login and stays the same after refreshing the token.
## Potential issues
The refresh token being rotated on each refresh is strongly recommended in the OAuth 2.0 world for unauthenticated clients to avoid token replay attacks.
This can however make the deployment of CLI tools for Matrix a bit harder, since the credentials can't be statically defined anymore.
This is not an issue in OAuth 2.0 because usually CLI tools use the client credentials flow, also known as service accounts.
An alternative would be to make the refresh token non-rotating for now but recommend clients to support rotation of refresh tokens and enforce it later on.
## Alternatives
This MSC defines a new endpoint for token refresh, but it could also be integrated as a new authentication mechanism.
## Security considerations
The time to live (TTL) of access tokens isn't enforced in this MSC but is advised to be kept relatively short.
Servers might choose to have stateless, digitally signed access tokens (JWT are good examples of this), which makes them non-revocable.
The TTL of access tokens should be around 15 minutes if they are revocable and should not exceed 5 minutes if they are not.
## Unstable prefix
While this MSC is not in a released version of the specification, clients should use the `org.matrix.msc2918.refresh_token` field in place of the `refresh_token` field in requests to the login and registration endpoints.
The refresh token endpoint should be served and used using the unstable prefix: `POST /_matrix/client/unstable/org.matrix.msc2918/refresh`.
## Detailed rationale
This MSC does not aim to protect against a completely compromised client.
More specifically, it does not protect against an attacker that managed to distribute an alternate, compromised version of the client to users.
In contrast, it protects against a whole range of attacks where the access token and/or refresh token get leaked but the client isn't completely compromised.
For example, those tokens can leak from user backups (user backs up his device on a NAS, the NAS gets compromised and leaks a backup of the client's secret storage), but one can assume those backups could be at least 5 min old.
If the leak only includes the access token, it is useless to the attacker since it would have expired.
If it also includes the refresh token, it is useless *if* the token was refreshed before (which will happen if the user just opens their Matrix client in between).
Worst case scenario, the leaked refresh token is still valid: in this case, the attacker would consume the refresh token to get a valid access token, but when the original client tries to use the same refresh token, the homeserver can detect it, consider the session has been compromised, end the session and warn the user.
This kind of attack also applies to leakage from the server, which could happen from database backups, for example.
The important thing here is while it does not completely prevent attacks in case of a token leakage, it does make this range of attack a lot more time-sensitive and detectable.
A homeserver will notice if a refresh token is being used twice.
The IETF has interesting [guidelines for refresh tokens](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics#section-4.13.2).
They recommend that either:
- the refresh tokens are sender-bound and require client authentication (making token leakage completely useless if the client credentials are not leaked at the same time)
- or make them rotate to make the attack a lot harder, as described just above.
Since all clients are "public" in the Matrix world, there are no client-bound credentials that could be used, hence the rotation of refresh tokens.
---
The other kind of scenario where this change makes sense is to help further changes in the homeservers.
A good, recent example of this, is in Synapse v1.34.0 [they moved away from macaroons for access tokens](https://github.com/matrix-org/synapse/pull/5588) to random, shorter, saved in database tokens, similar to [what GitHub did recently](https://github.blog/2021-04-05-behind-githubs-new-authentication-token-formats/).
Because there is no refresh token mechanism in the C2S API, most Synapse instances now have a mix of the two formats of tokens, and for a long time.
It makes it impossible to enforce the new format of tokens without invalidating all existing sessions, making it impossible to roll out changes like a web-app firewall in front of Synapse that verifies the shape and checksums of tokens even before reaching Synapse.
---
Lastly, expiring tokens already exist in Synapse (via the `session_lifetime` configuration parameter).
Before this MSC, clients had no idea when the session would end and relied on the server replying with a 401 error with `soft_logout: true` in the response on a random request to trigger a soft logout and go through the authentication process again.
A side effect of this MSC (although it could have been introduced separately) is that the login responses can now include a `expires_in_ms` to inform the clients when the token will expire.

Some files were not shown because too many files have changed in this diff Show More

Write Preview
Loading…
Cancel
Save