diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 00000000..2f48efb5 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,7 @@ +# EditorConfig is awesome: https://EditorConfig.org +root = true + +[*] +insert_final_newline = true +charset = utf-8 +max_line_length = 120 diff --git a/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md b/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md index b8e262a4..216c58c3 100644 --- a/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md +++ b/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md @@ -16,6 +16,7 @@ assignees: '' * [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-spec-proposals/blob/master/CONTRIBUTING.md#sign-off) * [ ] Update the title and file name of your proposal to match this PR's number (after opening). * [ ] Pull request includes a ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above. +* [ ] Your MSC adheres to each point in the [MSC Checklist](MSC_CHECKLIST.md). * [ ] Ask in [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) to get feedback on this PR. diff --git a/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md b/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md index 4744c746..3ea86ba5 100644 --- a/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md +++ b/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md @@ -16,5 +16,6 @@ assignees: '' * [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-spec-proposals/blob/master/CONTRIBUTING.md#sign-off) * [ ] Update the title and file name of your proposal to match this PR's number (after opening). * [ ] Pull request includes a ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above. +* [ ] Have a look at the [MSC Checklist](MSC_CHECKLIST.md) for guidelines on various aspects of your MSC. diff --git a/.github/_typos.toml b/.github/_typos.toml index 7c107218..2e7159b7 100644 --- a/.github/_typos.toml +++ b/.github/_typos.toml @@ -1,2 +1,11 @@ [default] check-filename = true + +[default.extend-identifiers] +OTK = "OTK" +OTKs = "OTKs" + +[default.extend-words] +OTK = "OTK" +OTKs = "OTKs" +Iy = "Iy" diff --git a/.github/workflows/spell-check.yaml b/.github/workflows/spell-check.yaml index 7b1fd92d..ff444616 100644 --- a/.github/workflows/spell-check.yaml +++ b/.github/workflows/spell-check.yaml @@ -11,9 +11,9 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository - uses: actions/checkout@v2 + uses: actions/checkout@v4 - name: Check spelling of proposals - uses: crate-ci/typos@9be36f97fdbe645ee9a12449fb13aca856c2516a + uses: crate-ci/typos@f2c1f08a7b3c1b96050cb786baaa2a94797bdb7d # v1.20.10 with: config: ${{github.workspace}}/.github/_typos.toml diff --git a/MSC_CHECKLIST.md b/MSC_CHECKLIST.md new file mode 100644 index 00000000..30e36436 --- /dev/null +++ b/MSC_CHECKLIST.md @@ -0,0 +1,50 @@ +# MSC Checklist + +This document contains a list of final checks to perform on an MSC before it +is accepted. The purpose is to prevent small clarifications needing to be +made to the MSC after it has already been accepted. + +Spec Core Team (SCT) members, please ensure that all of the following checks +pass before accepting a given Matrix Spec Change (MSC). + +MSC authors, feel free to ask in a thread on your PR or in the +[#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) room for +clarification of any of these points. + +- [ ] Are [appropriate + implementation(s)](https://spec.matrix.org/proposals/#implementing-a-proposal) + specified in the MSC’s PR description? +- [ ] Are all MSCs that this MSC depends on already accepted? +- [ ] For each new endpoint that is introduced: + - [ ] Have authentication requirements been specified? + - [ ] Have rate-limiting requirements been specified? + - [ ] Have guest access requirements been specified? + - [ ] Are error responses specified? + - [ ] Does each error case have a specified `errcode` (e.g. `M_FORBIDDEN`) and HTTP status code? + - [ ] If a new `errcode` is introduced, is it clear that it is new? +- [ ] Will the MSC require a new room version, and if so, has that been made clear? + - [ ] Is the reason for a new room version clearly stated? For example, + modifying the set of redacted fields changes how event IDs are calculated, + thus requiring a new room version. +- [ ] Are backwards-compatibility concerns appropriately addressed? +- [ ] Are the [endpoint conventions](https://spec.matrix.org/latest/appendices/#conventions-for-matrix-apis) honoured? + - [ ] Do HTTP endpoints `use_underscores_like_this`? + - [ ] Will the endpoint return unbounded data? If so, has pagination been considered? + - [ ] If the endpoint utilises pagination, is it consistent with [the + appendices](https://spec.matrix.org/v1.8/appendices/#pagination)? +- [ ] An introduction exists and clearly outlines the problem being solved. + Ideally, the first paragraph should be understandable by a non-technical + audience +- [ ] All outstanding threads are resolved + - [ ] All feedback is incorporated into the proposal text itself, either as a fix or noted as an alternative +- [ ] While the exact sections do not need to be present, the details implied by the proposal template are covered. Namely: + - [ ] Introduction + - [ ] Proposal text + - [ ] Potential issues + - [ ] Alternatives + - [ ] Security considerations + - [ ] Dependencies +- [ ] Stable identifiers are used throughout the proposal, except for the unstable prefix section + - [ ] Unstable prefixes [consider](README.md#unstable-prefixes) the awkward accepted-but-not-merged state + - [ ] Chosen unstable prefixes do not pollute any global namespace (use “org.matrix.mscXXXX”, not “org.matrix”). +- [ ] Changes have applicable [Sign Off](CONTRIBUTING.md#sign-off) from all authors/editors/contributors diff --git a/README.md b/README.md index 47435d6a..5b8e233b 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ Some tips for MSC writing: * Please wrap your lines to 120 characters maximum. This allows readers to review your markdown without needing to horizontally - scroll back and forth. Many markdown text editors have this a feature. + scroll back and forth. Many markdown text editors have this feature. * If you are referencing an existing endpoint in the spec, or another MSC, it is very helpful to add a link to them so the reader does not need to search themselves. Examples: @@ -108,6 +108,9 @@ Some tips for MSC writing: effort using [Mermaid](https://mermaid-js.github.io/mermaid/#/). See [this guide](https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/) for more information. +* Take a look at the [MSC Checklist](MSC_CHECKLIST.md). When it comes time for + the Spec Core Team to review your MSC for acceptance, they'll use the items + on this checklist as a guide. #### 2. Submitting a Pull Request @@ -184,19 +187,94 @@ like help with writing spec PRs, feel free to join and ask questions in the #### Unstable prefixes -*Unstable* prefixes are the namespaces which are used before an MSC has -completed FCP (see above). While the MSC might propose that a `m.space` or -`/_matrix/client/v1/account/whoami` endpoint should exist, the implementation -cannot use a *stable* identifier such as `/v1/` or `m.space` prior to the MSC -being accepted: it needs unstable prefixes. - -Typically for MSCs, one will use `org.matrix.msc0000` (using the real MSC -number once known) as a prefix. For the above examples, this would mean -`org.matrix.msc0000.space` and -`/_matrix/client/unstable/org.matrix.msc0000/account/whoami` to allow for -breaking compatibility changes between edits of the MSC itself, or indeed -another competing MSC that's attempting to add the same identifiers. - +"Unstable prefixes" are the namespaces which are used by implementations while +an MSC is not yet accepted. + +For instance, an MSC might propose that a `m.space` +event type or an `/_matrix/client/v1/account/whoami` endpoint should exist. +However, implementations cannot use these *stable* identifiers until the MSC +has been accepted, as the underlying design may change at any time; the design is +*unstable*. + +Instead, an MSC can define a namespace such as `org.matrix.msc1234` (using the real +MSC number once known) which is added to the stable identifier, allowing for +breaking changes between edits of the MSC itself, and preventing clashes with other +MSCs that might attempt to add the same stable identifiers. + +For the above examples, this would mean using `org.matrix.msc1234.space` and +`/_matrix/client/unstable/org.matrix.msc1234/account/whoami`. It is also fine to +use more traditional forms of namespace prefixes, such as `com.example.*` (e.g. +`com.example.space`). + +Note: not all MSCs need to make use of unstable prefixes. They are only needed if +implementations of your MSC need to exist in the wild before your MSC is accepted, +*and* the MSC defines new endpoints, field names, etc. + +#### Unstable feature flags + +It is common when implementing support for an MSC that a client may wish to check +if the homeserver it is communicating with supports an MSC. +Typically, this is handled by the MSC defining an +entry in the `unstable_features` dictionary of the +[`/_matrix/client/versions`](https://spec.matrix.org/v1.6/client-server-api/#get_matrixclientversions) +endpoint, in the form of a new entry: + +```json5 +{ + "unstable_features": { + "org.matrix.msc1234": true + } +} +``` + +... with a value of `true` indicating that the feature is supported, and `false` +or lack of the field altogether indicating the feature is not supported. + +#### When can I use stable identifiers? + +[According to the spec +process](https://spec.matrix.org/proposals/#early-release-of-an-mscidea): once +an MSC has been accepted, implementations are allowed to switch to *stable* +identifiers. However, the MSC is still not yet part of a released spec version. + +In most cases, this is not an issue. For instance, if your MSC specifies a new +event type, you can now start sending events with those types! + +Some MSCs introduce functionality where coordination between implementations is +needed. For instance, a client may want to know whether a homeserver supports +the stable version of a new endpoint before actually attempting to request it. +Or perhaps the new event type you're trying to send relies on the homeserver +recognising that new event type, and doing some work when it sees it. + +At this point, it may be best to wait until a new spec version is released with +your changes. Homeservers that support the changes will eventually advertise +that spec version under `/versions`, and your client can check for that. + +But if you really can't wait, then there is another option: the homeserver can +tell clients that it supports *stable* indentifiers for your MSC before it +enters a spec version, using yet another `unstable_features` flag: + +```json5 +{ + "unstable_features": { + "org.matrix.msc1234": true, + "org.matrix.msc1234.stable": true + } +} +``` + +If a client sees that `org.matrix.msc1234.stable` is `true`, it knows that it +can start using stable identifiers for the new MSC, and the homeserver will +accept and act on them accordingly. + +Note: While the general pattern of using the text ".stable" has emerged from +previous MSCs, you can pick any name you like. You need only to clearly state +their meaning, usually under an "Unstable prefixes" header in your MSC. + +See +[MSC3827](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3827-space-explore.md#unstable-prefix) +for a good example of an MSC that wanted to use such a flag to speed up +implementation rollout, and how it did so. #### Room versions diff --git a/proposals/1692-terms-at-registration.md b/proposals/1692-terms-at-registration.md new file mode 100644 index 00000000..1fc77d3b --- /dev/null +++ b/proposals/1692-terms-at-registration.md @@ -0,0 +1,92 @@ +# MSC1692: Terms of service at registration + +At registration, homeservers may wish to require the user to accept a given set of policy documents, +such as a terms of service and privacy policy. There may be many different types of documents, all of +which are versioned and presented in (potentially) multiple languages. + +This proposal covers requiring users to accept the list of documents during registration. Future +improvements could include informing the user *after* registration that a document has changed, which +has been spun out to [MSC3012](https://github.com/matrix-org/matrix-spec-proposals/pull/3012). + +## Proposal + +The [User-Interactive Authentication](https://spec.matrix.org/v1.9/client-server-api/#user-interactive-authentication-api) +API (UIA) is currently used during registration to create a new account. In future, it is expected +that OIDC will be used instead, which can include support for this MSC's principles without needing +to change the Matrix specification itself. As a measure until OIDC is here though, this MSC exists +to fill the need. + +A new `m.login.terms` authentication type is introduced, allowing servers to include it in registration +flows if it desires. Servers which do not require policy acceptance at registration are not required +to support this flow. + +The parameters for the new authentication type look like the following: + +```json +{ + "policies": { + "terms_of_service": { + "version": "1.2", + "en": { + "name": "Terms of Service", + "url": "https://example.org/somewhere/terms-1.2-en.html" + }, + "fr": { + "name": "Conditions d'utilisation", + "url": "https://example.org/somewhere/terms-1.2-fr.html" + } + }, + "privacy_policy": { + "version": "1.2", + "en": { + "name": "Privacy Policy", + "url": "https://example.org/somewhere/privacy-1.2-en.html" + }, + "fr": { + "name": "Politique de confidentialité", + "url": "https://example.org/somewhere/privacy-1.2-fr.html" + } + } + } +} +``` + +Each key under `policies` is a "Policy ID", and defined by the server. They are an opaque identifier +(described later in this proposal). Each policy object associated with the policy ID has a required +`version` as a convenience to the client, and is another opaque identifier. All other keys are language +codes to represent the same document. The client picks the language which best suits the user. + +Language codes *should* be formatted as per [Section 2.2 of RFC 5646](https://datatracker.ietf.org/doc/html/rfc5646#section-2.2), +noting that some implementation *may* use an underscore instead of dash. For example, `en_US` instead +of `en-US`. This recommendation is to ensure maximum compatibility with existing conventions around +language choices in (Matrix) clients. + +`name` and `url` for each policy document are required, and are arbitrary strings with no maximum +length. `url` *must* be a valid URI with scheme `https://` or `http://`. Insecure HTTP is discouraged. + +If a client encounters an invalid parameter, registration should stop with an error presented to the +user. + +To complete the stage, accepting *all* of the listed documents, the client submits an empty `auth` +dict. The client *should* present the user with a checkbox to accept each policy, including a link +to the provided `url`, or otherwise rely on [fallback auth](https://spec.matrix.org/v1.9/client-server-api/#fallback). + +The server is expected to track which document versions it presented to the user during registration, +if applicable. + +### Opaque identifier + +This definition is inherited from [MSC1597](https://github.com/matrix-org/matrix-spec-proposals/pull/1597). + +> Opaque IDs must be strings consisting entirely of the characters +> `[0-9a-zA-Z._~-]`. Their length must not exceed 255 characters and they must +> not be empty. + +## Unstable prefix + +Regrettably, this MSC was implemented with *stable* identifiers before an unstable identifiers process +was established. Implementation has existed in some capacity since 2018: https://github.com/matrix-org/synapse/pull/4004 + +Noting that the modern MSC process forbids such behaviour, new implementations should use the stable +`m.login.terms` identifier regardless of MSC status. If the MSC changes in a breaking way, a new +identifier *must* be chosen, and *must* include a proper unstable prefix. diff --git a/proposals/1711-x509-for-federation.md b/proposals/1711-x509-for-federation.md index 7fd90560..ef351ade 100644 --- a/proposals/1711-x509-for-federation.md +++ b/proposals/1711-x509-for-federation.md @@ -174,7 +174,7 @@ certificates comes with a number of downsides. Configuring a working, federating homeserver is a process fraught with pitfalls. This proposal adds the requirement to obtain a signed certificate to -that process. Even with modern intiatives such as Let's Encrypt, this is +that process. Even with modern initiatives such as Let's Encrypt, this is another procedure requiring manual intervention across several moving parts. On the other hand: obtaining an SSL certificate should be a familiar process to diff --git a/proposals/1884-replace-slashes-in-event_ids.md b/proposals/1884-replace-slashes-in-event_ids.md index bec8d7ad..7ae91102 100644 --- a/proposals/1884-replace-slashes-in-event_ids.md +++ b/proposals/1884-replace-slashes-in-event_ids.md @@ -88,7 +88,7 @@ right encoding. A potential extension would be to change *all* Base64 encodings to be URL-safe. This would address the inconsistency. However, it feels like a large job which would span the entire matrix ecosystem (far larger than - updating clients to URL-encode their URL prarameters), and again the + updating clients to URL-encode their URL parameters), and again the situation would be confusing while the transition was in progress. 2. Incompleteness. Event IDs are certainly not the only identifier which can diff --git a/proposals/1929-admin-contact.md b/proposals/1929-admin-contact.md new file mode 100644 index 00000000..7a5639ff --- /dev/null +++ b/proposals/1929-admin-contact.md @@ -0,0 +1,71 @@ +# MSC1929 Homeserver Admin Contact and Support page + +Currently, contacting a homeserver admin is difficult because you need to have insider knowledge +of who the admin actually is. This proposal aims to fix that by specifying a way to add contact details +of admins, as well as a link to a support page for users who are having issues with the service. + +This proposal aims to fix https://github.com/matrix-org/matrix-doc/issues/484 + +## Proposal + +The proposal suggests adding a new endpoint: `https://{hostname}/.well-known/matrix/support`, +where `hostname` is the server name, without the port number. This is the same as what is used +when performing server discovery. + +The response format should be: + +```json5 +{ + "contacts": [ + { + "matrix_id": "@admin:domain.tld", + "email_address": "admin@domain.tld", + "role": "m.role.admin" + }, + { + "email_address": "security@domain.tld", + "role": "m.role.security" + } + ], + "support_page": "https://domain.tld/support.html" +} +``` + +The `contacts` array is optional, but recommended. + +The `matrix_id` and `email_address` do NOT need to have the same domain as the homeserver. It is expected that +an admin will have a "backup" contact address if the server is down, like an email or alternative mxid on a different homeserver. + +Entries may have a `matrix_id` OR an `email_address`, but at least one MUST be specified. + +`role` is an informal description of what the address(es) are used for. The only two specified in this +proposal are `m.role.admin` and `m.role.security`. + + - `m.role.admin` is a catch-all user for any queries. + - `m.role.security` is intended for sensitive requests + +A value for `role` MUST be specified. Custom values are permitted using the +[common namespaced identifier format](https://spec.matrix.org/v1.8/appendices/#common-namespaced-identifier-grammar). + +`support_page` is an optional property to specify an affiliated page of the homeserver to give users help +specific to the homeserver, like extra login/registration steps. + +At least one valid key should be provided. This means `contacts` should have at least one entry, or the `support_page` should be defined. An empty object is not considered valid, however both `contacts `and `support_page` may be specified together. + +## Alternative solutions + +Hardcode a given user localpart that should be used as an admin address. + - The account would need to either internally redirect messages intended for @admin:domain.tld to another account(s) + - OR require an admin to regularly sign into this special account to check for messages. Neither of which is useful. + +Specify the same content inside a homeserver endpoint, rather than use `.well-known`. + - This requires the homeserver to be up or responsive, which might be not very useful if trying to report issues with + connectivity. + +Use vCards. + - vCards would add bloat, as the vast majority of a vcards contents is not useful for contacting an admin. + +## Security considerations + +If the host is compromised, any information could be specified in the well known file which may direct users to send +sensitive information to a malicious user. diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md index c7a4fbf8..df505d9f 100644 --- a/proposals/2140-terms-of-service-2.md +++ b/proposals/2140-terms-of-service-2.md @@ -34,7 +34,7 @@ way that Homeservers do. ## Proposal -Throuhgout this proposal, $prefix will be used to refer to the prefix of the +Throughout this proposal, $prefix will be used to refer to the prefix of the API in question, ie. `/_matrix/identity/v2` for the IS API and `/_matrix/integrations/v1` for the IM API. diff --git a/proposals/2191-maths.md b/proposals/2191-maths.md new file mode 100644 index 00000000..335e176a --- /dev/null +++ b/proposals/2191-maths.md @@ -0,0 +1,138 @@ +# MSC2191: Markup for mathematical messages + +Some people write using an odd language that has strange symbols. No, I'm not +talking about computer programmers; I'm talking about mathematicians. In order +to aid these people in communicating, Matrix should define a standard way of +including mathematical notation in messages. + +This proposal presents a format using LaTeX, in contrast with a [previous +proposal](https://github.com/matrix-org/matrix-doc/pull/1722/) that used +MathML. + +See also: + +- https://github.com/vector-im/riot-web/issues/1945 + + +## Proposal + +A new attribute `data-mx-maths` will be added for use in `` or `
` +elements. Its value will be mathematical notation in LaTeX format. `` +is used for inline math, and `
` for display math. The contents of the +`` or `
` will be a fallback representation or the desired notation +for clients that do not support mathematical display, or that are unable to +render the entire `data-mx-maths` attribute. The fallback representation is +left up to the sending client and could be, for example, an image, or an HTML +approximation, or the raw LaTeX source. When using an image as a fallback, the +sending client should be aware of issues that may arise from the receiving +client using a different background colour. + +Example (with line breaks and indentation added to `formatted_body` for clarity): + +```json +{ + "content": { + "body": "This is an equation: sin(x)=a/b", + "format": "org.matrix.custom.html", + "formatted_body": "This is an equation: + + sin(x)=a/b + ", + "msgtype": "m.text" + }, + "event_id": "$eventid:example.com", + "origin_server_ts": 1234567890, + "sender": "@alice:example.com", + "type": "m.room.message", + "room_id": "!soomeroom:example.com" +} +``` + + +## Other solutions + +[MSC1722](https://github.com/matrix-org/matrix-doc/pull/1722/) proposes using +MathML as the format of transporting mathematical notation. It also summarizes +some other solutions in its "Other Solutions" section. + +In comparison with MathML, LaTeX has several advantages and disadvantages. + +The first advantage, which is quite obvious, is that LaTeX is much less verbose +and more readable than MathML. In many cases, the LaTeX code is a suitable +fallback for the rendered notation. + +LaTeX is a suitable input method for many people, and so converting from a +user's input to the message format would be a no-op. + +However, balanced against these advantages, LaTeX has several disadvantages as +a message format. Some of these are covered in the "Potential issues" and +"Security considerations". + + +## Potential issues + +### "LaTeX" as a format is poorly defined + +There are several extensions to LaTeX that are commonly used, such as +AMS-LaTeX. It is unclear which extensions should be supported, and which +should not be supported. Different LaTeX-rendering libraries support different +sets of commands. + +This proposal suggests that the receiving client should render the LaTeX +version if possible, but if it contains unsupported commands, then it should +display the fallback. Thus, it is up to the receiving client to decide what +commands it will support, rather than dictating what commands must be +supported. This comes at a cost of possible inconsistency between clients, but +is somewhat mitigated by the use of a fallback. Clients should, however, aim +to support, at minimum, the basic LaTeX2e maths commands and the TeX maths +commands, with the possible exception of commands that could be security risks +(see below). + +To improve compatibility, the sender's client may warn the sender if they are +using a command that comes from another package, such as AMS-LaTeX. + +### Lack of libraries for displaying mathematics + +see the corresponding section in [MSC1722](https://github.com/matrix-org/matrix-spec-proposals/pull/1722/files#diff-4a271297299040dbfa622bfc6d2aab02f9bc82be0b28b2a92ce30b14c5621f94R148-R164) + + +## Security considerations + +LaTeX is a [Turing complete programming +language](https://web.archive.org/web/20160110102145/http://en.literateprograms.org/Turing_machine_simulator_%28LaTeX%29); +it is possible to write a LaTeX document that contains an infinite loop, or +that will require large amounts of memory. While it may be fun to write a +[LaTeX file that can control a Mars +Rover](https://wiki.haskell.org/wikiupload/8/85/TMR-Issue13.pdf#chapter.2), it +is not desirable for a mathematical formula embedded in a Matrix message to +control a Mars Rover. Clients should take precautions when rendering LaTeX. +Clients that use a rendering library should only use one that can process the +LaTeX safely. + +Clients should not render mathematics by calling the `latex` executable without +proper sandboxing, as the `latex` executable was not written to handle +untrusted input. (see, for example, , +, and +.) Some LaTeX rendering libraries +are better suited for processing untrusted input. + +Certain commands, such as [those that can create +macros](https://katex.org/docs/supported#macros), are potentially dangerous; +clients should either decline to process those commands, or should take care to +ensure that they are handled in safe ways (such as by limiting recursion). In +general, LaTeX commands should be filtered by allowing known-good commands +rather than forbidding known-bad commands. Some LaTeX libraries may have +options for doing this. + +In general, LaTeX places a heavy burden on client authors to ensure that it is +processed safely. Some LaTeX rendering libraries provide security advice, for +example, . + + +## Conclusion + +Math(s) is hard, but LaTeX makes it easier to write mathematical notation. +However, using LaTeX as a format for including mathematics in Matrix messages +has some serious downsides. Nevertheless, if clients handle the LaTeX +carefully, or rely on the fallback representation, the concerns can be +addressed. diff --git a/proposals/2403-knock.md b/proposals/2403-knock.md index 50978ddb..4cebca11 100644 --- a/proposals/2403-knock.md +++ b/proposals/2403-knock.md @@ -438,8 +438,8 @@ first we summarise the semantics of the proposed changes. ### Current membership Only users without a current membership or with their current membership -set to "knock", "leave", or "invite" can knock on a room. This means that a user that -is banned or is currently in the room cannot knock on it. +set to "knock" or "leave" can knock on a room. This means that a user that +is banned, is invited or is currently in the room cannot knock on it. ### Join Rules This proposal makes use of the existing "knock" join rule. The value of diff --git a/proposals/2530-body-as-caption.md b/proposals/2530-body-as-caption.md new file mode 100644 index 00000000..246b3f97 --- /dev/null +++ b/proposals/2530-body-as-caption.md @@ -0,0 +1,144 @@ +# Body field as media caption + +When sending images or other attachments, users often want to include text to +convey additional information. Most chat platforms offer media captions as a +first-class feature, allowing users to choose the attachment and write text, +then send both together in one message. + +Matrix currently does not enable this on the protocol level: at best, clients +can emulate the behavior by sending two messages quickly; at worst, the user +has to do that manually. Sending separate messages means it's possible for +the second message to be delayed or lost if something goes wrong. + +## Proposal + +This proposal allows the `filename` field from [`m.file`], and the `format` and +`formatted_body` fields from [`m.text`] for all media msgtypes (`m.image`, +`m.audio`, `m.video`, `m.file`). This proposal does not affect the `m.location` +msgtype, nor the separate `m.sticker` event type: stickers already use `body` +as a description, and locations don't have file names. + +If the `filename` field is present in a media message, clients should treat +`body` as a caption instead of a file name. If the `format`/`formatted_body` +fields are present in addition to `filename` and `body`, then they should take +priority as the caption text. Formatted text in media captions is rendered the +same way as formatted text in `m.text` messages. + +The current spec is somewhat ambiguous as to how `body` should be handled and +the definition varies across different message types. The current spec for +[`m.image`] describes `body` as + +> A textual representation of the image. This could be the alt text of the +> image, the filename of the image, or some kind of content description for +> accessibility e.g. ‘image attachment’. + +while [`m.audio`] describes it as + +> A description of the audio e.g. ‘Bee Gees - Stayin’ Alive’, or some kind of +> content description for accessibility e.g. ‘audio attachment’. + +In practice, clients (or at least Element) use it as the file name. As a part +of adding captions, the `body` field for all media message types is explicitly +defined to be used as the file name when the `filename` field is not present. + +For `m.file` messages, the [current (v1.9) spec][`m.file`] confusingly defines +`filename` as "The original filename of the uploaded file" and simultaneously +recommends that `body` is "the filename of the original upload", effectively +saying both fields should have the file name. In order to avoid (old) messages +with both fields being misinterpreted as having captions, the `body` field +should not be used as a caption when it's equal to `filename`. + +[`m.file`]: https://spec.matrix.org/v1.9/client-server-api/#mfile +[`m.text`]: https://spec.matrix.org/v1.9/client-server-api/#mtext +[`m.image`]: https://spec.matrix.org/v1.9/client-server-api/#mimage +[`m.audio`]: https://spec.matrix.org/v1.9/client-server-api/#maudio + +### Examples +
+Image with caption + +```json +{ + "msgtype": "m.image", + "url": "mxc://maunium.net/HaIrXlnKfEEHvMNKzuExiYlv", + "filename": "cat.jpeg", + "body": "this is a cat picture :3", + "info": { + "w": 479, + "h": 640, + "mimetype": "image/jpeg", + "size": 27253 + }, + "m.mentions": {} +} +``` + +
+
+File with formatted caption + +```json +{ + "msgtype": "m.file", + "url": "mxc://maunium.net/TizWsLhHfDCETKRXdDwHoAGn", + "filename": "hello.txt", + "body": "this caption is longer than the file itself 🤔", + "format": "org.matrix.custom.html", + "formatted_body": "this caption is longer than the file itself 🤔", + "info": { + "mimetype": "text/plain", + "size": 14 + }, + "m.mentions": {} +} +``` + +
+ +### Summary +* `filename` is defined for all media msgtypes. +* `body` is defined to be a caption when `filename` is present and not equal to `body`. + * `format` and `formatted_body` are allowed as well for formatted captions. +* `body` is defined to be the file name when `filename` is not present. + +## Potential issues + +In clients that don't show the file name anywhere, the caption would not be +visible at all. However, extensible events would run into the same issue. +Clients having captions implemented beforehand may even help eventually +implementing extensible events. + +Old clients may default to using the caption as the file name when the user +wants to download a file, which will be somewhat weird UX. + +## Alternatives + +### [MSC2529](https://github.com/matrix-org/matrix-spec-proposals/pull/2529) + +MSC2529 would allow existing clients to render captions without any changes, +but the use of relations makes implementation more difficult, especially for +bridges. It would require either waiting a predefined amount of time for the +caption to come through, or editing the message on the target platform (if +edits are supported). + +The format proposed by MSC2529 would also make it technically possible to use +other message types as captions without changing the format of the events, +which is not possible with this proposal. + +### Extensible events + +Like MSC2529, this would be obsoleted by [extensible events](https://github.com/matrix-org/matrix-spec-proposals/pull/3552). +However, fully switching to extensible events requires significantly more +implementation work, and it may take years for the necessary time to be +allocated for that. + +## Security considerations + +This proposal doesn't involve any security-sensitive components. + +## Unstable prefix + +The fields being added already exist in other msgtypes, so unstable prefixes +don't seem necessary. Additionally, using `body` as a caption could already be +considered spec-compliant due to the ambiguous definition of the field, and +only adding unstable prefixes for the other fields would be silly. diff --git a/proposals/2677-reactions.md b/proposals/2677-reactions.md index 41d85ebd..18e789c0 100644 --- a/proposals/2677-reactions.md +++ b/proposals/2677-reactions.md @@ -51,7 +51,7 @@ property should contains a `key` that indicates the annotation being applied. For example, when reacting with emojis, the `key` contains the emoji being used. -An event annotating another with the thumbs-up emoji would therefore have the following `m.relates_to` propperty: +An event annotating another with the thumbs-up emoji would therefore have the following `m.relates_to` property: ```json "m.relates_to": { diff --git a/proposals/2701-media-content-type.md b/proposals/2701-media-content-type.md new file mode 100644 index 00000000..467a800f --- /dev/null +++ b/proposals/2701-media-content-type.md @@ -0,0 +1,59 @@ +# MSC2701: Media and the `Content-Type` relationship + +The specification currently does not outline in great detail how `Content-Type` should be handled +with respect to media, particularly around uploads. The [`POST /upload`](https://spec.matrix.org/v1.9/client-server-api/#post_matrixmediav3upload) +and [`PUT /upload/:serverName/:mediaId`](https://spec.matrix.org/v1.9/client-server-api/#put_matrixmediav3uploadservernamemediaid) +endpoints mention that `Content-Type` is a header that can be set, but does not list it as required, +for example. Similarly, the `Content-Type` seems to entirely disappear when talking about +[downloads](https://spec.matrix.org/v1.9/client-server-api/#get_matrixmediav3downloadservernamemediaid). + +This proposal clarifies how the `Content-Type` header is used on upload and download, in line with +current best practices among server implementations. + +## Proposal + +For `POST` and `PUT` `/upload`, the `Content-Type` header becomes explicitly *optional*, defaulting +to `application/octet-stream`. [Synapse](https://github.com/element-hq/synapse/blob/742bae3761b7b2c638975f853ab6161527629240/synapse/rest/media/upload_resource.py#L91) +and [MMR](https://github.com/turt2live/matrix-media-repo/blob/fdb434dfd8b7ef7d93401d7b86791610fed72cb6/api/r0/upload_sync.go#L33) +both implement this behaviour. Clients SHOULD always supply a `Content-Type` header though, as this +may change in future iterations of the endpoints. + +**Note**: Synapse's behaviour was changed in October 2021 with [PR #11200](https://github.com/matrix-org/synapse/pull/11200). +Previously, Synapse required the header. + +For `GET /download`, the server MUST return a `Content-Type` which is either exactly the same as the +original upload, or reasonably close. The bounds of "reasonable" are: + +* Adding a `charset` to `text/*` content types. +* Detecting HTML and using `text/html` instead of `text/plain`. +* Using `application/octet-stream` when the server determines the content type is obviously wrong. For + example, an encrypted file being claimed as `image/png`. +* Returning `application/octet-stream` when the media has an unknown/unprovided `Content-Type`. For + example, being uploaded before the server tracked content types or when the remote server is non-compliantly + omitting the header entirely. + +Actions not in the spirit of the above are not considered "reasonable". Existing server implementations +are encouraged to downgrade their behaviour to be in line with this guidance. [Synapse](https://github.com/element-hq/synapse/blob/742bae3761b7b2c638975f853ab6161527629240/synapse/media/_base.py#L154) +already does very minimal post-processing while [MMR](https://github.com/turt2live/matrix-media-repo/blob/fdb434dfd8b7ef7d93401d7b86791610fed72cb6/api/_routers/98-use-rcontext.go#L110-L139) +actively ignores the uploaded `Content-Type` (the incorrect thing to do under this MSC). + +## Potential issues + +Some media may have already been uploaded to a server without a content type. Such media items are +returned as `application/octet-stream` under this proposal. + +## Alternatives + +No significant alternatives. + +## Security considerations + +No relevant security considerations, though server authors are encouraged to consider the impact of +[MSC2702](https://github.com/matrix-org/matrix-spec-proposals/pull/2702) in their threat model. + +## Unstable prefix + +This MSC is backwards compatible with existing specification and requires no particular unstable +prefix. Servers are already able to implement this proposal's behaviour legally. + +Additionally, cited in the proposal are examples of the behaviour being used in production today. diff --git a/proposals/2702-media-content-disposition.md b/proposals/2702-media-content-disposition.md new file mode 100644 index 00000000..a96eb6c9 --- /dev/null +++ b/proposals/2702-media-content-disposition.md @@ -0,0 +1,128 @@ +# MSC2702: `Content-Disposition` usage in the media repo + +The current specification does not clarify how to treat `Content-Disposition` on responses to +[`/download`](https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3downloadservernamemediaid) +and [`/thumbnail`](https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3thumbnailservernamemediaid) +requests. This has led to clients (and most notably, IRC bridges) relying on implementation-specific +behaviour for how their uploads will be represented. This reliance has caused issues in the past where +a security decision was made to use an `attachment` disposition on the download endpoint, however this +caused an IRC bridge's use of the media repo to break when trying to send large messages: users on the +IRC side were forced to download and open a text file instead of seeing it in their browser. Similar +problems have occurred in the past with respect to clients using the download endpoint as an "open in +browser" button. + +This proposal clarifies the exact behaviour and introduces a set of suggestions for servers to follow +with respect to `Content-Disposition`. + +## Context + +Headers: +* [`Content-Disposition`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition) + +Endpoints affected: + +* [`/download`](https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3downloadservernamemediaid) +* [`/thumbnail`](https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3thumbnailservernamemediaid) + +Upload endpoints: +* [`POST /upload`](https://spec.matrix.org/v1.8/client-server-api/#post_matrixmediav3upload) +* [`PUT /upload`](https://spec.matrix.org/v1.8/client-server-api/#put_matrixmediav3uploadservernamemediaid) + +## Proposal + +For `/download`: + +* `Content-Disposition` MUST be returned, and MUST be one of `inline` or `attachment`. + + * `inline` *should* be used when one of the "safe" content types listed below is being served. + +* When uploads are made with a `filename`, the `Content-Disposition` header MUST contain the same + `filename`. Otherwise, `filename` is excluded from the header. + + * If the media being downloaded is remote, the remote server's `filename` in the `Content-Disposition` + header is used as the `filename` instead. When the header is not supplied, or does not supply a + `filename`, the local download response does not include a `filename` (though does still contain + a generated `Content-Disposition`). + +For `/thumbnail`: + +* `Content-Disposition` MUST be returned, and MUST be `inline` (see below). +* `Content-Disposition` SHOULD include a server-generated `filename`. For example, `thumbnail.png`. + +Note that in both endpoints `Content-Disposition` becomes required, though the legal set of parameters is +intentionally different. Specifically, because `/thumbnail` returns server-generated content, that content +is safe to serve inline relative to a given user upload and therefore can be inline. It is however atypical +for a client to link to `/thumbnail` directly, but in the event they do we provide a safe default. + +The following content types are considered "safe" for `inline` usage. For `/download`, servers SHOULD +use `attachment` if the returned `Content-Type` is not on the list. For `/thumbnail`, servers SHOULD +only generate thumbnails with a `Content-Type` listed below. + +* `text/css` +* `text/plain` +* `text/csv` +* `application/json` +* `application/ld+json` +* `image/jpeg` +* `image/gif` +* `image/png` +* `image/apng` +* `image/webp` +* `image/avif` +* `video/mp4` +* `video/webm` +* `video/ogg` +* `video/quicktime` +* `audio/mp4` +* `audio/webm` +* `audio/aac` +* `audio/mpeg` +* `audio/ogg` +* `audio/wave` +* `audio/wav` +* `audio/x-wav` +* `audio/x-pn-wav` +* `audio/flac` +* `audio/x-flac` + +`Content-Type` additionally becomes a required header on responses to both `/download` and `/thumbnail`, as `Content-Disposition` +without `Content-Type` is effectively useless in HTTP. The `Content-Type` header is the `Content-Type` supplied by +the client during `/upload`. If no `Content-Type` was supplied during upload, `application/octet-stream` is used. + +Clients SHOULD NOT rely on servers returning `inline` rather than `attachment` on `/download`. Server +implementations might decide out of an abundance of caution that all downloads are responded to with +`attachment`, regardless of content type - clients should not be surprised by this behaviour. + +## Potential issues + +This proposal does not require the usage of `inline` on `/download`, making it harder for IRC and similar +bridges to rely on "pastebin" behaviour. For example, when a large message is posted on the Matrix side of +the bridge, the IRC bridge might upload it as a text file due to limits on the IRC side. Ideally, that text +file would be rendered inline by the server. Bridges are encouraged to use "proxy APIs" to serve the text +file instead, where they can better control the user experience. + +## Alternatives + +No major alternatives identified. + +## Security considerations + +This MSC fixes a possible Cross-Site Scripting issue in environments with insecure configurations. +For example, if a Matrix web client is also hosted on the same domain as the media download URL. This +is mitigated by only suggesting `inline` as a disposition on content types which are not likely to +execute code within the browser upon being viewed. A browser may still have further bugs which reveal +information, though those issues quickly become impractical for the Matrix specification to mitigate. + +No new security issues are identified, and careful consideration was put into `inline` to ensure an +extremely limited set of possible media types. + +The allowable content types for `inline` were inspired by [the react-sdk](https://github.com/matrix-org/matrix-react-sdk/blob/a70fcfd0bcf7f8c85986da18001ea11597989a7c/src/utils/blobs.ts#L51), +and extended to include what is present in [a related Synapse PR](https://github.com/matrix-org/synapse/pull/15988). +The react-sdk list of types has been known to be safe for several years now, and the added types in the Synapse +PR are not subject to XSS or similar attacks. Note that `image/svg`, `text/javascript`, and `text/html` are +explicitly not allowed. + +## Unstable prefix + +This MSC has no particular unstable prefix requirements. Servers are already able to return arbitrary +`Content-Disposition` headers on the affected endpoints. diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md new file mode 100644 index 00000000..71108af2 --- /dev/null +++ b/proposals/2705-thumbnail-requirements.md @@ -0,0 +1,52 @@ +# MSC2705: Animated thumbnails + +Users may already upload animated media to the media repository, such as gifs, webp images, and videos. +When this media is used in an event or avatar, many clients are stuck with a static thumbnail until +the user clicks on it to get the full, unedited, file. Some clients however would prefer to show an +animated thumbnail in certain conditions, like when the user is hovering over the message or avatar. + +This proposal introduces a new query parameter to the [`GET /_matrix/media/v3/thumbnail`](https://spec.matrix.org/v1.9/client-server-api/#get_matrixmediav3thumbnailservernamemediaid) +endpoint, allowing clients to specifically request an animated thumbnail. + +## Proposal + +A new query parameter, `animated`, is added to the `/thumbnail` endpoint. It has the following behaviour: + +* When `true`: the server SHOULD return an animated thumbnail if possible/supported. +* When `false`: the server MUST NOT return an animated thumbnail. +* When not provided: the server SHOULD NOT return an animated thumbnail. + +The default case is a relaxed version of the `false` behaviour to allow server owners to customize the +default behaviour when their users' clients do not support requesting animated thumbnails. + +Clients SHOULD respect a user's preference to [reduce motion](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion) +and request non-animated thumbnails in these cases. + +The content types which are able to be animated is left as an implementation detail. The following +SHOULD be supported at a minimum, however: + +* `image/gif` +* `image/png` or `image/apng` ("APNG" format) +* `image/webp` + +The returned content type for an animated thumbnail is additionally left as an implementation detail, +though servers SHOULD use `image/webp` whenever possible. + +When media cannot be animated, such as a PDF or JPEG, the server should return a thumbnail as though +`animated` was `false`. + +## Alternatives + +No significant alternatives. + +## Security considerations + +Server load could increase when the server tries to thumbnail a large file. Servers are expected to +mitigate this on their own by providing an option to disable the feature or limiting how/when +they will animate the media. + +## Unstable prefix + +While this proposal is not considered stable, `animated` is specified as `org.matrix.msc2705.animated` +on requests. No unstable endpoints are required due to backwards compatibility being built-in to the +proposal. diff --git a/proposals/2870-protect-acls-from-redaction.md b/proposals/2870-protect-acls-from-redaction.md new file mode 100644 index 00000000..48abc6f9 --- /dev/null +++ b/proposals/2870-protect-acls-from-redaction.md @@ -0,0 +1,66 @@ +# MSC2870: Protect server ACLs from redaction + +[Server ACLs](https://matrix.org/docs/spec/client_server/r0.6.1#server-access-control-lists-acls-for-rooms) +are a way to prevent malicious/untrusted homeservers from participating in a given room. These ACLs +are represented by a state event which is easily capable of demolishing all useful functionality of +a federated room. Typically this sort of design flaw would be worked out during the MSC process, and +in this particular case it was acknowledged as a potential source of accidental mistake, however the +impact of making a mistake appears to be larger than anticipated. + +Specifically, the ACLs do not default to allowing all servers when they are set but missing an `allow` +field. When an `allow` rule is missing, it means that all servers can no longer participate in the +room. The natural reaction to such a disaster is to try sending a new ACL event, however all the +receiving servers rightly refuse the event because the specification says so and it's not yet +possible to tell if a trusted server is publishing the new ACLs (due to the social nature of the +problem space rather than the technical side). Redacting a server ACL event causes the event content +to become `{}`, which means no `allow` rule is present. + +## Proposal + +In a future room version, the `allow`, `deny`, and `allow_ip_literals` fields of the `m.room.server_acl` +state event are protected from redaction. Typically this measure is only taken when a field is critical +to the operation of the protocol, such as in the case of protecting power levels from redactions. ACLs +are not as critical to the protocol as most of the other protected event fields, however the consequences +of accidentally redacting a server ACL event are disproportionately large. + +## Potential issues + +None foreseen - this MSC should remedy a problem encountered in the wild. + +## Alternatives + +We could instead dictate that a lack of `allow` rule implicitly means `allow: ["*"]`, however this is +a behavioural change which is easily missed between room versions. We could also define that `{}` means +applying the same mechanics of ACLs when the state event isn't set, however again this is subject to +being missed as a behavioural change. Behavioural changes are hard to represent in the specification as +room versions are not meant to contain information about how a room might react in the eyes of a room +administrator or client implementation, where possible. They are more intended to change server-side +algorithms, like the redaction algorithm, to change the functionality under the hood without impacting +the room administrator's understanding of their room's function. + +Another possible approach is to have servers prevent sending faulty ACL events by preventing their +local clients from doing so, such as by rejecting redaction requests. This doesn't solve the issue +over federation, but can reduce the likelihood of such events making it to the federation layer. This +is what Synapse currently does to help mitigate the issue. This is only effective if the server (or +client, locally) implements it - it's common for both clients and servers to forget to add these checks +during development, leading to occasional room breakage. This MSC instead tries to resolve the issue +more completely for future room versions, pending replacement of ACLs entirely. + +Meanwhile, [MSC4124](https://github.com/matrix-org/matrix-spec-proposals/pull/4124) exists as a possible +permanent replacement for ACLs. + +## Security considerations + +It may be desirable to redact server ACLs due to abusive server names needing to be banned. Clients +are encouraged to *not* display the differences to the ACLs without the user opting in to seeing the +changes (such as by clicking a 'show details' link). + +Of particular note is that redacted events may be provided to future users/servers, regardless of the +history visibility settings. If fields are protected from redaction, this means they will be visible +to those future joiners. This may be undesirable in some circumstances. This MSC does not attempt to +address this concern. + +## Unstable prefix + +Implementations looking to test this MSC before it lands in a released room version can use `org.matrix.msc2870` +as the room version, using [room version 11](https://spec.matrix.org/v1.9/rooms/v11/) as a base. diff --git a/proposals/3030-jump-to-date.md b/proposals/3030-jump-to-date.md index 75f3af15..189c9330 100644 --- a/proposals/3030-jump-to-date.md +++ b/proposals/3030-jump-to-date.md @@ -152,7 +152,7 @@ the room was created. ## Alternatives We chose the current `/timestamp_to_event` route because it sounded like the -easist path forward to bring it to fruition and get some real-world experience. +easiest path forward to bring it to fruition and get some real-world experience. And was on our mind during the [initial discussion](https://docs.google.com/document/d/1KCEmpnGr4J-I8EeaVQ8QJZKBDu53ViI7V62y5BzfXr0/edit#bookmark=id.qu9k9wje9pxm) because there was some prior art with a [WIP implementation](https://github.com/matrix-org/synapse/pull/9445/commits/91b1b3606c9fb9eede0a6963bc42dfb70635449f) from @erikjohnston. The alternatives haven't been thrown out for a particular diff --git a/proposals/3061-shareable-room-keys.md b/proposals/3061-shareable-room-keys.md new file mode 100644 index 00000000..39fc5b91 --- /dev/null +++ b/proposals/3061-shareable-room-keys.md @@ -0,0 +1,152 @@ +# MSC3061: Sharing room keys for past messages + +In Matrix, rooms can be configured via the `m.room.history_visibility` state +event such that historical messages can be visible to all Matrix users +(`world_readable`), all room members (`shared`), room members from the time +that they are invited to a room (`invited`), or room members from the time that +they join a room (`joined`). However, currently in encrypted rooms, rooms with +the history visibility set to `world_readable` or `shared` are effectively +set to `invited` since other members generally do not send new members the keys +to decrypt messages sent before they were invited or joined a room. + +We define a "shared-history" flag that identifies keys for messages that were +sent when the room's visibility setting was set to `world_readable` or +`shared`. This allows clients to know which keys are "safe" to share with new +members so that they can decrypt historical messages. We also give examples of +ways in which this flag can be used. + + +## Proposal + +A room key (such as a megolm session) is flagged as having been used for shared +history when it was used to encrypt a message while the room's history +visibility setting was set to `world_readable` or `shared`. + +If the client does not have an `m.room.history_visibility` state event for the +room, or its value is not understood, the client should treat it as if its +value is `joined` for the purposes of determining whether the key is used for +shared history. This is in contrast with the normal processing of +`m.room.history_visibility` which defaults to `world_readable` when there is no +`m.room.history_visibility` state event or its value is not understood. This +is done so that, in the event of a bug that causes the client to fail to obtain +the state event, the client will fail in a secure manner. + +Internally, a client may use any mechanism it wants to keep track of this flag. +When a room key is marked as having been used for shared history: + +- `m.room_key` and `m.forwarded_room_key` messages used to share this key have + a `shared_history` property set to `true` e.g. + + ```json + { + "type": "m.room_key", + "content": { + "algorithm": "m.megolm.v1.aes-sha2", + "room_id": "!room_id", + "session_id": "session_id", + "session_key": "session_key", + "shared_history": true + } + } + ``` + +- the [`SessionData` type](https://spec.matrix.org/unstable/client-server-api/#definition-sessiondata) + in key backups (that is, the plaintext object that gets encrypted into the + `session_data` field) of this key has a `shared_history` property set to + `true` in the decrypted JSON structure e.g. + + ```json + { + "algorithm": "m.megolm.v1.aes-sha2", + "forwarding_curve25519_key_chain": [ + "hPQNcabIABgGnx3/ACv/jmMmiQHoeFfuLB17tzWp6Hw" + ], + "sender_claimed_keys": { + "ed25519": "aj40p+aw64yPIdsxoog8jhPu9i7l7NcFRecuOQblE3Y" + }, + "sender_key": "RF3s+E7RkTQTGF2d8Deol0FkQvgII2aJDf3/Jp5mxVU", + "session_key": "AgAAAADxKHa9uFxcXzwYoNueL5Xqi69IkD4sni8Llf...", + "shared_history": true + } + ``` + + and, +- the [`SessionData` type](https://spec.matrix.org/unstable/client-server-api/#key-export-format) + used in key exports has a `shared_history` property that is set to `true` for + this key e.g. + + ```json + { + "algorithm": "m.megolm.v1.aes-sha2", + "forwarding_curve25519_key_chain": [ + "hPQNcabIABgGnx3/ACv/jmMmiQHoeFfuLB17tzWp6Hw" + ], + "sender_claimed_keys": { + "ed25519": "aj40p+aw64yPIdsxoog8jhPu9i7l7NcFRecuOQblE3Y" + }, + "sender_key": "RF3s+E7RkTQTGF2d8Deol0FkQvgII2aJDf3/Jp5mxVU", + "session_key": "AgAAAADxKHa9uFxcXzwYoNueL5Xqi69IkD4sni8Llf...", + "shared_history": true + } + ``` + +When a client obtains a key that has the `shared_history` property set to +`true`, then it flags the key internally as having been used for shared +history. Otherwise, the key should not be flagged as such. + +When the room's history visibility setting changes to `world_readable` or +`shared` from `invited` or `joined`, or changes to `invited` or `joined` from +`world_readable` or `shared`, senders that support this flag must rotate their +megolm sessions. + +Clients may use this flag to modify their behaviour with respect to sharing +keys. For example, when the user invites someone to the room, they may +preemptively share keys that have this flag with the invited user. Other +behaviours may be possible, but must be careful not to guard against malicious +homeservers. See the "Security Considerations" section. + +## Potential issues + +Room keys from clients that do not support this proposal will not be eligible +for the modified client behaviour. + +The suggested behaviour in this MSC is to only share additional keys when +inviting another user. This does not allow users who join the room but were +not invited (for example, if membership is restricted to another space, or if +the room is publicly joinable) to receive the keys. Also, if the inviter does +not have all the keys available for whatever reason, the invitee has no way of +receiving the keys. This may be solved in the future when we have a mechanism +for verifying room membership. + +## Alternatives + +Rather than having the sender flagging keys, a client can paginate through the +room's history to determine the room's history visibility settings when the +room key was used. This would not require any changes, but has performance +problems. In addition, the server could lie about the room history while the +user is paginating through the history. By having the sender flag keys, this +ensures that the key is treated in a manner consistent with the sender's view +of the room. + +Rather than using a boolean flag, we could include the history visibility +setting as-is. For example, a `history_visibility` field could be added, which +is set to the history visibility setting (e.g. `world_readable`). This +produces an equivalent effect, but it pushes the processing of the history +visibility setting to the receiver rather than the sender. For consistency, it +is better for as much of the decision-making done by the sender, rather than +the receiver. + +## Security considerations + +Clients should still ensure that keys are only shared with authorized users and +devices, as a malicious homeserver could inject fake room membership events. +One way to ensure that keys are only shared with authorized users is to only +share keys with users when the client invites them, as the client is then +certain that the user is allowed to be in the room. Another way is to have a +mechanism of verifying membership, such as the method proposed in +[MSC3917](https://github.com/matrix-org/matrix-spec-proposals/pull/3917). + +## Unstable prefix + +Until this feature lands in the spec, the property name to be used is +`org.matrix.msc3061.shared_history` rather than `shared_history`. diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md new file mode 100644 index 00000000..27f31a77 --- /dev/null +++ b/proposals/3077-multi-stream-voip.md @@ -0,0 +1,111 @@ +# MSC3077: Support for multi-stream VoIP + +This MSC proposes a method for differentiating WebRTC streams from each other. + +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) has improved VoIP +immeasurably. Yet, there is still no clear way to handle things such as +screen-sharing. + +Simple VoIP calls only ever feature one stream, though often clients will want +to send multiple - usermedia, screensharing and possibly more. In a situation +with more streams, it can be very helpful to provide the other side with +metadata about the content of the streams. + +## Proposal + +This MSC proposes adding an `sdp_stream_metadata` field to the events containing +a session description i.e.: + ++ [`m.call.invite`](https://spec.matrix.org/v1.7/client-server-api/#mcallinvite) ++ [`m.call.answer`](https://spec.matrix.org/v1.7/client-server-api/#mcallanswer) ++ [`m.call.negotiate`](https://spec.matrix.org/v1.7/client-server-api/#mcallnegotiate) + +The `sdp_stream_metadata` field is an object in which each key is one stream +`id` in the session description. The values are objects with the +following fields: + ++ `purpose` - a string indicating the purpose of the stream. For compatibility + between client the following values are defined: + + `m.usermedia` - stream that contains the webcam and/or microphone tracks + + `m.screenshare` - stream with the screen-sharing tracks + +### Example + +```JSON +{ + "type": "m.call.invite", + "room_id": "!roomId", + "content": { + "call_id": "1414213562373095", + "invitee": "@bob:matrix.org", + "party_id": "1732050807568877", + "lifetime": "60000", + "capabilities": { + "m.call.transferee": true, + }, + "offer": { + "sdp": "...", + "type": "offer", + }, + "sdp_stream_metadata": { + "271828182845": { + "purpose": "m.screenshare", + }, + "314159265358": { + "purpose": "m.usermedia", + }, + }, + "version": "1", + }, +} +``` + +### Edge cases + ++ If an incoming stream is not described in `sdp_stream_metadata` and + `sdp_stream_metadata` is present, the stream should be ignored. ++ If a stream has a `purpose` of an unknown type (i.e. not `m.usermedia` or + `m.screenshare`), it should be ignored. + +### Backwards compatibility + +During the initial invite and answer exchange clients find out if the field +`sdp_stream_metadata` is missing. If it is not present in the event sent by the +opponent, the client should ignore any new incoming streams (i.e. it should use +the first one) and it shouldn't send more than one stream (i.e. clients cannot send a video feed and a screenshare at the same time, as is the case in current clients). + +## Alternatives + +Setting the stream `id`s to custom values had been considered. Though this is +possible on some platforms, it is not in browsers. That is because the `id` +property of `MediaStream` is _read-only_ as the [MDN Documentation +states](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/id). +Similar is true for SDP attributes. + +This proposal is also more practical for cases where more complex metadata is +needed. For conferencing, a `user_id` field could be added to +the objects in `sdp_stream_metadata`; for differentiating between the front and rear camera of a +phone, a `camera_type` field could be added. + +Previously, it has been thought that the `purpose` field has to be unique (or +another unique field has to be added), though this could only ever be important +if we wanted to replace a stream with another one in-place. It was deemed as a +rather uncommon thing for which there doesn't seem to be any use-case, so +uniqueness is not required. + +## Unstable prefix + +During development, the following fields should be used: + +|Release |Development | +|----------------------------|-----------------------------------------------| +|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | +|`m.call.sdp_stream_metadata`|`org.matrix.msc3077.call.sdp_stream_metadata` | + +## Potential issues + +None that I can think of. + +## Security considerations + +None that I can think of. diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md new file mode 100644 index 00000000..6fd114d5 --- /dev/null +++ b/proposals/3291-muting.md @@ -0,0 +1,130 @@ +# MSC3291: Muting in VoIP calls + +During VoIP calls, it is common for a user to mute their microphone/camera. +Ideally, the other side should be able to see that the opponent's camera is +muted, so that it could reflect this in the UI (e.g. show the user's avatar +instead of their camera feed). We would also want the changes in the mutes state +to be quick. + +Using pure WebRTC there are two ways to do muting and both have their issues: + ++ Disabling the corresponding track ++ Setting the corresponding track as `recvonly`/`inactive` + +The Alternatives section describes the issues with using these alone. + +## Proposal + +This MSC proposes extending the `sdp_stream_metadata` object (see +[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow +indicating the mute state to the other side using the following fields: + ++ `audio_muted` - a boolean indicating the current audio mute state ++ `video_muted` - a boolean indicating the current video mute state + +This MSC also adds a new call event `m.call.sdp_stream_metadata_changed`, which +has the common VoIP fields as specified in +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`, +`call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same +thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and +`m.call.answer`. The client sends this event when the `sdp_stream_metadata` has +changed but no negotiation is required (e.g. the user mutes their +camera/microphone). + +All tracks should be assumed unmuted unless specified otherwise. + +Clients are recommended to not mute the audio of WebRTC tracks locally when a +incoming stream has the `audio_muted` field set to `true`. This is because when the +other user unmutes themselves, there may be a slight delay between their client +sending audio and the `m.call.sdp_stream_metadata_changed` event arriving. If +`enabled` is set to `false`, then any audio sent in between those two events +will not be heard. The other user will still stop transmitting audio once they +mute on their side, so no audio is sent without the user's knowledge. + +The same suggestion does not apply to `video_muted` - there clients _should_ +mute video locally, so that the receiving side doesn't see black video. + +### Example + +```JSON +{ + "type": "m.call.sdp_stream_metadata_changed", + "room_id": "!roomId", + "content": { + "version": "1", + "call_id": "1414213562373095", + "party_id": "1732050807568877", + "sdp_stream_metadata": { + "2311546231": { + "purpose": "m.usermedia", + "audio_muted:": true, + "video_muted": true + } + } + } +} +``` + +This event indicates that both audio and video are muted. It is suggested the +video track of stream `2311546231` should be hidden in the UI (probably replaced +by an avatar). It also suggests the UI should show an indication that the audio +track is muted but the client should not mute the audio on the receiving side. + +## Potential issues + +When the user mutes their camera, some browsers may keep sending meaningless data +which will waste bandwidth. + +## Alternatives + +### Only disabling the corresponding track + +This is the solution that some clients (e.g. Element Android) use at the moment. +While this is almost instantaneous, it doesn't allow the other side to know the +opponent's mute state. This leads to the opponent showing a black screen for a +muted video track and not doing anything for a muted audio track which is bad +for UX. + +### Setting the corresponding track as `recvonly`/`inactive` + +While this would be beneficial for low bandwidth connections, it takes time. The +delay might be acceptable for video but isn't for audio (with which you would +assume an instantaneous mute state change). This is also problematic since there +could be a confusion with holding (as defined in +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). + +### Using a separate event for muting + +While this might feel clearer initially, it doesn't have much real benefit. The +mute state is in fact a meta information about the stream and using +`sdp_stream_metadata` is also more flexible for cases where the user joins a +call already muted. It is also more flexible in general and would be useful if +we ever decided to do what is described in the next section. + +### A combination of disabling tracks, `sdp_stream_metadata` and SDP + +An option would be using the current method in combination with setting the +corresponding track as `recvonly`/`inactive`. Along with this clients would need +to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding +(as defined in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). +While this solution might be the most flexible solution as it would allow +clients to choose between bandwidth and a mute state change delay for each +track, it would be harder to implement and feels generally disjointed. + +## Security considerations + +None that I can think of. + +## Dependencies + ++ [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077) + +## Unstable prefix + +|Release |Development | +|------------------------------------|---------------------------------------------| +|`m.call.sdp_stream_metadata_changed`|`org.matrix.call.sdp_stream_metadata_changed`| +|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | + +We use an unstable prefix for `sdp_stream_metadata` to match +[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077). diff --git a/proposals/3381-polls.md b/proposals/3381-polls.md new file mode 100644 index 00000000..413b179f --- /dev/null +++ b/proposals/3381-polls.md @@ -0,0 +1,467 @@ +# MSC3381: Chat Polls + +Polls are additive functionality in a room, allowing someone to pose a question and others to answer +until the poll is closed. In chat, these are typically used for quick questionnaires such as what to +have for lunch or when the next office party should be, not elections or anything needing truly +secret ballot. + +[MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) does introduce a different way of doing +polls (originally related to inline widgets, but diverged into `m.room.message`-based design). That +MSC's approach is discussed at length in the alternatives section for why it is inferior. + +## Proposal + +Polls are intended to be handled completely client-side and encrypted when possible in a given room. +They are started by sending an event, responded to using events, and closed using more events - all +without involving the server (beyond being the natural vessel for sending the events). Other MSCs +related to polls might require changes from servers, however this MSC is intentionally scoped so that +it does not need server-side involvement. + +The events in this MSC make use of the following functionality: + +* [MSC1767](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/1767-extensible-events.md) (extensible events & `m.text`) +* [Event relationships](https://spec.matrix.org/v1.6/client-server-api/#forming-relationships-between-events) +* [Reference relations](https://spec.matrix.org/v1.6/client-server-api/#reference-relations) + +To start a poll, a user sends an `m.poll.start` event into the room. An example being: + +```json5 +{ + "type": "m.poll.start", + "sender": "@alice:example.org", + "content": { + "m.text": [ + // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is + // not specified, however something like the following is likely best. + { + "mimetype": "text/plain", + "body": "What should we order for the party?\n1. Pizza 🍕\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings 🔥" + } + ], + "m.poll": { + "kind": "m.disclosed", + "max_selections": 1, + "question": { + "m.text": [{"body": "What should we order for the party?"}] + }, + "answers": [ + {"m.id": "pizza", "m.text": [{"body": "Pizza 🍕"}]}, + {"m.id": "poutine", "m.text": [{"body": "Poutine 🍟"}]}, + {"m.id": "italian", "m.text": [{"body": "Italian 🍝"}]}, + {"m.id": "wings", "m.text": [{"body": "Wings 🔥"}]}, + ] + } + } +} +``` + +With consideration for extensible events, a new `m.poll` content block is defined: + +* `kind` - An optional namespaced string to represent a poll's general approach. Currently specified + values being `m.disclosed` and `m.undisclosed`. Clients which don't understand the `kind` should + assume `m.undisclosed` for maximum compatibility. The definitions for these values are specified + later in this proposal. +* `max_selections` - An optional integer to represent how many answers the user is allowed to select + from the poll. Must be greater than or equal to `1`, and defaults to `1`. +* `question` - A required object to represent the question being posed by the poll. Takes an `m.text` + content block within. More blocks might be added in the future. Clients should treat this similar + to how they would an `m.message` event. +* `answers` - Array of options users can select. Each entry is an object with an `m.text` content + block, similar to `question`, and an opaque string field `m.id` for use in response events. More + blocks might be added in the future. Clients should treat each entry similar to how they would an + `m.message` event. The array is truncated to 20 maximum options. + + Note that arrays are inherently ordered. Clients *should* render options in the order presented in + the array - a future MSC may add a flag to permit rendering in a different or random order. + +Together with content blocks from other proposals, an `m.poll.start` is described as: + +* **Required** - An `m.text` block to act as a fallback for clients which can't process polls. +* **Required** - An `m.poll` block to describe the poll itself. Clients use this to show the poll. + +The above describes the minimum requirements for sending an `m.poll.start` event. Senders can add additional +blocks, however as per the extensible events system, receivers which understand poll events should not +honour them. + +If a client does not support rendering polls inline, the client would instead typically represent +the event as a plain text message. This would allow users of such clients to participate in the poll, +even if they can not vote properly on it (ie: by using text messages or reactions). + +To respond or vote in a poll, a user sends an `m.poll.response` event into the room. An example being: + +```json5 +{ + "type": "m.poll.response", + "sender": "@bob:example.org", + "content": { + // Reference relationship formed per spec + // https://spec.matrix.org/v1.6/client-server-api/#reference-relations + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$poll_start_event_id" + }, + "m.selections": ["poutine"] + } +} +``` + +With consideration for extensible events, a new `m.selections` content block is defined: + +* An array of string identifiers to denote a user's selection. Can be empty to denote "no selection". + Identifiers are determined by the surrounding event type context, if available. + +Together with content blocks from other proposals, an `m.poll.response` is described as: + +* **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. +* **Required** - An `m.selections` block to list the user's preferred selections in the poll. Clients + must truncate this array to `max_selections` during processing. Each entry is the `m.id` of a poll + answer option from the poll start event. If *any* of the supplied answers is unknown, the sender's + vote is spoiled (as if they didn't make a selection). If an entry is repeated after truncation, only + one of those entries counts as the sender's vote (each sender gets 1 vote). + +The above describes the minimum requirements for sending an `m.poll.response` event. Senders can add +additional blocks, however as per the extensible events system, receivers which understand poll events +should not honour them. + +There is deliberately no textual or renderable fallback on poll responses: the intention is that clients +which don't understand how to process these events will hide/ignore them. This is to mirror what a +client which *does* support polls would do: they wouldn't render each vote as a new message, but would +aggregate them into a single result at the end of the poll. By not having a text fallback, the vote +is only revealed when the poll ends, which does have a text fallback. + +Only a user's most recent vote (by `origin_server_ts`) is accepted, even if that event is invalid. +Votes with timestamps after the poll has closed are ignored, as if they never happened. Note +that redaction currently removes the `m.relates_to` information from the event, causing the vote to be +detached from the poll. In this scenario, the user's vote is *reverted* to its previous state rather +than explicitly spoiled. To "unvote" or otherwise override the previous vote state, clients should send +a response with an empty `m.selections` array. + +To close a poll, a user sends an `m.poll.end` event into the room. An example being: + +```json5 +{ + "type": "m.poll.end", + "sender": "@alice:example.org", + "content": { + // Reference relationship formed per spec + // https://spec.matrix.org/v1.6/client-server-api/#reference-relations + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$poll_start_event_id" + }, + "m.text": [{ + // Simple text is used as a fallback for text-only clients which don't understand polls. Specific formatting is + // not specified, however something like the following is likely best. + "body": "The poll has closed. Top answer: Poutine 🍟" + }], + "m.poll.results": { // optional + "pizza": 5, + "poutine": 8, + "italian": 7, + "wings": 6 + } + } +} +``` + +With consideration for extensible events, a new `m.poll.results` content block is defined: + +* A dictionary object keyed by answer ID (`m.id` from the poll start event) and value being the integer + number of votes for that option as seen by the sender's client. Note that these values might not be + accurate, however other clients can easily validate the counts by retrieving all relations from the + server. + * User IDs which voted for each option are deliberately not included for brevity: clients requiring + more information about the poll are required to gather the relations themselves. + +Together with content blocks from other proposals, an `m.poll.end` is described as: + +* **Required** - An `m.relates_to` block to form a reference relationship to the poll start event. +* **Required** - An `m.text` block to act as a fallback for clients which can't process polls. +* **Optional** - An `m.poll.results` block to show the sender's perspective of the vote results. This + should not be used as a trusted block, but rather as a placeholder while the client's local results + are tabulated. + +The above describes the minimum requirements for sending an `m.poll.end` event. Senders can add additional +blocks, however as per the extensible events system, receivers which understand poll events should not +honour them. + +If a client does not support rendering polls (generally speaking), the client would instead typically +represent the poll start event as text (per above), and thus would likely do the same for the closure +event, keeping users in the loop with what is going on. + +If a `m.poll.end` event is received from someone other than the poll creator or user with permission to +redact other's messages in the room, the event must be ignored by clients due to being invalid. The +redaction power level is chosen to support moderation: while moderators can just remove the poll from the +timeline entirely, they may also wish to simply close it to keep context visible. + +**Rationale**: Although clock drift is possible, as is clock manipulation, it is not anticipated that +polls will be closed while they are still receiving high traffic. There are some cases where clients might +apply local timers to auto-close polls, though these are typically used in extremely high traffic cases +such as Twitch-style audience polls - rejecting even 100 responses is unlikely to significantly affect +the results. Further, if a server were to manipulate its clock so that poll responses are sent after the +poll was closed, but timestamped for when it was open, the server is violating a social contract and likely +will be facing a ban from the room. This MSC does not propose a mitigation strategy beyond telling people +not to ruin the fun. Also, don't use polls for things that are important. + +The `m.poll.end`'s `origin_server_ts` determines when the poll closes exactly: if no valid end event +is received, the poll is still open. If the poll is closed, only votes sent on or before that timestamp +are considered, even if those votes are from before the start event. This is to handle clock drift over +federation as gracefully as possible. + +Repeated end events are ignored - only the first (valid) closure event by `origin_server_ts` is counted. +Clients should disable voting interactions with polls once they are closed. + +### Poll kinds + +This proposal defines an `m.poll` content block with a `kind` field accepting namespaced strings, with +`m.disclosed` and `m.undisclosed` being mentioned (`m.undisclosed` being the default), however it does +not describe what these values represent. + +In short, `m.disclosed` means the votes for poll are shown to users while the poll is still open. An +`m.undisclosed` poll would only show results when the poll is closed. + +**Note**: because poll responses are sent into the room, non-compliant clients or curious users could +tally up results regardless of the poll being explicitly disclosed or not. This proposal acknowledges +the issue, but does not fix it. + +Custom poll kinds are possible using the [standardized namespace grammar](https://spec.matrix.org/v1.4/appendices/#common-namespaced-identifier-grammar), +and clients which do not recognize the kind are to assume `m.undisclosed` for maximum compatibility +with other poll kinds. + +#### Disclosed versus undisclosed polls + +Disclosed polls are most similar to what is seen on Twitch and often Twitter: members of the room are able +to see the results and vote accordingly. Clients are welcome to hide the poll results until after the user +has voted to avoid biasing the user. + +Undisclosed polls do track who voted for what, though don't reveal the results until the poll has been +closed, even after a user has voted themselves. This is enforced visually and not by the protocol given +the votes are sent to the room for local tallying - this is considered more of a social trust issue than +a technical one. This MSC expects that rooms (and clients) won't spoil the results of an undisclosed poll +before it is closed. + +In either case, once the poll ends the results are shown regardless of kind. Clients might wish to avoid +disclosing who voted for what in an undisclosed poll, though this MSC leaves that at just a suggestion. + +### Client implementation notes + +Clients can rely on the [`/relations`](https://spec.matrix.org/v1.4/client-server-api/#get_matrixclientv1roomsroomidrelationseventidreltype) +API to find votes which might have been received during limited ("gappy") syncs, or whenever they become +descynchronized and need to recalculate events. Ranged approaches, such as [MSC3523](https://github.com/matrix-org/matrix-spec-proposals/pull/3523), +are not suitable for this particular case because the gap between syncs might contain events which are not +revealed by the range. For example, if a remote server took an extra hour to send events and the receiving +client had a gappy sync over a span of 15 minutes: the client might not know that it needs to go back potentially +hours to see the missing event. + +This MSC does not describe an aggregation approach for poll events, hence the need for the client to retrieve +all referenced events rather than simply relying on bundles. + +## Potential issues + +As mentioned, poll responses are sent to the room regardless of the kind of poll. For open polls this +isn't a huge deal, but it can be considered an issue with undisclosed polls. This MSC strongly considers +the problem a social one: users who are looking to "cheat" at the results are unlikely to engage with the +poll in a productive way in the first place. And, of course, polls should never be used for something +important like electing a new leader for a country. + +Poll responses are also de-anonymized by nature of having the sender attached to a response. Clients +are strongly encouraged to demonstrate anonymization by not showing who voted for what, but should consider +warning the user that their vote is not anonymous. For example, saying "22 total responses, including +from TravisR, Matthew, and Alice" before the user casts their own vote. + +Limiting polls to client-side enforcement could be problematic if the MSC was interested in reliable +or provable votes, however as a chat feature this should reasonably be able to achieve user expectations. +Bolt-on support for signing, verification, validity, etc can be accomplished as well in the future. + +Due to the reference relationship between responses and the poll start event, it's possible that a +client facing an "unable to decrypt" error on the response won't know if it's a poll response specifically +or some other reference relationship. Clients are encouraged to tell users when there's a possibility +that not all responses are known, potentially impacting the results, such as where related events are +undecryptable. + +## Alternatives + +The primary competition to this MSC is the author's own [MSC2192](https://github.com/matrix-org/matrix-doc/pull/2192) +which describes not only polls but also inline widgets. The poll implementation in MSC2192 is primarily +based around `m.room.message` events, using `msgtype` to differentiate between the different states. As +[a thread](https://github.com/matrix-org/matrix-doc/pull/2192/files#r514497274) notes on the MSC, this +is an awful experience on clients which do not support polls properly, leaving an irritating amount of +contextless messages in the timeline. Though not directly mentioned on that thread, polls also cannot be +closed under that MSC which leads to people picking options hours or even days after the poll has "ended". +This MSC instead proposes to only supply fallback on the start and end of a poll, leading to enough context +for unsupporting clients without flooding the room with messages. + +Originally, MSC2192 was intended to propose polls as a sort of widget with access to timeline events +and other important information, however the widget infrastructure is just not ready for this sort of +thing to exist. First, we'd need to be able to send events to the widget which reference itself (for +counting votes), and allow the widget to self-close if needed. This is surprisingly difficult when widgets +can be "popped out" or have a link clicked in lieu of rendering (for desktop clients): there's no +communication channel back to the client to get the information back and forth. Some of this can be solved +with scoped access tokens for widgets, though at the time of writing those are a long ways out. In the +end, it's simply more effective to use Extensible Events and Matrix directly rather than building out +the widgets infrastructure to cope - MSC2192 is a demonstration of this, considering it ended up taking +out all the widget aspects and replacing them with fields in the content. + +Finally, MSC2192 is simply inferior due to not being able to restrict who can post a poll. Responses +and closures can also be limited arbitrarily by room admins, so clients might want to check to make +sure that the sender has a good chance of being able to close the poll they're about to create just +to avoid future issues. + +### Aggregations instead of references? + +A brief moment in this MSC's history described an approach which used aggregations (annotations/reactions) +instead of the proposed reference relationships, though this had immediate concerns of being too +complicated for practical use. + +While it is beneficial for votes to be quickly tallied by the server, the client still needs to do +post-processing on the data from the server in order to accurately represent the valid votes. The +server should not be made aware of the poll rules as it can lead to over-dependence on the server, +potentially causing excessive network requests from clients. + +As such, the reference relationship is maintained by this proposal in order to remain consistent with +how the poll close event is sent: instead of clients having to process two paginated requests they can +use a single request to get the same information, but in a more valuable form. + +For completeness, the approach of aggregations-based responses is summarized as: + +* `m.annotation` `rel_type` +* `key` is an answer ID +* Multiple response events for multi-select polls. Only the most recent duplicate is considered valid. +* Unvoting is done through redaction. + +Additional concerns are how the client needs to ensure that the answer IDs won't collide with a reaction +or other annotation, adding additional complexity in the form of magic strings. + +## Security considerations + +As mentioned a multitude of times throughout this proposal, this MSC's approach is prone to disclosure +of votes and has a couple abuse vectors which make it not suitable for important or truly secret votes. +Do not use this functionality to vote for presidents. + +Clients should apply a large amount of validation to each field when interacting with polls. Event +bodies are already declared as completely untrusted, though not all clients apply a layer of validation. +In general, this MSC aims to try and show something of use to users so they can at least figure out +what the sender intended, though clients are also welcome to just hide invalid events/responses (with +the exception of spoiled votes: those are treated as "unvoting" or choosing nothing). Clients are +encouraged to try and fall back to something sensible, even if just an error message saying the poll +is invalid. + +Users should be wary of polls changing their question after they have voted. Considering polls can be +edited, responses might no longer be relevant. For example, if a poll was opened for "do you like +cupcakes?" and you select "yes", the question may very well become "should we outlaw cupcakes?" where +your "yes" no longer applies. This MSC considers this problem more of a social issue than a technical +one, and reminds the reader that polls should not be used for anything important/serious at the moment. + +## Future considerations + +Some aspects of polls are explicitly not covered by this MSC, and are intended for another future MSC +to solve: + +* Allowing voters/room members to add their own freeform options. The edits system doesn't prevent other + members from editing messages, though clients tend to reject edits which are not made by the original + author. Altering this rule to allow it on some polls could be useful in a future MSC. + +* Verifiable or cryptographically secret polls. There is interest in a truly enforceable undisclosed poll + where even if the client wanted to it could not reveal the results before the poll is closed. Approaches + like [MSC3184](https://github.com/matrix-org/matrix-doc/pull/3184) or Public Key Infrastructure (PKI) + might be worthwhile to investigate in a future MSC. + +## Other notes + +If a client/user wishes to make a poll statically visible, they should check out +[pinned messages](https://spec.matrix.org/v1.4/client-server-api/#mroompinned_events). + +Notifications support for polls have been moved to [MSC3930](https://github.com/matrix-org/matrix-spec-proposals/pull/3930). + +Normally extensible events would only be permitted in a specific room version, however as a known-lossy chat +feature, this proposal's events are permitted in any room version. The stable event types should only be sent +in a room version which supports extensible events, however. + +## Unstable prefix + +While this MSC is not considered stable, implementations should use `org.matrix.msc3381.*` as a namespace +instead of `m.*` throughout this proposal, with the added considerations below. Note that extensible events +and content blocks might have their own prefixing requirements. + +Unstable implementations should note that a previous draft is responsible for defining the event format/schema +for the unstable prefix. The differences are rooted in a change in MSC1767 (Extensible Events) where the approach +and naming of fields changed. The differences are: + +* For `m.poll.start` / `org.matrix.msc3381.poll.start`: + * `m.text` throughout becomes a single string, represented as `org.matrix.msc1767.text` + * `m.poll` becomes `org.matrix.msc3381.poll.start`, retaining all other fields as described. Note the `m.text` + under `question` and `answers`, and the `org.matrix.msc3381.poll` prefix for `kind` enum values. `m.id` under + `answers` additionally becomes `id`, without prefix. +* For `m.poll.response` / `org.matrix.msc3381.poll.response`: + * `m.selections` becomes an `org.matrix.msc3381.poll.response` object with a single key `answers` being the + array of selections. + * `m.relates_to` is unchanged. +* For `m.poll.end` / `org.matrix.msc3381.poll.end`: + * `m.text` has the same change as `m.poll.start`. + * `m.poll.results` is removed. + * `org.matrix.msc3381.poll.end` is added as an empty object, and is required. + +Examples of unstable events are: + +```json +{ + "type": "org.matrix.msc3381.poll.start", + "content": { + "org.matrix.msc1767.text": "What should we order for the party?\n1. Pizza 🍕\n2. Poutine 🍟\n3. Italian 🍝\n4. Wings 🔥", + "org.matrix.msc3381.poll.start": { + "kind": "org.matrix.msc3381.poll.disclosed", + "max_selections": 1, + "question": { + "org.matrix.msc1767.text": "What should we order for the party?", + }, + "answers": [ + {"id": "pizza", "org.matrix.msc1767.text": "Pizza 🍕"}, + {"id": "poutine", "org.matrix.msc1767.text": "Poutine 🍟"}, + {"id": "italian", "org.matrix.msc1767.text": "Italian 🍝"}, + {"id": "wings", "org.matrix.msc1767.text": "Wings 🔥"} + ] + } + } +} +``` + +```json +{ + "type": "org.matrix.msc3381.poll.response", + "content": { + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$fw8dod4VdLCkakmKiD6XiVj7-RrFir9Jwc9RW6llJhU" + }, + "org.matrix.msc3381.poll.response": { + "answers": ["pizza"] + } + } +} +``` + +```json +{ + "type": "org.matrix.msc3381.poll.end", + "content": { + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$fw8dod4VdLCkakmKiD6XiVj7-RrFir9Jwc9RW6llJhU" + }, + "org.matrix.msc1767.text": "The poll has ended. Top answer: Italian 🍝", + "org.matrix.msc3381.poll.end": {}, + } +} +``` + +### Implementation considerations + +Client authors should note that as a feature using the Extensible Events system, +usage of the *stable* event types in regular room versions is not permitted. As +of writing (December 2023), Extensible Events does not have a *stable* room version +which supports such events, therefore meaning that clients will have to use the +*unstable* event types if they intend to support polls in existing room versions. + +When Extensible Events as a system is released in a dedicated room version, clients +will be able to use the stable event types there. The unstable event types should +not be used in that dedicated room version. diff --git a/proposals/3820-rooms-v11.md b/proposals/3820-rooms-v11.md new file mode 100644 index 00000000..84243825 --- /dev/null +++ b/proposals/3820-rooms-v11.md @@ -0,0 +1,28 @@ +# MSC3604: Room Version 11 + +A new room version, `11`, is proposed using [room version 10](https://spec.matrix.org/v1.7/rooms/v10/) as a base +and incorporating the following MSCs: + +* [MSC2174](https://github.com/matrix-org/matrix-spec-proposals/pull/2174) - Move `redacts` to sane place +* [MSC2175](https://github.com/matrix-org/matrix-spec-proposals/pull/2175) - Remove `creator` field from `m.room.create` events +* [MSC2176](https://github.com/matrix-org/matrix-spec-proposals/pull/2176) - Updates to redaction rules +* [MSC3989](https://github.com/matrix-org/matrix-spec-proposals/pull/3989) - Redact `origin` on events +* [MSC3821](https://github.com/matrix-org/matrix-spec-proposals/pull/3821) - More updates to redaction rules + +Though other MSCs are capable of being included in this version, they do not have sufficient implementation to be +considered stable enough for v11 rooms. A future room version may still include them. Such examples of ineligible MSCs +are: + +* [MSC2244](https://github.com/matrix-org/matrix-spec-proposals/pull/2244) - Mass redactions +* A number of MSCs which have not yet been accepted (they are not iterated here). + +Room version 11 upon being added to the specification shall be considered stable. No other room versions are affected +by this MSC. + +## Unstable prefix + +Implementations looking to test v11 before written into the specification should use `org.matrix.msc3820.opt2` +as the room version, treating it as unstable. + +Note: `org.matrix.msc3820.opt1` is from a prior draft of this proposal, consisting of different MSCs than the +current iteration. Implementations should not treat opt1 the same as other unstable-for-MSC3820 versions. diff --git a/proposals/3821-update-redaction-rules-again.md b/proposals/3821-update-redaction-rules-again.md new file mode 100644 index 00000000..2cc33405 --- /dev/null +++ b/proposals/3821-update-redaction-rules-again.md @@ -0,0 +1,37 @@ +# MSC3821: Update redaction rules, again + +[MSC2176](https://github.com/matrix-org/matrix-spec-proposals/pull/2176) aimed to fix inconsistencies +with the [redaction algorithm](https://spec.matrix.org/v1.6/rooms/v10/#redactions) where server-side +event auth required properties to exist that were being removed. While MSC2176 fixed a number of cases, +one was unfortunately missed. + +This MSC aims to fix that missing case, originally identified long ago by [synapse#1831](https://github.com/matrix-org/synapse/issues/1831). + +## Proposal + +*Note*: It is recommended to read [MSC2176](https://github.com/matrix-org/matrix-spec-proposals/pull/2176) +before this proposal as MSC2176 contains a lot of backing context. + +In a future room version, the following changes are made to the [redaction algorithm](https://spec.matrix.org/v1.6/rooms/v10/#redactions). +Note that this requires a new room version because changing the redaction algorithm changes how +[event IDs](https://spec.matrix.org/v1.6/rooms/v10/#event-ids) are calculated, as they are +[reference hashes](https://spec.matrix.org/v1.6/server-server-api/#calculating-the-reference-hash-for-an-event) +which redact the event during calculation. + +* [`m.room.member`](https://spec.matrix.org/v1.6/client-server-api/#mroommember) events preserve a portion + of `third_party_invite` under `content`, if present. Those properties being: + + * `signed`: the block is required for the server to validate the event, however excess adjacent properties + such as `display_name` are not important. + +Clients should note that because `display_name` is *not* preserved during redaction they might need to change +how that event is rendered/presented to users. For example, when rendering such a redacted event the client +might show it as "Bob accepted a third party invite". + +## Unstable prefix + +Implementations looking to test these changes before adopted into a stable room version should use +`org.matrix.msc3821.opt1` as the room version, using **v9** as a base and treating it as unstable. + +**History**: This MSC was originally written before room version 10 existed. Implementations wishing to use +v10 as a base instead of v9 should use `org.matrix.msc3821.opt1.v10` as the room version instead. diff --git a/proposals/3930-polls-notifications.md b/proposals/3930-polls-notifications.md new file mode 100644 index 00000000..d5dc0e9a --- /dev/null +++ b/proposals/3930-polls-notifications.md @@ -0,0 +1,142 @@ +# MSC3930: Polls push rules/notifications + +[MSC3381](https://github.com/matrix-org/matrix-spec-proposals/pull/3381) describes how chat polls can work, +though deliberately leaves out details on how push rules or notifications could work for such a feature. +This proposal aims to address that specific feature gap. + +Readers should review MSC3381 to better understand how polls are intended to operate in a room. + +Push rules are currently defined [here](https://spec.matrix.org/v1.7/client-server-api/#push-rules) in +specification. + +## Proposal + +Polls should behave similar to message events. To achieve this effect, we define the following underride +push rules which mimic their `m.room.message` partners. + +Note that [order matters](https://github.com/matrix-org/matrix-spec/issues/1406) for push rules - these +underride rules are to be inserted immediately after the `.m.rule.encrypted` underride push rule, in the +order presented by this MSC. + +```jsonc +{ + "rule_id": ".m.rule.poll_start_one_to_one", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "room_member_count", "is": "2"}, + + // Note: `.` is escaped once, but for valid JSON we need to escape the escape. + {"kind": "event_property_is", "key": "type", "value": "m\\.poll\\.start"} + ], + "actions": [ + "notify", + {"set_tweak": "sound", "value": "default"} + ] +} +``` + +```jsonc +{ + "rule_id": ".m.rule.poll_start", + "default": true, + "enabled": true, + "conditions": [ + // Note: `.` is escaped once, but for valid JSON we need to escape the escape. + {"kind": "event_property_is", "key": "type", "value": "m\\.poll\\.start"} + ], + "actions": [ + "notify" + ] +} +``` + +```jsonc +{ + "rule_id": ".m.rule.poll_end_one_to_one", + "default": true, + "enabled": true, + "conditions": [ + {"kind": "room_member_count", "is": "2"}, + + // Note: `.` is escaped once, but for valid JSON we need to escape the escape. + {"kind": "event_property_is", "key": "type", "value": "m\\.poll\\.end"} + ], + "actions": [ + "notify", + {"set_tweak": "sound", "value": "default"} + ] +} +``` + +```jsonc +{ + "rule_id": ".m.rule.poll_end", + "default": true, + "enabled": true, + "conditions": [ + // Note: `.` is escaped once, but for valid JSON we need to escape the escape. + {"kind": "event_property_is", "key": "type", "value": "m\\.poll\\.end"} + ], + "actions": [ + "notify" + ] +} +``` + +Additionally, a new override rule is defined to suppress poll responses by default, inserted immediately +after the `.m.rule.room.server_acl` override rule. + +```jsonc +{ + "rule_id": ".m.rule.poll_response", + "default": true, + "enabled": true, + "conditions": [ + // Note: `.` is escaped once, but for valid JSON we need to escape the escape. + {"kind": "event_property_is", "key": "type", "value": "m\\.poll\\.response"} + ], + "actions": [] +} +``` + +*Note*: Lack of `actions` means "don't do anything with matching events", or "don't notify". + +Typically these rules would be kept in sync with the `m.room.message` rules they are based upon, +however there is no requirement to do so. A client's approach might be to only keep them in sync +while setting the values, rather than monitoring for changes to push rules. + +Clients might find [MSC3934](https://github.com/matrix-org/matrix-spec-proposals/pull/3934) of value +for keeping the rules in sync, though this MSC does not require MSC3934. + +For the purposes of [MSC3932](https://github.com/matrix-org/matrix-spec-proposals/pull/3932), the +push rules defined in this proposal are *not* affected by the room version limitations. This is due +to polls not inherently being room version-specific, unlike other extensible event structures. For +clarity, this means the push rules described here are treated the same as `.m.rule.master` (for +example) - they always apply, regardless of room version. + +## Potential issues + +The push rules for this feature are complex and not ideal. The author believes that it solves a short +term need while other MSCs work on improving the notifications system. Most importantly, the author +believes future MSCs which aim to fix notifications for extensible events in general will be a more +preferred approach over this MSC's (hopefully) short-term solution. + +## General considerations + +While the order within the MSC is deliberate for the new rules, the positioning relative to other rules +already in the spec is fairly arbitrary. The new rules are placed at the end of each list (underride and +override) for simplicity, but could realistically go anywhere in the list. + +See ["Dot-separated property paths"](https://spec.matrix.org/v1.7/appendices/#dot-separated-property-paths) +for more information on why we escape the `event_property_is` condition values. + +## Security considerations + +None applicable - no new considerations need to be made with this proposal. + +## Unstable prefix + +While this MSC is not considered stable, implementations should use `org.matrix.msc3930.*` as a prefix +in place of `m.*` for the push rule IDs. The event types for those associated push rules are described +by the [Unstable Prefix section of MSC3381](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3381-polls.md#unstable-prefix). diff --git a/proposals/3958-suppress-notifications-of-message-edits.md b/proposals/3958-suppress-notifications-of-message-edits.md new file mode 100644 index 00000000..52351b1a --- /dev/null +++ b/proposals/3958-suppress-notifications-of-message-edits.md @@ -0,0 +1,142 @@ +# MSC3958: Suppress notifications from message edits + +[Event replacement](https://spec.matrix.org/v1.7/client-server-api/#event-replacements) +(more commonly known as message edits) signals that a message is intended to +be replaced with new content. + +This works well for fixing typos or other minor correction, but can cause +spurious notifications if the event mentions a user's display name / localpart or +if it includes `@room` (which is particularly bad in large rooms as every user +is re-notified). This contributes to notification fatigue as the additional +notifications contain no new information. + +Additionally for users which have a room set to "all messages" then every event +edit results in an additional notification.[^1] + +## Proposal + +A new default push rule is added to suppress notifications due to [edits](https://spec.matrix.org/v1.7/client-server-api/#event-replacements). + +```json +{ + "rule_id": ".m.rule.suppress_edits", + "default": true, + "enabled": true, + "conditions": [ + { + "kind": "event_property_is", + "key": "content.m\\.relates_to.rel_type", + "value": "m.replace" + } + ], + "actions": [] +} +``` + +This rule should be placed after the [`.m.rule.room.server_acl` rule](https://spec.matrix.org/v1.7/client-server-api/#default-override-rules) +as the last override rule. + +It would match events such as those given in [event replacements](https://spec.matrix.org/v1.7/client-server-api/#event-replacements) +portion of the spec: + +```json5 +{ + "type": "m.room.message", + "content": { + "body": "* Hello! My name is bar", + "msgtype": "m.text", + "m.new_content": { + "body": "Hello! My name is bar", + "msgtype": "m.text" + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$some_event_id" + } + }, + // ... other fields required by events +} +``` + +With the [updated mentions behavior in Matrix 1.7](https://spec.matrix.org/v1.7/client-server-api/#user-and-room-mentions), +this would allow the [`.m.rule.is_user_mention`](https://spec.matrix.org/v1.7/client-server-api/#_m_rule_is_user_mention) +and the [`.m.rule.is_room_mention`](https://spec.matrix.org/v1.7/client-server-api/#_m_rule_is_room_mention) +rules to continue matching, even for edited events, while suppressing notifications +from other edits. + +Some users may be depending on notifications of edits. If a user would like to +revert to the old behavior they can disable the `.m.rule.suppress_edits` push rule. + +## Potential issues + +### Edits of invites and tombstones + +The [`.m.rule.invite_for_me` and `.m.rule.tombstone`](https://spec.matrix.org/v1.7/client-server-api/#default-override-rules) +rules may still cause spurious notifications if events which match those rules +are edited. Both of those are state events and +[not subject to valid edits](https://spec.matrix.org/v1.7/client-server-api/#validity-of-replacement-events). + +### Keeping notifications up-to-date + +Mobile clients currently depend on the push notifications of edited events to update the +text of on-screen notifications. The proposed push rule would result in mobile clients no +longer receiving these edits; but showing slightly outdated text on a notification screen. That +is only a minor impact and it would be better to separate when (& why) we send pushes vs. +when we generate notifications. + +### Suppression of notifications to a new keyword + +If an event is edited and the new event (but not the original event) matches a keyword +then the notification would erroneously be suppressed. + +## Alternatives + +An alternative solution would be to add a push rule with no actions and a condition to +check whether a notification was generated for the original message. + +This would be placed as an override rule before the `.m.rule.contains_display_name` +and the `.m.rule.roomnotif` [push rules](https://spec.matrix.org/v1.7/client-server-api/#push-rules). + +This would suppress duplicate notifications, while still allow for new notifications due +to new mentions or keywords changing. + +## Security considerations + +None forseen. + +## Future extensions + +If message edits by other senders were allowed than it would be useful to +know when your own message was edited, but this +[is not currently allowed](https://spec.matrix.org/v1.7/client-server-api/#validity-of-replacement-events). +A future MSC to define this behavior should take into account notifying +users in this situation. + +## Unstable prefix + +The unstable prefix of `.org.matrix.msc3958.suppress_edits` should be used in place of +`.m.rule.suppress_edits`. + +A previous version of this MSC used `.com.beeper.suppress_edits` with a different condition +(which should match the same events), but different rule placement. + +## Dependencies + +N/A + + + +[^1]: A room can be configured to be notify for "all messages" by creating a [room-specific push rule](https://spec.matrix.org/v1.7/client-server-api/#push-rules) +with an `rule_id` of the room ID & has `actions` set to "notify" , e.g.: + + ```json + { + "rule_id" : "!abcdef:example.com", + "default" : false, + "enabled" : true, + "actions" : ["notify"] + } + ``` + + See the [Element Web](https://github.com/matrix-org/matrix-react-sdk/blob/da7aa4055e04f291be9b5141b704bd12aec03d0c/src/RoomNotifs.ts#L162-L170) + implementation. diff --git a/proposals/3981-relations-recursion.md b/proposals/3981-relations-recursion.md new file mode 100644 index 00000000..9a7ae79b --- /dev/null +++ b/proposals/3981-relations-recursion.md @@ -0,0 +1,165 @@ +# MSC3981: `/relations` recursion + +The [`/relations`] API allows clients to retrieve events which directly relate +to a given event. + +This API has been used as basis of many other features and MSCs since then, +including threads. + +Threads was one of the first usages of this API that allowed nested relations - +an event may have an [`m.reaction`] or [`m.replace`] relation to another event, +which in turn may have an `m.thread` relation to the thread root. + +This forms a tree of relations. A client wanting to display a thread will want +to display reactions and edits to messages in the thread, and will therefore need +the second-order related events in addition to just the events with a direct thread +relation to the root. + +Clients can recursively perform the /relations queries on each event but this is +very slow and does not give the client any information on how the events are ordered +for the purpose of sending read receipts. + +## Proposal + +It is proposed to add the `recurse` parameter to the `/relations` API, defined +as follows: + +> Whether to additionally include events which only relate indirectly to the +> given event, +> ie. events related to the root events via one or more direct relationships. +> +> If set to false, only events which have direct a relation with the given +> event will be included. +> +> If set to true, all events which relate to the given event, or relate to +> events that relate to the given event, will be included. +> +> It is recommended that at least 3 levels of relationships are traversed. +> Implementations may perform more but should be careful to not infinitely recurse. +> +> One of: `[true false]`. + +In order to be backwards compatible the `recurse` parameter must be +optional (defaulting to `false`). + +Regardless of the value of the `recurse` parameter, events will always be +returned in topological ordering, ie. the same order in which the `/messages` API +would return them (given the same `dir` parameter). + +It is also proposed to add a response parameter, `recursion_depth` to the response +which gives the actual depth limit the server used in its recursion. This key is mandatory if +the `recurse` parameter was passed and is absent otherwise. eg: + +```json +{ + "chunk": [...], + "recursion_depth": 3 +} +``` + +Note that there no way in this MSC for a client to affect how much the server recurses. +If the client decides that the server's recursion level is insufficient, it could, for example, +perform the recursion manually, or disable whatever feature requires more recursion. + +Filters specified via `event_type` or `rel_type` will be applied to all events +returned, whether direct or indirect relations. Events that would match the filter, +but whose only relation to the original given event is through a non-matching +intermediate event, will not be included. This means that supplying a `rel_type` +parameter of `m.thread` is not appropriate for fetching all events in a thread since +relations to the threaded events would be filtered out. For this purpose, clients should +omit the `rel_type` parameter and perform any necessary filtering on the client side. + +## Potential issues + +Naive implementations might be tempted to provide support for this parameter +through a thin shim which is functionally identical to the client doing +separate recursive `/relations` requests itself. However this would allow a +client to craft a set of events that would cause unreasonable load. + +## Alternatives + +1. Clients could fetch all relations recursively client-side, which would + increase network traffic and server load significantly. +2. A new, specialised endpoint could be created for threads, specifically + designed to present separate timelines that, in all other ways, would + behave identically to `/messages`. +3. Twitter-style threads (see [MSC2836]). +4. Alternatively a `depth` parameter could have been specified, as in [MSC2836]. + We believe that a customizable depth would add unnecessary constraints to + server implementers, as different server implementations may have different + performance considerations and may choose different limits. Additionally, + the maximum currently achievable depth is still low enough to avoid this + becoming an issue. + +## Security considerations + +Security considerations are discussed inline throughout this proposal. To summarise: + * Allowing a client to control recursion depth could allow a client to cause outsize + load on the server if the server doesn't check the recursion depth. + * Naive server implementations could allow a client to craft a set of events that would + cause high load. + +## Examples + +Given the following graph: + +```mermaid +flowchart RL + subgraph Thread + G + E-->D + B-->A + end + B-.->|m.thread|A + G-.->|m.thread|A + E-.->|m.annotation|B + D-.->|m.edit|A + G-->F-->E + D-->C-->B +``` + +`/messages` with `dir=f` would +return `[A, B, C, D, E, F, G]`. + +`/relations` on event `A` with `rel_type=m.thread` and `dir=f` would +return `[B, G]`. + +`/relations` on event `A` with `recurse=true` and `dir=f` would +return `[B, D, E, G]`. + +`/relations` on event `A` with `recurse=true`, `dir=b` and `limit=2` would +return `[G, E]`. + +`/relations` on event `A` with `rel_type=m.annotation`, +`event_type=m.reaction` and `recurse=true` would return `[G, E]`. + +## Unstable prefix + +### While the MSC is not yet part of a spec version + +During this period, to detect server support, clients should check for the +presence of the `org.matrix.msc3981` flag in the `unstable_features` map +on [`/versions`](https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientversions). + +Clients are also required to use `org.matrix.msc3981.recurse` in place +of `recurse` at this time. + +`recursion_depth` is always used un-namespaced (it would only ever be sent +if the client had already sent the recurse parameter). + +### Once the MSC is in a spec version + +Once this MSC becomes a part of a spec version, clients should rely on the +presence of the spec version that supports this MSC in the `/version` response +to determine support. + +Servers are encouraged to keep the `org.matrix.msc3827` flag around for a +reasonable amount of time to help smooth over the transition for clients. +"Reasonable" is intentionally left as an implementation detail, however the MSC +process currently recommends at most 2 months from the date of spec release. + +[MSC2836]: https://github.com/matrix-org/matrix-spec-proposals/pull/2836 +[MSC3771]: https://github.com/matrix-org/matrix-spec-proposals/pull/3771 +[`/relations`]: https://spec.matrix.org/v1.6/client-server-api/#get_matrixclientv1roomsroomidrelationseventid +[`m.reaction`]: https://github.com/matrix-org/matrix-spec-proposals/pull/2677 +[`m.replace`]: https://spec.matrix.org/v1.6/client-server-api/#event-replacements diff --git a/proposals/3989-redact-origin-field.md b/proposals/3989-redact-origin-field.md new file mode 100644 index 00000000..8d362122 --- /dev/null +++ b/proposals/3989-redact-origin-field.md @@ -0,0 +1,61 @@ +# MSC3989: Redact `origin` property on events + +The [current redaction algorithm](https://spec.matrix.org/v1.6/rooms/v10/#redactions) *keeps* the +top-level `origin` property on events during redaction, however, as of this writing, the only use within the +spec of `origin` as a top-level event property is a malformed example of event format. The property has no +significant meaning in modern room versions. + +Within the ecosystem, it's clear that we'd [prefer the property to disappear](https://github.com/matrix-org/matrix-spec/issues/374), +and have [tried to do so](https://github.com/matrix-org/matrix-spec/pull/998) in the past. The +malformed examples are even [known to us](https://github.com/matrix-org/matrix-spec/issues/1480). + +What's not clear, and mentioned in [a comment](https://github.com/matrix-org/matrix-spec/issues/1480#issuecomment-1495183789), +is whether the `origin` property is *actually* used. There do not appear to be any auth rules or similar +which would use the property, however it'd hardly be the first time that the spec was wrong about an +ancient room version like v1. What is clear is that Synapse, where this question would be asked, +wants to [drop support](https://github.com/matrix-org/synapse/issues/3816) for the property and has +taken [steps](https://github.com/matrix-org/synapse/pull/8324) towards that mission by fixing bugs +in the area. In a [quick audit](https://github.com/matrix-org/matrix-spec-proposals/pull/3989#issuecomment-1497659507) +of the Synapse codebase during implementation of this MSC, the `origin` property appears unused. + +Given the above context, this proposal removes the `origin` property from the [redaction algorithm](https://spec.matrix.org/v1.7/rooms/v10/#redactions) +in a future room version, leaving it as-is for existing versions (not that an MSC can change the behaviour +of an already-stable room version anyways). + +Some other properties are additionally useless in modern room versions, however they are already adapted +by [MSC2176](https://github.com/matrix-org/matrix-spec-proposals/pull/2176). + +## Proposal + +In a future room version, the `origin` property is *removed* from the list of *event* keys which are +kept during redaction. Note that this requires a new room version because changing the redaction +algorithm changes how [event IDs](https://spec.matrix.org/v1.6/rooms/v10/#event-ids) are calculated, +as they are [reference hashes](https://spec.matrix.org/v1.6/server-server-api/#calculating-the-reference-hash-for-an-event) +which redact the event during calculation. + +## Potential issues + +No major concerns. + +## Alternatives + +No significant alternatives. + +## Security considerations + +No major concerns. + +## Unstable prefix + +While this MSC is not considered stable, implementations should use `org.matrix.msc3989` as the room +version identifier, using v10 as a base. + +## Dependencies + +No blocking dependencies. + +This MSC would partner well with the following MSCs, however: +* [MSC2176](https://github.com/matrix-org/matrix-spec-proposals/pull/2176) +* [MSC2175](https://github.com/matrix-org/matrix-spec-proposals/pull/2175) +* [MSC2174](https://github.com/matrix-org/matrix-spec-proposals/pull/2174) +* [MSC3821](https://github.com/matrix-org/matrix-spec-proposals/pull/3821) diff --git a/proposals/4009-e.164-matrix-ids.md b/proposals/4009-e.164-matrix-ids.md new file mode 100644 index 00000000..ca86a8cd --- /dev/null +++ b/proposals/4009-e.164-matrix-ids.md @@ -0,0 +1,103 @@ +# MSC4009: Expanding the Matrix ID grammar to enable E.164 IDs + +[E.164](https://www.itu.int/rec/T-REC-E.164) is a set of international standards +for telephone numbering. This defines a phone number as a `+` followed by a country +code (e.g. `1` for the US, `44` for the UK) followed by a local phone number. +For example, a E.164 phone number would look like: `+15558675309`. + +It is somewhat common[^1] for social networks or messaging applications to use phone +numbers as identifiers instead of relying on users remembering separate usernames +(and needing to tell other users how to find them). + +[Matrix user identifiers](https://spec.matrix.org/v1.6/appendices/#user-identifiers) +are *almost* compatible with E.164: + +> The `localpart` of a user ID is an opaque identifier for that user. It MUST NOT +> be empty, and MUST contain only the characters `a-z`, `0-9`, `.`, `_`, `=`, `-`, +> and `/`. + +## Proposal + +Add `+` to the list of allowed characters in a Matrix user identifier. This would +allow a full E.164 phone number as a user's username on a server, which is common +practice for many applications. This should hopefully ease bridging of those +services to Matrix or help them natively speak Matrix in the future. Users would +not need to learn a new Matrix ID, but can continue using their phone number, as +today. + +Although E.164 IDs are meant to be globally unique they are still namespaced under +a domain under this proposal, e.g. `@+15558675309:example.com`, as the same user may +already be registered on multiple service providers. + +## Potential issues + +Homeservers and clients must already be +[more tolerant of Matrix user IDs](https://spec.matrix.org/v1.6/appendices/#historical-user-ids): + +> Older versions of this specification were more tolerant of the characters +> permitted in user ID localparts. [...] clients and servers MUST accept user IDs +> with localparts from the expanded character set: +> +> `extended_user_id_char = %x21-39 / %x3B-7E ; all ASCII printing chars except :` + +Thus, it is expected to not cause any issues, although clients will need to identify +that the `+` character is valid for a homeserver. This could be from the supported +Matrix versions which the homeserver advertises. + +---- + +A user having two accounts with the same identifier on different services is not +ideal, but is not different than today where a user may log into both WhatsApp +and Signal with their phone number. This MSC does *not* attempt to help with mapping +of an E.164 identifier to an actual Matrix ID, that's best left to the current +[identity service `/lookup` endpoint](https://spec.matrix.org/v1.6/identity-service-api/#post_matrixidentityv2lookup) +or future discovery services. + +---- + +If a service uses E.164 identifiers as Matrix IDs then it must be careful to avoid +leaking history when reassigning IDs to new users. This is related to [#234](https://github.com/matrix-org/matrix-spec/issues/234), +but only applies to the localpart of the Matrix ID, not the entire domain. The +solution remains the same: using portable identifiers ([MSC1228](https://github.com/matrix-org/matrix-spec-proposals/pull/1228) +or [MSC4014](https://github.com/matrix-org/matrix-spec-proposals/pull/4014)). + +## Alternatives + +The `+` could continue to be disallowed and left off the beginning of the Matrix +IDs. Note that Synapse reserves digit-only usernames for guest users, although this +seems to be an implementation detail and not mandated by the specification. + +Another option would be to [map from other character sets](https://spec.matrix.org/v1.6/appendices/#mapping-from-other-character-sets) +or prefix the Matrix ID with something (e.g. `msisdn`). +This would generate a Matrix ID like `@=2b15558675309:example.com` or +`@msisdn_5558675309:example.com`, which would dramatically impact usability +for the users on homeservers which use phone numbers as identifiers. + +---- + +Although E.164 limits the `+` character to the initial character there seems to +be no reason to limit that for Matrix identifiers. + +## Security considerations + +E.164 IDs are globally unique, but the proposed change only enforces per-homeserver +uniqueness. If a homeserver does not diligently check that the phone number belongs +to a user then this may allow additional spoofing attacks. The author does not consider +this to be much worse than today's situation: + +1. Many current applications need to verify that a phone number truly is owned + by an account. Any current best practices should be followed by a service + taking advantage of this MSC. +2. It is already quite easy today to spoof a user's ID by registering the same + user localpart on a different homeserver. The same issue occurs with email or + other distributed system (Mastodon, etc.). + +## Unstable prefix + +N/A + +## Dependencies + +N/A + +[^1]: E.g. Signal, WhatsApp, etc. diff --git a/proposals/4010-push-rules-and-account-data.md b/proposals/4010-push-rules-and-account-data.md new file mode 100644 index 00000000..cd59b575 --- /dev/null +++ b/proposals/4010-push-rules-and-account-data.md @@ -0,0 +1,80 @@ +# MSC4010: Push rules and account data + +Push rules have a [bespoke API](https://spec.matrix.org/v1.6/client-server-api/#push-rules-api) +which clients use to retrieve, add, modify, and remove push rules. Any modifications +made using this API are sent to all clients via a `m.push_rules` event in the +`account_data` section in the +[the next `/sync` response](https://spec.matrix.org/v1.6/client-server-api/#push-rules-events). + +The client-server API does not have any special behavior around using the +[account data APIs](https://spec.matrix.org/v1.6/client-server-api/#client-behaviour-17) +with the `m.push_rules` type leading to different behavior on different homeservers: + +* Synapse will accept the data and shadow the real push rules in `/sync`, but + *won't use the data when evaluating push rules*. +* Dendrite will return an HTTP 403 if you attempt to set `m.push_rules` via + `/account_data`. +* Conduit has no protections for special account data. It will accept `m.push_rules` via + `/account_data` *and* use those when evaluating push rules. + +The [fully read marker](https://spec.matrix.org/v1.6/client-server-api/#fully-read-markers) +operates in a similar way and +[servers must reject updating `m.fully_read` via `/account_data`](https://spec.matrix.org/v1.6/client-server-api/#server-behaviour-10). + +Note that when attempting to set `m.fully_read` via `/account_data` the following +behavior is observed: + +* Synapse will reject it with a 405 error (only for room account data). +* Dendrite will reject it with an HTTP 403 error. + +## Proposal + +To make push rules data consistent with fully read markers, the following +clarifications are offered: + +* The `m.push_rules` account data type becomes protected and cannot be set using + the `/account_data` API, similarly to `m.fully_read`. +* "Rejected" means to use the 405 error response as + [documented](https://spec.matrix.org/v1.6/client-server-api/#put_matrixclientv3useruseridaccount_datatype): + + > This `type` of account data is controlled by the server; it cannot be modified + > by clients. Errcode: `M_BAD_JSON`. +* `m.push_rules` and `m.fully_read` should be rejected for both global and room + account data. +* Reading `m.push_rules` and `m.fully_read` should be allowed (although note that + currently `m.push_rules` only makes sense for global account data and `m.fully_read` + only makes sense for room account data). The format should match what is currently + [returned via `/sync`](https://spec.matrix.org/v1.6/client-server-api/#push-rules-events). + +The above rules shall also apply when deleting account data if [MSC3391](https://github.com/matrix-org/matrix-spec-proposals/pull/3391) +is merged before this MSC. + +## Potential issues + +It is possible that a client is currently storing data in the `m.push_rules` +(or global `m.fully_read`) account data. After this change it could no longer +be updated, deleted, or retrieved. It seems unlikely that the data stored here +is done purposefully (and is likely the result of undefined behavior that this +MSC is attempting to fix). + +## Alternatives + +An alternative would be to remove the current push rules API and have clients +store all push rules in bulk. This would be subject to race conditions between +clients. + +A slight variation of the above would be to *additionally* define the `/account_data/m.push_rules` +endpoint as bulk modifying the push rules data. This could be seen as an alternative +to [MSC3934](https://github.com/matrix-org/matrix-spec-proposals/pull/3934). + +## Security considerations + +None foreseen. + +## Unstable prefix + +This is mostly clarifications and does not add any event types or new endpoints. + +## Dependencies + +N/A diff --git a/proposals/4025-local-user-erasure-requests.md b/proposals/4025-local-user-erasure-requests.md new file mode 100644 index 00000000..c6a16185 --- /dev/null +++ b/proposals/4025-local-user-erasure-requests.md @@ -0,0 +1,83 @@ +# MSC4025: Local user erasure requests + +Long ago a need arose for having a user specify that they'd like to not just deactivate their account, +but also *erase* as much of their content as possible for largely GDPR purposes. This got implemented +in the matrix-js-sdk as [PR #649](https://github.com/matrix-org/matrix-js-sdk/pull/649), but never +quite made it to the spec. + +Back in 2018 the proposal process was very different (technically didn't actually exist at the time +the js-sdk PR was written). A [spec omission issue](https://github.com/matrix-org/matrix-spec/issues/297) +was opened to track the missing property, however [an attempt](https://github.com/matrix-org/matrix-spec-proposals/pull/1290)[^1] +to do the documentation got blocked on having a larger GDPR-centric proposal. + +Eventually, [MSC2438](https://github.com/matrix-org/matrix-spec-proposals/pull/2438) was drafted, +presumably to be that GDPR/erasure-specific MSC the prior spec PR was waiting for. Unfortunately, it +appears to have gotten stuck in various stages of drafting. + +It's highly regrettable that yet another spec change from 2018 has managed to go unspecified for so +long. Theoretically, the change could go through as a spec PR (much like the one linked above) rather +than as a proposal, however there is significant interest in giving the feature a formal chance to be +*rejected* as a solution under the modern spec proposal process. + +This MSC serves as that opportunity. While a more comprehensive system could be designed, such as in +MSC2438, this MSC has a very narrow and specific scope of what was implemented back in 2018. For the +spec process, this translates to accepting the feature as-is, or rejecting it and flagging the client +behaviour non-compliant. + +## Proposal + +A new optional boolean is added to the request body of [`POST /deactivate`](https://spec.matrix.org/v1.7/client-server-api/#post_matrixclientv3accountdeactivate): +`erase`. It defaults to `false` and signifies whether the user would like their content to be erased +as much as possible. + +Erasure does *not* mean issuing redactions for all of the user's sent events, but does mean that any +users (or servers) which join the room after the erasure request are served redacted copies of those +events. Users which had visibility on the event prior to the erasure are able to see unredacted copies. + +The server should additionally erase any non-event data associated with the user, such as account +data and [contact 3PIDs](https://spec.matrix.org/v1.7/client-server-api/#adding-account-administrative-contact-information). + +This is in line with what Synapse does, as referenced [here](https://github.com/matrix-org/synapse/issues/8185). + +This is also what is described by the [matrix.org blog post](https://matrix.org/blog/2018/05/08/gdpr-compliance-in-matrix) +on GDPR compliance. + +## Potential issues + +Erasure requests are not sent over federation in this model, meaning servers which already have an +unredacted copy of the event will continue to serve that to their users after the erasure happened. +Servers are expected to be informed out of band of erasure requests that affect them, if they affect +them. + +## Alternatives + +[MSC2438](https://github.com/matrix-org/matrix-spec-proposals/pull/2438) is the leading alternative, +however as specified in the introduction for this proposal, the desired outcomes of this MSC are either +acceptance as-written or rejection. Ideally, if rejected, another MSC or idea is marked as the suitable +alternative. + +Redactions could be sent for all the user's events, however this has obvious performance impact on +servers and rooms. The [matrix.org blog](https://matrix.org/blog/2018/05/08/gdpr-compliance-in-matrix) +discusses how redactions and GDPR Right to Erasure interact (or rather, how they aren't the same). + +## Security considerations + +This feature was originally introduced primarily in response to the GDPR Right to Erasure requirement +within the European Union. The privacy benefits apply to all users of the ecosystem and there's no clear +reason to region-lock or otherwise leave this as an implementation detail for EU-operated homeservers. + +There are however consequences of GDPR erasure that are not covered by this proposal, such as the +deletion of server logs, forwarding the request, etc. Server operators are encouraged to seek legal +advice on how to handle this form of erasure request (and whether it qualifies under GDPR's Right to +Erasure requirements). The general recommendation is to enforce the erasure request as much as possible +on the local homeserver, including redacting/purging server logs, appservice data, etc. + +## Unstable prefix + +Even more regrettably than having unspecified behaviour, this feature was implemented before unstable +namespaces existed. Implementations can use `erase` without prefix. + + + +[^1]: Readers should note that the spec-proposals repo used to contain both the spec itself and proposals +to change the spec. This was later split into dedicated repos, but closed PRs and issues were not migrated. diff --git a/proposals/4026-optional-authed-versions.md b/proposals/4026-optional-authed-versions.md new file mode 100644 index 00000000..c9aba05e --- /dev/null +++ b/proposals/4026-optional-authed-versions.md @@ -0,0 +1,39 @@ +# MSC4026: Allow /versions to optionally accept authentication + +## Introduction + +Synapse is implementing the ability to turn some unstable features on per-user. Once this is +implemented, certain experimental features will be available to be enabled per-user via the [Admin API](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html). +The intention is to allow certain users to test the experimental feature without making it available to +all users before it is stable. +This is in addition to the current ability to toggle on/off those features system-wide in the configuration. + +However, this poses a problem when considering how to advertise that those features are enabled to clients. +Traditionally, to determine what unstable features were available from a server clients checked the [`/_matrix/client/versions`](https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientversions) +endpoint, which in turn checked the Synapse configuration to determine what experimental features were enabled. With the +changes being implemented this is no longer possible, as some experimental features may be enabled per-user. As the +`/_matrix/client/versions` endpoint does not require authentication there is no way to know which experimental features +are enabled - there is no access token that we can extract user info from to determine which unstable features are +currently enabled (as they may only be enabled for some users) - and thus there is no way to correctly communicate to +clients which experimental features are enabled. + +## Proposal + +The proposal to remedy this is to make `/_matrix/client/versions` optionally accept authentication, and ask clients +to use the authenticated version when determining which experimental features are enabled. + +## Potential issues + +This does raise the question of what the non-authenticated version of `/_matrix/client/versions` should return with +regard to unstable features if the proposal is accepted. + +## Alternatives + +An alternative to the proposal would be to move advertising the unstable features to the [`/_matrix/client/v3/capabilities`](https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientv3capabilities) +endpoint, which does require authentication. However, the spec is clear that `/_matrix/client/v3/capabilities` "should +not be used to advertise unstable or experimental features - this is better done by the `/versions` endpoint." Thus, +this seems like a less desirable option than the proposed solution. + +## Security considerations + +None that I am currently aware of. \ No newline at end of file diff --git a/proposals/4040-update-srv-service-name.md b/proposals/4040-update-srv-service-name.md new file mode 100644 index 00000000..5e41145d --- /dev/null +++ b/proposals/4040-update-srv-service-name.md @@ -0,0 +1,119 @@ +# MSC4040: Update SRV service name to IANA registration + +The Internet Assigned Numbers Authority (IANA) holds a registry of all service names and ports in use +by Internet applications. Unfortunately, the service name `matrix` was registered an eternity ago for +use in an unrelated project, which makes our usage of it improper at best. On August 4, 2023 our +registration of port 8448 was accepted, with an included service name of `matrix-fed`. This MSC +proposes a backwards-compatible change to use this newly registered service name. + +The registrations with keyword "matrix" can be found [here](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=matrix). + +This fixes https://github.com/matrix-org/matrix-spec/issues/400 + +See also https://github.com/matrix-org/matrix-spec/issues/1576 + +## Proposal + +The [current specification](https://spec.matrix.org/v1.7/server-server-api/#resolving-server-names) for +resolving server names requires an implementation to look up `_matrix._tcp` SRV records in some steps. +Those steps become *deprecated* and new steps which use `_matrix-fed._tcp` are added immediately before +each existing step. + +Deprecation in context of this MSC is to discourage continued use, and to queue the affected steps for eventual +removal from the specification under the [deprecation policy](https://spec.matrix.org/v1.7/#deprecation-policy). + +The new steps thus become: + +1. (unchanged) If the hostname is an IP literal, then that IP address should be used, together with the + given port number, or 8448 if no port is given. The target server must present a valid certificate for + the IP address. The `Host` header in the request should be set to the server name, including the port if + the server name included one. + +2. (unchanged) If the hostname is not an IP literal, and the server name includes an explicit port, resolve + the hostname to an IP address using CNAME, AAAA or A records. Requests are made to the resolved IP address + and given port with a `Host` header of the original server name (with port). The target server must present + a valid certificate for the hostname. + +3. (steps added/deprecated) If the hostname is not an IP literal, a regular HTTPS request is made to `https:///.well-known/matrix/server`, + expecting the schema defined later in this section. 30x redirects should be followed, however redirection + loops should be avoided. Responses (successful or otherwise) to the `/.well-known` endpoint should be + cached by the requesting server. Servers should respect the cache control headers present on the response, + or use a sensible default when headers are not present. The recommended sensible default is 24 hours. Servers + should additionally impose a maximum cache time for responses: 48 hours is recommended. Errors are recommended + to be cached for up to an hour, and servers are encouraged to exponentially back off for repeated failures. + The schema of the `/.well-known` request is later in this section. If the response is invalid (bad JSON, + missing properties, non-200 response, etc), skip to step 4. If the response is valid, the `m.server` + property is parsed as `[:]` and processed as follows: + + 1. (unchanged) If `` is an IP literal, then that IP address should be used together + with the `` or 8448 if no port is provided. The target server must present a valid TLS + certificate for the IP address. Requests must be made with a `Host` header containing the IP address, + including the port if one was provided. + + 2. (unchanged) If `` is not an IP literal, and `` is present, an IP + address is discovered by looking up CNAME, AAAA or A records for ``. The resulting IP + address is used, alongside the ``. Requests must be made with a `Host` header of + `:`. The target server must present a valid certificate for ``. + + 3. **(ADDED)** If `` is not an IP literal and no `` is present, an + SRV record is looked up for `_matrix-fed._tcp.`. This may result in another hostname + (to be resolved using AAAA or A records) and port. Requests should be made to the resolved IP address and + port with a `Host` header containing the ``. The target server must present a valid + certificate for ``. + + 4. **(DEPRECATED)** If `` is not an IP literal and no `` is present, an + SRV record is looked up for `_matrix._tcp.`. This may result in another hostname (to + be resolved using AAAA or A records) and port. Requests should be made to the resolved IP address and port + with a `Host` header containing the ``. The target server must present a valid certificate + for ``. + + 5. (unchanged) If no SRV record is found, an IP address is resolved using CNAME, AAAA or A records. Requests + are then made to the resolve IP address and a port of 8448, using a `Host` header of ``. + The target server must present a valid certificate for ``. + +4. **(ADDED)** If the `/.well-known` request resulted in an error response, a server is found by resolving + an SRV record for `_matrix-fed._tcp.`. This may result in a hostname (to be resolved using AAAA or A + records) and port. Requests are made to the resolved IP address and port, with a `Host` header of ``. + The target server must present a valid certificate for ``. + +5. **(DEPRECATED)** If the `/.well-known` request resulted in an error response, a server is found by resolving + an SRV record for `_matrix._tcp.`. This may result in a hostname (to be resolved using AAAA or A + records) and port. Requests are made to the resolved IP address and port, with a `Host` header of ``. + The target server must present a valid certificate for ``. + +6. (unchanged) If the `/.well-known` request returned an error response, and the SRV record was not found, an IP + address is resolved using CNAME, AAAA and A records. Requests are made to the resolved IP address using port + 8448 and a `Host` header containing the ``. The target server must present a valid certificate for + ``. + +## Potential issues + +Us using `matrix` as a service name has been an issue all along. This MSC aligns the specification with our +formal IANA registration. + +## Alternatives + +We could modify our IANA registration further to pick a different name, however the service name needs to be +descriptive and short. The author notes that `matrix-server` was attempted in registration, which was declined +during Expert Review of the application. + +`matrix` cannot be taken over via IANA process. It was assigned before dates were put on records in 2000. + +Non-options for names include `mxfed`, `matrixf`, and similar. + +A potential way forward is to remove SRV resolution entirely, however prior effort via +[MSC3922](https://github.com/matrix-org/matrix-spec-proposals/pull/3922) are expected to be *rejected* instead +(as of writing, August 4th, 2023). + +## Security considerations + +There is increased DNS load with this change, due to having to potentially look up 6+ records in the worst case +scenario. DNS servers should be more than capable of caching, and handling such a spike of requests. + +## Unstable prefix + +None applicable. + +## Dependencies + +None applicable. diff --git a/proposals/4041-retry-after-header-rate-limiting.md b/proposals/4041-retry-after-header-rate-limiting.md new file mode 100644 index 00000000..77966563 --- /dev/null +++ b/proposals/4041-retry-after-header-rate-limiting.md @@ -0,0 +1,50 @@ +# MSC4041: Use http header Retry-After to enable library-assisted retry handling + +The current Matrix Client-Server API (v1.7) recommends that homeservers should protect themselves from +being overloaded by enforcing rate-limits to selected API calls. +If homeservers limit access to an API they respond with an http error code 429 and a response body +that includes a property `retry_after_ms` to indicate how long a client has to wait before retrying. + +Some http libraries (like [Ky](https://github.com/sindresorhus/ky), [got](https://github.com/sindresorhus/got) +and [urllib3](https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry)) ease +the burden of handling retries by honoring the http header `Retry-After`. As explained in +[RFC 9119 - HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110#field.retry-after) this header is optional +and contains either a (relative) value for the delay in seconds or an (absolute) date. + +The current Matrix Client Server API specification does not take this header into account. This wastes the +potential that current http libraries offer in terms of automated retry handling. + +## Proposal + +In order to allow developers to make use of the automated retry handling capabilities of http libraries +homeservers shall use the http header `Retry-After` in case they respond with an http error 429. +The value of `Retry-After` (in __seconds__) is meant to be a delay after receiving the response and must be +calculated in order to comply with the specification in [RFC 9119 - HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110#field.retry-after). + +With the introduction of the http header `Retry-After` the usage of the existing `retry_after_ms` property in the response body becomes deprecated. + +## Potential issues + +In order to maintain backward compatibility with existing client libraries homeservers shall use both the `Retry-After` header and the +`retry_after_ms` property in the response body. + +Client libraries shall use the values in this order: + +1) `Retry-After` http header. +2) `retry_after_ms` property in the response body. + +## Alternatives + +N/A + +## Security considerations + +N/A + +## Unstable prefix + +Since this MSC is using a standard HTTP header, it will not use a unstable prefix. + +## Dependencies + +N/A diff --git a/proposals/4077-process-deprecated-html.md b/proposals/4077-process-deprecated-html.md new file mode 100644 index 00000000..80674090 --- /dev/null +++ b/proposals/4077-process-deprecated-html.md @@ -0,0 +1,58 @@ +# MSC4077: Improved process for handling deprecated HTML features + +The Matrix specification [defines](https://spec.matrix.org/v1.8/client-server-api/#mroommessage-msgtypes) +a subset of HTML which clients *should* use when rendering messages, limiting the possibility for +Cross-Site Scripting (XSS) and similar attacks. Clients are always welcome to use a different subset +in both their send and receive paths, however in practice the recommended set of HTML has become the +baseline standard for support among Matrix client developers. + +Additionally, to fix issues like [tags being deprecated](https://github.com/matrix-org/matrix-spec/issues/1492), +the existing spec process would require an entire MSC to add or remove the tag. This has precedent +through [MSC2422](https://github.com/matrix-org/matrix-spec-proposals/pull/2422) and +[MSC2184](https://github.com/matrix-org/matrix-spec-proposals/pull/2184), where additional HTML +features are analyzed from a security perspective. For example, whether XSS attacks are more probable +or flooding/disruption attempts are made easier. + +This proposal adjusts the MSC process to explicitly allow changes to the supported HTML tags where +the WHATWG has deprecated a feature in favour of another, enabling more rapid iteration and reducing +paperwork for everyone involved. + +## Proposal + +Where the WHATWG has deprecated a feature of the HTML specification, a Matrix spec PR *may* bypass the +MSC process to use the modern feature instead. In the case of [issue#1492](https://github.com/matrix-org/matrix-spec/issues/1492), +the spec PR is explicitly able to add the `s` tag and remove the `strike` tag. Note that such changes +can also bypass the [deprecation policy](https://spec.matrix.org/v1.8/#deprecation-policy) in this way. + +It is left to the discretion of the spec PR reviewer as to whether an MSC is required. In general, an +MSC should be requested when the replacement HTML feature is not quite the same as the old feature. +The reviewer *may* also decide whether to keep supporting the old, deprecated, feature for some time. +For example, replacing `strike` with `s` may mean that both tags are supported for 1-2 Matrix spec +versions before `strike` can be dropped. + +For clarity, an MSC is always required to add net-new features to the HTML subset, or to remove a +feature entirely. An example of this is [MSC2398](https://github.com/matrix-org/matrix-spec-proposals/pull/2398). +These changes are subject to interoperability and security review, and cannot bypass the MSC process. + +## Potential issues + +The author notes that this MSC might not need to exist under the banner of pragmatism, however for all +the effort to *not* write an MSC, an MSC has been written for review. + +## Alternatives + +Critically, without agreement that replacing deprecated HTML features can happen outside the MSC process, +changes to the HTML subset become slow and burdensome for no apparent gain. + +## Security considerations + +Special care and attention is expected to be taken by spec PR reviewers to ensure that changes are +safe and in line with the semantics of this proposal's guidance. + +## Unstable prefix + +Not applicable. This is a process MSC. + +## Dependencies + +None applicable. diff --git a/proposals/4126-deprecate-query-string-auth.md b/proposals/4126-deprecate-query-string-auth.md new file mode 100644 index 00000000..264555f7 --- /dev/null +++ b/proposals/4126-deprecate-query-string-auth.md @@ -0,0 +1,74 @@ +# MSC4126: Deprecation of query string auth + +Presently, the Client-Server API allows clients to provide their access token via the `Authorization` +request header or via an `access_token` query string parameter, described [here](https://spec.matrix.org/v1.10/client-server-api/#using-access-tokens). +Clients are already encouraged to use the header approach, though the query string option exists for +largely backwards compatibility reasons. + +The query string approach is subject a number of security, usability, and practical concerns, discussed +on [matrix-spec#1780](https://github.com/matrix-org/matrix-spec/issues/1780): + +* The query string of an HTTP request is often logged by the client itself, middleware reverse proxy, + and application/homeserver as well. Though some of these layers may be aware of this issue, they + can trivially accidentally break to log sensitive credentials again. By contrast, headers are not + typically logged by default. + +* Users often copy and paste URLs from their clients to either get support or provide direct links + to content/media. While the media angle is largely expected to be resolved with [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/pull/3916), + users are currently able to right click images in their client and copy the URL - if this URL + includes authentication in the query string, the user will likely end up disclosing their access + token. The same scenario applies when copy/pasting request logs out of a client when getting + support. + +* Having two ways of doing things could lead to compatibility issues, where a client using the query + string approach is tried against a server which only supports the header. The client ends up not + working, leading to subpar user experience. + +* Most clients have already adopted the header approach, largely forgetting that the query string + even exists. Continuing to support the query string option leaves some maintenance burden for what + is effectively unused code. + +* Matrix has [decided](https://matrix.org/blog/2023/09/matrix-2-0/) to adopt OIDC for authentication, + which is based on OAuth 2.0, which [advises against](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics#section-4.3.2) + the query string approach. + +With these conditions in mind, this proposal sets the query string approach on a path towards removal +from the Matrix specification. This affects the Client-Server API and [Identity Service API](https://spec.matrix.org/v1.10/identity-service-api/#authentication) +as both support the approaches described above. + +## Proposal + +For both the Client-Server API and Identity Service API, the `access_token` query string authentication +parameter becomes *deprecated*, and SHOULD NOT be used by clients (as already stated in the specification). +Deprecation is required for at least 1 spec version before removal under the [deprecation policy](https://spec.matrix.org/v1.10/#deprecation-policy). + +Removal from the specification requires a second MSC and at least 1 specification release to pass. This +is currently described as [MSC4127](https://github.com/matrix-org/matrix-spec-proposals/pull/4127). + +## Potential issues + +Clients which rely on the query string approach may stop working. This is considered acceptable for +the purposes of this MSC. + +## Alternatives + +Most alternatives are not practical as they would maintain the security risk described in the introduction +for this proposal. + +Alterations to the deprecation policy may be discussed in a future MSC to make this sort of removal +easier. + +## Security considerations + +Security considerations are described throughout this proposal. + +## Unstable prefix + +This proposal cannot feasibly have an unstable prefix. Clients are already discouraged from using +query string authentication and should switch to `Authorization` as soon as possible, regardless of +this MSC. + +## Dependencies + +This MSC has no direct dependencies itself. [MSC4127](https://github.com/matrix-org/matrix-spec-proposals/pull/4127) +requires this MSC to land first.