diff --git a/.gitignore b/.gitignore index cce5848fa..a0c5cef07 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,4 @@ /api/node_modules -/assets /assets.tar.gz /data/msc /env* @@ -18,4 +17,4 @@ _rendered.rst /.vscode/ /.idea/ -/spec/ \ No newline at end of file +/spec/ diff --git a/README.md b/README.md index c13d4eafd..75f55affa 100644 --- a/README.md +++ b/README.md @@ -5,44 +5,44 @@ This repository contains the Matrix Specification, rendered at [spec.matrix.org] Developers looking to use Matrix should join [#matrix-dev:matrix.org](https://matrix.to/#/#matrix-dev:matrix.org) on Matrix for help. -Spec authors and proposal writers are welcome to join [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org). +Spec authors and proposal writers are welcome to join [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org). We welcome contributions! See [CONTRIBUTING.rst](./CONTRIBUTING.rst) for details. ## Structure The Matrix spec is compiled with [Hugo](https://gohugo.io/) (a static site generator) with the following structure: -* `/assets`: assets that need postprocessing using [Hugo Pipes](https://gohugo.io/hugo-pipes/introduction/). +* `/assets`: assets that need postprocessing using [Hugo Pipes](https://gohugo.io/hugo-pipes/introduction/). For example, Sass files would go here. -* `/content`: files that will become pages in the site go here. Typically these are Markdown files with some YAML front - matter indicating, [among other things](https://gohugo.io/content-management/front-matter/), what layout should be - applied to this page. The organization of files under `/content` determines the organization of pages in the built +* `/content`: files that will become pages in the site go here. Typically these are Markdown files with some YAML front + matter indicating, [among other things](https://gohugo.io/content-management/front-matter/), what layout should be + applied to this page. The organization of files under `/content` determines the organization of pages in the built site. -* `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as - [data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and +* `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as + [data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and parse them. This is also where our Swagger/OpenAPI definitions and schemas are. -* `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of +* `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of a page: for example, whether it has header, footer, sidebar, and so on. - * `/layouts/partials`: these templates can be called from other templates, so they can be used to factor out - template code that's used in more than one template. An obvious example here is something like a sidebar, where - several different page layouts might all include the sidebar. But also, partial templates can return values: this + * `/layouts/partials`: these templates can be called from other templates, so they can be used to factor out + template code that's used in more than one template. An obvious example here is something like a sidebar, where + several different page layouts might all include the sidebar. But also, partial templates can return values: this means they can be used like functions, that can be called by multiple templates to do some common processing. * `/layouts/shortcodes`: these templates can be called directly from files in `/content`. * `/static`: static files which don't need preprocessing. JS or CSS files could live here. -* `/themes`: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are - supplied in a `/themes/$theme_name/layouts` directory. You can use a theme but customise it by providing your own - versions of any of the theme layouts in the base `/layouts` directory. That is, if a theme provides - `/themes/$theme_name/layouts/sidebar.html` and you provide `/layouts/sidebar.html`, then your version of the +* `/themes`: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are + supplied in a `/themes/$theme_name/layouts` directory. You can use a theme but customise it by providing your own + versions of any of the theme layouts in the base `/layouts` directory. That is, if a theme provides + `/themes/$theme_name/layouts/sidebar.html` and you provide `/layouts/sidebar.html`, then your version of the template will be used. It also has the following top-level file: -* `config.toml`: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings +* `config.toml`: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings defined here are available in templates. All these directories above are configurable via `config.toml` settings. Additionally, the following directories may be of interest: @@ -59,34 +59,40 @@ Additionally, the following directories may be of interest: Please read [CONTRIBUTING.rst](./CONTRIBUTING.rst) before authoring a change to the spec. Note that spec authoring takes place after an MSC has been accepted, not as part of a proposal itself. -1. Install the extended version (often the OS default) of Hugo: +1. Install the extended version (often the OS default) of Hugo: + . Note that at least Hugo + v0.74 is required. + + Alternatively, use the Docker image at https://hub.docker.com/r/klakegg/hugo/. 2. Run `git submodule update --init --recursive` for good measure. 3. Run `npm i` to install the dependencies. Note that this will require NodeJS to be installed. 4. Run `npm run get-proposals` to seed proposal data. This is merely for populating the content of the "Spec Change Proposals" page and is not required. -5. Run `hugo serve` to run a local webserver which builds whenever a file change is detected. If watching doesn't appear - to be working for you, try `hugo serve --disableFastRender` instead. +5. Run `hugo serve` (or `docker run --rm -it -v $(pwd):/src -p 1313:1313 + klakegg/hugo serve`) to run a local webserver which builds whenever a file + change is detected. If watching doesn't appear to be working for you, try + adding `--disableFastRender` to the commandline. 6. Edit the specification 🙂 -We use a highly customized [Docsy](https://www.docsy.dev/) theme for our generated site, which uses Bootstrap and Font -Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in +We use a highly customized [Docsy](https://www.docsy.dev/) theme for our generated site, which uses Bootstrap and Font +Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in [#matrix-docs:matrix.org](https://matrix.to/#/#matrix-docs:matrix.org) before opening a PR. ## Building the specification -If for some reason you're not a CI/CD system and want to render a static version of the spec for yourself, follow the above -steps for authoring changes to the specification and instead of `hugo serve` run `hugo -d "spec"` - this will generate the -spec to `/spec`. If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"` +If for some reason you're not a CI/CD system and want to render a static version of the spec for yourself, follow the above +steps for authoring changes to the specification and instead of `hugo serve` run `hugo -d "spec"` - this will generate the +spec to `/spec`. If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"` to the `hugo -d "spec"` command. -For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt` -and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`. To make use of the generated file, +For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt` +and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`. To make use of the generated file, there are a number of options: * It can be uploaded from your filesystem to an online editor/viewer such as [on the swagger website](http://editor.swagger.io/). * You can run a local HTTP server by running `./scripts/swagger-http-server.py`, and then view the documentation via an online viewer; for example, at . -* You can host the swagger UI yourself. See for advice on how to +* You can host the swagger UI yourself. See for advice on how to do so. ## Issue tracking diff --git a/assets-hugo/icons/logo.svg b/assets/icons/logo.svg similarity index 100% rename from assets-hugo/icons/logo.svg rename to assets/icons/logo.svg diff --git a/assets-hugo/scss/_variables_project.scss b/assets/scss/_variables_project.scss similarity index 100% rename from assets-hugo/scss/_variables_project.scss rename to assets/scss/_variables_project.scss diff --git a/assets-hugo/scss/custom.scss b/assets/scss/custom.scss similarity index 97% rename from assets-hugo/scss/custom.scss rename to assets/scss/custom.scss index 3116762ed..62ae4532f 100644 --- a/assets-hugo/scss/custom.scss +++ b/assets/scss/custom.scss @@ -45,9 +45,6 @@ Custom SCSS for the Matrix spec scroll-behavior: smooth; overscroll-behavior: contain; - /* This overrides calc(100vh - 10rem);, which gives us a blank space at the bottom of the sidebar */ - max-height: calc(100vh - 6rem); - &>.td-sidebar-nav__section { margin-top: 1rem; } @@ -92,6 +89,15 @@ Custom SCSS for the Matrix spec } } +@media (min-width: 768px) { + @supports (position: sticky) { + .td-sidebar-nav { + /* This overrides calc(100vh - 10rem);, which gives us a blank space at the bottom of the sidebar */ + max-height: calc(100vh - 6rem); + } + } +} + /* Customise footer */ footer { box-shadow: 0px 0px 8px rgba(179, 179, 179, 0.25); diff --git a/changelogs/client_server/newsfragments/3225.clarification b/changelogs/client_server/newsfragments/3225.clarification new file mode 100644 index 000000000..046b25cf1 --- /dev/null +++ b/changelogs/client_server/newsfragments/3225.clarification @@ -0,0 +1 @@ +Update `Access-Control-Allow-Headers` recommendation to fit CORS specification. \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3233.clarification b/changelogs/client_server/newsfragments/3233.clarification new file mode 100644 index 000000000..72a58f1b6 --- /dev/null +++ b/changelogs/client_server/newsfragments/3233.clarification @@ -0,0 +1 @@ +Explicitly state that `replacment_room` is a room ID in `m.room.tombstone` events. diff --git a/changelogs/client_server/newsfragments/3238.clarification b/changelogs/client_server/newsfragments/3238.clarification new file mode 100644 index 000000000..63e54ba6c --- /dev/null +++ b/changelogs/client_server/newsfragments/3238.clarification @@ -0,0 +1 @@ +Clarify that all request bodies are required. diff --git a/changelogs/client_server/newsfragments/3254.feature b/changelogs/client_server/newsfragments/3254.feature new file mode 100644 index 000000000..c73c27ec6 --- /dev/null +++ b/changelogs/client_server/newsfragments/3254.feature @@ -0,0 +1 @@ +Add support for knocking, as per [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403). \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3324.feature b/changelogs/client_server/newsfragments/3324.feature new file mode 100644 index 000000000..346de35df --- /dev/null +++ b/changelogs/client_server/newsfragments/3324.feature @@ -0,0 +1 @@ +Extend `/_matrix/client/r0/login` to accept a `m.login.appservice`, as per [MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778). \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3327.breaking b/changelogs/client_server/newsfragments/3327.breaking new file mode 100644 index 000000000..85c0da8ba --- /dev/null +++ b/changelogs/client_server/newsfragments/3327.breaking @@ -0,0 +1 @@ +Fix key_backup operation IDs in OpenAPI spec. diff --git a/changelogs/client_server/newsfragments/3330.clarification b/changelogs/client_server/newsfragments/3330.clarification new file mode 100644 index 000000000..5df676a14 --- /dev/null +++ b/changelogs/client_server/newsfragments/3330.clarification @@ -0,0 +1 @@ +Add titles for OpenAPI objects. diff --git a/changelogs/client_server/newsfragments/3331.clarification b/changelogs/client_server/newsfragments/3331.clarification new file mode 100644 index 000000000..4e0836b88 --- /dev/null +++ b/changelogs/client_server/newsfragments/3331.clarification @@ -0,0 +1 @@ +Add auth property to UIA endpoint uploadCrossSigningKeys. diff --git a/changelogs/client_server/newsfragments/3332.clarification b/changelogs/client_server/newsfragments/3332.clarification new file mode 100644 index 000000000..63e54ba6c --- /dev/null +++ b/changelogs/client_server/newsfragments/3332.clarification @@ -0,0 +1 @@ +Clarify that all request bodies are required. diff --git a/changelogs/client_server/newsfragments/3336.clarification b/changelogs/client_server/newsfragments/3336.clarification new file mode 100644 index 000000000..bb06ffec9 --- /dev/null +++ b/changelogs/client_server/newsfragments/3336.clarification @@ -0,0 +1 @@ +Disambiguate getEvents and peekEvents, and include both in swagger. diff --git a/changelogs/client_server/newsfragments/3337.clarification b/changelogs/client_server/newsfragments/3337.clarification new file mode 100644 index 000000000..9f8eb2f25 --- /dev/null +++ b/changelogs/client_server/newsfragments/3337.clarification @@ -0,0 +1 @@ +Mention that a canonical alias event should be added when a room is created with an alias. \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3339.clarification b/changelogs/client_server/newsfragments/3339.clarification new file mode 100644 index 000000000..3ccb23339 --- /dev/null +++ b/changelogs/client_server/newsfragments/3339.clarification @@ -0,0 +1 @@ +Fix various typos throughout the specification. diff --git a/changelogs/client_server/newsfragments/3350.clarification b/changelogs/client_server/newsfragments/3350.clarification new file mode 100644 index 000000000..c5524f19b --- /dev/null +++ b/changelogs/client_server/newsfragments/3350.clarification @@ -0,0 +1 @@ +Add an 'API conventions' section to the Appendices. \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3353.clarification b/changelogs/client_server/newsfragments/3353.clarification new file mode 100644 index 000000000..79f15dc71 --- /dev/null +++ b/changelogs/client_server/newsfragments/3353.clarification @@ -0,0 +1 @@ +Clarify the documentation around the pagination tokens used by `/sync`, `/rooms/{room_id}/messages`, `/initialSync`, `/rooms/{room_id}/initialSync`, and `/notifications`. \ No newline at end of file diff --git a/changelogs/client_server/newsfragments/3366.clarification b/changelogs/client_server/newsfragments/3366.clarification new file mode 100644 index 000000000..459c5075b --- /dev/null +++ b/changelogs/client_server/newsfragments/3366.clarification @@ -0,0 +1 @@ +Remove the inaccurate 'Pagination' section. diff --git a/changelogs/server_server/newsfragments/3312.clarification b/changelogs/server_server/newsfragments/3312.clarification new file mode 100644 index 000000000..a9e1fe8df --- /dev/null +++ b/changelogs/server_server/newsfragments/3312.clarification @@ -0,0 +1 @@ +Correct the `/_matrix/federation/v1/user/devices/{userId}` response which actually returns `"self_signing_key"` instead of `"self_signing_keys"`. diff --git a/changelogs/server_server/newsfragments/3322.clarification b/changelogs/server_server/newsfragments/3322.clarification new file mode 100644 index 000000000..fdcd7db9b --- /dev/null +++ b/changelogs/server_server/newsfragments/3322.clarification @@ -0,0 +1 @@ +Explain the reasons why `` TLS certificate is needed rather than `` for SRV delegation. \ No newline at end of file diff --git a/changelogs/server_server/newsfragments/3340.clarification b/changelogs/server_server/newsfragments/3340.clarification new file mode 100644 index 000000000..8128a2234 --- /dev/null +++ b/changelogs/server_server/newsfragments/3340.clarification @@ -0,0 +1 @@ +Tweak the example PDU diagram to better demonstrate situations with multiple `prev_events`. diff --git a/config.toml b/config.toml index 1e4cb70e7..21108fb84 100644 --- a/config.toml +++ b/config.toml @@ -12,10 +12,6 @@ theme = ["docsy"] disableKinds = ["taxonomy", "taxonomyTerm"] -# Change the default for assets, because the old Python toolchain uses "assets" for build output. -# When the old toolchain is retired we can switch back to the default here. -assetDir = "assets-hugo" - [languages] [languages.en] title = "Matrix Specification" @@ -43,7 +39,13 @@ privacy_policy = "https://matrix.org/legal/privacy-notice" # must be one of "unstable", "current", "historical" # this is used to decide whether to show a banner pointing to the current release status = "unstable" +# A URL pointing to the latest, stable release of the spec. To be shown in the unstable version warning banner. current_version_url = "https://matrix.org/docs/spec/" +# The following is used when status = "stable", and is displayed in various UI elements on a released version +# of the spec. CI will set these values here automatically when a release git tag (i.e `v1.5`) is created. +#major = "1" +#minor = "0" +#release_date = "April 01, 2021" # User interface configuration [params.ui] diff --git a/content/_index.md b/content/_index.md index 4f7c30515..7d978fd41 100644 --- a/content/_index.md +++ b/content/_index.md @@ -41,7 +41,7 @@ browsing the Client-Server API. {{% boxes/note %}} As of June 10th 2019, the Matrix specification is considered out of beta --indicating that all currently released APIs are considered stable and +- indicating that all currently released APIs are considered stable and secure to the best of our knowledge, and the spec should contain the complete information necessary to develop production-grade implementations of Matrix without the need for external reference. @@ -328,7 +328,7 @@ Federation maintains *shared data structures* per-room between multiple homeservers. The data is split into `message events` and `state events`. Message events: -These describe transient 'once-off' activity in a room such as an +These describe transient 'one-off' activity in a room such as an instant messages, VoIP call setups, file transfers, etc. They generally describe communication activity. diff --git a/content/appendices.md b/content/appendices.md index f3be0cc67..31c5fbb86 100644 --- a/content/appendices.md +++ b/content/appendices.md @@ -47,6 +47,27 @@ When decoding Base64, implementations SHOULD accept input with or without padding characters wherever possible, to ensure maximum interoperability. +## Binary data + +In some cases it is necessary to encapsulate binary data, for example, +public keys or signatures. Given that JSON cannot safely represent raw +binary data, all binary values should be encoded and represented in +JSON as unpadded Base64 strings as described above. + +In cases where the Matrix specification refers to either opaque byte +or opaque Base64 values, the value is considered to be opaque AFTER +Base64 decoding, rather than the encoded representation itself. + +It is safe for a client or homeserver implementation to check for +correctness of a Base64-encoded value at any point, and to altogether +reject a value which is not encoded properly. However, this is optional +and is considered to be an implementation detail. + +Special consideration is given for future protocol transformations, +such as those which do not use JSON, where Base64 encoding may not be +necessary in order to represent a binary value safely. In these cases, +Base64 encoding of binary values may be skipped altogether. + ## Signing JSON Various points in the Matrix specification require JSON objects to be @@ -1039,3 +1060,48 @@ The event signing algorithm should emit the following signed event: } } ``` + +## Conventions for Matrix APIs + +This section is intended primarily to guide API designers when adding to Matrix, +setting guidelines to follow for how those APIs should work. This is important to +maintain consistency with the Matrix protocol, and thus improve developer +experience. + +### HTTP endpoint and JSON property naming + +The names of the API endpoints for the HTTP transport follow a convention of +using underscores to separate words (for example `/delete_devices`). + +The key names in JSON objects passed over the API also follow this convention. + +{{% boxes/note %}} +There are a few historical exceptions to this rule, such as `/createRoom`. +These inconsistencies may be addressed in future versions of this specification. +{{% /boxes/note %}} + +### Pagination + +REST API endpoints which can return multiple "pages" of results should adopt the +following conventions. + + * If more results are available, the endpoint should return a property named + `next_batch`. The value should be a string token which can be passed into + a subsequent call to the endpoint to retrieve the next page of results. + + If no more results are available, this is indicated by *omitting* the + `next_batch` property from the results. + + * The endpoint should accept a query-parameter named `from` which the client + is expected to set to the value of a previous `next_batch`. + + * Some endpoints might support pagination in two directions (example: + `/messages`, which can be used to move forward or backwards in the timeline + from a known point). In this case, the endpoint should return a `prev_batch` + property which can be passed into `from` to receive the previous page of + results. + + Avoid having a separate "direction" parameter, which is generally redundant: + the tokens returned by `next_batch` and `prev_batch` should contain enough + information for subsequent calls to the API to know which page of results + they should return. diff --git a/content/application-service-api.md b/content/application-service-api.md index e324bba05..02d7ed6fa 100644 --- a/content/application-service-api.md +++ b/content/application-service-api.md @@ -312,10 +312,11 @@ information. The homeserver needs to give the application service *full control* over its namespace, both for users and for room aliases. This means that the -AS should be able to create/edit/delete any room alias in its namespace, -as well as create/delete any user in its namespace. No additional API +AS should be able to manage any users and room alias in its namespace. No additional API changes need to be made in order for control of room aliases to be -granted to the AS. Creation of users needs API changes in order to: +granted to the AS. + +Creation of users needs API changes in order to: - Work around captchas. - Have a 'passwordless' user. @@ -334,8 +335,26 @@ user ID without a password. username: "_irc_example" } +Similarly, logging in as users needs API changes in order to allow the AS to +log in without needing the user's password. This is achieved by including the +`as_token` on a `/login` request, along with a login type of +`m.login.application_service`. + + POST /_matrix/client/%CLIENT_MAJOR_VERSION%/login + Authorization: Bearer YourApplicationServiceTokenHere + + Content: + { + type: "m.login.application_service", + "identifier": { + "type": "m.id.user", + "user": "_irc_example" + } + } + Application services which attempt to create users or aliases *outside* -of their defined namespaces will receive an error code `M_EXCLUSIVE`. +of their defined namespaces, or log in as users outside of their defined +namespaces will receive an error code `M_EXCLUSIVE`. Similarly, normal users who attempt to create users or aliases *inside* an application service-defined namespace will receive the same `M_EXCLUSIVE` error code, but only if the application service has diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index 07fe9d962..f9ef60632 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -4,7 +4,7 @@ weight: 10 type: docs --- -The client-server API provides a simple lightweight API to let clients +The client-server API allows clients to send messages, control rooms and synchronise conversation history. It is designed to support both lightweight clients which store no state and lazy-load data from the server as required - as well as heavyweight @@ -20,20 +20,15 @@ supported as optional extensions - e.g. a packed binary encoding over stream-cipher encrypted TCP socket for low-bandwidth/low-roundtrip mobile usage. For the default HTTP transport, all API calls use a Content-Type of `application/json`. In addition, all strings MUST be -encoded as UTF-8. Clients are authenticated using opaque `access_token` -strings (see [Client Authentication](#client-authentication) for -details), passed as a query string parameter on all requests. +encoded as UTF-8. -The names of the API endpoints for the HTTP transport follow a -convention of using underscores to separate words (for example -`/delete_devices`). The key names in JSON objects passed over the API -also follow this convention. +Clients are authenticated using opaque `access_token` strings (see [Client +Authentication](#client-authentication) for details). -{{% boxes/note %}} -There are a few historical exceptions to this rule, such as -`/createRoom`. A future version of this specification will address the -inconsistency. -{{% /boxes/note %}} +See also [Conventions for Matrix APIs](/appendices#conventions-for-matrix-apis) +in the Appendices for conventions which all Matrix APIs are expected to follow. + +### Standard error response Any errors which occur at the Matrix API level MUST return a "standard error response". This is a JSON object which looks like: @@ -46,15 +41,17 @@ error response". This is a JSON object which looks like: ``` The `error` string will be a human-readable error message, usually a -sentence explaining what went wrong. The `errcode` string will be a -unique string which can be used to handle an error message e.g. -`M_FORBIDDEN`. These error codes should have their namespace first in -ALL CAPS, followed by a single \_ to ease separating the namespace from -the error code. For example, if there was a custom namespace -`com.mydomain.here`, and a `FORBIDDEN` code, the error code should look -like `COM.MYDOMAIN.HERE_FORBIDDEN`. There may be additional keys -depending on the error, but the keys `error` and `errcode` MUST always -be present. +sentence explaining what went wrong. + +The `errcode` string will be a unique string which can be used to handle an +error message e.g. `M_FORBIDDEN`. Error codes should have their namespace +first in ALL CAPS, followed by a single `_`. For example, if there was a custom +namespace `com.mydomain.here`, and a `FORBIDDEN` code, the error code should +look like `COM.MYDOMAIN.HERE_FORBIDDEN`. Error codes defined by this +specification should start `M_`. + +Some `errcode`s define additional keys which should be present in the error +response object, but the keys `error` and `errcode` MUST always be present. Errors are generally best expressed by their error code rather than the HTTP status code returned. When encountering the error code `M_UNKNOWN`, @@ -66,7 +63,9 @@ found. However, if the client were to receive an error code of `M_UNKNOWN` with a 400 Bad Request, the client should assume that the request being made was invalid. -The common error codes are: +#### Common error codes + +These error codes can be returned by any API endpoint: `M_FORBIDDEN` Forbidden access, e.g. joining a room without permission, failed login. @@ -98,7 +97,11 @@ then try again. `M_UNKNOWN` An unknown error has occurred. -Other error codes the client might encounter are: +#### Other error codes + +The following error codes are specific to certain endpoints. + +. `M_UNRECOGNIZED` The server did not understand the request. @@ -226,7 +229,7 @@ headers to be returned by servers on all requests are: Access-Control-Allow-Origin: * Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS - Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization + Access-Control-Allow-Headers: X-Requested-With, Content-Type, Authorization ## Server Discovery @@ -1021,6 +1024,41 @@ client supports it, the client should redirect the user to the is complete, the client will need to submit a `/login` request matching `m.login.token`. +#### Appservice Login + +An appservice can log in by providing a valid appservice token and a user within the appservice's +namespace. + +{{% boxes/note %}} +Appservices do not need to log in as individual users in all cases, as they +can perform [Identity Assertion](/application-service-api#identity-assertion) +using the appservice token. However, if the appservice needs a scoped token +for a single user then they can use this API instead. +{{% /boxes/note %}} + +This request must be authenticated by the [appservice `as_token`](/application-service-api#registration) +(see [Client Authentication](#client-authentication) on how to provide the token). + +To use this login type, clients should submit a `/login` request as follows: + +```json +{ + "type": "m.login.appservice", + "identifier": { + "type": "m.id.user", + "user": "" + } +} +``` + +If the access token is not valid, does not correspond to an appservice +or the user has not previously been registered then the homeserver will +respond with an errcode of `M_FORBIDDEN`. + +If the access token does correspond to an appservice, but the user id does +not lie within its namespace then the homeserver will respond with an +errcode of `M_EXCLUSIVE`. + {{% http-api spec="client-server" api="login" %}} {{% http-api spec="client-server" api="logout" %}} @@ -1211,88 +1249,6 @@ using an `unstable` version. When this capability is not listed, clients should use `"1"` as the default and only stable `available` room version. -## Pagination - -{{% boxes/note %}} -The paths referred to in this section are not actual endpoints. They -only serve as examples to explain how pagination functions. -{{% /boxes/note %}} - -Pagination is the process of dividing a dataset into multiple discrete -pages. Matrix makes use of pagination to allow clients to view extremely -large datasets. These datasets are not limited to events in a room (for -example clients may want to paginate a list of rooms in addition to -events within those rooms). Regardless of what is being paginated, there -is a common approach which is used to give clients an easy way of -selecting subsets of a potentially changing dataset. Each endpoint that -uses pagination may use different parameters. However the theme among -them is that they take a `from` and `to` token, and occasionally a -`limit` and `dir`. Together, these parameters describe the position in a -data set, where `from` and `to` are known as "stream tokens" matching -the regular expression `[a-zA-Z0-9.=_-]+`. If supported, the `dir` -defines the direction of events to return: either forwards (`f`) or -backwards (`b`). The response may contain tokens that can be used for -retrieving results before or after the returned set. These tokens may be -called start or prev\_batch for retrieving the previous result -set, or end, next\_batch or next\_token for retrieving the next result set. - -In the following examples, 'START' and 'END' are placeholders to signify -the start and end of the data sets respectively. - -For example, if an endpoint had events E1 -> E15. The client wants -the last 5 events and doesn't know any previous events: - -``` - S E - |-E1-E2-E3-E4-E5-E6-E7-E8-E9-E10-E11-E12-E13-E14-E15-| - | | | - | _____| <--backwards-- | - |__________________ | | ________| - | | | | - GET /somepath?to=START&limit=5&dir=b&from=END - Returns: - E15,E14,E13,E12,E11 -``` - -Another example: a public room list has rooms R1 -> R17. The client -is showing 5 rooms at a time on screen, and is on page 2. They want to -now show page 3 (rooms R11 -> 15): - -``` - S E - | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | stream token - |-R1-R2-R3-R4-R5-R6-R7-R8-R9-R10-R11-R12-R13-R14-R15-R16-R17| room - |____________| |________________| - | | - Currently | - viewing | - | - GET /roomslist?from=9&to=END&limit=5 - Returns: R11,R12,R13,R14,R15 -``` - -Note that tokens are treated in an *exclusive*, not inclusive, manner. -The end token from the initial request was '9' which corresponded to -R10. When the 2nd request was made, R10 did not appear again, even -though from=9 was specified. If you know the token, you already have the -data. - -Responses for pagination-capable endpoints SHOULD have a `chunk` array -alongside the applicable stream tokens to represent the result set. - -In general, when the end of a result set is reached the applicable -stream token will be excluded from the response. For example, if a user -was backwards-paginating events in a room they'd eventually reach the -first event in the room. In this scenario, the `prev_batch` token would -be excluded from the response. Some paginated endpoints are open-ended -in one direction, such as endpoints which expose an event stream for an -active room. In this case, it is not possible for the client to reach -the true "end" of the data set and therefore should always be presented -with a token to keep moving forwards. - ## Filtering Filters can be created on the server and can be passed as a parameter to diff --git a/content/server-server-api.md b/content/server-server-api.md index 9013d6c0f..9c6552ab2 100644 --- a/content/server-server-api.md +++ b/content/server-server-api.md @@ -134,6 +134,15 @@ to send. The process overall is as follows: 8448 and a `Host` header containing the ``. The target server must present a valid certificate for ``. +{{% boxes/note %}} +The reasons we require `` rather than `` for SRV +delegation are: + 1. DNS is insecure (not all domains have DNSSEC), so the target of the delegation + must prove that it is a valid delegate for `` via TLS. + 2. Consistency with the recommendations in [RFC6125](https://datatracker.ietf.org/doc/html/rfc6125#section-6.2.1) + and other applications using SRV records such [XMPP](https://datatracker.ietf.org/doc/html/rfc6120#section-13.7.2.1). +{{% /boxes/note %}} + The TLS certificate provided by the target server must be signed by a known Certificate Authority. Servers are ultimately responsible for determining the trusted Certificate Authorities, however are strongly @@ -324,15 +333,16 @@ after all other known events. For example, consider a room whose events form the DAG shown below. A server creating a new event in this room should populate the new event's -`prev_events` field with `E4` and `E5`, since neither event yet has a -child: +`prev_events` field with both `E4` and `E6`, since neither event yet has +a child: E1 ^ | - +-> E2 <-+ - | | - E3 E5 + E2 <--- E5 + ^ ^ + | | + E3 E6 ^ | E4 diff --git a/data/api/client-server/administrative_contact.yaml b/data/api/client-server/administrative_contact.yaml index 9966015b0..e6e1267b8 100644 --- a/data/api/client-server/administrative_contact.yaml +++ b/data/api/client-server/administrative_contact.yaml @@ -87,7 +87,7 @@ paths: associated the third party identifier with the user. required: ['medium', 'address', 'validated_at', 'added_at'] tags: - - User data + - Account management post: summary: Adds contact information to the user's account. description: |- @@ -154,6 +154,7 @@ paths: properties: submit_url: type: string + format: uri description: |- An optional field containing a URL where the client must submit the validation token to, with identical parameters @@ -178,7 +179,7 @@ paths: schema: "$ref": "definitions/errors/error.yaml" tags: - - User data + - Account management "/account/3pid/add": post: summary: Adds contact information to the user's account. @@ -234,7 +235,7 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management "/account/3pid/bind": post: summary: Binds a 3PID to the user's account through an Identity Service. @@ -288,7 +289,7 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management "/account/3pid/delete": post: summary: Deletes a third party identifier from the user's account @@ -345,7 +346,7 @@ paths: description: |- An indicator as to whether or not the homeserver was able to unbind the 3PID from the identity server. `success` indicates that the - indentity server has unbound the identifier whereas `no-support` + identity server has unbound the identifier whereas `no-support` indicates that the identity server refuses to support the request or the homeserver was not able to determine an identity server to unbind from. @@ -353,7 +354,7 @@ paths: required: - id_server_unbind_result tags: - - User data + - Account management "/account/3pid/unbind": post: summary: Removes a user's third party identifier from an identity server. @@ -417,7 +418,7 @@ paths: required: - id_server_unbind_result tags: - - User data + - Account management "/account/3pid/email/requestToken": post: summary: Begins the validation process for an email address for association with the user's account. @@ -469,6 +470,8 @@ paths: "errcode": "M_THREEPID_IN_USE", "error": "Third party identifier already in use" } + tags: + - Account management "/account/3pid/msisdn/requestToken": post: summary: Begins the validation process for a phone number for association with the user's account. @@ -517,3 +520,5 @@ paths: "errcode": "M_THREEPID_IN_USE", "error": "Third party identifier already in use" } + tags: + - Account management diff --git a/data/api/client-server/content-repo.yaml b/data/api/client-server/content-repo.yaml index c36fa837d..fb91c7c53 100644 --- a/data/api/client-server/content-repo.yaml +++ b/data/api/client-server/content-repo.yaml @@ -66,6 +66,7 @@ paths: properties: content_uri: type: string + format: uri description: "The [MXC URI](/client-server-api/#matrix-content-mxc-uris) to the uploaded content." examples: application/json: { @@ -360,6 +361,7 @@ paths: parameters: - in: query type: string + format: uri x-example: "https://matrix.org" name: url description: "The URL to get a preview of." @@ -389,6 +391,7 @@ paths: The byte-size of the image. Omitted if there is no image attached. "og:image": type: string + format: uri description: |- An [MXC URI](/client-server-api/#matrix-content-mxc-uris) to the image. Omitted if there is no image. examples: diff --git a/data/api/client-server/create_room.yaml b/data/api/client-server/create_room.yaml index fc558c03d..207c86202 100644 --- a/data/api/client-server/create_room.yaml +++ b/data/api/client-server/create_room.yaml @@ -48,16 +48,18 @@ paths: (and not other members) permission to send state events. Overridden by the `power_level_content_override` parameter. - 4. Events set by the `preset`. Currently these are the `m.room.join_rules`, + 4. An `m.room.canonical_alias` event if `room_alias_name` is given. + + 5. Events set by the `preset`. Currently these are the `m.room.join_rules`, `m.room.history_visibility`, and `m.room.guest_access` state events. - 5. Events listed in `initial_state`, in the order that they are + 6. Events listed in `initial_state`, in the order that they are listed. - 6. Events implied by `name` and `topic` (`m.room.name` and `m.room.topic` + 7. Events implied by `name` and `topic` (`m.room.name` and `m.room.topic` state events). - 7. Invite events implied by `invite` and `invite_3pid` (`m.room.member` with + 8. Invite events implied by `invite` and `invite_3pid` (`m.room.member` with `membership: invite` and `m.room.third_party_invite`). The available presets do the following with respect to room state: @@ -112,7 +114,8 @@ paths: would be `#foo:example.com`. The complete room alias will become the canonical alias for - the room. + the room and an `m.room.canonical_alias` event will be sent + into the room. name: type: string description: |- @@ -171,7 +174,7 @@ paths: type: object description: |- Extra keys, such as `m.federate`, to be added to the content - of the [`m.room.create`](client-server-api/#mroomcreate) event. The server will clobber the following + of the [`m.room.create`](/client-server-api/#mroomcreate) event. The server will clobber the following keys: `creator`, `room_version`. Future versions of the specification may allow the server to clobber other keys. initial_state: @@ -221,7 +224,7 @@ paths: description: |- The power level content to override in the default power level event. This object is applied on top of the generated - [`m.room.power_levels`](client-server-api/#mroompower_levels) + [`m.room.power_levels`](/client-server-api/#mroompower_levels) event content prior to it being sent to the room. Defaults to overriding nothing. responses: diff --git a/data/api/client-server/cross_signing.yaml b/data/api/client-server/cross_signing.yaml index a0516ef2c..3c17cd116 100644 --- a/data/api/client-server/cross_signing.yaml +++ b/data/api/client-server/cross_signing.yaml @@ -42,6 +42,7 @@ paths: name: keys description: |- The keys to be published. + required: true schema: type: object properties: @@ -66,6 +67,12 @@ paths: request. allOf: - $ref: definitions/cross_signing_key.yaml + auth: + description: |- + Additional authentication information for the + user-interactive authentication API. + allOf: + - $ref: "definitions/auth_data.yaml" example: { "master_key": { "user_id": "@alice:example.com", @@ -129,6 +136,8 @@ paths: "errcode": "M_FORBIDDEN", "error": "Key ID in use" } + tags: + - End-to-end encryption "/keys/signatures/upload": post: summary: Upload cross-signing signatures. @@ -143,6 +152,7 @@ paths: name: signatures description: |- The signatures to be published. + required: true schema: type: object title: Signatures @@ -222,3 +232,5 @@ paths: } } } + tags: + - End-to-end encryption diff --git a/data/api/client-server/definitions/errors/error.yaml b/data/api/client-server/definitions/errors/error.yaml index 7471da6f6..89a0d44f9 100644 --- a/data/api/client-server/definitions/errors/error.yaml +++ b/data/api/client-server/definitions/errors/error.yaml @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. type: object +title: Error description: A Matrix-level Error properties: errcode: @@ -22,4 +23,4 @@ properties: type: string description: A human-readable error message. example: An unknown error occurred -required: ["errcode"] \ No newline at end of file +required: ["errcode"] diff --git a/data/api/client-server/definitions/errors/rate_limited.yaml b/data/api/client-server/definitions/errors/rate_limited.yaml index aca82ce7b..1530458ba 100644 --- a/data/api/client-server/definitions/errors/rate_limited.yaml +++ b/data/api/client-server/definitions/errors/rate_limited.yaml @@ -13,6 +13,7 @@ # limitations under the License. $ref: error.yaml type: object +title: RateLimitError description: The rate limit was reached for this request properties: errcode: @@ -29,4 +30,4 @@ properties: The amount of time in milliseconds the client should wait before trying the request again. example: 2000 -required: ["errcode"] \ No newline at end of file +required: ["errcode"] diff --git a/data/api/client-server/definitions/event_batch.yaml b/data/api/client-server/definitions/event_batch.yaml index 1694652ca..e326184c5 100644 --- a/data/api/client-server/definitions/event_batch.yaml +++ b/data/api/client-server/definitions/event_batch.yaml @@ -21,3 +21,4 @@ properties: type: object type: array type: object +title: EventBatch diff --git a/data/api/client-server/definitions/openid_token.yaml b/data/api/client-server/definitions/openid_token.yaml index 32929ef6f..e74ddfff5 100644 --- a/data/api/client-server/definitions/openid_token.yaml +++ b/data/api/client-server/definitions/openid_token.yaml @@ -13,6 +13,7 @@ # See the License for the specific language governing permissions and # limitations under the License. type: object +title: OpenIdCredentials properties: access_token: type: string diff --git a/data/api/client-server/definitions/public_rooms_response.yaml b/data/api/client-server/definitions/public_rooms_response.yaml index 8cf9935c2..406f83b0e 100644 --- a/data/api/client-server/definitions/public_rooms_response.yaml +++ b/data/api/client-server/definitions/public_rooms_response.yaml @@ -68,6 +68,7 @@ properties: rules like any other user. avatar_url: type: string + format: uri description: The URL for the room's avatar, if one is set. join_rule: type: string @@ -110,4 +111,4 @@ example: { "next_batch": "p190q", "prev_batch": "p1902", "total_room_count_estimate": 115 -} \ No newline at end of file +} diff --git a/data/api/client-server/definitions/request_token_response.yaml b/data/api/client-server/definitions/request_token_response.yaml index b801af984..0f8c8cab9 100644 --- a/data/api/client-server/definitions/request_token_response.yaml +++ b/data/api/client-server/definitions/request_token_response.yaml @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. type: object +title: RequestTokenResponse properties: sid: type: string @@ -22,6 +23,7 @@ properties: example: "123abc" submit_url: type: string + format: uri description: |- An optional field containing a URL where the client must submit the validation token to, with identical parameters to the Identity Service diff --git a/data/api/client-server/definitions/room_event_batch.yaml b/data/api/client-server/definitions/room_event_batch.yaml index 45bcb4231..4a7dab193 100644 --- a/data/api/client-server/definitions/room_event_batch.yaml +++ b/data/api/client-server/definitions/room_event_batch.yaml @@ -25,3 +25,4 @@ properties: - origin_server_ts type: array type: object +title: RoomEventBatch diff --git a/data/api/client-server/definitions/state_event_batch.yaml b/data/api/client-server/definitions/state_event_batch.yaml index 9518300fb..f3966dcda 100644 --- a/data/api/client-server/definitions/state_event_batch.yaml +++ b/data/api/client-server/definitions/state_event_batch.yaml @@ -26,3 +26,4 @@ properties: - state_key type: array type: object +title: StateEventBatch diff --git a/data/api/client-server/definitions/timeline_batch.yaml b/data/api/client-server/definitions/timeline_batch.yaml index e243b485f..356325628 100644 --- a/data/api/client-server/definitions/timeline_batch.yaml +++ b/data/api/client-server/definitions/timeline_batch.yaml @@ -21,6 +21,11 @@ properties: type: boolean prev_batch: description: A token that can be supplied to the `from` parameter of the - rooms/{roomId}/messages endpoint. + [`/rooms//messages`](#get_matrixclientr0roomsroomidmessages) + endpoint in order to retrieve earlier events. + + If no earlier events are available, this property may be omitted from + the response. type: string type: object +title: TimelineBatch diff --git a/data/api/client-server/definitions/wellknown/homeserver.yaml b/data/api/client-server/definitions/wellknown/homeserver.yaml index 92ff34ed3..6f6d3cb9a 100644 --- a/data/api/client-server/definitions/wellknown/homeserver.yaml +++ b/data/api/client-server/definitions/wellknown/homeserver.yaml @@ -18,6 +18,7 @@ type: object properties: base_url: type: string + format: uri description: The base URL for the homeserver for client-server connections. example: https://matrix.example.com required: diff --git a/data/api/client-server/definitions/wellknown/identity_server.yaml b/data/api/client-server/definitions/wellknown/identity_server.yaml index a8f7c31cf..a86ad8b1b 100644 --- a/data/api/client-server/definitions/wellknown/identity_server.yaml +++ b/data/api/client-server/definitions/wellknown/identity_server.yaml @@ -18,6 +18,7 @@ type: object properties: base_url: type: string + format: uri description: The base URL for the identity server for client-server connections. example: https://identity.example.com required: diff --git a/data/api/client-server/key_backup.yaml b/data/api/client-server/key_backup.yaml index 7c42f6b53..438517913 100644 --- a/data/api/client-server/key_backup.yaml +++ b/data/api/client-server/key_backup.yaml @@ -375,7 +375,7 @@ paths: summary: Store a key in the backup. description: |- Store a key in the backup. - operationId: postRoomKeysKeyRoomIdSessionId + operationId: putRoomKeysBySessionId security: - accessToken: [] parameters: @@ -409,6 +409,7 @@ paths: description: The update succeeded. schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -443,10 +444,10 @@ paths: tags: - End-to-end encryption get: - summary: Retrieve a key from the backup + summary: Retrieve a key from the backup. description: |- Retrieve a key from the backup. - operationId: getRoomKeysKeyRoomIdSessionId + operationId: getRoomKeysBySessionId security: - accessToken: [] parameters: @@ -487,11 +488,13 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption delete: - summary: Delete a key from the backup + summary: Delete a key from the backup. description: |- Delete a key from the backup. - operationId: deleteRoomKeysKeyRoomIdSessionId + operationId: deleteRoomKeysBySessionId security: - accessToken: [] parameters: @@ -519,6 +522,7 @@ paths: description: The update succeeded schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -547,12 +551,14 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption "/room_keys/keys/{roomId}": put: summary: Store several keys in the backup for a given room. description: |- - Store a key in the backup. - operationId: postRoomKeysKeyRoomId + Store several keys in the backup for a given room. + operationId: putRoomKeysByRoomId security: - accessToken: [] parameters: @@ -580,6 +586,7 @@ paths: description: The update succeeded schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -624,10 +631,10 @@ paths: tags: - End-to-end encryption get: - summary: Retrieve the keys from the backup for a given room + summary: Retrieve the keys from the backup for a given room. description: |- - Retrieve the keys from the backup for a given room - operationId: getRoomKeysKeyRoomId + Retrieve the keys from the backup for a given room. + operationId: getRoomKeysByRoomId security: - accessToken: [] parameters: @@ -666,11 +673,13 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption delete: - summary: Delete a key from the backup + summary: Delete the keys from the backup for a given room. description: |- - Delete a key from the backup. - operationId: deleteRoomKeysKeyRoomId + Delete the keys from the backup for a given room. + operationId: deleteRoomKeysByRoomId security: - accessToken: [] parameters: @@ -692,6 +701,7 @@ paths: description: The update succeeded schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -720,12 +730,14 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption "/room_keys/keys": put: summary: Store several keys in the backup. description: |- Store several keys in the backup. - operationId: postRoomKeysKey + operationId: putRoomKeys security: - accessToken: [] parameters: @@ -773,6 +785,7 @@ paths: description: The update succeeded schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -817,10 +830,10 @@ paths: tags: - End-to-end encryption get: - summary: Retrieve the keys from the backup for a given room + summary: Retrieve the keys from the backup. description: |- - Retrieve the keys from the backup for a given room - operationId: getRoomKeysKeyRoomId + Retrieve the keys from the backup. + operationId: getRoomKeys security: - accessToken: [] parameters: @@ -875,11 +888,13 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption delete: - summary: Delete a key from the backup + summary: Delete the keys from the backup. description: |- - Delete a key from the backup. - operationId: deleteRoomKeysKeyRoomId + Delete the keys from the backup. + operationId: deleteRoomKeys security: - accessToken: [] parameters: @@ -895,6 +910,7 @@ paths: description: The update succeeded schema: type: object + title: RoomKeysUpdateResponse properties: etag: description: |- @@ -923,3 +939,5 @@ paths: description: This request was rate-limited. schema: "$ref": "definitions/errors/rate_limited.yaml" + tags: + - End-to-end encryption diff --git a/data/api/client-server/list_public_rooms.yaml b/data/api/client-server/list_public_rooms.yaml index 4175b0a96..87806e04a 100644 --- a/data/api/client-server/list_public_rooms.yaml +++ b/data/api/client-server/list_public_rooms.yaml @@ -61,6 +61,8 @@ paths: } schema: "$ref": "definitions/errors/error.yaml" + tags: + - Room discovery put: summary: Sets the visibility of a room in the room directory description: |- @@ -111,6 +113,8 @@ paths: } schema: "$ref": "definitions/errors/error.yaml" + tags: + - Room discovery "/publicRooms": get: summary: Lists the public rooms on the server. diff --git a/data/api/client-server/message_pagination.yaml b/data/api/client-server/message_pagination.yaml index 5b86ce901..c3c8d8882 100644 --- a/data/api/client-server/message_pagination.yaml +++ b/data/api/client-server/message_pagination.yaml @@ -51,9 +51,12 @@ paths: name: from description: |- The token to start returning events from. This token can be obtained - from a `prev_batch` token returned for each room by the sync API, - or from a `start` or `end` token returned by a previous request - to this endpoint. + from a `prev_batch` or `next_batch` token returned by the `/sync` endpoint, + or from an `end` token returned by a previous request to this endpoint. + + This endpoint can also accept a value returned as a `start` token + by a previous request to this endpoint, though servers are not + required to support this. Clients should not rely on the behaviour. required: true x-example: "s345_678_333" - in: query @@ -61,16 +64,18 @@ paths: name: to description: |- The token to stop returning events at. This token can be obtained from - a `prev_batch` token returned for each room by the sync endpoint, - or from a `start` or `end` token returned by a previous request to - this endpoint. + a `prev_batch` or `next_batch` token returned by the `/sync` endpoint, + or from an `end` token returned by a previous request to this endpoint. required: false - in: query type: string enum: ["b", "f"] name: dir description: |- - The direction to return events from. + The direction to return events from. If this is set to `f`, events + will be returned in chronological order starting at `from`. If it + is set to `b`, events will be returned in *reverse* chronolgical + order, again starting at `from`. required: true x-example: "b" - in: query @@ -96,20 +101,29 @@ paths: start: type: string description: |- - The token the pagination starts from. If `dir=b` this will be - the token supplied in `from`. + A token corresponding to the start of `chunk`. This will be the same as + the value given in `from`. end: type: string description: |- - The token the pagination ends at. If `dir=b` this token should - be used again to request even earlier events. + A token corresponding to the end of `chunk`. This token can be passed + back to this endpoint to request further events. + + If no further events are available (either because we have + reached the start of the timeline, or because the user does + not have permission to see any more events), this property + is omitted from the response. chunk: type: array description: |- A list of room events. The order depends on the `dir` parameter. For `dir=b` events will be in reverse-chronological order, - for `dir=f` in chronological order, so that events start - at the `from` point. + for `dir=f` in chronological order. (The exact definition of `chronological` + is dependent on the server implementation.) + + Note that an empty `chunk` does not *necessarily* imply that no more events + are available. Clients should continue to paginate until no `end` property + is returned. items: "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" state: @@ -125,6 +139,7 @@ paths: the membership of those members has not changed. items: $ref: "../../event-schemas/schema/core-event-schema/state_event.yaml" + required: [start, chunk] examples: application/json: { "start": "t47429-4392820_219380_26003_2265", diff --git a/data/api/client-server/notifications.yaml b/data/api/client-server/notifications.yaml index 7d0174037..0176a9b4d 100644 --- a/data/api/client-server/notifications.yaml +++ b/data/api/client-server/notifications.yaml @@ -41,7 +41,9 @@ paths: - in: query type: string name: from - description: Pagination token given to retrieve the next set of events. + description: |- + Pagination token to continue from. This should be the `next_token` + returned from an earlier call to this endpoint. required: false x-example: "xxxxx" - in: query diff --git a/data/api/client-server/old_sync.yaml b/data/api/client-server/old_sync.yaml index 522f0b1a4..d961077be 100644 --- a/data/api/client-server/old_sync.yaml +++ b/data/api/client-server/old_sync.yaml @@ -36,7 +36,7 @@ paths: This endpoint was deprecated in r0 of this specification. Clients should instead call the [`/sync`](/client-server-api/#get_matrixclientr0sync) - API with a `since` parameter. See + endpoint with a `since` parameter. See the [migration guide](https://matrix.org/docs/guides/migrating-from-client-server-api-v-1#deprecated-endpoints). operationId: getEvents security: @@ -73,12 +73,12 @@ paths: start: type: string description: |- - A token which correlates to the first value in `chunk`. This + A token which correlates to the start of `chunk`. This is usually the same token supplied to `from=`. end: type: string description: |- - A token which correlates to the last value in `chunk`. This + A token which correlates to the end of `chunk`. This token should be used in the next request to `/events`. chunk: type: array @@ -102,7 +102,7 @@ paths: This endpoint was deprecated in r0 of this specification. Clients should instead call the [`/sync`](/client-server-api/#get_matrixclientr0sync) - API with no `since` parameter. See + endpoint with no `since` parameter. See the [migration guide](https://matrix.org/docs/guides/migrating-from-client-server-api-v-1#deprecated-endpoints). operationId: initialSync security: @@ -199,8 +199,8 @@ paths: end: type: string description: |- - A token which correlates to the last value in `chunk`. This - token should be used with the `/events` API to listen for new + A token which correlates to the end of the timelines returned. This + token should be used with the `/events` endpoint to listen for new events. presence: type: array @@ -237,13 +237,20 @@ paths: start: type: string description: |- - A token which correlates to the first value in `chunk`. - Used for pagination. + A token which correlates to the start of `chunk`. + Can be passed to + [`/rooms//messages`](#get_matrixclientr0roomsroomidmessages) + to retrieve earlier events. + + If no earlier events are available, this property may be omitted from + the response. end: type: string description: |- - A token which correlates to the last value in `chunk`. - Used for pagination. + A token which correlates to the end of `chunk`. + Can be passed to + [`/rooms//messages`](#get_matrixclientr0roomsroomidmessages) + to retrieve later events. chunk: type: array description: |- @@ -257,7 +264,7 @@ paths: title: RoomEvent allOf: - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" - required: ["start", "end", "chunk"] + required: ["end", "chunk"] state: type: array description: |- diff --git a/data/api/client-server/peeking_events.yaml b/data/api/client-server/peeking_events.yaml index 85eb87683..0bf303ef1 100644 --- a/data/api/client-server/peeking_events.yaml +++ b/data/api/client-server/peeking_events.yaml @@ -27,9 +27,11 @@ produces: securityDefinitions: $ref: definitions/security.yaml paths: - "/events": + # With an extra " " to disambiguate from the getEvents endpoint + # The extra space makes it sort first for what I'm sure is a good reason. + "/events ": get: - summary: Listen on the event stream. + summary: Listen on the event stream of a particular room. description: |- This will listen for new events related to a particular room and return them to the caller. This will block until an event is received, or until @@ -103,4 +105,5 @@ paths: - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" 400: description: "Bad pagination `from` parameter." - # No tags to exclude this from the swagger UI - use the normal version instead. + tags: + - Room participation diff --git a/data/api/client-server/profile.yaml b/data/api/client-server/profile.yaml index 6a8d6a775..d1d4306cf 100644 --- a/data/api/client-server/profile.yaml +++ b/data/api/client-server/profile.yaml @@ -129,6 +129,7 @@ paths: properties: avatar_url: type: string + format: uri description: The new avatar URL for this user. responses: 200: @@ -170,6 +171,7 @@ paths: properties: avatar_url: type: string + format: uri description: The user's avatar URL if they have set one, otherwise not present. 404: description: There is no avatar URL for this user or this user does not exist. @@ -204,6 +206,7 @@ paths: properties: avatar_url: type: string + format: uri description: The user's avatar URL if they have set one, otherwise not present. displayname: type: string diff --git a/data/api/client-server/pusher.yaml b/data/api/client-server/pusher.yaml index 69882066f..9feccb4f0 100644 --- a/data/api/client-server/pusher.yaml +++ b/data/api/client-server/pusher.yaml @@ -113,6 +113,7 @@ paths: properties: url: type: string + format: uri description: |- Required if `kind` is `http`. The URL to use to send notifications to. @@ -220,6 +221,7 @@ paths: properties: url: type: string + format: uri description: |- Required if `kind` is `http`. The URL to use to send notifications to. MUST be an HTTPS URL with a path of diff --git a/data/api/client-server/receipts.yaml b/data/api/client-server/receipts.yaml index 716946ace..66610092b 100644 --- a/data/api/client-server/receipts.yaml +++ b/data/api/client-server/receipts.yaml @@ -61,6 +61,7 @@ paths: description: |- Extra receipt information to attach to `content` if any. The server will automatically set the `ts` field. + required: true schema: type: object example: { diff --git a/data/api/client-server/registration.yaml b/data/api/client-server/registration.yaml index 925ad00b8..526233fa4 100644 --- a/data/api/client-server/registration.yaml +++ b/data/api/client-server/registration.yaml @@ -214,7 +214,7 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management "/register/email/requestToken": post: summary: Begins the validation process for an email to be used during registration. @@ -265,6 +265,8 @@ paths: } schema: "$ref": "definitions/errors/error.yaml" + tags: + - Account management "/register/msisdn/requestToken": post: summary: Requests a validation token be sent to the given phone number for the purpose of registering an account @@ -315,6 +317,8 @@ paths: } schema: "$ref": "definitions/errors/error.yaml" + tags: + - Account management "/account/password": post: summary: "Changes a user's password." @@ -378,7 +382,7 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management "/account/password/email/requestToken": post: summary: Requests a validation token be sent to the given email address for the purpose of resetting a user's password @@ -434,6 +438,8 @@ paths: "errcode": "M_THREEPID_NOT_FOUND", "error": "Email not found" } + tags: + - Account management "/account/password/msisdn/requestToken": post: summary: Requests a validation token be sent to the given phone number for the purpose of resetting a user's password. @@ -489,6 +495,8 @@ paths: "errcode": "M_THREEPID_NOT_FOUND", "error": "Phone number not found" } + tags: + - Account management "/account/deactivate": post: summary: "Deactivate a user's account." @@ -565,7 +573,7 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management "/register/available": get: summary: Checks to see if a username is available on the server. @@ -627,4 +635,4 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Account management diff --git a/data/api/client-server/report_content.yaml b/data/api/client-server/report_content.yaml index 380995ead..19d6b6d0c 100644 --- a/data/api/client-server/report_content.yaml +++ b/data/api/client-server/report_content.yaml @@ -49,6 +49,7 @@ paths: x-example: "$something:example.org" - in: body name: body + required: true schema: type: object example: { diff --git a/data/api/client-server/room_initial_sync.yaml b/data/api/client-server/room_initial_sync.yaml index bab0e3d92..f8660e69f 100644 --- a/data/api/client-server/room_initial_sync.yaml +++ b/data/api/client-server/room_initial_sync.yaml @@ -98,13 +98,18 @@ paths: start: type: string description: |- - A token which correlates to the first value in `chunk`. - Used for pagination. + A token which correlates to the start of `chunk`. Can be passed to + [`/rooms//messages`](#get_matrixclientr0roomsroomidmessages) + to retrieve earlier events. + + If no earlier events are available, this property may be omitted from + the response. end: type: string description: |- - A token which correlates to the last value in `chunk`. - Used for pagination. + A token which correlates to the end of `chunk`. Can be passed to + [`/rooms//messages`](#get_matrixclientr0roomsroomidmessages) + to retrieve later events. chunk: type: array description: |- @@ -118,7 +123,7 @@ paths: title: RoomEvent allOf: - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" - required: ["start", "end", "chunk"] + required: ["end", "chunk"] state: type: array description: |- diff --git a/data/api/client-server/room_send.yaml b/data/api/client-server/room_send.yaml index c22fcfe47..18658f612 100644 --- a/data/api/client-server/room_send.yaml +++ b/data/api/client-server/room_send.yaml @@ -65,6 +65,7 @@ paths: x-example: "35" - in: body name: body + required: true schema: type: object example: { diff --git a/data/api/client-server/room_state.yaml b/data/api/client-server/room_state.yaml index 381c5c57e..016a82736 100644 --- a/data/api/client-server/room_state.yaml +++ b/data/api/client-server/room_state.yaml @@ -74,6 +74,7 @@ paths: x-example: "@alice:example.com" - in: body name: body + required: true schema: type: object example: { diff --git a/data/api/client-server/rooms.yaml b/data/api/client-server/rooms.yaml index 0dd60d8e6..2f161e2bf 100644 --- a/data/api/client-server/rooms.yaml +++ b/data/api/client-server/rooms.yaml @@ -206,6 +206,7 @@ paths: enum: - join - invite + - knock - leave - ban description: |- @@ -220,6 +221,7 @@ paths: enum: - join - invite + - knock - leave - ban description: |- @@ -301,6 +303,7 @@ paths: description: The display name of the user this object is representing. avatar_url: type: string + format: uri description: The mxc avatar url of the user this object is representing. description: A map from user ID to a RoomMember object. type: object diff --git a/data/api/client-server/search.yaml b/data/api/client-server/search.yaml index 1dc9804e2..8bc255286 100644 --- a/data/api/client-server/search.yaml +++ b/data/api/client-server/search.yaml @@ -238,6 +238,7 @@ paths: title: Display name avatar_url: type: string + format: uri title: Avatar Url events_before: type: array diff --git a/data/api/client-server/sso_login_redirect.yaml b/data/api/client-server/sso_login_redirect.yaml index 8c92a3ce6..b58d88fe4 100644 --- a/data/api/client-server/sso_login_redirect.yaml +++ b/data/api/client-server/sso_login_redirect.yaml @@ -46,6 +46,8 @@ paths: headers: Location: type: "string" + tags: + - Session management "/login/sso/redirect/{idpId}": get: summary: Redirect the user's browser to the SSO interface for an IdP. @@ -56,7 +58,7 @@ paths: The server MUST respond with an HTTP redirect to the SSO interface for that IdP. - operationId: redirectToSSO + operationId: redirectToIdP parameters: - in: path type: string @@ -83,3 +85,5 @@ paths: The IdP ID was not recognized by the server. The server is encouraged to provide a user-friendly page explaining the error given the user will be navigated to it. + tags: + - Session management diff --git a/data/api/client-server/sync.yaml b/data/api/client-server/sync.yaml index 140843a0c..53c521737 100644 --- a/data/api/client-server/sync.yaml +++ b/data/api/client-server/sync.yaml @@ -73,7 +73,8 @@ paths: name: since type: string description: |- - A point in time to continue a sync from. + A point in time to continue a sync from. This should be the + `next_batch` token returned by an earlier call to this endpoint. x-example: "s72594_4483_1934" - in: query name: full_state diff --git a/data/api/client-server/third_party_lookup.yaml b/data/api/client-server/third_party_lookup.yaml index 3d348df2f..6f8829655 100644 --- a/data/api/client-server/third_party_lookup.yaml +++ b/data/api/client-server/third_party_lookup.yaml @@ -42,6 +42,8 @@ paths: description: The protocols supported by the homeserver. schema: $ref: ../application-service/definitions/protocol_metadata.yaml + tags: + - Third Party Lookup "/thirdparty/protocol/{protocol}": get: summary: Retrieve metadata about a specific protocol that the homeserver supports. @@ -71,6 +73,8 @@ paths: } schema: $ref: definitions/errors/error.yaml + tags: + - Third Party Lookup "/thirdparty/location/{protocol}": get: summary: Retrieve Matrix-side portals rooms leading to a third party location. @@ -112,6 +116,8 @@ paths: } schema: $ref: definitions/errors/error.yaml + tags: + - Third Party Lookup "/thirdparty/user/{protocol}": get: summary: Retrieve the Matrix User ID of a corresponding third party user. @@ -147,6 +153,8 @@ paths: } schema: $ref: definitions/errors/error.yaml + tags: + - Third Party Lookup "/thirdparty/location": get: summary: Reverse-lookup third party locations given a Matrix room alias. @@ -177,6 +185,8 @@ paths: } schema: $ref: definitions/errors/error.yaml + tags: + - Third Party Lookup "/thirdparty/user": get: summary: Reverse-lookup third party users given a Matrix User ID. @@ -206,3 +216,5 @@ paths: } schema: $ref: definitions/errors/error.yaml + tags: + - Third Party Lookup diff --git a/data/api/client-server/users.yaml b/data/api/client-server/users.yaml index 255c538b1..ae015e7a5 100644 --- a/data/api/client-server/users.yaml +++ b/data/api/client-server/users.yaml @@ -47,6 +47,7 @@ paths: parameters: - in: body name: body + required: true schema: type: object properties: @@ -95,6 +96,7 @@ paths: description: The display name of the user, if one exists. avatar_url: type: string + format: uri example: "mxc://bar.com/foo" description: The avatar url, as an MXC, if one exists. limited: @@ -105,4 +107,4 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - User directory diff --git a/data/api/client-server/whoami.yaml b/data/api/client-server/whoami.yaml index 3458c4d0d..54d9cde90 100644 --- a/data/api/client-server/whoami.yaml +++ b/data/api/client-server/whoami.yaml @@ -89,4 +89,4 @@ paths: schema: "$ref": "definitions/errors/rate_limited.yaml" tags: - - User data + - Session management diff --git a/data/api/server-server/user_devices.yaml b/data/api/server-server/user_devices.yaml index 0c2348dab..9fb15777f 100644 --- a/data/api/server-server/user_devices.yaml +++ b/data/api/server-server/user_devices.yaml @@ -96,7 +96,7 @@ paths: "ed25519:base64+master+public+key": "base64+master+public+key", } } - self_signing_keys: + self_signing_key: type: object description: |- The user\'s self-signing key. diff --git a/data/event-schemas/schema/m.room.member.yaml b/data/event-schemas/schema/m.room.member.yaml index 928e1528f..37975cbe1 100644 --- a/data/event-schemas/schema/m.room.member.yaml +++ b/data/event-schemas/schema/m.room.member.yaml @@ -45,6 +45,7 @@ properties: avatar_url: description: 'The avatar URL for this user, if any.' type: string + format: uri displayname: description: 'The display name for this user, if any.' type: diff --git a/data/event-schemas/schema/m.room.tombstone.yaml b/data/event-schemas/schema/m.room.tombstone.yaml index 0fd8ba452..1c590e1fd 100644 --- a/data/event-schemas/schema/m.room.tombstone.yaml +++ b/data/event-schemas/schema/m.room.tombstone.yaml @@ -10,7 +10,7 @@ properties: description: A server-defined message. replacement_room: type: string - description: The new room the client should be visiting. + description: The room ID of the new room the client should be visiting. required: - replacement_room - body diff --git a/layouts/partials/navbar.html b/layouts/partials/navbar.html index 8a7364401..e78ee9c41 100644 --- a/layouts/partials/navbar.html +++ b/layouts/partials/navbar.html @@ -45,8 +45,10 @@ {{ $status := .Site.Params.version.status }} {{ if ne $status "unstable"}} - {{ $ret = .Site.Params.version.number }} - {{ $ret = delimit (slice "version" $ret) " " }} + {{ $path := path.Join "changelogs" }} + + {{/* produces a string similar to "version v1.5" */}} + {{ $ret = delimit (slice "version v" .Site.Params.version.major "." .Site.Params.version.minor) "" }} {{ end }} {{ return $ret }} diff --git a/layouts/shortcodes/changelog/changelog-changes.html b/layouts/shortcodes/changelog/changelog-changes.html index a1e277283..11a598580 100644 --- a/layouts/shortcodes/changelog/changelog-changes.html +++ b/layouts/shortcodes/changelog/changelog-changes.html @@ -7,22 +7,26 @@ it expects to find newsfragments describing changes to that API. If the `version.status` setting in config.toml is anything other than - "unstable", then it also expects to find a "release.yaml" file in /changelogs, - which contains: - - `tag`: Git tag for this release - - `date`: date of this release - It then renders this info a table, before the list of changes. + "unstable", then it also expects to find additional settings under + `version` in config.toml: + - `major`: the major version number of the release + - `minor`: the minor version number of the release + - `release_date`: the date of the release + + The release tag is calculated as `v.`; for example `v1.5`. + + It then renders this into a table displayed before the list of changes. */}} {{ $path := path.Join "changelogs" }} {{ $status := .Site.Params.version.status }} +{{ $release_tag := delimit (slice "v" .Site.Params.version.major "." .Site.Params.version.minor) "" }} {{ if ne $status "unstable" }} -{{ $release_info := readFile (path.Join $path "release.yaml") | transform.Unmarshal }} - - + +
Git commithttps://github.com/matrix-org/matrix-doc/tree/{{ $release_info.tag }}
Release date{{ $release_info.date }}
Git commithttps://github.com/matrix-org/matrix-doc/tree/{{ $release_tag }}
Release date{{ .Site.Params.version.release_date }}
{{ end }} diff --git a/layouts/shortcodes/changelog/changelog-description.html b/layouts/shortcodes/changelog/changelog-description.html index 738ff675b..3c7197252 100644 --- a/layouts/shortcodes/changelog/changelog-description.html +++ b/layouts/shortcodes/changelog/changelog-description.html @@ -8,8 +8,12 @@ {{ $status := .Site.Params.version.status }} {{ if eq $status "unstable"}} +

This is the unstable version of the Matrix specification.

This changelog lists changes made since the last release of the specification.

+ {{ else }} -

This is version {{ .Site.Params.version.number }} of the Matrix specification.

+ +

This is version v{{ .Site.Params.version.major }}.{{ .Site.Params.version.minor }} of the Matrix specification.

+ {{ end }} diff --git a/meta/documentation_style.rst b/meta/documentation_style.rst index cfec5e3ba..aea495bd3 100644 --- a/meta/documentation_style.rst +++ b/meta/documentation_style.rst @@ -8,7 +8,7 @@ in. Format ------ -Documentation is written in github-flavored markdown. +Documentation is written in Commonmark markdown. Sections -------- @@ -66,6 +66,12 @@ Lists should: * Be used where they provide clarity. * Contain entries which start with a capital and end with a full stop. +When talking about properties in JSON objects, prefer the word "property" to "field", +"member", or various other alternatives. For example: "this property will be set to +X if ...". Also avoid the term "key" unless you are specifically talking about the +*name* of a property - and be mindful of the scope for confusion with cryptographic +keys. + OpenAPI ~~~~~~~ diff --git a/package-lock.json b/package-lock.json index b41f83d66..73994715f 100644 --- a/package-lock.json +++ b/package-lock.json @@ -97,22 +97,10 @@ "postcss-value-parser": "^4.1.0" }, "dependencies": { - "postcss": { - "version": "7.0.32", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-7.0.32.tgz", - "integrity": "sha512-03eXong5NLnNCD05xscnGKGDZ98CyzoqPSMjOe6SuoQY7Z2hIj0Ld1g/O/UQRuOle2aRtiIRDg9tDcTGAkLfKw==", - "dev": true, - "requires": { - "chalk": "^2.4.2", - "source-map": "^0.6.1", - "supports-color": "^6.1.0" - } - }, "supports-color": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-6.1.0.tgz", "integrity": "sha512-qe1jfm1Mg7Nq/NSh6XE24gPXROEVsWHxC1LIx//XNlD9iw7YZQGjZNjYN7xGaEG6iKdA8EtNFW6R0gjnVXp+wQ==", - "dev": true, "requires": { "has-flag": "^3.0.0" } @@ -408,9 +396,9 @@ "dev": true }, "glob-parent": { - "version": "5.1.1", - "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.1.tgz", - "integrity": "sha512-FnI+VGOpnlGHWZxthPGR+QhR78fuiK0sNLkHQv+bL9fQi57lNNdquIbna/WrfROrolq8GK5Ek6BiMwqL/voRYQ==", + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", + "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", "dev": true, "requires": { "is-glob": "^4.0.1" @@ -439,8 +427,7 @@ "has-flag": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", - "integrity": "sha1-tdRU3CGZriJWmfNGfloH87lVuv0=", - "dev": true + "integrity": "sha1-tdRU3CGZriJWmfNGfloH87lVuv0=" }, "ignore": { "version": "5.1.8", @@ -673,9 +660,9 @@ "dev": true }, "postcss": { - "version": "7.0.32", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-7.0.32.tgz", - "integrity": "sha512-03eXong5NLnNCD05xscnGKGDZ98CyzoqPSMjOe6SuoQY7Z2hIj0Ld1g/O/UQRuOle2aRtiIRDg9tDcTGAkLfKw==", + "version": "7.0.36", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-7.0.36.tgz", + "integrity": "sha512-BebJSIUMwJHRH0HAQoxN4u1CN86glsrwsW0q7T+/m44eXOUAxSNdHRkNZPYz5vVUbg17hFgOQDE7fZk7li3pZw==", "dev": true, "requires": { "chalk": "^2.4.2", diff --git a/proposals/1767-extensible-events.md b/proposals/1767-extensible-events.md new file mode 100644 index 000000000..b0681b8a3 --- /dev/null +++ b/proposals/1767-extensible-events.md @@ -0,0 +1,410 @@ +# MSC1767: Extensible events in Matrix + +While events are currently JSON blobs which accept additional metadata appended to them, +there is no formal structure for how to represent this information or interpret it on the +client side, particularly in the case of unknown event types. + +When specifying new events, the proposals often reinvent the same wheel instead of reusing +existing blocks or types, such as in cases where captions, thumbnails, etc need to be +considered for an event. This has further issues of clients not knowing how to render these +newly-specified events, leading to mixed compatibility within the ecosystem. + +The above seriously hinders the uptake of new event types (and therefore features) within +the Matrix ecosystem. In the current system, a new event type would be introduced and +all implementations slowly gain support for it - if we instead had reusable types then +clients could automatically support a "good enough" version of that new event type while +"proper" support is written in over time. Such an example could be polls: not every +client will want polls right away, but it would be quite limiting as a user experience +if some users can't even see the question being posed. + +This proposal introduces a structure for how extensible events are represented, using +the existing extensible nature of events today, laying the groundwork for more reusable +blocks of content in future events. + +With text being the simplest form of representation for events today, this MSC also +specifies a relatively basic text schema for room messages that can be reused in other +events. Other building block types are specified by other MSCs: + +* [MSC3954 - Emotes](https://github.com/matrix-org/matrix-doc/pull/3954) +* [MSC3955 - Notices / automated events](https://github.com/matrix-org/matrix-doc/pull/3955) +* [MSC3956 - Encryption](https://github.com/matrix-org/matrix-doc/pull/3956) +* [MSC3927 - Audio](https://github.com/matrix-org/matrix-doc/pull/3927) +* [MSC3551 - Files](https://github.com/matrix-org/matrix-doc/pull/3551) +* [MSC3552 - Images and Stickers](https://github.com/matrix-org/matrix-doc/pull/3552) +* [MSC3553 - Videos](https://github.com/matrix-org/matrix-doc/pull/3553) +* [MSC3554 - Translatable text](https://github.com/matrix-org/matrix-doc/pull/3554) + +Some examples of new features/events using extensible events are: + +* [MSC3488 - Location data](https://github.com/matrix-org/matrix-doc/pull/3488) +* [MSC3381 - Polls](https://github.com/matrix-org/matrix-doc/pull/3381) +* [MSC3245 - Voice messages](https://github.com/matrix-org/matrix-doc/pull/3245) +* [MSC2192 - Inline widgets](https://github.com/matrix-org/matrix-doc/pull/2192) +* [MSC3765 - Rich text topics](https://github.com/matrix-org/matrix-doc/pull/3765) + +**Note**: Readers might find [Andy's blog](https://www.artificialworlds.net/blog/2022/03/08/comparison-of-matrix-events-before-and-after-extensible-events/) +useful for understanding the problem space. Unfortunately, for those who need to +understand the changes to the protocol/specification, the best option is to read +this proposal. + +## Proposal + +In a new room version (why is described later in this proposal), events are declared +to be represented by their extensible form, as described by this MSC. `m.room.message` +is formally deprecated by this MSC, with removal from the specification happening as +part of a room version adopting the feature. Clients are expected to use extensible +events only in rooms versions which explicitly declare such support (in both unstable +and stable settings), except where noted later in this proposal. + +An extensible event is made up of two critical parts: an event type and zero or more +content blocks. The event type defines which content blocks a receiver can expect, +and the content blocks carry the information needed to render the event (whether the +client understands the event type or not). + +Content blocks are simply any top-level key in `content` on the event. They can have +any value type (that is also legal in an event generally: string, integer, etc), and +are namespaced using the +[Matrix conventions for namespacing](https://spec.matrix.org/v1.4/appendices/#common-namespaced-identifier-grammar). + +Content blocks can be invented independent of event types and *should* be reusable +in nature. For example, this proposal introduces an `m.text` content block which +can be reused by other event types to represent textual fallback. + +When a client encounters an extensible event (any event sent in a supported room +version) that it does *not* understand, the client begins searching for a best match +based on event type schemas it *does* know. This may mean combining multiple different +content blocks to match a suitable schema, such as in the case of +[MSC3553](https://github.com/matrix-org/matrix-doc/pull/3553) video events. +Which schemas to try, and in what order, is left as a deliberate implementation detail. +A client might decide to try parsing the event as a video, then image, then file, then +text message, for example. + +It is generally not expected that a single content block will describe an entire event, +except in the exceedingly trivial cases (like text messages in this proposal). Multiple +content blocks will usually fully describe the information in the event, and mixins +(described later) can further change how an event is represented or processed. + +Note that a "client" in an extensible events sense will typically mean an application +using the Client-Server API, however in reality a client will be anything which needs +to parse and understand event contents (servers for some functions like push rules, +application services, etc). + +Per the introduction, text is the baseline format that most/all Matrix clients support +today, often through use of HTML and `m.room.message`. Instead of using `m.room.message` +to represent this content, clients would instead use an `m.message` event with, at +a minimum, a `m.text` content block: + +```json5 +{ + // irrelevant fields not shown + "type": "m.message", + "content": { + "m.text": [ + { "body": "Hello world", "mimetype": "text/html" }, + { "body": "Hello world" } + ] + } +} +``` + +`m.text` has the following definitions associated with it: +* An ordered array of mimetypes and applicable string content to represent a single + marked-up blob of text. Each element is known as a representation. +* `body` in a representation is required, and must be a string. +* `mimetype` is optional in a representation, and defaults to `text/plain`. +* Zero representations are permitted, however senders should aim to always specify + at least one. +* Invalid representations are skipped by clients (missing `body`, not an object, etc). +* The first representation a renderer understands should be used. +* Senders are strongly encouraged to always include a plaintext representation. +* The `mimetype` of a representation determines its `body` - no effort is made to + limit what is allowed in the `body`, however clients are still strongly encouraged + to validate/sanitize the content further, like in the + [existing spec](https://spec.matrix.org/v1.4/client-server-api/#mroommessage-msgtypes) + for HTML. +* Custom text formats in a representation are specified by a suitably custom `mimetype`. + For example, a representation might use a text format extending HTML or XML, or an + all-new markup. This can be used to create bridge-compatible clients where the + destination network's markup is first in the array, followed by more common HTML + and text formats. + +Like with the event described above, all event types now describe which content blocks +they expect to see on their events. These content blocks could be required, as is the +case of `m.text` in `m.message`, or they could be optional depending on the situation. +Of course, senders are welcome to send even more blocks which aren't specified in the +schema for an event type, however clients which understand that event type might not +consider them at all. + +In `m.message`'s case, `m.text` is the only required content block. The `m.text` +block can be reused by other events to include a text-like format for the event, such +as a text fallback for clients which do not understand how to render a custom event +type. + +To reiterate, when a client encounters an unknown event type it first tries to see +if there's a set of content blocks present that it can associate with a known event +type. If it finds suitable content blocks, it parses the event as though the event +were of the known type. If it doesn't find anything useful, the event is left as +unrenderable, just as it likely would today. + +To avoid a situation where events end up being unrenderable, it is strongly +recommended that all event types support at least an `m.text` content block in +their schema, thus allowing all events to theoretically be rendered as message +events (in a worst case scenario). + +For clarity, events are not able to specify *how* they are handled when the receiver +doesn't know how to render the event type: the sender simply includes all possible or +feasible representations for the data, hoping the receiver will pick the richest form +for the user. As an example, a special medical imaging event type might also be +represented as a video, static image, or text (URL to some healthcare platform): the +sender includes all 3 fallbacks by specifying the needed content blocks, and the +receiver may pick the video, image, or text depending on its own rules. + +Events must still only represent a single logical piece of information, thus encouraging +sensible fallback options in the form of content blocks. The information being represented +is described by the event type, as it always has been before this MSC. It is explicitly +not permitted to represent two or more pieces of information in a single event, such +as a livestream reference and poll: senders should look into +[relationships](https://spec.matrix.org/v1.5/client-server-api/#forming-relationships-between-events) +instead. + +### Worked example: Custom temperature event + +In a hypothetical scenario, a temperature event might look as such: + +```json5 +{ + // irrelevant fields not shown + "type": "org.example.temperature", + "content": { + "m.text": [{"body": "It is 22 degrees at Home"}], + "org.example.probe_value": { + "label": "Home", + "units": "org.example.celsius", + "value": 22 + } + } +} +``` + +In this scenario, clients which understand how to render an `org.example.temperature` +event might use the information in `org.example.probe_value` exclusively, leaving the +`m.text` block for clients which *don't* understand the temperature event type. + +Another event type might find inspiration and use the probe value block for their +event as well. Such an example might be in a more industrial control application: + +```json5 +{ + // irrelevant fields not shown + "type": "org.example.tank.level", + "content": { + "m.text": [{"body": "[Danger] The water tank is 90% full."}], + "org.example.probe_value": { + "label": "Tank 3", + "units": "org.example.litres", + "value": 9037 + }, + "org.example.danger_level": "alert" + } +} +``` + +This event also demonstrates a `org.example.danger_level` block, which uses a string +value type instead of the previously demonstrated objects and values - this is a legal +content block, as blocks can be of any type. + +Clients should be cautious and avoid reusing too many unspecified types as it can create +opportunities for confusion and inconsistency. There should always be an effort to get +useful event types into the Matrix spec for others to benefit from. + +### Room version + +This MSC requires a room version to make the transition process clear and coordinated. +Normally for a feature such as this, an effort would be made to attempt to support +backwards compatibility for a duration of time, however for a feature that requires +significant overhaul of clients, servers, and Matrix as a whole it feels more important +to bias towards a clear switch between legacy and modern (extensible) events. + +**Note**: A previous draft of this proposal (codenamed "v1 extensible events") did attempt +to describe a timeline-based approach, allowing for event types to mix concepts of content +blocks and legacy fields, however that approach did not give sufficient reason for clients +to fully adopt the extensible events changes. + +In room versions supporting extensible events, clients MUST only send extensible events. +Deprecated event types (to be enumerated at the time of making the room version) MUST NOT +be sent into extensible event-supporting room versions, and clients MUST treat deprecated +event types as unrenderable by force. For example, if a client sees an `m.room.message` in +an extensible event-supporting room version, it must not render it, even if it knows how +to render that type. + +While full enforcement of this restriction is not feasible, servers are encouraged to block +Client-Server API requests for sending known-banned event types into applicable rooms. This +obviously does not help when the room is encrypted, or the client is sending custom events +in a non-extensible form, hence the requirement that clients treat the events as invalid too. + +Using the usual MSC process, the Spec Core Team (SCT) will be responsible for determining +the minimum scope of extensible events in a published (stable) room version. + +Meanwhile, clients are welcome to use the unstable implementations of extensible event-supporting +features, provided they are in an appropriate room version. Some event type MSCs declare +explicit support for what would normally be an unsupported room version - client authors +should check the applicable MSC or specification for the feature to determine if they are +allowed to do this. Such examples include MSC3381 Polls and MSC3245 Voice Messages. + +### State events + +Unknown state event types generally should not be parsed by clients. This is to prevent situations +where the sender masks a state change as some other, non-state, event. For example, even +if a state event has an `m.text` content block, it should not be treated as a room message. + +Note that state events MUST still make use of content blocks in applicable room versions, and that +any top-level key in `content` is defined as a content block under this proposal. As such, this +MSC implicitly promotes all existing content fields of `m.*` state events to independent content +blocks as needed. Other MSCs may override this decision on a per-event type basis (ie: redeclaring +how room topics work to support content blocks, deprecating the existing `m.room.topic` event in +the process, like in [MSC3765](https://github.com/matrix-org/matrix-spec-proposals/pull/3765)). +Unlike most content blocks, these promoted-to-content-blocks are not realistically meant to be +reused: it is simply a formality given this MSC's scope. + +### Notifications + +Currently [push notifications](https://spec.matrix.org/v1.5/client-server-api/#push-notifications) +describe how an event can cause a notification to the user, though it makes the assumption +that there are `m.room.message` events flying around to denote "messages" which can trigger +keyword/mention-style alerts. With extensible events, the same might not be possible as it +relies on understanding how/when the client will render the event to cause notifications. + +For simplicity, when `content.body` is used in an `event_match` condition, it now looks for +an `m.text` block's `text/plain` representation (implied or explicit) in room versions +supporting extensible events. This is not an easy rule to represent in the existing push +rules schema, and this MSC has no interest in designing a better schema. Note that other +conditions applied to push notifications, such as an event type check, are not affected by +this: clients/servers will have to alter applicable push rules to handle the new event types +(see also: [MSC3933](https://github.com/matrix-org/matrix-spec-proposals/pull/3933) and friends). + +### Power levels + +This MSC proposes no changes to how power levels interact with events: they are still +capable of restricting which users can send an event type. Though events might be rendered +as a different logical type (ie: unknown event being rendered as a message), this does not +materially impact the room's ability to function. Thus, considerations for how to handle +power levels more intelligently are details left for a future MSC. + +As of writing, most rooms fit into two categories: any event type is possible to send, or +specific cherry-picked event types are allowed (announcement rooms: reactions & redactions). +Extensible events don't materially change the situation implied by this power levels structure. + +### Mixins specifically allowed + +A **mixin** is a specific type of content block which can be added to any type of event to +change how that event is processed. Content blocks which are +mixins will be called out as such in the spec. Mixins are meant to be purely additive, +thus all event types MUST support being rendered/processed *without* the use of mixins. + +See also the [Wikipedia entry on mixins](https://en.wikipedia.org/wiki/Mixin). + +Note that mixins differ from optional content blocks in an event type's schema: a mixin +is able to be applied to *any* event type sensibly while optional content blocks are +generally only valuable to the applicable event types. + +Though this MSC does not describe any such mixins itself, +[MSC3955](https://github.com/matrix-org/matrix-spec-proposals/pull/3955) does by allowing any +event to be flagged as "automated" - a strictly additive annotation on events. + +Another possible mixin would be `m.relates_to` (not described by this MSC). Currently, +some features like the [key verification framework](https://spec.matrix.org/v1.5/client-server-api/#key-verification-framework) +rely on relationships as part of making the feature work. The expectation is that +these features would be adapted to meet the "purely additive" condition (assuming +`m.relates_to` does actually end up being a mixin). + +### Uses of HTML & text throughout the spec + +For an abundance of clarity, all functionality not explicitly called out in this MSC which +relies on the `formatted_body` of an `m.room.message` is expected to transition to using +an appropriate `m.text` representation instead. For example, the HTML representation of +a [mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) will +now appear under `m.text`'s `text/html` representation (adding one if required). + +A similar condition is applied to `body` in `m.room.message`: all existing functionality +will instead use the `text/plain` representation within `m.text`, if not explicitly +called out by this MSC. + +## Potential issues + +It's a bit ugly to not know whether a given key in `content` will take a string, object, +boolean, integer, or array. + +It's a bit ugly to not know at a glance if a content block is a mixin or not. + +It's a bit ugly that you have to look over the keys of contents to see what blocks +are present, but better than duplicating this into an explicit `blocks` list within the +event content (on balance). + +We're skipping over defining rules for which fallback combinations to display +(i.e. "display hints") for now; these can be added in a future MSC if needed. +[MSC1225](https://github.com/matrix-org/matrix-doc/issues/1225) contains a proposal for this. + +Placing content blocks at the top level of `content` is a bit unfortunate, though mixes +nicely thanks to namespacing. Potentially conflicting cases in the wild would be +namespaced fields, which would get translated as unrenderable events if the value type +doesn't meet the client's known schema. + +This MSC does not rewrite or redefine all possible events in the specification: this is +deliberately left as an exercise for several future MSCs. + +## Security considerations + +Like today, it's possible to have the different representations of an event not match, +thus introducing a potential for malicious payloads (text-only clients seeing something +different to HTML-friendly ones). Clients could try to do similarity comparisons, though +this is complicated with features like HTML and arbitrary custom markup (markdown, etc) +showing up in the plaintext or in tertiary formats on the events. Historically, room +moderators have been pretty good about removing these malicious senders from their rooms +when other users point out (quite quickly) that the event is appearing funky to them. + +## Note about spec process + +Extensible events as a spec feature requires dozens of different MSCs, with this MSC being +the structure definition and text baseline. It is *not* expected that this MSC will be +written into spec once it has passed FCP. Instead, it is expected that all of the "core" +extensible events MSCs will pass FCP and extensible events be assigned a stable room version +before any spec authoring begins. Thus, this particular MSC should be anticipated to sit +in accepted-but-not-merged (stable, not formal spec yet) for a while, and that's okay. + +The Spec Core Team (SCT) has decision making power over what is considered core for extensible +events, though the recommendation is to ensure replacements for all non-state `m.room.*` types +have accepted (successful FCP) MSCs to replace them. + +## Unstable prefix + +While this MSC is not considered stable by the specification, implementations *must* use +`org.matrix.msc1767` as a prefix to denote the unstable functionality. For example, sending +an `m.message` event would mean sending an `org.matrix.msc1767.message` event instead. + +For purposes of testing, implementations can use a dynamically-assigned unstable room version +`org.matrix.msc1767.` to use extensible events within. For example, `org.matrix.msc1767.10` +for room version 10 or `org.matrix.msc1767.org.example.cool_ver` for a hypothetical +`org.example.cool_ver` room version. Any events sent in these room versions *can* use stable +identifiers given the entire room version itself is unstable, however senders *must* take care +to ensure stable identifiers do not leak out to other room versions - it may be simpler to not +send stable identifiers at all. + +## Changes from MSC1225 + +* converted from googledoc to MD, and to be a single PR rather than split PR/Issue. +* simplifies it by removing displayhints (for now - deferred to a future MSC). +* replaces the clunky m.text.1 idea with lists for types which support fallbacks. +* removes the concept of optional compact form for m.text by instead having m.text always in expanded form. +* tries to accomodate most of the feedback on GH and Google Docs from MSC1225. + +## Historical changes + +* Anything that wasn't simple text rendering was broken out to dedicated MSCs in an effort to get the + structure approved and adopted while the more complex types get implemented independently. +* Renamed subtypes/reusable types to just "content blocks". +* Allow content blocks to be nested. +* Fix push rules in the most basic sense, deferring to a future MSC on better support. +* Explicitly make no changes to power levels, deferring to a future MSC on better support. +* Drop timeline for transition in favour of an explicit room version. +* Move most push rule changes and such into their own/future MSCs. +* Move emotes, notices, and encryption out to their own dedicated MSCs. diff --git a/proposals/2285-hidden-read-receipts.md b/proposals/2285-hidden-read-receipts.md new file mode 100644 index 000000000..7c3ed9eeb --- /dev/null +++ b/proposals/2285-hidden-read-receipts.md @@ -0,0 +1,151 @@ +# MSC2285: Private read receipts + +Currently users must send read receipts in order to affect their notification +counts, which alerts other people that the user has read their message. For +primarily privacy reasons, it may be desirable to users to not advertise to +others that they've read a message. + +## Proposal + +This MSC proposes adding a new `receiptType` (see [the receipts +spec](https://spec.matrix.org/v1.3/client-server-api/#receipts)) of +`m.read.private`. This `receiptType` is used when the user wants to affect their +notification count but doesn't want other users to see their read receipt. + +To move the user's private read receipt to `$123` the client can make a POST +request to the [`/receipt` +endpoint](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3roomsroomidreceiptreceipttypeeventid). +For example: + +```HTTP +POST /_matrix/client/v3/rooms/!a:example.org/receipt/m.read.private/$123 +{} +``` + +The MSC also proposes adding `m.fully_read` and `m.read.private` as a possible +`receiptType` for `/receipt` to make this endpoint consistent with +`/read_markers`. (we have two endpoints that do essentially the same thing, so +it would make sense for them to be consistent) + +Alternatively, the client can move the user's `m.fully_read` marker and/or +`m.read` receipt at the same time as `m.read.private` by making a POST request +to the [`/read_markers` +endpoint](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3roomsroomidread_markers). +For example: + +```HTTP +POST /_matrix/client/r0/rooms/!a:example.org/read_markers +{ + "m.fully_read": "$123", + "m.read": "$123", + "m.read.private": "$123" +} +``` + +Both `m.read` and `m.read.private` clear notifications in the same way. If the +user sent two receipts into a room, the later one should be the one that decides +the notification count. + +The receipt that is more "ahead" of the other takes precedence when considering +notifications and a client's rendering of read receipts. This means that given +an ordered set of events A, B, C, and D the public read receipt could be at +point C, private at point A. If the user moves the private receipt from A to B +then the user's notification count is still considered from point C as the public +receipt is further ahead, still. Other users would also see the user's public read +receipt as not having moved. The user can then move the private read receipt +to point D, hopping over the public receipt, to change their notification count. + +For clarity, if the public receipt is "fast forwarded" to be at the same position +as the private receipt then the public receipt is broadcast to other users, even +if previously considered private. + +Note that like regular read receipts today, neither receipt can cause a backwards +movement: both receipts can only move forwards, but do not have to be ahead of +each other. It's valid to, for example, update a public read receipt which lags +20 messages behind the private one. + +The `m.fully_read` property is now optional for the [`/read_markers` +endpoint](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3roomsroomidread_markers) +as sometimes we only want to send `m.read.private`. + +The MSC proposes that from now on, not all things sent over `/receipt` are +federated. Servers MUST NOT send receipts of `receiptType` `m.read.private` to +any other user than the sender. Servers also MUST NOT send receipts of +`receiptType` `m.read.private` to any server over federation. + +## Security considerations + +Servers could act as if `m.read.private` is the same as `m.read` so the user +must already trust the homeserver to a degree however, and the methods of +notifying the user to the problem are difficult to implement. Users can always +run their own homeservers to ensure it behaves correctly. + +## Potential issues + +Clients which support read receipts would end up rendering the user's receipt as +jumping down when they send a message. This is no different from how IRC and +similarly bridged users are perceived today. + +## Alternatives + +It has been suggested to use account data to store the setting that controls +whether read receipts should be private on a per-account/per-room basis. While +this might have some benefits, it is much less flexible. + +Previous iterations of this MSC additionally suggested that having an `m.hidden` +flag on existing read receipts could work, however this feels like assigning too +much responsibility to an existing structure. + +## Unstable prefix + +While this MSC is not considered stable, implementations should use +`org.matrix.msc2285` as a namespace. + +|Stable (post-FCP)|Unstable | +|-----------------|---------------------------------| +|`m.read.private` |`org.matrix.msc2285.read.private`| + +Clients should check for server support before sending private read receipts: +if the server does not support them, then a private read receipt will not clear +any notifications for the user. + +The presence of `org.matrix.msc2285` or `org.matrix.msc2285.stable` in +`unstable_features` is a reliable indication that a server supports private read +receipts; however the converse is not true: their absence does not necessarily +mean that the server does *not* support private read receipts. In particular, +the server may have been updated to a future spec version which includes +private read receipts, and hence removed the `unstable_features` entry. + +Therefore, if a client has this feature enabled, but the server does not advertise +support for this MSC in `unstable_features`, the client should either keep sending +private read receipts with the risk that notifications will not be clearing, or it +should warn the user and start sending public read receipts instead. + +To mitigate this problem, once this MSC gets merged and once it becomes a part of a +spec version, clients should update their implementations as fast as possible to +accommodate the fact that the way of detecting server support will change: clients +will now be looking for that spec version in `/versions`. + +### While the MSC is unstable + +During this period, to detect server support clients should check for the +presence of the `org.matrix.msc2285` flag in `unstable_features` on `/versions`. +Clients are also required to use the unstable prefixes (see [unstable +prefix](#unstable-prefix)) during this time. + +### Once the MSC is merged but not in a spec version + +Once this MSC is merged, but is not yet part of the spec, clients should rely on +the presence of the `org.matrix.msc2285.stable` flag in `unstable_features` to +determine server support. If the flag is present, clients are required to use +stable prefixes (see [unstable prefix](#unstable-prefix)). + +### Once the MSC is in a spec version + +Once this MSC becomes a part of a spec version, clients should rely on the +presence of the spec version, that supports the MSC, in `versions` on +`/versions`, to determine support. Servers are encouraged to keep the +`org.matrix.msc2285.stable` flag around for a reasonable amount of time +to help smooth over the transition for clients. "Reasonable" is intentionally +left as an implementation detail, however the MSC process currently recommends +*at most* 2 months from the date of spec release. diff --git a/proposals/2674-event-relationships.md b/proposals/2674-event-relationships.md new file mode 100644 index 000000000..95f077bad --- /dev/null +++ b/proposals/2674-event-relationships.md @@ -0,0 +1,250 @@ +# MSC2674: Event relationships + +It's common to want to send events in Matrix which relate to existing events - +for instance, reactions, edits and even replies/threads. + +This proposal is one in a series of proposals that defines a mechanism for +events to relate to each other. Together, these proposals replace +[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849). + +* This proposal defines a standard shape for indicating events which relate to + other events. +* [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines APIs to + let the server calculate the aggregations on behalf of the client, and so + bundle the related events with the original event where appropriate. +* [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676) defines how + users can edit messages using this mechanism. +* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how + users can annotate events, such as reacting to events with emoji, using this + mechanism. +* [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267) defines how events + can make a reference to other events. +* [MSC3389](https://github.com/matrix-org/matrix-doc/pull/3389) defines changes to + the redaction algorithm, to preserve the type and target id of a relation. + + +## Proposal + +This proposal introduces the concept of relations, which can be used to +associate new information with an existing event. + +A relationship is an object with a field `rel_type`, which is a string describing the type of relation, +and a field `event_id`, which is a string that represents the event_id of the target event of this relation. +The target event must exist in the same room as the relating event is sent. +Both of those fields are required. An event is said to contain a relationship if its `content` contains +a relationship with all the required fields under the `m.relates_to` key. If any of these conditions is not met, +clients and servers should treat the event as if it does not contain a relationship. +Servers should reject events not meeting these conditions with an HTTP 400 error when +they are received via the client-server API. + +Here's a (partial) example of an event relating to another event: + +```json +{ + "content": { + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$abc:server.tld" + } + } +} +``` + +All the information about the relationship lives under the `m.relates_to` key. + +If it helps, you can think of relations as a "subject verb object" triple, +where the subject is the relation event itself; the verb is the `rel_type` +field of the `m.relates_to` and the object is the `event_id` field. + +We consciously do not support multiple different relations within a single event, +in order to keep the API simple. This means that if event A relates to event B +in two different ways you would send two events to describe the two relations, +rather than bundling them into a single event. Another MSC, +like [MSC 3051](https://github.com/matrix-org/matrix-doc/pull/3051), +can propose a change to add support for multiple relations if it turns out that +this would facilitate certain use cases. + +Relations do not yet replace the +[reply mechanism currently defined in the spec](https://matrix.org/docs/spec/client_server/r0.6.1#rich-replies). + +### Relation types + +Any values for `rel_type` should abide the +[general guidelines for identifiers](https://github.com/matrix-org/matrix-doc/pull/3171). + +The `rel_type` property determines how an event relates to another and can be used +by clients to determine how and in what context a relation should be displayed. + +[MSC 2675](https://github.com/matrix-org/matrix-doc/pull/2675) proposes to also interpret the `rel_type` server-side. + +It is left up to the discretion of other MSCs building on this one whether they introduce +`rel_type`s that are specific to their use case or that can serve a broad range +of use cases. MSCs may define additional properties on the relation object for a given `rel_type`. + +Currently, a few `rel_type`s are already proposed. Here's a non-exhaustive list: + + - `m.replace` in [MSC 2676](https://github.com/matrix-org/matrix-doc/pull/2676). + - `m.annotation` in [MSC 2677](https://github.com/matrix-org/matrix-doc/pull/2677). + - `m.reference` in [MSC 3267](https://github.com/matrix-org/matrix-doc/pull/3267). + - `m.thread` in [MSC 3440](https://github.com/matrix-org/matrix-doc/pull/3440). + + +### Sending relations + +Related events are normal Matrix events, and can be sent by the normal `/send` +API. + +The server should postprocess relations if needed before sending them into a +room, as defined by the relationship type. For example, a relationship type +might only allow a user to send one related message to a given event. + +### Receiving relations + +Relations are received like other non-state events, with `/sync`, +`/messages` and `/context`, as normal discrete Matrix events. As explained +in the limitations, clients may be unaware of some relations using just these endpoints. + +[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines ways in +which the server may aid clients in processing relations by aggregating the +events. + +### Redactions + +Events with a relation may be redacted like any other event. + +[MSC3389](https://github.com/matrix-org/matrix-doc/pull/3389) proposes that +the redaction algorithm should preserve the type and target id of a relation. + +However, event relationships can still be used in existing room versions, but +the user experience may be worse if redactions are performed. + +## Potential issues + +### Federation considerations + +We have a problem with resynchronising relations after a gap in federation: +We have no way of knowing that an edit happened in the gap to one of the events +we already have. So, we'll show inconsistent data until we backfill the gap. + * We could write this off as a limitation. + * Or we could make *ALL* relations a DAG, so we can spot holes at the next + relation, and go walk the DAG to pull in the missing relations? Then, the + next relation for an event could pull in any of the missing relations. + Socially this probably doesn't work as reactions will likely drop off over + time, so by the time your server comes back there won't be any more reactions + pulling the missing ones in. + * Could we also ask the server, after a gap, to provide all the relations which + happened during the gap to events whose root preceeded the gap. + * "Give me all relations which happened between this set of + forward-extremities when I lost sync, and the point i've rejoined the DAG, + for events which preceeded the gap"? + * Would be hard to auth all the relations which this api coughed up. + * We could auth them based only the auth events of the relation, except we + lose the context of the nearby DAG which we'd have if it was a normal + backfilled event. + * As a result it would be easier for a server to retrospectively lie about + events on behalf of its users. + * This probably isn't the end of the world, plus it's more likely to be + consistent than if we leave a gap. + * i.e. it's better to consistent with a small chance of being maliciously + wrong, than inconsistent with a guaranteed chance of being innocently + wrong. + * We'd need to worry about pagination. + * This is probably the best solution, but can also be added as a v2. + * In practice this seems to not be an issue, which is worth complicating + the s-s API over. Clients very rarely jump over the federation gap to an edit. + In most cases they scroll up, which backfills the server and we have all the + edits, when we reach the event before the gap. + +## Limitations + +Based solely on this MSC, relations are only received as discrete events in +the timeline, so clients may only have an incomplete image of all the relations +with an event if they do not fill gaps (syncs with a since token that have +`limited: true` set in the sync response for a room) in the timeline. + +In practice, this has proven not to be too big of a problem, as reactions +(as proposed in [MSC 2677](https://github.com/matrix-org/matrix-doc/pull/2677)) +tend to be posted close after the target event in the timeline. + +A more complete solution to this has been deferred to +[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). + +## Tradeoffs + +### Event shape + +Shape of + +```json +"content": { + "m.relates_to": { + "m.reference": { + "event_id": "$another:event.com" + } + } +} +``` +versus + +```json +"content": { + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$another:event.com" + } +} +``` + +The reasons to go with `rel_type` is: + * This format is now in use in the wider matrix ecosystem without a prefix, + in spite of the original MSC 1849 not being merged. This situation is not ideal + but we still don't want to break compatibility with several clients. + * We don't need the extra indirection to let multiple relations apply to a given pair of + events, as that should be expressed as separate relation events. + * If we want 'adverbs' to apply to 'verbs' in the subject-verb-object triples which + relations form, then we apply it as mixins to the relation data itself rather than trying + to construct subject-verb-verb-object sentences. + * We decided to not adopt the format used by `m.in_reply_to` as it allows for multiple relations + and is hence overly flexible. Also, the relation type of `m.in_reply_to` is also overly specific + judged by the guidelines for `rel_type`s laid out in this MSC. Having replies use the same + format as relations is postponed to a later MSC, but it would likely involve replies + adopting the relation format with a more broadly useful `rel_type` (possibly the `m.reference` + type proposed in [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267)), + rather than relations adopting the replies format. + +## Historical context + +pik's MSC441 has: + +Define the JSON schema for the aggregation event, so the server can work out +which fields should be aggregated. + +```json +"type": "m.room._aggregation.emoticon", +"content": { + "emoticon": "::smile::", + "msgtype": "?", + "target_id": "$another:event.com" +} +``` + +These would then be aggregated, based on target_id, and returned as annotations on +the source event in an `aggregation_data` field: + +```json +"content": { + ... + "aggregation_data": { + "m.room._aggregation.emoticon": { + "aggregation_data": [ + { + "emoticon": "::smile::", + "event_id": "$14796538949JTYis:pik-test", + "sender": "@pik:pik-test" + } + ], + "latest_event_id": "$14796538949JTYis:pik-test" + } + } +} +``` diff --git a/proposals/2675-aggregations-server.md b/proposals/2675-aggregations-server.md new file mode 100644 index 000000000..5808dbec0 --- /dev/null +++ b/proposals/2675-aggregations-server.md @@ -0,0 +1,320 @@ +# MSC2675: Serverside aggregations of message relationships + +It's common to want to send events in Matrix which relate to existing events - +for instance, reactions, edits and even replies/threads. + +Clients typically need to track the related events alongside the original +event they relate to, in order to correctly display them. For instance, +reaction events need to be aggregated together by summing and be shown next to +the event they react to; edits need to be aggregated together by replacing the +original event and subsequent edits, etc. + +It is possible to treat relations as normal events and aggregate them +clientside, but to do so comprehensively could be very resource intensive, as +the client would need to spider all possible events in a room to find +relationships and maintain a correct view. + +Instead, this proposal seeks to solve this problem by defining APIs to let the +server calculate the aggregations on behalf of the client, and so bundle the +aggregated data with the original event where appropriate. It also proposes an +API to let clients paginate through all relations of an event. + +This proposal is one in a series of proposals that defines a mechanism for +events to relate to each other. Together, these proposals replace +[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849). + +* [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674) defines a + standard shape for indicating events which relate to other events. +* This proposal defines APIs to let the server calculate the aggregations on + behalf of the client, and so bundle the aggregated data with the original + event where appropriate. +* [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676) defines how + users can edit messages using this mechanism. +* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how + users can annotate events, such as reacting to events with emoji, using this + mechanism. + +## Proposal + +### Aggregations + +Relation events can be aggregated per relation type by the server. +The format of the aggregated value (hereafter called "aggregation") +depends on the relation type. + +Some relation types might group the aggregations by the `key` property +in the relation and aggregate to an array, while +others might aggregate to a single object or any other value really. + +Here are some non-normative examples of what aggregations can look like: + +Example aggregation for [`m.thread`](https://github.com/matrix-org/matrix-doc/pull/3440) (which +aggregates all relations into a single object): +``` +{ + "latest_event": { + "content": { ... }, + ... + }, + "count": 7, + "current_user_participated": true +} +``` + +Example aggregation for [`m.annotation`](https://github.com/matrix-org/matrix-doc/pull/2677) (which +aggregates relations into a list of objects, grouped by `key`). +``` +[ + { + "key": "👍", + "origin_server_ts": 1562763768320, + "count": 3 + }, + { + "key": "👎", + "origin_server_ts": 1562763768320, + "count": 2 + } +] +``` + +#### Bundling + +Other than during non-gappy incremental syncs, timeline events that have other +events relate to them should include the aggregation of those related events +in the `m.relations` property of their unsigned data. This process is +referred to as "bundling", and the aggregated relations included via +this mechanism are called "bundled aggregations". + +By sending a summary of the relations, bundling +avoids us having to always send lots of individual relation events +to the client. + +Aggregations are never bundled into state events. This is a current +implementation detail that could be revisited later, +rather than a specific design decision. + +The following client-server APIs should bundle aggregations +with events they return: + + - `GET /rooms/{roomId}/messages` + - `GET /rooms/{roomId}/context/{eventId}` + - `GET /rooms/{roomId}/event/{eventId}` + - `GET /sync`, only for room sections in the response where `limited` field + is `true`; this amounts to all rooms in the response if + the `since` request parameter was not passed, also known as an initial sync. + - `GET /relations`, as proposed in this MSC. + +Deprecated APIs like `/initialSync` and `/events/{eventId}` are *not* required +to bundle aggregations. + +The bundled aggregations are grouped according to their relation type. +The format of `m.relations` (here with *non-normative* examples of +the [`m.replace`](https://github.com/matrix-org/matrix-doc/pull/2676) and +[`m.annotation`](https://github.com/matrix-org/matrix-doc/pull/2677) relation +types) is as follows: + +```json +{ + "event_id": "abc", + "unsigned": { + "m.relations": { + "m.annotation": { + "key": "👍", + "origin_server_ts": 1562763768320, + "count": 3 + }, + "m.replace": { + "event_id": "$edit_event_id", + "origin_server_ts": 1562763768320, + "sender": "@alice:localhost" + }, + } + } +} +``` + +#### Client-side aggregation + +Bundled aggregations on an event give a snapshot of what relations were known +at the time the event was received. When relations are received through `/sync`, +clients should locally aggregate (as they might have done already before +supporting this MSC) the relation on top of any bundled aggregation the server +might have sent along previously with the target event, to get an up to date +view of the aggregations for the target event. The aggregation algorithm is the +same as the one described here for the server. + +### Querying relations + +A single event can have lots of associated relations, and we do not want to +overload the client by, for example, including them all bundled with the +related-to event. Instead, we also provide a new `/relations` API in +order to paginate over the relations, which behaves in a similar way to +`/messages`, except using `next_batch` and `prev_batch` names +(in line with `/sync` API). Tokens from `/sync` or `/messages` can be +passed to `/relations` to only get relating events from a section of +the timeline. + +The `/relations` API returns the discrete relation events +associated with an event that the server is aware of +in standard topological order. Note that events may be missing, +see [limitations](#servers-might-not-be-aware-of-all-relations-of-an-event). +You can optionally filter by a given relation type and the event type of the +relating event: + +``` +GET /_matrix/client/v1/rooms/{roomID}/relations/{event_id}[/{rel_type}[/{event_type}]][?from=token][&to=token][&limit=amount] +``` + +``` +{ + "chunk": [ + { + "type": "m.reaction", + "sender": "...", + "content": { + "m.relates_to": { + "rel_type": "m.annotation", + ... + } + } + } + ], + "prev_batch": "some_token", + "next_batch": "some_token", +} +``` + +The endpoint does not have any trailing slashes. It requires authentication +and is not rate-limited. + +The `from` and `limit` query parameters are used for pagination, and work +just like described for the `/messages` endpoint. + +Note that [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676) +adds the related-to event in `original_event` property of the response. +This way the full history (e.g. also the first, original event) of the event +is obtained without further requests. See that MSC for further details. + +### End to end encryption + +Since the server has to be able to aggregate relation events, structural +information about relations must be visible to the server, and so the +`m.relates_to` field must be included in the plaintext. + +A future MSC may define a method for encrypting certain parts of the +`m.relates_to` field that may contain sensitive information. + +### Redactions + +Redacted relations should not be taken into consideration in +bundled aggregations, nor should they be returned from `/relations`. + +Requesting `/relations` on a redacted event should +still return any existing relation events. +This is in line with other APIs like `/context` and `/messages`. + +### Local echo + +For the best possible user experience, clients should also include unsent +relations into the client-side aggregation. When adding a relation to the send +queue, clients should locally aggregate it into the relations of the target +event, ideally regardless of the target event having received an `event_id` +already or still being pending. If the client gives up on sending the relation +for some reason, the relation should be de-aggregated from the relations of +the target event. If the client offers the user a possibility of manually +retrying to send the relation, it should be re-aggregated when the user does so. + +De-aggregating a relation refers to rerunning the aggregation for a given +target event while not considering the de-aggregated event any more. + +Upon receiving the remote echo for any relations, a client is likely to remove +the pending event from the send queue. Here, it should also de-aggregate the +pending event from the target event's relations, and re-aggregate the received +remote event from `/sync` to make sure the client-side aggregation happens with +the same event data as on the server. + +When adding a redaction for a relation to the send queue, the relation +referred to should be de-aggregated from the relations of the target of the +relation. Similar to a relation, when the sending of the redaction fails or +is cancelled, the relation should be aggregated again. + +If the target event is still pending and hasn't received its `event_id` yet, +clients can locally relate relation events to their target by using +`transaction_id` like they already do for detecting remote echos when sending +events. + +## Edge cases + +### How do you handle ignored users? + + * Information about relations sent from ignored users must never be sent to + the client, either in aggregations or discrete relation events. + This is to let you block someone from harassing you with emoji reactions + (or using edits as a side-channel to harass you). Therefore, it is possible + that different users will see different aggregations (a different last edit, + or a different reaction count) on an event. + +## Limitations + +### Relations can be missed while not being in the room + +Relation events behave no different from other events in terms of room history visibility, +which means that some relations might not be visible to a user while they are not invited +or have not joined the room. This can cause a user to see an incomplete edit history or reaction count +based on discrete relation events upon (re)joining a room. + +Ideally the server would not include these events in aggregations, as it would mean breaking +the room history visibility rules, but this MSC defers addressing this limitation and +specifying the exact server behaviour to [MSC3570](https://github.com/matrix-org/matrix-doc/pull/3570). + +### Servers might not be aware of all relations of an event + +The response of `/relations` might be incomplete because the homeserver +potentially doesn't have the full DAG of the room. The federation API doens't +have an equivalent of the `/relations` API, so has no way but to fetch the +full DAG over federation to assure itself that it is aware of all relations. + +[MSC2836](https://github.com/matrix-org/matrix-doc/pull/2836) provided a proposal for following relationships over federation in the "Querying relationships over federation" section via a `/_matrix/federation/v1/event_relationships` API + +### Event type based aggregation and filtering won't work well in encrypted rooms + +The `/relations` endpoint allows filtering by event type, +which for encrypted rooms will be `m.room.encrypted`, rendering this filtering +less useful for encrypted rooms. Aggregation algorithms that take the type of +the relating events they aggregate into account will suffer from the same +limitation. + +## Future extensions + +### Handling limited (gappy) syncs + +For the special case of a gappy incremental sync, many relations (particularly +reactions) may have occurred during the gap. It would be inefficient to send +each one individually to the client, but it would also be inefficient to send +all possible bundled aggregations to the client. + +The server could tell the client the event IDs of events which +predate the gap which received relations during the gap. This means that the +client could invalidate its copy of those events (if any) and then requery them +(including their bundled relations) from the server if/when needed, +for example using an extension of the `/event` API for batch requests. + +The server could do this with a new `stale_events` field of each room object +in the sync response. The `stale_events` field would list all the event IDs +prior to the gap which had updated relations during the gap. The event IDs +would be grouped by relation type, +and paginated as per the normal Matrix pagination model. + +This was originally part of this MSC but left out to limit the scope +to what is implemented at the time of writing. + +## Prefix + +While this MSC is not considered stable, the endpoints become: + + - `GET /_matrix/client/unstable/rooms/{roomID}/relations/{eventID}[/{relationType}[/{eventType}]]` + +None of the newly introduced identifiers should use a prefix though, as this MSC +tries to document relation support already being used in +the wider matrix ecosystem. \ No newline at end of file diff --git a/proposals/2676-message-editing.md b/proposals/2676-message-editing.md new file mode 100644 index 000000000..680563b50 --- /dev/null +++ b/proposals/2676-message-editing.md @@ -0,0 +1,407 @@ +# MSC2676: Message editing + +Users may wish to edit previously sent messages, for example to correct typos. +This can be done by sending a new message with an indication that it replaces +the previously sent message. + +This proposal is one in a series of proposals that defines a mechanism for +events to relate to each other. Together, these proposals replace +[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849). + +* [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674) defines a + standard shape for indicating events which relate to other events. +* [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) defines APIs to + let the server calculate the aggregations on behalf of the client, and so + bundle the related events with the original event where appropriate. +* This proposal defines how users can edit messages using this mechanism. +* [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677) defines how + users can annotate events, such as reacting to events with emoji, using this + mechanism. + +## Background + +Element-Web (then Riot-Web) and Synapse both implemented initial support for +message editing, following the proposals of MSC1849, in May 2019 +([matrix-react-sdk](https://github.com/matrix-org/matrix-react-sdk/pull/2952), +[synapse](https://github.com/matrix-org/synapse/pull/5209)). Element-Android +and Element-iOS also added implementations around that time. Unfortunately, +those implementations presented the feature as "production-ready", despite it +not yet having been adopted into the Matrix specification. + +The current situation is therefore that client or server implementations hoping +to interact with Element users must simply follow the examples of that +implementation. In other words, message edits form part of the *de-facto* spec +despite not being formalised in the written spec. This is clearly a regrettable +situation. Hopefully, processes have improved over the last three years so that +this situation will not arise again. Nevertheless there is little we can do +now other than formalise the status quo. + +This MSC, along with the others mentioned above, therefore seeks primarily to +do that. Although there is plenty of scope for improvement, we consider that +better done in *future* MSCs, based on a shared understanding of the *current* +implementation. + +In short, this MSC prefers fidelity to the current implementations over +elegance of design. + +## Proposal + +### `m.replace` event relationship type + +A new `rel_type` of `m.replace` is defined for use with the `m.relates_to` +field as defined in +[MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674). This is +intended primarily for handling edits, and lets you define an event which +replaces an existing event. + +Such an event, with `rel_type: m.replace`, is referred to as a "message edit event". + +### `m.new_content` property + +The `content` of a message edit event must contain a `m.new_content` property +which defines the replacement content. (This allows the normal `body` fields to +be used for a fallback for clients who do not understand replacement events.) + +For instance, an `m.room.message` which replaces an existing event might look like: + +```json +{ + "type": "m.room.message", + "content": { + "body": "* Hello! My name is bar", + "msgtype": "m.text", + "m.new_content": { + "body": "Hello! My name is bar", + "msgtype": "m.text" + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$some_event_id" + } + } +} +``` + +The `m.new_content` can include any properties that would normally be found in +an event's `content` property, such as `formatted_body`. + +#### Encrypted events + +If the original event was encrypted, the replacement should be too. In that +case, `m.new_content` is placed in the `content` of the encrypted payload. The +`m.relates_to` property remains unencrypted, as required by the +[relationships](https://spec.matrix.org/v1.3/client-server-api/#forming-relationships-between-events) +section of the Client-Server API specification. + +For example, an encrypted replacement event might look like this: + +```json +{ + "type": "m.room.encrypted", + "content": { + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$some_event_id" + }, + "algorithm": "m.megolm.v1.aes-sha2", + "sender_key": "", + "device_id": "", + "session_id": "", + "ciphertext": "" + } +} +``` + +... and, once decrypted, the payload might look like this: + + +```json +{ + "type": "m.room.", + "room_id": "!some_room_id", + "content": { + "body": "* Hello! My name is bar", + "msgtype": "m.text", + "m.new_content": { + "body": "Hello! My name is bar", + "msgtype": "m.text" + } + } +} +``` + +Note that: + * There is no `m.relates_to` property in the encrypted payload. (Any such + property would be ignored.) + * There is no `m.new_content` property in the cleartext `content` of the + `m.room.encrypted` event. (Again, any such property would be ignored.) + +For clarity: the payload must be encrypted as normal, ratcheting the Megolm session +as normal. The original Megolm ratchet entry should **not** be re-used. + +#### Applying `m.new_content` + +When applying a replacement, the `content` property of the original event is +replaced entirely by the `m.new_content`, with the exception of `m.relates_to`, +which is left *unchanged*. Any `m.relates_to` property within `m.new_content` +is ignored. + +For example, given a pair of events: + +```json +{ + "event_id": "$original_event", + "type": "m.room.message", + "content": { + "body": "I *really* like cake", + "msgtype": "m.text", + "formatted_body": "I really like cake", + } +} +``` + +```json +{ + "event_id": "$edit_event", + "type": "m.room.message", + "content": { + "body": "* I *really* like *chocolate* cake", + "msgtype": "m.text", + "m.new_content": { + "body": "I *really* like *chocolate* cake", + "msgtype": "m.text", + "com.example.extension_property": "chocolate" + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$original_event_id" + } + } +} +``` + +... then the end result is an event as shown below. Note that `formatted_body` +is now absent, because it was absent in the replacement event, but +`m.relates_to` remains unchanged (ie, absent). + +```json +{ + "event_id": "$original_event", + "type": "m.room.message", + "content": { + "body": "I *really* like *chocolate* cake", + "msgtype": "m.text", + "com.example.extension_property": "chocolate" + } +} +``` + +Note that the `msgtype` property of `m.room.message` events need not be the +same as in the original event. For example, if a user intended to send a +message beginning with "/me", but their client sends an `m.emote` event +instead, they could edit the message to send be an `m.text` event as they had +originally intended. + +### Validity of message edit events + +Some message edit events are defined to be invalid. To be considered valid, all +of the following criteria must be satisfied: + + * The replacement and original events must have the same `type`. + * Neither the replacement nor original events can be state events (ie, neither + may have a `state_key`). + * The original event must not, itself, have a `rel_type` of `m.replace`. + * The original event and replacement event must have the same `sender`. + * The replacement event (once decrypted, if appropriate) must have an + `m.new_content` property. + +The original event and replacement event must also have the same `room_id`, as +required by the +[relationships](https://spec.matrix.org/v1.3/client-server-api/#forming-relationships-between-events) +section of the Client-Server API specification. + +If any of these criteria are not satisfied, implementations should ignore the +replacement event (the content of the original should not be replaced, and the +edit should not be included in the server-side aggregation). + +### Server behaviour + +#### Server-side aggregation of `m.replace` relationships + +Note that there can be multiple events with an `m.replace` relationship to a +given event (for example, if an event is edited multiple times). These should +be [aggregated](https://spec.matrix.org/v1.3/client-server-api/#aggregations) +by the homeserver. + +The format of the aggregation for `m.replace` simply gives gives the +`event_id`, `origin_server_ts`, and `sender` of the most recent replacement +event (as determined by `origin_server_ts`, falling back to a lexicographic +ordering of `event_id`). + +This aggregation is bundled into the `unsigned/m.relations` property of any +event that is the target of an `m.replace` relationship. For example: + +```json5 + +{ + "event_id": "$original_event_id", + // ... + "unsigned": { + "m.relations": { + "m.replace": { + "event_id": "$latest_edit_event_id", + "origin_server_ts": 1649772304313, + "sender": "@editing_user:localhost" + } + } + } +} +``` + +If the original event is redacted, any `m.replace` relationship should **not** +be bundled with it (whether or not any subsequent edits are themselves +redacted). Note that this behaviour is specific to the `m.replace` +relationship. + +#### Server-side replacement of content + +Whenever an `m.replace` is to be bundled with an event as above, the server should +also modify the `content` of the original event according +to the `m.new_content` of the most recent edit (determined as above). + +An exception applies to [`GET +/_matrix/client/v3/rooms/{roomId}/event/{eventId}`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomideventeventid), +which should return the *unmodified* event (though the relationship should +still be bundled, as described above). + +The endpoints where this behaviour takes place is the same as those where +aggregations are bundled, with the exception of +`/room/{roomId}/event/{eventId}`. This includes: + + * `GET /rooms/{roomId}/messages` + * `GET /rooms/{roomId}/context/{eventId}` + * `GET /rooms/{roomId}/relations/{eventId}` + * `GET /rooms/{roomId}/relations/{eventId}/{relType}` + * `GET /rooms/{roomId}/relations/{eventId}/{relType}/{eventType}` + * `GET /sync` when the relevant section has a `limited` value of `true` + * `POST /search` for any matching events under `room_events`. + +### Client behaviour + +Clients can often ignore message edit events, since any events the server +returns via the C-S API will be updated by the server to account for subsequent +edits. + +However, clients should apply the replacement themselves when the server is +unable to do so. This happens in the following situations: + +1. The client has already received and stored the original event before the message + edit event arrives. + +2. The original event (and hence its replacement) are encrypted. + +Client authors are reminded to take note of the requirements for [Validity of +message edit events](#validity-of-message-edit-events), and to ignore any +invalid edit events that may be received. + +### Permalinks + +Permalinks to edited events should capture the event ID that the creator of the +permalink is viewing at that point (which might be a message edit event). + +The client viewing the permalink should resolve this ID to the original event +ID, and then display the most recent version of that event. + +### Redactions + +When a message using a `rel_type` of `m.replace` is redacted, it removes that +edit revision. This has little effect if there were subsequent edits, however +if it was the most recent edit, the event is in effect reverted to its content +before the redacted edit. + +Redacting the original message in effect removes the message, including all +subsequent edits, from the visible timeline. In this situation, homeservers +will return an empty `content` for the original event as with any other +redacted event. It must be noted that, although they are not immediately +visible in Element, subsequent edits remain unredacted and can be seen via API +calls. See [Future considerations](#future-considerations). + +### Edits of replies + +Some particular constraints apply to events which replace a +[reply](https://spec.matrix.org/v1.3/client-server-api/#rich-replies). In +particular: + + * There should be no `m.in_reply_to` property in the the `m.relates_to` + object, since it would be redundant (see [Applying + `m.new_content`](#applying-mnew_content) above, which notes that the original + event's `m.relates_to` is preserved), as well as being contrary to the + spirit of + [MSC2674](https://github.com/matrix-org/matrix-spec-proposals/pull/2674) + which expects only one relationship per event. + + * `m.new_content` should **not** contain any ["reply + fallback"](https://spec.matrix.org/v1.3/client-server-api/#fallbacks-for-rich-replies), + since it is assumed that any client which can handle edits can also + display replies natively. + +An example of an edit to a reply is as follows: + +```json +{ + "type": "m.room.message", + "content": { + "body": "> <@richvdh:sw1v.org> ab\n\n * ef", + "msgtype": "m.text", + "format": "org.matrix.custom.html", + "formatted_body": "
In reply to @richvdh:sw1v.org
ab
 * ef", + "m.new_content": { + "body": "ef", + "msgtype": "m.text", + "format": "org.matrix.custom.html", + "formatted_body": "ef" + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$original_reply_event" + } + } +} +``` + + +## Future considerations + +### Ordering of edits + +In future we may wish to consider ordering replacements (or relations in +general) via a DAG rather than using `origin_server_ts` to determine ordering - +particularly to mitigate potential abuse of edits applied by moderators. +Whatever, care must be taken by the server to ensure that if there are multiple +replacement events, the server must consistently choose the same one as all +other servers. + +### Redaction of edits + +It is highly unintuitive that redacting the original event leaves subsequent +edits visible to curious eyes even though they are hidden from the +timeline. This is considered a bug which this MSC makes no attempt to +resolve. See also +[element-web#11978](https://github.com/vector-im/element-web/issues/11978) and +[synapse#5594](https://github.com/matrix-org/synapse/issues/5594). + +### Edits to state events + +There are various issues which would need to be resolved before edits to state +events could be supported. In particular, we would need to consider how the +semantically-meaningful fields of the content of a state event relate to +`m.new_content`. Variation between implementations could easily lead to +security problems (See +[element-web#21851](https://github.com/vector-im/element-web/issues/21851) for +example.) + +### Editing other users' events + +There is a usecase for users with sufficient power-level to edit other peoples' +events. For now, no attempt is made to support this. If it is supported in the +future, we would need to find a way to make it clear in the timeline. diff --git a/proposals/2677-reactions.md b/proposals/2677-reactions.md new file mode 100644 index 000000000..97b06c34e --- /dev/null +++ b/proposals/2677-reactions.md @@ -0,0 +1,294 @@ +# MSC2677: Annotations and Reactions + +Users sometimes wish to respond to a message using emojis. When such responses +are grouped visually below the message being reacted to, this provides a +(visually) light-weight way for users to react to messages. + +This proposal was originally part of [MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849). + +## Background + +As with [message +edits](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/2676-message-editing.md#background), +support for reactions were landed in the Element clients and Synapse in May +2019, following the proposals of +[MSC1849](https://github.com/matrix-org/matrix-doc/pull/1849) and then +presented as being "production-ready", despite them not yet having been adopted +into the Matrix specification. + +Again as with edits, the current situation is therefore that client or server +implementations hoping to interact with Element users must simply follow the +examples of that implementation. + +To rectify the situation, this MSC therefore seeks primarily to formalise the +status quo. Although there is plenty of scope for improvement, we consider +that better done in *future* MSCs, based on a shared understanding of the +*current* implementation. + +In short, this MSC prefers fidelity to the current implementations over +elegance of design. + +On the positive side: this MSC is the last part of the former MSC1849 to be +formalised, and is by far the most significant feature implemented by the +Element clients which has yet to be specified. + +## Proposal + +### `m.annotation` event relationship type + +A new [event relationship type](https://spec.matrix.org/v1.6/client-server-api/#relationship-types) +with a `rel_type` of `m.annotation`. + +This relationship type is intended primarily for handling emoji reactions, allowing clients to +send an event which annotates an existing event. + +Another potential usage of annotations is for bots, which could use them to +report the success/failure or progress of a command. + +Along with the normal properties `event_id` and `rel_type`, the +[`m.relates_to`](https://spec.matrix.org/v1.6/client-server-api/#definition-mrelates_to) +property should contains a `key` that indicates the annotation being +applied. For example, when reacting with emojis, the `key` contains the emoji +being used. + +An event annotating another with the thumbs-up emoji would therefore have the following `m.relates_to` propperty: + +```json +"m.relates_to": { + "rel_type": "m.annotation", + "event_id": "$some_event_id", + "key": "👍" +} +``` + +When sending emoji reactions, the `key` property should include the unicode +[emoji presentation +selector](https://www.unicode.org/reports/tr51/#def_emoji_presentation_selector) +(`\uFE0F`) for codepoints which allow it (see the [emoji variation sequences +list](https://www.unicode.org/Public/UCD/latest/ucd/emoji/emoji-variation-sequences.txt)). + +Any `type` of event is eligible for an annotation, including state events. + +### `m.reaction` event type + +A new message type `m.reaction` is proposed to indicate that a user is reacting +to a message. No `content` properties are defined for this event type: it serves +only to hold a relationship to another event. + +For example, an `m.reaction` event which annotates an existing event with a 👍 +looks like: + +```json +{ + "type": "m.reaction", + "content": { + "m.relates_to": { + "rel_type": "m.annotation", + "event_id": "$some_event_id", + "key": "👍" + } + } +} +``` + +Since they contain no `content` other than `m.relates_to`, `m.reaction` events +are normally not encrypted, as there would be no benefit in doing so. (However, +see [Encrypted reactions](#encrypted-reactions) below.) + +### Interation with edited events + +It is not considered valid to send an annotation for a [replacement +event](https://spec.matrix.org/v1.6/client-server-api/#event-replacements) +(i.e., a message edit event): any reactions should refer to the original +event. Annotations of replacement events will be ignored according to the rules +for [counting annotations](#counting-annotations). + +As an aside, note that it is not possible to edit a reaction, since replacement +events do not change `m.relates_to` (see [Applying +`m.new_content`](https://spec.matrix.org/v1.6/client-server-api/#applying-mnew_content)), +and there is no other meaningful content within `m.reaction`. If a user wishes +to change their reaction, the original reaction should be redacted and a new +one sent in its place. + +### Counting annotations + +The intention of annotations is that they are counted up, rather than being displayed individually. + +Clients must keep count of the number of annotations with a given event `type` +and annotation `key` they observe for each event; these counts are typically +presented alongside the event in the timeline. + +When performing this count: + + * Each event `type` and annotation `key` should normally be counted separately, + though whether to actually do so is an implementation decision. + + * Annotation events sent by [ignored users](https://spec.matrix.org/v1.6/client-server-api/#ignoring-users) + should be excluded from the count. + + * Multiple identical annotations (i.e., with the same event `type` and + annotation `key`) from the same user (i.e., events with the same `sender`) should + be treated as a single annotation. + + * It is not considered valid to annotate an event which itself has an + `m.relates_to` with `rel_type: m.annotation` or `rel_type: + m.replace`. Implementations should ignore any such annotation events. + + * When an annotation is redacted, it is removed from the count. + +### Push rules + +Since reactions are considered "metadata" that annotate an existing event, they +should not by default trigger notifications. Thus a new [default override +rule](https://spec.matrix.org/v1.6/client-server-api/#default-override-rules) +is to be added that ignores reaction events: + +```json +{ + "rule_id": ".m.rule.reaction", + "default": true, + "enabled": true, + "conditions": [ + { + "kind": "event_match", + "key": "type", + "pattern": "m.reaction" + } + ], + "actions": [] +} +``` + +The rule is added between `.m.rule.tombstone` and `.m.rule.room.server_acl`. + +(Synapse implementation: [base_rules.rs](https://github.com/matrix-org/synapse/blob/157c571f3e9d3d09cd763405b6a9eb967f2807e7/rust/src/push/base_rules.rs#L216-L229)) + +### Server support + +#### Avoiding duplicate annotations + +Homeservers should prevent users from sending a second annotation for a given +event with identical event `type` and annotation `key` (unless the first event +has been redacted). + +Attempts to send such an annotation should be rejected with a 400 error and an +error code of `M_DUPLICATE_ANNOTATION`. + +Note that this does not guarantee that duplicate annotations will not arrive +over federation. Clients and servers are responsible for deduplicating received +annotations when [counting annotations](#counting-annotations). + +#### Server-side aggregation of `m.annotation` relationships + +`m.annotation` relationships are *not* [aggregated](https://spec.matrix.org/v1.6/client-server-api/#aggregations) +by the server. In other words, `m.annotation` is not included in the `m.relations` property. + +## Alternatives + +### Encrypted reactions + +[matrix-spec#660](https://github.com/matrix-org/matrix-spec/issues/660) +discusses the possibility of encrypting message relationships in general. + +Given that reactions do not rely on server-side aggregation support, an easier +solution to encrypting reactions might be not to use the relationships +framework at all and instead just use a keys within `m.reaction` events, which +could then be encrypted. For example, a reaction could instead be formatted as: + +```json5 +{ + "type": "m.reaction", + "content": { + "event_id": "$some_event_id", + "key": "👍" + } +} +``` + +### Extended annotation use case + +In future it might be useful to be able to annotate events with more +information, some examples include: + + * Annotate commit/PR notification messages with their associated CI state, e.g. + pending/passed/failed. + * If a user issues a command to a bot, e.g. `!deploy-site` the bot could + annotate that event with current state, like "acknowledged", + "redeploying...", "success", "failed", etc. + * Other use cases...? + +However, this doesn't really work with the proposed grouping, as the aggregation +key wouldn't contain the right information needed to display it (unlike for +reactions). + +One way to potentially support this is to include the events (or a subset of the +event) when grouping, so that clients have enough information to render them. +However this dramatically inceases the size of the parent event if we bundle the +full events inside, even if limit the number we bundle in. To reduce the +overhead the annotation event could include a `m.result` field which gets +included. + +This would look something like the following, where the annotation is: + +```json +{ + "type": "m.bot_command_response", + "content": { + "m.result": { + "state": "success", + }, + "m.relates_to": { + "type": "m.annotation", + "key": "" + } + } +} +``` + +and gets bundled into an event like: + +```json +{ + "unsigned": { + "m.relations": { + "m.annotation": [ + { + "type": "m.bot_command_response", + "key": "", + "count": 1, + "chunk": [ + { + "m.result": { + "state": "success", + }, + } + ], + "limited": false, + } + ] + } + } +} +``` + +This is something that could be added later on. A few issues with this are: + + * How does this work with E2EE? How do we encrypt the `m.result`? + * We would end up including old annotations that had been superceded, should + these be done via edits instead? + +## Security considerations + +Clients should render reactions that have a long `key` field in a sensible +manner. For example, clients can elide overly-long reactions. + +If using reactions for upvoting/downvoting purposes we would almost certainly want to anonymise the +reactor, at least from other users if not server admins, to avoid retribution problems. +This gives an unfair advantage to people who run their own servers however and +can cheat and deanonymise (and publish) reactor details. In practice, reactions may +not be best used for upvote/downvote as at the unbundled level they are intrinsically +private data. + +Or in a MSC1228 world... we could let users join the room under an anonymous +persona from a big public server in order to vote? However, such anonymous personae +would lack any reputation data. diff --git a/proposals/2732-olm-fallback-keys.md b/proposals/2732-olm-fallback-keys.md new file mode 100644 index 000000000..5ab90117c --- /dev/null +++ b/proposals/2732-olm-fallback-keys.md @@ -0,0 +1,97 @@ +# MSC2732: Olm fallback keys + +Olm uses a set of one-time keys when initializing a session between two +devices: Alice uploads one-time keys to her homeserver, and Bob claims one of +them to perform a Diffie-Hellman to generate a shared key. As implied by the +name, a one-time key is only to be used once. However, if all of Alice's +one-time keys are claimed, Bob will not be able to create a session with Alice. + +This can be addressed by Alice uploading a fallback key that is used in place +of a one-time key when no one-time keys are available. + +## Proposal + +A new request parameter, `fallback_keys`, is added to the body of the +[`/keys/upload` client-server API](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-keys-upload), which is in the same format as the +`one_time_keys` parameter with the exception that there must be at most one key +per key algorithm. If the user had previously uploaded a fallback key for a +given algorithm, it is replaced -- the server will only keep one fallback key +per algorithm for each user. + +When uploading fallback keys for algorithms whose key format is a signed JSON +object, client should include a property named `fallback` with a value of +`true`. + +Example: + +`POST /keys/upload` + +```json +{ + "fallback_keys": { + "signed_curve25519:AAAAAA": { + "key": "base64+public+key", + "fallback": true, + "signatures": { + "@alice:example.org": { + "ed25519:DEVICEID": "base64+signature" + } + } + } + } +} +``` + +When Bob calls `/keys/claim` to claim one of Alice's one-time keys, but Alice +has no one-time keys left, the homeserver will return the fallback key instead, +if Alice had previously uploaded one. Unlike with one-time keys, fallback keys +are not deleted when they are returned by `/keys/claim`. However, the server +marks that they have been used. + +A new response parameter, `device_unused_fallback_key_types`, is added to +`/sync`. This is an array listing the key algorithms for which the server has +an unused fallback key for the device. If the client wants the server to have a +fallback key for a given key algorithm, but that algorithm is not listed in +`device_unused_fallback_key_types`, the client will upload a new key as above. + +The `device_unused_fallback_key_types` parameter must be present if the server +supports fallback keys. Clients can thus treat this field as an indication +that the server supports fallback keys, and so only upload fallback keys to +servers that support them. + +Example: + +`GET /sync` + +Response: + +```jsonc +{ + // other fields... + "device_unused_fallback_key_types": ["signed_curve25519"] +} +``` + +## Security considerations + +Using a fallback key rather than a one-time key has security implications. An +attacker can replay a message that was originally sent with a fallback key, and +the receiving client will accept it as a new message if the fallback key is +still active. Also, an attacker that compromises a client may be able to +retrieve the private part of the fallback key to decrypt past messages if the +client has still retained the private part of the fallback key. + +For this reason, clients should not store the private part of the fallback key +indefinitely. For example, client should only store at most two fallback keys: +the current fallback key (that it has not yet received any messages for) and +the previous fallback key, and should remove the previous fallback key once it +is reasonably certain that it has received all the messages that use it (for +example, one hour after receiving the first message that used it). + +For addressing replay attacks, clients can also keep track of inbound sessions +to detect replays. + +## Unstable prefix + +The `fallback_keys` request parameter and the `device_unused_fallback_key_types` +response parameter will be prefixed by `org.matrix.msc2732.`. diff --git a/proposals/2778-appservice-login.md b/proposals/2778-appservice-login.md new file mode 100644 index 000000000..fd509ab89 --- /dev/null +++ b/proposals/2778-appservice-login.md @@ -0,0 +1,135 @@ +# MSC2778: Providing authentication method for appservice users + +Appservices within Matrix are increasingly attempting to support End-to-End Encryption. As such, they +need a way to generate devices for their users so that they can participate in E2E rooms. In order to +do so, this proposal suggests implementing an appservice extension to the +[`POST /login` endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-login). + +Appservice users do not usually need to log in as they do not need their own access token, and do not +traditionally need a "device". However, E2E encryption demands that at least one user in a room has a +Matrix device which means bridge users need to be able to generate a device on demand. In the past, +bridge developers have used the bridge bot's device for all bridge users in the room, but this causes +problems should the bridge wish to only join ghosts to a room (e.g. for DMs). + +Another advantage this provides is that an appservice can now be used to generate access tokens for +any user in its namespace without having to set a password for that user, which may be useful where +maintaining password(s) in the configuration is undesirable. + +## Proposal + +A new `type` is to be added to `POST /login`: `m.login.application_service` + +The `/login` endpoint may now take an `access_token` in the same way that other +authenticated endpoints do. No additional parameters should be specified in the request body. + +Example request + +```json +{ + "type": "m.login.application_service", + "identifier": { + "type": "m.id.user", + "user": "_bridge_alice" + } +} +``` + +Note: Implementations MUST use the `identifier.type`=`m.id.user` method of specifying the +localpart. The deprecated top-level `user` field **cannot** use this login flow type. This +is deliberate so as to coax developers into using the new identifier format when implementing +new flows. + +The response body should be unchanged from the existing `/login` specification. + +If one of the following conditions are true: + +- The access token is not provided +- The access token does not correspond to an appservice +- Or the user has not previously been registered + +Then the servers MUST reject with HTTP 403, with an `errcode` of `"M_FORBIDDEN"`. + +If the access token DOES correspond to an appservice but the user is not inside its namespace, +then the `errcode` must be `"M_EXCLUSIVE"`. + +Homeservers should ignore the `access_token` parameter if a type other than +`m.login.application_service` has been provided. + +Appservices creating **new** users can still use the `/register` endpoint to generate an `access_token` / `device_id` +but for existing users, the `/login` endpoint can be used instead. + +## Potential issues + +This proposal means that there will be more calls to make when setting up a appservice user, when +using encryption. While this could be done during the registration step, this would prohibit creating +new devices should the appservice intentionally or inadvertently have lost the client-side device data. + +## Alternatives + +### 1. Include the token in the `/login` request body + +One minor tweak to the current proposal could be to include the token as part of the auth data, rather than +being part of the header/params to the request. An argument could be made for either, but since the specification +expects the appservice to pass the token this way in all requests, including `/register`, it seems wise to keep +it that way. + +### 2. Use implementation specific "shared secret" authentication + +Some community members have used homeserver implementation details such as a "shared secret" authentication method to +log into the accounts without having to use the /login process at all. Synapse provides such a function, +but also means the appservice can now authenticate as any user on the homeserver. This is undesirable from a +security standpoint. + +### 3. Keep using `/register` solely + +A third option could be to create a new endpoint that simply creates a new device for an appservice user on demand. +Given the rest of the matrix eco-system does this with /login, and /login is already extensible with `type`, it would +create more work for all parties involved for little benefit. + +Finally, `POST /register` does already return a `device_id` and `access_token` so appservices +could store this information rather than calling `POST /login` at all. This does however present a few problems: + +- Quite a few appservices which only support unencrypted messaging do not use/store the `device_id`/`access_token` from a register call. + In the event that an appservice eventually gains the ability to support encryption, they would be unable to fetch a new `device_id`/ + `access_token` for any existing users (as `/register` would fail for an existing user). +- If user tokens were lost or exposed, there is no way to programattically create new access tokens for these users. +- Finally, if a user was registered externally and the appservice would like to masquerade as it, it would be unable to fetch + an access token for that user. + +While `POST /register` does work, it is impactical as the sole method of fetching an access token. + +## Security considerations + +Appservices could use this new functionality to generate devices for any userId that are within its namespace e.g. setting the +user namespace regex to `@.*:example.com` would allow appservice to control anyone on the homeserver. While this sounds scary, in practice +this is not a problem because: + +- Appservice namespaces are maintained by the homeserver admin. If the namespace were to change, then it's reasonable + to assume that the server admin is aware. There is no defense mechanism to stop a malicious server admin from creating new + devices for a given user's account as they could also do so by simply modifying the database. + +- While an appservice *could* try to masquerade as a user maliciously without the server admin expecting it, it would still + be bound by the restrictions of the namespace. Server admins are expected to be aware of the implications of adding new + appservices to their server so the burden of responsibility lies with the server admin. + +- Appservices already can /sync as any user using the `as_token` and send any messages as any user in the namespace, the only + difference is that without a dedicated access token they are unable to receive device messages. While in theory this + does make them unable to see encrypted messages, this is not designed to be a security mechanism. + +In conclusion this MSC only automates the creation of new devices for users inside an AS namespace, which is something +a server admin could already do. Appservices should always be treated with care and so with these facts in mind the MSC should +be considered secure. + +## Unstable prefix + +Implementations should use `uk.half-shot.msc2778.login.application_service` for `type` given in the +`POST /login` until this lands in a released version of the specification. + +## Implementations + +The proposal has been implemented by a homeserver, a bridge SDK and two bridges: + +- [synapse](https://github.com/matrix-org/synapse/pull/8320) +- [mautrix-python](https://github.com/tulir/mautrix-python/commit/12d7c48ca7c15fd3ff61608369af1cf69e289aeb) +- [mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp/commit/ead8a869c84d07fadc7cfcf3d522452c99faaa36) +- [matrix-appservice-bridge](https://github.com/matrix-org/matrix-appservice-bridge/pull/231/files#diff-5e93f1b51d50a44fcf0ca46ea1793c1cR851-R864) diff --git a/proposals/2832-appservice-auth-fix.md b/proposals/2832-appservice-auth-fix.md new file mode 100644 index 000000000..412b4d036 --- /dev/null +++ b/proposals/2832-appservice-auth-fix.md @@ -0,0 +1,28 @@ +# MSC2832: Homeserver -> Application Service authorization header +Most of the auth tokens in the spec are passed in the `Authorization` header, +with the `access_token` query parameter supported for backwards-compatibility. +For some reason, the application service spec was not updated in the same way +and it still requires using the archaic query parameter when the homeserver +pushes transactions to the appservice. + +## Proposal +The `access_token` query parameter is removed from all requests made by the +homeserver to appservice and is replaced with the `Authorization` header with +`Bearer ` as the value. + +### Backwards-compatibility +Homeservers which want to support old spec versions in the appservice API may +send both the query parameter and header. Similarly, appservices may accept the +token from either source. + +## Security considerations +Not fixing this causes access tokens to be logged in many bridges. + +## Alternatives +We could add a way for appservices to explicitly specify which spec version +they want in order to implement backwards-compatibility without sending both +tokens. + +## Unstable prefix +The authorization header is already used in the client-server spec, and an +unstable prefix would just unnecessarily complicate things. diff --git a/proposals/2918-refreshtokens.md b/proposals/2918-refreshtokens.md new file mode 100644 index 000000000..8caaef8fa --- /dev/null +++ b/proposals/2918-refreshtokens.md @@ -0,0 +1,163 @@ +# MSC2918: Refresh tokens + +In Matrix, requests to the Client-Server API are currently authenticated using non-expiring, revocable access tokens. +An access token might leak for various reasons, including: + + - leaking from the server database (and its backups) + - intercepting it with a man-in-the-middle attack + - leaking from the client storage (and its backups) + +In the OAuth 2.0 world, this vector of attack is partly mitigated by having expiring access tokens with short lifetimes and rotating refresh tokens to renew them. +This MSC adds support for expiring access tokens and introduces refresh tokens to renew them. +A more [detailed rationale](#detailed-rationale) of what kind of attacks it mitigates lives at the end of this document. + +## Proposal + +Homeservers can choose to have access tokens expire after a short amount of time, forcing the client to renew them with a refresh token. +A refresh token is issued on login and rotates on each usage. + +It allows homeservers to opt for signed and non-revocable access tokens (JWTs, Macaroon, etc.) for performance reasons if their expiration is short enough (less than 5 minutes). + +It is heavily recommended for clients to support refreshing tokens for additional security. +They can advertise their support by adding a `"refresh_token": true` field in the request body on the `/login` and `/register` APIs. + +Handling of clients that do *not* support refreshing access tokens is up to individual homeserver deployments. +For example, server administrators may choose to support such clients for backwards-compatibility, or to expire access tokens anyway for improved security at the cost of inferior user experience in legacy clients. + +If a client uses an access token that has expired, the server will respond with an `M_UNKNOWN_TOKEN` error, preferably with the `soft_logout` parameter set to `true` to improve the user experience in legacy clients. +Thus, if a client receives an `M_UNKNOWN_TOKEN` error, and it has a refresh token available, it should no longer assume that it has been logged out, and instead attempt to refresh the token. +If the client was in fact logged out, then the server will respond with an `M_UNKNOWN_TOKEN` error to the token refresh request, possibly with the `soft_logout` parameter set. + +### Login API changes + +The login API returns two additional fields: + +- `expires_in_ms`: The lifetime in milliseconds of the access token. +- `refresh_token`: The refresh token, which can be used to obtain new access tokens. + +This also applies to logins done by application services. + +Both fields are optional. +If `expires_in_ms` is missing, the client can assume the access token won't expire. +If `refresh_token` is missing but `expires_in_ms` is present, the client can assume the access token will expire but it won't have a way to refresh the access token without re-logging in. + +Clients advertise their support for refreshing tokens by setting the `refresh_token` field to `true` in the request body. + +### Account registration API changes + +Unless `inhibit_login` is `true`, the account registration API returns two additional fields: + +- `expires_in_ms`: The lifetime in milliseconds of the access token. +- `refresh_token`: The refresh token, which can be used to obtain new access tokens. + +This also applies to registrations done by application services. + +As in the login API, both field are optional. + +Clients advertise their support for refreshing tokens by setting the `refresh_token` field to `true` in the request body. + +### Token refresh API + +This API lets the client refresh the access token. +A new refresh token is also issued. +The existing refresh token remains valid until the new access token (or refresh token) is used, at which point it is revoked. +This allows for the request to get lost in flight. +The Matrix server can revoke the old access token right away, but does not have to since its lifetime is short enough that it will expire anyway soon after. + +`POST /_matrix/client/r0/refresh` + +```json +{ + "refresh_token": "aaaabbbbccccdddd" +} +``` + +response: + +```json +{ + "access_token": "xxxxyyyyzzz", + "expires_in_ms": 60000, + "refresh_token": "eeeeffffgggghhhh" +} +``` + +If the `refresh_token` is missing from the response, the client can assume the refresh token has not changed and use the same token in subsequent token refresh API requests. + +The `refresh_token` parameter can be invalid for two reasons: + + - if it does not exist + - if it was already used once + +In both cases, the server must reply with a `401` HTTP status code and an `M_UNKNOWN_TOKEN` error code. +This new use case of the `M_UNKNOWN_TOKEN` error code must be reflected in the spec. +As with other endpoints, the server can include an extra `soft_logout` parameter in the response to signify the client it should do a soft logout. + +This new API should be rate-limited and does not require authentication since only the `refresh_token` parameter is needed. +Identity assertion via the `user_id` query parameter as defined by the Application Service API specification is disabled on this endpoint. + +### Device handling + +The current spec states that "Matrix servers should record which device each access token is assigned to". +This must be updated to reflect that devices are bound to a session, which are created during login and stays the same after refreshing the token. + +## Potential issues + +The refresh token being rotated on each refresh is strongly recommended in the OAuth 2.0 world for unauthenticated clients to avoid token replay attacks. +This can however make the deployment of CLI tools for Matrix a bit harder, since the credentials can't be statically defined anymore. +This is not an issue in OAuth 2.0 because usually CLI tools use the client credentials flow, also known as service accounts. +An alternative would be to make the refresh token non-rotating for now but recommend clients to support rotation of refresh tokens and enforce it later on. + +## Alternatives + +This MSC defines a new endpoint for token refresh, but it could also be integrated as a new authentication mechanism. + +## Security considerations + +The time to live (TTL) of access tokens isn't enforced in this MSC but is advised to be kept relatively short. +Servers might choose to have stateless, digitally signed access tokens (JWT are good examples of this), which makes them non-revocable. +The TTL of access tokens should be around 15 minutes if they are revocable and should not exceed 5 minutes if they are not. + +## Unstable prefix + +While this MSC is not in a released version of the specification, clients should use the `org.matrix.msc2918.refresh_token` field in place of the `refresh_token` field in requests to the login and registration endpoints. +The refresh token endpoint should be served and used using the unstable prefix: `POST /_matrix/client/unstable/org.matrix.msc2918/refresh`. + +## Detailed rationale + +This MSC does not aim to protect against a completely compromised client. +More specifically, it does not protect against an attacker that managed to distribute an alternate, compromised version of the client to users. +In contrast, it protects against a whole range of attacks where the access token and/or refresh token get leaked but the client isn't completely compromised. + +For example, those tokens can leak from user backups (user backs up his device on a NAS, the NAS gets compromised and leaks a backup of the client's secret storage), but one can assume those backups could be at least 5 min old. +If the leak only includes the access token, it is useless to the attacker since it would have expired. +If it also includes the refresh token, it is useless *if* the token was refreshed before (which will happen if the user just opens their Matrix client in between). + +Worst case scenario, the leaked refresh token is still valid: in this case, the attacker would consume the refresh token to get a valid access token, but when the original client tries to use the same refresh token, the homeserver can detect it, consider the session has been compromised, end the session and warn the user. + +This kind of attack also applies to leakage from the server, which could happen from database backups, for example. + +The important thing here is while it does not completely prevent attacks in case of a token leakage, it does make this range of attack a lot more time-sensitive and detectable. +A homeserver will notice if a refresh token is being used twice. + +The IETF has interesting [guidelines for refresh tokens](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics#section-4.13.2). +They recommend that either: + + - the refresh tokens are sender-bound and require client authentication (making token leakage completely useless if the client credentials are not leaked at the same time) + - or make them rotate to make the attack a lot harder, as described just above. + +Since all clients are "public" in the Matrix world, there are no client-bound credentials that could be used, hence the rotation of refresh tokens. + +--- + +The other kind of scenario where this change makes sense is to help further changes in the homeservers. +A good, recent example of this, is in Synapse v1.34.0 [they moved away from macaroons for access tokens](https://github.com/matrix-org/synapse/pull/5588) to random, shorter, saved in database tokens, similar to [what GitHub did recently](https://github.blog/2021-04-05-behind-githubs-new-authentication-token-formats/). + +Because there is no refresh token mechanism in the C2S API, most Synapse instances now have a mix of the two formats of tokens, and for a long time. +It makes it impossible to enforce the new format of tokens without invalidating all existing sessions, making it impossible to roll out changes like a web-app firewall in front of Synapse that verifies the shape and checksums of tokens even before reaching Synapse. + +--- + +Lastly, expiring tokens already exist in Synapse (via the `session_lifetime` configuration parameter). +Before this MSC, clients had no idea when the session would end and relied on the server replying with a 401 error with `soft_logout: true` in the response on a random request to trigger a soft logout and go through the authentication process again. +A side effect of this MSC (although it could have been introduced separately) is that the login responses can now include a `expires_in_ms` to inform the clients when the token will expire. diff --git a/proposals/2946-spaces-summary.md b/proposals/2946-spaces-summary.md new file mode 100644 index 000000000..fc0c136b8 --- /dev/null +++ b/proposals/2946-spaces-summary.md @@ -0,0 +1,389 @@ +# MSC2946: Spaces Summary + +This MSC depends on [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772), which +describes why a Space is useful: + +> Collecting rooms together into groups is useful for a number of purposes. Examples include: +> +> * Allowing users to discover different rooms related to a particular topic: for example "official matrix.org rooms". +> * Allowing administrators to manage permissions across a number of rooms: for example "a new employee has joined my company and needs access to all of our rooms". +> * Letting users classify their rooms: for example, separating "work" from "personal" rooms. +> +> We refer to such collections of rooms as "spaces". + +This MSC attempts to solve how a member of a space discovers rooms in that space. This +is useful for quickly exposing a user to many aspects of an entire community, using the +examples above, joining the "official matrix.org rooms" space might suggest joining a few +rooms: + +* A room to discuss development of the Matrix Spec. +* An announcements room for news related to matrix.org. +* An off-topic room for members of the space. + +## Proposal + +A new client-server API (and corresponding server-server API) is added which allows +for querying for the rooms and spaces contained within a space. This allows a client +to efficiently display a hierarchy of rooms to a user (i.e. without having +to walk the full state of each room). + +### Client-server API + +An endpoint is provided to walk the space tree, starting at the provided room ID +("the root room"), and visiting other rooms/spaces found via `m.space.child` +events. It recurses into the children and into their children, etc. + +Any child room that the user is joined or is potentially joinable (per +[MSC3173](https://github.com/matrix-org/matrix-doc/pull/3173)) is included in +the response. When a room with a `type` of `m.space` is found, it is searched +for valid `m.space.child` events to recurse into. + +In order to provide a consistent experience, the space tree should be walked in +a depth-first manner, e.g. whenever a space is found it should be recursed into +by sorting the children rooms and iterating through them. + +There could be loops in the returned child events; clients and servers should +handle this gracefully. Similarly, note that a child room might appear multiple +times (e.g. also be a grandchild). Clients and servers should handle this +appropriately. + +This endpoint requires authentication and is subject to rate-limiting. + +#### Request format + +```text +GET /_matrix/client/v1/rooms/{roomID}/hierarchy +``` + +Query Parameters: + +* **`suggested_only`**: Optional. If `true`, return only child events and rooms + where the `m.space.child` event has `suggested: true`. Must be a boolean, + defaults to `false`. + + This applies transitively, i.e. if a `suggested_only` is `true` and a space is + not suggested then it should not be searched for children. The inverse is also + true, if a space is suggested, but a child of that space is not then the child + should not be included. +* **`limit`**: Optional: a client-defined limit to the maximum + number of rooms to return per page. Must an integer greater than zero. + + Server implementations should impose a maximum value to avoid resource + exhaustion. +* **`max_depth`**: Optional: The maximum depth in the tree (from the root room) + to return. The deepest depth returned will not include children events. Defaults + to no-limit. Must be a non-negative integer. + + Server implementations may wish to impose a maximum value to avoid resource + exhaustion. +* **`from`**: Optional. Pagination token given to retrieve the next set of rooms. + + Note that if a pagination token is provided, then the parameters given for + `suggested_only` and `max_depth` must be the same. + +#### Response Format + +* **`rooms`**: `[object]` For each room/space, starting with the root room, a + summary of that room. The fields are the same as those returned by + `/publicRooms` (see + [spec](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-publicrooms)), + with the addition of: + * **`room_type`**: the value of the `m.type` field from the room's + `m.room.create` event, if any. + * **`children_state`**: The stripped state of the `m.space.child` events of + the room per [MSC3173](https://github.com/matrix-org/matrix-doc/pull/3173). + In addition to the standard stripped state fields, the following is included: + * **`origin_server_ts`**: `integer`. The `origin_server_ts` field from the + room's `m.space.child` event. This is required for sorting of rooms as + specified below. +* **`next_batch`**: Optional `string`. The token to supply in the `from` param + of the next `/hierarchy` request in order to request more rooms. If this is absent, + there are no more results. + +#### Example request: + +```text +GET /_matrix/client/v1/rooms/%21ol19s%3Ableecker.street/hierarchy? + limit=30& + suggested_only=true& + max_depth=4 +``` + +#### Example response: + +```jsonc +{ + "rooms": [ + { + "room_id": "!ol19s:bleecker.street", + "avatar_url": "mxc://bleecker.street/CHEDDARandBRIE", + "guest_can_join": false, + "name": "CHEESE", + "num_joined_members": 37, + "topic": "Tasty tasty cheese", + "world_readable": true, + "join_rules": "public", + "room_type": "m.space", + "children_state": [ + { + "type": "m.space.child", + "state_key": "!efgh:example.com", + "content": { + "via": ["example.com"], + "suggested": true + }, + "room_id": "!ol19s:bleecker.street", + "sender": "@alice:bleecker.street", + "origin_server_ts": 1432735824653 + }, + { ... } + ] + }, + { ... } + ], + "next_batch": "abcdef" +} +``` + +#### Errors: + +An HTTP response with a status code of 403 and an error code of `M_FORBIDDEN` +should be returned if the user doesn't have permission to view/peek the root room. +This should also be returned if that room does not exist, which matches the +behavior of other room endpoints (e.g. +[`/_matrix/client/r0/rooms/{roomID}/aliases`](https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-rooms-roomid-aliases)) +to not divulge that a room exists which the user doesn't have permission to view. + +An HTTP response with a status code of 400 and an error code of `M_INVALID_PARAM` +should be returned if the `from` token provided is unknown to the server or if +the `suggested_only` or `max_depth` parameters are modified during pagination. + +#### Server behaviour + +The server should generate the response as discussed above, by doing a depth-first +search (starting at the "root" room) for any `m.space.child` events. Any +`m.space.child` with an invalid `via` are discarded (invalid is defined as in +[MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772): missing, not an +array or an empty array). + +In the case of the homeserver not having access to the state of a room, the +server-server API (see below) can be used to query for this information over +federation from one of the servers provided in the `via` key of the +`m.space.child` event. It is recommended to cache the federation response for a +period of time. The federation results may contain information on a room +that the requesting server is already participating in; the requesting server +should use its local data for such rooms rather than the data returned over +federation. + +When the current response page is full, the current state should be persisted +and a pagination token should be generated (if there is more data to return). +To prevent resource exhaustion, the server may expire persisted data that it +deems to be stale. + +The persisted state will include: + +* The processed rooms. +* Rooms to process (in depth-first order with rooms at the same depth + ordered [according to MSC1772, as updated to below](#msc1772-ordering)). +* Room information from federation responses for rooms which have yet to be + processed. + +### Server-server API + +The Server-Server API has a similar interface to the Client-Server API, but a +simplified response. It is used when a homeserver is not participating in a room +(and cannot summarize room due to not having the state). + +The main difference is that it does *not* recurse into spaces and does not support +pagination. This is somewhat equivalent to a Client-Server request with a `max_depth=1`. + +Additional federation requests are made to recurse into sub-spaces. This allows +for trivially caching responses for a short period of time (since it is not +easily known the room summary might have changed). + +Since the server-server API does not know the requesting user, the response should +divulge information based on if any member of the requesting server could join +the room. The requesting server is trusted to properly filter this information +using the `world_readable`, `join_rules`, and `allowed_room_ids` fields from the +response. + +If the target server is not a member of some children rooms (so would have to send +another request over federation to inspect them), no attempt is made to recurse +into them. They are simply omitted from the `children` key of the response. +(Although they will still appear in the `children_state`key of the `room`.) + +Similarly, if a server-set limit on the size of the response is reached, additional +rooms are not added to the response and can be queried individually. + +#### Request format + +```text +GET /_matrix/federation/v1/hierarchy/{roomID} +``` + +Query Parameters: + +* **`suggested_only`**: The same as the Client-Server API. + +#### Response format + +The response format is similar to the Client-Server API: + +* **`room`**: `object` The summary of the requested room, see below for details. +* **`children`**: `[object]` For each room/space, a summary of that room, see + below for details. +* **`inaccessible_children`**: Optional `[string]`. A list of room IDs which are + children of the requested room, but are inaccessible to the requesting server. + Assuming the target server is non-malicious and well-behaved, then other + non-malicious servers should respond with the same set of inaccessible rooms. + Thus the requesting server can consider the rooms inaccessible from everywhere. + + This is used to differentiate between rooms which the requesting server does + not have access to from those that the target server cannot include in the + response (which will simply be missing in the response). + +For both the `room` and `children` fields the summary of the room/space includes +the fields returned by `/publicRooms` (see [spec](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-publicrooms)), +with the addition of: + +* **`room_type`**: the value of the `m.type` field from the room's `m.room.create` + event, if any. +* **`allowed_room_ids`**: A list of room IDs which give access to this room per + [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083).[1](#f1) + +#### Example request: + +```jsonc +GET /_matrix/federation/v1/hierarchy/{roomID}? + suggested_only=true +``` + +#### Errors: + +An HTTP response with a status code of 404 and an error code of `M_NOT_FOUND` is +returned if the target server is not a member of the requested room or the +requesting server is not allowed to access the room. + +### MSC1772 Ordering + +[MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) defines the ordering +of "default ordering of siblings in the room list" using the `order` key: + +> Rooms are sorted based on a lexicographic ordering of the Unicode codepoints +> of the characters in `order` values. Rooms with no `order` come last, in +> ascending numeric order of the `origin_server_ts` of their `m.room.create` +> events, or ascending lexicographic order of their `room_id`s in case of equal +> `origin_server_ts`. `order`s which are not strings, or do not consist solely +> of ascii characters in the range `\x20` (space) to `\x7F` (~), or consist of +> more than 50 characters, are forbidden and the field should be ignored if +> received. + +Unfortunately there are situations when a homeserver comes across a reference to +a child room that is unknown to it and must decide the ordering. Without being +able to see the `m.room.create` event (which it might not have permission to see) +no proper ordering can be given. + +Consider the following case of a space with 3 child rooms: + +``` + Space A + | + +--------+--------+ + | | | +Room B Room C Room D +``` + +HS1 has users in Space A, Room B, and Room C, while HS2 has users in Room D. HS1 has no users +in Room D (and thus has no state from it). Room B, C, and D do not have an +`order` field set (and default to using the ordering rules above). + +When a user asks HS1 for the space summary with a `limit` equal to `2` it cannot +fulfill this request since it is unsure how to order Room B, Room C, and Room D, +but it can only return 2 of them. It *can* reach out over federation to HS2 and +request a space summary for Room D, but this is undesirable: + +* HS1 might not have the permissions to know any of the state of Room D, so might + receive a 404 error. +* If we expand the example above to many rooms than this becomes expensive to + query a remote server simply for ordering. + +This proposes changing the ordering rules from MSC1772 to the following: + +> Rooms are sorted based on a lexicographic ordering of the Unicode codepoints +> of the characters in `order` values. Rooms with no `order` come last, in +> ascending numeric order of the `origin_server_ts` of their `m.space.child` +> events, or ascending lexicographic order of their `room_id`s in case of equal +> `origin_server_ts`. `order`s which are not strings, or do not consist solely +> of ascii characters in the range `\x20` (space) to `\x7E` (~), or consist of +> more than 50 characters, are forbidden and the field should be ignored if +> received. + +This modifies the clause for calculating the order to use the `origin_server_ts` +of the `m.space.child` event instead of the `m.room.create` event. This allows +for a defined sorting of siblings based purely on the information available in +the state of the space while still allowing for a natural ordering due to the +age of the relationship. + +## Potential issues + +A large flat space (a single room with many `m.space.child` events) could cause +a large federation response. + +Room version upgrades of rooms in a space are unsolved and left to a future MSC. +When upgrading a room it is unclear if the old room should be removed (in which +case users who have not yet joined the new room will no longer see it in the space) +or leave the old room (in which case users who have joined the new room will see +both). The current recommendation is for clients de-duplicate rooms which are +known old versions of rooms in the space. + +## Alternatives + +Peeking to explore the room state could be used to build the tree of rooms/spaces, +but this would be significantly more expensive for both clients and servers. It +would also require peeking over federation (which is explored in +[MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444)). + +## Security considerations + +A space with many sub-spaces and rooms on different homeservers could cause +a large number of federation requests. A carefully crafted space with inadequate +server enforced limits could be used in a denial of service attack. Generally +this is mitigated by enforcing server limits and caching of responses. + +The requesting server over federation is trusted to filter the response for the +requesting user. The alternative, where the requesting server sends the requesting +`user_id`, and the target server does the filtering, is unattractive because it +rules out a caching of the result. This does not decrease security since a server +could lie and make a request on behalf of a user in the proper space to see the +given information. I.e. the calling server must be trusted anyway. + +## Unstable prefix + +During development of this feature it will be available at unstable endpoints. + +The client-server API will be: +`/_matrix/client/unstable/org.matrix.msc2946/rooms/{roomID}/hierarchy` + +The server-server API will be: +`/_matrix/federation/unstable/org.matrix.msc2946/hierarchy/{roomID}` + +## Footnotes + +[1]: As a worked example, in the context of +[MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083), consider that Alice +and Bob share a server; Alice is a member of a space, but Bob is not. A remote +server will not know whether the request is on behalf of Alice or Bob (and hence +whether it should share details of restricted rooms within that space). + +Consider if the space is modified to include a restricted room on a different server +which allows access from the space. When summarizing the space, the homeserver must make +a request over federation for information on the room. The response should include +the room (since Alice is able to join it). Without additional information the +calling server does not know *why* they received the room and cannot properly +filter the returned results. + +Note that there are still potential situations where each server individually +doesn't have enough information to properly return the full summary, but these +do not seem reasonable in what is considered a normal structure of spaces. (E.g. +in the above example, if the remote server is not in the space and does not know +whether the server is in the space or not it cannot return the room.)[↩](#a1) diff --git a/proposals/3030-jump-to-date.md b/proposals/3030-jump-to-date.md new file mode 100644 index 000000000..e19ce415b --- /dev/null +++ b/proposals/3030-jump-to-date.md @@ -0,0 +1,286 @@ +# MSC3030: Jump to date API endpoint + +Add an API that makes it easy to find the closest messages for a given +timestamp. + +The goal of this change is to have clients be able to implement a jump to date +feature in order to see messages back at a given point in time. Pick a date from +a calender, heatmap, or paginate next/previous between days and view all of the +messages that were sent on that date. + +Alongside the [roadmap of feature parity with +Gitter](https://github.com/vector-im/roadmap/issues/26), we're also interested +in using this for a new better static Matrix archive. Our idea is to server-side +render [Hydrogen](https://github.com/vector-im/hydrogen-web) and this new +endpoint would allow us to jump back on the fly without having to paginate and +keep track of everything in order to display the selected date. + +Also useful for archiving and backup use cases. This new endpoint can be used to +slice the messages by day and persist to file. + +Related issue: [*URL for an arbitrary day of history and navigation for next and +previous days* +(vector-im/element-web#7677)](https://github.com/vector-im/element-web/issues/7677) + + +## Problem + +These types of use cases are not supported by the current Matrix API because it +has no way to fetch or filter older messages besides a manual brute force +pagination from the most recent event in the room. Paginating is time-consuming +and expensive to process every event as you go (not practical for clients). +Imagine wanting to get a message from 3 years ago 😫 + + +## Proposal + +Add new client API endpoint `GET +/_matrix/client/v1/rooms/{roomId}/timestamp_to_event?ts=&dir=[f|b]` +which fetches the closest `event_id` to the given timestamp `ts` query parameter +in the direction specified by the `dir` query parameter. The direction `dir` +query parameter accepts `f` for forward-in-time from the timestamp and `b` for +backward-in-time from the timestamp. This endpoint also returns +`origin_server_ts` to make it easy to do a quick comparison to see if the +`event_id` fetched is too far out of range to be useful for your use case. + +When an event can't be found in the given direction, the endpoint throws a 404 +`"errcode":"M_NOT_FOUND",` (example error message `"error":"Unable to find event +from 1672531200000 in direction f"`). + +In order to solve the problem where a homeserver does not have all of the history in a +room and no suitably close event, we also add a server API endpoint `GET +/_matrix/federation/v1/timestamp_to_event/{roomId}?ts=?dir=[f|b]` which other +homeservers can use to ask about their closest `event_id` to the timestamp. This +endpoint also returns `origin_server_ts` to make it easy to do a quick comparison to see +if the remote `event_id` fetched is closer than the local one. After the local +homeserver receives a response from the federation endpoint, it probably should +try to backfill this event via the federation `/event/` endpoint so that it's +available to query with `/context` from a client in order to get a pagination token. + +The heuristics for deciding when to ask another homeserver for a closer event if +your homeserver doesn't have something close, are left up to the homeserver +implementation, although the heuristics will probably be based on whether the +closest event is a forward/backward extremity indicating it's next to a gap of +events which are potentially closer. + +A good heuristic for which servers to try first is to sort by servers that have +been in the room the longest because they're most likely to have anything we ask +about. + +These endpoints are authenticated and should be rate-limited like similar client +and federation endpoints to prevent resource exhaustion abuse. + +``` +GET /_matrix/client/v1/rooms//timestamp_to_event?ts=&dir= +{ + "event_id": ... + "origin_server_ts": ... +} +``` + +Federation API endpoint: +``` +GET /_matrix/federation/v1/timestamp_to_event/?ts=&dir= +{ + "event_id": ... + "origin_server_ts": ... +} +``` + +--- + +In order to paginate `/messages`, we need a pagination token which we can get +using `GET /_matrix/client/r0/rooms/{roomId}/context/{eventId}?limit=0` for the +`event_id` returned by `/timestamp_to_event`. + +We can always iterate on `/timestamp_to_event` later and return a pagination +token directly in another MSC ⏩ + + +## Potential issues + +### Receiving a rogue random delayed event ID + +Since `origin_server_ts` is not enforcably accurate, we can only hope that an event's +`origin_server_ts` is relevant enough to its `prev_events` and descendants. + +If you ask for "the message with `origin_server_ts` closest to Jan 1st 2018" you +might actually get a rogue random delayed one that was backfilled from a +federated server, but the human can figure that out by trying again with a +slight variation on the date or something. + +Since there isn't a good or fool-proof way to combat this, it's probably best to just go +with `origin_server_ts` and not let perfect be the enemy of good. + + +### Receiving an unrenderable event ID + +Another issue is that clients could land on an event they can't/won't render, +such as a reaction, then they'll be forced to desperately seek around the +timeline until they find an event they can do something with. + +Eg: + - Client wants to jump to January 1st, 2022 + - Server says there's an event on January 2nd, 2022 that is close enough + - Client finds out there's a ton of unrenderable events like memberships, poll responses, reactions, etc at that time + - Client starts paginating forwards, finally finding an event on January 27th it can render + - Client wasn't aware that the actual nearest neighbouring event was backwards on December 28th, 2021 because it didn't paginate in that direction + - User is confused that they are a month past the target date when the message is *right there*. + +Clients can be smarter here though. Clients can see when events were sent as +they paginate and if they see they're going more than a couple days out, they +can also try the other direction before going further and further away. + +Clients can also just explain to the user what happened with a little toast: "We +were unable to find an event to display on January 1st, 2022. The closest event +after that date is on January 27th." + + +### Abusing the `/timestamp_to_event` API to get the `m.room.create` event + +Although it's possible to jump to the start of the room and get the first event in the +room (`m.room.create`) with `/timestamp_to_event?dir=f&ts=0`, clients should still use +`GET /_matrix/client/v3/rooms/{roomId}/state/m.room.create/` to get the room creation +event. + +In the future, with things like importing history via +[MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716), the first +event you encounter with `/timestamp_to_event?dir=f&ts=0` could be an imported event before +the room was created. + + +## Alternatives + +We chose the current `/timestamp_to_event` route because it sounded like the +easist path forward to bring it to fruition and get some real-world experience. +And was on our mind during the [initial discussion](https://docs.google.com/document/d/1KCEmpnGr4J-I8EeaVQ8QJZKBDu53ViI7V62y5BzfXr0/edit#bookmark=id.qu9k9wje9pxm) because there was some prior art with a [WIP +implementation](https://github.com/matrix-org/synapse/pull/9445/commits/91b1b3606c9fb9eede0a6963bc42dfb70635449f) +from @erikjohnston. The alternatives haven't been thrown out for a particular +reason and we could still go down those routes depending on how people like the +current design. + + +### Paginate `/messages?around=` from timestamp + +Add the `?around=` query parameter to the `GET +/_matrix/client/r0/rooms/{roomId}/messages` endpoint. This will start the +response at the message with `origin_server_ts` closest to the provided `around` +timestamp. The direction is determined by the existing `?dir` query parameter. + +Use topological ordering, just as Element would use if you follow a permalink. + +This alternative could be confusing to the end-user around how this plays with +the existing query parameters +`/messages?from={paginationToken}&to={paginationToken}` which also determine +what part of the timeline to query. Those parameters could be extended to accept +timestamps in addition to pagination tokens but then could get confusing again +when you start mixing timestamps and pagination tokens. The homeserver also has +to disambiguate what a pagination token looks like vs a unix timestamp. Since +pagination tokens don't follow a certain convention, some homeserver +implementations may already be using arbitrary number tokens already which would +be impossible to distinguish from a timestamp. + +A related alternative is to use `/messages` with a `from_time`/`to_time` (or +`from_ts`/`to_ts`) query parameters that only accept timestamps which solves the +confusion and disambigution problem of trying to re-use the existing `from`/`to` +query paramters. Re-using `/messages` would reduce the number of round-trips and +potentially client-side implementations for the use case where you want to fetch +a window of messages from a given time. But has the same round-trip problem if +you want to use the returned `event_id` with `/context` or another endpoint +instead. + + +### Filter by date in `RoomEventFilter` + +Extend `RoomEventFilter` to be able to specify a timestamp or a date range. The +`RoomEventFilter` can be passed via the `?filter` query param on the `/messages` +endpoint. + +This suffers from the same confusion to the end-user of how it plays with how +this plays with `/messages?from={paginationToken}&to={paginationToken}` which +also determines what part of the timeline to query. + + +### Return the closest event in any direction + +We considered omitting the `dir` parameter (or allowing `dir=c`) to have the server +return the closest event to the timestamp, regardless of direction. However, this seems +to offer little benefit. + +Firstly, for some usecases (such as archive viewing, where we want to show all the +messages that happened on a particular day), an explicit direction is important, so this +would have to be optional behaviour. + +For a regular messaging client, "directionless" search also offers little benefit: it is +easy for the client to repeat the request in the other direction if the returned event +is "too far away", and in any case it needs to manage an iterative search to handle +unrenderable events, as discussed above. + +Implementing a directionless search on the server carries a performance overhead, since +it must search both forwards and backwards on every request. In short, there is little +reason to expect that a single `dir=c` request would be any more efficient than a pair of +requests with `dir=b` and `dir=f`. + +### New `destination_server_ts` field + +Add a new field and index on messages called `destination_server_ts` which +indicates when the message was received from federation. This gives a more +"real" time for how someone would actually consume those messages. + +The contract of the API is "show me messages my server received at time T" +rather than the messy confusion of showing a delayed message which happened to +originally be sent at time T. + +We've decided against this approach because the backfill from federated servers +could be horribly late. + +--- + +Related issue around `/sync` vs `/messages`, +https://github.com/matrix-org/synapse/issues/7164 + +> Sync returns things in the order they arrive at the server; backfill returns +> them in the order determined by the event graph. +> +> *-- @richvdh, https://github.com/matrix-org/synapse/issues/7164#issuecomment-605877176* + +> The general idea is that, if you're following a room in real-time (ie, +> `/sync`), you probably want to see the messages as they arrive at your server, +> rather than skipping any that arrived late; whereas if you're looking at a +> historical section of timeline (ie, `/messages`), you want to see the best +> representation of the state of the room as others were seeing it at the time. +> +> *-- @richvdh , https://github.com/matrix-org/synapse/issues/7164#issuecomment-605953296* + + +## Security considerations + +We're only going to expose messages according to the existing message history +setting in the room (`m.room.history_visibility`). No extra data is exposed, +just a new way to sort through it all. + + + +## Unstable prefix + +While this MSC is not considered stable, the endpoints are available at `/unstable/org.matrix.msc3030` instead of their `/v1` description from above. + +``` +GET /_matrix/client/unstable/org.matrix.msc3030/rooms//timestamp_to_event?ts=&dir= +{ + "event_id": ... + "origin_server_ts": ... +} +``` + +``` +GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/?ts=&dir= +{ + "event_id": ... + "origin_server_ts": ... +} +``` + +Servers will indicate support for the new endpoint via a non-empty value for feature flag +`org.matrix.msc3030` in `unstable_features` in the response to `GET +/_matrix/client/versions`. diff --git a/proposals/3069-guests-whoami.md b/proposals/3069-guests-whoami.md new file mode 100644 index 000000000..7666dfb23 --- /dev/null +++ b/proposals/3069-guests-whoami.md @@ -0,0 +1,28 @@ +# MSC3069: Allow guests to use /account/whoami + +Currently the [/account/whoami](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-account-whoami) +endpoint does not mention anything about guests, which is a bit of an oversight. The implementation +of the endpoint got created such that guest access was declined. + +## Proposal + +Guests are allowed to use `/account/whoami`. When a guest makes a request, the response will have +an added `is_guest: true` field - this field is optional (default `false`) otherwise. + +## Potential issues + +None forseen. This corrects a mistake. + +## Alternatives + +None relevant. + +## Security considerations + +Guests will be able to know their user ID, as they would when they registered in the first place. + +## Unstable prefix + +While this MSC is not in a stable version of the specification, implementations should use +`org.matrix.msc3069.is_guest` in place of `is_guest`. Callers should note that they might see +`M_GUEST_ACCESS_FORBIDDEN` errors if the server is not implementing this MSC. diff --git a/proposals/3083-restricted-rooms.md b/proposals/3083-restricted-rooms.md new file mode 100644 index 000000000..1caa9f970 --- /dev/null +++ b/proposals/3083-restricted-rooms.md @@ -0,0 +1,291 @@ +# Restricting room membership based on membership in other rooms + +A desirable feature is to give room admins the power to restrict membership of +their room based on the membership of one or more rooms. + +Potential usecases include: + +* Private spaces (allowing any member of a [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) + space to join child rooms in that space), for example: + + > members of the #doglovers:example.com space can join this room without an invitation[1](#f1) +* Room upgrades for private rooms (instead of issuing invites to each user). +* Allowing all users in a private room to be able to join a private breakout room. + +This does not preclude members from being directly invited to the room, which is +still a useful discovery feature. + +## Proposal + +In a future room version a new `join_rule` (`restricted`) will be used to reflect +a cross between `invite` and `public` join rules. The content of the join rules +would include the rooms to trust for membership. For example: + +```json +{ + "type": "m.room.join_rules", + "state_key": "", + "content": { + "join_rule": "restricted", + "allow": [ + { + "type": "m.room_membership", + "room_id": "!mods:example.org" + }, + { + "type": "m.room_membership", + "room_id": "!users:example.org" + } + ] + } +} +``` + +This means that a user must be a member of the `!mods:example.org` room or +`!users:example.org` room in order to join without an invite[2](#f2). +Membership in a single allowed room is enough. + +If the `allow` key is an empty list (or not a list at all), then no users are +allowed to join without an invite. Each entry is expected to be an object with the +following keys: + +* `type`: `"m.room_membership"` to describe that we are allowing access via room + membership. Future MSCs may define other types. +* `room_id`: The room ID to check the membership of. + +Any entries in the list which do not match the expected format are ignored. Thus, +if all entries are invalid, the list behaves as if empty and all users without +an invite are rejected. + +The `allow` key is to be protected when redacting an event. + +When a homeserver receives a `/join` request from a client or a `/make_join` / +`/send_join` request from another homeserver, the request should only be permitted +if the user is invited to this room, or is joined to one of the listed rooms. If +the user is not a member of at least one of the rooms, the homeserver should return +an error response with HTTP status code of 403 and an `errcode` of `M_FORBIDDEN`. + +It is possible for a resident homeserver (one which receives a `/make_join` / +`/send_join` request) to not know if the user is in some of the allowed rooms (due +to not participating in them). If the user is not in any of the allowed rooms that +are known to the homeserver, and the homeserver is not participating in all listed +rooms, then it should return an error response with HTTP status code of 400 with an `errcode` of `M_UNABLE_TO_AUTHORISE_JOIN`. The joining server should +attempt to join via another resident homeserver. If the resident homeserver knows +that the user is not in *any* of the allowed rooms it should return an error response +with HTTP status code of 403 and an `errcode` of `M_FORBIDDEN`. Note that it is a +configuration error if there are allowed rooms with no participating authorised +servers. + +A chosen resident homeserver might also be unable to issue invites (which, as below, +is a pre-requisite for generating a correctly-signed join event). In this case +it should return an error response with HTTP status code of 400 and an `errcode` +of `M_UNABLE_TO_GRANT_JOIN`. The joining server should attempt to join via another +resident homeserver. + +From the perspective of the [auth rules](https://spec.matrix.org/unstable/rooms/v1/#authorization-rules), +the `restricted` join rule has the same behavior as `public`, with the additional +caveat that servers must ensure that, for `m.room.member` events with a `membership` of `join`: + +* The user's previous membership was `invite` or `join`, or +* The join event has a valid signature from a homeserver whose users have the + power to issue invites. + + When generating a join event for `/join` or `/make_join`, the server should + include the MXID of a local user who could issue an invite in the content with + the key `join_authorised_via_users_server`. The actual user chosen is arbitrary. + +The changes to the auth rules imply that: + +* A join event issued via `/send_join` is signed by not just the requesting + server, but also the resident server.[3](#f3) + + In order for the joining server to receive the proper signatures the join + event will be returned via `/send_join` in the `event` field. +* The auth chain of the join event needs to include events which prove + the homeserver can be issuing the join. This can be done by including: + + * The `m.room.power_levels` event. + * The join event of the user specified in `join_authorised_via_users_server`. + + It should be confirmed that the authorising user is in the room. (This + prevents situations where any homeserver could process the join, even if + they weren't in the room, under certain power level conditions.) + +The above creates a new restriction on the relationship between the resident +servers used for `/make_join` and `/send_join` -- they must now both go to +the same server (since the `join_authorised_via_users_server` is added in +the call to `/make_join`, while the final signature is added during +the call to `/send_join`). If a request to `/send_join` is received that includes +an event from a different resident server it should return an error response with +HTTP status code of 400. + +Note that the homeservers whose users can issue invites are trusted to confirm +that the `allow` rules were properly checked (since this cannot easily be +enforced over federation by event authorisation).[4](#f4) + +To better cope with joining via aliases, homeservers should use the list of +authorised servers (not the list of candidate servers) when a user attempts to +join a room. + +## Summary of the behaviour of join rules + +See the [join rules](https://matrix.org/docs/spec/client_server/r0.6.1#m-room-join-rules) +specification for full details; the summary below is meant to highlight the differences +between `public`, `invite`, and `restricted` from a user perspective. Note that +all join rules are subject to `ban` and `server_acls`. + +* `public`: anyone can join, as today. +* `invite`: only people with membership `invite` can join, as today. +* `knock`: the same as `invite`, except anyone can knock. See + [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403). +* `private`: This is reserved, but unspecified. +* `restricted`: the same as `invite`, except users may also join if they are a + member of a room listed in the `allow` rules. + +## Security considerations + +Increased trust to enforce the join rules during calls to `/join`, `/make_join`, +and `/send_join` is placed in the homeservers whose users can issue invites. +Although it is possible for those homeservers to issue a join event in bad faith, +there is no real-world benefit to doing this as those homeservers could easily +side-step the restriction by issuing an invite first anyway. + +## Unstable prefix + +The `restricted` join rule will be included in a future room version to allow +servers and clients to opt-into the new functionality. + +During development, an unstable room version of `org.matrix.msc3083.v2` will be used. +Since the room version namespaces the behaviour, the `allow` key and value, as well +as the `restricted` join rule value do not need unstable prefixes. + +An unstable key of `org.matrix.msc3083.v2.event` will be used in the response +from `/send_join` in place of `event` during development. + +## Alternatives + +It may seem that just having the `allow` key with `public` join rules is enough +(as originally suggested in [MSC2962](https://github.com/matrix-org/matrix-doc/pull/2962)), +but there are concerns that changing the behaviour of a pre-existing `public` +join rule may cause security issues in older implementations (that do not yet +understand the new behaviour). This could be solved by introducing a new room +version, thus it seems clearer to introduce a new join rule -- `restricted`. + +Using an `allow` key with the `invite` join rules to broaden who can join was rejected +as an option since it requires weakening the [auth rules](https://spec.matrix.org/unstable/rooms/v1/#authorization-rules). +From the perspective of the auth rules, the `restricted` join rule is identical +to `public` with additional checks on the signature of the event. + +## Future extensions + +### Checking room membership over federation + +If a homeserver is not in an allowed room (and thus doesn't know the +membership of it) then the server cannot enforce the membership checks while +generating a join event. Peeking over federation, as described in +[MSC2444](https://github.com/matrix-org/matrix-doc/pull/2444), +could be used to establish if the user is in any of the proper rooms. + +This would then delegate power out to a (potentially) untrusted server, giving that +peek server significant power. For example, a poorly chosen peek +server could lie about the room membership and add an `@evil_user:example.org` +to an allowed room to gain membership to a room. + +As iterated above, this MSC recommends rejecting the join, potentially allowing +the requesting homeserver to retry via another homeserver. + +### Kicking users out when they leave the allowed room + +In the above example, suppose `@bob:server.example` leaves `!users:example.org`: +should they be removed from the room? Likely not, by analogy with what happens +when you switch the join rules from `public` to `invite`. Join rules currently govern +joins, not existing room membership. + +It is left to a future MSC to consider this, but some potential thoughts are +given below. + +If you assume that a user *should* be removed in this case, one option is to +leave the departure up to Bob's server `server.example`, but this places a +relatively high level of trust in that server. Additionally, if `server.example` +were offline, other users in the room would still see Bob in the room (and their +servers would attempt to send message traffic to it). + +Another consideration is that users may have joined via a direct invite, not via +access through a room. + +Fixing this is thorny. Some sort of annotation on the membership events might +help, but it's unclear what the desired semantics are: + +* Assuming that users in an allowed room are *not* kicked when that room is + removed from `allow`, are those users then given a pass to remain + in the room indefinitely? What happens if the room is added back to + `allow` and *then* the user leaves it? +* Suppose a user joins a room via an allowed room (RoomA). Later, RoomB is added + to the `allow` list and RoomA is removed. What should happen when the + user leaves RoomB? Are they exempt from the kick? + +It is possible that completely different state should be kept, or a different +`m.room.member` state could be used in a more reasonable way to track this. + +### Inheriting join rules + +If an allowed room is a space and you make a parent space invite-only, should that +(optionally?) cascade into child rooms? This would have some of the same problems +as inheriting power levels, as discussed in [MSC2962](https://github.com/matrix-org/matrix-doc/pull/2962). + +### Additional allow types + +Future MSCs may wish to define additional values for the `type` argument, potentially +restricting access via: + +* MXIDs or servers. +* A shared secret (room password). + +These are just examples and are not fully thought through for this MSC, but it should +be possible to add these behaviors in the future. + +### Client considerations + +[MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) defines a `via` +key in the content of `m.space.child` events: + +> the content must contain a via `key` which gives a list of candidate servers +> that can be used to join the room. + +It is possible for the list of candidate servers and the list of authorised +servers to diverge. It may not be possible for a user to join a room if there's +no overlap between these lists. + +If there is some overlap between the lists of servers the join request should +complete successfully. + +Clients should also consider the authorised servers when generating candidate +servers to embed in links to the room, e.g. via matrix.to. + +A future MSC may define a way to override or update the `via` key in a coherent +manner. + +## Footnotes + +[1]: The converse restriction, "anybody can join, provided they are not members +of the #catlovers:example.com space" is less useful since: + +1. Users in the banned room could simply leave it at any time +2. This functionality is already partially provided by + [Moderation policy lists](https://matrix.org/docs/spec/client_server/r0.6.1#moderation-policy-lists). [↩](#a1) + +[2]: Note that there is nothing stopping users sending and +receiving invites in `public` rooms today, and they work as you might expect. +The only difference is that you are not *required* to hold an invite when +joining the room. [↩](#a2) + +[3]: This seems like an improvement regardless since the resident server +is accepting the event on behalf of the joining server and ideally this should be +verifiable after the fact, even for current room versions. Requiring all events +to be signed and verified in this way is left to a future MSC. [↩](#a3) + +[4]: This has the downside of increased centralisation, as some +homeservers that are already in the room may not issue a join event for another +user on that server. (It must go through the `/make_join` / `/send_join` flow of +a server whose users may issue invites.) This is considered a reasonable +trade-off. [↩](#a4) diff --git a/proposals/3173-expose-stripped-state-events.md b/proposals/3173-expose-stripped-state-events.md new file mode 100644 index 000000000..830a9388a --- /dev/null +++ b/proposals/3173-expose-stripped-state-events.md @@ -0,0 +1,195 @@ +# MSC3173: Expose stripped state events to any potential joiner + +It can be useful to view the partial state of a room before joining to allow a user +to know *what* they're joining. For example, it improves the user experience to +show the user the room name and avatar before joining. + +It is already allowed to partially view the room state without being joined to +the room in some situations: + +* If the room has `history_visibility: world_readable`, then anyone can inspect + it (by calling `/state` on it). +* Rooms in the [room directory](https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-publicrooms) + expose some of their state publicly. +* [Invited users](https://matrix.org/docs/spec/server_server/r0.1.4#put-matrix-federation-v2-invite-roomid-eventid) + and [knocking users](https://github.com/matrix-org/matrix-doc/pull/2403) + receive stripped state events to display metadata to users. + +This MSC proposes formalizing that the stripped state that is currently available +to invited and knocking users is available to any user who could potentially join +a room. It also defines "stripped state" and consolidates the recommendation on +which events to include in the stripped state. + +## Background + +When creating an invite it is [currently recommended](https://matrix.org/docs/spec/server_server/r0.1.4#put-matrix-federation-v2-invite-roomid-eventid) +to include stripped state events which are useful for displaying the invite to a user: + +> An optional list of simplified events to help the receiver of the invite identify +> the room. The recommended events to include are the join rules, canonical alias, +> avatar, and name of the room. + +The invited user receives these [stripped state events](https://spec.matrix.org/unstable/client-server-api/#get_matrixclientr0sync) +as part of the `/sync` response: + +> The state of a room that the user has been invited to. These state events may +> only have the `sender`, `type`, `state_key` and `content` keys present. These +> events do not replace any state that the client already has for the room, for +> example if the client has archived the room. + +These are sent as part of the [`unsigned` content of the `m.room.member` event](https://spec.matrix.org/unstable/client-server-api/#mroommember) +containing the invite. + +[MSC2403: Add "knock" feature](https://github.com/matrix-org/matrix-doc/pull/2403) +extends this concept to also include the stripped state events in the `/sync` response +for knocking users: + +> It is proposed to add a fourth possible key to rooms, called `knock`. Its value +> is a mapping from room ID to room information. The room information is a mapping +> from a key `knock_state` to another mapping with key events being a list of +> `StrippedStateEvent`. + +It is also provides an extended rationale of why this is useful: + +> These stripped state events contain information about the room, most notably the +> room's name and avatar. A client will need this information to show a nice +> representation of pending knocked rooms. The recommended events to include are the +> join rules, canonical alias, avatar, name and encryption state of the room, rather +> than all room state. This behaviour matches the information sent to remote +> homeservers when remote users are invited to a room. + +[MSC1772: Spaces](https://github.com/matrix-org/matrix-doc/pull/1772) additionally +recommends including the `m.room.create` event as one of the stripped state events: + +> Join rules, invites and 3PID invites work as for a normal room, with the exception +> that `invite_state` sent along with invites should be amended to include the +> `m.room.create` event, to allow clients to discern whether an invite is to a +> space-room or not. + +## Proposal + +The specification does not currently define what "stripped state" is or formally +describe who can access it, instead it is specified that certain situations (e.g. +an invite or knock) provide a potential joiner with the stripped state of a room. + +This MSC clarifies what "stripped state" is and formalizes who can access the +stripped state of a room in future cases. + +Potential ways that a user might be able to join a room include, but are not +limited to, the following mechanisms: + +* A room that has `join_rules` set to `public` or `knock`. +* A room that the user is in possession of an invite to (regardless of the `join_rules`). + +This MSC proposes a formal definition for the stripped state of a room[1](#f1): + +> The stripped state of a room is a list of simplified state events to help a +> potential joiner identify the room. These state events may only have the +> `sender`, `type`, `state_key` and `content` keys present. + +### Client behavior + +These events do not replace any state that the client already has for the room, +for example if the client has archived the room. Instead the client should keep +two separate copies of the state: the one from the stripped state and one from the +archived state. If the client joins the room then the current state will be given +as a delta against the archived state not the stripped state. + +### Server behavior + +It is recommended (although not required[2](#f2)) that +homeserver implementations include the following events as part of the stripped +state of a room: + +* Create event (`m.room.create`)[3](#f3) +* Join rules (`m.room.join_rules`) +* Canonical alias (`m.room.canonical_alias`) +* Room avatar (`m.room.avatar`) +* Room name (`m.room.name`) +* Encryption information (`m.room.encryption`)[4](#f4) +* Room topic (`m.room.topic`)[5](#f5) + +## Potential issues + +This is a formalization of current behavior and should not introduce new issues. + +## Alternatives + +A different approach would be to continue with what is done today for invites, +knocking, the room directory: separately specify that a user is allowed to see +the stripped state (and what events the stripped state should contain). + +## Security considerations + +This would allow for invisibly accessing the stripped state of a room with `public` +or `knock` join rules. + +In the case of a public room, if the room has `history_visibility` set to `world_readable` +then this is no change. Additionally, this information is already provided by the +room directory (if the room is listed there). Otherwise, it is trivial to access +the state of the room by joining, but currently users in the room would know +that the join occurred. + +Similarly, in the case of knocking, a user is able to trivially access the +stripped state of the room by knocking, but users in the room would know that +the knock occurred. + +This does not seem to weaken the security expectations of either join rule. + +## Future extensions + +### Revisions to the room directory + +A future MSC could include additional information from the stripped state events +in the [room directory](https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-publicrooms). +The main missing piece seems to be the encryption information, but there may also +be other pieces of information to include. + +### Additional ways to access the stripped state + +[MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946) proposes including +the stripped state in the spaces summary. Not needing to rationalize what state +can be included for a potential joiner would simplify this (and future) MSCs. + +### Additional ways to join a room + +[MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083) proposes a new +join rule due to membership in a space. This MSC would clarify that the stripped +state of a room is available to those joiners. + +### Dedicated API for accessing the stripped state + +Dedicated client-server and server-server APIs could be added to request the +stripped state events, but that is considered out-of-scope for the current +proposal. + +This API would allow any potential joiner to query for the stripped state. If +the server does not know the room's state it would need to query other servers +for it. + +## Unstable prefix + +N/A + +## Footnotes + +[1]: No changes are proposed to +[the definition of `history_visibility`](https://matrix.org/docs/spec/client_server/latest#room-history-visibility). +The state of a room which is `world_readable` is available to anyone. This somewhat +implies that the stripped state is also available to anyone, regardless of the join +rules, but having a `world_readable`, `invite` room does not seem valuable. [↩](#a1) + +[2]: Privacy conscious deployments may wish to limit the metadata +available to users who are not in a room as the trade-off against user experience. +There seems to be no reason to not allow this. [↩](#a2) + +[3]: As updated in [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772). [↩](#a3) + +[4]: The encryption information (`m.room.encryption`) is already sent +from Synapse and generally seems useful for a user to know before joining a room. +[↩](#a4) + +[5]: The room topic (`m.room.topic`) is included as part of the +[room directory](https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-publicrooms) +response for public rooms. It is also planned to be included as part of [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946) +in the spaces summary response. [↩](#a5) diff --git a/proposals/3231-token-authenticated-registration.md b/proposals/3231-token-authenticated-registration.md new file mode 100644 index 000000000..a86b9fbc5 --- /dev/null +++ b/proposals/3231-token-authenticated-registration.md @@ -0,0 +1,138 @@ +# MSC3231: Token Authenticated Registration + +Currently, homeserver administrators must choose between allowing anyone to +register and completely disabling registrations. This is a problem for +administrators who want to let certain people have an account on their server, +but do not want to register the accounts manually (possibly because the +initial password may not be changed by the user). + +There are some existing external solutions (see the **Alternatives** section), +but these require additional effort from administrators and are disconnected +from Matrix clients. It would be useful for administrators to have a convenient +method of limiting registrations to certain people which requires minimal setup +and is integrated with existing clients. + +## Proposal + +The [/\_matrix/client/r0/register](https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-register) +endpoint uses the [User-Interactive Authentication API](https://matrix.org/docs/spec/client_server/r0.6.1#user-interactive-authentication-api). +A new authentication type `m.login.registration_token` will be defined which requires +a `token` key to be present in the submitted `auth` dict. The token will be a +string of no more than 64 characters, and contain only characters matched by the +regex `[A-Za-z0-9._~-]`. +This will avoid URL encoding issues with the validity checking endpoint, and +prevent DoS attacks from extremely long tokens. + +For example, when a client attempts registration with no `auth` dict, a server +may respond with: + +``` +HTTP/1.1 401 Unauthorized + +{ + "flows": [ + { + "stages": [ "m.login.registration_token" ] + } + ], + "params": {}, + "session": "xxxxx" +} +``` + +The client would then prompt the user to enter a token and send a new request +with an `auth` dict: + +``` +POST /_matrix/client/r0/register + +{ + "auth": { + "type": "m.login.registration_token", + "token": "fBVFdqVE", + "session": "xxxxx" + } + "device_id": "ABC", + "initial_device_display_name": "Some Client", + "password": "badpassword", + "username": "bob" +} +``` + +If the server verifies that `fBVFdqVE` is a valid token then the account is +registered as normal assuming all other required auth stages have been completed, otherwise a `401` status is returned. Once registration of +the user has completed, the server may alter the validity of the token. +For example, the token may be completely invalidated, or its number of permitted +uses reduced. Management of the tokens is left to the server implementation. + +Using the User-Interactive Authentication API means clients' existing +registration logic will be unaffected, with a fallback available for clients +without native support for the new authentication type. + + +### Checking the validity of a token + +A Client may wish to present username and password inputs only after it has +checked the token is valid. + +Clients would be able to check the validity of a token in advance of +registration with a `GET` request to +`/_matrix/client/r0/register/m.login.registration_token/validity`. +This endpoint would take a required `token` query parameter, and validity would be +indicated by the boolean `valid` key in the response. + +For example, a client would send: + +``` +GET /_matrix/client/r0/register/m.login.registration_token/validity?token=abcd +``` + +If `abcd` is a valid token, the server would respond with: + +``` +HTTP/1.1 200 OK + +{ + "valid": true +} +``` + +This does not perform any actual authentication, and the validity of the token +may change between the check and the User-Interactive Authentication. + +This endpoint should be rate limited in order to prevent dictionary attacks. + +## Potential issues + +The new authentication type would only apply to the +`/_matrix/client/r0/register` endpoint. This should not cause problems, but it +would be worth noting in any change to the specification. + + +## Alternatives + +[matrix-registration](https://github.com/ZerataX/matrix-registration/) is an +application which provides token management and a standalone web interface. +It uses Synapse's admin API to do registrations. + +[Midnight](https://github.com/KombuchaPrivacy/midnight) sits in front of a +homeserver and handles the `/_matrix/client/r0/register` endpoint. It provides +token management and does additional checks on the requested username. +Registration requests are forwarded to the homeserver once authenticated. +It uses the User-Interactive Authentication API in a very similar way to this +MSC, which could allow existing Matrix clients to be used. + +[matrix-register-bot](https://github.com/krombel/matrix-register-bot) is a +Matrix bot which allows registrations done through the provided web interface +to be accepted or denied by users in a certain room. When a registration is +approved temporary credentials are sent to the user's verified email address. +It can use Synapse's admin API or [matrix-synapse-rest-auth](https://github.com/kamax-matrix/matrix-synapse-rest-password-provider#integrate) +to do the registration. + + +## Unstable prefix + +Implementations should use `org.matrix.msc3231.login.registration_token` as the +authentication type until this MSC has passed FCP and been merged. +Similarly, `/_matrix/client/unstable/org.matrix.msc3231/register/org.matrix.msc3231.login.registration_token/validity` +should be used as the endpoint for checking the validity of a token in advance. diff --git a/proposals/3267-reference-relations.md b/proposals/3267-reference-relations.md new file mode 100644 index 000000000..7b11d649f --- /dev/null +++ b/proposals/3267-reference-relations.md @@ -0,0 +1,92 @@ +# MSC3267: reference relationships + +## Proposal + +This proposal defines a relation type (using +[MSC2674 relations](https://github.com/matrix-org/matrix-doc/pull/2674)) +for events to make a reference to another event. + +A `rel_type` of `m.reference` is defined as a generic way to associate an +event with another event. As a bundle, `m.reference` relations appear as +an object with a single `chunk` field. The `chunk` is an array of objects +with a single `event_id` field for all the child events which `m.reference` +the parent. + +There are no implied semantics by a reference relation: the feature or event +type which makes use of the `rel_type` should specify what sort of semantic +behaviour there is, if any. For example, describing that a poll response event +*references* the poll start event, or that a location update *references* a +previous location update. + +Reference relations are used by [MSC2241](https://github.com/matrix-org/matrix-doc/pull/2241) +to tie all events together for the same verification request. + +For instance, an `m.room.message` which references an existing event +would look like: + +```json5 +{ + // Unimportant fields omitted + "type": "m.room.message", + "content": { + "msgtype": "m.text", + "body": "i <3 shelties", + "m.relates_to": { + "rel_type": "m.reference", + "event_id": "$another_event_id" + } + } +} +``` + +## Server aggregation + +[MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674) states +that values for `rel_type` should define how the server should aggregate the +`rel_type`. For `m.reference`, child events are bundled to appear as follows +under `m.relations`: + +```json5 +{ + "m.reference": { + "chunk": [ + {"event_id": "$one"}, + {"event_id": "$two"}, + {"event_id": "$three"} + ] + } +} +``` + +Note that currently only the `event_id` is noted in the chunk, however a future +MSC might add more fields. + +## Limitations + +Different subtypes of references could be defined through additional fields on +the `m.relates_to` object, to distinguish between other forms of semantic behaviour +independent of type (hypothetical threads, replies, etc if we didn't have a system +for them already). This MSC doesn't attempt to define these subtypes. + +This relation cannot be used in conjunction with another relation due to `rel_type` +being a single value. This is known and unfortunately not resolved by this MSC. +A future MSC might address the concern. + +## Edge Cases + +Can you reference an event which doesn't have logical meaning? Eg to a [reaction](https://github.com/matrix-org/matrix-doc/pull/2677)? + * Yes, at the protocol level. But you shouldn't expect clients to do anything + useful with it. + * The relationship is effectively pointless, so the event would appear as though + there was no reference relationship. + +Do we need to support retrospective references? + * For something like "m.duplicate" to retrospectively declare that one event + dupes another, we might need to bundle two-levels deep (subject+ref and then + ref+target). We can cross this bridge when we get there though, as another + aggregation type + +## Unstable prefix + +Unfortunately not applicable - this MSC was used in production and appears in the +specified version of the [key verification framework](https://spec.matrix.org/v1.2/client-server-api/#key-verification-framework). diff --git a/proposals/3283-enable_set_displayname-capabilities.md b/proposals/3283-enable_set_displayname-capabilities.md new file mode 100644 index 000000000..e59746d8b --- /dev/null +++ b/proposals/3283-enable_set_displayname-capabilities.md @@ -0,0 +1,46 @@ +# MSC3283: Expose enable_set_displayname, enable_set_avatar_url and enable_3pid_changes in capabilities response + +Some home servers like [Synapse](https://github.com/matrix-org/synapse/blob/756fd513dfaebddd28bf783eafa95b4505ce8745/docs/sample_config.yaml#L1207) +can be configured to `enable_set_displayname: false`, `enable_set_avatar_url: false` or `enable_3pid_changes: false`. +To enable clients to handle that gracefully in the UI this setting should be exposed. + +## Proposal + +The `/_matrix/client/r0/capabilities` endpoint should be decorated to provide more information on capabilities. +```jsonc +{ + "capabilities": { + "m.set_displayname": { "enabled": false }, + "m.set_avatar_url": { "enabled": false }, + "m.3pid_changes": { "enabled": false }, + "m.room_versions": {...}, + } +} +``` +As part of this MSC, a capability for each setting will be added that exposes the server setting: +- `m.set_displayname` + +Whether users are allowed to change their displayname after it has been initially set. +Useful when provisioning users based on the contents of a third-party directory. + +- `m.set_avatar_url` + +Whether users are allowed to change their avatar after it has been initially set. +Useful when provisioning users based on the contents of a third-party directory. + +- `m.3pid_changes` + +Whether users can change the 3PIDs associated with their accounts +(email address and msisdn). +Useful when provisioning users based on the contents of a third-party directory. + +## Client recommendations +When presenting profile settings, clients should use capabilities in order to display the correct UI. + +Capability should always be present. +Servers should always send these capabilities. If they aren't (because the server does not support +a new enough spec version or for any other reason), clients should behave as if they were present and set to true. + +## Unstable prefix + +While this MSC is not considered stable, implementations should use `org.matrix.msc3283.` in place of `m.` throughout this proposal. diff --git a/proposals/3288-pass_room_type_in_3pid_invite.md b/proposals/3288-pass_room_type_in_3pid_invite.md new file mode 100644 index 000000000..408573c08 --- /dev/null +++ b/proposals/3288-pass_room_type_in_3pid_invite.md @@ -0,0 +1,69 @@ +# MSC3288: Add room type to `/_matrix/identity/v2/store-invite` API + +Currently when inviting via 3pid, the Identity Server receives some information about the room, +like for example the room name and avatar as well as the inviter name. +This allows the identity server to generate a rich email to the invitee. + +Now that the matrix spec supports spaces, it would be nice to also provide this information to the identity server +so that the email invite could be customized for spaces. The current implementation would say wrongly that +you are invited to a room when the room is actually a space. + +The goal of this proposal is to make 3pid invites space aware. + + +## Proposal + +Homeservers should also send the `room_type` to the identity server when performing a third party invite (__Invitation storage__). + + +__Proposed change:__ + +Add a new `room_type` field in json body of [`POST /_matrix/identity/v2/store-invite`](https://matrix.org/docs/spec/identity_service/r0.3.0#post-matrix-identity-v2-store-invite): + +| Parameter | Type | Description | +|--|--|--| +| room_type | string | The room type for the room to which the user is invited. This should be retrieved from the value of `type` in the `m.room.create` event's `content`. Do not include parameter if `type` is not present in `m.room.create`. + +```` +POST /_matrix/identity/v2/store-invite HTTP/1.1 +Content-Type: application/json + +{ + "medium": "email", + "address": "foo@example.com", + "room_id": "!something:example.org", + "sender": "@bob:example.com", + "room_alias": "#somewhere:exmaple.org", + "room_avatar_url": "mxc://example.org/s0meM3dia", + "room_join_rules": "public", + "room_name": "The Bob Project", + "room_type": "m.space", + "sender_display_name": "Bob Smith", + "sender_avatar_url": "mxc://example.org/an0th3rM3dia" +} +```` + +The identity server could then use room type to customize the email depending on the room type. + +__Email Generation__ + +The link in the generated email should also pass over the `room_type` to clients ( like it is doing for +`inviter_name`, `room_name`, `room_avatar`) + +## Potential issues + +None. + + +## Security considerations + +None. + +## Unstable prefix + +The following mapping will be used for identifiers in this MSC during development: + + +Proposed final identifier | Purpose | Development identifier +------------------------------- | ------- | ---- +`room_type` | POST body | `org.matrix.msc3288.room_type` diff --git a/proposals/3289-rooms-v8.md b/proposals/3289-rooms-v8.md new file mode 100644 index 000000000..7212e0a8c --- /dev/null +++ b/proposals/3289-rooms-v8.md @@ -0,0 +1,15 @@ +# MSC3289: Room Version 8 + +A new room version, `8`, is proposed using [room version 7](https://spec.matrix.org/unstable/rooms/v7/) +as a base and incorporating the following MSCs: + +* [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083) - Restricting room + membership based on membership in other rooms + +Though other MSCs are capable of being included in this version, they do not have +sufficient implementation to be considered for this room version. A future room +version may include them. + +Room version `8` upon being added to the specification shall be considered stable. +No other room versions are affected by this MSC. Before room version `8` can enter +the specification, MSC3083 needs to complete its final comment period. diff --git a/proposals/3316-appservice-timestamp-massaging.md b/proposals/3316-appservice-timestamp-massaging.md new file mode 100644 index 000000000..b954e59a1 --- /dev/null +++ b/proposals/3316-appservice-timestamp-massaging.md @@ -0,0 +1,51 @@ +# Proposal to add timestamp massaging to the spec +Bridges often want to override message timestamps to preserve the timestamps from +the remote network. The spec used to have a concept of [timestamp massaging], but +it was excluded from the release due to not being properly specified. Synapse +still implements it and it is widely used in bridges. + +[MSC2716] was originally going to add timestamp massaging to the spec, but it +pivoted to focusing solely on batch sending history. This MSC simply copies the +proposed `ts` query param from the [original MSC2716]. + +[timestamp massaging]: https://matrix.org/docs/spec/application_service/r0.1.2#timestamp-massaging +[MSC2716]: https://github.com/matrix-org/matrix-doc/pull/2716 +[original MSC2716]: https://github.com/matrix-org/matrix-doc/blob/94514392b118dfae8ee6840b13b83d2f8ce8fcfc/proposals/2716-importing-history-into-existing-rooms.md + +## Proposal +As per the original version of MSC2716: + +> We let the AS API override ('massage') the `origin_server_ts` timestamp +> applied to sent events. We do this by adding a `ts` querystring parameter on +> the `PUT /_matrix/client/r0/rooms/{roomId}/send/{eventType}/{txnId}` +> and `PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}` +> endpoints, specifying the value to apply to `origin_server_ts` on the event +> (UNIX epoch milliseconds). +> +> We consciously don't support the `ts` parameter on the various helper +> syntactic-sugar APIs like /kick and /ban. If a bridge/bot is smart enough to +> be faking history, it is already in the business of dealing with raw events, +> and should not be using the syntactic sugar APIs. + +The spec should also make it clear that the `ts` query param won't affect DAG +ordering, and MSC2716's batch sending should be used when the intention is to +insert history somewhere else than the end of the room. + +## Potential issues +None. + +## Alternatives +The new MSC2716 could technically be considered an alternative, but it can only +be used for history, while this proposal also supports overriding timestamps of +new messages. In practice, bridges will likely use both: Batch sending for +filling history and timestamp massaging for new messages. + +## Security considerations +Timestamps should already be considered untrusted over federation, and +application services are trusted server components, so allowing appservices +to override timestamps does not create any new security considerations. + +## Unstable prefix +`org.matrix.msc3316.ts` may be used as the query parameter. However, the `ts` +parameter is already used in production for the `/send` endpoint, which means +the unstable prefix should only be used for the `/state` endpoint. diff --git a/schemas/server-signatures.yaml b/schemas/server-signatures.yaml deleted file mode 100644 index a18552566..000000000 --- a/schemas/server-signatures.yaml +++ /dev/null @@ -1,24 +0,0 @@ -# Copyright 2018 New Vector Ltd -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -type: object -example: { - "example.com": { - "ed25519:0": "these86bytesofbase64signaturecoveressentialfieldsincludinghashessocancheckredactedpdus" - } -} -additionalProperties: - type: object - title: Server Signatures - additionalProperties: - type: string \ No newline at end of file diff --git a/scripts/dump-swagger.py b/scripts/dump-swagger.py index 4b4922fff..68a9356aa 100755 --- a/scripts/dump-swagger.py +++ b/scripts/dump-swagger.py @@ -98,10 +98,9 @@ for filename in os.listdir(cs_api_dir): path = (basePath + path).replace('%CLIENT_MAJOR_VERSION%', major_version) for method, spec in methods.items(): - if "tags" in spec.keys(): - if path not in output["paths"]: - output["paths"][path] = {} - output["paths"][path][method] = spec + if path not in output["paths"]: + output["paths"][path] = {} + output["paths"][path][method] = spec print("Generating %s" % output_file) diff --git a/static/css/fonts/README.md b/static/css/fonts/README.md index e6eb74eb2..aca333042 100644 --- a/static/css/fonts/README.md +++ b/static/css/fonts/README.md @@ -22,9 +22,9 @@ python3 download_google_fonts_css.py \ Which would pop out a `Inter.css` file that should be `@import url("Inter.css")`d somewhere in the site's SCSS (currently in -[/assets-hugo/scss/_variables_project.scss](/assets-hugo/scss/_variables_project.scss)). +[/assets/scss/_variables_project.scss](/assets/scss/_variables_project.scss)). Re-running the script and committing any new files is only necessary when a desired font updates (not very often), or we want to change the font we're using. In that case, remove the existing font files at `/static/fonts/*.woff2` and re-run the script with a -different URL. \ No newline at end of file +different URL.