diff --git a/.circleci/config.yml b/.circleci/config.yml index 659380b0..bf4404ce 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -97,7 +97,7 @@ jobs: command: DOCS_URL="${CIRCLE_BUILD_URL}/artifacts/${CIRCLE_NODE_INDEX}/${CIRCLE_WORKING_DIRECTORY/#\~/$HOME}/api/client-server/index.html"; echo $DOCS_URL build-dev-scripts: docker: - - image: golang:1.8 + - image: golang:1.10 steps: - checkout - run: @@ -121,4 +121,4 @@ workflows: notify: webhooks: - - url: https://giles.cadair.com/circleci + - url: https://giles.cadair.dev/circleci diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 00000000..afc29f01 --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1,2 @@ +patreon: matrixdotorg +liberapay: matrixdotorg diff --git a/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md b/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md new file mode 100644 index 00000000..afa808b8 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/ready-proposal.md @@ -0,0 +1,19 @@ +--- +name: Proposal ready for review +about: A proposal that is ready for review by the core team and community. +title: '' +labels: proposal, proposal-in-review +assignees: '' + +--- + + + +### Pull Request Checklist + + + +* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog) +* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off) +* [ ] Pull request includes ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above. +* [ ] Pull request title and file name include this PR's number as the MSC number. diff --git a/.github/PULL_REQUEST_TEMPLATE/spec-change.md b/.github/PULL_REQUEST_TEMPLATE/spec-change.md new file mode 100644 index 00000000..d2f6e874 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/spec-change.md @@ -0,0 +1,16 @@ +--- +name: Spec clarification/not a proposal +about: A change that's not a spec proposal, such as a clarification to the spec itself. +title: '' +labels: '' +assignees: '' + +--- + +### Pull Request Checklist + + + +* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog) +* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off) +* [ ] Pull request is classified as ['other changes'](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#other-changes) diff --git a/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md b/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md new file mode 100644 index 00000000..34e26187 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/wip-proposal.md @@ -0,0 +1,20 @@ +--- +name: WIP Proposal +about: A proposal that isn't quite ready for formal review yet. +title: '[WIP] Your Proposal Title' +labels: proposal +assignees: '' + +--- + + + +### Pull Request Checklist + + + +* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog) +* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off) +* [ ] A ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above. +* [ ] Update the title and file name of your proposal to match this PR's number (after opening). +* [ ] Ask in [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) to get this marked as ready for review, once it is ready for review. diff --git a/.gitignore b/.gitignore index 800be2fa..9cc27b85 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,4 @@ *.swp _rendered.rst /.vscode/ +/.idea/ diff --git a/README.rst b/README.rst index b8847bfb..01ea8e72 100644 --- a/README.rst +++ b/README.rst @@ -66,12 +66,14 @@ The above will write the rendered version of the specification to Windows users ~~~~~~~~~~~~~ +The ``source`` program does not exist on Windows, so instead run one of the +``activate`` files in ``.\env\Scripts\`` to activate the virtual environment. If you're on Windows Vista or higher, be sure that the "Symbolic Links" option was selected when installing Git prior to cloning this repository. If you're still seeing errors about files not being found it is likely because the symlink at ``api/client-server/definitions/event-schemas`` looks like a -file. To correct the problem, open an Administrative/Elevated shell in your +file. To correct the problem, open an Administrative/Elevated Command Prompt in your cloned matrix-doc directory and run the following:: cd api\client-server\definitions @@ -138,4 +140,4 @@ Issue tracking Issues with the Matrix specification are tracked in `GitHub `_. -See `meta/labels.rst `_ for notes on what the labels mean. +See `meta/github-labels.rst `_ for notes on what the labels mean. diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 0e93e4cd..fc231b60 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -1,4 +1,5 @@ # Copyright 2016 OpenMarket Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -89,8 +90,19 @@ paths: - User data post: summary: Adds contact information to the user's account. - description: Adds contact information to the user's account. + description: |- + Adds contact information to the user's account. + + This endpoint is deprecated in favour of the more specific ``/3pid/add`` + and ``/3pid/bind`` endpoints. + + .. Note:: + Previously this endpoint supported a ``bind`` parameter. This parameter + has been removed, making this endpoint behave as though it was ``false``. + This results in this endpoint being an equivalent to ``/3pid/bind`` rather + than dual-purpose. operationId: post3PIDs + deprecated: true security: - accessToken: [] parameters: @@ -110,25 +122,24 @@ paths: id_server: type: string description: The identity server to use. + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. Servers + can treat this as optional to distinguish between r0.5-compatible clients + and this specification version. sid: type: string description: The session identifier given by the identity server. - required: ["client_secret", "id_server", "sid"] - bind: - type: boolean - description: |- - Whether the homeserver should also bind this third party - identifier to the account's Matrix ID with the passed identity - server. Default: ``false``. - x-example: true + required: ["client_secret", "id_server", "id_access_token", "sid"] required: ["three_pid_creds"] example: { "three_pid_creds": { "id_server": "matrix.org", + "id_access_token": "abc123_OpaqueString", "sid": "abc123987", "client_secret": "d0n'tT3ll" - }, - "bind": false + } } responses: 200: @@ -146,9 +157,10 @@ paths: An optional field containing a URL where the client must submit the validation token to, with identical parameters to the Identity Service API's ``POST - /validate/email/submitToken`` endpoint. The homeserver must - send this token to the user (if applicable), who should - then be prompted to provide it to the client. + /validate/email/submitToken`` endpoint (without the requirement + for an access token). The homeserver must send this token to the + user (if applicable), who should then be prompted to provide it + to the client. If this field is not present, the client can assume that verification will happen without the client's involvement @@ -166,12 +178,121 @@ paths: "$ref": "definitions/errors/error.yaml" tags: - User data + "/account/3pid/add": + post: + summary: Adds contact information to the user's account. + description: |- + This API endpoint uses the `User-Interactive Authentication API`_. + + Adds contact information to the user's account. Homeservers should use 3PIDs added + through this endpoint for password resets instead of relying on the identity server. + + Homeservers should prevent the caller from adding a 3PID to their account if it has + already been added to another user's account on the homeserver. + operationId: add3PID + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + auth: + description: |- + Additional authentication information for the + user-interactive authentication API. + $ref: "definitions/auth_data.yaml" + client_secret: + type: string + description: The client secret used in the session with the homeserver. + sid: + type: string + description: The session identifier given by the homeserver. + required: ["client_secret", "sid"] + example: { + "sid": "abc123987", + "client_secret": "d0n'tT3ll" + } + responses: + 200: + description: The addition was successful. + examples: + application/json: {} + schema: + type: object + 401: + description: |- + The homeserver requires additional authentication information. + schema: + "$ref": "definitions/auth_response.yaml" + 429: + description: This request was rate-limited. + schema: + "$ref": "definitions/errors/rate_limited.yaml" + "/account/3pid/bind": + post: + summary: Binds a 3PID to the user's account through an Identity Service. + description: |- + Binds a 3PID to the user's account through the specified identity server. + + Homeservers should not prevent this request from succeeding if another user + has bound the 3PID. Homeservers should simply proxy any errors received by + the identity server to the caller. + + Homeservers should track successful binds so they can be unbound later. + operationId: bind3PID + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + client_secret: + type: string + description: The client secret used in the session with the identity server. + id_server: + type: string + description: The identity server to use. + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. + sid: + type: string + description: The session identifier given by the identity server. + required: ["client_secret", "id_server", "id_access_token", "sid"] + example: { + "id_server": "example.org", + "id_access_token": "abc123_OpaqueString", + "sid": "abc123987", + "client_secret": "d0n'tT3ll" + } + responses: + 200: + description: The addition was successful. + examples: + application/json: {} + schema: + type: object + 429: + description: This request was rate-limited. + schema: + "$ref": "definitions/errors/rate_limited.yaml" + tags: + - User data "/account/3pid/delete": post: summary: Deletes a third party identifier from the user's account description: |- Removes a third party identifier from the user's account. This might not cause an unbind of the identifier from the identity server. + + Unlike other endpoints, this endpoint does not take an ``id_access_token`` + parameter because the homeserver is expected to sign the request to the + identity server instead. operationId: delete3pidFromAccount security: - accessToken: [] @@ -226,6 +347,69 @@ paths: - id_server_unbind_result tags: - User data + "/account/3pid/unbind": + post: + summary: Removes a user's third party identifier from an identity server. + description: |- + Removes a user's third party identifier from the provided identity server + without removing it from the homeserver. + + Unlike other endpoints, this endpoint does not take an ``id_access_token`` + parameter because the homeserver is expected to sign the request to the + identity server instead. + operationId: unbind3pidFromAccount + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + id_server: + type: string + description: |- + The identity server to unbind from. If not provided, the homeserver + MUST use the ``id_server`` the identifier was added through. If the + homeserver does not know the original ``id_server``, it MUST return + a ``id_server_unbind_result`` of ``no-support``. + example: "example.org" + medium: + type: string + description: The medium of the third party identifier being removed. + enum: ["email", "msisdn"] + example: "email" + address: + type: string + description: The third party address being removed. + example: "example@example.org" + required: ['medium', 'address'] + responses: + 200: + description: |- + The identity server has disassociated the third party identifier from the + user. + schema: + type: object + properties: + id_server_unbind_result: + type: string + enum: + # XXX: I don't know why, but the order matters here so that "no-support" + # doesn't become "no- support" by the renderer. + - "no-support" + - "success" + description: |- + An indicator as to whether or not the identity server was able to unbind + the 3PID. ``success`` indicates that the identity server has unbound the + identifier whereas ``no-support`` indicates that the identity server + refuses to support the request or the homeserver was not able to determine + an identity server to unbind from. + example: "success" + required: + - id_server_unbind_result + tags: + - User data "/account/3pid/email/requestToken": post: summary: Begins the validation process for an email address for association with the user's account. @@ -234,12 +418,9 @@ paths: already associated with an account on this homeserver. This API should be used to request validation tokens when adding an email address to an account. This API's parameters and response are identical to that of - the |/register/email/requestToken|_ endpoint. The homeserver has the - choice of validating the email address itself, or proxying the request - to the ``/validate/email/requestToken`` Identity Service API as - identified by ``id_server``. It is imperative that the - homeserver keep a list of trusted Identity Servers and only proxies to - those that it trusts. + the |/register/email/requestToken|_ endpoint. The homeserver should validate + the email itself, either by sending a validation email itself or by using + a service it has control over. operationId: requestTokenTo3PIDEmail parameters: - in: body @@ -269,7 +450,9 @@ paths: 400: description: |- The third party identifier is already in use on the homeserver, or - the request was invalid. + the request was invalid. The error code ``M_SERVER_NOT_TRUSTED`` + can be returned if the server does not trust/support the identity server + provided in the request. schema: $ref: "definitions/errors/error.yaml" examples: @@ -285,12 +468,9 @@ paths: already associated with an account on this homeserver. This API should be used to request validation tokens when adding a phone number to an account. This API's parameters and response are identical to that of - the |/register/msisdn/requestToken|_ endpoint. The homeserver has the - choice of validating the phone number itself, or proxying the request - to the ``/validate/msisdn/requestToken`` Identity Service API as - identified by ``id_server``. It is imperative that the - homeserver keep a list of trusted Identity Servers and only proxies to - those that it trusts. + the |/register/msisdn/requestToken|_ endpoint. The homeserver should validate + the phone number itself, either by sending a validation message itself or by using + a service it has control over. operationId: requestTokenTo3PIDMSISDN parameters: - in: body @@ -317,7 +497,9 @@ paths: 400: description: |- The third party identifier is already in use on the homeserver, or - the request was invalid. + the request was invalid. The error code ``M_SERVER_NOT_TRUSTED`` + can be returned if the server does not trust/support the identity server + provided in the request. schema: $ref: "definitions/errors/error.yaml" examples: diff --git a/api/client-server/capabilities.yaml b/api/client-server/capabilities.yaml index a50908f7..0d6b0677 100644 --- a/api/client-server/capabilities.yaml +++ b/api/client-server/capabilities.yaml @@ -96,6 +96,7 @@ paths: example: "1" available: type: object + title: AvailableRoomVersions description: |- A detailed description of the room versions the server supports. additionalProperties: diff --git a/api/client-server/content-repo.yaml b/api/client-server/content-repo.yaml index a9a0c2f6..d596dbda 100644 --- a/api/client-server/content-repo.yaml +++ b/api/client-server/content-repo.yaml @@ -340,6 +340,16 @@ paths: "/preview_url": get: summary: "Get information about a URL for a client" + description: |- + Get information about a URL for the client. Typically this is called when a + client sees a URL in a message and wants to render a preview for the user. + + .. Note:: + Clients should consider avoiding this endpoint for URLs posted in encrypted + rooms. Encrypted rooms often contain more sensitive information the users + do not want to share with the homeserver, and this can mean that the URLs + being shared should also not be shared with the homeserver. + operationId: getUrlPreview produces: ["application/json"] security: diff --git a/api/client-server/create_room.yaml b/api/client-server/create_room.yaml index bce61aad..2487707e 100644 --- a/api/client-server/create_room.yaml +++ b/api/client-server/create_room.yaml @@ -139,6 +139,12 @@ paths: id_server: type: string description: The hostname+port of the identity server which should be used for third party identifier lookups. + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. Servers + can treat this as optional to distinguish between r0.5-compatible clients + and this specification version. medium: type: string # TODO: Link to Identity Service spec when it eixsts @@ -146,7 +152,7 @@ paths: address: type: string description: The invitee's third party identifier. - required: ["id_server", "medium", "address"] + required: ["id_server", "id_access_token", "medium", "address"] room_version: type: string description: |- diff --git a/api/client-server/definitions/openid_token.yaml b/api/client-server/definitions/openid_token.yaml new file mode 100644 index 00000000..b50fcd54 --- /dev/null +++ b/api/client-server/definitions/openid_token.yaml @@ -0,0 +1,36 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +properties: + access_token: + type: string + description: |- + An access token the consumer may use to verify the identity of + the person who generated the token. This is given to the federation + API ``GET /openid/userinfo`` to verify the user's identity. + token_type: + type: string + description: The string ``Bearer``. + matrix_server_name: + type: string + description: |- + The homeserver domain the consumer should use when attempting to + verify the user's identity. + expires_in: + type: integer + description: |- + The number of seconds before this token expires and a new one must + be generated. +required: ['access_token', 'token_type', 'matrix_server_name', 'expires_in'] diff --git a/api/client-server/definitions/request_email_validation.yaml b/api/client-server/definitions/request_email_validation.yaml index 15bc5b3a..d24c42b5 100644 --- a/api/client-server/definitions/request_email_validation.yaml +++ b/api/client-server/definitions/request_email_validation.yaml @@ -22,5 +22,15 @@ allOf: The hostname of the identity server to communicate with. May optionally include a port. This parameter is ignored when the homeserver handles 3PID verification. + + This parameter is deprecated with a plan to be removed in a future specification + version for ``/account/password`` and ``/register`` requests. example: "id.example.com" - required: ["id_server"] + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. Servers + can treat this as optional to distinguish between r0.5-compatible clients + and this specification version. + + Required if an ``id_server`` is supplied. diff --git a/api/client-server/definitions/request_msisdn_validation.yaml b/api/client-server/definitions/request_msisdn_validation.yaml index 370a10cc..54897e63 100644 --- a/api/client-server/definitions/request_msisdn_validation.yaml +++ b/api/client-server/definitions/request_msisdn_validation.yaml @@ -22,5 +22,15 @@ allOf: The hostname of the identity server to communicate with. May optionally include a port. This parameter is ignored when the homeserver handles 3PID verification. + + This parameter is deprecated with a plan to be removed in a future specification + version for ``/account/password`` and ``/register`` requests. example: "id.example.com" - required: ["id_server"] + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. Servers + can treat this as optional to distinguish between r0.5-compatible clients + and this specification version. + + Required if an ``id_server`` is supplied. diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index e47db8a0..45201a20 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -25,9 +25,9 @@ properties: description: |- An optional field containing a URL where the client must submit the validation token to, with identical parameters to the Identity Service - API's ``POST /validate/email/submitToken`` endpoint. The homeserver must - send this token to the user (if applicable), who should then be - prompted to provide it to the client. + API's ``POST /validate/email/submitToken`` endpoint (without the requirement + for an access token). The homeserver must send this token to the user (if + applicable), who should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's involvement provided the homeserver diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 3bea351c..5fbfc198 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -59,6 +59,18 @@ paths: description: |- The maximum number of events to return. Default: 10. x-example: 3 + - in: query + name: filter + type: string + description: |- + A JSON ``RoomEventFilter`` to filter the returned events with. The + filter is only applied to ``events_before``, ``events_after``, and + ``state``. It is not applied to the ``event`` itself. The filter may + be applied before or/and after the ``limit`` parameter - whichever the + homeserver prefers. + + See `Filtering <#filtering>`_ for more information. + x-example: "66696p746572" responses: 200: description: The events and state surrounding the requested event. diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 69e39def..e172ea8a 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -101,7 +101,7 @@ paths: responses: 200: description: - The provided keys were sucessfully uploaded. + The provided keys were successfully uploaded. schema: type: object properties: diff --git a/api/client-server/openid.yaml b/api/client-server/openid.yaml index cb982fb3..01fdf68f 100644 --- a/api/client-server/openid.yaml +++ b/api/client-server/openid.yaml @@ -73,28 +73,7 @@ paths: "expires_in": 3600, } schema: - type: object - properties: - access_token: - type: string - description: |- - An access token the consumer may use to verify the identity of - the person who generated the token. This is given to the federation - API ``GET /openid/userinfo``. - token_type: - type: string - description: The string ``Bearer``. - matrix_server_name: - type: string - description: |- - The homeserver domain the consumer should use when attempting to - verify the user's identity. - expires_in: - type: integer - description: |- - The number of seconds before this token expires and a new one must - be generated. - required: ['access_token', 'token_type', 'matrix_server_name', 'expires_in'] + $ref: "definitions/openid_token.yaml" 429: description: This request was rate-limited. schema: diff --git a/api/client-server/pushrules.yaml b/api/client-server/pushrules.yaml index a1cbc354..32c3c9a1 100644 --- a/api/client-server/pushrules.yaml +++ b/api/client-server/pushrules.yaml @@ -498,6 +498,8 @@ paths: type: boolean description: Whether the push rule is enabled or not. required: ["enabled"] + tags: + - Push notifications put: summary: "Enable or disable a push rule." description: |- @@ -601,6 +603,8 @@ paths: items: type: string required: ["actions"] + tags: + - Push notifications put: summary: "Set the actions for a push rule." description: |- diff --git a/api/client-server/redaction.yaml b/api/client-server/redaction.yaml index 58141049..907b1d16 100644 --- a/api/client-server/redaction.yaml +++ b/api/client-server/redaction.yaml @@ -37,7 +37,7 @@ paths: This cannot be undone. Users may redact their own events, and any user with a power level - greater than or equal to the `redact` power level of the room may + greater than or equal to the ``redact`` power level of the room may redact events there. operationId: redactEvent security: diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 71177d0c..50ce4a96 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -95,18 +95,6 @@ paths: should be authenticated, but is instead used to authenticate the ``register`` call itself. "$ref": "definitions/auth_data.yaml" - bind_email: - type: boolean - description: |- - If true, the server binds the email used for authentication to - the Matrix ID with the identity server. - example: false - bind_msisdn: - type: boolean - description: |- - If true, the server binds the phone number used for authentication - to the Matrix ID with the identity server. - example: false username: type: string description: |- @@ -231,11 +219,8 @@ paths: description: |- The homeserver must check that the given email address is **not** already associated with an account on this homeserver. The homeserver - has the choice of validating the email address itself, or proxying the - request to the ``/validate/email/requestToken`` Identity Service API. The - request should be proxied to the domain that is sent by the client in - the ``id_server``. It is imperative that the homeserver keep a list of - trusted Identity Servers and only proxies to those it trusts. + should validate the email itself, either by sending a validation email + itself or by using a service it has control over. operationId: requestTokenToRegisterEmail parameters: - in: body @@ -284,11 +269,8 @@ paths: description: |- The homeserver must check that the given phone number is **not** already associated with an account on this homeserver. The homeserver - has the choice of validating the phone number itself, or proxying the - request to the ``/validate/msisdn/requestToken`` Identity Service API. The - request should be proxied to the domain that is sent by the client in - the ``id_server``. It is imperative that the homeserver keep a list of - trusted Identity Servers and only proxies to those it trusts. + should validate the phone number itself, either by sending a validation + message itself or by using a service it has control over. operationId: requestTokenToRegisterMSISDN parameters: - in: body @@ -346,8 +328,8 @@ paths: The homeserver may change the flows available depending on whether a valid access token is provided. The homeserver SHOULD NOT revoke the - access token provided in the request, however all other access tokens - for the user should be revoked if the request succeeds. + access token provided in the request. Whether other access tokens for + the user are revoked depends on the request parameters. security: - accessToken: [] operationId: changePassword @@ -361,6 +343,12 @@ paths: type: string description: The new password for the account. example: "ihatebananas" + logout_devices: + type: boolean + description: |- + Whether the other access tokens, and their associated devices, for the user should be + revoked if the request succeeds. Defaults to true. + example: true auth: description: |- Additional authentication information for the user-interactive authentication API. @@ -400,12 +388,8 @@ paths: email to the given address prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - The homeserver has the choice of validating the email address itself, - or proxying the request to the ``/validate/email/requestToken`` - Identity Service API. The request should be proxied to the domain that - is sent by the client in the ``id_server``. It is imperative that the - homeserver keep a list of trusted Identity Servers and only proxies to - those that it trusts. + The homeserver should validate the email itself, either by sending a + validation email itself or by using a service it has control over. .. |/register/email/requestToken| replace:: ``/register/email/requestToken`` @@ -437,7 +421,9 @@ paths: 400: description: |- The referenced third party identifier is not recognised by the - homeserver, or the request was invalid + homeserver, or the request was invalid. The error code ``M_SERVER_NOT_TRUSTED`` + can be returned if the server does not trust/support the identity server + provided in the request. schema: $ref: "definitions/errors/error.yaml" examples: @@ -461,12 +447,8 @@ paths: to the given phone number prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - The homeserver has the choice of validating the phone number itself, or - proxying the request to the ``/validate/msisdn/requestToken`` Identity - Service API. The request should be proxied to the domain that is sent - by the client in the ``id_server``. It is imperative that the - homeserver keep a list of trusted Identity Servers and only proxies to - those that it trusts. + The homeserver should validate the phone number itself, either by sending a + validation message itself or by using a service it has control over. .. |/register/msisdn/requestToken| replace:: ``/register/msisdn/requestToken`` @@ -497,7 +479,9 @@ paths: 400: description: |- The referenced third party identifier is not recognised by the - homeserver, or the request was invalid + homeserver, or the request was invalid. The error code ``M_SERVER_NOT_TRUSTED`` + can be returned if the server does not trust/support the identity server + provided in the request. schema: $ref: "definitions/errors/error.yaml" examples: @@ -519,6 +503,10 @@ paths: The homeserver may change the flows available depending on whether a valid access token is provided. + + Unlike other endpoints, this endpoint does not take an ``id_access_token`` + parameter because the homeserver is expected to sign the request to the + identity server instead. security: - accessToken: [] operationId: deactivateAccount diff --git a/api/client-server/room_send.yaml b/api/client-server/room_send.yaml index 6963f76c..fc8f3339 100644 --- a/api/client-server/room_send.yaml +++ b/api/client-server/room_send.yaml @@ -85,5 +85,7 @@ paths: type: string description: |- A unique identifier for the event. + required: + - event_id tags: - Room participation diff --git a/api/client-server/room_state.yaml b/api/client-server/room_state.yaml index 62168f26..640e2009 100644 --- a/api/client-server/room_state.yaml +++ b/api/client-server/room_state.yaml @@ -92,6 +92,8 @@ paths: type: string description: |- A unique identifier for the event. + required: + - event_id 403: description: |- The sender doesn't have permission to send the event into the room. diff --git a/api/client-server/room_upgrades.yaml b/api/client-server/room_upgrades.yaml index 55ee14dc..3aaaadcb 100644 --- a/api/client-server/room_upgrades.yaml +++ b/api/client-server/room_upgrades.yaml @@ -90,4 +90,4 @@ paths: schema: "$ref": "definitions/errors/error.yaml" tags: - - Room ugprades + - Room upgrades diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 00a332d9..bf804792 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -136,7 +136,8 @@ paths: title: Joined Rooms type: object description: |- - The rooms that the user has joined. + The rooms that the user has joined, mapped as room ID to + room information. additionalProperties: title: Joined Room type: object @@ -249,7 +250,8 @@ paths: title: Invited Rooms type: object description: |- - The rooms that the user has been invited to. + The rooms that the user has been invited to, mapped as room ID to + room information. additionalProperties: title: Invited Room type: object @@ -280,7 +282,8 @@ paths: title: Left rooms type: object description: |- - The rooms that the user has left or been banned from. + The rooms that the user has left or been banned from, mapped as room ID to + room information. additionalProperties: title: Left Room type: object diff --git a/api/client-server/third_party_membership.yaml b/api/client-server/third_party_membership.yaml index 077e1f6a..4c5890d1 100644 --- a/api/client-server/third_party_membership.yaml +++ b/api/client-server/third_party_membership.yaml @@ -92,6 +92,7 @@ paths: type: object example: { "id_server": "matrix.org", + "id_access_token": "abc123_OpaqueString", "medium": "email", "address": "cheeky@monkey.com" } @@ -99,6 +100,12 @@ paths: id_server: type: string description: The hostname+port of the identity server which should be used for third party identifier lookups. + id_access_token: + type: string + description: |- + An access token previously registered with the identity server. Servers + can treat this as optional to distinguish between r0.5-compatible clients + and this specification version. medium: type: string # TODO: Link to Identity Service spec when it eixsts @@ -106,7 +113,7 @@ paths: address: type: string description: The invitee's third party identifier. - required: ["id_server", "medium", "address"] + required: ["id_server", "id_access_token", "medium", "address"] responses: 200: description: The user has been invited to join the room. diff --git a/api/client-server/users.yaml b/api/client-server/users.yaml index 98744719..da8cf03f 100644 --- a/api/client-server/users.yaml +++ b/api/client-server/users.yaml @@ -31,7 +31,7 @@ paths: post: summary: Searches the user directory. description: |- - Performs a search for users on the homeserver. The homeserver may + Performs a search for users. The homeserver may determine which subset of users are searched, however the homeserver MUST at a minimum consider the users the requesting user shares a room with and those who reside in public rooms (known to the homeserver). diff --git a/api/identity/associations.yaml b/api/identity/associations.yaml index 8ff4a9ed..82c70fb8 100644 --- a/api/identity/associations.yaml +++ b/api/identity/associations.yaml @@ -30,6 +30,7 @@ paths: description: |- Determines if a given 3pid has been validated by a user. operationId: getValidated3pid + deprecated: true parameters: - in: query type: string @@ -104,6 +105,7 @@ paths: ``application/x-form-www-urlencoded`` data. However, this usage is deprecated. operationId: bind + deprecated: true parameters: - in: body name: body @@ -221,6 +223,7 @@ paths: through to the client requesting an unbind through a homeserver, if the homeserver is acting on behalf of a client. operationId: unbind + deprecated: true parameters: - in: body name: body diff --git a/api/identity/definitions/request_msisdn_validation.yaml b/api/identity/definitions/request_msisdn_validation.yaml index 018bd733..93cea2ed 100644 --- a/api/identity/definitions/request_msisdn_validation.yaml +++ b/api/identity/definitions/request_msisdn_validation.yaml @@ -30,8 +30,8 @@ properties: country: type: string description: |- - The two-letter uppercase ISO country code that the number in - ``phone_number`` should be parsed as if it were dialled from. + The two-letter uppercase ISO-3166-1 alpha-2 country code that the + number in ``phone_number`` should be parsed as if it were dialled from. example: "GB" phone_number: type: string diff --git a/api/identity/definitions/security.yaml b/api/identity/definitions/security.yaml new file mode 100644 index 00000000..ef49ff5c --- /dev/null +++ b/api/identity/definitions/security.yaml @@ -0,0 +1,18 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +accessToken: + type: apiKey + description: The access_token returned by a call to ``/register``. + name: access_token + in: query diff --git a/api/identity/email_associations.yaml b/api/identity/email_associations.yaml index 38186432..69ec7c58 100644 --- a/api/identity/email_associations.yaml +++ b/api/identity/email_associations.yaml @@ -46,6 +46,7 @@ paths: ``application/x-form-www-urlencoded`` data. However, this usage is deprecated. operationId: emailRequestToken + deprecated: true parameters: - in: body name: body @@ -92,6 +93,7 @@ paths: ``application/x-form-www-urlencoded`` data. However, this usage is deprecated. operationId: emailSubmitTokenPost + deprecated: true parameters: - in: body name: body @@ -142,6 +144,7 @@ paths: Note that, in contrast with the POST version, this endpoint will be used by end-users, and so the response should be human-readable. operationId: emailSubmitTokenGet + deprecated: true parameters: - in: query type: string @@ -162,7 +165,7 @@ paths: description: The token generated by the ``requestToken`` call and emailed to the user. x-example: atoken responses: - "200": + 200: description: Email address is validated. "3xx": description: |- diff --git a/api/identity/invitation_signing.yaml b/api/identity/invitation_signing.yaml index 4e10e2b5..e2ac28b0 100644 --- a/api/identity/invitation_signing.yaml +++ b/api/identity/invitation_signing.yaml @@ -33,6 +33,7 @@ paths: The identity server will look up ``token`` which was stored in a call to ``store-invite``, and fetch the sender of the invite. operationId: blindlySignStuff + deprecated: true parameters: - in: body name: body diff --git a/api/identity/lookup.yaml b/api/identity/lookup.yaml index fd50f94b..78fa3e3e 100644 --- a/api/identity/lookup.yaml +++ b/api/identity/lookup.yaml @@ -16,7 +16,7 @@ # limitations under the License. swagger: '2.0' info: - title: "Matrix Identity Service Lookup API" + title: "Matrix Identity Service Lookup API" version: "1.0.0" host: localhost:8090 schemes: @@ -32,6 +32,7 @@ paths: summary: Look up the Matrix user ID for a 3pid. description: Look up the Matrix user ID for a 3pid. operationId: lookupUser + deprecated: true parameters: - in: query type: string @@ -101,6 +102,7 @@ paths: summary: Lookup Matrix user IDs for a list of 3pids. description: Lookup Matrix user IDs for a list of 3pids. operationId: lookupUsers + deprecated: true parameters: - in: body name: body diff --git a/api/identity/phone_associations.yaml b/api/identity/phone_associations.yaml index b9933f1a..65bbff0c 100644 --- a/api/identity/phone_associations.yaml +++ b/api/identity/phone_associations.yaml @@ -46,6 +46,7 @@ paths: ``application/x-form-www-urlencoded`` data. However, this usage is deprecated. operationId: msisdnRequestToken + deprecated: true parameters: - in: body name: body @@ -94,6 +95,7 @@ paths: ``application/x-form-www-urlencoded`` data. However, this usage is deprecated. operationId: msisdnSubmitTokenPost + deprecated: true parameters: - in: body name: body @@ -144,6 +146,7 @@ paths: Note that, in contrast with the POST version, this endpoint will be used by end-users, and so the response should be human-readable. operationId: msisdnSubmitTokenGet + deprecated: true parameters: - in: query type: string @@ -164,7 +167,7 @@ paths: description: The token generated by the ``requestToken`` call and sent to the user. x-example: atoken responses: - "200": + 200: description: Phone number is validated. "3xx": description: |- diff --git a/api/identity/ping.yaml b/api/identity/ping.yaml index 05e12a87..d7249a77 100644 --- a/api/identity/ping.yaml +++ b/api/identity/ping.yaml @@ -36,6 +36,7 @@ paths: This is primarly used for auto-discovery and health check purposes by entities acting as a client for the identity server. operationId: ping + deprecated: true responses: 200: description: An identity server is ready to serve requests. diff --git a/api/identity/pubkey.yaml b/api/identity/pubkey.yaml index e657c61c..a585e7ec 100644 --- a/api/identity/pubkey.yaml +++ b/api/identity/pubkey.yaml @@ -30,6 +30,7 @@ paths: description: |- Get the public key for the passed key ID. operationId: getPubKey + deprecated: true parameters: - in: path type: string @@ -72,6 +73,7 @@ paths: Check whether a long-term public key is valid. The response should always be the same, provided the key exists. operationId: isPubKeyValid + deprecated: true parameters: - in: query type: string @@ -101,6 +103,7 @@ paths: description: |- Check whether a short-term public key is valid. operationId: isEphemeralPubKeyValid + deprecated: true parameters: - in: query type: string diff --git a/api/identity/store_invite.yaml b/api/identity/store_invite.yaml index bca78d7e..4a5d525d 100644 --- a/api/identity/store_invite.yaml +++ b/api/identity/store_invite.yaml @@ -55,6 +55,7 @@ paths: server's ability. Identity servers may use these variables when notifying the ``address`` of the pending invite for display purposes. operationId: storeInvite + deprecated: true parameters: - in: body name: body diff --git a/api/identity/v2_associations.yaml b/api/identity/v2_associations.yaml new file mode 100644 index 00000000..b9039a18 --- /dev/null +++ b/api/identity/v2_associations.yaml @@ -0,0 +1,333 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Establishing Associations API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/3pid/getValidated3pid": + get: + summary: Check whether ownership of a 3pid was validated. + description: |- + Determines if a given 3pid has been validated by a user. + operationId: getValidated3pidV2 + security: + - accessToken: [] + parameters: + - in: query + type: string + name: sid + description: The Session ID generated by the ``requestToken`` call. + required: true + x-example: 1234 + - in: query + type: string + name: client_secret + description: The client secret passed to the ``requestToken`` call. + required: true + x-example: monkeys_are_GREAT + responses: + 200: + description: Validation information for the session. + examples: + application/json: { + "medium": "email", + "validated_at": 1457622739026, + "address": "louise@bobs.burgers" + } + schema: + type: object + properties: + medium: + type: string + description: The medium type of the 3pid. + address: + type: string + description: The address of the 3pid being looked up. + validated_at: + type: integer + description: |- + Timestamp, in milliseconds, indicating the time that the 3pid + was validated. + required: ['medium', 'address', 'validated_at'] + 400: + description: |- + The session has not been validated. + + If the session has not been validated, then ``errcode`` will be + ``M_SESSION_NOT_VALIDATED``. If the session has timed out, then + ``errcode`` will be ``M_SESSION_EXPIRED``. + examples: + application/json: { + "errcode": "M_SESSION_NOT_VALIDATED", + "error": "This validation session has not yet been completed" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 404: + description: The Session ID or client secret were not found. + examples: + application/json: { + "errcode": "M_NO_VALID_SESSION", + "error": "No valid session was found matching that sid and client secret" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/3pid/bind": + post: + summary: Publish an association between a session and a Matrix user ID. + description: |- + Publish an association between a session and a Matrix user ID. + + Future calls to ``/lookup`` for any of the session\'s 3pids will return + this association. + + Note: for backwards compatibility with previous drafts of this + specification, the parameters may also be specified as + ``application/x-form-www-urlencoded`` data. However, this usage is + deprecated. + operationId: bindV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "mxid": "@ears:matrix.org" + } + properties: + sid: + type: string + description: The Session ID generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret passed to the ``requestToken`` call. + mxid: + type: string + description: The Matrix user ID to associate with the 3pids. + required: ["sid", "client_secret", "mxid"] + responses: + 200: + description: The association was published. + examples: + application/json: { + "address": "louise@bobs.burgers", + "medium": "email", + "mxid": "@ears:matrix.org", + "not_before": 1428825849161, + "not_after": 4582425849161, + "ts": 1428825849161, + "signatures": { + "matrix.org": { + "ed25519:0": "ENiU2YORYUJgE6WBMitU0mppbQjidDLanAusj8XS2nVRHPu+0t42OKA/r6zV6i2MzUbNQ3c3MiLScJuSsOiVDQ" + } + } + } + schema: + type: object + properties: + address: + type: string + description: The 3pid address of the user being looked up. + medium: + type: string + description: The medium type of the 3pid. + mxid: + type: string + description: The Matrix user ID associated with the 3pid. + not_before: + type: integer + description: A unix timestamp before which the association is not known to be valid. + not_after: + type: integer + description: A unix timestamp after which the association is not known to be valid. + ts: + type: integer + description: The unix timestamp at which the association was verified. + signatures: + type: object + description: |- + The signatures of the verifying identity servers which show that the + association should be trusted, if you trust the verifying identity + services. + $ref: "../../schemas/server-signatures.yaml" + required: + - address + - medium + - mxid + - not_before + - not_after + - ts + - signatures + 400: + description: |- + The association was not published. + + If the session has not been validated, then ``errcode`` will be + ``M_SESSION_NOT_VALIDATED``. If the session has timed out, then + ``errcode`` will be ``M_SESSION_EXPIRED``. + examples: + application/json: { + "errcode": "M_SESSION_NOT_VALIDATED", + "error": "This validation session has not yet been completed" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 404: + description: The Session ID or client secret were not found + examples: + application/json: { + "errcode": "M_NO_VALID_SESSION", + "error": "No valid session was found matching that sid and client secret" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/3pid/unbind": + post: + summary: Remove an association between a session and a Matrix user ID. + description: |- + Remove an association between a session and a Matrix user ID. + + Future calls to ``/lookup`` for any of the session's 3pids will not + return the removed association. + + The identity server should authenticate the request in one of two + ways: + + 1. The request is signed by the homeserver which controls the ``user_id``. + 2. The request includes the ``sid`` and ``client_secret`` parameters, + as per ``/3pid/bind``, which proves ownership of the 3PID. + + If this endpoint returns a JSON Matrix error, that error should be passed + through to the client requesting an unbind through a homeserver, if the + homeserver is acting on behalf of a client. + operationId: unbindV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "mxid": "@ears:example.org", + "threepid": { + "medium": "email", + "address": "monkeys_have_ears@example.org" + } + } + properties: + sid: + type: string + description: The Session ID generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret passed to the ``requestToken`` call. + mxid: + type: string + description: The Matrix user ID to remove from the 3pids. + threepid: + type: object + title: 3PID + description: |- + The 3PID to remove. Must match the 3PID used to generate the session + if using ``sid`` and ``client_secret`` to authenticate this request. + properties: + medium: + type: string + description: |- + A medium from the `3PID Types`_ Appendix, matching the medium + of the identifier to unbind. + address: + type: string + description: The 3PID address to remove. + required: ['medium', 'address'] + required: ["threepid", "mxid"] + responses: + 200: + description: The association was successfully removed. + examples: + application/json: {} + schema: + type: object + 400: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 404: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 403: + description: |- + The credentials supplied to authenticate the request were invalid. + This may also be returned if the identity server does not support + the chosen authentication method (such as blocking homeservers from + unbinding identifiers). + + Another common error code is ``M_TERMS_NOT_SIGNED`` where the user + needs to `agree to more terms`_ in order to continue. + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "Invalid homeserver signature" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 501: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. diff --git a/api/identity/v2_auth.yaml b/api/identity/v2_auth.yaml new file mode 100644 index 00000000..a34f679c --- /dev/null +++ b/api/identity/v2_auth.yaml @@ -0,0 +1,131 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Authentication API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/account/register": + post: + summary: Exchanges an OpenID token for an access token. + description: |- + Exchanges an OpenID token from the homeserver for an access token to + access the identity server. The request body is the same as the values + returned by ``/openid/request_token`` in the Client-Server API. + operationId: registerAccount + parameters: + - in: body + name: body + schema: + $ref: "../client-server/definitions/openid_token.yaml" + responses: + 200: + description: |- + A token which can be used to authenticate future requests to the + identity server. + examples: + application/json: { + "token": "abc123_OpaqueString" + } + schema: + type: object + properties: + token: + type: string + description: |- + An opaque string representing the token to authenticate future + requests to the identity server with. + required: ['token'] + "/account": + get: + summary: Gets account holder information for a given token. + description: |- + Gets information about what user owns the access token used in the request. + operationId: getAccount + security: + - accessToken: [] + parameters: [] + responses: + 200: + description: The token holder's information. + examples: + application/json: { + "user_id": "@alice:example.org" + } + schema: + type: object + properties: + user_id: + type: string + description: The user ID which registered the token. + required: ['user_id'] + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/account/logout": + post: + summary: Logs out an access token, rendering it unusable. + description: |- + Logs out the access token, preventing it from being used to authenticate + future requests to the server. + operationId: logout + security: + - accessToken: [] + parameters: [] + responses: + 200: + description: The token was successfully logged out. + examples: + application/json: {} + schema: + type: object + 401: + description: |- + The token is not registered or is otherwise unknown to the server. + examples: + application/json: { + "errcode": "M_UNKNOWN_TOKEN", + "error": "Unrecognised access token" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_email_associations.yaml b/api/identity/v2_email_associations.yaml new file mode 100644 index 00000000..76c3747e --- /dev/null +++ b/api/identity/v2_email_associations.yaml @@ -0,0 +1,216 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Email Associations API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/validate/email/requestToken": + post: + summary: Request a token for validating an email address. + description: |- + Create a session for validating an email address. + + The identity server will send an email containing a token. If that + token is presented to the identity server in the future, it indicates + that that user was able to read the email for that email address, and + so we validate ownership of the email address. + + Note that homeservers offer APIs that proxy this API, adding + additional behaviour on top, for example, + ``/register/email/requestToken`` is designed specifically for use when + registering an account and therefore will inform the user if the email + address given is already registered on the server. + + Note: for backwards compatibility with previous drafts of this + specification, the parameters may also be specified as + ``application/x-form-www-urlencoded`` data. However, this usage is + deprecated. + operationId: emailRequestTokenV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + $ref: "definitions/request_email_validation.yaml" + responses: + 200: + description: Session created. + schema: + $ref: "definitions/sid.yaml" + 400: + description: | + An error ocurred. Some possible errors are: + + - ``M_INVALID_EMAIL``: The email address provided was invalid. + - ``M_EMAIL_SEND_ERROR``: The validation email could not be sent. + examples: + application/json: { + "errcode": "M_INVALID_EMAIL", + "error": "The email address is not valid" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/validate/email/submitToken": + post: + summary: Validate ownership of an email address. + description: |- + Validate ownership of an email address. + + If the three parameters are consistent with a set generated by a + ``requestToken`` call, ownership of the email address is considered to + have been validated. This does not publish any information publicly, or + associate the email address with any Matrix user ID. Specifically, + calls to ``/lookup`` will not show a binding. + + The identity server is free to match the token case-insensitively, or + carry out other mapping operations such as unicode + normalisation. Whether to do so is an implementation detail for the + identity server. Clients must always pass on the token without + modification. + + Note: for backwards compatibility with previous drafts of this + specification, the parameters may also be specified as + ``application/x-form-www-urlencoded`` data. However, this usage is + deprecated. + operationId: emailSubmitTokenPostV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "token": "atoken" + } + properties: + sid: + type: string + description: The session ID, generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret that was supplied to the ``requestToken`` call. + token: + type: string + description: The token generated by the ``requestToken`` call and emailed to the user. + required: ["sid", "client_secret", "token"] + responses: + 200: + description: + The success of the validation. + examples: + application/json: { + "success": true + } + schema: + type: object + properties: + success: + type: boolean + description: Whether the validation was successful or not. + required: ['success'] + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + get: + summary: Validate ownership of an email address. + description: |- + Validate ownership of an email address. + + If the three parameters are consistent with a set generated by a + ``requestToken`` call, ownership of the email address is considered to + have been validated. This does not publish any information publicly, or + associate the email address with any Matrix user ID. Specifically, + calls to ``/lookup`` will not show a binding. + + Note that, in contrast with the POST version, this endpoint will be + used by end-users, and so the response should be human-readable. + operationId: emailSubmitTokenGetV2 + security: + - accessToken: [] + parameters: + - in: query + type: string + name: sid + required: true + description: The session ID, generated by the ``requestToken`` call. + x-example: 1234 + - in: query + type: string + name: client_secret + required: true + description: The client secret that was supplied to the ``requestToken`` call. + x-example: monkeys_are_GREAT + - in: query + type: string + name: token + required: true + description: The token generated by the ``requestToken`` call and emailed to the user. + x-example: atoken + responses: + 200: + description: Email address is validated. + "3xx": + description: |- + Email address is validated, and the ``next_link`` parameter was + provided to the ``requestToken`` call. The user must be redirected + to the URL provided by the ``next_link`` parameter. + "4xx": + description: + Validation failed. + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_invitation_signing.yaml b/api/identity/v2_invitation_signing.yaml new file mode 100644 index 00000000..0431233a --- /dev/null +++ b/api/identity/v2_invitation_signing.yaml @@ -0,0 +1,112 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Ephemeral Invitation Signing API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/sign-ed25519": + post: + summary: Sign invitation details + description: |- + Sign invitation details. + + The identity server will look up ``token`` which was stored in a call + to ``store-invite``, and fetch the sender of the invite. + operationId: blindlySignStuffV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + example: { + "mxid": "@foo:bar.com", + "token": "sometoken", + "private_key": "base64encodedkey" + } + properties: + mxid: + type: string + description: The Matrix user ID of the user accepting the invitation. + token: + type: string + description: The token from the call to ``store-invite``. + private_key: + type: string + description: The private key, encoded as `Unpadded base64`_. + required: ["mxid", "token", "private_key"] + responses: + 200: + description: The signed JSON of the mxid, sender, and token. + schema: + type: object + properties: + mxid: + type: string + description: The Matrix user ID of the user accepting the invitation. + sender: + type: string + description: The Matrix user ID of the user who sent the invitation. + signatures: + type: object + description: The signature of the mxid, sender, and token. + $ref: "../../schemas/server-signatures.yaml" + token: + type: string + description: The token for the invitation. + required: ['mxid', 'sender', 'signatures', 'token'] + examples: + application/json: { + "mxid": "@foo:bar.com", + "sender": "@baz:bar.com", + "signatures": { + "my.id.server": { + "ed25519:0": "def987" + } + }, + "token": "abc123" + } + 404: + description: The token was not found. + examples: + application/json: { + "errcode": "M_UNRECOGNIZED", + "error": "Didn't recognize token" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_lookup.yaml b/api/identity/v2_lookup.yaml new file mode 100644 index 00000000..8acafd75 --- /dev/null +++ b/api/identity/v2_lookup.yaml @@ -0,0 +1,148 @@ +# Copyright 2016 OpenMarket Ltd +# Copyright 2017 Kamax.io +# Copyright 2017 New Vector Ltd +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Lookup API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/hash_details": + get: + summary: Gets hash function information from the server. + description: |- + Gets parameters for hashing identifiers from the server. This can include + any of the algorithms defined in this specification. + operationId: getHashDetails + security: + - accessToken: [] + parameters: [] + responses: + 200: + description: The hash function information. + examples: + application/json: { + "lookup_pepper": "matrixrocks", + "algorithms": ["none", "sha256"] + } + schema: + type: object + properties: + lookup_pepper: + type: string + description: |- + The pepper the client MUST use in hashing identifiers, and MUST + supply to the ``/lookup`` endpoint when performing lookups. + + Servers SHOULD rotate this string often. + algorithms: + type: array + items: + type: string + description: |- + The algorithms the server supports. Must contain at least ``sha256``. + required: ['lookup_pepper', 'algorithms'] + "/lookup": + post: + summary: Look up Matrix User IDs for a set of 3PIDs. + description: |- + Looks up the set of Matrix User IDs which have bound the 3PIDs given, if + bindings are available. Note that the format of the addresses is defined + later in this specification. + operationId: lookupUsersV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + algorithm: + type: string + description: |- + The algorithm the client is using to encode the ``addresses``. This + should be one of the available options from ``/hash_details``. + example: "sha256" + pepper: + type: string + description: |- + The pepper from ``/hash_details``. This is required even when the + ``algorithm`` does not make use of it. + example: "matrixrocks" + addresses: + type: array + items: + type: string + description: |- + The addresses to look up. The format of the entries here depend on + the ``algorithm`` used. Note that queries which have been incorrectly + hashed or formatted will lead to no matches. + example: [ + "4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc", + "nlo35_T5fzSGZzJApqu8lgIudJvmOQtDaHtr-I4rU7I" + ] + required: ['algorithm', 'pepper', 'addresses'] + responses: + 200: + description: + The associations for any matched ``addresses``. + examples: + application/json: { + "mappings": { + "4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc": "@alice:example.org" + } + } + schema: + type: object + properties: + mappings: + type: object + description: |- + Any applicable mappings of ``addresses`` to Matrix User IDs. Addresses + which do not have associations will not be included, which can make + this property be an empty object. + title: AssociatedMappings + additionalProperties: + type: string + required: ['mappings'] + 400: + description: + The client's request was invalid in some way. One possible problem could + be the ``pepper`` being invalid after the server has rotated it - this is + presented with the ``M_INVALID_PEPPER`` error code. Clients SHOULD make + a call to ``/hash_details`` to get a new pepper in this scenario, being + careful to avoid retry loops. + + ``M_INVALID_PARAM`` can also be returned to indicate the client supplied + an ``algorithm`` that is unknown to the server. + examples: + application/json: { + "errcode": "M_INVALID_PEPPER", + "error": "Unknown or invalid pepper - has it been rotated?" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_phone_associations.yaml b/api/identity/v2_phone_associations.yaml new file mode 100644 index 00000000..6d4ad79b --- /dev/null +++ b/api/identity/v2_phone_associations.yaml @@ -0,0 +1,218 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Phone Number Associations API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/validate/msisdn/requestToken": + post: + summary: Request a token for validating a phone number. + description: |- + Create a session for validating a phone number. + + The identity server will send an SMS message containing a token. If + that token is presented to the identity server in the future, it + indicates that that user was able to read the SMS for that phone + number, and so we validate ownership of the phone number. + + Note that homeservers offer APIs that proxy this API, adding + additional behaviour on top, for example, + ``/register/msisdn/requestToken`` is designed specifically for use when + registering an account and therefore will inform the user if the phone + number given is already registered on the server. + + Note: for backwards compatibility with previous drafts of this + specification, the parameters may also be specified as + ``application/x-form-www-urlencoded`` data. However, this usage is + deprecated. + operationId: msisdnRequestTokenV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + $ref: "definitions/request_msisdn_validation.yaml" + responses: + 200: + description: Session created. + schema: + $ref: "definitions/sid.yaml" + 400: + description: | + An error ocurred. Some possible errors are: + + - ``M_INVALID_ADDRESS``: The phone number provided was invalid. + - ``M_SEND_ERROR``: The validation SMS could not be sent. + - ``M_DESTINATION_REJECTED``: The identity server cannot deliver an + SMS to the provided country or region. + examples: + application/json: { + "errcode": "M_INVALID_ADDRESS", + "error": "The phone number is not valid" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/validate/msisdn/submitToken": + post: + summary: Validate ownership of a phone number. + description: |- + Validate ownership of a phone number. + + If the three parameters are consistent with a set generated by a + ``requestToken`` call, ownership of the phone number is considered to + have been validated. This does not publish any information publicly, or + associate the phone number address with any Matrix user + ID. Specifically, calls to ``/lookup`` will not show a binding. + + The identity server is free to match the token case-insensitively, or + carry out other mapping operations such as unicode + normalisation. Whether to do so is an implementation detail for the + identity server. Clients must always pass on the token without + modification. + + Note: for backwards compatibility with previous drafts of this + specification, the parameters may also be specified as + ``application/x-form-www-urlencoded`` data. However, this usage is + deprecated. + operationId: msisdnSubmitTokenPostV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "token": "atoken" + } + properties: + sid: + type: string + description: The session ID, generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret that was supplied to the ``requestToken`` call. + token: + type: string + description: The token generated by the ``requestToken`` call and sent to the user. + required: ["sid", "client_secret", "token"] + responses: + 200: + description: + The success of the validation. + examples: + application/json: { + "success": true + } + schema: + type: object + properties: + success: + type: boolean + description: Whether the validation was successful or not. + required: ['success'] + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + get: + summary: Validate ownership of a phone number. + description: |- + Validate ownership of a phone number. + + If the three parameters are consistent with a set generated by a + ``requestToken`` call, ownership of the phone number address is + considered to have been validated. This does not publish any + information publicly, or associate the phone number with any Matrix + user ID. Specifically, calls to ``/lookup`` will not show a binding. + + Note that, in contrast with the POST version, this endpoint will be + used by end-users, and so the response should be human-readable. + operationId: msisdnSubmitTokenGetV2 + security: + - accessToken: [] + parameters: + - in: query + type: string + name: sid + required: true + description: The session ID, generated by the ``requestToken`` call. + x-example: 1234 + - in: query + type: string + name: client_secret + required: true + description: The client secret that was supplied to the ``requestToken`` call. + x-example: monkeys_are_GREAT + - in: query + type: string + name: token + required: true + description: The token generated by the ``requestToken`` call and sent to the user. + x-example: atoken + responses: + 200: + description: Phone number is validated. + "3xx": + description: |- + Phone number address is validated, and the ``next_link`` parameter + was provided to the ``requestToken`` call. The user must be + redirected to the URL provided by the ``next_link`` parameter. + "4xx": + description: + Validation failed. + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_ping.yaml b/api/identity/v2_ping.yaml new file mode 100644 index 00000000..61f5d35b --- /dev/null +++ b/api/identity/v2_ping.yaml @@ -0,0 +1,46 @@ +# Copyright 2018 Kamax Sàrl +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +swagger: "2.0" +info: + title: "Matrix Identity Service Ping API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity +produces: + - application/json +paths: + "/v2": + get: + summary: Checks that an identity server is available at this API endpoint. + description: |- + Checks that an identity server is available at this API endpoint. + + To discover that an identity server is available at a specific URL, + this endpoint can be queried and will return an empty object. + + This is primarly used for auto-discovery and health check purposes + by entities acting as a client for the identity server. + operationId: pingV2 + responses: + 200: + description: An identity server is ready to serve requests. + examples: + application/json: {} + schema: + type: object diff --git a/api/identity/v2_pubkey.yaml b/api/identity/v2_pubkey.yaml new file mode 100644 index 00000000..68facd68 --- /dev/null +++ b/api/identity/v2_pubkey.yaml @@ -0,0 +1,127 @@ +# Copyright 2016 OpenMarket Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Public Key API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +paths: + "/pubkey/{keyId}": + get: + summary: Get a public key. + description: |- + Get the public key for the passed key ID. + operationId: getPubKeyV2 + parameters: + - in: path + type: string + name: keyId + required: true + description: |- + The ID of the key. This should take the form algorithm:identifier + where algorithm identifies the signing algorithm, and the identifier + is an opaque string. + x-example: "ed25519:0" + responses: + 200: + description: + The public key exists. + examples: + application/json: { + "public_key": "VXuGitF39UH5iRfvbIknlvlAVKgD1BsLDMvBf0pmp7c" + } + schema: + type: object + properties: + public_key: + type: string + description: Unpadded Base64 encoded public key. + required: ['public_key'] + 404: + description: + The public key was not found. + examples: + application/json: { + "errcode": "M_NOT_FOUND", + "error": "The public key was not found" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + "/pubkey/isvalid": + get: + summary: Check whether a long-term public key is valid. + description: |- + Check whether a long-term public key is valid. The response should always + be the same, provided the key exists. + operationId: isPubKeyValidV2 + parameters: + - in: query + type: string + name: public_key + required: true + description: |- + The unpadded base64-encoded public key to check. + x-example: "VXuGitF39UH5iRfvbIknlvlAVKgD1BsLDMvBf0pmp7c" + responses: + 200: + description: + The validity of the public key. + examples: + application/json: { + "valid": true + } + schema: + type: object + properties: + valid: + type: boolean + description: Whether the public key is recognised and is currently valid. + required: ['valid'] + "/pubkey/ephemeral/isvalid": + get: + summary: Check whether a short-term public key is valid. + description: |- + Check whether a short-term public key is valid. + operationId: isEphemeralPubKeyValidV2 + parameters: + - in: query + type: string + name: public_key + required: true + description: |- + The unpadded base64-encoded public key to check. + x-example: "VXuGitF39UH5iRfvbIknlvlAVKgD1BsLDMvBf0pmp7c" + responses: + 200: + description: + The validity of the public key. + examples: + application/json: { + "valid": true + } + schema: + type: object + properties: + valid: + type: boolean + description: Whether the public key is recognised and is currently valid. + required: ['valid'] diff --git a/api/identity/v2_store_invite.yaml b/api/identity/v2_store_invite.yaml new file mode 100644 index 00000000..9b7a653c --- /dev/null +++ b/api/identity/v2_store_invite.yaml @@ -0,0 +1,176 @@ +# Copyright 2018 New Vector Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Store Invitations API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/store-invite": + post: + summary: Store pending invitations to a user's 3pid. + description: |- + Store pending invitations to a user's 3pid. + + In addition to the request parameters specified below, an arbitrary + number of other parameters may also be specified. These may be used in + the invite message generation described below. + + The service will generate a random token and an ephemeral key used for + accepting the invite. + + The service also generates a ``display_name`` for the inviter, which is + a redacted version of ``address`` which does not leak the full contents + of the ``address``. + + The service records persistently all of the above information. + + It also generates an email containing all of this data, sent to the + ``address`` parameter, notifying them of the invitation. + + Also, the generated ephemeral public key will be listed as valid on + requests to ``/_matrix/identity/v2/pubkey/ephemeral/isvalid``. + + Currently, invites may only be issued for 3pids of the ``email`` medium. + + Optional fields in the request should be populated to the best of the + server's ability. Identity servers may use these variables when notifying + the ``address`` of the pending invite for display purposes. + operationId: storeInviteV2 + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + medium: + type: string + description: The literal string ``email``. + example: "email" + address: + type: string + description: The email address of the invited user. + example: "foo@example.com" + room_id: + type: string + description: The Matrix room ID to which the user is invited + example: "!something:example.org" + sender: + type: string + description: The Matrix user ID of the inviting user + example: "@bob:example.com" + room_alias: + type: string + description: |- + The Matrix room alias for the room to which the user is + invited. This should be retrieved from the ``m.room.canonical_alias`` + state event. + example: "#somewhere:exmaple.org" + room_avatar_url: + type: string + description: |- + The Content URI for the room to which the user is invited. This should + be retrieved from the ``m.room.avatar`` state event. + example: "mxc://example.org/s0meM3dia" + room_join_rules: + type: string + description: |- + The ``join_rule`` for the room to which the user is invited. This should + be retrieved from the ``m.room.join_rules`` state event. + example: "public" + room_name: + type: string + description: |- + The name of the room to which the user is invited. This should be retrieved + from the ``m.room.name`` state event. + example: "Bob's Emporium of Messages" + sender_display_name: + type: string + description: The display name of the user ID initiating the invite. + example: "Bob Smith" + sender_avatar_url: + type: string + description: The Content URI for the avatar of the user ID initiating the invite. + example: "mxc://example.org/an0th3rM3dia" + required: ["medium", "address", "room_id", "sender"] + responses: + 200: + description: The invitation was stored. + schema: + type: object + properties: + token: + type: string + description: | + The generated token. Must be a string consisting of the + characters ``[0-9a-zA-Z.=_-]``. Its length must not exceed + 255 characters and it must not be empty. + public_keys: + type: array + description: | + A list of [server's long-term public key, generated ephemeral + public key]. + items: + type: string + display_name: + type: string + description: The generated (redacted) display_name. + required: ['token', 'public_keys', 'display_name'] + example: + application/json: { + "token": "sometoken", + "public_keys": [ + "serverpublickey", + "ephemeralpublickey" + ], + "display_name": "f...@b..." + } + 400: + description: | + An error has occured. + + If the 3pid is already bound to a Matrix user ID, the error code + will be ``M_THREEPID_IN_USE``. If the medium is unsupported, the + error code will be ``M_UNRECOGNIZED``. + examples: + application/json: { + "errcode": "M_THREEPID_IN_USE", + "error": "Binding already known", + "mxid": "@alice:example.com" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" + 403: + description: | + The user must do something in order to use this endpoint. One example + is an ``M_TERMS_NOT_SIGNED`` error where the user must `agree to more terms`_. + examples: + application/json: { + "errcode": "M_TERMS_NOT_SIGNED", + "error": "Please accept our updated terms of service before continuing" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" diff --git a/api/identity/v2_terms.yaml b/api/identity/v2_terms.yaml new file mode 100644 index 00000000..9b831fba --- /dev/null +++ b/api/identity/v2_terms.yaml @@ -0,0 +1,149 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +swagger: '2.0' +info: + title: "Matrix Identity Service Terms of Service API" + version: "2.0.0" +host: localhost:8090 +schemes: + - https +basePath: /_matrix/identity/v2 +consumes: + - application/json +produces: + - application/json +securityDefinitions: + $ref: definitions/security.yaml +paths: + "/terms": + get: + summary: Gets the terms of service offered by the server. + description: |- + Gets all the terms of service offered by the server. The client is expected + to filter through the terms to determine which terms need acceptance from the + user. Note that this endpoint does not require authentication. + operationId: getTerms + parameters: [] + responses: + 200: + description: |- + The terms of service offered by the server. + examples: + application/json: { + "policies": { + "terms_of_service": { + "version": "2.0", + "en": { + "name": "Terms of Service", + "url": "https://example.org/somewhere/terms-2.0-en.html" + }, + "fr": { + "name": "Conditions d'utilisation", + "url": "https://example.org/somewhere/terms-2.0-fr.html" + } + }, + "privacy_policy": { + "version": "1.2", + "en": { + "name": "Privacy Policy", + "url": "https://example.org/somewhere/privacy-1.2-en.html" + }, + "fr": { + "name": "Politique de confidentialité", + "url": "https://example.org/somewhere/privacy-1.2-fr.html" + } + } + } + } + schema: + type: object + properties: + policies: + type: object + title: Policy Map + description: |- + The policies the server offers. Mapped from arbitrary ID (unused in + this version of the specification) to a Policy Object. + additionalProperties: + type: object + title: Policy Object + description: |- + The policy. Includes a map of language (ISO 639-2) to language-specific + policy information. + properties: + version: + type: string + description: |- + The version for the policy. There are no requirements on what this + might be and could be "alpha", semantically versioned, or arbitrary. + required: ['version'] + # TODO: TravisR - Make this render + additionalProperties: + type: object + title: Internationalised Policy + description: |- + The policy information for the specified language. + properties: + name: + type: string + description: The translated name of the policy. + url: + type: string + description: |- + The URL, which should include the policy ID, version, and language + in it, to be presented to the user as the policy. URLs should have + all three criteria to avoid conflicts when the policy is updated + in the future: for example, if this was "https://example.org/terms.html" + then the server would be unable to update it because the client would + have already added that URL to the ``m.accepted_terms`` collection. + required: ['name', 'url'] + required: ['policies'] + post: + summary: Indicates acceptance of terms to the server. + description: |- + Called by a client to indicate that the user has accepted/agreed to the included + set of URLs. Servers MUST NOT assume that the client will be sending all previously + accepted URLs and should therefore append the provided URLs to what the server + already knows has been accepted. + + Clients MUST provide the URL of the policy in the language that was presented + to the user. Servers SHOULD consider acceptance of any one language's URL as + acceptance for all other languages of that policy. + + The server should avoid returning ``M_TERMS_NOT_SIGNED`` because the client + may not be accepting all terms at once. + operationId: agreeToTerms + security: + - accessToken: [] + parameters: + - in: body + name: body + schema: + type: object + properties: + user_accepts: + type: array + items: + type: string + description: The URLs the user is accepting in this request. + example: "https://example.org/somewhere/terms-2.0-en.html" + required: ['user_accepts'] + responses: + 200: + description: |- + The server has considered the user as having accepted the provided URLs. + examples: + application/json: {} + schema: + type: object diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index 8bc6c563..1e025a52 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -20,7 +20,6 @@ properties: server_name: type: string description: DNS name of the homeserver. - required: true example: "example.org" verify_keys: type: object @@ -31,7 +30,6 @@ properties: algorithm and ``abc123`` being the version in the example below). Together, this forms the Key ID. The version must have characters matching the regular expression ``[a-zA-Z0-9_]``. - required: true additionalProperties: type: object title: Verify Key @@ -44,8 +42,8 @@ properties: key: type: string description: The `Unpadded Base64`_ encoded key. - required: true example: "VGhpcyBzaG91bGQgYmUgYSByZWFsIGVkMjU1MTkgcGF5bG9hZA" + required: ["key"] old_verify_keys: type: object description: |- @@ -69,13 +67,12 @@ properties: type: integer format: int64 description: POSIX timestamp in milliseconds for when this key expired. - required: true example: 1532645052628 key: type: string description: The `Unpadded Base64`_ encoded key. - required: true example: "VGhpcyBzaG91bGQgYmUgeW91ciBvbGQga2V5J3MgZWQyNTUxOSBwYXlsb2FkLg" + required: ["expired_ts", "key"] signatures: type: object description: Digital signatures for this object signed using the ``verify_keys``. @@ -103,3 +100,4 @@ properties: publishes a key which is valid for a significant amount of time without a way for the homeserver owner to revoke it. example: 1052262000000 +required: ["server_name", "verify_keys"] diff --git a/api/server-server/event_auth.yaml b/api/server-server/event_auth.yaml index 0db0d401..8f545e1d 100644 --- a/api/server-server/event_auth.yaml +++ b/api/server-server/event_auth.yaml @@ -72,137 +72,3 @@ paths: example: $ref: "examples/minimal_pdu.json" required: ['auth_chain'] - "/query_auth/{roomId}/{eventId}": - post: - summary: Compare auth chains with the receiving server - description: |- - Compares the auth chain provided with what the receiving server has for the - room ID and event ID combination. - - The auth difference can be calculated in two parts, where the "remote auth" - is the auth chain provided by the sending server and the "local auth" is the - auth chain the receiving server has. With those lists, the algorithm works - bottom-up after sorting each chain by depth then by event ID. The differences - are then discovered and returned as the response to this API call. - operationId: compareEventAuth - security: - - signedRequest: [] - parameters: - - in: path - name: roomId - type: string - description: The room ID to compare the auth chain in. - required: true - x-example: "!abc123:matrix.org" - - in: path - name: eventId - type: string - description: The event ID to compare the auth chain of. - required: true - x-example: "$helloworld:example.org" - - in: body - name: body - schema: - type: object - properties: - auth_chain: - type: array - description: |- - The auth chain (the "remote auth"). Note that events have a different - format depending on the room version - check the `room version specification`_ - for precise event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ contained in the auth chain. The event format - varies depending on the room version - check the `room version specification`_ - for precise event formats. - properties: [] - example: - $ref: "examples/minimal_pdu.json" - missing: - type: array - description: |- - A list of event IDs that the sender thinks the receiver is missing. - items: - type: string - example: [] - rejects: - type: object - description: |- - The set of events that the sending server has rejected from the provided - auth chain. - - The ``string`` key is the event ID that was rejected. - additionalProperties: - type: object - title: Rejection Reason - properties: - reason: - type: enum - enum: ['auth_error', 'replaced', 'not_ancestor'] - description: |- - The reason for the event being rejected. - required: ['reason'] - example: { - "$some_event:example.org": { - "reason": "auth_error" - } - } - required: ['auth_chain'] - responses: - 200: - description: The auth chain differences, as determined by the receiver. - schema: - type: object - properties: - auth_chain: - type: array - description: |- - The auth chain the receiver has, and used to determine the auth - chain differences (the "local auth"). Note that events have a different - format depending on the room version - check the `room version specification`_ - for precise event formats. - items: - type: object - title: PDU - description: |- - The `PDUs <#pdus>`_ contained in the auth chain. The event format - varies depending on the room version - check the `room version specification`_ - for precise event formats. - properties: [] - example: - $ref: "examples/minimal_pdu.json" - missing: - type: array - description: |- - The list of event IDs that the receiver believes it is missing, - after comparing the "remote auth" and "local auth" chains. - items: - type: string - example: ["$a_missing_event:example.org"] - rejects: - type: object - description: |- - The set of events that the receiving server has rejected from the - auth chain, not including events that the sending server is missing - as determined from the difference algorithm. - - The ``string`` key is the event ID that was rejected. - additionalProperties: - type: object - title: Rejection Reason - properties: - reason: - type: enum - enum: ['auth_error', 'replaced', 'not_ancestor'] - description: |- - The reason for the event being rejected. - required: ['reason'] - example: { - "$some_event:example.org": { - "reason": "auth_error" - } - } - required: ['auth_chain', 'missing', 'rejects'] diff --git a/api/server-server/public_rooms.yaml b/api/server-server/public_rooms.yaml index b7691023..0216f0c3 100644 --- a/api/server-server/public_rooms.yaml +++ b/api/server-server/public_rooms.yaml @@ -68,3 +68,163 @@ paths: description: The public room list for the homeserver. schema: $ref: "../client-server/definitions/public_rooms_response.yaml" + post: + summary: Gets the public rooms on the server with optional filter. + description: |- + Lists the public rooms on the server, with optional filter. + + This API returns paginated responses. The rooms are ordered by the number + of joined members, with the largest rooms first. + + Note that this endpoint receives and returns the same format that is seen + in the Client-Server API's ``POST /publicRooms`` endpoint. + operationId: queryPublicRooms + security: + - signedRequest: [] + parameters: + - in: body + name: body + required: true + description: |- + Options for which rooms to return. + schema: + type: object + properties: + limit: + type: integer + description: |- + Limit the number of results returned. + since: + type: string + description: |- + A pagination token from a previous request, allowing servers + to get the next (or previous) batch of rooms. The direction + of pagination is specified solely by which token is supplied, + rather than via an explicit flag. + filter: + type: object + title: "Filter" + description: |- + Filter to apply to the results. + properties: + generic_search_term: + type: string + description: |- + A string to search for in the room metadata, e.g. name, + topic, canonical alias etc. (Optional). + include_all_networks: + type: boolean + description: |- + Whether or not to include all known networks/protocols from + application services on the homeserver. Defaults to false. + example: false + third_party_instance_id: + type: string + description: |- + The specific third party network/protocol to request from the + homeserver. Can only be used if ``include_all_networks`` is false. + example: "irc" + example: { + "limit": 10, + "filter": { + "generic_search_term": "foo" + }, + "include_all_networks": false, + "third_party_instance_id": "irc" + } + responses: + 200: + description: A list of the rooms on the server. + schema: + type: object + description: A list of the rooms on the server. + required: ["chunk"] + properties: + chunk: + title: "PublicRoomsChunks" + type: array + description: |- + A paginated chunk of public rooms. + items: + type: object + title: "PublicRoomsChunk" + required: + - room_id + - num_joined_members + - world_readable + - guest_can_join + properties: + aliases: + type: array + description: |- + Aliases of the room. May be empty. + items: + type: string + canonical_alias: + type: string + description: |- + The canonical alias of the room, if any. + name: + type: string + description: |- + The name of the room, if any. + num_joined_members: + type: integer + description: |- + The number of members joined to the room. + room_id: + type: string + description: |- + The ID of the room. + topic: + type: string + description: |- + The topic of the room, if any. + world_readable: + type: boolean + description: |- + Whether the room may be viewed by guest users without joining. + guest_can_join: + type: boolean + description: |- + Whether guest users may join the room and participate in it. + If they can, they will be subject to ordinary power level + rules like any other user. + avatar_url: + type: string + description: The URL for the room's avatar, if one is set. + next_batch: + type: string + description: |- + A pagination token for the response. The absence of this token + means there are no more results to fetch and the client should + stop paginating. + prev_batch: + type: string + description: |- + A pagination token that allows fetching previous results. The + absence of this token means there are no results before this + batch, i.e. this is the first batch. + total_room_count_estimate: + type: integer + description: |- + An estimate on the total number of public rooms, if the + server has an estimate. + examples: + application/json: { + "chunk": [ + { + "aliases": ["#murrays:cheese.bar"], + "avatar_url": "mxc://bleeker.street/CHEDDARandBRIE", + "guest_can_join": false, + "name": "CHEESE", + "num_joined_members": 37, + "room_id": "!ol19s:bleecker.street", + "topic": "Tasty tasty cheese", + "world_readable": true + } + ], + "next_batch": "p190q", + "prev_batch": "p1902", + "total_room_count_estimate": 115 + } diff --git a/changelogs/README.md b/changelogs/README.md index a5fb1fb7..5a5b6271 100644 --- a/changelogs/README.md +++ b/changelogs/README.md @@ -2,14 +2,14 @@ # Changelogs -[Towncrier](https://github.com/hawkowl/towncrier) is used to manage the changelog and +[Towncrier](https://github.com/hawkowl/towncrier) is used to manage the changelog and keep it up to date. Because of this, updating a changelog is really easy. ## How to update a changelog when releasing an API 1. Ensure you're in your Python 3 virtual environment 2. `cd` your way to the API you're releasing (eg: `cd changelogs/client_server`) -3. Run `towncrier --version "r0.4.0" --name "client-server" --yes` substituting the +3. Run `towncrier --version "r0.4.0" --name "client-server" --yes` substituting the variables as approprite. Note that `--name` is required although the value is ignored. 4. Commit the changes and finish the release process. @@ -26,27 +26,32 @@ For this example, we're going to pretend that the `server_server` API doesn't ex directory = "newsfragments" issue_format = "`#{issue} `_" title_format = "{version}" - + [[tool.towncrier.type]] directory = "breaking" name = "Breaking Changes" showcontent = true - + [[tool.towncrier.type]] directory = "deprecation" name = "Deprecations" showcontent = true - + [[tool.towncrier.type]] directory = "new" name = "New Endpoints" showcontent = true - + + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" showcontent = true - + [[tool.towncrier.type]] directory = "clarification" name = "Spec Clarifications" diff --git a/changelogs/application_service/pyproject.toml b/changelogs/application_service/pyproject.toml index 44d430e8..278def78 100644 --- a/changelogs/application_service/pyproject.toml +++ b/changelogs/application_service/pyproject.toml @@ -3,27 +3,32 @@ directory = "newsfragments" issue_format = "`#{issue} `_" title_format = "{version}" - + [[tool.towncrier.type]] directory = "breaking" name = "Breaking Changes" showcontent = true - + [[tool.towncrier.type]] directory = "deprecation" name = "Deprecations" showcontent = true - + [[tool.towncrier.type]] directory = "new" name = "New Endpoints" showcontent = true - + + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" showcontent = true - + [[tool.towncrier.type]] directory = "clarification" name = "Spec Clarifications" diff --git a/changelogs/client_server.rst b/changelogs/client_server.rst index c56a1073..bc12da5c 100644 --- a/changelogs/client_server.rst +++ b/changelogs/client_server.rst @@ -1,3 +1,47 @@ +r0.6.0 +====== + +Breaking Changes +---------------- + +- Add ``id_access_token`` as a required request parameter to a few endpoints which require an ``id_server`` parameter as part of `MSC2140 `_. (`#2255 `_) + + +New Endpoints +------------- + +- Add ``POST /account/3pid/unbind`` for removing a 3PID from an identity server. (`#2282 `_) + + +Backwards Compatible Changes +---------------------------- + +- Add ``M_USER_DEACTIVATED`` error code. (`#2234 `_) +- Remove ``bind_msisdn`` and ``bind_email`` from ``/register`` now that the identity server's bind endpoint requires authentication. (`#2279 `_) +- Add ``m.identity_server`` account data for tracking the user's preferred identity server. (`#2281 `_) +- Deprecate ``id_server`` and make it optional in several places. (`#2310 `_) + + +Spec Clarifications +------------------- + +- Add missing format fields to ``m.room.message$m.notice`` schema. (`#2125 `_) +- Remove "required" designation from the ``url`` field of certain ``m.room.message`` msgtypes. (`#2129 `_) +- Fix various typos throughout the specification. (`#2131 `_, `#2136 `_, `#2148 `_, `#2215 `_) +- Clarify the distinction between ``m.key.verification.start`` and its ``m.sas.v1`` variant. (`#2132 `_) +- Fix link to Olm signing specification. (`#2133 `_) +- Clarify the conditions for the ``.m.rule.room_one_to_one`` push rule. (`#2152 `_) +- Clarify the encryption algorithms supported by the device of the device keys example. (`#2157 `_) +- Clarify that ``/rooms/:roomId/event/:eventId`` returns a Matrix error. (`#2204 `_) +- Add a missing ``state_key`` check on ``.m.rule.tombstone``. (`#2223 `_) +- Fix the ``m.room_key_request`` ``action`` value, setting it from ``cancel_request`` to ``request_cancellation``. (`#2247 `_) +- Clarify that the ``submit_url`` field is without authentication. (`#2341 `_) +- Clarify the expected phone number format. (`#2342 `_) +- Clarify that clients should consider not requesting URL previews in encrypted rooms. (`#2343 `_) +- Add missing information on how filters are meant to work with ``/context``. (`#2344 `_) +- Clarify what the keys are for rooms in ``/sync``. (`#2345 `_) + + r0.5.0 ====== @@ -30,7 +74,7 @@ Backwards Compatible Changes - Support optional features by having clients query for capabilities. (`#1829 `_, `#1879 `_) - Add ``M_RESOURCE_LIMIT_EXCEEDED`` as an error code for when homeservers exceed limits imposed on them. (`#1874 `_) - Emit ``M_UNSUPPORTED_ROOM_VERSION`` error codes where applicable on ``/createRoom`` and ``/invite`` APIs. (`#1908 `_) -- Add a ``.m.rule.tombstone`` default push rule for room ugprade notifications. (`#2020 `_) +- Add a ``.m.rule.tombstone`` default push rule for room upgrade notifications. (`#2020 `_) - Add support for sending server notices to clients. (`#2026 `_) - Add MSISDN (phone number) support to User-Interactive Authentication. (`#2030 `_) - Add the option to lazy-load room members for increased client performance. (`#2035 `_) diff --git a/changelogs/client_server/newsfragments/2125.clarification b/changelogs/client_server/newsfragments/2125.clarification deleted file mode 100644 index c71cdfff..00000000 --- a/changelogs/client_server/newsfragments/2125.clarification +++ /dev/null @@ -1 +0,0 @@ -Add missing format fields to ``m.room.message$m.notice`` schema. diff --git a/changelogs/client_server/newsfragments/2129.clarification b/changelogs/client_server/newsfragments/2129.clarification deleted file mode 100644 index 9d67deac..00000000 --- a/changelogs/client_server/newsfragments/2129.clarification +++ /dev/null @@ -1 +0,0 @@ -Remove "required" designation from the ``url`` field of certain ``m.room.message`` msgtypes. diff --git a/changelogs/client_server/newsfragments/2131.clarification b/changelogs/client_server/newsfragments/2131.clarification deleted file mode 100644 index 3c41fb60..00000000 --- a/changelogs/client_server/newsfragments/2131.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix typo in key verification framework section. diff --git a/changelogs/client_server/newsfragments/2132.clarification b/changelogs/client_server/newsfragments/2132.clarification deleted file mode 100644 index b8a4cc8a..00000000 --- a/changelogs/client_server/newsfragments/2132.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the distinction between ``m.key.verification.start`` and its ``m.sas.v1`` variant. diff --git a/changelogs/client_server/newsfragments/2133.clarification b/changelogs/client_server/newsfragments/2133.clarification deleted file mode 100644 index 3a003179..00000000 --- a/changelogs/client_server/newsfragments/2133.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix link to Olm signing specification. diff --git a/changelogs/client_server/newsfragments/2136.clarification b/changelogs/client_server/newsfragments/2136.clarification deleted file mode 100644 index 3ccb2333..00000000 --- a/changelogs/client_server/newsfragments/2136.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various typos throughout the specification. diff --git a/changelogs/client_server/newsfragments/2148.misc b/changelogs/client_server/newsfragments/2148.misc deleted file mode 100644 index d5514131..00000000 --- a/changelogs/client_server/newsfragments/2148.misc +++ /dev/null @@ -1 +0,0 @@ -Fix a small duplicated "as". diff --git a/changelogs/client_server/newsfragments/2152.clarification b/changelogs/client_server/newsfragments/2152.clarification deleted file mode 100644 index 03fde9ff..00000000 --- a/changelogs/client_server/newsfragments/2152.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the conditions for the ``.m.rule.room_one_to_one`` push rule. diff --git a/changelogs/client_server/newsfragments/2157.clarification b/changelogs/client_server/newsfragments/2157.clarification deleted file mode 100644 index 5c1d549b..00000000 --- a/changelogs/client_server/newsfragments/2157.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the encryption algorithms supported by the device of the device keys example. diff --git a/changelogs/client_server/newsfragments/2204.clarification b/changelogs/client_server/newsfragments/2204.clarification deleted file mode 100644 index 7a66b08b..00000000 --- a/changelogs/client_server/newsfragments/2204.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that ``/rooms/:roomId/event/:eventId`` returns a Matrix error. diff --git a/changelogs/client_server/newsfragments/2215.clarification b/changelogs/client_server/newsfragments/2215.clarification deleted file mode 100644 index 3ccb2333..00000000 --- a/changelogs/client_server/newsfragments/2215.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various typos throughout the specification. diff --git a/changelogs/client_server/newsfragments/2223.clarification b/changelogs/client_server/newsfragments/2223.clarification deleted file mode 100644 index 165b7e13..00000000 --- a/changelogs/client_server/newsfragments/2223.clarification +++ /dev/null @@ -1 +0,0 @@ -Add a missing ``state_key`` check on ``.m.rule.tombstone``. diff --git a/changelogs/client_server/newsfragments/2234.feature b/changelogs/client_server/newsfragments/2234.feature deleted file mode 100644 index bb1883b3..00000000 --- a/changelogs/client_server/newsfragments/2234.feature +++ /dev/null @@ -1 +0,0 @@ -Add ``M_USER_DEACTIVATED`` error code. diff --git a/changelogs/client_server/newsfragments/2245.clarification b/changelogs/client_server/newsfragments/2245.clarification new file mode 100644 index 00000000..31e5c2df --- /dev/null +++ b/changelogs/client_server/newsfragments/2245.clarification @@ -0,0 +1 @@ +List available enum values for the room versions capability. diff --git a/changelogs/client_server/newsfragments/2351.clarification b/changelogs/client_server/newsfragments/2351.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/client_server/newsfragments/2351.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2369.clarification b/changelogs/client_server/newsfragments/2369.clarification new file mode 100644 index 00000000..c7fd0fd9 --- /dev/null +++ b/changelogs/client_server/newsfragments/2369.clarification @@ -0,0 +1 @@ +Minor clarifications to token-based User-Interactive Authentication. diff --git a/changelogs/client_server/newsfragments/2381.clarification b/changelogs/client_server/newsfragments/2381.clarification new file mode 100644 index 00000000..70bd6d18 --- /dev/null +++ b/changelogs/client_server/newsfragments/2381.clarification @@ -0,0 +1 @@ +Minor clarification for what the user directory searches. diff --git a/changelogs/client_server/newsfragments/2415.clarification b/changelogs/client_server/newsfragments/2415.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/client_server/newsfragments/2415.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2430.clarification b/changelogs/client_server/newsfragments/2430.clarification new file mode 100644 index 00000000..56be0c1a --- /dev/null +++ b/changelogs/client_server/newsfragments/2430.clarification @@ -0,0 +1 @@ +Fix key export format example to match the specification. diff --git a/changelogs/client_server/newsfragments/2434.feature b/changelogs/client_server/newsfragments/2434.feature new file mode 100644 index 00000000..2b8f36f5 --- /dev/null +++ b/changelogs/client_server/newsfragments/2434.feature @@ -0,0 +1 @@ +Added data structures for defining moderation policies in rooms per `MSC2313 `_. diff --git a/changelogs/client_server/newsfragments/2453.clarification b/changelogs/client_server/newsfragments/2453.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/client_server/newsfragments/2453.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2492.clarification b/changelogs/client_server/newsfragments/2492.clarification new file mode 100644 index 00000000..88c478ec --- /dev/null +++ b/changelogs/client_server/newsfragments/2492.clarification @@ -0,0 +1 @@ +Clarify the IV data type for encrypted files. diff --git a/changelogs/client_server/newsfragments/2519.clarification b/changelogs/client_server/newsfragments/2519.clarification new file mode 100644 index 00000000..4b32da1e --- /dev/null +++ b/changelogs/client_server/newsfragments/2519.clarification @@ -0,0 +1 @@ +Fix the ``.m.rule.contains_user_name`` default push rule to set the highlight tweak. diff --git a/changelogs/client_server/newsfragments/2523.feature b/changelogs/client_server/newsfragments/2523.feature new file mode 100644 index 00000000..e45d1c2f --- /dev/null +++ b/changelogs/client_server/newsfragments/2523.feature @@ -0,0 +1 @@ +Optionally invalidate other access tokens during password modification per `MSC2457 `_. diff --git a/changelogs/client_server/newsfragments/2524.clarification b/changelogs/client_server/newsfragments/2524.clarification new file mode 100644 index 00000000..902a9c3f --- /dev/null +++ b/changelogs/client_server/newsfragments/2524.clarification @@ -0,0 +1 @@ +Fix various spelling errors throughout the specification. diff --git a/changelogs/client_server/newsfragments/2525.clarification b/changelogs/client_server/newsfragments/2525.clarification new file mode 100644 index 00000000..df1fc538 --- /dev/null +++ b/changelogs/client_server/newsfragments/2525.clarification @@ -0,0 +1 @@ +Clarify that an ``event_id`` is returned when sending events. diff --git a/changelogs/client_server/newsfragments/2532.feature b/changelogs/client_server/newsfragments/2532.feature new file mode 100644 index 00000000..cf74a289 --- /dev/null +++ b/changelogs/client_server/newsfragments/2532.feature @@ -0,0 +1 @@ +Add User-Interactive Authentication for SSO-backed homeserver per `MSC2454 `_. diff --git a/changelogs/client_server/pyproject.toml b/changelogs/client_server/pyproject.toml index 8fa3f6b5..eb9e7b4e 100644 --- a/changelogs/client_server/pyproject.toml +++ b/changelogs/client_server/pyproject.toml @@ -3,27 +3,32 @@ directory = "newsfragments" issue_format = "`#{issue} `_" title_format = "{version}" - + [[tool.towncrier.type]] directory = "breaking" name = "Breaking Changes" showcontent = true - + [[tool.towncrier.type]] directory = "deprecation" name = "Deprecations" showcontent = true - + [[tool.towncrier.type]] directory = "new" name = "New Endpoints" showcontent = true - + + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" showcontent = true - + [[tool.towncrier.type]] directory = "clarification" name = "Spec Clarifications" diff --git a/changelogs/identity_service.rst b/changelogs/identity_service.rst index cb06709d..7cccc0cf 100644 --- a/changelogs/identity_service.rst +++ b/changelogs/identity_service.rst @@ -1,3 +1,20 @@ +r0.3.0 +====== + +New Endpoints +------------- + +- Add ``/account``, ``/account/register``, and ``/account/logout`` to authenticate with the identity server. (`#2255 `_) +- Add endpoints for accepting and handling terms of service. (`#2258 `_) +- Add ``/hash_details`` and a new ``/lookup`` endpoint for performing hashed association lookups. (`#2287 `_) + + +Backwards Compatible Changes +---------------------------- + +- Deprecate the v1 API in favour of an authenticated v2 API. (`#2254 `_) + + r0.2.1 ====== diff --git a/changelogs/identity_service/pyproject.toml b/changelogs/identity_service/pyproject.toml index a7fe582d..2e50b9df 100644 --- a/changelogs/identity_service/pyproject.toml +++ b/changelogs/identity_service/pyproject.toml @@ -19,6 +19,11 @@ name = "New Endpoints" showcontent = true + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" diff --git a/changelogs/push_gateway/pyproject.toml b/changelogs/push_gateway/pyproject.toml index dad1bc04..9f2595c9 100644 --- a/changelogs/push_gateway/pyproject.toml +++ b/changelogs/push_gateway/pyproject.toml @@ -3,27 +3,32 @@ directory = "newsfragments" issue_format = "`#{issue} `_" title_format = "{version}" - + [[tool.towncrier.type]] directory = "breaking" name = "Breaking Changes" showcontent = true - + [[tool.towncrier.type]] directory = "deprecation" name = "Deprecations" showcontent = true - + [[tool.towncrier.type]] directory = "new" name = "New Endpoints" showcontent = true - + + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" showcontent = true - + [[tool.towncrier.type]] directory = "clarification" name = "Spec Clarifications" diff --git a/changelogs/server_server/newsfragments/2035.new b/changelogs/server_server/newsfragments/2035.new new file mode 100644 index 00000000..6794c4ea --- /dev/null +++ b/changelogs/server_server/newsfragments/2035.new @@ -0,0 +1 @@ +Add new ``POST /publicRooms`` endpoint for filtering the room directory. diff --git a/changelogs/server_server/newsfragments/2392.clarification b/changelogs/server_server/newsfragments/2392.clarification new file mode 100644 index 00000000..b09420e4 --- /dev/null +++ b/changelogs/server_server/newsfragments/2392.clarification @@ -0,0 +1 @@ +Move auth event selection to a more obvious location. diff --git a/changelogs/server_server/newsfragments/2470.removal b/changelogs/server_server/newsfragments/2470.removal new file mode 100644 index 00000000..51cdadd4 --- /dev/null +++ b/changelogs/server_server/newsfragments/2470.removal @@ -0,0 +1 @@ +Remove the unused ``query_auth`` API per `MSC2451 `_. diff --git a/changelogs/server_server/newsfragments/2510.clarification b/changelogs/server_server/newsfragments/2510.clarification new file mode 100644 index 00000000..9c96c5a5 --- /dev/null +++ b/changelogs/server_server/newsfragments/2510.clarification @@ -0,0 +1 @@ +Fix typo in Request Authentication python example. diff --git a/changelogs/server_server/newsfragments/2527.clarification b/changelogs/server_server/newsfragments/2527.clarification new file mode 100644 index 00000000..329d5da2 --- /dev/null +++ b/changelogs/server_server/newsfragments/2527.clarification @@ -0,0 +1 @@ +Clarify which fields are required on the key server endpoints. diff --git a/changelogs/server_server/pyproject.toml b/changelogs/server_server/pyproject.toml index 98478527..6a9dca1d 100644 --- a/changelogs/server_server/pyproject.toml +++ b/changelogs/server_server/pyproject.toml @@ -3,27 +3,32 @@ directory = "newsfragments" issue_format = "`#{issue} `_" title_format = "{version}" - + [[tool.towncrier.type]] directory = "breaking" name = "Breaking Changes" showcontent = true - + [[tool.towncrier.type]] directory = "deprecation" name = "Deprecations" showcontent = true - + [[tool.towncrier.type]] directory = "new" name = "New Endpoints" showcontent = true - + + [[tool.towncrier.type]] + directory = "removal" + name = "Removed Endpoints" + showcontent = true + [[tool.towncrier.type]] directory = "feature" name = "Backwards Compatible Changes" showcontent = true - + [[tool.towncrier.type]] directory = "clarification" name = "Spec Clarifications" diff --git a/event-schemas/examples/m.accepted_terms b/event-schemas/examples/m.accepted_terms new file mode 100644 index 00000000..5e8dad16 --- /dev/null +++ b/event-schemas/examples/m.accepted_terms @@ -0,0 +1,10 @@ +{ + "$ref": "core/event.json", + "type": "m.accepted_terms", + "content": { + "accepted": [ + "https://example.org/somewhere/terms-1.2-en.html", + "https://example.org/somewhere/privacy-1.2-en.html" + ] + } +} diff --git a/event-schemas/examples/m.identity_server b/event-schemas/examples/m.identity_server new file mode 100644 index 00000000..32855e9c --- /dev/null +++ b/event-schemas/examples/m.identity_server @@ -0,0 +1,7 @@ +{ + "$ref": "core/event.json", + "type": "m.identity_server", + "content": { + "base_url": "https://example.org" + } +} diff --git a/event-schemas/examples/m.policy.rule.room b/event-schemas/examples/m.policy.rule.room new file mode 100644 index 00000000..5a532cb5 --- /dev/null +++ b/event-schemas/examples/m.policy.rule.room @@ -0,0 +1,10 @@ +{ + "$ref": "core/state_event.json", + "type": "m.policy.rule.room", + "state_key": "rule:#*:example.org", + "content": { + "entity": "#*:example.org", + "recommendation": "m.ban", + "reason": "undesirable content" + } +} diff --git a/event-schemas/examples/m.policy.rule.server b/event-schemas/examples/m.policy.rule.server new file mode 100644 index 00000000..3d740a28 --- /dev/null +++ b/event-schemas/examples/m.policy.rule.server @@ -0,0 +1,10 @@ +{ + "$ref": "core/state_event.json", + "type": "m.policy.rule.server", + "state_key": "rule:*.example.org", + "content": { + "entity": "*.example.org", + "recommendation": "m.ban", + "reason": "undesirable engagement" + } +} diff --git a/event-schemas/examples/m.policy.rule.user b/event-schemas/examples/m.policy.rule.user new file mode 100644 index 00000000..eb3832da --- /dev/null +++ b/event-schemas/examples/m.policy.rule.user @@ -0,0 +1,10 @@ +{ + "$ref": "core/state_event.json", + "type": "m.policy.rule.user", + "state_key": "rule:@alice*:example.org", + "content": { + "entity": "@alice*:example.org", + "recommendation": "m.ban", + "reason": "undesirable behaviour" + } +} diff --git a/event-schemas/examples/m.room_key_request$cancel_request b/event-schemas/examples/m.room_key_request$cancel_request index c6eb25de..afc1c350 100644 --- a/event-schemas/examples/m.room_key_request$cancel_request +++ b/event-schemas/examples/m.room_key_request$cancel_request @@ -1,6 +1,6 @@ { "content": { - "action": "cancel_request", + "action": "request_cancellation", "requesting_device_id": "RJYKSTBOIE", "request_id": "1495474790150.19" }, diff --git a/event-schemas/moderation_policy_rule.yaml b/event-schemas/moderation_policy_rule.yaml new file mode 100644 index 00000000..34ad4d9a --- /dev/null +++ b/event-schemas/moderation_policy_rule.yaml @@ -0,0 +1,30 @@ +# Copyright 2020 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +properties: + entity: + description: |- + The entity affected by this rule. Glob characters ``*`` and ``?`` can be used + to match zero or more and one or more characters respectively. + type: string + recommendation: + description: The suggested action to take. Currently only ``m.ban`` is specified. + type: string + reason: + description: The human-readable description for the ``recommendation``. + type: string +type: object +required: + - entity + - recommendation + - reason diff --git a/event-schemas/schema/m.accepted_terms b/event-schemas/schema/m.accepted_terms new file mode 100644 index 00000000..510e741d --- /dev/null +++ b/event-schemas/schema/m.accepted_terms @@ -0,0 +1,23 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml +description: |- + A list of terms URLs the user has previously accepted. Clients SHOULD use this + to avoid presenting the user with terms they have already agreed to. +properties: + content: + type: object + properties: + accepted: + type: array + items: + type: string + description: |- + The list of URLs the user has previously accepted. Should be appended to + when the user agrees to new terms. + type: + enum: + - m.accepted_terms + type: string +title: Accepted Terms of Service URLs +type: object diff --git a/event-schemas/schema/m.identity_server b/event-schemas/schema/m.identity_server new file mode 100644 index 00000000..75d2409f --- /dev/null +++ b/event-schemas/schema/m.identity_server @@ -0,0 +1,23 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml +description: |- + Persists the user's preferred identity server, or preference to not use + an identity server at all, in the user's account data. +properties: + content: + type: object + properties: + base_url: + type: string + description: |- + The URL of the identity server the user prefers to use, or ``null`` + if the user does not want to use an identity server. This value is + similar in structure to the ``base_url`` for identity servers in the + ``.well-known/matrix/client`` schema. + type: + enum: + - m.identity_server + type: string +title: Identity Server Preference +type: object diff --git a/event-schemas/schema/m.policy.rule.room b/event-schemas/schema/m.policy.rule.room new file mode 100644 index 00000000..ff81543e --- /dev/null +++ b/event-schemas/schema/m.policy.rule.room @@ -0,0 +1,15 @@ +--- +allOf: + - $ref: core-event-schema/state_event.yaml +description: A moderation policy rule which affects room IDs and room aliases. +properties: + content: + $ref: "../moderation_policy_rule.yaml" + state_key: + description: An arbitrary string decided upon by the sender. + type: string + type: + enum: + - m.policy.rule.room + type: string +type: object diff --git a/event-schemas/schema/m.policy.rule.server b/event-schemas/schema/m.policy.rule.server new file mode 100644 index 00000000..ca37e8ff --- /dev/null +++ b/event-schemas/schema/m.policy.rule.server @@ -0,0 +1,15 @@ +--- +allOf: + - $ref: core-event-schema/state_event.yaml +description: A moderation policy rule which affects servers. +properties: + content: + $ref: "../moderation_policy_rule.yaml" + state_key: + description: An arbitrary string decided upon by the sender. + type: string + type: + enum: + - m.policy.rule.server + type: string +type: object diff --git a/event-schemas/schema/m.policy.rule.user b/event-schemas/schema/m.policy.rule.user new file mode 100644 index 00000000..4fa65ad8 --- /dev/null +++ b/event-schemas/schema/m.policy.rule.user @@ -0,0 +1,15 @@ +--- +allOf: + - $ref: core-event-schema/state_event.yaml +description: A moderation policy rule which affects users. +properties: + content: + $ref: "../moderation_policy_rule.yaml" + state_key: + description: An arbitrary string decided upon by the sender. + type: string + type: + enum: + - m.policy.rule.user + type: string +type: object diff --git a/event-schemas/schema/m.room_key_request b/event-schemas/schema/m.room_key_request index 007d0086..c08ac0e3 100644 --- a/event-schemas/schema/m.room_key_request +++ b/event-schemas/schema/m.room_key_request @@ -38,7 +38,7 @@ properties: action: enum: - request - - cancel_request + - request_cancellation type: string requesting_device_id: description: ID of the device requesting the key. diff --git a/meta/documentation_style.rst b/meta/documentation_style.rst index e59ad229..e3d363c4 100644 --- a/meta/documentation_style.rst +++ b/meta/documentation_style.rst @@ -8,10 +8,7 @@ in. Format ------ -We use restructured text throughout all the documentation. You should NOT use -markdown (github-flavored or otherwise). This format was chosen partly because -Sphinx only supports RST. - +Documentation is written either in github-flavored markdown or RST. Sections -------- diff --git a/meta/github-labels.rst b/meta/github-labels.rst new file mode 100644 index 00000000..f674b81b --- /dev/null +++ b/meta/github-labels.rst @@ -0,0 +1,91 @@ +The following labels are used to help categorize issues: + +`spec-omission `_ +-------------------------------------------------------------------------------- + +Things which have been implemented but not currently specified. These may range +from entire API endpoints, to particular options or return parameters. + +Issues with this label will have been implemented in `Synapse +`_. Normally there will be a design +document in Google Docs or similar which describes the feature. + +Examples: + +* `Spec PUT /directory/list `_ +* `Unspec'd server_name request param for /join/{roomIdOrAlias} + `_ + +`clarification `_ +-------------------------------------------------------------------------------- + +An area where the spec could do with being more explicit. + +Examples: + +* `Spec the implicit limit on /syncs + `_ + +* `Clarify the meaning of the currently_active flags in presence events + `_ + +`spec-bug `_ +---------------------------------------------------------------------- + +Something which is in the spec, but is wrong. + +Note: this is *not* for things that are badly designed or don't work well +(for which see 'improvement' or 'feature') - it is for places where the +spec doesn't match reality. + +Examples: + +* `swagger is wrong for directory PUT + `_ + +* `receipts section still refers to initialSync + `_ + +`improvement `_ +---------------------------------------------------------------------------- + +A suggestion for a relatively simple improvement to the protocol. + +Examples: + +* `We need a 'remove 3PID' API so that users can remove mappings + `_ +* `We should mandate that /publicRooms requires an access_token + `_ + +`feature `_ +-------------------------------------------------------------------- + +A suggestion for a significant extension to the matrix protocol which +needs considerable consideration before implementation. + +Examples: + +* `Peer-to-peer Matrix `_ +* `Specify a means for clients to "edit" previous messages + `_ + + +`wart `_ +-------------------------------------------------------------- + +A point where the protocol is inconsistent or inelegant, but which isn't really +causing anybody any problems right now. Might be nice to consider fixing one +day. + + +`question `_ +---------------------------------------------------------------------- + +A thought or idea about the protocol which we aren't really sure whether to +pursue or not. + +Examples: + +* `Should we prepend anti-eval code to our json responses? + `_ diff --git a/proposals/0000-proposal-template.md b/proposals/0000-proposal-template.md index 8c8cdcea..e50aed8b 100644 --- a/proposals/0000-proposal-template.md +++ b/proposals/0000-proposal-template.md @@ -41,34 +41,17 @@ The template should have the following sections: * **Introduction** - This should cover the primary problem and broad description of the solution. * **Proposal** - The gory details of the proposal. -* **Tradeoffs** - Any items of the proposal that are less desirable should be listed here. Alternative - solutions to the same problem could also be listed here. * **Potential issues** - This is where problems with the proposal would be listed, such as changes that are not backwards compatible. +* **Alternatives** - This section lists alternative solutions to the same + problem which have been considered and dismsissed. * **Security considerations** - Discussion of what steps were taken to avoid security issues in the future and any potential risks in the proposal. -* **Conclusion** - A repeat of the problem and solution. Furthermore, the template should not be required to be followed. However it is strongly recommended to maintain some sense of consistency between proposals. -## Tradeoffs - -*This is where alternative solutions could be listed. There's almost always another way to do things -and this section gives you the opportunity to highlight why those ways are not as desirable. The -argument made in this example is that all of the text provided by the template could be integrated -into the proposals introduction, although with some risk of losing clarity.* - -Instead of adding a template to the repository, the assistance it provides could be integrated into -the proposal process itself. There is an argument to be had that the proposal process should be as -descriptive as possible, although having even more detail in the proposals introduction could lead to -some confusion or lack of understanding. Not to mention if the document is too large then potential -authors could be scared off as the process suddenly looks a lot more complicated than it is. For those -reasons, this proposal does not consider integrating the template in the proposals introduction a good -idea. - - ## Potential issues *Not all proposals are perfect. Sometimes there's a known disadvantage to implementing the proposal, @@ -84,6 +67,22 @@ is beneficial and not considered a significant problem because it will lead to a can follow. +## Alternatives + +*This is where alternative solutions could be listed. There's almost always another way to do things +and this section gives you the opportunity to highlight why those ways are not as desirable. The +argument made in this example is that all of the text provided by the template could be integrated +into the proposals introduction, although with some risk of losing clarity.* + +Instead of adding a template to the repository, the assistance it provides could be integrated into +the proposal process itself. There is an argument to be had that the proposal process should be as +descriptive as possible, although having even more detail in the proposals introduction could lead to +some confusion or lack of understanding. Not to mention if the document is too large then potential +authors could be scared off as the process suddenly looks a lot more complicated than it is. For those +reasons, this proposal does not consider integrating the template in the proposals introduction a good +idea. + + ## Security considerations *Some proposals may have some security aspect to them that was addressed in the proposed solution. This @@ -95,19 +94,12 @@ of concerns where possible.* By having a template available, people would know what the desired detail for a proposal is. This is not considered a risk because it is important that people understand the proposal process from start to end. +## Unstable prefix -## Conclusion - -*Repeating the problem and solution in different words helps reviewers understand the problem a bit more. -This section should wrap up any loose ends left in the document, as well as cover a brief overview of the -content in each section. Note that the example here doesn't touch on the specific implementation details -described in the "Proposal" section - just the high-level points made there.* - -Not having a template for people to follow when making their proposals could lead to large differences -between each MSC. This would make it difficult for reviewers, and there's a potential that some information -could be left out by accident. A template written in the same format the proposal process requires would -give authors the ability to understand how to better explain their own proposal. - -A descriptive template would help potential authors comprehend what the proposal process requires by -demonstrating what is expected of a proposal. Although this is more effort up front, it would lead to more -time saved in the future due to questions about the process. +*If a proposal is implemented before it is included in the spec, then implementers must ensure that the +implementation is compatible with the final version that lands in the spec. This generally means that +experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary. +For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section +should be used to document things such as what endpoints and names are being used while the feature is +in development, the name of the unstable feature flag to use to detect support for the feature, or what +migration steps are needed to switch to newer versions of the proposal.* diff --git a/proposals/1219-storing-megolm-keys-serverside.md b/proposals/1219-storing-megolm-keys-serverside.md new file mode 100644 index 00000000..95430fe2 --- /dev/null +++ b/proposals/1219-storing-megolm-keys-serverside.md @@ -0,0 +1,562 @@ +Storing megolm keys serverside +============================== + +Background +---------- + +A user who uses end-to-end encryption will usually have many inbound group session +keys. Users who log into new devices and want to read old messages will need a +convenient way to transfer the session keys from one device to another. While +users can currently export their keys from one device and import them to +another, this is involves several steps and may be cumbersome for many users. +Users can also share keys from one device to another, but this has several +limitations, such as the fact that key shares only share one key at a time, and +require another logged-in device to be active. + +To help resolve this, we *optionally* let clients store an encrypted copy of +their megolm inbound session keys on the homeserver. Clients can keep the +backup up to date, so that users will always have the keys needed to decrypt +their conversations. The backup could be used not just for new logins, but +also to support clients with limited local storage for keys (clients can store +old keys to the backup, and remove their local copy, retrieving the key from +the backup when needed). + +To recover keys from the backup, a user will need to enter a recovery key to +decrypt the backup. The backup will be encrypted using public key +cryptography, so that any of a user's devices can back up keys without needing +the user to enter the recovery key until they need to read from the backup. + +See also: + +* https://github.com/matrix-org/matrix-doc/issues/1219 +* https://github.com/vector-im/riot-web/issues/3661 +* https://github.com/vector-im/riot-web/issues/5675 +* https://docs.google.com/document/d/1MOoIA9qEKIhUQ3UmKZG-loqA8e0BzgWKKlKRUGMynVc/edit# + (old version of proposal) + +Proposal +-------- + +This proposal creates new APIs to allow clients to back up room decryption keys +on the server. Room decryption keys are encrypted (using public key crypto) +before being sent to the server along with some unencrypted metadata to allow +the server to manage the backups. If a key for a new megolm session is +uploaded, it is added to the current backup. If a key is uploaded for a megolm +session is that is already present in the backup, the server will use the +metadata to determine which version of the key is "better". The way in which +the server determines which key is "better" is described in the [Storing +Keys](#storing-keys) section. The user is given a private recovery key in +order to recover the keys from the backup in the future. + +Clients can create new key backups (sometimes also referred to in the API as +backup versions) to replace the current backup. Aside from the initial backup +creation, a client might start a new a backup when, for example, a user loses a +device and wants to ensure that that device does not get any new decryption +keys. In this case, the client will then create a new backup using a new key +that the device does not have access to. + +Once one client has created a backup, other clients can fetch the public part +of the recovery key from the server and add keys to the backup, if they trust +that the backup was not created by a malicious device. + +### Possible UX for interactive clients + +This section gives an example of how a client might handle key backups. Clients +may behave differently. + +On receipt of encryption keys (1st time): + +1. client checks if there is an existing backup: `GET /room_keys/version` + 1. if not, ask if the user wants to back up keys + 1. if yes: + 1. generate new curve25519 key pair, which will be the recovery key + 2. create new backup: `POST /room_keys/version` + 3. display private key for user to save (see below for the + [format of the recovery key](#recovery-key)) + 2. if no, exit and remember decision (user can change their mind later) + 3. while prompting, continue to poll `GET /room_keys/versions`, as + another device may have created a backup. If so, go to 1.2. + 2. if yes, either get the public part of the recovery key and check that it + is signed by the master cross-signing key, or prompt user to enter the + private part of the recovery key (which can derive the public part). + 1. User can also decide to create a new backup, in which case, go to 1.1. +2. send key to backup: `PUT /room_keys/keys/${roomId}/${sessionId}?version=$v` +3. continue backing up keys as we receive them (may receive a + `M_WRONG_ROOM_KEYS_VERSION` error if a new backup has been created: + see below) + +On `M_WRONG_ROOM_KEYS_VERSION` error when trying to `PUT` keys: + +1. get the current version +2. notify the user that there is a new backup, and display relevant information +3. confirm with user that they want to use the backup (user may want use the + backup, to stop backing up keys, or to create a new backup) +4. ensure the public part of the recovery key is signed by the user's master + key, or prompt the user to enter the private part of the recovery key + +On receipt of undecryptable message: + +1. ask user if they want to restore backup (ask whether to get individual key, + room keys, or all keys). (This can be done in the same place as asking if + the user wants to request keys from other devices.) +2. if yes, prompt for private key, and get keys: `GET /room_keys/keys` + +Users can also set up, disable, or rotate backups, or restore from backup via user +settings. + +### Recovery key + +The recovery key can be saved by the user directly, stored encrypted on the +server (using the method proposed in +[MSC1946](https://github.com/matrix-org/matrix-doc/issues/1946)), or both. If +the key is saved directly by the user, then the code is constructed as follows: + +1. The 256-bit curve25519 private key is prepended by the bytes `0x8B` and + `0x01` +2. All the bytes in the string above, including the two header bytes, are XORed together to form a parity + byte. This parity byte is appended to the byte string. +3. The byte string is encoded using base58, using the same mapping as is used + for Bitcoin addresses. + +This 58-character string is presented to the user to save. Implementations may +add whitespace to the recovery key; adding a space every 4th character is +recommended. + +When reading in a recovery key, clients must disregard whitespace. Clients +must base58-decode the code, ensure that the first two bytes of the decoded +string are `0x8B` and `0x01`, ensure that XOR-ing all the bytes together +results in 0, and ensure that the total length of the decoded string +is 35 bytes. Clients must then remove the first two bytes and the last byte, +and use the resulting string as the private key to decrypt backups. + +#### Encoding the recovery key for server-side storage via MSC1946 + +If MSC1946 is used to store the key on the server, it must be stored using the +`account_data` type `m.megolm_backup.v1`. + +As a special case, if the recovery key is the same as the curve25519 key used +for storing the key, then the contents of the `m.megolm_backup.v1` +`account_data` for that key will be an object with a `passthrough` property +whose value is `true`. For example, if `m.megolm_backup.v1` is set to: + +```json +{ + "encrypted": { + "key_id": { + "passthrough": true + } + } +} +``` + +means that the recovery key for the backup is the same as the private key for +the key with ID `key_id`. (This is mostly intended to provide a migration path +for for backups that were created using an earlier draft that stored the +recovery information in the `auth_data`.) + +### API + +#### Backup versions + +##### `POST /room_keys/version` + +Create a new backup version. + +Body parameters: + +- `algorithm` (string): Required. The algorithm used for storing backups. + Currently, only `m.megolm_backup.v1.curve25519-aes-sha2` is defined. +- `auth_data` (object): Required. algorithm-dependent data. For + `m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of + this property](#auth_data-backup-versions). + +Example: + +```javascript +{ + "algorithm": "m.megolm_backup.v1.curve25519-aes-sha2", + "auth_data": { + "public_key": "abcdefg", + "signatures": { + "something": { + "ed25519:something": "hijklmnop" + } + } + } +} +``` + +On success, returns a JSON object with keys: + +- `version` (string): the backup version + +##### `GET /room_keys/version/{version}` + +Get information about the given version, or the current version if `/{version}` +is omitted. + +On success, returns a JSON object with keys: + +- `algorithm` (string): Required. Same as in the body parameters for `POST + /room_keys/version`. +- `auth_data` (object): Required. Same as in the body parameters for + `POST /room_keys/version`. +- `version` (string): Required. The backup version. +- `etag` (string): Required. The etag value which is an opaque string + representing stored keys in the backup. Clients can compare it with the + `etag` value they received in the response of their last key storage request. + If not equal, another client has pushed new keys to the backup. +- `count` (number): Required. The number of keys stored in the backup. + +Error codes: + +- `M_NOT_FOUND`: No backup version has been created. (with HTTP status code 404) + +##### `PUT /room_keys/version/{version}` + +Update information about the given version. Only `auth_data` can be updated. + +Body parameters: + +- `algorithm` (string): Required. Must be the same as in the body parameters for `GET + /room_keys/version`. +- `auth_data` (object): Required. algorithm-dependent data. For + `m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of + this property](#auth_data-backup-versions). +- `version` (string): Optional. The backup version. If present, must be the same as the path parameter. + +Example: + +```javascript +{ + "algorithm": "m.megolm_backup.v1.curve25519-aes-sha2", + "auth_data": { + "public_key": "abcdefg", + "signatures": { + "something": { + "ed25519:something": "hijklmnop" + "ed25519:anotherthing": "abcdef" + } + } + }, + "version": "42" +} +``` + +On success, returns the empty JSON object. + +Error codes: + +- `M_NOT_FOUND`: This backup version was not found. (with HTTP status code 404) + +#### Storing keys + +##### `PUT /room_keys/keys/${roomId}/${sessionId}?version=$v` + +Store the key for the given session in the given room, using the given backup +version. + +If the server already has a backup in the backup version for the given session +and room, then it will keep the "better" one. To determine which one is +"better", keys are compared: + +- first by the `is_verified` flag (`true` is better than `false`), +- then, if `is_verified` is equal, by the `first_message_index` (a lower number is better), +- and finally, is `is_verified` and `first_message_index` are equal, by + `forwarded_count` (a lower number is better). + +If neither key is better than the other (that is, if all three fields are +equal), then the server should keep the existing key. + +Body parameters: + +- `first_message_index` (integer): Required. The index of the first message + in the session that the key can decrypt. +- `forwarded_count` (integer): Required. The number of times this key has been + forwarded. +- `is_verified` (boolean): Required. Whether the device backing up the key has + verified the device that the key is from. +- `session_data` (object): Required. Algorithm-dependent data. For + `m.megolm_backup.v1.curve25519-aes-sha2`, see below for the [definition of + this property](#auth_data-backup-versions). + +On success, returns a JSON object with keys: + +- `etag` (string): Required. The new etag value representing stored keys. See + `GET /room_keys/version/{version}` for more details. +- `count` (number): Required. The new count of keys stored in the backup. + +Error codes: + +- `M_WRONG_ROOM_KEYS_VERSION`: the version specified does not match the current + backup version (with HTTP status code 403). The current backup version will + be included in the `current_version` field of the HTTP result. + +Example: + +`PUT /room_keys/keys/!room_id:example.com/sessionid?version=1` + +```javascript +{ + "first_message_index": 1, + "forwarded_count": 0, + "is_verified": true, + "session_data": { + "ephemeral": "base64+ephemeral+key", + "ciphertext": "base64+ciphertext+of+JSON+data", + "mac": "base64+mac+of+ciphertext" + } +} +``` + +Result: + +```javascript +{ + "etag": "abcdefghi", + "count": 10 +} +``` + +##### `PUT /room_keys/keys/${roomId}?version=$v` + +Store several keys for the given room, using the given backup version. + +Behaves the same way as if the keys were added individually using `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v`. + +Body parameters: +- `sessions` (object): an object where the keys are the session IDs, and the + values are objects of the same form as the body in `PUT + /room_keys/keys/${roomId}/${sessionId}?version=$v`. + +Returns the same as `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v`. + +Example: + +`PUT /room_keys/keys/!room_id:example.com?version=1` + +```javascript +{ + "sessions": { + "sessionid": { + "first_message_index": 1, + "forwarded_count": 0, + "is_verified": true, + "session_data": { + "ephemeral": "base64+ephemeral+key", + "ciphertext": "base64+ciphertext+of+JSON+data", + "mac": "base64+mac+of+ciphertext" + } + } + } +} +``` + +Result: + +```javascript +{ + "etag": "abcdefghi", + "count": 10 +} +``` + +##### `PUT /room_keys/keys?version=$v` + +Store several keys, using the given backup version. + +Behaves the same way as if the keys were added individually using `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v`. + +Body parameters: +- `rooms` (object): an object where the keys are the room IDs, and the values + are objects of the same form as the body in `PUT + /room_keys/keys/${roomId}/?version=$v`. + +Returns the same as `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v` + +Example: + +`PUT /room_keys/keys/!room_id:example.com?version=1` + +```javascript +{ + "rooms": { + "!room_id:example.com": { + "sessions": { + "sessionid": { + "first_message_index": 1, + "forwarded_count": 0, + "is_verified": true, + "session_data": { + "ephemeral": "base64+ephemeral+key", + "ciphertext": "base64+ciphertext+of+JSON+data", + "mac": "base64+mac+of+ciphertext" + } + } + } + } + } +} +``` + +Result: + +```javascript +{ + "etag": "abcdefghi", + "count": 10 +} +``` + +#### Retrieving keys + +When retrieving keys, the `version` parameter is optional, and defaults to +retrieving keys from the latest backup version. + +##### `GET /room_keys/keys/${roomId}/${sessionId}?version=$v` + +Retrieve the key for the given session in the given room from the backup. + +On success, returns a JSON object in the same form as the request body of `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v`. + +Error codes: + +- M_NOT_FOUND: The session is not present in the backup, or the requested + backup version does not exist. (with HTTP status code 404) + +##### `GET /room_keys/keys/${roomId}?version=$v` + +Retrieve the all the keys for the given room from the backup. + +On success, returns a JSON object in the same form as the request body of `PUT +/room_keys/keys/${roomId}?version=$v`. + +If the backup version exists but no keys are found, then this endpoint returns +a successful response with body: + +``` +{ + "sessions": {} +} +``` + +Error codes: + +- `M_NOT_FOUND`: The requested backup version does not exist. (with HTTP status code 404) + +##### `GET /room_keys/keys?version=$v` + +Retrieve all the keys from the backup. + +On success, returns a JSON object in the same form as the request body of `PUT +/room_keys/keys?version=$v`. + +If the backup version exists but no keys are found, then this endpoint returns +a successful response with body: + +``` +{ + "rooms": {} +} +``` + +Error codes: + +- `M_NOT_FOUND`: The requested backup version does not exist. (with HTTP status code 404) + +#### Deleting keys + +##### `DELETE /room_keys/keys/${roomId}/${sessionId}?version=$v` +##### `DELETE /room_keys/keys/${roomId}?version=$v` +##### `DELETE /room_keys/keys/?version=$v` + +Deletes keys from the backup. + +Returns the same as `PUT +/room_keys/keys/${roomId}/${sessionId}?version=$v`. + +#### `m.megolm_backup.v1.curve25519-aes-sha2` definitions + +##### `auth_data` for backup versions + +The `auth_data` property for the backup versions endpoints for +`m.megolm_backup.v1.curve25519-aes-sha2` is a [signed +json](https://matrix.org/docs/spec/appendices#signing-json) object with the +following keys: + +- `public_key` (string): the curve25519 public key used to encrypt the backups +- `signatures` (object): signatures of the `auth_data`. + +The `auth_data` should be signed by the user's [master cross-signing +key](https://github.com/matrix-org/matrix-doc/pull/1756), and may also be +signed by the user's device key. This allows clients to ensure that the public +key is valid, and prevents an attacker from being able to change the backup to +use a public key that they have the private key for. + +##### `session_data` for key backups + +The `session_data` field in the backups is constructed as follows: + +1. Encode the session key to be backed up as a JSON object with the properties: + - `algorithm` (string): `m.megolm.v1.aes-sha2` + - `sender_key` (string): base64-encoded device curve25519 key + - `sender_claimed_keys` (object): object containing the identity keys for the + sending device + - `forwarding_curve25519_key_chain` (array): zero or more curve25519 keys + for devices who forwarded the session key + - `session_key` (string): base64-encoded (unpadded) session key in + [session-sharing + format](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/megolm.md#session-sharing-format) +2. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral + key and the backup's public key to generate a shared secret. The public + half of the ephemeral key, encoded using base64, becomes the `ephemeral` + property of the `session_data`. +3. Using the shared secret, generate 80 bytes by performing an HKDF using + SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string + as the info. The first 32 bytes are used as the AES key, the next 32 bytes + are used as the MAC key, and the last 16 bytes are used as the AES + initialization vector. +4. Stringify the JSON object, and encrypt it using AES-CBC-256 with PKCS#7 + padding. This encrypted data, encoded using base64, becomes the + `ciphertext` property of the `session_data`. +5. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 + using the MAC key generated above. The first 8 bytes of the resulting MAC + are base64-encoded, and become the `mac` property of the `session_data`. + +(The key HKDF, AES, and HMAC steps are the same as what are used for encryption +in olm and megolm.) + +Security Considerations +----------------------- + +An attacker who gains access to a user's account can delete or corrupt their +key backup. This proposal does not attempt to protect against that. + +An attacker who gains access to a user's account can create a new backup +version using a key that they control. For this reason, clients SHOULD confirm +with users before sending keys to a new backup version or verify that it was +created by a trusted device by checking the signature. Alternatively, if the +signature cannot be verified, the backup can be validated by prompting the user +to enter the recovery key, and confirming that the backup's public key +corresponds to the recovery key. + +Other Issues +------------ + +Since many clients will receive encryption keys at around the same time, they +will all want to back up their copies of the keys at around the same time, +which may increase load on the server if this happens in a big room. (TODO: +how much of an issue is this?) For this reason, clients should offset their +backup requests randomly. + +Conclusion +---------- + +This proposal allows users to securely and conveniently back up and restore +their decryption keys so that users logging into a new device can decrypt old +messages. diff --git a/proposals/1756-cross-signing.md b/proposals/1756-cross-signing.md new file mode 100644 index 00000000..de08422a --- /dev/null +++ b/proposals/1756-cross-signing.md @@ -0,0 +1,565 @@ +# Cross-signing devices with device signing keys + +## Background + +If a user has multiple devices, each device will have a different key for +end-to-end encryption. Other users who want to communicate securely with this +user must then verify each key on each of their own devices. If Alice has *n* +devices, and Bob has *m* devices, then for Alice to be able to communicate with +Bob on any of their devices, this involves *n×m* key verifications. + +One way to address this is for each user to use a device signing key to sign +all of their devices. Thus another user who wishes to verify their identity +only needs to verify the device signing key and can use the signatures created +by the device signing key to verify their devices. + +[MSC1680](https://github.com/matrix-org/matrix-doc/issues/1680) presents a +different solution to the problem. A comparison between this proposal and +MSC1680 is presented below. + +## Proposal + +Each user has three key pairs: + +- a *master* cross-signing key pair that is used to identify themselves and to + sign their other cross-signing keys, +- a *self-signing* key pair that is used to sign their own devices, and +- a *user-signing* key pair that is used to sign other users' master keys. + +When one user (e.g. Alice) verifies another user's (Bob's) identity, Alice will +sign Bob's master key with her user-signing key. (This will mean that +verification methods will need to be modified to pass along the public part of +Bob's master key.) Alice's device will trust Bob's device if: + +- Alice's device is using a master key that has signed her user-signing key, +- Alice's user-signing key has signed Bob's master key, +- Bob's master key has signed Bob's self-signing key, and +- Bob's self-signing key has signed Bob's device key. + +### Key security + +A user's master key could allow an attacker to impersonate that user to other +users, or other users to that user. Thus clients must ensure that the private +part of the master key is treated securely. If clients do not have a secure +means of storing the master key (such as a secret storage system provided by +the operating system), then clients must not store the private part. If a user +changes their master key, clients of users that they communicate with must +notify their users about the change. + +A user's user-signing and self-signing keys are intended to be easily +replaceable if they are compromised by re-issuing a new key signed by the +user's master key and possibly by re-verifying devices or users. However, +doing so relies on the user being able to notice when their keys have been +compromised, and it involves extra work for the user, and so although clients +do not have to treat the private parts as sensitively as the master key, +clients should still make efforts to store the private part securely, or not +store it at all. Clients will need to balance the security of the keys with +the usability of signing users and devices when performing key verification. + +The private halves of a user's cross-signing keys may be stored encrypted on the +server so that they may be retrieved by new devices, or shared between devices +using [MSC1946](https://github.com/matrix-org/matrix-doc/pull/1946). When +handled in this way, the keys must be base64-encoded, and use the names +`m.cross_signing.master`, `m.cross_signing.self_signing`, and +`m.cross_signing.user_signing` for the master, self-signing, and user-signing +keys, respectively. + +### Signature distribution + +Currently, users will only be allowed to see +* signatures made by their own master, self-signing or user-signing keys, +* signatures made by their own devices about their own master key, +* signatures made by other users' self-signing keys about their own respective + devices, +* signatures made by other users' master keys about their respective + self-signing key, or +* signatures made by other users' devices about their respective master keys + (these signatures are used for [migrating from device + verifications](#migrating-from-device-verifications)). + +This is done in order to preserve the privacy of social connections. Future +proposals may define mechanisms for distributing signatures to other users in +order to allow for other web-of-trust use cases. + +### Migrating from device verifications + +Users who have verified individual devices may wish to migrate these +verifications to use cross-signing instead. In order to aid with this, +signatures of a user's master key, made by their own devices, may be uploaded +to the server. If another user's client sees that that a given user's master key has a valid +signature from a device that was previously verified, then the client may +choose to trust and sign the master key. The client should take precautions to +ensure that a stolen device cannot be used to cause it to trust a malicious +master key. For example, a client could prompt the user before signing the +master key, or it could only do this migration on the first master key that it +sees from a user. + +### API description + +#### Uploading signing keys + +Public keys for the cross-signing keys are uploaded to the servers using +`/keys/device_signing/upload`. This endpoint requires [UI +Auth](https://matrix.org/docs/spec/client_server/r0.4.0.html#user-interactive-authentication-api). + +`POST /keys/device_signing/upload` + +``` json +{ + "master_key": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+self+master+key", + } + }, + "self_signing_key": { + "user_id": "@alice:example.com", + "usage": ["self_signing"], + "keys": { + "ed25519:base64+self+signing+public+key": "base64+self+signing+public+key", + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + }, + "user_signing_key": { + "user_id": "@alice:example.com", + "keys": { + "ed25519:base64+device+signing+public+key": "base64+device+signing+public+key", + }, + "usage": ["user_signing"], + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + } +} +``` + +Cross-signing keys are JSON objects with the following properties: + +* `user_id` (string): The user who owns the key +* `usage` ([string]): Allowed uses for the key. Must contain `"master"` for + master keys, `"self_signing"` for self-signing keys, and `"user_signing"` + for user-signing keys. +* `keys` ({string: string}): an object that must have one entry, whose name is + "`ed25519:`" followed by the unpadded base64 encoding of the public key, and + whose value is the unpadded base64 encoding of the public key. +* `signatures` ({string: {string: string}}): signatures of the key. A + self-signing or user-signing key must be signed by the master key. A master + key may be signed by a device. + +In order to ensure that there will be no collisions in the `signatures` +property, the server must respond with an `M_FORBIDDEN` error if any of +the uploaded public keys match an existing device ID for the user. Similarly, +if a user attempts to log in specifying a device ID matching one of the signing +keys, the server must respond with an `M_FORBIDDEN` error. + +If a self-signing or user-signing key is uploaded, it must be signed by the +master key that is included in the request, or the current master key if no +master key is included. If the signature from the master key is incorrect, the +server should respond with an error code of `M_INVALID_SIGNATURE`. + +After uploading cross-signing keys, they will be included under the +`/keys/query` endpoint under the `master_keys`, `self_signing_keys` and +`user_signing_keys` properties. The `user_signing_keys` property will only be +included when a user requests their own keys. + +`POST /keys/query` + +``` json +{ + "device_keys": { + "@alice:example.com": [] + }, + "token": "string" +} +``` + +response: + +``` json +{ + "failures": {}, + "device_keys": { + "@alice:example.com": { + // ... + } + }, + "master_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+master+public+key" + } + } + }, + "self_signing_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["self_signing"], + "keys": { + "ed25519:base64+self+signing+public+key": "base64+self+signing+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + } + } +} +``` + +Similarly, the federation endpoints `POST /user/keys/query` and `GET +/user/devices/{userId}` will include the master and self-signing keys. (It +will not include the user-signing key because it is not intended to be visible +to other users.) + +`POST /user/keys/query` + +``` json +{ + "device_keys": { + "@alice:example.com": [] + } +} +``` + +response: + +``` json +{ + "device_keys": { + "@alice:example.com": { + // ... + } + }, + "master_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+master+public+key" + } + } + }, + "self_signing_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["self_signing"], + "keys": { + "ed25519:base64+self+signing+public+key": "base64+self+signing+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + } + } +} +``` + +`GET /user/devices/%40alice%3Aexample.com` + +response: + +``` json +{ + "user_id": "@alice:example.com", + "stream_id": 5, + "devices": [ + // ... + ], + "master_key": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+master+public+key" + } + }, + "self_signing_key": { + "user_id": "@alice:example.com", + "usage": ["self_signing"], + "keys": { + "ed25519:base64+self+signing+public+key": "base64+self+signing+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + } +} +``` + +In addition, Alice's homeserver will send a `m.signing_key_update` EDU to +servers that have users who share encrypted rooms with Alice. The `content` of +that EDU has the following properties: + +* `user_id` (string): Required. The user ID who owns the signing key +* `master_key` (object): The master key, as above. +* `self_signing_key` (object): The self-signing key, as above. + +After uploading self-signing and user-signing keys, the user will show up in +the `changed` property of the `device_lists` field of the sync result of any +others users who share an encrypted room with that user. + +#### Uploading signatures + +Signatures of device keys can be uploaded using `/keys/signatures/upload`. + +For example, Alice signs one of her devices (HIJKLMN) (using her self-signing +key), her own master key (using her HIJKLMN device), Bob's master key (using +her user-signing key). + +`POST /keys/signatures/upload` + +``` json +{ + "@alice:example.com": { + "HIJKLMN": { + "user_id": "@alice:example.com", + "device_id": "HIJKLMN", + "algorithms": [ + "m.olm.curve25519-aes-sha256", + "m.megolm.v1.aes-sha" + ], + "keys": { + "curve25519:HIJKLMN": "base64+curve25519+key", + "ed25519:HIJKLMN": "base64+ed25519+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+self+signing+public+key": "base64+signature+of+HIJKLMN" + } + } + }, + "base64+master+public+key": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+master+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:HIJKLMN": "base64+signature+of+master+key" + } + } + } + }, + "@bob:example.com": { + "bobs+base64+self+signing+public+key": { + "user_id": "@bob:example.com", + "keys": { + "ed25519:bobs+base64+master+public+key": "bobs+base64+master+public+key" + }, + "usage": ["master"], + "signatures": { + "@alice:example.com": { + "ed25519:base64+user+signing+public+key": "base64+signature+of+bobs+master+key" + } + } + } + } +} +``` + +response: + +``` json +{ + "failures": {} +} +``` + +The response contains a `failures` property, which is a map of user ID to +device ID to failure reason, if any of the uploaded keys failed. The +homeserver should verify that the signatures on the uploaded keys are valid. +If a signature is not valid, the homeserver should set the corresponding entry +in `failures` to a JSON object with the `errcode` property set to +`M_INVALID_SIGNATURE`. + +After Alice uploads a signature for her own devices or master key, her +signature will be included in the results of the `/keys/query` request when +*anyone* requests her keys. However, signatures made for other users' keys, +made by her user-signing key, will not be included. + +`POST /keys/query` + +``` json +{ + "device_keys": { + "@alice:example.com": [] + }, + "token": "string" +} +``` + +response: + +``` json +{ + "failures": {}, + "device_keys": { + "@alice:example.com": { + "HIJKLMN": { + "user_id": "@alice:example.com", + "device_id": "HIJKLMN", + "algorithms": [ + "m.olm.v1.curve25519-aes-sha256", + "m.megolm.v1.aes-sha" + ], + "keys": { + "curve25519:HIJKLMN": "base64+curve25519+key", + "ed25519:HIJKLMN": "base64+ed25519+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:HIJKLMN": "base64+self+signature", + "ed25519:base64+self+signing+public+key": "base64+signature+of+HIJKLMN" + } + }, + "unsigned": { + "device_display_name": "Alice's Osborne 2" + } + } + } + }, + "master_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["master"], + "keys": { + "ed25519:base64+master+public+key": "base64+master+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:HIJKLMN": "base64+signature+of+master+key" + } + } + } + }, + "self_signing_keys": { + "@alice:example.com": { + "user_id": "@alice:example.com", + "usage": ["self_signing"], + "keys": { + "ed25519:base64+self+signing+public+key": "base64+self+signing+public+key" + }, + "signatures": { + "@alice:example.com": { + "ed25519:base64+master+public+key": "base64+signature" + } + } + } + } +} +``` + +Similarly, the federation endpoints `POST /user/keys/query` and `GET +/user/devices/{userId}` will include the new signatures for her own devices or +master key, but not signatures made by her user-signing key. + +In addition, when Alice uploads signatures for her own device, Alice's server +will send an `m.device_list_update` EDU to servers that have users who share +encrypted rooms with Alice, updating her device to include her new signature. +And when a signature of a master key is uploaded, Alice's server will send an +`m.signing_key_update` EDU, updating her master key to include her new +signature. + +After Alice uploads a signature for Bob's user-signing key, her signature will +be included in the results of the `/keys/query` request when Alice requests +Bob's key, but will not be included when anyone else requests Bob's key: + +`GET /keys/query` + +``` json +{ + "failures": {}, + "device_keys": { + "@bob:example.com": { + // ... + } + }, + "master_keys": { + "@bob:example.com": { + "user_id": "@bob:example.com", + "keys": { + "ed25519:bobs+base64+master+public+key": "bobs+base64+master+public+key" + }, + "usage": ["master"], + "signatures": { + "@alice:example.com": { + "ed25519:base64+user+signing+public+key": "base64+signature+of+bobs+master+key" + } + } + } + } +} +``` + +## Comparison with MSC1680 + +MSC1680 suffers from the fact that the attestation graph may be arbitrarily +complex and may become ambiguous how the graph should be interpreted. In +particular, it is not obvious exactly how revocations should be interpreted -- +should they be interpreted as only revoking the signature created previously by +the device making the revocation, or should it be interpreted as a statement +that the device should not be trusted at all? As well, a revocation may split +the attestation graph, causing devices that were previously trusted to possibly +become untrusted. Logging out a device may also split the attestation graph. +Moreover, it may not be clear to a user what device verifications would be +needed to reattach the parts of the graph. + +One way to solve this is by registering a "virtual device", which is used to +sign other devices. This solution would be similar to this proposal. However, +real devices would still form an integral part of the attestation graph. For +example, if Alice's Osborne 2 verifies Bob's Dynabook, the attestation graph might +look like: + +![](images/1756-graph1.dot.png) + +If Bob replaces his Dynabook without re-verifying with Alice, this will split +the graph and Alice will not be able to verify Bob's other devices. In +contrast, in this proposal, Alice and Bob sign each other's self-signing key +with their user-signing keys, and the attestation graph would look like: + +![](images/1756-graph2.dot.png) + +In this case, Bob's Dynabook can be replaced without breaking the graph. + +With normal cross-signing, it is not clear how to recover from a stolen device. +For example, if Mallory steals one of Alice's devices and revokes Alice's other +devices, it is unclear how Alice can rebuild the attestation graph with her +devices, as there may be stale attestations and revocations lingering around. +(This also relates to the question of whether a revocation should only revoke +the signature created previously by the device making the attestation, or +whether it should be a statement that the device should not be trusted at all.) +In contrast, with this proposal, if a device is stolen, then only the +user-signing key must be re-issued. + +## Security considerations + +This proposal relies on servers to communicate when cross-signing keys are +deleted and replaced. An attacker who is able to both steal a user's device +and control their homeserver could prevent that device from being marked as +untrusted. + +An attacker may be able to upload a large number of signatures in a DoS attack +against clients or servers, similar to the [attack against the SKS keyserver +network](https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f). +Since clients are only sent a subset of signatures, and the attestation graph +is limited, a DoS attack is less likely to be successful in this case. + +## Conclusion + +This proposal presents an alternative cross-signing mechanism to MSC1680, +allowing users to trust another user's devices without needing to verify each +one individually. diff --git a/proposals/1802-standardised-federation-response-format.md b/proposals/1802-standardised-federation-response-format.md new file mode 100644 index 00000000..6ddaa080 --- /dev/null +++ b/proposals/1802-standardised-federation-response-format.md @@ -0,0 +1,50 @@ +# Remove the '200' value from some federation responses + +Some responses formats in the federation API specifications use the form `[200, +res]` in which `res` is the JSON object containing the actual response for the +affected endpoints. This was due to a mishap while building synapse's federation +features, and has been left as is because fixing it would induce backward +incompatibility. + +This proposal proposes a backward compatible alternative + +## Proposal + +Add a new version of the following endpoints under the prefix +`/_matrix/federation/v2`: + +* `PUT /_matrix/federation/v2/send_join/{roomId}/{eventId}` +* `PUT /_matrix/federation/v2/send_leave/{roomId}/{eventId}` + +Which are the exact same endpoints as their equivalents under the `v1` prefix, +except for the response format, which changes from: + +``` +[ + 200, + res +] +``` + +To: + +``` +res +``` + +Where `res` is the JSON object containing the response to a request directed at +one of the affected endpoints. + +This proposal doesn't address the `PUT +/_matrix/federation/v1/invite/{roomId}/{eventId}` endpoint since +[MSC1794](https://github.com/matrix-org/matrix-doc/pull/1794) already takes care +of it. + +If a call to any of the `v2` endpoints described in this proposal results in an +unrecognised request exception (i.e. in a response with a 400 or a 404 status +code), then the sending server should retry the request with the `v1` API. + +## Alternative solutions + +An alternative solution would be to make the change in the `v1` federation API, +but would break backward compatibility, thus would be harder to manage. diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md new file mode 100644 index 00000000..3f386d5e --- /dev/null +++ b/proposals/1946-secure_server-side_storage.md @@ -0,0 +1,287 @@ +# Secure Secret Storage and Sharing + +Some features may require clients to store encrypted data on the server so that +it can be shared securely between clients. Clients may also wish to securely +send such data directly to each other. For example, key backups +([MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219)) can store the +decryption key for the backups on the server, or cross-signing +([MSC1756](https://github.com/matrix-org/matrix-doc/pull/1756)) can store the +signing keys. This proposal presents a standardized way of storing such data. + +## Proposal + +Secrets are data that clients need to use and that are sent through or stored +on the server, but should not be visible to server operators. Secrets are +plain strings -- if clients need to use more complicated data, they must be +encoded as a string, such as by encoding as JSON. + +### Storage + +If secret data is stored on the server, it must be encrypted in order to +prevent homeserver administrators from being able to read it. A user can have +multiple keys used for encrypting data. This allows the user to selectively +decrypt data on clients. For example, the user could have one key that can +decrypt everything, and another key that can only decrypt their user-signing +key for cross-signing. + +Key descriptions and secret data are both stored in the user's account_data. + +#### Key storage + +Each key has an ID, and the description of the key is stored in the user's +account_data using the event type `m.secret_storage.key.[key ID]`. The contents +of the account data for the key will include an `algorithm` property, which +indicates the encryption algorithm used, as well as a `name` property, which is +a human-readable name. The contents will be signed as signed JSON using the +user's master cross-signing key. Other properties depend on the encryption +algorithm, and are described below. + +Example: + +A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` + +```json +{ + "name": "Some key", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` + +A key can be marked as the "default" key by setting the user's account_data +with event type `m.secret_storage.default_key` to an object that has the ID of +the key as its `key` property. The default key will be used to encrypt all +secrets that the user would expect to be available on all their clients. +Unless the user specifies otherwise, clients will try to use the default key to +decrypt secrets. + +Clients MUST ensure that the key is trusted before using it to encrypt secrets. +One way to do that is to have the client that creates the key sign the key +description (as signed JSON) using the user's master cross-signing key. +Another way to do that is to prompt the user to enter the passphrase used to +generate the encryption key and ensure that the generated private key +corresponds to the public key. + +#### Secret storage + +Encrypted data is stored in the user's account_data using the event type +defined by the feature that uses the data. For example, decryption keys for +key backups could be stored under the type `m.megolm_backup.v1`, +or the self-signing key for cross-signing could be stored under the type +`m.cross_signing.self_signing`. + +The account_data will have an `encrypted` property that is a map from key ID +to an object. The algorithm from the `m.secret_storage.key.[key ID]` data for +the given key defines how the other properties are interpreted, though it's +expected that most encryption schemes would have `ciphertext` and `mac` +properties, where the `ciphertext` property is the unpadded base64-encoded +ciphertext, and the `mac` is used to ensure the integrity of the data. + +Example: + +Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: + +`org.example.some.secret`: + +```json +{ + "encrypted": { + "key_id_1": { + "ciphertext": "base64+encoded+encrypted+data", + "mac": "base64+encoded+mac", + // ... other properties according to algorithm property in + // m.secret_storage.key.key_id_1 + }, + "key_id_2": { + // ... + } + } +} +``` + +and the key descriptions for the keys would be: + +`m.secret_storage.key.key_id_1`: + +```json +{ + "name": "Some key", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` + +`m.secret_storage.key.key_id_2`: + +```json +{ + "name": "Some other key", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` + +#### Encryption algorithms + +##### `m.secret_storage.v1.curve25519-aes-sha2` + +The public key is stored in the `pubkey` property of the `m.secret_storage.key.[key +ID]` account_data as a base64-encoded string. + +The data is encrypted and MACed as follows: + +1. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral + key and the public key to generate a shared secret. The public half of the + ephemeral key, encoded using base64, becomes the `ephemeral` property. +2. Using the shared secret, generate 80 bytes by performing an HKDF using + SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string + as the info. The first 32 bytes are used as the AES key, the next 32 bytes + are used as the MAC key, and the last 16 bytes are used as the AES + initialization vector. +4. Encrypt the data using AES-CBC-256 with PKCS#7 padding. This encrypted + data, encoded using base64, becomes the `ciphertext` property. +5. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 + using the MAC key generated above. The first 8 bytes of the resulting MAC + are base64-encoded, and become the `mac` property. + +(The key HKDF, AES, and HMAC steps are the same as what are used for encryption +in olm and megolm.) + +For example, the `m.secret_storage.key.[key ID]` for a key using this algorithm +could look like: + +```json +{ + "name": "m.default", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + "pubkey": "base64+public+key" +} +``` + +and data encrypted using this algorithm could look like this: + +```json +{ + "encrypted": { + "key_id": { + "ciphertext": "base64+encoded+encrypted+data", + "ephemeral": "base64+ephemeral+key", + "mac": "base64+encoded+mac" + } + } +} +``` + +###### Keys + +When a user is given a raw key for `m.secret_storage.v1.curve25519-aes-sha2`, +it will be encoded as follows (this is the same as what is proposed in MSC1703): + +* prepend the two bytes 0x8b and 0x01 to the key +* compute a parity byte by XORing all bytes of the resulting string, and append + the parity byte to the string +* base58-encode the resulting byte string with the alphabet + '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'. +* format the resulting ASCII string into groups of 4 characters separated by + spaces. + +When decoding a raw key, the process should be reversed, with the exception +that whitespace is insignificant in the user's ASCII input. + +###### Passphrase + +A user may wish to use a chosen passphrase rather than a randomly generated +key. In this case, information on how to generate the key from a passphrase +will be stored in the `passphrase` property of the `m.secret_storage.key.[key +ID]` account-data: + +```json +{ + "passphrase": { + "algorithm": "m.pbkdf2", + "salt": "MmMsAlty", + "iterations": 100000 + }, + ... +} +``` + +**`m.pbkdf2`** + +The key is generated using PBKDF2 using the salt given in the `salt` parameter, +and the number of iterations given in the `iterations` parameter. + +### Sharing + +Rather than (or in addition to) storing secrets on the server encrypted by a +shared key, devices can send secrets to each other, encrypted using olm. + +To request a secret, a client sends a `m.secret.request` device event with `action` +set to `request` to other devices, and `name` set to the name of the secret +that it wishes to retrieve. A device that wishes to share the secret will +reply with a `m.secret.send` event, encrypted using olm. When the original +client obtains the secret, it sends a `m.secret.request` event with `action` +set to `request_cancellation` to all devices other than the one that it received the +secret from. Clients should ignore `m.secret.send` events received from +devices that it did not send an `m.secret.request` event to. + +Clients MUST ensure that they only share secrets with other devices that are +allowed to see them. For example, clients SHOULD only share secrets with +the user’s own devices that are verified and MAY prompt the user to confirm sharing the +secret. + +If a feature allows secrets to be stored or shared, then for consistency it +SHOULD use the same name for both the account_data event type and the `name` in +the `m.secret.request`. + +#### Event definitions + +##### `m.secret.request` + +Sent by a client to request a secret from another device. It is sent as an +unencrypted to-device event. + +- `name`: (string) Required if `action` is `request`. The name of the secret + that is being requested. +- `action`: (enum) Required. One of ["request", "request_cancellation"]. +- `requesting_device_id`: (string) Required. ID of the device requesting the + secret. +- `request_id`: (string) Required. A random string uniquely identifying the + request for a secret. If the secret is requested multiple times, it should be + reused. It should also reused in order to cancel a request. + +##### `m.secret.send` + +Sent by a client to share a secret with another device, in response to an +`m.secret.request` event. It MUST be encrypted as an `m.room.encrypted` event, +then sent as a to-device event. + +- `request_id`: (string) Required. The ID of the request that this a response to. +- `secret`: (string) Required. The contents of the secret. + +## Tradeoffs + +Currently, only a public/private key mechanism is defined. It may be useful to +also define a secret key mechanism. + +## Potential issues + +Keeping all the data and keys in account data means that it may clutter up +`/sync` requests. However, clients can filter out the data that they are not interested +in. One possibility for addressing this would be to add a flag to the account +data to indicate whether it should come down the `/sync` or not. + +## Security considerations + +By storing information encrypted on the server, this allows the server operator +to read the information if they manage to get hold of the decryption keys. +In particular, if the key is based on a passphrase and the passphrase can be +guessed, then the secrets could be compromised. In order to help protect the +secrets, clients should provide feedback to the user when their chosen +passphrase is considered weak, and may also wish to prevent the user from +reusing their login password. + +## Conclusion + +This proposal presents a common way for bits of encrypted data to be stored on +a user's homeserver for use by various features. diff --git a/proposals/1957-integrations-discovery.md b/proposals/1957-integrations-discovery.md new file mode 100644 index 00000000..85939726 --- /dev/null +++ b/proposals/1957-integrations-discovery.md @@ -0,0 +1,189 @@ +# MSC1957: Integration manager discovery + +**Note**: this proposal is part of a larger "Integrations API" which has not yet been defined. +See [MSC1956](https://github.com/matrix-org/matrix-doc/pull/1956) for details. + +**Note**: this proposal makes use of the existing Widget API proposed by +[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236). + +Users should have the freedom to choose which integration manager they want to use in their client, while +also accepting suggestions from their homeserver and client. Clients need to know where to find the different +integration managers and how to contact them. + + +## Proposal + +A single logged in user may be influenced by zero or more integration managers at any given time. Managers +are sourced from the client's own configuration, homeserver discovery information, and the user's personal +account data in the form of widgets. Clients should support users using more than one integration manager +at a given time, although the rules for how this can be handled are defined later in this proposal. + +#### Client-configured integration managers + +This is left as an implementation detail. In the case of Riot, this is likely to be part of the existing +`config.json` options, although likely modified to support multiple managers instead of one. + +#### Homeserver-configured integration managers + +The integration managers suggested by a homeserver are done through the existing +[.well-known](https://matrix.org/docs/spec/client_server/r0.4.0.html#get-well-known-matrix-client) discovery +mechanism. The added optional fields, which should not affect a client's ability to log a user in, are: +```json +{ + "m.integrations": { + "managers": [ + { + "api_url": "https://integrations.example.org", + "ui_url": "https://integrations.example.org/ui" + }, + { + "api_url": "https://bots.example.org" + } + ] + } +} +``` + +As shown, the homeserver is able to suggest multiple integration managers through this method. Each manager +must have an `api_url` which must be an `http` or `https` URL. The `ui_url` is optional and if not provided +is the same as the `api_url`. Like the `api_url`, the `ui_url` must be `http` or `https` if supplied. + +The `ui_url` is ultimately treated the same as a widget, except that the `data` object from the widget is not +present and must not be templated here. Variables like `$matrix_display_name` are able to function, however. +Integration managers should never use the `$matrix_user_id` as authoritative and instead seek other ways to +determine the user ID. This is covered by other proposals. + +The `api_url` is the URL clients will use when *not* embedding the integration manager, and instead showing +its own purpose-built interface. + +Clients should query the `.well-known` information for the homeserver periodically to update the integration +manager settings for that homeserver. The client is not expected to validate or use any other information +contained in the response. Current recommendations are to query the configuration when the client starts up +and every 8 hours after that. Clients can additionally refresh the configuration whenever they feel is +necessary (such as every time the user opens the integration manager). + +#### User-configured integration managers + +Users can specify integration managers in the form of account widgets. The `type` is to be `m.integration_manager` +and the content would look something similar to: +```json +{ + "url": "https://integrations.example.org/ui?displayName=$matrix_display_name", + "data": { + "api_url": "https://integrations.example.org" + } +} +``` + +The `api_url` in the `data` object is required and has the same meaning as the homeserver-defined `api_url`. +The `url` of the widget is analogous to the `ui_url` from the homeserver configuration above, however normal +widget rules apply here. + +The user is able to have multiple integration managers through use of multiple widgets. + +The query string shown in the example is to demonstrate that integration managers are widgets and can +make use of the template options provided to widgets. + +#### Display order of integration managers + +Clients which have support for integration managers should display at least 1 manager, but should +display multiple via something like tabs. Clients must prefer to display the user's configured +integration managers over any defaults, and if only displaying one manager must pick the first +manager after sorting the `state_key`s of the applicable widgets in lexicographical order. Clients +can additionally display default managers if they so wish, and should preserve the order defined in +the various defaults. If the user has no configured integration managers, the client must prefer +to display one or more of the managers suggested by the homeserver over the managers recommended +by the client. + +The client can optionally support a way to entirely disable integration manager support, even if the +user and homeserver have managers defined. + +The rationale for having the client prefer to use the user's integration managers first is so that +the user can tailor their experience within Matrix if desired. Similarly, a homeserver may wish to +subject all of their users to the same common integration manager as would be common in some organizations. +The client's own preference is a last ditch effort to have an integration manager available to the +user so they don't get left out. + +#### Displaying integration managers + +Clients simply open the `ui_url` (or equivalent) in an `iframe` or similar. In the current ecosystem, +integration managers would receive a `scalar_token` to identify the user - this is no longer the case +and instead integration managers must seek other avenues for determining the user ID. Other proposals +cover how to do this in the context of the integrations API. + +Integration managers shown in this way must be treated like widgets, regardless of source. In practice +this means exposing the Widget API to the manager and applying necessary scoping to keep the manager +as an account widget rather than a room widget. + +#### Discovering a manager by only the domain name + +Clients may wish to ask users for a single canonical domain name so they can find the manager to add +to the user's account transparently. This differs from the .well-known discovery which allows homeservers +to recommend their own integration manager: the homeserver is not recommending a default here. The +user has instead opted to pick an integration manager (identified only by domain name) and the client +is expected to resolve that to a set of URLs it can use for the manager. + +Similar to the .well-known discovery done by servers (and clients during login), clients which have an +integrations domain (eg: "example.org") make a regular HTTPS request to +`https://example.org/.well-known/matrix/integrations` which returns an object which looks like the +following: +```json +{ + "m.integrations_widget": { + "url": "https://integrations.example.org/ui?displayName=$matrix_display_name", + "data": { + "api_url": "https://integrations.example.org" + } + } +} +``` + +The response should be parsed as JSON. If the endpoint returns an error or is missing the `m.integrations_widget` +property, the client should assume there is no integrations manager running on that domain. The +`m.integrations_widget` is an object which has the exact same format as the account widget for +an integration manager, described above. The client should wrap the object verbatim into the appropriate +account data location. + +Because the .well-known file would be accessed by web browsers, among other platforms, the server +should be using appropriate CORS headers for the request. The recommended headers are the same as those +which are already recommended for homeserver discovery in the Client-Server API. + +*Note*: this could reuse the client-server mechanic for discovery and just omit the homeserver information +however that conflates many concerns together on the one endpoint. A new endpoint is instead proposed +to keep the concerns isolated. + +The query string shown in the example is to demonstrate that integration managers are widgets and can +make use of the template options provided to widgets. + +## Tradeoffs + +We could limit the user (and by extension, the homeserver and client) to exactly 1 integration manager +and not worry about tabs or other concepts, however this restricts a user's access to integrations. +In a scenario where the user wants to use widgets from Service A and bots from Service B, they'd +end up switching accounts or clients to gain access to either service, or potentially just give up +and walk away from the problem. Instead of having the user switch between clients, we might as well +support this use case, even if it is moderately rare. + +We could also define the integration managers in a custom account data event rather than defining them +as a widget. Doing so just adds clutter to the account data and risks duplicating code in clients as +using widgets gets us URL templating for free (see the section earlier on in this proposal about account +widgets for more information: "User-configured integration managers"). + + +## Future extensions + +Some things which may be desirable in the future are: +* Avatars for the different managers +* Human-readable names for the different managers +* Supporting `ui_url`s targeting specific clients for a more consistent design + + +## Security considerations + +When displaying integration managers, clients should not trust that the input is sanitary. Per the +proposal above, an integration manager is only permitted to be served from HTTP(S) URIs. A given +integration manager can still have malicious intent however, and clients should ensure any sandboxing +on the manager is appropriate such that it can communicate with the client, but cannot perform +unauthorized actions. Other URI schemes are just as dangerous and could potentially be allowed by +this proposal - use cases are less defined and desirable for schemes like `file://` and are excluded +by this proposal. They can be added in a future proposal if a use case arises. diff --git a/proposals/1961-integrations-auth.md b/proposals/1961-integrations-auth.md new file mode 100644 index 00000000..1f868e4c --- /dev/null +++ b/proposals/1961-integrations-auth.md @@ -0,0 +1,73 @@ +# MSC1961: Integration manager authentication + +A set of common APIs needs to be defined for clients to be able to interact with an integration +manager. This proposal covers the authentication portion of that API. + +**Note**: this proposal is part of a larger "Integrations API" which has not yet been defined. +See [MSC1956](https://github.com/matrix-org/matrix-doc/pull/1956) for details. + + +## Proposal + +All specified APIs (except `/register`) will take an `Authorization` header with a `Bearer` token returned +from a call to `/register`. This token is used to authorize the request and to identify who is making the +request. The token may also be specified as the `access_token` query string parameter, similar to the +Client-Server API. + +#### POST `/_matrix/integrations/v1/account/register` + +Exchanges an OpenID object for a token which can be used to authorize future requests to the manager. + +Request body is an OpenID object as returned by `/_matrix/client/r0/user/:userId/openid/request_token`. + +Response is: +```json +{ + "token": "OpaqueString" +} +``` + +The token should consist of URL-safe base64 characters. Integration managers should be careful to validate +the OpenID object by ensuring the `/_matrix/federation/v1/openid/userinfo` response has a `sub` which belongs +to the `matrix_server_name` provided in the original OpenID object. + +Applications which register for a token are responsible for tracking which integration manager they are for. +This can usually be done by tracking the hostname of the integration manager and matching a token with it. + +#### GET `/_matrix/integrations/v1/account` + +Gets information about the token's owner, such as the user ID for which it belongs. + +Besides a token, no other information is required for the request. + +Response is: +```json +{ + "user_id": "@alice:example.org" +} +``` + +The `user_id` is the user ID which was represented in the OpenID object provided to `/register`. Integration +managers may be interested in also supplying information about the user's credit balance for paid integrations +here. Preferably, custom information is stored under a namespaced key like so: +```json +{ + "user_id": "@alice:example.org", + "org.example.paid.integrations": { + "credit": 20000 + } +} +``` + +#### POST `/_matrix/integrations/v1/account/logout` + +Logs the token out, rendering it useless for future requests. + +Request body is an empty object. Response body is also an empty object if successful. + + +## Security considerations + +Clients should be sure to call `/logout` where possible when the user is logging out or no longer needs access +to a given manager. Clients should additionally be cautious about which managers they register for tokens with, +as some integration managers may be untrusted. diff --git a/proposals/1983-leave-reasons.md b/proposals/1983-leave-reasons.md new file mode 100644 index 00000000..8e0710b8 --- /dev/null +++ b/proposals/1983-leave-reasons.md @@ -0,0 +1,39 @@ +# Proposal to add reasons for leaving a room + +This proposal intends to solve [#1295](https://github.com/matrix-org/matrix-doc/issues/1295). Currently +people can be kicked from a room for a human-readable reason, but when voluntarily leaving the `reason` +is not considered. + +## Proposal + +Like when kicking someone, `m.room.member` events with `membership: leave` can have a string `reason`. +The reason is a human-readable string of text which clients should display to users of the room, providing +users with the ability to leave a room for a reason of their choosing. The field is optional. + +Having such a field gives Matrix better compatibility with other networks such as IRC which use the leave +reason to communicate connection problems ("ping timeout", etc). Although Matrix doesn't have a need for a +persistent connection to remain in the room, having a reason can improve the bridge experience. + +An example of a partial leave event with reason would be: +```json +{ + "type": "m.room.member", + "sender": "@travis:t2l.io", + "state_key": "@travis:t2l.io", + "content": { + "membership": "leave", + "reason": "I'm in too many rooms and never talk here" + } +} +``` + + +## Potential issues + +There is a chance that quality of communication may deteriorate as a result of adding this field. Arguments +in rooms may result in the leave reason having personal attacks against other room members or rude remarks. +Moderators would be expected to handle these situations for these cases, including redacting the state event +if needed. + +This also opens up an avenue for spam by having a very large reason for leaving. Clients are encouraged to +trim/hide the reason at a reasonable length. diff --git a/proposals/2140-terms-of-service-2.md b/proposals/2140-terms-of-service-2.md new file mode 100644 index 00000000..6bff8ebb --- /dev/null +++ b/proposals/2140-terms-of-service-2.md @@ -0,0 +1,348 @@ +# MSC2140: Terms of Service API for Identity Servers and Integration Managers + +*Note*: This MSC was added to in [MSC2264](https://github.com/matrix-org/matrix-doc/pull/2264) + +[MSC1692](https://github.com/matrix-org/matrix-doc/issues/1692) introduces a +method for homeservers to require that users read and agree to certain +documents before being permitted to use the service. This proposal introduces a +corresponding method that can be used with Identity Servers and Integration +Managers. + +Requirements for this proposal are: + * ISes and IMs should be able to give multiple documents a user must agree to + abide by + * Each document shoud be versioned + * ISes and IMs must, for each request that they handle, know that the user + making the request has agreed to their data being used. This need not be + absolute proof (we will always have to trust that the client actually + showed the document to the user) but it must be reasonably demonstrable that + the user has given informed consent for the client to use that service. + * ISes and IMs must be able to prevent users from using the service if they + have not provided agreement. + * A user should only have to agree to each version of each document once for + their Matrix ID, ie. having agreed to a set of terms in one client, they + should not have to agree to them again when using a different client. + * Documents should be de-duplicated between services. If two or more services + are hosted by the same organisation, the organisation should have the + option to give their users a single document that encompasses both services + (bearing in mind that the user must be able to opt-out of components of a + service whilst still being able to use the service without that component). + +Identity Servers do not currently require any kind of user login to access the +service and so are unable to track what users have agreed to what terms in the +way that Homeservers do. + +## Proposal + +Throuhgout this proposal, $prefix will be used to refer to the prefix of the +API in question, ie. `/_matrix/identity/v2` for the IS API and +`/_matrix/integrations/v1` for the IM API. + +Note the removal of the `/api` prefix and migration to v2 in the IS API +following convention from +[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134). + +This proposal introduces: + * A v2 API prefix, with authentication, for the Identity Service + * The `$prefix/terms` endpoint + * The `m.accepted_terms` section in account data + * `POST /_matrix/client/r0/account/3pid/unbind` endpoints on the client/server + API + +This proposal removes: + * The `bind_email` and `bind_msisdn` on the Homeserver `/register` endpoint + +This proposal relies on both Integration Managers and Identity Servers being +able to identify users by their MXID and store the fact that a given MXID has +indicated that they accept the terms given. Integration Managers already +identify users in this way by authenticating them using the OpenID endpoint on +the Homeserver. This proposal introduces the same mechanism to Identity Servers +and adds authentication across the Identity Service API. + +### IS API Authentication + +All current endpoints within `/_matrix/identity/api/v1/` will be duplicated +into `/_matrix/identity/v2`, noting that MSC2134 changes the behaviour of +lookups. Authentication is still expected on MSC2134's proposed endpoints. +Support for `application/x-form-www-urlencoded` parameters in requests will +be dropped from all endpoints. + +Any request to any endpoint within `/_matrix/identity/v2`, with the exception +of: + * `/_matrix/identity/v2` + * `/_matrix/identity/v2/pubkey/*` + * The new `$prefix/account/register` endpoint + * The new `GET /_matrix/identity/v2/terms` + * `$prefix/account/logout` + +...may return an error with `M_UNAUTHORIZED` errcode with HTTP status code 401. +This indicates that the user must authenticate with OpenID and supply a valid +`access_token`. + +Clients authenticate either via an `Authorization` header with a `Bearer` token +or an `access_token` query parameter. + +The existing endpoints under `/_matrix/identity/api/v1/` continue to be +unauthenticated but will be deprecated. ISes may support the old v1 API for as +long as they wish. Once ISes remove support for the old APIs, those endpoints +must return HTTP Status 404. Clients must update to use the v2 API as soon as +possible. + +OpenID authentication in the IS API will work the same as in the Integration Manager +API, as specified in [MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961). + +When clients supply an identity server to the Homeserver in order for the +Homeserver to make calls to the IS on its behalf, it must also supply its +access token for the Identity Server alongside in the `id_access_token` key of +the same JSON object. That is, in the main request object for `requestToken` +and `/_matrix/client/r0/rooms/{roomId}/invite` requests and in the +`threepidCreds` object when supplying 3PID credentials (eg. in the +`m.email.identity` UI auth stage). The server must also relay +`M_TERMS_NOT_SIGNED` errors back to the client. Exceptions to this are any +requests where the only IS operation the Homeserver may perform is unbinding, +ie. `/_matrix/client/r0/account/deactivate` and +`/_matrix/client/r0/account/3pid/delete`, in which case the unbind will be +authenticated by a signed request from the Homeserver. + +### HS Register API + +The `bind_email` and `bind_msisdn` options to `/_matrix/client/r0/register` in +the client/server API will be removed. Due to the fact that +`/_matrix/identity/v2/3pid/bind` requires authentication, it will no longer be +possible for the Homeserver to bind 3PIDs as part of the registration process. + +### IS Register API + +The following new APIs will be introduced to support OpenID auth as per +[MSC1961](https://github.com/matrix-org/matrix-doc/issues/1961): + + * `/_matrix/identity/v2/account/register` + * `/_matrix/identity/v2/account` + * `/_matrix/identity/v2/account/logout` + +Note again the removal of the `/api` prefix and migration to v2 following +convention from +[MSC2134](https://github.com/matrix-org/matrix-doc/issues/2134). + +### Terms API + +New API endpoints will be introduced: + +#### `GET $prefix/terms`: +This returns a set of documents that the user must agree to abide by in order +to use the service. Its response is similar to the structure used in the +`m.terms` UI auth flow of the Client/Server API: + +```json +{ + "policies": { + "terms_of_service": { + "version": "2.0", + "en": { + "name": "Terms of Service", + "url": "https://example.org/somewhere/terms-2.0-en.html" + }, + "fr": { + "name": "Conditions d'utilisation", + "url": "https://example.org/somewhere/terms-2.0-fr.html" + } + }, + "privacy_policy": { + "version": "1.2", + "en": { + "name": "Privacy Policy", + "url": "https://example.org/somewhere/privacy-1.2-en.html" + }, + "fr": { + "name": "Politique de confidentialité", + "url": "https://example.org/somewhere/privacy-1.2-fr.html" + } + } + } +} +``` + +Each document (ie. key/value pair in the 'policies' object) MUST be +uniquely identified by its URL. It is therefore strongly recommended +that the URL contains the version number of the document. The name +and version keys, however, are used only to provide a human-readable +description of the document to the user. + +This endpoint does *not* require authentication. + +#### `POST $prefix/terms`: +Requests to this endpoint have a single key, `user_accepts` whose value is +a list of URLs (given by the `url` field in the GET response) of documents that +the user has agreed to: + +```json +{ + "user_accepts": ["https://example.org/somewhere/terms-2.0-en.html"] +} +``` + +This endpoint requires authentication. + +The clients MUST include the correct URL for the language of the document that +was presented to the user and they agreed to. Servers should accept agreement +of any one language of each document as sufficient, regardless of what language +a client is operating in: users should not have to re-consent to documents if +they change their client to a different language. + +The server responds with an empty JSON object. The server must not assume that +the client will agree to all documents in a single request. + +### Accepted Terms Account Data + +This proposal also defines the `m.accepted_terms` section in User Account +Data in the client/server API that clients SHOULD use to track what sets of +terms the user has consented to. This has an array of URLs under the 'accepted' +key to which the user has agreed to. + +An `m.accepted_terms` section therefore resembles the following: + +```json +{ + "accepted": [ + "https://example.org/somewhere/terms-1.2-en.html", + "https://example.org/somewhere/privacy-1.2-en.html" + ] +} +``` + +Whenever a client submits a `POST $prefix/terms` request to an IS or IM or +completes an `m.terms` flow on the HS (or as soon as possible afterwards, ie. +after registration is complete), it SHOULD update this account data section +adding any the URLs of any additional documents that the user agreed to to this +list. + +### Terms Acceptance in the API + +Before any requests are made to an Identity Server or Integration Manager, +the client must use the `GET $prefix/terms` endpoint to fetch the set of +documents that the user must agree to in order to use the service. + +It then cross-references this set of documents against the `m.accepted_terms` +account data and presents to the user any documents that they have not already +agreed to, along with UI for them to indicate their agreement. If there are no +such documents (ie. if the `policies` dict is empty or the user has already +agreed to all documents) the client proceeds to perform the OpenID +registration. If there are new terms documents, the client prompts the user for +agreement, then once the user has indicated their agreement, it adds these URLs +to `m.accepted_terms` account data and then proceeds with OpenID +authentication, getting a token from the Homeserver and submitting this to the +service using the `register` endpoint. + +Having done this, if the user agreed to any new documents, it performs a `POST +$prefix/terms` request to signal to the server the set of documents that the +user has agreed to. + +Any request to any endpoint in the IM API, and the `/_matrix/identity/v2/` +namespace of the IS API, with the exception of `/_matrix/identity/v2` itself, +may return: + + * `M_UNAUTHORIZED` errcode with HTTP status code 401. This indicates that + the user must authenticate with OpenID and supply a valid `access_token`. + * `M_TERMS_NOT_SIGNED` errcode with HTTP status code 403. This indicates + that the user must agree to (new) terms in order to use or continue to + use the service. + +The `/_matrix/identity/v2/3pid/unbind` endpoint must not return either of these +errors if the request has a valid signature from a Homeserver, and is being authenticated as such. + +In summary, the process for using a service that has not previously been used +in the current login session is: + + * `GET $prefix/terms` + * Compare result with `m.accepted_terms` account data, get set of documents + pending agreement. + * If non-empty, show this set of documents to the user and wait for the user + to indicate their agreement. + * Add the newly agreed documents to `m.accepted_terms`. + * On success, or if there were no documents pending agreement, get an OpenID + token from the Homeserver and submit this token to the `register` endpoint. + Store the resulting access token. + * If the set of documents pending agreement was non-empty, Perform a + `POST $prefix/terms` request to the service with these documents. + +### `POST /_matrix/client/r0/account/3pid/unbind` + +A client uses this client/server API endpoint to request that the Homeserver +removes the given 3PID from the given Identity Server, or all Identity Servers. +Takes the same parameters as +`POST /_matrix/client/r0/account/3pid/delete`, ie. `id_server`, `medium`, +`address`. Similar to the other unbind endpoints, this endpoint does not +require an `id_access_token` because the homeserver can only unbind. + +Returns the same as `POST /_matrix/client/r0/account/3pid/delete`. + +Clients may add IS bindings for 3PIDs that already exist on the user's +Homeserver account by using the `POST /_matrix/client/r0/account/3pid` +to re-add the 3PID. + +### Unstable feature flag for transition + +In order to allow client implementations to determine if the homeserver they are developed +against supports `id_access_token`, an unstable feature flag of `m.id_access_token` +is to be added to `/versions`. When the flag is `false` or not present, clients must assume +that the homeserver does not support being given `id_access_token` and may receive an error +for doing so. Clients are expected to use the supported specification versions the homeserver +advertises instead of the feature flag's presence once this proposal is included in a release +of the specification. + +## Tradeoffs + +The Identity Service API previously did not require authentication, and OpenID +is reasonably complex, adding a significant burden to both clients and servers. +A custom HTTP header was also considered that could be added to assert that the +client agrees to a particular set of terms. We decided against this in favour +of re-using existing primitives that already exist in the Matrix ecosystem. +Custom HTTP headers are not used anywhere else within Matrix. This also gives a +very simple and natural way for ISes to enforce that users may only bind 3PIDs +to their own MXIDs. + +This introduces a different way of accepting terms from the client/server API +which uses User-Interactive Authentication. In the client/server API, the use +of UI auth allows terms acceptance to be integrated into the registration flow +in a simple and backwards-compatible way. Another option here would be to use +UI Auth on the register endpoint. This would also not allow users to register +before accepting the terms. However, this would then make the OpenID +registration process different and non-standard. + +The `m.accepted_terms` section contains only URLs of the documents that +have been agreed to. This loses information like the name and version of +the document, but: + * It would be up to the clients to copy this information correctly into + account data. + * Having just the URLs makes it much easier for clients to make a list + of URLs and find documents not already agreed to. + +## Potential issues + +This change deprecates all v1 endpoints and so will require clients to update +to continue working. + +## Security considerations + +Requiring authentication on the IS API means it will no longer be possible to +use it anonymously. + +It is assumed that once servers publish a given version of a document at a +given URL, the contents of that URL will not change. This could be mitigated by +identifying documents based on a hash of their contents rather than their URLs. +Agreement to terms in the client/server API makes this assumption, so this +proposal aims to be consistent. + + +## Conclusion + +This proposal adds an error response to all endpoints on the API and a custom +HTTP header on all requests that is used to signal agreement to a set of terms +and conditions. The use of the header is only necessary if the server has no +other means of tracking acceptance of terms per-user. The IS API is not +authenticated so ISes will have no choice but to use the header. The IM API is +authenticated so IMs may either use the header or store acceptance per-user. + +A separate endpoint is specified with a GET request for retrieving the set +of terms required and a POST to indicate that the user consents to those +terms. diff --git a/proposals/2176-update-redaction-rules.md b/proposals/2176-update-redaction-rules.md new file mode 100644 index 00000000..574028f2 --- /dev/null +++ b/proposals/2176-update-redaction-rules.md @@ -0,0 +1,97 @@ +# MSC2176: Update the redaction rules + +The current [redaction +algorithm](https://matrix.org/docs/spec/client_server/r0.5.0#redactions) is now +somewhat dated. This MSC proposes a number of changes to the rules which will +improve the security and reliability of the Matrix protocol. + +## Proposal + +The following changes will require a new room version, since changes to the +redaction algorithm also change the way that [event +hashes](https://matrix.org/docs/spec/server_server/r0.1.2#calculating-the-reference-hash-for-an-event) +(and hence event IDs) are calculated. + +The following *event* keys are to be *removed* from the list of those to be +preserved by a redaction: + + * `membership` + * `prev_state` + +(Note this refers to the *event-level* `membership` property, rather than the +similarly-named sub-property under the `content` key.) + +Rationale: neither of the above properties have defined meanings any more in the Matrix +protocol, so there is no reason for them to be special-cased in this way. + +The following are to be added to the list of subkeys of the content property +which are preserved: + + * `m.room.create` preserves *all* content. Rationale: the values in a + `create` event are deliberately intended to last the lifetime of the room, + and if values are redacted, there is no way to add correct settings + afterwards. It therefore seems non-sensical to allow redaction of a `create` + event. + + * `m.room.redaction` should allow the `redacts` key (assuming + [MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174) is merged). + Rationale: currently, redacting a redaction can lead to inconsistent results + among homservers, depending on whether they receive the `m.room.redaction` + result before or after it is redacted (and therefore may or may not redact + the original event). + + * `m.room.power_levels` should allow (in addition to the keys already listed + in the spec): + + * the `invite` key. Rationale: this is required to authenticate + `m.room.member` events with the `invite` membership. Currently, redacting + a `power_levels` event will mean that such events cannot be authenticated, + potentially leading to a split-brain room. + +## Other properties considered for preservation + +Currently it is *not* proposed to add these to the list of properties which are +proposed for a redaction: + + * The `notifications` key of `m.room.power_levels`. Unlike the other + properties in `power_levels`, `notifications` does not play a part in + authorising the events in the room graph. Once the `power_levels` are + replaced, historical values of the `notifications` property are + irrelevant. There is therefore no need for it to be protected from + redactions. + + * The `algorithm` key of `m.room.encryption`. Again, historical values of + `m.room.encryption` have no effect, and servers do not use the value of the + property to authenticate events. + + The effect of redacting an `m.room.encryption` event is much the same as that + of sending a new `m.room.encryption` event with no `algorithm` key. It's + unlikely to be what was intended, but adding rules to the redaction + algorithm will not help this. + +### Background to things not included in the proposal + +The approach taken here has been to minimise the list of properties preserved +by redaction; in general, the list is limited to those which are required by +servers to authenticate events in the room. One reason for this is to simplify +the implementation of servers and clients, but a more important philosophical +reason is as follows. + +Changing the redaction algorithm requires changes to both servers and clients, +so changes are difficult and will happen rarely. Adding additional keys now +sets an awkward precedent. + +It is likely that in the future more properties will be defined which might be +convenient to preserve under redaction. One of the two scenarios would then +happen: + + * We would be forced to issue yet more updates to the redaction algorithm, + with a new room versions and mandatory updates to all servers and clients, or: + + * We would end up with an awkward asymmetry between properties which were + preserved under this MSC, and those which were introduced later so were not + preserved. + +In short, I consider it important for the elegance of the Matrix protocol that +we do not add unnecessary properties to the list of those to be preserved by +redaction. diff --git a/proposals/2229-rebind-existing-3pid.md b/proposals/2229-rebind-existing-3pid.md index 83883c78..398e6513 100644 --- a/proposals/2229-rebind-existing-3pid.md +++ b/proposals/2229-rebind-existing-3pid.md @@ -1,5 +1,16 @@ # Allowing 3PID Owners to Rebind +## Note: This MSC has been made obsolete by MSC2290. + +MSC2290 provides two separate API endpoints, one for adding a 3PID to the +homeserver, and another for binding to an identity server. These new +endpoints will allow the homeserver to enforce rules on emails that already +exist on the homeserver, only when modifying homeserver email, while only +needing to forward requests when binding to an identity server. This removes +the problem MSC2229 is trying to solve, and it is thus made obselete. + +--- + ``` 3PID noun diff --git a/proposals/2230-identity-server-account-data.md b/proposals/2230-identity-server-account-data.md new file mode 100644 index 00000000..19fb3062 --- /dev/null +++ b/proposals/2230-identity-server-account-data.md @@ -0,0 +1,71 @@ +# MSC2230: Store Identity Server in Account Data + +The URL of the Identity Server to use is currently specified at registration and +login time and then used for the lifetime of a login session. If users wish to +specify a custom one, they must do so each time they log in on every client. +Once they have chosen an Identity Server to advertise their 3PIDs on, it would +be normal that they would wish to continue using this Identity Server for all +Identity requests in their account accross all clients. This proposal aims to +make this easier. + +## Proposal + +The base URL of the Identity Server is to be stored in user account data. It +shall be stored in the same format as in a .well-known file under the event type +`m.identity_server` and shall comprise a single key, `base_url` which is the +base URL of the ID Server to use (that is, the part before `/_matrix`, including +`https://`). + +Upon registration or login, a client SHOULD refrain from performing any requests +to the Identity Server until the account data has been fetched from the server. +Once it has the account data, it SHOULD check for the presence of the +`m.identity_server` key. If present, the `base_url` in this key SHOULD be used +as the Identity Server base URL for the duration of the login session. If this +key is not present, the client SHOULD use whatever value it would have used prior +to this MSC. It should not update the account data in this situation. + +Client SHOULD listen for changes in the `m.identity_server` account data value +and update the URL that they use for ID Server requests accordingly. + +Clients can offer a way for users to change the ID server being used. If they +do, the client MUST update the value of `m.identity_server` accordingly. + +The `m.identity_server` may be present with a `base_url` of `null`. In this case, +clients MUST treat this as no ID Server URL being set and not perform ID +Server requests, disabling any functionality that requires such requests. + +Conversely, if a user wishes to disable ID Server functionality, the client +shall action this by setting the `base_url` of the `m.identity_server` +account data entry to `null`. + +### Transition Period + +Clients will continue to use whatever IS URLs they currently use until the +user sets one explicitly, at which point it will be written to account data +and all clients will start using this value. + +## Tradeoffs + +There are a number of ways to transition to this new scheme. Clients could +populate the account data with their current ID Server URL as soon as +possible, and immediately use any new value seen on account data. This +would a much faster migration but any users with clients using different +ID Servers would suddenly find all their clients using the ID Server of +whichever client they updated first. + +## Potential issues + +Users will no longer be able to have different clients configured with +different ID Servers. + +## Security considerations + +An attacker would be able to force all a user clients to use a given ID Server +if they gained control of any of a user's logins. + +## Conclusion + +This makes the ID server an account setting which means it persists between +logins. The intention would be to phase out clients ever asking for an ID +Server URL at registration or login: this will be much easier for users to +understand whilst still retaining the flexibilty for those who want it. diff --git a/proposals/2244-mass-redactions.md b/proposals/2244-mass-redactions.md new file mode 100644 index 00000000..4f410356 --- /dev/null +++ b/proposals/2244-mass-redactions.md @@ -0,0 +1,78 @@ +# Mass redactions +Matrix, like any platform with public chat rooms, has spammers. Currently, +redacting spam essentially requires spamming redaction events in a 1:1 ratio, +which is not optimal[1](images/2244-redaction-spam.png). Most +clients do not even have any mass redaction tools, likely in part due to the +lack of a mass redaction API. A mass redaction API on the other hand has not +been implemented as it would require sending lots of events at once. However, +this problem could be solved by allowing a single redaction event to redact +many events instead of sending many redaction events. + +## Proposal +This proposal builds upon [MSC2174](https://github.com/matrix-org/matrix-doc/pull/2174) +and suggests making the `redacts` field in the content of `m.room.redaction` +events an array of event ID strings instead of a single event ID string. + +It would be easiest to do this before MSC2174 is written into the spec, as then +only one migration would be needed: from an event-level redacts string to a +content-level redacts array. + +### Backwards compatibility +There is no easy way to stay fully compatible with *older* clients, so the +proposed solution is to not support them. In order to not break old clients +completely, servers should still add a `redacts` string containing one of the +redacted event IDs to the top level of `m.room.redaction` events in *newer* +room versions when serving such events over the Client-Server API. + +Like MSC2174, for improved compatibility with *newer* clients, servers should +add a `redacts` array to the `content` of `m.room.redaction` events in *older* +room versions when serving such events over the Client-Server API. + +### Number of redactions +Room v4+ event IDs are 44 bytes long, which means the federation event size +limit would cap a single redaction event at a bit less than 1500 targets. +Redactions are not intrinsically heavy, so a separate limit should not be +necessary. + +Due to the possible large number of redaction targets per redaction event, +servers should omit the list of redaction targets from the `unsigned` -> +`redacted_because` field of redacted events. If clients want to get the list +of targets of a redaction event in `redacted_because`, they should read the +`event_id` field of the `redacted_because` event and use the +`/rooms/{roomId}/event/{eventId}` endpoint to fetch the content. + +### Client behavior +Clients shall apply existing `m.room.redaction` target behavior over an array +of event ID strings. + +### Server behavior (auth rules) +The target events of an `m.room.redaction` shall no longer be considered when +authorizing an `m.room.redaction` event. Any other existing rules remain +unchanged. + +After a server accepts an `m.room.redaction` using the modified auth rules, it +evaluates individually whether each target can be redacted under the existing +room v5 auth rules. Servers MUST NOT include failing and unknown entries to +clients. + +> Servers do not know whether redaction targets are authorized at the time they + receive the `m.room.redaction` unless they are in possession of the target + event. Implementations retain entries in the original list which were not + shared with clients to later evaluate the target's redaction status. + +When the implementation receives a belated target from an earlier +`m.room.redaction`, it evaluates at that point whether the redaction is +authorized. + +> Servers should not send belated target events to clients if their redaction + was found to be in effect, as clients were not made aware of the redaction. + That fact is also used to simply ignore unauthorized targets and send the + events to clients normally. + +## Tradeoffs + +## Potential issues + +## Security considerations +Server implementations should ensure that large redaction events do not become +a DoS vector, e.g. by processing redactions in the background. diff --git a/proposals/2263-homeserver-pw-resets.md b/proposals/2263-homeserver-pw-resets.md new file mode 100644 index 00000000..3b58af17 --- /dev/null +++ b/proposals/2263-homeserver-pw-resets.md @@ -0,0 +1,56 @@ +# MSC2263: Give homeservers the ability to handle their own 3PID registrations/password resets + +In order to better protect the privacy of a user, Matrix is wanting to shift to +a model where identity servers have less control over the affairs of the homeserver. +Identity servers are currently used to reset the passwords of users on a given homeserver +as an identity verification technique, however there is no reason why the homeserver +itself can't handle the verification. This proposal allows for a homeserver to verify +the identity of users itself, without the use of an identity server. + +## Proposal + +The `id_server` parameter is to become optional on the following endpoints: + +* `/_matrix/client/:version/account/3pid/:medium/requestToken` +* `/_matrix/client/:version/register/:medium/requestToken` +* `/_matrix/client/:version/account/password/:medium/requestToken` + +The `id_server` parameter is additionally deprecated with intention of being removed +in a future specification release on the `/register/:medium` and `/account/password/:medium` +endpoints. Once appropriate adoption has been achieved, the specification can safely +remove the parameter as supported. The reason for this deprecation is to completely +remove the identity server's ability to be involved in password resets/registration. +Users wishing to bind their 3rd party identifiers can do so after registration, and +clients can automate this if they so desire. + +Note that `bind_email` and `bind_msisdn` on `/register` have already been removed +by [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140). + +As per [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140), an `id_access_token` +is required only if an `id_server` is supplied. + +Although not specified as required in the specification currently, the `id_server` +as part of User-Interactive Authentication is also optional if this proposal is accepted. +When the client requests a token without an `id_server`, it should not specify an +`id_server` in UIA. + +Homeservers can reuse HTTP 400 `M_SERVER_NOT_TRUSTED` as an error code on the `/requestToken` +endpoints listed above if they do not trust the identity server the user is supplying. + +In order to allow client implementations to determine if the homeserver they are developed +against supports `id_server` being optional, an unstable feature flag of `m.require_identity_server` +is to be added to `/versions`. When the flag is `true` or not present, clients must assume +that the homeserver requires an `id_server` (ie: it has not yet considered it optional). +If this proposal is accepted, clients are expected to use the supported specification versions +the homeserver advertises instead of the feature flag's presence. + +## Tradeoffs + +Homeservers may have to set up MSISDN/email support to their implementations. This is believed +to be of minimal risk compared to allowing the identity server to continue being involved +with password reset/registration. + +## Security considerations + +The identity server was previously involved with affairs only the homeserver cares about. +This is no longer the case. diff --git a/proposals/2284-optional-identity-server-discovery.md b/proposals/2284-optional-identity-server-discovery.md new file mode 100644 index 00000000..940f3381 --- /dev/null +++ b/proposals/2284-optional-identity-server-discovery.md @@ -0,0 +1,24 @@ +# MSC2284: Making the identity server optional during discovery + +Currently the specification requires that clients `FAIL_ERROR` (hard failure - do not continue) +when the `.well-known` file for `m.identity_server` points to somewhere invalid or is invalid +itself. This can cause problems for clients if they either don't need an identity server to +function (and are forced to validate it anyways) or the client ends up having to disable all +their login UX because the identity server is misconfigured/down. + +This proposal aims to change that by allowing clients to make a conscious decision to continue +with the invalid identity server configuration, provided the homeserver configuration is valid. + +## Proposal + +Instead of `FAIL_ERROR` for an invalid `m.identity_server` schema/server, clients are to move +to the `FAIL_PROMPT` (inform the user, ask for input if applicable) state. Clients can decide +to show a warning that the identity server is unavailable and allow the user to continue with +the invalid (or client's default) configuration. + +## Tradeoffs + +Clients can end up being configured with an invalid or inoperable identity server. This is +considered a feature by this proposal to allow for less intelligent clients to have their +identity server disabled. Intelligent clients could interpret the lack of identity server +as the homeserver/user asking that identity server functionality be disabled in the client. diff --git a/proposals/2290-separate-threepid-bind-hs.md b/proposals/2290-separate-threepid-bind-hs.md new file mode 100644 index 00000000..8c899699 --- /dev/null +++ b/proposals/2290-separate-threepid-bind-hs.md @@ -0,0 +1,223 @@ +# Separate Endpoints for Binding Threepids + +*Note: This MSC obsoletes +[MSC2229](https://github.com/matrix-org/matrix-doc/pull/2229), which dealt +with changing the rules of the `bind` flag on [POST +/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid). +That endpoint is being deprecated as part of this MSC, thus MSC2229 is no +longer relevant.* + +On the Client Server API there is currently a single endpoint for binding a +threepid (an email or a phone number): [POST +/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid). +Depending on whether the `bind` flag is `true` or `false`, the threepid will +be bound to either a user's account on the homeserver, or both the homeserver +and an identity server. Note that we use the term `add` when talking about +adding a threepid to a homeserver, and `bind` when binding a threepid to an +identity server. This terminology will be used throughout the rest of this +proposal. + +Typically, when using the `/account/3pid` endpoint, the identity server +handles the verification -- either by sending an email to an email address, +or a SMS message to a phone number. Once completed, the homeserver will check +with the identity server that verification had indeed happened, and if so, +the threepid would be either added to the homeserver, or added to the +homeserver and bound to the identity server simultaneously. + +Now, consider the fact that the identity server used in this process is +provided by the user, using the endpoint's `id_server` parameter. If the user were +to supply a malicious identity server that would immediately answer "yes" to +any threepid validation, then the user could add any threepid to their +account on the homeserver (which is likely not something homeserver admins want). + +To be clear, this is not a long-standing security issue. It is not a problem +in any released version of Synapse, as Synapse keeps a list of "trusted +identity servers" that acts a whitelist for what identity servers a user can +specify. + +The concept of this whitelist is being removed in this MSC however, as part +of lessening the reliance of homeservers on identity servers. This cannot be +done while the homeserver is still trusting an identity server for validation +of threepids. If the endpoints are split, the homeserver will handle the +validation of threepids being added to user accounts, and identity servers +will validate threepids being bound to themselves. + +## Proposal + +To solve this problem, two new endpoints will be added to the Client Server +API: `POST /account/3pid/bind` and `POST /account/3pid/add`. Binding to an +identity server will require standard authentication, whereas adding a 3pid +to a user account will require [User-Interactive +Authentication](https://matrix.org/docs/spec/client_server/r0.5.0#user-interactive-authentication-api). +The latter is to prevent someone from adding a 3pid (which can be used to +reset passwords) to someone who's left their account open on a public +computer, without needing their password to do so. + +Both endpoints will be rate-limited. The request parameters of `POST +/account/3pid/bind` are the same as [POST +/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid), +minus the `bind` flag, and the contents of `three_pid_creds` have been +brought to the top level of the request body. The request parameters of `POST +/account/3pid/add` will simply consist of a JSON body containing +`client_secret` and `sid`. + +The homeserver should prevent a threepid being added to a user's account if +it's already part of another user's account. However, the homeserver should not +check for existing threepids when binding to an identity server. Identity +servers do not enforce this requirement and neither should the proxying +homeserver. + +An example of binding a threepid to an identity server with this new endpoint +is as follows: + +First the client must request the threepid be validated by its chosen +identity server. + +``` +POST https://identity.server/_matrix/identity/v2/validate/email/requestToken + +{ + "client_secret": "don'tT3ll", + "email": "bob@example.com", + "send_attempt": 1 +} +``` + +The identity server must send an email to the specified address, including a +link to a URL on the identity server which will accept the validation session +ID, the given client_secret, and a randomly-generated token. + +Once an email has been sent, the user clicks the link in the email, which +notifies the identity server that the email has been verified. + +Next, the client completes the bind by calling the new endpoint on the +homeserver: + +``` +POST https://home.server/_matrix/client/r0/account/3pid/bind + +{ + "id_server": "example.org", + "id_access_token": "abc123_OpaqueString", + "sid": "abc123987", + "client_secret": "don'tT3ll" +} +``` + +The homeserver will then make a bind request to the specified identity server +on behalf of the user. The homeserver will record if the bind was successful +and notify the user. The homeserver will remember this bind and the identity +server it occurred on so that it can perform an unbind later if the user +requests it or their account is deactivated. + +The threepid has now been bound on the user's requested identity server +without causing that threepid to be used for password resets or any other +homeserver-related functions. + +For completeness, here is an example of adding a threepid to the homeserver +only, using the `/account/3pid/add` endpoint: + +The homeserver is validating the threepid in this instance, so the client +must use the `/requestToken` endpoint of the homeserver: + +``` +POST https://home.server/_matrix/client/r0/account/3pid/email/requestToken + +{ + "client_secret": "don'tT3ll", + "email": "bob@example.com", + "send_attempt": 1, +} +``` + +Here the homeserver must send an email to the specified address, including a +link to a URL on the homeserver which will accept the validation session ID, +the given client_secret, and a randomly-generated token. + +Once an email has been sent, the user clicks the link in the email, which +notifies the homeserver that the threepid has been verified. + +The client then sends a request to the endpoint on the homeserver to add +the threepid to a user's account. + +``` +POST https://home.server/_matrix/client/r0/account/3pid/add + +{ + "sid": "abc123987", + "client_secret": "don'tT3ll" +} +``` + +The homeserver checks the threepid validation session referred to by the +given ID and client_secret was validated, and if so adds the threepid to the +user's account. + +To achieve the above flows, some changes need to be made to existing +endpoints. The [POST +/account/3pid](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid) +endpoint is deprecated as the two new endpoints replace its functionality. +The `bind` parameter is to be removed, with the endpoint functioning as if +`bind` was `false`. Allowing an endpoint to add a threepid to both the +identity server and homeserver at the same time requires one to trust the +other, which is the exact behaviour we're trying to eliminate. Doing this +also helps backward compatibility, as explained in [Backwards +compatibility](#backwards-compatibility). + +Either the homeserver itself or a service that the homeserver delegates to +should be handling the sending of validation messages, not a user-provided +server. Any mention of the homeserver being able to proxy to an identity +server in the below endpoint descriptions: + +* [POST /account/3pid/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-email-requesttoken) +* [POST /account/3pid/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-3pid-msisdn-requesttoken) +* [POST /register/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-email-requesttoken) +* [POST /register/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-register-msisdn-requesttoken) +* [POST /account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-email-requesttoken) +* [POST /account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.5.0#post-matrix-client-r0-account-password-msisdn-requesttoken) + +As well as the text "It is imperative that the homeserver keep a list of +trusted Identity Servers and only proxies to those that it trusts." is to be +removed from all parts of the spec, as the homeserver should no longer need +to trust any identity servers. + +## Tradeoffs + +One may question why clients don't just contact an identity server directly +to bind a threepid, bypassing the implications of binding through a +homeserver. While this will work, binds should still occur through a +homeserver such that the homeserver can keep track of which binds were made, +which is important when a user wishes to deactivate their account (and remove +all of their bindings made on different identity servers). + +A verification could occur on an identity server, which could then tell the +homeserver that a validation happened, but then there are security +considerations about how to authenticate an identity server in that instance +(and prevent people pretending to be identity servers and telling homeservers +about hundreds of fake threepid additions to a user's account). + +## Backwards compatibility + +A new flag will be added to `/versions`' unstable_features section, +`m.separate_add_and_bind`. If this flag is present and set to `true`, then +clients should use the new API endpoints to carry out threepid adds and +binds. If this flag is not present or set to `false`, clients should use +`/account/3pid`, being aware that they can only bind threepids to the +homeserver, not the identity server. + +Old matrix clients will continue to use the `/account/3pid` endpoint. This +MSC removes the `bind` parameter and forces `/account/3pid` calls to act as +if `bind` was set to `false`. Old clients will still be able to add 3pids to +the homeserver, but not bind to the identity server. New homeservers must +ignore any `id_server` information passed to this endpoint. + +## Security considerations + +Reducing the homeserver's trust in identity servers should be a boost to +security and improve decentralisation in the Matrix ecosystem to boot. + +Some endpoints of the Client Server API allow a user to provide an +`id_server` parameter. Caution should be taken for homeserver developers to +stop using these user-provided identity servers for any sensitive tasks where +possible, such as password reset or account registration, if it removes the +concept of a trusted identity server list. diff --git a/proposals/2313-moderation-policy-rooms.md b/proposals/2313-moderation-policy-rooms.md new file mode 100644 index 00000000..0fe9d7b1 --- /dev/null +++ b/proposals/2313-moderation-policy-rooms.md @@ -0,0 +1,199 @@ +# MSC2313: Moderation policies as rooms (ban lists) + +Matrix is an open network and anyone can participate in it. As a result, a +very wide range of content exists, and it is important to empower users to be +able to select which content they wish to see, and which they wish to block. By +extension, room moderators and server admins should also be able to select +which content they do not wish to host in their rooms and servers. + +The protocol's position in this solution should be one of neutrality: it +should not be deciding what content is undesirable for any particular entity and +should instead be empowering those entities to make their own decisions. This +proposal introduces "moderation policy rooms" as a basic mechanism to help users +manage this process, by providing a way of modelling sets of servers, rooms and users +which can then be used to make filtering decisions. This proposal makes no +attempt at interpreting the model and actually making those decisions however. + +To reaffirm: where this proposal says that some content is undesirable it does not intend to +bias the reader into what that could entail. Undesirability is purely in the hands of the +entity perceiving the content. For example, someone who believes birthday cake is undesirable +is perfectly valid in taking that position and is encouraged by this proposal to set up or +use a policy room which prevents birthday cake from coming across their field of view. + +## Proposal + +Moderation policy lists, also known as ban lists in this proposal, are stored as room state events, +allowing for structures and concepts to be reused without defining a new room version. This +proposal does not make any restrictions on how the rooms are configured, just that the state +events described here are represented in a room. For example, a room which is invite only is +just as valid as a room that is not: the important details are specific state events and not +the accessibility, retention, or other aspects of the room. + +Ban lists are stored as `m.policy.rule.` state events, with state keys being arbitrary IDs +assigned by the sender. The `` is currently one of `user`, `room`, and `server`. Three +fields are defined in `content`: + +* `entity` (`string`) - **Required.** The entity/entities the recommendation applies to. Simple globs are supported + for defining entities (`*` and `?` as wildcards, just like `m.room.server_acl`). +* `recommendation` (`enum`) - **Required.** The action subscribed entities should take against + the described entity. Currently only `m.ban` is defined (see below). +* `reason` (`string`) - **Required.** The human-readable description for the recommendation. + +Invalid or missing fields are to be treated as though the rule doesn't exist. This is to +allow for rules to be deleted while state events cannot be deleted in Matrix. + +An example set of minimal state events for banning `@alice:example.org`, `!matrix:example.org`, +`evil.example.org`, and `*.evil.example.org` would be: + +```json +[ + { + "type": "m.policy.rule.user", + "state_key": "rule_1", + "content": { + "entity": "@alice:example.org", + "recommendation": "m.ban", + "reason": "undesirable behaviour" + } + }, + { + "type": "m.policy.rule.room", + "state_key": "rule_2", + "content": { + "entity": "!matrix:example.org", + "recommendation": "m.ban", + "reason": "undesirable content" + } + }, + { + "type": "m.policy.rule.server", + "state_key": "rule_3", + "content": { + "entity": "evil.example.org", + "recommendation": "m.ban", + "reason": "undesirable engagement" + } + }, + { + "type": "m.policy.rule.server", + "state_key": "rule_4", + "content": { + "entity": "*.evil.example.org", + "recommendation": "m.ban", + "reason": "undesirable engagement" + } + } +] +``` + +When the entity is a room, it can be a room alias or ID - the subscriber is responsible for +resolving it to a room ID (if it wants to). + +Non-standard recommendations are permitted using the Java package naming convention for +namespacing. A `recommendation` is just a recommendation: how implementations apply the rules +is left as a concern for those implementations. The only standard `recommendation` currently +defined is `m.ban`: The `entity` should be banned from participation where possible. + +The enforcement mechanism of `m.ban` is deliberately left as an implementation detail to avoid the +protocol imposing its opinion on how the lists are interpreted. However, a suggestion for +a simple implementation is as follows: + +* Is a user... + * Applied to a user: The user should be added to the subscriber's ignore list. + * Applied to a room: The user should be banned from the room (either on sight or immediately). + * Applied to a server: The user should not be allowed to send invites to users on the server. +* Is a room... + * Applied to a user: The user should leave the room and not join it + ([MSC2270](https://github.com/matrix-org/matrix-doc/pull/2270)-style ignore). + * Applied to a room: No-op because a room cannot ban itself. + * Applied to a server: The server should prevent users from joining the room and from receiving + invites to it (similar to the `shutdown_room` API in Synapse). +* Is a server... + * Applied to a user: The user should not receive events or invites from the server. + * Applied to a room: The server is added as a denied server in the ACLs. + * Applied to a server: The subscriber should avoid federating with the server as much as + possible by blocking invites from the server and not sending traffic unless strictly + required (no outbound invites). + +Other entities could be represented by this recommendation as well, however as per the +introduction to this proposal they are strictly out of scope. An example would be an integration +manager which doesn't want to offer integrations to banned entities - this is an implementation +detail for the integration manager to solve. + +A new event type is introduced here instead of reusing existing events (membership, ACLs, etc) +because the implication of a recommendation/rule is less clear when using the more narrow-scoped +events. For example, a philosophical question arises over what a `membership` of `ban` means to a server +subscribed to the list. More questions get raised when the `membership` of a user isn't `ban`, +but is `join` or similar instead - if the subscriber was mirroring events, it would be inclined +to try and sync membership lists, which this proposal attempts to avoid by using a more generic +and neutral event type. + +How subscriptions to ban lists are handled is also left as an implementation +detail (to avoid unnecessarily blocking progress on this MSC). The subscriber +could be anything that speaks Matrix, therefore this proposal makes no attempt +to describe how this should work for everything. Some ideas for how this could +be implemented include joining the ban list room to watch for updates and +applying them automatically, however there is no requirement that the +subscriber needs to join the room: they could peek or poll at an interval +instead, which is just as valid. + +To ease sharing of these ban list rooms, a system very similar to [MSC1951's sharable URLs]( +https://github.com/matrix-org/matrix-doc/pull/1951/files#diff-4ee6ed0ee1f2df73efac5fa9a9835642R50-R70) +is defined. There are two ways to share the ban list: a link to the room as one would when +sharing any reference to any other room ("please add `#bans:example.org` to your block list"), +or by using plain `http` or `https` URLs. Just like in MSC1951, the URL when approached with +a `Accept: application/json` header or has `.json` appended to the end of the URL should return +a `room_uri` which is a reference to the ban list room. Currently, the reference would be a +`matrix.to` URI, however in the future this could be a `mx://` or similar URL. When not approached +with the intent of JSON, the service could return a user-friendly page describing what is included +in the ban list. + +## Future considerations + +This proposal notably does not define specific behaviour for AI or machine learning applications. +Implementations are currently able to apply AI/ML to their systems if they see fit (for example, +spam detection or undesirable content being uploaded), however no specification is proposed +here to make the interaction standardized. + +This proposal additionally does not describe how a server could subscribe to a ban list: this +is left for the server to figure out (possibly by using a utility user account?) potentially +with the support of other proposals, such as [MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777). + +Further work on reputation systems could enhance ban lists by adding additional metrics to +assert their validity. This proposal assumes social trust ("don't use it if you +don't trust the creator") over verifiable/provable trust - a future proposal can easily add +such systems to these ban lists. + +This proposal intentionally does not handle how a server could assist the user in preventing +undesirable content or subscribing to ban lists. Users already have some tools at their disposal, +such as being able to ignore other users, and are encouraged to make use of those first. Other +proposals are encouraged to specify what additional tools might look like to better handle +ban lists. + +Media (uploaded content) is not handled by this proposal either. This is a concern left for +other proposals like [MSC2278](https://github.com/matrix-org/matrix-doc/pull/2278) and +[MSC701](https://github.com/matrix-org/matrix-doc/issues/701). + +## Security considerations + +Using this solution one can build a social system of shared blacklists, which +may create a divide within established communities if not carefully deployed. +This may well not be a suitable answer for all communities. + +Depending on how the implementations handle subscriptions, user IDs may be linked to ban +lists and therefore expose the views of that user. Using the example from the introduction, +if a user who does not like birthday cake were to join the ban list room for blocking +birthday cake, that user's preference would be exposed to any other observers of that ban +list. Proposals like [MSC1228](https://github.com/matrix-org/matrix-doc/issues/1228) and +[MSC1777](https://github.com/matrix-org/matrix-doc/pull/1777) could help solve this. + +## Implementation notes + +This proposal is partially implemented by [mjolnir](https://github.com/matrix-org/mjolnir) +using the `org.matrix.mjolnir.*` namespace until this becomes stable. This results in +the following mappings: + +* `m.policy.rule.user` => `org.matrix.mjolnir.rule.user` +* `m.policy.rule.room` => `org.matrix.mjolnir.rule.room` +* `m.policy.rule.server` => `org.matrix.mjolnir.rule.server` +* `m.ban` => `org.matrix.mjolnir.ban` diff --git a/proposals/2324-when-to-ship.md b/proposals/2324-when-to-ship.md new file mode 100644 index 00000000..1335bfab --- /dev/null +++ b/proposals/2324-when-to-ship.md @@ -0,0 +1,124 @@ +# MSC2324: Facilitating early releases of software dependent on spec + +*Note*: This is a process change MSC, not a change to the spec itself. + +There's currently an unanswered question by the spec process: when is it +safe to start using stable endpoints or to present a feature as "stable"? +Historically this question would receive very different answers depending +on who you asked, so in an effort to come up with a concise answer the +following process change is proposed. + +## Proposal + +The new process, from start to finish, is proposed as: + +1. Have an idea for a feature. +2. Optionally: implement the feature using unstable endpoints, vendor prefixes, + and unstable feature flags as appropriate. + * When using unstable endpoints, they MUST include a vendor prefix. For + example: `/_matrix/client/unstable/com.example/login`. Vendor prefixes + throughout this proposal always use the Java package naming convention. + * Unstable endpoints **do not** inherit from stable (`/r0`) APIs. Previously, + one could access the entirety of the Matrix API through `/unstable` however + this is generally considered a bad practice. Therefore an implementation + can no longer assume that because its feature-specific endpoint exists that + any other endpoint will exist in the same unstable namespace. + * If the client needs to be sure the server supports the feature, an unstable + feature flag that MUST be vendor prefixed is to be used. This kind of flag + shows up in the `unstable_features` field of `/versions` as, for example, + `com.example.new_login`. + * You can ship the feature at *any* time, so long as you are able to accept + the technical debt that results from needing to provide adequate backwards + and forwards compatibility for the vendor prefixed implementation. The + implementation MUST support the flag disappearing and be generally safe for + users. Note that implementations early in the MSC review process may also be + required to provide backwards compatibility with earlier editions of the + proposal. + * If you don't want to support the technical debt (or if it's impossible to + provide adequate backwards/forwards compatibility - e.g. a user authentication + change which can't be safely rolled back), do not implement the feature and + wait for Step 7. + * If at any point the idea changes, the feature flag should also change so + that implementations can adapt as needed. +3. In parallel, or ahead of implementation, open an MSC and solicit review. +4. Before a FCP (Final Comment Period) can be called, the Spec Core Team will + require that evidence to prove the MSC works be presented. A typical example + of this is an implementation of the MSC (which does not necessarily need to have been shipped anywhere). +5. FCP is gone through, and assuming nothing is flagged the MSC lands. +6. A spec PR is written to incorporate the changes into Matrix. +7. A spec release happens. +8. Implementations switch to using stable prefixes (e.g.: `/r0`) if the server + supports the specification version released. If the server doesn't advertise + the specification version, but does have the feature flag, unstable prefixes + should still be used. +9. A transition period of about 2 months starts immediately after the spec release, before + implementations start to loudly encourage other implementations to switch to stable + endpoints. For example, the Synapse team should start asking the Riot team to + support the stable endpoints (as per Step 8) 2 months after the spec release if they + haven't already. + +It's worth repeating that this process generally only applies if the implementation +wants to ship the feature ahead of the spec being available. By doing so, it takes +on the risk that the spec/MSC may change and it must adapt. If the implementation +is unable to take on that risk, or simply doesn't mind waiting, it should go through +the spec process without shipping an unstable implementation. + +To help MSCs get incorporated by implementations as stable features, the spec core +team plans to release the specification more often. How often is undefined and is +largely a case-by-case basis. + +To reiterate: + +* Implementations MUST NOT use stable endpoints before the MSC is in the spec. This + includes NOT using stable endpoints post-FCP. +* Implementations CAN ship features that are exposed by default to users before an + MSC has been merged to the spec, provided they follow the process above. +* Implementations SHOULD be wary of the technical debt they are incurring by moving + faster than the spec. + +To clarify: + +* The vendor prefix is chosen by the developer of the feature, using the Java package + naming convention. For example, `org.matrix` is the foundation's vendor prefix. +* The vendor prefixes, unstable feature flags, and unstable endpoints should be included + in the MSC so other developers can benefit. The MSC MUST still say what the stable + endpoints are to look like. + +### Specific examples outside of the client-server API + +There are some instances where a change might be made outside of the client-server API, +which is where much of this proposal is targetted. The general spirit of the process +should be followed where possible, if implementations decide to work ahead of spec releases. + +#### Room versions + +When a new room version is needed, implementations MUST use vendor-prefixed versions +before using the namespace reserved for Matrix (see https://matrix.org/docs/spec/#room-versions). +A room version is considered released once it is listed as an "available room version" in +the spec. Often a new room version is accompanied with a server-server API release, but +doesn't have to be. + +#### Server-server / Identity / Push / Appservice API + +These APIs don't yet have a `/versions` endpoint or similar. Typically behaviour changes in +these APIs are introduced with backwards compatibility in mind (try X and if that fails fall +back to Y) and therefore don't always need a flag to indicate support. If a flag were to +be required, an `unstable_features` or similar array would need to be exposed somewhere. + +#### Changes to request/response parameters + +Parameters being added to request/response bodies and query strings MUST be vendor-prefixed +per the proposed process. For example, a new JSON field might be `{"org.matrix.example": true}` +with the proposal being for `example` being added. A query string parameter would be prefixed +the same way: `?org.matrix.example=true`. + +If the MSC is simply adding fields to already-versioned endpoints, it does not need to put +the whole endpoint into the `/unstable` namespace provided the new parameters are prefixed +appropriately. + +#### .well-known and other APIs that can't be versioned + +Best effort is appreciated. Typically these endpoints will be receiving minor behavioural +changes or new fields. New fields should be appropriately prefixed, and behaviour changes +should be rolled out cautiously by implementations (waiting until after FCP has concluded +is probably best to ensure there's no major problems with the new behaviour). diff --git a/proposals/2334-default-room-version-v5.md b/proposals/2334-default-room-version-v5.md new file mode 100644 index 00000000..e511aafd --- /dev/null +++ b/proposals/2334-default-room-version-v5.md @@ -0,0 +1,29 @@ +# [MSC2334](https://github.com/matrix-org/matrix-doc/pull/2334) - Change default room version to v5 + +This MSC proposes changing the recommended default room version from v4 to v5. +This implements steps 5 and 6 of the plan from +[MSC2002](https://github.com/matrix-org/matrix-doc/issues/2002). + +Room version 5 enforces the `valid_until_ts` timestamp on signing keys as +proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). + +## Proposal + +When [MSC2077](https://github.com/matrix-org/matrix-doc/issues/2077) proposed +room version 5, the default version continued to be v4 to allow old servers to +continue to chat in newly created rooms. Server-Server API r0.1.2 (which +contains the details on room version 5) and Synapse 1.0.0 (with support for room +version 5) were both released more than 4 months ago which has given people +plenty of time to update. Room version 5 should be the default room version so +that newly created rooms enforce key-validity periods. + +## Potential issues + +Servers which do not support room version 5 will not be able to participate in +newly created rooms on other servers. Hopefully this will encourage them to +update their server. + +## Security considerations + +Room version 5 fixes known security vulnerabilities but that doesn't do much +good if newly created rooms aren't using room version 5 by default. \ No newline at end of file diff --git a/proposals/2367-membership-reasons.md b/proposals/2367-membership-reasons.md new file mode 100644 index 00000000..b43c909c --- /dev/null +++ b/proposals/2367-membership-reasons.md @@ -0,0 +1,81 @@ +# Allowing Reasons in all Membership Events + +Currently users can specify a reason for kicking or banning users in a room +that both the target and other users in a room can see. This is useful to +explain *why* an action without having to send separate messages. + +The proposal extends this to all events, including invites, invite rejections +and leaves for similar reasons. + +## Proposal + +Allow `reason` field to be specified for all of the following APIs: + +``` +POST /_matrix/client/r0/rooms/{roomId}/invite +POST /_matrix/client/r0/rooms/{roomId}/leave +POST /_matrix/client/r0/rooms/{roomId}/kick +POST /_matrix/client/r0/rooms/{roomId}/ban +POST /_matrix/client/r0/rooms/{roomId}/unban +POST /_matrix/client/r0/rooms/{roomId}/join +POST /_matrix/client/r0/join/{roomIdOrAlias} +PUT /_matrix/client/r0/rooms/{roomId}/state/m.room.member/{userID} +``` + +If specified the `reason` field will be added to the generated membership +event's content. + +*Note: `/state/m.room.member` API currently allows this as clients can specify +arbitrary content already* + +Clients may choose to display the reason for membership events in a room, +though may not do so if e.g. they have collapsed a set of membership changes. + +Clients should not display an invite reason by default to the invitee as this +allows a classic abuse and harassment vector. However, clients may wish to show +invite reasons from known¹ senders, or have a button that allows viewing any +invite reason. + +## Use Cases + +Some basic use cases and examples are given below. + +### Kick/ban + +Kicking and banning already allow specifying a reason to allow giving a reason +for the moderation action (e.g. "Banned for spamming"). + +### Leaving a Room + +A user may wish to leave a room e.g. because the room is no longer relevant +to them, allowing them to specify a reason means they can easily step out of +the room quietly without having to send a message to explain their actions. + +### Invite + +This can be useful to give some context for an invite, e.g. "Alice invites Bob +to get some feedback on the membership reasons MSC". + +### Rejecting an Invite + +If Alice has invited Bob (and many others) to a room to discuss going to a +concert then Bob may wish to simply reject the invite if he can't make it. +Adding a "will be out of town" reason to the rejection helps Alice to know why +her invite was rejected. + +### Joining room + +Adding a reason for joining could be used e.g. by automated bots to say why +they're joining. For example a bridge bot may join a room when asked to bridge +the room to an external network, in which case they may have a reason such as +"BridgeBot joined to bridge the room to RemoteNetwork at the request of Alice". + +## Potential Issues + +The main issue here is ensuring that the invite reason cannot be used as an +abuse vector, however if clients follow the recommendations above this concern +should be mitigated. + +--- + +¹ This is left up to implementations to decide, if they wish to do so. diff --git a/proposals/2422-allow-color-attribute-on-font-tag.md b/proposals/2422-allow-color-attribute-on-font-tag.md new file mode 100644 index 00000000..4e6a9ced --- /dev/null +++ b/proposals/2422-allow-color-attribute-on-font-tag.md @@ -0,0 +1,39 @@ +# MSC2422: Allow `color` as attribute for `` in messages + +Currently the spec recommends that you to use `data-mx-color` instead of the standard +`color` html attribute for the `` tag. This is probably done to make it +consistent with ``, where you may not want to allow a generic style tag for. + +On the other hand the /rainbow command on almost every client just uses the +`color` attribute of the `` tag. While some clients support +`data-mx-color` (i.e. Riot Web), most clients don't. Most clients support +rendering `color` however. + +It would probably be for the best to allow or even prefer `color` on the +`` tag. + +## Proposal + +Add the `color` attribute to the allowed attributes of `` in section +13.2.1.7. No changes to the allowable values from the HTML spec are made here. + +## Potential issues + +- We now have a redundant attribute in the spec. While it matches what the + clients currently do, it may be better to fix each client instead. +- Clients may not sanitize the color attribute and will let other color values + through, increasing compatibility issues again. +- Clients may never support the data-mx-* attributes now. +- Old messages could loose their color +- This proposal doesn't touch span at all, maybe it should? + +## Alternatives + +- fix the clients + -> This currently seems not feasible. Multiple clients started using color first (i.e. RiotX, Gomuks) and if it isn't spelled out explicitly in the spec, this will probably continue. +- remove the `data-mx-color` and `data-mx-bg-color` attributes entirely, leaving us just with `color` for `` + -> This would break old messages and can be done independently of this proposal at a later date, if it is deemed useful. +- Add a section to tell the clients to prefer `color` over `mx-data-color` + -> I don't really know, why mx-data-* was chosen, but I assume there was a reason, so I don't want to change that. +- Spec an entirely different format for messages (that would probably not make this proposal obsolete) + -> This wouldn't fix the issue, where some client may choose to remove the color tag, since it is discouraged in the spec. Migration would probably also take a while, so this proposal is a quick solution, that doesn't prevent other solutions at a later date. diff --git a/proposals/2432-revised-alias-publishing.md b/proposals/2432-revised-alias-publishing.md new file mode 100644 index 00000000..4c2f010f --- /dev/null +++ b/proposals/2432-revised-alias-publishing.md @@ -0,0 +1,247 @@ +# MSC2432: Updated semantics for publishing room aliases + +This MSC offers an alternative to [MSC2260](https://github.com/matrix-org/matrix-doc/issues/2260). + +## Background + +The [`m.room.aliases`](https://matrix.org/docs/spec/client_server/r0.6.0#m-room-aliases) +state event exists to list the available aliases for a given room. This serves +two purposes: + + * It allows existing members of the room to discover alternative aliases, + which may be useful for them to pass this knowledge on to others trying to + join. + + * Secondarily, it helps to educate users about how Matrix works by + illustrating multiple aliases per room and giving a perception of the size + of the network. + +However, it has problems: + + * Any user in the entire ecosystem can create aliases for rooms, which are + then unilaterally added to `m.room.aliases`, and room admins are unable to + remove them. This is an abuse + vector (https://github.com/matrix-org/matrix-doc/issues/625). + + * For various reasons, the `m.room.aliases` event tends to get out of sync + with the actual aliases (https://github.com/matrix-org/matrix-doc/issues/2262). + +## Proposal + +We propose that that room moderators should be able to manually curate a list +of "official" aliases for their room, instead of matrix servers automatically +publishing lists of all room aliases into the room state. No particular +guarantees are offered that this alias list is entirely accurate: it becomes +room moderators' responsibility to keep it so. + +Meanwhile, the aliases that map to a given room on a given server become +the ultimate responsibility of the administrators of that server. We give them +tools to inspect the alias list and clean it up when necessary, in addition to +the current tools which allow restriction of who can create aliases in the +first place. + +A detailed list of proposed modifications to the Matrix spec follows: + + * `m.room.aliases` loses any special meaning within the spec. In particular: + + * Clients should no longer format it specially in room timelines. + + * Clients and servers should no longer consider `m.room.aliases` when + [calculating the display name for a + room](https://matrix.org/docs/spec/client_server/r0.6.0#calculating-the-display-name-for-a-room). + + (Note: servers follow the room display-name algorithm when calculating + room names for certain types of push notification.) + + * A future room version will remove the special [authorization +rules](https://matrix.org/docs/spec/rooms/v1#authorization-rules) and +[redaction rules](https://matrix.org/docs/spec/client_server/r0.6.0#redactions). + + * [`m.room.canonical_alias`](https://matrix.org/docs/spec/client_server/r0.6.0#m-room-canonical-alias) + is extended to include a new `alt_aliases` property. This, if present, + should be a list of alternative aliases for the room. An example event might + look like: + + ```json + { + "content": { + "alias": "#somewhere:localhost", + "alt_aliases": [ + "#somewhere:overthere.com", + "#somewhereelse:example.com" + ] + }, + "room_id": "!jEsUZKDJdhlrceRyVU:example.org", + "state_key": "", + "type": "m.room.canonical_alias" + } + ``` + + It is valid for `alt_aliases` to be non-empty even if `alias` is absent or + empty. This means that no alias has been picked out as the 'main' alias. + + (Note: although the spec currently claims that `alias` is mandatory, Synapse + generates `m.room.canonical_alias` events with no `alias` property when the + main alias is deleted. This change would legitimise that behaviour.) + + (For clarity: it is not proposed that the `alt_aliases` be considered when + calculating the displayname for a room.) + + * [`PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-rooms-roomid-state-eventtype-statekey) + is extended to recommend that servers validate any *new* aliases added to + `m.room.canonical_alias` by checking that it is a valid alias according to + the [syntax](https://matrix.org/docs/spec/appendices#room-aliases), and by + looking up the alias and and that it corresponds to the expected room ID. + + (Note: Synapse currently implements this check on the main alias, though + this is unspecced.) + + The following error codes are specified: + + * HTTP 400, with `errcode: M_INVALID_PARAMETER` if an attempt is made to add + an entry which is not a well-formed alias (examples: too long, doesn't + start with `#`, doesn't contain a `:`). + + * HTTP 400, with `errcode: M_BAD_ALIAS` if an added alias does not point at + the given room (either because the alias doesn't exist, because it can't + be resolved due to an unreachable server, or because the alias points at a + different room). + + * Currently, [`PUT /_matrix/client/r0/directory/room/{roomAlias}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-directory-room-roomalias) + attempts to send updated `m.room.aliases` events on the caller's + behalf. (This is implemented in Synapse but does not appear to be explicitly + specced.) This functionality should be removed. + + * Currently, [`DELETE /_matrix/client/r0/directory/room/{roomAlias}`](https://matrix.org/docs/spec/client_server/r0.6.0#delete-matrix-client-r0-directory-room-roomalias), + attempts to send updated `m.room.aliases` and/or `m.room.canonical_alias` + events on the caller's behalf, removing any aliases which have been + deleted. (Again, this is implemented in Synapse but does not appear to be + explicitly specced.) The `m.room.aliases` functionality should be removed, + and the `m.room.canonical_alias` functionality should be extended to cover + `alt_aliases`. + + The behaviour if the calling user has permission to delete the alias but + does not have permission to send `m.room.canonical_alias` events in the room + (for example, by virtue of being a "server administrator", or by being the + user that created the alias) is implementation-defined. It is *recommended* + that in this case, the alias is deleted anyway, and a successful response is + returned to the client. + + * A new api endpoint, `GET /_matrix/client/r0/rooms/{roomId}/aliases` is + added, which returns the list of aliases currently defined on the local + server for the given room. The response looks like: + + ```json + { + "aliases": [ + "#somewhere:example.com", + "#somewhereelse:example.com", + "#special_alias:example.com" + ] + } + ``` + + This API can be called by any current member of the room (calls from other + users result in `M_FORBIDDEN`). For rooms with `history_visibility` set to + `world_readable`, it can also be called by users outside the room. + + Servers might also choose to allow access to other users such as server + administrators. + +Various APIs are currently subject to implementation-defined access +restrictions. No change to the specification is introduced in this regard +(implementations will continue to be free to impose their own +restrictions). Nevertheless as part of this MSC it is useful to consider some +proposed changes to Synapse's implementation: + + * No change: `PUT /_matrix/client/r0/directory/room/{roomAlias}`: Synapse + only allows access to current members of the room, and also exposes some + configuration options which allow restriction of which users are allowed to + create aliases in general. + + * `DELETE /_matrix/client/r0/directory/room/{roomAlias}`: in this case, + currently Synapse restricts its use to the user that created the alias, and + server admins. + + It is proposed to extend this to local users who have a power-level + sufficient to send an `m.room.canonical_alias` event in the room that the + alias currently points to. + + * [`PUT /_matrix/client/r0/directory/list/room/{roomId}`](https://matrix.org/docs/spec/client_server/r0.6.0#put-matrix-client-r0-directory-list-room-roomid) + and the corresponding unspecced `DELETE` api (both of which set the + visibility of a room in the public directory): currently Synapse restricts + their use to server admins and local users who have a PL sufficient to send + an `m.room.aliases` event in the room (ignoring the special auth + rules). This will be changed to check against the PL required to send an + `m.room.canonical_alias` event. + +It is envisaged that Matrix clients will then change their "Room Settings" user +interface to display the aliases from `m.room.canonical_alias` instead of those +in `m.room.aliases`, as well as giving moderators the ability to update that +list. Clients might also wish to use the new `GET +/_matrix/client/r0/rooms/{roomId}/aliases` endpoint to obtain and display the +currently-available local aliases, though given that this list may be subject +to abuse, it should probably not be shown by default. + +### Future work + +This work isn't considered part of this MSC, but rather a potential extension +for the future. + + * It may be useful to be able to query remote servers for their alias + list. This could be done by extending `GET + /_matrix/client/r0/rooms/{roomId}/aliases` to take a `server_name` + parameter, and defining an API in the server_server spec which will expose + the requested information, subject to the calling homeserver having at least + one user with a right to see it. + + * Similarly, room moderators may wish to be able to delete aliases on a remote + server for their room. We could envisage a federation API which allows such + a request to be made, subject to the calling homeserver having at least one + moderator in the room. + +## Potential issues + +The biggest problem with this proposal is that existing clients, which rely on +`m.room.aliases` in one way or another, will lose functionality. In particular, +they may not know about aliases that exist, or they may look at outdated +`m.room.aliases` events that list aliases that no longer exist. However, since +`m.room.aliases` is best-effort anyway, these are both problems that exist to +some extent today. + +## Alternatives + +We considered continuing to use `m.room.aliases` to advertise room aliases +instead of `m.room.canonical_alias`, but the significant changes in semantics +made that seem inappropriate. + +We also considered using separate state events for each advertised alias, +rather than listing them all in one event. This might increase the number of +aliases which can be advertised, and help to reduce races when editing the +list. However, the 64KB limit of an event still allows room for hundreds of +aliases of any sane length, and we don't expect the list to be changing +frequently enough for races to be a practical concern. Ultimately the added +complexity seemed redundant. + +A previous suggestion was +[MSC2260](https://github.com/matrix-org/matrix-doc/issues/2260), which proposed +keeping `m.room.aliases` largely as-is, but giving room moderators tools to +control who can send them via room power-levels. We dismissed it for the +reasons set out at +https://github.com/matrix-org/matrix-doc/pull/2260#issuecomment-584207073. + +## Security considerations + +None currently identified. + +## Unstable prefix + +While this feature is in development, the following names will be in use: + +| Proposed final name | Name while in development | +| --- | --- | +| `GET /_matrix/client/r0/rooms/{roomId}/aliases` | `GET /_matrix/client/unstable/org.matrix.msc2432/rooms/{roomId}/aliases` | + +Servers will indicate support for the new endpoint via a non-empty value for feature flag +`org.matrix.msc2432` in `unstable_features` in the response to `GET +/_matrix/client/versions`. diff --git a/proposals/2451-remove-query_auth-federation-endpoint.md b/proposals/2451-remove-query_auth-federation-endpoint.md new file mode 100644 index 00000000..b2d8bc9f --- /dev/null +++ b/proposals/2451-remove-query_auth-federation-endpoint.md @@ -0,0 +1,56 @@ +# MSC2451: Remove the `query_auth` federation endpoint + +This API was added without sufficient thought nor testing. The endpoint isn't +used in any known implementations, and we do not believe it to be necessary +for federation to work. The only known implementation (in Synapse) was not fully +fleshed out and is broken. + +For background, the idea behind this endpoint was that two homeservers would be +able to share state events with the hope of filling in missing state from one +of homeservers allowing state resolution to complete. This was to protect +against a joining server not providing the full (or providing stale) state. + +In addition to the ideas above not coming to fruition, it is unclear whether the +current design of this endpoint would be sufficient. If this state negotiation +feature is needed in the future it should be redesigned from scratch via the MSC +proposal process. + +## Proposal + +Remove the following endpoint: + +* [POST `/_matrix/federation/v1/query_auth/{roomId}/{eventId}`](https://matrix.org/docs/spec/server_server/r0.1.3#post-matrix-federation-v1-query-auth-roomid-eventid) + +## Potential issues + +Removing this endpoint impacts backwards compatibility, in practice removing +this endpoint should have minimal impact as it was an unused error path in +Synapse. The federation client code to call this endpoint was removed in Synapse +v1.5.0rc1. + +There is no evidence of other homeserver implementations having implemented this +endpoint. + +### History + +This endpoint (and the federation client code) to call it was initially +added in Synapse v0.7.0 (see [#43](https://github.com/matrix-org/synapse/pull/43)). +The federation client code was heavily modified for v1.0.0rc1 (see +[#5227](https://github.com/matrix-org/synapse/pull/5227/)), + +The federation client code to call this endpoint was removed in v1.5.0rc1 of +Synapse (see [#6214](https://github.com/matrix-org/synapse/pull/6214). After +that point this endpoint is not called). + +During removal it was noted that the code to call this endpoint was already +unreachable. It seems that this code was never reachable and was meant for an +error situation which was never built out (see `git log -S NOT_ANCESTOR`, the +error condition is never assigned). + +## Alternatives + +The endpoint could be deprecated and removed in a future version of the specification. + +## Security considerations + +None. diff --git a/proposals/2454-ui-interactive-auth-for-sso.md b/proposals/2454-ui-interactive-auth-for-sso.md new file mode 100644 index 00000000..18112c5f --- /dev/null +++ b/proposals/2454-ui-interactive-auth-for-sso.md @@ -0,0 +1,230 @@ +# User-Interactive Authentication for SSO-backed homeserver + +Certain endpoints, such as `DELETE /_matrix/client/r0/devices/{deviceId}` and +`POST /_matrix/client/r0/account/3pid/add`, require the user to reconfirm their +identity, as a guard against a leaked access token being used to take over an +entire account. + +On a normal homeserver, this is done by prompting the user to enter their +password. However, on a homeserver where users authenticate via a single-sign-on +system, the user doesn't have a password registered with the homeserver. Instead +we need to delegate that check to the SSO system. + +At the protocol level, this means adding support for SSO to the +[user-interactive authentication API](https://matrix.org/docs/spec/client_server/r0.6.0#user-interactive-authentication-api). + +In theory, once SSO is added as a possible flow for authentication, any clients +that already implement the [fallback process for unknown authentication types](https://matrix.org/docs/spec/client_server/r0.6.0#fallback) +will work fine without modification. It is unknown whether this is widely +supported among clients. + +## Proposal + +An [additional authentication type](https://matrix.org/docs/spec/client_server/r0.6.0#authentication-types) +of `m.login.sso` is added to the user-interactive authentication specification. + +There are no additional parameters as part of this authentication type. As per +the user-interactive authentication specification, the only parameter included in +the `auth` dictionary should be the session ID from the homeserver, e.g.: + +```json +{ + "auth": { + "session": "" + } +} +``` + +### Detailed fallback authentication flow: + +The following is a re-iteration of the [fallback authentication flow](https://matrix.org/docs/spec/client_server/r0.6.0#fallback), +but with details filled in for the proposed new authentication type. + +When choosing this authentication flow, the following should occur: + +1. If the client wants to complete authentication using SSO, it opens a browser + window for `/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>` + with session set to the UI-Auth session id (from the "auth" dict). + + The homeserver returns a page which asks for the user's confirmation before + proceeding. See the security considerations section below for why this is + necessary. For example, the page could say words to the effect of: + + > A client is trying to remove a device/add an email address/take over your + > account. To confirm this action, **re-authenticate with single sign-on**. + > If you did not expect this, your account may be compromised! +2. The link, once the user clicks on it, goes to the SSO provider's page. +3. The SSO provider validates the user, and redirects the browser back to the + homeserver. +4. The homeserver validates the response from the SSO provider, updates the + user-interactive auth session to show that the SSO has completed, and + [serves the fallback auth completion page as specced](https://matrix.org/docs/spec/client_server/r0.6.0#fallback). +5. The client resubmits its original request, with its original session id, + which now should complete. + +Note that the post-SSO URL on the homeserver is left up to the homeserver +implementation rather than forming part of the specification, choices might be +limited by the chosen SSO implementation, for example: + +* SAML2 servers typically only support one URL per service provider, so in + practice it will need to be the same as that already used for the login flow + (for synapse, it's `/_matrix/saml2/authn_response`) - and the server needs to + be able to figure out if it's doing SSO for a login attempt or an SSO + attempt. +* CAS doesn't have the same restriction. + +### Example flow: + +A more complete example is provided below in which a user attempts to delete +a device and is pushed into the user interactive authentication process with +SSO being the only possible flow. + +0. Client submits the request, which the server says requires SSO: + + ``` + POST /_matrix/client/r0/delete_devices HTTP/1.1 + Content-Type: application/json + Authorization: Bearer xyzzy + + { + "devices": ["FSVVTZRRAA"] + } + + HTTP/1.1 401 Unauthorized + Content-Type: application/json + + { + "flows": [ + { + "stages": [ + "m.login.sso" + ] + } + ], + "params": {}, + "session": "dTKfsLHSAJeAhqfxUsvrIVJd" + } + ``` + +1. Client opens a browser window for the fallback endpoint: + + ``` + GET /_matrix/client/r0/auth/m.login.sso/fallback/web + ?session=dTKfsLHSAJeAhqfxUsvrIVJd HTTP/1.1 + + HTTP/1.1 200 OK + + + A client is trying to remove a device from your account. To confirm this + action, re-authenticate with single sign-on. + If you did not expect this, your account may be compromised! + + ``` + +2. The user clicks the confirmation link which goes to the SSO provider's site: + + ``` + GET https://sso_provider/validate?SAMLRequest= HTTP/1.1 + + + ``` + +3. The SSO provider validates the user and ends up redirecting the browser back + to the homeserver. The example below shows a 302 for simplicity, this might + vary based on SSO implementation. + + ``` + HTTP/1.1 302 Moved + Location: https://homeserver/_matrix/saml2/authn_response? + SAMLResponse= + ``` + +4. The browser sends the SSO response to the homeserver, which validates it and + shows the fallback auth completion page: + + ``` + GET /_matrix/saml2/authn_response?SAMLResponse= + + + HTTP/1.1 200 OK + + + +

Thank you.

+

You may now close this window and return to the application.

+ ``` + +5. The client closes the browser popup if it is still open, and resubmits its + original request, which now succeeds: + + ``` + POST /_matrix/client/r0/delete_devices HTTP/1.1 + Content-Type: application/json + Authorization: Bearer xyzzy + + { + "auth": { + "session": "dTKfsLHSAJeAhqfxUsvrIVJd" + } + } + + HTTP/1.1 200 OK + Content-Type: application/json + + {} + ``` + +## Alternatives + +An alternative client flow where the fallback auth ends up redirecting to a +given URI, instead of doing JavaScript `postMessage` foo could be considered. +This is probably an orthogonal change to the fallback auth though. + +## Security considerations + +### Why we need user to confirm before the SSO flow + +Recall that the thing we are trying to guard against here is a single leaked +access-token being used to take over an entire account. So let's assume the +attacker has got hold of an access token for your account. What happens if the +confirmation step is skipped? + +The attacker, who has your access token, starts a UI Authentication session to +add their email address to your account. + +They then sends you a link "hey, check out this cool video!"; the link leads (via +an innocent-looking URL shortener or some other phishing technique) to +`/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`, with the ID of +the session that he just created. + +Since there is no confirmation step, the server redirects directly to the SSO +provider. + +It's common for SSO providers to redirect straight back to the app if you've +recently authenticated with them; even in the best case, the SSO provider shows +an innocent message along the lines of "Confirm that you want to sign in to +\". + +After redirecting back to the homeserver, the SSO is completed and the +attacker's session is validated. They are now able to make their malicious +change to your account. + +This problem can be mitigated by clearly telling the user what is about to happen. + +### Reusing User Interactive Authentication sessions + +The security of this relies on User Interactive Authentication sessions only +being used for the same request as they were initiated for. This security is not +only a concern for the proposed SSO authentication type. It is not believed +that this is currently enforced in implementations. + +## Unstable prefix + +A vendor prefix of `org.matrix.login.sso` is proposed (instead of `m.login.sso`) +until this is part of the specification. diff --git a/proposals/2457-password-modification-invalidating-devices.md b/proposals/2457-password-modification-invalidating-devices.md new file mode 100644 index 00000000..ccde2ac6 --- /dev/null +++ b/proposals/2457-password-modification-invalidating-devices.md @@ -0,0 +1,56 @@ +# Invalidating devices during password modification + +There are multiple use cases for why a user might want to modify their password: + +* Adopting a password manager (to use a unique password or more secure password). +* Password rotation. +* Re-secure a compromised account. +* ... probably tons of others ... + +These can be summarized into two groups: + +1. "My account has been compromised and I need to re-secure it." +2. "I just want to change my password." + +The [current Matrix specification](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) +does not provide a way to differentiate between these use cases. It currently +specifies behavior that fits well into the first use-case above: that the +sessions except the current session should be revoked. + +It is reasonable for a client to want to specify this behavior to offer two +different workflows: + +1. Modify a password and log all other devices out (for use when an account has + been compromised). +2. Modify a password and do not touch any session data (for use in a + non-malicious situations). + +Alternately a client may default to whichever workflow is best for their users. + +## Proposal + +An optional field is added to the JSON body of the [password reset endpoint](https://matrix.org/docs/spec/client_server/r0.6.0#post-matrix-client-r0-account-password) +called `logout_devices`. This is a boolean flag (defaulting to `true`) that +signals to whether other devices and sessions should be invalidated after +modifying the password. + +## Potential issues + +The specification states: + +> The homeserver SHOULD NOT revoke the access token provided in the request, +> however all other access tokens for the user should be revoked if the request +> succeeds. + +Defaulting `logout_devices` to `true` should be backwards compatible. + +## Alternatives + +A new endpoint could be provided in a future version of the specification that +supports an additional field (as described above). + +## Security considerations + +By defaulting to invalidating devices and sessions the security considerations +of this endpoint should remain intact. A client will need to be modified to +choose to keep other devices active. diff --git a/proposals/images/1756-graph1.dot b/proposals/images/1756-graph1.dot new file mode 100644 index 00000000..8a8c9641 --- /dev/null +++ b/proposals/images/1756-graph1.dot @@ -0,0 +1,13 @@ +graph { +A1 [label="A's PDP-11"] +AV [label="A's virtual device"] +A2 [label="A's Osborne 2"] +B1 [label="B's Dynabook"] +BV [label="B's virtual device"] +B2 [label="B's VAX"] +A1 -- AV +AV -- A2 +A2 -- B1 +B1 -- BV +BV -- B2 +} \ No newline at end of file diff --git a/proposals/images/1756-graph1.dot.png b/proposals/images/1756-graph1.dot.png new file mode 100644 index 00000000..87a78f83 Binary files /dev/null and b/proposals/images/1756-graph1.dot.png differ diff --git a/proposals/images/1756-graph2.dot b/proposals/images/1756-graph2.dot new file mode 100644 index 00000000..a8074125 --- /dev/null +++ b/proposals/images/1756-graph2.dot @@ -0,0 +1,22 @@ +digraph { +A1 [label="A's PDP-11"] +A2 [label="A's Osborne 2"] +AM [label="A's master key"] +AS [label="A's self-signing key"] +AU [label="A's user-signing key"] +BM [label="B's master key"] +BU [label="B's user-signing key"] +BS [label="B's self-signing key"] +B1 [label="B's Dynabook"] +B2 [label="B's VAX"] +AS -> A1 +AS -> A2 +AM -> AS +AM -> AU +AU -> BM +BM -> BS +BM -> BU +BU -> AM +BS -> B1 +BS -> B2 +} diff --git a/proposals/images/1756-graph2.dot.png b/proposals/images/1756-graph2.dot.png new file mode 100644 index 00000000..e7591f04 Binary files /dev/null and b/proposals/images/1756-graph2.dot.png differ diff --git a/proposals/images/2244-redaction-spam.png b/proposals/images/2244-redaction-spam.png new file mode 100644 index 00000000..386d9576 Binary files /dev/null and b/proposals/images/2244-redaction-spam.png differ diff --git a/pyproject.toml b/pyproject.toml index b53982b8..060a44fc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,9 +1,9 @@ -[ tool.giles ] +[ tool.gilesbot ] - [ tool.giles.circleci_artifacts.docs ] + [ tool.gilesbot.circleci_artifacts.docs ] url = "gen/index.html" message = "Click details to preview the HTML documentation." - [ tool.giles.circleci_artifacts.swagger ] + [ tool.gilesbot.circleci_artifacts.swagger ] url = "client-server/index.html" message = "Click to preview the swagger build." diff --git a/scripts/proposals.py b/scripts/proposals.py index 27cc6cfb..faa10a83 100755 --- a/scripts/proposals.py +++ b/scripts/proposals.py @@ -38,7 +38,10 @@ def getpage(url): pagecount = 1 for link in resp.links.values(): if link['rel'] == 'last': - pagecount = int(re.search('page=(.+?)', link['url']).group(1)) + # we extract the pagecount from the `page` param of the last url + # in the response, eg + # 'https://api.github.com/repositories/24998719/issues?state=all&labels=proposal&page=10' + pagecount = int(re.search('page=(\d+)', link['url']).group(1)) val = resp.json() if not isinstance(val, list): diff --git a/scripts/templating/matrix_templates/templates/http-api.tmpl b/scripts/templating/matrix_templates/templates/http-api.tmpl index 74836045..d2ee3ff7 100644 --- a/scripts/templating/matrix_templates/templates/http-api.tmpl +++ b/scripts/templating/matrix_templates/templates/http-api.tmpl @@ -1,11 +1,15 @@ {% import 'tables.tmpl' as tables -%} -``{{endpoint.method}} {{endpoint.path}}`` -{{(5 + (endpoint.path | length) + (endpoint.method | length)) * title_kind}} {% if "deprecated" in endpoint and endpoint.deprecated -%} +Deprecated: ``{{endpoint.method}} {{endpoint.path}}`` +{{(17 + (endpoint.path | length) + (endpoint.method | length)) * title_kind}} + .. WARNING:: This API is deprecated and will be removed from a future release. +{% else %} +``{{endpoint.method}} {{endpoint.path}}`` +{{(5 + (endpoint.path | length) + (endpoint.method | length)) * title_kind}} {% endif -%} {{endpoint.desc}} diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index 04e6f8a9..9d2c69f2 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -211,9 +211,17 @@ def get_json_schema_object_fields(obj, enforce_title=False): key_type = additionalProps.get("x-pattern", "string") res = process_data_type(additionalProps) + tables = res["tables"] + val_title = res["title"] + if res.get("enum_desc") and val_title != "enum": + # A map to enum needs another table with enum description + tables.append(TypeTable( + title=val_title, + rows=[TypeTableRow(key="(mapped value)", title="enum", desc=res["desc"])] + )) return { - "title": "{%s: %s}" % (key_type, res["title"]), - "tables": res["tables"], + "title": "{%s: %s}" % (key_type, val_title), + "tables": tables, } if not props: @@ -338,8 +346,8 @@ def process_data_type(prop, required=False, enforce_title=True): prop_title = prop_type if prop.get("enum"): + prop_title = prop.get("title", "enum") if len(prop["enum"]) > 1: - prop_title = "enum" enum_desc = ( "One of: %s" % json.dumps(prop["enum"]) ) @@ -586,7 +594,21 @@ class MatrixUnits(Units): raise Exception("Error handling parameter %s" % param_name, e) # endfor[param] good_response = None - for code in sorted(endpoint_swagger.get("responses", {}).keys()): + endpoint_status_codes = endpoint_swagger.get("responses", {}).keys() + # Check to see if any of the status codes are strings ("4xx") and if + # so convert everything to a string to avoid comparing ints and strings. + has_string_status = False + for code in endpoint_status_codes: + if isinstance(code, str): + has_string_status = True + break + if has_string_status: + endpoint_status_codes = [str(i) for i in endpoint_status_codes] + for code in sorted(endpoint_status_codes): + # Convert numeric responses to ints, assuming they got converted + # above. + if isinstance(code, str) and code.isdigit(): + code = int(code) res = endpoint_swagger["responses"][code] if not good_response and code == 200: good_response = res diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 916604a3..fce879a2 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -45,6 +45,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.6.0 `_ - `r0.5.0 `_ - `r0.4.0 `_ - `r0.3.0 `_ @@ -642,6 +643,7 @@ This specification defines the following auth types: - ``m.login.password`` - ``m.login.recaptcha`` - ``m.login.oauth2`` + - ``m.login.sso`` - ``m.login.email.identity`` - ``m.login.msisdn`` - ``m.login.token`` @@ -739,22 +741,23 @@ To use this authentication type, clients should submit an auth dict as follows: "session": "" } -The ``nonce`` should be a random string generated by the client for the -request. The same ``nonce`` should be used if retrying the request. A client may receive a login ``token`` via some external service, such as email or SMS. Note that a login token is separate from an access token, the latter providing general authentication to various API endpoints. -The ``txn_id`` may be used by the server to disallow other devices from using -the token, thus providing "single use" tokens while still allowing the device -to retry the request. This would be done by tying the token to the ``txn_id`` +Additionally, the server must encode the user ID in the ``token``; there is +therefore no need for the client to submit a separate username. + +The ``txn_id`` should be a random string generated by the client for the +request. The same ``txn_id`` should be used if retrying the request. The +``txn_id`` may be used by the server to disallow other devices from using the +token, thus providing "single use" tokens while still allowing the device to +retry the request. This would be done by tying the token to the ``txn_id`` server side, as well as potentially invalidating the token completely once the device has successfully logged in (e.g. when we receive a request from the newly provisioned access_token). -The server must encode the user id in the ``token``. There is therefore no need -for the client to submit a separate username. OAuth2-based <<<<<<<<<<<< @@ -780,6 +783,38 @@ the auth code. Homeservers can choose any path for the ``redirect URI``. Once the OAuth flow has completed, the client retries the request with the session only, as above. +Single Sign-On +<<<<<<<<<<<<<< +:Type: + ``m.login.sso`` +:Description: + Authentication is supported by authorising with an external single sign-on + provider. + +A client wanting to complete authentication using SSO should use the +`Fallback`_ authentication flow by opening a browser window for +``/_matrix/client/r0/auth/m.login.sso/fallback/web?session=<...>`` with the +session parameter set to the session ID provided by the server. + +The homeserver should return a page which asks for the user's confirmation +before proceeding. For example, the page could say words to the effect of: + + A client is trying to remove a device/add an email address/take over your + account. To confirm this action, re-authenticate with single sign-on. If you + did not expect this, your account may be compromised! + +Once the user has confirmed they should be redirected to the single sign-on +provider's login page. Once the provider has validated the user, the browser is +redirected back to the homeserver. + +The homeserver then validates the response from the single sign-on provider and +updates the user-interactive authentication session to mark the single sign-on +stage has been completed. The browser is shown the fallback authentication +completion page. + +Once the flow has completed, the client retries the request with the session +only, as above. + Email-based (identity / homeserver) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< :Type: @@ -802,12 +837,16 @@ To use this authentication type, clients should submit an auth dict as follows: { "sid": "", "client_secret": "", - "id_server": "" + "id_server": "", + "id_access_token": "" } ], "session": "" } +Note that ``id_server`` (and therefore ``id_access_token``) is optional if the +``/requestToken`` request did not include them. + Phone number/MSISDN-based (identity / homeserver) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< :Type: @@ -830,12 +869,16 @@ To use this authentication type, clients should submit an auth dict as follows: { "sid": "", "client_secret": "", - "id_server": "" + "id_server": "", + "id_access_token": "" } ], "session": "" } +Note that ``id_server`` (and therefore ``id_access_token``) is optional if the +``/requestToken`` request did not include them. + Dummy Auth <<<<<<<<<< :Type: @@ -1034,6 +1077,9 @@ wishes to canonicalise the phone number, then it can use the "phone": "" } +The ``country`` is the two-letter uppercase ISO-3166-1 alpha-2 country code +that the number in ``phone`` should be parsed as if it were dialled from. + Login ~~~~~ @@ -1130,7 +1176,15 @@ Adding Account Administrative Contact Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A homeserver may keep some contact information for administrative use. -This is independent of any information kept by any identity servers. +This is independent of any information kept by any identity servers, though +can be proxied (bound) to the identity server in many cases. + +.. Note:: + This section deals with two terms: "add" and "bind". Where "add" (or "remove") + is used, it is speaking about an identifier that was not bound to an identity + server. As a result, "bind" (or "unbind") references an identifier that is found + in an identity server. Note that an identifer can be added and bound at the same + time, depending on context. {{administrative_contact_cs_http_api}} @@ -1139,6 +1193,41 @@ Current account information {{whoami_cs_http_api}} +Notes on identity servers ++++++++++++++++++++++++++ + +Identity servers in Matrix store bindings (relationships) between a user's third +party identifier, typically email or phone number, and their user ID. Once a user +has chosen an identity server, that identity server should be used by all clients. + +Clients can see which identity server the user has chosen through the ``m.identity_server`` +account data event, as described below. Clients SHOULD refrain from making requests +to any identity server until the presence of ``m.identity_server`` is confirmed as +(not) present. If present, the client SHOULD check for the presence of the ``base_url`` +property in the event's content. If the ``base_url`` is present, the client SHOULD +use the identity server in that property as the identity server for the user. If the +``base_url`` is missing, or the account data event is not present, the client SHOULD +use whichever default value it normally would for an identity server, if applicable. +Clients SHOULD NOT update the account data with the default identity server when the +user is missing an identity server in their account data. + +Clients SHOULD listen for changes to the ``m.identity_server`` account data event +and update the identity server they are contacting as a result. + +If the client offers a way to set the identity server to use, it MUST update the +value of ``m.identity_server`` accordingly. A ``base_url`` of ``null`` MUST be +treated as though the user does not want to use an identity server, disabling all +related functionality as a result. + +Clients SHOULD refrain from populating the account data as a migration step for users +who are lacking the account data, unless the user sets the identity server within +the client to a value. For example, a user which has no ``m.identity_server`` account +data event should not end up with the client's default identity server in their +account data, unless the user first visits their account settings to set the identity +server. + +{{m_identity_server_event}} + Capabilities negotiation ------------------------ diff --git a/specification/feature_profiles.rst b/specification/feature_profiles.rst index bb638380..8a3caf87 100644 --- a/specification/feature_profiles.rst +++ b/specification/feature_profiles.rst @@ -61,6 +61,7 @@ Summary `Stickers`_ Optional Optional Optional Optional Optional `Server ACLs`_ Optional Optional Optional Optional Optional `Server Notices`_ Optional Optional Optional Optional Optional + `Moderation policies`_ Optional Optional Optional Optional Optional ===================================== ========== ========== ========== ========== ========== *Please see each module for more details on what clients need to implement.* @@ -94,6 +95,7 @@ Summary .. _Stickers: `module:stickers`_ .. _Server ACLs: `module:server-acls`_ .. Server Notices already has a link elsewhere. +.. _Moderation Policies: `module:moderation-policies`_ Clients ------- diff --git a/specification/identity_service_api.rst b/specification/identity_service_api.rst index 89604644..704f763e 100644 --- a/specification/identity_service_api.rst +++ b/specification/identity_service_api.rst @@ -51,6 +51,8 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.3.0 `_ +- `r0.2.1 `_ - `r0.2.0 `_ - `r0.1.0 `_ @@ -155,6 +157,23 @@ should allow a 3PID to be mapped to a Matrix user identity, but not in the other direction (i.e. one should not be able to get all 3PIDs associated with a Matrix user ID, or get all 3PIDs associated with a 3PID). +Version 1 API deprecation +------------------------- + +.. TODO: Remove this section when the v1 API is removed. + +As described on each of the version 1 endpoints, the v1 API is deprecated in +favour of the v2 API described here. The major difference, with the exception +of a few isolated cases, is that the v2 API requires authentication to ensure +the user has given permission for the identity server to operate on their data. + +The v1 API is planned to be removed from the specification in a future version. + +Clients SHOULD attempt the v2 endpoints first, and if they receive a ``404``, +``400``, or similar error they should try the v1 endpoint or fail the operation. +Clients are strongly encouraged to warn the user of the risks in using the v1 API, +if they are planning on using it. + Web browser clients ------------------- @@ -171,11 +190,67 @@ to be returned by servers on all requests are:: Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization +Authentication +-------------- + +Most ``v2`` endpoints in the Identity Service API require authentication in order +to ensure that the requesting user has accepted all relevant policies and is otherwise +permitted to make the request. The ``v1`` API (currently deprecated) does not require +this authentication, however using ``v1`` is strongly discouraged as it will be removed +in a future release. + +Identity Servers use a scheme similar to the Client-Server API's concept of access +tokens to authenticate users. The access tokens provided by an Identity Server cannot +be used to authenticate Client-Server API requests. + +An access token is provided to an endpoint in one of two ways: + +1. Via a query string parameter, ``access_token=TheTokenHere``. +2. Via a request header, ``Authorization: Bearer TheTokenHere``. + +Clients are encouraged to the use the ``Authorization`` header where possible to prevent +the access token being leaked in access/HTTP logs. The query string should only be used +in cases where the ``Authorization`` header is inaccessible for the client. + +When credentials are required but missing or invalid, the HTTP call will return with a +status of 401 and the error code ``M_UNAUTHORIZED``. + +{{v2_auth_is_http_api}} + + +.. _`agree to more terms`: + +Terms of service +---------------- + +Identity Servers are encouraged to have terms of service (or similar policies) to +ensure that users have agreed to their data being processed by the server. To facilitate +this, an identity server can respond to almost any authenticated API endpoint with a +HTTP 403 and the error code ``M_TERMS_NOT_SIGNED``. The error code is used to indicate +that the user must accept new terms of service before being able to continue. + +All endpoints which support authentication can return the ``M_TERMS_NOT_SIGNED`` error. +When clients receive the error, they are expected to make a call to ``GET /terms`` to +find out what terms the server offers. The client compares this to the ``m.accepted_terms`` +account data for the user (described later) and presents the user with option to accept +the still-missing terms of service. After the user has made their selection, if applicable, +the client sends a request to ``POST /terms`` to indicate the user's acceptance. The +server cannot expect that the client will send acceptance for all pending terms, and the +client should not expect that the server will not respond with another ``M_TERMS_NOT_SIGNED`` +on their next request. The terms the user has just accepted are appended to ``m.accepted_terms``. + +{{m_accepted_terms_event}} + +{{v2_terms_is_http_api}} + + Status check ------------ {{ping_is_http_api}} +{{v2_ping_is_http_api}} + Key management -------------- @@ -195,11 +270,142 @@ service's long-term keys. {{pubkey_is_http_api}} +{{v2_pubkey_is_http_api}} + Association lookup ------------------ {{lookup_is_http_api}} +{{v2_lookup_is_http_api}} + +Client behaviour +~~~~~~~~~~~~~~~~ + +.. TODO: Remove this note when v1 is removed completely +.. Note:: + This section only covers the v2 lookup endpoint. The v1 endpoint is described + in isolation above. + +Prior to performing a lookup clients SHOULD make a request to the ``/hash_details`` +endpoint to determine what algorithms the server supports (described in more detail +below). The client then uses this information to form a ``/lookup`` request and +receive known bindings from the server. + +Clients MUST support at least the ``sha256`` algorithm. + +Server behaviour +~~~~~~~~~~~~~~~~ + +.. TODO: Remove this note when v1 is removed completely +.. Note:: + This section only covers the v2 lookup endpoint. The v1 endpoint is described + in isolation above. + +Servers, upon receipt of a ``/lookup`` request, will compare the query against +known bindings it has, hashing the identifiers it knows about as needed to +verify exact matches to the request. + +Servers MUST support at least the ``sha256`` algorithm. + +Algorithms +~~~~~~~~~~ + +Some algorithms are defined as part of the specification, however other formats +can be negotiated between the client and server using ``/hash_details``. + +``sha256`` +++++++++++ + +This algorithm MUST be supported by clients and servers at a minimum. It is +additionally the preferred algorithm for lookups. + +When using this algorithm, the client converts the query first into strings +separated by spaces in the format ``
``. The ```` +is retrieved from ``/hash_details``, the ```` is typically ``email`` or +``msisdn`` (both lowercase), and the ``
`` is the 3PID to search for. +For example, if the client wanted to know about ``alice@example.org``'s bindings, +it would first format the query as ``alice@example.org email ThePepperGoesHere``. + +.. admonition:: Rationale + + Mediums and peppers are appended to the address to prevent a common prefix + for each 3PID, helping prevent attackers from pre-computing the internal state + of the hash function. + +After formatting each query, the string is run through SHA-256 as defined by +`RFC 4634 `_. The resulting bytes are then +encoded using URL-Safe `Unpadded Base64`_ (similar to `room version 4's +event ID format <../rooms/v4.html#event-ids>`_). + +An example set of queries when using the pepper ``matrixrocks`` would be:: + + "alice@example.com email matrixrocks" -> "4kenr7N9drpCJ4AfalmlGQVsOn3o2RHjkADUpXJWZUc" + "bob@example.com email matrixrocks" -> "LJwSazmv46n0hlMlsb_iYxI0_HXEqy_yj6Jm636cdT8" + "18005552067 msisdn matrixrocks" -> "nlo35_T5fzSGZzJApqu8lgIudJvmOQtDaHtr-I4rU7I" + + +The set of hashes is then given as the ``addresses`` array in ``/lookup``. Note +that the pepper used MUST be supplied as ``pepper`` in the ``/lookup`` request. + +``none`` +++++++++ + +This algorithm performs plaintext lookups on the identity server. Typically this +algorithm should not be used due to the security concerns of unhashed identifiers, +however some scenarios (such as LDAP-backed identity servers) prevent the use of +hashed identifiers. Identity servers (and optionally clients) can use this algorithm +to perform those kinds of lookups. + +Similar to the ``sha256`` algorithm, the client converts the queries into strings +separated by spaces in the format ``
`` - note the lack of ````. +For example, if the client wanted to know about ``alice@example.org``'s bindings, +it would format the query as ``alice@example.org email``. + +The formatted strings are then given as the ``addresses`` in ``/lookup``. Note that +the ``pepper`` is still required, and must be provided to ensure the client has made +an appropriate request to ``/hash_details`` first. + +Security considerations +~~~~~~~~~~~~~~~~~~~~~~~ + +.. Note:: + `MSC2134 `_ has much more + information about the security considerations made for this section of the + specification. This section covers the high-level details for why the specification + is the way it is. + +Typically the lookup endpoint is used when a client has an unknown 3PID it wants to +find a Matrix User ID for. Clients normally do this kind of lookup when inviting new +users to a room or searching a user's address book to find any Matrix users they may +not have discovered yet. Rogue or malicious identity servers could harvest this +unknown information and do nefarious things with it if it were sent in plain text. +In order to protect the privacy of users who might not have a Matrix identifier bound +to their 3PID addresses, the specification attempts to make it difficult to harvest +3PIDs. + +.. admonition:: Rationale + + Hashing identifiers, while not perfect, helps make the effort required to harvest + identifiers significantly higher. Phone numbers in particular are still difficult + to protect with hashing, however hashing is objectively better than not. + + An alternative to hashing would be using bcrypt or similar with many rounds, however + by nature of needing to serve mobile clients and clients on limited hardware the + solution needs be kept relatively lightweight. + +Clients should be cautious of servers not rotating their pepper very often, and +potentially of servers which use a weak pepper - these servers may be attempting to +brute force the identifiers or use rainbow tables to mine the addresses. Similarly, +clients which support the ``none`` algorithm should consider at least warning the user +of the risks in sending identifiers in plain text to the identity server. + +Addresses are still potentially reversable using a calculated rainbow table given +some identifiers, such as phone numbers, common email address domains, and leaked +addresses are easily calculated. For example, phone numbers can have roughly 12 +digits to them, making them an easier target for attack than email addresses. + + Establishing associations ------------------------- @@ -243,16 +449,22 @@ Email associations {{email_associations_is_http_api}} +{{v2_email_associations_is_http_api}} + Phone number associations ~~~~~~~~~~~~~~~~~~~~~~~~~ {{phone_associations_is_http_api}} +{{v2_phone_associations_is_http_api}} + General ~~~~~~~ {{associations_is_http_api}} +{{v2_associations_is_http_api}} + Invitation storage ------------------ @@ -267,6 +479,8 @@ long-term private key for the identity server. {{store_invite_is_http_api}} +{{v2_store_invite_is_http_api}} + Ephemeral invitation signing ---------------------------- @@ -277,6 +491,8 @@ this isn't possible. {{invitation_signing_is_http_api}} +{{v2_invitation_signing_is_http_api}} + .. _`Unpadded Base64`: ../appendices.html#unpadded-base64 .. _`3PID Types`: ../appendices.html#pid-types .. _`Signing JSON`: ../appendices.html#signing-json diff --git a/specification/index.rst b/specification/index.rst index bf554e4a..728383ab 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -106,7 +106,7 @@ The principles that Matrix attempts to follow are: - Empowering the end-user + The user should be able to choose the server and clients they use - + The user should be control how private their communication is + + The user should be able to control how private their communication is + The user should know precisely where their data is stored - Fully decentralised - no single points of control over conversations or the @@ -540,7 +540,7 @@ some other reason. Versions can switch between stable and unstable periodically for a variety of reasons, including discovered security vulnerabilities and age. Clients should not ask room administrators to upgrade their rooms if the room is -running a stable version. Servers SHOULD use room version 4 as the default room +running a stable version. Servers SHOULD use room version 5 as the default room version when creating new rooms. The available room versions are: diff --git a/specification/modules/content_repo.rst b/specification/modules/content_repo.rst index 1e3d1866..192250d2 100644 --- a/specification/modules/content_repo.rst +++ b/specification/modules/content_repo.rst @@ -19,7 +19,7 @@ Content repository .. _module:content: The content repository (or "media repository") allows users to upload -files to their homeserver for later user. For example, files which the +files to their homeserver for later use. For example, files which the user wants to send to a room would be uploaded here, as would an avatar the user wants to use. diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 329c0170..0223d9f5 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -277,7 +277,7 @@ Parameter Type Description ========= ================ ===================================================== url string **Required.** The URL to the file. key JWK **Required.** A `JSON Web Key`_ object. -iv string **Required.** The Initialisation Vector used by +iv string **Required.** The 128-bit unique counter block used by AES-CTR, encoded as unpadded base64. hashes {string: string} **Required.** A map from an algorithm name to a hash of the ciphertext, encoded as unpadded base64. Clients @@ -505,7 +505,7 @@ framework outlined above. SAS verification is intended to be a highly interactiv process for users, and as such exposes verfiication methods which are easier for users to use. -The verification process is heavily inspired by Phil Zimmerman's ZRTP key agreement +The verification process is heavily inspired by Phil Zimmermann's ZRTP key agreement handshake. A key part of key agreement in ZRTP is the hash commitment: the party that begins the Diffie-Hellman key sharing sends a hash of their part of the Diffie-Hellman exchange, and does not send their part of the Diffie-Hellman exchange until they have @@ -565,9 +565,9 @@ The process between Alice and Bob verifying each other would be: they match or not. #. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). + to verify, using SHA-256 as the hash function. HMAC is defined in `RFC 2104 `_. The key for the HMAC is different for each item and is calculated by generating - 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. + 32 bytes (256 bits) using `the key verification HKDF <#sas-hkdf>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does the same for Bob's device keys and key IDs concurrently with Alice. @@ -653,14 +653,14 @@ are used in addition to those already specified: {{m_key_verification_mac_event}} -.. _`SAS-HKDF`: +.. _sas-hkdf: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) +In all of the SAS methods, HKDF is as defined in `RFC 5869 `_ and uses the previously agreed-upon hash function for the hash function. The shared -secret is supplied as the input keying material. No salt is used, and the input +secret is supplied as the input keying material. No salt is used, and the info parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_SAS``. @@ -677,7 +677,7 @@ parameter is the concatenation of: For verification of each party's device keys, HKDF is as defined in RFC 5869 and uses SHA-256 as the hash function. The shared secret is supplied as the input keying -material. No salt is used, and in the input parameter is the concatenation of: +material. No salt is used, and in the info parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_MAC``. * The Matrix ID of the user whose key is being MAC-ed. @@ -691,7 +691,7 @@ material. No salt is used, and in the input parameter is the concatenation of: SAS method: ``decimal`` <<<<<<<<<<<<<<<<<<<<<<< -Generate 5 bytes using `HKDF <#SAS-HKDF>`_ then take sequences of 13 bits to +Generate 5 bytes using `HKDF <#sas-hkdf>`_ then take sequences of 13 bits to convert to decimal numbers (resulting in 3 numbers between 0 and 8191 inclusive each). Add 1000 to each calculated number. @@ -708,7 +708,7 @@ such as dashes, or with the numbers on individual lines. SAS method: ``emoji`` <<<<<<<<<<<<<<<<<<<<< -Generate 6 bytes using `HKDF <#SAS-HKDF>`_ then split the first 42 bits into +Generate 6 bytes using `HKDF <#sas-hkdf>`_ then split the first 42 bits into 7 groups of 6 bits, similar to how one would base64 encode something. Convert each group of 6 bits to a number and use the following table to get the corresponding emoji: @@ -756,8 +756,8 @@ sending `m.room_key_request`_ to-device messages to other devices with device, it can forward the keys to the first device by sending an encrypted `m.forwarded_room_key`_ to-device message. The first device should then send an `m.room_key_request`_ to-device message with ``action`` set to -``cancel_request`` to the other devices that it had originally sent the key -request to; a device that receives a ``cancel_request`` should disregard any +``request_cancellation`` to the other devices that it had originally sent the key +request to; a device that receives a ``request_cancellation`` should disregard any previously-received ``request`` message with the same ``request_id`` and ``requesting_device_id``. @@ -844,24 +844,22 @@ Example: .. code:: json - { - "sessions": [ - { - "algorithm": "m.megolm.v1.aes-sha2", - "forwarding_curve25519_key_chain": [ - "hPQNcabIABgGnx3/ACv/jmMmiQHoeFfuLB17tzWp6Hw" - ], - "room_id": "!Cuyf34gef24t:localhost", - "sender_key": "RF3s+E7RkTQTGF2d8Deol0FkQvgII2aJDf3/Jp5mxVU", - "sender_claimed_keys": { - "ed25519": "", - }, - "session_id": "X3lUlvLELLYxeTx4yOVu6UDpasGEVO0Jbu+QFnm0cKQ", - "session_key": "AgAAAADxKHa9uFxcXzwYoNueL5Xqi69IkD4sni8Llf..." + [ + { + "algorithm": "m.megolm.v1.aes-sha2", + "forwarding_curve25519_key_chain": [ + "hPQNcabIABgGnx3/ACv/jmMmiQHoeFfuLB17tzWp6Hw" + ], + "room_id": "!Cuyf34gef24t:localhost", + "sender_key": "RF3s+E7RkTQTGF2d8Deol0FkQvgII2aJDf3/Jp5mxVU", + "sender_claimed_keys": { + "ed25519": "", }, - ... - ] - } + "session_id": "X3lUlvLELLYxeTx4yOVu6UDpasGEVO0Jbu+QFnm0cKQ", + "session_key": "AgAAAADxKHa9uFxcXzwYoNueL5Xqi69IkD4sni8Llf..." + }, + ... + ] Messaging Algorithms -------------------- diff --git a/specification/modules/moderation_policies.rst b/specification/modules/moderation_policies.rst new file mode 100644 index 00000000..2e85fb07 --- /dev/null +++ b/specification/modules/moderation_policies.rst @@ -0,0 +1,128 @@ +.. Copyright 2020 The Matrix.org Foundation C.I.C. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Moderation policy lists +======================= + +.. _module:moderation-policies: + +With Matrix being an open network where anyone can participate, a very wide +range of content exists and it is important that users are empowered to select +which content they wish to see, and which content they wish to block. By +extension, room moderators and server admins should also be able to select +which content they do not wish to host in their rooms and servers. + +The protocol's position on this is one of neutrality: it should not be deciding +what content is undesirable for any particular entity and should instead be +empowering those entities to make their own decisions. As such, a generic +framework for communicating "moderation policy lists" or "moderation policy rooms" +is described. Note that this module only describes the data structures and not +how they should be interpreting: the entity making the decisions on filtering +is best positioned to interpret the rules how it sees fit. + +Moderation policy lists are stored as room state events. There are no restrictions +on how the rooms can be configured (they could be public, private, encrypted, etc). + +There are currently 3 kinds of entities which can be affected by rules: ``user``, +``server``, and ``room``. All 3 are described with ``m.policy.rule.`` state +events. The ``state_key`` for a policy rule is an arbitrary string decided by the +sender of the rule. + +Rules contain recommendations and reasons for the rule existing. The ``reason`` +is a human-readable string which describes the ``recommendation``. Currently only +one recommendation, ``m.ban``, is specified. + +``m.ban`` recommendation +------------------------ + +When this recommendation is used, the entities affected by the rule should be +banned from participation where possible. The enforcement of this is deliberately +left as an implementation detail to avoid the protocol imposing its opinion on how +the policy list is to be interpreted. However, a suggestion for a simple implementation +is as follows: + +* Is a ``user`` rule... + + * Applied to a user: The user should be added to the subscriber's ignore list. + * Applied to a room: The user should be banned from the room (either on sight or immediately). + * Applied to a server: The user should not be allowed to send invites to users on the server. + +* Is a ``room`` rule... + + * Applied to a user: The user should leave the room and not join it + (`MSC2270 `_-style ignore). + * Applied to a room: No-op because a room cannot ban itself. + * Applied to a server: The server should prevent users from joining the room and from receiving + invites to it. + +* Is a ``server`` rule... + + * Applied to a user: The user should not receive events or invites from the server. + * Applied to a room: The server is added as a denied server in the ACLs. + * Applied to a server: The subscriber should avoid federating with the server as much as + possible by blocking invites from the server and not sending traffic unless strictly + required (no outbound invites). + +Subscribing to policy lists +--------------------------- + +This is deliberatly left as an implementation detail. For implementations using the +Client-Server API, this could be as easy as joining or peeking the room. Joining or peeking +is not required, however: an implementation could poll for updates or use a different +technique for receiving updates to the policy's rules. + +Sharing +------- + +In addition to sharing a direct reference to the room which contains the policy's rules, +plain http or https URLs can be used to share links to the list. When the URL is approached +with a ``Accept: application/json`` header or has ``.json`` appended to the end of the URL, it +should return a JSON object containing a ``room_uri`` property which references the room. +Currently this would be a ``matrix.to`` URI, however in future it could be a Matrix-schemed +URI instead. When not approached with the intent of JSON, the service could return a +user-friendly page describing what is included in the ban list. + +Events +------ + +The ``entity`` described by the state events can contain ``*`` and ``?`` to match zero or more +and one or more characters respectively. Note that rules against rooms can describe a room ID +or room alias - the subscriber is responsible for resolving the alias to a room ID if desired. + +{{m_policy_rule_user_event}} + +{{m_policy_rule_room_event}} + +{{m_policy_rule_server_event}} + +Client behaviour +---------------- +As described above, the client behaviour is deliberatly left undefined. + +Server behaviour +---------------- +Servers have no additional requirements placed on them by this module. + +Security considerations +----------------------- +This module could be used to build a system of shared blacklists, which may create +a divide within established communities if not carefully deployed. This may well not +be a suitable solution for all communities. + +Depending on how implementations handle subscriptions, user IDs may be linked to +policy lists and therefore expose the views of that user. For example, a client implementation +which joins the user to the policy room would expose the user's ID to observers of the +policy room. In future, `MSC1228 `_ +and `MSC1777 `_ (or similar) could +help solve this concern. diff --git a/specification/modules/push.rst b/specification/modules/push.rst index c9489987..ebf1aef2 100644 --- a/specification/modules/push.rst +++ b/specification/modules/push.rst @@ -473,6 +473,9 @@ Definition (as a ``content`` rule): { "set_tweak": "sound", "value": "default" + }, + { + "set_tweak": "highlight" } ] } diff --git a/specification/modules/third_party_invites.rst b/specification/modules/third_party_invites.rst index 282b06f4..04c3b180 100644 --- a/specification/modules/third_party_invites.rst +++ b/specification/modules/third_party_invites.rst @@ -255,4 +255,4 @@ these is left to the implementer's discretion. -.. _`identity server /isvalid`: ../identity_service/%IDENTITY_RELEASE_LABEL%.html#get-matrix-identity-api-v1-pubkey-isvalid +.. _`identity server /isvalid`: ../identity_service/%IDENTITY_RELEASE_LABEL%.html#get-matrix-identity-v2-pubkey-isvalid diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 20cfd48d..771d2d6a 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -79,7 +79,7 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture -Please see [MSC1779](https://github.com/matrix-org/matrix-doc/blob/matthew/msc1779/proposals/1779-open-governance.md) +Please `see MSC1779 `_ for full details of the project's Guiding Principles. Technical notes @@ -341,6 +341,35 @@ Closed proposal-closed A proposal which Obsolete obsolete A proposal which has been made obsolete by another proposal or decision elsewhere. =============================== ============================= ==================================== +Categories +---------- + +We use category labels on MSCs to place them into a track of work. The Spec Core Team +decides which of the tracks they are focusing on for the next while and generally makes +an effort to pull MSCs out of that category when possible. + +The current categories are: + +============ ================= ====================================== +Name Github Label Description +============ ================= ====================================== +Core kind:core Important for the protocol's success. +Feature kind:feature Nice to have additions to the spec. +Maintenance kind:maintenance Fixes or clarifies existing spec. +============ ================= ====================================== + +Some examples of core MSCs would be aggregations, cross-signing, and groups/communities. +These are the sorts of things that if not implemented could cause the protocol to +fail or become second-class. Features would be areas like enhanced media APIs, +new transports, and bookmarks in comparison. Finally, maintenance MSCs would include +improving error codes, clarifying what is required of an API, and adding properties +to an API which makes it easier to use. + +The Spec Core Team assigns a category to each MSC based on the descriptions above. +This can mean that new MSCs get categorized into an area the team isn't focused on, +though that can always change as priorities evolve. We still encourage that MSCs be +opened, even if not the focus for the time being, as they can still make progress and +even be merged without the Spec Core Team focusing on them specifically. Proposal Tracking ----------------- diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index a720b41a..10eee54d 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -94,7 +94,7 @@ results of the resolution so far. passes authentication in :math:`R` and add it to :math:`R`. A *conflict* occurs between states where those states have different -``event_ids`` for the same ``(state_type, state_key)``. The events thus +``event_ids`` for the same ``(event_type, state_key)``. The events thus affected are said to be *conflicting* events. diff --git a/specification/rooms/v2.rst b/specification/rooms/v2.rst index 4e8365bf..afc114f8 100644 --- a/specification/rooms/v2.rst +++ b/specification/rooms/v2.rst @@ -148,8 +148,8 @@ The *resolution* of a set of states is obtained as follows: 1. Take all *power events* and any events in their auth chains, recursively, that appear in the *full conflicted set* and order them by the *reverse topological power ordering*. -2. Apply the *iterative auth checks algorithm* on the *unconflicted state map* - and the list of events from the previous step to get a partially resolved +2. Apply the *iterative auth checks algorithm*, starting from the *unconflicted state map*, + to the list of events from the previous step to get a partially resolved state. 3. Take all remaining events that weren't picked in step 1 and order them by the mainline ordering based on the power level in the partially resolved diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 62f6916a..655a8cfc 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -316,8 +316,8 @@ Example python code: "destination": destination_name, } - if content_json is not None: - request["content"] = content + if content is not None: + request_json["content"] = content signed_json = sign_json(request_json, origin_name, origin_signing_key) @@ -391,28 +391,9 @@ creating a new event in this room should populate the new event's | E4 -.. _`auth events selection`: - -The ``auth_events`` field of a PDU identifies the set of events which give the -sender permission to send the event. The ``auth_events`` for the -``m.room.create`` event in a room is empty; for other events, it should be the -following subset of the room state: - -- The ``m.room.create`` event. -- The current ``m.room.power_levels`` event, if any. -- The sender's current ``m.room.member`` event, if any. -- If type is ``m.room.member``: - - - The target's current ``m.room.member`` event, if any. - - If ``membership`` is ``join`` or ``invite``, the current - ``m.room.join_rules`` event, if any. - - If membership is ``invite`` and ``content`` contains a - ``third_party_invite`` property, the current - ``m.room.third_party_invite`` event with ``state_key`` matching - ``content.third_party_invite.signed.token``, if any. - For a full schema of what a PDU looks like, see the `room version specification`_. + Checks performed on receipt of a PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -466,6 +447,29 @@ specified above. Each room version can have a different algorithm for how the rules work, and which rules are applied. For more detailed information, please see the `room version specification`_. + +Auth events selection +^^^^^^^^^^^^^^^^^^^^^ + +The ``auth_events`` field of a PDU identifies the set of events which give the +sender permission to send the event. The ``auth_events`` for the +``m.room.create`` event in a room is empty; for other events, it should be the +following subset of the room state: + +- The ``m.room.create`` event. +- The current ``m.room.power_levels`` event, if any. +- The sender's current ``m.room.member`` event, if any. +- If type is ``m.room.member``: + + - The target's current ``m.room.member`` event, if any. + - If ``membership`` is ``join`` or ``invite``, the current + ``m.room.join_rules`` event, if any. + - If membership is ``invite`` and ``content`` contains a + ``third_party_invite`` property, the current + ``m.room.third_party_invite`` event with ``state_key`` matching + ``content.third_party_invite.signed.token``, if any. + + Rejection +++++++++ @@ -1073,7 +1077,6 @@ The following endpoint prefixes MUST be protected: * ``/_matrix/federation/v1/state_ids`` * ``/_matrix/federation/v1/backfill`` * ``/_matrix/federation/v1/event_auth`` -* ``/_matrix/federation/v1/query_auth`` * ``/_matrix/federation/v1/get_missing_events`` diff --git a/specification/targets.yaml b/specification/targets.yaml index 2cd911ba..4e0b068d 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -91,6 +91,7 @@ groups: # reusable blobs of files when prefixed with 'group:' - modules/mentions.rst - modules/room_upgrades.rst - modules/server_notices.rst + - modules/moderation_policies.rst title_styles: ["=", "-", "~", "+", "^", "`", "@", ":"]