From 8521c2d6963e878c60a91ffa5ab3d03c245e1cfa Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 13 Nov 2018 11:31:41 -0500 Subject: [PATCH 001/282] initial proposal for key verification methods --- proposals/xxxx-key_verification.md | 126 +++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 proposals/xxxx-key_verification.md diff --git a/proposals/xxxx-key_verification.md b/proposals/xxxx-key_verification.md new file mode 100644 index 00000000..b71fcf34 --- /dev/null +++ b/proposals/xxxx-key_verification.md @@ -0,0 +1,126 @@ +# Key verification mechanisms + +Key verification is an essential part of ensuring that end-to-end encrypted +messages are secure. Matrix may support multiple verification methods that +require sending events; in fact, two such methods have already been proposed. + +This proposal tries to present a common framework for verification methods to +use, and presents a way to request key verification. + +## Proposal + +Each key verification method is identified by a name. Verification method +names defined in the Matrix spec will begin with `m.`, and verification method +names that are not defined in the Matrix spec must be namespaced following the +Java package naming convention. + +If Alice wants to verify keys with Bob, Alice's device may send `to_device` +events to Bob's devices with the `type` set to `m.key.verification.request`, as +described below. The event lists the verification methods that Alice's device +supports. Upon receipt of this message, Bob's client should prompt him to +verify keys with Alice using one of the applicable methods. In order to avoid +displaying stale key verification prompts, if Bob does not interact with the +prompt, it should be automatically hidden 10 minutes after the message is sent +(according to the `timestamp` field), or 2 minutes after the client receives +the message, whichever comes first. The prompt should also be hidden if an +appropriate `m.key.verification.cancel` message is received. If Bob chooses to +reject the key verification request, Bob's client should send a +`m.key.verification.cancel` message to Alice's device. If Bob's client does +not understand any of the methods offered, it should display a message to Bob +saying so. + +To initiate a key verification process, Bob's device sends a `to_device` event +to one of Alice's devices with the `type` set to `m.key.verification.start`. +This may either be done in response to an `m.key.verification.request` message, +or can be done independently. If Alice's device receives an +`m.key.verification.start` message in response to an +`m.key.verification.request` message, it should send an +`m.key.verification.cancel` message to Bob's other devices that it had +originally sent an `m.key.verification.request` to, in order to cancel the key +verification request. + +Verification methods can define other events required to complete the +verification. Event types for verification methods defined in the Matrix spec +should be in the `m.key.verification` namespace. Event types that are not +defined in the Matrix spec must be namespaced following the Java package naming +convention. + +Alice's or Bob's devices can cancel a key verification process or a key +verification request by sending a `to_device` event with `type` set to +`m.key.verification.cancel`. + +### Event Definitions + +#### `m.key.verification.request` + +Requests a key verification. + +Properties: + +- `from_device` (string): the device ID of the device requesting verification. +- `transaction_id` (string): an identifier for the verification request. Must + be unique with respect to the pair of devices. +- `methods` ([string]): the verification methods supported by the sender. +- `timestamp` (integer): the time when the request was made. If the timestamp + is in the future (by more than 5 minutes, to allow for clock skew), or more + than 10 minutes in the past, then the message must be ignored. + +#### `m.key.verification.start` + +Begins a key verification process. + +Properties: + +- `method` (string): the verification method to use. +- `from_device` (string): The device ID of the device starting the verification + process. +- `transaction_id` (string): an identifier for the verification process. If + this message is sent in reponse to an `m.key.verification.request` event, then + it must use the same `transaction_id` as the one given in the + `m.key.verification.request`. + +Key verification methods can define additional properties to be included. + +#### `m.key.verification.cancel` + +Cancels a key verification process or a key verification request. Upon +receiving an `m.key.verification.cancel` message, the receiving device must +cancel the verification or the request. If it is a verification process that +is cancelled, or a verification request initiated by the recipient of the +cancellation message, the device should inform the user of the reason. + +Properties: + +- `transaction_id` (string): the identifier for the request or key verification + to cancel. +- `code` (string): machine-readable reason for cancelling. Possible reasons + are: + - `m.user`: the user cancelled the verification. + - `m.timeout`: the verification process has timed out. Different verification + methods may define their own timeouts. + - `m.unknown_transaction`: the device does not know about the given transaction + ID. + - `m.unknown_method`: the device does not know how to handle the given method. + This can be sent in response to an `m.key.verification.start` message, or + can be sent in response to other verification method-specific messages. + - `m.unexpected_message`: the device received an unexpected message. For + example, a message for a verification method may have been received when it + was not expected. +- `reason` (string): human-readable reason for cancelling. This should only be + used if the recieving client does not understand the code given. + +Verification methods may define their own additional cancellation codes. +Cancellation codes defined in the Matrix spec will begin with `m.`; other +cancellation codes must be namespaced following the Java package naming +convention. + +## Tradeoffs + +## Potential issues + +## Security considerations + +## Conclusion + +This proposal presents common event definitions for use by key verification +methods and defines a way for users to request key verification. From ec2e02e8b5479b4734b0d8a6ed5aa8953f495a67 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 13 Nov 2018 11:45:18 -0500 Subject: [PATCH 002/282] rename to match MSC number --- proposals/{xxxx-key_verification.md => 1717-key_verification.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{xxxx-key_verification.md => 1717-key_verification.md} (100%) diff --git a/proposals/xxxx-key_verification.md b/proposals/1717-key_verification.md similarity index 100% rename from proposals/xxxx-key_verification.md rename to proposals/1717-key_verification.md From b535226d407b28709c8d126ef6852f3bb9344c10 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 13 Nov 2018 22:00:42 -0500 Subject: [PATCH 003/282] initial work on documenting olm unwedging --- proposals/xxxx-olm_unwedging.md | 45 +++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 proposals/xxxx-olm_unwedging.md diff --git a/proposals/xxxx-olm_unwedging.md b/proposals/xxxx-olm_unwedging.md new file mode 100644 index 00000000..60f45fe1 --- /dev/null +++ b/proposals/xxxx-olm_unwedging.md @@ -0,0 +1,45 @@ +# Olm unwedging + +Olm sessions sometimes get out of sync, resulting in undecryptable messages. +This proposal documents a method for devices to create a new session to replace +the broken session. + +## Proposal + +When a device receives an olm-encrypted message that it cannot decrypt, it +should assume that the olm session has become corrupted and create a new olm +session to replace it. It should then send a dummy message, using that +session, to the other party in order to inform them of the new session. To +send a dummy message, clients may send an event with type `m.dummy`, and with +empty contents. + +If the corrupted session has already been replaced, the receiving device should +do nothing, under the assumption that the message from the corrupted session +was sent before the sender was informed of the replacement session, in order to +avoid creating too many extra sessions. + +The spec currently says, "If a client has multiple sessions established with +another device, it should use the session from which it last received a +message." (the last paragraph of the [`m.olm.v1.curve25519-aes-sha2` +section](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-olm-v1-curve25519-aes-sha2)). +When comparing the time of the last received message for each session, the +client should consider only consider messages that were successfully decrypted, +and for sessions that have never received a message, it should use the creation +time of the session. The spec will be changed to read: + +> If a client has multiple sessions established with another device, it should +> use the session from which it last received and successfully decrypted a +> message. For these purposes, a session that has not received any messages +> should consider its creation time to be the time that it last received a +> message. + +## Tradeoffs + +## Potential issues + +## Security considerations + +## Conclusion + +This proposal outlines how wedged olm sessions can be replaced by a new +session. From d0bfdc13af8b4b8e5bba695c6f6bb3a2f6d7d275 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 13 Nov 2018 22:03:07 -0500 Subject: [PATCH 004/282] rename to match MSC number --- proposals/{xxxx-olm_unwedging.md => 1719-olm_unwedging.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{xxxx-olm_unwedging.md => 1719-olm_unwedging.md} (100%) diff --git a/proposals/xxxx-olm_unwedging.md b/proposals/1719-olm_unwedging.md similarity index 100% rename from proposals/xxxx-olm_unwedging.md rename to proposals/1719-olm_unwedging.md From 495df02da6efb5d821fad69abb4fb66600db20b4 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 14 Nov 2018 10:19:11 -0500 Subject: [PATCH 005/282] we don't actually know which session got stuck, so rate-limit by device --- proposals/1719-olm_unwedging.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 60f45fe1..13c4ea4b 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -13,10 +13,11 @@ session, to the other party in order to inform them of the new session. To send a dummy message, clients may send an event with type `m.dummy`, and with empty contents. -If the corrupted session has already been replaced, the receiving device should -do nothing, under the assumption that the message from the corrupted session -was sent before the sender was informed of the replacement session, in order to -avoid creating too many extra sessions. +In order to avoid creating too many extra sessions, a client should rate-limit +the number of new sessions it creates per device that it receives a message +from; the client should not create a new session with another device if it has +already created one for that given device in the past 1 hour. (TODO: is 1 hour +the right amount of time?) The spec currently says, "If a client has multiple sessions established with another device, it should use the session from which it last received a From 2b5805255564bb80a63c73dd977b487c67c9007c Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 14 Nov 2018 10:28:26 -0500 Subject: [PATCH 006/282] add note about re-requesting megolm keys --- proposals/1719-olm_unwedging.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 13c4ea4b..e696c1e2 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -19,6 +19,12 @@ from; the client should not create a new session with another device if it has already created one for that given device in the past 1 hour. (TODO: is 1 hour the right amount of time?) +Clients may wish to ask the sender of the undecryptable messages to re-send the +message. For exampe, if the undecryptable message was a megolm session, then +the client can send an +[`m.room_key_request`](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-room-key-request) +message to request that the sender re-send the key. + The spec currently says, "If a client has multiple sessions established with another device, it should use the session from which it last received a message." (the last paragraph of the [`m.olm.v1.curve25519-aes-sha2` From 4538745809d15dbdc6107466f88193f555f3a948 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:43:46 +0000 Subject: [PATCH 007/282] MSC1779: Open Governance for Matrix.org --- proposals/1779-open-governance.md | 386 ++++++++++++++++++++++++++++++ 1 file changed, 386 insertions(+) create mode 100644 proposals/1779-open-governance.md diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md new file mode 100644 index 00000000..a79d175b --- /dev/null +++ b/proposals/1779-open-governance.md @@ -0,0 +1,386 @@ +# Proposal for Open Governance of Matrix.org + +This whole document is a **work in progress** draft of a constitution proposal +for open governance for Matrix.org, and forms the basis of the first full +Articles of Association (AoA) for [The Matrix.org Foundation +C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit legal +entity incorporated to act as the neutral guardian of the Matrix decentralised +communication standard on behalf of the whole Matrix community. + +See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ +for more context. + +This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318) + +## Introduction + +Historically the core team of Matrix has been paid to work on it by the same +employer (currently New Vector; the startup incorporated to hire the core +team in Aug 2017). Whilst convenient in initially getting Matrix built, we +recognise that this could create a potential conflict of interest between the +core team’s responsibilities to neutrally support the wider Matrix.org ecosystem +versus the need for New Vector to be able to support the team, and it has always +been the plan to set up a completely neutral custodian for the standard once it +had reached sufficient maturity. + +This proposal seeks to establish a new open governance process for Matrix.org, +such that once the specification has finally been ‘born’ and reached an initial +‘r0’ release across all APIs, control of Matrix.org can be decoupled from New +Vector and better support contributions from the whole ecosystem. + +The concepts here are somewhat inspired by [Rust’s Governance +Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust- +governance.md); a highly regarded solution to a similar problem: an ambitious +open-source project which has been too many years in the making, incubated at +first by a single company (Mozilla Corporation), which also enjoys a very +enthusiastic community! + +## Overview + +Governance of the project is split into two teams: the Spec Core Team and the +Guardians of the Foundation. In brief: + +The Spec Core Team are the technical experts who curate and edit the Matrix +Specification from day to day, and so steer the evolution of the protocol by +having final review over which Matrix Spec Changes (MSCs) are merged into the +core spec. + +The Guardians are the legal directors of the non-profit Foundation, and are +responsible for ensuring that the Foundation (and by extension the Spec Core +Team) keeps on mission and neutrally protects the development of Matrix. +Guardians are typically independent of the commercial Matrix ecosystem and may +even not be members of today’s Matrix community, but are deeply aligned with the +mission of the project, and who are respected and trusted by the wider community +to uphold the guiding principles of the Foundation and keep the other Guardians +honest. + +In other words; the Spec Core Team builds the spec, and the Guardians provide an +independent backstop to ensure the spec evolves in line with the Foundation's +mission. + +## Guiding Principles + +The guiding principles define the core philosophy of the project, and will be a +formal part of the final Articles of Association of the Matrix.org Foundation. + +### Matrix Manifesto + +We believe: + + * People should have full control over their own communication. + + * People should not be locked into centralised communication silos, but free to + pick who they choose to host their communication without limiting who they + can reach. + + * The ability to converse securely and privately is a basic human right. + + * Communication should be available to everyone as an free and open, + unencumbered, standard and global network. + +### Mission + +The Matrix.org Foundation exists to act as a neutral custodian for Matrix and +nurture it as efficiently as possible as a single unfragmented standard, for the +greater benefit of the whole ecosystem; not benefiting or privileging any single +player or subset of players. + +For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix +protocol. This includes (non-exhaustively): + + * End-users of Matrix clients + * Matrix client developers and testers + * Spec developers + * Server admins + * Matrix packagers & maintainers + * Companies building products or services on Matrix + * Bridge developers + * Bot developers + * Widget developers + * Server developers + * Matrix room and community moderators + * End-users who are using Matrix indirectly via bridges + * External systems which are bridged into Matrix + * Anyone using Matrix for data communications + +"Greater benefit" is defined as maximising: + + * the number of end-users reachable on the open Matrix network + * the number of regular users on the Matrix network (e.g. 30-day retained federated users) + * the number of end-users reachable by Matrix (natively or via bridges) + * the number of online servers in the open federation + * the number of developers building on Matrix + * the number of independent implementations which use Matrix + * the quality and utility of the Matrix spec (as defined by ease and ability + with which a developer can implement spec-compliant clients, servers, bots, + bridges, and other integrations without needing to refer to any other + external material) + +N.B. that we consider success to be the growth of the open federated network +rather than closed deployments. For example, if WhatsApp adopted Matrix it +wouldn’t be a complete win unless they openly federated with the rest of the +Matrix network. + +### Values + +As Matrix evolves, it's critical that the Spec Core Team and Guardians are +aligned on the overall philosophy of the project, particularly in more +subjective areas. The values we follow are: + + * Supporting the whole long-term ecosystem rather than individual stakeholder gain + * Openness rather than proprietariness + * Collaboration rather than competition + * Accessibility rather than elitism + * Transparency rather than stealth + * Empathy rather than contrariness + * Pragmatism rather than perfection + * Proof rather than conjecture + +Patent encumbered IP is strictly prohibited from being added to the standard. + +## The Spec Core Team + +The contents and direction of the Matrix Spec is governed by the Spec Core Team; +a set of experts from across the whole Matrix community, representing all +aspects of the Matrix ecosystem. The Spec Core Team acts as a subcommittee of +the Foundation. + +Members of the Spec Core Team pledge to act as a neutral custodian for Matrix on +behalf of the whole ecosystem and uphold the Guiding Principles of the project +as outlined above. In particular, they agree to drive the adoption of Matrix as +a single global federation, an open standard unencumbered from any proprietary +IP or software patents, minimising fragmentation (whilst encouraging +experimentation), evolving rapidly, and prioritising the long-term success and +growth of the overall network over individual commercial concerns. + +Spec Core Team members need to have significant proven domain experience/skill +and have had clear dedication and commitment to the project and community for >6 +months. (In future, once we have subteams a la Rust, folks need to have proven +themselves there first). + +Members need to demonstrate ability to work constructively with the rest of the +team; we want participation in the Spec Core Team to be an efficient, pleasant and +productive place, even in the face of inevitable disagreement. We do not want a +toxic culture of bullying or competitive infighting. Folks need to be able to +compromise; we are not building a culture of folks pushing their personal +agendas at the expense of the overall project. + +We are particularly vigilant against 'trojan horse' additions to the spec - +features which only benefit particular players, or are designed to somehow +cripple or fragment the open protocol and ecosystem in favour of competitive +advantage. Commercial players are of course encouraged to build proprietary +implementations, or use custom event types, or even custom API extensions (e.g. +more efficient network transports) - but implementations must fall back to +interoperating correctly with the rest of the ecosystem. + +### Spec Core Team logistics + +The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. +Roughly half the members are expected to be from the historical core team +(similar to Rust). + +In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; +IS API; Crypto), but as a starting point we are beginning with a single core +team in the interests of not over-engineering it and scaling up elastically. + +Spec Core Team members need to be able to commit to at least 1 hour a week of +availability to work on the spec and (where relevant) reference implementations. +Members of the team volunteer their time for free to work on the project. + +Responsibilities include: + + * Reviewing Matrix Spec Change proposals and Spec PRs + + * Contributing to and reviewing reference implementations of Matrix Spec Change + proposals + + * Shepherding Matrix Spec Changes on behalf of authors where needed + + * Triaging Matrix Spec issues + + * Coordinating reference implementations + + * Ensuring the code of conduct for +matrix:matrix.org community rooms is + maintained and applied + +If members are absent for more than 8 weeks without prior agreement, they will +be assumed to have left the project. + +Spec Core Team members can resign whenever they want, but must notify the rest +of the team and the Guardians on doing so. + +New additions to the team require 100% consent from the current team members. +Membership has to be formally proposed by someone already on the Spec Core Team. + +Members can be removed from the team if X% of the team agrees they are no longer +following the goals and guiding principles of the project. + +Guardians act as a backstop, and can appoint or remove Spec Core Team members +(requiring a 75% consensus threshold between the Guardians) if the Spec Core +Team is unable to reach consensus or is failing to align with the Foundation's +mission. + +It's suggested that one of the Spec Core Team members is also be a Guardian, to +facilitate information between the Guardians and the Spec Core Team and +represent the technical angle of the project to the other Guardians. + +The project lead role acts to coordinate the team and to help tie-break in the +event of failing to get acceptance on a Matrix Spec Change. The project lead is +reviewed every 12 months and requires the confidence of 75% of the team to be +renewed. There is no maximum term for the project lead. The lead may be removed +by the core team at any point (with 75% majority), and may resign the role at +any point (notifying the team and the Guardians). The lead automatically resigns +the role if they resign from the Spec Core Team. + +The initial Spec Core Team (and their domain areas) is: + + * Matthew Hodgson (Lead) + * Erik Johnston (Servers) + * Richard van der Hoff (Servers, Cryptography) + * David Baker (Clients, IS API, Push API, Media) + * Hubert Chathi (Cryptography, General) + * Andrew Morgan (Servers, AS API, Spec Process) + * Travis Ralston (Bots and Bridges & AS API, Media, acting with Dimension hat on) + * kitsune (Clients on behalf of Community) + * TBD + +MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment +Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC +process). + +The above governance process for the Spec Core Team is considered as part of the +spec and is updated using the Matrix Spec Change process. However, changes to +the governance process also require a 75% positive approval from the Guardians +(acting as a formal decision of the Foundation's Directors), in order to ensure +changes are aligned with the Foundation's mission. + +## The Guardians + +*This section will be used as the basis for the legal responsibilities of +Directors in the Articles of Association of the Foundation.* + +The Guardians form the legal Board of Directors of The Matrix.org Foundation CIC +(Community Interest Company). They are responsible for ensuring the Foundation +is following its guiding principles, and provide a safety mechanism if the +structure of the Spec Core Team runs into trouble. + +In practice, this means that: + * Guardians must approve changes to the Spec Core Team + * Guardians must keep each other honest, providing a ‘checks and balances’ + mechanism between each other to ensure that all Guardians and the Spec Core + Team act in the best interests of the protocol and ecosystem. + * Guardians may appoint/dismiss members of the Spec Core Team who are in serious + breach of the guiding principles. + * Guardians must approve changes to the Guiding Principles (above) + * Guardians are responsible for approving use of the Foundation's assets + (e.g. redistributing donations) + * In future, Guardians may also be responsible for ensuring staff are hired by + the Foundation to support administrative functions + * As well as the Spec Core Team committee, they may also oversee committees for + other areas such as marketing Matrix.org, registering custom event types, + or "Made for Matrix" certification. + * It's likely a subset of Guardians will be hands-on for day-to-day + administrative purposes, whilst the others act to keep them in balance. + +Guardians are chosen typically to be independent of the commercial Matrix +ecosystem (and especially independent from New Vector), and may even not be +members of today’s Matrix community. However, they should be deeply aligned with +the mission of the project, and respected and trusted by the wider community to +uphold the guiding principles of the Foundation and keep the other Guardians +honest. + +Guardians are responsible for maintaining and updating the Guiding +Principles and Articles of Association of the Foundation if/when +necessary. Changes to the Guiding Principles require a 75% majority from the +Guardians and are passed as a 'special resolution' of the board. + +New Guardians may be appointed with a 75% majority by the board. + +Guardians may resign at any time, with notification to the board. + +Guardians may be removed due to serious breach of the guiding principles with a +75% majority of the other Guardians, or if absent from 3 consecutive board +meetings, or if they are legally disqualified from acting as a Director. + +We aim to recruit roughly 5 Guardians. The initial Guardians are: + + * Matthew Hodgson (CEO/CTO, New Vector) + * Amandine Le Pape (COO, New Vector) + * TBA (agreed, needs paperwork) + * TBD + * TBD + +The intention is for Matthew & Amandine (the original founders of Matrix) to +form a minority of the Guardians, in order to ensure the neutrality of the +Foundation relative to Matthew & Amandine’s day jobs at New Vector. + +Guardians volunteer their time for free to work on the project. + +## The Core Team + +"The Core Team" is a loose term that describes the set of people with access to +commit code to the public https://github.com/matrix-org repositories, who are +either working on matrix.org's reference implementations or the spec itself. +Commit access is decided by those responsible for the projects in question, much +like any other open source project. Anyone is eligible for commit access if +they have proved themselves a valuable long-term contributor, upholds the +guiding principles and mission of the project and have proved themselves able to +collaborate constructively with the existing core team. + +## Responsibilities for the Foundation + + * Independent legal entity to act as neutral custodian of Matrix + * Gathering donations + * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: + * Owns the matrix.org domain and branding + * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). + By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. + * Owns the IP of the website + * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) + * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) + * Publishing the spec + * Responsible for sytest + * Manages IANA-style allocations for Matrix + * mx:// URI scheme? + * TCP port 8448 + * .well-known URIs…? + +In future: + + * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) + * manage a Matrix certification process? + * promote Matrix (e.g. organise meetups & events & fund community activity)? + +## Timings + +The Foundation was incorporated in October 2018 as a UK limited by guarantee +private company, using generic non-profit articles of association combined with +a high-level mission lock aligned with the above: + +> 4. The objects of the Foundation are for the benefit of the community as a whole +> to: + +> 4.1.1 empower users to control their communication data and have freedom over +> their communications infrastructure by creating, maintaining and promoting +> Matrix as an openly standardised secure decentralised communication protocol and +> network, open to all, and available to the public for no charge; + +> 4.1.2 build and develop an appropriate governance model for Matrix through the +> Foundation, in order to drive the adoption of Matrix as a single global +> federation, an open standard unencumbered from any proprietary intellectual +> property and/or software patents, minimising fragmentation (whilst encouraging +> experimentation), maximising speed of development, and prioritising the long- +> term success and growth of the overall network over the commercial concerns of +> an individual person or persons. + +The foundation was then converted into a Community Interest Company, formalising +its non-profit status under the approval of the independent [Community Interest +Companies Regulator](https://www.gov.uk/government/organisations/office-of-the- +regulator- of-community-interest-companies), which took effect Jan 2019. + +We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and +finalise the Foundation's articles of association shortly afterwards based on the +contents of this MSC once passed FCP. + +This will coincide with the formal asset transfer of Matrix.org's assets from +New Vector Ltd, and the appointment of the remaining Guardians. From 303e1081f8e0647b2acd67a4050793789cb5e2a7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:44:59 +0000 Subject: [PATCH 008/282] unbreak wordwrap --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index a79d175b..cf5d0d20 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -375,8 +375,8 @@ a high-level mission lock aligned with the above: The foundation was then converted into a Community Interest Company, formalising its non-profit status under the approval of the independent [Community Interest -Companies Regulator](https://www.gov.uk/government/organisations/office-of-the- -regulator- of-community-interest-companies), which took effect Jan 2019. +Companies Regulator](https://www.gov.uk/government/organisations/office-of-the-regulator-of-community-interest-companies), +which took effect Jan 2019. We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and finalise the Foundation's articles of association shortly afterwards based on the From e6fb403dd0919018fced9525412da6f9eaef6701 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:47:37 +0000 Subject: [PATCH 009/282] add TODO for clarifying features v. extensions --- proposals/1779-open-governance.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index cf5d0d20..006d65b3 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -121,6 +121,10 @@ rather than closed deployments. For example, if WhatsApp adopted Matrix it wouldn’t be a complete win unless they openly federated with the rest of the Matrix network. +TODO: spell out when features should land in the spec, versus via +integration/widget or other non-core extension. e.g. should video conferencing +be in the spec itself, or done via Jitsi? + ### Values As Matrix evolves, it's critical that the Spec Core Team and Guardians are From c073adac9bbdaac30cc24e941185163da19cda12 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:51:04 +0000 Subject: [PATCH 010/282] incorporate review from #1318 --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 006d65b3..4fb564a5 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,6 +344,7 @@ collaborate constructively with the existing core team. * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) * Publishing the spec * Responsible for sytest + * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? * TCP port 8448 @@ -352,7 +353,7 @@ collaborate constructively with the existing core team. In future: * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) - * manage a Matrix certification process? + * manage a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) * promote Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From de6a8b20ff26f317d8a3ca8988f15cd59bd1d802 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:53:49 +0000 Subject: [PATCH 011/282] grammar --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4fb564a5..79213f89 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -10,7 +10,7 @@ communication standard on behalf of the whole Matrix community. See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ for more context. -This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318) +This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318). ## Introduction From b8249067090fc533676d23fff7a13fd16fc4ee4f Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 03:23:36 +0000 Subject: [PATCH 012/282] typoes --- proposals/1779-open-governance.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 79213f89..34230d70 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -29,8 +29,8 @@ such that once the specification has finally been ‘born’ and reached an init Vector and better support contributions from the whole ecosystem. The concepts here are somewhat inspired by [Rust’s Governance -Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust- -governance.md); a highly regarded solution to a similar problem: an ambitious +Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust-governance.md); +a highly regarded solution to a similar problem: an ambitious open-source project which has been too many years in the making, incubated at first by a single company (Mozilla Corporation), which also enjoys a very enthusiastic community! @@ -224,8 +224,8 @@ Guardians act as a backstop, and can appoint or remove Spec Core Team members Team is unable to reach consensus or is failing to align with the Foundation's mission. -It's suggested that one of the Spec Core Team members is also be a Guardian, to -facilitate information between the Guardians and the Spec Core Team and +It's suggested that one of the Spec Core Team members should also be a Guardian, +to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help tie-break in the From 4994fa115e41fd4b8777b474465c3e3702589f4a Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:21:19 +0000 Subject: [PATCH 013/282] X=75 --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 34230d70..4181d7de 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -216,8 +216,8 @@ of the team and the Guardians on doing so. New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. -Members can be removed from the team if X% of the team agrees they are no longer -following the goals and guiding principles of the project. +Members can be removed from the team if >= 75% of the team agrees they are no +longer following the goals and guiding principles of the project. Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core From 4fcd38a3a03a48d0610225453ab5adbc51636ab7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:27:03 +0000 Subject: [PATCH 014/282] clarify guardians' right to override spec core team membership --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4181d7de..07e275dc 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -274,7 +274,8 @@ In practice, this means that: mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. * Guardians may appoint/dismiss members of the Spec Core Team who are in serious - breach of the guiding principles. + breach of the guiding principles. This overrides the unanimous consent + requirement for the Spec Core Team when appointing new members. * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) From 7831c04e4caa9459cb527338e9ff6ea75ff8703f Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:29:24 +0000 Subject: [PATCH 015/282] clarify sytest responsibilities --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 07e275dc..6ba52f4d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,7 +344,8 @@ collaborate constructively with the existing core team. * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) * Publishing the spec - * Responsible for sytest + * Responsible for tools and documentation that supports the spec + * Responsible for ensuring reference implementations and test suite exists for the spec * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? From e730cc02a9d5ec77a70296b30c66a45e8ae41bf6 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:34:20 +0000 Subject: [PATCH 016/282] specify how to select a new spec core team lead --- proposals/1779-open-governance.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 6ba52f4d..67d66b2a 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -234,7 +234,9 @@ reviewed every 12 months and requires the confidence of 75% of the team to be renewed. There is no maximum term for the project lead. The lead may be removed by the core team at any point (with 75% majority), and may resign the role at any point (notifying the team and the Guardians). The lead automatically resigns -the role if they resign from the Spec Core Team. +the role if they resign from the Spec Core Team. Resignation automatically +triggers selection of a new lead, who must be selected from the existing core +spec team. The initial Spec Core Team (and their domain areas) is: @@ -276,6 +278,8 @@ In practice, this means that: * Guardians may appoint/dismiss members of the Spec Core Team who are in serious breach of the guiding principles. This overrides the unanimous consent requirement for the Spec Core Team when appointing new members. + * Guardians may also override deadlocks when appointing a Spec Core Team leader + (with a >= 75% majority) * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) From 103d2f4ed223be24f798b7d45a4f5cb87c6e8720 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:36:32 +0000 Subject: [PATCH 017/282] clarify that the project lead doesn't have casting vote --- proposals/1779-open-governance.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 67d66b2a..6280a45b 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -228,15 +228,15 @@ It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. -The project lead role acts to coordinate the team and to help tie-break in the -event of failing to get acceptance on a Matrix Spec Change. The project lead is -reviewed every 12 months and requires the confidence of 75% of the team to be -renewed. There is no maximum term for the project lead. The lead may be removed -by the core team at any point (with 75% majority), and may resign the role at -any point (notifying the team and the Guardians). The lead automatically resigns -the role if they resign from the Spec Core Team. Resignation automatically -triggers selection of a new lead, who must be selected from the existing core -spec team. +The project lead role acts to coordinate the team and to help steer the team to +consensus in the event of failing to get agreement on a Matrix Spec Change. The +project lead is reviewed every 12 months and requires the confidence of 75% of +the team to be renewed. There is no maximum term for the project lead. The lead +may be removed by the core team at any point (with 75% majority), and may resign +the role at any point (notifying the team and the Guardians). The lead +automatically resigns the role if they resign from the Spec Core Team. +Resignation automatically triggers selection of a new lead, who must be selected +from the existing core spec team. The initial Spec Core Team (and their domain areas) is: From 2047ba59daf0632b04ed974dc3598c2224b9335e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:42:49 +0000 Subject: [PATCH 018/282] spell out domain spread requirement for spec core team --- proposals/1779-open-governance.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 6280a45b..fb15027a 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -238,6 +238,10 @@ automatically resigns the role if they resign from the Spec Core Team. Resignation automatically triggers selection of a new lead, who must be selected from the existing core spec team. +It is vital that the core spec team has strong domain expertise covering all +different domains of the spec (e.g. we don't want to end up with a core spec +team where nobody has strong experience in cryptography) + The initial Spec Core Team (and their domain areas) is: * Matthew Hodgson (Lead) From c05000b38dd7529fe79f582091de6bea1590509d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:43:44 +0000 Subject: [PATCH 019/282] broaden reasons for dysfunctional core spec teams --- proposals/1779-open-governance.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index fb15027a..2c67a5ca 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -221,8 +221,7 @@ longer following the goals and guiding principles of the project. Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core -Team is unable to reach consensus or is failing to align with the Foundation's -mission. +Team is unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information between the Guardians and the Spec Core Team and From 0e246b147717b1b7f7d40968211a39b365d6effa Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 8 Jan 2019 14:44:18 +0000 Subject: [PATCH 020/282] Update proposals/1779-open-governance.md Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2c67a5ca..1a836322 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -243,7 +243,7 @@ team where nobody has strong experience in cryptography) The initial Spec Core Team (and their domain areas) is: - * Matthew Hodgson (Lead) + * Matthew Hodgson (Lead, Guardian) * Erik Johnston (Servers) * Richard van der Hoff (Servers, Cryptography) * David Baker (Clients, IS API, Push API, Media) From 5235293623bfbe62301d8514c6055d321f0aad06 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:50:18 +0000 Subject: [PATCH 021/282] spell out the Core Team a bit more --- proposals/1779-open-governance.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2c67a5ca..6f8af97d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -328,16 +328,27 @@ Foundation relative to Matthew & Amandine’s day jobs at New Vector. Guardians volunteer their time for free to work on the project. -## The Core Team - -"The Core Team" is a loose term that describes the set of people with access to -commit code to the public https://github.com/matrix-org repositories, who are -either working on matrix.org's reference implementations or the spec itself. -Commit access is decided by those responsible for the projects in question, much -like any other open source project. Anyone is eligible for commit access if -they have proved themselves a valuable long-term contributor, upholds the -guiding principles and mission of the project and have proved themselves able to -collaborate constructively with the existing core team. +## The Code Core Team (aka The Core Team) + +The "Core Team" (or the "Code Core Team", to disambiguate from the Spec Core +Team) is a loose term that describes the set of people with access to commit +code to the public https://github.com/matrix-org repositories, who are either +working on matrix.org's reference implementations or the spec itself. Commit +access is decided by those responsible for the projects in question, much like +any other open source project. Anyone is eligible for commit access if they +have proved themselves a valuable long-term contributor, upholds the guiding +principles and mission of the project and have proved themselves able to +collaborate constructively with the existing core team. Active participation in +the core team is also signified by membership of the +matrix:matrix.org Matrix +community. + +TODO: spell out some responsibilities. Erik suggests something like: + * Helping to ensure the quality of the projects' repositories + * That all projects follow the Matrix spec + * Engaging with the people in a way that fosters a healthy and happy community + * Following the Guiding Principles and promoting them within the community + +Code Core Team members volunteer their time for free to work on the project. ## Responsibilities for the Foundation From 3a5d56467bcaacf3f1672f3ed522c61da8c097a0 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:51:45 +0000 Subject: [PATCH 022/282] reword lead renewals --- proposals/1779-open-governance.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f7712197..1e78d8e0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -228,14 +228,14 @@ to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to -consensus in the event of failing to get agreement on a Matrix Spec Change. The -project lead is reviewed every 12 months and requires the confidence of 75% of -the team to be renewed. There is no maximum term for the project lead. The lead -may be removed by the core team at any point (with 75% majority), and may resign -the role at any point (notifying the team and the Guardians). The lead -automatically resigns the role if they resign from the Spec Core Team. -Resignation automatically triggers selection of a new lead, who must be selected -from the existing core spec team. +consensus in the event of failing to get agreement on a Matrix Spec Change. +Every 12 months, a vote of confidence is held in the project lead, requiring the +confidence of 75% of the team for the lead to be renewed. There is no maximum +term for the project lead. The lead may be removed by the core team at any +point (with 75% majority), and may resign the role at any point (notifying the +team and the Guardians). The lead automatically resigns the role if they resign +from the Spec Core Team. Resignation automatically triggers selection of a new +lead, who must be selected from the existing core spec team. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec From c02ecb58aec48778d9ca4531bb84f5e875d71210 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 8 Jan 2019 16:11:00 -0500 Subject: [PATCH 023/282] mark which fields are required --- proposals/1717-key_verification.md | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index b71fcf34..fa1d7373 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -57,13 +57,15 @@ Requests a key verification. Properties: -- `from_device` (string): the device ID of the device requesting verification. -- `transaction_id` (string): an identifier for the verification request. Must - be unique with respect to the pair of devices. -- `methods` ([string]): the verification methods supported by the sender. -- `timestamp` (integer): the time when the request was made. If the timestamp - is in the future (by more than 5 minutes, to allow for clock skew), or more - than 10 minutes in the past, then the message must be ignored. +- `from_device` (string): Required. The device ID of the device requesting + verification. +- `transaction_id` (string): Required. An identifier for the verification + request. Must be unique with respect to the pair of devices. +- `methods` ([string]): Required. The verification methods supported by the + sender. +- `timestamp` (integer): Required. The time when the request was made. If the + timestamp is in the future (by more than 5 minutes, to allow for clock skew), + or more than 10 minutes in the past, then the message must be ignored. #### `m.key.verification.start` @@ -71,13 +73,13 @@ Begins a key verification process. Properties: -- `method` (string): the verification method to use. -- `from_device` (string): The device ID of the device starting the verification - process. -- `transaction_id` (string): an identifier for the verification process. If - this message is sent in reponse to an `m.key.verification.request` event, then - it must use the same `transaction_id` as the one given in the - `m.key.verification.request`. +- `method` (string): Required. The verification method to use. +- `from_device` (string): Required. The device ID of the device starting the + verification process. +- `transaction_id` (string): Required. An identifier for the verification + process. If this message is sent in reponse to an + `m.key.verification.request` event, then it must use the same + `transaction_id` as the one given in the `m.key.verification.request`. Key verification methods can define additional properties to be included. From ff0b9eac76cfa65222dd82d45ff3358570ab1fd1 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 8 Jan 2019 16:11:15 -0500 Subject: [PATCH 024/282] add ability to start verifications that happen in two stages --- proposals/1717-key_verification.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index fa1d7373..8acc4654 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -80,6 +80,10 @@ Properties: process. If this message is sent in reponse to an `m.key.verification.request` event, then it must use the same `transaction_id` as the one given in the `m.key.verification.request`. +- `next_method` (string): Optional. If the selected verification method only + verifies one user's key, then this property can be used to indicate the + method to use to verify the other user's key, which will be started + immediately after after the current key verification is complete. Key verification methods can define additional properties to be included. From 09a547d67e0a4b835405af0be9ce739f9dfe400a Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 8 Jan 2019 17:45:13 -0500 Subject: [PATCH 025/282] add some cancellation codes, and mention existing verification MSCs --- proposals/1717-key_verification.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index 8acc4654..b536be0e 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -2,7 +2,8 @@ Key verification is an essential part of ensuring that end-to-end encrypted messages are secure. Matrix may support multiple verification methods that -require sending events; in fact, two such methods have already been proposed. +require sending events; in fact, two such methods (such as MSC1267 and MSC1543) +have already been proposed. This proposal tries to present a common framework for verification methods to use, and presents a way to request key verification. @@ -112,6 +113,12 @@ Properties: - `m.unexpected_message`: the device received an unexpected message. For example, a message for a verification method may have been received when it was not expected. + - `m.key_mismatch`: the key was not verified. + - `m.user_mismatch`: the expected user did not match the user verified. + - `m.invalid_message`: an invalid message was received. + - `m.accepted`: when an `m.key.verification.request` is accepted by one + device, an `m.key.verification.cancel` message with `code` set to + `m.accepted` is sent to the other devices - `reason` (string): human-readable reason for cancelling. This should only be used if the recieving client does not understand the code given. From 03802701715d4ac1e404e14d619c1c80be4fca2e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Wed, 9 Jan 2019 23:29:59 +0000 Subject: [PATCH 026/282] spell out that hypothetical employees could come in any size --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 1e78d8e0..8834da30 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -287,7 +287,8 @@ In practice, this means that: * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) * In future, Guardians may also be responsible for ensuring staff are hired by - the Foundation to support administrative functions + the Foundation to support administrative functions and other roles required + to facilitate the Foundation's mission. * As well as the Spec Core Team committee, they may also oversee committees for other areas such as marketing Matrix.org, registering custom event types, or "Made for Matrix" certification. From c053996a7749b810affe871f36541de481b152c7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:12:56 +0000 Subject: [PATCH 027/282] add new technical guidelines to proposals.rst this was originally a todo for MSC1779, but belongs better in proposals.rst --- specification/proposals_intro.rst | 69 +++++++++++++++++++++++++++++-- 1 file changed, 65 insertions(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 73ace98e..049ad0c8 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -44,15 +44,18 @@ moderators and admins, companies/projects building products or services on Matrix, spec contributors, translators, and those who created it in the first place. -"Greater benefit" could include maximising: +"Greater benefit" includes maximising: * the number of end-users reachable on the open Matrix network -* the number of regular users on the Matrix network (e.g. 30-day retained - federated users) +* the number of regular users on the Matrix network (e.g. 30-day retained federated users) +* the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix -* the quality and utility of the Matrix spec +* the quality and utility of the Matrix spec (as defined by ease and ability + with which a developer can implement spec-compliant clients, servers, bots, + bridges, and other integrations without needing to refer to any other + external material) In addition, proposal authors are expected to uphold the following values in their proposed changes to the Matrix protocol: @@ -66,6 +69,64 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture +Technical notes +--------------- + +Proposals **must** develop Matrix as a layered protocol: with new features +building on layers of shared abstractions rather than introducing tight vertical +coupling within the stack. This ensures that new features can evolve rapidly by +building on existing layers and swapping out old features without impacting the +rest of the stack or requiring substantial upgrades to the whole ecosystem. +This is critical for Matrix to rapidly evolve and compete effectively with +centralised systems, despite being a federated protocol. + +For instance, new features should be implemented using the highest layer +abstractions possible (e.g. new event types, which layer on top of the existing +room semantics, and so don't even require any API changes). Failing that, the +next recourse would be backwards-compatible changes to the next layer down (e.g. +room APIs); failing that, considering changes to the format of events or the +DAG; etc. It would be a very unusual feature which doesn't build on the +existing infrastructure provided by the spec and instead created new primitives +or low level APIs. + +Backwards compatibility is very important for Matrix, but not at the expense of +hindering the protocol's evolution. Backwards incompatible changes to endpoints +are allowed when no other alternative exists, and must be versioned under a new +major release of the API. Backwards incompatible changes to the room algorithm +are also allowed when no other alternative exists, and must be versioned under a +new version of the room algorithm. + +There is sometimes a dilemma over where to include higher level features: for +instance, should video conferencing be formalised in the spec, or should it be +implemented via widgets (if one assumes that widgets have landed in the spec and +[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) is merged)? +Should reputation systems be specified? Should search engine behaviour be +specified? + +There is no universal answer to this, but the following guidelines should be +applied: + * If the feature would benefit the whole Matrix ecosystem and is aligned with + the guiding principles above, then it should be supported by the spec. + For instance, video conferencing is clearly a feature which would benefit + the whole ecosystem, and so the spec should find a way to make it happen. + * If the spec already makes the feature possible without changing any of the + spec *or implementations*, then it may not need to be added to the spec. + For instance, video conferencing done by widgets requires no compulsory + changes to clients nor servers to work, and so could be omitted. + * However, if the best user experience for a feature does require custom + implementation behaviour - e.g. embedding Jitsi into your client rather than + using a widget, then the behaviour should be defined in the spec to allow + implementations to do so. + * However, the spec must never add dependencies on unspecified/nonstandardised + 3rd party behaviour. For instance, defining how to embed Jitsi is unlikely to + ever make it into the spec, given Jitsi does not implement a standardised + interface (although a URL-based calling standard may emerge in future, which + could be used as an extension to the current widget-based approach). + * Therefore, our two options in the specific case of video conferencing are + either to spec SFU conferencing semantics on WebRTC (or refer to an existing spec + for doing so), or to keep it as a widget-based approach (optionally with widget + extensions specific for more deeply integrating video conferencing use cases). + Process ------- From edaf3596f470323a66c74383afb70332d26c28d9 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:14:36 +0000 Subject: [PATCH 028/282] fix 'which features should go in the spec?' TODO and reword intro --- proposals/1779-open-governance.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 8834da30..9db96756 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -1,11 +1,11 @@ # Proposal for Open Governance of Matrix.org -This whole document is a **work in progress** draft of a constitution proposal -for open governance for Matrix.org, and forms the basis of the first full -Articles of Association (AoA) for [The Matrix.org Foundation -C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit legal -entity incorporated to act as the neutral guardian of the Matrix decentralised -communication standard on behalf of the whole Matrix community. +This whole document is the proposed constitution proposal for Matrix.org, and +will form the basis of the first full Articles of Association (AoA) for [The +Matrix.org Foundation +C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit +legal entity incorporated to act as the neutral guardian of the Matrix +decentralised communication standard on behalf of the whole Matrix community. See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ for more context. @@ -121,10 +121,6 @@ rather than closed deployments. For example, if WhatsApp adopted Matrix it wouldn’t be a complete win unless they openly federated with the rest of the Matrix network. -TODO: spell out when features should land in the spec, versus via -integration/widget or other non-core extension. e.g. should video conferencing -be in the spec itself, or done via Jitsi? - ### Values As Matrix evolves, it's critical that the Spec Core Team and Guardians are From 21a781b1d2d5c035763da2d734613d323888eb3d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:16:24 +0000 Subject: [PATCH 029/282] fix proprietary wording --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9db96756..f829f1d8 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -168,7 +168,7 @@ agendas at the expense of the overall project. We are particularly vigilant against 'trojan horse' additions to the spec - features which only benefit particular players, or are designed to somehow cripple or fragment the open protocol and ecosystem in favour of competitive -advantage. Commercial players are of course encouraged to build proprietary +advantage. Commercial players are of course free to build proprietary implementations, or use custom event types, or even custom API extensions (e.g. more efficient network transports) - but implementations must fall back to interoperating correctly with the rest of the ecosystem. From 265a3dc49b3a00f7ce80e00e8b34bc13a5612adc Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:20:09 +0000 Subject: [PATCH 030/282] clarify removing members --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f829f1d8..9013f6b6 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -213,7 +213,8 @@ New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. Members can be removed from the team if >= 75% of the team agrees they are no -longer following the goals and guiding principles of the project. +longer following the goals and guiding principles of the project. (The 75% is +measured of the whole team, including the member in question) Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core From e584ae31f4e1e5bf806250a06e28b1bb3d9abcee Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:21:02 +0000 Subject: [PATCH 031/282] fix missing words --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9013f6b6..808e8368 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -221,8 +221,8 @@ Guardians act as a backstop, and can appoint or remove Spec Core Team members Team is unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, -to facilitate information between the Guardians and the Spec Core Team and -represent the technical angle of the project to the other Guardians. +to facilitate information exchange between the Guardians and the Spec Core Team, +and to represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to consensus in the event of failing to get agreement on a Matrix Spec Change. From ed820ca27bdc00845f085c48fbf73c06f42c5202 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sat, 12 Jan 2019 23:21:52 +0000 Subject: [PATCH 032/282] s/core spec team/Spec Core Team/ Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 808e8368..2d0e21ed 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -232,7 +232,7 @@ term for the project lead. The lead may be removed by the core team at any point (with 75% majority), and may resign the role at any point (notifying the team and the Guardians). The lead automatically resigns the role if they resign from the Spec Core Team. Resignation automatically triggers selection of a new -lead, who must be selected from the existing core spec team. +lead, who must be selected from the existing Spec Core Team. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec From b758ceea5b3976f0ae64da5ad7cf8855f0ee5ab7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:23:46 +0000 Subject: [PATCH 033/282] add erik's quorum --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2d0e21ed..58142027 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -177,7 +177,8 @@ interoperating correctly with the rest of the ecosystem. The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. Roughly half the members are expected to be from the historical core team -(similar to Rust). +(similar to Rust). The team must have 5 members to be quorate, with the aim of +generally having between 7 and 9 members. In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; IS API; Crypto), but as a starting point we are beginning with a single core From 107d96c50e4882924d4823fef92967e09e9bc223 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:24:55 +0000 Subject: [PATCH 034/282] spell out consensus ftw --- proposals/1779-open-governance.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 58142027..76c6ffa9 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -255,6 +255,9 @@ MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC process). +Even though a threshold of only 75% is required for approval, the Spec Core Team +is expected to seek consensus on MSCs. + The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to the governance process also require a 75% positive approval from the Guardians From cccd62f03530fcaa78fa38b8fa4992518058100a Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sat, 12 Jan 2019 23:25:37 +0000 Subject: [PATCH 035/282] grammar Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 76c6ffa9..4de598a9 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -338,7 +338,7 @@ code to the public https://github.com/matrix-org repositories, who are either working on matrix.org's reference implementations or the spec itself. Commit access is decided by those responsible for the projects in question, much like any other open source project. Anyone is eligible for commit access if they -have proved themselves a valuable long-term contributor, upholds the guiding +have proved themselves a valuable long-term contributor, uphold the guiding principles and mission of the project and have proved themselves able to collaborate constructively with the existing core team. Active participation in the core team is also signified by membership of the +matrix:matrix.org Matrix From d00a5eba93926b768555d53a7b081c3a4ccfc9b0 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:26:58 +0000 Subject: [PATCH 036/282] consistent tenses --- proposals/1779-open-governance.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4de598a9..16047a2d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -354,19 +354,19 @@ Code Core Team members volunteer their time for free to work on the project. ## Responsibilities for the Foundation - * Independent legal entity to act as neutral custodian of Matrix - * Gathering donations + * Independent legal entity which acts as neutral custodian of Matrix + * Gathers donations * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: * Owns the matrix.org domain and branding * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. * Owns the IP of the website * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) - * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) - * Publishing the spec + * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector) + * Publishes the spec * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec - * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) + * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? * TCP port 8448 @@ -374,9 +374,9 @@ Code Core Team members volunteer their time for free to work on the project. In future: - * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) - * manage a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) - * promote Matrix (e.g. organise meetups & events & fund community activity)? + * contracts entities to work on Matrix? (e.g. redistributing donations back to fund development) + * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) + * promotes Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From ddc3921318262ad7134b37253030ac5a182833ca Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:31:01 +0000 Subject: [PATCH 037/282] un-todo code core team responsibilities --- proposals/1779-open-governance.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 16047a2d..f30d50d0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,10 +344,14 @@ collaborate constructively with the existing core team. Active participation in the core team is also signified by membership of the +matrix:matrix.org Matrix community. -TODO: spell out some responsibilities. Erik suggests something like: - * Helping to ensure the quality of the projects' repositories - * That all projects follow the Matrix spec - * Engaging with the people in a way that fosters a healthy and happy community +Responsibilities include: + * Helping ensure the quality of the projects' code repositories + * Ensuring all commits are reviewed + * Ensuring that all projects follow the Matrix spec + * Helping architect the implementations + * Contributing code to the implementations + * Fostering contributions and engaging with contributors constructively in a + way that fosters a healthy and happy community * Following the Guiding Principles and promoting them within the community Code Core Team members volunteer their time for free to work on the project. From 156488384c75aa23290da0cd845902fffb043d1d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:49:27 +0000 Subject: [PATCH 038/282] add more examples for spec inclusion; add interoperability as a core value --- proposals/1779-open-governance.md | 1 + specification/proposals_intro.rst | 16 +++++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f30d50d0..f5a89f5e 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -129,6 +129,7 @@ subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness + * Interoperability rather than fragmentation * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 049ad0c8..b7663e66 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -62,6 +62,7 @@ their proposed changes to the Matrix protocol: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness +* Interoperability rather than fragmentation * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -110,7 +111,7 @@ applied: For instance, video conferencing is clearly a feature which would benefit the whole ecosystem, and so the spec should find a way to make it happen. * If the spec already makes the feature possible without changing any of the - spec *or implementations*, then it may not need to be added to the spec. + implementations and spec, then it may not need to be added to the spec. For instance, video conferencing done by widgets requires no compulsory changes to clients nor servers to work, and so could be omitted. * However, if the best user experience for a feature does require custom @@ -127,6 +128,19 @@ applied: for doing so), or to keep it as a widget-based approach (optionally with widget extensions specific for more deeply integrating video conferencing use cases). +As an alternative example: it's very unlikely that "how to visualise Magnetic +Resonsance Imaging data over Matrix" would ever be added to the Matrix spec +(other than perhaps a custom event type in a wider standardised Matrix event +registry) given that the spec's existing primitives of file transfer and +extensible events (MSC1767) give excellent tools for transferring and +visualising arbitrary rich data. + +Conversely, features such as reactions, threaded messages, editable messages, +spam/abuse/content filtering, are all features which would clearly benefit the +whole Matrix ecosystem and require both client & server implementation +changes across the board to be implemented in an interoperable way, and so +necessitate a spec change. + Process ------- From f3085812e9c160f63e0cb542378eb827112a54c2 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:51:49 +0000 Subject: [PATCH 039/282] more examples; remove widget MSC ref --- specification/proposals_intro.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index b7663e66..c8a2fd32 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -99,10 +99,8 @@ new version of the room algorithm. There is sometimes a dilemma over where to include higher level features: for instance, should video conferencing be formalised in the spec, or should it be -implemented via widgets (if one assumes that widgets have landed in the spec and -[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) is merged)? -Should reputation systems be specified? Should search engine behaviour be -specified? +implemented via widgets? Should reputation systems be specified? Should search +engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: @@ -135,11 +133,17 @@ registry) given that the spec's existing primitives of file transfer and extensible events (MSC1767) give excellent tools for transferring and visualising arbitrary rich data. +Supporting public search engines are likely to not require custom spec features +(other than possibly better bulk access APIs), given they can be implemented as +clients using the existing CS API. An exception could be API features required +by decentralised search infrastructure (avoiding centralisation of power by +a centralised search engine). + Conversely, features such as reactions, threaded messages, editable messages, -spam/abuse/content filtering, are all features which would clearly benefit the -whole Matrix ecosystem and require both client & server implementation -changes across the board to be implemented in an interoperable way, and so -necessitate a spec change. +spam/abuse/content filtering (and reputation systems), are all features which +would clearly benefit the whole Matrix ecosystem and require both client & +server implementation changes across the board to be implemented in an +interoperable way, and so necessitate a spec change. Process ------- From 09813fc3a080600c13ccaaa74deb61c9df07c1ee Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 12:41:19 +0000 Subject: [PATCH 040/282] improve wording around compensation for team members. tweak Greater Benefit (particularly adding SnR and filtering goals --- proposals/1779-open-governance.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f5a89f5e..63684e40 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -105,12 +105,14 @@ protocol. This includes (non-exhaustively): "Greater benefit" is defined as maximising: - * the number of end-users reachable on the open Matrix network + * the number of Matrix-native end-users reachable on the open Matrix network * the number of regular users on the Matrix network (e.g. 30-day retained federated users) - * the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix + * the number of bridged end-users reachable on the open Matrix network + * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) + * the ability for users to discover content on their terms (empowering them to select what to see and what not to see) * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other @@ -130,6 +132,7 @@ subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness * Interoperability rather than fragmentation + * Cross-platform rather than platform-specific * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -187,7 +190,7 @@ team in the interests of not over-engineering it and scaling up elastically. Spec Core Team members need to be able to commit to at least 1 hour a week of availability to work on the spec and (where relevant) reference implementations. -Members of the team volunteer their time for free to work on the project. +Members must arrange their own funding for their time. Responsibilities include: @@ -329,7 +332,7 @@ The intention is for Matthew & Amandine (the original founders of Matrix) to form a minority of the Guardians, in order to ensure the neutrality of the Foundation relative to Matthew & Amandine’s day jobs at New Vector. -Guardians volunteer their time for free to work on the project. +Guardians must arrange their own funding for their time. ## The Code Core Team (aka The Core Team) @@ -355,7 +358,7 @@ Responsibilities include: way that fosters a healthy and happy community * Following the Guiding Principles and promoting them within the community -Code Core Team members volunteer their time for free to work on the project. +Code Core Team members must arrange their own funding for their time. ## Responsibilities for the Foundation From 2f20679db514ec6b58970bcf91e18a3f3c46270a Mon Sep 17 00:00:00 2001 From: Kitsune Ral Date: Mon, 14 Jan 2019 12:48:12 +0000 Subject: [PATCH 041/282] deanonymise kitsune Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 63684e40..a98f36a0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -252,7 +252,7 @@ The initial Spec Core Team (and their domain areas) is: * Hubert Chathi (Cryptography, General) * Andrew Morgan (Servers, AS API, Spec Process) * Travis Ralston (Bots and Bridges & AS API, Media, acting with Dimension hat on) - * kitsune (Clients on behalf of Community) + * Alexey Rusakov (Clients on behalf of Community) * TBD MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment From a38e1e6adff27c649b6ac964f7787535b1aa988e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 12:54:21 +0000 Subject: [PATCH 042/282] incorporate kitsune & dbkr review --- proposals/1779-open-governance.md | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index a98f36a0..0b8c0aeb 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -75,7 +75,7 @@ We believe: * The ability to converse securely and privately is a basic human right. - * Communication should be available to everyone as an free and open, + * Communication should be available to everyone as a free and open, unencumbered, standard and global network. ### Mission @@ -130,7 +130,7 @@ aligned on the overall philosophy of the project, particularly in more subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain - * Openness rather than proprietariness + * Openness rather than proprietary lock-in * Interoperability rather than fragmentation * Cross-platform rather than platform-specific * Collaboration rather than competition @@ -142,6 +142,10 @@ subjective areas. The values we follow are: Patent encumbered IP is strictly prohibited from being added to the standard. +Making the specification rely on non-standard/unspecified behaviour of other +systems or actors (such as SaaS services, even open-sourced, not governed by a +standard protocol) shall not be accepted, either. + ## The Spec Core Team The contents and direction of the Matrix Spec is governed by the Spec Core Team; @@ -208,8 +212,8 @@ Responsibilities include: * Ensuring the code of conduct for +matrix:matrix.org community rooms is maintained and applied -If members are absent for more than 8 weeks without prior agreement, they will -be assumed to have left the project. +If members are absent (uncontactable) for more than 8 weeks without prior +agreement, they will be assumed to have left the project. Spec Core Team members can resign whenever they want, but must notify the rest of the team and the Guardians on doing so. @@ -375,16 +379,20 @@ Code Core Team members must arrange their own funding for their time. * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) - * Manages IANA-style allocations for Matrix - * mx:// URI scheme? + * Manages IANA-style allocations for Matrix, including: + * mx:// URI scheme * TCP port 8448 - * .well-known URIs…? + * .well-known URIs +* Ensures that Matrix promotion is happening (e.g. ensuring that meetups & + events & community activity is supported) In future: - * contracts entities to work on Matrix? (e.g. redistributing donations back to fund development) + * contracts entities to work on Matrix if such contracts help the Foundation to + fulfil its mission and obey the Guiding Principles (e.g. redistributing + donations back to fund development of reference implementations or compliance + kits) * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) - * promotes Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From 70be8393a391baecc23a322800f6843d63d8187a Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 13:01:50 +0000 Subject: [PATCH 043/282] clarify the technical note and guiding principles --- specification/proposals_intro.rst | 73 ++++++++++++++++++------------- 1 file changed, 42 insertions(+), 31 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index c8a2fd32..393a43a2 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -33,25 +33,28 @@ Guiding Principles Proposals **must** act to the greater benefit of the entire Matrix ecosystem, rather than benefiting or privileging any single player or subset of players - -and must not contain any patent encumbered intellectual property. Members of the Core Team pledge to act as +and must not contain any patent encumbered intellectual property. +Members of the Spec Core Team pledge to act as a neutral custodian for Matrix on behalf of the whole ecosystem. For clarity: the Matrix ecosystem is anyone who uses the Matrix protocol. That -includes client users, server admins, client developers, bot developers, -bridge and application service developers, users and admins who are indirectly using Matrix via -3rd party networks which happen to be bridged, server developers, room -moderators and admins, companies/projects building products or services on -Matrix, spec contributors, translators, and those who created it in -the first place. +includes client users, server admins, client developers, bot developers, bridge +and application service developers, users and admins who are indirectly using +Matrix via 3rd party networks which happen to be bridged, server developers, +room moderators and admins, companies/projects building products or services on +Matrix, spec contributors, translators, and those who created it in the first +place. -"Greater benefit" includes maximising: +"Greater benefit" is defined as maximising: -* the number of end-users reachable on the open Matrix network +* the number of Matrix-native end-users reachable on the open Matrix network * the number of regular users on the Matrix network (e.g. 30-day retained federated users) -* the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix +* the number of bridged end-users reachable on the open Matrix network +* the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) +* the ability for users to discover content on their terms (empowering them to select what to see and what not to see) * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other @@ -61,8 +64,9 @@ In addition, proposal authors are expected to uphold the following values in their proposed changes to the Matrix protocol: * Supporting the whole long-term ecosystem rather than individual stakeholder gain -* Openness rather than proprietariness +* Openness rather than proprietary lock-in * Interoperability rather than fragmentation +* Cross-platform rather than platform-specific * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -70,6 +74,9 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture +Please see [MSC1779](https://github.com/matrix-org/matrix-doc/pull/1779) +for full details of the project's Guiding Principles. + Technical notes --------------- @@ -104,33 +111,37 @@ engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: - * If the feature would benefit the whole Matrix ecosystem and is aligned with +1. If the feature would benefit the whole Matrix ecosystem and is aligned with the guiding principles above, then it should be supported by the spec. - For instance, video conferencing is clearly a feature which would benefit - the whole ecosystem, and so the spec should find a way to make it happen. - * If the spec already makes the feature possible without changing any of the +2. If the spec already makes the feature possible without changing any of the implementations and spec, then it may not need to be added to the spec. - For instance, video conferencing done by widgets requires no compulsory - changes to clients nor servers to work, and so could be omitted. - * However, if the best user experience for a feature does require custom - implementation behaviour - e.g. embedding Jitsi into your client rather than - using a widget, then the behaviour should be defined in the spec to allow - implementations to do so. - * However, the spec must never add dependencies on unspecified/nonstandardised - 3rd party behaviour. For instance, defining how to embed Jitsi is unlikely to - ever make it into the spec, given Jitsi does not implement a standardised - interface (although a URL-based calling standard may emerge in future, which - could be used as an extension to the current widget-based approach). - * Therefore, our two options in the specific case of video conferencing are - either to spec SFU conferencing semantics on WebRTC (or refer to an existing spec - for doing so), or to keep it as a widget-based approach (optionally with widget - extensions specific for more deeply integrating video conferencing use cases). +3. However, if the best user experience for a feature does require custom + implementation behaviour then the behaviour should be defined in the spec + such that all implementations may implement it. +4. However, the spec must never add dependencies on unspecified/nonstandardised + 3rd party behaviour. + +As a worked example: +1. Video conferencing is clearly a feature which would benefit + the whole ecosystem, and so the spec should find a way to make it happen. +2. Video conferencing can be achieved by widgets without requiring any + compulsory changes to changes to clients nor servers to work, and so could be + omitted from the spec. +3. A better experience could be achieved by embedding Jitsi natively into clients + rather than using a widget... +4. ...except that would add a dependency on unspecified/nonstandardised 3rd party + behaviour, so must not be added to the spec. + +Therefore, our two options in the specific case of video conferencing are +either to spec SFU conferencing semantics for WebRTC (or refer to an existing spec +for doing so), or to keep it as a widget-based approach (optionally with widget +extensions specific for more deeply integrating video conferencing use cases). As an alternative example: it's very unlikely that "how to visualise Magnetic Resonsance Imaging data over Matrix" would ever be added to the Matrix spec (other than perhaps a custom event type in a wider standardised Matrix event registry) given that the spec's existing primitives of file transfer and -extensible events (MSC1767) give excellent tools for transferring and +extensible events (MSC1767) give excellent tools for transfering and visualising arbitrary rich data. Supporting public search engines are likely to not require custom spec features From 811e65a4d1cf92b1314889330a01f86a16d23148 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 13:14:43 +0000 Subject: [PATCH 044/282] kitsune review on when to add things to the spec --- specification/proposals_intro.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 393a43a2..1e93a22a 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -150,11 +150,10 @@ clients using the existing CS API. An exception could be API features required by decentralised search infrastructure (avoiding centralisation of power by a centralised search engine). -Conversely, features such as reactions, threaded messages, editable messages, +Features such as reactions, threaded messages, editable messages, spam/abuse/content filtering (and reputation systems), are all features which -would clearly benefit the whole Matrix ecosystem and require both client & -server implementation changes across the board to be implemented in an -interoperable way, and so necessitate a spec change. +would clearly benefit the whole Matrix ecosystem, and cannot be implemented in an +interoperable way using the current spec; so they necessitate a spec change. Process ------- From 9e435d6dabdeb2e482d15e1ceb9bcda0bbe9b432 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Mon, 14 Jan 2019 23:33:03 +0000 Subject: [PATCH 045/282] scifi IANA Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 0b8c0aeb..f6d5b959 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -379,7 +379,7 @@ Code Core Team members must arrange their own funding for their time. * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) - * Manages IANA-style allocations for Matrix, including: + * Manages any future IANA-style allocations for Matrix, such as: * mx:// URI scheme * TCP port 8448 * .well-known URIs From 6ff0155a324510b863f3c66843817f4d45e5b9e7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:16:33 +0000 Subject: [PATCH 046/282] s/responsibilities/functions --- proposals/1779-open-governance.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f6d5b959..28c18791 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -364,7 +364,7 @@ Responsibilities include: Code Core Team members must arrange their own funding for their time. -## Responsibilities for the Foundation +## Functions of the Foundation * Independent legal entity which acts as neutral custodian of Matrix * Gathers donations @@ -392,7 +392,8 @@ In future: fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance kits) - * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) + * manages a "Made for Matrix" certification process? (to confirm that products + are actually compatible with Matrix) ## Timings From 822d84e50c46f335efbd04af8f565fbf98af7b96 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:29:07 +0000 Subject: [PATCH 047/282] append fullstops to lists to make vdh happy --- proposals/1779-open-governance.md | 128 +++++++++++++++--------------- 1 file changed, 64 insertions(+), 64 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 28c18791..58e43b80 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -88,35 +88,35 @@ player or subset of players. For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix protocol. This includes (non-exhaustively): - * End-users of Matrix clients - * Matrix client developers and testers - * Spec developers - * Server admins - * Matrix packagers & maintainers - * Companies building products or services on Matrix - * Bridge developers - * Bot developers - * Widget developers - * Server developers - * Matrix room and community moderators - * End-users who are using Matrix indirectly via bridges - * External systems which are bridged into Matrix - * Anyone using Matrix for data communications + * End-users of Matrix clients. + * Matrix client developers and testers. + * Spec developers. + * Server admins. + * Matrix packagers & maintainers. + * Companies building products or services on Matrix. + * Bridge developers. + * Bot developers. + * Widget developers. + * Server developers. + * Matrix room and community moderators. + * End-users who are using Matrix indirectly via bridges. + * External systems which are bridged into Matrix. + * Anyone using Matrix for data communications. "Greater benefit" is defined as maximising: - * the number of Matrix-native end-users reachable on the open Matrix network - * the number of regular users on the Matrix network (e.g. 30-day retained federated users) - * the number of online servers in the open federation - * the number of developers building on Matrix - * the number of independent implementations which use Matrix - * the number of bridged end-users reachable on the open Matrix network - * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) - * the ability for users to discover content on their terms (empowering them to select what to see and what not to see) + * the number of Matrix-native end-users reachable on the open Matrix network. + * the number of regular users on the Matrix network (e.g. 30-day retained federated users). + * the number of online servers in the open federation. + * the number of developers building on Matrix. + * the number of independent implementations which use Matrix. + * the number of bridged end-users reachable on the open Matrix network. + * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam). + * the ability for users to discover content on their terms (empowering them to select what to see and what not to see). * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other - external material) + external material). N.B. that we consider success to be the growth of the open federated network rather than closed deployments. For example, if WhatsApp adopted Matrix it @@ -129,16 +129,16 @@ As Matrix evolves, it's critical that the Spec Core Team and Guardians are aligned on the overall philosophy of the project, particularly in more subjective areas. The values we follow are: - * Supporting the whole long-term ecosystem rather than individual stakeholder gain - * Openness rather than proprietary lock-in - * Interoperability rather than fragmentation - * Cross-platform rather than platform-specific - * Collaboration rather than competition - * Accessibility rather than elitism - * Transparency rather than stealth - * Empathy rather than contrariness - * Pragmatism rather than perfection - * Proof rather than conjecture + * Supporting the whole long-term ecosystem rather than individual stakeholder gain. + * Openness rather than proprietary lock-in. + * Interoperability rather than fragmentation. + * Cross-platform rather than platform-specific. + * Collaboration rather than competition. + * Accessibility rather than elitism. + * Transparency rather than stealth. + * Empathy rather than contrariness. + * Pragmatism rather than perfection. + * Proof rather than conjecture. Patent encumbered IP is strictly prohibited from being added to the standard. @@ -198,19 +198,19 @@ Members must arrange their own funding for their time. Responsibilities include: - * Reviewing Matrix Spec Change proposals and Spec PRs + * Reviewing Matrix Spec Change proposals and Spec PRs. * Contributing to and reviewing reference implementations of Matrix Spec Change - proposals + proposals. - * Shepherding Matrix Spec Changes on behalf of authors where needed + * Shepherding Matrix Spec Changes on behalf of authors where needed. - * Triaging Matrix Spec issues + * Triaging Matrix Spec issues. - * Coordinating reference implementations + * Coordinating reference implementations. * Ensuring the code of conduct for +matrix:matrix.org community rooms is - maintained and applied + maintained and applied. If members are absent (uncontactable) for more than 8 weeks without prior agreement, they will be assumed to have left the project. @@ -283,18 +283,18 @@ is following its guiding principles, and provide a safety mechanism if the structure of the Spec Core Team runs into trouble. In practice, this means that: - * Guardians must approve changes to the Spec Core Team - * Guardians must keep each other honest, providing a ‘checks and balances’ + * Guardians must approve changes to the Spec Core Team. + * Guardians must keep each other honest, providing a ‘checks and balances’. mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. * Guardians may appoint/dismiss members of the Spec Core Team who are in serious breach of the guiding principles. This overrides the unanimous consent requirement for the Spec Core Team when appointing new members. * Guardians may also override deadlocks when appointing a Spec Core Team leader - (with a >= 75% majority) + (with a >= 75% majority). * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets - (e.g. redistributing donations) + (e.g. redistributing donations). * In future, Guardians may also be responsible for ensuring staff are hired by the Foundation to support administrative functions and other roles required to facilitate the Foundation's mission. @@ -354,46 +354,46 @@ community. Responsibilities include: * Helping ensure the quality of the projects' code repositories - * Ensuring all commits are reviewed - * Ensuring that all projects follow the Matrix spec - * Helping architect the implementations - * Contributing code to the implementations + * Ensuring all commits are reviewed. + * Ensuring that all projects follow the Matrix spec. + * Helping architect the implementations. + * Contributing code to the implementations. * Fostering contributions and engaging with contributors constructively in a - way that fosters a healthy and happy community - * Following the Guiding Principles and promoting them within the community + way that fosters a healthy and happy community. + * Following the Guiding Principles and promoting them within the community. Code Core Team members must arrange their own funding for their time. ## Functions of the Foundation - * Independent legal entity which acts as neutral custodian of Matrix - * Gathers donations + * Independent legal entity which acts as neutral custodian of Matrix. + * Gathers donations. * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: - * Owns the matrix.org domain and branding + * Owns the matrix.org domain and branding. * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. - * Owns the IP of the website - * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) - * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector) - * Publishes the spec - * Responsible for tools and documentation that supports the spec - * Responsible for ensuring reference implementations and test suite exists for the spec - * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) + * Owns the IP of the website. + * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc). + * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector). + * Publishes the spec. + * Responsible for tools and documentation that supports the spec. + * Responsible for ensuring reference implementations and test suite exists for the spec. + * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects). * Manages any future IANA-style allocations for Matrix, such as: - * mx:// URI scheme - * TCP port 8448 + * mx:// URI scheme. + * TCP port 8448. * .well-known URIs * Ensures that Matrix promotion is happening (e.g. ensuring that meetups & - events & community activity is supported) + events & community activity is supported). In future: * contracts entities to work on Matrix if such contracts help the Foundation to fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance - kits) + kits). * manages a "Made for Matrix" certification process? (to confirm that products - are actually compatible with Matrix) + are actually compatible with Matrix). ## Timings From cc6b6ea021645f93b339bc17df1cfaf92f079fd2 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:31:08 +0000 Subject: [PATCH 048/282] clarify wording for trojan horsen --- proposals/1779-open-governance.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 58e43b80..38469a06 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -173,13 +173,13 @@ toxic culture of bullying or competitive infighting. Folks need to be able to compromise; we are not building a culture of folks pushing their personal agendas at the expense of the overall project. -We are particularly vigilant against 'trojan horse' additions to the spec - -features which only benefit particular players, or are designed to somehow -cripple or fragment the open protocol and ecosystem in favour of competitive -advantage. Commercial players are of course free to build proprietary -implementations, or use custom event types, or even custom API extensions (e.g. -more efficient network transports) - but implementations must fall back to -interoperating correctly with the rest of the ecosystem. +The team should be particularly vigilant against 'trojan horse' additions to the +spec - features which only benefit particular players, or are designed to +somehow cripple or fragment the open protocol and ecosystem in favour of +competitive advantage. Commercial players are of course free to build +proprietary implementations, or use custom event types, or even custom API +extensions (e.g. more efficient network transports) - but implementations must +fall back to interoperating correctly with the rest of the ecosystem. ### Spec Core Team logistics From 8f4e1d9686993097df415ee71bf4dd16d4257b34 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:32:14 +0000 Subject: [PATCH 049/282] clarify guardian selection wording --- proposals/1779-open-governance.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 38469a06..7fbb786f 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -50,9 +50,9 @@ responsible for ensuring that the Foundation (and by extension the Spec Core Team) keeps on mission and neutrally protects the development of Matrix. Guardians are typically independent of the commercial Matrix ecosystem and may even not be members of today’s Matrix community, but are deeply aligned with the -mission of the project, and who are respected and trusted by the wider community -to uphold the guiding principles of the Foundation and keep the other Guardians -honest. +mission of the project. Guardians are selected to be respected and trusted by +the wider community to uphold the guiding principles of the Foundation and keep +the other Guardians honest. In other words; the Spec Core Team builds the spec, and the Guardians provide an independent backstop to ensure the spec evolves in line with the Foundation's From 80b9c83ccea64d31732a35642653eeb326cf2f20 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 10:00:17 +0000 Subject: [PATCH 050/282] clarify dual-hatted guardians & spec core teamistas --- proposals/1779-open-governance.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 7fbb786f..9f6a49d3 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -270,7 +270,9 @@ The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to the governance process also require a 75% positive approval from the Guardians (acting as a formal decision of the Foundation's Directors), in order to ensure -changes are aligned with the Foundation's mission. +changes are aligned with the Foundation's mission. For avoidance of doubt, Spec +Core Team votes and Guardians' votes are distinct and a person having both hats +has to vote independently on both forums with the respective hat on. ## The Guardians From 3b86fa0e3c5ef217e112cd1bcf3dc4c082638520 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Thu, 17 Jan 2019 23:56:49 +0000 Subject: [PATCH 051/282] incorporating delph & vdh reviews --- proposals/1779-open-governance.md | 100 ++++++++++++++++++------------ specification/proposals_intro.rst | 22 +++---- 2 files changed, 72 insertions(+), 50 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9f6a49d3..dde9aeac 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -69,9 +69,9 @@ We believe: * People should have full control over their own communication. - * People should not be locked into centralised communication silos, but free to - pick who they choose to host their communication without limiting who they - can reach. + * People should not be locked into centralised communication silos, but instead + be free to pick who they choose to host their communication without limiting + who they can reach. * The ability to converse securely and privately is a basic human right. @@ -80,9 +80,9 @@ We believe: ### Mission -The Matrix.org Foundation exists to act as a neutral custodian for Matrix and +The Matrix.org Foundation exists to act as a neutral custodian for Matrix and to nurture it as efficiently as possible as a single unfragmented standard, for the -greater benefit of the whole ecosystem; not benefiting or privileging any single +greater benefit of the whole ecosystem, not benefiting or privileging any single player or subset of players. For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix @@ -185,8 +185,8 @@ fall back to interoperating correctly with the rest of the ecosystem. The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. Roughly half the members are expected to be from the historical core team -(similar to Rust). The team must have 5 members to be quorate, with the aim of -generally having between 7 and 9 members. +(similar to Rust). The team must have 5 members to be able to function, with +the aim of generally having between 7 and 9 members. In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; IS API; Crypto), but as a starting point we are beginning with a single core @@ -221,13 +221,14 @@ of the team and the Guardians on doing so. New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. -Members can be removed from the team if >= 75% of the team agrees they are no -longer following the goals and guiding principles of the project. (The 75% is -measured of the whole team, including the member in question) +Members can be removed from the team if 75% of the current members approves and +agrees they are no longer following the goals and guiding principles of the +project. (The 75% is measured of the whole team, including the member in +question). -Guardians act as a backstop, and can appoint or remove Spec Core Team members -(requiring a 75% consensus threshold between the Guardians) if the Spec Core -Team is unable to function or is failing to align with the Foundation's mission. +Guardians act as a safety net, and can appoint or remove Spec Core Team members +(requiring approval by 75% of the current Guardians) if the Spec Core Team is +unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information exchange between the Guardians and the Spec Core Team, @@ -236,12 +237,14 @@ and to represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to consensus in the event of failing to get agreement on a Matrix Spec Change. Every 12 months, a vote of confidence is held in the project lead, requiring the -confidence of 75% of the team for the lead to be renewed. There is no maximum -term for the project lead. The lead may be removed by the core team at any -point (with 75% majority), and may resign the role at any point (notifying the -team and the Guardians). The lead automatically resigns the role if they resign -from the Spec Core Team. Resignation automatically triggers selection of a new -lead, who must be selected from the existing Spec Core Team. +approval of 75% of the current Spec Core Team members for the lead to be +renewed. There is no maximum term for the project lead. The lead may be +removed by the core team at any point (requiring 75% approval of current +members), and may resign the role at any point (notifying the team and the +Guardians). The lead automatically resigns the role if they resign from the +Spec Core Team. Resignation automatically triggers selection of a new lead, who +must be selected from the existing Spec Core Team with 75% approval from current +members within 14 days. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec @@ -259,21 +262,24 @@ The initial Spec Core Team (and their domain areas) is: * Alexey Rusakov (Clients on behalf of Community) * TBD -MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment -Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC -process). +MSCs require approval by 75% of the current members of the Spec Core Team to +proceed to Final Comment Period (see https://matrix.org/docs/spec/proposals for +the rest of the MSC process). Even though a threshold of only 75% is required for approval, the Spec Core Team is expected to seek consensus on MSCs. The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to -the governance process also require a 75% positive approval from the Guardians +the governance process also require approval by 75% of the current Guardians (acting as a formal decision of the Foundation's Directors), in order to ensure changes are aligned with the Foundation's mission. For avoidance of doubt, Spec Core Team votes and Guardians' votes are distinct and a person having both hats has to vote independently on both forums with the respective hat on. +Spec Core Team decisions (e.g. appointing/removing members and lead) +should be published openly and transparently for the public. + ## The Guardians *This section will be used as the basis for the legal responsibilities of @@ -285,16 +291,23 @@ is following its guiding principles, and provide a safety mechanism if the structure of the Spec Core Team runs into trouble. In practice, this means that: - * Guardians must approve changes to the Spec Core Team. + + * Guardians are responsible for ensuring the Spec Core Team continues to + function, and have the power to appoint/dismiss members of the spec core team + (with the agreement of 75% of the Guardians) to address issues with the Spec + Core Team. * Guardians must keep each other honest, providing a ‘checks and balances’. mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. - * Guardians may appoint/dismiss members of the Spec Core Team who are in serious - breach of the guiding principles. This overrides the unanimous consent - requirement for the Spec Core Team when appointing new members. + * Guardians may dismiss members of the Spec Core Team who are in serious + breach of the guiding principles. + * Guardians may appoint members of the Spec Core Team to break deadlocks in the + unanimous consent requirement for the Spec Core Team when appointing new + members. * Guardians may also override deadlocks when appointing a Spec Core Team leader - (with a >= 75% majority). - * Guardians must approve changes to the Guiding Principles (above) + (with approval of 75% of the current Guardians). + * Guardians must approve changes to the above Guiding Principles (with approval + of 75% of the current Guardians) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations). * In future, Guardians may also be responsible for ensuring staff are hired by @@ -303,8 +316,14 @@ In practice, this means that: * As well as the Spec Core Team committee, they may also oversee committees for other areas such as marketing Matrix.org, registering custom event types, or "Made for Matrix" certification. - * It's likely a subset of Guardians will be hands-on for day-to-day - administrative purposes, whilst the others act to keep them in balance. + * Guardians are responsible for choosing if, when and how staff are located by + the Foundation to fill administrative and other functions required to + facilitate the Foundations' mission. + * Guardians are responsible for choosing if and when additional committees are + formed, and to oversee those committees. + * Guardians are not required to be involved on a day-to-day basis, however + those not taking a hands on approach are required to monitor to ensure a + suitable balance is kept by those that do. Guardians are chosen typically to be independent of the commercial Matrix ecosystem (and especially independent from New Vector), and may even not be @@ -313,18 +332,18 @@ the mission of the project, and respected and trusted by the wider community to uphold the guiding principles of the Foundation and keep the other Guardians honest. -Guardians are responsible for maintaining and updating the Guiding -Principles and Articles of Association of the Foundation if/when -necessary. Changes to the Guiding Principles require a 75% majority from the -Guardians and are passed as a 'special resolution' of the board. +Guardians are responsible for maintaining and updating the Guiding Principles +and Articles of Association of the Foundation if/when necessary. Changes to the +Guiding Principles require approval from 75% of the current Guardians and are +passed as a 'special resolution' of the board. -New Guardians may be appointed with a 75% majority by the board. +New Guardians may be appointed with approval from 75% of the current Guardians. Guardians may resign at any time, with notification to the board. -Guardians may be removed due to serious breach of the guiding principles with a -75% majority of the other Guardians, or if absent from 3 consecutive board -meetings, or if they are legally disqualified from acting as a Director. +Guardians may be removed due to serious breach of the guiding principles with +approval by 75% of the other current Guardians, or if absent from 3 consecutive +board meetings, or if they are legally disqualified from acting as a Director. We aim to recruit roughly 5 Guardians. The initial Guardians are: @@ -340,6 +359,9 @@ Foundation relative to Matthew & Amandine’s day jobs at New Vector. Guardians must arrange their own funding for their time. +Guardian decisions (e.g. appointing/removing guardians; changes to the spec core +team; etc) should be published openly and transparently for the public. + ## The Code Core Team (aka The Core Team) The "Core Team" (or the "Code Core Team", to disambiguate from the Spec Core diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 1e93a22a..87828916 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -19,10 +19,10 @@ proposal being accepted, then actually having your ideas implemented as committed changes to the `Specification repository `_. -Meet the `members of the Core Team +Meet the `members of the Spec Core Team `_, a group of individuals tasked with ensuring the spec process is as smooth and painless as -possible. Members of the Core Team will do their best to participate in +possible. Members of the Spec Core Team will do their best to participate in discussion, summarise when things become long-winded, and generally try to act towards the benefit of everyone. As a majority, team members have the ability to change the state of a proposal, and individually have the final say in @@ -74,7 +74,7 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture -Please see [MSC1779](https://github.com/matrix-org/matrix-doc/pull/1779) +Please see [MSC1779](https://github.com/matrix-org/matrix-doc/blob/matthew/msc1779/proposals/1779-open-governance.md) for full details of the project's Guiding Principles. Technical notes @@ -213,25 +213,25 @@ follows: viewpoints and get consensus, but this can sometimes be time-consuming (or the author may be biased), in which case an impartial 'shepherd' can be assigned to help guide the proposal through this process instead. A shepherd is - typically a neutral party from the Core Team or an experienced member of + typically a neutral party from the Spec Core Team or an experienced member of the community. There is no formal process for assignment. Simply ask for a shepherd to help get your proposal through and one will be assigned based on availability. Having a shepherd is not a requirement for proposal acceptance. -- Members of the Core Team and community will review and discuss the PR in the +- Members of the Spec Core Team and community will review and discuss the PR in the comments and in relevant rooms on Matrix. Discussion outside of GitHub should be summarised in a comment on the PR. -- When a member of the Core Team believes that no new discussion points are +- When a member of the Spec Core Team believes that no new discussion points are being made, they will propose a motion for a final comment period (FCP), along with a *disposition* of either merge, close or postpone. This FCP is provided to allow a short period of time for any invested party to provide a final objection before a major decision is made. If sufficient reasoning is given, an FCP can be cancelled. It is often preceded by a comment summarising the current state of the discussion, along with reasoning for its occurrence. -- A concern can be raised by a Core Team member at any time, which will block - an FCP from beginning. An FCP will only begin when a **majority** of core - team members agree on its outcome, and all existing concerns have been +- A concern can be raised by a Spec Core Team member at any time, which will block + an FCP from beginning. An FCP will only begin when 75% of the members of the + Spec Core Team team agree on its outcome, and all existing concerns have been resolved. - The FCP will then begin and last for 5 days, giving anyone else some time to speak up before it concludes. On its conclusion, the disposition of the FCP @@ -321,7 +321,7 @@ Lifetime States Name GitHub Label Description =============================== ============================= ==================================== Proposal Drafting and Feedback N/A A proposal document which is still work-in-progress but is being shared to incorporate feedback -Proposal In Review proposal-in-review A proposal document which is now ready and waiting for review by the Core Team and community +Proposal In Review proposal-in-review A proposal document which is now ready and waiting for review by the Spec Core Team and community Proposed Final Comment Period proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period Final Comment Period final-comment-period A proposal document which has reached final comment period either for merge, closure or postponement Final Commment Period Complete finished-final-comment-period The final comment period has been completed. Waiting for a demonstration implementation @@ -342,7 +342,7 @@ pull request trackers of the `matrix-doc `_ repo. We use labels and some metadata in MSC PR descriptions to generate this page. -Labels are assigned by the Core Team whilst triaging the proposals based on those +Labels are assigned by the Spec Core Team whilst triaging the proposals based on those which exist in the `matrix-doc `_ repo already. From 20b9a33b1240a45cae4419ba19ff5670195cfb99 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 18 Jan 2019 22:39:26 +0000 Subject: [PATCH 052/282] clarify 100% threshold for new spec core team additions --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index dde9aeac..779e94ab 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -218,7 +218,7 @@ agreement, they will be assumed to have left the project. Spec Core Team members can resign whenever they want, but must notify the rest of the team and the Guardians on doing so. -New additions to the team require 100% consent from the current team members. +New additions to the team must be approved by all current members of the team. Membership has to be formally proposed by someone already on the Spec Core Team. Members can be removed from the team if 75% of the current members approves and From 41c14c9e7df76efbbcc70eab2b93669a64909a40 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 18 Jan 2019 22:56:19 +0000 Subject: [PATCH 053/282] uhoreg tweaks Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 779e94ab..0a1cfca4 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -377,7 +377,7 @@ the core team is also signified by membership of the +matrix:matrix.org Matrix community. Responsibilities include: - * Helping ensure the quality of the projects' code repositories + * Helping ensure the quality of the projects' code repositories. * Ensuring all commits are reviewed. * Ensuring that all projects follow the Matrix spec. * Helping architect the implementations. @@ -400,7 +400,7 @@ Code Core Team members must arrange their own funding for their time. * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc). * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector). * Publishes the spec. - * Responsible for tools and documentation that supports the spec. + * Responsible for tools and documentation that support the spec. * Responsible for ensuring reference implementations and test suite exists for the spec. * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects). * Manages any future IANA-style allocations for Matrix, such as: @@ -412,11 +412,11 @@ Code Core Team members must arrange their own funding for their time. In future: - * contracts entities to work on Matrix if such contracts help the Foundation to + * Contracts entities to work on Matrix if such contracts help the Foundation to fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance kits). - * manages a "Made for Matrix" certification process? (to confirm that products + * Manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix). ## Timings From a358e2d4d8e8d1128dc308306eaec91b188ed547 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 28 Jan 2019 14:57:59 +0000 Subject: [PATCH 054/282] rst --- specification/proposals_intro.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 9274b2a9..44fcf4b1 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -111,6 +111,7 @@ engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: + 1. If the feature would benefit the whole Matrix ecosystem and is aligned with the guiding principles above, then it should be supported by the spec. 2. If the spec already makes the feature possible without changing any of the @@ -122,6 +123,7 @@ applied: 3rd party behaviour. As a worked example: + 1. Video conferencing is clearly a feature which would benefit the whole ecosystem, and so the spec should find a way to make it happen. 2. Video conferencing can be achieved by widgets without requiring any From f9a00fc943d9ae49ae32845a7a971578ff84c4a0 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 13 Mar 2019 23:25:11 -0400 Subject: [PATCH 055/282] clarify that not understanding a verification method should not auto-cancel --- proposals/1717-key_verification.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index b536be0e..10ccf785 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -28,7 +28,10 @@ appropriate `m.key.verification.cancel` message is received. If Bob chooses to reject the key verification request, Bob's client should send a `m.key.verification.cancel` message to Alice's device. If Bob's client does not understand any of the methods offered, it should display a message to Bob -saying so. +saying so. However, it should not send a `m.key.verification.cancel` message +to Alice's device unless Bob chooses to reject the verification request, as Bob +may have another device that is capable of verifying using one of the given +methods. To initiate a key verification process, Bob's device sends a `to_device` event to one of Alice's devices with the `type` set to `m.key.verification.start`. From 4842a718d8ab3c4000e86270d3ef3aea6cf42154 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 13 Mar 2019 23:39:00 -0400 Subject: [PATCH 056/282] fill in some of the other sections --- proposals/1717-key_verification.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index 10ccf785..9296cf40 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -132,10 +132,18 @@ convention. ## Tradeoffs -## Potential issues +Rather than broadcasting verification requests to Bob's devices, Alice could +simply send an `m.key.verification.start` request to a single device. However, +this would require Alice to choose the right device to send to, which may be +hard for Alice to do if, for example, Bob has many devices, or if his devices +have similar names. ## Security considerations +An attacker could try to spam a user with verification requests. Clients +should take care that such requests do not interfere with a user's use of the +client. + ## Conclusion This proposal presents common event definitions for use by key verification From 1749a91344786815ecbae43a53e19fb53157c307 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 13 Mar 2019 23:41:43 -0400 Subject: [PATCH 057/282] make MSC mentions into links --- proposals/1717-key_verification.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index 9296cf40..02018ef9 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -2,8 +2,10 @@ Key verification is an essential part of ensuring that end-to-end encrypted messages are secure. Matrix may support multiple verification methods that -require sending events; in fact, two such methods (such as MSC1267 and MSC1543) -have already been proposed. +require sending events; in fact, two such methods (such as [MSC +1267](https://github.com/matrix-org/matrix-doc/issues/1267) and [MSC +1543](https://github.com/matrix-org/matrix-doc/issues/1543)) have already been +proposed. This proposal tries to present a common framework for verification methods to use, and presents a way to request key verification. From 696e568fb2101480826d873fac83897ae904472a Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 26 Mar 2019 15:10:14 -0400 Subject: [PATCH 058/282] add some clarifications --- proposals/1717-key_verification.md | 46 +++++++++++++++++++----------- 1 file changed, 30 insertions(+), 16 deletions(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index 02018ef9..6c9e2341 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -19,26 +19,40 @@ Java package naming convention. If Alice wants to verify keys with Bob, Alice's device may send `to_device` events to Bob's devices with the `type` set to `m.key.verification.request`, as -described below. The event lists the verification methods that Alice's device -supports. Upon receipt of this message, Bob's client should prompt him to -verify keys with Alice using one of the applicable methods. In order to avoid -displaying stale key verification prompts, if Bob does not interact with the -prompt, it should be automatically hidden 10 minutes after the message is sent -(according to the `timestamp` field), or 2 minutes after the client receives -the message, whichever comes first. The prompt should also be hidden if an -appropriate `m.key.verification.cancel` message is received. If Bob chooses to -reject the key verification request, Bob's client should send a -`m.key.verification.cancel` message to Alice's device. If Bob's client does -not understand any of the methods offered, it should display a message to Bob -saying so. However, it should not send a `m.key.verification.cancel` message -to Alice's device unless Bob chooses to reject the verification request, as Bob -may have another device that is capable of verifying using one of the given -methods. +described below. The `m.key.verification.request` messages should all have the +same `transaction_id`, and are considered to be a single request. Thus, for +example, if Bob rejects the request on one device, then the entire request +should be considered as rejected across all of his devices. Similarly, if Bob +accepts the request on one device, that device is now in charge of completing +the key verification, and Bob's other devices no longer need to be involved. + +The `m.key.verification.request` event lists the verification methods that +Alice's device supports, and upon receipt of this message, Bob's client should +prompt him to verify keys with Alice using one of the applicable methods. In +order to avoid displaying stale key verification prompts, if Bob does not +interact with the prompt, it should be automatically hidden 10 minutes after +the message is sent (according to the `timestamp` field), or 2 minutes after +the client receives the message, whichever comes first. The prompt should also +be hidden if an appropriate `m.key.verification.cancel` message is received. + +If Bob chooses to reject the key verification request, Bob's client should send +a `m.key.verification.cancel` message to Alice's device. This indicates to +Alice that Bob does not wish to verify keys with her. In this case, Alice's +device should send an `m.key.verification.cancel` message to all of Bob's +devices to notify them that the request has been rejected. + +If one of Bob's clients does not understand any of the methods offered, it +should display a message to Bob saying so. However, it should not send a +`m.key.verification.cancel` message to Alice's device unless Bob chooses to +reject the verification request, as Bob may have another device that is capable +of verifying using one of the given methods. To initiate a key verification process, Bob's device sends a `to_device` event to one of Alice's devices with the `type` set to `m.key.verification.start`. This may either be done in response to an `m.key.verification.request` message, -or can be done independently. If Alice's device receives an +or can be done independently. If it is done in response to an +`m.key.verification.request` messsage, it should use the same `transaction_id` +as the `m.key.verification.request` message. If Alice's device receives an `m.key.verification.start` message in response to an `m.key.verification.request` message, it should send an `m.key.verification.cancel` message to Bob's other devices that it had From 743eeca27a821b0939780d4fd3713c3f514b3660 Mon Sep 17 00:00:00 2001 From: Neil Johnson Date: Fri, 5 Apr 2019 15:19:22 +0100 Subject: [PATCH 059/282] MSC to remove prev_content from the essential keys list --- ...ove-prev_event-from-essential-keys-list.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 proposals/1954-remove-prev_event-from-essential-keys-list.md diff --git a/proposals/1954-remove-prev_event-from-essential-keys-list.md b/proposals/1954-remove-prev_event-from-essential-keys-list.md new file mode 100644 index 00000000..992b3e80 --- /dev/null +++ b/proposals/1954-remove-prev_event-from-essential-keys-list.md @@ -0,0 +1,64 @@ +# Remove prev_content from the essential keys list + +Matrix supports the concept of event redaction. The ability to redact rather +than delete is necessary because some events e.g. membership events are +essential to the protocol and _cannot_ be deleted. Therefore we do not delete +events outright and instead redact them. This involves removing all keys from +an event that are not required by the protocol. The stripped down event is +thereafter returned anytime a client or remote server requests it. + + +## Proposal + +[The redaction algorithm](https://matrix.org/docs/spec/client_server/r0.4.0.html#redactions) +defines which keys must be retained through a redaction. Currently it lists +```prev_content``` as a key to retain, though in practice there is no need to +do so at the protocol level. + +The proposal is simply to remove ```prev_content``` from the essential keys +list. + +Note: the inclusion of ```prev_content``` in the essential keys list was +unintentional and should be considered a spec bug. Synapse (and other server +implementations) have not implemented the bug and already omit +```prev_content``` from redacted events. + + +## Tradeoffs + +When sending events over federation the events are [hashed and +signed](https://matrix.org/docs/spec/server_server/unstable.html#adding-hashes-and-signatures-to-outgoing-events), +this involves operating not only on the original event but also the redacted +form of the event. The redacted hash and redacted signed event are necessary if +the event is ever redacted in future. As a result, any change of the essential +keys list must be managed carefully. If disparate servers implement different +versions of the redaction algorithm (for a given event) attempts to send the +event over federation will fail. + +We _could_ manage this change via room versioning and create a new room +version that implements this MSC. However, because the federation already +omits the ```prev_content``` key by convention, implementing this MSC only in +the new room version would mean that the entire existing federation would not +be spec compliant. + +As a result it seems pragmatic to have the spec reflect reality, acknowledge +that the spec and federation have deviated and instead update the spec +retrospectively to describe the de-facto redaction algorithm. + +## Potential issues + +It is theoretically possible that a closed federation could exist whose servers +do follow the spec as is. This MSC would render those servers uncompliant with +the spec. On balance this seems unlikely and in the worst case those +implementors could add the change to a subsequent room version, eventually +reaching spec consistency as older room versions are deprecated. + +## Security considerations + +I am unaware of any security issues related to this proposal, but can certainly +see issues with a precedent that the federation deviates from the spec. + +## Conclusions +Removing ```prev_content``` is pragmatic response to the current situation. It +alligns the federation and the spec, and does so in a way that removes +unecessary overhead. From b41fbc86b6d07a93c10f074c97befe9fe7e4733e Mon Sep 17 00:00:00 2001 From: Neil Johnson Date: Tue, 9 Apr 2019 13:56:45 +0100 Subject: [PATCH 060/282] add further potential issues and security concerns --- ...emove-prev_event-from-essential-keys-list.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/proposals/1954-remove-prev_event-from-essential-keys-list.md b/proposals/1954-remove-prev_event-from-essential-keys-list.md index 992b3e80..1cd0fd36 100644 --- a/proposals/1954-remove-prev_event-from-essential-keys-list.md +++ b/proposals/1954-remove-prev_event-from-essential-keys-list.md @@ -53,10 +53,23 @@ the spec. On balance this seems unlikely and in the worst case those implementors could add the change to a subsequent room version, eventually reaching spec consistency as older room versions are deprecated. +Another scenario is that a client may redact events according to the spec as is +and persist prev_content through the redaction, thereby diverting from that on +the server(s). Client authors will have to update their code to drop +```prev_content``` - however, given that prev_content should not be used in +important calculations and/or visualisations, this ought to be relatively +uninvaisive change. + + ## Security considerations -I am unaware of any security issues related to this proposal, but can certainly -see issues with a precedent that the federation deviates from the spec. +A further reason to support removal of ```prev_content``` is the case where a +malicious user adds illegal or abusive content into a state event and then +overwrites that state event. The content would then be preserved through the +redaction. + +Additionally, there are plenty of reasons to have security concerns over a +precedent that the federation can deviate from the spec. ## Conclusions Removing ```prev_content``` is pragmatic response to the current situation. It From 24e0ec4bcecee55e34d54bd8a747f619b57384b5 Mon Sep 17 00:00:00 2001 From: aqtusia <47819902+aqtusia@users.noreply.github.com> Date: Sun, 14 Apr 2019 18:17:44 +0200 Subject: [PATCH 061/282] Replace /bind with /3pid/bind --- api/identity/associations.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/identity/associations.yaml b/api/identity/associations.yaml index edd43f5d..152a0a9b 100644 --- a/api/identity/associations.yaml +++ b/api/identity/associations.yaml @@ -90,7 +90,7 @@ paths: } schema: $ref: "../client-server/definitions/errors/error.yaml" - "/bind": + "/3pid/bind": post: summary: Publish an association between a session and a Matrix user ID. description: |- From 2eb9708f7fcdead5e21ad22bd8a054e1ae4bfd31 Mon Sep 17 00:00:00 2001 From: aqtusia <47819902+aqtusia@users.noreply.github.com> Date: Sun, 14 Apr 2019 18:21:59 +0200 Subject: [PATCH 062/282] Replace /unbind with /3pid/unbind --- proposals/1915-unbind-identity-server-param.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1915-unbind-identity-server-param.md b/proposals/1915-unbind-identity-server-param.md index 053552f6..6817ece3 100644 --- a/proposals/1915-unbind-identity-server-param.md +++ b/proposals/1915-unbind-identity-server-param.md @@ -55,12 +55,12 @@ from the given identity server. ### Identity Server 3PID Unbind API -Add `POST /_matrix/identity/api/v1/unbind` with `mxid` and `threepid` fields. +Add `POST /_matrix/identity/api/v1/3pid/unbind` with `mxid` and `threepid` fields. The `mxid` is the user's `user_id` and `threepid` is a dict with the usual `medium` and `address` fields. If the server returns a 400, 404 or 501 HTTP error code then the homeserver -should assume that the identity server doesn't support the `/unbind` API, unless +should assume that the identity server doesn't support the `/3pid/unbind` API, unless it returns a specific matrix error response (i.e. the body is a JSON object with `error` and `errcode` fields). @@ -73,7 +73,7 @@ The identity server should authenticate the request in one of two ways: Example: ``` -POST /_matrix/identity/api/v1/unbind HTTP/1.1 +POST /_matrix/identity/api/v1/3pid/unbind HTTP/1.1 { "mxid": "@foobar:example.com", From 911fb94ea0d7213ea363293c1db39ca4391669f7 Mon Sep 17 00:00:00 2001 From: Neil Johnson Date: Mon, 15 Apr 2019 17:08:09 +0100 Subject: [PATCH 063/282] typos --- .../1954-remove-prev_event-from-essential-keys-list.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1954-remove-prev_event-from-essential-keys-list.md b/proposals/1954-remove-prev_event-from-essential-keys-list.md index 1cd0fd36..735fac01 100644 --- a/proposals/1954-remove-prev_event-from-essential-keys-list.md +++ b/proposals/1954-remove-prev_event-from-essential-keys-list.md @@ -48,7 +48,7 @@ retrospectively to describe the de-facto redaction algorithm. ## Potential issues It is theoretically possible that a closed federation could exist whose servers -do follow the spec as is. This MSC would render those servers uncompliant with +do follow the spec as is. This MSC would render those servers non-compliant with the spec. On balance this seems unlikely and in the worst case those implementors could add the change to a subsequent room version, eventually reaching spec consistency as older room versions are deprecated. @@ -57,8 +57,8 @@ Another scenario is that a client may redact events according to the spec as is and persist prev_content through the redaction, thereby diverting from that on the server(s). Client authors will have to update their code to drop ```prev_content``` - however, given that prev_content should not be used in -important calculations and/or visualisations, this ought to be relatively -uninvaisive change. +important calculations and/or visualisations, this ought to be a relatively +non-invasive change. ## Security considerations From 043dddc49061f4a8837a0b9d5dc0fb0c0d79f85d Mon Sep 17 00:00:00 2001 From: Jonas Platte Date: Sat, 20 Apr 2019 21:56:43 +0200 Subject: [PATCH 064/282] Fix a typo in m.call.invite --- event-schemas/schema/m.call.invite | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/event-schemas/schema/m.call.invite b/event-schemas/schema/m.call.invite index ebf09267..65796e1e 100644 --- a/event-schemas/schema/m.call.invite +++ b/event-schemas/schema/m.call.invite @@ -10,7 +10,7 @@ "properties": { "call_id": { "type": "string", - "description": "A unique identifer for the call." + "description": "A unique identifier for the call." }, "offer": { "type": "object", From 14715468bb6aa157601fa271bd3a57507f90b8dd Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Fri, 26 Apr 2019 14:41:19 +0100 Subject: [PATCH 065/282] Make thumbnail dimensions mandatory Fixes #1883 --- api/client-server/content-repo.yaml | 2 ++ changelogs/client_server/newsfragments/1975.clarification | 1 + 2 files changed, 3 insertions(+) create mode 100644 changelogs/client_server/newsfragments/1975.clarification diff --git a/api/client-server/content-repo.yaml b/api/client-server/content-repo.yaml index 5f4e9111..07df18fe 100644 --- a/api/client-server/content-repo.yaml +++ b/api/client-server/content-repo.yaml @@ -202,6 +202,7 @@ paths: type: integer x-example: 64 name: width + required: true description: |- The *desired* width of the thumbnail. The actual thumbnail may not match the size specified. @@ -209,6 +210,7 @@ paths: type: integer x-example: 64 name: height + required: true description: |- The *desired* height of the thumbnail. The actual thumbnail may not match the size specified. diff --git a/changelogs/client_server/newsfragments/1975.clarification b/changelogs/client_server/newsfragments/1975.clarification new file mode 100644 index 00000000..03a9b754 --- /dev/null +++ b/changelogs/client_server/newsfragments/1975.clarification @@ -0,0 +1 @@ +Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. It is somewhat meaningless to request a thumbnail without specifying a desired size, and Synapse has never permitted such requests. \ No newline at end of file From 6cdc8982fa84fc25b6ea9d20d241df8e2795b643 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Fri, 26 Apr 2019 15:58:31 +0100 Subject: [PATCH 066/282] jenkins is dead, long live buildkite --- .buildkite/pipeline.yaml | 11 +++++++++++ jenkins.sh | 3 --- scripts/requirements.txt | 8 ++++++-- 3 files changed, 17 insertions(+), 5 deletions(-) create mode 100644 .buildkite/pipeline.yaml delete mode 100755 jenkins.sh diff --git a/.buildkite/pipeline.yaml b/.buildkite/pipeline.yaml new file mode 100644 index 00000000..e98d7026 --- /dev/null +++ b/.buildkite/pipeline.yaml @@ -0,0 +1,11 @@ +steps: + - label: ":books: Build spec" + command: + - python3 -m venv env + - env/bin/pip install -r scripts/requirements.txt + - ". env/bin/activate; scripts/generate-matrix-org-assets" + artifact_paths: + - assets.tar.gz + plugins: + - docker#v3.0.1: + image: "python:3.6" diff --git a/jenkins.sh b/jenkins.sh deleted file mode 100755 index 79b77acb..00000000 --- a/jenkins.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh - -exec ./scripts/test-and-build.sh diff --git a/scripts/requirements.txt b/scripts/requirements.txt index 2a7d7ff8..66027f91 100644 --- a/scripts/requirements.txt +++ b/scripts/requirements.txt @@ -4,8 +4,12 @@ docutils >= 0.14 pygments >= 2.2.0 Jinja2 >= 2.9.6 -jsonschema >= 2.6.0 + +# jsonschema 3.0.0 objects to the $refs in our schema file. TODO: figure out +# why. +jsonschema >= 2.6.0, < 3.0.0 + PyYAML >= 3.12 requests >= 2.18.4 towncrier == 18.6.0 -six >= 1.11.0 \ No newline at end of file +six >= 1.11.0 From 4e9dc2098ff65acda435797b8d824653289f0e8b Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Tue, 30 Apr 2019 17:43:21 +0100 Subject: [PATCH 067/282] Fix comments which refer to jenkins. (#1981) * Fix comments which refer to jenkins. * Spelling Co-Authored-By: turt2live --- scripts/test-and-build.sh | 2 +- specification/proposals.rst | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/test-and-build.sh b/scripts/test-and-build.sh index 710b03dd..f45e2da6 100755 --- a/scripts/test-and-build.sh +++ b/scripts/test-and-build.sh @@ -30,5 +30,5 @@ go get gopkg.in/fsnotify/fsnotify.v1 # build the spec for matrix.org. # (we don't actually use it on travis, but it's still useful to check we -# can build it. On Jenkins, this is then used to deploy to matrix.org). +# can build it. On Buildkite, this is then used to deploy to matrix.org). ./scripts/generate-matrix-org-assets diff --git a/specification/proposals.rst b/specification/proposals.rst index 371850ab..94878f80 100644 --- a/specification/proposals.rst +++ b/specification/proposals.rst @@ -1,6 +1,6 @@ Tables of Tracked Proposals --------------------------- -This file is autogenerated by a jenkins build process +This file is generated by an automated process on our build server. -View the current live version `at https://matrix.org/docs/spec/proposals `_ +View the current live version `at https://matrix.org/docs/spec/proposals `_. From 7c7bc677fb485d3df0e2dcb52554eb1bf2bd7f28 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 7 May 2019 00:11:14 +0100 Subject: [PATCH 068/282] Trigger matrix.org rebuild --- .buildkite/pipeline.yaml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.buildkite/pipeline.yaml b/.buildkite/pipeline.yaml index e98d7026..6e595120 100644 --- a/.buildkite/pipeline.yaml +++ b/.buildkite/pipeline.yaml @@ -9,3 +9,8 @@ steps: plugins: - docker#v3.0.1: image: "python:3.6" + + - label: "rebuild matrix.org" + trigger: "matrix-dot-org" + async: true + branches: "master" From da82a423084d4bd5a209e73548001fb89024aa35 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 15:40:39 -0700 Subject: [PATCH 069/282] fix grammatical error --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 59d241b6..8d8a2d06 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -182,7 +182,7 @@ process: field as the ``from`` parameter. If the client is tracking the device list of any of the users listed in the response, it marks them as outdated. It combines this list with those already flagged as outdated, and initiates a - |/keys/query|_ requests for all of them. + |/keys/query|_ request for all of them. .. Warning:: From c233535de00e8c66d80fa807535b68f3e9ec291b Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 15:51:32 -0700 Subject: [PATCH 070/282] fix typo --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 59d241b6..9ad80a12 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -460,7 +460,7 @@ Keys can be manually exported from one device to an encrypted file, copied to another device, and imported. The file is encrypted using a user-supplied passphrase, and is created as follows: -1. Encode the sessions a JSON object, formatted as described in `Key export +1. Encode the sessions as a JSON object, formatted as described in `Key export format`_. 2. Generate a 512-bit key from the user-entered passphrase by computing `PBKDF2`_\(HMAC-SHA-512, passphrase, S, N, 512), where S is a 128-bit From 40482f76163951aaf95a2ccef848fc43789ec3f9 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 15:55:07 -0700 Subject: [PATCH 071/282] Add missing period --- specification/appendices/signing_json.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/appendices/signing_json.rst b/specification/appendices/signing_json.rst index 795d6669..8036950e 100644 --- a/specification/appendices/signing_json.rst +++ b/specification/appendices/signing_json.rst @@ -59,7 +59,7 @@ Grammar +++++++ Adapted from the grammar in http://tools.ietf.org/html/rfc7159 removing -insignificant whitespace, fractions, exponents and redundant character escapes +insignificant whitespace, fractions, exponents and redundant character escapes. .. code:: From 3c62b90dfbf58a4ff99c4e7fc403d8737816dc2b Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 16:59:10 -0700 Subject: [PATCH 072/282] Add missing punctuation --- api/client-server/keys.yaml | 2 +- api/server-server/user_keys.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 55f8a5a5..718703fd 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -253,7 +253,7 @@ paths: responses: 200: description: - The claimed keys + The claimed keys. schema: type: object properties: diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 63c74d20..3c59cf81 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -63,7 +63,7 @@ paths: - one_time_keys responses: 200: - description: The claimed keys + description: The claimed keys. schema: type: object properties: From 57cf1aaa96f9767444f42a32f0f7c7d21b92062e Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 19:37:25 -0700 Subject: [PATCH 073/282] Add missing period --- specification/modules/send_to_device.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/send_to_device.rst b/specification/modules/send_to_device.rst index 86288546..cca0338a 100644 --- a/specification/modules/send_to_device.rst +++ b/specification/modules/send_to_device.rst @@ -108,7 +108,7 @@ to_device ToDevice Optional. Information on the send-to-device messages ========= ========= ============================================= Parameter Type Description ========= ========= ============================================= -events [Event] List of send-to-device messages +events [Event] List of send-to-device messages. ========= ========= ============================================= ``Event`` From 20d2fdc288e3e79909db449f99895f1a2ad77764 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 19:51:11 -0700 Subject: [PATCH 074/282] Add changelog --- changelogs/client_server/newsfragments/1988.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/1988.clarification diff --git a/changelogs/client_server/newsfragments/1988.clarification b/changelogs/client_server/newsfragments/1988.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/client_server/newsfragments/1988.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. From abd770419b6169510293e963c373625bfefd0646 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 19:52:38 -0700 Subject: [PATCH 075/282] Add changelog --- changelogs/client_server/newsfragments/1989.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/1989.clarification diff --git a/changelogs/client_server/newsfragments/1989.clarification b/changelogs/client_server/newsfragments/1989.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/client_server/newsfragments/1989.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. From 713e4401b47ff71da6aad21ce2bd781281f48007 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 7 May 2019 19:54:38 -0700 Subject: [PATCH 076/282] Add changelogs --- changelogs/client_server/newsfragments/1991.clarification | 1 + changelogs/server_server/newsfragments/1991.clarification | 1 + 2 files changed, 2 insertions(+) create mode 100644 changelogs/client_server/newsfragments/1991.clarification create mode 100644 changelogs/server_server/newsfragments/1991.clarification diff --git a/changelogs/client_server/newsfragments/1991.clarification b/changelogs/client_server/newsfragments/1991.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/client_server/newsfragments/1991.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. diff --git a/changelogs/server_server/newsfragments/1991.clarification b/changelogs/server_server/newsfragments/1991.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/server_server/newsfragments/1991.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. From 00b7b70c062e78c8bdbce060a56d099540654f9e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 7 May 2019 23:09:39 -0600 Subject: [PATCH 077/282] Create 1992.clarification --- changelogs/client_server/newsfragments/1992.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/1992.clarification diff --git a/changelogs/client_server/newsfragments/1992.clarification b/changelogs/client_server/newsfragments/1992.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/client_server/newsfragments/1992.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. From 383e02835e5c9e60d0f574e69cf7cd6fa2cae262 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 14 May 2019 18:07:58 +0100 Subject: [PATCH 078/282] Words on using m.login.dummy for disambiguation Add some text on how m.login.dummy can be used to distinguish a flow that would otherwise be a subset of other flows. --- specification/client_server_api.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index f9f815f7..b2ff3dbd 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -789,7 +789,14 @@ Dummy Auth :Description: Dummy authentication always succeeds and requires no extra parameters. Its purpose is to allow servers to not require any form of User-Interactive - Authentication to perform a request. + Authentication to perform a request. It can also be used to differentiate + flows where otherwise one flow would be a subset of another flow. eg. if + a server offers flows ``m.login.recaptcha`` and ``m.login.recaptcha, + m.login.email.identity`` and the client completes the recaptcha stage first, + the auth would succeed with the former flow, even if the client was intending + to then complete the email auth stage. A server can instead send flows + ``m.login.recaptcha, m.login.dummy`` and ``m.login.recaptcha, + m.login.email.identity`` to fix the ambiguity. To use this authentication type, clients should submit an auth dict with just the type and session, if provided: From c38581fb868c0d198297d82932d2b8117264bad3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 14 May 2019 19:20:01 +0100 Subject: [PATCH 079/282] Too many spaces Co-Authored-By: Travis Ralston --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index b2ff3dbd..6f65ebb1 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -790,7 +790,7 @@ Dummy Auth Dummy authentication always succeeds and requires no extra parameters. Its purpose is to allow servers to not require any form of User-Interactive Authentication to perform a request. It can also be used to differentiate - flows where otherwise one flow would be a subset of another flow. eg. if + flows where otherwise one flow would be a subset of another flow. eg. if a server offers flows ``m.login.recaptcha`` and ``m.login.recaptcha, m.login.email.identity`` and the client completes the recaptcha stage first, the auth would succeed with the former flow, even if the client was intending From 37871106c6fc6013c17b5711fb93853fff140267 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 21 May 2019 16:21:47 +0100 Subject: [PATCH 080/282] MSC2002: Proposal for adopting MSC1884 as v4 rooms (#2002) --- proposals/2002-rooms-v4.md | 54 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 proposals/2002-rooms-v4.md diff --git a/proposals/2002-rooms-v4.md b/proposals/2002-rooms-v4.md new file mode 100644 index 00000000..defacee1 --- /dev/null +++ b/proposals/2002-rooms-v4.md @@ -0,0 +1,54 @@ +# MSC 2002 - Rooms V4 + +This MSC proposes creating a new room version named v4 to allow servers to switch +event ID grammar to that proposed in MSC1884. + +## Proposal + +The new room version is called "4". The only difference between v4 and v3 is +that v4 rooms use the grammar for defining event IDs as proposed in MSC1884 - +expressing event IDs as url-safe base64 rather than plain base64 for the +convenience of client implementors. + +It is not proposed that servers change the default room version used when +creating new rooms, and it is not proposed that servers recommend upgrading +existing rooms to v4. + +## Rationale and Context + +We would like to default to creating rooms with a reasonably secure room +algorithm in upcoming Matrix 1.0. We do not want to default to creating v3 +rooms due to the inconvenience the v3 event ID grammar might cause, so instead +we are proposing favouring v4. + +Ideally we would also include other room algorithm changes in v4 (e.g. +honouring server signing key validity periods, as per +https://github.com/matrix-org/synapse/issues/4364), but as spec & +implementation work is still ongoing there, we are proposing using v4 as a +room version which can be supported in the wild before Matrix 1.0 and then +used as the initial default for creating rooms. The expectation is for the +versions of the spec which coincide with 1.0 to also support v5 rooms, but in +practice v5 will not be marked as default until it has sufficient adoption on +the public network. + +The expectation is never to recommend upgrading existing +rooms to v4, but instead v5 (once ready), to avoid overburdening room admins +with upgrade notifications. + +To conclude, the proposed plan is: + 1. Define room v4 as MSC1884 + 2. Introduce servers with v4 support into the public network as rapidly as possible + 3. Wait for enough servers to speak v4. Meanwhile: + 1. Define an MSC for honouring server signing key validity periods + 2. Implement this MSC + 3. Define room v5 as this MSC + 4. Release Matrix 1.0, defining room v4 as the new default for creating rooms, + but also shipping support for room v5 for the first time. + 5. Wait for enough servers to speak v5 rooms. + 6. Define room v5 as the new default for creating rooms. + 7. Define room versions prior to v5 as unsafe, thus prompting users to upgrade their + rooms to v5. + +The reason we don't wait for v5 to be widespread before releasing 1.0 is to avoid +delaying the 1.0 yet further. It is good enough for 1.0 to support v5 without it +also being the default for creating rooms. From 77050062601cad901623660de151081608fb5ca3 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 22 May 2019 12:43:43 -0400 Subject: [PATCH 081/282] 1 hour seems to be fine --- proposals/1719-olm_unwedging.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index e696c1e2..50b6f0e6 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -16,8 +16,7 @@ empty contents. In order to avoid creating too many extra sessions, a client should rate-limit the number of new sessions it creates per device that it receives a message from; the client should not create a new session with another device if it has -already created one for that given device in the past 1 hour. (TODO: is 1 hour -the right amount of time?) +already created one for that given device in the past 1 hour. Clients may wish to ask the sender of the undecryptable messages to re-send the message. For exampe, if the undecryptable message was a megolm session, then From d39baba21b64c0dfe3b57e2bbbe8b9609fb69f2d Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 22 May 2019 12:44:04 -0400 Subject: [PATCH 082/282] add a security note --- proposals/1719-olm_unwedging.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 50b6f0e6..3e0ce35d 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -45,6 +45,10 @@ time of the session. The spec will be changed to read: ## Security considerations +An attacker could use this to create a new session on a device that they are +able to read. However, this would require the attacker to have compromised the +device's keys. + ## Conclusion This proposal outlines how wedged olm sessions can be replaced by a new From dd74baa5d0fb6c2f15878a929c1a70e2edf69e62 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 22 May 2019 12:54:01 -0400 Subject: [PATCH 083/282] clarify wording --- proposals/1719-olm_unwedging.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 3e0ce35d..67bd6010 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -36,8 +36,7 @@ time of the session. The spec will be changed to read: > If a client has multiple sessions established with another device, it should > use the session from which it last received and successfully decrypted a > message. For these purposes, a session that has not received any messages -> should consider its creation time to be the time that it last received a -> message. +> should use its creation time as the time that it last received a message. ## Tradeoffs From ac08c8461240e2fc9f88bc59c34263d8add4115f Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Thu, 23 May 2019 14:39:46 -0400 Subject: [PATCH 084/282] remove duplicate consideration Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/1719-olm_unwedging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 67bd6010..c181e3f7 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -29,7 +29,7 @@ another device, it should use the session from which it last received a message." (the last paragraph of the [`m.olm.v1.curve25519-aes-sha2` section](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-olm-v1-curve25519-aes-sha2)). When comparing the time of the last received message for each session, the -client should consider only consider messages that were successfully decrypted, +client should only consider messages that were successfully decrypted, and for sessions that have never received a message, it should use the creation time of the session. The spec will be changed to read: From 9c2a789d34da6a465ace23875d9d9e89ff7ced48 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 23 May 2019 22:13:57 -0600 Subject: [PATCH 085/282] Add missing changelogs and make existing ones match conventions The conventions are not set in stone, however the changelog should not be a mixed bag of voices. --- changelogs/application_service/newsfragments/1650.clarification | 1 + changelogs/client_server/newsfragments/1650.clarification | 1 + changelogs/client_server/newsfragments/1656.clarification | 1 + changelogs/client_server/newsfragments/1701.feature | 2 +- changelogs/client_server/newsfragments/1744.clarification | 2 +- changelogs/client_server/newsfragments/1889.clarification | 2 +- changelogs/client_server/newsfragments/1891.clarification | 1 + changelogs/client_server/newsfragments/1969.clarification | 1 + changelogs/client_server/newsfragments/1975.clarification | 2 +- changelogs/client_server/newsfragments/1999.clarification | 1 + changelogs/identity_service/newsfragments/1967.clarification | 1 + changelogs/server_server/newsfragments/1650.clarification | 1 + 12 files changed, 12 insertions(+), 4 deletions(-) create mode 100644 changelogs/application_service/newsfragments/1650.clarification create mode 100644 changelogs/client_server/newsfragments/1650.clarification create mode 100644 changelogs/client_server/newsfragments/1656.clarification create mode 100644 changelogs/client_server/newsfragments/1891.clarification create mode 100644 changelogs/client_server/newsfragments/1969.clarification create mode 100644 changelogs/client_server/newsfragments/1999.clarification create mode 100644 changelogs/identity_service/newsfragments/1967.clarification create mode 100644 changelogs/server_server/newsfragments/1650.clarification diff --git a/changelogs/application_service/newsfragments/1650.clarification b/changelogs/application_service/newsfragments/1650.clarification new file mode 100644 index 00000000..617b7ab6 --- /dev/null +++ b/changelogs/application_service/newsfragments/1650.clarification @@ -0,0 +1 @@ +Change examples to use example.org instead of a real domain. diff --git a/changelogs/client_server/newsfragments/1650.clarification b/changelogs/client_server/newsfragments/1650.clarification new file mode 100644 index 00000000..617b7ab6 --- /dev/null +++ b/changelogs/client_server/newsfragments/1650.clarification @@ -0,0 +1 @@ +Change examples to use example.org instead of a real domain. diff --git a/changelogs/client_server/newsfragments/1656.clarification b/changelogs/client_server/newsfragments/1656.clarification new file mode 100644 index 00000000..0c8f4ad0 --- /dev/null +++ b/changelogs/client_server/newsfragments/1656.clarification @@ -0,0 +1 @@ +Clarify that ``state_default`` in ``m.room.power_levels`` always defaults to 50. diff --git a/changelogs/client_server/newsfragments/1701.feature b/changelogs/client_server/newsfragments/1701.feature index 39c22dd7..cf6084ae 100644 --- a/changelogs/client_server/newsfragments/1701.feature +++ b/changelogs/client_server/newsfragments/1701.feature @@ -1 +1 @@ -Documented megolm session export format. \ No newline at end of file +Add megolm session export format. diff --git a/changelogs/client_server/newsfragments/1744.clarification b/changelogs/client_server/newsfragments/1744.clarification index dc103920..dfe838f1 100644 --- a/changelogs/client_server/newsfragments/1744.clarification +++ b/changelogs/client_server/newsfragments/1744.clarification @@ -1 +1 @@ -Add missing status_msg to m.presence schema. +Add missing ``status_msg`` to ``m.presence`` schema. diff --git a/changelogs/client_server/newsfragments/1889.clarification b/changelogs/client_server/newsfragments/1889.clarification index 5026dab3..2737a7ee 100644 --- a/changelogs/client_server/newsfragments/1889.clarification +++ b/changelogs/client_server/newsfragments/1889.clarification @@ -1 +1 @@ -Add the missing `m.push_rules` event schema. +Add the missing ``m.push_rules`` event schema. diff --git a/changelogs/client_server/newsfragments/1891.clarification b/changelogs/client_server/newsfragments/1891.clarification new file mode 100644 index 00000000..ef4edfb4 --- /dev/null +++ b/changelogs/client_server/newsfragments/1891.clarification @@ -0,0 +1 @@ +Clarify how modern day local echo is meant to be solved by clients. diff --git a/changelogs/client_server/newsfragments/1969.clarification b/changelogs/client_server/newsfragments/1969.clarification new file mode 100644 index 00000000..b0f05203 --- /dev/null +++ b/changelogs/client_server/newsfragments/1969.clarification @@ -0,0 +1 @@ +Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1975.clarification b/changelogs/client_server/newsfragments/1975.clarification index 03a9b754..ac118bfd 100644 --- a/changelogs/client_server/newsfragments/1975.clarification +++ b/changelogs/client_server/newsfragments/1975.clarification @@ -1 +1 @@ -Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. It is somewhat meaningless to request a thumbnail without specifying a desired size, and Synapse has never permitted such requests. \ No newline at end of file +Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. diff --git a/changelogs/client_server/newsfragments/1999.clarification b/changelogs/client_server/newsfragments/1999.clarification new file mode 100644 index 00000000..748c55f2 --- /dev/null +++ b/changelogs/client_server/newsfragments/1999.clarification @@ -0,0 +1 @@ +Clarify how ``m.login.dummy`` can be used to disambiguate login flows. diff --git a/changelogs/identity_service/newsfragments/1967.clarification b/changelogs/identity_service/newsfragments/1967.clarification new file mode 100644 index 00000000..b080caeb --- /dev/null +++ b/changelogs/identity_service/newsfragments/1967.clarification @@ -0,0 +1 @@ +Fix route for ``/3pid/bind``. diff --git a/changelogs/server_server/newsfragments/1650.clarification b/changelogs/server_server/newsfragments/1650.clarification new file mode 100644 index 00000000..617b7ab6 --- /dev/null +++ b/changelogs/server_server/newsfragments/1650.clarification @@ -0,0 +1 @@ +Change examples to use example.org instead of a real domain. From 3c38956510511d97dc770c0ec0be4d681b382af0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 24 May 2019 11:57:22 -0600 Subject: [PATCH 086/282] Remove prev_content from the redaction essential keys list As per [MSC1954](https://github.com/matrix-org/matrix-doc/pull/1954) No known changes since the proposal was accepted. --- changelogs/client_server/newsfragments/2016.clarification | 1 + specification/client_server_api.rst | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2016.clarification diff --git a/changelogs/client_server/newsfragments/2016.clarification b/changelogs/client_server/newsfragments/2016.clarification new file mode 100644 index 00000000..77ea0d4c --- /dev/null +++ b/changelogs/client_server/newsfragments/2016.clarification @@ -0,0 +1 @@ +Remove ``prev_content`` from the redaction algorithm's essential keys list. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 6f65ebb1..604c2b1c 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1500,7 +1500,6 @@ the following list: - ``room_id`` - ``sender`` - ``state_key`` -- ``prev_content`` - ``content`` - ``hashes`` - ``signatures`` From 3b0e194ff7a5a4b6e20bbf1572337738904feda5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 24 May 2019 15:50:05 -0600 Subject: [PATCH 087/282] Add version 4 rooms to the spec As per [MSC1884](https://github.com/matrix-org/matrix-doc/pull/1884) and [MSC2002](https://github.com/matrix-org/matrix-doc/pull/2002). No known changes since the proposals were accepted. Due to being in the area: This fixes https://github.com/matrix-org/matrix-doc/issues/1863 --- api/server-server/definitions/pdu_v3.yaml | 4 ++ api/server-server/definitions/pdu_v4.yaml | 47 ++++++++++++++ api/server-server/examples/pdu_v3.json | 3 +- api/server-server/examples/pdu_v4.json | 12 ++++ specification/index.rst | 1 + specification/rooms/v4.rst | 76 +++++++++++++++++++++++ specification/targets.yaml | 4 ++ 7 files changed, 146 insertions(+), 1 deletion(-) create mode 100644 api/server-server/definitions/pdu_v4.yaml create mode 100644 api/server-server/examples/pdu_v4.json create mode 100644 specification/rooms/v4.rst diff --git a/api/server-server/definitions/pdu_v3.yaml b/api/server-server/definitions/pdu_v3.yaml index 8d41fbda..38105098 100644 --- a/api/server-server/definitions/pdu_v3.yaml +++ b/api/server-server/definitions/pdu_v3.yaml @@ -20,6 +20,10 @@ allOf: - $ref: "unsigned_pdu_base.yaml" - type: object properties: + redacts: + type: string + description: For redaction events, the ID of the event being redacted. + example: "$def/456+oldevent" auth_events: type: array items: diff --git a/api/server-server/definitions/pdu_v4.yaml b/api/server-server/definitions/pdu_v4.yaml new file mode 100644 index 00000000..55b1c26f --- /dev/null +++ b/api/server-server/definitions/pdu_v4.yaml @@ -0,0 +1,47 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +title: Persistent Data Unit +description: A persistent data unit (event) for room version 3 and beyond. +example: + $ref: "../examples/pdu_v4.json" +allOf: + - $ref: "pdu_v3.yaml" + - type: object + properties: + redacts: + type: string + description: For redaction events, the ID of the event being redacted. + example: "$def_456-oldevent" + auth_events: + type: array + items: + type: string + description: Event ID. + description: |- + Event IDs for the authorization events that would + allow this event to be in the room. + example: ["$URLsafe-base64EncodedHash", "$Another_Event"] + prev_events: + type: array + items: + type: string + description: Event ID. + description: |- + Event IDs for the most recent events in the room + that the homeserver was aware of when it made this event. + example: ["$URLsafe-base64EncodedHash", "$Another_Event"] + required: + - auth_events + - prev_events diff --git a/api/server-server/examples/pdu_v3.json b/api/server-server/examples/pdu_v3.json index 6a454b4e..acffdf26 100644 --- a/api/server-server/examples/pdu_v3.json +++ b/api/server-server/examples/pdu_v3.json @@ -15,5 +15,6 @@ "prev_events": [ "$base64encodedeventid", "$adifferenteventid" - ] + ], + "redacts": "$some/old+event" } diff --git a/api/server-server/examples/pdu_v4.json b/api/server-server/examples/pdu_v4.json new file mode 100644 index 00000000..3c2f0e22 --- /dev/null +++ b/api/server-server/examples/pdu_v4.json @@ -0,0 +1,12 @@ +{ + "$ref": "pdu_v3.json", + "auth_events": [ + "$urlsafe_base64_encoded_eventid", + "$a-different-event-id" + ], + "prev_events": [ + "$urlsafe_base64_encoded_eventid", + "$a-different-event-id" + ], + "redacts": "$some-old_event" +} diff --git a/specification/index.rst b/specification/index.rst index 19c5ad22..33dff5a3 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -493,6 +493,7 @@ The available room versions are: * `Version 1 `_ - **Stable**. The current version of most rooms. * `Version 2 `_ - **Stable**. Implements State Resolution Version 2. * `Version 3 `_ - **Stable**. Introduces events whose IDs are the event's hash. +* `Version 4 `_ - **Stable**. Builds on v3 by using URL-safe base64 for event IDs. Specification Versions ---------------------- diff --git a/specification/rooms/v4.rst b/specification/rooms/v4.rst new file mode 100644 index 00000000..8d2cc59b --- /dev/null +++ b/specification/rooms/v4.rst @@ -0,0 +1,76 @@ +.. Copyright 2019 The Matrix.org Foundation C.I.C. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Room Version 4 +============== + +This room version builds on `version 3 `_ using a different encoding for +event IDs. + +.. contents:: Table of Contents +.. sectnum:: + + +Client considerations +--------------------- + +This room version changes the format form event IDs sent to clients. Clients should +already be treating event IDs as opaque identifiers, and should not be concerned with +the format of them. Clients should still encode the event ID when including it in a +request path. + +Clients should expect to see event IDs changed from the format of ``$randomstring:example.org`` +to something like ``$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg`` (note the lack of domain). + + +Server implementation components +-------------------------------- + +.. WARNING:: + The information contained in this section is strictly for server implementors. + Applications which use the Client-Server API are generally unaffected by the + intricacies contained here. The section above regarding client considerations + is the resource that Client-Server API use cases should reference. + + +Room version 4 uses the same algorithms defined in `room version 3 `_, however +using URL-safe base64 to generate the event ID. + +Event IDs +~~~~~~~~~ + +.. admonition:: Rationale + + Room version 3 generated event IDs that were difficult for client implementations + which were not encoding the event ID to function in those rooms. It additionally + raised concern due to the ``/`` character being interpretted differently by some + reverse proxy software, and generally made administration harder. + +The event ID is the `reference hash`_ of the event encoded using a variation of +`Unpadded Base64`_ which replaces the 62nd and 63rd characters with ``-`` and ``_`` +instead of using ``+`` and ``/``. This matches `RFC4648's definition of URL-safe base64 +`_. Event IDs are still prefixed +with ``$`` and may result in looking like ``$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg``. + +Just like in room version 3, event IDs should not be sent over federation to servers +when the room uses this room version. On the receiving end of an event, the server +should compute the relevant event ID for itself. Room version 3 also changes the format +of ``auth_events`` and ``prev_events`` in a PDU. + +{{definition_ss_pdu_v4}} + +.. _`Unpadded Base64`: ../appendices.html#unpadded-base64 +.. _`Canonical JSON`: ../appendices.html#canonical-json +.. _`Signing Events`: ../server_server/r0.1.1.html#signing-events +.. _`reference hash`: ../server_server/r0.1.1.html#reference-hashes diff --git a/specification/targets.yaml b/specification/targets.yaml index 830449ae..be4a063a 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -38,6 +38,10 @@ targets: files: - rooms/v3.rst version_label: v3 + rooms@v4: # this is translated to be rooms/v4.html + files: + - rooms/v4.rst + version_label: v4 appendices: files: - appendices.rst From ceaccffdf7244e79aa91e3916811f93881be9bca Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 24 May 2019 17:14:23 -0600 Subject: [PATCH 088/282] Add a .m.rule.tombstone default push rule for room upgrades As per [MSC1930](https://github.com/matrix-org/matrix-doc/pull/1930) There are no known changes to this proposal since it was accepted. --- .../client_server/newsfragments/2020.feature | 1 + specification/modules/push.rst | 35 +++++++++++++++++-- 2 files changed, 34 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2020.feature diff --git a/changelogs/client_server/newsfragments/2020.feature b/changelogs/client_server/newsfragments/2020.feature new file mode 100644 index 00000000..0d7c7eb8 --- /dev/null +++ b/changelogs/client_server/newsfragments/2020.feature @@ -0,0 +1 @@ +Add a ``.m.rule.tombstone`` default push rule for room ugprade notifications. diff --git a/specification/modules/push.rst b/specification/modules/push.rst index 9830729a..1bac0c2e 100644 --- a/specification/modules/push.rst +++ b/specification/modules/push.rst @@ -177,7 +177,7 @@ notification is delivered for a matching event. The following actions are define This prevents each matching event from generating a notification ``coalesce`` This enables notifications for matching events but activates homeserver - specific behaviour to intelligently coalesce multiple events into a single + specific behaviour to intelligently coalesce multiple events into a single notification. Not all homeservers may support this. Those that do not support it should treat it as the ``notify`` action. ``set_tweak`` @@ -369,6 +369,37 @@ Definition: } +``.m.rule.tombstone`` +````````````````````` +Matches any event whose type is ``m.room.tombstone``. This is intended +to notify users of a room when it is upgraded, similar to what an +``@room`` notification would accomplish. + +Definition: + +.. code:: json + + { + "rule_id": ".m.rule.tombstone", + "default": true, + "enabled": true, + "conditions": [ + { + "kind": "event_match", + "key": "type", + "pattern": "m.room.tombstone" + } + ], + "actions": [ + "notify", + { + "set_tweak": "highlight", + "value": true + } + ] + } + + ``.m.rule.roomnotif`` ````````````````````` Matches any message whose content is unencrypted and contains the @@ -599,7 +630,7 @@ Definition: Conditions ++++++++++ -Override, Underride and Default Rules MAY have a list of 'conditions'. +Override, Underride and Default Rules MAY have a list of 'conditions'. All conditions must hold true for an event in order to apply the ``action`` for the event. A rule with no conditions always matches. Room, Sender, User and Content rules do not have conditions in the same way, but instead have From 2b96d73305328188f584070f550ce1591d08bbff Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 15:39:35 -0600 Subject: [PATCH 089/282] Fix third party signed definitions for join APIs Fixes https://github.com/matrix-org/matrix-doc/issues/1978 --- .../definitions/third_party_signed.yaml | 41 +++++++++++ api/client-server/joining.yaml | 69 +------------------ specification/modules/third_party_invites.rst | 3 +- 3 files changed, 45 insertions(+), 68 deletions(-) create mode 100644 api/client-server/definitions/third_party_signed.yaml diff --git a/api/client-server/definitions/third_party_signed.yaml b/api/client-server/definitions/third_party_signed.yaml new file mode 100644 index 00000000..c9c761a1 --- /dev/null +++ b/api/client-server/definitions/third_party_signed.yaml @@ -0,0 +1,41 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +title: Third Party Signed +description: |- + A signature of an ``m.third_party_invite`` token to prove that this user + owns a third party identity which has been invited to the room. +properties: + sender: + type: string + description: The Matrix ID of the user who issued the invite. + example: "@alice:example.org" + mxid: + type: string + description: The Matrix ID of the invitee. + example: "@bob:example.org" + token: + type: string + description: The state key of the m.third_party_invite event. + example: "random8nonce" + signatures: + type: object + description: A signatures object containing a signature of the entire signed object. + title: Signatures + example: { + "example.org": { + "ed25519:0": "some9signature" + } + } +required: ["sender", "mxid", "token", "signatures"] diff --git a/api/client-server/joining.yaml b/api/client-server/joining.yaml index 1dcf769f..af38d6f9 100644 --- a/api/client-server/joining.yaml +++ b/api/client-server/joining.yaml @@ -58,38 +58,9 @@ paths: name: third_party_signed schema: type: object - example: { - "third_party_signed": { - "sender": "@cat:the.hat", - "mxid": "@green:eggs.ham", - "token": "random8nonce", - "signatures": { - "horton.hears": { - "ed25519:0": "some9signature" - } - } - } - } properties: third_party_signed: - type: object - title: ThirdPartySigned - description: A signature of an ``m.third_party_invite`` token to prove that this user owns a third party identity which has been invited to the room. - properties: - sender: - type: string - description: The Matrix ID of the user who issued the invite. - mxid: - type: string - description: The Matrix ID of the invitee. - token: - type: string - description: The state key of the m.third_party_invite event. - signatures: - type: object - description: A signatures object containing a signature of the entire signed object. - title: Signatures - required: ["sender", "mxid", "token", "signatures"] + $ref: "definitions/third_party_signed.yaml" responses: 200: description: |- @@ -163,45 +134,9 @@ paths: name: third_party_signed schema: type: object - example: { - "third_party_signed": { - "signed": { - "sender": "@cat:the.hat", - "mxid": "@green:eggs.ham", - "token": "random8nonce", - "signatures": { - "horton.hears": { - "ed25519:0": "some9signature" - } - } - } - } - } properties: third_party_signed: - type: object - title: ThirdPartySigned - description: A signature of an ``m.third_party_invite`` token to prove that this user owns a third party identity which has been invited to the room. - properties: - signed: - type: object - title: Signed - properties: - sender: - type: string - description: The Matrix ID of the user who issued the invite. - mxid: - type: string - description: The Matrix ID of the invitee. - token: - type: string - description: The state key of the m.third_party_invite event. - signatures: - type: object - description: A signatures object containing a signature of the entire signed object. - title: Signatures - required: ["sender", "mxid", "token", "signatures"] - required: ["signed"] + $ref: "definitions/third_party_signed.yaml" responses: 200: description: |- diff --git a/specification/modules/third_party_invites.rst b/specification/modules/third_party_invites.rst index 3e11d929..b8ab9657 100644 --- a/specification/modules/third_party_invites.rst +++ b/specification/modules/third_party_invites.rst @@ -38,7 +38,8 @@ When the invitee's homeserver receives the notification of the binding, it should insert an ``m.room.member`` event into the room's graph for that user, with ``content.membership`` = ``invite``, as well as a ``content.third_party_invite`` property which contains proof that the invitee -does indeed own that third party identifier. +does indeed own that third party identifier. See the `m.room.member <#m-room-member>`_ +schema for more information. Events From 17a9524cad8774c9cfe12157742d0f196391de44 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 15:40:29 -0600 Subject: [PATCH 090/282] changelog --- changelogs/client_server/newsfragments/2025.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2025.clarification diff --git a/changelogs/client_server/newsfragments/2025.clarification b/changelogs/client_server/newsfragments/2025.clarification new file mode 100644 index 00000000..9e99b23d --- /dev/null +++ b/changelogs/client_server/newsfragments/2025.clarification @@ -0,0 +1 @@ +Fix the ``third_party_signed`` definitions for the join APIs. From 5eea4a477f98d79371791ab153bdaae91883f4b8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 20:42:21 -0600 Subject: [PATCH 091/282] Add server notices support As per [MSC1452](https://github.com/matrix-org/matrix-doc/issues/1452) Fixes https://github.com/matrix-org/matrix-doc/issues/1254 Although MSC1452 focuses on just the warnings part of the server notices, the base for notices has not been established in the spec. This commit adds the needed support to be able to handle notices. No intentional divergences from the proposal are included in this changeset. There are a few additions which are used in practice although not defined in the proposal, such as who is responsible for aesthetics, sending notices, and other misc rules. --- .../client_server/newsfragments/2026.feature | 1 + .../examples/m.room.message#m.server_notice | 11 +++ .../schema/m.room.message#m.server_notice | 39 ++++++++++ .../templating/matrix_templates/sections.py | 15 +++- .../matrix_templates/templates/events.tmpl | 7 ++ scripts/templating/matrix_templates/units.py | 2 + specification/client_server_api.rst | 5 +- specification/modules/server_notices.rst | 78 +++++++++++++++++++ specification/targets.yaml | 1 + 9 files changed, 154 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2026.feature create mode 100644 event-schemas/examples/m.room.message#m.server_notice create mode 100644 event-schemas/schema/m.room.message#m.server_notice create mode 100644 specification/modules/server_notices.rst diff --git a/changelogs/client_server/newsfragments/2026.feature b/changelogs/client_server/newsfragments/2026.feature new file mode 100644 index 00000000..f82b9aea --- /dev/null +++ b/changelogs/client_server/newsfragments/2026.feature @@ -0,0 +1 @@ +Add support for sending server notices to clients. diff --git a/event-schemas/examples/m.room.message#m.server_notice b/event-schemas/examples/m.room.message#m.server_notice new file mode 100644 index 00000000..0eb44ea7 --- /dev/null +++ b/event-schemas/examples/m.room.message#m.server_notice @@ -0,0 +1,11 @@ +{ + "$ref": "core/room_event.json", + "type": "m.room.message", + "content": { + "body": "Human-readable message to explain the notice", + "msgtype": "m.server_notice", + "server_notice_type": "m.server_notice.usage_limit_reached", + "admin_contact": "mailto:server.admin@example.org", + "limit_type": "monthly_active_user" + } +} diff --git a/event-schemas/schema/m.room.message#m.server_notice b/event-schemas/schema/m.room.message#m.server_notice new file mode 100644 index 00000000..f1848821 --- /dev/null +++ b/event-schemas/schema/m.room.message#m.server_notice @@ -0,0 +1,39 @@ +--- +allOf: + - $ref: core-event-schema/room_event.yaml +description: Represents a server notice for a user. +properties: + content: + properties: + body: + description: A human-readable description of the notice. + type: string + msgtype: + enum: + - m.server_notice + type: string + server_notice_type: + description: |- + The type of notice being represented. + type: string + admin_contact: + description: |- + A URI giving a contact method for the server administrator. Required if the + notice type is ``m.server_notice.usage_limit_reached``. + type: string + limit_type: + description: |- + The kind of usage limit the server has exceeded. Required if the notice type is + ``m.server_notice.usage_limit_reached``. + type: string + required: + - msgtype + - body + - server_notice_type + type: object + type: + enum: + - m.room.message + type: string +title: ServerNoticeMessage +type: object diff --git a/scripts/templating/matrix_templates/sections.py b/scripts/templating/matrix_templates/sections.py index 5961aa24..92afa5ff 100644 --- a/scripts/templating/matrix_templates/sections.py +++ b/scripts/templating/matrix_templates/sections.py @@ -110,12 +110,13 @@ class MatrixSections(Sections): # Special function: Returning a dict will specify multiple sections where # the key is the section name and the value is the value of the section def render_group_events(self): - # map all event schemata to the form $EVENTTYPE_event with s/./_/g - # e.g. m_room_topic_event + # map all event schemata to the form $EVENTTYPE_event with s/.#/_/g + # e.g. m_room_topic_event or m_room_message_m_text_event schemas = self.units.get("event_schemas") renders = {} for event_type in schemas: - renders[event_type.replace(".", "_") + "_event"] = self._render_events( + underscored_event_type = event_type.replace(".", "_").replace("#", "_") + renders[underscored_event_type + "_event"] = self._render_events( lambda x: x == event_type, sorted ) return renders @@ -141,9 +142,15 @@ class MatrixSections(Sections): "m.room.message#m.notice", "m.room.message#m.image", "m.room.message#m.file" ] + excluded_types = [ + # We exclude server notices from here because we handle them in a + # dedicated module. We do not want to confuse developers this early + # in the spec. + "m.room.message#m.server_notice", + ] other_msgtypes = [ k for k in schemas.keys() if k.startswith("m.room.message#") and - k not in msgtype_order + k not in msgtype_order and k not in excluded_types ] for event_name in (msgtype_order + other_msgtypes): if not event_name.startswith("m.room.message#m."): diff --git a/scripts/templating/matrix_templates/templates/events.tmpl b/scripts/templating/matrix_templates/templates/events.tmpl index 0955cf4c..f55be73f 100644 --- a/scripts/templating/matrix_templates/templates/events.tmpl +++ b/scripts/templating/matrix_templates/templates/events.tmpl @@ -1,7 +1,14 @@ {% import 'tables.tmpl' as tables -%} +{% if (event.type_with_msgtype) %} +``{{event.type_with_msgtype}}`` +{{(4 + event.type_with_msgtype | length) * title_kind}} +{% endif -%} + +{% if (not event.type_with_msgtype) %} ``{{event.type}}`` {{(4 + event.type | length) * title_kind}} +{% endif -%} {% if (event.typeof | length) %} *{{event.typeof}}* diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index c1755119..466110f8 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -846,6 +846,7 @@ class MatrixUnits(Units): "title": None, "desc": None, "msgtype": None, + "type_with_msgtype": None, # for the template's sake "content_fields": [ # ] @@ -884,6 +885,7 @@ class MatrixUnits(Units): ) if msgtype: schema["msgtype"] = msgtype[0] # enum prop + schema["type_with_msgtype"] = schema["type"] + " (" + msgtype[0] + ")" # link to msgtypes for m.room.message if schema["type"] == "m.room.message" and not msgtype: diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 604c2b1c..a8bbfca0 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -219,9 +219,12 @@ Other error codes the client might encounter are: to modify state (eg: sending messages, account data, etc) and not routes which only read state (eg: ``/sync``, get account data, etc). +:``M_CANNOT_LEAVE_SERVER_NOTICE_ROOM``: + The user is unable to reject an invite to join the server notices room. See the + `Server Notices <#server-notices>`_ module for more information. + .. TODO: More error codes (covered by other issues) .. * M_CONSENT_NOT_GIVEN - GDPR: https://github.com/matrix-org/matrix-doc/issues/1512 -.. * M_CANNOT_LEAVE_SERVER_NOTICE_ROOM - GDPR: https://github.com/matrix-org/matrix-doc/issues/1254 .. _sect:txn_ids: diff --git a/specification/modules/server_notices.rst b/specification/modules/server_notices.rst new file mode 100644 index 00000000..63b7bfc5 --- /dev/null +++ b/specification/modules/server_notices.rst @@ -0,0 +1,78 @@ +.. Copyright 2019 The Matrix.org Foundation C.I.C. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Server Notices +============== + +.. _module:server-notices: + +Homeserver hosts often want to send messages to users in an official capacity, +or have resource limits which affect a user's ability to use the homeserver. +For example, the homeserver may be limited to a certain number of active users +per month and has exceeded that limit. To communicate this failure to users, +the homeserver would use the Server Notices room. + +The aesthetics of the room (name, topic, avatar, etc) are left as an implementation +detail. It is recommended that the homeserver decorate the room such that it looks +like an official room to users. + +Events +------ +Notices are sent to the client as normal ``m.room.message`` events with a +``msgtype`` of ``m.server_notice`` in the server notices room. Events with +a ``m.server_notice`` ``msgtype`` outside of the server notice room must +be ignored by clients. + +The specified values for ``server_notice_type`` are: + +:``m.server_notice.usage_limit_reached``: + The server has exceeded some limit which requires the server administrator + to intervene. The ``limit_type`` describes the kind of limit reached. + The specified values for ``limit_type`` are: + + :``monthly_active_user``: + The server's number of active users in the last 30 days has exceeded the + maximum. New connections are being refused by the server. What defines + "active" is left as an implementation detail, however servers are encouraged + to treat syncing users as "active". + + +{{m_room_message_m_server_notice_event}} + +Client behaviour +---------------- +Clients can identify the server notices room by the ``m.server_notice`` tag +on the room. Active notices are represented by the `pinned events <#m-room-pinned-events>`_ +in the server notices room. Server notice events pinned in that room should +be shown to the user through special UI and not through the normal pinned +events interface in the client. For example, clients may show warning banners +or bring up dialogs to get the user's attention. Events which are not server +notice events and are pinned in the server notices room should be shown just +like any other pinned event in a room. + +The client must not expect to be able to reject an invite to join the server +notices room. Attempting to reject the invite must result in a +``M_CANNOT_LEAVE_SERVER_NOTICE_ROOM`` error. Servers should not prevent the user +leaving the room after joining the server notices room, however the same error +code must be used if the server will prevent leaving the room. + +Server behaviour +---------------- +Servers should manage exactly 1 server notices room per user. Servers must +identify this room to clients with the ``m.server_notice`` tag. Servers should +invite the target user rather than automatically join them to the server notice +room. + +How servers send notices to clients, and which user they use to send the events, +is left as an implementation detail for the server. diff --git a/specification/targets.yaml b/specification/targets.yaml index 830449ae..aa53e9c3 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -83,6 +83,7 @@ groups: # reusable blobs of files when prefixed with 'group:' - modules/server_acls.rst - modules/mentions.rst - modules/room_upgrades.rst + - modules/server_notices.rst title_styles: ["=", "-", "~", "+", "^", "`", "@", ":"] From 1d33adf62d8d56d1bfb88a0ff13da7e00fb0cc60 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:20:04 -0600 Subject: [PATCH 092/282] Add rationale for UIA on change password, and how access tokens behave Fixes https://github.com/matrix-org/matrix-doc/issues/680 --- api/client-server/registration.yaml | 8 ++++++-- changelogs/client_server/newsfragments/2027.clarification | 1 + 2 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2027.clarification diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index e2d35d2c..3195ab41 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -326,13 +326,17 @@ paths: description: |- Changes the password for an account on this homeserver. - This API endpoint uses the `User-Interactive Authentication API`_. + This API endpoint uses the `User-Interactive Authentication API`_ to + ensure the user changing the password is actually the owner of the + account. An access token should be submitted to this endpoint if the client has an active session. The homeserver may change the flows available depending on whether a - valid access token is provided. + valid access token is provided. The homeserver SHOULD NOT revoke the + access token provided in the request, however all other access tokens + for the user should be revoked if the request succeeds. security: - accessToken: [] operationId: changePassword diff --git a/changelogs/client_server/newsfragments/2027.clarification b/changelogs/client_server/newsfragments/2027.clarification new file mode 100644 index 00000000..db74ea56 --- /dev/null +++ b/changelogs/client_server/newsfragments/2027.clarification @@ -0,0 +1 @@ +Clarify why User Interactive Auth is used on password changes and how access tokens are handled. From 221d9f24fd5699c78072feeab335c9a389a4de41 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:30:55 -0600 Subject: [PATCH 093/282] Clarify that logging out deletes devices too Fixes https://github.com/matrix-org/matrix-doc/issues/1651 --- api/client-server/logout.yaml | 5 +++-- changelogs/client_server/newsfragments/2028.clarification | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2028.clarification diff --git a/api/client-server/logout.yaml b/api/client-server/logout.yaml index 2dfd6d97..8451f739 100644 --- a/api/client-server/logout.yaml +++ b/api/client-server/logout.yaml @@ -32,7 +32,7 @@ paths: summary: Invalidates a user access token description: |- Invalidates an existing access token, so that it can no longer be used for - authorization. + authorization. The device associated with the access token is also deleted. operationId: logout security: - accessToken: [] @@ -49,7 +49,8 @@ paths: summary: Invalidates all access tokens for a user description: |- Invalidates all access tokens for a user, so that they can no longer be used for - authorization. This includes the access token that made this request. + authorization. This includes the access token that made this request. All devices + for the user are also deleted. This endpoint does not require UI authorization because UI authorization is designed to protect against attacks where the someone gets hold of a single access diff --git a/changelogs/client_server/newsfragments/2028.clarification b/changelogs/client_server/newsfragments/2028.clarification new file mode 100644 index 00000000..75e21e74 --- /dev/null +++ b/changelogs/client_server/newsfragments/2028.clarification @@ -0,0 +1 @@ +Clarify that devices are deleted upon logout. From d52fcdacfa7a98e6ab56916dd7d331e53d94ab77 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:36:31 -0600 Subject: [PATCH 094/282] Add M_NOT_FOUND definition for deleting non-existent aliases Fixes https://github.com/matrix-org/matrix-doc/issues/1675 --- api/client-server/directory.yaml | 9 +++++++++ .../client_server/newsfragments/2029.clarification | 1 + 2 files changed, 10 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2029.clarification diff --git a/api/client-server/directory.yaml b/api/client-server/directory.yaml index 78ddfa29..2c7c8386 100644 --- a/api/client-server/directory.yaml +++ b/api/client-server/directory.yaml @@ -148,5 +148,14 @@ paths: } schema: type: object + 404: + description: There is no mapped room ID for this room alias. + examples: + application/json: { + "errcode": "M_NOT_FOUND", + "error": "Room alias #monkeys:example.org not found." + } + schema: + "$ref": "definitions/errors/error.yaml" tags: - Room directory diff --git a/changelogs/client_server/newsfragments/2029.clarification b/changelogs/client_server/newsfragments/2029.clarification new file mode 100644 index 00000000..95b65481 --- /dev/null +++ b/changelogs/client_server/newsfragments/2029.clarification @@ -0,0 +1 @@ +Add ``M_NOT_FOUND`` error definition for deleting room aliases. From a30dbc590da00727e71920af0df7f5e140783aba Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:39:09 -0600 Subject: [PATCH 095/282] Clarify that e2e keys are also obliterated --- api/client-server/logout.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/api/client-server/logout.yaml b/api/client-server/logout.yaml index 8451f739..75a3ec87 100644 --- a/api/client-server/logout.yaml +++ b/api/client-server/logout.yaml @@ -33,6 +33,7 @@ paths: description: |- Invalidates an existing access token, so that it can no longer be used for authorization. The device associated with the access token is also deleted. + `Device keys <#device-keys>`_ for the device are deleted alongside the device. operationId: logout security: - accessToken: [] @@ -50,7 +51,8 @@ paths: description: |- Invalidates all access tokens for a user, so that they can no longer be used for authorization. This includes the access token that made this request. All devices - for the user are also deleted. + for the user are also deleted. `Device keys <#device-keys>`_ for the device are + deleted alongside the device. This endpoint does not require UI authorization because UI authorization is designed to protect against attacks where the someone gets hold of a single access From 1bda3fe2b20427be2e129ac2a91d7dc9bbd72215 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:49:46 -0600 Subject: [PATCH 096/282] Spec MSISDN UIA support Fixes https://github.com/matrix-org/matrix-doc/issues/1702 1702 describes the lack of `bind_msisdn` parameter, however the whole login type was missing from UIA. --- api/client-server/registration.yaml | 6 ++++ .../client_server/newsfragments/2030.feature | 1 + specification/client_server_api.rst | 29 +++++++++++++++++++ 3 files changed, 36 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2030.feature diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index e2d35d2c..741008bb 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -94,6 +94,12 @@ paths: If true, the server binds the email used for authentication to the Matrix ID with the identity server. example: false + bind_msisdn: + type: boolean + description: |- + If true, the server binds the phone number used for authentication + to the Matrix ID with the identity server. + example: false username: type: string description: |- diff --git a/changelogs/client_server/newsfragments/2030.feature b/changelogs/client_server/newsfragments/2030.feature new file mode 100644 index 00000000..b5975a73 --- /dev/null +++ b/changelogs/client_server/newsfragments/2030.feature @@ -0,0 +1 @@ +Add MSISDN (phone number) support to User-Interactive Authentication. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 604c2b1c..a40d463c 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -618,6 +618,7 @@ This specification defines the following auth types: - ``m.login.recaptcha`` - ``m.login.oauth2`` - ``m.login.email.identity`` + - ``m.login.msisdn`` - ``m.login.token`` - ``m.login.dummy`` @@ -782,6 +783,34 @@ To use this authentication type, clients should submit an auth dict as follows: "session": "" } +Phone number/MSISDN-based (identity server) +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< +:Type: + ``m.login.msisdn`` +:Description: + Authentication is supported by authorising a phone number with an identity + server. + +Prior to submitting this, the client should authenticate with an identity +server. After authenticating, the session information should be submitted to +the homeserver. + +To use this authentication type, clients should submit an auth dict as follows: + +.. code:: json + + { + "type": "m.login.msisdn", + "threepidCreds": [ + { + "sid": "", + "client_secret": "", + "id_server": "" + } + ], + "session": "" + } + Dummy Auth <<<<<<<<<< :Type: From 00f97636a22b28de26ffaafd674e1014374210e0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 22:08:04 -0600 Subject: [PATCH 097/282] Add missing reason property to m.call.hangup Fixes https://github.com/matrix-org/matrix-doc/issues/1689 --- .../client_server/newsfragments/2031.clarification | 1 + event-schemas/schema/m.call.hangup | 10 +++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2031.clarification diff --git a/changelogs/client_server/newsfragments/2031.clarification b/changelogs/client_server/newsfragments/2031.clarification new file mode 100644 index 00000000..9bed3bcc --- /dev/null +++ b/changelogs/client_server/newsfragments/2031.clarification @@ -0,0 +1 @@ +Add missing ``reason`` to ``m.call.hangup``. diff --git a/event-schemas/schema/m.call.hangup b/event-schemas/schema/m.call.hangup index 9d45d179..0328e03a 100644 --- a/event-schemas/schema/m.call.hangup +++ b/event-schemas/schema/m.call.hangup @@ -1,6 +1,6 @@ { "type": "object", - "description": "Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call.", + "description": "Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call. The ``reason`` for the hangup is expected to be provided when there was an error in the call negotiation, such as ``ice_failed`` for when ICE negotiation fails or ``invite_timeout`` for when the other party did not answer in time.", "allOf": [{ "$ref": "core-event-schema/room_event.yaml" }], @@ -15,6 +15,14 @@ "version": { "type": "integer", "description": "The version of the VoIP specification this message adheres to. This specification is version 0." + }, + "reason": { + "type": "string", + "description": "Optional error reason for the hangup. This should not be provided when the user naturally ends or rejects the call.", + "enum": [ + "ice_failed", + "invite_timeout" + ] } }, "required": ["call_id", "version"] From 23ab1c527a4864f632ce07d80d80fa23eae81ed6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 22:16:12 -0600 Subject: [PATCH 098/282] Clarify how redactions affect room state Fixes https://github.com/matrix-org/matrix-doc/issues/1726 --- .../client_server/newsfragments/2032.clarification | 1 + specification/client_server_api.rst | 9 +++++++++ 2 files changed, 10 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2032.clarification diff --git a/changelogs/client_server/newsfragments/2032.clarification b/changelogs/client_server/newsfragments/2032.clarification new file mode 100644 index 00000000..e497b8be --- /dev/null +++ b/changelogs/client_server/newsfragments/2032.clarification @@ -0,0 +1 @@ +Clarify how redactions affect room state. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 604c2b1c..a254a85d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1531,6 +1531,15 @@ property of the redacted event, under the ``redacted_because`` key. When a client receives a redaction event it should change the redacted event in the same way a server does. +.. NOTE:: + + Redacted events can still affect the state of the room. For example, a + redacted ``join`` event will still result in the user being considered + joined. Similarly, a redacted topic does not necessarily cause the topic + to revert to what is was prior to the event - it causes the topic to be + removed from the room. + + Events ++++++ From e7ed8a23ce1e2e4c11484e1ffcda728730c56b22 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 14:42:00 -0600 Subject: [PATCH 099/282] Move lazy loading filter options to event filter The options also work on /messages --- .../definitions/room_event_filter.yaml | 25 ++++++++++++++++ .../definitions/sync_filter.yaml | 29 +------------------ 2 files changed, 26 insertions(+), 28 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index aa217d2b..fd399913 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -16,6 +16,31 @@ allOf: - type: object title: RoomEventFilter properties: + lazy_load_members: + type: boolean + description: |- + If ``true``, the only ``m.room.member`` events returned in + the ``state`` section of the ``/sync`` response are those + which are definitely necessary for a client to display + the ``sender`` of the timeline events in that response. + If ``false``, ``m.room.member`` events are not filtered. + By default, servers should suppress duplicate redundant + lazy-loaded ``m.room.member`` events from being sent to a given + client across multiple calls to ``/sync``, given that most clients + cache membership events (see ``include_redundant_members`` + to change this behaviour). + include_redundant_members: + type: boolean + description: |- + If ``true``, the ``state`` section of the ``/sync`` response will + always contain the ``m.room.member`` events required to display + the ``sender`` of the timeline events in that response, assuming + ``lazy_load_members`` is enabled. This means that redundant + duplicate member events may be returned across multiple calls to + ``/sync``. This is useful for naive clients who never track + membership data. If ``false``, duplicate ``m.room.member`` events + may be suppressed by the server across multiple calls to ``/sync``. + If ``lazy_load_members`` is ``false`` this field is ignored. not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` diff --git a/api/client-server/definitions/sync_filter.yaml b/api/client-server/definitions/sync_filter.yaml index 65b18ba6..45a269c7 100644 --- a/api/client-server/definitions/sync_filter.yaml +++ b/api/client-server/definitions/sync_filter.yaml @@ -47,7 +47,7 @@ properties: not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` - filter. This filter is applied before the filters in ``ephemeral``, + filter. This filter is applied before the filters in ``ephemeral``, ``state``, ``timeline`` or ``account_data`` items: type: string @@ -73,33 +73,6 @@ properties: allOf: - $ref: room_event_filter.yaml description: The state events to include for rooms. - properties: - lazy_load_members: - type: boolean - description: |- - If ``true``, the only ``m.room.member`` events returned in - the ``state`` section of the ``/sync`` response are those - which are definitely necessary for a client to display - the ``sender`` of the timeline events in that response. - If ``false``, ``m.room.member`` events are not filtered. - By default, servers should suppress duplicate redundant - lazy-loaded ``m.room.member`` events from being sent to a given - client across multiple calls to ``/sync``, given that most clients - cache membership events (see ``include_redundant_members`` - to change this behaviour). - include_redundant_members: - type: boolean - description: |- - If ``true``, the ``state`` section of the ``/sync`` response will - always contain the ``m.room.member`` events required to display - the ``sender`` of the timeline events in that response, assuming - ``lazy_load_members`` is enabled. This means that redundant - duplicate member events may be returned across multiple calls to - ``/sync``. This is useful for naive clients who never track - membership data. If ``false``, duplicate ``m.room.member`` events - may be suppressed by the server across multiple calls to ``/sync``. - If ``lazy_load_members`` is ``false`` this field is ignored. - timeline: allOf: - $ref: room_event_filter.yaml From d56df3238c58abde17ca28165baa9b904cb743c0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 14:35:37 -0600 Subject: [PATCH 100/282] Generalize wording to fit /messages and /sync --- .../definitions/room_event_filter.yaml | 30 +++++++++++-------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index fd399913..5258ea30 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -19,28 +19,32 @@ allOf: lazy_load_members: type: boolean description: |- - If ``true``, the only ``m.room.member`` events returned in - the ``state`` section of the ``/sync`` response are those - which are definitely necessary for a client to display - the ``sender`` of the timeline events in that response. + If ``true``, the only ``m.room.member`` events returned + are those which are definitely necessary for a client to + display the ``sender`` of the timeline events in the response. If ``false``, ``m.room.member`` events are not filtered. By default, servers should suppress duplicate redundant lazy-loaded ``m.room.member`` events from being sent to a given - client across multiple calls to ``/sync``, given that most clients + client across multiple calls, given that most clients cache membership events (see ``include_redundant_members`` to change this behaviour). + + Only applicable when filtering state events, such as the + ``state`` section of a ``/sync`` or ``/messages``. include_redundant_members: type: boolean description: |- - If ``true``, the ``state`` section of the ``/sync`` response will - always contain the ``m.room.member`` events required to display - the ``sender`` of the timeline events in that response, assuming - ``lazy_load_members`` is enabled. This means that redundant - duplicate member events may be returned across multiple calls to - ``/sync``. This is useful for naive clients who never track + If ``true``, response will always contain the ``m.room.member`` + events required to display the ``sender`` of the timeline events + in that response, assuming ``lazy_load_members`` is enabled. This + means that redundant duplicate member events may be returned across + multiple calls to. This is useful for naive clients who never track membership data. If ``false``, duplicate ``m.room.member`` events - may be suppressed by the server across multiple calls to ``/sync``. - If ``lazy_load_members`` is ``false`` this field is ignored. + may be suppressed by the server across multiple calls. If + ``lazy_load_members`` is ``false`` this field is ignored. + + Only applicable when filtering state events, such as the + ``state`` section of a ``/sync`` or ``/messages``. not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` From 34d6c1f4ad60d7d753ac703613034a8756662ae8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 14:46:59 -0600 Subject: [PATCH 101/282] Clarify wording further for how to handle redundant members Note: This makes assumptions on what the TODO comment in Synapse means: https://github.com/matrix-org/synapse/blob/e26e6b3230f0b55376f0f3bf823dd789ac7064d0/synapse/handlers/pagination.py#L262 Due to lack of implementation, it is assumed that using the same filter across multiple calls to /sync OR /messages will result in the redundant members being excluded in the next request. For example, calling /sync, then /messages which returns some members, then /sync again will exclude the members due to them being in /messages. --- .../definitions/room_event_filter.yaml | 25 ++++++++++--------- api/client-server/message_pagination.yaml | 17 +++++++++++++ 2 files changed, 30 insertions(+), 12 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 5258ea30..35b3edf9 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -24,27 +24,28 @@ allOf: display the ``sender`` of the timeline events in the response. If ``false``, ``m.room.member`` events are not filtered. By default, servers should suppress duplicate redundant - lazy-loaded ``m.room.member`` events from being sent to a given - client across multiple calls, given that most clients - cache membership events (see ``include_redundant_members`` - to change this behaviour). + lazy-loaded ``m.room.member`` events from being sent to a + given client in a previous request using the same filter, + given that most clients cache membership events (see + ``include_redundant_members`` to change this behaviour). Only applicable when filtering state events, such as the - ``state`` section of a ``/sync`` or ``/messages``. + ``state`` section of a ``/sync`` or ``/messages`` response. include_redundant_members: type: boolean description: |- If ``true``, response will always contain the ``m.room.member`` events required to display the ``sender`` of the timeline events - in that response, assuming ``lazy_load_members`` is enabled. This - means that redundant duplicate member events may be returned across - multiple calls to. This is useful for naive clients who never track - membership data. If ``false``, duplicate ``m.room.member`` events - may be suppressed by the server across multiple calls. If - ``lazy_load_members`` is ``false`` this field is ignored. + in that response, assuming ``lazy_load_members`` is enabled. + This means that redundant duplicate member events will be returned + across multiple calls using the same filter. This is useful for + naive clients who never track membership data. If ``false`` or + not provided, duplicate ``m.room.member`` events should be + suppressed by the server across multiple calls. If ``lazy_load_members`` + is ``false`` this field is ignored. Only applicable when filtering state events, such as the - ``state`` section of a ``/sync`` or ``/messages``. + ``state`` section of a ``/sync`` or ``/messages`` response. not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 941e61fb..ff6d970d 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -108,6 +108,23 @@ paths: type: object title: RoomEvent "$ref": "definitions/event-schemas/schema/core-event-schema/room_event.yaml" + state: + type: array + description: |- + A list of state events relevant to showing the ``chunk``. For example, if + lazy-loading members is enabled in the filter then this will contain any + applicable membership events. Servers should be careful to not exclude + membership events which are older than ones already sent to the client. + Likewise, clients should be cautious and avoid using older membership + events as the current membership event when paginating backwards. + + Unless ``include_redundant_members`` is ``true``, the server should remove + redundant members which would have already been sent to clients in prior calls + to ``/messages`` or ``/sync`` with the same filter. + items: + type: object + title: RoomStateEvent + $ref: "definitions/event-schemas/schema/core-event-schema/state_event.yaml" examples: application/json: { "start": "t47429-4392820_219380_26003_2265", From b67161cf97badda0c37ccaa482634e5c5fa4093b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 14:59:59 -0600 Subject: [PATCH 102/282] List the endpoints which are lazy-loading aware --- api/client-server/definitions/room_event_filter.yaml | 9 +++++++-- api/client-server/event_context.yaml | 3 +++ api/client-server/message_pagination.yaml | 5 ++++- api/client-server/sync.yaml | 3 +++ 4 files changed, 17 insertions(+), 3 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 35b3edf9..4790643e 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -30,7 +30,12 @@ allOf: ``include_redundant_members`` to change this behaviour). Only applicable when filtering state events, such as the - ``state`` section of a ``/sync`` or ``/messages`` response. + ``state`` section of a lazy-loading aware endpoint. + + The endpoints which support lazy-loading are: + `/sync <#get-matrix-client-%CLIENT_RELEASE_LABEL%-sync>`_, + `/messages <#get-matrix-client-%CLIENT_RELEASE_LABEL%-rooms-roomid-messages>`_, + and `/context <#get-matrix-client-%CLIENT_RELEASE_LABEL%-rooms-roomid-context-eventid>`_. include_redundant_members: type: boolean description: |- @@ -45,7 +50,7 @@ allOf: is ``false`` this field is ignored. Only applicable when filtering state events, such as the - ``state`` section of a ``/sync`` or ``/messages`` response. + ``state`` section of a lazy-loading aware endpoint. not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 91da3cf4..c0f391e5 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -34,6 +34,9 @@ paths: This API returns a number of events that happened just before and after the specified event. This allows clients to get the context surrounding an event. + + *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + for more information. operationId: getEventContext security: - accessToken: [] diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index ff6d970d..716afb65 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -33,6 +33,9 @@ paths: description: |- This API returns a list of message and state events for a room. It uses pagination query parameters to paginate history in the room. + + *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + for more information. operationId: getRoomEvents security: - accessToken: [] @@ -120,7 +123,7 @@ paths: Unless ``include_redundant_members`` is ``true``, the server should remove redundant members which would have already been sent to clients in prior calls - to ``/messages`` or ``/sync`` with the same filter. + to lazy-loading aware endpoints with the same filter. items: type: object title: RoomStateEvent diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index f204152a..87e7bf74 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -34,6 +34,9 @@ paths: Clients use this API when they first log in to get an initial snapshot of the state on the server, and then continue to call this API to get incremental deltas to the state, and to receive new messages. + + *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + for more information. operationId: sync security: - accessToken: [] From ba520df004eafdf3fc58d051a567708d7419fe14 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 17:38:04 -0600 Subject: [PATCH 103/282] Move lazy loading to a section in Filtering --- .../definitions/room_event_filter.yaml | 36 ++++--------------- api/client-server/sync.yaml | 3 +- specification/client_server_api.rst | 33 +++++++++++++++++ 3 files changed, 42 insertions(+), 30 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 4790643e..0659be8e 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -19,38 +19,16 @@ allOf: lazy_load_members: type: boolean description: |- - If ``true``, the only ``m.room.member`` events returned - are those which are definitely necessary for a client to - display the ``sender`` of the timeline events in the response. - If ``false``, ``m.room.member`` events are not filtered. - By default, servers should suppress duplicate redundant - lazy-loaded ``m.room.member`` events from being sent to a - given client in a previous request using the same filter, - given that most clients cache membership events (see - ``include_redundant_members`` to change this behaviour). - - Only applicable when filtering state events, such as the - ``state`` section of a lazy-loading aware endpoint. - - The endpoints which support lazy-loading are: - `/sync <#get-matrix-client-%CLIENT_RELEASE_LABEL%-sync>`_, - `/messages <#get-matrix-client-%CLIENT_RELEASE_LABEL%-rooms-roomid-messages>`_, - and `/context <#get-matrix-client-%CLIENT_RELEASE_LABEL%-rooms-roomid-context-eventid>`_. + If ``true``, enables lazy-loading of membership events. See + `Lazy-loading room members <#lazy-loading-room-members>`_ + for more information. Defaults to ``false``. include_redundant_members: type: boolean description: |- - If ``true``, response will always contain the ``m.room.member`` - events required to display the ``sender`` of the timeline events - in that response, assuming ``lazy_load_members`` is enabled. - This means that redundant duplicate member events will be returned - across multiple calls using the same filter. This is useful for - naive clients who never track membership data. If ``false`` or - not provided, duplicate ``m.room.member`` events should be - suppressed by the server across multiple calls. If ``lazy_load_members`` - is ``false`` this field is ignored. - - Only applicable when filtering state events, such as the - ``state`` section of a lazy-loading aware endpoint. + If ``true``, enables redudant membership events. Does not + apply unless ``lazy_load_members`` is ``true``. See + `Lazy-loading room members <#lazy-loading-room-members>`_ + for more information. Defaults to ``false``. not_rooms: description: A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the ``'rooms'`` diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 87e7bf74..bd659369 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -36,7 +36,8 @@ paths: incremental deltas to the state, and to receive new messages. *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ - for more information. + for more information. Lazy-loading members is only supported on a ``StateFilter`` + for this endpoint. operationId: sync security: - accessToken: [] diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 604c2b1c..a4b8dd5c 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1268,6 +1268,39 @@ Filters can be created on the server and can be passed as as a parameter to APIs which return events. These filters alter the data returned from those APIs. Not all APIs accept filters. +Lazy-loading room members +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Membership events often take significant resources for clients to track. In an +effort to reduce the number of resources used, clients can enable "lazy-loading" +for room members. By doing this, servers will only ever send membership events +which are relevant to the client. + +In terms of filters, this means enabling ``lazy_load_members`` on a ``StateFilter`` +or ``RoomEventFilter``. When enabled, lazy-loading aware endpoints (see below) +will only include membership events for the ``sender`` of events being included +in the response. For example, if a client makes a ``/sync`` request with lazy-loading +enabled, the server will only return membership events for the ``sender`` of events +in the timeline, not all members of a room. + +Repeated calls to lazy-loading aware endpoints will result in redundant membership +events being excluded by default. Clients often track which membership events they +already have, therefore making the extra information not as useful to the client. +Clients can always request redundant members by setting ``include_redundant_members`` +to true in the filter. + +Servers should be cautious about which events they consider redundant. Membership +events can change over time, and should be included as relevant to maintain the +historical record. Likewise, clients should be cautious about treating an older event +as the current membership event for a user. + +.. Note:: + Repeated calls using the same filter to *any* lazy-loading aware endpoint may + result in redundant members being excluded from future calls. For example, a + request to ``/sync`` followed by a request to ``/messages`` may result in a + future call to ``/sync`` excluding members included by the ``/messages`` call. + + {{filter_cs_http_api}} Events From 7b266b33da0cdabcfa6e2d2fdf4d3fc36f83ed0f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 19:26:26 -0600 Subject: [PATCH 104/282] Add membership params Fixes https://github.com/matrix-org/matrix-doc/issues/1945 --- api/client-server/rooms.yaml | 38 ++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/api/client-server/rooms.yaml b/api/client-server/rooms.yaml index cc1f2bf7..acddf891 100644 --- a/api/client-server/rooms.yaml +++ b/api/client-server/rooms.yaml @@ -288,6 +288,44 @@ paths: description: The room to get the member events for. required: true x-example: "!636q39766251:example.com" + - in: query + name: at + type: string + description: |- + The point in time (pagination token) to return members for in the room. + This token can be obtained from a ``prev_batch`` token returned for + each room by the sync API. Defaults to the current state of the room, + as determined by the server. + x-example: "YWxsCgpOb25lLDM1ODcwOA" + # XXX: As mentioned in MSC1227, replacing `[not_]membership` with a JSON + # filter might be a better alternative. + # See https://github.com/matrix-org/matrix-doc/issues/1337 + - in: query + name: membership + type: string + enum: + - join + - invite + - leave + - ban + description: |- + The kind of membership to filter for. Defaults to no filtering if + unspecified. When specified alongside ``not_membership``, the two + parameters create an 'or' condition: either the membership *is* + the same as ``membership`` **or** *is not* the same as ``not_membership``. + x-example: "join" + - in: query + name: not_membership + type: string + enum: + - join + - invite + - leave + - ban + description: |- + The kind of membership to exclude from the results. Defaults to no + filtering if unspecified. + x-example: leave security: - accessToken: [] responses: From b3d86f99b9d21ad98e26b37cf6afcc73d7064964 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 20:20:00 -0600 Subject: [PATCH 105/282] Add room summary spec --- api/client-server/sync.yaml | 52 +++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index bd659369..2c504295 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -129,6 +129,50 @@ paths: title: Joined Room type: object properties: + summary: + title: RoomSummary + type: object + description: |- + Information about the room which clients may need to + correctly render it to users. + properties: + "m.heroes": + type: array + description: |- + The users which can be used to generate a room name + if the room does not have one. Required if the room + does not have a ``m.room.name`` or ``m.room.canonical_alias`` + state event with non-empty content. + + This should be the first 5 members of the room, ordered + by stream ordering, which are joined or invited. The + list must never include the client's own user ID. When + no joined or invited members are available, this should + consist of the banned and left users. More than 5 members + may be provided, however less than 5 should only be provided + when there are less than 5 members to represent. + + When lazy-loading room members is enabled, the membership + events for the heroes MUST be included in the ``state``, + unless they are redundant. When the list of users changes, + the server notifies the client by sending a fresh list of + heroes. If there are no changes since the last sync, this + field may be omitted. + items: + type: string + "m.joined_member_count": + type: integer + description: |- + The number of users with ``membership`` of ``join``, + including the client's own user ID. If this field has + not changed since the last sync, it may be omitted. + Required otherwise. + "m.invited_member_count": + type: integer + description: |- + The number of users with ``membership`` of ``invite``. + If this field has not changed since the last sync, it + may be omitted. Required otherwise. state: title: State type: object @@ -334,6 +378,14 @@ paths: "rooms": { "join": { "!726s6s6q:example.com": { + "summary": { + "m.heroes": [ + "@alice:example.com", + "@bob:example.com" + ], + "m.joined_member_count": 2, + "m.invited_member_count": 0 + }, "state": { "events": [ { From 8330810e957bf33991274b9fad6cd258deaf670d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 20:47:03 -0600 Subject: [PATCH 106/282] Specify the new room naming scheme --- specification/modules/instant_messaging.rst | 98 ++++++++------------- 1 file changed, 35 insertions(+), 63 deletions(-) diff --git a/specification/modules/instant_messaging.rst b/specification/modules/instant_messaging.rst index 86daa700..dd3e9c6c 100644 --- a/specification/modules/instant_messaging.rst +++ b/specification/modules/instant_messaging.rst @@ -278,70 +278,42 @@ choose a name: #. If the room has an `m.room.canonical_alias`_ state event with a non-empty ``alias`` field, use the alias given by that field as the name. -#. If neither of the above conditions are met, a name should be composed based - on the members of the room. Clients should consider `m.room.member`_ events - for users other than the logged-in user, with ``membership: join`` or - ``membership: invite``. - - .. _active_members: - - i. If there is only one such event, the display name for the room should be - the `disambiguated display name`_ of the corresponding user. - - #. If there are two such events, they should be lexicographically sorted by - their ``state_key`` (i.e. the corresponding user IDs), and the display - name for the room should be the `disambiguated display name`_ of both - users: " and ", or a localised variant thereof. - - #. If there are three or more such events, the display name for the room - should be based on the disambiguated display name of the user - corresponding to the first such event, under a lexicographical sorting - according to their ``state_key``. The display name should be in the - format " and others" (or a localised variant thereof), where N - is the number of `m.room.member`_ events with ``membership: join`` or - ``membership: invite``, excluding the logged-in user and "user1". - - For example, if Alice joins a room, where Bob (whose user id is - ``@superuser:example.com``), Carol (user id ``@carol:example.com``) and - Dan (user id ``@dan:matrix.org``) are in conversation, Alice's - client should show the room name as "Carol and 2 others". - - .. TODO-spec - Sorting by user_id certainly isn't ideal, as IDs at the start of the - alphabet will end up dominating room names: they will all be called - "Arathorn and 15 others". Furthermore - user_ids are not necessarily - ASCII, which means we need to either specify a collation order, or specify - how to choose one. - - Ideally we might sort by the time when the user was first invited to, or - first joined the room. But we don't have this information. - - See https://matrix.org/jira/browse/SPEC-267 for further discussion. - -#. If the room has no valid ``m.room.name`` or ``m.room.canonical_alias`` - event, and no active members other than the current user, clients should - consider ``m.room.member`` events with ``membership: leave``. If such events - exist, a display name such as "Empty room (was and others)" (or - a localised variant thereof) should be used, following similar rules as for - active members (see `above `_). - -#. A complete absence of room name, canonical alias, and room members is likely - to indicate a problem with creating the room or synchronising the state - table; however clients should still handle this situation. A display name - such as "Empty room" (or a localised variant thereof) should be used in this - situation. - -.. _`disambiguated display name`: `Calculating the display name for a user`_ - -Clients SHOULD NOT use `m.room.aliases`_ events as a source for room names, as -it is difficult for clients to agree on the best alias to use, and aliases can -change unexpectedly. - -.. TODO-spec - How can we make this less painful for clients to implement, without forcing - an English-language implementation on them all? See - https://matrix.org/jira/browse/SPEC-425. +#. If neither of the above conditions are met, the client can optionally guess + an alias from the ``m.room.alias`` events in the room. This is a temporary + measure while clients do not promote canonical aliases as prominently. This + step may be removed in a future version of the specification. +#. If none of the above conditions are met, a name should be composed based + on the members of the room. Clients should consider `m.room.member`_ events + for users other than the logged-in user, as defined below. + + i. If the ``m.heroes`` for the room are greater or equal to + ``m.joined_member_count + m.invited_member_count - 1``, then use the + membership events for the heroes to calculate display names for the + users (`disambiguating them if required`_) and concatenating them. For + example, the client may choose to show "Alice, Bob, and Charlie + (@charlie:example.org)" as the room name. The client may optionally + limit the number + + #. If there are fewer heroes than ``m.joined_member_count + m.invited_member_count + - 1``, and ``m.joined_member_count + m.invited_member_count`` is greater + than 1, the client should use the heroes to calculate display names for + the users (`disambiguating them if required`_) and concatenating them + alongside a count of the remaining users. For example, "Alice, Bob, and + 1234 others". + + #. If ``m.joined_member_count + m.invited_member_count`` is less than or + equal to 1 (indicating the member is alone), the client should use the + rules above to indicate that the room was empty. For example, "Empty + Room (was Alice)", "Empty Room (was Alice and 1234 others)", or + "Empty Room" if there are no heroes. + +Clients SHOULD internationalise the room name to the user's language when using +the ``m.heroes`` to calculate the name. Clients SHOULD use minimum 5 heroes to +calculate room names where possible, but may use more or less to fit better with +their user experience. + +.. _`disambiguating them if required`: `Calculating the display name for a user`_ Forming relationships between events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From b1dccda49ddae41ec67e193b7da1f0bd1f345360 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 20:47:53 -0600 Subject: [PATCH 107/282] changelog --- changelogs/client_server/newsfragments/2035.feature | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2035.feature diff --git a/changelogs/client_server/newsfragments/2035.feature b/changelogs/client_server/newsfragments/2035.feature new file mode 100644 index 00000000..47029c28 --- /dev/null +++ b/changelogs/client_server/newsfragments/2035.feature @@ -0,0 +1 @@ +Add the option to lazy-load room members for increased client performance. From d14dc1d8e15a12b91cb20592d6e46c6b20ea678a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 21:15:24 -0600 Subject: [PATCH 108/282] Clarify that redacted state events affect the room with default values --- specification/client_server_api.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index a254a85d..22cc9425 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1533,11 +1533,12 @@ same way a server does. .. NOTE:: - Redacted events can still affect the state of the room. For example, a - redacted ``join`` event will still result in the user being considered - joined. Similarly, a redacted topic does not necessarily cause the topic - to revert to what is was prior to the event - it causes the topic to be - removed from the room. + Redacted events can still affect the state of the room. When redacted, + state events behave as though their properties were simply not specified + unless their properties are protected by the redaction algorithm. For example, + a redacted ``join`` event will still result in the user being considered joined. + Similarly, a redacted topic does not necessarily cause the topic to revert to + what is was prior to the event - it causes the topic to be removed from the room. Events From 229893589a30f21560cf55fac00d0112fa041413 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 21:18:04 -0600 Subject: [PATCH 109/282] Move wording for reason definitions --- event-schemas/schema/m.call.hangup | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/event-schemas/schema/m.call.hangup b/event-schemas/schema/m.call.hangup index 0328e03a..c0478f5a 100644 --- a/event-schemas/schema/m.call.hangup +++ b/event-schemas/schema/m.call.hangup @@ -1,6 +1,6 @@ { "type": "object", - "description": "Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call. The ``reason`` for the hangup is expected to be provided when there was an error in the call negotiation, such as ``ice_failed`` for when ICE negotiation fails or ``invite_timeout`` for when the other party did not answer in time.", + "description": "Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call.", "allOf": [{ "$ref": "core-event-schema/room_event.yaml" }], @@ -18,7 +18,7 @@ }, "reason": { "type": "string", - "description": "Optional error reason for the hangup. This should not be provided when the user naturally ends or rejects the call.", + "description": "Optional error reason for the hangup. This should not be provided when the user naturally ends or rejects the call. When there was an error in the call negotiation, this should be ``ice_failed`` for when ICE negotiation fails or ``invite_timeout`` for when the other party did not answer in time.", "enum": [ "ice_failed", "invite_timeout" From ade346f8cc173a532a0e991b61ed153a3ea93e9c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 21:19:58 -0600 Subject: [PATCH 110/282] Add m.server_notice to reserved tags --- specification/modules/tags.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specification/modules/tags.rst b/specification/modules/tags.rst index a4b0becf..9d0cd554 100644 --- a/specification/modules/tags.rst +++ b/specification/modules/tags.rst @@ -34,9 +34,9 @@ The JSON object associated with each tag gives information about the tag, e.g ho to order the rooms with a given tag. Ordering information is given under the ``order`` key as a number between 0 and -1. The numbers are compared such that 0 is displayed first. Therefore a room -with an ``order`` of ``0.2`` would be displayed before a room with an ``order`` -of ``0.7``. If a room has a tag without an ``order`` key then it should appear +1. The numbers are compared such that 0 is displayed first. Therefore a room +with an ``order`` of ``0.2`` would be displayed before a room with an ``order`` +of ``0.7``. If a room has a tag without an ``order`` key then it should appear after the rooms with that tag that have an ``order`` key. The name of a tag MUST NOT exceed 255 bytes. @@ -60,6 +60,7 @@ The following tags are defined in the ``m.*`` namespace: * ``m.favourite``: The user's favourite rooms. These should be shown with higher precedence than other rooms. * ``m.lowpriority``: These should be shown with lower precedence than others. +* ``m.server_notice``: Used to identify `Server Notice Rooms <#module-server-notices>`_. {{m_tag_event}} From 9acd960cf6afc8ddb011f7db5ca00fbf57cba5a5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 21:22:07 -0600 Subject: [PATCH 111/282] Update specification/client_server_api.rst Co-Authored-By: Kitsune Ral --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 22cc9425..c18f4d03 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1535,7 +1535,7 @@ same way a server does. Redacted events can still affect the state of the room. When redacted, state events behave as though their properties were simply not specified - unless their properties are protected by the redaction algorithm. For example, + except those protected by the redaction algorithm. For example, a redacted ``join`` event will still result in the user being considered joined. Similarly, a redacted topic does not necessarily cause the topic to revert to what is was prior to the event - it causes the topic to be removed from the room. From 699cafe6701d2206160170c6eaafeb789405507a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 21:27:54 -0600 Subject: [PATCH 112/282] v4 is v4, not v3 --- api/server-server/definitions/pdu_v4.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/server-server/definitions/pdu_v4.yaml b/api/server-server/definitions/pdu_v4.yaml index 55b1c26f..a045e657 100644 --- a/api/server-server/definitions/pdu_v4.yaml +++ b/api/server-server/definitions/pdu_v4.yaml @@ -13,7 +13,7 @@ # limitations under the License. type: object title: Persistent Data Unit -description: A persistent data unit (event) for room version 3 and beyond. +description: A persistent data unit (event) for room version 4 and beyond. example: $ref: "../examples/pdu_v4.json" allOf: From 10648aa9e8ce9556d3b881539acca7c9a7962156 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 22:30:37 -0600 Subject: [PATCH 113/282] Clarify that FAIL_ERROR is not limited to just homeservers. Fixes https://github.com/matrix-org/matrix-doc/issues/1735 --- changelogs/client_server/newsfragments/2036.clarification | 1 + specification/client_server_api.rst | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2036.clarification diff --git a/changelogs/client_server/newsfragments/2036.clarification b/changelogs/client_server/newsfragments/2036.clarification new file mode 100644 index 00000000..96058b7b --- /dev/null +++ b/changelogs/client_server/newsfragments/2036.clarification @@ -0,0 +1 @@ +Clarify that ``FAIL_ERROR`` in autodiscovery is not limited to just homeservers. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index a8bbfca0..4158bad8 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -289,7 +289,7 @@ In this section, the following terms are used with specific meanings: ``FAIL_ERROR`` Inform the user that auto-discovery did not return any usable URLs. Do not continue further with the current login process. At this point, valid data - was obtained, but no homeserver is available to serve the client. No further + was obtained, but no server is available to serve the client. No further guess should be attempted and the user should make a conscientious decision what to do next. From d0fd20fdb4b412c9bb596cfc2b06915d46c631a3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 22:40:07 -0600 Subject: [PATCH 114/282] Clarify how homeservers are meant to auth themselves to appservices Fixes https://github.com/matrix-org/matrix-doc/issues/1765 Note that the swagger definitions already say that authorization is required. It just wasn't mentioned in the spec. --- .../application_service/newsfragments/2037.clarification | 1 + specification/application_service_api.rst | 8 ++++++++ 2 files changed, 9 insertions(+) create mode 100644 changelogs/application_service/newsfragments/2037.clarification diff --git a/changelogs/application_service/newsfragments/2037.clarification b/changelogs/application_service/newsfragments/2037.clarification new file mode 100644 index 00000000..f425b1c1 --- /dev/null +++ b/changelogs/application_service/newsfragments/2037.clarification @@ -0,0 +1 @@ +Add missing definition for how appservices verify requests came from a homeserver. diff --git a/specification/application_service_api.rst b/specification/application_service_api.rst index 865544dd..81077bcf 100644 --- a/specification/application_service_api.rst +++ b/specification/application_service_api.rst @@ -187,6 +187,14 @@ An example registration file for an IRC-bridging application service is below: Homeserver -> Application Service API ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Authorization ++++++++++++++ + +Homeservers MUST include a query parameter named ``access_token`` containing the +``hs_token`` from the application service's registration when making requests to +the application service. Application services MUST verify the provided ``access_token`` +matches their known ``hs_token``, failing the request with a ``M_FORBIDDEN`` error. + Legacy routes +++++++++++++ From d8eb2949066c9c53f7501da9fb86f76fa762e960 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 22:44:24 -0600 Subject: [PATCH 115/282] Fix s2s changelog entries --- changelogs/server_server/newsfragments/1904.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/server_server/newsfragments/1904.clarification b/changelogs/server_server/newsfragments/1904.clarification index 94174ebd..531fdb94 100644 --- a/changelogs/server_server/newsfragments/1904.clarification +++ b/changelogs/server_server/newsfragments/1904.clarification @@ -1 +1 @@ -Fix the `access_token` parameter in the open_id endpoint. +Fix the ``access_token`` parameter in the open_id endpoint. From 88d47fd57e423c3e2e958fc547f7ef4812eb3ca2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 22:51:52 -0600 Subject: [PATCH 116/282] Rename changelog entry to point to PR We use the PR for every other entry, so here is no exception. --- .../newsfragments/{1904.clarification => 1906.clarification} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/server_server/newsfragments/{1904.clarification => 1906.clarification} (100%) diff --git a/changelogs/server_server/newsfragments/1904.clarification b/changelogs/server_server/newsfragments/1906.clarification similarity index 100% rename from changelogs/server_server/newsfragments/1904.clarification rename to changelogs/server_server/newsfragments/1906.clarification From 62890d21b25016dc008a5cc2e2f993ee3a0873ab Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 23:08:02 -0600 Subject: [PATCH 117/282] Fix example Content-Type for /media/upload request Fixes https://github.com/matrix-org/matrix-doc/issues/1770 --- api/client-server/content-repo.yaml | 11 ++++++----- .../client_server/newsfragments/2041.clarification | 1 + scripts/templating/matrix_templates/units.py | 9 +++++++-- 3 files changed, 14 insertions(+), 7 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2041.clarification diff --git a/api/client-server/content-repo.yaml b/api/client-server/content-repo.yaml index 07df18fe..4460bb69 100644 --- a/api/client-server/content-repo.yaml +++ b/api/client-server/content-repo.yaml @@ -41,7 +41,7 @@ paths: name: Content-Type type: string description: The content type of the file being uploaded - x-example: "Content-Type: audio/mpeg" + x-example: "Content-Type: application/pdf" - in: query type: string x-example: "War and Peace.pdf" @@ -51,6 +51,7 @@ paths: name: "" description: The content to be uploaded. required: true + x-example: "" # so the spec shows "" without quotes. schema: type: string example: "" @@ -103,7 +104,7 @@ paths: default: true description: | Indicates to the server that it should not attempt to fetch the media if it is deemed - remote. This is to prevent routing loops where the server contacts itself. Defaults to + remote. This is to prevent routing loops where the server contacts itself. Defaults to true if not provided. responses: 200: @@ -158,7 +159,7 @@ paths: default: true description: | Indicates to the server that it should not attempt to fetch the media if it is deemed - remote. This is to prevent routing loops where the server contacts itself. Defaults to + remote. This is to prevent routing loops where the server contacts itself. Defaults to true if not provided. responses: 200: @@ -228,7 +229,7 @@ paths: default: true description: | Indicates to the server that it should not attempt to fetch the media if it is deemed - remote. This is to prevent routing loops where the server contacts itself. Defaults to + remote. This is to prevent routing loops where the server contacts itself. Defaults to true if not provided. responses: 200: @@ -330,7 +331,7 @@ paths: m.upload.size: type: integer format: int64 - description: |- + description: |- The maximum size an upload can be in bytes. Clients SHOULD use this as a guide when uploading content. If not listed or null, the size limit should be treated as unknown. diff --git a/changelogs/client_server/newsfragments/2041.clarification b/changelogs/client_server/newsfragments/2041.clarification new file mode 100644 index 00000000..39bbddb5 --- /dev/null +++ b/changelogs/client_server/newsfragments/2041.clarification @@ -0,0 +1 @@ +Fix example ``Content-Type`` for ``/media/upload`` request. diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index 466110f8..d4745e4d 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -521,6 +521,7 @@ class MatrixUnits(Units): path_template = path example_query_params = [] example_body = "" + example_mime = "application/json" for param in endpoint_swagger.get("parameters", []): # even body params should have names, otherwise the active docs don't work. param_name = param["name"] @@ -533,6 +534,10 @@ class MatrixUnits(Units): example_body = get_example_for_param(param) continue + if param_loc == "header": + if param["name"] == "Content-Type" and param["x-example"]: + example_mime = param["x-example"] + # description desc = param.get("description", "") if param.get("required"): @@ -610,8 +615,8 @@ class MatrixUnits(Units): example_query_params) if example_body: endpoint["example"][ - "req"] = "%s %s%s HTTP/1.1\nContent-Type: application/json\n\n%s" % ( - method.upper(), path_template, query_string, example_body + "req"] = "%s %s%s HTTP/1.1\nContent-Type: %s\n\n%s" % ( + method.upper(), path_template, query_string, example_mime, example_body ) else: endpoint["example"]["req"] = "%s %s%s HTTP/1.1\n\n" % ( From 572a6056ad64c5f3f654f91a27cf8a42c19d6ee5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 23:15:02 -0600 Subject: [PATCH 118/282] Clarify that login flows must be completed in order Fixes https://github.com/matrix-org/matrix-doc/issues/1134 Evidence of this being the case is shown here: https://github.com/matrix-org/synapse/pull/5174 --- changelogs/client_server/newsfragments/2042.clarification | 1 + specification/client_server_api.rst | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2042.clarification diff --git a/changelogs/client_server/newsfragments/2042.clarification b/changelogs/client_server/newsfragments/2042.clarification new file mode 100644 index 00000000..4e17b99f --- /dev/null +++ b/changelogs/client_server/newsfragments/2042.clarification @@ -0,0 +1 @@ +Clarify that login flows are meant to be completed in order. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index a8bbfca0..4b7065b3 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -406,8 +406,9 @@ an additional stage. This exchange continues until the final success. For each endpoint, a server offers one or more 'flows' that the client can use to authenticate itself. Each flow comprises a series of stages, as described -above. The client is free to choose which flow it follows. When all stages in a -flow are complete, authentication is complete and the API call succeeds. +above. The client is free to choose which flow it follows, however the flow's +stages must be completed in order. When all stages in a flow are complete, +authentication is complete and the API call succeeds. User-interactive API in the REST API <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< From 4e58414b26d9d2bc145393091b7e141574c36450 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 23:20:09 -0600 Subject: [PATCH 119/282] Clarify that clients should not ack their own messages Fixes https://github.com/matrix-org/matrix-doc/issues/567 --- changelogs/client_server/newsfragments/2043.clarification | 1 + specification/modules/receipts.rst | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2043.clarification diff --git a/changelogs/client_server/newsfragments/2043.clarification b/changelogs/client_server/newsfragments/2043.clarification new file mode 100644 index 00000000..9bb975fa --- /dev/null +++ b/changelogs/client_server/newsfragments/2043.clarification @@ -0,0 +1 @@ +Clarify that clients should not send read receipts for their own messages. diff --git a/specification/modules/receipts.rst b/specification/modules/receipts.rst index faba7b62..ee2b697a 100644 --- a/specification/modules/receipts.rst +++ b/specification/modules/receipts.rst @@ -60,7 +60,8 @@ Clients should send read receipts when there is some certainty that the event in question has been **displayed** to the user. Simply receiving an event does not provide enough certainty that the user has seen the event. The user SHOULD need to *take some action* such as viewing the room that the event was sent to or -dismissing a notification in order for the event to count as "read". +dismissing a notification in order for the event to count as "read". Clients +SHOULD NOT send read receipts for events sent by their own user. A client can update the markers for its user by interacting with the following HTTP APIs. @@ -94,4 +95,3 @@ Security considerations As receipts are sent outside the context of the event graph, there are no integrity checks performed on the contents of ``m.receipt`` events. - From bf86b4b83ca9ee97ef8bc7edcc17a663996bcffd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 May 2019 23:42:08 -0600 Subject: [PATCH 120/282] Fix incorrect PR reference on changelog --- .../client_server/newsfragments/{1903.feature => 1908.feature} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/client_server/newsfragments/{1903.feature => 1908.feature} (100%) diff --git a/changelogs/client_server/newsfragments/1903.feature b/changelogs/client_server/newsfragments/1908.feature similarity index 100% rename from changelogs/client_server/newsfragments/1903.feature rename to changelogs/client_server/newsfragments/1908.feature From 0580f5120693e2535920f5e57e4218e9467b7d18 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 12:49:50 -0600 Subject: [PATCH 121/282] Clarify that failing to follow the flows == 401 --- specification/client_server_api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 4b7065b3..3ea1411e 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -407,8 +407,9 @@ an additional stage. This exchange continues until the final success. For each endpoint, a server offers one or more 'flows' that the client can use to authenticate itself. Each flow comprises a series of stages, as described above. The client is free to choose which flow it follows, however the flow's -stages must be completed in order. When all stages in a flow are complete, -authentication is complete and the API call succeeds. +stages must be completed in order. Failing to follow the flows in order must +result in an HTTP 401 response, as defined below. When all stages in a flow +are complete, authentication is complete and the API call succeeds. User-interactive API in the REST API <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< From 8151aa331f6b553ffa755f38aaffb5f5b466b07a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 12:51:36 -0600 Subject: [PATCH 122/282] Update specification/client_server_api.rst Co-Authored-By: Hubert Chathi --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index c18f4d03..a8246578 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1534,7 +1534,7 @@ same way a server does. .. NOTE:: Redacted events can still affect the state of the room. When redacted, - state events behave as though their properties were simply not specified + state events behave as though their properties were simply not specified, except those protected by the redaction algorithm. For example, a redacted ``join`` event will still result in the user being considered joined. Similarly, a redacted topic does not necessarily cause the topic to revert to From e2da3728a0631f95bae08f7ceb4a18b695841d3b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 12:53:48 -0600 Subject: [PATCH 123/282] Only error if the token doesn't match --- specification/application_service_api.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specification/application_service_api.rst b/specification/application_service_api.rst index 81077bcf..3220df2d 100644 --- a/specification/application_service_api.rst +++ b/specification/application_service_api.rst @@ -193,7 +193,8 @@ Authorization Homeservers MUST include a query parameter named ``access_token`` containing the ``hs_token`` from the application service's registration when making requests to the application service. Application services MUST verify the provided ``access_token`` -matches their known ``hs_token``, failing the request with a ``M_FORBIDDEN`` error. +matches their known ``hs_token``, failing the request with a ``M_FORBIDDEN`` error +if it does not match. Legacy routes +++++++++++++ From 3ade2a9ae78406c2804d57f4dbda09e0655bcb41 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 13:56:25 -0600 Subject: [PATCH 124/282] List the endpoints which support LL --- specification/client_server_api.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index a4b8dd5c..740023fb 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1300,6 +1300,11 @@ as the current membership event for a user. request to ``/sync`` followed by a request to ``/messages`` may result in a future call to ``/sync`` excluding members included by the ``/messages`` call. +The current endpoints which support lazy-loading room members are: + +* |/sync|_ +* |/rooms//messages|_ +* |/rooms/{roomId}/context/{eventId}|_ {{filter_cs_http_api}} From 551806a8ad797895a53f4555a252f1d59048ae43 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 13:54:22 -0600 Subject: [PATCH 125/282] Add a reference to the filtering module to /sync --- api/client-server/sync.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 2c504295..a9f1f714 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -53,6 +53,8 @@ paths: requests. Creating a filter using the filter API is recommended for clients that reuse the same filter multiple times, for example in long poll requests. + + See `Filtering <#filtering>`_ for more information. x-example: "66696p746572" - in: query name: since From c5fdd5cb0bef008945cde2e915245ade190541c5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 13:55:37 -0600 Subject: [PATCH 126/282] Change note style --- api/client-server/event_context.yaml | 2 +- api/client-server/message_pagination.yaml | 2 +- api/client-server/sync.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index c0f391e5..327c8f43 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -35,7 +35,7 @@ paths: after the specified event. This allows clients to get the context surrounding an event. - *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ for more information. operationId: getEventContext security: diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 716afb65..8469eec4 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -34,7 +34,7 @@ paths: This API returns a list of message and state events for a room. It uses pagination query parameters to paginate history in the room. - *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ for more information. operationId: getRoomEvents security: diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index a9f1f714..4fe22d50 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -35,7 +35,7 @@ paths: of the state on the server, and then continue to call this API to get incremental deltas to the state, and to receive new messages. - *Note*: this endpoint supports lazy-loading. See `Filtering <#filtering>`_ + *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ for more information. Lazy-loading members is only supported on a ``StateFilter`` for this endpoint. operationId: sync From 0463084924543ee5b024adb62ac32ee5339855c5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 14:19:56 -0600 Subject: [PATCH 127/282] Spec 3PID unbind API As per [MSC1915](https://github.com/matrix-org/matrix-doc/pull/1915) Implementation proof: * https://github.com/matrix-org/synapse/pull/4982 * https://github.com/matrix-org/sydent/pull/160 The only alteration made which differs from the proposal is clarity on how to handle homeservers not knowing the `id_server`. All other differences are unintentional. --- api/client-server/administrative_contact.yaml | 27 +++++- api/client-server/registration.yaml | 30 ++++++- api/identity/associations.yaml | 83 +++++++++++++++++++ .../client_server/newsfragments/2046.feature | 1 + .../identity_service/newsfragments/2046.new | 1 + .../1915-unbind-identity-server-param.md | 4 +- 6 files changed, 142 insertions(+), 4 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2046.feature create mode 100644 changelogs/identity_service/newsfragments/2046.new diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 33ea9786..5cf01805 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -163,6 +163,14 @@ paths: schema: type: object properties: + id_server: + type: string + description: |- + The identity server to unbind from. If not provided, the homeserver + MUST use the ``id_server`` the identifier was added through. If the + homeserver does not know the original ``id_server``, it MUST return + a ``id_server_unbind_result`` of ``no-support``. + example: "example.org" medium: type: string description: The medium of the third party identifier being removed. @@ -180,7 +188,24 @@ paths: user. schema: type: object - properties: {} + properties: + id_server_unbind_result: + type: string + enum: + # XXX: I don't know why, but the order matters here so that "no-support" + # doesn't become "no- support" by the renderer. + - "no-support" + - "success" + description: |- + An indicator as to whether or not the homeserver was able to unbind + the 3PID from the identity server. ``success`` indicates that the + indentity server has unbound the identifier whereas ``no-support`` + indicates that the identity server refuses to support the request + or the homeserver was not able to determine an identity server to + unbind from. + example: "success" + required: + - id_server_unbind_result tags: - User data "/account/3pid/email/requestToken": diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..f003cf1f 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -524,13 +524,39 @@ paths: description: |- Additional authentication information for the user-interactive authentication API. "$ref": "definitions/auth_data.yaml" + id_server: + type: string + description: |- + The identity server to unbind all of the user's 3PIDs from. + If not provided, the homeserver MUST use the ``id_server`` + that was originally use to bind each identifier. If the + homeserver does not know which ``id_server`` that was, + it must return an ``id_server_unbind_result`` of + ``no-support``. + example: "example.org" responses: 200: description: The account has been deactivated. - examples: - application/json: {} schema: type: object + properties: + id_server_unbind_result: + type: string + enum: + - "success" + - "no-support" + description: |- + An indicator as to whether or not the homeserver was able to unbind + the user's 3PIDs from the identity server(s). ``success`` indicates + that all identifiers have been unbound from the identity server while + ``no-support`` indicates that one or more identifiers failed to unbind + due to the identity server refusing the request or the homeserver + being unable to determine an identity server to unbind from. This + must be ``success`` if the homeserver has no identifiers to unbind + for the user. + example: "success" + required: + - id_server_unbind_result 401: description: |- The homeserver requires additional authentication information. diff --git a/api/identity/associations.yaml b/api/identity/associations.yaml index 152a0a9b..f44fe3cc 100644 --- a/api/identity/associations.yaml +++ b/api/identity/associations.yaml @@ -201,3 +201,86 @@ paths: } schema: $ref: "../client-server/definitions/errors/error.yaml" + "/3pid/unbind": + post: + summary: Remove an association between a session and a Matrix user ID. + description: |- + Remove an association between a session and a Matrix user ID. + + Future calls to ``/lookup`` for any of the session's 3pids will not + return the removed association. + + The identity server should authenticate the request in one of two + ways: + + 1. The request is signed by the homeserver which controls the ``user_id``. + 2. The request includes the ``sid`` and ``client_secret`` parameters, + as per ``/3pid/bind``, which proves ownership of the 3PID. + + If this endpoint returns a JSON Matrix error, that error should be passed + through to the client requesting an unbind through a homeserver, if the + homeserver is acting on behalf of a client. + operationId: unbind + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "mxid": "@ears:example.org", + "threepid": { + "medium": "email", + "address": "monkeys_have_ears@example.org" + } + } + properties: + sid: + type: string + description: The Session ID generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret passed to the ``requestToken`` call. + mxid: + type: string + description: The Matrix user ID to remove from the 3pids. + threepid: + type: object + title: 3PID + description: |- + The 3PID to remove. Must match the 3PID used to generate the session + if using ``sid`` and ``client_secret`` to authenticate this request. + properties: + medium: + type: string + description: |- + A medium from the `3PID Types`_ Appendix, matching the medium + of the identifier to unbind. + address: + type: string + description: The 3PID address to remove. + required: ['medium', 'address'] + required: ["threepid", "mxid"] + responses: + 200: + description: The association was successfully removed. + examples: + application/json: {} + schema: + type: object + 400: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 404: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 501: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. diff --git a/changelogs/client_server/newsfragments/2046.feature b/changelogs/client_server/newsfragments/2046.feature new file mode 100644 index 00000000..e54df535 --- /dev/null +++ b/changelogs/client_server/newsfragments/2046.feature @@ -0,0 +1 @@ +Add ``id_server`` to ``/deactivate`` and ``/3pid/delete`` endpoints to unbind from a specific identity server. diff --git a/changelogs/identity_service/newsfragments/2046.new b/changelogs/identity_service/newsfragments/2046.new new file mode 100644 index 00000000..7146799b --- /dev/null +++ b/changelogs/identity_service/newsfragments/2046.new @@ -0,0 +1 @@ +Add ``/3pid/unbind`` for removing 3PIDs. diff --git a/proposals/1915-unbind-identity-server-param.md b/proposals/1915-unbind-identity-server-param.md index 6817ece3..5b7a1a50 100644 --- a/proposals/1915-unbind-identity-server-param.md +++ b/proposals/1915-unbind-identity-server-param.md @@ -27,7 +27,9 @@ known by the homeserver). The 200 response is a JSON object with an `id_server_unbind_result` field whose value is either `success` or `no-support`, where the latter indicates that the identity server (IS) does not support unbinding 3PIDs directly. If the identity -server returns an error then that should be returned to the client. +server returns an error then that should be returned to the client. If the homeserver +is unable to determine an `id_server` to use, it should return `no-support` for +the `id_server_unbind_result`. Example: From 84f0d9d7e4467460a9caf9128cd739ddd2fb03b6 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 28 May 2019 20:07:45 -0400 Subject: [PATCH 128/282] add clarification --- proposals/1717-key_verification.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1717-key_verification.md b/proposals/1717-key_verification.md index 6c9e2341..429e3a97 100644 --- a/proposals/1717-key_verification.md +++ b/proposals/1717-key_verification.md @@ -139,7 +139,8 @@ Properties: device, an `m.key.verification.cancel` message with `code` set to `m.accepted` is sent to the other devices - `reason` (string): human-readable reason for cancelling. This should only be - used if the recieving client does not understand the code given. + used if the recieving client does not understand the code given in the `code` + property. Verification methods may define their own additional cancellation codes. Cancellation codes defined in the Matrix spec will begin with `m.`; other From 6706d772c91be7c4ab5c8089f4579695e461ffd4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 00:46:00 -0600 Subject: [PATCH 129/282] Fix test vectors with invalid JSON and signature Fixes https://github.com/matrix-org/matrix-doc/issues/2023 The content hashes appear correct, however applying the algorithm defined in the spec never resulted in the signatures previously demonstrated. --- specification/appendices/test_vectors.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index e2b8fb58..7759fa88 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -114,7 +114,7 @@ The event signing algorithm should emit the following signed event: "origin_server_ts": 1000000, "signatures": { "domain": { - "ed25519:1": "2Wptgo4CwmLo/Y8B8qinxApKaCkBG2fjTWB7AbP5Uy+aIbygsSdLOFzvdDjww8zUVKCmI02eP9xtyJxc/cLiBA" + "ed25519:1": "JV2dlZUASAefSdywnyCxzykHlyr7xkKGK7IRir1cF8eYsnONrCSb+GRn7aXXstr1UHKvzYjRXPx0001+boD1Ag" } }, "type": "X", @@ -129,7 +129,7 @@ Given the following event containing redactable content: { "content": { - "body": "Here is the message content", + "body": "Here is the message content" }, "event_id": "$0:domain", "origin": "domain", @@ -149,7 +149,7 @@ The event signing algorithm should emit the following signed event: { "content": { - "body": "Here is the message content", + "body": "Here is the message content" }, "event_id": "$0:domain", "hashes": { @@ -162,7 +162,7 @@ The event signing algorithm should emit the following signed event: "sender": "@u:domain", "signatures": { "domain": { - "ed25519:1": "Wm+VzmOUOz08Ds+0NTWb1d4CZrVsJSikkeRxh6aCcUwu6pNC78FunoD7KNWzqFn241eYHYMGCA5McEiVPdhzBA" + "ed25519:1": "4zc79tH2cU6Y+eg4YbbF7KiDOrnwEDjlhTqIKiH4k7L9zD9XCiomD7x9odL9eEwnyy1144QyMBe8O3HK++GHBg" } }, "unsigned": { From ffb70a2fabd46f5197cf123d6ed8bcd5d613566d Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 29 May 2019 15:01:35 -0400 Subject: [PATCH 130/282] fix typo Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/1719-olm_unwedging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index c181e3f7..5de6463f 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -19,7 +19,7 @@ from; the client should not create a new session with another device if it has already created one for that given device in the past 1 hour. Clients may wish to ask the sender of the undecryptable messages to re-send the -message. For exampe, if the undecryptable message was a megolm session, then +message. For example, if the undecryptable message was a megolm session, then the client can send an [`m.room_key_request`](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-room-key-request) message to request that the sender re-send the key. From 6929579360113e4e653e884f7e4cb5bfa68fbbd7 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 29 May 2019 15:18:54 -0400 Subject: [PATCH 131/282] add some clarifications --- proposals/1719-olm_unwedging.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/proposals/1719-olm_unwedging.md b/proposals/1719-olm_unwedging.md index 5de6463f..dffdc8b1 100644 --- a/proposals/1719-olm_unwedging.md +++ b/proposals/1719-olm_unwedging.md @@ -1,8 +1,13 @@ # Olm unwedging Olm sessions sometimes get out of sync, resulting in undecryptable messages. -This proposal documents a method for devices to create a new session to replace -the broken session. +This can happen for several reasons. For example, if a user restores their +client state from a backup, the client will be using an old ratchet state +([riot-web#3822](https://github.com/vector-im/riot-web/issues/3822)). Or a +client might expire a one-time key that another client is trying to use +([riot-web#3309](https://github.com/vector-im/riot-web/issues/3309)). This +proposal documents a method for devices to create a new session to replace the +broken session. ## Proposal @@ -18,11 +23,11 @@ the number of new sessions it creates per device that it receives a message from; the client should not create a new session with another device if it has already created one for that given device in the past 1 hour. -Clients may wish to ask the sender of the undecryptable messages to re-send the -message. For example, if the undecryptable message was a megolm session, then -the client can send an -[`m.room_key_request`](https://matrix.org/docs/spec/client_server/r0.4.0.html#m-room-key-request) -message to request that the sender re-send the key. +Clients may wish to take steps to mitigate the loss of the undecryptable +messages. For example, megolm sessions that were sent using the old session +would have been lost, so the client can send +[`m.room_key_request`](https://matrix.org/docs/spec/client_server/latest.html#m-room-key-request) +messages to re-request any megolm sessions that it is unable to decrypt. The spec currently says, "If a client has multiple sessions established with another device, it should use the session from which it last received a From 15b8011f63a687bc8961c881c1dcb73fd4318c6b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 13:37:58 -0600 Subject: [PATCH 132/282] Add missing aesthetic parameters to /store-invite Fixes https://github.com/matrix-org/matrix-doc/issues/2048 --- api/identity/store_invite.yaml | 47 ++++++++++++++++--- .../newsfragments/2049.clarification | 1 + 2 files changed, 42 insertions(+), 6 deletions(-) create mode 100644 changelogs/identity_service/newsfragments/2049.clarification diff --git a/api/identity/store_invite.yaml b/api/identity/store_invite.yaml index 69103294..bca78d7e 100644 --- a/api/identity/store_invite.yaml +++ b/api/identity/store_invite.yaml @@ -50,31 +50,66 @@ paths: requests to ``/_matrix/identity/api/v1/pubkey/ephemeral/isvalid``. Currently, invites may only be issued for 3pids of the ``email`` medium. + + Optional fields in the request should be populated to the best of the + server's ability. Identity servers may use these variables when notifying + the ``address`` of the pending invite for display purposes. operationId: storeInvite parameters: - in: body name: body schema: type: object - example: { - "medium": "email", - "address": "foo@bar.baz", - "room_id": "!something:example.tld", - "sender": "@bob:example.com" - } properties: medium: type: string description: The literal string ``email``. + example: "email" address: type: string description: The email address of the invited user. + example: "foo@example.com" room_id: type: string description: The Matrix room ID to which the user is invited + example: "!something:example.org" sender: type: string description: The Matrix user ID of the inviting user + example: "@bob:example.com" + room_alias: + type: string + description: |- + The Matrix room alias for the room to which the user is + invited. This should be retrieved from the ``m.room.canonical_alias`` + state event. + example: "#somewhere:exmaple.org" + room_avatar_url: + type: string + description: |- + The Content URI for the room to which the user is invited. This should + be retrieved from the ``m.room.avatar`` state event. + example: "mxc://example.org/s0meM3dia" + room_join_rules: + type: string + description: |- + The ``join_rule`` for the room to which the user is invited. This should + be retrieved from the ``m.room.join_rules`` state event. + example: "public" + room_name: + type: string + description: |- + The name of the room to which the user is invited. This should be retrieved + from the ``m.room.name`` state event. + example: "Bob's Emporium of Messages" + sender_display_name: + type: string + description: The display name of the user ID initiating the invite. + example: "Bob Smith" + sender_avatar_url: + type: string + description: The Content URI for the avatar of the user ID initiating the invite. + example: "mxc://example.org/an0th3rM3dia" required: ["medium", "address", "room_id", "sender"] responses: 200: diff --git a/changelogs/identity_service/newsfragments/2049.clarification b/changelogs/identity_service/newsfragments/2049.clarification new file mode 100644 index 00000000..403ac8d0 --- /dev/null +++ b/changelogs/identity_service/newsfragments/2049.clarification @@ -0,0 +1 @@ +Add missing aesthetic parameters to ``/store-invite``. From 78d93432f4cf3fde1cb11af37b95fe16b46bbe0c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 14:00:34 -0600 Subject: [PATCH 133/282] Synchronize proposals_intro.rst and CONTRIBUTING.rst --- CONTRIBUTING.rst | 28 +++++++++++++++------------- specification/proposals_intro.rst | 26 +++++++++++++++++--------- 2 files changed, 32 insertions(+), 22 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 7a6c6be2..0b814fb9 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -26,10 +26,11 @@ For this to be effective, the APIs need to be present and working correctly in a server before they can be documented in the specification. This process can take some time to complete. -For this reason, we have not found the github pull-request model effective for -discussing changes to the specification. Instead, we have adopted the workflow -as described at https://matrix.org/docs/spec/proposals - *please read this for -details on how to contribute spec changes*. +Changes to the protocol (new endpoints, ideas, etc) need to go through the +`proposals process `_. Other changes, +such as fixing bugs, typos, or clarifying existing behaviour do not need a proposal. +If you're not sure, visit us at `#matrix-spec:matrix.org`_ +and ask. Other changes @@ -51,8 +52,7 @@ following: `_ label. (If there is any doubt about whether it is the spec or the implementations - that need fixing, please discuss it with us first in `#matrix-dev:matrix.org - `_.) + that need fixing, please discuss it with us first in `#matrix-spec:matrix.org`_.) * Clarifications to the specification which do not change the behaviour of Matrix servers or clients in a way which might introduce compatibility @@ -60,14 +60,16 @@ following: `clarification `_ label. - For example, recommendations for UI behaviour do not require a proposal - document. On the other hand, changes to event contents would be best - discussed in a proposal document even though no changes would be necessary to - server implementations. + For example, areas where the specification is unclear do not require a proposal + to fix. On the other hand, introducing new behaviour is best represented by a + proposal. -For such changes, please do just open a `pull request`_. +For such changes, please do just open a `pull request`_. If you're not sure if +your change is covered by the above, please visit `#matrix-spec:matrix.org` and +ask. -.. _pull request: https://help.github.com/articles/about-pull-requests +.. _`pull request`: https://help.github.com/articles/about-pull-requests +.. _`#matrix-spec:matrix.org`: https://matrix.to/#/#matrix-spec:matrix.org Adding to the changelog @@ -96,7 +98,7 @@ the ``newsfragments`` directory. The ``type`` can be one of the following: * ``breaking`` - Used when the change is not backwards compatible. -* ``deprecation`` - Used when deprecating something +* ``deprecation`` - Used when deprecating something. All news fragments must have a brief summary explaining the change in the contents of the file. The summary must end in a full stop to be in line with diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index d95128b7..d492700e 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -13,12 +13,18 @@ Proposals for Spec Changes to Matrix If you are interested in submitting a change to the Matrix Specification, please take note of the following guidelines. -All changes to Specification content require a formal proposal process. This -involves writing a proposal, having it reviewed by everyone, having the -proposal being accepted, then actually having your ideas implemented as -committed changes to the `Specification repository +Most changes to the Specification require a formal proposal. Bug fixes, typos, +and clarifications to existing behaviour do not need proposals - see the +`contributing guide `_ +for more information on what does and does not need a proposal. + +The proposal process involves some technical writing, having it reviewed by +everyone, having the proposal being accepted, then actually having your ideas +implemented as committed changes to the `Specification repository `_. +.. TODO: Replace GH team link with https://matrix.org/foundation or something + Meet the `members of the Core Team `_, a group of individuals tasked with ensuring the spec process is as smooth and painless as @@ -33,14 +39,15 @@ Guiding Principles Proposals **must** act to the greater benefit of the entire Matrix ecosystem, rather than benefiting or privileging any single player or subset of players - -and must not contain any patent encumbered intellectual property. Members of the Core Team pledge to act as -a neutral custodian for Matrix on behalf of the whole ecosystem. +and must not contain any patent encumbered intellectual property. Members of +the Core Team pledge to act as a neutral custodian for Matrix on behalf of the +whole ecosystem. For clarity: the Matrix ecosystem is anyone who uses the Matrix protocol. That includes client users, server admins, client developers, bot developers, -bridge and application service developers, users and admins who are indirectly using Matrix via -3rd party networks which happen to be bridged, server developers, room -moderators and admins, companies/projects building products or services on +bridge and application service developers, users and admins who are indirectly +using Matrix via 3rd party networks which happen to be bridged, server developers, +room moderators and admins, companies/projects building products or services on Matrix, spec contributors, translators, and those who created it in the first place. @@ -242,6 +249,7 @@ Spec PR Merged merged A proposal with Postponed proposal-postponed A proposal that is temporarily blocked or a feature that may not be useful currently but perhaps sometime in the future Closed proposal-closed A proposal which has been reviewed and deemed unsuitable for acceptance +Obsolete obsolete A proposal which has been made obsolete by another proposal or decision elsewhere. =============================== ============================= ==================================== From c25afa663ef619a82b4d7f93f96118974a59d4a0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 14:56:42 -0600 Subject: [PATCH 134/282] Use consistent event examples throughout the Client-Server API Fixes https://github.com/matrix-org/matrix-doc/issues/1968 --- api/client-server/event_context.yaml | 88 ++++-------- api/client-server/message_pagination.yaml | 39 +---- api/client-server/notifications.yaml | 11 +- api/client-server/old_sync.yaml | 136 ++---------------- api/client-server/peeking_events.yaml | 12 +- api/client-server/room_initial_sync.yaml | 89 +----------- api/client-server/rooms.yaml | 136 +++--------------- api/client-server/search.yaml | 13 +- api/client-server/sync.yaml | 44 ++---- .../newsfragments/2050.clarification | 1 + 10 files changed, 94 insertions(+), 475 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2050.clarification diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 91da3cf4..549902a0 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -101,65 +101,35 @@ paths: - "$ref": "definitions/event-schemas/schema/core-event-schema/state_event.yaml" examples: application/json: { - "end": "t29-57_2_0_2", - "events_after": [ - { - "age": 91911336, - "content": { - "body": "7", - "msgtype": "m.text" - }, - "event_id": "$14460306086CiUaL:localhost:8480", - "origin_server_ts": 1446030608551, - "room_id": "!sCDvXTtzjpiPxaqkkt:localhost:8480", - "type": "m.room.message", - "sender": "@test:localhost:8480" - } - ], - "events_before": [ - { - "age": 91911903, - "content": { - "body": "5", - "msgtype": "m.text" - }, - "event_id": "$14460306074UYTlh:localhost:8480", - "origin_server_ts": 1446030607984, - "room_id": "!sCDvXTtzjpiPxaqkkt:localhost:8480", - "type": "m.room.message", - "sender": "@test:localhost:8480" - } - ], - "start": "t27-54_2_0_2", - "state": [ - { - "age": 3123715284, - "content": { - "creator": "@test:localhost:8480" - }, - "event_id": "$14429988040dgQAE:localhost:8480", - "origin_server_ts": 1442998804603, - "room_id": "!sCDvXTtzjpiPxaqkkt:localhost:8480", - "state_key": "", - "type": "m.room.create", - "sender": "@test:localhost:8480" - }, - { - "age": 2067105053, - "content": { - "avatar_url": "mxc://localhost:8480/tVWZTAIIfqtXMZZtmGCkVjTD#auto", - "displayname": "Bob2", - "membership": "join" - }, - "event_id": "$14440554144URDbf:localhost:8480", - "origin_server_ts": 1444055414834, - "replaces_state": "$14440552472PgiGk:localhost:8480", - "room_id": "!sCDvXTtzjpiPxaqkkt:localhost:8480", - "state_key": "@test:localhost:8480", - "type": "m.room.member", - "sender": "@test:localhost:8480" - } - ] + "end": "t29-57_2_0_2", + "events_after": [ + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" } + ], + "event": { + "event_id": "$f3h4d129462ha:example.com", + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.image" + }, + "events_before": [ + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.file" + } + ], + "start": "t27-54_2_0_2", + "state": [ + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.create" + }, + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.member" + } + ] + } tags: - Room participation diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 941e61fb..f29f1f23 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -114,43 +114,16 @@ paths: "end": "t47409-4357353_219380_26003_2265", "chunk": [ { - "origin_server_ts": 1444812213737, - "sender": "@alice:example.com", - "event_id": "$1444812213350496Caaaa:example.com", - "content": { - "body": "hello world", - "msgtype":"m.text" - }, - "room_id":"!Xq3620DUiqCaoxq:example.com", - "type":"m.room.message", - "age": 1042 + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" }, { - "origin_server_ts": 1444812194656 , - "sender": "@bob:example.com", - "event_id": "$1444812213350496Cbbbb:example.com", - "content": { - "body": "the world is big", - "msgtype":"m.text" - }, - "room_id":"!Xq3620DUiqCaoxq:example.com", - "type":"m.room.message", - "age": 20123 + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.name" }, { - "origin_server_ts": 1444812163990, - "sender": "@bob:example.com", - "event_id": "$1444812213350496Ccccc:example.com", - "content": { - "name": "New room name" - }, - "prev_content": { - "name": "Old room name" - }, - "state_key": "", - "room_id":"!Xq3620DUiqCaoxq:example.com", - "type":"m.room.name", - "age": 50789 + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.video" } ] } diff --git a/api/client-server/notifications.yaml b/api/client-server/notifications.yaml index b450885b..6d9366be 100644 --- a/api/client-server/notifications.yaml +++ b/api/client-server/notifications.yaml @@ -75,16 +75,7 @@ paths: "room_id": "!abcdefg:example.com", "ts": 1475508881945, "event": { - "sender": "@alice:example.com", - "type": "m.room.message", - "age": 124524, - "txn_id": "1234", - "content": { - "body": "I am a fish", - "msgtype": "m.text" - }, - "origin_server_ts": 1417731086797, - "event_id": "$74686972643033:example.com" + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" } } ] diff --git a/api/client-server/old_sync.yaml b/api/client-server/old_sync.yaml index c502c239..c9b7586b 100644 --- a/api/client-server/old_sync.yaml +++ b/api/client-server/old_sync.yaml @@ -64,18 +64,7 @@ paths: "start": "s3456_9_0", "end": "s3457_9_0", "chunk": [ - { - "age": 32, - "content": { - "body": "incoming message", - "msgtype": "m.text" - }, - "event_id": "$14328055551tzaee:localhost", - "origin_server_ts": 1432804485886, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "type": "m.room.message", - "sender": "@bob:localhost" - } + {"$ref": "definitions/event-schemas/examples/m.room.message#m.text"} ] } schema: @@ -142,16 +131,7 @@ paths: application/json: { "end": "s3456_9_0", "presence": [ - { - "content": { - "avatar_url": "mxc://localhost/GCmhgzMPRjqgpODLsNQzVuHZ#auto", - "displayname": "Bob", - "last_active_ago": 31053, - "presence": "online", - "user_id": "@bob:localhost" - }, - "type": "m.presence" - } + {"$ref": "definitions/event-schemas/examples/m.presence"} ], "account_data": [ { @@ -167,28 +147,12 @@ paths: "messages": { "chunk": [ { - "age": 343513403, - "content": { - "body": "foo", - "msgtype": "m.text" - }, - "event_id": "$14328044851tzTJS:localhost", - "origin_server_ts": 1432804485886, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "type": "m.room.message", - "sender": "@alice:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" }, { - "age": 343511809, - "content": { - "body": "bar", - "msgtype": "m.text" - }, - "event_id": "$14328044872spjFg:localhost", - "origin_server_ts": 1432804487480, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "type": "m.room.message", - "sender": "@bob:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.message#m.video" } ], "end": "s3456_9_0", @@ -197,81 +161,20 @@ paths: "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", "state": [ { - "age": 7148266897, - "content": { - "join_rule": "public" - }, - "event_id": "$14259997323TLwtb:localhost", - "origin_server_ts": 1425999732392, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "state_key": "", - "type": "m.room.join_rules", - "sender": "@alice:localhost" - }, - { - "age": 6547561012, - "content": { - "avatar_url": "mxc://localhost/fzysBrHpPEeTGANCVLXWXNMI#auto", - "membership": "join" - }, - "event_id": "$1426600438280zExKY:localhost", - "membership": "join", - "origin_server_ts": 1426600438277, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "state_key": "@alice:localhost", - "type": "m.room.member", - "sender": "@alice:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.join_rules" }, { - "age": 7148267200, - "content": { - "creator": "@alice:localhost" - }, - "event_id": "$14259997320KhbwJ:localhost", - "origin_server_ts": 1425999732089, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "state_key": "", - "type": "m.room.create", - "sender": "@alice:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.member" }, { - "age": 1622568720, - "content": { - "avatar_url": "mxc://localhost/GCmhgzMPRjqgpODLsNQzVuHZ#auto", - "displayname": "Bob", - "membership": "join" - }, - "event_id": "$1431525430134MxlLX:localhost", - "origin_server_ts": 1431525430569, - "replaces_state": "$142652023736BSXcM:localhost", - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "state_key": "@bob:localhost", - "type": "m.room.member", - "sender": "@bob:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.create" }, { - "age": 7148267004, - "content": { - "ban": 50, - "events": { - "m.room.name": 100, - "m.room.power_levels": 100 - }, - "events_default": 0, - "kick": 50, - "redact": 50, - "state_default": 50, - "users": { - "@alice:localhost": 100 - }, - "users_default": 0 - }, - "event_id": "$14259997322mqfaq:localhost", - "origin_server_ts": 1425999732285, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "state_key": "", - "type": "m.room.power_levels", - "sender": "@alice:localhost" + "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", + "$ref": "definitions/event-schemas/examples/m.room.power_levels" } ], "visibility": "private", @@ -423,16 +326,7 @@ paths: 200: description: The full event. examples: - application/json: { - "content": { - "body": "Hello world!", - "msgtype": "m.text" - }, - "room_id": "!wfgy43Sg4a:matrix.org", - "sender": "@bob:matrix.org", - "event_id": "$asfDuShaf7Gafaw:matrix.org", - "type": "m.room.message" - } + application/json: {"$ref": "definitions/event-schemas/examples/m.room.message#m.text"} schema: allOf: - "$ref": "definitions/event-schemas/schema/core-event-schema/event.yaml" diff --git a/api/client-server/peeking_events.yaml b/api/client-server/peeking_events.yaml index 2f66bae7..e3dc5777 100644 --- a/api/client-server/peeking_events.yaml +++ b/api/client-server/peeking_events.yaml @@ -75,16 +75,8 @@ paths: "end": "s3457_9_0", "chunk": [ { - "age": 32, - "content": { - "body": "incoming message", - "msgtype": "m.text" - }, - "event_id": "$14328055551tzaee:localhost", - "origin_server_ts": 1432804485886, - "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "type": "m.room.message", - "sender": "@bob:localhost" + "room_id": "!somewhere:over.the.rainbow", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" } ] } diff --git a/api/client-server/room_initial_sync.yaml b/api/client-server/room_initial_sync.yaml index c27f0f24..2a354fc0 100644 --- a/api/client-server/room_initial_sync.yaml +++ b/api/client-server/room_initial_sync.yaml @@ -43,28 +43,12 @@ paths: "messages": { "chunk": [ { - "age": 343513403, - "content": { - "body": "foo", - "msgtype": "m.text" - }, - "event_id": "$14328044851tzTJS:example.com", - "origin_server_ts": 1432804485886, "room_id": "!636q39766251:example.com", - "type": "m.room.message", - "sender": "@alice:example.com" + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" }, { - "age": 343511809, - "content": { - "body": "bar", - "msgtype": "m.text" - }, - "event_id": "$14328044872spjFg:example.com", - "origin_server_ts": 1432804487480, "room_id": "!636q39766251:example.com", - "type": "m.room.message", - "sender": "@bob:example.com" + "$ref": "definitions/event-schemas/examples/m.room.message#m.file" } ], "end": "s3456_9_0", @@ -73,81 +57,20 @@ paths: "room_id": "!636q39766251:example.com", "state": [ { - "age": 7148266897, - "content": { - "join_rule": "public" - }, - "event_id": "$14259997323TLwtb:example.com", - "origin_server_ts": 1425999732392, - "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.join_rules", - "sender": "@alice:example.com" - }, - { - "age": 6547561012, - "content": { - "avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto", - "membership": "join" - }, - "event_id": "$1426600438280zExKY:example.com", - "membership": "join", - "origin_server_ts": 1426600438277, "room_id": "!636q39766251:example.com", - "state_key": "@alice:example.com", - "type": "m.room.member", - "sender": "@alice:example.com" + "$ref": "definitions/event-schemas/examples/m.room.join_rules" }, { - "age": 7148267200, - "content": { - "creator": "@alice:example.com" - }, - "event_id": "$14259997320KhbwJ:example.com", - "origin_server_ts": 1425999732089, "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.create", - "sender": "@alice:example.com" + "$ref": "definitions/event-schemas/examples/m.room.member" }, { - "age": 1622568720, - "content": { - "avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto", - "displayname": "Bob", - "membership": "join" - }, - "event_id": "$1431525430134MxlLX:example.com", - "origin_server_ts": 1431525430569, - "replaces_state": "$142652023736BSXcM:example.com", "room_id": "!636q39766251:example.com", - "state_key": "@bob:example.com", - "type": "m.room.member", - "sender": "@bob:example.com" + "$ref": "definitions/event-schemas/examples/m.room.create" }, { - "age": 7148267004, - "content": { - "ban": 50, - "events": { - "m.room.name": 100, - "m.room.power_levels": 100 - }, - "events_default": 0, - "kick": 50, - "redact": 50, - "state_default": 50, - "users": { - "@alice:example.com": 100 - }, - "users_default": 0 - }, - "event_id": "$14259997322mqfaq:example.com", - "origin_server_ts": 1425999732285, "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.power_levels", - "sender": "@alice:example.com" + "$ref": "definitions/event-schemas/examples/m.room.power_levels" } ], "visibility": "private", diff --git a/api/client-server/rooms.yaml b/api/client-server/rooms.yaml index cc1f2bf7..55456436 100644 --- a/api/client-server/rooms.yaml +++ b/api/client-server/rooms.yaml @@ -42,7 +42,7 @@ paths: name: roomId description: The ID of the room the event is in. required: true - x-example: "!asfDuShaf7Gafaw:matrix.org" + x-example: "!636q39766251:matrix.org" - in: path type: string name: eventId @@ -54,15 +54,9 @@ paths: description: The full event. examples: application/json: { - "content": { - "body": "Hello world!", - "msgtype": "m.text" - }, - "room_id": "!wfgy43Sg4a:matrix.org", - "sender": "@bob:matrix.org", - "event_id": "$asfDuShaf7Gafaw:matrix.org", - "type": "m.room.message" - } + "room_id": "!636q39766251:matrix.org", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + } schema: allOf: - "$ref": "definitions/event-schemas/schema/core-event-schema/event.yaml" @@ -178,84 +172,23 @@ paths: description: The current state of the room examples: application/json: [ - { - "age": 7148266897, - "content": { - "join_rule": "public" - }, - "event_id": "$14259997323TLwtb:example.com", - "origin_server_ts": 1425999732392, - "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.join_rules", - "sender": "@alice:example.com" - }, - { - "age": 6547561012, - "content": { - "avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto", - "membership": "join" - }, - "event_id": "$1426600438280zExKY:example.com", - "membership": "join", - "origin_server_ts": 1426600438277, - "room_id": "!636q39766251:example.com", - "state_key": "@alice:example.com", - "type": "m.room.member", - "sender": "@alice:example.com" - }, - { - "age": 7148267200, - "content": { - "creator": "@alice:example.com" - }, - "event_id": "$14259997320KhbwJ:example.com", - "origin_server_ts": 1425999732089, - "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.create", - "sender": "@alice:example.com" - }, - { - "age": 1622568720, - "content": { - "avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto", - "displayname": "Bob", - "membership": "join" - }, - "event_id": "$1431525430134MxlLX:example.com", - "origin_server_ts": 1431525430569, - "replaces_state": "$142652023736BSXcM:example.com", - "room_id": "!636q39766251:example.com", - "state_key": "@bob:example.com", - "type": "m.room.member", - "sender": "@bob:example.com" - }, - { - "age": 7148267004, - "content": { - "ban": 50, - "events": { - "m.room.name": 100, - "m.room.power_levels": 100 - }, - "events_default": 0, - "kick": 50, - "redact": 50, - "state_default": 50, - "users": { - "@alice:example.com": 100 - }, - "users_default": 0 - }, - "event_id": "$14259997322mqfaq:example.com", - "origin_server_ts": 1425999732285, - "room_id": "!636q39766251:example.com", - "state_key": "", - "type": "m.room.power_levels", - "sender": "@alice:example.com" - } - ] + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.join_rules" + }, + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.member" + }, + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.create" + }, + { + "room_id": "!636q39766251:example.com", + "$ref": "definitions/event-schemas/examples/m.room.power_levels" + } + ] schema: type: array title: RoomState @@ -300,33 +233,8 @@ paths: application/json: { "chunk": [ { - "age": 6547561012, - "content": { - "avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto", - "membership": "join" - }, - "event_id": "$1426600438280zExKY:example.com", - "membership": "join", - "origin_server_ts": 1426600438277, - "room_id": "!636q39766251:example.com", - "state_key": "@alice:example.com", - "type": "m.room.member", - "sender": "@alice:example.com" - }, - { - "age": 1622568720, - "content": { - "avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto", - "displayname": "Bob", - "membership": "join" - }, - "event_id": "$1431525430134MxlLX:example.com", - "origin_server_ts": 1431525430569, - "replaces_state": "$142652023736BSXcM:example.com", "room_id": "!636q39766251:example.com", - "state_key": "@bob:example.com", - "type": "m.room.member", - "sender": "@bob:example.com" + "$ref": "definitions/event-schemas/examples/m.room.member" } ] } diff --git a/api/client-server/search.yaml b/api/client-server/search.yaml index 4a5f4515..9d5d2074 100644 --- a/api/client-server/search.yaml +++ b/api/client-server/search.yaml @@ -280,7 +280,7 @@ paths: Any groups that were requested. The outer ``string`` key is the group key requested (eg: ``room_id`` - or ``sender``). The inner ``string`` key is the grouped value (eg: + or ``sender``). The inner ``string`` key is the grouped value (eg: a room's ID or a user's ID). additionalProperties: type: object @@ -347,16 +347,9 @@ paths: { "rank": 0.00424866, "result": { - "age": 526228296, - "content": { - "body": "Test content martians and men", - "msgtype": "m.text" - }, - "event_id": "$144429830826TWwbB:localhost", - "origin_server_ts": 1444298308034, "room_id": "!qPewotXpIctQySfjSy:localhost", - "type": "m.room.message", - "sender": "@test:localhost" + "event_id": "$144429830826TWwbB:localhost", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" } } ] diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index f204152a..ad5a6075 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -310,11 +310,7 @@ paths: "next_batch": "s72595_4483_1934", "presence": { "events": [ - { - "sender": "@alice:example.com", - "type": "m.presence", - "content": {"presence": "online"} - } + {"$ref": "definitions/event-schemas/examples/m.presence"} ] }, "account_data": { @@ -333,36 +329,20 @@ paths: "state": { "events": [ { - "sender": "@alice:example.com", - "type": "m.room.member", - "state_key": "@alice:example.com", - "content": {"membership": "join"}, - "origin_server_ts": 1417731086795, - "event_id": "$66697273743031:example.com" + "room_id": "!726s6s6q:example.com", + "$ref": "definitions/event-schemas/examples/m.room.member" } ] }, "timeline": { "events": [ { - "sender": "@bob:example.com", - "type": "m.room.member", - "state_key": "@bob:example.com", - "content": {"membership": "join"}, - "prev_content": {"membership": "invite"}, - "origin_server_ts": 1417731086795, - "event_id": "$7365636s6r6432:example.com" + "room_id": "!726s6s6q:example.com", + "$ref": "definitions/event-schemas/examples/m.room.member" }, { - "sender": "@alice:example.com", - "type": "m.room.message", - "txn_id": "1234", - "content": { - "body": "I am a fish", - "msgtype": "m.text" - }, - "origin_server_ts": 1417731086797, - "event_id": "$74686972643033:example.com" + "room_id": "!726s6s6q:example.com", + "$ref": "definitions/event-schemas/examples/m.room.message#m.text" } ], "limited": true, @@ -370,18 +350,12 @@ paths: }, "ephemeral": { "events": [ - { - "type": "m.typing", - "content": {"user_ids": ["@alice:example.com"]} - } + {"$ref": "definitions/event-schemas/examples/m.typing"} ] }, "account_data": { "events": [ - { - "type": "m.tag", - "content": {"tags": {"work": {"order": 1}}} - }, + {"$ref": "definitions/event-schemas/examples/m.tag"}, { "type": "org.example.custom.room.config", "content": { diff --git a/changelogs/client_server/newsfragments/2050.clarification b/changelogs/client_server/newsfragments/2050.clarification new file mode 100644 index 00000000..384daa11 --- /dev/null +++ b/changelogs/client_server/newsfragments/2050.clarification @@ -0,0 +1 @@ +Use consistent examples of events throughout the specification. From d6d74c4cbef04e194826cf3565b21bf5959a7d31 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 15:23:45 -0600 Subject: [PATCH 135/282] Switch to using $ instead of # for sub-types # is reserved by the swagger validator as a way to include partial content from a JSON object (eg: "#/path" would include {"test": true} from the object {"path":{"test":true}}). Instead of trying to convince the validator that it is wrong, we'll just use a different character. Note that our rendering tools do not care about #-style references to objects. It's still somewhat worth changing the character though. --- api/application-service/transactions.yaml | 2 +- api/client-server/event_context.yaml | 6 +++--- api/client-server/message_pagination.yaml | 4 ++-- api/client-server/notifications.yaml | 2 +- api/client-server/old_sync.yaml | 8 ++++---- api/client-server/peeking_events.yaml | 2 +- api/client-server/room_initial_sync.yaml | 4 ++-- api/client-server/rooms.yaml | 2 +- api/client-server/search.yaml | 2 +- api/client-server/sync.yaml | 2 +- event-schemas/check_examples.py | 4 ++-- ....encrypted#megolm => m.room.encrypted$megolm} | 0 ...m.room.encrypted#olm => m.room.encrypted$olm} | 0 ...oom_state => m.room.member$invite_room_state} | 0 ...y_invite => m.room.member$third_party_invite} | 0 ...om.message#m.audio => m.room.message$m.audio} | 2 +- ...om.message#m.emote => m.room.message$m.emote} | 0 ...room.message#m.file => m.room.message$m.file} | 0 ...om.message#m.image => m.room.message$m.image} | 0 ...sage#m.location => m.room.message$m.location} | 0 ....message#m.notice => m.room.message$m.notice} | 0 ...ver_notice => m.room.message$m.server_notice} | 0 ...room.message#m.text => m.room.message$m.text} | 0 ...om.message#m.video => m.room.message$m.video} | 0 ...request => m.room_key_request$cancel_request} | 0 ...equest#request => m.room_key_request$request} | 0 ...om.message#m.audio => m.room.message$m.audio} | 0 ...om.message#m.emote => m.room.message$m.emote} | 0 ...room.message#m.file => m.room.message$m.file} | 0 ...om.message#m.image => m.room.message$m.image} | 0 ...sage#m.location => m.room.message$m.location} | 0 ....message#m.notice => m.room.message$m.notice} | 0 ...ver_notice => m.room.message$m.server_notice} | 0 ...room.message#m.text => m.room.message$m.text} | 0 ...om.message#m.video => m.room.message$m.video} | 0 scripts/templating/matrix_templates/sections.py | 16 ++++++++-------- scripts/templating/matrix_templates/units.py | 2 +- 37 files changed, 29 insertions(+), 29 deletions(-) rename event-schemas/examples/{m.room.encrypted#megolm => m.room.encrypted$megolm} (100%) rename event-schemas/examples/{m.room.encrypted#olm => m.room.encrypted$olm} (100%) rename event-schemas/examples/{m.room.member#invite_room_state => m.room.member$invite_room_state} (100%) rename event-schemas/examples/{m.room.member#third_party_invite => m.room.member$third_party_invite} (100%) rename event-schemas/examples/{m.room.message#m.audio => m.room.message$m.audio} (99%) rename event-schemas/examples/{m.room.message#m.emote => m.room.message$m.emote} (100%) rename event-schemas/examples/{m.room.message#m.file => m.room.message$m.file} (100%) rename event-schemas/examples/{m.room.message#m.image => m.room.message$m.image} (100%) rename event-schemas/examples/{m.room.message#m.location => m.room.message$m.location} (100%) rename event-schemas/examples/{m.room.message#m.notice => m.room.message$m.notice} (100%) rename event-schemas/examples/{m.room.message#m.server_notice => m.room.message$m.server_notice} (100%) rename event-schemas/examples/{m.room.message#m.text => m.room.message$m.text} (100%) rename event-schemas/examples/{m.room.message#m.video => m.room.message$m.video} (100%) rename event-schemas/examples/{m.room_key_request#cancel_request => m.room_key_request$cancel_request} (100%) rename event-schemas/examples/{m.room_key_request#request => m.room_key_request$request} (100%) rename event-schemas/schema/{m.room.message#m.audio => m.room.message$m.audio} (100%) rename event-schemas/schema/{m.room.message#m.emote => m.room.message$m.emote} (100%) rename event-schemas/schema/{m.room.message#m.file => m.room.message$m.file} (100%) rename event-schemas/schema/{m.room.message#m.image => m.room.message$m.image} (100%) rename event-schemas/schema/{m.room.message#m.location => m.room.message$m.location} (100%) rename event-schemas/schema/{m.room.message#m.notice => m.room.message$m.notice} (100%) rename event-schemas/schema/{m.room.message#m.server_notice => m.room.message$m.server_notice} (100%) rename event-schemas/schema/{m.room.message#m.text => m.room.message$m.text} (100%) rename event-schemas/schema/{m.room.message#m.video => m.room.message$m.video} (100%) diff --git a/api/application-service/transactions.yaml b/api/application-service/transactions.yaml index 98181196..09f15276 100644 --- a/api/application-service/transactions.yaml +++ b/api/application-service/transactions.yaml @@ -56,7 +56,7 @@ paths: example: { "events": [ {"$ref": "../../event-schemas/examples/m.room.member"}, - {"$ref": "../../event-schemas/examples/m.room.message#m.text"} + {"$ref": "../../event-schemas/examples/m.room.message$m.text"} ] } description: Transaction information diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 549902a0..e2018028 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -105,18 +105,18 @@ paths: "events_after": [ { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } ], "event": { "event_id": "$f3h4d129462ha:example.com", "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.image" + "$ref": "definitions/event-schemas/examples/m.room.message$m.image" }, "events_before": [ { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.file" + "$ref": "definitions/event-schemas/examples/m.room.message$m.file" } ], "start": "t27-54_2_0_2", diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index f29f1f23..c9f9d0ae 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -115,7 +115,7 @@ paths: "chunk": [ { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" }, { "room_id": "!636q39766251:example.com", @@ -123,7 +123,7 @@ paths: }, { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.video" + "$ref": "definitions/event-schemas/examples/m.room.message$m.video" } ] } diff --git a/api/client-server/notifications.yaml b/api/client-server/notifications.yaml index 6d9366be..87341d41 100644 --- a/api/client-server/notifications.yaml +++ b/api/client-server/notifications.yaml @@ -75,7 +75,7 @@ paths: "room_id": "!abcdefg:example.com", "ts": 1475508881945, "event": { - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } } ] diff --git a/api/client-server/old_sync.yaml b/api/client-server/old_sync.yaml index c9b7586b..a79c3b32 100644 --- a/api/client-server/old_sync.yaml +++ b/api/client-server/old_sync.yaml @@ -64,7 +64,7 @@ paths: "start": "s3456_9_0", "end": "s3457_9_0", "chunk": [ - {"$ref": "definitions/event-schemas/examples/m.room.message#m.text"} + {"$ref": "definitions/event-schemas/examples/m.room.message$m.text"} ] } schema: @@ -148,11 +148,11 @@ paths: "chunk": [ { "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" }, { "room_id": "!TmaZBKYIFrIPVGoUYp:localhost", - "$ref": "definitions/event-schemas/examples/m.room.message#m.video" + "$ref": "definitions/event-schemas/examples/m.room.message$m.video" } ], "end": "s3456_9_0", @@ -326,7 +326,7 @@ paths: 200: description: The full event. examples: - application/json: {"$ref": "definitions/event-schemas/examples/m.room.message#m.text"} + application/json: {"$ref": "definitions/event-schemas/examples/m.room.message$m.text"} schema: allOf: - "$ref": "definitions/event-schemas/schema/core-event-schema/event.yaml" diff --git a/api/client-server/peeking_events.yaml b/api/client-server/peeking_events.yaml index e3dc5777..feac36f4 100644 --- a/api/client-server/peeking_events.yaml +++ b/api/client-server/peeking_events.yaml @@ -76,7 +76,7 @@ paths: "chunk": [ { "room_id": "!somewhere:over.the.rainbow", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } ] } diff --git a/api/client-server/room_initial_sync.yaml b/api/client-server/room_initial_sync.yaml index 2a354fc0..72e56ba9 100644 --- a/api/client-server/room_initial_sync.yaml +++ b/api/client-server/room_initial_sync.yaml @@ -44,11 +44,11 @@ paths: "chunk": [ { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" }, { "room_id": "!636q39766251:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.file" + "$ref": "definitions/event-schemas/examples/m.room.message$m.file" } ], "end": "s3456_9_0", diff --git a/api/client-server/rooms.yaml b/api/client-server/rooms.yaml index 55456436..377783c6 100644 --- a/api/client-server/rooms.yaml +++ b/api/client-server/rooms.yaml @@ -55,7 +55,7 @@ paths: examples: application/json: { "room_id": "!636q39766251:matrix.org", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } schema: allOf: diff --git a/api/client-server/search.yaml b/api/client-server/search.yaml index 9d5d2074..4fe72d5b 100644 --- a/api/client-server/search.yaml +++ b/api/client-server/search.yaml @@ -349,7 +349,7 @@ paths: "result": { "room_id": "!qPewotXpIctQySfjSy:localhost", "event_id": "$144429830826TWwbB:localhost", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } } ] diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index ad5a6075..02fddb84 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -342,7 +342,7 @@ paths: }, { "room_id": "!726s6s6q:example.com", - "$ref": "definitions/event-schemas/examples/m.room.message#m.text" + "$ref": "definitions/event-schemas/examples/m.room.message$m.text" } ], "limited": true, diff --git a/event-schemas/check_examples.py b/event-schemas/check_examples.py index 3e536ec3..2baa3ef0 100755 --- a/event-schemas/check_examples.py +++ b/event-schemas/check_examples.py @@ -112,8 +112,8 @@ def check_example_dir(exampledir, schemadir): continue examplepath = os.path.join(root, filename) schemapath = examplepath.replace(exampledir, schemadir) - if schemapath.find("#") >= 0: - schemapath = schemapath[:schemapath.find("#")] + if schemapath.find("$") >= 0: + schemapath = schemapath[:schemapath.find("$")] try: check_example_file(examplepath, schemapath) except Exception as e: diff --git a/event-schemas/examples/m.room.encrypted#megolm b/event-schemas/examples/m.room.encrypted$megolm similarity index 100% rename from event-schemas/examples/m.room.encrypted#megolm rename to event-schemas/examples/m.room.encrypted$megolm diff --git a/event-schemas/examples/m.room.encrypted#olm b/event-schemas/examples/m.room.encrypted$olm similarity index 100% rename from event-schemas/examples/m.room.encrypted#olm rename to event-schemas/examples/m.room.encrypted$olm diff --git a/event-schemas/examples/m.room.member#invite_room_state b/event-schemas/examples/m.room.member$invite_room_state similarity index 100% rename from event-schemas/examples/m.room.member#invite_room_state rename to event-schemas/examples/m.room.member$invite_room_state diff --git a/event-schemas/examples/m.room.member#third_party_invite b/event-schemas/examples/m.room.member$third_party_invite similarity index 100% rename from event-schemas/examples/m.room.member#third_party_invite rename to event-schemas/examples/m.room.member$third_party_invite diff --git a/event-schemas/examples/m.room.message#m.audio b/event-schemas/examples/m.room.message$m.audio similarity index 99% rename from event-schemas/examples/m.room.message#m.audio rename to event-schemas/examples/m.room.message$m.audio index 2f743d49..58e874e0 100644 --- a/event-schemas/examples/m.room.message#m.audio +++ b/event-schemas/examples/m.room.message$m.audio @@ -11,4 +11,4 @@ }, "msgtype": "m.audio" } -} +} diff --git a/event-schemas/examples/m.room.message#m.emote b/event-schemas/examples/m.room.message$m.emote similarity index 100% rename from event-schemas/examples/m.room.message#m.emote rename to event-schemas/examples/m.room.message$m.emote diff --git a/event-schemas/examples/m.room.message#m.file b/event-schemas/examples/m.room.message$m.file similarity index 100% rename from event-schemas/examples/m.room.message#m.file rename to event-schemas/examples/m.room.message$m.file diff --git a/event-schemas/examples/m.room.message#m.image b/event-schemas/examples/m.room.message$m.image similarity index 100% rename from event-schemas/examples/m.room.message#m.image rename to event-schemas/examples/m.room.message$m.image diff --git a/event-schemas/examples/m.room.message#m.location b/event-schemas/examples/m.room.message$m.location similarity index 100% rename from event-schemas/examples/m.room.message#m.location rename to event-schemas/examples/m.room.message$m.location diff --git a/event-schemas/examples/m.room.message#m.notice b/event-schemas/examples/m.room.message$m.notice similarity index 100% rename from event-schemas/examples/m.room.message#m.notice rename to event-schemas/examples/m.room.message$m.notice diff --git a/event-schemas/examples/m.room.message#m.server_notice b/event-schemas/examples/m.room.message$m.server_notice similarity index 100% rename from event-schemas/examples/m.room.message#m.server_notice rename to event-schemas/examples/m.room.message$m.server_notice diff --git a/event-schemas/examples/m.room.message#m.text b/event-schemas/examples/m.room.message$m.text similarity index 100% rename from event-schemas/examples/m.room.message#m.text rename to event-schemas/examples/m.room.message$m.text diff --git a/event-schemas/examples/m.room.message#m.video b/event-schemas/examples/m.room.message$m.video similarity index 100% rename from event-schemas/examples/m.room.message#m.video rename to event-schemas/examples/m.room.message$m.video diff --git a/event-schemas/examples/m.room_key_request#cancel_request b/event-schemas/examples/m.room_key_request$cancel_request similarity index 100% rename from event-schemas/examples/m.room_key_request#cancel_request rename to event-schemas/examples/m.room_key_request$cancel_request diff --git a/event-schemas/examples/m.room_key_request#request b/event-schemas/examples/m.room_key_request$request similarity index 100% rename from event-schemas/examples/m.room_key_request#request rename to event-schemas/examples/m.room_key_request$request diff --git a/event-schemas/schema/m.room.message#m.audio b/event-schemas/schema/m.room.message$m.audio similarity index 100% rename from event-schemas/schema/m.room.message#m.audio rename to event-schemas/schema/m.room.message$m.audio diff --git a/event-schemas/schema/m.room.message#m.emote b/event-schemas/schema/m.room.message$m.emote similarity index 100% rename from event-schemas/schema/m.room.message#m.emote rename to event-schemas/schema/m.room.message$m.emote diff --git a/event-schemas/schema/m.room.message#m.file b/event-schemas/schema/m.room.message$m.file similarity index 100% rename from event-schemas/schema/m.room.message#m.file rename to event-schemas/schema/m.room.message$m.file diff --git a/event-schemas/schema/m.room.message#m.image b/event-schemas/schema/m.room.message$m.image similarity index 100% rename from event-schemas/schema/m.room.message#m.image rename to event-schemas/schema/m.room.message$m.image diff --git a/event-schemas/schema/m.room.message#m.location b/event-schemas/schema/m.room.message$m.location similarity index 100% rename from event-schemas/schema/m.room.message#m.location rename to event-schemas/schema/m.room.message$m.location diff --git a/event-schemas/schema/m.room.message#m.notice b/event-schemas/schema/m.room.message$m.notice similarity index 100% rename from event-schemas/schema/m.room.message#m.notice rename to event-schemas/schema/m.room.message$m.notice diff --git a/event-schemas/schema/m.room.message#m.server_notice b/event-schemas/schema/m.room.message$m.server_notice similarity index 100% rename from event-schemas/schema/m.room.message#m.server_notice rename to event-schemas/schema/m.room.message$m.server_notice diff --git a/event-schemas/schema/m.room.message#m.text b/event-schemas/schema/m.room.message$m.text similarity index 100% rename from event-schemas/schema/m.room.message#m.text rename to event-schemas/schema/m.room.message$m.text diff --git a/event-schemas/schema/m.room.message#m.video b/event-schemas/schema/m.room.message$m.video similarity index 100% rename from event-schemas/schema/m.room.message#m.video rename to event-schemas/schema/m.room.message$m.video diff --git a/scripts/templating/matrix_templates/sections.py b/scripts/templating/matrix_templates/sections.py index 92afa5ff..c88959ed 100644 --- a/scripts/templating/matrix_templates/sections.py +++ b/scripts/templating/matrix_templates/sections.py @@ -115,7 +115,7 @@ class MatrixSections(Sections): schemas = self.units.get("event_schemas") renders = {} for event_type in schemas: - underscored_event_type = event_type.replace(".", "_").replace("#", "_") + underscored_event_type = event_type.replace(".", "_").replace("$", "_") renders[underscored_event_type + "_event"] = self._render_events( lambda x: x == event_type, sorted ) @@ -125,7 +125,7 @@ class MatrixSections(Sections): def filterFn(eventType): return ( eventType.startswith("m.room") and - not eventType.startswith("m.room.message#m.") + not eventType.startswith("m.room.message$m.") ) return self._render_events(filterFn, sorted) @@ -138,22 +138,22 @@ class MatrixSections(Sections): ]["subtitle"] sections = [] msgtype_order = [ - "m.room.message#m.text", "m.room.message#m.emote", - "m.room.message#m.notice", "m.room.message#m.image", - "m.room.message#m.file" + "m.room.message$m.text", "m.room.message$m.emote", + "m.room.message$m.notice", "m.room.message$m.image", + "m.room.message$m.file" ] excluded_types = [ # We exclude server notices from here because we handle them in a # dedicated module. We do not want to confuse developers this early # in the spec. - "m.room.message#m.server_notice", + "m.room.message$m.server_notice", ] other_msgtypes = [ - k for k in schemas.keys() if k.startswith("m.room.message#") and + k for k in schemas.keys() if k.startswith("m.room.message$") and k not in msgtype_order and k not in excluded_types ] for event_name in (msgtype_order + other_msgtypes): - if not event_name.startswith("m.room.message#m."): + if not event_name.startswith("m.room.message$m."): continue sections.append(template.render( example=examples[event_name][0], diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index d4745e4d..ddb65efe 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -795,7 +795,7 @@ class MatrixUnits(Units): if not filename.startswith("m."): continue - event_name = filename.split("#")[0] + event_name = filename.split("$")[0] filepath = os.path.join(path, filename) logger.info("Reading event example: %s" % filepath) try: From f9472bae411e8db020247eb7394d7bd42fe50d5a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 16:27:16 -0600 Subject: [PATCH 136/282] Change reference for definition of push rule condition kinds Fixes https://github.com/matrix-org/matrix-doc/issues/1970 --- api/client-server/definitions/push_condition.yaml | 12 ++++++++---- .../client_server/newsfragments/2052.clarification | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2052.clarification diff --git a/api/client-server/definitions/push_condition.yaml b/api/client-server/definitions/push_condition.yaml index 796a51f4..8752274e 100644 --- a/api/client-server/definitions/push_condition.yaml +++ b/api/client-server/definitions/push_condition.yaml @@ -16,16 +16,20 @@ title: PushCondition type: object properties: kind: - enum: - - event_match - - contains_display_name - - room_member_count type: string + description: |- + The kind of condition to apply. See `conditions <#conditions>`_ for + more information on the allowed kinds and how they work. key: type: string description: |- Required for ``event_match`` conditions. The dot-separated field of the event to match. + + Required for ``sender_notification_permission`` conditions. The field in + the power level event the user needs a minimum power level for. Fields + must be specified under the ``notifications`` property in the power level + event's ``content``. x-example: content.body pattern: type: string diff --git a/changelogs/client_server/newsfragments/2052.clarification b/changelogs/client_server/newsfragments/2052.clarification new file mode 100644 index 00000000..95bdc928 --- /dev/null +++ b/changelogs/client_server/newsfragments/2052.clarification @@ -0,0 +1 @@ +Clarify which push rule condition kinds exist. From 792bb8faa49c4551067fd824e805ba4f5c195509 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 16:28:36 -0600 Subject: [PATCH 137/282] Renumber changelog to match PR --- .../newsfragments/{2050.clarification => 2051.clarification} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/client_server/newsfragments/{2050.clarification => 2051.clarification} (100%) diff --git a/changelogs/client_server/newsfragments/2050.clarification b/changelogs/client_server/newsfragments/2051.clarification similarity index 100% rename from changelogs/client_server/newsfragments/2050.clarification rename to changelogs/client_server/newsfragments/2051.clarification From 464845feb0695c99f1b81e665e980e0289ec119c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 16:36:43 -0600 Subject: [PATCH 138/282] Make url required for m.file-like messages Fixes https://github.com/matrix-org/matrix-doc/issues/2008 This also removes `filename` from `m.file` because it has never been used in practice. --- changelogs/client_server/newsfragments/2053.clarification | 1 + event-schemas/schema/m.room.message#m.audio | 1 + event-schemas/schema/m.room.message#m.file | 2 +- event-schemas/schema/m.room.message#m.image | 1 + event-schemas/schema/m.room.message#m.video | 1 + 5 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2053.clarification diff --git a/changelogs/client_server/newsfragments/2053.clarification b/changelogs/client_server/newsfragments/2053.clarification new file mode 100644 index 00000000..2a72a88e --- /dev/null +++ b/changelogs/client_server/newsfragments/2053.clarification @@ -0,0 +1 @@ +Clarify the required fields on ``m.file`` (and similar) messages. diff --git a/event-schemas/schema/m.room.message#m.audio b/event-schemas/schema/m.room.message#m.audio index c258b85f..99e28110 100644 --- a/event-schemas/schema/m.room.message#m.audio +++ b/event-schemas/schema/m.room.message#m.audio @@ -38,6 +38,7 @@ properties: required: - msgtype - body + - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message#m.file b/event-schemas/schema/m.room.message#m.file index 2fb4fe50..2389d8a9 100644 --- a/event-schemas/schema/m.room.message#m.file +++ b/event-schemas/schema/m.room.message#m.file @@ -53,7 +53,7 @@ properties: required: - msgtype - body - - filename + - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message#m.image b/event-schemas/schema/m.room.message#m.image index 349f78f4..1e6ebeaa 100644 --- a/event-schemas/schema/m.room.message#m.image +++ b/event-schemas/schema/m.room.message#m.image @@ -28,6 +28,7 @@ properties: required: - msgtype - body + - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message#m.video b/event-schemas/schema/m.room.message#m.video index 8a66fdeb..2da7e0bc 100644 --- a/event-schemas/schema/m.room.message#m.video +++ b/event-schemas/schema/m.room.message#m.video @@ -59,6 +59,7 @@ properties: required: - msgtype - body + - url type: object type: enum: From 0f623113f10cfc50d99305a8d61ce02371636f5c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 18:35:18 -0600 Subject: [PATCH 139/282] Clarify that UIA stages cannot be attempted twice Fixes https://github.com/matrix-org/matrix-doc/issues/1987 Note: Synapse currently does not care, however the spirit of the text in the spec implies that completed == done forever, so we're just reinforcing it here. --- .../client_server/newsfragments/2054.clarification | 1 + specification/client_server_api.rst | 11 ++++++----- 2 files changed, 7 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2054.clarification diff --git a/changelogs/client_server/newsfragments/2054.clarification b/changelogs/client_server/newsfragments/2054.clarification new file mode 100644 index 00000000..e43aea2d --- /dev/null +++ b/changelogs/client_server/newsfragments/2054.clarification @@ -0,0 +1 @@ +Clarify that User-Interactive Authentication stages cannot be attempted more than once. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 6c2e364a..318ac08d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -453,11 +453,10 @@ params presented, that type may be present as a key in this dictionary. For example, the public part of an OAuth client ID could be given here. session - This is a session identifier that the client must pass back to the home - server, if one is provided, in subsequent attempts to authenticate in the same - API call. + This is a session identifier that the client must pass back to the homeserver, + if one is provided, in subsequent attempts to authenticate in the same API call. -The client then chooses a flow and attempts to complete one of the stages. It +The client then chooses a flow and attempts to complete the first stage. It does this by resubmitting the same request with the addition of an ``auth`` key in the object that it submits. This dictionary contains a ``type`` key whose value is the name of the authentication type that the client is attempting to complete. @@ -558,7 +557,9 @@ message in the standard format. For example: } If the client has completed all stages of a flow, the homeserver performs the -API call and returns the result as normal. +API call and returns the result as normal. Completed stages cannot be re-tried; +The client must abandon the current session and start over. Homeservers should +treat retries as authentication errors. Some authentication types may be completed by means other than through the Matrix client, for example, an email confirmation may be completed when the user From 2ed37f5bf464d03a322184221a89f6ebe4c61630 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 21:00:37 -0600 Subject: [PATCH 140/282] Clarify guest accounts and `auth` usage on /register Fixes https://github.com/matrix-org/matrix-doc/issues/1980 Fixes https://github.com/matrix-org/matrix-doc/issues/1984 --- api/client-server/registration.yaml | 34 ++++++++++++++----- .../newsfragments/2055.clarification | 1 + 2 files changed, 26 insertions(+), 9 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2055.clarification diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..10b661a3 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -29,7 +29,8 @@ paths: post: summary: Register for an account on this homeserver. description: |- - This API endpoint uses the `User-Interactive Authentication API`_. + This API endpoint uses the `User-Interactive Authentication API`_, except in + the cases where a guest account is being registered. Register for an account on this homeserver. @@ -59,6 +60,11 @@ paths: supplied by the client or generated by the server. The server may invalidate any access token previously associated with that device. See `Relationship between access tokens and devices`_. + + When registering a guest account, all parameters in the request body + with the exception of ``initial_device_display_name`` are ignored by + the server. The server will pick a ``device_id`` for the account + regardless of input. operationId: register parameters: - in: query @@ -72,7 +78,7 @@ paths: enum: - guest - user - description: The kind of account to register. Defaults to `user`. + description: The kind of account to register. Defaults to ``user``. - in: body name: body schema: @@ -80,13 +86,11 @@ paths: properties: auth: description: |- - Additional authentication information for the - user-interactive authentication API. Note that this - information is *not* used to define how the registered user - should be authenticated, but is instead used to - authenticate the ``register`` call itself. It should be - left empty, or omitted, unless an earlier call returned an - response with status code 401. + Additional authentication information for the + user-interactive authentication API. Note that this + information is *not* used to define how the registered user + should be authenticated, but is instead used to + authenticate the ``register`` call itself. "$ref": "definitions/auth_data.yaml" bind_email: type: boolean @@ -194,6 +198,18 @@ paths: The homeserver requires additional authentication information. schema: "$ref": "definitions/auth_response.yaml" + 403: + description: |- + The homeserver does not permit registering the account. This response + can be used to identify that a particular ``kind`` of account is not + allowed, or that registration is generally not supported by the homeserver. + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "Registration is disabled" + } + schema: + "$ref": "definitions/errors/error.yaml" 429: description: This request was rate-limited. schema: diff --git a/changelogs/client_server/newsfragments/2055.clarification b/changelogs/client_server/newsfragments/2055.clarification new file mode 100644 index 00000000..3a57ef7e --- /dev/null +++ b/changelogs/client_server/newsfragments/2055.clarification @@ -0,0 +1 @@ +Clarify which parameters apply in what scenarios on ``/register``. From f3c0c5232fb812d90ceeda1348c1fc2d17558e28 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 21:44:50 -0600 Subject: [PATCH 141/282] Add a table to show how changes in membership should be interpreted Fixes https://github.com/matrix-org/matrix-doc/issues/876 --- .../newsfragments/2056.clarification | 1 + event-schemas/schema/m.room.member | 23 +++++++++++++++++++ scripts/css/tables.css | 4 ++++ 3 files changed, 28 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2056.clarification create mode 100644 scripts/css/tables.css diff --git a/changelogs/client_server/newsfragments/2056.clarification b/changelogs/client_server/newsfragments/2056.clarification new file mode 100644 index 00000000..12521867 --- /dev/null +++ b/changelogs/client_server/newsfragments/2056.clarification @@ -0,0 +1 @@ +Clarify how to interpret changes of ``membership`` over time. diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index de14644d..20d3cbcc 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -21,6 +21,29 @@ description: |- This event may also include an ``invite_room_state`` key inside the event's ``unsigned`` data. If present, this contains an array of ``StrippedState`` Events. These events provide information on a subset of state events such as the room name. + + The user for which a membership applies is represented by the ``state_key``. Under some conditions, + the ``sender`` and ``state_key`` may not match - this may be interpreted as the ``sender`` affecting + the membership state of the ``state_key`` user. + + The ``membership`` for a given user can change over time. The table below represents the various changes + over time and how clients and servers must interpret those changes. Previous membership can be retrieved + from the ``prev_content`` object on an event. If not present, the user's previous membership must be assumed + as ``leave``. + + .. TODO: Improve how this table is written? We use a csv-table to get around vertical header restrictions. + + .. csv-table:: + :header-rows: 1 + :stub-columns: 1 + + "","to ``invite``","to ``join``","to ``leave``","to ``ban``","to ``knock``" + "from ``invite``","No change.","User joined the room.","If the ``state_key`` is the same as the ``sender``, the user rejected the invite. Otherwise, the ``state_key`` user had their invite revoked.","User was banned.","Not implemented." + "from ``join``","Must never happen.","``displayname`` or ``avatar_url`` changed.","If the ``state_key`` is the same as the ``sender``, the user left. Otherwise, the ``state_key`` user was kicked.","User was kicked and banned.","Not implemented." + "from ``leave``","New invitation sent.","User joined.","Must never happen.","User was banned.","Not implemented." + "from ``ban``","Must never happen.","Must never happen.","User was unbanned.","No change.","Not implemented." + "from ``knock``","Not implemented.","Not implemented.","Not implemented.","Not implemented.","Not implemented." + properties: content: properties: diff --git a/scripts/css/tables.css b/scripts/css/tables.css new file mode 100644 index 00000000..03ee1d85 --- /dev/null +++ b/scripts/css/tables.css @@ -0,0 +1,4 @@ +/* Column with header cells */ +table.docutils tbody th.stub { + background: #eeeeee; +} From 6ae7e49522e34b32fa317756b906a784317ac829 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Thu, 30 May 2019 12:30:43 +0100 Subject: [PATCH 142/282] Clarify what the client should receiving on email resending It was a little unclear what the client should do when asked to send a validation email and provides a `send_attempt` value that is not greater than previous attempts. As this is intended to be for when a client mistakenly sends the request twice, it makes logical sense that the implication was to simply resend a success value so the client doesn't error even when an email may have been sent on the first attempt. This behaviour was also mimicked in Synapse/Sydent. --- api/identity/definitions/request_email_validation.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/api/identity/definitions/request_email_validation.yaml b/api/identity/definitions/request_email_validation.yaml index 1194a18e..d6606f03 100644 --- a/api/identity/definitions/request_email_validation.yaml +++ b/api/identity/definitions/request_email_validation.yaml @@ -39,7 +39,8 @@ properties: avoid repeatedly sending the same email in the case of request retries between the POSTing user and the identity server. The client should increment this value if they desire a new - email (e.g. a reminder) to be sent. + email (e.g. a reminder) to be sent. If they do not, the server + should return a success but not resend the email. example: 1 next_link: type: string From c0c462999374a51b8430b88f7abe58a5a2b4386b Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Thu, 30 May 2019 12:31:26 +0100 Subject: [PATCH 143/282] Slight word change --- api/identity/definitions/request_email_validation.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/identity/definitions/request_email_validation.yaml b/api/identity/definitions/request_email_validation.yaml index d6606f03..5d1aac8a 100644 --- a/api/identity/definitions/request_email_validation.yaml +++ b/api/identity/definitions/request_email_validation.yaml @@ -40,7 +40,7 @@ properties: retries between the POSTing user and the identity server. The client should increment this value if they desire a new email (e.g. a reminder) to be sent. If they do not, the server - should return a success but not resend the email. + should response with success but not resend the email. example: 1 next_link: type: string From dda8976d2544a22cb280fce243d23107e2bd4c88 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 30 May 2019 12:33:29 +0100 Subject: [PATCH 144/282] Add changelog --- changelogs/client_server/newsfragments/2057.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2057.clarification diff --git a/changelogs/client_server/newsfragments/2057.clarification b/changelogs/client_server/newsfragments/2057.clarification new file mode 100644 index 00000000..de72c201 --- /dev/null +++ b/changelogs/client_server/newsfragments/2057.clarification @@ -0,0 +1 @@ +Clarify what the client should receive upon sending an identical email validation request multiple times. From 54f74cd877eb166ca53d52bd1a16f51a6df68461 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 30 May 2019 16:46:55 -0600 Subject: [PATCH 145/282] Add Olm unwedging As per [MSC1719](https://github.com/matrix-org/matrix-doc/pull/1719) No known alterations have been made to the proposal. Implementation proof: https://github.com/matrix-org/matrix-js-sdk/pull/780 --- .../client_server/newsfragments/2059.feature | 1 + event-schemas/examples/m.dummy | 4 ++ event-schemas/schema/m.dummy | 23 +++++++++++ .../modules/end_to_end_encryption.rst | 41 ++++++++++++++++--- 4 files changed, 64 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2059.feature create mode 100644 event-schemas/examples/m.dummy create mode 100644 event-schemas/schema/m.dummy diff --git a/changelogs/client_server/newsfragments/2059.feature b/changelogs/client_server/newsfragments/2059.feature new file mode 100644 index 00000000..fde106ce --- /dev/null +++ b/changelogs/client_server/newsfragments/2059.feature @@ -0,0 +1 @@ +Add support for Olm sessions becoming un-stuck. diff --git a/event-schemas/examples/m.dummy b/event-schemas/examples/m.dummy new file mode 100644 index 00000000..0cd39166 --- /dev/null +++ b/event-schemas/examples/m.dummy @@ -0,0 +1,4 @@ +{ + "content": {}, + "type": "m.dummy" +} diff --git a/event-schemas/schema/m.dummy b/event-schemas/schema/m.dummy new file mode 100644 index 00000000..5bebe430 --- /dev/null +++ b/event-schemas/schema/m.dummy @@ -0,0 +1,23 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + This event type is used to indicate new Olm sessions for end-to-end encryption. + Typically it is encrypted as an ``m.room.encrypted`` event, then sent as a `to-device`_ + event. + + The event does not have any content associated with it. The sending client is expected + to send a key share request shortly after this message, causing the receiving client to + process this ``m.dummy`` event as the most recent event and using the keyshare request + to set up the session. The keyshare request and ``m.dummy`` combination should result + in the original sending client receiving keys over the newly establish session. +properties: + content: + properties: {} + type: object + type: + enum: + - m.dummy + type: string +type: object diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 38a0a06b..a77dbad9 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -1,4 +1,5 @@ .. Copyright 2016 OpenMarket Ltd +.. Copyright 2019 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -18,7 +19,7 @@ End-to-End Encryption .. _module:e2e: Matrix optionally supports end-to-end encryption, allowing rooms to be created -whose conversation contents is not decryptable or interceptable on any of the +whose conversation contents are not decryptable or interceptable on any of the participating homeservers. Key Distribution @@ -549,6 +550,31 @@ Example: ] } + +Recovering from undecryptable messages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Occasionally messages may be undecryptable by clients due to a variety of reasons. +When this happens to an Olm-encrypted message, the client should assume that the Olm +session has become corrupted and create a new one to replace it. + +.. Note:: + Megolm-encrypted messages generally do not have the same problem. Usually the key + for an undecryptable Megolm-encrypted message will come later, allowing the client + to decrypt it successfully. Olm does not have a way to recover from the failure, + making this session replacement process required. + +To establish a new session, the client sends a `m.dummy <#m-dummy>`_ to-device event +to the other party to notify them of the new session details. + +Clients should rate-limit the number of sessions it creates per device that it receives +a message from. Clients should not create a new session with another device if it has +already created on for that given device in the past 1 hour. + +Clients should attempt to mitigate loss of the undecryptable messages. For example, +Megolm sessions that were sent using the old session would have been lost. The client +can attempt to retrieve the lost sessions through ``m.room_key_request`` messages. + Messaging Algorithms -------------------- @@ -658,10 +684,13 @@ part of the ed25519 key it claims to have in the Olm payload. This is crucial when the ed25519 key corresponds to a verified device. If a client has multiple sessions established with another device, it should -use the session from which it last received a message. A client may expire old -sessions by defining a maximum number of olm sessions that it will maintain for -each device, and expiring sessions on a Least Recently Used basis. The maximum -number of olm sessions maintained per device should be at least 4. +use the session from which it last received and successfully decrypted a +message. For these purposes, a session that has not received any messages +should use its creation time as the time that it last received a message. +A client may expire old sessions by defining a maximum number of olm sessions +that it will maintain for each device, and expiring sessions on a Least Recently +Used basis. The maximum number of olm sessions maintained per device should +be at least 4. ``m.megolm.v1.aes-sha2`` ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -740,6 +769,8 @@ Events {{m_forwarded_room_key_event}} +{{m_dummy_event}} + Key management API ~~~~~~~~~~~~~~~~~~ From d48265f49be7ba8b4fd4471396dd4deb0796ec25 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 30 May 2019 16:51:24 -0600 Subject: [PATCH 146/282] typo --- event-schemas/schema/m.dummy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/event-schemas/schema/m.dummy b/event-schemas/schema/m.dummy index 5bebe430..8e4b6f94 100644 --- a/event-schemas/schema/m.dummy +++ b/event-schemas/schema/m.dummy @@ -11,7 +11,7 @@ description: |- to send a key share request shortly after this message, causing the receiving client to process this ``m.dummy`` event as the most recent event and using the keyshare request to set up the session. The keyshare request and ``m.dummy`` combination should result - in the original sending client receiving keys over the newly establish session. + in the original sending client receiving keys over the newly established session. properties: content: properties: {} From 41e07ff7d64ba3bb667532bb0f69c250ba93d149 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 30 May 2019 16:54:19 -0600 Subject: [PATCH 147/282] Fix incorrect state of leave->leave As mentioned in PR review, it is possible for moderators in a room to race at kicking someone, resulting in multiple leave events. --- event-schemas/schema/m.room.member | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index 20d3cbcc..8984ac5c 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -40,7 +40,7 @@ description: |- "","to ``invite``","to ``join``","to ``leave``","to ``ban``","to ``knock``" "from ``invite``","No change.","User joined the room.","If the ``state_key`` is the same as the ``sender``, the user rejected the invite. Otherwise, the ``state_key`` user had their invite revoked.","User was banned.","Not implemented." "from ``join``","Must never happen.","``displayname`` or ``avatar_url`` changed.","If the ``state_key`` is the same as the ``sender``, the user left. Otherwise, the ``state_key`` user was kicked.","User was kicked and banned.","Not implemented." - "from ``leave``","New invitation sent.","User joined.","Must never happen.","User was banned.","Not implemented." + "from ``leave``","New invitation sent.","User joined.","No change.","User was banned.","Not implemented." "from ``ban``","Must never happen.","Must never happen.","User was unbanned.","No change.","Not implemented." "from ``knock``","Not implemented.","Not implemented.","Not implemented.","Not implemented.","Not implemented." From 754b19bb929601fa5253470eea5dbfc45361d81b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 31 May 2019 10:21:16 -0600 Subject: [PATCH 148/282] typo Co-Authored-By: Hubert Chathi --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index a77dbad9..628c1a60 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -569,7 +569,7 @@ to the other party to notify them of the new session details. Clients should rate-limit the number of sessions it creates per device that it receives a message from. Clients should not create a new session with another device if it has -already created on for that given device in the past 1 hour. +already created one for that given device in the past 1 hour. Clients should attempt to mitigate loss of the undecryptable messages. For example, Megolm sessions that were sent using the old session would have been lost. The client From 57e3b152b042183f1bbb424fdaec7518c820dc66 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 31 May 2019 11:37:09 -0600 Subject: [PATCH 149/282] Move section to under Olm stuff --- .../modules/end_to_end_encryption.rst | 50 +++++++++---------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 628c1a60..e6048c28 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -550,31 +550,6 @@ Example: ] } - -Recovering from undecryptable messages -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Occasionally messages may be undecryptable by clients due to a variety of reasons. -When this happens to an Olm-encrypted message, the client should assume that the Olm -session has become corrupted and create a new one to replace it. - -.. Note:: - Megolm-encrypted messages generally do not have the same problem. Usually the key - for an undecryptable Megolm-encrypted message will come later, allowing the client - to decrypt it successfully. Olm does not have a way to recover from the failure, - making this session replacement process required. - -To establish a new session, the client sends a `m.dummy <#m-dummy>`_ to-device event -to the other party to notify them of the new session details. - -Clients should rate-limit the number of sessions it creates per device that it receives -a message from. Clients should not create a new session with another device if it has -already created one for that given device in the past 1 hour. - -Clients should attempt to mitigate loss of the undecryptable messages. For example, -Megolm sessions that were sent using the old session would have been lost. The client -can attempt to retrieve the lost sessions through ``m.room_key_request`` messages. - Messaging Algorithms -------------------- @@ -692,6 +667,31 @@ that it will maintain for each device, and expiring sessions on a Least Recently Used basis. The maximum number of olm sessions maintained per device should be at least 4. +Recovering from undecryptable messages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Occasionally messages may be undecryptable by clients due to a variety of reasons. +When this happens to an Olm-encrypted message, the client should assume that the Olm +session has become corrupted and create a new one to replace it. + +.. Note:: + Megolm-encrypted messages generally do not have the same problem. Usually the key + for an undecryptable Megolm-encrypted message will come later, allowing the client + to decrypt it successfully. Olm does not have a way to recover from the failure, + making this session replacement process required. + +To establish a new session, the client sends a `m.dummy <#m-dummy>`_ to-device event +to the other party to notify them of the new session details. + +Clients should rate-limit the number of sessions it creates per device that it receives +a message from. Clients should not create a new session with another device if it has +already created one for that given device in the past 1 hour. + +Clients should attempt to mitigate loss of the undecryptable messages. For example, +Megolm sessions that were sent using the old session would have been lost. The client +can attempt to retrieve the lost sessions through ``m.room_key_request`` messages. + + ``m.megolm.v1.aes-sha2`` ~~~~~~~~~~~~~~~~~~~~~~~~ From 7a07a6b358cb15cc6ef81fba07037b440ade2879 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:41:39 -0600 Subject: [PATCH 150/282] Clarify exactly what StrippedState is Fixes https://github.com/matrix-org/matrix-doc/issues/2066 The expectation everywhere is that the `sender` is required. `/initialSync` references StrippedState through a `m.room.member` event reference, and does not need editing. --- api/client-server/sync.yaml | 25 +---------- api/server-server/invites-v1.yaml | 20 +-------- api/server-server/invites-v2.yaml | 20 +-------- .../newsfragments/2067.clarification | 1 + .../newsfragments/2067.clarification | 1 + .../examples/m.room.member$invite_room_state | 19 ++------ event-schemas/examples/stripped_state.json | 18 ++++++++ event-schemas/schema/m.room.member | 19 +------- event-schemas/schema/stripped_state.yaml | 44 +++++++++++++++++++ 9 files changed, 71 insertions(+), 96 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2067.clarification create mode 100644 changelogs/server_server/newsfragments/2067.clarification create mode 100644 event-schemas/examples/stripped_state.json create mode 100644 event-schemas/schema/stripped_state.yaml diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 02fddb84..3db1fa54 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -212,30 +212,7 @@ paths: events: description: The StrippedState events that form the invite state. items: - description: |- - A stripped down state event, with only the ``type``, ``state_key``, - ``sender``, and ``content`` keys. - properties: - content: - description: The ``content`` for the event. - title: EventContent - type: object - state_key: - description: The ``state_key`` for the event. - type: string - type: - description: The ``type`` for the event. - type: string - sender: - description: The ``sender`` for the event. - type: string - required: - - type - - state_key - - content - - sender - title: StrippedState - type: object + $ref: "definitions/event-schemas/schema/stripped_state.yaml" type: array leave: title: Left rooms diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 2ad0f220..867d7b05 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -82,25 +82,7 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - type: object - title: Invite Room State Event - properties: - type: - type: string - description: The type of event. - example: "m.room.join_rules" - state_key: - type: string - description: The state key for the event. May be an empty string. - example: "" - content: - type: object - description: The content for the event. - sender: - type: string - description: The sender of the event. - example: "@someone:matrix.org" - required: ['type', 'state_key', 'content', 'sender'] + $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" example: [ { "type": "m.room.join_rules", diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index c459a848..6d5b102e 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -83,25 +83,7 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - type: object - title: Invite Room State Event - properties: - type: - type: string - description: The type of event. - example: "m.room.join_rules" - state_key: - type: string - description: The state key for the event. May be an empty string. - example: "" - content: - type: object - description: The content for the event. - sender: - type: string - description: The sender of the event. - example: "@someone:matrix.org" - required: ['type', 'state_key', 'content', 'sender'] + $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" example: [ { "type": "m.room.join_rules", diff --git a/changelogs/client_server/newsfragments/2067.clarification b/changelogs/client_server/newsfragments/2067.clarification new file mode 100644 index 00000000..cc706274 --- /dev/null +++ b/changelogs/client_server/newsfragments/2067.clarification @@ -0,0 +1 @@ +Clarify exactly what invite_room_state consists of. diff --git a/changelogs/server_server/newsfragments/2067.clarification b/changelogs/server_server/newsfragments/2067.clarification new file mode 100644 index 00000000..cc706274 --- /dev/null +++ b/changelogs/server_server/newsfragments/2067.clarification @@ -0,0 +1 @@ +Clarify exactly what invite_room_state consists of. diff --git a/event-schemas/examples/m.room.member$invite_room_state b/event-schemas/examples/m.room.member$invite_room_state index f8f05484..b60abf95 100644 --- a/event-schemas/examples/m.room.member$invite_room_state +++ b/event-schemas/examples/m.room.member$invite_room_state @@ -7,21 +7,8 @@ }, "unsigned": { "age": 1234, - "invite_room_state": [ - { - "type": "m.room.name", - "state_key": "", - "content": { - "name": "Forest of Magic" - } - }, - { - "type": "m.room.join_rules", - "state_key": "", - "content": { - "join_rule": "invite" - } - } - ] + "invite_room_state": { + "$ref": "stripped_state.json" + } } } diff --git a/event-schemas/examples/stripped_state.json b/event-schemas/examples/stripped_state.json new file mode 100644 index 00000000..9d8c1b2b --- /dev/null +++ b/event-schemas/examples/stripped_state.json @@ -0,0 +1,18 @@ +[ + { + "type": "m.room.name", + "sender": "@bob:example.org", + "state_key": "", + "content": { + "name": "Example Room" + } + }, + { + "type": "m.room.join_rules", + "sender": "@bob:example.org", + "state_key": "", + "content": { + "join_rule": "invite" + } + } +] diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index de14644d..f846196b 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -81,24 +81,7 @@ properties: invite_room_state: description: 'A subset of the state of the room at the time of the invite, if ``membership`` is ``invite``. Note that this state is informational, and SHOULD NOT be trusted; once the client has joined the room, it SHOULD fetch the live state from the server and discard the invite_room_state. Also, clients must not rely on any particular state being present here; they SHOULD behave properly (with possibly a degraded but not a broken experience) in the absence of any particular events here. If they are set on the room, at least the state for ``m.room.avatar``, ``m.room.canonical_alias``, ``m.room.join_rules``, and ``m.room.name`` SHOULD be included.' items: - description: 'A stripped down state event, with only the ``type``, ``state_key`` and ``content`` keys.' - properties: - content: - description: The ``content`` for the event. - title: EventContent - type: object - state_key: - description: The ``state_key`` for the event. - type: string - type: - description: The ``type`` for the event. - type: string - required: - - type - - state_key - - content - title: StrippedState - type: object + $ref: "stripped_state.yaml" type: array required: - membership diff --git a/event-schemas/schema/stripped_state.yaml b/event-schemas/schema/stripped_state.yaml new file mode 100644 index 00000000..ec591bf1 --- /dev/null +++ b/event-schemas/schema/stripped_state.yaml @@ -0,0 +1,44 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Note: this, and the example, are in the `event-schemas` directory because +# the CS API uses a symlink. In order for the `m.room.member` event to +# reference this, we'd need to use relative pathing. The symlink makes this +# difficult because the schema would be at two different locations, with +# different relative pathing. + +title: StrippedState +type: object +description: |- + A stripped down state event, with only the ``type``, ``state_key``, + ``sender``, and ``content`` keys. +properties: + content: + description: The ``content`` for the event. + title: EventContent + type: object + state_key: + description: The ``state_key`` for the event. + type: string + type: + description: The ``type`` for the event. + type: string + sender: + description: The ``sender`` for the event. + type: string +required: + - type + - state_key + - content + - sender From b9c4a2561ff8692b8e960e9eb977efb27b86e57b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:50:27 -0600 Subject: [PATCH 151/282] Fix examples of StrippedState in s2s spec --- api/server-server/invites-v1.yaml | 53 +++---------------------------- api/server-server/invites-v2.yaml | 53 ++++--------------------------- 2 files changed, 11 insertions(+), 95 deletions(-) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 867d7b05..2f08dd12 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -83,16 +83,8 @@ paths: canonical alias, avatar, and name of the room. items: $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" - example: [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - } - ] + example: + $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -100,26 +92,6 @@ paths: "origin": "example.org", "origin_server_ts": 1549041175876, "sender": "@someone:example.org", - "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] - }, "content": { "membership": "invite" }, @@ -162,24 +134,9 @@ paths: "origin_server_ts": 1549041175876, "sender": "@someone:example.org", "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + "invite_room_state": { + "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + } }, "content": { "membership": "invite" diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index 6d5b102e..57ca99ff 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -84,16 +84,8 @@ paths: canonical alias, avatar, and name of the room. items: $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" - example: [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - } - ] + example: + $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -112,25 +104,7 @@ paths: "ed25519:key_version": "SomeSignatureHere" }, } - }, - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + } } responses: 200: @@ -156,24 +130,9 @@ paths: "origin_server_ts": 1549041175876, "sender": "@someone:example.org", "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + "invite_room_state": { + "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + } }, "content": { "membership": "invite" From 237d585e07b376d69e0f7e887b7135bc8250aba6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:59:19 -0600 Subject: [PATCH 152/282] Shorten references to StrippedState in s2s spec --- api/server-server/invites-v1.yaml | 6 +++--- api/server-server/invites-v2.yaml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 2f08dd12..83aafb3a 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -82,9 +82,9 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" + $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/stripped_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -135,7 +135,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/stripped_state.json" } }, "content": { diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index 57ca99ff..d2cc63a2 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -83,9 +83,9 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" + $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/stripped_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -131,7 +131,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/stripped_state.json" } }, "content": { From 0b45f3795bb6c561266f9388f850bae460c204fd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 18:02:22 -0600 Subject: [PATCH 153/282] Rename example to invite_room_state This is a better representation of what it actually is --- api/server-server/invites-v1.yaml | 4 ++-- api/server-server/invites-v2.yaml | 4 ++-- .../examples/{stripped_state.json => invite_room_state.json} | 0 event-schemas/examples/m.room.member$invite_room_state | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) rename event-schemas/examples/{stripped_state.json => invite_room_state.json} (100%) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 83aafb3a..8e1c861d 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -84,7 +84,7 @@ paths: items: $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../../event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/invite_room_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -135,7 +135,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../../event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/invite_room_state.json" } }, "content": { diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index d2cc63a2..cae14bb4 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -85,7 +85,7 @@ paths: items: $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../../event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/invite_room_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -131,7 +131,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../../event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/invite_room_state.json" } }, "content": { diff --git a/event-schemas/examples/stripped_state.json b/event-schemas/examples/invite_room_state.json similarity index 100% rename from event-schemas/examples/stripped_state.json rename to event-schemas/examples/invite_room_state.json diff --git a/event-schemas/examples/m.room.member$invite_room_state b/event-schemas/examples/m.room.member$invite_room_state index b60abf95..2c93eb9b 100644 --- a/event-schemas/examples/m.room.member$invite_room_state +++ b/event-schemas/examples/m.room.member$invite_room_state @@ -8,7 +8,7 @@ "unsigned": { "age": 1234, "invite_room_state": { - "$ref": "stripped_state.json" + "$ref": "invite_room_state.json" } } } From 86019c9adeb05a3998a03b7e12089b151293015a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 18:02:36 -0600 Subject: [PATCH 154/282] Skip over partial event definitions in examples --- event-schemas/check_examples.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/event-schemas/check_examples.py b/event-schemas/check_examples.py index 2baa3ef0..31daa478 100755 --- a/event-schemas/check_examples.py +++ b/event-schemas/check_examples.py @@ -106,6 +106,9 @@ def check_example_dir(exampledir, schemadir): if filename.startswith("."): # Skip over any vim .swp files. continue + if filename.endswith(".json"): + # Skip over any explicit examples (partial event definitions) + continue cwd = os.path.basename(os.path.dirname(os.path.join(root, filename))) if cwd == "core": # Skip checking the underlying definitions From 1f86e8e31b11db7514db2d139cb3836f5af0078e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 21:23:59 -0600 Subject: [PATCH 155/282] Refactor documentation for content/media repository Fixes https://github.com/matrix-org/matrix-doc/issues/2060 Fixes https://github.com/matrix-org/matrix-doc/issues/772 Fixes https://github.com/matrix-org/matrix-doc/issues/888 --- api/client-server/content-repo.yaml | 156 ++++++++++-------- api/client-server/room_state.yaml | 2 +- .../newsfragments/2068.clarification | 1 + event-schemas/examples/m.room.member | 2 +- .../examples/m.room.member$invite_room_state | 2 +- .../examples/m.room.member$third_party_invite | 2 +- .../msgtype_infos/image_info.yaml | 2 +- event-schemas/schema/m.room.message$m.audio | 2 +- event-schemas/schema/m.room.message$m.file | 2 +- event-schemas/schema/m.room.message$m.image | 2 +- event-schemas/schema/m.room.message$m.video | 4 +- specification/modules/content_repo.rst | 87 +++++++--- 12 files changed, 163 insertions(+), 101 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2068.clarification diff --git a/api/client-server/content-repo.yaml b/api/client-server/content-repo.yaml index 4460bb69..576d29ef 100644 --- a/api/client-server/content-repo.yaml +++ b/api/client-server/content-repo.yaml @@ -1,4 +1,5 @@ # Copyright 2016 OpenMarket Ltd +# Copyright 2019 The Matrix.org Foundation C.I.C. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -58,66 +59,41 @@ paths: format: byte responses: 200: - description: The MXC URI for the uploaded content. + description: The `MXC URI`_ for the uploaded content. schema: type: object required: ["content_uri"] properties: content_uri: type: string - description: "The MXC URI to the uploaded content." + description: "The `MXC URI`_ to the uploaded content." examples: application/json: { - "content_uri": "mxc://example.com/AQwafuaFswefuhsfAFAgsw" - } - 429: - description: This request was rate-limited. + "content_uri": "mxc://example.com/AQwafuaFswefuhsfAFAgsw" + } + 403: + description: |- + The user does not have permission to upload the content. Some reasons for this error include: + + - The server does not permit the file type. + - The user has reached a quota for uploaded content. + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "Cannot upload this content" + } schema: - "$ref": "definitions/errors/rate_limited.yaml" - tags: - - Media - "/download/{serverName}/{mediaId}": - get: - summary: "Download content from the content repository." - operationId: getContent - produces: ["*/*"] - parameters: - - in: path - type: string - name: serverName - x-example: matrix.org - required: true - description: | - The server name from the ``mxc://`` URI (the authoritory component) - - in: path - type: string - name: mediaId - x-example: ascERGshawAWawugaAcauga - required: true - description: | - The media ID from the ``mxc://`` URI (the path component) - - in: query - type: boolean - name: allow_remote - x-example: false - required: false - default: true - description: | - Indicates to the server that it should not attempt to fetch the media if it is deemed - remote. This is to prevent routing loops where the server contacts itself. Defaults to - true if not provided. - responses: - 200: - description: "The content that was previously uploaded." - headers: - Content-Type: - description: "The content type of the file that was previously uploaded." - type: "string" - Content-Disposition: - description: "The name of the file that was previously uploaded, if set." - type: "string" + "$ref": "definitions/errors/error.yaml" + 413: + description: |- + The uploaded content is too large for the server. + examples: + application/json: { + "errcode": "M_TOO_LARGE", + "error": "Cannot upload files larger than 100mb" + } schema: - type: file + "$ref": "definitions/errors/error.yaml" 429: description: This request was rate-limited. schema: @@ -126,8 +102,8 @@ paths: - Media "/download/{serverName}/{mediaId}/{fileName}": get: - summary: "Download content from the content repository as a given filename." - operationId: getContentOverrideName + summary: "Download content from the content repository." + operationId: getContent produces: ["*/*"] parameters: - in: path @@ -148,9 +124,7 @@ paths: type: string name: fileName x-example: filename.jpg - required: true - description: | - The filename to give in the Content-Disposition + description: An optional filename to give in the ``Content-Disposition`` header. - in: query type: boolean name: allow_remote @@ -169,10 +143,24 @@ paths: description: "The content type of the file that was previously uploaded." type: "string" Content-Disposition: - description: "The name of file given in the request" + description: |- + The ``fileName`` requested or the name of the file that was previously + uploaded, if set. type: "string" schema: type: file + # This is a workaround for us not being able to say the response is required. + description: "**Required.** The bytes for the uploaded file." + 502: + description: |- + The content is too large for the server to serve. + examples: + application/json: { + "errcode": "M_TOO_LARGE", + "error": "Content is too large to serve" + } + schema: + "$ref": "definitions/errors/error.yaml" 429: description: This request was rate-limited. schema: @@ -181,7 +169,9 @@ paths: - Media "/thumbnail/{serverName}/{mediaId}": get: - summary: "Download a thumbnail of the content from the content repository." + summary: |- + Download a thumbnail of content from the content repository. See the `thumbnailing <#thumbnails>`_ + section for more information. operationId: getContentThumbnail produces: ["image/jpeg", "image/png"] parameters: @@ -189,7 +179,7 @@ paths: type: string name: serverName required: true - x-example: matrix.org + x-example: example.org description: | The server name from the ``mxc://`` URI (the authoritory component) - in: path @@ -205,22 +195,24 @@ paths: name: width required: true description: |- - The *desired* width of the thumbnail. The actual thumbnail may not - match the size specified. + The *desired* width of the thumbnail. The actual thumbnail may be + larger than the size specified. - in: query type: integer x-example: 64 name: height required: true description: |- - The *desired* height of the thumbnail. The actual thumbnail may not - match the size specified. + The *desired* height of the thumbnail. The actual thumbnail may be + larger than the size specified. - in: query type: string enum: ["crop", "scale"] name: method x-example: "scale" - description: The desired resizing method. + description: |- + The desired resizing method. See the `thumbnailing <#thumbnails>`_ + section for more information. - in: query type: boolean name: allow_remote @@ -241,6 +233,40 @@ paths: enum: ["image/jpeg", "image/png"] schema: type: file + # This is a workaround for us not being able to say the response is required. + description: "**Required.** The bytes for the thumbnail." + 400: + description: |- + The request does not make sense to the server, or the server cannot thumbnail + the content. For example, the client requested non-integer dimensions or asked + for negatively-sized images. + examples: + application/json: { + "errcode": "M_UNKNOWN", + "error": "Cannot generate thumbnails for the requested content" + } + schema: + "$ref": "definitions/errors/error.yaml" + 413: + description: |- + The local content is too large for the server to thumbnail. + examples: + application/json: { + "errcode": "M_TOO_LARGE", + "error": "Content is too large to thumbnail" + } + schema: + "$ref": "definitions/errors/error.yaml" + 502: + description: |- + The remote content is too large for the server to thumbnail. + examples: + application/json: { + "errcode": "M_TOO_LARGE", + "error": "Content is too large to thumbnail" + } + schema: + "$ref": "definitions/errors/error.yaml" 429: description: This request was rate-limited. schema: @@ -259,7 +285,7 @@ paths: type: string x-example: "https://matrix.org" name: url - description: "The URL to get a preview of" + description: "The URL to get a preview of." required: true - in: query type: integer @@ -287,7 +313,7 @@ paths: "og:image": type: string description: |- - An MXC URI to the image. Omitted if there is no image. + An `MXC URI`_ to the image. Omitted if there is no image. examples: application/json: { "og:title": "Matrix Blog Post", diff --git a/api/client-server/room_state.yaml b/api/client-server/room_state.yaml index bda66eb8..4b159a3c 100644 --- a/api/client-server/room_state.yaml +++ b/api/client-server/room_state.yaml @@ -70,7 +70,7 @@ paths: type: object example: { "membership": "join", - "avatar_url": "mxc://localhost/SEsfnsuifSDFSSEF#auto", + "avatar_url": "mxc://localhost/SEsfnsuifSDFSSEF", "displayname": "Alice Margatroid" } responses: diff --git a/changelogs/client_server/newsfragments/2068.clarification b/changelogs/client_server/newsfragments/2068.clarification new file mode 100644 index 00000000..77ad7125 --- /dev/null +++ b/changelogs/client_server/newsfragments/2068.clarification @@ -0,0 +1 @@ +Clarify how the content repository works, and what it is used for. diff --git a/event-schemas/examples/m.room.member b/event-schemas/examples/m.room.member index b0aa59dd..18bc457b 100644 --- a/event-schemas/examples/m.room.member +++ b/event-schemas/examples/m.room.member @@ -4,7 +4,7 @@ "type": "m.room.member", "content": { "membership": "join", - "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF#auto", + "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF", "displayname": "Alice Margatroid" } } diff --git a/event-schemas/examples/m.room.member$invite_room_state b/event-schemas/examples/m.room.member$invite_room_state index f8f05484..9045dffd 100644 --- a/event-schemas/examples/m.room.member$invite_room_state +++ b/event-schemas/examples/m.room.member$invite_room_state @@ -2,7 +2,7 @@ "$ref": "m.room.member", "content": { "membership": "invite", - "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF#auto", + "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF", "displayname": "Alice Margatroid" }, "unsigned": { diff --git a/event-schemas/examples/m.room.member$third_party_invite b/event-schemas/examples/m.room.member$third_party_invite index c688a283..a40d44f9 100644 --- a/event-schemas/examples/m.room.member$third_party_invite +++ b/event-schemas/examples/m.room.member$third_party_invite @@ -2,7 +2,7 @@ "$ref": "m.room.member", "content": { "membership": "invite", - "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF#auto", + "avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF", "displayname": "Alice Margatroid", "third_party_invite": { "display_name": "alice", diff --git a/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml b/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml index 8ff27b1e..b6a45007 100644 --- a/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml +++ b/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml @@ -19,7 +19,7 @@ properties: type: integer thumbnail_url: description: |- - The URL to a thumbnail of the image. Only present if the + The `MXC URI`_ to a thumbnail of the image. Only present if the thumbnail is unencrypted. type: string thumbnail_file: diff --git a/event-schemas/schema/m.room.message$m.audio b/event-schemas/schema/m.room.message$m.audio index 99e28110..40075541 100644 --- a/event-schemas/schema/m.room.message$m.audio +++ b/event-schemas/schema/m.room.message$m.audio @@ -27,7 +27,7 @@ properties: - m.audio type: string url: - description: Required if the file is not encrypted. The URL to the audio clip. + description: Required if the file is not encrypted. The `MXC URI`_ to the audio clip. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.file b/event-schemas/schema/m.room.message$m.file index 2389d8a9..225ca61d 100644 --- a/event-schemas/schema/m.room.message$m.file +++ b/event-schemas/schema/m.room.message$m.file @@ -42,7 +42,7 @@ properties: - m.file type: string url: - description: Required if the file is unencrypted. The URL to the file. + description: Required if the file is unencrypted. The `MXC URI`_ to the file. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.image b/event-schemas/schema/m.room.message$m.image index 1e6ebeaa..8bf9c5fa 100644 --- a/event-schemas/schema/m.room.message$m.image +++ b/event-schemas/schema/m.room.message$m.image @@ -17,7 +17,7 @@ properties: - m.image type: string url: - description: Required if the file is unencrypted. The URL to the image. + description: Required if the file is unencrypted. The `MXC URI`_ to the image. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.video b/event-schemas/schema/m.room.message$m.video index 2da7e0bc..01286ce2 100644 --- a/event-schemas/schema/m.room.message$m.video +++ b/event-schemas/schema/m.room.message$m.video @@ -28,7 +28,7 @@ properties: type: integer thumbnail_url: description: |- - The URL to an image thumbnail of the video clip. Only present if the + The `MXC URI`_ to an image thumbnail of the video clip. Only present if the thumbnail is unencrypted. type: string thumbnail_file: @@ -48,7 +48,7 @@ properties: - m.video type: string url: - description: Required if the file is unencrypted. The URL to the video clip. + description: Required if the file is unencrypted. The `MXC URI`_ to the video clip. type: string file: description: |- diff --git a/specification/modules/content_repo.rst b/specification/modules/content_repo.rst index e7bdb044..823efb3c 100644 --- a/specification/modules/content_repo.rst +++ b/specification/modules/content_repo.rst @@ -1,4 +1,5 @@ .. Copyright 2016 OpenMarket Ltd +.. Copyright 2019 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -17,27 +18,38 @@ Content repository .. _module:content: -This module allows users to upload content to their homeserver which is -retrievable from other homeservers. Its' purpose is to allow users to share -attachments in a room. Content locations are represented as Matrix Content (MXC) -URIs. They look like:: +The content repository (or "media repository") allows users to upload +files to their homeserver for later user. For example, files which the +user wants to send to a room would be uploaded here, as would an avatar +the user wants to use. - mxc:/// - - : The name of the homeserver where this content originated, e.g. matrix.org - : An opaque ID which identifies the content. - -Uploads are POSTed to a resource on the user's local homeserver which returns a -token which is used to GET the download. Content is downloaded from the -recipient's local homeserver, which must first transfer the content from the -origin homeserver using the same API (unless the origin and destination -homeservers are the same). +Uploads are POSTed to a resource on the user's local homeserver which +returns a MXC URI which can later be used to GET the download. Content +is downloaded from the recipient's local homeserver, which must first +transfer the content from the origin homeserver using the same API +(unless the origin and destination homeservers are the same). When serving content, the server SHOULD provide a ``Content-Security-Policy`` header. The recommended policy is ``sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';``. +Content in the repository should be treated as bytes as it may be encrypted. + +Matrix Content (MXC) URIs +------------------------- + +.. _`MXC URI`: + +Content locations are represented as Matrix Content (MXC) URIs. They look +like:: + + mxc:/// + + : The name of the homeserver where this content originated, e.g. matrix.org + : An opaque ID which identifies the content. + + Client behaviour ---------------- @@ -47,6 +59,11 @@ Clients can upload and download content using the following HTTP APIs. Thumbnails ~~~~~~~~~~ +The homeserver SHOULD be able to supply thumbnails for uploaded images and +videos. The exact file types which can be thumbnailed are not currently +specified - see `Issue #1938 `_ +for more information. + The thumbnail methods are "crop" and "scale". "scale" tries to return an image where either the width or the height is smaller than the requested size. The client should then scale and letterbox the image if it needs to @@ -55,18 +72,32 @@ width and height are close to the requested size and the aspect matches the requested size. The client should scale the image if it needs to fit within a given rectangle. +The dimensions given to the thumbnail API are the minimum size the client +would prefer. Servers must never return thumbnails smaller than the client's +requested dimensions, unless the content being thumbnailed is smaller than +the dimensions. When the content is smaller than the requested dimensions, +servers should return the original content rather than thumbnail it. + +Servers SHOULD pre-calculate or have a list of set dimensions for which they +will thumbnail content at. For example, the server may choose that it will +only create thumbnails sized 96x96 or 512x512. When the client requests a +thumbnail, the server will pick the size which is larger than the requested +dimensions. Servers SHOULD pre-calculate the following thumbnails for uploaded +content, and limit thumbnails to the same sizes: + +* 32x32, crop +* 96x96, crop +* 320x240, scale +* 640x480, scale +* 800x600, scale + In summary: * "scale" maintains the original aspect ratio of the image * "crop" provides an image in the aspect ratio of the sizes given in the request + * The server will return an image larger than or equal to the dimensions requested + where possible. -Server behaviour ----------------- - -Homeservers may generate thumbnails for content uploaded to remote -homeservers themselves or may rely on the remote homeserver to thumbnail -the content. Homeservers may return thumbnails of a different size to that -requested. However homeservers should provide exact matches where reasonable. -Homeservers must never upscale images. +Servers MUST NOT upscale thumbnails under any circumstance. Security considerations ----------------------- @@ -88,16 +119,20 @@ UTF-8 encoded traversals, etc). Homeservers have additional content-specific concerns: - Clients may try to upload very large files. Homeservers should not store files - that are too large and should not serve them to clients. + that are too large and should not serve them to clients, returning a HTTP 413 + error with the ``M_TOO_LARGE`` code. - Clients may try to upload very large images. Homeservers should not attempt to - generate thumbnails for images that are too large. + generate thumbnails for images that are too large, returning a HTTP 413 error + with the ``M_TOO_LARGE`` code. - Remote homeservers may host very large files or images. Homeservers should not - proxy or thumbnail large files or images from remote homeservers. + proxy or thumbnail large files or images from remote homeservers, returning a + HTTP 502 error with the ``M_TOO_LARGE`` code. - Clients may try to upload a large number of files. Homeservers should limit the - number and total size of media that can be uploaded by clients. + number and total size of media that can be uploaded by clients, returning a + HTTP 403 error with the ``M_FORBIDDEN`` code. - Clients may try to access a large number of remote files through a homeserver. Homeservers should restrict the number and size of remote files that it caches. From dc6d89caca5ea165bfe472d82f696529e80feaab Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 21:31:06 -0600 Subject: [PATCH 156/282] Split download endpoints back apart Apparently you can't have an optional path parameter. --- api/client-server/content-repo.yaml | 68 ++++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/api/client-server/content-repo.yaml b/api/client-server/content-repo.yaml index 576d29ef..a9a0c2f6 100644 --- a/api/client-server/content-repo.yaml +++ b/api/client-server/content-repo.yaml @@ -100,11 +100,74 @@ paths: "$ref": "definitions/errors/rate_limited.yaml" tags: - Media - "/download/{serverName}/{mediaId}/{fileName}": + "/download/{serverName}/{mediaId}": get: summary: "Download content from the content repository." operationId: getContent produces: ["*/*"] + parameters: + - in: path + type: string + name: serverName + x-example: matrix.org + required: true + description: | + The server name from the ``mxc://`` URI (the authoritory component) + - in: path + type: string + name: mediaId + x-example: ascERGshawAWawugaAcauga + required: true + description: | + The media ID from the ``mxc://`` URI (the path component) + - in: query + type: boolean + name: allow_remote + x-example: false + required: false + default: true + description: | + Indicates to the server that it should not attempt to fetch the media if it is deemed + remote. This is to prevent routing loops where the server contacts itself. Defaults to + true if not provided. + responses: + 200: + description: "The content that was previously uploaded." + headers: + Content-Type: + description: "The content type of the file that was previously uploaded." + type: "string" + Content-Disposition: + description: |- + The name of the file that was previously uploaded, if set. + type: "string" + schema: + type: file + # This is a workaround for us not being able to say the response is required. + description: "**Required.** The bytes for the uploaded file." + 502: + description: |- + The content is too large for the server to serve. + examples: + application/json: { + "errcode": "M_TOO_LARGE", + "error": "Content is too large to serve" + } + schema: + "$ref": "definitions/errors/error.yaml" + 429: + description: This request was rate-limited. + schema: + "$ref": "definitions/errors/rate_limited.yaml" + tags: + - Media + "/download/{serverName}/{mediaId}/{fileName}": + get: + summary: |- + Download content from the content repository. This is the same as + the download endpoint above, except permitting a desired file name. + operationId: getContentOverrideName + produces: ["*/*"] parameters: - in: path type: string @@ -124,7 +187,8 @@ paths: type: string name: fileName x-example: filename.jpg - description: An optional filename to give in the ``Content-Disposition`` header. + required: true + description: A filename to give in the ``Content-Disposition`` header. - in: query type: boolean name: allow_remote From 53aa8fe8ecc7e4d2028894a57395fe3e9340fc2c Mon Sep 17 00:00:00 2001 From: Bruno Windels Date: Mon, 3 Jun 2019 09:47:09 +0200 Subject: [PATCH 157/282] clarify the order events in chunk for /messages --- api/client-server/message_pagination.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index c9f9d0ae..7457530b 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -103,7 +103,7 @@ paths: chunk: type: array description: |- - A list of room events. + A list of room events. The order depends on the ``dir`` parameter. For ``dir=b`` events will be in reverse-chronological order, for ``dir=f`` in chronological order, so that events start at the ``from`` point. items: type: object title: RoomEvent From 0b42a17352356eb879ee21f264dddfdcc78ef647 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Mon, 3 Jun 2019 09:48:38 +0100 Subject: [PATCH 158/282] Update api/identity/definitions/request_email_validation.yaml Co-Authored-By: Travis Ralston --- api/identity/definitions/request_email_validation.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/identity/definitions/request_email_validation.yaml b/api/identity/definitions/request_email_validation.yaml index 5d1aac8a..5f15bd41 100644 --- a/api/identity/definitions/request_email_validation.yaml +++ b/api/identity/definitions/request_email_validation.yaml @@ -40,7 +40,7 @@ properties: retries between the POSTing user and the identity server. The client should increment this value if they desire a new email (e.g. a reminder) to be sent. If they do not, the server - should response with success but not resend the email. + should respond with success but not resend the email. example: 1 next_link: type: string From 49b9bfc9f6ed5041807d8d893a29f9753381ad06 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 3 Jun 2019 09:48:53 +0100 Subject: [PATCH 159/282] Move changelog to the right place --- .../newsfragments/2057.clarification | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/{client_server => identity_service}/newsfragments/2057.clarification (100%) diff --git a/changelogs/client_server/newsfragments/2057.clarification b/changelogs/identity_service/newsfragments/2057.clarification similarity index 100% rename from changelogs/client_server/newsfragments/2057.clarification rename to changelogs/identity_service/newsfragments/2057.clarification From 39144942da1b235627b652909c0f21c8f57c33ea Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 3 Jun 2019 20:39:41 -0600 Subject: [PATCH 160/282] Fix title ordering --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index e6048c28..1fd28e93 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -668,7 +668,7 @@ Used basis. The maximum number of olm sessions maintained per device should be at least 4. Recovering from undecryptable messages -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Occasionally messages may be undecryptable by clients due to a variety of reasons. When this happens to an Olm-encrypted message, the client should assume that the Olm From 8b28972a2d38935f3c5c7953ba40dbc5c36ddbcb Mon Sep 17 00:00:00 2001 From: Bruno Windels Date: Tue, 4 Jun 2019 09:28:01 +0200 Subject: [PATCH 161/282] PR feedback --- api/client-server/message_pagination.yaml | 5 ++++- changelogs/client_server/newsfragments/2069.clarification | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2069.clarification diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 7457530b..9a24537c 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -103,7 +103,10 @@ paths: chunk: type: array description: |- - A list of room events. The order depends on the ``dir`` parameter. For ``dir=b`` events will be in reverse-chronological order, for ``dir=f`` in chronological order, so that events start at the ``from`` point. + A list of room events. The order depends on the ``dir`` parameter. + For ``dir=b`` events will be in reverse-chronological order, + for ``dir=f`` in chronological order, so that events start + at the ``from`` point. items: type: object title: RoomEvent diff --git a/changelogs/client_server/newsfragments/2069.clarification b/changelogs/client_server/newsfragments/2069.clarification new file mode 100644 index 00000000..353b545d --- /dev/null +++ b/changelogs/client_server/newsfragments/2069.clarification @@ -0,0 +1 @@ +Clarify the order events in chunk are returned in for /messages From 042455d954699b486b5c732d848697d11fd6a5e4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 4 Jun 2019 10:58:39 -0600 Subject: [PATCH 162/282] Update changelog to appease style guidelines --- changelogs/client_server/newsfragments/2069.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2069.clarification b/changelogs/client_server/newsfragments/2069.clarification index 353b545d..127573a6 100644 --- a/changelogs/client_server/newsfragments/2069.clarification +++ b/changelogs/client_server/newsfragments/2069.clarification @@ -1 +1 @@ -Clarify the order events in chunk are returned in for /messages +Clarify the order events in chunk are returned in for ``/messages``. From a3364ff35712be278fc4f5914a89dbb27f41d08a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 4 Jun 2019 12:41:30 -0600 Subject: [PATCH 163/282] Spec SAS verification and the common key verification framework Reference implementations: * https://gitlab.matrix.org/matrix-org/olm/commit/94f664e7256215f33639dbbad6aaf87ada082a9f * https://github.com/matrix-org/matrix-react-sdk/pull/2461 * https://github.com/matrix-org/matrix-js-sdk/pull/818 * https://github.com/matrix-org/matrix-react-sdk/pull/2596 * https://github.com/matrix-org/matrix-js-sdk/pull/837 Proposals: * [MSC1717](https://github.com/matrix-org/matrix-doc/pull/1717) * [MSC1267](https://github.com/matrix-org/matrix-doc/issues/1267) No alterations to either proposal have been made intentionally here. --- .../client_server/newsfragments/2072.feature | 1 + data-definitions/sas-emoji.json | 66 ++++ .../examples/m.key.verification.accept | 12 + .../examples/m.key.verification.cancel | 8 + event-schemas/examples/m.key.verification.key | 7 + event-schemas/examples/m.key.verification.mac | 10 + .../examples/m.key.verification.request | 11 + .../examples/m.key.verification.start | 8 + .../m.key.verification.start$m.sas.v1 | 12 + .../schema/m.key.verification.accept | 64 ++++ .../schema/m.key.verification.cancel | 70 ++++ event-schemas/schema/m.key.verification.key | 28 ++ event-schemas/schema/m.key.verification.mac | 38 ++ .../schema/m.key.verification.request | 43 +++ event-schemas/schema/m.key.verification.start | 39 +++ .../schema/m.key.verification.start$m.sas.v1 | 69 ++++ scripts/continuserv/main.go | 2 +- .../templating/matrix_templates/sections.py | 17 + scripts/templating/matrix_templates/units.py | 20 ++ .../modules/end_to_end_encryption.rst | 325 +++++++++++++++++- 20 files changed, 847 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2072.feature create mode 100644 data-definitions/sas-emoji.json create mode 100644 event-schemas/examples/m.key.verification.accept create mode 100644 event-schemas/examples/m.key.verification.cancel create mode 100644 event-schemas/examples/m.key.verification.key create mode 100644 event-schemas/examples/m.key.verification.mac create mode 100644 event-schemas/examples/m.key.verification.request create mode 100644 event-schemas/examples/m.key.verification.start create mode 100644 event-schemas/examples/m.key.verification.start$m.sas.v1 create mode 100644 event-schemas/schema/m.key.verification.accept create mode 100644 event-schemas/schema/m.key.verification.cancel create mode 100644 event-schemas/schema/m.key.verification.key create mode 100644 event-schemas/schema/m.key.verification.mac create mode 100644 event-schemas/schema/m.key.verification.request create mode 100644 event-schemas/schema/m.key.verification.start create mode 100644 event-schemas/schema/m.key.verification.start$m.sas.v1 diff --git a/changelogs/client_server/newsfragments/2072.feature b/changelogs/client_server/newsfragments/2072.feature new file mode 100644 index 00000000..c7d8bd76 --- /dev/null +++ b/changelogs/client_server/newsfragments/2072.feature @@ -0,0 +1 @@ +Add interactive device verification, including a common framework for device verification. diff --git a/data-definitions/sas-emoji.json b/data-definitions/sas-emoji.json new file mode 100644 index 00000000..060fbd49 --- /dev/null +++ b/data-definitions/sas-emoji.json @@ -0,0 +1,66 @@ +[ + {"number": 0, "emoji": "🐶", "description": "Dog", "unicode": "U+1F436"}, + {"number": 1, "emoji": "🐱", "description": "Cat", "unicode": "U+1F431"}, + {"number": 2, "emoji": "🦁", "description": "Lion", "unicode": "U+1F981"}, + {"number": 3, "emoji": "🐎", "description": "Horse", "unicode": "U+1F40E"}, + {"number": 4, "emoji": "🦄", "description": "Unicorn", "unicode": "U+1F984"}, + {"number": 5, "emoji": "🐷", "description": "Pig", "unicode": "U+1F437"}, + {"number": 6, "emoji": "🐘", "description": "Elephant", "unicode": "U+1F418"}, + {"number": 7, "emoji": "🐰", "description": "Rabbit", "unicode": "U+1F430"}, + {"number": 8, "emoji": "🐼", "description": "Panda", "unicode": "U+1F43C"}, + {"number": 9, "emoji": "🐓", "description": "Rooster", "unicode": "U+1F413"}, + {"number": 10, "emoji": "🐧", "description": "Penguin", "unicode": "U+1F427"}, + {"number": 11, "emoji": "🐢", "description": "Turtle", "unicode": "U+1F422"}, + {"number": 12, "emoji": "🐟", "description": "Fish", "unicode": "U+1F41F"}, + {"number": 13, "emoji": "🐙", "description": "Octopus", "unicode": "U+1F419"}, + {"number": 14, "emoji": "🦋", "description": "Butterfly", "unicode": "U+1F98B"}, + {"number": 15, "emoji": "🌷", "description": "Flower", "unicode": "U+1F337"}, + {"number": 16, "emoji": "🌳", "description": "Tree", "unicode": "U+1F333"}, + {"number": 17, "emoji": "🌵", "description": "Cactus", "unicode": "U+1F335"}, + {"number": 18, "emoji": "🍄", "description": "Mushroom", "unicode": "U+1F344"}, + {"number": 19, "emoji": "🌏", "description": "Globe", "unicode": "U+1F30F"}, + {"number": 20, "emoji": "🌙", "description": "Moon", "unicode": "U+1F319"}, + {"number": 21, "emoji": "☁️", "description": "Cloud", "unicode": "U+2601U+FE0F"}, + {"number": 22, "emoji": "🔥", "description": "Fire", "unicode": "U+1F525"}, + {"number": 23, "emoji": "🍌", "description": "Banana", "unicode": "U+1F34C"}, + {"number": 24, "emoji": "🍎", "description": "Apple", "unicode": "U+1F34E"}, + {"number": 25, "emoji": "🍓", "description": "Strawberry", "unicode": "U+1F353"}, + {"number": 26, "emoji": "🌽", "description": "Corn", "unicode": "U+1F33D"}, + {"number": 27, "emoji": "🍕", "description": "Pizza", "unicode": "U+1F355"}, + {"number": 28, "emoji": "🎂", "description": "Cake", "unicode": "U+1F382"}, + {"number": 29, "emoji": "❤️", "description": "Heart", "unicode": "U+2764U+FE0F"}, + {"number": 30, "emoji": "😀", "description": "Smiley", "unicode": "U+1F600"}, + {"number": 31, "emoji": "🤖", "description": "Robot", "unicode": "U+1F916"}, + {"number": 32, "emoji": "🎩", "description": "Hat", "unicode": "U+1F3A9"}, + {"number": 33, "emoji": "👓", "description": "Glasses", "unicode": "U+1F453"}, + {"number": 34, "emoji": "🔧", "description": "Spanner", "unicode": "U+1F527"}, + {"number": 35, "emoji": "🎅", "description": "Santa", "unicode": "U+1F385"}, + {"number": 36, "emoji": "👍", "description": "Thumbs Up", "unicode": "U+1F44D"}, + {"number": 37, "emoji": "☂️", "description": "Umbrella", "unicode": "U+2602U+FE0F"}, + {"number": 38, "emoji": "⌛", "description": "Hourglass", "unicode": "U+231B"}, + {"number": 39, "emoji": "⏰", "description": "Clock", "unicode": "U+23F0"}, + {"number": 40, "emoji": "🎁", "description": "Gift", "unicode": "U+1F381"}, + {"number": 41, "emoji": "💡", "description": "Light Bulb", "unicode": "U+1F4A1"}, + {"number": 42, "emoji": "📕", "description": "Book", "unicode": "U+1F4D5"}, + {"number": 43, "emoji": "✏️", "description": "Pencil", "unicode": "U+270FU+FE0F"}, + {"number": 44, "emoji": "📎", "description": "Paperclip", "unicode": "U+1F4CE"}, + {"number": 45, "emoji": "✂️", "description": "Scissors", "unicode": "U+2702U+FE0F"}, + {"number": 46, "emoji": "🔒", "description": "Lock", "unicode": "U+1F512"}, + {"number": 47, "emoji": "🔑", "description": "Key", "unicode": "U+1F511"}, + {"number": 48, "emoji": "🔨", "description": "Hammer", "unicode": "U+1F528"}, + {"number": 49, "emoji": "☎️", "description": "Telephone", "unicode": "U+260EU+FE0F"}, + {"number": 50, "emoji": "🏁", "description": "Flag", "unicode": "U+1F3C1"}, + {"number": 51, "emoji": "🚂", "description": "Train", "unicode": "U+1F682"}, + {"number": 52, "emoji": "🚲", "description": "Bicycle", "unicode": "U+1F6B2"}, + {"number": 53, "emoji": "✈️", "description": "Aeroplane", "unicode": "U+2708U+FE0F"}, + {"number": 54, "emoji": "🚀", "description": "Rocket", "unicode": "U+1F680"}, + {"number": 55, "emoji": "🏆", "description": "Trophy", "unicode": "U+1F3C6"}, + {"number": 56, "emoji": "⚽", "description": "Ball", "unicode": "U+26BD"}, + {"number": 57, "emoji": "🎸", "description": "Guitar", "unicode": "U+1F3B8"}, + {"number": 58, "emoji": "🎺", "description": "Trumpet", "unicode": "U+1F3BA"}, + {"number": 59, "emoji": "🔔", "description": "Bell", "unicode": "U+1F514"}, + {"number": 60, "emoji": "⚓", "description": "Anchor", "unicode": "U+2693"}, + {"number": 61, "emoji": "🎧", "description": "Headphones", "unicode": "U+1F3A7"}, + {"number": 62, "emoji": "📁", "description": "Folder", "unicode": "U+1F4C1"}, + {"number": 63, "emoji": "📌", "description": "Pin", "unicode": "U+1F4CC"} +] diff --git a/event-schemas/examples/m.key.verification.accept b/event-schemas/examples/m.key.verification.accept new file mode 100644 index 00000000..98e89c06 --- /dev/null +++ b/event-schemas/examples/m.key.verification.accept @@ -0,0 +1,12 @@ +{ + "type": "m.key.verification.accept", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1", + "key_agreement_protocol": "curve25519", + "hash": "sha256", + "message_authentication_code": "hkdf-hmac-sha256", + "short_authentication_string": ["decimal", "emoji"], + "commitment": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } +} diff --git a/event-schemas/examples/m.key.verification.cancel b/event-schemas/examples/m.key.verification.cancel new file mode 100644 index 00000000..9d78f67c --- /dev/null +++ b/event-schemas/examples/m.key.verification.cancel @@ -0,0 +1,8 @@ +{ + "type": "m.key.verification.cancel", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "code": "m.user", + "reason": "User rejected the key verification request" + } +} diff --git a/event-schemas/examples/m.key.verification.key b/event-schemas/examples/m.key.verification.key new file mode 100644 index 00000000..608a2ebd --- /dev/null +++ b/event-schemas/examples/m.key.verification.key @@ -0,0 +1,7 @@ +{ + "type": "m.key.verification.key", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "key": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } +} diff --git a/event-schemas/examples/m.key.verification.mac b/event-schemas/examples/m.key.verification.mac new file mode 100644 index 00000000..c77c3a8d --- /dev/null +++ b/event-schemas/examples/m.key.verification.mac @@ -0,0 +1,10 @@ +{ + "type": "m.key.verification.mac", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "keys": "2Wptgo4CwmLo/Y8B8qinxApKaCkBG2fjTWB7AbP5Uy+aIbygsSdLOFzvdDjww8zUVKCmI02eP9xtyJxc/cLiBA", + "mac": { + "ed25519:ABCDEF": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } + } +} diff --git a/event-schemas/examples/m.key.verification.request b/event-schemas/examples/m.key.verification.request new file mode 100644 index 00000000..258471d2 --- /dev/null +++ b/event-schemas/examples/m.key.verification.request @@ -0,0 +1,11 @@ +{ + "type": "m.key.verification.request", + "content": { + "from_device": "AliceDevice2", + "transaction_id": "S0meUniqueAndOpaqueString", + "methods": [ + "m.sas.v1" + ], + "timestamp": 1559598944869 + } +} diff --git a/event-schemas/examples/m.key.verification.start b/event-schemas/examples/m.key.verification.start new file mode 100644 index 00000000..52f16150 --- /dev/null +++ b/event-schemas/examples/m.key.verification.start @@ -0,0 +1,8 @@ +{ + "type": "m.key.verification.start", + "content": { + "from_device": "BobDevice1", + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1" + } +} diff --git a/event-schemas/examples/m.key.verification.start$m.sas.v1 b/event-schemas/examples/m.key.verification.start$m.sas.v1 new file mode 100644 index 00000000..dae1d405 --- /dev/null +++ b/event-schemas/examples/m.key.verification.start$m.sas.v1 @@ -0,0 +1,12 @@ +{ + "type": "m.key.verification.start", + "content": { + "from_device": "BobDevice1", + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1", + "key_agreement_protocols": ["curve25519"], + "hashes": ["sha256"], + "message_authentication_codes": ["hkdf-hmac-sha256"], + "short_authentication_string": ["decimal", "emoji"] + } +} diff --git a/event-schemas/schema/m.key.verification.accept b/event-schemas/schema/m.key.verification.accept new file mode 100644 index 00000000..e52df39e --- /dev/null +++ b/event-schemas/schema/m.key.verification.accept @@ -0,0 +1,64 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Accepts a previously sent ``m.key.verification.start`` messge. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + method: + type: string + enum: ["m.sas.v1"] + description: |- + The verification method to use. Must be ``m.sas.v1``. + key_agreement_protocol: + type: string + description: |- + The key agreement protocol the device is choosing to use, out of the + options in the ``m.key.verification.start`` message. + hash: + type: string + description: |- + The hash method the device is choosing to use, out of the options in + the ``m.key.verification.start`` message. + message_authentication_code: + type: string + description: |- + The message authentication code the device is choosing to use, out of + the options in the ``m.key.verification.start`` message. + short_authentication_string: + type: array + description: |- + The SAS methods both devices involved in the verification process + understand. Must be a subset of the options in the ``m.key.verification.start`` + message. + items: + type: string + enum: ["decimal", "emoji"] + commitment: + type: string + description: |- + The hash (encoded as unpadded base64) of the concatenation of the device's + ephemeral public key (encoded as unpadded base64) and the canonical JSON + representation of the ``m.key.verification.start`` message. + required: + - transaction_id + - method + - key_agreement_protocol + - hash + - message_authentication_code + - short_authentication_string + - commitment + type: object + type: + enum: + - m.key.verification.accept + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.cancel b/event-schemas/schema/m.key.verification.cancel new file mode 100644 index 00000000..36ffc9ea --- /dev/null +++ b/event-schemas/schema/m.key.verification.cancel @@ -0,0 +1,70 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Cancels a key verification process/request. Typically sent as a `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + The opaque identifier for the verification process/request. + reason: + type: string + description: |- + A human readable description of the ``code``. The client should only rely on this + string if it does not understand the ``code``. + code: + type: string + # Note: this is not an enum because we go into detail about the different + # error codes. If we made this an enum, we'd be repeating information. + # Also, we can't put a real bulleted list in here because the HTML2RST parser + # cuts the text at weird points, breaking the list completely. + description: |- + The error code for why the process/request was cancelled by the user. Error + codes should use the Java package naming convention if not in the following + list: + + ``m.user``: The user cancelled the verification. + + ``m.timeout``: The verification process timed out. Verification processes + can define their own timeout parameters. + + ``m.unknown_transaction``: The device does not know about the given transaction + ID. + + ``m.unknown_method``: The device does not know how to handle the requested + method. This should be sent for ``m.key.verification.start`` messages and + messages defined by individual verification processes. + + ``m.unexpected_message``: The device received an unexpected message. Typically + raised when one of the parties is handling the verification out of order. + + ``m.key_mismatch``: The key was not verified. + + ``m.user_mismatch``: The expected user did not match the user verified. + + ``m.invalid_message``: The message received was invalid. + + ``m.accepted``: A ``m.key.verification.request`` was accepted by a different + device. The device receiving this error can ignore the verification request. + + Clients should be careful to avoid error loops. For example, if a device sends + an incorrect message and the client returns ``m.invalid_message`` to which it + gets an unexpected response with ``m.unexpected_message``, the client should not + respond again with ``m.unexpected_message`` to avoid the other device potentially + sending another error response. + + .. The above blank line is important for RST. + required: + - transaction_id + - code + - reason + type: object + type: + enum: + - m.key.verification.cancel + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.key b/event-schemas/schema/m.key.verification.key new file mode 100644 index 00000000..6dc4954b --- /dev/null +++ b/event-schemas/schema/m.key.verification.key @@ -0,0 +1,28 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Sends the ephemeral public key for a device to the partner device. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + key: + type: string + description: |- + The device's ephemeral public key, encoded as unpadded base64. + required: + - transaction_id + - key + type: object + type: + enum: + - m.key.verification.key + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.mac b/event-schemas/schema/m.key.verification.mac new file mode 100644 index 00000000..769ebe15 --- /dev/null +++ b/event-schemas/schema/m.key.verification.mac @@ -0,0 +1,38 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Sends the MAC of a device's key to the partner device. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + mac: + type: object + description: |- + A map of the key ID to the MAC of the key, using the algorithm in the + verification process. The MAC is encoded as unpadded base64. + additionalProperties: + type: string + description: The key's MAC, encoded as unpadded base64. + keys: + type: string + description: |- + The MAC of the comma-separated, sorted, list of key IDs given in the ``mac`` + property, encoded as unpadded base64. + required: + - transaction_id + - mac + - keys + type: object + type: + enum: + - m.key.verification.mac + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.request b/event-schemas/schema/m.key.verification.request new file mode 100644 index 00000000..c9efa14e --- /dev/null +++ b/event-schemas/schema/m.key.verification.request @@ -0,0 +1,43 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Requests a key verification with another user's devices. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the request. + transaction_id: + type: string + description: |- + An opaque identifier for the verification request. Must be unique + with respect to the devices involved. + methods: + type: array + description: |- + The verification methods supported by the sender. + items: + type: string + timestamp: + type: integer + format: int64 + description: |- + The POSIX timestamp in milliseconds for when the request was made. If + the request is in the future by more than 5 minutes or more than 10 + minutes in the past, the message should be ignored by the receiver. + required: + - from_device + - transaction_id + - methods + - timestamp + type: object + type: + enum: + - m.key.verification.request + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.start b/event-schemas/schema/m.key.verification.start new file mode 100644 index 00000000..ad59d6c7 --- /dev/null +++ b/event-schemas/schema/m.key.verification.start @@ -0,0 +1,39 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Begins a key verification process. Typically sent as a `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the process. + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be unique + with respect to the devices involved. Must be the same as the + ``transaction_id`` given in the ``m.key.verification.request`` + if this process is originating from a request. + method: + type: string + description: |- + The verification method to use. + next_method: + type: string + description: |- + Optional method to use to verify the other user's key with. Applicable + when the ``method`` chosen only verifies one user's key. + required: + - from_device + - transaction_id + - method + type: object + type: + enum: + - m.key.verification.start + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.start$m.sas.v1 b/event-schemas/schema/m.key.verification.start$m.sas.v1 new file mode 100644 index 00000000..867ca820 --- /dev/null +++ b/event-schemas/schema/m.key.verification.start$m.sas.v1 @@ -0,0 +1,69 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Begins a SAS key verification process. Typically sent as a `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the process. + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be unique + with respect to the devices involved. Must be the same as the + ``transaction_id`` given in the ``m.key.verification.request`` + if this process is originating from a request. + method: + type: string + enum: ["m.sas.v1"] + description: |- + The verification method to use. Must be ``m.sas.v1``. + key_agreement_protocols: + type: array + description: |- + The key agreement protocols the sending device understands. Must + include at least ``curve25519``. + items: + type: string + hashes: + type: array + description: |- + The hash methods the sending device understands. Must include at least + ``sha256``. + items: + type: string + message_authentication_codes: + type: array + description: |- + The message authentication codes that the sending device understands. + Must include at least ``hkdf-hmac-sha256``. + items: + type: string + short_authentication_string: + type: array + description: |- + The SAS methods the sending device (and the sending device's user) + understands. Must include at least ``decimal``. Optionally can include + ``emoji``. + items: + type: string + enum: ["decimal", "emoji"] + required: + - from_device + - transaction_id + - method + - key_agreement_protocols + - hashes + - message_authentication_codes + - short_authentication_string + type: object + type: + enum: + - m.key.verification.start + type: string +type: object diff --git a/scripts/continuserv/main.go b/scripts/continuserv/main.go index 2ef6fed9..1bd07e6e 100644 --- a/scripts/continuserv/main.go +++ b/scripts/continuserv/main.go @@ -52,7 +52,7 @@ func main() { walker := makeWalker(dir, w) paths := []string{"api", "changelogs", "event-schemas", "scripts", - "specification"} + "specification", "schemas", "data-definitions"} for _, p := range paths { filepath.Walk(path.Join(dir, p), walker) diff --git a/scripts/templating/matrix_templates/sections.py b/scripts/templating/matrix_templates/sections.py index c88959ed..7000916b 100644 --- a/scripts/templating/matrix_templates/sections.py +++ b/scripts/templating/matrix_templates/sections.py @@ -18,6 +18,7 @@ import inspect import json import os import logging +import re logger = logging.getLogger(__name__) @@ -225,3 +226,19 @@ class MatrixSections(Sections): examples=swagger_def['examples'], title_kind=subtitle_title_char) return rendered + + def render_sas_emoji_table(self): + emoji = self.units.get("sas_emoji") + rendered = ".. csv-table::\n" + rendered += " :header: \"Number\", \"Emoji\", \"Unicode\", \"Description\"\n" + rendered += " :widths: 10, 10, 15, 20\n" + rendered += "\n" + for row in emoji: + rendered += " %d, \"%s\", \"``%s``\", \"%s\"\n" % ( + row['number'], + row['emoji'], + row['unicode'], + row['description'], + ) + rendered += "\n" + return rendered diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index ddb65efe..fe3ba5d2 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -59,6 +59,8 @@ TARGETS = os.path.join(matrix_doc_dir, "specification/targets.yaml") ROOM_EVENT = "core-event-schema/room_event.yaml" STATE_EVENT = "core-event-schema/state_event.yaml" +SAS_EMOJI_JSON = os.path.join(matrix_doc_dir, "data-definitions/sas-emoji.json") + logger = logging.getLogger(__name__) # a yaml Loader which loads mappings into OrderedDicts instead of regular @@ -1088,3 +1090,21 @@ class MatrixUnits(Units): "string": git_version, "revision": git_commit } + + def load_sas_emoji(self): + with open(SAS_EMOJI_JSON, 'r', encoding='utf-8') as sas_json: + emoji = json.load(sas_json) + + # Verify the emoji matches the unicode + for c in emoji: + e = c['emoji'] + logger.info("Checking emoji %s (%s)", e, c['description']) + u = re.sub(r'U\+([0-9a-fA-F]+)', lambda m: chr(int(m.group(1), 16)), c['unicode']) + if e != u: + raise Exception("Emoji %s should be %s not %s" % ( + c['description'], + repr(e), + c['unicode'], + )) + + return emoji diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 1fd28e93..4bd12b71 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -396,8 +396,8 @@ recommended that clients provide mechanisms by which the user can see: Ed25519 signing key for each device, again encoded using unpadded Base64. Alice can then meet Bob in person, or contact him via some other trusted -medium, and ask him to read out the Ed25519 key shown on his device. She -compares this with the value shown for his device on her client. +medium, and use `SAS Verification`_ or ask him to read out the Ed25519 key +shown on his device, comparing it to the one shown on Alice's device. Device verification may reach one of several conclusions. For example: @@ -423,6 +423,327 @@ Device verification may reach one of several conclusions. For example: decrypted by such a device. For the Olm protocol, this is documented at https://matrix.org/git/olm/about/docs/signing.rst. + +Key verification framework +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Verifying keys manually by reading out the Ed25519 key is not very user friendly, +and can lead to errors. In order to help mitigate errors, and to make the process +eaiser for users, some verification methods are supported by the specification. +The methods all use a common framework for negotiating the key verification. + +To use this framework, Alice's client would send ``m.key.verification.request`` +events to Bob's devices. All of the ``to_device`` messages sent to Bob MUST have +the same ``transaction_id`` to indicate they are part of the same request. This +allows Bob to reject the request on one device, and have it apply to all of his +devices. Similarly, it allows Bob to process the verification on one device without +having to involve all of his devices. + +When Bob's device receives a ``m.key.verification.request``, it should prompt Bob +to verify keys with Alice using one of the supported methods in the request. If +Bob's device does not understand any of the methods, it should not cancel the request +as one of his other devices may support the request. Instead, Bob's device should +tell Bob that an unsupported method was used for starting key verification. The +prompt for Bob to accept/reject Alice's request (or the unsupported method prompt) +should be automatically dismissed 10 minutes after the ``timestamp`` field or 2 +minutes after Bob's client receives the message, whichever comes first, if Bob +does not interact with the prompt. The prompt should additionally be hidden if +an appropriate ``m.key.verification.cancel`` message is received. + +If Bob rejects the request, Bob's client must send a ``m.key.verification.cancel`` +message to Alice's device. Upon receipt, Alice's device should tell her that Bob +does not want to verify her device and send ``m.key.verification.cancel`` messages +to all of Bob's devices to notify them that the request was rejected. + +If Bob accepts the request, Bob's device starts the key verification process by +sending a ``m.key.verification.start`` message to Alice's device. Upon receipt +of this message, Alice's device should send a ``m.key.verification.cancel`` message +to all of Bob's other devices to indicate the process has been started. The start +message must use the same ``transaction_id`` from the original key verification +request if it is in response to the request. The start message can be sent indepdently +of any request. + +Individual verification methods may add additional steps, events, and properties to +the verification messages. Event types for methods defined in this specification must +be under the ``m.key.verification`` namespace and any other event types must be namespaced +according to the Java package naming convention. + +Any of Alice's or Bob's devices can cancel the key verification request or process +at any time with a ``m.key.verification.cancel`` message to all applicable devices. + +This framework yields the following handshake, assuming both Alice and Bob each have +2 devices, Bob's first device accepts the key verification request, and Alice's second +device initiates the request. Note how Alice's first device is not involved in the +request or verification process. + +:: + + +---------------+ +---------------+ +-------------+ +-------------+ + | AliceDevice1 | | AliceDevice2 | | BobDevice1 | | BobDevice2 | + +---------------+ +---------------+ +-------------+ +-------------+ + | | | | + | | m.key.verification.request | | + | |---------------------------------->| | + | | | | + | | m.key.verification.request | | + | |-------------------------------------------------->| + | | | | + | | m.key.verification.start | | + | |<----------------------------------| | + | | | | + | | m.key.verification.cancel | | + | |-------------------------------------------------->| + | | | | + + +After the handshake, the verification process begins. + +{{m_key_verification_request_event}} + +{{m_key_verification_start_event}} + +{{m_key_verification_cancel_event}} + + +.. _`SAS Verification`: + +Short Authentication String (SAS) verification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +SAS verification is a user-friendly key verification process built off the common +framework outlined above. SAS verification is intended to be a highly interactive +process for users, and as such exposes verfiication methods which are easier for +users to use. + +The verification process is heavily inspired by Phil Zimmerman's ZRTP key agreement +handshake. A key part of key agreement in ZRTP is the hash commitment: the party that +begins the Diffie-Hellman key sharing sends a hash of their part of the Diffie-Hellman +exchange, and does not send their part of the Diffie-Hellman exchange until they have +received the other party's part. Thus an attacker essentially only has one attempt to +attack the Diffie-Hellman exchange, and hence we can verify fewer bits while still +achieving a high degree of security: if we verify n bits, then an attacker has a 1 in +2\ :sup:`n` chance of success. For example, if we verify 40 bits, then an attacker has +a 1 in 1,099,511,627,776 chance (or less than 1 in 1012 chance) of success. A failed +attack would result in a mismatched Short Authentication String, alerting users to the +attack. + +The verification process takes place over `to-device`_ messages in two phases: + +1. Key agreement phase (based on `ZRTP key agreement `_). +#. Key verification phase (based on HMAC). + +The process between Alice and Bob verifying each other would be: + +.. |AlicePublicKey| replace:: :math:`K_{A}^{public}` +.. |AlicePrivateKey| replace:: :math:`K_{A}^{private}` +.. |AliceCurve25519| replace:: :math:`K_{A}^{private}K_{A}^{public}` +.. |BobPublicKey| replace:: :math:`K_{B}^{public}` +.. |BobPrivateKey| replace:: :math:`K_{B}^{private}` +.. |BobCurve25519| replace:: :math:`K_{B}^{private}K_{B}^{public}` +.. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` +.. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` +.. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` + +1. Alice and Bob establish a secure connection, likely meeting in-person. "Secure" + here means that either party cannot be impersonated, not explicit secrecy. +#. Alice and Bob communicate which devices they'd like to verify with each other. +#. Alice selects Bob's device from the device list and begins verification. +#. Alice's client ensures it has a copy of Bob's device key. +#. Alice's device sends Bob's device a ``m.key.verification.start`` message. +#. Bob's device receives the message and selects a key agreement protocol, hash + algorithm, message authentication code, and SAS method supported by Alice's + device. +#. Bob's device ensures it has a copy of Alice's device key. +#. Bob's device creates an ephemeral Curve25519 key pair (|BobCurve25519|), and + calculates the hash (using the chosen algorithm) of the public key |BobPublicKey|. +#. Bob's device replies to Alice's device with a ``m.key.verification.accept`` message. +#. Alice's device receives Bob's message and stores the commitment hash for later use. +#. Alice's device creates an ephemeral Curve25519 key pair (|AliceCurve25519|) and + replies to Bob's device with a ``m.key.verification.key``, sending only the public + key |AlicePublicKey|. +#. Bob's device receives Alice's message and replies with its own ``m.key.verification.key`` + message containing its public key |BobPublicKey|. +#. Alice's device receives Bob's message and verifies the commitment hash from earlier + matches the hash of the key Bob's device just sent and the content of Alice's + ``m.key.verification.start`` message. +#. Both Alice and Bob's devices perform an Elliptic-curve Diffie-Hellman (|AliceBobECDH|), + using the result as the shared secret. +#. Both Alice and Bob's devices display a SAS to their users, which is derived + from the shared key using one of the methods in this section. If multiple SAS + methods are available, clients should allow the users to select a method. +#. Alice and Bob compare the strings shown by their devices, and tell their devices if + they match or not. +#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device + and a comma-separated sorted list of of the key IDs that they wish the other user + to verify. HMAC is defined in RFC 2104, and SHA-256 as the hash function. The key for + the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) + using `the key verification HKDF <#SAS-HKDF>`_. +#. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the + MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does + the same for Bob's device keys and key IDs concurrently with Alice. +#. When the other device receives the ``m.key.verification.mac`` message, the device + calculates the HMAC of its copies of the other device's keys given in the message, + as well as the HMAC of the comma-seperated, sorted, list of key IDs in the message. + The device compares these with the HMAC values given in the message, and if everything + matches then the device keys are verified. + +The wire protocol looks like the following between Alice and Bob's devices:: + + +-------------+ +-----------+ + | AliceDevice | | BobDevice | + +-------------+ +-----------+ + | | + | m.key.verification.start | + |-------------------------------->| + | | + | m.key.verification.accept | + |<--------------------------------| + | | + | m.key.verification.key | + |-------------------------------->| + | | + | m.key.verification.key | + |<--------------------------------| + | | + | m.key.verification.mac | + |-------------------------------->| + | | + | m.key.verification.mac | + |<--------------------------------| + | | + +Error and exception handling +<<<<<<<<<<<<<<<<<<<<<<<<<<<< + +At any point the interactive verfication can go wrong. The following describes what +to do when an error happens: + +* Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` + message must be sent to signify the cancelation. +* The verification can time out. Clients should time out a verification that does not + complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` + which goes unused for 5 minutes after having last sent/received it. The client should + inform the user that the verification timed out, and send an appropriate ``m.key.verification.cancel`` + message to the other device. +* When the same device attempts to intiate multiple verification attempts, cancel all + attempts with that device. +* When a device receives an unknown ``transaction_id``, it should send an appropriate + ``m.key.verfication.cancel`` message to the other device indicating as such. This + does not apply for inbound ``m.key.verification.start`` or ``m.key.verification.cancel`` + messages. +* If the two devices do not share a common key share, hash, HMAC, or SAS method then + the device should notify the other device with an appropriate ``m.key.verification.cancel`` + message. +* If the user claims the Short Authentication Strings do not match, the device should + send an appropriate ``m.key.verification.cancel`` message to the other device. +* If the device receives a message out of sequence or that it was not expecting, it should + notify the other device with an appropriate ``m.key.verification.cancel`` message. + + +Verification messages specific to SAS +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< + +Building off the common framework, the following events are involved in SAS verification. + +The ``m.key.verification.cancel`` event is unchanged, however the following error codes +are used in addition to those already specified: + +* ``m.unknown_method``: The devices are unable to agree on the key agreement, hash, MAC, + or SAS method. +* ``m.mismatched_commitment``: The hash commitment did not match. +* ``m.mismatched_sas``: The SAS did not match. + + +{{m_key_verification_start_m_sas_v1_event}} + +{{m_key_verification_accept_event}} + +{{m_key_verification_key_event}} + +{{m_key_verification_mac_event}} + + +.. _`SAS-HKDF`: + +HKDF calculation +<<<<<<<<<<<<<<<< + +In all of the SAS methods, HKDF is as defined in RFC 5869 and uses the previously +agreed upon hash function for the hash function. The shared secret is supplied +as the input keying material. No salt is used, and the input parameter is the +concatenation of: + + * The string ``MATRIX_KEY_VERIFICATION_SAS``. + * The Matrix ID of the user who sent the ``m.key.verification.start`` message. + * The Device ID of the device which sent the ``m.key.verification.start`` message. + * The Matrix ID of the user who sent the ``m.key.verification.accept`` message. + * The Device ID of the device which sent the ``m.key.verification.accept`` message. + * The ``transaction_id`` being used. + +HKDF is used over the plain shared secret as it results in a harder attack +as well as more uniform data to work with. + +For verification of each party's device keys, HKDF is as defined in RFC 5869 and +uses SHA-256 as the hash function. The shared secret is supplied as the input keying +material. No salt is used, and in the input parameter is the concatenation of: + + * The string ``MATRIX_KEY_VERIFICATION_MAC``. + * The Matrix ID of the user whose key is being MAC-ed. + * The Device ID of the device sending the MAC. + * The Matrix ID of the other user. + * The Device ID of the device receiving the MAC. + * The ``transaction_id`` being used. + * The Key ID of the key being MAC-ed, or the string ``KEY_IDS`` if the item + being MAC-ed is the list of key IDs. + +SAS method: ``decimal`` +<<<<<<<<<<<<<<<<<<<<<<< + +Generate 5 bytes using `HKDF <#SAS-HKDF>`_ then take sequences of 13 bits to +convert to decimal numbers (resulting in 3 numbers between 0 and 8191 inclusive +each). Add 1000 to each calculated number. + +The bitwise operations to get the numbers given the 5 bytes +:math:`B_{0}, B_{1}, B_{2}, B_{3}, B_{4}` would be: + +* First: :math:`(B_{0} \ll 5 | B_{1} \gg 3) + 1000` +* Second: :math:`(B_{1} \& 0x7 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` +* Third: :math:`((B_{3} \& 0x3F) \ll 7 | B_{4} \gg 1) + 1000` + +The digits are displayed to the user either with an appropriate separator, +such as dashes, or with the numbers on individual lines. + +SAS method: ``emoji`` +<<<<<<<<<<<<<<<<<<<<< + +Generate 6 bytes using `HKDF <#SAS-HKDF>`_ then split the first 42 bits into +7 groups of 6 bits, similar to how one would base64 encode something. Convert +each group of 6 bits to a number and use the following table to get the corresponding +emoji: + +{{sas_emoji_table}} + +.. Note:: + This table is available as JSON at + https://github.com/matrix-org/matrix-doc/blob/master/data-definitions/sas-emoji.json + +.. admonition:: Rationale + + The emoji above were chosen to: + + * Be recognisable without colour. + * Be recognisable at a small size. + * Be recognisable by most cultures. + * Be distinguishable from each other. + * Easily described by a few words. + * Avoid symbols with negative connotations. + * Be likely similar across multiple platforms. + +Clients SHOULD show the emoji with the descriptions from the table, or appropriate +translation of those descriptions. Client authors SHOULD collaborate to create a +common set of translations for all languages. + + .. section name changed, so make sure that old links keep working .. _key-sharing: From 37b1e171fc182d3a055190b02f3cfe7c7ea9cd99 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 4 Jun 2019 23:51:10 +0100 Subject: [PATCH 164/282] MSC2076: Enforce key-validity periods when validating event signatures --- proposals/2076-enforce-validity-periods.md | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 proposals/2076-enforce-validity-periods.md diff --git a/proposals/2076-enforce-validity-periods.md b/proposals/2076-enforce-validity-periods.md new file mode 100644 index 00000000..b09aa2f4 --- /dev/null +++ b/proposals/2076-enforce-validity-periods.md @@ -0,0 +1,42 @@ +# MSC2076: Enforce key-validity periods when validating event signatures + +## Background + +The [Federation API +specification](https://matrix.org/docs/spec/server_server/r0.1.1.html#validating-hashes-and-signatures-on-received-events) +specifies that events should be validated via the signature verification +algorithm, but does not specify how the keys for that check should be obtained +and validated. + +In practice, the implementation has been as follows. The receiving server +first requests a copy of the key via the [`GET /_matrix/key/v2/server/` +API](https://matrix.org/docs/spec/server_server/r0.1.1.html#get-matrix-key-v2-server-keyid) +directly from the server which created the signature, or via the [`POST +/_matrix/key/v2/query` API](https://matrix.org/docs/spec/server_server/r0.1.1.html#post-matrix-key-v2-query) +from a trusted key server. Once such a key is obtained, it is then cached +forever. No check is made on the `valid_until_ts` field, and +`minimum_valid_until_ts` is set to zero for calls to `POST +/_matrix/key/v2/query`. + +This is highly unsatisfactory, as it means that, should a key be compromised, +then an attacker can spoof arbitrary events claiming to be from the compromised +server forever, since there is no revocation mechanism. + +## Proposal + +This MSC proposes to enforce the `valid_until_ts` property when validating +event signatures. In particular, the server must ensure that it has a copy of +the key with a `valid_until_ts` at least as large as the `origin_server_ts` of +the event being validated. If it does not have such a copy, it must try to +obtain one via the `GET /_matrix/key/v2/server/` or `POST +/_matrix/key/v2/query` APIs. For the latter, it must set +`minimum_valid_until_ts` to prompt the notary server to attempt to refresh the +key if appropriate. + +Since this changes the rules used to validate events, it will be introduced +with a new room version. This will reduce the risk of divergence between +servers in a room due to some servers accepting events which others reject. + +This MSC also proposes that the current situation - where `valid_until_ts` is +ignored - be formalised for the existing room versions v1-v4, rather than be +left as implementation-specific behaviour. From d2ccd6b268e28437a0a2d70c2fb2d1135fc76757 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 4 Jun 2019 23:53:42 +0100 Subject: [PATCH 165/282] MSC2077: room v5 --- proposals/2077-rooms-v5.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 proposals/2077-rooms-v5.md diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md new file mode 100644 index 00000000..034c118a --- /dev/null +++ b/proposals/2077-rooms-v5.md @@ -0,0 +1,20 @@ +# MSC2077 - Rooms V6 + +This MSC proposes creating a new room version named v5, which will enforce the +signing key `valid_until_ts` timestamps proposed in +[MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). + +## Proposal + +The new room version is called "5". The only difference between v5 and v4 is +that v5 rooms enforce the `valid_until_ts` timestamp on signing keys as +proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). + +It is not yet proposed that servers change the default room version used when +creating new rooms, and it is not yet proposed that servers recommend upgrading +existing rooms to v5. + +## Notes + +See also [MSC2002](./2002-rooms-v4.md), which proposed room v4 but also +mentioned that a v5 was anticipated and gave some context for this change. From 3347a480eb0db602d964073129b7e567de84e7e8 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 5 Jun 2019 00:07:05 +0100 Subject: [PATCH 166/282] fix typo --- proposals/2077-rooms-v5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md index 034c118a..0a7eff2b 100644 --- a/proposals/2077-rooms-v5.md +++ b/proposals/2077-rooms-v5.md @@ -1,4 +1,4 @@ -# MSC2077 - Rooms V6 +# MSC2077 - Rooms V5 This MSC proposes creating a new room version named v5, which will enforce the signing key `valid_until_ts` timestamps proposed in From 40b10f254b5c82e11b56f96260e7e9b4f184d52d Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 5 Jun 2019 07:18:25 +0100 Subject: [PATCH 167/282] clarifications --- proposals/2077-rooms-v5.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md index 0a7eff2b..9f8c1654 100644 --- a/proposals/2077-rooms-v5.md +++ b/proposals/2077-rooms-v5.md @@ -1,18 +1,17 @@ -# MSC2077 - Rooms V5 +# MSC2077 - Room version 5 -This MSC proposes creating a new room version named v5, which will enforce the -signing key `valid_until_ts` timestamps proposed in +This MSC proposes creating room version 5, which will enforce the signing key +`valid_until_ts` timestamps proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). ## Proposal -The new room version is called "5". The only difference between v5 and v4 is +The new room version is called `5`. The only difference between v5 and v4 is that v5 rooms enforce the `valid_until_ts` timestamp on signing keys as proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). -It is not yet proposed that servers change the default room version used when -creating new rooms, and it is not yet proposed that servers recommend upgrading -existing rooms to v5. +It is not yet proposed to change the default room version to v5. Version 5 will +be considered a "stable" version. ## Notes From cf932ad4f817b05ffbaf2464ad3eb1bc6e781d14 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 11:20:28 +0100 Subject: [PATCH 168/282] msc2078 - proposal for homeservers sending passwords reset requests --- proposals/2078-homeserver-password-resets.md | 21 ++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 proposals/2078-homeserver-password-resets.md diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md new file mode 100644 index 00000000..f5523cd2 --- /dev/null +++ b/proposals/2078-homeserver-password-resets.md @@ -0,0 +1,21 @@ +# MSC2078 - Sending Password Reset Emails via the Homeserver + +This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's password if that user has registered a third-party identifier (3PID) with that identity server, due to itself also handling the job of confirming the user's control of that identity. + +The MSC aims to simply clarify that homeservers can take on the responisibility of sending password reset tokens themselves. + +## Proposal + +Currently when a client requests a password reset, they make a call to either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). This request is supplied all the necessary details as well as a `id_server` field containing the address of a trusted identity server which the user has used in the past to bind their 3PID. Understand that it is recommended for the homeserver to only grant the request if the given identity server is in a trusted list. + +The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field, but instead to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can both send password reset tokens (via email or sms), as well as accept requests to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. + +Thus, this proposal really only requests that it be clear that a homeserver does not need to proxy requests to `/requestToken`, and instead can ignore the `id_server` field and carry out the request itself. + +## Tradeoffs + +If homeservers choose to not proxy the request, they will need to implement the ability to send emails and/or sms messages. This is left as a detail for the homeserver implementation. + +## Future Considerations + +At some point we should look into removing the `id_server` field altogether and removing any email/sms message sending from the identity server. This would drastically reduce the amount of trust needed in the identity server and its required ability. This is, however, a good first step. From 4e692735f55f18c86033edb00d61f5d1ce17bc38 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 11:25:26 +0100 Subject: [PATCH 169/282] Update some wording --- proposals/2078-homeserver-password-resets.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index f5523cd2..f1c5928a 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -8,9 +8,9 @@ The MSC aims to simply clarify that homeservers can take on the responisibility Currently when a client requests a password reset, they make a call to either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). This request is supplied all the necessary details as well as a `id_server` field containing the address of a trusted identity server which the user has used in the past to bind their 3PID. Understand that it is recommended for the homeserver to only grant the request if the given identity server is in a trusted list. -The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field, but instead to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can both send password reset tokens (via email or sms), as well as accept requests to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. +The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field. Instead, it asks to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can both send password reset tokens (via email or sms), as well as accept requests to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. -Thus, this proposal really only requests that it be clear that a homeserver does not need to proxy requests to `/requestToken`, and instead can ignore the `id_server` field and carry out the request itself. +Thus, this proposal really only requests that it be clear that a homeserver does not need to proxy requests to `/requestToken`, and instead can ignore the `id_server` field and perform emailing/sms message sending by itself. ## Tradeoffs From c9711acbc5fe231e67c2dbfe15a8c795219a25d5 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 12:52:01 +0100 Subject: [PATCH 170/282] Remove attacker bit --- proposals/2078-homeserver-password-resets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index f1c5928a..8ef560e9 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -1,6 +1,6 @@ # MSC2078 - Sending Password Reset Emails via the Homeserver -This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's password if that user has registered a third-party identifier (3PID) with that identity server, due to itself also handling the job of confirming the user's control of that identity. +This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. The MSC aims to simply clarify that homeservers can take on the responisibility of sending password reset tokens themselves. From 1956f1a916d5fbb512c834740e9e5cb0a027e6e9 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 12:59:58 +0100 Subject: [PATCH 171/282] Revert "Remove attacker bit" This reverts commit c9711acbc5fe231e67c2dbfe15a8c795219a25d5. --- proposals/2078-homeserver-password-resets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 8ef560e9..f1c5928a 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -1,6 +1,6 @@ # MSC2078 - Sending Password Reset Emails via the Homeserver -This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. +This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's password if that user has registered a third-party identifier (3PID) with that identity server, due to itself also handling the job of confirming the user's control of that identity. The MSC aims to simply clarify that homeservers can take on the responisibility of sending password reset tokens themselves. From 085c5667a478fe0f0f64973b3627863d8e656057 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 13:42:23 +0100 Subject: [PATCH 172/282] wrap lines --- proposals/2078-homeserver-password-resets.md | 50 ++++++++++++++++---- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index f1c5928a..e64c8651 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -1,21 +1,51 @@ # MSC2078 - Sending Password Reset Emails via the Homeserver -This MSC proposes removing the current requirement of the identity server to send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's password if that user has registered a third-party identifier (3PID) with that identity server, due to itself also handling the job of confirming the user's control of that identity. - -The MSC aims to simply clarify that homeservers can take on the responisibility of sending password reset tokens themselves. +This MSC proposes removing the current requirement of the identity server to +send password reset tokens, and allows homeservers to implement the +functionality instead. The intention is to put less trust in the identity +server which is currently one of the most centralised components of Matrix. As +it stands, an attacker in control of a identity server can reset a user's +password if that user has registered a third-party identifier (3PID) with that +identity server, due to itself also handling the job of confirming the user's +control of that identity. + +The MSC aims to simply clarify that homeservers can take on the responisibility +of sending password reset tokens themselves. ## Proposal -Currently when a client requests a password reset, they make a call to either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). This request is supplied all the necessary details as well as a `id_server` field containing the address of a trusted identity server which the user has used in the past to bind their 3PID. Understand that it is recommended for the homeserver to only grant the request if the given identity server is in a trusted list. - -The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field. Instead, it asks to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can both send password reset tokens (via email or sms), as well as accept requests to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. - -Thus, this proposal really only requests that it be clear that a homeserver does not need to proxy requests to `/requestToken`, and instead can ignore the `id_server` field and perform emailing/sms message sending by itself. +Currently when a client requests a password reset, they make a call to either +[/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) +or +[/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). +This request is supplied all the necessary details as well as a `id_server` +field containing the address of a trusted identity server which the user has +used in the past to bind their 3PID. Understand that it is recommended for the +homeserver to only grant the request if the given identity server is in a +trusted list. + +The `id_server` field is currently required as the homeserver must know where +to proxy the request to. This MSC proposes not to change the requirements of +this field. Instead, it asks to clarify that the homeserver is allowed to not +proxy the request, but carry it out itself. This would mean the homeserver can +both send password reset tokens (via email or sms), as well as accept requests +to +[/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) +to verify that token. + +Thus, this proposal really only requests that it be clear that a homeserver +does not need to proxy requests to `/requestToken`, and instead can ignore the +`id_server` field and perform emailing/sms message sending by itself. ## Tradeoffs -If homeservers choose to not proxy the request, they will need to implement the ability to send emails and/or sms messages. This is left as a detail for the homeserver implementation. +If homeservers choose to not proxy the request, they will need to implement the +ability to send emails and/or sms messages. This is left as a detail for the +homeserver implementation. ## Future Considerations -At some point we should look into removing the `id_server` field altogether and removing any email/sms message sending from the identity server. This would drastically reduce the amount of trust needed in the identity server and its required ability. This is, however, a good first step. +At some point we should look into removing the `id_server` field altogether and +removing any email/sms message sending from the identity server. This would +drastically reduce the amount of trust needed in the identity server and its +required ability. This is, however, a good first step. From 8cba7adcdf47ed69ece187f472d3237e532dae32 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 13:52:02 +0100 Subject: [PATCH 173/282] Clarify conditions for attack --- proposals/2078-homeserver-password-resets.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index e64c8651..0d2207b0 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -5,8 +5,9 @@ send password reset tokens, and allows homeservers to implement the functionality instead. The intention is to put less trust in the identity server which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's -password if that user has registered a third-party identifier (3PID) with that -identity server, due to itself also handling the job of confirming the user's +password if the identity server is considered trusted by that homeserver, and +the user has registered at least one third-party identifier (3PID). This is due +to the identity server currently handling the job of confirming the user's control of that identity. The MSC aims to simply clarify that homeservers can take on the responisibility From 7e18c5d5a8a6cd392561eb260a4ab66eaa759fad Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 14:38:16 +0100 Subject: [PATCH 174/282] Add new submit_url response field --- proposals/2078-homeserver-password-resets.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 0d2207b0..b780d188 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -21,9 +21,7 @@ or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). This request is supplied all the necessary details as well as a `id_server` field containing the address of a trusted identity server which the user has -used in the past to bind their 3PID. Understand that it is recommended for the -homeserver to only grant the request if the given identity server is in a -trusted list. +used in the past to bind their 3PID. The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of @@ -34,9 +32,11 @@ to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. -Thus, this proposal really only requests that it be clear that a homeserver -does not need to proxy requests to `/requestToken`, and instead can ignore the -`id_server` field and perform emailing/sms message sending by itself. +An additional complication is that in the case of sms, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? + +In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the homeserver has not sent out the entire link (for instance in the case of a short code through sms). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. + +If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a request (either POST or GET, depending on whether it desires a machine- or human-readable response) to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. This data should be submitted as query parameters for `GET` request, and a JSON body for a `POST`. ## Tradeoffs From 8259ae292a010b6453f6a7b4f40609370b80dc1d Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 14:39:36 +0100 Subject: [PATCH 175/282] Capitalise SMS --- proposals/2078-homeserver-password-resets.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index b780d188..1a224fe8 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -27,26 +27,26 @@ The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field. Instead, it asks to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can -both send password reset tokens (via email or sms), as well as accept requests +both send password reset tokens (via email or SMS), as well as accept requests to [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) to verify that token. -An additional complication is that in the case of sms, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? +An additional complication is that in the case of SMS, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? -In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the homeserver has not sent out the entire link (for instance in the case of a short code through sms). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. +In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the homeserver has not sent out the entire link (for instance in the case of a short code through SMS). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a request (either POST or GET, depending on whether it desires a machine- or human-readable response) to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. This data should be submitted as query parameters for `GET` request, and a JSON body for a `POST`. ## Tradeoffs If homeservers choose to not proxy the request, they will need to implement the -ability to send emails and/or sms messages. This is left as a detail for the +ability to send emails and/or SMS messages. This is left as a detail for the homeserver implementation. ## Future Considerations At some point we should look into removing the `id_server` field altogether and -removing any email/sms message sending from the identity server. This would +removing any email/SMS message sending from the identity server. This would drastically reduce the amount of trust needed in the identity server and its required ability. This is, however, a good first step. From 4174b612794b77a35eee0c9ea4cac8bdee40a62a Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 14:50:40 +0100 Subject: [PATCH 176/282] submit_url only if the user has to enter the code somewhere --- proposals/2078-homeserver-password-resets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 1a224fe8..47e12224 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -34,7 +34,7 @@ to verify that token. An additional complication is that in the case of SMS, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? -In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the homeserver has not sent out the entire link (for instance in the case of a short code through SMS). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. +In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the verification message contains a code the user is expected to enter into the client (for instance in the case of a short code through SMS). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a request (either POST or GET, depending on whether it desires a machine- or human-readable response) to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. This data should be submitted as query parameters for `GET` request, and a JSON body for a `POST`. From 6e0af5e64cda74dad65c602e2425a148c35ddda9 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 15:07:06 +0100 Subject: [PATCH 177/282] If no submit_url, just send it to the IS as before --- proposals/2078-homeserver-password-resets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 47e12224..1bb5dbc4 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -34,7 +34,7 @@ to verify that token. An additional complication is that in the case of SMS, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? -In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the verification message contains a code the user is expected to enter into the client (for instance in the case of a short code through SMS). If this field is omitted, the client knows that the link has been sent in its entirety and the verification will be handled out of band. +In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the verification message contains a code the user is expected to enter into the client (for instance in the case of a short code through SMS). If this field is omitted, the client should continue the same behaviour from before, which is to send the token to the identity server directly. This is intended for backwards compatibility with older servers. If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a request (either POST or GET, depending on whether it desires a machine- or human-readable response) to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. This data should be submitted as query parameters for `GET` request, and a JSON body for a `POST`. From 6bb48723e8fe7aa88bf1f5f7f42b1f047b057aa0 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 15:43:57 +0100 Subject: [PATCH 178/282] The HS can send any URL --- proposals/2078-homeserver-password-resets.md | 28 ++++++++++++++------ 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 1bb5dbc4..6f7386d4 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -19,24 +19,36 @@ Currently when a client requests a password reset, they make a call to either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). -This request is supplied all the necessary details as well as a `id_server` -field containing the address of a trusted identity server which the user has -used in the past to bind their 3PID. +This request is supplied all the necessary details as well as an `id_server` +field containing the address of a identity server trusted by the homeserver. The `id_server` field is currently required as the homeserver must know where to proxy the request to. This MSC proposes not to change the requirements of this field. Instead, it asks to clarify that the homeserver is allowed to not proxy the request, but carry it out itself. This would mean the homeserver can both send password reset tokens (via email or SMS), as well as accept requests -to -[/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) +an endpoint (with the same parameters as +[/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken)) to verify that token. -An additional complication is that in the case of SMS, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should it send it to the identity server or the homeserver? Which sent out the code? +An additional complication is that in the case of SMS, a full link to reset +passwords is not sent, but a short code. The client then asks the user to enter +this code, however the client may now not know where to send the code. Should +it send it to the identity server or the homeserver? Which sent out the code? -In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the verification message contains a code the user is expected to enter into the client (for instance in the case of a short code through SMS). If this field is omitted, the client should continue the same behaviour from before, which is to send the token to the identity server directly. This is intended for backwards compatibility with older servers. +In order to combat this problem, the field `submit_url` should be added in the +response from both the email and msisdn variants of the `/requestToken` +Client-Server API, if and only if the verification message contains a code the +user is expected to enter into the client (for instance in the case of a short +code through SMS). If this field is omitted, the client should continue the +same behaviour from before, which is to send the token to the identity server +directly. This is intended for backwards compatibility with older servers. -If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a request (either POST or GET, depending on whether it desires a machine- or human-readable response) to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. This data should be submitted as query parameters for `GET` request, and a JSON body for a `POST`. +If the client receives a response to `/requestToken` with `submit_url`, it +should accept the token from user input, then make a POST request to the +content of `submit_url` with the `sid`, `client_secret` and user-entered token. +This data should be submitted as query parameters for `GET` request, and a JSON +body for a `POST`. ## Tradeoffs From 395acf8e0650a147dbcc7340e48d2d4f21ea20c5 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Wed, 5 Jun 2019 15:58:14 +0100 Subject: [PATCH 179/282] Update proposals/2078-homeserver-password-resets.md Co-Authored-By: Hubert Chathi --- proposals/2078-homeserver-password-resets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 6f7386d4..e92d679e 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -15,7 +15,7 @@ of sending password reset tokens themselves. ## Proposal -Currently when a client requests a password reset, they make a call to either +Currently when a client requests a password reset, it makes a call to either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or [/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). From e49518099de29e8d092b2fbb675fc941fb1124fc Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 17:37:09 +0100 Subject: [PATCH 180/282] Be explicit with request/responses --- proposals/2078-homeserver-password-resets.md | 53 +++++++++++++++++++- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 6f7386d4..0052c48e 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -47,8 +47,57 @@ directly. This is intended for backwards compatibility with older servers. If the client receives a response to `/requestToken` with `submit_url`, it should accept the token from user input, then make a POST request to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. -This data should be submitted as query parameters for `GET` request, and a JSON -body for a `POST`. +`submit_url` can lead to anywhere the homeserver deems necessary for +verification. This data should be submitted as a JSON body. + +An example exchange from the client's perspective is shown below: + +``` +POST https://homeserver.tld/_matrix/client/r0/account/password/email/requestToken + +{ + "client_secret": "monkeys_are_AWESOME", + "email": "alice@homeserver.tld", + "send_attempt": 1, + "id_server": "id.example.com" +} +``` + +If the server responds with a `submit_url` field, it means the client should +collect a token from the user and then submit it to the provided URL. + +``` +{ + "sid": "123abc", + "submit_url": "https://homeserver.tld/path/to/submitToken" +} +``` + +Since a `submit_url` was provided, the client will now collect a token from the +user, say "123456", and then submit that as a POST request to the +`"submit_url"`. + +``` +POST https://homeserver.tld/path/to/submitToken + +{ + "sid": "123abc", + "client_secret": "monkeys_are_AWESOME", + "token": "123456" +} +``` + +The client will then receive an appropriate response: + +``` +{ + "success": true +} +``` + +If the client did not receive a `submit_url` field, they should instead assume +that verification will be completed out of band (e.g. the user clicks a link in +their email and makes the submitToken request with their web browser). ## Tradeoffs From de725c26ccebe93faf556305f4803215f0aab50d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 10:37:22 -0600 Subject: [PATCH 181/282] Add more clarity to the media repo --- specification/modules/content_repo.rst | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/specification/modules/content_repo.rst b/specification/modules/content_repo.rst index 823efb3c..1e3d1866 100644 --- a/specification/modules/content_repo.rst +++ b/specification/modules/content_repo.rst @@ -34,8 +34,6 @@ header. The recommended policy is ``sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';``. -Content in the repository should be treated as bytes as it may be encrypted. - Matrix Content (MXC) URIs ------------------------- @@ -78,12 +76,7 @@ requested dimensions, unless the content being thumbnailed is smaller than the dimensions. When the content is smaller than the requested dimensions, servers should return the original content rather than thumbnail it. -Servers SHOULD pre-calculate or have a list of set dimensions for which they -will thumbnail content at. For example, the server may choose that it will -only create thumbnails sized 96x96 or 512x512. When the client requests a -thumbnail, the server will pick the size which is larger than the requested -dimensions. Servers SHOULD pre-calculate the following thumbnails for uploaded -content, and limit thumbnails to the same sizes: +Servers SHOULD produce thumbnails with the following dimensions and methods: * 32x32, crop * 96x96, crop @@ -97,7 +90,9 @@ In summary: * The server will return an image larger than or equal to the dimensions requested where possible. -Servers MUST NOT upscale thumbnails under any circumstance. +Servers MUST NOT upscale thumbnails under any circumstance. Servers MUST NOT +return a smaller thumbnail than requested, unless the original content makes +that impossible. Security considerations ----------------------- From fc4965f2748a97312cfe9930aa456ba0caea7302 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 10:40:51 -0600 Subject: [PATCH 182/282] Stronger spec words --- api/client-server/registration.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 10b661a3..093bac6b 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -62,8 +62,8 @@ paths: `Relationship between access tokens and devices`_. When registering a guest account, all parameters in the request body - with the exception of ``initial_device_display_name`` are ignored by - the server. The server will pick a ``device_id`` for the account + with the exception of ``initial_device_display_name`` MUST BE ignored + by the server. The server MUST pick a ``device_id`` for the account regardless of input. operationId: register parameters: From d3f21e03605ad1dafb548b0386716103905d6282 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 17:54:18 +0100 Subject: [PATCH 183/282] Address review comments --- proposals/2078-homeserver-password-resets.md | 35 ++++++++++++++------ 1 file changed, 25 insertions(+), 10 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 0052c48e..64916931 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -31,6 +31,17 @@ an endpoint (with the same parameters as [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken)) to verify that token. +Consideration was taken not to make `id_server` and optional field. Let's +assume for a moment that it was optional. Now, a client could send a request to +`/requestToken` omitting the `id_server` field. The homeserver however has +opted to continue proxying `/requestToken` to the identity server, even though +it knows this is potentially insecure. The homeserver now has no idea which +identity server to proxy the request to, and must return a failure to the +client. The client could then make another request with an `id_server`, but +we've now made two requests that ended up in the same outcome, instead of one, +in hopes of saving a very small amount of bandwidth by omitting the field +originally. + An additional complication is that in the case of SMS, a full link to reset passwords is not sent, but a short code. The client then asks the user to enter this code, however the client may now not know where to send the code. Should @@ -40,15 +51,19 @@ In order to combat this problem, the field `submit_url` should be added in the response from both the email and msisdn variants of the `/requestToken` Client-Server API, if and only if the verification message contains a code the user is expected to enter into the client (for instance in the case of a short -code through SMS). If this field is omitted, the client should continue the -same behaviour from before, which is to send the token to the identity server -directly. This is intended for backwards compatibility with older servers. - -If the client receives a response to `/requestToken` with `submit_url`, it -should accept the token from user input, then make a POST request to the -content of `submit_url` with the `sid`, `client_secret` and user-entered token. +code through SMS). It SHOULD be in the form of +`/_matrix/identity/api/v1/validate/{3pid_type}/submitToken`, similar to the +[same endpoint that exists in the Identity-Server +API](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken). +If this field is omitted, the client MUST continue the same behaviour from +before, which is to send the token to the identity server directly. This is +intended for backwards compatibility with older servers. + +If the client receives a response to `/requestToken` with `submit_url`, it MUST +accept the token from user input, then make a POST request to the content of +`submit_url` with the `sid`, `client_secret` and user-entered token. `submit_url` can lead to anywhere the homeserver deems necessary for -verification. This data should be submitted as a JSON body. +verification. This data MUST be submitted as a JSON body. An example exchange from the client's perspective is shown below: @@ -69,7 +84,7 @@ collect a token from the user and then submit it to the provided URL. ``` { "sid": "123abc", - "submit_url": "https://homeserver.tld/path/to/submitToken" + "submit_url": "https://homeserver.tld/_matrix/identity/api/v1/validate/msisdn/submitToken" } ``` @@ -78,7 +93,7 @@ user, say "123456", and then submit that as a POST request to the `"submit_url"`. ``` -POST https://homeserver.tld/path/to/submitToken +POST https://homeserver.tld/_matrix/identity/api/v1/validate/msisdn/submitToken { "sid": "123abc", From 3e23dde341b114117a448261d02d45ed495b3e34 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 5 Jun 2019 21:49:02 +0100 Subject: [PATCH 184/282] Be clear that any 3PID token request can now be done by the hs --- proposals/2078-homeserver-password-resets.md | 28 ++++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index e9e53151..f8dfc53e 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -1,16 +1,22 @@ -# MSC2078 - Sending Password Reset Emails via the Homeserver +# MSC2078 - Sending Third-Party Request Tokens via the Homeserver This MSC proposes removing the current requirement of the identity server to -send password reset tokens, and allows homeservers to implement the -functionality instead. The intention is to put less trust in the identity -server which is currently one of the most centralised components of Matrix. As -it stands, an attacker in control of a identity server can reset a user's -password if the identity server is considered trusted by that homeserver, and -the user has registered at least one third-party identifier (3PID). This is due -to the identity server currently handling the job of confirming the user's -control of that identity. - -The MSC aims to simply clarify that homeservers can take on the responisibility +send third-party request tokens, and allows homeservers to implement the +functionality instead. These request tokens are used to verify the identity of +the request auther as an owner of the third-party identity (3PID). This can be +used for binding a 3PID to an account, or for resetting passwords via email or +SMS. The latter is what this proposal mainly focuses on, but be aware that it +allows for any task that requires requesting a token through a 3PID to be +taken on by the homeserver instead of the identity server. + +The intention is to put less trust in the identity server, which is currently +one of the most centralised components of Matrix. As it stands, an attacker in +control of a identity server can reset a user's password if the identity server +is considered trusted by that homeserver, and the user has registered at least +one 3PID. This is due to the identity server currently handling the job of +confirming the user's control of that identity. + +The MSC aims to simply clarify that homeservers can take on the responsibility of sending password reset tokens themselves. ## Proposal From a6314df44ceb63bd65a9508464e454a3a224d9ff Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 17:03:19 -0600 Subject: [PATCH 185/282] Spec v5 rooms: Key validity Proposals: * [MSC2076](https://github.com/matrix-org/matrix-doc/pull/2076) * [MSC2077](https://github.com/matrix-org/matrix-doc/pull/2077) Implementation references: * https://github.com/matrix-org/synapse/commit/00bf99fa628e173ff14c99e5ffd02e8317ee1656 * https://github.com/matrix-org/synapse/pull/5354 * https://github.com/matrix-org/synapse/pull/5321 No known differences from the proposals are included here - alterations are accidental. --- api/server-server/definitions/keys.yaml | 10 +++- .../newsfragments/2080.clarification | 1 + specification/index.rst | 1 + specification/rooms/v5.rst | 54 +++++++++++++++++++ specification/targets.yaml | 4 ++ 5 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2080.clarification create mode 100644 specification/rooms/v5.rst diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index 06619641..c48c320a 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -94,6 +94,12 @@ properties: type: integer format: int64 description: |- - POSIX timestamp when the list of valid keys should be refreshed. Keys used beyond this - timestamp are no longer valid. + POSIX timestamp when the list of valid keys should be refreshed. This field MUST + be ignored in room versions 1, 2, 3, and 4. Keys used beyond this timestamp MUST + be considered invalid, depending on the `room version specification`_. + + Servers SHOULD use the lesser of this field and 7 days into the future when + determining if a key is valid. This is to avoid a situation where an attacker + publishes a key which is valid for a significant amount of time without a way + for the homeserver owner to revoke it. example: 1052262000000 diff --git a/changelogs/server_server/newsfragments/2080.clarification b/changelogs/server_server/newsfragments/2080.clarification new file mode 100644 index 00000000..c568fa13 --- /dev/null +++ b/changelogs/server_server/newsfragments/2080.clarification @@ -0,0 +1 @@ +Clarify how ``valid_until_ts`` behaves with respect to room version. diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..2e1ffd27 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -494,6 +494,7 @@ The available room versions are: * `Version 2 `_ - **Stable**. Implements State Resolution Version 2. * `Version 3 `_ - **Stable**. Introduces events whose IDs are the event's hash. * `Version 4 `_ - **Stable**. Builds on v3 by using URL-safe base64 for event IDs. +* `Version 5 `_ - **Stable**. Introduces enforcement of signing key validity periods. Specification Versions ---------------------- diff --git a/specification/rooms/v5.rst b/specification/rooms/v5.rst new file mode 100644 index 00000000..9b9fad7e --- /dev/null +++ b/specification/rooms/v5.rst @@ -0,0 +1,54 @@ +.. Copyright 2019 The Matrix.org Foundation C.I.C. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Room Version 5 +============== + +This room version builds on `version 4 `_, enforcing signing key validity +periods for events. + +.. contents:: Table of Contents +.. sectnum:: + + +Client considerations +--------------------- + +There are no specific requirements for clients in this room version. Clients should +be aware of event ID changes in `room version 4 `_, however. + + +Server implementation components +-------------------------------- + +.. WARNING:: + The information contained in this section is strictly for server implementors. + Applications which use the Client-Server API are generally unaffected by the + intricacies contained here. The section above regarding client considerations + is the resource that Client-Server API use cases should reference. + + +Room version 5 uses the same algorithms defined in `room version 4 `_, ensuring +that signing key validity is respected. + +Signing key validity period +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When validating event signatures, servers MUST enforce the ``valid_until_ts`` property +from a key request is at least as large as the ``origin_server_ts`` for the event being +validated. Servers missing a copy of the signing key MUST try to obtain one via the +`GET /_matrix/key/v2/server <../server_server/r0.1.1.html#get-matrix-key-v2-server-keyid>`_ +or `POST /_matrix/key/v2/query <../server_server/r0.1.1.html#post-matrix-key-v2-query>`_ +APIs. When using the ``/query`` endpoint, servers MUST set the ``minimum_valid_until_ts`` +property to prompt the notary server to attempt to refresh the key if appropriate. diff --git a/specification/targets.yaml b/specification/targets.yaml index abcdc240..11d69fc0 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -42,6 +42,10 @@ targets: files: - rooms/v4.rst version_label: v4 + rooms@v5: # this is translated to be rooms/v5.html + files: + - rooms/v5.rst + version_label: v5 appendices: files: - appendices.rst From 7ce1ff21351d61cf9cac88637ac9ac389ee12255 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 20:30:21 -0600 Subject: [PATCH 186/282] Clarify which servers are supposed to sign events Fixes https://github.com/matrix-org/matrix-doc/issues/2074 --- .../server_server/newsfragments/2081.clarification | 1 + specification/server_server_api.rst | 14 +++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2081.clarification diff --git a/changelogs/server_server/newsfragments/2081.clarification b/changelogs/server_server/newsfragments/2081.clarification new file mode 100644 index 00000000..fd291273 --- /dev/null +++ b/changelogs/server_server/newsfragments/2081.clarification @@ -0,0 +1 @@ +Clarify which servers are supposed to sign events. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b765e36a..812f0ffc 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -421,9 +421,8 @@ must ensure that the event: Further details of these checks, and how to handle failures, are described below. -.. TODO: - Flesh this out a bit more, and probably change the doc to group the various - checks in one place, rather than have them spread out. +The `Signing Events <#signing-events>`_ section has more information on which hashes +and signatures are expected on events, and how to calculate them. Definitions @@ -1099,6 +1098,15 @@ originating server, following the algorithm described in `Checking for a signatu Note that this step should succeed whether we have been sent the full event or a redacted copy. +The signatures expected on an event are: + +* The sender's server, unless the invite was created as a result of 3rd party invite. + The sender must already match the 3rd party invite, and the server which actually + sends the event may be a different server. +* For room versions 1 and 2, the server which created the ``event_id``. Other room + versions do not track the ``event_id`` over federation and therefore do not need + a signature from those servers. + If the signature is found to be valid, the expected content hash is calculated as described below. The content hash in the ``hashes`` property of the received event is base64-decoded, and the two are compared for equality. From 80aa5a24dc55433bd95f37cd03dd20fab55c280b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 20:33:09 -0600 Subject: [PATCH 187/282] Make v4 the "default" room version As per [MSC2002](https://github.com/matrix-org/matrix-doc/pull/2002). This was missed in https://github.com/matrix-org/matrix-doc/pull/2019 Fixes https://github.com/matrix-org/matrix-doc/issues/2071 --- specification/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..a6f6f62b 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -485,7 +485,7 @@ some other reason. Versions can switch between stable and unstable periodically for a variety of reasons, including discovered security vulnerabilities and age. Clients should not ask room administrators to upgrade their rooms if the room is -running a stable version. Servers SHOULD use room version 1 as the default room +running a stable version. Servers SHOULD use room version 4 as the default room version when creating new rooms. The available room versions are: From a19eb59f13058afcb6f7b2bb9947a173fb004f5a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 21:10:38 -0600 Subject: [PATCH 188/282] Clarify the key object definition for the key management API Fixes https://github.com/matrix-org/matrix-doc/issues/1907 One too far --- api/client-server/keys.yaml | 99 +++++++++++++------ api/server-server/user_keys.yaml | 37 ++++--- .../newsfragments/2083.clarification | 1 + .../newsfragments/2083.clarification | 1 + 4 files changed, 96 insertions(+), 42 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2083.clarification create mode 100644 changelogs/server_server/newsfragments/2083.clarification diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 718703fd..30056259 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -59,22 +59,41 @@ paths: by the key algorithm. May be absent if no new one-time keys are required. - additionalProperties: - type: - - string - - object - example: - "curve25519:AAAAAQ": "/qyvZvwjiTxGdGU0RCguDCLeR+nmsb3FfNG3/Ve4vU8" - signed_curve25519:AAAAHg: - key: "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs" - signatures: - "@alice:example.com": - ed25519:JLAFKJWSCS: "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" - signed_curve25519:AAAAHQ: - key: "j3fR3HemM16M7CWhoI4Sk5ZsdmdfQHsKL1xuSft6MSw" - signatures: - "@alice:example.com": - ed25519:JLAFKJWSCS: "IQeCEPb9HFk217cU9kw9EOiusC6kMIkoIRnbnfOh5Oc63S1ghgyjShBGpu34blQomoalCyXWyhaaT3MrLZYQAA" + additionalProperties: + type: + - string + - type: object + title: KeyObject + properties: + key: + type: string + description: The key, encoded using unpadded base64. + signatures: + type: object + description: |- + Signature for the device. Mapped from user ID to signature object. + additionalProperties: + type: string + required: ['key', 'signatures'] + example: { + "curve25519:AAAAAQ": "/qyvZvwjiTxGdGU0RCguDCLeR+nmsb3FfNG3/Ve4vU8", + "signed_curve25519:AAAAHg": { + "key": "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs", + "signatures": { + "@alice:example.com": { + "ed25519:JLAFKJWSCS": "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" + } + } + }, + "signed_curve25519:AAAAHQ": { + "key": "j3fR3HemM16M7CWhoI4Sk5ZsdmdfQHsKL1xuSft6MSw", + "signatures": { + "@alice:example.com": { + "ed25519:JLAFKJWSCS": "IQeCEPb9HFk217cU9kw9EOiusC6kMIkoIRnbnfOh5Oc63S1ghgyjShBGpu34blQomoalCyXWyhaaT3MrLZYQAA" + } + } + } + } responses: 200: description: @@ -205,12 +224,12 @@ paths: "@alice:example.com": { "ed25519:JLAFKJWSCS": "dSO80A01XiigH3uBiDVx/EjzaoycHcjq9lfQX0uWsqxl2giMIiSPR8a4d291W1ihKJL/a+myXS367WT6NAIcBA" } - }, + }, "unsigned": { "device_display_name": "Alice's mobile phone" } } - + tags: - End-to-end encryption "/keys/claim": @@ -246,8 +265,9 @@ paths: type: string description: algorithm example: "signed_curve25519" - example: + example: { "@alice:example.com": { "JLAFKJWSCS": "signed_curve25519" } + } required: - one_time_keys responses: @@ -263,7 +283,7 @@ paths: If any remote homeservers could not be reached, they are recorded here. The names of the properties are the names of the unreachable servers. - + If the homeserver could be reached, but the user or device was unknown, no failure is recorded. Instead, the corresponding user or device is missing from the ``one_time_keys`` result. @@ -281,14 +301,37 @@ paths: type: - string - object - example: - "@alice:example.com": - JLAFKJWSCS: - signed_curve25519:AAAAHg: - key: "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs" - signatures: - "@alice:example.com": - ed25519:JLAFKJWSCS: "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" + # XXX: We can't define an actual object here, so we have to hope + # that people will look at the swagger source or can figure it out + # from the other endpoints/example. + # - type: object + # title: KeyObject + # properties: + # key: + # type: string + # description: The key, encoded using unpadded base64. + # signatures: + # type: object + # description: |- + # Signature for the device. Mapped from user ID to signature object. + # additionalProperties: + # type: string + # required: ['key', 'signatures'] + example: { + "@alice:example.com": { + "JLAFKJWSCS": { + "signed_curve25519:AAAAHg": { + "key": "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs", + "signatures": { + "@alice:example.com": { + "ed25519:JLAFKJWSCS": "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" + } + } + } + } + } + } + required: ['one_time_keys'] tags: - End-to-end encryption "/keys/changes": diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 3c59cf81..613948c3 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -77,25 +77,34 @@ paths: additionalProperties: type: - string - - object - required: ['one_time_keys'] - examples: - application/json: { - "one_time_keys": { - "@alice:example.com": { - "JLAFKJWSCS": { - "signed_curve25518:AAAAHg": { - "key": "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs", - "signatures": { - "@alice:example.com": { - "ed25519:JLAFKJWSCS": "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" + - type: object + title: KeyObject + properties: + key: + type: string + description: The key, encoded using unpadded base64. + signatures: + type: object + description: |- + Signature for the device. Mapped from user ID to signature object. + additionalProperties: + type: string + required: ['key', 'signatures'] + example: { + "@alice:example.com": { + "JLAFKJWSCS": { + "signed_curve25519:AAAAHg": { + "key": "zKbLg+NrIjpnagy+pIY6uPL4ZwEG2v+8F9lmgsnlZzs", + "signatures": { + "@alice:example.com": { + "ed25519:JLAFKJWSCS": "FLWxXqGbwrb8SM3Y795eB6OA8bwBcoMZFXBqnTn58AYWZSqiD45tlBVcDa2L7RwdKXebW/VzDlnfVJ+9jok1Bw" + } } } } } } - } - } + required: ['one_time_keys'] "/user/keys/query": post: summary: Download device identity keys. diff --git a/changelogs/client_server/newsfragments/2083.clarification b/changelogs/client_server/newsfragments/2083.clarification new file mode 100644 index 00000000..8083d85d --- /dev/null +++ b/changelogs/client_server/newsfragments/2083.clarification @@ -0,0 +1 @@ +Clarify the key object definition for the key management API. diff --git a/changelogs/server_server/newsfragments/2083.clarification b/changelogs/server_server/newsfragments/2083.clarification new file mode 100644 index 00000000..8083d85d --- /dev/null +++ b/changelogs/server_server/newsfragments/2083.clarification @@ -0,0 +1 @@ +Clarify the key object definition for the key management API. From e115e3439dd84d195496f740d6951531524450ec Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:13:25 -0600 Subject: [PATCH 189/282] Touchups on the s2s swagger definitions Duplicate properties, wrong types, etc. --- api/server-server/leaving.yaml | 1 - api/server-server/query.yaml | 6 +++--- api/server-server/user_devices.yaml | 3 +-- api/server-server/version.yaml | 1 + api/server-server/wellknown.yaml | 1 + 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/api/server-server/leaving.yaml b/api/server-server/leaving.yaml index c088cb5d..e0882fe8 100644 --- a/api/server-server/leaving.yaml +++ b/api/server-server/leaving.yaml @@ -57,7 +57,6 @@ paths: `room version specification`_ for precise event formats. **The response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** - schema: schema: type: object properties: diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml index dc14724c..29826b32 100644 --- a/api/server-server/query.yaml +++ b/api/server-server/query.yaml @@ -81,7 +81,7 @@ paths: servers: type: array description: |- - An array of server names that are likely to hold the given room. This + An array of server names that are likely to hold the given room. This list may or may not include the server answering the query. items: type: string @@ -128,7 +128,7 @@ paths: x-example: "@someone:example.org" - in: query name: field - type: enum + type: string enum: ['displayname', 'avatar_url'] description: |- The field to query. If specified, the server will only return the given field @@ -139,7 +139,7 @@ paths: description: |- The profile for the user. If a ``field`` is specified in the request, only the matching field should be included in the response. If no ``field`` was specified, - the response should include the fields of the user's profile that can be made + the response should include the fields of the user's profile that can be made public, such as the display name and avatar. If the user does not have a particular field set on their profile, the server diff --git a/api/server-server/user_devices.yaml b/api/server-server/user_devices.yaml index 4805deb4..362f9baa 100644 --- a/api/server-server/user_devices.yaml +++ b/api/server-server/user_devices.yaml @@ -42,7 +42,6 @@ paths: description: |- The user ID to retrieve devices for. Must be a user local to the receiving homeserver. - required: true x-example: "@alice:example.org" responses: 200: @@ -82,4 +81,4 @@ paths: description: Optional display name for the device. example: "Alice's Mobile Phone" required: ['device_id', 'keys'] - required: ['user_id', 'stream_id', 'devices'] \ No newline at end of file + required: ['user_id', 'stream_id', 'devices'] diff --git a/api/server-server/version.yaml b/api/server-server/version.yaml index 19975529..929f7b91 100644 --- a/api/server-server/version.yaml +++ b/api/server-server/version.yaml @@ -27,6 +27,7 @@ paths: get: summary: Get the implementation name and version of this homeserver. description: Get the implementation name and version of this homeserver. + operationId: getVersion responses: 200: description: diff --git a/api/server-server/wellknown.yaml b/api/server-server/wellknown.yaml index 75676646..bc390bd5 100644 --- a/api/server-server/wellknown.yaml +++ b/api/server-server/wellknown.yaml @@ -29,6 +29,7 @@ paths: Gets information about the delegated server for server-server communication between Matrix homeservers. Servers should follow 30x redirects, carefully avoiding redirect loops, and use normal X.509 certificate validation. + operationId: getWellKnown responses: 200: description: From 500f3d3bf121f30171422566a46dbbeab3d88608 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:28:57 -0600 Subject: [PATCH 190/282] Clarify that the default s2s transport is JSON over HTTP Fixes https://github.com/matrix-org/matrix-doc/issues/1713 --- .../identity_service/newsfragments/2086.clarification | 1 + specification/client_server_api.rst | 3 ++- specification/server_server_api.rst | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 changelogs/identity_service/newsfragments/2086.clarification diff --git a/changelogs/identity_service/newsfragments/2086.clarification b/changelogs/identity_service/newsfragments/2086.clarification new file mode 100644 index 00000000..7016308b --- /dev/null +++ b/changelogs/identity_service/newsfragments/2086.clarification @@ -0,0 +1 @@ +Clarify that the default transport is JSON over HTTP. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..d5b6491f 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -57,6 +57,8 @@ The following other versions are also available, in reverse chronological order: API Standards ------------- +.. TODO: Move a lot of this to a common area for all specs. + .. TODO Need to specify any HMAC or access_token lifetime/ratcheting tricks We need to specify capability negotiation for extensible transports @@ -82,7 +84,6 @@ names in JSON objects passed over the API also follow this convention. ``/createRoom``. A future version of this specification will address the inconsistency. - Any errors which occur at the Matrix API level MUST return a "standard error response". This is a JSON object which looks like: diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b765e36a..28876e44 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -83,6 +83,17 @@ The following other versions are also available, in reverse chronological order: - `r0.1.1 `_ - `r0.1.0 `_ + +API standards +------------- + +The mandatory baseline for client-server communication in Matrix is exchanging +JSON objects over HTTP APIs. More efficient optional transports will in future +be supported as optional extensions - e.g. a packed binary encoding over +stream-cipher encrypted TCP socket for low-bandwidth/low-roundtrip mobile usage. +For the default HTTP transport, all API calls use a Content-Type of +``application/json``. In addition, all strings MUST be encoded as UTF-8. + Server discovery ---------------- From 8fd5b15594ebe41f15334078507edc956c3bc8fb Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:55:11 -0600 Subject: [PATCH 191/282] Reorganize event structure in c2s spec and clarify event capabilities Fixes https://github.com/matrix-org/matrix-doc/issues/1166 Fixes https://github.com/matrix-org/matrix-doc/issues/1527 Fixes https://github.com/matrix-org/matrix-doc/issues/1827 Note: In order to fix the "state events have the following fields: [no words]" bug (1827) we need to resolve references on common event types. When doing this we ultimately end up with more fields than may be required to explain the section, however this commit alters the section descriptions to just say "these fields" instead of "these additional fields". This is also preferable over trying to get the inheritance reversed in the common event types, as the `/sync` endpoint has a high amount of reliance on partial events definitions. --- .../newsfragments/2087.clarification | 1 + .../schema/core-event-schema/room_event.yaml | 3 +- .../schema/core-event-schema/state_event.yaml | 3 +- scripts/templating/matrix_templates/units.py | 1 + specification/client_server_api.rst | 57 +++++++++++++++ specification/events.rst | 73 ------------------- specification/index.rst | 5 ++ specification/targets.yaml | 1 - 8 files changed, 66 insertions(+), 78 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2087.clarification delete mode 100644 specification/events.rst diff --git a/changelogs/client_server/newsfragments/2087.clarification b/changelogs/client_server/newsfragments/2087.clarification new file mode 100644 index 00000000..1974127d --- /dev/null +++ b/changelogs/client_server/newsfragments/2087.clarification @@ -0,0 +1 @@ +Reorganize information about events into a common section. diff --git a/event-schemas/schema/core-event-schema/room_event.yaml b/event-schemas/schema/core-event-schema/room_event.yaml index 007372a5..231d5c65 100644 --- a/event-schemas/schema/core-event-schema/room_event.yaml +++ b/event-schemas/schema/core-event-schema/room_event.yaml @@ -1,7 +1,6 @@ allOf: - $ref: sync_room_event.yaml -description: In addition to the Event fields, Room Events have the following additional - fields. +description: Room Events have the following fields. properties: room_id: description: |- diff --git a/event-schemas/schema/core-event-schema/state_event.yaml b/event-schemas/schema/core-event-schema/state_event.yaml index 37d4426f..816f925f 100644 --- a/event-schemas/schema/core-event-schema/state_event.yaml +++ b/event-schemas/schema/core-event-schema/state_event.yaml @@ -1,7 +1,6 @@ allOf: - $ref: room_event.yaml - $ref: sync_state_event.yaml -description: In addition to the Room Event fields, State Events have the following - additional fields. +description: State Events have the following fields. title: State Event type: object diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index ddb65efe..d44fb9c8 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -748,6 +748,7 @@ class MatrixUnits(Units): with open(filepath, encoding="utf-8") as f: event_schema = yaml.load(f, OrderedLoader) + event_schema = resolve_references(filepath, event_schema) schema_info = process_data_type( event_schema, diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..80e4a2f4 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1326,6 +1326,63 @@ the event ``type`` key SHOULD follow the Java package naming convention, e.g. ``com.example.myapp.event``. This ensures event types are suitably namespaced for each application and reduces the risk of clashes. +.. Note:: + Events are not limited to the types defined in this specification. New or custom + event types can be created on a whim using the Java package naming convention. + For example, a ``com.example.game.score`` event can be sent by clients and other + clients would receive it through Matrix. + +Note that the structure of these events may be different than those in the +server-server API. + +{{common_event_fields}} + +{{common_room_event_fields}} + +{{common_state_event_fields}} + + +Size limits +~~~~~~~~~~~ + +The complete event MUST NOT be larger than 65535 bytes, when formatted as a +`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_, +including any signatures, and encoded as `Canonical JSON`_. + +There are additional restrictions on sizes per key: + +- ``sender`` MUST NOT exceed 255 bytes (including domain). +- ``room_id`` MUST NOT exceed 255 bytes. +- ``state_key`` MUST NOT exceed 255 bytes. +- ``type`` MUST NOT exceed 255 bytes. +- ``event_id`` MUST NOT exceed 255 bytes. + +Some event types have additional size restrictions which are specified in +the description of the event. Additional keys have no limit other than that +implied by the total 65 KB limit on events. + +Room Events +~~~~~~~~~~~ +.. NOTE:: + This section is a work in progress. + +This specification outlines several standard event types, all of which are +prefixed with ``m.`` + +{{m_room_aliases_event}} + +{{m_room_canonical_alias_event}} + +{{m_room_create_event}} + +{{m_room_join_rules_event}} + +{{m_room_member_event}} + +{{m_room_power_levels_event}} + +{{m_room_redaction_event}} + Syncing ~~~~~~~ diff --git a/specification/events.rst b/specification/events.rst deleted file mode 100644 index c5e4a288..00000000 --- a/specification/events.rst +++ /dev/null @@ -1,73 +0,0 @@ -.. Copyright 2016 OpenMarket Ltd -.. -.. Licensed under the Apache License, Version 2.0 (the "License"); -.. you may not use this file except in compliance with the License. -.. You may obtain a copy of the License at -.. -.. http://www.apache.org/licenses/LICENSE-2.0 -.. -.. Unless required by applicable law or agreed to in writing, software -.. distributed under the License is distributed on an "AS IS" BASIS, -.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -.. See the License for the specific language governing permissions and -.. limitations under the License. - -Event Structure -=============== - -All communication in Matrix is expressed in the form of data objects called -Events. These are the fundamental building blocks common to the client-server, -server-server and application-service APIs, and are described below. - -Note that the structure of these events may be different than those in the -server-server API. - -{{common_event_fields}} - -{{common_room_event_fields}} - -{{common_state_event_fields}} - - -Size limits ------------ - -The complete event MUST NOT be larger than 65535 bytes, when formatted as a -`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_, -including any signatures, and encoded as `Canonical JSON`_. - -There are additional restrictions on sizes per key: - -- ``sender`` MUST NOT exceed 255 bytes (including domain). -- ``room_id`` MUST NOT exceed 255 bytes. -- ``state_key`` MUST NOT exceed 255 bytes. -- ``type`` MUST NOT exceed 255 bytes. -- ``event_id`` MUST NOT exceed 255 bytes. - -Some event types have additional size restrictions which are specified in -the description of the event. Additional keys have no limit other than that -implied by the total 65 KB limit on events. - -Room Events ------------ -.. NOTE:: - This section is a work in progress. - -This specification outlines several standard event types, all of which are -prefixed with ``m.`` - -{{m_room_aliases_event}} - -{{m_room_canonical_alias_event}} - -{{m_room_create_event}} - -{{m_room_join_rules_event}} - -{{m_room_member_event}} - -{{m_room_power_levels_event}} - -{{m_room_redaction_event}} - -.. _`Canonical JSON`: ../appendices.html#canonical-json diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..7aa6a672 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -348,6 +348,11 @@ pushed over federation to the participating servers in a room, currently using full mesh topology. Servers may also request backfill of events over federation from the other servers participating in a room. +.. Note:: + Events are not limited to the types defined in this specification. New or custom + event types can be created on a whim using the Java package naming convention. + For example, a ``com.example.game.score`` event can be sent by clients and other + clients would receive it through Matrix. Room Aliases ++++++++++++ diff --git a/specification/targets.yaml b/specification/targets.yaml index abcdc240..ed3dcee3 100644 --- a/specification/targets.yaml +++ b/specification/targets.yaml @@ -5,7 +5,6 @@ targets: client_server: files: - client_server_api.rst - - { 1: events.rst } - { 1: modules.rst } - { 2: feature_profiles.rst } - { 2: "group:modules" } # reference a group of files From c8a3850598cfa11fbfd1f30005ded4a854eec270 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:06:00 -0600 Subject: [PATCH 192/282] Merge GET/PUT /state/:event_type endpoints Clarifying that the state key is optional, and how that works. Fixes https://github.com/matrix-org/matrix-doc/issues/1182 --- api/client-server/room_state.yaml | 72 ++----------------- api/client-server/rooms.yaml | 49 ++----------- .../newsfragments/2088.clarification | 1 + 3 files changed, 13 insertions(+), 109 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2088.clarification diff --git a/api/client-server/room_state.yaml b/api/client-server/room_state.yaml index bda66eb8..37da0335 100644 --- a/api/client-server/room_state.yaml +++ b/api/client-server/room_state.yaml @@ -31,6 +31,9 @@ paths: put: summary: Send a state event to the given room. description: | + .. For backwards compatibility with older links... + .. _`put-matrix-client-%CLIENT_MAJOR_VERSION%-rooms-roomid-state-eventtype`: + State events can be sent using this endpoint. These events will be overwritten if ````, ```` and ```` all match. @@ -61,7 +64,9 @@ paths: - in: path type: string name: stateKey - description: The state_key for the state to send. Defaults to the empty string. + description: |- + The state_key for the state to send. Defaults to the empty string. When + an empty string, the trailing slash on this endpoint is optional. required: true x-example: "@alice:example.com" - in: body @@ -99,68 +104,3 @@ paths: } tags: - Room participation - "/rooms/{roomId}/state/{eventType}": - put: - summary: Send a state event to the given room. - description: | - State events can be sent using this endpoint. This endpoint is - equivalent to calling `/rooms/{roomId}/state/{eventType}/{stateKey}` - with an empty `stateKey`. Previous state events with matching - `` and ``, and empty ``, will be overwritten. - - Requests to this endpoint **cannot use transaction IDs** - like other ``PUT`` paths because they cannot be differentiated from the - ``state_key``. Furthermore, ``POST`` is unsupported on state paths. - - The body of the request should be the content object of the event; the - fields in this object will vary depending on the type of event. See - `Room Events`_ for the ``m.`` event specification. - operationId: setRoomState - security: - - accessToken: [] - parameters: - - in: path - type: string - name: roomId - description: The room to set the state in - required: true - x-example: "!636q39766251:example.com" - - in: path - type: string - name: eventType - description: The type of event to send. - required: true - x-example: "m.room.name" - - in: body - name: body - schema: - type: object - example: { - "name": "New name for the room" - } - responses: - 200: - description: "An ID for the sent event." - examples: - application/json: { - "event_id": "$YUwRidLecu:example.com" - } - schema: - type: object - properties: - event_id: - type: string - description: |- - A unique identifier for the event. - 403: - description: |- - The sender doesn't have permission to send the event into the room. - schema: - $ref: "definitions/errors/error.yaml" - examples: - application/json: { - "errcode": "M_FORBIDDEN", - "error": "You do not have permission to send the event." - } - tags: - - Room participation diff --git a/api/client-server/rooms.yaml b/api/client-server/rooms.yaml index 377783c6..f29a1860 100644 --- a/api/client-server/rooms.yaml +++ b/api/client-server/rooms.yaml @@ -68,6 +68,9 @@ paths: get: summary: Get the state identified by the type and key. description: |- + .. For backwards compatibility with older links... + .. _`get-matrix-client-%CLIENT_MAJOR_VERSION%-rooms-roomid-state-eventtype`: + Looks up the contents of a state event in a room. If the user is joined to the room then the state is taken from the current state of the room. If the user has left the room then the state is @@ -91,7 +94,9 @@ paths: - in: path type: string name: stateKey - description: The key of the state to look up. + description: |- + The key of the state to look up. Defaults to an empty string. When + an empty string, the trailing slash on this endpoint is optional. required: true x-example: "" responses: @@ -110,48 +115,6 @@ paths: member of the room. tags: - Room participation - "/rooms/{roomId}/state/{eventType}": - get: - summary: Get the state identified by the type, with the empty state key. - description: |- - Looks up the contents of a state event in a room. If the user is - joined to the room then the state is taken from the current - state of the room. If the user has left the room then the state is - taken from the state of the room when they left. - - This looks up the state event with the empty state key. - operationId: getRoomStateByType - security: - - accessToken: [] - parameters: - - in: path - type: string - name: roomId - description: The room to look up the state in. - required: true - x-example: "!636q39766251:example.com" - - in: path - type: string - name: eventType - description: The type of state to look up. - required: true - x-example: "m.room.name" - responses: - 200: - description: The content of the state event. - examples: - application/json: { - "name": "Example room name"} - schema: - type: object - 404: - description: The room has no state with the given type or key. - 403: - description: > - You aren't a member of the room and weren't previously a - member of the room. - tags: - - Room participation "/rooms/{roomId}/state": get: summary: Get all state events in the current state of a room. diff --git a/changelogs/client_server/newsfragments/2088.clarification b/changelogs/client_server/newsfragments/2088.clarification new file mode 100644 index 00000000..ae22d66a --- /dev/null +++ b/changelogs/client_server/newsfragments/2088.clarification @@ -0,0 +1 @@ +De-duplicate ``/state/`` endpoints, clarifying that the ```` is optional. From bbc740197390091f9d9a83fb0a12c79b1a2e620f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:14:33 -0600 Subject: [PATCH 193/282] Clarify when and where CORS headers should be returned Fixes https://github.com/matrix-org/matrix-doc/issues/1736 Fixes https://github.com/matrix-org/matrix-doc/issues/2013 --- .../newsfragments/2089.clarification | 1 + specification/client_server_api.rst | 18 +++++++++++++++--- 2 files changed, 16 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2089.clarification diff --git a/changelogs/client_server/newsfragments/2089.clarification b/changelogs/client_server/newsfragments/2089.clarification new file mode 100644 index 00000000..17405adc --- /dev/null +++ b/changelogs/client_server/newsfragments/2089.clarification @@ -0,0 +1 @@ +Clarify when and where CORS headers should be returned. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..e38c8ac0 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -242,6 +242,9 @@ recommended. {{versions_cs_http_api}} + +.. _`CORS`: + Web Browser Clients ------------------- @@ -250,9 +253,14 @@ web browser or similar environment. In these cases, the homeserver should respon to pre-flight requests and supply Cross-Origin Resource Sharing (CORS) headers on all requests. -When a client approaches the server with a pre-flight (``OPTIONS``) request, the -server should respond with the CORS headers for that route. The recommended CORS -headers to be returned by servers on all requests are: +Servers MUST expect that clients will approach them with ``OPTIONS`` requests, +allowing clients to discover the CORS headers. All endpoints in this specification s +upport the ``OPTIONS`` method, however the server MUST NOT perform any logic defined +for the endpoints when approached with an ``OPTIONS`` request. + +When a client approaches the server with a request, the server should respond with +the CORS headers for that route. The recommended CORS headers to be returned by +servers on all requests are: .. code:: @@ -296,6 +304,10 @@ In this section, the following terms are used with specific meanings: Well-known URI ~~~~~~~~~~~~~~ +.. Note:: + Servers hosting the ``.well-known`` JSON file SHOULD offer CORS headers, as + per the `CORS`_ section in this specification. + The ``.well-known`` method uses a JSON file at a predetermined location to specify parameter values. The flow for this method is as follows: From 79bbb47d9f200c3d99d3c6d8e2f7456cd66ac94f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:18:04 -0600 Subject: [PATCH 194/282] Clarify when authorization and rate-limiting are not applicable Fixes https://github.com/matrix-org/matrix-doc/issues/1971 --- .../newsfragments/2090.clarification | 1 + .../matrix_templates/templates/http-api.tmpl | 18 +++++++++--------- 2 files changed, 10 insertions(+), 9 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2090.clarification diff --git a/changelogs/client_server/newsfragments/2090.clarification b/changelogs/client_server/newsfragments/2090.clarification new file mode 100644 index 00000000..23ab50f7 --- /dev/null +++ b/changelogs/client_server/newsfragments/2090.clarification @@ -0,0 +1 @@ +Clarify when authorization and rate-limiting are not applicable. diff --git a/scripts/templating/matrix_templates/templates/http-api.tmpl b/scripts/templating/matrix_templates/templates/http-api.tmpl index 0b9207d9..74836045 100644 --- a/scripts/templating/matrix_templates/templates/http-api.tmpl +++ b/scripts/templating/matrix_templates/templates/http-api.tmpl @@ -10,13 +10,13 @@ {{endpoint.desc}} -{{":Rate-limited: Yes." if endpoint.rate_limited else "" }} -{{":Requires auth: Yes." if endpoint.requires_auth else "" }} +{{":Rate-limited: Yes." if endpoint.rate_limited else ":Rate-limited: No." }} +{{":Requires auth: Yes." if endpoint.requires_auth else ":Requires auth: No." }} .. class:: httpheaders - + Request format: - + {% if (endpoint.req_param_by_loc | length) %} {{ tables.split_paramtable(endpoint.req_param_by_loc) }} {% if (endpoint.req_body_tables) %} @@ -33,7 +33,7 @@ {% if endpoint.res_headers is not none -%} .. class:: httpheaders - + Response headers: {{ tables.paramtable(endpoint.res_headers.rows) }} @@ -42,7 +42,7 @@ {% if endpoint.res_tables|length > 0 -%} .. class:: httpheaders - + Response format: {% for table in endpoint.res_tables -%} @@ -54,7 +54,7 @@ {% endif -%} .. class:: httpheaders - + Example request: .. code:: http @@ -64,7 +64,7 @@ {% if endpoint.responses|length > 0 -%} .. class:: httpheaders - + Response{{"s" if endpoint.responses|length > 1 else "" }}: {% endif -%} @@ -78,7 +78,7 @@ {% if res["example"] -%} .. class:: httpheaders - + Example .. code:: json From 976f32fcab51fde2de0301a7844c9470a9fd7ab4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:23:23 -0600 Subject: [PATCH 195/282] Clarify that /register must produce valid Matrix User IDs Fixes https://github.com/matrix-org/matrix-doc/issues/1793 --- api/client-server/registration.yaml | 5 ++++- changelogs/client_server/newsfragments/2091.clarification | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2091.clarification diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..6fa99550 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -59,6 +59,9 @@ paths: supplied by the client or generated by the server. The server may invalidate any access token previously associated with that device. See `Relationship between access tokens and devices`_. + + Any user ID returned by this API must conform to the grammar given in the + `Matrix specification <../appendices.html#user-identifiers>`_. operationId: register parameters: - in: query @@ -142,7 +145,7 @@ paths: The fully-qualified Matrix user ID (MXID) that has been registered. Any user ID returned by this API must conform to the grammar given in the - `Matrix specification `_. + `Matrix specification <../appendices.html#user-identifiers>`_. access_token: type: string description: |- diff --git a/changelogs/client_server/newsfragments/2091.clarification b/changelogs/client_server/newsfragments/2091.clarification new file mode 100644 index 00000000..2c4a276e --- /dev/null +++ b/changelogs/client_server/newsfragments/2091.clarification @@ -0,0 +1 @@ +Clarify that ``/register`` must produce valid Matrix User IDs. From e644227f4b607b3438cb37b21616a68e7f007645 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 14:13:20 -0600 Subject: [PATCH 196/282] Clarify that the server shouldn't process retries for UIA --- specification/client_server_api.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 318ac08d..4593311d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -557,9 +557,10 @@ message in the standard format. For example: } If the client has completed all stages of a flow, the homeserver performs the -API call and returns the result as normal. Completed stages cannot be re-tried; -The client must abandon the current session and start over. Homeservers should -treat retries as authentication errors. +API call and returns the result as normal. Completed stages cannot be retried +by clients, therefore servers must return either a 401 response with the completed +stages, or the result of the API call if all stages were completed when a client +retries a stage. Some authentication types may be completed by means other than through the Matrix client, for example, an email confirmation may be completed when the user From afead2eb1bb578d6ca01af3ed3a8f729b852e795 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 14:18:41 -0600 Subject: [PATCH 197/282] Clarify LL in /sync a bit more --- api/client-server/sync.yaml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 4fe22d50..a0d2fd60 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -37,7 +37,13 @@ paths: *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ for more information. Lazy-loading members is only supported on a ``StateFilter`` - for this endpoint. + for this endpoint. When lazy-loading is enabled, servers MUST include the + syncing user's own membership event when they join a room, or when the + full state of rooms is requested. The user's own membership event is eligible + for being considered redudant by the server. When a sync is ``limited``, + the server MUST return membership events for the timeline, even if the + applicable events are not in the response, regardless as to whether or not + they are redundant. operationId: sync security: - accessToken: [] From 9bf0103ef359202026ffce958c243b898a828f9e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 14:36:30 -0600 Subject: [PATCH 198/282] Clarify how many PDUs are in a given transaction object Fixes https://github.com/matrix-org/matrix-doc/issues/2093 --- api/server-server/backfill.yaml | 2 +- .../definitions/single_pdu_transaction.yaml | 32 ++++++++++++++++++ .../unlimited_pdu_transaction.yaml | 33 +++++++++++++++++++ api/server-server/events.yaml | 2 +- .../newsfragments/2095.clarification | 1 + 5 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 api/server-server/definitions/single_pdu_transaction.yaml create mode 100644 api/server-server/definitions/unlimited_pdu_transaction.yaml create mode 100644 changelogs/server_server/newsfragments/2095.clarification diff --git a/api/server-server/backfill.yaml b/api/server-server/backfill.yaml index 0da0e234..2ed6298c 100644 --- a/api/server-server/backfill.yaml +++ b/api/server-server/backfill.yaml @@ -64,7 +64,7 @@ paths: A transaction containing the PDUs that preceded the given event(s), including the given event(s), up to the given limit. schema: - $ref: "definitions/transaction.yaml" + $ref: "definitions/unlimited_pdu_transaction.yaml" "/get_missing_events/{roomId}": post: summary: Retrieves events that the sender is missing diff --git a/api/server-server/definitions/single_pdu_transaction.yaml b/api/server-server/definitions/single_pdu_transaction.yaml new file mode 100644 index 00000000..ff682a44 --- /dev/null +++ b/api/server-server/definitions/single_pdu_transaction.yaml @@ -0,0 +1,32 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: + - $ref: "transaction.yaml" +properties: + pdus: + type: array + description: |- + A single PDU. Note that events have a different format depending on the room + version - check the `room version specification`_ for precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ contained in the transaction. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + properties: [] + example: + $ref: "../examples/minimal_pdu.json" +required: ['origin', 'origin_server_ts', 'pdus'] diff --git a/api/server-server/definitions/unlimited_pdu_transaction.yaml b/api/server-server/definitions/unlimited_pdu_transaction.yaml new file mode 100644 index 00000000..0fc31ee4 --- /dev/null +++ b/api/server-server/definitions/unlimited_pdu_transaction.yaml @@ -0,0 +1,33 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: + - $ref: "transaction.yaml" +properties: + pdus: + type: array + description: |- + List of persistent updates to rooms. Note that events have a different format + depending on the room version - check the `room version specification`_ for + precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ contained in the transaction. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + properties: [] + example: + $ref: "../examples/minimal_pdu.json" +required: ['origin', 'origin_server_ts', 'pdus'] diff --git a/api/server-server/events.yaml b/api/server-server/events.yaml index 1f1a802d..1f8ee537 100644 --- a/api/server-server/events.yaml +++ b/api/server-server/events.yaml @@ -156,4 +156,4 @@ paths: 200: description: A transaction containing a single PDU which is the event requested. schema: - $ref: "definitions/transaction.yaml" + $ref: "definitions/single_pdu_transaction.yaml" diff --git a/changelogs/server_server/newsfragments/2095.clarification b/changelogs/server_server/newsfragments/2095.clarification new file mode 100644 index 00000000..66257e17 --- /dev/null +++ b/changelogs/server_server/newsfragments/2095.clarification @@ -0,0 +1 @@ +Clarify how many PDUs are contained in transaction objects for various endpoints. From f0eb495ceebfced779d54b5fcfab2aabdea35ad2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 15:41:28 -0600 Subject: [PATCH 199/282] Clarify how notification counts are calculated in /sync Fixes https://github.com/matrix-org/matrix-doc/issues/2015 --- api/client-server/sync.yaml | 6 ++++-- changelogs/client_server/newsfragments/2097.clarification | 1 + specification/modules/receipts.rst | 4 +++- 3 files changed, 8 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2097.clarification diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 02fddb84..4514d417 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -167,11 +167,13 @@ paths: this room. allOf: - $ref: "definitions/event_batch.yaml" - "unread_notifications": + unread_notifications: title: Unread Notification Counts type: object description: |- - Counts of unread notifications for this room + Counts of unread notifications for this room. See the + `Receipts module <#module-receipts>`_ for more information + on how these are calculated. properties: highlight_count: title: Highlighted notification count diff --git a/changelogs/client_server/newsfragments/2097.clarification b/changelogs/client_server/newsfragments/2097.clarification new file mode 100644 index 00000000..68d89bcc --- /dev/null +++ b/changelogs/client_server/newsfragments/2097.clarification @@ -0,0 +1 @@ +Clarify how ``unread_notifications`` is calculated. diff --git a/specification/modules/receipts.rst b/specification/modules/receipts.rst index ee2b697a..1745706e 100644 --- a/specification/modules/receipts.rst +++ b/specification/modules/receipts.rst @@ -26,7 +26,9 @@ to a homeserver. To prevent this from becoming a problem, receipts are implement using "up to" markers. This marker indicates that the acknowledgement applies to all events "up to and including" the event specified. For example, marking an event as "read" would indicate that the user had read all events *up to* the -referenced event. +referenced event. Servers MUST decremement the number of pending notifications +for a user if the events are up to or including the read receipt. This is typically +done by adjusting the ``unread_notifications`` value in a ``/sync`` response. Events ------ From b9c9396c111c08a893dda37225b1073d1d2ce4cd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 15:59:54 -0600 Subject: [PATCH 200/282] Specify some of the common concepts for Matrix in the index See https://github.com/matrix-org/matrix-doc/pull/2061 Fixes https://github.com/matrix-org/matrix-doc/issues/1468 Fixes https://github.com/matrix-org/matrix-doc/issues/1528 The section is not referenced by the specifications yet - they do a fairly good job of explaining it over and over. In future, it would be good to point all the references to the index. --- api/server-server/keys_query.yaml | 8 +++-- api/server-server/keys_server.yaml | 2 ++ .../newsfragments/2097.clarification | 1 + specification/index.rst | 36 +++++++++++++++++++ 4 files changed, 44 insertions(+), 3 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2097.clarification diff --git a/api/server-server/keys_query.yaml b/api/server-server/keys_query.yaml index e616915b..4989f7fa 100644 --- a/api/server-server/keys_query.yaml +++ b/api/server-server/keys_query.yaml @@ -44,8 +44,10 @@ paths: type: string description: |- **Deprecated**. Servers should not use this parameter and instead - opt to return all keys, not just the requested one. The key ID to + opt to return all keys, not just the requested one. The key ID to look up. + + When excluded, the trailing slash on this endpoint is optional. required: false x-example: "ed25519:abc123" - in: query @@ -53,7 +55,7 @@ paths: type: integer format: int64 description: |- - A millisecond POSIX timestamp in milliseconds indicating when the returned + A millisecond POSIX timestamp in milliseconds indicating when the returned certificates will need to be valid until to be useful to the requesting server. If not supplied, the current time as determined by the notary server is used. @@ -114,7 +116,7 @@ paths: format: int64 description: |- A millisecond POSIX timestamp in milliseconds indicating when - the returned certificates will need to be valid until to be + the returned certificates will need to be valid until to be useful to the requesting server. If not supplied, the current time as determined by the notary diff --git a/api/server-server/keys_server.yaml b/api/server-server/keys_server.yaml index 69985ab7..465bb294 100644 --- a/api/server-server/keys_server.yaml +++ b/api/server-server/keys_server.yaml @@ -51,6 +51,8 @@ paths: **Deprecated**. Servers should not use this parameter and instead opt to return all keys, not just the requested one. The key ID to look up. + + When excluded, the trailing slash on this endpoint is optional. required: false x-example: "ed25519:abc123" deprecated: true diff --git a/changelogs/server_server/newsfragments/2097.clarification b/changelogs/server_server/newsfragments/2097.clarification new file mode 100644 index 00000000..10dcecb6 --- /dev/null +++ b/changelogs/server_server/newsfragments/2097.clarification @@ -0,0 +1 @@ +Clarify that the trailing slash is optional on ``/keys/*`` endpoints when no key ID is requested. diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..375e19c0 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -425,6 +425,42 @@ dedicated API. The API is symmetrical to managing Profile data. Would it really be overengineered to use the same API for both profile & private user data, but with different ACLs? + +Common concepts +--------------- + +Various things are common throughout all of the Matrix APIs. They are +documented here. + +Trailing slashes on API endpoints +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Unless the endpoint is explicltly specified to have a trailing slash, the +slash is optional. For example, an endpoint specified as ``/_matrix/example/`` +would require a trailing slash, however an endpoint specified as ``/_matrix/example`` +has an optional slash (can be excluded when making requests). + +Namespacing +~~~~~~~~~~~ + +Namespacing helps prevent conflicts between multiple applications and the specification +itself. Where namespacing is used, ``m.`` prefixes are used by the specification to +indicate that the field is controlled by the specification. Custom or non-specified +namespaces used in the wild SHOULD use the Java package naming convention to prevent +conflicts. + +As an example, event types are namespaced under ``m.`` in the specification however +any client can send a custom event type, such as ``com.example.game.score`` without +needing to put the event into the ``m.`` namespace. + +Timestamps +~~~~~~~~~~ + +Unless otherwise stated, timestamps are measured as milliseconds since the Unix epoch. +Throughout the specification this may be referred to as POSIX, Unix, or just "time in +milliseconds". + + .. _`room versions`: Room Versions From cd6b012523c5ea11974617036ec1ee189babdb99 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 17:07:49 -0600 Subject: [PATCH 201/282] Clarify what a "module" is and update feature profiles for clients Fixes https://github.com/matrix-org/matrix-doc/issues/2009 --- .../newsfragments/2098.clarification | 1 + specification/feature_profiles.rst | 36 ++++++++++++++++++- specification/modules.rst | 8 +++++ specification/modules/_template.rst | 2 +- 4 files changed, 45 insertions(+), 2 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2098.clarification diff --git a/changelogs/client_server/newsfragments/2098.clarification b/changelogs/client_server/newsfragments/2098.clarification new file mode 100644 index 00000000..1c8ba3ea --- /dev/null +++ b/changelogs/client_server/newsfragments/2098.clarification @@ -0,0 +1 @@ +Clarify what a "module" is and update feature profiles for clients. diff --git a/specification/feature_profiles.rst b/specification/feature_profiles.rst index c6b8ef4c..bb638380 100644 --- a/specification/feature_profiles.rst +++ b/specification/feature_profiles.rst @@ -1,4 +1,5 @@ .. Copyright 2016 OpenMarket Ltd +.. Copyright 2019 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -32,33 +33,67 @@ Summary Module / Profile Web Mobile Desktop CLI Embedded ===================================== ========== ========== ========== ========== ========== `Instant Messaging`_ Required Required Required Required Optional + `Direct Messaging`_ Required Required Required Required Optional + `Mentions`_ Required Required Required Optional Optional `Presence`_ Required Required Required Required Optional `Push Notifications`_ Optional Required Optional Optional Optional `Receipts`_ Required Required Required Required Optional + `Fully read markers`_ Optional Optional Optional Optional Optional `Typing Notifications`_ Required Required Required Required Optional `VoIP`_ Required Required Required Optional Optional + `Ignoring Users`_ Required Required Required Optional Optional + `Reporting Content`_ Optional Optional Optional Optional Optional `Content Repository`_ Required Required Required Optional Optional `Managing History Visibility`_ Required Required Required Required Optional `Server Side Search`_ Optional Optional Optional Optional Optional + `Room Upgrades`_ Required Required Required Required Optional `Server Administration`_ Optional Optional Optional Optional Optional `Event Context`_ Optional Optional Optional Optional Optional `Third Party Networks`_ Optional Optional Optional Optional Optional + `Send-to-Device Messaging`_ Optional Optional Optional Optional Optional + `Device Management`_ Optional Optional Optional Optional Optional + `End-to-End Encryption`_ Optional Optional Optional Optional Optional + `Guest Accounts`_ Optional Optional Optional Optional Optional + `Room Previews`_ Optional Optional Optional Optional Optional + `Client Config`_ Optional Optional Optional Optional Optional + `SSO Login`_ Optional Optional Optional Optional Optional + `OpenID`_ Optional Optional Optional Optional Optional + `Stickers`_ Optional Optional Optional Optional Optional + `Server ACLs`_ Optional Optional Optional Optional Optional + `Server Notices`_ Optional Optional Optional Optional Optional ===================================== ========== ========== ========== ========== ========== *Please see each module for more details on what clients need to implement.* .. _Instant Messaging: `module:im`_ +.. _Direct Messaging: `module:dm`_ +.. _Mentions: `module:mentions`_ .. _Presence: `module:presence`_ .. _Push Notifications: `module:push`_ .. _Receipts: `module:receipts`_ +.. _Fully read markers: `module:read-markers`_ .. _Typing Notifications: `module:typing`_ .. _VoIP: `module:voip`_ +.. _Ignoring Users: `module:ignore_users`_ +.. _Reporting Content: `module:report_content`_ .. _Content Repository: `module:content`_ .. _Managing History Visibility: `module:history-visibility`_ .. _Server Side Search: `module:search`_ +.. _Room Upgrades: `module:room-upgrades`_ .. _Server Administration: `module:admin`_ .. _Event Context: `module:event-context`_ .. _Third Party Networks: `module:third-party-networks`_ +.. _Send-to-Device Messaging: `module:to_device`_ +.. _Device Management: `module:device-management`_ +.. _End-to-End Encryption: `module:e2e`_ +.. _Guest Accounts: `module:guest-access`_ +.. _Room Previews: `module:room-previews`_ +.. _Client Config: `module:account_data`_ +.. _SSO Login: `module:sso_login`_ +.. _OpenID: `module:openid`_ +.. _Stickers: `module:stickers`_ +.. _Server ACLs: `module:server-acls`_ +.. Server Notices already has a link elsewhere. Clients ------- @@ -110,4 +145,3 @@ This is a client which is typically running on an embedded device such as a kettle, fridge or car. These clients tend to perform a few operations and run in a resource constrained environment. Like embedded applications, they are not intended to be fully-fledged communication systems. - diff --git a/specification/modules.rst b/specification/modules.rst index 36f79cfd..f2269a5a 100644 --- a/specification/modules.rst +++ b/specification/modules.rst @@ -1,4 +1,5 @@ .. Copyright 2016 OpenMarket Ltd +.. Copyright 2019 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -15,3 +16,10 @@ Modules ======= +Modules are parts of the Client-Server API which are not universal to all +endpoints and are accessible to all clients. Modules are strictly defined +within this specification and should not be mistaken for XEP or equivalent +extensions from other protocols - in order for an implementation to be +compliant with the Client-Server specification it MUST support all modules +and supporting specification. The exception being clients, which are governed +by `Feature Profiles <#feature-profiles>`_. diff --git a/specification/modules/_template.rst b/specification/modules/_template.rst index aa4f93db..d1fef7f5 100644 --- a/specification/modules/_template.rst +++ b/specification/modules/_template.rst @@ -15,6 +15,7 @@ Module Heading ============== +.. NOTE: Prefer to identify-modules-with-dashes despite historical examples. .. _module:short-name: A short summary of the module. What features does this module provide? An anchor @@ -67,4 +68,3 @@ This includes privacy leaks: for example leaking presence info. How do misbehaving clients or servers impact this module? This section should always be included, if only to say "we've thought about it but there isn't anything to do here". - From 87d8d970745e107f4494256dcbe475fa2c73eda3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 17:19:08 -0600 Subject: [PATCH 202/282] Remove incomplete and weird sentence --- specification/modules.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/specification/modules.rst b/specification/modules.rst index f2269a5a..6a2564ce 100644 --- a/specification/modules.rst +++ b/specification/modules.rst @@ -17,9 +17,8 @@ Modules ======= Modules are parts of the Client-Server API which are not universal to all -endpoints and are accessible to all clients. Modules are strictly defined -within this specification and should not be mistaken for XEP or equivalent -extensions from other protocols - in order for an implementation to be -compliant with the Client-Server specification it MUST support all modules -and supporting specification. The exception being clients, which are governed -by `Feature Profiles <#feature-profiles>`_. +endpoints. Modules are strictly defined within this specification and +should not be mistaken for XEP or equivalent extensions from other protocols +- in order for an implementation to be compliant with the Client-Server +specification it MUST support all modules and supporting specification. +The exception being clients, which are governed by `Feature Profiles <#feature-profiles>`_. From 49dbb3ee4fa25ff5a2c5b43115471bb32410982b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 17:26:27 -0600 Subject: [PATCH 203/282] Declare Matrix 1.0 in the index Fixes https://github.com/matrix-org/matrix-doc/issues/2044 --- specification/index.rst | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..67d17a11 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -52,14 +52,11 @@ is useful for browsing the Client-Server API. Introduction to the Matrix APIs ------------------------------- -.. WARNING:: - The Matrix specification is still evolving: the APIs are not yet frozen - and this document is in places a work in progress or stale. We have made every - effort to clearly flag areas which are still being finalised. - We're publishing it at this point because it's complete enough to be more than - useful and provide a canonical reference to how Matrix is evolving. Our end - goal is to mirror WHATWG's `Living Standard - `_. +.. Note:: + The Matrix specification is currently at version 1.0 - a milestone indicating that + anything built on top of Matrix can safely rely on the APIs in the specification + not changing drastically. The specification continues to evolve, however, mirroring + something similar to WHATWG's `Living Standard `_. Matrix is a set of open APIs for open-federated Instant Messaging (IM), Voice over IP (VoIP) and Internet of Things (IoT) communication, designed to create From 06ee60f0046b80d7d85f80e1b23d2c2e1147d0cd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 17:32:59 -0600 Subject: [PATCH 204/282] Clarify what compliant implementations are --- specification/modules.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/specification/modules.rst b/specification/modules.rst index 6a2564ce..1bc88445 100644 --- a/specification/modules.rst +++ b/specification/modules.rst @@ -18,7 +18,10 @@ Modules Modules are parts of the Client-Server API which are not universal to all endpoints. Modules are strictly defined within this specification and -should not be mistaken for XEP or equivalent extensions from other protocols -- in order for an implementation to be compliant with the Client-Server -specification it MUST support all modules and supporting specification. -The exception being clients, which are governed by `Feature Profiles <#feature-profiles>`_. +should not be mistaken for experimental extensions or optional features. +A compliant server implementation MUST support all modules and supporting +specification (unless the implementation only targets clients of certain +profiles, in which case only the required modules for those feature profiles +MUST be implemented). A compliant client implementation MUST support all +the required modules and supporting specification for the `Feature Profile <#feature-profiles>`_ +it targets. From 4f665f06e6cbdca0310c2cc5b533b1cdd1cf50d0 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 17:42:37 -0600 Subject: [PATCH 205/282] Clarify what Matrix 1.0 is further And add a table for minimum versions. --- specification/index.rst | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/specification/index.rst b/specification/index.rst index 67d17a11..37c6d682 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -50,13 +50,34 @@ one of the above APIs. The `Matrix Client-Server API Swagger Viewer `_ is useful for browsing the Client-Server API. + +Matrix versions +~~~~~~~~~~~~~~~ + +.. Note:: + As of June 10th 2019, the Matrix specification is considered out of beta - + indicating that all currently released APIs are considered stable and secure + to the best of our knowledge, and the spec should contain the complete + information necessary to develop production-grade implementations of Matrix + without the need for external reference. + +Matrix 1.0 (released June 10th, 2019) consists of the following minimum API +versions: + +======================= ======= +API/Specification Version +======================= ======= +Client-Server API r0.5.0 +Server-Server API r0.1.2 +Application Service API r0.1.1 +Identity Service API r0.1.1 +Push Gateway API r0.1.0 +Room Version v5 +======================= ======= + + Introduction to the Matrix APIs ------------------------------- -.. Note:: - The Matrix specification is currently at version 1.0 - a milestone indicating that - anything built on top of Matrix can safely rely on the APIs in the specification - not changing drastically. The specification continues to evolve, however, mirroring - something similar to WHATWG's `Living Standard `_. Matrix is a set of open APIs for open-federated Instant Messaging (IM), Voice over IP (VoIP) and Internet of Things (IoT) communication, designed to create From ae9abe798ef07de9c48f739b50b01e955725c63b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 23:41:51 -0600 Subject: [PATCH 206/282] Revert signature change for redactable event test The previous signature was calculated on the unredacted event, which means the signature produced was wrong. --- specification/appendices/test_vectors.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 7759fa88..790ed878 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -162,7 +162,7 @@ The event signing algorithm should emit the following signed event: "sender": "@u:domain", "signatures": { "domain": { - "ed25519:1": "4zc79tH2cU6Y+eg4YbbF7KiDOrnwEDjlhTqIKiH4k7L9zD9XCiomD7x9odL9eEwnyy1144QyMBe8O3HK++GHBg" + "ed25519:1": "Wm+VzmOUOz08Ds+0NTWb1d4CZrVsJSikkeRxh6aCcUwu6pNC78FunoD7KNWzqFn241eYHYMGCA5McEiVPdhzBA" } }, "unsigned": { From 7f01346bbad3f0cc31bf2d2fcfde4dd7daa0b2f6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 23:50:26 -0600 Subject: [PATCH 207/282] Provide a more complete example of a "minimally-sized event" Using all the required fields of a v1 event. --- specification/appendices/test_vectors.rst | 25 +++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 790ed878..1de458cc 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -91,11 +91,22 @@ Given the following minimally-sized event: .. code:: json { + "room_id": "!x:domain", + "sender": "@a:domain", "event_id": "$0:domain", "origin": "domain", "origin_server_ts": 1000000, "signatures": {}, + "hashes": {}, "type": "X", + "content": {}, + "prev_events": [ + ["$1:domain", "ExampleHash"] + ], + "auth_events": [ + ["$2", "ExampleHash2"] + ], + "depth": 3, "unsigned": { "age_ts": 1000000 } @@ -106,15 +117,25 @@ The event signing algorithm should emit the following signed event: .. code:: json { + "auth_events": [ + ["$2", "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI"] + ], + "content": {}, + "depth": 3, "event_id": "$0:domain", "hashes": { - "sha256": "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI" + "sha256": "6AaJICN1NJURTtaomDYfJlCPMIU+0gtkwg7qzd8FiJM" }, "origin": "domain", "origin_server_ts": 1000000, + "prev_events": [ + ["$1:domain", "onLKD1bGljeBWQhWZ1kaP9SorVmRQNdN5aM2JYU2n/g"] + ], + "room_id": "!x:domain", + "sender": "@a:domain", "signatures": { "domain": { - "ed25519:1": "JV2dlZUASAefSdywnyCxzykHlyr7xkKGK7IRir1cF8eYsnONrCSb+GRn7aXXstr1UHKvzYjRXPx0001+boD1Ag" + "ed25519:1": "51U0wpKYsaNLTQRbha2v5EGO2cVA6pCtnAKEXguu3j3efCLlmq/53vEfWhsk3tY6gnLsV0YM4Lx2NGZkzmV2Ag" } }, "type": "X", From d7858354f26aa3bb8a6cc58e0b1db1d280eee21e Mon Sep 17 00:00:00 2001 From: Jamie McClymont Date: Fri, 7 Jun 2019 20:54:47 +1200 Subject: [PATCH 208/282] Fix 404s in links from room v1 spec --- specification/rooms/v1.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index 1c7a56c4..e8cbf663 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -293,5 +293,5 @@ Events in version 1 rooms have the following structure: {{definition_ss_pdu}} -.. _`auth events selection`: ../../server_server/r0.1.1.html#auth-events-selection -.. _`Signing Events`: ../../server_server/r0.1.1.html#signing-events +.. _`auth events selection`: ../server_server/r0.1.1.html#auth-events-selection +.. _`Signing Events`: ../server_server/r0.1.1.html#signing-events From 7f65704ebc23c8ebcd8f38b7e5c11172a28e2254 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 7 Jun 2019 12:45:11 +0100 Subject: [PATCH 209/282] Update wording and answer review comments --- proposals/2078-homeserver-password-resets.md | 132 +++++++++++-------- 1 file changed, 75 insertions(+), 57 deletions(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index f8dfc53e..5064be20 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -3,70 +3,76 @@ This MSC proposes removing the current requirement of the identity server to send third-party request tokens, and allows homeservers to implement the functionality instead. These request tokens are used to verify the identity of -the request auther as an owner of the third-party identity (3PID). This can be -used for binding a 3PID to an account, or for resetting passwords via email or -SMS. The latter is what this proposal mainly focuses on, but be aware that it -allows for any task that requires requesting a token through a 3PID to be -taken on by the homeserver instead of the identity server. +the request author as an owner of the third-party ID (3PID). This can be used +for binding a 3PID to an account, or for resetting passwords via email or SMS. +The latter is what this proposal mainly focuses on, but be aware that it allows +for any task that requires requesting a token for a 3PID to be taken on by the +homeserver instead of the identity server. The intention is to put less trust in the identity server, which is currently one of the most centralised components of Matrix. As it stands, an attacker in control of a identity server can reset a user's password if the identity server is considered trusted by that homeserver, and the user has registered at least -one 3PID. This is due to the identity server currently handling the job of -confirming the user's control of that identity. +one 3PID. This is due to the identity server handling the job of confirming the +user's control of that identity. -The MSC aims to simply clarify that homeservers can take on the responsibility -of sending password reset tokens themselves. +The MSC seeks to clarify that homeservers can take on the responsibility of +sending password reset tokens themselves, and a new response field that will +aid homeservers in doing so. -## Proposal +# Background -Currently when a client requests a password reset, it makes a call to either +Currently when a client requests a 3PID token, it makes a call to one of the +`/requestToken` endpoints on the homeserver. For instance, during password +resets, a token is requested from either [/_matrix/client/r0/account/password/email/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-email-requesttoken) or -[/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken). -This request is supplied all the necessary details as well as an `id_server` -field containing the address of a identity server trusted by the homeserver. - -The `id_server` field is currently required as the homeserver must know where -to proxy the request to. This MSC proposes not to change the requirements of -this field. Instead, it asks to clarify that the homeserver is allowed to not -proxy the request, but carry it out itself. This would mean the homeserver can -both send password reset tokens (via email or SMS), as well as accept requests -an endpoint (with the same parameters as +[/_matrix/client/r0/account/password/msisdn/requestToken](https://matrix.org/docs/spec/client_server/r0.4.0.html#post-matrix-client-r0-account-password-msisdn-requesttoken), +depending on the medium of the 3PID. These requests are supplied all the +necessary details as well as an `id_server` field containing the domain address +of a identity server trusted by the homeserver. + +In order to facilitate these requests, the homeserver will simply proxy them to +the identity server. The IS will send out a token via email or sms, the user +will click a link or enter the token into their client, and either the client +or the user's browser will make a request **directly to the identity server** +with the token for verification. The IS then informs the homeserver that +verification was successful. At this point you can likely see that there is +potential for abuse here, so instead Homeservers should be given the option to +stop proxying the request to the identity server, and instead just send and +validate the token themselves. + +## Proposal + +The homeserver should be allowed to either proxy `/requestToken` requests or +handle them itself. Specifically, this means that the homeserver can both send +password reset tokens (via email or SMS), as well as accept requests on an +arbitrary endpoint (with the same parameters as [/_matrix/identity/api/v1/validate/email/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken)) to verify that token. -Consideration was taken not to make `id_server` and optional field. Let's -assume for a moment that it was optional. Now, a client could send a request to -`/requestToken` omitting the `id_server` field. The homeserver however has -opted to continue proxying `/requestToken` to the identity server, even though -it knows this is potentially insecure. The homeserver now has no idea which -identity server to proxy the request to, and must return a failure to the -client. The client could then make another request with an `id_server`, but -we've now made two requests that ended up in the same outcome, instead of one, -in hopes of saving a very small amount of bandwidth by omitting the field -originally. - -An additional complication is that in the case of SMS, a full link to reset -passwords is not sent, but a short code. The client then asks the user to enter -this code, however the client may now not know where to send the code. Should -it send it to the identity server or the homeserver? Which sent out the code? - -In order to combat this problem, the field `submit_url` should be added in the -response from both the email and msisdn variants of the `/requestToken` -Client-Server API, if and only if the verification message contains a code the -user is expected to enter into the client (for instance in the case of a short -code through SMS). It SHOULD be in the form of -`/_matrix/identity/api/v1/validate/{3pid_type}/submitToken`, similar to the -[same endpoint that exists in the Identity-Server -API](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken). -If this field is omitted, the client MUST continue the same behaviour from -before, which is to send the token to the identity server directly. This is -intended for backwards compatibility with older servers. +One additional complication that in the case of SMS, just a code is sent to a +person's phone. This is then given to the client, but the client may not know +where to send the code now, as it doesn't know whether the homeserver or +identity server generated it. + +In order to combat this problem, the field `submit_url` MUST be added in the +response from all of the variants of `/requestToken` in the Client-Server API, +if and only if the verification message contains a code the user is expected to +enter into the client (for instance in the case of a short code through SMS). +This URL is simply where the client should submit this token. The endpoint +should accept the same parameters as +[/_matrix/identity/api/v1/validate/{3pid_type}/submitToken](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-validate-email-submittoken) +in the Identity Service API. The only recommendation to homeserver developers +for this endpoint's path is to not be exactly the same as that of the identity +server, in order to prevent clashes between setups running both an identity +server and homeserver on the same domain. If `submit_url` is omitted, the +client MUST continue the same behaviour from before, which is to send the token +to the identity server directly. This is intended for backwards compatibility +with older servers. If the client receives a response to `/requestToken` with `submit_url`, it MUST -accept the token from user input, then make a POST request to the content of +accept a token from user input, then make a POST request to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. `submit_url` can lead to anywhere the homeserver deems necessary for verification. This data MUST be submitted as a JSON body. @@ -90,7 +96,7 @@ collect a token from the user and then submit it to the provided URL. ``` { "sid": "123abc", - "submit_url": "https://homeserver.tld/_matrix/identity/api/v1/validate/msisdn/submitToken" + "submit_url": "https://homeserver.tld/_homeserver/password_reset/msisdn/submitToken" } ``` @@ -99,7 +105,7 @@ user, say "123456", and then submit that as a POST request to the `"submit_url"`. ``` -POST https://homeserver.tld/_matrix/identity/api/v1/validate/msisdn/submitToken +POST https://homeserver.tld/_homeserver/password_reset/msisdn/submitToken { "sid": "123abc", @@ -120,15 +126,27 @@ If the client did not receive a `submit_url` field, they should instead assume that verification will be completed out of band (e.g. the user clicks a link in their email and makes the submitToken request with their web browser). -## Tradeoffs +## Dismissed Alternatives -If homeservers choose to not proxy the request, they will need to implement the -ability to send emails and/or SMS messages. This is left as a detail for the -homeserver implementation. - -## Future Considerations +Consideration was taken not to make `id_server` an optional field. Let's +assume for a moment that it was optional. Now, a client could send a request to +`/requestToken` omitting the `id_server` field. The homeserver however has +opted to continue proxying `/requestToken` to the identity server, even though +it knows this is potentially insecure. The homeserver now has no idea which +identity server to proxy the request to, and must return a failure to the +client. The client could then make another request with an `id_server`, but +we've now made two requests that ended up in the same outcome, instead of one, +in hopes of saving a very small amount of bandwidth by omitting the field +originally. At some point we should look into removing the `id_server` field altogether and removing any email/SMS message sending from the identity server. This would drastically reduce the amount of trust needed in the identity server and its required ability. This is, however, a good first step. + +## Tradeoffs + +If homeservers choose to not proxy the request, they will need to implement the +ability to send emails and/or SMS messages. This is left as a detail for the +homeserver implementation. + From 45e271c0f7d1a79b37dce2fe9c73d0b8bb742874 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 7 Jun 2019 13:29:22 +0100 Subject: [PATCH 210/282] be super explicit --- proposals/2078-homeserver-password-resets.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/2078-homeserver-password-resets.md b/proposals/2078-homeserver-password-resets.md index 5064be20..8dd7d9c9 100644 --- a/proposals/2078-homeserver-password-resets.md +++ b/proposals/2078-homeserver-password-resets.md @@ -75,7 +75,9 @@ If the client receives a response to `/requestToken` with `submit_url`, it MUST accept a token from user input, then make a POST request to the content of `submit_url` with the `sid`, `client_secret` and user-entered token. `submit_url` can lead to anywhere the homeserver deems necessary for -verification. This data MUST be submitted as a JSON body. +verification. To be clear the content of `id_server` does not matter here, the +client should just submit a POST request to the value of `submit_url`. Additionally +data MUST be submitted as a JSON body. An example exchange from the client's perspective is shown below: From 00fee7463623b38516687d1d4ea5463a5e5e3ec8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 07:40:52 -0600 Subject: [PATCH 211/282] Update example --- specification/appendices/test_vectors.rst | 22 ++++++---------------- 1 file changed, 6 insertions(+), 16 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 1de458cc..05b115db 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -93,19 +93,14 @@ Given the following minimally-sized event: { "room_id": "!x:domain", "sender": "@a:domain", - "event_id": "$0:domain", "origin": "domain", "origin_server_ts": 1000000, "signatures": {}, "hashes": {}, "type": "X", "content": {}, - "prev_events": [ - ["$1:domain", "ExampleHash"] - ], - "auth_events": [ - ["$2", "ExampleHash2"] - ], + "prev_events": [], + "auth_events": [], "depth": 3, "unsigned": { "age_ts": 1000000 @@ -117,25 +112,20 @@ The event signing algorithm should emit the following signed event: .. code:: json { - "auth_events": [ - ["$2", "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI"] - ], + "auth_events": [], "content": {}, "depth": 3, - "event_id": "$0:domain", "hashes": { - "sha256": "6AaJICN1NJURTtaomDYfJlCPMIU+0gtkwg7qzd8FiJM" + "sha256": "5jM4wQpv6lnBo7CLIghJuHdW+s2CMBJPUOGOC89ncos" }, "origin": "domain", "origin_server_ts": 1000000, - "prev_events": [ - ["$1:domain", "onLKD1bGljeBWQhWZ1kaP9SorVmRQNdN5aM2JYU2n/g"] - ], + "prev_events": [], "room_id": "!x:domain", "sender": "@a:domain", "signatures": { "domain": { - "ed25519:1": "51U0wpKYsaNLTQRbha2v5EGO2cVA6pCtnAKEXguu3j3efCLlmq/53vEfWhsk3tY6gnLsV0YM4Lx2NGZkzmV2Ag" + "ed25519:1": "KxwGjPSDEtvnFgU00fwFz+l6d2pJM6XBIaMEn81SXPTRl16AqLAYqfIReFGZlHi5KLjAWbOoMszkwsQma+lYAg" } }, "type": "X", From d49c7fb3b02db22a4275bfb0f147b0b4a0477b6d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:01:17 -0600 Subject: [PATCH 212/282] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- event-schemas/schema/m.key.verification.accept | 2 +- specification/modules/end_to_end_encryption.rst | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/event-schemas/schema/m.key.verification.accept b/event-schemas/schema/m.key.verification.accept index e52df39e..41c59968 100644 --- a/event-schemas/schema/m.key.verification.accept +++ b/event-schemas/schema/m.key.verification.accept @@ -17,7 +17,7 @@ properties: type: string enum: ["m.sas.v1"] description: |- - The verification method to use. Must be ``m.sas.v1``. + The verification method to use. key_agreement_protocol: type: string description: |- diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 4bd12b71..fb3d2ba5 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -536,15 +536,15 @@ The process between Alice and Bob verifying each other would be: .. |AlicePublicKey| replace:: :math:`K_{A}^{public}` .. |AlicePrivateKey| replace:: :math:`K_{A}^{private}` -.. |AliceCurve25519| replace:: :math:`K_{A}^{private}K_{A}^{public}` +.. |AliceCurve25519| replace:: :math:`K_{A}^{private},K_{A}^{public}` .. |BobPublicKey| replace:: :math:`K_{B}^{public}` .. |BobPrivateKey| replace:: :math:`K_{B}^{private}` -.. |BobCurve25519| replace:: :math:`K_{B}^{private}K_{B}^{public}` +.. |BobCurve25519| replace:: :math:`K_{B}^{private},K_{B}^{public}` .. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` .. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` .. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` -1. Alice and Bob establish a secure connection, likely meeting in-person. "Secure" +1. Alice and Bob establish a secure out-of-band connection, such as meeting in-person or a video call. "Secure" here means that either party cannot be impersonated, not explicit secrecy. #. Alice and Bob communicate which devices they'd like to verify with each other. #. Alice selects Bob's device from the device list and begins verification. @@ -573,9 +573,9 @@ The process between Alice and Bob verifying each other would be: methods are available, clients should allow the users to select a method. #. Alice and Bob compare the strings shown by their devices, and tell their devices if they match or not. -#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device +#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify. HMAC is defined in RFC 2104, and SHA-256 as the hash function. The key for + to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). The key for the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the @@ -619,7 +619,7 @@ At any point the interactive verfication can go wrong. The following describes w to do when an error happens: * Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` - message must be sent to signify the cancelation. + message must be sent to signify the cancellation. * The verification can time out. Clients should time out a verification that does not complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` which goes unused for 5 minutes after having last sent/received it. The client should @@ -668,8 +668,8 @@ are used in addition to those already specified: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in RFC 5869 and uses the previously -agreed upon hash function for the hash function. The shared secret is supplied +In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) and uses the previously +agreed-upon hash function for the hash function. The shared secret is supplied as the input keying material. No salt is used, and the input parameter is the concatenation of: From 3877896a4cb232ce760f1556d7a1fc6de001b95f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:10:52 -0600 Subject: [PATCH 213/282] Clarify how we now expect verification to be done --- .../modules/end_to_end_encryption.rst | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index fb3d2ba5..fb320dc4 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -384,20 +384,10 @@ man-in-the-middle. This verification process requires an out-of-band channel: there is no way to do it within Matrix without trusting the administrators of the homeservers. -In Matrix, the basic process for device verification is for Alice to verify -that the public Ed25519 signing key she received via ``/keys/query`` for Bob's -device corresponds to the private key in use by Bob's device. For now, it is -recommended that clients provide mechanisms by which the user can see: - -1. The public part of their device's Ed25519 signing key, encoded using - `unpadded Base64`_. - -2. The list of devices in use for each user in a room, along with the public - Ed25519 signing key for each device, again encoded using unpadded Base64. - -Alice can then meet Bob in person, or contact him via some other trusted -medium, and use `SAS Verification`_ or ask him to read out the Ed25519 key -shown on his device, comparing it to the one shown on Alice's device. +In Matrix, verification works by Alice meeting Bob in person, or contact him +via some other trusted medium, and use `SAS Verification`_ to interactively +verify Bob's devices. Alice and Bob may also read aloud their unpadded base64 +encoded Ed25519 public key, as returned by ``/keys/query``. Device verification may reach one of several conclusions. For example: From 5ec1a50b94cca9a92951b7f3139c4ff76ed6c2b2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:12:09 -0600 Subject: [PATCH 214/282] Linefeeds and other clarifications --- .../modules/end_to_end_encryption.rst | 39 ++++++++++--------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index fb320dc4..0de1328b 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -530,12 +530,12 @@ The process between Alice and Bob verifying each other would be: .. |BobPublicKey| replace:: :math:`K_{B}^{public}` .. |BobPrivateKey| replace:: :math:`K_{B}^{private}` .. |BobCurve25519| replace:: :math:`K_{B}^{private},K_{B}^{public}` -.. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` .. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` .. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` -1. Alice and Bob establish a secure out-of-band connection, such as meeting in-person or a video call. "Secure" - here means that either party cannot be impersonated, not explicit secrecy. +1. Alice and Bob establish a secure out-of-band connection, such as meeting + in-person or a video call. "Secure" here means that either party cannot be + impersonated, not explicit secrecy. #. Alice and Bob communicate which devices they'd like to verify with each other. #. Alice selects Bob's device from the device list and begins verification. #. Alice's client ensures it has a copy of Bob's device key. @@ -565,9 +565,9 @@ The process between Alice and Bob verifying each other would be: they match or not. #. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). The key for - the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) - using `the key verification HKDF <#SAS-HKDF>`_. + to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). + The key for the HMAC is different for each item and is calculated by generating + 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does the same for Bob's device keys and key IDs concurrently with Alice. @@ -611,12 +611,12 @@ to do when an error happens: * Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` message must be sent to signify the cancellation. * The verification can time out. Clients should time out a verification that does not - complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` - which goes unused for 5 minutes after having last sent/received it. The client should - inform the user that the verification timed out, and send an appropriate ``m.key.verification.cancel`` - message to the other device. -* When the same device attempts to intiate multiple verification attempts, cancel all - attempts with that device. + complete within 10 minutes. Additionally, clients should expire a ``transaction_id`` + which goes unused for 10 minutes after having last sent/received it. The client should + inform the user that the verification timed out, and send an appropriate + ``m.key.verification.cancel`` message to the other device. +* When the same device attempts to intiate multiple verification attempts, the receipient + should cancel all attempts with that device. * When a device receives an unknown ``transaction_id``, it should send an appropriate ``m.key.verfication.cancel`` message to the other device indicating as such. This does not apply for inbound ``m.key.verification.start`` or ``m.key.verification.cancel`` @@ -658,10 +658,10 @@ are used in addition to those already specified: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) and uses the previously -agreed-upon hash function for the hash function. The shared secret is supplied -as the input keying material. No salt is used, and the input parameter is the -concatenation of: +In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) +and uses the previously agreed-upon hash function for the hash function. The shared +secret is supplied as the input keying material. No salt is used, and the input +parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_SAS``. * The Matrix ID of the user who sent the ``m.key.verification.start`` message. @@ -670,8 +670,9 @@ concatenation of: * The Device ID of the device which sent the ``m.key.verification.accept`` message. * The ``transaction_id`` being used. -HKDF is used over the plain shared secret as it results in a harder attack -as well as more uniform data to work with. +.. admonition:: Rationale + HKDF is used over the plain shared secret as it results in a harder attack + as well as more uniform data to work with. For verification of each party's device keys, HKDF is as defined in RFC 5869 and uses SHA-256 as the hash function. The shared secret is supplied as the input keying @@ -697,7 +698,7 @@ The bitwise operations to get the numbers given the 5 bytes :math:`B_{0}, B_{1}, B_{2}, B_{3}, B_{4}` would be: * First: :math:`(B_{0} \ll 5 | B_{1} \gg 3) + 1000` -* Second: :math:`(B_{1} \& 0x7 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` +* Second: :math:`((B_{1} \& 0x7) \ll 10 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` * Third: :math:`((B_{3} \& 0x3F) \ll 7 | B_{4} \gg 1) + 1000` The digits are displayed to the user either with an appropriate separator, From 77c4c4b07cb1cfe30d7a3477944eb2229e62ab2f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:27:18 -0600 Subject: [PATCH 215/282] Add general clarity --- api/client-server/definitions/room_event_filter.yaml | 2 +- api/client-server/sync.yaml | 9 +++++---- specification/client_server_api.rst | 3 +++ specification/modules/instant_messaging.rst | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 0659be8e..7045396d 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -25,7 +25,7 @@ allOf: include_redundant_members: type: boolean description: |- - If ``true``, enables redudant membership events. Does not + If ``true``, enables redundant membership events. Does not apply unless ``lazy_load_members`` is ``true``. See `Lazy-loading room members <#lazy-loading-room-members>`_ for more information. Defaults to ``false``. diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index a0d2fd60..ed9d7420 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -40,10 +40,11 @@ paths: for this endpoint. When lazy-loading is enabled, servers MUST include the syncing user's own membership event when they join a room, or when the full state of rooms is requested. The user's own membership event is eligible - for being considered redudant by the server. When a sync is ``limited``, - the server MUST return membership events for the timeline, even if the - applicable events are not in the response, regardless as to whether or not - they are redundant. + for being considered redundant by the server. When a sync is ``limited``, + the server MUST return membership events for events in the gap (from ``since``), + even if the applicable events are not in the response, regardless as to whether + or not they are redundant. ``include_redundant_members`` is ignored for limited + syncs. operationId: sync security: - accessToken: [] diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 740023fb..5290ec20 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1306,6 +1306,9 @@ The current endpoints which support lazy-loading room members are: * |/rooms//messages|_ * |/rooms/{roomId}/context/{eventId}|_ +API endpoints +~~~~~~~~~~~~~ + {{filter_cs_http_api}} Events diff --git a/specification/modules/instant_messaging.rst b/specification/modules/instant_messaging.rst index dd3e9c6c..c514f481 100644 --- a/specification/modules/instant_messaging.rst +++ b/specification/modules/instant_messaging.rst @@ -293,7 +293,7 @@ choose a name: users (`disambiguating them if required`_) and concatenating them. For example, the client may choose to show "Alice, Bob, and Charlie (@charlie:example.org)" as the room name. The client may optionally - limit the number + limit the number of users it uses to generate a room name. #. If there are fewer heroes than ``m.joined_member_count + m.invited_member_count - 1``, and ``m.joined_member_count + m.invited_member_count`` is greater From a0e82018161f82a723e2d57eb3e14eec42d66610 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:27:53 -0600 Subject: [PATCH 216/282] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- api/client-server/event_context.yaml | 2 +- api/client-server/message_pagination.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 327c8f43..0e7fa531 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -35,7 +35,7 @@ paths: after the specified event. This allows clients to get the context surrounding an event. - *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ + *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_ for more information. operationId: getEventContext security: diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 8469eec4..c5d7b8fc 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -34,7 +34,7 @@ paths: This API returns a list of message and state events for a room. It uses pagination query parameters to paginate history in the room. - *Note*: This endpoint supports lazy-loading. See `Filtering <#filtering>`_ + *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_ for more information. operationId: getRoomEvents security: From 360ac0b90088a0b9fcb6876fb9bac847fb3f0038 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:54:21 -0600 Subject: [PATCH 217/282] Update specification/server_server_api.rst Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/server_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 812f0ffc..7acc71e4 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -1100,7 +1100,7 @@ a redacted copy. The signatures expected on an event are: -* The sender's server, unless the invite was created as a result of 3rd party invite. +* The ``sender``'s server, unless the invite was created as a result of 3rd party invite. The sender must already match the 3rd party invite, and the server which actually sends the event may be a different server. * For room versions 1 and 2, the server which created the ``event_id``. Other room From 31481840525620c8ac657d31956cfa9428f8f2b0 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 7 Jun 2019 10:58:43 +0100 Subject: [PATCH 218/282] HS' can send 3PID token. Add a new submit_url field --- api/client-server/administrative_contact.yaml | 116 +++++++++++-- api/client-server/registration.yaml | 162 ++++++++++++++---- 2 files changed, 228 insertions(+), 50 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 33ea9786..d62a4fab 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -132,11 +132,30 @@ paths: } responses: 200: - description: The addition was successful. + description: |- + The addition was successful. + + ``submit_url`` is an optional field containing a URL where the + client must submit a validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` + endpoint. The homeserver will send this token to the user, which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. examples: - application/json: {} + application/json: { + "submit_url": "https://example.org/path/to/submitToken" + } schema: type: object + properties: + submit_url: + type: string + description: |- + An optional URL to submit information to to verify a + third-party identifier. + example: "https://example.org/path/to/submitToken" 403: description: The credentials could not be verified with the identity server. examples: @@ -187,12 +206,14 @@ paths: post: summary: Begins the validation process for an email address for association with the user's account. description: |- - Proxies the Identity Service API ``validate/email/requestToken``, but - first checks that the given email address is **not** already associated - with an account on this homeserver. This API should be used to request - validation tokens when adding an email address to an account. This API's - parameters and response are identical to that of the |/register/email/requestToken|_ - endpoint. + The homeserver should check that the given email address is **not** + already associated with an account on this homeserver. This API should + be used to request validation tokens when adding an email address to an + account. This API's parameters and response are identical to that of + the |/register/email/requestToken|_ endpoint. The homeserver has the + choice of validating the email address itself, or proxying the request + to the ``validate/email/requestToken`` Identity Server API on the + server sent in ``id_server``. operationId: requestTokenTo3PIDEmail parameters: - in: body @@ -212,9 +233,38 @@ paths: required: ['id_server'] responses: 200: - description: An email was sent to the given address. + description: |- + An email was sent to the given address. + Note that this may be an email containing the validation token or + it may be informing the user of an error. + + ``submit_url`` is an optional field containing a URL where the + client must submit a validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` + endpoint. The homeserver will send this token to the user, which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client + must submit a validation token to, with identical + parameters to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" 403: description: |- The homeserver does not allow the third party identifier as a @@ -241,12 +291,14 @@ paths: post: summary: Begins the validation process for a phone number for association with the user's account. description: |- - Proxies the Identity Service API ``validate/msisdn/requestToken``, but - first checks that the given phone number is **not** already associated - with an account on this homeserver. This API should be used to request - validation tokens when adding a phone number to an account. This API's - parameters and response are identical to that of the |/register/msisdn/requestToken|_ - endpoint. + The homeserver should check that the given phone number is **not** + already associated with an account on this homeserver. This API should + be used to request validation tokens when adding a phone number to an + account. This API's parameters and response are identical to that of + the |/register/msisdn/requestToken|_ endpoint. The homeserver has the + choice of validating the phone number itself, or proxying the request + to the ``validate/msisdn/requestToken`` Identity Server API on the + server sent in ``id_server``. operationId: requestTokenTo3PIDMSISDN parameters: - in: body @@ -266,9 +318,37 @@ paths: required: ['id_server'] responses: 200: - description: An SMS message was sent to the given phone number. + description: |- + An SMS message was sent to the given phone number. + + ``submit_url`` is an optional field containing a URL where the + client must submit a validation token to, with identical parameters + to the Identity Service API's ``/validate/msisdn/submitToken`` + endpoint. The homeserver will send this token to the user, which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client + must submit a validation token to, with identical + parameters to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" + 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..00d2e90f 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -204,10 +204,11 @@ paths: post: summary: Begins the validation process for an email to be used during registration. description: |- - Proxies the Identity Service API ``validate/email/requestToken``, but - first checks that the given email address is not already associated - with an account on this homeserver. See the Identity Service API for - further information. + The homeserver should check that the given email address is **not** + already associated with an account on this homeserver. The homeserver + has the choice of validating the email address itself, or proxying the + request to the ``validate/email/requestToken`` Identity Server API on + the server sent in ``id_server``. operationId: requestTokenToRegisterEmail parameters: - in: body @@ -231,8 +232,34 @@ paths: An email has been sent to the specified address. Note that this may be an email containing the validation token or it may be informing the user of an error. + + ``submit_url`` is an optional field containing a URL where the + client must submit a validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` + endpoint. The homeserver will send this token to the user, which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client + must submit a validation token to, with identical + parameters to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" 403: description: The homeserver does not permit the address to be bound. schema: @@ -264,10 +291,11 @@ paths: post: summary: Requests a validation token be sent to the given phone number for the purpose of registering an account description: |- - Proxies the Identity Service API ``validate/msisdn/requestToken``, but - first checks that the given phone number is not already associated - with an account on this homeserver. See the Identity Service API for - further information. + The homeserver should check that the given phone number is **not** + already associated with an account on this homeserver. The homeserver + has the choice of validating the phone number itself, or proxying the + request to the ``validate/msisdn/requestToken`` Identity Server API on + the server sent in ``id_server``. operationId: requestTokenToRegisterMSISDN parameters: - in: body @@ -291,8 +319,34 @@ paths: An SMS message has been sent to the specified phone number. Note that this may be an SMS message containing the validation token or it may be informing the user of an error. + + ``submit_url`` is an optional field containing a URL where the + client must submit a validation token to, with identical parameters + to the Identity Service API's ``/validate/msisdn/submitToken`` + endpoint. The homeserver will send this token to the user, which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client + must submit a validation token to, with identical + parameters to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" 403: description: The homeserver does not permit the address to be bound. schema: @@ -377,16 +431,21 @@ paths: post: summary: Requests a validation token be sent to the given email address for the purpose of resetting a user's password description: |- - Proxies the Identity Service API ``validate/email/requestToken``, but - first checks that the given email address **is** associated with an account - on this homeserver. This API should be used to request - validation tokens when authenticating for the - `account/password` endpoint. This API's parameters and response are - identical to that of the HS API |/register/email/requestToken|_ except that - `M_THREEPID_NOT_FOUND` may be returned if no account matching the - given email address could be found. The server may instead send an - email to the given address prompting the user to create an account. - `M_THREEPID_IN_USE` may not be returned. + The homeserver should check that the given email address **is + associated** with an account on this homeserver. This API should be + used to request validation tokens when authenticating for the + ``account/password`` endpoint. + + This API's parameters and response are identical to that of the HS API + |/register/email/requestToken|_ except that ``M_THREEPID_NOT_FOUND`` + may be returned if no account matching the given email address could be + found. The server may instead send an email to the given address + prompting the user to create an account. ``M_THREEPID_IN_USE`` may not + be returned. + + The homeserver has the choice of validating the email address itself, + or proxying the request to the ``validate/email/requestToken`` Identity + server api on the server sent in ``id_server``. .. |/register/email/requestToken| replace:: ``/register/email/requestToken`` @@ -412,7 +471,24 @@ paths: 200: description: An email was sent to the given address. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client must + submit a validation token to, with identical parameters + to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" 403: description: |- The homeserver does not allow the third party identifier as a @@ -439,16 +515,21 @@ paths: post: summary: Requests a validation token be sent to the given phone number for the purpose of resetting a user's password. description: |- - Proxies the Identity Service API ``validate/msisdn/requestToken``, but - first checks that the given phone number **is** associated with an account - on this homeserver. This API should be used to request - validation tokens when authenticating for the - `account/password` endpoint. This API's parameters and response are - identical to that of the HS API |/register/msisdn/requestToken|_ except that - `M_THREEPID_NOT_FOUND` may be returned if no account matching the - given phone number could be found. The server may instead send an - SMS message to the given address prompting the user to create an account. - `M_THREEPID_IN_USE` may not be returned. + The homeserver should check that the given phone number **is + associated** with an account on this homeserver. This API should be + used to request validation tokens when authenticating for the + ``account/password`` endpoint. + + This API's parameters and response are identical to that of the HS API + |/register/msisdn/requestToken|_ except that ``M_THREEPID_NOT_FOUND`` may + be returned if no account matching the given phone number could be + found. The server may instead send the SMS to the given phone number + prompting the user to create an account. ``M_THREEPID_IN_USE`` may not + be returned. + + The homeserver has the choice of validating the phone number itself, or + proxying the request to the ``validate/msisdn/requestToken`` Identity + server api on the server sent in ``id_server``. .. |/register/msisdn/requestToken| replace:: ``/register/msisdn/requestToken`` @@ -474,7 +555,24 @@ paths: 200: description: An SMS message was sent to the given phone number. schema: - $ref: "../identity/definitions/sid.yaml" + allOf: + - $ref: "../identity/definitions/sid.yaml" + - type: object + properties: + submit_url: + type: string + description: |- + An optional field containing a URL where the client must + submit a validation token to, with identical parameters + to the Identity Service API's + ``/validate/msisdn/submitToken`` endpoint. The homeserver + will send this token to the user, which should then be + prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" 403: description: |- The homeserver does not allow the third party identifier as a From e4339fd68755344527cce85895ac131ee8618e6e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 09:01:14 -0600 Subject: [PATCH 219/282] More clarity --- specification/client_server_api.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 5290ec20..c62740d4 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1276,12 +1276,12 @@ effort to reduce the number of resources used, clients can enable "lazy-loading" for room members. By doing this, servers will only ever send membership events which are relevant to the client. -In terms of filters, this means enabling ``lazy_load_members`` on a ``StateFilter`` -or ``RoomEventFilter``. When enabled, lazy-loading aware endpoints (see below) -will only include membership events for the ``sender`` of events being included -in the response. For example, if a client makes a ``/sync`` request with lazy-loading -enabled, the server will only return membership events for the ``sender`` of events -in the timeline, not all members of a room. +In terms of filters, this means enabling ``lazy_load_members`` on a ``RoomEventFilter`` +(or a ``StateFilter`` in the case of ``/sync`` only). When enabled, lazy-loading +aware endpoints (see below) will only include membership events for the ``sender`` +of events being included in the response. For example, if a client makes a ``/sync`` +request with lazy-loading enabled, the server will only return membership events +for the ``sender`` of events in the timeline, not all members of a room. Repeated calls to lazy-loading aware endpoints will result in redundant membership events being excluded by default. Clients often track which membership events they From 46747e897ecc0b7c4c06e6488a02b5c208c87d3d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 10:32:06 -0600 Subject: [PATCH 220/282] ing --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 0de1328b..62881967 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -384,7 +384,7 @@ man-in-the-middle. This verification process requires an out-of-band channel: there is no way to do it within Matrix without trusting the administrators of the homeservers. -In Matrix, verification works by Alice meeting Bob in person, or contact him +In Matrix, verification works by Alice meeting Bob in person, or contacting him via some other trusted medium, and use `SAS Verification`_ to interactively verify Bob's devices. Alice and Bob may also read aloud their unpadded base64 encoded Ed25519 public key, as returned by ``/keys/query``. From a38af2009f72fc91f3aa4a90a323a050ba8dde06 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 10:37:57 -0600 Subject: [PATCH 221/282] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- api/client-server/message_pagination.yaml | 6 +++--- api/client-server/sync.yaml | 4 ++-- specification/client_server_api.rst | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index c5d7b8fc..3e01437a 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -115,14 +115,14 @@ paths: type: array description: |- A list of state events relevant to showing the ``chunk``. For example, if - lazy-loading members is enabled in the filter then this will contain any - applicable membership events. Servers should be careful to not exclude + ``lazy_load_members`` is enabled in the filter then this will contain any + the membership events for the the senders of events in the ``chunk``. Servers should be careful to not exclude membership events which are older than ones already sent to the client. Likewise, clients should be cautious and avoid using older membership events as the current membership event when paginating backwards. Unless ``include_redundant_members`` is ``true``, the server should remove - redundant members which would have already been sent to clients in prior calls + membership events which would have already been sent to clients in prior calls to lazy-loading aware endpoints with the same filter. items: type: object diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index ed9d7420..45829f68 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -41,8 +41,8 @@ paths: syncing user's own membership event when they join a room, or when the full state of rooms is requested. The user's own membership event is eligible for being considered redundant by the server. When a sync is ``limited``, - the server MUST return membership events for events in the gap (from ``since``), - even if the applicable events are not in the response, regardless as to whether + the server MUST return membership events for events in the gap (between ``since`` and the start of the returned timeline), + regardless as to whether or not they are redundant. ``include_redundant_members`` is ignored for limited syncs. operationId: sync diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index c62740d4..42f34849 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1286,7 +1286,7 @@ for the ``sender`` of events in the timeline, not all members of a room. Repeated calls to lazy-loading aware endpoints will result in redundant membership events being excluded by default. Clients often track which membership events they already have, therefore making the extra information not as useful to the client. -Clients can always request redundant members by setting ``include_redundant_members`` +Clients can always request redundant membership events by setting ``include_redundant_members`` to true in the filter. Servers should be cautious about which events they consider redundant. Membership @@ -1296,9 +1296,9 @@ as the current membership event for a user. .. Note:: Repeated calls using the same filter to *any* lazy-loading aware endpoint may - result in redundant members being excluded from future calls. For example, a + result in redundant membership events being excluded from future calls. For example, a request to ``/sync`` followed by a request to ``/messages`` may result in a - future call to ``/sync`` excluding members included by the ``/messages`` call. + future call to ``/sync`` excluding membership events returned by the ``/messages`` call. The current endpoints which support lazy-loading room members are: From 8a6ef187db803e8150670b6b3b16a496eccfd9db Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 7 Jun 2019 17:52:17 +0100 Subject: [PATCH 222/282] more object! --- api/client-server/administrative_contact.yaml | 2 ++ api/client-server/registration.yaml | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 71ea3e6a..c2985f66 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -272,6 +272,7 @@ paths: If this field is not present, the client can assume that verification will happen without the client's involvement. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object @@ -355,6 +356,7 @@ paths: If this field is not present, the client can assume that verification will happen without the client's involvement. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index c8afed83..769614c1 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -248,6 +248,7 @@ paths: If this field is not present, the client can assume that verification will happen without the client's involvement. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object @@ -335,6 +336,7 @@ paths: If this field is not present, the client can assume that verification will happen without the client's involvement. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object @@ -477,6 +479,7 @@ paths: 200: description: An email was sent to the given address. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object @@ -561,6 +564,7 @@ paths: 200: description: An SMS message was sent to the given phone number. schema: + type: object allOf: - $ref: "../identity/definitions/sid.yaml" - type: object From a8edb066aa52bdadff63c2fa9d9b5ad50a8f5b99 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Fri, 7 Jun 2019 18:00:12 +0100 Subject: [PATCH 223/282] Clear up some wording --- api/client-server/registration.yaml | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 769614c1..d97766e2 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -213,8 +213,10 @@ paths: The homeserver should check that the given email address is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the email address itself, or proxying the - request to the ``validate/email/requestToken`` Identity Server API on - the server sent in ``id_server``. + request to the ``validate/email/requestToken`` Identity Server API. The + request should be proxied to the domain that is sent by the client in + the ``id_server``. It is imperative that the homeserver keep a list of + trusted Identity Servers and only proxies to those it trusts. operationId: requestTokenToRegisterEmail parameters: - in: body @@ -301,8 +303,10 @@ paths: The homeserver should check that the given phone number is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the phone number itself, or proxying the - request to the ``validate/msisdn/requestToken`` Identity Server API on - the server sent in ``id_server``. + request to the ``validate/msisdn/requestToken`` Identity Server API. The + request should be proxied to the domain that is sent by the client in + the ``id_server``. It is imperative that the homeserver keep a list of + trusted Identity Servers and only proxies to those it trusts. operationId: requestTokenToRegisterMSISDN parameters: - in: body @@ -453,7 +457,11 @@ paths: The homeserver has the choice of validating the email address itself, or proxying the request to the ``validate/email/requestToken`` Identity - server api on the server sent in ``id_server``. + Server API. The request should be proxied to the domain that is sent by + the client in the ``id_server``. It is imperative that the homeserver + keep a list of trusted Identity Servers and only proxies to those it + trusts. + .. |/register/email/requestToken| replace:: ``/register/email/requestToken`` @@ -536,9 +544,12 @@ paths: prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - The homeserver has the choice of validating the phone number itself, or - proxying the request to the ``validate/msisdn/requestToken`` Identity - server api on the server sent in ``id_server``. + The homeserver has the choice of validating the phone number itself, + or proxying the request to the ``validate/msisdn/requestToken`` Identity + Server API. The request should be proxied to the domain that is sent by + the client in the ``id_server``. It is imperative that the homeserver + keep a list of trusted Identity Servers and only proxies to those it + trusts. .. |/register/msisdn/requestToken| replace:: ``/register/msisdn/requestToken`` From be568ba9ab8c8f4342220331eb66fe70ae09e5c7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 7 Jun 2019 19:10:26 +0100 Subject: [PATCH 224/282] link to the legalified version of MSC1779 --- proposals/1779-open-governance.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 0a1cfca4..bfe86eb8 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -12,6 +12,9 @@ for more context. This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318). +**This MSC is now formalised in the official Rules of the Matrix.org Foundation, +maintained at https://docs.google.com/document/d/1MhqsuIUxPc7Vf_y8D250mKZlLeQS6E39DPY6Azpc2NY** + ## Introduction Historically the core team of Matrix has been paid to work on it by the same From 1776ba28d3d98d3a63c7b7becc032621aab9632b Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sat, 8 Jun 2019 18:52:38 +0100 Subject: [PATCH 225/282] Address review comments --- api/client-server/administrative_contact.yaml | 70 +++++--------- api/client-server/registration.yaml | 94 ++++++++----------- 2 files changed, 63 insertions(+), 101 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index c2985f66..bc6f98c7 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -134,15 +134,6 @@ paths: 200: description: |- The addition was successful. - - ``submit_url`` is an optional field containing a URL where the - client must submit a validation token to, with identical parameters - to the Identity Service API's ``/validate/email/submitToken`` - endpoint. The homeserver will send this token to the user, which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's involvement. examples: application/json: { "submit_url": "https://example.org/path/to/submitToken" @@ -153,8 +144,15 @@ paths: submit_url: type: string description: |- - An optional URL to submit information to to verify a - third-party identifier. + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's involvement. example: "https://example.org/path/to/submitToken" 403: description: The credentials could not be verified with the identity server. @@ -231,14 +229,14 @@ paths: post: summary: Begins the validation process for an email address for association with the user's account. description: |- - The homeserver should check that the given email address is **not** + The homeserver must check that the given email address is **not** already associated with an account on this homeserver. This API should be used to request validation tokens when adding an email address to an account. This API's parameters and response are identical to that of the |/register/email/requestToken|_ endpoint. The homeserver has the choice of validating the email address itself, or proxying the request - to the ``validate/email/requestToken`` Identity Server API on the - server sent in ``id_server``. + to the ``validate/email/requestToken`` Identity Service API as + identified by ``id_server``. operationId: requestTokenTo3PIDEmail parameters: - in: body @@ -262,15 +260,6 @@ paths: An email was sent to the given address. Note that this may be an email containing the validation token or it may be informing the user of an error. - - ``submit_url`` is an optional field containing a URL where the - client must submit a validation token to, with identical parameters - to the Identity Service API's ``/validate/email/submitToken`` - endpoint. The homeserver will send this token to the user, which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's involvement. schema: type: object allOf: @@ -280,11 +269,11 @@ paths: submit_url: type: string description: |- - An optional field containing a URL where the client - must submit a validation token to, with identical - parameters to the Identity Service API's + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be + must send this token to the user, which should then be prompted to provide it to the client. If this field is not present, the client can assume that @@ -317,14 +306,14 @@ paths: post: summary: Begins the validation process for a phone number for association with the user's account. description: |- - The homeserver should check that the given phone number is **not** + The homeserver must check that the given phone number is **not** already associated with an account on this homeserver. This API should be used to request validation tokens when adding a phone number to an account. This API's parameters and response are identical to that of the |/register/msisdn/requestToken|_ endpoint. The homeserver has the choice of validating the phone number itself, or proxying the request - to the ``validate/msisdn/requestToken`` Identity Server API on the - server sent in ``id_server``. + to the ``validate/msisdn/requestToken`` Identity Service API as + identified by ``id_server``. operationId: requestTokenTo3PIDMSISDN parameters: - in: body @@ -345,16 +334,7 @@ paths: responses: 200: description: |- - An SMS message was sent to the given phone number. - - ``submit_url`` is an optional field containing a URL where the - client must submit a validation token to, with identical parameters - to the Identity Service API's ``/validate/msisdn/submitToken`` - endpoint. The homeserver will send this token to the user, which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's involvement. + A SMS message was sent to the given phone number. schema: type: object allOf: @@ -364,12 +344,12 @@ paths: submit_url: type: string description: |- - An optional field containing a URL where the client - must submit a validation token to, with identical - parameters to the Identity Service API's + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be - prompted to provide it to the client. + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index d97766e2..c8615d29 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -210,10 +210,10 @@ paths: post: summary: Begins the validation process for an email to be used during registration. description: |- - The homeserver should check that the given email address is **not** + The homeserver must check that the given email address is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the email address itself, or proxying the - request to the ``validate/email/requestToken`` Identity Server API. The + request to the ``validate/email/requestToken`` Identity Service API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it trusts. @@ -240,15 +240,6 @@ paths: An email has been sent to the specified address. Note that this may be an email containing the validation token or it may be informing the user of an error. - - ``submit_url`` is an optional field containing a URL where the - client must submit a validation token to, with identical parameters - to the Identity Service API's ``/validate/email/submitToken`` - endpoint. The homeserver will send this token to the user, which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's involvement. schema: type: object allOf: @@ -258,12 +249,12 @@ paths: submit_url: type: string description: |- - An optional field containing a URL where the client - must submit a validation token to, with identical - parameters to the Identity Service API's + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be - prompted to provide it to the client. + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's @@ -300,10 +291,10 @@ paths: post: summary: Requests a validation token be sent to the given phone number for the purpose of registering an account description: |- - The homeserver should check that the given phone number is **not** + The homeserver must check that the given phone number is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the phone number itself, or proxying the - request to the ``validate/msisdn/requestToken`` Identity Server API. The + request to the ``validate/msisdn/requestToken`` Identity Service API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it trusts. @@ -327,18 +318,9 @@ paths: responses: 200: description: |- - An SMS message has been sent to the specified phone number. - Note that this may be an SMS message containing the validation token or it may be informing - the user of an error. - - ``submit_url`` is an optional field containing a URL where the - client must submit a validation token to, with identical parameters - to the Identity Service API's ``/validate/msisdn/submitToken`` - endpoint. The homeserver will send this token to the user, which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's involvement. + A SMS message has been sent to the specified phone number. Note + that this may be an SMS message containing the validation token or + it may be informing the user of an error. schema: type: object allOf: @@ -348,12 +330,12 @@ paths: submit_url: type: string description: |- - An optional field containing a URL where the client - must submit a validation token to, with identical - parameters to the Identity Service API's + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's ``/validate/email/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be - prompted to provide it to the client. + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's @@ -443,17 +425,17 @@ paths: post: summary: Requests a validation token be sent to the given email address for the purpose of resetting a user's password description: |- - The homeserver should check that the given email address **is + The homeserver must check that the given email address **is associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the ``account/password`` endpoint. - This API's parameters and response are identical to that of the HS API - |/register/email/requestToken|_ except that ``M_THREEPID_NOT_FOUND`` - may be returned if no account matching the given email address could be - found. The server may instead send an email to the given address - prompting the user to create an account. ``M_THREEPID_IN_USE`` may not - be returned. + This API's parameters and response are identical to that of the + |/register/email/requestToken|_ endpoint, except that + ``M_THREEPID_NOT_FOUND`` may be returned if no account matching the + given email address could be found. The server may instead send an + email to the given address prompting the user to create an account. + ``M_THREEPID_IN_USE`` may not be returned. The homeserver has the choice of validating the email address itself, or proxying the request to the ``validate/email/requestToken`` Identity @@ -496,11 +478,11 @@ paths: type: string description: |- An optional field containing a URL where the client must - submit a validation token to, with identical parameters + submit the validation token to, with identical parameters to the Identity Service API's ``/validate/email/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be - prompted to provide it to the client. + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's @@ -532,17 +514,17 @@ paths: post: summary: Requests a validation token be sent to the given phone number for the purpose of resetting a user's password. description: |- - The homeserver should check that the given phone number **is + The homeserver must check that the given phone number **is associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the ``account/password`` endpoint. - This API's parameters and response are identical to that of the HS API - |/register/msisdn/requestToken|_ except that ``M_THREEPID_NOT_FOUND`` may - be returned if no account matching the given phone number could be - found. The server may instead send the SMS to the given phone number - prompting the user to create an account. ``M_THREEPID_IN_USE`` may not - be returned. + This API's parameters and response are identical to that of the + |/register/msisdn/requestToken|_ endpoint, except that + ``M_THREEPID_NOT_FOUND`` may be returned if no account matching the + given phone number could be found. The server may instead send the SMS + to the given phone number prompting the user to create an account. + ``M_THREEPID_IN_USE`` may not be returned. The homeserver has the choice of validating the phone number itself, or proxying the request to the ``validate/msisdn/requestToken`` Identity @@ -573,7 +555,7 @@ paths: required: ['id_server'] responses: 200: - description: An SMS message was sent to the given phone number. + description: A SMS message was sent to the given phone number. schema: type: object allOf: @@ -584,11 +566,11 @@ paths: type: string description: |- An optional field containing a URL where the client must - submit a validation token to, with identical parameters + submit the validation token to, with identical parameters to the Identity Service API's ``/validate/msisdn/submitToken`` endpoint. The homeserver - will send this token to the user, which should then be - prompted to provide it to the client. + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's From 0ded48427652857f92e323d5d85580adf0b6c7bd Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sun, 9 Jun 2019 16:10:44 +0100 Subject: [PATCH 226/282] Address review comments --- api/client-server/administrative_contact.yaml | 4 ++-- api/client-server/registration.yaml | 12 ++++++------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index bc6f98c7..99451294 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -235,7 +235,7 @@ paths: account. This API's parameters and response are identical to that of the |/register/email/requestToken|_ endpoint. The homeserver has the choice of validating the email address itself, or proxying the request - to the ``validate/email/requestToken`` Identity Service API as + to the ``/validate/email/requestToken`` Identity Service API as identified by ``id_server``. operationId: requestTokenTo3PIDEmail parameters: @@ -312,7 +312,7 @@ paths: account. This API's parameters and response are identical to that of the |/register/msisdn/requestToken|_ endpoint. The homeserver has the choice of validating the phone number itself, or proxying the request - to the ``validate/msisdn/requestToken`` Identity Service API as + to the ``/validate/msisdn/requestToken`` Identity Service API as identified by ``id_server``. operationId: requestTokenTo3PIDMSISDN parameters: diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index c8615d29..21626815 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -213,7 +213,7 @@ paths: The homeserver must check that the given email address is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the email address itself, or proxying the - request to the ``validate/email/requestToken`` Identity Service API. The + request to the ``/validate/email/requestToken`` Identity Service API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it trusts. @@ -294,7 +294,7 @@ paths: The homeserver must check that the given phone number is **not** already associated with an account on this homeserver. The homeserver has the choice of validating the phone number itself, or proxying the - request to the ``validate/msisdn/requestToken`` Identity Service API. The + request to the ``/validate/msisdn/requestToken`` Identity Service API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it trusts. @@ -428,7 +428,7 @@ paths: The homeserver must check that the given email address **is associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the - ``account/password`` endpoint. + ``/account/password`` endpoint. This API's parameters and response are identical to that of the |/register/email/requestToken|_ endpoint, except that @@ -438,7 +438,7 @@ paths: ``M_THREEPID_IN_USE`` may not be returned. The homeserver has the choice of validating the email address itself, - or proxying the request to the ``validate/email/requestToken`` Identity + or proxying the request to the ``/validate/email/requestToken`` Identity Server API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it @@ -517,7 +517,7 @@ paths: The homeserver must check that the given phone number **is associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the - ``account/password`` endpoint. + ``/account/password`` endpoint. This API's parameters and response are identical to that of the |/register/msisdn/requestToken|_ endpoint, except that @@ -527,7 +527,7 @@ paths: ``M_THREEPID_IN_USE`` may not be returned. The homeserver has the choice of validating the phone number itself, - or proxying the request to the ``validate/msisdn/requestToken`` Identity + or proxying the request to the ``/validate/msisdn/requestToken`` Identity Server API. The request should be proxied to the domain that is sent by the client in the ``id_server``. It is imperative that the homeserver keep a list of trusted Identity Servers and only proxies to those it From fba61941af471e885fb2310ddb443660e92f0929 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sun, 9 Jun 2019 20:02:57 +0100 Subject: [PATCH 227/282] Move submit_url responses to ref --- api/client-server/administrative_contact.yaml | 41 +--------- api/client-server/registration.yaml | 80 +------------------ api/identity/definitions/sid.yaml | 14 ++++ 3 files changed, 20 insertions(+), 115 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 99451294..d26e5a96 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -261,25 +261,7 @@ paths: Note that this may be an email containing the validation token or it may be informing the user of an error. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user, which should then be - prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" + - $ref: "../identity/definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -336,26 +318,7 @@ paths: description: |- A SMS message was sent to the given phone number. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" - + - $ref: "../identity/definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 21626815..a2a0d47f 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -241,25 +241,7 @@ paths: Note that this may be an email containing the validation token or it may be informing the user of an error. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" + - $ref: "../identity/definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -322,25 +304,7 @@ paths: that this may be an SMS message containing the validation token or it may be informing the user of an error. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" + - $ref: "../identity/definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -469,25 +433,7 @@ paths: 200: description: An email was sent to the given address. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" + - $ref: "../identity/definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -557,25 +503,7 @@ paths: 200: description: A SMS message was sent to the given phone number. schema: - type: object - allOf: - - $ref: "../identity/definitions/sid.yaml" - - type: object - properties: - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/msisdn/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" + - $ref: "../identity/definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/identity/definitions/sid.yaml b/api/identity/definitions/sid.yaml index c1f1ae64..c09ed395 100644 --- a/api/identity/definitions/sid.yaml +++ b/api/identity/definitions/sid.yaml @@ -21,4 +21,18 @@ properties: ``[0-9a-zA-Z.=_-]``. Their length must not exceed 255 characters and they must not be empty. example: "123abc" + submit_url: + type: string + description: |- + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" required: ['sid'] From 525bedf8e14327556153d19933efcd488fa61017 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sun, 9 Jun 2019 20:10:44 +0100 Subject: [PATCH 228/282] Fix refs --- api/client-server/administrative_contact.yaml | 6 ++++-- api/client-server/registration.yaml | 12 ++++++++---- api/identity/definitions/sid.yaml | 14 -------------- 3 files changed, 12 insertions(+), 20 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index d26e5a96..c19872e8 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -261,7 +261,8 @@ paths: Note that this may be an email containing the validation token or it may be informing the user of an error. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -318,7 +319,8 @@ paths: description: |- A SMS message was sent to the given phone number. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index a2a0d47f..4ea8b252 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -241,7 +241,8 @@ paths: Note that this may be an email containing the validation token or it may be informing the user of an error. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -304,7 +305,8 @@ paths: that this may be an SMS message containing the validation token or it may be informing the user of an error. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -433,7 +435,8 @@ paths: 200: description: An email was sent to the given address. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -503,7 +506,8 @@ paths: 200: description: A SMS message was sent to the given phone number. schema: - - $ref: "../identity/definitions/sid.yaml" + type: object + $ref: "definitions/sid.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/identity/definitions/sid.yaml b/api/identity/definitions/sid.yaml index c09ed395..c1f1ae64 100644 --- a/api/identity/definitions/sid.yaml +++ b/api/identity/definitions/sid.yaml @@ -21,18 +21,4 @@ properties: ``[0-9a-zA-Z.=_-]``. Their length must not exceed 255 characters and they must not be empty. example: "123abc" - submit_url: - type: string - description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. - - If this field is not present, the client can assume that - verification will happen without the client's - involvement. - example: "https://example.org/path/to/submitToken" required: ['sid'] From 2551ff6ce0b4100a3436344c2043241998635fa0 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sun, 9 Jun 2019 20:12:23 +0100 Subject: [PATCH 229/282] Actually check in code --- api/client-server/definitions/sid.yaml | 38 ++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 api/client-server/definitions/sid.yaml diff --git a/api/client-server/definitions/sid.yaml b/api/client-server/definitions/sid.yaml new file mode 100644 index 00000000..50b45ca9 --- /dev/null +++ b/api/client-server/definitions/sid.yaml @@ -0,0 +1,38 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +properties: + sid: + type: string + description: |- + The session ID. Session IDs are opaque strings generated by the identity + server. They must consist entirely of the characters + ``[0-9a-zA-Z.=_-]``. Their length must not exceed 255 characters and they + must not be empty. + example: "123abc" + submit_url: + type: string + description: |- + An optional field containing a URL where the client must + submit the validation token to, with identical parameters + to the Identity Service API's + ``/validate/email/submitToken`` endpoint. The homeserver + must send this token to the user (if applicable), which + should then be prompted to provide it to the client. + + If this field is not present, the client can assume that + verification will happen without the client's + involvement. + example: "https://example.org/path/to/submitToken" +required: ['sid'] From 046a83448862d000048b644386cfacd5df6190e3 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Sun, 9 Jun 2019 22:04:58 +0100 Subject: [PATCH 230/282] Slight code cleanup --- api/client-server/administrative_contact.yaml | 14 +++++--------- api/client-server/registration.yaml | 12 ++++-------- 2 files changed, 9 insertions(+), 17 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index c19872e8..f0a86762 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -132,8 +132,7 @@ paths: } responses: 200: - description: |- - The addition was successful. + description: The addition was successful. examples: application/json: { "submit_url": "https://example.org/path/to/submitToken" @@ -257,11 +256,10 @@ paths: responses: 200: description: |- - An email was sent to the given address. - Note that this may be an email containing the validation token or - it may be informing the user of an error. + An email was sent to the given address. Note that this may be an + email containing the validation token or it may be informing the + user of an error. schema: - type: object $ref: "definitions/sid.yaml" 403: description: |- @@ -316,10 +314,8 @@ paths: required: ['id_server'] responses: 200: - description: |- - A SMS message was sent to the given phone number. + description: A SMS message was sent to the given phone number. schema: - type: object $ref: "definitions/sid.yaml" 403: description: |- diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 4ea8b252..c892b08d 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -237,11 +237,10 @@ paths: responses: 200: description: |- - An email has been sent to the specified address. - Note that this may be an email containing the validation token or it may be informing - the user of an error. + An email has been sent to the specified address. Note that this + may be an email containing the validation token or it may be + informing the user of an error. schema: - type: object $ref: "definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. @@ -301,11 +300,10 @@ paths: responses: 200: description: |- - A SMS message has been sent to the specified phone number. Note + A SMS message has been sent to the specified phone number. Note that this may be an SMS message containing the validation token or it may be informing the user of an error. schema: - type: object $ref: "definitions/sid.yaml" 403: description: The homeserver does not permit the address to be bound. @@ -435,7 +433,6 @@ paths: 200: description: An email was sent to the given address. schema: - type: object $ref: "definitions/sid.yaml" 403: description: |- @@ -506,7 +503,6 @@ paths: 200: description: A SMS message was sent to the given phone number. schema: - type: object $ref: "definitions/sid.yaml" 403: description: |- From fe23de7d7f43f6d7ed30f0a9a320b940071e89f9 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 05:20:15 +0100 Subject: [PATCH 231/282] A SMS -> An SMS --- api/client-server/administrative_contact.yaml | 2 +- api/client-server/registration.yaml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index f0a86762..4c1ea2e7 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -314,7 +314,7 @@ paths: required: ['id_server'] responses: 200: - description: A SMS message was sent to the given phone number. + description: An SMS message was sent to the given phone number. schema: $ref: "definitions/sid.yaml" 403: diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index c892b08d..00334434 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -300,7 +300,7 @@ paths: responses: 200: description: |- - A SMS message has been sent to the specified phone number. Note + An SMS message has been sent to the specified phone number. Note that this may be an SMS message containing the validation token or it may be informing the user of an error. schema: @@ -501,7 +501,7 @@ paths: required: ['id_server'] responses: 200: - description: A SMS message was sent to the given phone number. + description: An SMS message was sent to the given phone number. schema: $ref: "definitions/sid.yaml" 403: From 29340c6eb815dc2fedeec9f18468c58e385cda78 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 9 Jun 2019 22:50:23 -0600 Subject: [PATCH 232/282] Clarify that the common identifier format can be broken We already reference in the Event IDs section that the format depends on the room version, so we just need to link there. Fixes https://github.com/matrix-org/matrix-doc/issues/2103 --- specification/appendices/identifier_grammar.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/specification/appendices/identifier_grammar.rst b/specification/appendices/identifier_grammar.rst index ea805955..b5b0537a 100644 --- a/specification/appendices/identifier_grammar.rst +++ b/specification/appendices/identifier_grammar.rst @@ -88,8 +88,8 @@ Common Identifier Format ~~~~~~~~~~~~~~~~~~~~~~~~ The Matrix protocol uses a common format to assign unique identifiers to a -number of entities, including users, events and rooms. Each identifier takes -the form:: +number of entities, including users, events and rooms. Each identifier typically +takes the form:: &localpart:domain @@ -106,7 +106,9 @@ The sigil characters are as follows: * ``#``: Room alias The precise grammar defining the allowable format of an identifier depends on -the type of identifier. +the type of identifier. For example, event IDs can be represented without a +``domain`` component under some conditions - see the `Event IDs <#room-ids-and-event-ids>`_ +section below for more information. User Identifiers ++++++++++++++++ From ab0c1bc054247f732e66dd2b424d8c3a12a5e492 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 08:41:35 +0100 Subject: [PATCH 233/282] Add changelog, update imperative check, rename sid.yaml --- api/client-server/administrative_contact.yaml | 14 +++++---- .../{sid.yaml => request_token_response.yaml} | 0 api/client-server/registration.yaml | 30 +++++++++---------- .../client_server/newsfragments/2101.breaking | 1 + 4 files changed, 25 insertions(+), 20 deletions(-) rename api/client-server/definitions/{sid.yaml => request_token_response.yaml} (100%) create mode 100644 changelogs/client_server/newsfragments/2101.breaking diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 4c1ea2e7..4438b643 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -135,7 +135,7 @@ paths: description: The addition was successful. examples: application/json: { - "submit_url": "https://example.org/path/to/submitToken" + "submit_url": "https://example.org/path/to/submitToken" } schema: type: object @@ -235,7 +235,9 @@ paths: the |/register/email/requestToken|_ endpoint. The homeserver has the choice of validating the email address itself, or proxying the request to the ``/validate/email/requestToken`` Identity Service API as - identified by ``id_server``. + identified by ``id_server``. It is imperative that the + homeserver keep a list of trusted Identity Servers and only proxies to + those that it trusts. operationId: requestTokenTo3PIDEmail parameters: - in: body @@ -260,7 +262,7 @@ paths: email containing the validation token or it may be informing the user of an error. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -294,7 +296,9 @@ paths: the |/register/msisdn/requestToken|_ endpoint. The homeserver has the choice of validating the phone number itself, or proxying the request to the ``/validate/msisdn/requestToken`` Identity Service API as - identified by ``id_server``. + identified by ``id_server``. It is imperative that the + homeserver keep a list of trusted Identity Servers and only proxies to + those that it trusts. operationId: requestTokenTo3PIDMSISDN parameters: - in: body @@ -316,7 +320,7 @@ paths: 200: description: An SMS message was sent to the given phone number. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/api/client-server/definitions/sid.yaml b/api/client-server/definitions/request_token_response.yaml similarity index 100% rename from api/client-server/definitions/sid.yaml rename to api/client-server/definitions/request_token_response.yaml diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 00334434..5ec657e0 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -241,7 +241,7 @@ paths: may be an email containing the validation token or it may be informing the user of an error. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -304,7 +304,7 @@ paths: that this may be an SMS message containing the validation token or it may be informing the user of an error. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: The homeserver does not permit the address to be bound. schema: @@ -402,11 +402,11 @@ paths: ``M_THREEPID_IN_USE`` may not be returned. The homeserver has the choice of validating the email address itself, - or proxying the request to the ``/validate/email/requestToken`` Identity - Server API. The request should be proxied to the domain that is sent by - the client in the ``id_server``. It is imperative that the homeserver - keep a list of trusted Identity Servers and only proxies to those it - trusts. + or proxying the request to the ``/validate/email/requestToken`` + Identity Service API. The request should be proxied to the domain that + is sent by the client in the ``id_server``. It is imperative that the + homeserver keep a list of trusted Identity Servers and only proxies to + those that it trusts. .. |/register/email/requestToken| replace:: ``/register/email/requestToken`` @@ -433,7 +433,7 @@ paths: 200: description: An email was sent to the given address. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: |- The homeserver does not allow the third party identifier as a @@ -472,12 +472,12 @@ paths: to the given phone number prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - The homeserver has the choice of validating the phone number itself, - or proxying the request to the ``/validate/msisdn/requestToken`` Identity - Server API. The request should be proxied to the domain that is sent by - the client in the ``id_server``. It is imperative that the homeserver - keep a list of trusted Identity Servers and only proxies to those it - trusts. + The homeserver has the choice of validating the phone number itself, or + proxying the request to the ``/validate/msisdn/requestToken`` Identity + Service API. The request should be proxied to the domain that is sent + by the client in the ``id_server``. It is imperative that the + homeserver keep a list of trusted Identity Servers and only proxies to + those that it trusts. .. |/register/msisdn/requestToken| replace:: ``/register/msisdn/requestToken`` @@ -503,7 +503,7 @@ paths: 200: description: An SMS message was sent to the given phone number. schema: - $ref: "definitions/sid.yaml" + $ref: "definitions/request_token_response.yaml" 403: description: |- The homeserver does not allow the third party identifier as a diff --git a/changelogs/client_server/newsfragments/2101.breaking b/changelogs/client_server/newsfragments/2101.breaking new file mode 100644 index 00000000..84c645ab --- /dev/null +++ b/changelogs/client_server/newsfragments/2101.breaking @@ -0,0 +1 @@ +Add a new ``submit_url`` field to the response of various `.../requestToken` endpoints. From 0f82056ca2a46bd35483ee83c18c75035d85056f Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 08:43:34 +0100 Subject: [PATCH 234/282] Highlight breaking change --- changelogs/client_server/newsfragments/2101.breaking | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2101.breaking b/changelogs/client_server/newsfragments/2101.breaking index 84c645ab..bcbdcd55 100644 --- a/changelogs/client_server/newsfragments/2101.breaking +++ b/changelogs/client_server/newsfragments/2101.breaking @@ -1 +1 @@ -Add a new ``submit_url`` field to the response of various `.../requestToken` endpoints. +Breaking change: Add a new ``submit_url`` field to the response of various `.../requestToken` endpoints. Clients should request a token from the user to submit 3PID validation when they received this field. Older clients will not know about this field and thus may display a deceptive message thinking that validation will be handled out of bounds. From d1fde0837af4ecfde83532f0c027bde4f4c5ea10 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 09:06:13 +0100 Subject: [PATCH 235/282] Consolidate id_server into ref. id_server/next_link are ignored --- api/client-server/administrative_contact.yaml | 24 ++----------------- .../definitions/request_email_validation.yaml | 14 ++++++++--- .../request_msisdn_validation.yaml | 14 ++++++++--- 3 files changed, 24 insertions(+), 28 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 4438b643..6ee4a152 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -244,17 +244,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_email_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_email_validation.yaml" responses: 200: description: |- @@ -305,17 +295,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_msisdn_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_msisdn_validation.yaml" responses: 200: description: An SMS message was sent to the given phone number. diff --git a/api/identity/definitions/request_email_validation.yaml b/api/identity/definitions/request_email_validation.yaml index 5f15bd41..b99fe121 100644 --- a/api/identity/definitions/request_email_validation.yaml +++ b/api/identity/definitions/request_email_validation.yaml @@ -45,7 +45,15 @@ properties: next_link: type: string description: |- - Optional. When the validation is completed, the identity - server will redirect the user to this URL. + Optional. When the validation is completed, the identity server will + redirect the user to this URL. This option is ignored when submitting + 3PID validation information through a POST request. example: "https://example.org/congratulations.html" -required: ["client_secret", "email", "send_attempt"] + id_server: + type: string + description: |- + The hostname of the identity server to communicate with. May optionally + include a port. This parameter is ignored when the homeserver handles + 3PID verification. + example: "id.example.com" +required: ["client_secret", "email", "send_attempt", "id_server"] diff --git a/api/identity/definitions/request_msisdn_validation.yaml b/api/identity/definitions/request_msisdn_validation.yaml index 1a8c0cf8..08dd0482 100644 --- a/api/identity/definitions/request_msisdn_validation.yaml +++ b/api/identity/definitions/request_msisdn_validation.yaml @@ -51,7 +51,15 @@ properties: next_link: type: string description: |- - Optional. When the validation is completed, the identity - server will redirect the user to this URL. + Optional. When the validation is completed, the identity server will + redirect the user to this URL. This option is ignored when submitting + 3PID validation information through a POST request. example: "https://example.org/congratulations.html" -required: ["client_secret", "country", "phone_number", "send_attempt"] + id_server: + type: string + description: |- + The hostname of the identity server to communicate with. May optionally + include a port. This parameter is ignored when the homeserver handles + 3PID verification. + example: "id.example.com" +required: ["client_secret", "country", "phone_number", "send_attempt", "id_server"] From 7116f9334e8925e2c710c529dd1d0ab8c44209de Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 09:09:26 +0100 Subject: [PATCH 236/282] More consolidation --- api/client-server/registration.yaml | 48 +++-------------------------- 1 file changed, 4 insertions(+), 44 deletions(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 5ec657e0..edc52c6a 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -223,17 +223,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_email_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_email_validation.yaml" responses: 200: description: |- @@ -286,17 +276,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_msisdn_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_msisdn_validation.yaml" responses: 200: description: |- @@ -418,17 +398,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_email_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_email_validation.yaml" responses: 200: description: An email was sent to the given address. @@ -488,17 +458,7 @@ paths: name: body required: true schema: - allOf: - - $ref: "../identity/definitions/request_msisdn_validation.yaml" - - type: object - properties: - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May - optionally include a port. - example: "id.example.com" - required: ['id_server'] + $ref: "../identity/definitions/request_msisdn_validation.yaml" responses: 200: description: An SMS message was sent to the given phone number. From 2068cba598b3e3630950f95d0d730a1a2e081a56 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 09:14:25 +0100 Subject: [PATCH 237/282] Clients should submit a POST request --- api/client-server/administrative_contact.yaml | 8 ++++---- .../definitions/request_token_response.yaml | 16 +++++++--------- 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 6ee4a152..a3a391bc 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -145,10 +145,10 @@ paths: description: |- An optional field containing a URL where the client must submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. + to the Identity Service API's ``POST + /validate/email/submitToken`` endpoint. The homeserver must + send this token to the user (if applicable), which should + then be prompted to provide it to the client. If this field is not present, the client can assume that verification will happen without the client's involvement. diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index 50b45ca9..48f818e5 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -24,15 +24,13 @@ properties: submit_url: type: string description: |- - An optional field containing a URL where the client must - submit the validation token to, with identical parameters - to the Identity Service API's - ``/validate/email/submitToken`` endpoint. The homeserver - must send this token to the user (if applicable), which - should then be prompted to provide it to the client. + An optional field containing a URL where the client must submit the + validation token to, with identical parameters to the Identity Service + API's ``POST /validate/email/submitToken`` endpoint. The homeserver must + send this token to the user (if applicable), which should then be + prompted to provide it to the client. - If this field is not present, the client can assume that - verification will happen without the client's - involvement. + If this field is not present, the client can assume that verification + will happen without the client's involvement. example: "https://example.org/path/to/submitToken" required: ['sid'] From 0506d09cf7735ac01ffe30e1783cd0429a761c28 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 10 Jun 2019 19:38:44 +0100 Subject: [PATCH 238/282] incorporate LL review from matthew --- .../definitions/room_event_filter.yaml | 3 +- api/client-server/message_pagination.yaml | 14 ++-- api/client-server/sync.yaml | 19 +++--- specification/client_server_api.rst | 67 ++++++++++++------- specification/modules/instant_messaging.rst | 2 +- 5 files changed, 63 insertions(+), 42 deletions(-) diff --git a/api/client-server/definitions/room_event_filter.yaml b/api/client-server/definitions/room_event_filter.yaml index 7045396d..880cb173 100644 --- a/api/client-server/definitions/room_event_filter.yaml +++ b/api/client-server/definitions/room_event_filter.yaml @@ -25,7 +25,8 @@ allOf: include_redundant_members: type: boolean description: |- - If ``true``, enables redundant membership events. Does not + If ``true``, sends all membership events for all events, even if they have already + been sent to the client. Does not apply unless ``lazy_load_members`` is ``true``. See `Lazy-loading room members <#lazy-loading-room-members>`_ for more information. Defaults to ``false``. diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index 3e01437a..35555375 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -115,15 +115,13 @@ paths: type: array description: |- A list of state events relevant to showing the ``chunk``. For example, if - ``lazy_load_members`` is enabled in the filter then this will contain any - the membership events for the the senders of events in the ``chunk``. Servers should be careful to not exclude - membership events which are older than ones already sent to the client. - Likewise, clients should be cautious and avoid using older membership - events as the current membership event when paginating backwards. + ``lazy_load_members`` is enabled in the filter then this may contain + the membership events for the senders of events in the ``chunk``. - Unless ``include_redundant_members`` is ``true``, the server should remove - membership events which would have already been sent to clients in prior calls - to lazy-loading aware endpoints with the same filter. + Unless ``include_redundant_members`` is ``true``, the server + may remove membership events which would have already been + sent to the client in prior calls to this endpoint, assuming + the membership of those members has not changed. items: type: object title: RoomStateEvent diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 45829f68..00ed562d 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -39,12 +39,15 @@ paths: for more information. Lazy-loading members is only supported on a ``StateFilter`` for this endpoint. When lazy-loading is enabled, servers MUST include the syncing user's own membership event when they join a room, or when the - full state of rooms is requested. The user's own membership event is eligible + full state of rooms is requested, to aid discovering the user's avatar & + displayname. + + Like other members, the user's own membership event is eligible for being considered redundant by the server. When a sync is ``limited``, - the server MUST return membership events for events in the gap (between ``since`` and the start of the returned timeline), - regardless as to whether - or not they are redundant. ``include_redundant_members`` is ignored for limited - syncs. + the server MUST return membership events for events in the gap + (between ``since`` and the start of the returned timeline), regardless + as to whether or not they are redundant. This ensures that joins/leaves + and profile changes which occur during the gap are not lost. operationId: sync security: - accessToken: [] @@ -149,9 +152,9 @@ paths: type: array description: |- The users which can be used to generate a room name - if the room does not have one. Required if the room - does not have a ``m.room.name`` or ``m.room.canonical_alias`` - state event with non-empty content. + if the room does not have one. Required if the room's + ``m.room.name`` or ``m.room.canonical_alias`` state events + are unset or empty. This should be the first 5 members of the room, ordered by stream ordering, which are joined or invited. The diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 42f34849..a292eaeb 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1273,32 +1273,51 @@ Lazy-loading room members Membership events often take significant resources for clients to track. In an effort to reduce the number of resources used, clients can enable "lazy-loading" -for room members. By doing this, servers will only ever send membership events +for room members. By doing this, servers will attempt to only send membership events which are relevant to the client. -In terms of filters, this means enabling ``lazy_load_members`` on a ``RoomEventFilter`` -(or a ``StateFilter`` in the case of ``/sync`` only). When enabled, lazy-loading -aware endpoints (see below) will only include membership events for the ``sender`` -of events being included in the response. For example, if a client makes a ``/sync`` -request with lazy-loading enabled, the server will only return membership events -for the ``sender`` of events in the timeline, not all members of a room. - -Repeated calls to lazy-loading aware endpoints will result in redundant membership -events being excluded by default. Clients often track which membership events they -already have, therefore making the extra information not as useful to the client. -Clients can always request redundant membership events by setting ``include_redundant_members`` -to true in the filter. - -Servers should be cautious about which events they consider redundant. Membership -events can change over time, and should be included as relevant to maintain the -historical record. Likewise, clients should be cautious about treating an older event -as the current membership event for a user. - -.. Note:: - Repeated calls using the same filter to *any* lazy-loading aware endpoint may - result in redundant membership events being excluded from future calls. For example, a - request to ``/sync`` followed by a request to ``/messages`` may result in a - future call to ``/sync`` excluding membership events returned by the ``/messages`` call. +It is important to understand that lazy-loading is not intended to be a +perfect optimisation, and that it may not be practical for the server to +calculate precisely which membership events are relevant to the client. As a +result, it is valid for the server to send redundant membership events to the +client to ease implementation, although such redundancy should be minimised +where possible to conserve bandwidth. + +In terms of filters, lazy-loading is enabled by enabling ``lazy_load_members`` +on a ``RoomEventFilter`` (or a ``StateFilter`` in the case of ``/sync`` only). +When enabled, lazy-loading aware endpoints (see below) will only include +membership events for the ``sender`` of events being included in the response. +For example, if a client makes a ``/sync`` request with lazy-loading enabled, +the server will only return membership events for the ``sender`` of events in +the timeline, not all members of a room. + +When processing a sequence of events (e.g. by looping on ``/sync`` or +paginating ``/messages``), it is common for blocks of events in the sequence +to share a similar set of senders. Rather than responses in the sequence +sending duplicate membership events for these senders to the client, the +server MAY assume that clients will remember membership events they have +already been sent, and choose to skip sending membership events for members +whose membership has not changed. These are called 'redundant membership +events'. Clients may request that redundant membership events are always +included in responses by setting ``include_redundant_members`` to true in the +filter. + +The expected pattern for using lazy-loading is currently: + +* Client performs an initial /sync with lazy-loading enabled, and receives + only the membership events which relate to the senders of the events it + receives. +* Clients which support display-name tab-completion or other operations which + require rapid access to all members in a room should call /members for the + currently selected room, with an ``?at`` parameter set to the /sync + response's from token. The member list for the room is then maintained by + the state in subsequent incremental /sync responses. +* Clients which do not support tab-completion may instead pull in profiles for + arbitrary users (e.g. read receipts, typing notifications) on demand by + querying the room state or ``/profile``. + +.. TODO-spec + This implies that GET /state should also take an ``?at`` param The current endpoints which support lazy-loading room members are: diff --git a/specification/modules/instant_messaging.rst b/specification/modules/instant_messaging.rst index c514f481..679eabdc 100644 --- a/specification/modules/instant_messaging.rst +++ b/specification/modules/instant_messaging.rst @@ -287,7 +287,7 @@ choose a name: on the members of the room. Clients should consider `m.room.member`_ events for users other than the logged-in user, as defined below. - i. If the ``m.heroes`` for the room are greater or equal to + i. If the number of ``m.heroes`` for the room are greater or equal to ``m.joined_member_count + m.invited_member_count - 1``, then use the membership events for the heroes to calculate display names for the users (`disambiguating them if required`_) and concatenating them. For From cf19f525767a9db9a982a966e47f11a983f4eab3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 13:21:24 -0600 Subject: [PATCH 239/282] Enforce 7 day validity requirement everywhere also misc formatting fixes --- api/server-server/definitions/keys.yaml | 2 +- specification/rooms/v5.rst | 9 +++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/api/server-server/definitions/keys.yaml b/api/server-server/definitions/keys.yaml index c48c320a..8bc6c563 100644 --- a/api/server-server/definitions/keys.yaml +++ b/api/server-server/definitions/keys.yaml @@ -98,7 +98,7 @@ properties: be ignored in room versions 1, 2, 3, and 4. Keys used beyond this timestamp MUST be considered invalid, depending on the `room version specification`_. - Servers SHOULD use the lesser of this field and 7 days into the future when + Servers MUST use the lesser of this field and 7 days into the future when determining if a key is valid. This is to avoid a situation where an attacker publishes a key which is valid for a significant amount of time without a way for the homeserver owner to revoke it. diff --git a/specification/rooms/v5.rst b/specification/rooms/v5.rst index 9b9fad7e..97ebf2fe 100644 --- a/specification/rooms/v5.rst +++ b/specification/rooms/v5.rst @@ -15,8 +15,8 @@ Room Version 5 ============== -This room version builds on `version 4 `_, enforcing signing key validity -periods for events. +This room version builds on `version 4 `_ while enforcing signing +key validity periods for events. .. contents:: Table of Contents .. sectnum:: @@ -52,3 +52,8 @@ validated. Servers missing a copy of the signing key MUST try to obtain one via or `POST /_matrix/key/v2/query <../server_server/r0.1.1.html#post-matrix-key-v2-query>`_ APIs. When using the ``/query`` endpoint, servers MUST set the ``minimum_valid_until_ts`` property to prompt the notary server to attempt to refresh the key if appropriate. + +Servers MUST use the lesser of ``valid_until_ts`` and 7 days into the future when +determining if a key is valid. This is to avoid a situation where an attacker +publishes a key which is valid for a significant amount of time without a way for +the homeserver owner to revoke it. From 1886a2346a27579cd378b4f112181d880626b460 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 13:38:43 -0600 Subject: [PATCH 240/282] Move explanation of notifications to notifications module --- api/client-server/sync.yaml | 4 ++-- specification/modules/push.rst | 11 +++++++++++ specification/modules/receipts.rst | 5 ++--- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 4514d417..89292673 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -172,8 +172,8 @@ paths: type: object description: |- Counts of unread notifications for this room. See the - `Receipts module <#module-receipts>`_ for more information - on how these are calculated. + `Receiving notifications section <#receiving-notifications>`_ + for more information on how these are calculated. properties: highlight_count: title: Highlighted notification count diff --git a/specification/modules/push.rst b/specification/modules/push.rst index 1bac0c2e..33ca7fd7 100644 --- a/specification/modules/push.rst +++ b/specification/modules/push.rst @@ -1,4 +1,5 @@ .. Copyright 2016 OpenMarket Ltd +.. Copyright 2019 The Matrix.org Foundation C.I.C. .. .. Licensed under the Apache License, Version 2.0 (the "License"); .. you may not use this file except in compliance with the License. @@ -116,6 +117,16 @@ have received. {{notifications_cs_http_api}} +Receiving notifications +~~~~~~~~~~~~~~~~~~~~~~~ + +Servers MUST include the number of unread notifications in a client's ``/sync`` +stream, and MUST update it as it changes. Notifications are determined by the +push rules which apply to an event. + +When the user updates their read receipt (either by using the API or by sending an +event), notifications prior to and including that event MUST be marked as read. + Push Rules ~~~~~~~~~~ A push rule is a single rule that states under what *conditions* an event should diff --git a/specification/modules/receipts.rst b/specification/modules/receipts.rst index 1745706e..4630091f 100644 --- a/specification/modules/receipts.rst +++ b/specification/modules/receipts.rst @@ -26,9 +26,8 @@ to a homeserver. To prevent this from becoming a problem, receipts are implement using "up to" markers. This marker indicates that the acknowledgement applies to all events "up to and including" the event specified. For example, marking an event as "read" would indicate that the user had read all events *up to* the -referenced event. Servers MUST decremement the number of pending notifications -for a user if the events are up to or including the read receipt. This is typically -done by adjusting the ``unread_notifications`` value in a ``/sync`` response. +referenced event. See the `Receiving notifications <#receiving-notifications>`_ +section for more information on how read receipts affect notification counts. Events ------ From 49831fb74fbde025c6d1c95a57ae5d15d0919fd5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 13:48:55 -0600 Subject: [PATCH 241/282] Clarifications about namespaces --- specification/index.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/specification/index.rst b/specification/index.rst index 375e19c0..547375e3 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -446,12 +446,13 @@ Namespacing Namespacing helps prevent conflicts between multiple applications and the specification itself. Where namespacing is used, ``m.`` prefixes are used by the specification to indicate that the field is controlled by the specification. Custom or non-specified -namespaces used in the wild SHOULD use the Java package naming convention to prevent +namespaces used in the wild MUST use the Java package naming convention to prevent conflicts. -As an example, event types are namespaced under ``m.`` in the specification however -any client can send a custom event type, such as ``com.example.game.score`` without -needing to put the event into the ``m.`` namespace. +As an example, event types defined in the specification are namespaced under the +special ``m.`` prefix, however any client can send a custom event type, such as +``com.example.game.score`` (assuming the client has rights to the ``com.example`` +namespace) without needing to put the event into the ``m.`` namespace. Timestamps ~~~~~~~~~~ From 19c827e581476a18875ea471263453d11a6d5cc3 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 13:49:05 -0600 Subject: [PATCH 242/282] Remove trailing slashes section --- specification/index.rst | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/specification/index.rst b/specification/index.rst index 547375e3..5acec2ea 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -432,13 +432,7 @@ Common concepts Various things are common throughout all of the Matrix APIs. They are documented here. -Trailing slashes on API endpoints -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Unless the endpoint is explicltly specified to have a trailing slash, the -slash is optional. For example, an endpoint specified as ``/_matrix/example/`` -would require a trailing slash, however an endpoint specified as ``/_matrix/example`` -has an optional slash (can be excluded when making requests). +.. TODO: Some words about trailing slashes. See https://github.com/matrix-org/matrix-doc/issues/2107 Namespacing ~~~~~~~~~~~ From 51698a5dd55232d39746269bfd164647ec56548f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 14:04:14 -0600 Subject: [PATCH 243/282] Clarify that people must own the namespace to use it --- specification/client_server_api.rst | 3 ++- specification/index.rst | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 80e4a2f4..b67beca1 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1330,7 +1330,8 @@ namespaced for each application and reduces the risk of clashes. Events are not limited to the types defined in this specification. New or custom event types can be created on a whim using the Java package naming convention. For example, a ``com.example.game.score`` event can be sent by clients and other - clients would receive it through Matrix. + clients would receive it through Matrix, assuming the client has access to the + ``com.example`` namespace. Note that the structure of these events may be different than those in the server-server API. diff --git a/specification/index.rst b/specification/index.rst index 7aa6a672..bf12a271 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -352,7 +352,8 @@ from the other servers participating in a room. Events are not limited to the types defined in this specification. New or custom event types can be created on a whim using the Java package naming convention. For example, a ``com.example.game.score`` event can be sent by clients and other - clients would receive it through Matrix. + clients would receive it through Matrix, assuming the client has access to the + ``com.example`` namespace. Room Aliases ++++++++++++ From 37e2d81d580083f8d7ed2f1a99e368231aab299c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 14:04:27 -0600 Subject: [PATCH 244/282] Hardcode the state event fields table into the spec See comment for why. --- specification/client_server_api.rst | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index b67beca1..ad58fd1a 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1340,7 +1340,28 @@ server-server API. {{common_room_event_fields}} -{{common_state_event_fields}} +.. This is normally where we'd put the common_state_event_fields variable for the +.. magic table of what makes up a state event, however the table is verbose in our +.. custom rendering of swagger. To combat this, we just hardcode this particular +.. table. + +State Event Fields +++++++++++++++++++ + +In addition to the fields of a Room Event, State Events have the following fields. + ++-----------+--------+-------------------------------------------------------------+ +| Key | Type | Description | ++===========+========+=============================================================+ +| state_key | string | **Required.** A unique key which defines the overwriting | +| | | semantics for this piece of room state. This value is often | +| | | a zero-length string. The presence of this key makes this | +| | | event a State Event. State keys starting with an ``@`` are | +| | | reserved for referencing user IDs, such as room members. | +| | | With the exception of a few events, state events set with | +| | | a given user's ID as the state key MUST only be set by that | +| | | user. | ++-----------+--------+-------------------------------------------------------------+ Size limits From ad2eefdde9b065800b5bc33e7697e14c8de82d90 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 10 Jun 2019 14:23:04 -0600 Subject: [PATCH 245/282] Shuffle and clarify identifier grammar Specifically, make it clearer that event IDs are the weird exception in this section. --- .../appendices/identifier_grammar.rst | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/specification/appendices/identifier_grammar.rst b/specification/appendices/identifier_grammar.rst index b5b0537a..c2c734ab 100644 --- a/specification/appendices/identifier_grammar.rst +++ b/specification/appendices/identifier_grammar.rst @@ -88,14 +88,13 @@ Common Identifier Format ~~~~~~~~~~~~~~~~~~~~~~~~ The Matrix protocol uses a common format to assign unique identifiers to a -number of entities, including users, events and rooms. Each identifier typically -takes the form:: +number of entities, including users, events and rooms. Each identifier takes +the form:: - &localpart:domain + &string -where ``&`` represents a 'sigil' character; ``domain`` is the `server name`_ of -the homeserver which allocated the identifier, and ``localpart`` is an -identifier allocated by that homeserver. +where ``&`` represents a 'sigil' character; ``string`` is the string which makes +up the identifier. The sigil characters are as follows: @@ -105,9 +104,16 @@ The sigil characters are as follows: * ``+``: Group ID * ``#``: Room alias +User IDs, group IDs, room IDs, room aliases, and sometimes event IDs take the form:: + + &localpart:domain + +where ``domain`` is the `server name`_ of the homeserver which allocated the +identifier, and ``localpart`` is an identifier allocated by that homeserver. + The precise grammar defining the allowable format of an identifier depends on -the type of identifier. For example, event IDs can be represented without a -``domain`` component under some conditions - see the `Event IDs <#room-ids-and-event-ids>`_ +the type of identifier. For example, event IDs can sometimes be represented with +a ``domain`` component under some conditions - see the `Event IDs <#room-ids-and-event-ids>`_ section below for more information. User Identifiers From 572d29348c293012a03326c9ee0bfac74b76bf98 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 11 Jun 2019 00:18:37 +0100 Subject: [PATCH 246/282] Update changelogs/client_server/newsfragments/2101.breaking Co-Authored-By: Travis Ralston --- changelogs/client_server/newsfragments/2101.breaking | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2101.breaking b/changelogs/client_server/newsfragments/2101.breaking index bcbdcd55..68971171 100644 --- a/changelogs/client_server/newsfragments/2101.breaking +++ b/changelogs/client_server/newsfragments/2101.breaking @@ -1 +1 @@ -Breaking change: Add a new ``submit_url`` field to the response of various `.../requestToken` endpoints. Clients should request a token from the user to submit 3PID validation when they received this field. Older clients will not know about this field and thus may display a deceptive message thinking that validation will be handled out of bounds. +Add a new ``submit_url`` field to the responses of ``/requestToken`` which older clients will not be able to handle correctly. From 1a22508e150071a275c08e3a13ace15211924f74 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 19:18:58 -0400 Subject: [PATCH 247/282] identity server -> the server --- api/client-server/definitions/request_token_response.yaml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index 48f818e5..8776d143 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -16,10 +16,9 @@ properties: sid: type: string description: |- - The session ID. Session IDs are opaque strings generated by the identity - server. They must consist entirely of the characters - ``[0-9a-zA-Z.=_-]``. Their length must not exceed 255 characters and they - must not be empty. + The session ID. Session IDs are opaque strings generated by a server. + They must consist entirely of the characters ``[0-9a-zA-Z.=_-]``. Their + length must not exceed 255 characters and they must not be empty. example: "123abc" submit_url: type: string From 5f24f63338f94a08c51754877a708fb596154e09 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Mon, 10 Jun 2019 19:21:14 -0400 Subject: [PATCH 248/282] Better wording --- api/client-server/definitions/request_token_response.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index 8776d143..98fefe07 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -16,9 +16,9 @@ properties: sid: type: string description: |- - The session ID. Session IDs are opaque strings generated by a server. - They must consist entirely of the characters ``[0-9a-zA-Z.=_-]``. Their - length must not exceed 255 characters and they must not be empty. + The session ID. Session IDs are opaque strings that must consist entirely + of the characters ``[0-9a-zA-Z.=_-]``. Their length must not exceed 255 + characters and they must not be empty. example: "123abc" submit_url: type: string From 04930c6ddfb0a5fadbde2a8255a2a8d59c430c98 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 08:29:50 -0600 Subject: [PATCH 249/282] Don't enforce MXC URIs, but also don't confuse people --- .../core-event-schema/msgtype_infos/image_info.yaml | 4 ++-- event-schemas/schema/m.room.message$m.audio | 4 +++- event-schemas/schema/m.room.message$m.file | 4 +++- event-schemas/schema/m.room.message$m.image | 4 +++- event-schemas/schema/m.room.message$m.video | 8 +++++--- 5 files changed, 16 insertions(+), 8 deletions(-) diff --git a/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml b/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml index b6a45007..ff40efcb 100644 --- a/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml +++ b/event-schemas/schema/core-event-schema/msgtype_infos/image_info.yaml @@ -19,8 +19,8 @@ properties: type: integer thumbnail_url: description: |- - The `MXC URI`_ to a thumbnail of the image. Only present if the - thumbnail is unencrypted. + The URL (typically `MXC URI`_) to a thumbnail of the image. + Only present if the thumbnail is unencrypted. type: string thumbnail_file: description: |- diff --git a/event-schemas/schema/m.room.message$m.audio b/event-schemas/schema/m.room.message$m.audio index 40075541..88b459ec 100644 --- a/event-schemas/schema/m.room.message$m.audio +++ b/event-schemas/schema/m.room.message$m.audio @@ -27,7 +27,9 @@ properties: - m.audio type: string url: - description: Required if the file is not encrypted. The `MXC URI`_ to the audio clip. + description: |- + Required if the file is not encrypted. The URL (typically `MXC URI`_) + to the audio clip. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.file b/event-schemas/schema/m.room.message$m.file index 225ca61d..9f4fdf07 100644 --- a/event-schemas/schema/m.room.message$m.file +++ b/event-schemas/schema/m.room.message$m.file @@ -42,7 +42,9 @@ properties: - m.file type: string url: - description: Required if the file is unencrypted. The `MXC URI`_ to the file. + description: |- + Required if the file is unencrypted. The URL (typically `MXC URI`_) + to the file. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.image b/event-schemas/schema/m.room.message$m.image index 8bf9c5fa..a466562a 100644 --- a/event-schemas/schema/m.room.message$m.image +++ b/event-schemas/schema/m.room.message$m.image @@ -17,7 +17,9 @@ properties: - m.image type: string url: - description: Required if the file is unencrypted. The `MXC URI`_ to the image. + description: |- + Required if the file is unencrypted. The URL (typically `MXC URI`_) + to the image. type: string file: description: |- diff --git a/event-schemas/schema/m.room.message$m.video b/event-schemas/schema/m.room.message$m.video index 01286ce2..b23c2392 100644 --- a/event-schemas/schema/m.room.message$m.video +++ b/event-schemas/schema/m.room.message$m.video @@ -28,8 +28,8 @@ properties: type: integer thumbnail_url: description: |- - The `MXC URI`_ to an image thumbnail of the video clip. Only present if the - thumbnail is unencrypted. + The URL (typically `MXC URI`_) to an image thumbnail of + the video clip. Only present if the thumbnail is unencrypted. type: string thumbnail_file: description: |- @@ -48,7 +48,9 @@ properties: - m.video type: string url: - description: Required if the file is unencrypted. The `MXC URI`_ to the video clip. + description: |- + Required if the file is unencrypted. The URL (typically `MXC URI`_) + to the video clip. type: string file: description: |- From ca8b539b2fe51e01c6042519aa04aa78b7e55d72 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 08:47:25 -0600 Subject: [PATCH 250/282] humans prefer to be treated as people --- api/client-server/administrative_contact.yaml | 2 +- api/client-server/definitions/request_token_response.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index a3a391bc..af7cdcd6 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -147,7 +147,7 @@ paths: submit the validation token to, with identical parameters to the Identity Service API's ``POST /validate/email/submitToken`` endpoint. The homeserver must - send this token to the user (if applicable), which should + send this token to the user (if applicable), who should then be prompted to provide it to the client. If this field is not present, the client can assume that diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index 98fefe07..bca57227 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -26,7 +26,7 @@ properties: An optional field containing a URL where the client must submit the validation token to, with identical parameters to the Identity Service API's ``POST /validate/email/submitToken`` endpoint. The homeserver must - send this token to the user (if applicable), which should then be + send this token to the user (if applicable), who should then be prompted to provide it to the client. If this field is not present, the client can assume that verification From b32f0e768f3a478e01913c997f97b5522b2baad5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 08:47:40 -0600 Subject: [PATCH 251/282] clarify that UIA can now be done by the HS --- specification/client_server_api.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 2e979cf7..fd63339e 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -762,17 +762,17 @@ the auth code. Homeservers can choose any path for the ``redirect URI``. Once the OAuth flow has completed, the client retries the request with the session only, as above. -Email-based (identity server) -<<<<<<<<<<<<<<<<<<<<<<<<<<<<< +Email-based (identity / homeserver) +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< :Type: ``m.login.email.identity`` :Description: Authentication is supported by authorising an email address with an identity - server. + server, or homeserver if supported. Prior to submitting this, the client should authenticate with an identity -server. After authenticating, the session information should be submitted to -the homeserver. +server (or homeserver). After authenticating, the session information should +be submitted to the homeserver. To use this authentication type, clients should submit an auth dict as follows: @@ -790,17 +790,17 @@ To use this authentication type, clients should submit an auth dict as follows: "session": "" } -Phone number/MSISDN-based (identity server) -<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< +Phone number/MSISDN-based (identity / homeserver) +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< :Type: ``m.login.msisdn`` :Description: Authentication is supported by authorising a phone number with an identity - server. + server, or homeserver if supported. Prior to submitting this, the client should authenticate with an identity -server. After authenticating, the session information should be submitted to -the homeserver. +server (or homeserver). After authenticating, the session information should +be submitted to the homeserver. To use this authentication type, clients should submit an auth dict as follows: From f7aa2adfb4bd01ffb10cc76d45ef18c44469ffe5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:16:44 -0600 Subject: [PATCH 252/282] Fix indentation --- api/client-server/keys.yaml | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 30056259..0cf074b0 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -59,22 +59,22 @@ paths: by the key algorithm. May be absent if no new one-time keys are required. - additionalProperties: - type: - - string - - type: object - title: KeyObject - properties: - key: + additionalProperties: + type: + - string + - type: object + title: KeyObject + properties: + key: + type: string + description: The key, encoded using unpadded base64. + signatures: + type: object + description: |- + Signature for the device. Mapped from user ID to signature object. + additionalProperties: type: string - description: The key, encoded using unpadded base64. - signatures: - type: object - description: |- - Signature for the device. Mapped from user ID to signature object. - additionalProperties: - type: string - required: ['key', 'signatures'] + required: ['key', 'signatures'] example: { "curve25519:AAAAAQ": "/qyvZvwjiTxGdGU0RCguDCLeR+nmsb3FfNG3/Ve4vU8", "signed_curve25519:AAAAHg": { From 85f34f942f4c1980658d147d5002f44fb76885fd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:18:29 -0600 Subject: [PATCH 253/282] Fix invalid doc error Another annoying case of Swagger fighting us --- api/client-server/keys.yaml | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 0cf074b0..0ea52e4a 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -62,19 +62,23 @@ paths: additionalProperties: type: - string - - type: object - title: KeyObject - properties: - key: - type: string - description: The key, encoded using unpadded base64. - signatures: - type: object - description: |- - Signature for the device. Mapped from user ID to signature object. - additionalProperties: - type: string - required: ['key', 'signatures'] + - object + # XXX: We can't define an actual object here, so we have to hope + # that people will look at the swagger source or can figure it out + # from the other endpoints/example. + # - type: object + # title: KeyObject + # properties: + # key: + # type: string + # description: The key, encoded using unpadded base64. + # signatures: + # type: object + # description: |- + # Signature for the device. Mapped from user ID to signature object. + # additionalProperties: + # type: string + # required: ['key', 'signatures'] example: { "curve25519:AAAAAQ": "/qyvZvwjiTxGdGU0RCguDCLeR+nmsb3FfNG3/Ve4vU8", "signed_curve25519:AAAAHg": { From 56745c76aa24c3d358574540b0f011aff6f0b15f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:22:02 -0600 Subject: [PATCH 254/282] Clarify that submit_url being not present only matters for r0.5 --- api/client-server/administrative_contact.yaml | 4 +++- api/client-server/definitions/request_token_response.yaml | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index af7cdcd6..c196c109 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -151,7 +151,9 @@ paths: then be prompted to provide it to the client. If this field is not present, the client can assume that - verification will happen without the client's involvement. + verification will happen without the client's involvement + provided the homeserver advertises this specification version + in the ``/versions`` response (ie: r0.5.0). example: "https://example.org/path/to/submitToken" 403: description: The credentials could not be verified with the identity server. diff --git a/api/client-server/definitions/request_token_response.yaml b/api/client-server/definitions/request_token_response.yaml index bca57227..e47db8a0 100644 --- a/api/client-server/definitions/request_token_response.yaml +++ b/api/client-server/definitions/request_token_response.yaml @@ -30,6 +30,8 @@ properties: prompted to provide it to the client. If this field is not present, the client can assume that verification - will happen without the client's involvement. + will happen without the client's involvement provided the homeserver + advertises this specification version in the ``/versions`` response + (ie: r0.5.0). example: "https://example.org/path/to/submitToken" required: ['sid'] From 8b7887dfc21615386c0b80e14550d0faf2e5505b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:47:02 -0600 Subject: [PATCH 255/282] Link to key algorithms section --- api/client-server/keys.yaml | 5 ++++- api/server-server/user_keys.yaml | 3 +++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/api/client-server/keys.yaml b/api/client-server/keys.yaml index 0ea52e4a..cce0edca 100644 --- a/api/client-server/keys.yaml +++ b/api/client-server/keys.yaml @@ -56,7 +56,7 @@ paths: One-time public keys for "pre-key" messages. The names of the properties should be in the format ``:``. The format of the key is determined - by the key algorithm. + by the `key algorithm <#key-algorithms>`_. May be absent if no new one-time keys are required. additionalProperties: @@ -299,6 +299,9 @@ paths: description: |- One-time keys for the queried devices. A map from user ID, to a map from devices to a map from ``:`` to the key object. + + See the `key algorithms <#key-algorithms>`_ section for information + on the Key Object format. additionalProperties: type: object additionalProperties: diff --git a/api/server-server/user_keys.yaml b/api/server-server/user_keys.yaml index 613948c3..ea59de2d 100644 --- a/api/server-server/user_keys.yaml +++ b/api/server-server/user_keys.yaml @@ -72,6 +72,9 @@ paths: description: |- One-time keys for the queried devices. A map from user ID, to a map from devices to a map from ``:`` to the key object. + + See the Client-Server Key Algorithms section for more information on + the Key Object format. additionalProperties: type: object additionalProperties: From c9345ba6a32618ded7fbfd992469164f4f239496 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 09:53:36 -0600 Subject: [PATCH 256/282] Never forget the past --- specification/client_server_api.rst | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index ad58fd1a..948ddc22 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1350,18 +1350,21 @@ State Event Fields In addition to the fields of a Room Event, State Events have the following fields. -+-----------+--------+-------------------------------------------------------------+ -| Key | Type | Description | -+===========+========+=============================================================+ -| state_key | string | **Required.** A unique key which defines the overwriting | -| | | semantics for this piece of room state. This value is often | -| | | a zero-length string. The presence of this key makes this | -| | | event a State Event. State keys starting with an ``@`` are | -| | | reserved for referencing user IDs, such as room members. | -| | | With the exception of a few events, state events set with | -| | | a given user's ID as the state key MUST only be set by that | -| | | user. | -+-----------+--------+-------------------------------------------------------------+ ++--------------+--------------+-------------------------------------------------------------+ +| Key | Type | Description | ++==============+==============+=============================================================+ +| state_key | string | **Required.** A unique key which defines the overwriting | +| | | semantics for this piece of room state. This value is often | +| | | a zero-length string. The presence of this key makes this | +| | | event a State Event. State keys starting with an ``@`` are | +| | | reserved for referencing user IDs, such as room members. | +| | | With the exception of a few events, state events set with | +| | | a given user's ID as the state key MUST only be set by that | +| | | user. | ++--------------+--------------+-------------------------------------------------------------+ +| prev_content | EventContent | Optional. The previous ``content`` for this event. If there | +| | | is no previous content, this key will be missing. | ++--------------+--------------+-------------------------------------------------------------+ Size limits From 4f915f27604c8ce9d3e347d991060a7561dbc4f9 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 10:15:51 -0600 Subject: [PATCH 257/282] Appservice r0.1.1 --- changelogs/application_service.rst | 10 ++++++++++ .../newsfragments/1650.clarification | 1 - .../newsfragments/2037.clarification | 1 - specification/application_service_api.rst | 1 + 4 files changed, 11 insertions(+), 2 deletions(-) delete mode 100644 changelogs/application_service/newsfragments/1650.clarification delete mode 100644 changelogs/application_service/newsfragments/2037.clarification diff --git a/changelogs/application_service.rst b/changelogs/application_service.rst index 9d098837..bbc461b2 100644 --- a/changelogs/application_service.rst +++ b/changelogs/application_service.rst @@ -1,3 +1,13 @@ +r0.1.1 +====== + +Spec Clarifications +------------------- + +- Change examples to use example.org instead of a real domain. (`#1650 `_) +- Add missing definition for how appservices verify requests came from a homeserver. (`#2037 `_) + + r0.1.0 ====== diff --git a/changelogs/application_service/newsfragments/1650.clarification b/changelogs/application_service/newsfragments/1650.clarification deleted file mode 100644 index 617b7ab6..00000000 --- a/changelogs/application_service/newsfragments/1650.clarification +++ /dev/null @@ -1 +0,0 @@ -Change examples to use example.org instead of a real domain. diff --git a/changelogs/application_service/newsfragments/2037.clarification b/changelogs/application_service/newsfragments/2037.clarification deleted file mode 100644 index f425b1c1..00000000 --- a/changelogs/application_service/newsfragments/2037.clarification +++ /dev/null @@ -1 +0,0 @@ -Add missing definition for how appservices verify requests came from a homeserver. diff --git a/specification/application_service_api.rst b/specification/application_service_api.rst index 3220df2d..ee7e9de7 100644 --- a/specification/application_service_api.rst +++ b/specification/application_service_api.rst @@ -52,6 +52,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.1.1 `_ - `r0.1.0 `_ From 34ad81e81b60775db732edacbd5af59d95579d24 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 10:30:56 -0600 Subject: [PATCH 258/282] Identity r0.2.0 --- changelogs/identity_service.rst | 19 +++++++++++++++++++ .../newsfragments/1853.clarification | 1 - .../newsfragments/1967.clarification | 1 - .../identity_service/newsfragments/2046.new | 1 - .../newsfragments/2049.clarification | 1 - .../newsfragments/2057.clarification | 1 - .../newsfragments/2086.clarification | 1 - specification/identity_service_api.rst | 1 + specification/modules/third_party_invites.rst | 2 +- 9 files changed, 21 insertions(+), 7 deletions(-) delete mode 100644 changelogs/identity_service/newsfragments/1853.clarification delete mode 100644 changelogs/identity_service/newsfragments/1967.clarification delete mode 100644 changelogs/identity_service/newsfragments/2046.new delete mode 100644 changelogs/identity_service/newsfragments/2049.clarification delete mode 100644 changelogs/identity_service/newsfragments/2057.clarification delete mode 100644 changelogs/identity_service/newsfragments/2086.clarification diff --git a/changelogs/identity_service.rst b/changelogs/identity_service.rst index 0405d515..d60812b8 100644 --- a/changelogs/identity_service.rst +++ b/changelogs/identity_service.rst @@ -1,3 +1,22 @@ +r0.2.0 +====== + +New Endpoints +------------- + +- Add ``/3pid/unbind`` for removing 3PIDs. (`#2046 `_) + + +Spec Clarifications +------------------- + +- Fix various spelling mistakes throughout the specification. (`#1853 `_) +- Fix route for ``/3pid/bind``. (`#1967 `_) +- Add missing aesthetic parameters to ``/store-invite``. (`#2049 `_) +- Clarify what the client should receive upon sending an identical email validation request multiple times. (`#2057 `_) +- Clarify that the default transport is JSON over HTTP. (`#2086 `_) + + r0.1.0 ====== diff --git a/changelogs/identity_service/newsfragments/1853.clarification b/changelogs/identity_service/newsfragments/1853.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/identity_service/newsfragments/1853.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/identity_service/newsfragments/1967.clarification b/changelogs/identity_service/newsfragments/1967.clarification deleted file mode 100644 index b080caeb..00000000 --- a/changelogs/identity_service/newsfragments/1967.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix route for ``/3pid/bind``. diff --git a/changelogs/identity_service/newsfragments/2046.new b/changelogs/identity_service/newsfragments/2046.new deleted file mode 100644 index 7146799b..00000000 --- a/changelogs/identity_service/newsfragments/2046.new +++ /dev/null @@ -1 +0,0 @@ -Add ``/3pid/unbind`` for removing 3PIDs. diff --git a/changelogs/identity_service/newsfragments/2049.clarification b/changelogs/identity_service/newsfragments/2049.clarification deleted file mode 100644 index 403ac8d0..00000000 --- a/changelogs/identity_service/newsfragments/2049.clarification +++ /dev/null @@ -1 +0,0 @@ -Add missing aesthetic parameters to ``/store-invite``. diff --git a/changelogs/identity_service/newsfragments/2057.clarification b/changelogs/identity_service/newsfragments/2057.clarification deleted file mode 100644 index de72c201..00000000 --- a/changelogs/identity_service/newsfragments/2057.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify what the client should receive upon sending an identical email validation request multiple times. diff --git a/changelogs/identity_service/newsfragments/2086.clarification b/changelogs/identity_service/newsfragments/2086.clarification deleted file mode 100644 index 7016308b..00000000 --- a/changelogs/identity_service/newsfragments/2086.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that the default transport is JSON over HTTP. diff --git a/specification/identity_service_api.rst b/specification/identity_service_api.rst index 88c7a8ab..952a239d 100644 --- a/specification/identity_service_api.rst +++ b/specification/identity_service_api.rst @@ -51,6 +51,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.2.0 `_ - `r0.1.0 `_ General principles diff --git a/specification/modules/third_party_invites.rst b/specification/modules/third_party_invites.rst index b8ab9657..282b06f4 100644 --- a/specification/modules/third_party_invites.rst +++ b/specification/modules/third_party_invites.rst @@ -255,4 +255,4 @@ these is left to the implementer's discretion. -.. _`identity server /isvalid`: ../identity_service/unstable.html#get-matrix-identity-api-v1-pubkey-isvalid +.. _`identity server /isvalid`: ../identity_service/%IDENTITY_RELEASE_LABEL%.html#get-matrix-identity-api-v1-pubkey-isvalid From 8e6ccf0a0e79eca83e4e968f7b7c9b837df3d4a7 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 10:38:12 -0600 Subject: [PATCH 259/282] Federation r0.1.2 --- changelogs/server_server.rst | 17 +++++++++++++++++ .../newsfragments/1650.clarification | 1 - .../newsfragments/1906.clarification | 1 - .../newsfragments/1991.clarification | 1 - .../newsfragments/2067.clarification | 1 - .../newsfragments/2080.clarification | 1 - .../newsfragments/2081.clarification | 1 - .../newsfragments/2083.clarification | 1 - .../newsfragments/2095.clarification | 1 - .../newsfragments/2097.clarification | 1 - specification/identity_service_api.rst | 2 +- specification/modules/send_to_device.rst | 2 +- specification/rooms/v1.rst | 4 ++-- specification/rooms/v2.rst | 2 +- specification/rooms/v3.rst | 4 ++-- specification/rooms/v4.rst | 4 ++-- specification/rooms/v5.rst | 4 ++-- specification/server_server_api.rst | 1 + 18 files changed, 29 insertions(+), 20 deletions(-) delete mode 100644 changelogs/server_server/newsfragments/1650.clarification delete mode 100644 changelogs/server_server/newsfragments/1906.clarification delete mode 100644 changelogs/server_server/newsfragments/1991.clarification delete mode 100644 changelogs/server_server/newsfragments/2067.clarification delete mode 100644 changelogs/server_server/newsfragments/2080.clarification delete mode 100644 changelogs/server_server/newsfragments/2081.clarification delete mode 100644 changelogs/server_server/newsfragments/2083.clarification delete mode 100644 changelogs/server_server/newsfragments/2095.clarification delete mode 100644 changelogs/server_server/newsfragments/2097.clarification diff --git a/changelogs/server_server.rst b/changelogs/server_server.rst index a21da177..11f99a5f 100644 --- a/changelogs/server_server.rst +++ b/changelogs/server_server.rst @@ -1,3 +1,20 @@ +r0.1.2 +====== + +Spec Clarifications +------------------- + +- Change examples to use example.org instead of a real domain. (`#1650 `_) +- Fix the ``access_token`` parameter in the open_id endpoint. (`#1906 `_) +- Fix various spelling mistakes throughout the specification. (`#1991 `_) +- Clarify exactly what invite_room_state consists of. (`#2067 `_) +- Clarify how ``valid_until_ts`` behaves with respect to room version. (`#2080 `_) +- Clarify which servers are supposed to sign events. (`#2081 `_) +- Clarify the key object definition for the key management API. (`#2083 `_) +- Clarify how many PDUs are contained in transaction objects for various endpoints. (`#2095 `_) +- Clarify that the trailing slash is optional on ``/keys/*`` endpoints when no key ID is requested. (`#2097 `_) + + r0.1.1 ====== diff --git a/changelogs/server_server/newsfragments/1650.clarification b/changelogs/server_server/newsfragments/1650.clarification deleted file mode 100644 index 617b7ab6..00000000 --- a/changelogs/server_server/newsfragments/1650.clarification +++ /dev/null @@ -1 +0,0 @@ -Change examples to use example.org instead of a real domain. diff --git a/changelogs/server_server/newsfragments/1906.clarification b/changelogs/server_server/newsfragments/1906.clarification deleted file mode 100644 index 531fdb94..00000000 --- a/changelogs/server_server/newsfragments/1906.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix the ``access_token`` parameter in the open_id endpoint. diff --git a/changelogs/server_server/newsfragments/1991.clarification b/changelogs/server_server/newsfragments/1991.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/server_server/newsfragments/1991.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/server_server/newsfragments/2067.clarification b/changelogs/server_server/newsfragments/2067.clarification deleted file mode 100644 index cc706274..00000000 --- a/changelogs/server_server/newsfragments/2067.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify exactly what invite_room_state consists of. diff --git a/changelogs/server_server/newsfragments/2080.clarification b/changelogs/server_server/newsfragments/2080.clarification deleted file mode 100644 index c568fa13..00000000 --- a/changelogs/server_server/newsfragments/2080.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how ``valid_until_ts`` behaves with respect to room version. diff --git a/changelogs/server_server/newsfragments/2081.clarification b/changelogs/server_server/newsfragments/2081.clarification deleted file mode 100644 index fd291273..00000000 --- a/changelogs/server_server/newsfragments/2081.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which servers are supposed to sign events. diff --git a/changelogs/server_server/newsfragments/2083.clarification b/changelogs/server_server/newsfragments/2083.clarification deleted file mode 100644 index 8083d85d..00000000 --- a/changelogs/server_server/newsfragments/2083.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the key object definition for the key management API. diff --git a/changelogs/server_server/newsfragments/2095.clarification b/changelogs/server_server/newsfragments/2095.clarification deleted file mode 100644 index 66257e17..00000000 --- a/changelogs/server_server/newsfragments/2095.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how many PDUs are contained in transaction objects for various endpoints. diff --git a/changelogs/server_server/newsfragments/2097.clarification b/changelogs/server_server/newsfragments/2097.clarification deleted file mode 100644 index 10dcecb6..00000000 --- a/changelogs/server_server/newsfragments/2097.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that the trailing slash is optional on ``/keys/*`` endpoints when no key ID is requested. diff --git a/specification/identity_service_api.rst b/specification/identity_service_api.rst index 88c7a8ab..b0b186a4 100644 --- a/specification/identity_service_api.rst +++ b/specification/identity_service_api.rst @@ -279,4 +279,4 @@ this isn't possible. .. _`Unpadded Base64`: ../appendices.html#unpadded-base64 .. _`3PID Types`: ../appendices.html#pid-types .. _`Signing JSON`: ../appendices.html#signing-json -.. _`/3pid/onbind`: ../server_server/r0.1.1.html#put-matrix-federation-v1-3pid-onbind +.. _`/3pid/onbind`: ../server_server/%SERVER_RELEASE_LABEL%.html#put-matrix-federation-v1-3pid-onbind diff --git a/specification/modules/send_to_device.rst b/specification/modules/send_to_device.rst index cca0338a..7ab622bc 100644 --- a/specification/modules/send_to_device.rst +++ b/specification/modules/send_to_device.rst @@ -63,7 +63,7 @@ If the client sends messages to users on remote domains, those messages should be sent on to the remote servers via `federation`_. -.. _`federation`: ../server_server/latest.html#send-to-device-messaging +.. _`federation`: ../server_server/%SERVER_RELEASE_LABEL%.html#send-to-device-messaging .. TODO-spec: diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index e8cbf663..b5ef217a 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -293,5 +293,5 @@ Events in version 1 rooms have the following structure: {{definition_ss_pdu}} -.. _`auth events selection`: ../server_server/r0.1.1.html#auth-events-selection -.. _`Signing Events`: ../server_server/r0.1.1.html#signing-events +.. _`auth events selection`: ../server_server/%SERVER_RELEASE_LABEL%.html#auth-events-selection +.. _`Signing Events`: ../server_server/%SERVER_RELEASE_LABEL%.html#signing-events diff --git a/specification/rooms/v2.rst b/specification/rooms/v2.rst index 247f6909..4e8365bf 100644 --- a/specification/rooms/v2.rst +++ b/specification/rooms/v2.rst @@ -161,7 +161,7 @@ The *resolution* of a set of states is obtained as follows: resolved state. -.. _`authorization rules`: ../server_server/r0.1.1.html#authorization-rules +.. _`authorization rules`: ../server_server/%SERVER_RELEASE_LABEL%.html#authorization-rules Rejected events +++++++++++++++ diff --git a/specification/rooms/v3.rst b/specification/rooms/v3.rst index 733c6984..8ef52acc 100644 --- a/specification/rooms/v3.rst +++ b/specification/rooms/v3.rst @@ -120,5 +120,5 @@ The remaining rules are the same as `room version 1 `_ -or `POST /_matrix/key/v2/query <../server_server/r0.1.1.html#post-matrix-key-v2-query>`_ +`GET /_matrix/key/v2/server <../server_server/%SERVER_RELEASE_LABEL%.html#get-matrix-key-v2-server-keyid>`_ +or `POST /_matrix/key/v2/query <../server_server/%SERVER_RELEASE_LABEL%.html#post-matrix-key-v2-query>`_ APIs. When using the ``/query`` endpoint, servers MUST set the ``minimum_valid_until_ts`` property to prompt the notary server to attempt to refresh the key if appropriate. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index d608965b..03c624ab 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -80,6 +80,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.1.2 `_ - `r0.1.1 `_ - `r0.1.0 `_ From 1ca60bf5d7658316aa0e917770f06ee4771af364 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 10:40:48 -0600 Subject: [PATCH 260/282] Client-Server r0.5.0 --- changelogs/client_server.rst | 82 +++++++++++++++++++ .../newsfragments/1650.clarification | 1 - .../newsfragments/1656.clarification | 1 - .../client_server/newsfragments/1701.feature | 1 - .../newsfragments/1744.clarification | 1 - .../client_server/newsfragments/1786.feature | 1 - .../client_server/newsfragments/1789.feature | 1 - .../client_server/newsfragments/1790.feature | 1 - .../client_server/newsfragments/1791.feature | 1 - .../newsfragments/1817.deprecation | 1 - .../client_server/newsfragments/1829.feature | 1 - .../newsfragments/1838.clarification | 1 - .../newsfragments/1853.clarification | 1 - .../newsfragments/1860.clarification | 1 - .../client_server/newsfragments/1873.new | 1 - .../client_server/newsfragments/1874.feature | 1 - .../client_server/newsfragments/1875.feature | 1 - .../client_server/newsfragments/1879.feature | 1 - .../newsfragments/1889.clarification | 1 - .../newsfragments/1891.clarification | 1 - .../client_server/newsfragments/1908.feature | 1 - .../newsfragments/1933.clarification | 1 - .../newsfragments/1969.clarification | 1 - .../newsfragments/1975.clarification | 1 - .../newsfragments/1988.clarification | 1 - .../newsfragments/1989.clarification | 1 - .../newsfragments/1991.clarification | 1 - .../newsfragments/1992.clarification | 1 - .../newsfragments/1999.clarification | 1 - .../newsfragments/2016.clarification | 1 - .../client_server/newsfragments/2020.feature | 1 - .../newsfragments/2025.clarification | 1 - .../client_server/newsfragments/2026.feature | 1 - .../newsfragments/2027.clarification | 1 - .../newsfragments/2028.clarification | 1 - .../newsfragments/2029.clarification | 1 - .../client_server/newsfragments/2030.feature | 1 - .../newsfragments/2031.clarification | 1 - .../newsfragments/2032.clarification | 1 - .../client_server/newsfragments/2035.feature | 1 - .../newsfragments/2036.clarification | 1 - .../newsfragments/2041.clarification | 1 - .../newsfragments/2042.clarification | 1 - .../newsfragments/2043.clarification | 1 - .../client_server/newsfragments/2046.feature | 1 - .../newsfragments/2051.clarification | 1 - .../newsfragments/2052.clarification | 1 - .../newsfragments/2053.clarification | 1 - .../newsfragments/2054.clarification | 1 - .../newsfragments/2055.clarification | 1 - .../newsfragments/2056.clarification | 1 - .../client_server/newsfragments/2059.feature | 1 - .../newsfragments/2067.clarification | 1 - .../newsfragments/2068.clarification | 1 - .../newsfragments/2069.clarification | 1 - .../client_server/newsfragments/2072.feature | 1 - .../newsfragments/2083.clarification | 1 - .../newsfragments/2087.clarification | 1 - .../newsfragments/2088.clarification | 1 - .../newsfragments/2089.clarification | 1 - .../newsfragments/2090.clarification | 1 - .../newsfragments/2091.clarification | 1 - .../newsfragments/2097.clarification | 1 - .../newsfragments/2098.clarification | 1 - .../client_server/newsfragments/2101.breaking | 1 - specification/client_server_api.rst | 1 + 66 files changed, 83 insertions(+), 64 deletions(-) delete mode 100644 changelogs/client_server/newsfragments/1650.clarification delete mode 100644 changelogs/client_server/newsfragments/1656.clarification delete mode 100644 changelogs/client_server/newsfragments/1701.feature delete mode 100644 changelogs/client_server/newsfragments/1744.clarification delete mode 100644 changelogs/client_server/newsfragments/1786.feature delete mode 100644 changelogs/client_server/newsfragments/1789.feature delete mode 100644 changelogs/client_server/newsfragments/1790.feature delete mode 100644 changelogs/client_server/newsfragments/1791.feature delete mode 100644 changelogs/client_server/newsfragments/1817.deprecation delete mode 100644 changelogs/client_server/newsfragments/1829.feature delete mode 100644 changelogs/client_server/newsfragments/1838.clarification delete mode 100644 changelogs/client_server/newsfragments/1853.clarification delete mode 100644 changelogs/client_server/newsfragments/1860.clarification delete mode 100644 changelogs/client_server/newsfragments/1873.new delete mode 100644 changelogs/client_server/newsfragments/1874.feature delete mode 100644 changelogs/client_server/newsfragments/1875.feature delete mode 100644 changelogs/client_server/newsfragments/1879.feature delete mode 100644 changelogs/client_server/newsfragments/1889.clarification delete mode 100644 changelogs/client_server/newsfragments/1891.clarification delete mode 100644 changelogs/client_server/newsfragments/1908.feature delete mode 100644 changelogs/client_server/newsfragments/1933.clarification delete mode 100644 changelogs/client_server/newsfragments/1969.clarification delete mode 100644 changelogs/client_server/newsfragments/1975.clarification delete mode 100644 changelogs/client_server/newsfragments/1988.clarification delete mode 100644 changelogs/client_server/newsfragments/1989.clarification delete mode 100644 changelogs/client_server/newsfragments/1991.clarification delete mode 100644 changelogs/client_server/newsfragments/1992.clarification delete mode 100644 changelogs/client_server/newsfragments/1999.clarification delete mode 100644 changelogs/client_server/newsfragments/2016.clarification delete mode 100644 changelogs/client_server/newsfragments/2020.feature delete mode 100644 changelogs/client_server/newsfragments/2025.clarification delete mode 100644 changelogs/client_server/newsfragments/2026.feature delete mode 100644 changelogs/client_server/newsfragments/2027.clarification delete mode 100644 changelogs/client_server/newsfragments/2028.clarification delete mode 100644 changelogs/client_server/newsfragments/2029.clarification delete mode 100644 changelogs/client_server/newsfragments/2030.feature delete mode 100644 changelogs/client_server/newsfragments/2031.clarification delete mode 100644 changelogs/client_server/newsfragments/2032.clarification delete mode 100644 changelogs/client_server/newsfragments/2035.feature delete mode 100644 changelogs/client_server/newsfragments/2036.clarification delete mode 100644 changelogs/client_server/newsfragments/2041.clarification delete mode 100644 changelogs/client_server/newsfragments/2042.clarification delete mode 100644 changelogs/client_server/newsfragments/2043.clarification delete mode 100644 changelogs/client_server/newsfragments/2046.feature delete mode 100644 changelogs/client_server/newsfragments/2051.clarification delete mode 100644 changelogs/client_server/newsfragments/2052.clarification delete mode 100644 changelogs/client_server/newsfragments/2053.clarification delete mode 100644 changelogs/client_server/newsfragments/2054.clarification delete mode 100644 changelogs/client_server/newsfragments/2055.clarification delete mode 100644 changelogs/client_server/newsfragments/2056.clarification delete mode 100644 changelogs/client_server/newsfragments/2059.feature delete mode 100644 changelogs/client_server/newsfragments/2067.clarification delete mode 100644 changelogs/client_server/newsfragments/2068.clarification delete mode 100644 changelogs/client_server/newsfragments/2069.clarification delete mode 100644 changelogs/client_server/newsfragments/2072.feature delete mode 100644 changelogs/client_server/newsfragments/2083.clarification delete mode 100644 changelogs/client_server/newsfragments/2087.clarification delete mode 100644 changelogs/client_server/newsfragments/2088.clarification delete mode 100644 changelogs/client_server/newsfragments/2089.clarification delete mode 100644 changelogs/client_server/newsfragments/2090.clarification delete mode 100644 changelogs/client_server/newsfragments/2091.clarification delete mode 100644 changelogs/client_server/newsfragments/2097.clarification delete mode 100644 changelogs/client_server/newsfragments/2098.clarification delete mode 100644 changelogs/client_server/newsfragments/2101.breaking diff --git a/changelogs/client_server.rst b/changelogs/client_server.rst index eb92c29d..c56a1073 100644 --- a/changelogs/client_server.rst +++ b/changelogs/client_server.rst @@ -1,3 +1,85 @@ +r0.5.0 +====== + +Breaking Changes +---------------- + +- Add a new ``submit_url`` field to the responses of ``/requestToken`` which older clients will not be able to handle correctly. (`#2101 `_) + + +Deprecations +------------ + +- Remove references to presence lists. (`#1817 `_) + + +New Endpoints +------------- + +- ``GET /account_data`` routes. (`#1873 `_) + + +Backwards Compatible Changes +---------------------------- + +- Add megolm session export format. (`#1701 `_) +- Add support for advertising experimental features to clients. (`#1786 `_) +- Add a generic SSO login API. (`#1789 `_) +- Add a mechanism for servers to redirect clients to an alternative homeserver after logging in. (`#1790 `_) +- Add room version upgrades. (`#1791 `_, `#1875 `_) +- Support optional features by having clients query for capabilities. (`#1829 `_, `#1879 `_) +- Add ``M_RESOURCE_LIMIT_EXCEEDED`` as an error code for when homeservers exceed limits imposed on them. (`#1874 `_) +- Emit ``M_UNSUPPORTED_ROOM_VERSION`` error codes where applicable on ``/createRoom`` and ``/invite`` APIs. (`#1908 `_) +- Add a ``.m.rule.tombstone`` default push rule for room ugprade notifications. (`#2020 `_) +- Add support for sending server notices to clients. (`#2026 `_) +- Add MSISDN (phone number) support to User-Interactive Authentication. (`#2030 `_) +- Add the option to lazy-load room members for increased client performance. (`#2035 `_) +- Add ``id_server`` to ``/deactivate`` and ``/3pid/delete`` endpoints to unbind from a specific identity server. (`#2046 `_) +- Add support for Olm sessions becoming un-stuck. (`#2059 `_) +- Add interactive device verification, including a common framework for device verification. (`#2072 `_) + + +Spec Clarifications +------------------- + +- Change examples to use example.org instead of a real domain. (`#1650 `_) +- Clarify that ``state_default`` in ``m.room.power_levels`` always defaults to 50. (`#1656 `_) +- Add missing ``status_msg`` to ``m.presence`` schema. (`#1744 `_) +- Fix various spelling mistakes throughout the specification. (`#1838 `_, `#1853 `_, `#1860 `_, `#1933 `_, `#1969 `_, `#1988 `_, `#1989 `_, `#1991 `_, `#1992 `_) +- Add the missing ``m.push_rules`` event schema. (`#1889 `_) +- Clarify how modern day local echo is meant to be solved by clients. (`#1891 `_) +- Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. (`#1975 `_) +- Clarify how ``m.login.dummy`` can be used to disambiguate login flows. (`#1999 `_) +- Remove ``prev_content`` from the redaction algorithm's essential keys list. (`#2016 `_) +- Fix the ``third_party_signed`` definitions for the join APIs. (`#2025 `_) +- Clarify why User Interactive Auth is used on password changes and how access tokens are handled. (`#2027 `_) +- Clarify that devices are deleted upon logout. (`#2028 `_) +- Add ``M_NOT_FOUND`` error definition for deleting room aliases. (`#2029 `_) +- Add missing ``reason`` to ``m.call.hangup``. (`#2031 `_) +- Clarify how redactions affect room state. (`#2032 `_) +- Clarify that ``FAIL_ERROR`` in autodiscovery is not limited to just homeservers. (`#2036 `_) +- Fix example ``Content-Type`` for ``/media/upload`` request. (`#2041 `_) +- Clarify that login flows are meant to be completed in order. (`#2042 `_) +- Clarify that clients should not send read receipts for their own messages. (`#2043 `_) +- Use consistent examples of events throughout the specification. (`#2051 `_) +- Clarify which push rule condition kinds exist. (`#2052 `_) +- Clarify the required fields on ``m.file`` (and similar) messages. (`#2053 `_) +- Clarify that User-Interactive Authentication stages cannot be attempted more than once. (`#2054 `_) +- Clarify which parameters apply in what scenarios on ``/register``. (`#2055 `_) +- Clarify how to interpret changes of ``membership`` over time. (`#2056 `_) +- Clarify exactly what invite_room_state consists of. (`#2067 `_) +- Clarify how the content repository works, and what it is used for. (`#2068 `_) +- Clarify the order events in chunk are returned in for ``/messages``. (`#2069 `_) +- Clarify the key object definition for the key management API. (`#2083 `_) +- Reorganize information about events into a common section. (`#2087 `_) +- De-duplicate ``/state/`` endpoints, clarifying that the ```` is optional. (`#2088 `_) +- Clarify when and where CORS headers should be returned. (`#2089 `_) +- Clarify when authorization and rate-limiting are not applicable. (`#2090 `_) +- Clarify that ``/register`` must produce valid Matrix User IDs. (`#2091 `_) +- Clarify how ``unread_notifications`` is calculated. (`#2097 `_) +- Clarify what a "module" is and update feature profiles for clients. (`#2098 `_) + + r0.4.0 ====== diff --git a/changelogs/client_server/newsfragments/1650.clarification b/changelogs/client_server/newsfragments/1650.clarification deleted file mode 100644 index 617b7ab6..00000000 --- a/changelogs/client_server/newsfragments/1650.clarification +++ /dev/null @@ -1 +0,0 @@ -Change examples to use example.org instead of a real domain. diff --git a/changelogs/client_server/newsfragments/1656.clarification b/changelogs/client_server/newsfragments/1656.clarification deleted file mode 100644 index 0c8f4ad0..00000000 --- a/changelogs/client_server/newsfragments/1656.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that ``state_default`` in ``m.room.power_levels`` always defaults to 50. diff --git a/changelogs/client_server/newsfragments/1701.feature b/changelogs/client_server/newsfragments/1701.feature deleted file mode 100644 index cf6084ae..00000000 --- a/changelogs/client_server/newsfragments/1701.feature +++ /dev/null @@ -1 +0,0 @@ -Add megolm session export format. diff --git a/changelogs/client_server/newsfragments/1744.clarification b/changelogs/client_server/newsfragments/1744.clarification deleted file mode 100644 index dfe838f1..00000000 --- a/changelogs/client_server/newsfragments/1744.clarification +++ /dev/null @@ -1 +0,0 @@ -Add missing ``status_msg`` to ``m.presence`` schema. diff --git a/changelogs/client_server/newsfragments/1786.feature b/changelogs/client_server/newsfragments/1786.feature deleted file mode 100644 index 6f21778c..00000000 --- a/changelogs/client_server/newsfragments/1786.feature +++ /dev/null @@ -1 +0,0 @@ -Add support for advertising experimental features to clients. diff --git a/changelogs/client_server/newsfragments/1789.feature b/changelogs/client_server/newsfragments/1789.feature deleted file mode 100644 index 97c1e5ca..00000000 --- a/changelogs/client_server/newsfragments/1789.feature +++ /dev/null @@ -1 +0,0 @@ -Add a generic SSO login API. diff --git a/changelogs/client_server/newsfragments/1790.feature b/changelogs/client_server/newsfragments/1790.feature deleted file mode 100644 index 26dccd05..00000000 --- a/changelogs/client_server/newsfragments/1790.feature +++ /dev/null @@ -1 +0,0 @@ -Add a mechanism for servers to redirect clients to an alternative homeserver after logging in. diff --git a/changelogs/client_server/newsfragments/1791.feature b/changelogs/client_server/newsfragments/1791.feature deleted file mode 100644 index 0a854c8f..00000000 --- a/changelogs/client_server/newsfragments/1791.feature +++ /dev/null @@ -1 +0,0 @@ -Add room version upgrades. diff --git a/changelogs/client_server/newsfragments/1817.deprecation b/changelogs/client_server/newsfragments/1817.deprecation deleted file mode 100644 index 2c52d198..00000000 --- a/changelogs/client_server/newsfragments/1817.deprecation +++ /dev/null @@ -1 +0,0 @@ -Remove references to presence lists. diff --git a/changelogs/client_server/newsfragments/1829.feature b/changelogs/client_server/newsfragments/1829.feature deleted file mode 100644 index 107291f3..00000000 --- a/changelogs/client_server/newsfragments/1829.feature +++ /dev/null @@ -1 +0,0 @@ -Support optional features by having clients query for capabilities. diff --git a/changelogs/client_server/newsfragments/1838.clarification b/changelogs/client_server/newsfragments/1838.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1838.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1853.clarification b/changelogs/client_server/newsfragments/1853.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1853.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1860.clarification b/changelogs/client_server/newsfragments/1860.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1860.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1873.new b/changelogs/client_server/newsfragments/1873.new deleted file mode 100644 index 724a4308..00000000 --- a/changelogs/client_server/newsfragments/1873.new +++ /dev/null @@ -1 +0,0 @@ -``GET /account_data`` routes. diff --git a/changelogs/client_server/newsfragments/1874.feature b/changelogs/client_server/newsfragments/1874.feature deleted file mode 100644 index bdab5464..00000000 --- a/changelogs/client_server/newsfragments/1874.feature +++ /dev/null @@ -1 +0,0 @@ -Add ``M_RESOURCE_LIMIT_EXCEEDED`` as an error code for when homeservers exceed limits imposed on them. diff --git a/changelogs/client_server/newsfragments/1875.feature b/changelogs/client_server/newsfragments/1875.feature deleted file mode 100644 index 0a854c8f..00000000 --- a/changelogs/client_server/newsfragments/1875.feature +++ /dev/null @@ -1 +0,0 @@ -Add room version upgrades. diff --git a/changelogs/client_server/newsfragments/1879.feature b/changelogs/client_server/newsfragments/1879.feature deleted file mode 100644 index 107291f3..00000000 --- a/changelogs/client_server/newsfragments/1879.feature +++ /dev/null @@ -1 +0,0 @@ -Support optional features by having clients query for capabilities. diff --git a/changelogs/client_server/newsfragments/1889.clarification b/changelogs/client_server/newsfragments/1889.clarification deleted file mode 100644 index 2737a7ee..00000000 --- a/changelogs/client_server/newsfragments/1889.clarification +++ /dev/null @@ -1 +0,0 @@ -Add the missing ``m.push_rules`` event schema. diff --git a/changelogs/client_server/newsfragments/1891.clarification b/changelogs/client_server/newsfragments/1891.clarification deleted file mode 100644 index ef4edfb4..00000000 --- a/changelogs/client_server/newsfragments/1891.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how modern day local echo is meant to be solved by clients. diff --git a/changelogs/client_server/newsfragments/1908.feature b/changelogs/client_server/newsfragments/1908.feature deleted file mode 100644 index 1c64d826..00000000 --- a/changelogs/client_server/newsfragments/1908.feature +++ /dev/null @@ -1 +0,0 @@ -Emit ``M_UNSUPPORTED_ROOM_VERSION`` error codes where applicable on ``/createRoom`` and ``/invite`` APIs. diff --git a/changelogs/client_server/newsfragments/1933.clarification b/changelogs/client_server/newsfragments/1933.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1933.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1969.clarification b/changelogs/client_server/newsfragments/1969.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1969.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1975.clarification b/changelogs/client_server/newsfragments/1975.clarification deleted file mode 100644 index ac118bfd..00000000 --- a/changelogs/client_server/newsfragments/1975.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. diff --git a/changelogs/client_server/newsfragments/1988.clarification b/changelogs/client_server/newsfragments/1988.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1988.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1989.clarification b/changelogs/client_server/newsfragments/1989.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1989.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1991.clarification b/changelogs/client_server/newsfragments/1991.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1991.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1992.clarification b/changelogs/client_server/newsfragments/1992.clarification deleted file mode 100644 index b0f05203..00000000 --- a/changelogs/client_server/newsfragments/1992.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix various spelling mistakes throughout the specification. diff --git a/changelogs/client_server/newsfragments/1999.clarification b/changelogs/client_server/newsfragments/1999.clarification deleted file mode 100644 index 748c55f2..00000000 --- a/changelogs/client_server/newsfragments/1999.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how ``m.login.dummy`` can be used to disambiguate login flows. diff --git a/changelogs/client_server/newsfragments/2016.clarification b/changelogs/client_server/newsfragments/2016.clarification deleted file mode 100644 index 77ea0d4c..00000000 --- a/changelogs/client_server/newsfragments/2016.clarification +++ /dev/null @@ -1 +0,0 @@ -Remove ``prev_content`` from the redaction algorithm's essential keys list. diff --git a/changelogs/client_server/newsfragments/2020.feature b/changelogs/client_server/newsfragments/2020.feature deleted file mode 100644 index 0d7c7eb8..00000000 --- a/changelogs/client_server/newsfragments/2020.feature +++ /dev/null @@ -1 +0,0 @@ -Add a ``.m.rule.tombstone`` default push rule for room ugprade notifications. diff --git a/changelogs/client_server/newsfragments/2025.clarification b/changelogs/client_server/newsfragments/2025.clarification deleted file mode 100644 index 9e99b23d..00000000 --- a/changelogs/client_server/newsfragments/2025.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix the ``third_party_signed`` definitions for the join APIs. diff --git a/changelogs/client_server/newsfragments/2026.feature b/changelogs/client_server/newsfragments/2026.feature deleted file mode 100644 index f82b9aea..00000000 --- a/changelogs/client_server/newsfragments/2026.feature +++ /dev/null @@ -1 +0,0 @@ -Add support for sending server notices to clients. diff --git a/changelogs/client_server/newsfragments/2027.clarification b/changelogs/client_server/newsfragments/2027.clarification deleted file mode 100644 index db74ea56..00000000 --- a/changelogs/client_server/newsfragments/2027.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify why User Interactive Auth is used on password changes and how access tokens are handled. diff --git a/changelogs/client_server/newsfragments/2028.clarification b/changelogs/client_server/newsfragments/2028.clarification deleted file mode 100644 index 75e21e74..00000000 --- a/changelogs/client_server/newsfragments/2028.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that devices are deleted upon logout. diff --git a/changelogs/client_server/newsfragments/2029.clarification b/changelogs/client_server/newsfragments/2029.clarification deleted file mode 100644 index 95b65481..00000000 --- a/changelogs/client_server/newsfragments/2029.clarification +++ /dev/null @@ -1 +0,0 @@ -Add ``M_NOT_FOUND`` error definition for deleting room aliases. diff --git a/changelogs/client_server/newsfragments/2030.feature b/changelogs/client_server/newsfragments/2030.feature deleted file mode 100644 index b5975a73..00000000 --- a/changelogs/client_server/newsfragments/2030.feature +++ /dev/null @@ -1 +0,0 @@ -Add MSISDN (phone number) support to User-Interactive Authentication. diff --git a/changelogs/client_server/newsfragments/2031.clarification b/changelogs/client_server/newsfragments/2031.clarification deleted file mode 100644 index 9bed3bcc..00000000 --- a/changelogs/client_server/newsfragments/2031.clarification +++ /dev/null @@ -1 +0,0 @@ -Add missing ``reason`` to ``m.call.hangup``. diff --git a/changelogs/client_server/newsfragments/2032.clarification b/changelogs/client_server/newsfragments/2032.clarification deleted file mode 100644 index e497b8be..00000000 --- a/changelogs/client_server/newsfragments/2032.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how redactions affect room state. diff --git a/changelogs/client_server/newsfragments/2035.feature b/changelogs/client_server/newsfragments/2035.feature deleted file mode 100644 index 47029c28..00000000 --- a/changelogs/client_server/newsfragments/2035.feature +++ /dev/null @@ -1 +0,0 @@ -Add the option to lazy-load room members for increased client performance. diff --git a/changelogs/client_server/newsfragments/2036.clarification b/changelogs/client_server/newsfragments/2036.clarification deleted file mode 100644 index 96058b7b..00000000 --- a/changelogs/client_server/newsfragments/2036.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that ``FAIL_ERROR`` in autodiscovery is not limited to just homeservers. diff --git a/changelogs/client_server/newsfragments/2041.clarification b/changelogs/client_server/newsfragments/2041.clarification deleted file mode 100644 index 39bbddb5..00000000 --- a/changelogs/client_server/newsfragments/2041.clarification +++ /dev/null @@ -1 +0,0 @@ -Fix example ``Content-Type`` for ``/media/upload`` request. diff --git a/changelogs/client_server/newsfragments/2042.clarification b/changelogs/client_server/newsfragments/2042.clarification deleted file mode 100644 index 4e17b99f..00000000 --- a/changelogs/client_server/newsfragments/2042.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that login flows are meant to be completed in order. diff --git a/changelogs/client_server/newsfragments/2043.clarification b/changelogs/client_server/newsfragments/2043.clarification deleted file mode 100644 index 9bb975fa..00000000 --- a/changelogs/client_server/newsfragments/2043.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that clients should not send read receipts for their own messages. diff --git a/changelogs/client_server/newsfragments/2046.feature b/changelogs/client_server/newsfragments/2046.feature deleted file mode 100644 index e54df535..00000000 --- a/changelogs/client_server/newsfragments/2046.feature +++ /dev/null @@ -1 +0,0 @@ -Add ``id_server`` to ``/deactivate`` and ``/3pid/delete`` endpoints to unbind from a specific identity server. diff --git a/changelogs/client_server/newsfragments/2051.clarification b/changelogs/client_server/newsfragments/2051.clarification deleted file mode 100644 index 384daa11..00000000 --- a/changelogs/client_server/newsfragments/2051.clarification +++ /dev/null @@ -1 +0,0 @@ -Use consistent examples of events throughout the specification. diff --git a/changelogs/client_server/newsfragments/2052.clarification b/changelogs/client_server/newsfragments/2052.clarification deleted file mode 100644 index 95bdc928..00000000 --- a/changelogs/client_server/newsfragments/2052.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which push rule condition kinds exist. diff --git a/changelogs/client_server/newsfragments/2053.clarification b/changelogs/client_server/newsfragments/2053.clarification deleted file mode 100644 index 2a72a88e..00000000 --- a/changelogs/client_server/newsfragments/2053.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the required fields on ``m.file`` (and similar) messages. diff --git a/changelogs/client_server/newsfragments/2054.clarification b/changelogs/client_server/newsfragments/2054.clarification deleted file mode 100644 index e43aea2d..00000000 --- a/changelogs/client_server/newsfragments/2054.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that User-Interactive Authentication stages cannot be attempted more than once. diff --git a/changelogs/client_server/newsfragments/2055.clarification b/changelogs/client_server/newsfragments/2055.clarification deleted file mode 100644 index 3a57ef7e..00000000 --- a/changelogs/client_server/newsfragments/2055.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify which parameters apply in what scenarios on ``/register``. diff --git a/changelogs/client_server/newsfragments/2056.clarification b/changelogs/client_server/newsfragments/2056.clarification deleted file mode 100644 index 12521867..00000000 --- a/changelogs/client_server/newsfragments/2056.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how to interpret changes of ``membership`` over time. diff --git a/changelogs/client_server/newsfragments/2059.feature b/changelogs/client_server/newsfragments/2059.feature deleted file mode 100644 index fde106ce..00000000 --- a/changelogs/client_server/newsfragments/2059.feature +++ /dev/null @@ -1 +0,0 @@ -Add support for Olm sessions becoming un-stuck. diff --git a/changelogs/client_server/newsfragments/2067.clarification b/changelogs/client_server/newsfragments/2067.clarification deleted file mode 100644 index cc706274..00000000 --- a/changelogs/client_server/newsfragments/2067.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify exactly what invite_room_state consists of. diff --git a/changelogs/client_server/newsfragments/2068.clarification b/changelogs/client_server/newsfragments/2068.clarification deleted file mode 100644 index 77ad7125..00000000 --- a/changelogs/client_server/newsfragments/2068.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how the content repository works, and what it is used for. diff --git a/changelogs/client_server/newsfragments/2069.clarification b/changelogs/client_server/newsfragments/2069.clarification deleted file mode 100644 index 127573a6..00000000 --- a/changelogs/client_server/newsfragments/2069.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the order events in chunk are returned in for ``/messages``. diff --git a/changelogs/client_server/newsfragments/2072.feature b/changelogs/client_server/newsfragments/2072.feature deleted file mode 100644 index c7d8bd76..00000000 --- a/changelogs/client_server/newsfragments/2072.feature +++ /dev/null @@ -1 +0,0 @@ -Add interactive device verification, including a common framework for device verification. diff --git a/changelogs/client_server/newsfragments/2083.clarification b/changelogs/client_server/newsfragments/2083.clarification deleted file mode 100644 index 8083d85d..00000000 --- a/changelogs/client_server/newsfragments/2083.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify the key object definition for the key management API. diff --git a/changelogs/client_server/newsfragments/2087.clarification b/changelogs/client_server/newsfragments/2087.clarification deleted file mode 100644 index 1974127d..00000000 --- a/changelogs/client_server/newsfragments/2087.clarification +++ /dev/null @@ -1 +0,0 @@ -Reorganize information about events into a common section. diff --git a/changelogs/client_server/newsfragments/2088.clarification b/changelogs/client_server/newsfragments/2088.clarification deleted file mode 100644 index ae22d66a..00000000 --- a/changelogs/client_server/newsfragments/2088.clarification +++ /dev/null @@ -1 +0,0 @@ -De-duplicate ``/state/`` endpoints, clarifying that the ```` is optional. diff --git a/changelogs/client_server/newsfragments/2089.clarification b/changelogs/client_server/newsfragments/2089.clarification deleted file mode 100644 index 17405adc..00000000 --- a/changelogs/client_server/newsfragments/2089.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify when and where CORS headers should be returned. diff --git a/changelogs/client_server/newsfragments/2090.clarification b/changelogs/client_server/newsfragments/2090.clarification deleted file mode 100644 index 23ab50f7..00000000 --- a/changelogs/client_server/newsfragments/2090.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify when authorization and rate-limiting are not applicable. diff --git a/changelogs/client_server/newsfragments/2091.clarification b/changelogs/client_server/newsfragments/2091.clarification deleted file mode 100644 index 2c4a276e..00000000 --- a/changelogs/client_server/newsfragments/2091.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that ``/register`` must produce valid Matrix User IDs. diff --git a/changelogs/client_server/newsfragments/2097.clarification b/changelogs/client_server/newsfragments/2097.clarification deleted file mode 100644 index 68d89bcc..00000000 --- a/changelogs/client_server/newsfragments/2097.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify how ``unread_notifications`` is calculated. diff --git a/changelogs/client_server/newsfragments/2098.clarification b/changelogs/client_server/newsfragments/2098.clarification deleted file mode 100644 index 1c8ba3ea..00000000 --- a/changelogs/client_server/newsfragments/2098.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify what a "module" is and update feature profiles for clients. diff --git a/changelogs/client_server/newsfragments/2101.breaking b/changelogs/client_server/newsfragments/2101.breaking deleted file mode 100644 index 68971171..00000000 --- a/changelogs/client_server/newsfragments/2101.breaking +++ /dev/null @@ -1 +0,0 @@ -Add a new ``submit_url`` field to the responses of ``/requestToken`` which older clients will not be able to handle correctly. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index bc2d66d7..dd6e3246 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -45,6 +45,7 @@ Other versions of this specification The following other versions are also available, in reverse chronological order: - `HEAD `_: Includes all changes since the latest versioned release. +- `r0.5.0 `_ - `r0.4.0 `_ - `r0.3.0 `_ - `r0.2.0 `_ From 8f1f8b4fe5bf5b21ca943678c54d30b1ffbf3025 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 11:28:35 -0600 Subject: [PATCH 261/282] Fix references to filtering We can't have two different backlinks for the same text, so use LL as the label --- api/client-server/event_context.yaml | 4 ++-- api/client-server/message_pagination.yaml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/api/client-server/event_context.yaml b/api/client-server/event_context.yaml index 802f5f27..3bea351c 100644 --- a/api/client-server/event_context.yaml +++ b/api/client-server/event_context.yaml @@ -35,8 +35,8 @@ paths: after the specified event. This allows clients to get the context surrounding an event. - *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_ - for more information. + *Note*: This endpoint supports lazy-loading of room member events. See + `Lazy-loading room members <#lazy-loading-room-members>`_ for more information. operationId: getEventContext security: - accessToken: [] diff --git a/api/client-server/message_pagination.yaml b/api/client-server/message_pagination.yaml index dc33b717..22828219 100644 --- a/api/client-server/message_pagination.yaml +++ b/api/client-server/message_pagination.yaml @@ -34,8 +34,8 @@ paths: This API returns a list of message and state events for a room. It uses pagination query parameters to paginate history in the room. - *Note*: This endpoint supports lazy-loading of room member events. See `Filtering <#lazy-loading-room-members>`_ - for more information. + *Note*: This endpoint supports lazy-loading of room member events. See + `Lazy-loading room members <#lazy-loading-room-members>`_ for more information. operationId: getRoomEvents security: - accessToken: [] From ba5479e46f1cd0aecdec54d200233a8d4f771882 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 11:30:04 -0600 Subject: [PATCH 262/282] Reference Canonical JSON --- specification/client_server_api.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index dd6e3246..9b16abc3 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1491,6 +1491,8 @@ Some event types have additional size restrictions which are specified in the description of the event. Additional keys have no limit other than that implied by the total 65 KB limit on events. +.. _`Canonical JSON`: ../appendices.html#canonical-json + Room Events ~~~~~~~~~~~ .. NOTE:: From 19a3d574b7e9b961b9df5ceeaa5e13e7f34a077a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 11 Jun 2019 11:31:24 -0600 Subject: [PATCH 263/282] Fix HKDF rationale --- specification/modules/end_to_end_encryption.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 62881967..36336c74 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -671,6 +671,7 @@ parameter is the concatenation of: * The ``transaction_id`` being used. .. admonition:: Rationale + HKDF is used over the plain shared secret as it results in a harder attack as well as more uniform data to work with. From e60d2defbd608476a4ae15816bf51e813c0e6713 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Jun 2019 13:47:09 -0600 Subject: [PATCH 264/282] Fix changelog generation Only include the target version, not all versions. Additionally, make sure the appservice spec isn't hardcoded as "unstable". --- scripts/templating/matrix_templates/units.py | 4 ++++ specification/application_service_api.rst | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index eeda9e63..157fa5a1 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -962,6 +962,9 @@ class MatrixUnits(Units): if re.match("^[=]{3,}$", line.strip()): # the last line was a header - use that as our new title_part title_part = prev_line.strip() + # take off the last line from the changelog_body_lines because it's the title + if len(changelog_body_lines) > 0: + changelog_body_lines = changelog_body_lines[:len(changelog_body_lines) - 1] continue if re.match("^[-]{3,}$", line.strip()): # the last line is a subheading - drop this line because it's the underline @@ -975,6 +978,7 @@ class MatrixUnits(Units): # that it renders correctly in the section. We also add newlines so that there's # intentionally blank lines that make rst2html happy. changelog_body_lines.append(" " + line + '\n') + prev_line = line if len(changelog_body_lines) > 0: changelogs[api_name] = "".join(changelog_body_lines) diff --git a/specification/application_service_api.rst b/specification/application_service_api.rst index ee7e9de7..11a07839 100644 --- a/specification/application_service_api.rst +++ b/specification/application_service_api.rst @@ -36,7 +36,7 @@ Changelog --------- -.. topic:: Version: unstable +.. topic:: Version: %APPSERVICE_RELEASE_LABEL% {{application_service_changelog}} This version of the specification is generated from From 18eca900220031bc92de970ce68662d5c3e9a89a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Jun 2019 13:48:50 -0600 Subject: [PATCH 265/282] Exclude DEL from historical user IDs The range is inclusive, so don't include 7F --- specification/appendices/identifier_grammar.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/appendices/identifier_grammar.rst b/specification/appendices/identifier_grammar.rst index c2c734ab..4cce1f90 100644 --- a/specification/appendices/identifier_grammar.rst +++ b/specification/appendices/identifier_grammar.rst @@ -190,7 +190,7 @@ history includes events with a ``sender`` which does not conform. In order to handle these rooms successfully, clients and servers MUST accept user IDs with localparts from the expanded character set:: - extended_user_id_char = %x21-39 / %x3B-7F ; all ascii printing chars except : + extended_user_id_char = %x21-39 / %x3B-7E ; all ascii printing chars except : Mapping from other character sets <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< From b8f1f3792744878b26f17b413c3c7936428a7f05 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Jun 2019 15:29:25 -0600 Subject: [PATCH 266/282] Remove incorrect id_server param from IS spec Fixes https://github.com/matrix-org/matrix-doc/issues/2118 --- api/client-server/administrative_contact.yaml | 4 +-- .../definitions/request_email_validation.yaml | 26 +++++++++++++++++++ .../request_msisdn_validation.yaml | 26 +++++++++++++++++++ api/client-server/registration.yaml | 14 +++++----- .../definitions/request_email_validation.yaml | 9 +------ .../request_msisdn_validation.yaml | 9 +------ .../newsfragments/2124.clarification | 1 + 7 files changed, 64 insertions(+), 25 deletions(-) create mode 100644 api/client-server/definitions/request_email_validation.yaml create mode 100644 api/client-server/definitions/request_msisdn_validation.yaml create mode 100644 changelogs/identity_service/newsfragments/2124.clarification diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index c196c109..0e93e4cd 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -246,7 +246,7 @@ paths: name: body required: true schema: - $ref: "../identity/definitions/request_email_validation.yaml" + $ref: "./definitions/request_email_validation.yaml" responses: 200: description: |- @@ -297,7 +297,7 @@ paths: name: body required: true schema: - $ref: "../identity/definitions/request_msisdn_validation.yaml" + $ref: "./definitions/request_msisdn_validation.yaml" responses: 200: description: An SMS message was sent to the given phone number. diff --git a/api/client-server/definitions/request_email_validation.yaml b/api/client-server/definitions/request_email_validation.yaml new file mode 100644 index 00000000..15bc5b3a --- /dev/null +++ b/api/client-server/definitions/request_email_validation.yaml @@ -0,0 +1,26 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: +- $ref: "../../identity/definitions/request_email_validation.yaml" +- type: object + properties: + id_server: + type: string + description: |- + The hostname of the identity server to communicate with. May optionally + include a port. This parameter is ignored when the homeserver handles + 3PID verification. + example: "id.example.com" + required: ["id_server"] diff --git a/api/client-server/definitions/request_msisdn_validation.yaml b/api/client-server/definitions/request_msisdn_validation.yaml new file mode 100644 index 00000000..370a10cc --- /dev/null +++ b/api/client-server/definitions/request_msisdn_validation.yaml @@ -0,0 +1,26 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: +- $ref: "../../identity/definitions/request_msisdn_validation.yaml" +- type: object + properties: + id_server: + type: string + description: |- + The hostname of the identity server to communicate with. May optionally + include a port. This parameter is ignored when the homeserver handles + 3PID verification. + example: "id.example.com" + required: ["id_server"] diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index d04e1a33..71177d0c 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -242,7 +242,7 @@ paths: name: body required: true schema: - $ref: "../identity/definitions/request_email_validation.yaml" + $ref: "./definitions/request_email_validation.yaml" responses: 200: description: |- @@ -295,7 +295,7 @@ paths: name: body required: true schema: - $ref: "../identity/definitions/request_msisdn_validation.yaml" + $ref: "./definitions/request_msisdn_validation.yaml" responses: 200: description: |- @@ -392,14 +392,14 @@ paths: associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the ``/account/password`` endpoint. - + This API's parameters and response are identical to that of the |/register/email/requestToken|_ endpoint, except that ``M_THREEPID_NOT_FOUND`` may be returned if no account matching the given email address could be found. The server may instead send an email to the given address prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - + The homeserver has the choice of validating the email address itself, or proxying the request to the ``/validate/email/requestToken`` Identity Service API. The request should be proxied to the domain that @@ -417,7 +417,7 @@ paths: name: body required: true schema: - $ref: "../identity/definitions/request_email_validation.yaml" + $ref: "./definitions/request_email_validation.yaml" responses: 200: description: An email was sent to the given address. @@ -453,14 +453,14 @@ paths: associated** with an account on this homeserver. This API should be used to request validation tokens when authenticating for the ``/account/password`` endpoint. - + This API's parameters and response are identical to that of the |/register/msisdn/requestToken|_ endpoint, except that ``M_THREEPID_NOT_FOUND`` may be returned if no account matching the given phone number could be found. The server may instead send the SMS to the given phone number prompting the user to create an account. ``M_THREEPID_IN_USE`` may not be returned. - + The homeserver has the choice of validating the phone number itself, or proxying the request to the ``/validate/msisdn/requestToken`` Identity Service API. The request should be proxied to the domain that is sent diff --git a/api/identity/definitions/request_email_validation.yaml b/api/identity/definitions/request_email_validation.yaml index b99fe121..1a7502c7 100644 --- a/api/identity/definitions/request_email_validation.yaml +++ b/api/identity/definitions/request_email_validation.yaml @@ -49,11 +49,4 @@ properties: redirect the user to this URL. This option is ignored when submitting 3PID validation information through a POST request. example: "https://example.org/congratulations.html" - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May optionally - include a port. This parameter is ignored when the homeserver handles - 3PID verification. - example: "id.example.com" -required: ["client_secret", "email", "send_attempt", "id_server"] +required: ["client_secret", "email", "send_attempt"] diff --git a/api/identity/definitions/request_msisdn_validation.yaml b/api/identity/definitions/request_msisdn_validation.yaml index 08dd0482..018bd733 100644 --- a/api/identity/definitions/request_msisdn_validation.yaml +++ b/api/identity/definitions/request_msisdn_validation.yaml @@ -55,11 +55,4 @@ properties: redirect the user to this URL. This option is ignored when submitting 3PID validation information through a POST request. example: "https://example.org/congratulations.html" - id_server: - type: string - description: |- - The hostname of the identity server to communicate with. May optionally - include a port. This parameter is ignored when the homeserver handles - 3PID verification. - example: "id.example.com" -required: ["client_secret", "country", "phone_number", "send_attempt", "id_server"] +required: ["client_secret", "country", "phone_number", "send_attempt"] diff --git a/changelogs/identity_service/newsfragments/2124.clarification b/changelogs/identity_service/newsfragments/2124.clarification new file mode 100644 index 00000000..384af82f --- /dev/null +++ b/changelogs/identity_service/newsfragments/2124.clarification @@ -0,0 +1 @@ +Remove incorrect ``id_server`` parameter from ``/requestToken`` endpoints. From 67ea3b9ce8321499a81961cdc1faddbf2cbacb2a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Jun 2019 15:41:10 -0600 Subject: [PATCH 267/282] Add 403 error to IS unbind Fixes https://github.com/matrix-org/matrix-doc/issues/2117 --- api/identity/associations.yaml | 13 +++++++++++++ .../newsfragments/2126.clarification | 1 + 2 files changed, 14 insertions(+) create mode 100644 changelogs/identity_service/newsfragments/2126.clarification diff --git a/api/identity/associations.yaml b/api/identity/associations.yaml index f44fe3cc..8ff4a9ed 100644 --- a/api/identity/associations.yaml +++ b/api/identity/associations.yaml @@ -279,6 +279,19 @@ paths: If the response body is not a JSON Matrix error, the identity server does not support unbinds. If a JSON Matrix error is in the response body, the requesting party should respect the error. + 403: + description: |- + The credentials supplied to authenticate the request were invalid. + This may also be returned if the identity server does not support + the chosen authentication method (such as blocking homeservers from + unbinding identifiers). + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "Invalid homeserver signature" + } + schema: + $ref: "../client-server/definitions/errors/error.yaml" 501: description: |- If the response body is not a JSON Matrix error, the identity server diff --git a/changelogs/identity_service/newsfragments/2126.clarification b/changelogs/identity_service/newsfragments/2126.clarification new file mode 100644 index 00000000..1475b4b4 --- /dev/null +++ b/changelogs/identity_service/newsfragments/2126.clarification @@ -0,0 +1 @@ +Clarify that identity servers can return 403 for unbind requests. From e670fb1f5a67daefdf4d4351ff33dd21bb945981 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Wed, 12 Jun 2019 14:32:33 -0700 Subject: [PATCH 268/282] Add missing format fields to m.room.message$m.notice schema. Signed-off-by: Jimmy Cuadra --- .../client_server/newsfragments/2125.clarification | 1 + event-schemas/schema/m.room.message$m.notice | 10 ++++++++++ 2 files changed, 11 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2125.clarification diff --git a/changelogs/client_server/newsfragments/2125.clarification b/changelogs/client_server/newsfragments/2125.clarification new file mode 100644 index 00000000..c71cdfff --- /dev/null +++ b/changelogs/client_server/newsfragments/2125.clarification @@ -0,0 +1 @@ +Add missing format fields to ``m.room.message$m.notice`` schema. diff --git a/event-schemas/schema/m.room.message$m.notice b/event-schemas/schema/m.room.message$m.notice index ef97e28a..19c4f985 100644 --- a/event-schemas/schema/m.room.message$m.notice +++ b/event-schemas/schema/m.room.message$m.notice @@ -12,6 +12,16 @@ properties: enum: - m.notice type: string + format: + description: |- + The format used in the ``formatted_body``. Currently only + ``org.matrix.custom.html`` is supported. + type: string + formatted_body: + description: |- + The formatted version of the ``body``. This is required if ``format`` + is specified. + type: string required: - msgtype - body From decb75555c103cd4c00066fe49ea9dc4757aabcc Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 12 Jun 2019 18:09:15 -0600 Subject: [PATCH 269/282] =?UTF-8?q?We're=20stable=20now=20=F0=9F=8E=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CONTRIBUTING.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 0b814fb9..fc5b146f 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -75,10 +75,8 @@ ask. Adding to the changelog ~~~~~~~~~~~~~~~~~~~~~~~ -Currently only changes to the client-server API need to end up in a changelog. The -other APIs are not yet stable and therefore do not have a changelog. Adding to the -changelog can only be done after you've opened your pull request, so be sure to do -that first. +All API specifications require a changelog entry. Adding to the changelog can only +be done after you've opened your pull request, so be sure to do that first. The changelog is managed by Towncrier (https://github.com/hawkowl/towncrier) in the form of "news fragments". The news fragments for the client-server API are stored From 2dd0da7d50ed8209ba277c5b10bacf7f1ebdecd2 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Fri, 14 Jun 2019 13:46:20 -0700 Subject: [PATCH 270/282] Fix typo in key verification framework section. Signed-off-by: Jimmy Cuadra --- changelogs/client_server/newsfragments/2131.clarification | 1 + specification/modules/end_to_end_encryption.rst | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2131.clarification diff --git a/changelogs/client_server/newsfragments/2131.clarification b/changelogs/client_server/newsfragments/2131.clarification new file mode 100644 index 00000000..3c41fb60 --- /dev/null +++ b/changelogs/client_server/newsfragments/2131.clarification @@ -0,0 +1 @@ +Fix typo in key verification framework section. diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 36336c74..f94fec2d 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -419,7 +419,7 @@ Key verification framework Verifying keys manually by reading out the Ed25519 key is not very user friendly, and can lead to errors. In order to help mitigate errors, and to make the process -eaiser for users, some verification methods are supported by the specification. +easier for users, some verification methods are supported by the specification. The methods all use a common framework for negotiating the key verification. To use this framework, Alice's client would send ``m.key.verification.request`` From b885714d94aa694fe267e3d36318b12d48499934 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Thu, 13 Jun 2019 17:21:11 -0700 Subject: [PATCH 271/282] Remove the "required" designation from the `url` field of certain m.room.message msgtypes. Now that content referenced by the *m.audio*, *m.file*, *m.image*, and *m.video* message types can be encrypted, the `url` field is required *only* if the content is unencrypted. The "required" designation in the event schemas (which prefixes the field description with "Required" in bold in the generated HTML) is used to indicate fields which must always be present, and this is no longer the case. Signed-off-by: Jimmy Cuadra --- changelogs/client_server/newsfragments/2129.clarification | 1 + event-schemas/schema/m.room.message$m.audio | 3 +-- event-schemas/schema/m.room.message$m.file | 1 - event-schemas/schema/m.room.message$m.image | 1 - event-schemas/schema/m.room.message$m.video | 1 - 5 files changed, 2 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2129.clarification diff --git a/changelogs/client_server/newsfragments/2129.clarification b/changelogs/client_server/newsfragments/2129.clarification new file mode 100644 index 00000000..82ed4fce --- /dev/null +++ b/changelogs/client_server/newsfragments/2129.clarification @@ -0,0 +1 @@ +Remove "required" designation from the ``url`` field of certain ``m`.room.message` msgtypes. diff --git a/event-schemas/schema/m.room.message$m.audio b/event-schemas/schema/m.room.message$m.audio index 88b459ec..fb049fc9 100644 --- a/event-schemas/schema/m.room.message$m.audio +++ b/event-schemas/schema/m.room.message$m.audio @@ -28,7 +28,7 @@ properties: type: string url: description: |- - Required if the file is not encrypted. The URL (typically `MXC URI`_) + Required if the file is unencrypted. The URL (typically `MXC URI`_) to the audio clip. type: string file: @@ -40,7 +40,6 @@ properties: required: - msgtype - body - - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message$m.file b/event-schemas/schema/m.room.message$m.file index 9f4fdf07..54a999ec 100644 --- a/event-schemas/schema/m.room.message$m.file +++ b/event-schemas/schema/m.room.message$m.file @@ -55,7 +55,6 @@ properties: required: - msgtype - body - - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message$m.image b/event-schemas/schema/m.room.message$m.image index a466562a..8944ce96 100644 --- a/event-schemas/schema/m.room.message$m.image +++ b/event-schemas/schema/m.room.message$m.image @@ -30,7 +30,6 @@ properties: required: - msgtype - body - - url type: object type: enum: diff --git a/event-schemas/schema/m.room.message$m.video b/event-schemas/schema/m.room.message$m.video index b23c2392..1a3c3e40 100644 --- a/event-schemas/schema/m.room.message$m.video +++ b/event-schemas/schema/m.room.message$m.video @@ -61,7 +61,6 @@ properties: required: - msgtype - body - - url type: object type: enum: From bc71dacaf4f65e2a27705a658fc4b303ecba16c9 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Fri, 14 Jun 2019 16:05:25 -0700 Subject: [PATCH 272/282] Clarify the distinction between *m.key.verification.start* and its *m.sas.v1* variant. Currently the *m.key.verification.start* event appears twice with the exact same title, in the "Key verification framework" section and the "Short Authentication (SAS) verification" section. It's not immediately clear that the first occurrence describes the format of the event in general terms and that the second occurrence describes the fields when the *m.sas.v1* verification method is being used. This is a similar relationship to the *m.room.message* event and its various *msgtype* variants. This commit does three things: * It tweaks the generation of the documentation to change the title of the second occurrence of *m.key.verification.start* to distinguish it from the first. * It updates the language in the description of the two versions of the event to better describe the relationship between the two. * It adds the optional `next_method` field to the schema of the *m.sas.v1* variant, as specified in the general form of *m.key.verification.start*. Signed-off-by: Jimmy Cuadra --- .../client_server/newsfragments/2132.clarification | 1 + event-schemas/schema/m.key.verification.start | 2 +- event-schemas/schema/m.key.verification.start$m.sas.v1 | 8 ++++++-- scripts/templating/matrix_templates/units.py | 9 +++++++++ 4 files changed, 17 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2132.clarification diff --git a/changelogs/client_server/newsfragments/2132.clarification b/changelogs/client_server/newsfragments/2132.clarification new file mode 100644 index 00000000..53063400 --- /dev/null +++ b/changelogs/client_server/newsfragments/2132.clarification @@ -0,0 +1 @@ +Clarify the distinction between *m.key.verification.start* and its *m.sas.v1* variant. diff --git a/event-schemas/schema/m.key.verification.start b/event-schemas/schema/m.key.verification.start index ad59d6c7..99b4998e 100644 --- a/event-schemas/schema/m.key.verification.start +++ b/event-schemas/schema/m.key.verification.start @@ -3,7 +3,7 @@ allOf: - $ref: core-event-schema/event.yaml description: |- - Begins a key verification process. Typically sent as a `to-device`_ event. + Begins a key verification process. Typically sent as a `to-device`_ event. The ``method`` field determines the type of verification. The fields in the event will differ depending on the ``method``. This definition includes fields that are in common among all variants. properties: content: properties: diff --git a/event-schemas/schema/m.key.verification.start$m.sas.v1 b/event-schemas/schema/m.key.verification.start$m.sas.v1 index 867ca820..a42f20e7 100644 --- a/event-schemas/schema/m.key.verification.start$m.sas.v1 +++ b/event-schemas/schema/m.key.verification.start$m.sas.v1 @@ -3,7 +3,7 @@ allOf: - $ref: core-event-schema/event.yaml description: |- - Begins a SAS key verification process. Typically sent as a `to-device`_ event. + Begins an SAS key verification process using the ``m.sas.v1`` method. Typically sent as a `to-device`_ event. properties: content: properties: @@ -22,7 +22,11 @@ properties: type: string enum: ["m.sas.v1"] description: |- - The verification method to use. Must be ``m.sas.v1``. + The verification method to use. + next_method: + type: string + description: |- + Optional method to use to verify the other user's key with. key_agreement_protocols: type: array description: |- diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index 157fa5a1..8538de2d 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -902,6 +902,15 @@ class MatrixUnits(Units): "`m.room.message msgtypes`_." ) + # method types for m.key.verification.start + if schema["type"] == "m.key.verification.start": + methods = Units.prop( + json_schema, "properties/content/properties/method/enum" + ) + if methods: + schema["type_with_msgtype"] = schema["type"] + " (" + methods[0] + ")" + + # Assign state key info if it has some if schema["typeof"] == "State Event": skey_desc = Units.prop( From 5384b61d95c82dcde6e924ce40f037f7f659d70f Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Fri, 14 Jun 2019 17:35:39 -0700 Subject: [PATCH 273/282] Fix style issues brought up during code review. Signed-off-by: Jimmy Cuadra --- changelogs/client_server/newsfragments/2132.clarification | 2 +- event-schemas/schema/m.key.verification.start | 4 +++- scripts/templating/matrix_templates/units.py | 1 - 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/changelogs/client_server/newsfragments/2132.clarification b/changelogs/client_server/newsfragments/2132.clarification index 53063400..1c458340 100644 --- a/changelogs/client_server/newsfragments/2132.clarification +++ b/changelogs/client_server/newsfragments/2132.clarification @@ -1 +1 @@ -Clarify the distinction between *m.key.verification.start* and its *m.sas.v1* variant. +Clarify the distinction between ``m`.key.verification.start` and its `m`.`sas.v1` variant. diff --git a/event-schemas/schema/m.key.verification.start b/event-schemas/schema/m.key.verification.start index 99b4998e..28926f4f 100644 --- a/event-schemas/schema/m.key.verification.start +++ b/event-schemas/schema/m.key.verification.start @@ -3,7 +3,9 @@ allOf: - $ref: core-event-schema/event.yaml description: |- - Begins a key verification process. Typically sent as a `to-device`_ event. The ``method`` field determines the type of verification. The fields in the event will differ depending on the ``method``. This definition includes fields that are in common among all variants. + Begins a key verification process. Typically sent as a `to-device`_ event. The ``method`` + field determines the type of verification. The fields in the event will differ depending + on the ``method``. This definition includes fields that are in common among all variants. properties: content: properties: diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index 8538de2d..04e6f8a9 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -910,7 +910,6 @@ class MatrixUnits(Units): if methods: schema["type_with_msgtype"] = schema["type"] + " (" + methods[0] + ")" - # Assign state key info if it has some if schema["typeof"] == "State Event": skey_desc = Units.prop( From 33ca891e71e6673ff6bdd3706d34c9c45d16da03 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 14 Jun 2019 22:22:43 -0600 Subject: [PATCH 274/282] Fix link to Olm signing specification See https://github.com/matrix-org/matrix.org/issues/478 --- changelogs/client_server/newsfragments/2133.clarification | 1 + specification/modules/end_to_end_encryption.rst | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelogs/client_server/newsfragments/2133.clarification diff --git a/changelogs/client_server/newsfragments/2133.clarification b/changelogs/client_server/newsfragments/2133.clarification new file mode 100644 index 00000000..3a003179 --- /dev/null +++ b/changelogs/client_server/newsfragments/2133.clarification @@ -0,0 +1 @@ +Fix link to Olm signing specification. diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index f94fec2d..27ba4998 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -411,7 +411,7 @@ Device verification may reach one of several conclusions. For example: protocol to verify that a given message was sent from a device holding that Ed25519 private key, or to encrypt a message so that it may only be decrypted by such a device. For the Olm protocol, this is documented at - https://matrix.org/git/olm/about/docs/signing.rst. + https://matrix.org/docs/spec/olm_signing.html. Key verification framework From 6f460ad70aa6c642b582bc9a3b140ffbcab41271 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 14 Jun 2019 22:26:29 -0600 Subject: [PATCH 275/282] minus spec according to the apache .htaccess we use --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 27ba4998..329c0170 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -411,7 +411,7 @@ Device verification may reach one of several conclusions. For example: protocol to verify that a given message was sent from a device holding that Ed25519 private key, or to encrypt a message so that it may only be decrypted by such a device. For the Olm protocol, this is documented at - https://matrix.org/docs/spec/olm_signing.html. + https://matrix.org/docs/olm_signing.html. Key verification framework From 8affb23e5ef62e01ba5c43d3d91f633748580994 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Sat, 15 Jun 2019 13:44:58 -0700 Subject: [PATCH 276/282] Address feedback from code review. * Switch "an SAS" back to "a SAS" * Remove the `next_method` field from m.key.verification.start$m.sas.v1 but add additional clarification to its description on m.key.verification.start that it is never present for methods that verify keys both ways. --- event-schemas/schema/m.key.verification.start | 3 ++- event-schemas/schema/m.key.verification.start$m.sas.v1 | 6 +----- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/event-schemas/schema/m.key.verification.start b/event-schemas/schema/m.key.verification.start index 28926f4f..faa7a96a 100644 --- a/event-schemas/schema/m.key.verification.start +++ b/event-schemas/schema/m.key.verification.start @@ -28,7 +28,8 @@ properties: type: string description: |- Optional method to use to verify the other user's key with. Applicable - when the ``method`` chosen only verifies one user's key. + when the ``method`` chosen only verifies one user's key. This field will + never be present if the ``method`` verifies keys both ways. required: - from_device - transaction_id diff --git a/event-schemas/schema/m.key.verification.start$m.sas.v1 b/event-schemas/schema/m.key.verification.start$m.sas.v1 index a42f20e7..daf6fa39 100644 --- a/event-schemas/schema/m.key.verification.start$m.sas.v1 +++ b/event-schemas/schema/m.key.verification.start$m.sas.v1 @@ -3,7 +3,7 @@ allOf: - $ref: core-event-schema/event.yaml description: |- - Begins an SAS key verification process using the ``m.sas.v1`` method. Typically sent as a `to-device`_ event. + Begins a SAS key verification process using the ``m.sas.v1`` method. Typically sent as a `to-device`_ event. properties: content: properties: @@ -23,10 +23,6 @@ properties: enum: ["m.sas.v1"] description: |- The verification method to use. - next_method: - type: string - description: |- - Optional method to use to verify the other user's key with. key_agreement_protocols: type: array description: |- From 802b90d1132407efb1b1ae6adc6511901f9ace46 Mon Sep 17 00:00:00 2001 From: Anatoly Sablin Date: Sun, 16 Jun 2019 22:24:06 +0300 Subject: [PATCH 277/282] Typo. --- event-schemas/schema/m.key.verification.accept | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/event-schemas/schema/m.key.verification.accept b/event-schemas/schema/m.key.verification.accept index 41c59968..ad54488e 100644 --- a/event-schemas/schema/m.key.verification.accept +++ b/event-schemas/schema/m.key.verification.accept @@ -3,7 +3,7 @@ allOf: - $ref: core-event-schema/event.yaml description: |- - Accepts a previously sent ``m.key.verification.start`` messge. Typically sent as a + Accepts a previously sent ``m.key.verification.start`` message. Typically sent as a `to-device`_ event. properties: content: From c63b5aff697c71c80b34b73ddf3ba327dae7c3b9 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 17 Jun 2019 00:10:59 -0600 Subject: [PATCH 278/282] Create 2136.clarification --- changelogs/client_server/newsfragments/2136.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2136.clarification diff --git a/changelogs/client_server/newsfragments/2136.clarification b/changelogs/client_server/newsfragments/2136.clarification new file mode 100644 index 00000000..3ccb2333 --- /dev/null +++ b/changelogs/client_server/newsfragments/2136.clarification @@ -0,0 +1 @@ +Fix various typos throughout the specification. From d09af5b191472fd7078aeff1b5c13558ab4a1212 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 17 Jun 2019 11:18:24 -0600 Subject: [PATCH 279/282] Identity r0.2.1 --- changelogs/identity_service.rst | 10 ++++++++++ .../identity_service/newsfragments/2124.clarification | 1 - .../identity_service/newsfragments/2126.clarification | 1 - 3 files changed, 10 insertions(+), 2 deletions(-) delete mode 100644 changelogs/identity_service/newsfragments/2124.clarification delete mode 100644 changelogs/identity_service/newsfragments/2126.clarification diff --git a/changelogs/identity_service.rst b/changelogs/identity_service.rst index d60812b8..cb06709d 100644 --- a/changelogs/identity_service.rst +++ b/changelogs/identity_service.rst @@ -1,3 +1,13 @@ +r0.2.1 +====== + +Spec Clarifications +------------------- + +- Remove incorrect ``id_server`` parameter from ``/requestToken`` endpoints. (`#2124 `_) +- Clarify that identity servers can return 403 for unbind requests. (`#2126 `_) + + r0.2.0 ====== diff --git a/changelogs/identity_service/newsfragments/2124.clarification b/changelogs/identity_service/newsfragments/2124.clarification deleted file mode 100644 index 384af82f..00000000 --- a/changelogs/identity_service/newsfragments/2124.clarification +++ /dev/null @@ -1 +0,0 @@ -Remove incorrect ``id_server`` parameter from ``/requestToken`` endpoints. diff --git a/changelogs/identity_service/newsfragments/2126.clarification b/changelogs/identity_service/newsfragments/2126.clarification deleted file mode 100644 index 1475b4b4..00000000 --- a/changelogs/identity_service/newsfragments/2126.clarification +++ /dev/null @@ -1 +0,0 @@ -Clarify that identity servers can return 403 for unbind requests. From 82463833300121a2ddc5b2676e5e8fd64f4263d3 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Tue, 18 Jun 2019 16:37:37 -0700 Subject: [PATCH 280/282] Fix typos in changelog entry. --- changelogs/client_server/newsfragments/2132.clarification | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelogs/client_server/newsfragments/2132.clarification b/changelogs/client_server/newsfragments/2132.clarification index 1c458340..b8a4cc8a 100644 --- a/changelogs/client_server/newsfragments/2132.clarification +++ b/changelogs/client_server/newsfragments/2132.clarification @@ -1 +1 @@ -Clarify the distinction between ``m`.key.verification.start` and its `m`.`sas.v1` variant. +Clarify the distinction between ``m.key.verification.start`` and its ``m.sas.v1`` variant. From 4997abc4b7cd36722003be98f7efae8d37f3f476 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 25 Jun 2019 16:47:10 +0100 Subject: [PATCH 281/282] as as -> as --- specification/client_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 9b16abc3..39a2fd3d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -1314,7 +1314,7 @@ to keep moving forwards. Filtering --------- -Filters can be created on the server and can be passed as as a parameter to APIs +Filters can be created on the server and can be passed as a parameter to APIs which return events. These filters alter the data returned from those APIs. Not all APIs accept filters. From e88a074c9ff9f2fa2f9c1aa129a0376a0bc5c617 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 25 Jun 2019 17:02:39 +0100 Subject: [PATCH 282/282] Add changelog --- changelogs/client_server/newsfragments/2148.misc | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/client_server/newsfragments/2148.misc diff --git a/changelogs/client_server/newsfragments/2148.misc b/changelogs/client_server/newsfragments/2148.misc new file mode 100644 index 00000000..d5514131 --- /dev/null +++ b/changelogs/client_server/newsfragments/2148.misc @@ -0,0 +1 @@ +Fix a small duplicated "as".