From 4538745809d15dbdc6107466f88193f555f3a948 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:43:46 +0000 Subject: [PATCH 01/79] MSC1779: Open Governance for Matrix.org --- proposals/1779-open-governance.md | 386 ++++++++++++++++++++++++++++++ 1 file changed, 386 insertions(+) create mode 100644 proposals/1779-open-governance.md diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md new file mode 100644 index 00000000..a79d175b --- /dev/null +++ b/proposals/1779-open-governance.md @@ -0,0 +1,386 @@ +# Proposal for Open Governance of Matrix.org + +This whole document is a **work in progress** draft of a constitution proposal +for open governance for Matrix.org, and forms the basis of the first full +Articles of Association (AoA) for [The Matrix.org Foundation +C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit legal +entity incorporated to act as the neutral guardian of the Matrix decentralised +communication standard on behalf of the whole Matrix community. + +See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ +for more context. + +This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318) + +## Introduction + +Historically the core team of Matrix has been paid to work on it by the same +employer (currently New Vector; the startup incorporated to hire the core +team in Aug 2017). Whilst convenient in initially getting Matrix built, we +recognise that this could create a potential conflict of interest between the +core team’s responsibilities to neutrally support the wider Matrix.org ecosystem +versus the need for New Vector to be able to support the team, and it has always +been the plan to set up a completely neutral custodian for the standard once it +had reached sufficient maturity. + +This proposal seeks to establish a new open governance process for Matrix.org, +such that once the specification has finally been ‘born’ and reached an initial +‘r0’ release across all APIs, control of Matrix.org can be decoupled from New +Vector and better support contributions from the whole ecosystem. + +The concepts here are somewhat inspired by [Rust’s Governance +Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust- +governance.md); a highly regarded solution to a similar problem: an ambitious +open-source project which has been too many years in the making, incubated at +first by a single company (Mozilla Corporation), which also enjoys a very +enthusiastic community! + +## Overview + +Governance of the project is split into two teams: the Spec Core Team and the +Guardians of the Foundation. In brief: + +The Spec Core Team are the technical experts who curate and edit the Matrix +Specification from day to day, and so steer the evolution of the protocol by +having final review over which Matrix Spec Changes (MSCs) are merged into the +core spec. + +The Guardians are the legal directors of the non-profit Foundation, and are +responsible for ensuring that the Foundation (and by extension the Spec Core +Team) keeps on mission and neutrally protects the development of Matrix. +Guardians are typically independent of the commercial Matrix ecosystem and may +even not be members of today’s Matrix community, but are deeply aligned with the +mission of the project, and who are respected and trusted by the wider community +to uphold the guiding principles of the Foundation and keep the other Guardians +honest. + +In other words; the Spec Core Team builds the spec, and the Guardians provide an +independent backstop to ensure the spec evolves in line with the Foundation's +mission. + +## Guiding Principles + +The guiding principles define the core philosophy of the project, and will be a +formal part of the final Articles of Association of the Matrix.org Foundation. + +### Matrix Manifesto + +We believe: + + * People should have full control over their own communication. + + * People should not be locked into centralised communication silos, but free to + pick who they choose to host their communication without limiting who they + can reach. + + * The ability to converse securely and privately is a basic human right. + + * Communication should be available to everyone as an free and open, + unencumbered, standard and global network. + +### Mission + +The Matrix.org Foundation exists to act as a neutral custodian for Matrix and +nurture it as efficiently as possible as a single unfragmented standard, for the +greater benefit of the whole ecosystem; not benefiting or privileging any single +player or subset of players. + +For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix +protocol. This includes (non-exhaustively): + + * End-users of Matrix clients + * Matrix client developers and testers + * Spec developers + * Server admins + * Matrix packagers & maintainers + * Companies building products or services on Matrix + * Bridge developers + * Bot developers + * Widget developers + * Server developers + * Matrix room and community moderators + * End-users who are using Matrix indirectly via bridges + * External systems which are bridged into Matrix + * Anyone using Matrix for data communications + +"Greater benefit" is defined as maximising: + + * the number of end-users reachable on the open Matrix network + * the number of regular users on the Matrix network (e.g. 30-day retained federated users) + * the number of end-users reachable by Matrix (natively or via bridges) + * the number of online servers in the open federation + * the number of developers building on Matrix + * the number of independent implementations which use Matrix + * the quality and utility of the Matrix spec (as defined by ease and ability + with which a developer can implement spec-compliant clients, servers, bots, + bridges, and other integrations without needing to refer to any other + external material) + +N.B. that we consider success to be the growth of the open federated network +rather than closed deployments. For example, if WhatsApp adopted Matrix it +wouldn’t be a complete win unless they openly federated with the rest of the +Matrix network. + +### Values + +As Matrix evolves, it's critical that the Spec Core Team and Guardians are +aligned on the overall philosophy of the project, particularly in more +subjective areas. The values we follow are: + + * Supporting the whole long-term ecosystem rather than individual stakeholder gain + * Openness rather than proprietariness + * Collaboration rather than competition + * Accessibility rather than elitism + * Transparency rather than stealth + * Empathy rather than contrariness + * Pragmatism rather than perfection + * Proof rather than conjecture + +Patent encumbered IP is strictly prohibited from being added to the standard. + +## The Spec Core Team + +The contents and direction of the Matrix Spec is governed by the Spec Core Team; +a set of experts from across the whole Matrix community, representing all +aspects of the Matrix ecosystem. The Spec Core Team acts as a subcommittee of +the Foundation. + +Members of the Spec Core Team pledge to act as a neutral custodian for Matrix on +behalf of the whole ecosystem and uphold the Guiding Principles of the project +as outlined above. In particular, they agree to drive the adoption of Matrix as +a single global federation, an open standard unencumbered from any proprietary +IP or software patents, minimising fragmentation (whilst encouraging +experimentation), evolving rapidly, and prioritising the long-term success and +growth of the overall network over individual commercial concerns. + +Spec Core Team members need to have significant proven domain experience/skill +and have had clear dedication and commitment to the project and community for >6 +months. (In future, once we have subteams a la Rust, folks need to have proven +themselves there first). + +Members need to demonstrate ability to work constructively with the rest of the +team; we want participation in the Spec Core Team to be an efficient, pleasant and +productive place, even in the face of inevitable disagreement. We do not want a +toxic culture of bullying or competitive infighting. Folks need to be able to +compromise; we are not building a culture of folks pushing their personal +agendas at the expense of the overall project. + +We are particularly vigilant against 'trojan horse' additions to the spec - +features which only benefit particular players, or are designed to somehow +cripple or fragment the open protocol and ecosystem in favour of competitive +advantage. Commercial players are of course encouraged to build proprietary +implementations, or use custom event types, or even custom API extensions (e.g. +more efficient network transports) - but implementations must fall back to +interoperating correctly with the rest of the ecosystem. + +### Spec Core Team logistics + +The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. +Roughly half the members are expected to be from the historical core team +(similar to Rust). + +In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; +IS API; Crypto), but as a starting point we are beginning with a single core +team in the interests of not over-engineering it and scaling up elastically. + +Spec Core Team members need to be able to commit to at least 1 hour a week of +availability to work on the spec and (where relevant) reference implementations. +Members of the team volunteer their time for free to work on the project. + +Responsibilities include: + + * Reviewing Matrix Spec Change proposals and Spec PRs + + * Contributing to and reviewing reference implementations of Matrix Spec Change + proposals + + * Shepherding Matrix Spec Changes on behalf of authors where needed + + * Triaging Matrix Spec issues + + * Coordinating reference implementations + + * Ensuring the code of conduct for +matrix:matrix.org community rooms is + maintained and applied + +If members are absent for more than 8 weeks without prior agreement, they will +be assumed to have left the project. + +Spec Core Team members can resign whenever they want, but must notify the rest +of the team and the Guardians on doing so. + +New additions to the team require 100% consent from the current team members. +Membership has to be formally proposed by someone already on the Spec Core Team. + +Members can be removed from the team if X% of the team agrees they are no longer +following the goals and guiding principles of the project. + +Guardians act as a backstop, and can appoint or remove Spec Core Team members +(requiring a 75% consensus threshold between the Guardians) if the Spec Core +Team is unable to reach consensus or is failing to align with the Foundation's +mission. + +It's suggested that one of the Spec Core Team members is also be a Guardian, to +facilitate information between the Guardians and the Spec Core Team and +represent the technical angle of the project to the other Guardians. + +The project lead role acts to coordinate the team and to help tie-break in the +event of failing to get acceptance on a Matrix Spec Change. The project lead is +reviewed every 12 months and requires the confidence of 75% of the team to be +renewed. There is no maximum term for the project lead. The lead may be removed +by the core team at any point (with 75% majority), and may resign the role at +any point (notifying the team and the Guardians). The lead automatically resigns +the role if they resign from the Spec Core Team. + +The initial Spec Core Team (and their domain areas) is: + + * Matthew Hodgson (Lead) + * Erik Johnston (Servers) + * Richard van der Hoff (Servers, Cryptography) + * David Baker (Clients, IS API, Push API, Media) + * Hubert Chathi (Cryptography, General) + * Andrew Morgan (Servers, AS API, Spec Process) + * Travis Ralston (Bots and Bridges & AS API, Media, acting with Dimension hat on) + * kitsune (Clients on behalf of Community) + * TBD + +MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment +Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC +process). + +The above governance process for the Spec Core Team is considered as part of the +spec and is updated using the Matrix Spec Change process. However, changes to +the governance process also require a 75% positive approval from the Guardians +(acting as a formal decision of the Foundation's Directors), in order to ensure +changes are aligned with the Foundation's mission. + +## The Guardians + +*This section will be used as the basis for the legal responsibilities of +Directors in the Articles of Association of the Foundation.* + +The Guardians form the legal Board of Directors of The Matrix.org Foundation CIC +(Community Interest Company). They are responsible for ensuring the Foundation +is following its guiding principles, and provide a safety mechanism if the +structure of the Spec Core Team runs into trouble. + +In practice, this means that: + * Guardians must approve changes to the Spec Core Team + * Guardians must keep each other honest, providing a ‘checks and balances’ + mechanism between each other to ensure that all Guardians and the Spec Core + Team act in the best interests of the protocol and ecosystem. + * Guardians may appoint/dismiss members of the Spec Core Team who are in serious + breach of the guiding principles. + * Guardians must approve changes to the Guiding Principles (above) + * Guardians are responsible for approving use of the Foundation's assets + (e.g. redistributing donations) + * In future, Guardians may also be responsible for ensuring staff are hired by + the Foundation to support administrative functions + * As well as the Spec Core Team committee, they may also oversee committees for + other areas such as marketing Matrix.org, registering custom event types, + or "Made for Matrix" certification. + * It's likely a subset of Guardians will be hands-on for day-to-day + administrative purposes, whilst the others act to keep them in balance. + +Guardians are chosen typically to be independent of the commercial Matrix +ecosystem (and especially independent from New Vector), and may even not be +members of today’s Matrix community. However, they should be deeply aligned with +the mission of the project, and respected and trusted by the wider community to +uphold the guiding principles of the Foundation and keep the other Guardians +honest. + +Guardians are responsible for maintaining and updating the Guiding +Principles and Articles of Association of the Foundation if/when +necessary. Changes to the Guiding Principles require a 75% majority from the +Guardians and are passed as a 'special resolution' of the board. + +New Guardians may be appointed with a 75% majority by the board. + +Guardians may resign at any time, with notification to the board. + +Guardians may be removed due to serious breach of the guiding principles with a +75% majority of the other Guardians, or if absent from 3 consecutive board +meetings, or if they are legally disqualified from acting as a Director. + +We aim to recruit roughly 5 Guardians. The initial Guardians are: + + * Matthew Hodgson (CEO/CTO, New Vector) + * Amandine Le Pape (COO, New Vector) + * TBA (agreed, needs paperwork) + * TBD + * TBD + +The intention is for Matthew & Amandine (the original founders of Matrix) to +form a minority of the Guardians, in order to ensure the neutrality of the +Foundation relative to Matthew & Amandine’s day jobs at New Vector. + +Guardians volunteer their time for free to work on the project. + +## The Core Team + +"The Core Team" is a loose term that describes the set of people with access to +commit code to the public https://github.com/matrix-org repositories, who are +either working on matrix.org's reference implementations or the spec itself. +Commit access is decided by those responsible for the projects in question, much +like any other open source project. Anyone is eligible for commit access if +they have proved themselves a valuable long-term contributor, upholds the +guiding principles and mission of the project and have proved themselves able to +collaborate constructively with the existing core team. + +## Responsibilities for the Foundation + + * Independent legal entity to act as neutral custodian of Matrix + * Gathering donations + * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: + * Owns the matrix.org domain and branding + * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). + By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. + * Owns the IP of the website + * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) + * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) + * Publishing the spec + * Responsible for sytest + * Manages IANA-style allocations for Matrix + * mx:// URI scheme? + * TCP port 8448 + * .well-known URIs…? + +In future: + + * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) + * manage a Matrix certification process? + * promote Matrix (e.g. organise meetups & events & fund community activity)? + +## Timings + +The Foundation was incorporated in October 2018 as a UK limited by guarantee +private company, using generic non-profit articles of association combined with +a high-level mission lock aligned with the above: + +> 4. The objects of the Foundation are for the benefit of the community as a whole +> to: + +> 4.1.1 empower users to control their communication data and have freedom over +> their communications infrastructure by creating, maintaining and promoting +> Matrix as an openly standardised secure decentralised communication protocol and +> network, open to all, and available to the public for no charge; + +> 4.1.2 build and develop an appropriate governance model for Matrix through the +> Foundation, in order to drive the adoption of Matrix as a single global +> federation, an open standard unencumbered from any proprietary intellectual +> property and/or software patents, minimising fragmentation (whilst encouraging +> experimentation), maximising speed of development, and prioritising the long- +> term success and growth of the overall network over the commercial concerns of +> an individual person or persons. + +The foundation was then converted into a Community Interest Company, formalising +its non-profit status under the approval of the independent [Community Interest +Companies Regulator](https://www.gov.uk/government/organisations/office-of-the- +regulator- of-community-interest-companies), which took effect Jan 2019. + +We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and +finalise the Foundation's articles of association shortly afterwards based on the +contents of this MSC once passed FCP. + +This will coincide with the formal asset transfer of Matrix.org's assets from +New Vector Ltd, and the appointment of the remaining Guardians. From 303e1081f8e0647b2acd67a4050793789cb5e2a7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:44:59 +0000 Subject: [PATCH 02/79] unbreak wordwrap --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index a79d175b..cf5d0d20 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -375,8 +375,8 @@ a high-level mission lock aligned with the above: The foundation was then converted into a Community Interest Company, formalising its non-profit status under the approval of the independent [Community Interest -Companies Regulator](https://www.gov.uk/government/organisations/office-of-the- -regulator- of-community-interest-companies), which took effect Jan 2019. +Companies Regulator](https://www.gov.uk/government/organisations/office-of-the-regulator-of-community-interest-companies), +which took effect Jan 2019. We are currently planning to release r0 of the Matrix Spec at the end of Jan 2019, and finalise the Foundation's articles of association shortly afterwards based on the From e6fb403dd0919018fced9525412da6f9eaef6701 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:47:37 +0000 Subject: [PATCH 03/79] add TODO for clarifying features v. extensions --- proposals/1779-open-governance.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index cf5d0d20..006d65b3 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -121,6 +121,10 @@ rather than closed deployments. For example, if WhatsApp adopted Matrix it wouldn’t be a complete win unless they openly federated with the rest of the Matrix network. +TODO: spell out when features should land in the spec, versus via +integration/widget or other non-core extension. e.g. should video conferencing +be in the spec itself, or done via Jitsi? + ### Values As Matrix evolves, it's critical that the Spec Core Team and Guardians are From c073adac9bbdaac30cc24e941185163da19cda12 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:51:04 +0000 Subject: [PATCH 04/79] incorporate review from #1318 --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 006d65b3..4fb564a5 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,6 +344,7 @@ collaborate constructively with the existing core team. * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) * Publishing the spec * Responsible for sytest + * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? * TCP port 8448 @@ -352,7 +353,7 @@ collaborate constructively with the existing core team. In future: * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) - * manage a Matrix certification process? + * manage a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) * promote Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From de6a8b20ff26f317d8a3ca8988f15cd59bd1d802 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 02:53:49 +0000 Subject: [PATCH 05/79] grammar --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4fb564a5..79213f89 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -10,7 +10,7 @@ communication standard on behalf of the whole Matrix community. See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ for more context. -This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318) +This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318). ## Introduction From b8249067090fc533676d23fff7a13fd16fc4ee4f Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 7 Jan 2019 03:23:36 +0000 Subject: [PATCH 06/79] typoes --- proposals/1779-open-governance.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 79213f89..34230d70 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -29,8 +29,8 @@ such that once the specification has finally been ‘born’ and reached an init Vector and better support contributions from the whole ecosystem. The concepts here are somewhat inspired by [Rust’s Governance -Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust- -governance.md); a highly regarded solution to a similar problem: an ambitious +Model](https://github.com/rust-lang/rfcs/blob/master/text/1068-rust-governance.md); +a highly regarded solution to a similar problem: an ambitious open-source project which has been too many years in the making, incubated at first by a single company (Mozilla Corporation), which also enjoys a very enthusiastic community! @@ -224,8 +224,8 @@ Guardians act as a backstop, and can appoint or remove Spec Core Team members Team is unable to reach consensus or is failing to align with the Foundation's mission. -It's suggested that one of the Spec Core Team members is also be a Guardian, to -facilitate information between the Guardians and the Spec Core Team and +It's suggested that one of the Spec Core Team members should also be a Guardian, +to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help tie-break in the From 4994fa115e41fd4b8777b474465c3e3702589f4a Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:21:19 +0000 Subject: [PATCH 07/79] X=75 --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 34230d70..4181d7de 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -216,8 +216,8 @@ of the team and the Guardians on doing so. New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. -Members can be removed from the team if X% of the team agrees they are no longer -following the goals and guiding principles of the project. +Members can be removed from the team if >= 75% of the team agrees they are no +longer following the goals and guiding principles of the project. Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core From 4fcd38a3a03a48d0610225453ab5adbc51636ab7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:27:03 +0000 Subject: [PATCH 08/79] clarify guardians' right to override spec core team membership --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4181d7de..07e275dc 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -274,7 +274,8 @@ In practice, this means that: mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. * Guardians may appoint/dismiss members of the Spec Core Team who are in serious - breach of the guiding principles. + breach of the guiding principles. This overrides the unanimous consent + requirement for the Spec Core Team when appointing new members. * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) From 7831c04e4caa9459cb527338e9ff6ea75ff8703f Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:29:24 +0000 Subject: [PATCH 09/79] clarify sytest responsibilities --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 07e275dc..6ba52f4d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,7 +344,8 @@ collaborate constructively with the existing core team. * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) * Publishing the spec - * Responsible for sytest + * Responsible for tools and documentation that supports the spec + * Responsible for ensuring reference implementations and test suite exists for the spec * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? From e730cc02a9d5ec77a70296b30c66a45e8ae41bf6 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:34:20 +0000 Subject: [PATCH 10/79] specify how to select a new spec core team lead --- proposals/1779-open-governance.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 6ba52f4d..67d66b2a 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -234,7 +234,9 @@ reviewed every 12 months and requires the confidence of 75% of the team to be renewed. There is no maximum term for the project lead. The lead may be removed by the core team at any point (with 75% majority), and may resign the role at any point (notifying the team and the Guardians). The lead automatically resigns -the role if they resign from the Spec Core Team. +the role if they resign from the Spec Core Team. Resignation automatically +triggers selection of a new lead, who must be selected from the existing core +spec team. The initial Spec Core Team (and their domain areas) is: @@ -276,6 +278,8 @@ In practice, this means that: * Guardians may appoint/dismiss members of the Spec Core Team who are in serious breach of the guiding principles. This overrides the unanimous consent requirement for the Spec Core Team when appointing new members. + * Guardians may also override deadlocks when appointing a Spec Core Team leader + (with a >= 75% majority) * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) From 103d2f4ed223be24f798b7d45a4f5cb87c6e8720 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:36:32 +0000 Subject: [PATCH 11/79] clarify that the project lead doesn't have casting vote --- proposals/1779-open-governance.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 67d66b2a..6280a45b 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -228,15 +228,15 @@ It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. -The project lead role acts to coordinate the team and to help tie-break in the -event of failing to get acceptance on a Matrix Spec Change. The project lead is -reviewed every 12 months and requires the confidence of 75% of the team to be -renewed. There is no maximum term for the project lead. The lead may be removed -by the core team at any point (with 75% majority), and may resign the role at -any point (notifying the team and the Guardians). The lead automatically resigns -the role if they resign from the Spec Core Team. Resignation automatically -triggers selection of a new lead, who must be selected from the existing core -spec team. +The project lead role acts to coordinate the team and to help steer the team to +consensus in the event of failing to get agreement on a Matrix Spec Change. The +project lead is reviewed every 12 months and requires the confidence of 75% of +the team to be renewed. There is no maximum term for the project lead. The lead +may be removed by the core team at any point (with 75% majority), and may resign +the role at any point (notifying the team and the Guardians). The lead +automatically resigns the role if they resign from the Spec Core Team. +Resignation automatically triggers selection of a new lead, who must be selected +from the existing core spec team. The initial Spec Core Team (and their domain areas) is: From 2047ba59daf0632b04ed974dc3598c2224b9335e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:42:49 +0000 Subject: [PATCH 12/79] spell out domain spread requirement for spec core team --- proposals/1779-open-governance.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 6280a45b..fb15027a 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -238,6 +238,10 @@ automatically resigns the role if they resign from the Spec Core Team. Resignation automatically triggers selection of a new lead, who must be selected from the existing core spec team. +It is vital that the core spec team has strong domain expertise covering all +different domains of the spec (e.g. we don't want to end up with a core spec +team where nobody has strong experience in cryptography) + The initial Spec Core Team (and their domain areas) is: * Matthew Hodgson (Lead) From c05000b38dd7529fe79f582091de6bea1590509d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:43:44 +0000 Subject: [PATCH 13/79] broaden reasons for dysfunctional core spec teams --- proposals/1779-open-governance.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index fb15027a..2c67a5ca 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -221,8 +221,7 @@ longer following the goals and guiding principles of the project. Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core -Team is unable to reach consensus or is failing to align with the Foundation's -mission. +Team is unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information between the Guardians and the Spec Core Team and From 0e246b147717b1b7f7d40968211a39b365d6effa Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 8 Jan 2019 14:44:18 +0000 Subject: [PATCH 14/79] Update proposals/1779-open-governance.md Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2c67a5ca..1a836322 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -243,7 +243,7 @@ team where nobody has strong experience in cryptography) The initial Spec Core Team (and their domain areas) is: - * Matthew Hodgson (Lead) + * Matthew Hodgson (Lead, Guardian) * Erik Johnston (Servers) * Richard van der Hoff (Servers, Cryptography) * David Baker (Clients, IS API, Push API, Media) From 5235293623bfbe62301d8514c6055d321f0aad06 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:50:18 +0000 Subject: [PATCH 15/79] spell out the Core Team a bit more --- proposals/1779-open-governance.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2c67a5ca..6f8af97d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -328,16 +328,27 @@ Foundation relative to Matthew & Amandine’s day jobs at New Vector. Guardians volunteer their time for free to work on the project. -## The Core Team - -"The Core Team" is a loose term that describes the set of people with access to -commit code to the public https://github.com/matrix-org repositories, who are -either working on matrix.org's reference implementations or the spec itself. -Commit access is decided by those responsible for the projects in question, much -like any other open source project. Anyone is eligible for commit access if -they have proved themselves a valuable long-term contributor, upholds the -guiding principles and mission of the project and have proved themselves able to -collaborate constructively with the existing core team. +## The Code Core Team (aka The Core Team) + +The "Core Team" (or the "Code Core Team", to disambiguate from the Spec Core +Team) is a loose term that describes the set of people with access to commit +code to the public https://github.com/matrix-org repositories, who are either +working on matrix.org's reference implementations or the spec itself. Commit +access is decided by those responsible for the projects in question, much like +any other open source project. Anyone is eligible for commit access if they +have proved themselves a valuable long-term contributor, upholds the guiding +principles and mission of the project and have proved themselves able to +collaborate constructively with the existing core team. Active participation in +the core team is also signified by membership of the +matrix:matrix.org Matrix +community. + +TODO: spell out some responsibilities. Erik suggests something like: + * Helping to ensure the quality of the projects' repositories + * That all projects follow the Matrix spec + * Engaging with the people in a way that fosters a healthy and happy community + * Following the Guiding Principles and promoting them within the community + +Code Core Team members volunteer their time for free to work on the project. ## Responsibilities for the Foundation From 3a5d56467bcaacf3f1672f3ed522c61da8c097a0 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 8 Jan 2019 14:51:45 +0000 Subject: [PATCH 16/79] reword lead renewals --- proposals/1779-open-governance.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f7712197..1e78d8e0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -228,14 +228,14 @@ to facilitate information between the Guardians and the Spec Core Team and represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to -consensus in the event of failing to get agreement on a Matrix Spec Change. The -project lead is reviewed every 12 months and requires the confidence of 75% of -the team to be renewed. There is no maximum term for the project lead. The lead -may be removed by the core team at any point (with 75% majority), and may resign -the role at any point (notifying the team and the Guardians). The lead -automatically resigns the role if they resign from the Spec Core Team. -Resignation automatically triggers selection of a new lead, who must be selected -from the existing core spec team. +consensus in the event of failing to get agreement on a Matrix Spec Change. +Every 12 months, a vote of confidence is held in the project lead, requiring the +confidence of 75% of the team for the lead to be renewed. There is no maximum +term for the project lead. The lead may be removed by the core team at any +point (with 75% majority), and may resign the role at any point (notifying the +team and the Guardians). The lead automatically resigns the role if they resign +from the Spec Core Team. Resignation automatically triggers selection of a new +lead, who must be selected from the existing core spec team. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec From 03802701715d4ac1e404e14d619c1c80be4fca2e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Wed, 9 Jan 2019 23:29:59 +0000 Subject: [PATCH 17/79] spell out that hypothetical employees could come in any size --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 1e78d8e0..8834da30 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -287,7 +287,8 @@ In practice, this means that: * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations) * In future, Guardians may also be responsible for ensuring staff are hired by - the Foundation to support administrative functions + the Foundation to support administrative functions and other roles required + to facilitate the Foundation's mission. * As well as the Spec Core Team committee, they may also oversee committees for other areas such as marketing Matrix.org, registering custom event types, or "Made for Matrix" certification. From c053996a7749b810affe871f36541de481b152c7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:12:56 +0000 Subject: [PATCH 18/79] add new technical guidelines to proposals.rst this was originally a todo for MSC1779, but belongs better in proposals.rst --- specification/proposals_intro.rst | 69 +++++++++++++++++++++++++++++-- 1 file changed, 65 insertions(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 73ace98e..049ad0c8 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -44,15 +44,18 @@ moderators and admins, companies/projects building products or services on Matrix, spec contributors, translators, and those who created it in the first place. -"Greater benefit" could include maximising: +"Greater benefit" includes maximising: * the number of end-users reachable on the open Matrix network -* the number of regular users on the Matrix network (e.g. 30-day retained - federated users) +* the number of regular users on the Matrix network (e.g. 30-day retained federated users) +* the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix -* the quality and utility of the Matrix spec +* the quality and utility of the Matrix spec (as defined by ease and ability + with which a developer can implement spec-compliant clients, servers, bots, + bridges, and other integrations without needing to refer to any other + external material) In addition, proposal authors are expected to uphold the following values in their proposed changes to the Matrix protocol: @@ -66,6 +69,64 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture +Technical notes +--------------- + +Proposals **must** develop Matrix as a layered protocol: with new features +building on layers of shared abstractions rather than introducing tight vertical +coupling within the stack. This ensures that new features can evolve rapidly by +building on existing layers and swapping out old features without impacting the +rest of the stack or requiring substantial upgrades to the whole ecosystem. +This is critical for Matrix to rapidly evolve and compete effectively with +centralised systems, despite being a federated protocol. + +For instance, new features should be implemented using the highest layer +abstractions possible (e.g. new event types, which layer on top of the existing +room semantics, and so don't even require any API changes). Failing that, the +next recourse would be backwards-compatible changes to the next layer down (e.g. +room APIs); failing that, considering changes to the format of events or the +DAG; etc. It would be a very unusual feature which doesn't build on the +existing infrastructure provided by the spec and instead created new primitives +or low level APIs. + +Backwards compatibility is very important for Matrix, but not at the expense of +hindering the protocol's evolution. Backwards incompatible changes to endpoints +are allowed when no other alternative exists, and must be versioned under a new +major release of the API. Backwards incompatible changes to the room algorithm +are also allowed when no other alternative exists, and must be versioned under a +new version of the room algorithm. + +There is sometimes a dilemma over where to include higher level features: for +instance, should video conferencing be formalised in the spec, or should it be +implemented via widgets (if one assumes that widgets have landed in the spec and +[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) is merged)? +Should reputation systems be specified? Should search engine behaviour be +specified? + +There is no universal answer to this, but the following guidelines should be +applied: + * If the feature would benefit the whole Matrix ecosystem and is aligned with + the guiding principles above, then it should be supported by the spec. + For instance, video conferencing is clearly a feature which would benefit + the whole ecosystem, and so the spec should find a way to make it happen. + * If the spec already makes the feature possible without changing any of the + spec *or implementations*, then it may not need to be added to the spec. + For instance, video conferencing done by widgets requires no compulsory + changes to clients nor servers to work, and so could be omitted. + * However, if the best user experience for a feature does require custom + implementation behaviour - e.g. embedding Jitsi into your client rather than + using a widget, then the behaviour should be defined in the spec to allow + implementations to do so. + * However, the spec must never add dependencies on unspecified/nonstandardised + 3rd party behaviour. For instance, defining how to embed Jitsi is unlikely to + ever make it into the spec, given Jitsi does not implement a standardised + interface (although a URL-based calling standard may emerge in future, which + could be used as an extension to the current widget-based approach). + * Therefore, our two options in the specific case of video conferencing are + either to spec SFU conferencing semantics on WebRTC (or refer to an existing spec + for doing so), or to keep it as a widget-based approach (optionally with widget + extensions specific for more deeply integrating video conferencing use cases). + Process ------- From edaf3596f470323a66c74383afb70332d26c28d9 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:14:36 +0000 Subject: [PATCH 19/79] fix 'which features should go in the spec?' TODO and reword intro --- proposals/1779-open-governance.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 8834da30..9db96756 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -1,11 +1,11 @@ # Proposal for Open Governance of Matrix.org -This whole document is a **work in progress** draft of a constitution proposal -for open governance for Matrix.org, and forms the basis of the first full -Articles of Association (AoA) for [The Matrix.org Foundation -C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit legal -entity incorporated to act as the neutral guardian of the Matrix decentralised -communication standard on behalf of the whole Matrix community. +This whole document is the proposed constitution proposal for Matrix.org, and +will form the basis of the first full Articles of Association (AoA) for [The +Matrix.org Foundation +C.I.C.](https://beta.companieshouse.gov.uk/company/11648710) - a non-profit +legal entity incorporated to act as the neutral guardian of the Matrix +decentralised communication standard on behalf of the whole Matrix community. See https://matrix.org/blog/2018/10/29/introducing-the-matrix-org-foundation-part-1-of-2/ for more context. @@ -121,10 +121,6 @@ rather than closed deployments. For example, if WhatsApp adopted Matrix it wouldn’t be a complete win unless they openly federated with the rest of the Matrix network. -TODO: spell out when features should land in the spec, versus via -integration/widget or other non-core extension. e.g. should video conferencing -be in the spec itself, or done via Jitsi? - ### Values As Matrix evolves, it's critical that the Spec Core Team and Guardians are From 21a781b1d2d5c035763da2d734613d323888eb3d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:16:24 +0000 Subject: [PATCH 20/79] fix proprietary wording --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9db96756..f829f1d8 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -168,7 +168,7 @@ agendas at the expense of the overall project. We are particularly vigilant against 'trojan horse' additions to the spec - features which only benefit particular players, or are designed to somehow cripple or fragment the open protocol and ecosystem in favour of competitive -advantage. Commercial players are of course encouraged to build proprietary +advantage. Commercial players are of course free to build proprietary implementations, or use custom event types, or even custom API extensions (e.g. more efficient network transports) - but implementations must fall back to interoperating correctly with the rest of the ecosystem. From 265a3dc49b3a00f7ce80e00e8b34bc13a5612adc Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:20:09 +0000 Subject: [PATCH 21/79] clarify removing members --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f829f1d8..9013f6b6 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -213,7 +213,8 @@ New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. Members can be removed from the team if >= 75% of the team agrees they are no -longer following the goals and guiding principles of the project. +longer following the goals and guiding principles of the project. (The 75% is +measured of the whole team, including the member in question) Guardians act as a backstop, and can appoint or remove Spec Core Team members (requiring a 75% consensus threshold between the Guardians) if the Spec Core From e584ae31f4e1e5bf806250a06e28b1bb3d9abcee Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:21:02 +0000 Subject: [PATCH 22/79] fix missing words --- proposals/1779-open-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9013f6b6..808e8368 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -221,8 +221,8 @@ Guardians act as a backstop, and can appoint or remove Spec Core Team members Team is unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, -to facilitate information between the Guardians and the Spec Core Team and -represent the technical angle of the project to the other Guardians. +to facilitate information exchange between the Guardians and the Spec Core Team, +and to represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to consensus in the event of failing to get agreement on a Matrix Spec Change. From ed820ca27bdc00845f085c48fbf73c06f42c5202 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sat, 12 Jan 2019 23:21:52 +0000 Subject: [PATCH 23/79] s/core spec team/Spec Core Team/ Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 808e8368..2d0e21ed 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -232,7 +232,7 @@ term for the project lead. The lead may be removed by the core team at any point (with 75% majority), and may resign the role at any point (notifying the team and the Guardians). The lead automatically resigns the role if they resign from the Spec Core Team. Resignation automatically triggers selection of a new -lead, who must be selected from the existing core spec team. +lead, who must be selected from the existing Spec Core Team. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec From b758ceea5b3976f0ae64da5ad7cf8855f0ee5ab7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:23:46 +0000 Subject: [PATCH 24/79] add erik's quorum --- proposals/1779-open-governance.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 2d0e21ed..58142027 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -177,7 +177,8 @@ interoperating correctly with the rest of the ecosystem. The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. Roughly half the members are expected to be from the historical core team -(similar to Rust). +(similar to Rust). The team must have 5 members to be quorate, with the aim of +generally having between 7 and 9 members. In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; IS API; Crypto), but as a starting point we are beginning with a single core From 107d96c50e4882924d4823fef92967e09e9bc223 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:24:55 +0000 Subject: [PATCH 25/79] spell out consensus ftw --- proposals/1779-open-governance.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 58142027..76c6ffa9 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -255,6 +255,9 @@ MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC process). +Even though a threshold of only 75% is required for approval, the Spec Core Team +is expected to seek consensus on MSCs. + The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to the governance process also require a 75% positive approval from the Guardians From cccd62f03530fcaa78fa38b8fa4992518058100a Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sat, 12 Jan 2019 23:25:37 +0000 Subject: [PATCH 26/79] grammar Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 76c6ffa9..4de598a9 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -338,7 +338,7 @@ code to the public https://github.com/matrix-org repositories, who are either working on matrix.org's reference implementations or the spec itself. Commit access is decided by those responsible for the projects in question, much like any other open source project. Anyone is eligible for commit access if they -have proved themselves a valuable long-term contributor, upholds the guiding +have proved themselves a valuable long-term contributor, uphold the guiding principles and mission of the project and have proved themselves able to collaborate constructively with the existing core team. Active participation in the core team is also signified by membership of the +matrix:matrix.org Matrix From d00a5eba93926b768555d53a7b081c3a4ccfc9b0 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:26:58 +0000 Subject: [PATCH 27/79] consistent tenses --- proposals/1779-open-governance.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 4de598a9..16047a2d 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -354,19 +354,19 @@ Code Core Team members volunteer their time for free to work on the project. ## Responsibilities for the Foundation - * Independent legal entity to act as neutral custodian of Matrix - * Gathering donations + * Independent legal entity which acts as neutral custodian of Matrix + * Gathers donations * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: * Owns the matrix.org domain and branding * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. * Owns the IP of the website * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) - * It's responsible for finding someone to run the Matrix.org homeserver (currently New Vector) - * Publishing the spec + * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector) + * Publishes the spec * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec - * Publishing the website (including ensuring This Week In Matrix and similar exist to promote independent projects) + * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) * Manages IANA-style allocations for Matrix * mx:// URI scheme? * TCP port 8448 @@ -374,9 +374,9 @@ Code Core Team members volunteer their time for free to work on the project. In future: - * contract entities to work on Matrix? (e.g. redistributing donations back to fund development) - * manage a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) - * promote Matrix (e.g. organise meetups & events & fund community activity)? + * contracts entities to work on Matrix? (e.g. redistributing donations back to fund development) + * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) + * promotes Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From ddc3921318262ad7134b37253030ac5a182833ca Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:31:01 +0000 Subject: [PATCH 28/79] un-todo code core team responsibilities --- proposals/1779-open-governance.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 16047a2d..f30d50d0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -344,10 +344,14 @@ collaborate constructively with the existing core team. Active participation in the core team is also signified by membership of the +matrix:matrix.org Matrix community. -TODO: spell out some responsibilities. Erik suggests something like: - * Helping to ensure the quality of the projects' repositories - * That all projects follow the Matrix spec - * Engaging with the people in a way that fosters a healthy and happy community +Responsibilities include: + * Helping ensure the quality of the projects' code repositories + * Ensuring all commits are reviewed + * Ensuring that all projects follow the Matrix spec + * Helping architect the implementations + * Contributing code to the implementations + * Fostering contributions and engaging with contributors constructively in a + way that fosters a healthy and happy community * Following the Guiding Principles and promoting them within the community Code Core Team members volunteer their time for free to work on the project. From 156488384c75aa23290da0cd845902fffb043d1d Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:49:27 +0000 Subject: [PATCH 29/79] add more examples for spec inclusion; add interoperability as a core value --- proposals/1779-open-governance.md | 1 + specification/proposals_intro.rst | 16 +++++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f30d50d0..f5a89f5e 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -129,6 +129,7 @@ subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness + * Interoperability rather than fragmentation * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 049ad0c8..b7663e66 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -62,6 +62,7 @@ their proposed changes to the Matrix protocol: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness +* Interoperability rather than fragmentation * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -110,7 +111,7 @@ applied: For instance, video conferencing is clearly a feature which would benefit the whole ecosystem, and so the spec should find a way to make it happen. * If the spec already makes the feature possible without changing any of the - spec *or implementations*, then it may not need to be added to the spec. + implementations and spec, then it may not need to be added to the spec. For instance, video conferencing done by widgets requires no compulsory changes to clients nor servers to work, and so could be omitted. * However, if the best user experience for a feature does require custom @@ -127,6 +128,19 @@ applied: for doing so), or to keep it as a widget-based approach (optionally with widget extensions specific for more deeply integrating video conferencing use cases). +As an alternative example: it's very unlikely that "how to visualise Magnetic +Resonsance Imaging data over Matrix" would ever be added to the Matrix spec +(other than perhaps a custom event type in a wider standardised Matrix event +registry) given that the spec's existing primitives of file transfer and +extensible events (MSC1767) give excellent tools for transferring and +visualising arbitrary rich data. + +Conversely, features such as reactions, threaded messages, editable messages, +spam/abuse/content filtering, are all features which would clearly benefit the +whole Matrix ecosystem and require both client & server implementation +changes across the board to be implemented in an interoperable way, and so +necessitate a spec change. + Process ------- From f3085812e9c160f63e0cb542378eb827112a54c2 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Sat, 12 Jan 2019 23:51:49 +0000 Subject: [PATCH 30/79] more examples; remove widget MSC ref --- specification/proposals_intro.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index b7663e66..c8a2fd32 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -99,10 +99,8 @@ new version of the room algorithm. There is sometimes a dilemma over where to include higher level features: for instance, should video conferencing be formalised in the spec, or should it be -implemented via widgets (if one assumes that widgets have landed in the spec and -[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) is merged)? -Should reputation systems be specified? Should search engine behaviour be -specified? +implemented via widgets? Should reputation systems be specified? Should search +engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: @@ -135,11 +133,17 @@ registry) given that the spec's existing primitives of file transfer and extensible events (MSC1767) give excellent tools for transferring and visualising arbitrary rich data. +Supporting public search engines are likely to not require custom spec features +(other than possibly better bulk access APIs), given they can be implemented as +clients using the existing CS API. An exception could be API features required +by decentralised search infrastructure (avoiding centralisation of power by +a centralised search engine). + Conversely, features such as reactions, threaded messages, editable messages, -spam/abuse/content filtering, are all features which would clearly benefit the -whole Matrix ecosystem and require both client & server implementation -changes across the board to be implemented in an interoperable way, and so -necessitate a spec change. +spam/abuse/content filtering (and reputation systems), are all features which +would clearly benefit the whole Matrix ecosystem and require both client & +server implementation changes across the board to be implemented in an +interoperable way, and so necessitate a spec change. Process ------- From 09813fc3a080600c13ccaaa74deb61c9df07c1ee Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 12:41:19 +0000 Subject: [PATCH 31/79] improve wording around compensation for team members. tweak Greater Benefit (particularly adding SnR and filtering goals --- proposals/1779-open-governance.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f5a89f5e..63684e40 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -105,12 +105,14 @@ protocol. This includes (non-exhaustively): "Greater benefit" is defined as maximising: - * the number of end-users reachable on the open Matrix network + * the number of Matrix-native end-users reachable on the open Matrix network * the number of regular users on the Matrix network (e.g. 30-day retained federated users) - * the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix + * the number of bridged end-users reachable on the open Matrix network + * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) + * the ability for users to discover content on their terms (empowering them to select what to see and what not to see) * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other @@ -130,6 +132,7 @@ subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain * Openness rather than proprietariness * Interoperability rather than fragmentation + * Cross-platform rather than platform-specific * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -187,7 +190,7 @@ team in the interests of not over-engineering it and scaling up elastically. Spec Core Team members need to be able to commit to at least 1 hour a week of availability to work on the spec and (where relevant) reference implementations. -Members of the team volunteer their time for free to work on the project. +Members must arrange their own funding for their time. Responsibilities include: @@ -329,7 +332,7 @@ The intention is for Matthew & Amandine (the original founders of Matrix) to form a minority of the Guardians, in order to ensure the neutrality of the Foundation relative to Matthew & Amandine’s day jobs at New Vector. -Guardians volunteer their time for free to work on the project. +Guardians must arrange their own funding for their time. ## The Code Core Team (aka The Core Team) @@ -355,7 +358,7 @@ Responsibilities include: way that fosters a healthy and happy community * Following the Guiding Principles and promoting them within the community -Code Core Team members volunteer their time for free to work on the project. +Code Core Team members must arrange their own funding for their time. ## Responsibilities for the Foundation From 2f20679db514ec6b58970bcf91e18a3f3c46270a Mon Sep 17 00:00:00 2001 From: Kitsune Ral Date: Mon, 14 Jan 2019 12:48:12 +0000 Subject: [PATCH 32/79] deanonymise kitsune Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 63684e40..a98f36a0 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -252,7 +252,7 @@ The initial Spec Core Team (and their domain areas) is: * Hubert Chathi (Cryptography, General) * Andrew Morgan (Servers, AS API, Spec Process) * Travis Ralston (Bots and Bridges & AS API, Media, acting with Dimension hat on) - * kitsune (Clients on behalf of Community) + * Alexey Rusakov (Clients on behalf of Community) * TBD MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment From a38e1e6adff27c649b6ac964f7787535b1aa988e Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 12:54:21 +0000 Subject: [PATCH 33/79] incorporate kitsune & dbkr review --- proposals/1779-open-governance.md | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index a98f36a0..0b8c0aeb 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -75,7 +75,7 @@ We believe: * The ability to converse securely and privately is a basic human right. - * Communication should be available to everyone as an free and open, + * Communication should be available to everyone as a free and open, unencumbered, standard and global network. ### Mission @@ -130,7 +130,7 @@ aligned on the overall philosophy of the project, particularly in more subjective areas. The values we follow are: * Supporting the whole long-term ecosystem rather than individual stakeholder gain - * Openness rather than proprietariness + * Openness rather than proprietary lock-in * Interoperability rather than fragmentation * Cross-platform rather than platform-specific * Collaboration rather than competition @@ -142,6 +142,10 @@ subjective areas. The values we follow are: Patent encumbered IP is strictly prohibited from being added to the standard. +Making the specification rely on non-standard/unspecified behaviour of other +systems or actors (such as SaaS services, even open-sourced, not governed by a +standard protocol) shall not be accepted, either. + ## The Spec Core Team The contents and direction of the Matrix Spec is governed by the Spec Core Team; @@ -208,8 +212,8 @@ Responsibilities include: * Ensuring the code of conduct for +matrix:matrix.org community rooms is maintained and applied -If members are absent for more than 8 weeks without prior agreement, they will -be assumed to have left the project. +If members are absent (uncontactable) for more than 8 weeks without prior +agreement, they will be assumed to have left the project. Spec Core Team members can resign whenever they want, but must notify the rest of the team and the Guardians on doing so. @@ -375,16 +379,20 @@ Code Core Team members must arrange their own funding for their time. * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) - * Manages IANA-style allocations for Matrix - * mx:// URI scheme? + * Manages IANA-style allocations for Matrix, including: + * mx:// URI scheme * TCP port 8448 - * .well-known URIs…? + * .well-known URIs +* Ensures that Matrix promotion is happening (e.g. ensuring that meetups & + events & community activity is supported) In future: - * contracts entities to work on Matrix? (e.g. redistributing donations back to fund development) + * contracts entities to work on Matrix if such contracts help the Foundation to + fulfil its mission and obey the Guiding Principles (e.g. redistributing + donations back to fund development of reference implementations or compliance + kits) * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) - * promotes Matrix (e.g. organise meetups & events & fund community activity)? ## Timings From 70be8393a391baecc23a322800f6843d63d8187a Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 13:01:50 +0000 Subject: [PATCH 34/79] clarify the technical note and guiding principles --- specification/proposals_intro.rst | 73 ++++++++++++++++++------------- 1 file changed, 42 insertions(+), 31 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index c8a2fd32..393a43a2 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -33,25 +33,28 @@ Guiding Principles Proposals **must** act to the greater benefit of the entire Matrix ecosystem, rather than benefiting or privileging any single player or subset of players - -and must not contain any patent encumbered intellectual property. Members of the Core Team pledge to act as +and must not contain any patent encumbered intellectual property. +Members of the Spec Core Team pledge to act as a neutral custodian for Matrix on behalf of the whole ecosystem. For clarity: the Matrix ecosystem is anyone who uses the Matrix protocol. That -includes client users, server admins, client developers, bot developers, -bridge and application service developers, users and admins who are indirectly using Matrix via -3rd party networks which happen to be bridged, server developers, room -moderators and admins, companies/projects building products or services on -Matrix, spec contributors, translators, and those who created it in -the first place. +includes client users, server admins, client developers, bot developers, bridge +and application service developers, users and admins who are indirectly using +Matrix via 3rd party networks which happen to be bridged, server developers, +room moderators and admins, companies/projects building products or services on +Matrix, spec contributors, translators, and those who created it in the first +place. -"Greater benefit" includes maximising: +"Greater benefit" is defined as maximising: -* the number of end-users reachable on the open Matrix network +* the number of Matrix-native end-users reachable on the open Matrix network * the number of regular users on the Matrix network (e.g. 30-day retained federated users) -* the number of end-users reachable by Matrix (natively or via bridges) * the number of online servers in the open federation * the number of developers building on Matrix * the number of independent implementations which use Matrix +* the number of bridged end-users reachable on the open Matrix network +* the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) +* the ability for users to discover content on their terms (empowering them to select what to see and what not to see) * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other @@ -61,8 +64,9 @@ In addition, proposal authors are expected to uphold the following values in their proposed changes to the Matrix protocol: * Supporting the whole long-term ecosystem rather than individual stakeholder gain -* Openness rather than proprietariness +* Openness rather than proprietary lock-in * Interoperability rather than fragmentation +* Cross-platform rather than platform-specific * Collaboration rather than competition * Accessibility rather than elitism * Transparency rather than stealth @@ -70,6 +74,9 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture +Please see [MSC1779](https://github.com/matrix-org/matrix-doc/pull/1779) +for full details of the project's Guiding Principles. + Technical notes --------------- @@ -104,33 +111,37 @@ engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: - * If the feature would benefit the whole Matrix ecosystem and is aligned with +1. If the feature would benefit the whole Matrix ecosystem and is aligned with the guiding principles above, then it should be supported by the spec. - For instance, video conferencing is clearly a feature which would benefit - the whole ecosystem, and so the spec should find a way to make it happen. - * If the spec already makes the feature possible without changing any of the +2. If the spec already makes the feature possible without changing any of the implementations and spec, then it may not need to be added to the spec. - For instance, video conferencing done by widgets requires no compulsory - changes to clients nor servers to work, and so could be omitted. - * However, if the best user experience for a feature does require custom - implementation behaviour - e.g. embedding Jitsi into your client rather than - using a widget, then the behaviour should be defined in the spec to allow - implementations to do so. - * However, the spec must never add dependencies on unspecified/nonstandardised - 3rd party behaviour. For instance, defining how to embed Jitsi is unlikely to - ever make it into the spec, given Jitsi does not implement a standardised - interface (although a URL-based calling standard may emerge in future, which - could be used as an extension to the current widget-based approach). - * Therefore, our two options in the specific case of video conferencing are - either to spec SFU conferencing semantics on WebRTC (or refer to an existing spec - for doing so), or to keep it as a widget-based approach (optionally with widget - extensions specific for more deeply integrating video conferencing use cases). +3. However, if the best user experience for a feature does require custom + implementation behaviour then the behaviour should be defined in the spec + such that all implementations may implement it. +4. However, the spec must never add dependencies on unspecified/nonstandardised + 3rd party behaviour. + +As a worked example: +1. Video conferencing is clearly a feature which would benefit + the whole ecosystem, and so the spec should find a way to make it happen. +2. Video conferencing can be achieved by widgets without requiring any + compulsory changes to changes to clients nor servers to work, and so could be + omitted from the spec. +3. A better experience could be achieved by embedding Jitsi natively into clients + rather than using a widget... +4. ...except that would add a dependency on unspecified/nonstandardised 3rd party + behaviour, so must not be added to the spec. + +Therefore, our two options in the specific case of video conferencing are +either to spec SFU conferencing semantics for WebRTC (or refer to an existing spec +for doing so), or to keep it as a widget-based approach (optionally with widget +extensions specific for more deeply integrating video conferencing use cases). As an alternative example: it's very unlikely that "how to visualise Magnetic Resonsance Imaging data over Matrix" would ever be added to the Matrix spec (other than perhaps a custom event type in a wider standardised Matrix event registry) given that the spec's existing primitives of file transfer and -extensible events (MSC1767) give excellent tools for transferring and +extensible events (MSC1767) give excellent tools for transfering and visualising arbitrary rich data. Supporting public search engines are likely to not require custom spec features From 811e65a4d1cf92b1314889330a01f86a16d23148 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 14 Jan 2019 13:14:43 +0000 Subject: [PATCH 35/79] kitsune review on when to add things to the spec --- specification/proposals_intro.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 393a43a2..1e93a22a 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -150,11 +150,10 @@ clients using the existing CS API. An exception could be API features required by decentralised search infrastructure (avoiding centralisation of power by a centralised search engine). -Conversely, features such as reactions, threaded messages, editable messages, +Features such as reactions, threaded messages, editable messages, spam/abuse/content filtering (and reputation systems), are all features which -would clearly benefit the whole Matrix ecosystem and require both client & -server implementation changes across the board to be implemented in an -interoperable way, and so necessitate a spec change. +would clearly benefit the whole Matrix ecosystem, and cannot be implemented in an +interoperable way using the current spec; so they necessitate a spec change. Process ------- From 9e435d6dabdeb2e482d15e1ceb9bcda0bbe9b432 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Mon, 14 Jan 2019 23:33:03 +0000 Subject: [PATCH 36/79] scifi IANA Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 0b8c0aeb..f6d5b959 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -379,7 +379,7 @@ Code Core Team members must arrange their own funding for their time. * Responsible for tools and documentation that supports the spec * Responsible for ensuring reference implementations and test suite exists for the spec * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) - * Manages IANA-style allocations for Matrix, including: + * Manages any future IANA-style allocations for Matrix, such as: * mx:// URI scheme * TCP port 8448 * .well-known URIs From 6ff0155a324510b863f3c66843817f4d45e5b9e7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:16:33 +0000 Subject: [PATCH 37/79] s/responsibilities/functions --- proposals/1779-open-governance.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index f6d5b959..28c18791 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -364,7 +364,7 @@ Responsibilities include: Code Core Team members must arrange their own funding for their time. -## Responsibilities for the Foundation +## Functions of the Foundation * Independent legal entity which acts as neutral custodian of Matrix * Gathers donations @@ -392,7 +392,8 @@ In future: fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance kits) - * manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix) + * manages a "Made for Matrix" certification process? (to confirm that products + are actually compatible with Matrix) ## Timings From 822d84e50c46f335efbd04af8f565fbf98af7b96 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:29:07 +0000 Subject: [PATCH 38/79] append fullstops to lists to make vdh happy --- proposals/1779-open-governance.md | 128 +++++++++++++++--------------- 1 file changed, 64 insertions(+), 64 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 28c18791..58e43b80 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -88,35 +88,35 @@ player or subset of players. For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix protocol. This includes (non-exhaustively): - * End-users of Matrix clients - * Matrix client developers and testers - * Spec developers - * Server admins - * Matrix packagers & maintainers - * Companies building products or services on Matrix - * Bridge developers - * Bot developers - * Widget developers - * Server developers - * Matrix room and community moderators - * End-users who are using Matrix indirectly via bridges - * External systems which are bridged into Matrix - * Anyone using Matrix for data communications + * End-users of Matrix clients. + * Matrix client developers and testers. + * Spec developers. + * Server admins. + * Matrix packagers & maintainers. + * Companies building products or services on Matrix. + * Bridge developers. + * Bot developers. + * Widget developers. + * Server developers. + * Matrix room and community moderators. + * End-users who are using Matrix indirectly via bridges. + * External systems which are bridged into Matrix. + * Anyone using Matrix for data communications. "Greater benefit" is defined as maximising: - * the number of Matrix-native end-users reachable on the open Matrix network - * the number of regular users on the Matrix network (e.g. 30-day retained federated users) - * the number of online servers in the open federation - * the number of developers building on Matrix - * the number of independent implementations which use Matrix - * the number of bridged end-users reachable on the open Matrix network - * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam) - * the ability for users to discover content on their terms (empowering them to select what to see and what not to see) + * the number of Matrix-native end-users reachable on the open Matrix network. + * the number of regular users on the Matrix network (e.g. 30-day retained federated users). + * the number of online servers in the open federation. + * the number of developers building on Matrix. + * the number of independent implementations which use Matrix. + * the number of bridged end-users reachable on the open Matrix network. + * the signal-to-noise ratio of the content on the open Matrix network (i.e. minimising spam). + * the ability for users to discover content on their terms (empowering them to select what to see and what not to see). * the quality and utility of the Matrix spec (as defined by ease and ability with which a developer can implement spec-compliant clients, servers, bots, bridges, and other integrations without needing to refer to any other - external material) + external material). N.B. that we consider success to be the growth of the open federated network rather than closed deployments. For example, if WhatsApp adopted Matrix it @@ -129,16 +129,16 @@ As Matrix evolves, it's critical that the Spec Core Team and Guardians are aligned on the overall philosophy of the project, particularly in more subjective areas. The values we follow are: - * Supporting the whole long-term ecosystem rather than individual stakeholder gain - * Openness rather than proprietary lock-in - * Interoperability rather than fragmentation - * Cross-platform rather than platform-specific - * Collaboration rather than competition - * Accessibility rather than elitism - * Transparency rather than stealth - * Empathy rather than contrariness - * Pragmatism rather than perfection - * Proof rather than conjecture + * Supporting the whole long-term ecosystem rather than individual stakeholder gain. + * Openness rather than proprietary lock-in. + * Interoperability rather than fragmentation. + * Cross-platform rather than platform-specific. + * Collaboration rather than competition. + * Accessibility rather than elitism. + * Transparency rather than stealth. + * Empathy rather than contrariness. + * Pragmatism rather than perfection. + * Proof rather than conjecture. Patent encumbered IP is strictly prohibited from being added to the standard. @@ -198,19 +198,19 @@ Members must arrange their own funding for their time. Responsibilities include: - * Reviewing Matrix Spec Change proposals and Spec PRs + * Reviewing Matrix Spec Change proposals and Spec PRs. * Contributing to and reviewing reference implementations of Matrix Spec Change - proposals + proposals. - * Shepherding Matrix Spec Changes on behalf of authors where needed + * Shepherding Matrix Spec Changes on behalf of authors where needed. - * Triaging Matrix Spec issues + * Triaging Matrix Spec issues. - * Coordinating reference implementations + * Coordinating reference implementations. * Ensuring the code of conduct for +matrix:matrix.org community rooms is - maintained and applied + maintained and applied. If members are absent (uncontactable) for more than 8 weeks without prior agreement, they will be assumed to have left the project. @@ -283,18 +283,18 @@ is following its guiding principles, and provide a safety mechanism if the structure of the Spec Core Team runs into trouble. In practice, this means that: - * Guardians must approve changes to the Spec Core Team - * Guardians must keep each other honest, providing a ‘checks and balances’ + * Guardians must approve changes to the Spec Core Team. + * Guardians must keep each other honest, providing a ‘checks and balances’. mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. * Guardians may appoint/dismiss members of the Spec Core Team who are in serious breach of the guiding principles. This overrides the unanimous consent requirement for the Spec Core Team when appointing new members. * Guardians may also override deadlocks when appointing a Spec Core Team leader - (with a >= 75% majority) + (with a >= 75% majority). * Guardians must approve changes to the Guiding Principles (above) * Guardians are responsible for approving use of the Foundation's assets - (e.g. redistributing donations) + (e.g. redistributing donations). * In future, Guardians may also be responsible for ensuring staff are hired by the Foundation to support administrative functions and other roles required to facilitate the Foundation's mission. @@ -354,46 +354,46 @@ community. Responsibilities include: * Helping ensure the quality of the projects' code repositories - * Ensuring all commits are reviewed - * Ensuring that all projects follow the Matrix spec - * Helping architect the implementations - * Contributing code to the implementations + * Ensuring all commits are reviewed. + * Ensuring that all projects follow the Matrix spec. + * Helping architect the implementations. + * Contributing code to the implementations. * Fostering contributions and engaging with contributors constructively in a - way that fosters a healthy and happy community - * Following the Guiding Principles and promoting them within the community + way that fosters a healthy and happy community. + * Following the Guiding Principles and promoting them within the community. Code Core Team members must arrange their own funding for their time. ## Functions of the Foundation - * Independent legal entity which acts as neutral custodian of Matrix - * Gathers donations + * Independent legal entity which acts as neutral custodian of Matrix. + * Gathers donations. * Owns the core Matrix IP in an asset lock, which shall be transferred from New Vector: - * Owns the matrix.org domain and branding + * Owns the matrix.org domain and branding. * Owns the copyright of the reference implementations of Matrix (i.e. everything in https://github.com/matrix-org). By assigning copyright to the Foundation, it’s protected against New Vector ever being tempted to relicense it. - * Owns the IP of the website - * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc) - * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector) - * Publishes the spec - * Responsible for tools and documentation that supports the spec - * Responsible for ensuring reference implementations and test suite exists for the spec - * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects) + * Owns the IP of the website. + * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc). + * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector). + * Publishes the spec. + * Responsible for tools and documentation that supports the spec. + * Responsible for ensuring reference implementations and test suite exists for the spec. + * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects). * Manages any future IANA-style allocations for Matrix, such as: - * mx:// URI scheme - * TCP port 8448 + * mx:// URI scheme. + * TCP port 8448. * .well-known URIs * Ensures that Matrix promotion is happening (e.g. ensuring that meetups & - events & community activity is supported) + events & community activity is supported). In future: * contracts entities to work on Matrix if such contracts help the Foundation to fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance - kits) + kits). * manages a "Made for Matrix" certification process? (to confirm that products - are actually compatible with Matrix) + are actually compatible with Matrix). ## Timings From cc6b6ea021645f93b339bc17df1cfaf92f079fd2 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:31:08 +0000 Subject: [PATCH 39/79] clarify wording for trojan horsen --- proposals/1779-open-governance.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 58e43b80..38469a06 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -173,13 +173,13 @@ toxic culture of bullying or competitive infighting. Folks need to be able to compromise; we are not building a culture of folks pushing their personal agendas at the expense of the overall project. -We are particularly vigilant against 'trojan horse' additions to the spec - -features which only benefit particular players, or are designed to somehow -cripple or fragment the open protocol and ecosystem in favour of competitive -advantage. Commercial players are of course free to build proprietary -implementations, or use custom event types, or even custom API extensions (e.g. -more efficient network transports) - but implementations must fall back to -interoperating correctly with the rest of the ecosystem. +The team should be particularly vigilant against 'trojan horse' additions to the +spec - features which only benefit particular players, or are designed to +somehow cripple or fragment the open protocol and ecosystem in favour of +competitive advantage. Commercial players are of course free to build +proprietary implementations, or use custom event types, or even custom API +extensions (e.g. more efficient network transports) - but implementations must +fall back to interoperating correctly with the rest of the ecosystem. ### Spec Core Team logistics From 8f4e1d9686993097df415ee71bf4dd16d4257b34 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 00:32:14 +0000 Subject: [PATCH 40/79] clarify guardian selection wording --- proposals/1779-open-governance.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 38469a06..7fbb786f 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -50,9 +50,9 @@ responsible for ensuring that the Foundation (and by extension the Spec Core Team) keeps on mission and neutrally protects the development of Matrix. Guardians are typically independent of the commercial Matrix ecosystem and may even not be members of today’s Matrix community, but are deeply aligned with the -mission of the project, and who are respected and trusted by the wider community -to uphold the guiding principles of the Foundation and keep the other Guardians -honest. +mission of the project. Guardians are selected to be respected and trusted by +the wider community to uphold the guiding principles of the Foundation and keep +the other Guardians honest. In other words; the Spec Core Team builds the spec, and the Guardians provide an independent backstop to ensure the spec evolves in line with the Foundation's From 80b9c83ccea64d31732a35642653eeb326cf2f20 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 15 Jan 2019 10:00:17 +0000 Subject: [PATCH 41/79] clarify dual-hatted guardians & spec core teamistas --- proposals/1779-open-governance.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 7fbb786f..9f6a49d3 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -270,7 +270,9 @@ The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to the governance process also require a 75% positive approval from the Guardians (acting as a formal decision of the Foundation's Directors), in order to ensure -changes are aligned with the Foundation's mission. +changes are aligned with the Foundation's mission. For avoidance of doubt, Spec +Core Team votes and Guardians' votes are distinct and a person having both hats +has to vote independently on both forums with the respective hat on. ## The Guardians From 3b86fa0e3c5ef217e112cd1bcf3dc4c082638520 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Thu, 17 Jan 2019 23:56:49 +0000 Subject: [PATCH 42/79] incorporating delph & vdh reviews --- proposals/1779-open-governance.md | 100 ++++++++++++++++++------------ specification/proposals_intro.rst | 22 +++---- 2 files changed, 72 insertions(+), 50 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 9f6a49d3..dde9aeac 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -69,9 +69,9 @@ We believe: * People should have full control over their own communication. - * People should not be locked into centralised communication silos, but free to - pick who they choose to host their communication without limiting who they - can reach. + * People should not be locked into centralised communication silos, but instead + be free to pick who they choose to host their communication without limiting + who they can reach. * The ability to converse securely and privately is a basic human right. @@ -80,9 +80,9 @@ We believe: ### Mission -The Matrix.org Foundation exists to act as a neutral custodian for Matrix and +The Matrix.org Foundation exists to act as a neutral custodian for Matrix and to nurture it as efficiently as possible as a single unfragmented standard, for the -greater benefit of the whole ecosystem; not benefiting or privileging any single +greater benefit of the whole ecosystem, not benefiting or privileging any single player or subset of players. For clarity: the Matrix ecosystem is defined as anyone who uses the Matrix @@ -185,8 +185,8 @@ fall back to interoperating correctly with the rest of the ecosystem. The Spec Core Team itself will be made up of roughly 8 members + 1 project lead. Roughly half the members are expected to be from the historical core team -(similar to Rust). The team must have 5 members to be quorate, with the aim of -generally having between 7 and 9 members. +(similar to Rust). The team must have 5 members to be able to function, with +the aim of generally having between 7 and 9 members. In future we may also have sub-teams (like Rust - e.g. CS/AS/Push API; SS API; IS API; Crypto), but as a starting point we are beginning with a single core @@ -221,13 +221,14 @@ of the team and the Guardians on doing so. New additions to the team require 100% consent from the current team members. Membership has to be formally proposed by someone already on the Spec Core Team. -Members can be removed from the team if >= 75% of the team agrees they are no -longer following the goals and guiding principles of the project. (The 75% is -measured of the whole team, including the member in question) +Members can be removed from the team if 75% of the current members approves and +agrees they are no longer following the goals and guiding principles of the +project. (The 75% is measured of the whole team, including the member in +question). -Guardians act as a backstop, and can appoint or remove Spec Core Team members -(requiring a 75% consensus threshold between the Guardians) if the Spec Core -Team is unable to function or is failing to align with the Foundation's mission. +Guardians act as a safety net, and can appoint or remove Spec Core Team members +(requiring approval by 75% of the current Guardians) if the Spec Core Team is +unable to function or is failing to align with the Foundation's mission. It's suggested that one of the Spec Core Team members should also be a Guardian, to facilitate information exchange between the Guardians and the Spec Core Team, @@ -236,12 +237,14 @@ and to represent the technical angle of the project to the other Guardians. The project lead role acts to coordinate the team and to help steer the team to consensus in the event of failing to get agreement on a Matrix Spec Change. Every 12 months, a vote of confidence is held in the project lead, requiring the -confidence of 75% of the team for the lead to be renewed. There is no maximum -term for the project lead. The lead may be removed by the core team at any -point (with 75% majority), and may resign the role at any point (notifying the -team and the Guardians). The lead automatically resigns the role if they resign -from the Spec Core Team. Resignation automatically triggers selection of a new -lead, who must be selected from the existing Spec Core Team. +approval of 75% of the current Spec Core Team members for the lead to be +renewed. There is no maximum term for the project lead. The lead may be +removed by the core team at any point (requiring 75% approval of current +members), and may resign the role at any point (notifying the team and the +Guardians). The lead automatically resigns the role if they resign from the +Spec Core Team. Resignation automatically triggers selection of a new lead, who +must be selected from the existing Spec Core Team with 75% approval from current +members within 14 days. It is vital that the core spec team has strong domain expertise covering all different domains of the spec (e.g. we don't want to end up with a core spec @@ -259,21 +262,24 @@ The initial Spec Core Team (and their domain areas) is: * Alexey Rusakov (Clients on behalf of Community) * TBD -MSCs require >= 75% approval from the Spec Core Team to proceed to Final Comment -Period (see https://matrix.org/docs/spec/proposals for the rest of the MSC -process). +MSCs require approval by 75% of the current members of the Spec Core Team to +proceed to Final Comment Period (see https://matrix.org/docs/spec/proposals for +the rest of the MSC process). Even though a threshold of only 75% is required for approval, the Spec Core Team is expected to seek consensus on MSCs. The above governance process for the Spec Core Team is considered as part of the spec and is updated using the Matrix Spec Change process. However, changes to -the governance process also require a 75% positive approval from the Guardians +the governance process also require approval by 75% of the current Guardians (acting as a formal decision of the Foundation's Directors), in order to ensure changes are aligned with the Foundation's mission. For avoidance of doubt, Spec Core Team votes and Guardians' votes are distinct and a person having both hats has to vote independently on both forums with the respective hat on. +Spec Core Team decisions (e.g. appointing/removing members and lead) +should be published openly and transparently for the public. + ## The Guardians *This section will be used as the basis for the legal responsibilities of @@ -285,16 +291,23 @@ is following its guiding principles, and provide a safety mechanism if the structure of the Spec Core Team runs into trouble. In practice, this means that: - * Guardians must approve changes to the Spec Core Team. + + * Guardians are responsible for ensuring the Spec Core Team continues to + function, and have the power to appoint/dismiss members of the spec core team + (with the agreement of 75% of the Guardians) to address issues with the Spec + Core Team. * Guardians must keep each other honest, providing a ‘checks and balances’. mechanism between each other to ensure that all Guardians and the Spec Core Team act in the best interests of the protocol and ecosystem. - * Guardians may appoint/dismiss members of the Spec Core Team who are in serious - breach of the guiding principles. This overrides the unanimous consent - requirement for the Spec Core Team when appointing new members. + * Guardians may dismiss members of the Spec Core Team who are in serious + breach of the guiding principles. + * Guardians may appoint members of the Spec Core Team to break deadlocks in the + unanimous consent requirement for the Spec Core Team when appointing new + members. * Guardians may also override deadlocks when appointing a Spec Core Team leader - (with a >= 75% majority). - * Guardians must approve changes to the Guiding Principles (above) + (with approval of 75% of the current Guardians). + * Guardians must approve changes to the above Guiding Principles (with approval + of 75% of the current Guardians) * Guardians are responsible for approving use of the Foundation's assets (e.g. redistributing donations). * In future, Guardians may also be responsible for ensuring staff are hired by @@ -303,8 +316,14 @@ In practice, this means that: * As well as the Spec Core Team committee, they may also oversee committees for other areas such as marketing Matrix.org, registering custom event types, or "Made for Matrix" certification. - * It's likely a subset of Guardians will be hands-on for day-to-day - administrative purposes, whilst the others act to keep them in balance. + * Guardians are responsible for choosing if, when and how staff are located by + the Foundation to fill administrative and other functions required to + facilitate the Foundations' mission. + * Guardians are responsible for choosing if and when additional committees are + formed, and to oversee those committees. + * Guardians are not required to be involved on a day-to-day basis, however + those not taking a hands on approach are required to monitor to ensure a + suitable balance is kept by those that do. Guardians are chosen typically to be independent of the commercial Matrix ecosystem (and especially independent from New Vector), and may even not be @@ -313,18 +332,18 @@ the mission of the project, and respected and trusted by the wider community to uphold the guiding principles of the Foundation and keep the other Guardians honest. -Guardians are responsible for maintaining and updating the Guiding -Principles and Articles of Association of the Foundation if/when -necessary. Changes to the Guiding Principles require a 75% majority from the -Guardians and are passed as a 'special resolution' of the board. +Guardians are responsible for maintaining and updating the Guiding Principles +and Articles of Association of the Foundation if/when necessary. Changes to the +Guiding Principles require approval from 75% of the current Guardians and are +passed as a 'special resolution' of the board. -New Guardians may be appointed with a 75% majority by the board. +New Guardians may be appointed with approval from 75% of the current Guardians. Guardians may resign at any time, with notification to the board. -Guardians may be removed due to serious breach of the guiding principles with a -75% majority of the other Guardians, or if absent from 3 consecutive board -meetings, or if they are legally disqualified from acting as a Director. +Guardians may be removed due to serious breach of the guiding principles with +approval by 75% of the other current Guardians, or if absent from 3 consecutive +board meetings, or if they are legally disqualified from acting as a Director. We aim to recruit roughly 5 Guardians. The initial Guardians are: @@ -340,6 +359,9 @@ Foundation relative to Matthew & Amandine’s day jobs at New Vector. Guardians must arrange their own funding for their time. +Guardian decisions (e.g. appointing/removing guardians; changes to the spec core +team; etc) should be published openly and transparently for the public. + ## The Code Core Team (aka The Core Team) The "Core Team" (or the "Code Core Team", to disambiguate from the Spec Core diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 1e93a22a..87828916 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -19,10 +19,10 @@ proposal being accepted, then actually having your ideas implemented as committed changes to the `Specification repository `_. -Meet the `members of the Core Team +Meet the `members of the Spec Core Team `_, a group of individuals tasked with ensuring the spec process is as smooth and painless as -possible. Members of the Core Team will do their best to participate in +possible. Members of the Spec Core Team will do their best to participate in discussion, summarise when things become long-winded, and generally try to act towards the benefit of everyone. As a majority, team members have the ability to change the state of a proposal, and individually have the final say in @@ -74,7 +74,7 @@ their proposed changes to the Matrix protocol: * Pragmatism rather than perfection * Proof rather than conjecture -Please see [MSC1779](https://github.com/matrix-org/matrix-doc/pull/1779) +Please see [MSC1779](https://github.com/matrix-org/matrix-doc/blob/matthew/msc1779/proposals/1779-open-governance.md) for full details of the project's Guiding Principles. Technical notes @@ -213,25 +213,25 @@ follows: viewpoints and get consensus, but this can sometimes be time-consuming (or the author may be biased), in which case an impartial 'shepherd' can be assigned to help guide the proposal through this process instead. A shepherd is - typically a neutral party from the Core Team or an experienced member of + typically a neutral party from the Spec Core Team or an experienced member of the community. There is no formal process for assignment. Simply ask for a shepherd to help get your proposal through and one will be assigned based on availability. Having a shepherd is not a requirement for proposal acceptance. -- Members of the Core Team and community will review and discuss the PR in the +- Members of the Spec Core Team and community will review and discuss the PR in the comments and in relevant rooms on Matrix. Discussion outside of GitHub should be summarised in a comment on the PR. -- When a member of the Core Team believes that no new discussion points are +- When a member of the Spec Core Team believes that no new discussion points are being made, they will propose a motion for a final comment period (FCP), along with a *disposition* of either merge, close or postpone. This FCP is provided to allow a short period of time for any invested party to provide a final objection before a major decision is made. If sufficient reasoning is given, an FCP can be cancelled. It is often preceded by a comment summarising the current state of the discussion, along with reasoning for its occurrence. -- A concern can be raised by a Core Team member at any time, which will block - an FCP from beginning. An FCP will only begin when a **majority** of core - team members agree on its outcome, and all existing concerns have been +- A concern can be raised by a Spec Core Team member at any time, which will block + an FCP from beginning. An FCP will only begin when 75% of the members of the + Spec Core Team team agree on its outcome, and all existing concerns have been resolved. - The FCP will then begin and last for 5 days, giving anyone else some time to speak up before it concludes. On its conclusion, the disposition of the FCP @@ -321,7 +321,7 @@ Lifetime States Name GitHub Label Description =============================== ============================= ==================================== Proposal Drafting and Feedback N/A A proposal document which is still work-in-progress but is being shared to incorporate feedback -Proposal In Review proposal-in-review A proposal document which is now ready and waiting for review by the Core Team and community +Proposal In Review proposal-in-review A proposal document which is now ready and waiting for review by the Spec Core Team and community Proposed Final Comment Period proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period Final Comment Period final-comment-period A proposal document which has reached final comment period either for merge, closure or postponement Final Commment Period Complete finished-final-comment-period The final comment period has been completed. Waiting for a demonstration implementation @@ -342,7 +342,7 @@ pull request trackers of the `matrix-doc `_ repo. We use labels and some metadata in MSC PR descriptions to generate this page. -Labels are assigned by the Core Team whilst triaging the proposals based on those +Labels are assigned by the Spec Core Team whilst triaging the proposals based on those which exist in the `matrix-doc `_ repo already. From 20b9a33b1240a45cae4419ba19ff5670195cfb99 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 18 Jan 2019 22:39:26 +0000 Subject: [PATCH 43/79] clarify 100% threshold for new spec core team additions --- proposals/1779-open-governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index dde9aeac..779e94ab 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -218,7 +218,7 @@ agreement, they will be assumed to have left the project. Spec Core Team members can resign whenever they want, but must notify the rest of the team and the Guardians on doing so. -New additions to the team require 100% consent from the current team members. +New additions to the team must be approved by all current members of the team. Membership has to be formally proposed by someone already on the Spec Core Team. Members can be removed from the team if 75% of the current members approves and From 41c14c9e7df76efbbcc70eab2b93669a64909a40 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 18 Jan 2019 22:56:19 +0000 Subject: [PATCH 44/79] uhoreg tweaks Co-Authored-By: ara4n --- proposals/1779-open-governance.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 779e94ab..0a1cfca4 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -377,7 +377,7 @@ the core team is also signified by membership of the +matrix:matrix.org Matrix community. Responsibilities include: - * Helping ensure the quality of the projects' code repositories + * Helping ensure the quality of the projects' code repositories. * Ensuring all commits are reviewed. * Ensuring that all projects follow the Matrix spec. * Helping architect the implementations. @@ -400,7 +400,7 @@ Code Core Team members must arrange their own funding for their time. * Owns the Matrix.org marketing swag (t-shirts, stickers, exhibition stands etc). * Responsible for finding someone to run the Matrix.org homeserver (currently New Vector). * Publishes the spec. - * Responsible for tools and documentation that supports the spec. + * Responsible for tools and documentation that support the spec. * Responsible for ensuring reference implementations and test suite exists for the spec. * Publishes the website (including ensuring This Week In Matrix and similar exist to promote independent projects). * Manages any future IANA-style allocations for Matrix, such as: @@ -412,11 +412,11 @@ Code Core Team members must arrange their own funding for their time. In future: - * contracts entities to work on Matrix if such contracts help the Foundation to + * Contracts entities to work on Matrix if such contracts help the Foundation to fulfil its mission and obey the Guiding Principles (e.g. redistributing donations back to fund development of reference implementations or compliance kits). - * manages a "Made for Matrix" certification process? (to confirm that products + * Manages a "Made for Matrix" certification process? (to confirm that products are actually compatible with Matrix). ## Timings From a358e2d4d8e8d1128dc308306eaec91b188ed547 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Mon, 28 Jan 2019 14:57:59 +0000 Subject: [PATCH 45/79] rst --- specification/proposals_intro.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/proposals_intro.rst b/specification/proposals_intro.rst index 9274b2a9..44fcf4b1 100644 --- a/specification/proposals_intro.rst +++ b/specification/proposals_intro.rst @@ -111,6 +111,7 @@ engine behaviour be specified? There is no universal answer to this, but the following guidelines should be applied: + 1. If the feature would benefit the whole Matrix ecosystem and is aligned with the guiding principles above, then it should be supported by the spec. 2. If the spec already makes the feature possible without changing any of the @@ -122,6 +123,7 @@ applied: 3rd party behaviour. As a worked example: + 1. Video conferencing is clearly a feature which would benefit the whole ecosystem, and so the spec should find a way to make it happen. 2. Video conferencing can be achieved by widgets without requiring any From 1bda3fe2b20427be2e129ac2a91d7dc9bbd72215 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 26 May 2019 21:49:46 -0600 Subject: [PATCH 46/79] Spec MSISDN UIA support Fixes https://github.com/matrix-org/matrix-doc/issues/1702 1702 describes the lack of `bind_msisdn` parameter, however the whole login type was missing from UIA. --- api/client-server/registration.yaml | 6 ++++ .../client_server/newsfragments/2030.feature | 1 + specification/client_server_api.rst | 29 +++++++++++++++++++ 3 files changed, 36 insertions(+) create mode 100644 changelogs/client_server/newsfragments/2030.feature diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index e2d35d2c..741008bb 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -94,6 +94,12 @@ paths: If true, the server binds the email used for authentication to the Matrix ID with the identity server. example: false + bind_msisdn: + type: boolean + description: |- + If true, the server binds the phone number used for authentication + to the Matrix ID with the identity server. + example: false username: type: string description: |- diff --git a/changelogs/client_server/newsfragments/2030.feature b/changelogs/client_server/newsfragments/2030.feature new file mode 100644 index 00000000..b5975a73 --- /dev/null +++ b/changelogs/client_server/newsfragments/2030.feature @@ -0,0 +1 @@ +Add MSISDN (phone number) support to User-Interactive Authentication. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 604c2b1c..a40d463c 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -618,6 +618,7 @@ This specification defines the following auth types: - ``m.login.recaptcha`` - ``m.login.oauth2`` - ``m.login.email.identity`` + - ``m.login.msisdn`` - ``m.login.token`` - ``m.login.dummy`` @@ -782,6 +783,34 @@ To use this authentication type, clients should submit an auth dict as follows: "session": "" } +Phone number/MSISDN-based (identity server) +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< +:Type: + ``m.login.msisdn`` +:Description: + Authentication is supported by authorising a phone number with an identity + server. + +Prior to submitting this, the client should authenticate with an identity +server. After authenticating, the session information should be submitted to +the homeserver. + +To use this authentication type, clients should submit an auth dict as follows: + +.. code:: json + + { + "type": "m.login.msisdn", + "threepidCreds": [ + { + "sid": "", + "client_secret": "", + "id_server": "" + } + ], + "session": "" + } + Dummy Auth <<<<<<<<<< :Type: From 0463084924543ee5b024adb62ac32ee5339855c5 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 28 May 2019 14:19:56 -0600 Subject: [PATCH 47/79] Spec 3PID unbind API As per [MSC1915](https://github.com/matrix-org/matrix-doc/pull/1915) Implementation proof: * https://github.com/matrix-org/synapse/pull/4982 * https://github.com/matrix-org/sydent/pull/160 The only alteration made which differs from the proposal is clarity on how to handle homeservers not knowing the `id_server`. All other differences are unintentional. --- api/client-server/administrative_contact.yaml | 27 +++++- api/client-server/registration.yaml | 30 ++++++- api/identity/associations.yaml | 83 +++++++++++++++++++ .../client_server/newsfragments/2046.feature | 1 + .../identity_service/newsfragments/2046.new | 1 + .../1915-unbind-identity-server-param.md | 4 +- 6 files changed, 142 insertions(+), 4 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2046.feature create mode 100644 changelogs/identity_service/newsfragments/2046.new diff --git a/api/client-server/administrative_contact.yaml b/api/client-server/administrative_contact.yaml index 33ea9786..5cf01805 100644 --- a/api/client-server/administrative_contact.yaml +++ b/api/client-server/administrative_contact.yaml @@ -163,6 +163,14 @@ paths: schema: type: object properties: + id_server: + type: string + description: |- + The identity server to unbind from. If not provided, the homeserver + MUST use the ``id_server`` the identifier was added through. If the + homeserver does not know the original ``id_server``, it MUST return + a ``id_server_unbind_result`` of ``no-support``. + example: "example.org" medium: type: string description: The medium of the third party identifier being removed. @@ -180,7 +188,24 @@ paths: user. schema: type: object - properties: {} + properties: + id_server_unbind_result: + type: string + enum: + # XXX: I don't know why, but the order matters here so that "no-support" + # doesn't become "no- support" by the renderer. + - "no-support" + - "success" + description: |- + An indicator as to whether or not the homeserver was able to unbind + the 3PID from the identity server. ``success`` indicates that the + indentity server has unbound the identifier whereas ``no-support`` + indicates that the identity server refuses to support the request + or the homeserver was not able to determine an identity server to + unbind from. + example: "success" + required: + - id_server_unbind_result tags: - User data "/account/3pid/email/requestToken": diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..f003cf1f 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -524,13 +524,39 @@ paths: description: |- Additional authentication information for the user-interactive authentication API. "$ref": "definitions/auth_data.yaml" + id_server: + type: string + description: |- + The identity server to unbind all of the user's 3PIDs from. + If not provided, the homeserver MUST use the ``id_server`` + that was originally use to bind each identifier. If the + homeserver does not know which ``id_server`` that was, + it must return an ``id_server_unbind_result`` of + ``no-support``. + example: "example.org" responses: 200: description: The account has been deactivated. - examples: - application/json: {} schema: type: object + properties: + id_server_unbind_result: + type: string + enum: + - "success" + - "no-support" + description: |- + An indicator as to whether or not the homeserver was able to unbind + the user's 3PIDs from the identity server(s). ``success`` indicates + that all identifiers have been unbound from the identity server while + ``no-support`` indicates that one or more identifiers failed to unbind + due to the identity server refusing the request or the homeserver + being unable to determine an identity server to unbind from. This + must be ``success`` if the homeserver has no identifiers to unbind + for the user. + example: "success" + required: + - id_server_unbind_result 401: description: |- The homeserver requires additional authentication information. diff --git a/api/identity/associations.yaml b/api/identity/associations.yaml index 152a0a9b..f44fe3cc 100644 --- a/api/identity/associations.yaml +++ b/api/identity/associations.yaml @@ -201,3 +201,86 @@ paths: } schema: $ref: "../client-server/definitions/errors/error.yaml" + "/3pid/unbind": + post: + summary: Remove an association between a session and a Matrix user ID. + description: |- + Remove an association between a session and a Matrix user ID. + + Future calls to ``/lookup`` for any of the session's 3pids will not + return the removed association. + + The identity server should authenticate the request in one of two + ways: + + 1. The request is signed by the homeserver which controls the ``user_id``. + 2. The request includes the ``sid`` and ``client_secret`` parameters, + as per ``/3pid/bind``, which proves ownership of the 3PID. + + If this endpoint returns a JSON Matrix error, that error should be passed + through to the client requesting an unbind through a homeserver, if the + homeserver is acting on behalf of a client. + operationId: unbind + parameters: + - in: body + name: body + schema: + type: object + example: { + "sid": "1234", + "client_secret": "monkeys_are_GREAT", + "mxid": "@ears:example.org", + "threepid": { + "medium": "email", + "address": "monkeys_have_ears@example.org" + } + } + properties: + sid: + type: string + description: The Session ID generated by the ``requestToken`` call. + client_secret: + type: string + description: The client secret passed to the ``requestToken`` call. + mxid: + type: string + description: The Matrix user ID to remove from the 3pids. + threepid: + type: object + title: 3PID + description: |- + The 3PID to remove. Must match the 3PID used to generate the session + if using ``sid`` and ``client_secret`` to authenticate this request. + properties: + medium: + type: string + description: |- + A medium from the `3PID Types`_ Appendix, matching the medium + of the identifier to unbind. + address: + type: string + description: The 3PID address to remove. + required: ['medium', 'address'] + required: ["threepid", "mxid"] + responses: + 200: + description: The association was successfully removed. + examples: + application/json: {} + schema: + type: object + 400: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 404: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. + 501: + description: |- + If the response body is not a JSON Matrix error, the identity server + does not support unbinds. If a JSON Matrix error is in the response + body, the requesting party should respect the error. diff --git a/changelogs/client_server/newsfragments/2046.feature b/changelogs/client_server/newsfragments/2046.feature new file mode 100644 index 00000000..e54df535 --- /dev/null +++ b/changelogs/client_server/newsfragments/2046.feature @@ -0,0 +1 @@ +Add ``id_server`` to ``/deactivate`` and ``/3pid/delete`` endpoints to unbind from a specific identity server. diff --git a/changelogs/identity_service/newsfragments/2046.new b/changelogs/identity_service/newsfragments/2046.new new file mode 100644 index 00000000..7146799b --- /dev/null +++ b/changelogs/identity_service/newsfragments/2046.new @@ -0,0 +1 @@ +Add ``/3pid/unbind`` for removing 3PIDs. diff --git a/proposals/1915-unbind-identity-server-param.md b/proposals/1915-unbind-identity-server-param.md index 6817ece3..5b7a1a50 100644 --- a/proposals/1915-unbind-identity-server-param.md +++ b/proposals/1915-unbind-identity-server-param.md @@ -27,7 +27,9 @@ known by the homeserver). The 200 response is a JSON object with an `id_server_unbind_result` field whose value is either `success` or `no-support`, where the latter indicates that the identity server (IS) does not support unbinding 3PIDs directly. If the identity -server returns an error then that should be returned to the client. +server returns an error then that should be returned to the client. If the homeserver +is unable to determine an `id_server` to use, it should return `no-support` for +the `id_server_unbind_result`. Example: From 6706d772c91be7c4ab5c8089f4579695e461ffd4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 00:46:00 -0600 Subject: [PATCH 48/79] Fix test vectors with invalid JSON and signature Fixes https://github.com/matrix-org/matrix-doc/issues/2023 The content hashes appear correct, however applying the algorithm defined in the spec never resulted in the signatures previously demonstrated. --- specification/appendices/test_vectors.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index e2b8fb58..7759fa88 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -114,7 +114,7 @@ The event signing algorithm should emit the following signed event: "origin_server_ts": 1000000, "signatures": { "domain": { - "ed25519:1": "2Wptgo4CwmLo/Y8B8qinxApKaCkBG2fjTWB7AbP5Uy+aIbygsSdLOFzvdDjww8zUVKCmI02eP9xtyJxc/cLiBA" + "ed25519:1": "JV2dlZUASAefSdywnyCxzykHlyr7xkKGK7IRir1cF8eYsnONrCSb+GRn7aXXstr1UHKvzYjRXPx0001+boD1Ag" } }, "type": "X", @@ -129,7 +129,7 @@ Given the following event containing redactable content: { "content": { - "body": "Here is the message content", + "body": "Here is the message content" }, "event_id": "$0:domain", "origin": "domain", @@ -149,7 +149,7 @@ The event signing algorithm should emit the following signed event: { "content": { - "body": "Here is the message content", + "body": "Here is the message content" }, "event_id": "$0:domain", "hashes": { @@ -162,7 +162,7 @@ The event signing algorithm should emit the following signed event: "sender": "@u:domain", "signatures": { "domain": { - "ed25519:1": "Wm+VzmOUOz08Ds+0NTWb1d4CZrVsJSikkeRxh6aCcUwu6pNC78FunoD7KNWzqFn241eYHYMGCA5McEiVPdhzBA" + "ed25519:1": "4zc79tH2cU6Y+eg4YbbF7KiDOrnwEDjlhTqIKiH4k7L9zD9XCiomD7x9odL9eEwnyy1144QyMBe8O3HK++GHBg" } }, "unsigned": { From 0f623113f10cfc50d99305a8d61ce02371636f5c Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 18:35:18 -0600 Subject: [PATCH 49/79] Clarify that UIA stages cannot be attempted twice Fixes https://github.com/matrix-org/matrix-doc/issues/1987 Note: Synapse currently does not care, however the spirit of the text in the spec implies that completed == done forever, so we're just reinforcing it here. --- .../client_server/newsfragments/2054.clarification | 1 + specification/client_server_api.rst | 11 ++++++----- 2 files changed, 7 insertions(+), 5 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2054.clarification diff --git a/changelogs/client_server/newsfragments/2054.clarification b/changelogs/client_server/newsfragments/2054.clarification new file mode 100644 index 00000000..e43aea2d --- /dev/null +++ b/changelogs/client_server/newsfragments/2054.clarification @@ -0,0 +1 @@ +Clarify that User-Interactive Authentication stages cannot be attempted more than once. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 6c2e364a..318ac08d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -453,11 +453,10 @@ params presented, that type may be present as a key in this dictionary. For example, the public part of an OAuth client ID could be given here. session - This is a session identifier that the client must pass back to the home - server, if one is provided, in subsequent attempts to authenticate in the same - API call. + This is a session identifier that the client must pass back to the homeserver, + if one is provided, in subsequent attempts to authenticate in the same API call. -The client then chooses a flow and attempts to complete one of the stages. It +The client then chooses a flow and attempts to complete the first stage. It does this by resubmitting the same request with the addition of an ``auth`` key in the object that it submits. This dictionary contains a ``type`` key whose value is the name of the authentication type that the client is attempting to complete. @@ -558,7 +557,9 @@ message in the standard format. For example: } If the client has completed all stages of a flow, the homeserver performs the -API call and returns the result as normal. +API call and returns the result as normal. Completed stages cannot be re-tried; +The client must abandon the current session and start over. Homeservers should +treat retries as authentication errors. Some authentication types may be completed by means other than through the Matrix client, for example, an email confirmation may be completed when the user From 2ed37f5bf464d03a322184221a89f6ebe4c61630 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 29 May 2019 21:00:37 -0600 Subject: [PATCH 50/79] Clarify guest accounts and `auth` usage on /register Fixes https://github.com/matrix-org/matrix-doc/issues/1980 Fixes https://github.com/matrix-org/matrix-doc/issues/1984 --- api/client-server/registration.yaml | 34 ++++++++++++++----- .../newsfragments/2055.clarification | 1 + 2 files changed, 26 insertions(+), 9 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2055.clarification diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 3195ab41..10b661a3 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -29,7 +29,8 @@ paths: post: summary: Register for an account on this homeserver. description: |- - This API endpoint uses the `User-Interactive Authentication API`_. + This API endpoint uses the `User-Interactive Authentication API`_, except in + the cases where a guest account is being registered. Register for an account on this homeserver. @@ -59,6 +60,11 @@ paths: supplied by the client or generated by the server. The server may invalidate any access token previously associated with that device. See `Relationship between access tokens and devices`_. + + When registering a guest account, all parameters in the request body + with the exception of ``initial_device_display_name`` are ignored by + the server. The server will pick a ``device_id`` for the account + regardless of input. operationId: register parameters: - in: query @@ -72,7 +78,7 @@ paths: enum: - guest - user - description: The kind of account to register. Defaults to `user`. + description: The kind of account to register. Defaults to ``user``. - in: body name: body schema: @@ -80,13 +86,11 @@ paths: properties: auth: description: |- - Additional authentication information for the - user-interactive authentication API. Note that this - information is *not* used to define how the registered user - should be authenticated, but is instead used to - authenticate the ``register`` call itself. It should be - left empty, or omitted, unless an earlier call returned an - response with status code 401. + Additional authentication information for the + user-interactive authentication API. Note that this + information is *not* used to define how the registered user + should be authenticated, but is instead used to + authenticate the ``register`` call itself. "$ref": "definitions/auth_data.yaml" bind_email: type: boolean @@ -194,6 +198,18 @@ paths: The homeserver requires additional authentication information. schema: "$ref": "definitions/auth_response.yaml" + 403: + description: |- + The homeserver does not permit registering the account. This response + can be used to identify that a particular ``kind`` of account is not + allowed, or that registration is generally not supported by the homeserver. + examples: + application/json: { + "errcode": "M_FORBIDDEN", + "error": "Registration is disabled" + } + schema: + "$ref": "definitions/errors/error.yaml" 429: description: This request was rate-limited. schema: diff --git a/changelogs/client_server/newsfragments/2055.clarification b/changelogs/client_server/newsfragments/2055.clarification new file mode 100644 index 00000000..3a57ef7e --- /dev/null +++ b/changelogs/client_server/newsfragments/2055.clarification @@ -0,0 +1 @@ +Clarify which parameters apply in what scenarios on ``/register``. From 7a07a6b358cb15cc6ef81fba07037b440ade2879 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:41:39 -0600 Subject: [PATCH 51/79] Clarify exactly what StrippedState is Fixes https://github.com/matrix-org/matrix-doc/issues/2066 The expectation everywhere is that the `sender` is required. `/initialSync` references StrippedState through a `m.room.member` event reference, and does not need editing. --- api/client-server/sync.yaml | 25 +---------- api/server-server/invites-v1.yaml | 20 +-------- api/server-server/invites-v2.yaml | 20 +-------- .../newsfragments/2067.clarification | 1 + .../newsfragments/2067.clarification | 1 + .../examples/m.room.member$invite_room_state | 19 ++------ event-schemas/examples/stripped_state.json | 18 ++++++++ event-schemas/schema/m.room.member | 19 +------- event-schemas/schema/stripped_state.yaml | 44 +++++++++++++++++++ 9 files changed, 71 insertions(+), 96 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2067.clarification create mode 100644 changelogs/server_server/newsfragments/2067.clarification create mode 100644 event-schemas/examples/stripped_state.json create mode 100644 event-schemas/schema/stripped_state.yaml diff --git a/api/client-server/sync.yaml b/api/client-server/sync.yaml index 02fddb84..3db1fa54 100644 --- a/api/client-server/sync.yaml +++ b/api/client-server/sync.yaml @@ -212,30 +212,7 @@ paths: events: description: The StrippedState events that form the invite state. items: - description: |- - A stripped down state event, with only the ``type``, ``state_key``, - ``sender``, and ``content`` keys. - properties: - content: - description: The ``content`` for the event. - title: EventContent - type: object - state_key: - description: The ``state_key`` for the event. - type: string - type: - description: The ``type`` for the event. - type: string - sender: - description: The ``sender`` for the event. - type: string - required: - - type - - state_key - - content - - sender - title: StrippedState - type: object + $ref: "definitions/event-schemas/schema/stripped_state.yaml" type: array leave: title: Left rooms diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 2ad0f220..867d7b05 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -82,25 +82,7 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - type: object - title: Invite Room State Event - properties: - type: - type: string - description: The type of event. - example: "m.room.join_rules" - state_key: - type: string - description: The state key for the event. May be an empty string. - example: "" - content: - type: object - description: The content for the event. - sender: - type: string - description: The sender of the event. - example: "@someone:matrix.org" - required: ['type', 'state_key', 'content', 'sender'] + $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" example: [ { "type": "m.room.join_rules", diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index c459a848..6d5b102e 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -83,25 +83,7 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - type: object - title: Invite Room State Event - properties: - type: - type: string - description: The type of event. - example: "m.room.join_rules" - state_key: - type: string - description: The state key for the event. May be an empty string. - example: "" - content: - type: object - description: The content for the event. - sender: - type: string - description: The sender of the event. - example: "@someone:matrix.org" - required: ['type', 'state_key', 'content', 'sender'] + $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" example: [ { "type": "m.room.join_rules", diff --git a/changelogs/client_server/newsfragments/2067.clarification b/changelogs/client_server/newsfragments/2067.clarification new file mode 100644 index 00000000..cc706274 --- /dev/null +++ b/changelogs/client_server/newsfragments/2067.clarification @@ -0,0 +1 @@ +Clarify exactly what invite_room_state consists of. diff --git a/changelogs/server_server/newsfragments/2067.clarification b/changelogs/server_server/newsfragments/2067.clarification new file mode 100644 index 00000000..cc706274 --- /dev/null +++ b/changelogs/server_server/newsfragments/2067.clarification @@ -0,0 +1 @@ +Clarify exactly what invite_room_state consists of. diff --git a/event-schemas/examples/m.room.member$invite_room_state b/event-schemas/examples/m.room.member$invite_room_state index f8f05484..b60abf95 100644 --- a/event-schemas/examples/m.room.member$invite_room_state +++ b/event-schemas/examples/m.room.member$invite_room_state @@ -7,21 +7,8 @@ }, "unsigned": { "age": 1234, - "invite_room_state": [ - { - "type": "m.room.name", - "state_key": "", - "content": { - "name": "Forest of Magic" - } - }, - { - "type": "m.room.join_rules", - "state_key": "", - "content": { - "join_rule": "invite" - } - } - ] + "invite_room_state": { + "$ref": "stripped_state.json" + } } } diff --git a/event-schemas/examples/stripped_state.json b/event-schemas/examples/stripped_state.json new file mode 100644 index 00000000..9d8c1b2b --- /dev/null +++ b/event-schemas/examples/stripped_state.json @@ -0,0 +1,18 @@ +[ + { + "type": "m.room.name", + "sender": "@bob:example.org", + "state_key": "", + "content": { + "name": "Example Room" + } + }, + { + "type": "m.room.join_rules", + "sender": "@bob:example.org", + "state_key": "", + "content": { + "join_rule": "invite" + } + } +] diff --git a/event-schemas/schema/m.room.member b/event-schemas/schema/m.room.member index de14644d..f846196b 100644 --- a/event-schemas/schema/m.room.member +++ b/event-schemas/schema/m.room.member @@ -81,24 +81,7 @@ properties: invite_room_state: description: 'A subset of the state of the room at the time of the invite, if ``membership`` is ``invite``. Note that this state is informational, and SHOULD NOT be trusted; once the client has joined the room, it SHOULD fetch the live state from the server and discard the invite_room_state. Also, clients must not rely on any particular state being present here; they SHOULD behave properly (with possibly a degraded but not a broken experience) in the absence of any particular events here. If they are set on the room, at least the state for ``m.room.avatar``, ``m.room.canonical_alias``, ``m.room.join_rules``, and ``m.room.name`` SHOULD be included.' items: - description: 'A stripped down state event, with only the ``type``, ``state_key`` and ``content`` keys.' - properties: - content: - description: The ``content`` for the event. - title: EventContent - type: object - state_key: - description: The ``state_key`` for the event. - type: string - type: - description: The ``type`` for the event. - type: string - required: - - type - - state_key - - content - title: StrippedState - type: object + $ref: "stripped_state.yaml" type: array required: - membership diff --git a/event-schemas/schema/stripped_state.yaml b/event-schemas/schema/stripped_state.yaml new file mode 100644 index 00000000..ec591bf1 --- /dev/null +++ b/event-schemas/schema/stripped_state.yaml @@ -0,0 +1,44 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Note: this, and the example, are in the `event-schemas` directory because +# the CS API uses a symlink. In order for the `m.room.member` event to +# reference this, we'd need to use relative pathing. The symlink makes this +# difficult because the schema would be at two different locations, with +# different relative pathing. + +title: StrippedState +type: object +description: |- + A stripped down state event, with only the ``type``, ``state_key``, + ``sender``, and ``content`` keys. +properties: + content: + description: The ``content`` for the event. + title: EventContent + type: object + state_key: + description: The ``state_key`` for the event. + type: string + type: + description: The ``type`` for the event. + type: string + sender: + description: The ``sender`` for the event. + type: string +required: + - type + - state_key + - content + - sender From b9c4a2561ff8692b8e960e9eb977efb27b86e57b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:50:27 -0600 Subject: [PATCH 52/79] Fix examples of StrippedState in s2s spec --- api/server-server/invites-v1.yaml | 53 +++---------------------------- api/server-server/invites-v2.yaml | 53 ++++--------------------------- 2 files changed, 11 insertions(+), 95 deletions(-) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 867d7b05..2f08dd12 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -83,16 +83,8 @@ paths: canonical alias, avatar, and name of the room. items: $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" - example: [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - } - ] + example: + $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -100,26 +92,6 @@ paths: "origin": "example.org", "origin_server_ts": 1549041175876, "sender": "@someone:example.org", - "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] - }, "content": { "membership": "invite" }, @@ -162,24 +134,9 @@ paths: "origin_server_ts": 1549041175876, "sender": "@someone:example.org", "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + "invite_room_state": { + "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + } }, "content": { "membership": "invite" diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index 6d5b102e..57ca99ff 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -84,16 +84,8 @@ paths: canonical alias, avatar, and name of the room. items: $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" - example: [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - } - ] + example: + $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -112,25 +104,7 @@ paths: "ed25519:key_version": "SomeSignatureHere" }, } - }, - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + } } responses: 200: @@ -156,24 +130,9 @@ paths: "origin_server_ts": 1549041175876, "sender": "@someone:example.org", "unsigned": { - "invite_room_state": [ - { - "type": "m.room.join_rules", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "join_rule": "public" - } - }, - { - "type": "m.room.name", - "sender": "@someone:matrix.org", - "state_key": "", - "content": { - "name": "Cool New Room" - } - } - ] + "invite_room_state": { + "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + } }, "content": { "membership": "invite" From 237d585e07b376d69e0f7e887b7135bc8250aba6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 17:59:19 -0600 Subject: [PATCH 53/79] Shorten references to StrippedState in s2s spec --- api/server-server/invites-v1.yaml | 6 +++--- api/server-server/invites-v2.yaml | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 2f08dd12..83aafb3a 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -82,9 +82,9 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" + $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/stripped_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -135,7 +135,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/stripped_state.json" } }, "content": { diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index 57ca99ff..d2cc63a2 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -83,9 +83,9 @@ paths: identify the room. The recommended events to include are the join rules, canonical alias, avatar, and name of the room. items: - $ref: "../client-server/definitions/event-schemas/schema/stripped_state.yaml" + $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../client-server/definitions/event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/stripped_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -131,7 +131,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../client-server/definitions/event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/stripped_state.json" } }, "content": { From 0b45f3795bb6c561266f9388f850bae460c204fd Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 18:02:22 -0600 Subject: [PATCH 54/79] Rename example to invite_room_state This is a better representation of what it actually is --- api/server-server/invites-v1.yaml | 4 ++-- api/server-server/invites-v2.yaml | 4 ++-- .../examples/{stripped_state.json => invite_room_state.json} | 0 event-schemas/examples/m.room.member$invite_room_state | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) rename event-schemas/examples/{stripped_state.json => invite_room_state.json} (100%) diff --git a/api/server-server/invites-v1.yaml b/api/server-server/invites-v1.yaml index 83aafb3a..8e1c861d 100644 --- a/api/server-server/invites-v1.yaml +++ b/api/server-server/invites-v1.yaml @@ -84,7 +84,7 @@ paths: items: $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../../event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/invite_room_state.json" example: { "$ref": "examples/minimal_pdu.json", "type": "m.room.member", @@ -135,7 +135,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../../event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/invite_room_state.json" } }, "content": { diff --git a/api/server-server/invites-v2.yaml b/api/server-server/invites-v2.yaml index d2cc63a2..cae14bb4 100644 --- a/api/server-server/invites-v2.yaml +++ b/api/server-server/invites-v2.yaml @@ -85,7 +85,7 @@ paths: items: $ref: "../../event-schemas/schema/stripped_state.yaml" example: - $ref: "../../event-schemas/examples/stripped_state.json" + $ref: "../../event-schemas/examples/invite_room_state.json" required: ['room_version', 'event'] example: { "room_version": "2", @@ -131,7 +131,7 @@ paths: "sender": "@someone:example.org", "unsigned": { "invite_room_state": { - "$ref": "../../../event-schemas/examples/stripped_state.json" + "$ref": "../../../event-schemas/examples/invite_room_state.json" } }, "content": { diff --git a/event-schemas/examples/stripped_state.json b/event-schemas/examples/invite_room_state.json similarity index 100% rename from event-schemas/examples/stripped_state.json rename to event-schemas/examples/invite_room_state.json diff --git a/event-schemas/examples/m.room.member$invite_room_state b/event-schemas/examples/m.room.member$invite_room_state index b60abf95..2c93eb9b 100644 --- a/event-schemas/examples/m.room.member$invite_room_state +++ b/event-schemas/examples/m.room.member$invite_room_state @@ -8,7 +8,7 @@ "unsigned": { "age": 1234, "invite_room_state": { - "$ref": "stripped_state.json" + "$ref": "invite_room_state.json" } } } From 86019c9adeb05a3998a03b7e12089b151293015a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Sun, 2 Jun 2019 18:02:36 -0600 Subject: [PATCH 55/79] Skip over partial event definitions in examples --- event-schemas/check_examples.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/event-schemas/check_examples.py b/event-schemas/check_examples.py index 2baa3ef0..31daa478 100755 --- a/event-schemas/check_examples.py +++ b/event-schemas/check_examples.py @@ -106,6 +106,9 @@ def check_example_dir(exampledir, schemadir): if filename.startswith("."): # Skip over any vim .swp files. continue + if filename.endswith(".json"): + # Skip over any explicit examples (partial event definitions) + continue cwd = os.path.basename(os.path.dirname(os.path.join(root, filename))) if cwd == "core": # Skip checking the underlying definitions From a3364ff35712be278fc4f5914a89dbb27f41d08a Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Tue, 4 Jun 2019 12:41:30 -0600 Subject: [PATCH 56/79] Spec SAS verification and the common key verification framework Reference implementations: * https://gitlab.matrix.org/matrix-org/olm/commit/94f664e7256215f33639dbbad6aaf87ada082a9f * https://github.com/matrix-org/matrix-react-sdk/pull/2461 * https://github.com/matrix-org/matrix-js-sdk/pull/818 * https://github.com/matrix-org/matrix-react-sdk/pull/2596 * https://github.com/matrix-org/matrix-js-sdk/pull/837 Proposals: * [MSC1717](https://github.com/matrix-org/matrix-doc/pull/1717) * [MSC1267](https://github.com/matrix-org/matrix-doc/issues/1267) No alterations to either proposal have been made intentionally here. --- .../client_server/newsfragments/2072.feature | 1 + data-definitions/sas-emoji.json | 66 ++++ .../examples/m.key.verification.accept | 12 + .../examples/m.key.verification.cancel | 8 + event-schemas/examples/m.key.verification.key | 7 + event-schemas/examples/m.key.verification.mac | 10 + .../examples/m.key.verification.request | 11 + .../examples/m.key.verification.start | 8 + .../m.key.verification.start$m.sas.v1 | 12 + .../schema/m.key.verification.accept | 64 ++++ .../schema/m.key.verification.cancel | 70 ++++ event-schemas/schema/m.key.verification.key | 28 ++ event-schemas/schema/m.key.verification.mac | 38 ++ .../schema/m.key.verification.request | 43 +++ event-schemas/schema/m.key.verification.start | 39 +++ .../schema/m.key.verification.start$m.sas.v1 | 69 ++++ scripts/continuserv/main.go | 2 +- .../templating/matrix_templates/sections.py | 17 + scripts/templating/matrix_templates/units.py | 20 ++ .../modules/end_to_end_encryption.rst | 325 +++++++++++++++++- 20 files changed, 847 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2072.feature create mode 100644 data-definitions/sas-emoji.json create mode 100644 event-schemas/examples/m.key.verification.accept create mode 100644 event-schemas/examples/m.key.verification.cancel create mode 100644 event-schemas/examples/m.key.verification.key create mode 100644 event-schemas/examples/m.key.verification.mac create mode 100644 event-schemas/examples/m.key.verification.request create mode 100644 event-schemas/examples/m.key.verification.start create mode 100644 event-schemas/examples/m.key.verification.start$m.sas.v1 create mode 100644 event-schemas/schema/m.key.verification.accept create mode 100644 event-schemas/schema/m.key.verification.cancel create mode 100644 event-schemas/schema/m.key.verification.key create mode 100644 event-schemas/schema/m.key.verification.mac create mode 100644 event-schemas/schema/m.key.verification.request create mode 100644 event-schemas/schema/m.key.verification.start create mode 100644 event-schemas/schema/m.key.verification.start$m.sas.v1 diff --git a/changelogs/client_server/newsfragments/2072.feature b/changelogs/client_server/newsfragments/2072.feature new file mode 100644 index 00000000..c7d8bd76 --- /dev/null +++ b/changelogs/client_server/newsfragments/2072.feature @@ -0,0 +1 @@ +Add interactive device verification, including a common framework for device verification. diff --git a/data-definitions/sas-emoji.json b/data-definitions/sas-emoji.json new file mode 100644 index 00000000..060fbd49 --- /dev/null +++ b/data-definitions/sas-emoji.json @@ -0,0 +1,66 @@ +[ + {"number": 0, "emoji": "🐶", "description": "Dog", "unicode": "U+1F436"}, + {"number": 1, "emoji": "🐱", "description": "Cat", "unicode": "U+1F431"}, + {"number": 2, "emoji": "🦁", "description": "Lion", "unicode": "U+1F981"}, + {"number": 3, "emoji": "🐎", "description": "Horse", "unicode": "U+1F40E"}, + {"number": 4, "emoji": "🦄", "description": "Unicorn", "unicode": "U+1F984"}, + {"number": 5, "emoji": "🐷", "description": "Pig", "unicode": "U+1F437"}, + {"number": 6, "emoji": "🐘", "description": "Elephant", "unicode": "U+1F418"}, + {"number": 7, "emoji": "🐰", "description": "Rabbit", "unicode": "U+1F430"}, + {"number": 8, "emoji": "🐼", "description": "Panda", "unicode": "U+1F43C"}, + {"number": 9, "emoji": "🐓", "description": "Rooster", "unicode": "U+1F413"}, + {"number": 10, "emoji": "🐧", "description": "Penguin", "unicode": "U+1F427"}, + {"number": 11, "emoji": "🐢", "description": "Turtle", "unicode": "U+1F422"}, + {"number": 12, "emoji": "🐟", "description": "Fish", "unicode": "U+1F41F"}, + {"number": 13, "emoji": "🐙", "description": "Octopus", "unicode": "U+1F419"}, + {"number": 14, "emoji": "🦋", "description": "Butterfly", "unicode": "U+1F98B"}, + {"number": 15, "emoji": "🌷", "description": "Flower", "unicode": "U+1F337"}, + {"number": 16, "emoji": "🌳", "description": "Tree", "unicode": "U+1F333"}, + {"number": 17, "emoji": "🌵", "description": "Cactus", "unicode": "U+1F335"}, + {"number": 18, "emoji": "🍄", "description": "Mushroom", "unicode": "U+1F344"}, + {"number": 19, "emoji": "🌏", "description": "Globe", "unicode": "U+1F30F"}, + {"number": 20, "emoji": "🌙", "description": "Moon", "unicode": "U+1F319"}, + {"number": 21, "emoji": "☁️", "description": "Cloud", "unicode": "U+2601U+FE0F"}, + {"number": 22, "emoji": "🔥", "description": "Fire", "unicode": "U+1F525"}, + {"number": 23, "emoji": "🍌", "description": "Banana", "unicode": "U+1F34C"}, + {"number": 24, "emoji": "🍎", "description": "Apple", "unicode": "U+1F34E"}, + {"number": 25, "emoji": "🍓", "description": "Strawberry", "unicode": "U+1F353"}, + {"number": 26, "emoji": "🌽", "description": "Corn", "unicode": "U+1F33D"}, + {"number": 27, "emoji": "🍕", "description": "Pizza", "unicode": "U+1F355"}, + {"number": 28, "emoji": "🎂", "description": "Cake", "unicode": "U+1F382"}, + {"number": 29, "emoji": "❤️", "description": "Heart", "unicode": "U+2764U+FE0F"}, + {"number": 30, "emoji": "😀", "description": "Smiley", "unicode": "U+1F600"}, + {"number": 31, "emoji": "🤖", "description": "Robot", "unicode": "U+1F916"}, + {"number": 32, "emoji": "🎩", "description": "Hat", "unicode": "U+1F3A9"}, + {"number": 33, "emoji": "👓", "description": "Glasses", "unicode": "U+1F453"}, + {"number": 34, "emoji": "🔧", "description": "Spanner", "unicode": "U+1F527"}, + {"number": 35, "emoji": "🎅", "description": "Santa", "unicode": "U+1F385"}, + {"number": 36, "emoji": "👍", "description": "Thumbs Up", "unicode": "U+1F44D"}, + {"number": 37, "emoji": "☂️", "description": "Umbrella", "unicode": "U+2602U+FE0F"}, + {"number": 38, "emoji": "⌛", "description": "Hourglass", "unicode": "U+231B"}, + {"number": 39, "emoji": "⏰", "description": "Clock", "unicode": "U+23F0"}, + {"number": 40, "emoji": "🎁", "description": "Gift", "unicode": "U+1F381"}, + {"number": 41, "emoji": "💡", "description": "Light Bulb", "unicode": "U+1F4A1"}, + {"number": 42, "emoji": "📕", "description": "Book", "unicode": "U+1F4D5"}, + {"number": 43, "emoji": "✏️", "description": "Pencil", "unicode": "U+270FU+FE0F"}, + {"number": 44, "emoji": "📎", "description": "Paperclip", "unicode": "U+1F4CE"}, + {"number": 45, "emoji": "✂️", "description": "Scissors", "unicode": "U+2702U+FE0F"}, + {"number": 46, "emoji": "🔒", "description": "Lock", "unicode": "U+1F512"}, + {"number": 47, "emoji": "🔑", "description": "Key", "unicode": "U+1F511"}, + {"number": 48, "emoji": "🔨", "description": "Hammer", "unicode": "U+1F528"}, + {"number": 49, "emoji": "☎️", "description": "Telephone", "unicode": "U+260EU+FE0F"}, + {"number": 50, "emoji": "🏁", "description": "Flag", "unicode": "U+1F3C1"}, + {"number": 51, "emoji": "🚂", "description": "Train", "unicode": "U+1F682"}, + {"number": 52, "emoji": "🚲", "description": "Bicycle", "unicode": "U+1F6B2"}, + {"number": 53, "emoji": "✈️", "description": "Aeroplane", "unicode": "U+2708U+FE0F"}, + {"number": 54, "emoji": "🚀", "description": "Rocket", "unicode": "U+1F680"}, + {"number": 55, "emoji": "🏆", "description": "Trophy", "unicode": "U+1F3C6"}, + {"number": 56, "emoji": "⚽", "description": "Ball", "unicode": "U+26BD"}, + {"number": 57, "emoji": "🎸", "description": "Guitar", "unicode": "U+1F3B8"}, + {"number": 58, "emoji": "🎺", "description": "Trumpet", "unicode": "U+1F3BA"}, + {"number": 59, "emoji": "🔔", "description": "Bell", "unicode": "U+1F514"}, + {"number": 60, "emoji": "⚓", "description": "Anchor", "unicode": "U+2693"}, + {"number": 61, "emoji": "🎧", "description": "Headphones", "unicode": "U+1F3A7"}, + {"number": 62, "emoji": "📁", "description": "Folder", "unicode": "U+1F4C1"}, + {"number": 63, "emoji": "📌", "description": "Pin", "unicode": "U+1F4CC"} +] diff --git a/event-schemas/examples/m.key.verification.accept b/event-schemas/examples/m.key.verification.accept new file mode 100644 index 00000000..98e89c06 --- /dev/null +++ b/event-schemas/examples/m.key.verification.accept @@ -0,0 +1,12 @@ +{ + "type": "m.key.verification.accept", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1", + "key_agreement_protocol": "curve25519", + "hash": "sha256", + "message_authentication_code": "hkdf-hmac-sha256", + "short_authentication_string": ["decimal", "emoji"], + "commitment": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } +} diff --git a/event-schemas/examples/m.key.verification.cancel b/event-schemas/examples/m.key.verification.cancel new file mode 100644 index 00000000..9d78f67c --- /dev/null +++ b/event-schemas/examples/m.key.verification.cancel @@ -0,0 +1,8 @@ +{ + "type": "m.key.verification.cancel", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "code": "m.user", + "reason": "User rejected the key verification request" + } +} diff --git a/event-schemas/examples/m.key.verification.key b/event-schemas/examples/m.key.verification.key new file mode 100644 index 00000000..608a2ebd --- /dev/null +++ b/event-schemas/examples/m.key.verification.key @@ -0,0 +1,7 @@ +{ + "type": "m.key.verification.key", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "key": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } +} diff --git a/event-schemas/examples/m.key.verification.mac b/event-schemas/examples/m.key.verification.mac new file mode 100644 index 00000000..c77c3a8d --- /dev/null +++ b/event-schemas/examples/m.key.verification.mac @@ -0,0 +1,10 @@ +{ + "type": "m.key.verification.mac", + "content": { + "transaction_id": "S0meUniqueAndOpaqueString", + "keys": "2Wptgo4CwmLo/Y8B8qinxApKaCkBG2fjTWB7AbP5Uy+aIbygsSdLOFzvdDjww8zUVKCmI02eP9xtyJxc/cLiBA", + "mac": { + "ed25519:ABCDEF": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg" + } + } +} diff --git a/event-schemas/examples/m.key.verification.request b/event-schemas/examples/m.key.verification.request new file mode 100644 index 00000000..258471d2 --- /dev/null +++ b/event-schemas/examples/m.key.verification.request @@ -0,0 +1,11 @@ +{ + "type": "m.key.verification.request", + "content": { + "from_device": "AliceDevice2", + "transaction_id": "S0meUniqueAndOpaqueString", + "methods": [ + "m.sas.v1" + ], + "timestamp": 1559598944869 + } +} diff --git a/event-schemas/examples/m.key.verification.start b/event-schemas/examples/m.key.verification.start new file mode 100644 index 00000000..52f16150 --- /dev/null +++ b/event-schemas/examples/m.key.verification.start @@ -0,0 +1,8 @@ +{ + "type": "m.key.verification.start", + "content": { + "from_device": "BobDevice1", + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1" + } +} diff --git a/event-schemas/examples/m.key.verification.start$m.sas.v1 b/event-schemas/examples/m.key.verification.start$m.sas.v1 new file mode 100644 index 00000000..dae1d405 --- /dev/null +++ b/event-schemas/examples/m.key.verification.start$m.sas.v1 @@ -0,0 +1,12 @@ +{ + "type": "m.key.verification.start", + "content": { + "from_device": "BobDevice1", + "transaction_id": "S0meUniqueAndOpaqueString", + "method": "m.sas.v1", + "key_agreement_protocols": ["curve25519"], + "hashes": ["sha256"], + "message_authentication_codes": ["hkdf-hmac-sha256"], + "short_authentication_string": ["decimal", "emoji"] + } +} diff --git a/event-schemas/schema/m.key.verification.accept b/event-schemas/schema/m.key.verification.accept new file mode 100644 index 00000000..e52df39e --- /dev/null +++ b/event-schemas/schema/m.key.verification.accept @@ -0,0 +1,64 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Accepts a previously sent ``m.key.verification.start`` messge. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + method: + type: string + enum: ["m.sas.v1"] + description: |- + The verification method to use. Must be ``m.sas.v1``. + key_agreement_protocol: + type: string + description: |- + The key agreement protocol the device is choosing to use, out of the + options in the ``m.key.verification.start`` message. + hash: + type: string + description: |- + The hash method the device is choosing to use, out of the options in + the ``m.key.verification.start`` message. + message_authentication_code: + type: string + description: |- + The message authentication code the device is choosing to use, out of + the options in the ``m.key.verification.start`` message. + short_authentication_string: + type: array + description: |- + The SAS methods both devices involved in the verification process + understand. Must be a subset of the options in the ``m.key.verification.start`` + message. + items: + type: string + enum: ["decimal", "emoji"] + commitment: + type: string + description: |- + The hash (encoded as unpadded base64) of the concatenation of the device's + ephemeral public key (encoded as unpadded base64) and the canonical JSON + representation of the ``m.key.verification.start`` message. + required: + - transaction_id + - method + - key_agreement_protocol + - hash + - message_authentication_code + - short_authentication_string + - commitment + type: object + type: + enum: + - m.key.verification.accept + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.cancel b/event-schemas/schema/m.key.verification.cancel new file mode 100644 index 00000000..36ffc9ea --- /dev/null +++ b/event-schemas/schema/m.key.verification.cancel @@ -0,0 +1,70 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Cancels a key verification process/request. Typically sent as a `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + The opaque identifier for the verification process/request. + reason: + type: string + description: |- + A human readable description of the ``code``. The client should only rely on this + string if it does not understand the ``code``. + code: + type: string + # Note: this is not an enum because we go into detail about the different + # error codes. If we made this an enum, we'd be repeating information. + # Also, we can't put a real bulleted list in here because the HTML2RST parser + # cuts the text at weird points, breaking the list completely. + description: |- + The error code for why the process/request was cancelled by the user. Error + codes should use the Java package naming convention if not in the following + list: + + ``m.user``: The user cancelled the verification. + + ``m.timeout``: The verification process timed out. Verification processes + can define their own timeout parameters. + + ``m.unknown_transaction``: The device does not know about the given transaction + ID. + + ``m.unknown_method``: The device does not know how to handle the requested + method. This should be sent for ``m.key.verification.start`` messages and + messages defined by individual verification processes. + + ``m.unexpected_message``: The device received an unexpected message. Typically + raised when one of the parties is handling the verification out of order. + + ``m.key_mismatch``: The key was not verified. + + ``m.user_mismatch``: The expected user did not match the user verified. + + ``m.invalid_message``: The message received was invalid. + + ``m.accepted``: A ``m.key.verification.request`` was accepted by a different + device. The device receiving this error can ignore the verification request. + + Clients should be careful to avoid error loops. For example, if a device sends + an incorrect message and the client returns ``m.invalid_message`` to which it + gets an unexpected response with ``m.unexpected_message``, the client should not + respond again with ``m.unexpected_message`` to avoid the other device potentially + sending another error response. + + .. The above blank line is important for RST. + required: + - transaction_id + - code + - reason + type: object + type: + enum: + - m.key.verification.cancel + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.key b/event-schemas/schema/m.key.verification.key new file mode 100644 index 00000000..6dc4954b --- /dev/null +++ b/event-schemas/schema/m.key.verification.key @@ -0,0 +1,28 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Sends the ephemeral public key for a device to the partner device. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + key: + type: string + description: |- + The device's ephemeral public key, encoded as unpadded base64. + required: + - transaction_id + - key + type: object + type: + enum: + - m.key.verification.key + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.mac b/event-schemas/schema/m.key.verification.mac new file mode 100644 index 00000000..769ebe15 --- /dev/null +++ b/event-schemas/schema/m.key.verification.mac @@ -0,0 +1,38 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Sends the MAC of a device's key to the partner device. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be the same as + the one used for the ``m.key.verification.start`` message. + mac: + type: object + description: |- + A map of the key ID to the MAC of the key, using the algorithm in the + verification process. The MAC is encoded as unpadded base64. + additionalProperties: + type: string + description: The key's MAC, encoded as unpadded base64. + keys: + type: string + description: |- + The MAC of the comma-separated, sorted, list of key IDs given in the ``mac`` + property, encoded as unpadded base64. + required: + - transaction_id + - mac + - keys + type: object + type: + enum: + - m.key.verification.mac + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.request b/event-schemas/schema/m.key.verification.request new file mode 100644 index 00000000..c9efa14e --- /dev/null +++ b/event-schemas/schema/m.key.verification.request @@ -0,0 +1,43 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Requests a key verification with another user's devices. Typically sent as a + `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the request. + transaction_id: + type: string + description: |- + An opaque identifier for the verification request. Must be unique + with respect to the devices involved. + methods: + type: array + description: |- + The verification methods supported by the sender. + items: + type: string + timestamp: + type: integer + format: int64 + description: |- + The POSIX timestamp in milliseconds for when the request was made. If + the request is in the future by more than 5 minutes or more than 10 + minutes in the past, the message should be ignored by the receiver. + required: + - from_device + - transaction_id + - methods + - timestamp + type: object + type: + enum: + - m.key.verification.request + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.start b/event-schemas/schema/m.key.verification.start new file mode 100644 index 00000000..ad59d6c7 --- /dev/null +++ b/event-schemas/schema/m.key.verification.start @@ -0,0 +1,39 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Begins a key verification process. Typically sent as a `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the process. + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be unique + with respect to the devices involved. Must be the same as the + ``transaction_id`` given in the ``m.key.verification.request`` + if this process is originating from a request. + method: + type: string + description: |- + The verification method to use. + next_method: + type: string + description: |- + Optional method to use to verify the other user's key with. Applicable + when the ``method`` chosen only verifies one user's key. + required: + - from_device + - transaction_id + - method + type: object + type: + enum: + - m.key.verification.start + type: string +type: object diff --git a/event-schemas/schema/m.key.verification.start$m.sas.v1 b/event-schemas/schema/m.key.verification.start$m.sas.v1 new file mode 100644 index 00000000..867ca820 --- /dev/null +++ b/event-schemas/schema/m.key.verification.start$m.sas.v1 @@ -0,0 +1,69 @@ +--- +allOf: + - $ref: core-event-schema/event.yaml + +description: |- + Begins a SAS key verification process. Typically sent as a `to-device`_ event. +properties: + content: + properties: + from_device: + type: string + description: |- + The device ID which is initiating the process. + transaction_id: + type: string + description: |- + An opaque identifier for the verification process. Must be unique + with respect to the devices involved. Must be the same as the + ``transaction_id`` given in the ``m.key.verification.request`` + if this process is originating from a request. + method: + type: string + enum: ["m.sas.v1"] + description: |- + The verification method to use. Must be ``m.sas.v1``. + key_agreement_protocols: + type: array + description: |- + The key agreement protocols the sending device understands. Must + include at least ``curve25519``. + items: + type: string + hashes: + type: array + description: |- + The hash methods the sending device understands. Must include at least + ``sha256``. + items: + type: string + message_authentication_codes: + type: array + description: |- + The message authentication codes that the sending device understands. + Must include at least ``hkdf-hmac-sha256``. + items: + type: string + short_authentication_string: + type: array + description: |- + The SAS methods the sending device (and the sending device's user) + understands. Must include at least ``decimal``. Optionally can include + ``emoji``. + items: + type: string + enum: ["decimal", "emoji"] + required: + - from_device + - transaction_id + - method + - key_agreement_protocols + - hashes + - message_authentication_codes + - short_authentication_string + type: object + type: + enum: + - m.key.verification.start + type: string +type: object diff --git a/scripts/continuserv/main.go b/scripts/continuserv/main.go index 2ef6fed9..1bd07e6e 100644 --- a/scripts/continuserv/main.go +++ b/scripts/continuserv/main.go @@ -52,7 +52,7 @@ func main() { walker := makeWalker(dir, w) paths := []string{"api", "changelogs", "event-schemas", "scripts", - "specification"} + "specification", "schemas", "data-definitions"} for _, p := range paths { filepath.Walk(path.Join(dir, p), walker) diff --git a/scripts/templating/matrix_templates/sections.py b/scripts/templating/matrix_templates/sections.py index c88959ed..7000916b 100644 --- a/scripts/templating/matrix_templates/sections.py +++ b/scripts/templating/matrix_templates/sections.py @@ -18,6 +18,7 @@ import inspect import json import os import logging +import re logger = logging.getLogger(__name__) @@ -225,3 +226,19 @@ class MatrixSections(Sections): examples=swagger_def['examples'], title_kind=subtitle_title_char) return rendered + + def render_sas_emoji_table(self): + emoji = self.units.get("sas_emoji") + rendered = ".. csv-table::\n" + rendered += " :header: \"Number\", \"Emoji\", \"Unicode\", \"Description\"\n" + rendered += " :widths: 10, 10, 15, 20\n" + rendered += "\n" + for row in emoji: + rendered += " %d, \"%s\", \"``%s``\", \"%s\"\n" % ( + row['number'], + row['emoji'], + row['unicode'], + row['description'], + ) + rendered += "\n" + return rendered diff --git a/scripts/templating/matrix_templates/units.py b/scripts/templating/matrix_templates/units.py index ddb65efe..fe3ba5d2 100644 --- a/scripts/templating/matrix_templates/units.py +++ b/scripts/templating/matrix_templates/units.py @@ -59,6 +59,8 @@ TARGETS = os.path.join(matrix_doc_dir, "specification/targets.yaml") ROOM_EVENT = "core-event-schema/room_event.yaml" STATE_EVENT = "core-event-schema/state_event.yaml" +SAS_EMOJI_JSON = os.path.join(matrix_doc_dir, "data-definitions/sas-emoji.json") + logger = logging.getLogger(__name__) # a yaml Loader which loads mappings into OrderedDicts instead of regular @@ -1088,3 +1090,21 @@ class MatrixUnits(Units): "string": git_version, "revision": git_commit } + + def load_sas_emoji(self): + with open(SAS_EMOJI_JSON, 'r', encoding='utf-8') as sas_json: + emoji = json.load(sas_json) + + # Verify the emoji matches the unicode + for c in emoji: + e = c['emoji'] + logger.info("Checking emoji %s (%s)", e, c['description']) + u = re.sub(r'U\+([0-9a-fA-F]+)', lambda m: chr(int(m.group(1), 16)), c['unicode']) + if e != u: + raise Exception("Emoji %s should be %s not %s" % ( + c['description'], + repr(e), + c['unicode'], + )) + + return emoji diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 1fd28e93..4bd12b71 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -396,8 +396,8 @@ recommended that clients provide mechanisms by which the user can see: Ed25519 signing key for each device, again encoded using unpadded Base64. Alice can then meet Bob in person, or contact him via some other trusted -medium, and ask him to read out the Ed25519 key shown on his device. She -compares this with the value shown for his device on her client. +medium, and use `SAS Verification`_ or ask him to read out the Ed25519 key +shown on his device, comparing it to the one shown on Alice's device. Device verification may reach one of several conclusions. For example: @@ -423,6 +423,327 @@ Device verification may reach one of several conclusions. For example: decrypted by such a device. For the Olm protocol, this is documented at https://matrix.org/git/olm/about/docs/signing.rst. + +Key verification framework +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Verifying keys manually by reading out the Ed25519 key is not very user friendly, +and can lead to errors. In order to help mitigate errors, and to make the process +eaiser for users, some verification methods are supported by the specification. +The methods all use a common framework for negotiating the key verification. + +To use this framework, Alice's client would send ``m.key.verification.request`` +events to Bob's devices. All of the ``to_device`` messages sent to Bob MUST have +the same ``transaction_id`` to indicate they are part of the same request. This +allows Bob to reject the request on one device, and have it apply to all of his +devices. Similarly, it allows Bob to process the verification on one device without +having to involve all of his devices. + +When Bob's device receives a ``m.key.verification.request``, it should prompt Bob +to verify keys with Alice using one of the supported methods in the request. If +Bob's device does not understand any of the methods, it should not cancel the request +as one of his other devices may support the request. Instead, Bob's device should +tell Bob that an unsupported method was used for starting key verification. The +prompt for Bob to accept/reject Alice's request (or the unsupported method prompt) +should be automatically dismissed 10 minutes after the ``timestamp`` field or 2 +minutes after Bob's client receives the message, whichever comes first, if Bob +does not interact with the prompt. The prompt should additionally be hidden if +an appropriate ``m.key.verification.cancel`` message is received. + +If Bob rejects the request, Bob's client must send a ``m.key.verification.cancel`` +message to Alice's device. Upon receipt, Alice's device should tell her that Bob +does not want to verify her device and send ``m.key.verification.cancel`` messages +to all of Bob's devices to notify them that the request was rejected. + +If Bob accepts the request, Bob's device starts the key verification process by +sending a ``m.key.verification.start`` message to Alice's device. Upon receipt +of this message, Alice's device should send a ``m.key.verification.cancel`` message +to all of Bob's other devices to indicate the process has been started. The start +message must use the same ``transaction_id`` from the original key verification +request if it is in response to the request. The start message can be sent indepdently +of any request. + +Individual verification methods may add additional steps, events, and properties to +the verification messages. Event types for methods defined in this specification must +be under the ``m.key.verification`` namespace and any other event types must be namespaced +according to the Java package naming convention. + +Any of Alice's or Bob's devices can cancel the key verification request or process +at any time with a ``m.key.verification.cancel`` message to all applicable devices. + +This framework yields the following handshake, assuming both Alice and Bob each have +2 devices, Bob's first device accepts the key verification request, and Alice's second +device initiates the request. Note how Alice's first device is not involved in the +request or verification process. + +:: + + +---------------+ +---------------+ +-------------+ +-------------+ + | AliceDevice1 | | AliceDevice2 | | BobDevice1 | | BobDevice2 | + +---------------+ +---------------+ +-------------+ +-------------+ + | | | | + | | m.key.verification.request | | + | |---------------------------------->| | + | | | | + | | m.key.verification.request | | + | |-------------------------------------------------->| + | | | | + | | m.key.verification.start | | + | |<----------------------------------| | + | | | | + | | m.key.verification.cancel | | + | |-------------------------------------------------->| + | | | | + + +After the handshake, the verification process begins. + +{{m_key_verification_request_event}} + +{{m_key_verification_start_event}} + +{{m_key_verification_cancel_event}} + + +.. _`SAS Verification`: + +Short Authentication String (SAS) verification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +SAS verification is a user-friendly key verification process built off the common +framework outlined above. SAS verification is intended to be a highly interactive +process for users, and as such exposes verfiication methods which are easier for +users to use. + +The verification process is heavily inspired by Phil Zimmerman's ZRTP key agreement +handshake. A key part of key agreement in ZRTP is the hash commitment: the party that +begins the Diffie-Hellman key sharing sends a hash of their part of the Diffie-Hellman +exchange, and does not send their part of the Diffie-Hellman exchange until they have +received the other party's part. Thus an attacker essentially only has one attempt to +attack the Diffie-Hellman exchange, and hence we can verify fewer bits while still +achieving a high degree of security: if we verify n bits, then an attacker has a 1 in +2\ :sup:`n` chance of success. For example, if we verify 40 bits, then an attacker has +a 1 in 1,099,511,627,776 chance (or less than 1 in 1012 chance) of success. A failed +attack would result in a mismatched Short Authentication String, alerting users to the +attack. + +The verification process takes place over `to-device`_ messages in two phases: + +1. Key agreement phase (based on `ZRTP key agreement `_). +#. Key verification phase (based on HMAC). + +The process between Alice and Bob verifying each other would be: + +.. |AlicePublicKey| replace:: :math:`K_{A}^{public}` +.. |AlicePrivateKey| replace:: :math:`K_{A}^{private}` +.. |AliceCurve25519| replace:: :math:`K_{A}^{private}K_{A}^{public}` +.. |BobPublicKey| replace:: :math:`K_{B}^{public}` +.. |BobPrivateKey| replace:: :math:`K_{B}^{private}` +.. |BobCurve25519| replace:: :math:`K_{B}^{private}K_{B}^{public}` +.. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` +.. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` +.. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` + +1. Alice and Bob establish a secure connection, likely meeting in-person. "Secure" + here means that either party cannot be impersonated, not explicit secrecy. +#. Alice and Bob communicate which devices they'd like to verify with each other. +#. Alice selects Bob's device from the device list and begins verification. +#. Alice's client ensures it has a copy of Bob's device key. +#. Alice's device sends Bob's device a ``m.key.verification.start`` message. +#. Bob's device receives the message and selects a key agreement protocol, hash + algorithm, message authentication code, and SAS method supported by Alice's + device. +#. Bob's device ensures it has a copy of Alice's device key. +#. Bob's device creates an ephemeral Curve25519 key pair (|BobCurve25519|), and + calculates the hash (using the chosen algorithm) of the public key |BobPublicKey|. +#. Bob's device replies to Alice's device with a ``m.key.verification.accept`` message. +#. Alice's device receives Bob's message and stores the commitment hash for later use. +#. Alice's device creates an ephemeral Curve25519 key pair (|AliceCurve25519|) and + replies to Bob's device with a ``m.key.verification.key``, sending only the public + key |AlicePublicKey|. +#. Bob's device receives Alice's message and replies with its own ``m.key.verification.key`` + message containing its public key |BobPublicKey|. +#. Alice's device receives Bob's message and verifies the commitment hash from earlier + matches the hash of the key Bob's device just sent and the content of Alice's + ``m.key.verification.start`` message. +#. Both Alice and Bob's devices perform an Elliptic-curve Diffie-Hellman (|AliceBobECDH|), + using the result as the shared secret. +#. Both Alice and Bob's devices display a SAS to their users, which is derived + from the shared key using one of the methods in this section. If multiple SAS + methods are available, clients should allow the users to select a method. +#. Alice and Bob compare the strings shown by their devices, and tell their devices if + they match or not. +#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device + and a comma-separated sorted list of of the key IDs that they wish the other user + to verify. HMAC is defined in RFC 2104, and SHA-256 as the hash function. The key for + the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) + using `the key verification HKDF <#SAS-HKDF>`_. +#. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the + MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does + the same for Bob's device keys and key IDs concurrently with Alice. +#. When the other device receives the ``m.key.verification.mac`` message, the device + calculates the HMAC of its copies of the other device's keys given in the message, + as well as the HMAC of the comma-seperated, sorted, list of key IDs in the message. + The device compares these with the HMAC values given in the message, and if everything + matches then the device keys are verified. + +The wire protocol looks like the following between Alice and Bob's devices:: + + +-------------+ +-----------+ + | AliceDevice | | BobDevice | + +-------------+ +-----------+ + | | + | m.key.verification.start | + |-------------------------------->| + | | + | m.key.verification.accept | + |<--------------------------------| + | | + | m.key.verification.key | + |-------------------------------->| + | | + | m.key.verification.key | + |<--------------------------------| + | | + | m.key.verification.mac | + |-------------------------------->| + | | + | m.key.verification.mac | + |<--------------------------------| + | | + +Error and exception handling +<<<<<<<<<<<<<<<<<<<<<<<<<<<< + +At any point the interactive verfication can go wrong. The following describes what +to do when an error happens: + +* Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` + message must be sent to signify the cancelation. +* The verification can time out. Clients should time out a verification that does not + complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` + which goes unused for 5 minutes after having last sent/received it. The client should + inform the user that the verification timed out, and send an appropriate ``m.key.verification.cancel`` + message to the other device. +* When the same device attempts to intiate multiple verification attempts, cancel all + attempts with that device. +* When a device receives an unknown ``transaction_id``, it should send an appropriate + ``m.key.verfication.cancel`` message to the other device indicating as such. This + does not apply for inbound ``m.key.verification.start`` or ``m.key.verification.cancel`` + messages. +* If the two devices do not share a common key share, hash, HMAC, or SAS method then + the device should notify the other device with an appropriate ``m.key.verification.cancel`` + message. +* If the user claims the Short Authentication Strings do not match, the device should + send an appropriate ``m.key.verification.cancel`` message to the other device. +* If the device receives a message out of sequence or that it was not expecting, it should + notify the other device with an appropriate ``m.key.verification.cancel`` message. + + +Verification messages specific to SAS +<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< + +Building off the common framework, the following events are involved in SAS verification. + +The ``m.key.verification.cancel`` event is unchanged, however the following error codes +are used in addition to those already specified: + +* ``m.unknown_method``: The devices are unable to agree on the key agreement, hash, MAC, + or SAS method. +* ``m.mismatched_commitment``: The hash commitment did not match. +* ``m.mismatched_sas``: The SAS did not match. + + +{{m_key_verification_start_m_sas_v1_event}} + +{{m_key_verification_accept_event}} + +{{m_key_verification_key_event}} + +{{m_key_verification_mac_event}} + + +.. _`SAS-HKDF`: + +HKDF calculation +<<<<<<<<<<<<<<<< + +In all of the SAS methods, HKDF is as defined in RFC 5869 and uses the previously +agreed upon hash function for the hash function. The shared secret is supplied +as the input keying material. No salt is used, and the input parameter is the +concatenation of: + + * The string ``MATRIX_KEY_VERIFICATION_SAS``. + * The Matrix ID of the user who sent the ``m.key.verification.start`` message. + * The Device ID of the device which sent the ``m.key.verification.start`` message. + * The Matrix ID of the user who sent the ``m.key.verification.accept`` message. + * The Device ID of the device which sent the ``m.key.verification.accept`` message. + * The ``transaction_id`` being used. + +HKDF is used over the plain shared secret as it results in a harder attack +as well as more uniform data to work with. + +For verification of each party's device keys, HKDF is as defined in RFC 5869 and +uses SHA-256 as the hash function. The shared secret is supplied as the input keying +material. No salt is used, and in the input parameter is the concatenation of: + + * The string ``MATRIX_KEY_VERIFICATION_MAC``. + * The Matrix ID of the user whose key is being MAC-ed. + * The Device ID of the device sending the MAC. + * The Matrix ID of the other user. + * The Device ID of the device receiving the MAC. + * The ``transaction_id`` being used. + * The Key ID of the key being MAC-ed, or the string ``KEY_IDS`` if the item + being MAC-ed is the list of key IDs. + +SAS method: ``decimal`` +<<<<<<<<<<<<<<<<<<<<<<< + +Generate 5 bytes using `HKDF <#SAS-HKDF>`_ then take sequences of 13 bits to +convert to decimal numbers (resulting in 3 numbers between 0 and 8191 inclusive +each). Add 1000 to each calculated number. + +The bitwise operations to get the numbers given the 5 bytes +:math:`B_{0}, B_{1}, B_{2}, B_{3}, B_{4}` would be: + +* First: :math:`(B_{0} \ll 5 | B_{1} \gg 3) + 1000` +* Second: :math:`(B_{1} \& 0x7 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` +* Third: :math:`((B_{3} \& 0x3F) \ll 7 | B_{4} \gg 1) + 1000` + +The digits are displayed to the user either with an appropriate separator, +such as dashes, or with the numbers on individual lines. + +SAS method: ``emoji`` +<<<<<<<<<<<<<<<<<<<<< + +Generate 6 bytes using `HKDF <#SAS-HKDF>`_ then split the first 42 bits into +7 groups of 6 bits, similar to how one would base64 encode something. Convert +each group of 6 bits to a number and use the following table to get the corresponding +emoji: + +{{sas_emoji_table}} + +.. Note:: + This table is available as JSON at + https://github.com/matrix-org/matrix-doc/blob/master/data-definitions/sas-emoji.json + +.. admonition:: Rationale + + The emoji above were chosen to: + + * Be recognisable without colour. + * Be recognisable at a small size. + * Be recognisable by most cultures. + * Be distinguishable from each other. + * Easily described by a few words. + * Avoid symbols with negative connotations. + * Be likely similar across multiple platforms. + +Clients SHOULD show the emoji with the descriptions from the table, or appropriate +translation of those descriptions. Client authors SHOULD collaborate to create a +common set of translations for all languages. + + .. section name changed, so make sure that old links keep working .. _key-sharing: From 37b1e171fc182d3a055190b02f3cfe7c7ea9cd99 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 4 Jun 2019 23:51:10 +0100 Subject: [PATCH 57/79] MSC2076: Enforce key-validity periods when validating event signatures --- proposals/2076-enforce-validity-periods.md | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 proposals/2076-enforce-validity-periods.md diff --git a/proposals/2076-enforce-validity-periods.md b/proposals/2076-enforce-validity-periods.md new file mode 100644 index 00000000..b09aa2f4 --- /dev/null +++ b/proposals/2076-enforce-validity-periods.md @@ -0,0 +1,42 @@ +# MSC2076: Enforce key-validity periods when validating event signatures + +## Background + +The [Federation API +specification](https://matrix.org/docs/spec/server_server/r0.1.1.html#validating-hashes-and-signatures-on-received-events) +specifies that events should be validated via the signature verification +algorithm, but does not specify how the keys for that check should be obtained +and validated. + +In practice, the implementation has been as follows. The receiving server +first requests a copy of the key via the [`GET /_matrix/key/v2/server/` +API](https://matrix.org/docs/spec/server_server/r0.1.1.html#get-matrix-key-v2-server-keyid) +directly from the server which created the signature, or via the [`POST +/_matrix/key/v2/query` API](https://matrix.org/docs/spec/server_server/r0.1.1.html#post-matrix-key-v2-query) +from a trusted key server. Once such a key is obtained, it is then cached +forever. No check is made on the `valid_until_ts` field, and +`minimum_valid_until_ts` is set to zero for calls to `POST +/_matrix/key/v2/query`. + +This is highly unsatisfactory, as it means that, should a key be compromised, +then an attacker can spoof arbitrary events claiming to be from the compromised +server forever, since there is no revocation mechanism. + +## Proposal + +This MSC proposes to enforce the `valid_until_ts` property when validating +event signatures. In particular, the server must ensure that it has a copy of +the key with a `valid_until_ts` at least as large as the `origin_server_ts` of +the event being validated. If it does not have such a copy, it must try to +obtain one via the `GET /_matrix/key/v2/server/` or `POST +/_matrix/key/v2/query` APIs. For the latter, it must set +`minimum_valid_until_ts` to prompt the notary server to attempt to refresh the +key if appropriate. + +Since this changes the rules used to validate events, it will be introduced +with a new room version. This will reduce the risk of divergence between +servers in a room due to some servers accepting events which others reject. + +This MSC also proposes that the current situation - where `valid_until_ts` is +ignored - be formalised for the existing room versions v1-v4, rather than be +left as implementation-specific behaviour. From d2ccd6b268e28437a0a2d70c2fb2d1135fc76757 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 4 Jun 2019 23:53:42 +0100 Subject: [PATCH 58/79] MSC2077: room v5 --- proposals/2077-rooms-v5.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 proposals/2077-rooms-v5.md diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md new file mode 100644 index 00000000..034c118a --- /dev/null +++ b/proposals/2077-rooms-v5.md @@ -0,0 +1,20 @@ +# MSC2077 - Rooms V6 + +This MSC proposes creating a new room version named v5, which will enforce the +signing key `valid_until_ts` timestamps proposed in +[MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). + +## Proposal + +The new room version is called "5". The only difference between v5 and v4 is +that v5 rooms enforce the `valid_until_ts` timestamp on signing keys as +proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). + +It is not yet proposed that servers change the default room version used when +creating new rooms, and it is not yet proposed that servers recommend upgrading +existing rooms to v5. + +## Notes + +See also [MSC2002](./2002-rooms-v4.md), which proposed room v4 but also +mentioned that a v5 was anticipated and gave some context for this change. From 3347a480eb0db602d964073129b7e567de84e7e8 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 5 Jun 2019 00:07:05 +0100 Subject: [PATCH 59/79] fix typo --- proposals/2077-rooms-v5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md index 034c118a..0a7eff2b 100644 --- a/proposals/2077-rooms-v5.md +++ b/proposals/2077-rooms-v5.md @@ -1,4 +1,4 @@ -# MSC2077 - Rooms V6 +# MSC2077 - Rooms V5 This MSC proposes creating a new room version named v5, which will enforce the signing key `valid_until_ts` timestamps proposed in From 40b10f254b5c82e11b56f96260e7e9b4f184d52d Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Wed, 5 Jun 2019 07:18:25 +0100 Subject: [PATCH 60/79] clarifications --- proposals/2077-rooms-v5.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/proposals/2077-rooms-v5.md b/proposals/2077-rooms-v5.md index 0a7eff2b..9f8c1654 100644 --- a/proposals/2077-rooms-v5.md +++ b/proposals/2077-rooms-v5.md @@ -1,18 +1,17 @@ -# MSC2077 - Rooms V5 +# MSC2077 - Room version 5 -This MSC proposes creating a new room version named v5, which will enforce the -signing key `valid_until_ts` timestamps proposed in +This MSC proposes creating room version 5, which will enforce the signing key +`valid_until_ts` timestamps proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). ## Proposal -The new room version is called "5". The only difference between v5 and v4 is +The new room version is called `5`. The only difference between v5 and v4 is that v5 rooms enforce the `valid_until_ts` timestamp on signing keys as proposed in [MSC2076](https://github.com/matrix-org/matrix-doc/issues/2076). -It is not yet proposed that servers change the default room version used when -creating new rooms, and it is not yet proposed that servers recommend upgrading -existing rooms to v5. +It is not yet proposed to change the default room version to v5. Version 5 will +be considered a "stable" version. ## Notes From fc4965f2748a97312cfe9930aa456ba0caea7302 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 10:40:51 -0600 Subject: [PATCH 61/79] Stronger spec words --- api/client-server/registration.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/api/client-server/registration.yaml b/api/client-server/registration.yaml index 10b661a3..093bac6b 100644 --- a/api/client-server/registration.yaml +++ b/api/client-server/registration.yaml @@ -62,8 +62,8 @@ paths: `Relationship between access tokens and devices`_. When registering a guest account, all parameters in the request body - with the exception of ``initial_device_display_name`` are ignored by - the server. The server will pick a ``device_id`` for the account + with the exception of ``initial_device_display_name`` MUST BE ignored + by the server. The server MUST pick a ``device_id`` for the account regardless of input. operationId: register parameters: From 7ce1ff21351d61cf9cac88637ac9ac389ee12255 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 20:30:21 -0600 Subject: [PATCH 62/79] Clarify which servers are supposed to sign events Fixes https://github.com/matrix-org/matrix-doc/issues/2074 --- .../server_server/newsfragments/2081.clarification | 1 + specification/server_server_api.rst | 14 +++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) create mode 100644 changelogs/server_server/newsfragments/2081.clarification diff --git a/changelogs/server_server/newsfragments/2081.clarification b/changelogs/server_server/newsfragments/2081.clarification new file mode 100644 index 00000000..fd291273 --- /dev/null +++ b/changelogs/server_server/newsfragments/2081.clarification @@ -0,0 +1 @@ +Clarify which servers are supposed to sign events. diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b765e36a..812f0ffc 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -421,9 +421,8 @@ must ensure that the event: Further details of these checks, and how to handle failures, are described below. -.. TODO: - Flesh this out a bit more, and probably change the doc to group the various - checks in one place, rather than have them spread out. +The `Signing Events <#signing-events>`_ section has more information on which hashes +and signatures are expected on events, and how to calculate them. Definitions @@ -1099,6 +1098,15 @@ originating server, following the algorithm described in `Checking for a signatu Note that this step should succeed whether we have been sent the full event or a redacted copy. +The signatures expected on an event are: + +* The sender's server, unless the invite was created as a result of 3rd party invite. + The sender must already match the 3rd party invite, and the server which actually + sends the event may be a different server. +* For room versions 1 and 2, the server which created the ``event_id``. Other room + versions do not track the ``event_id`` over federation and therefore do not need + a signature from those servers. + If the signature is found to be valid, the expected content hash is calculated as described below. The content hash in the ``hashes`` property of the received event is base64-decoded, and the two are compared for equality. From 80aa5a24dc55433bd95f37cd03dd20fab55c280b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 20:33:09 -0600 Subject: [PATCH 63/79] Make v4 the "default" room version As per [MSC2002](https://github.com/matrix-org/matrix-doc/pull/2002). This was missed in https://github.com/matrix-org/matrix-doc/pull/2019 Fixes https://github.com/matrix-org/matrix-doc/issues/2071 --- specification/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/index.rst b/specification/index.rst index 33dff5a3..a6f6f62b 100644 --- a/specification/index.rst +++ b/specification/index.rst @@ -485,7 +485,7 @@ some other reason. Versions can switch between stable and unstable periodically for a variety of reasons, including discovered security vulnerabilities and age. Clients should not ask room administrators to upgrade their rooms if the room is -running a stable version. Servers SHOULD use room version 1 as the default room +running a stable version. Servers SHOULD use room version 4 as the default room version when creating new rooms. The available room versions are: From e115e3439dd84d195496f740d6951531524450ec Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:13:25 -0600 Subject: [PATCH 64/79] Touchups on the s2s swagger definitions Duplicate properties, wrong types, etc. --- api/server-server/leaving.yaml | 1 - api/server-server/query.yaml | 6 +++--- api/server-server/user_devices.yaml | 3 +-- api/server-server/version.yaml | 1 + api/server-server/wellknown.yaml | 1 + 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/api/server-server/leaving.yaml b/api/server-server/leaving.yaml index c088cb5d..e0882fe8 100644 --- a/api/server-server/leaving.yaml +++ b/api/server-server/leaving.yaml @@ -57,7 +57,6 @@ paths: `room version specification`_ for precise event formats. **The response body here describes the common event fields in more detail and may be missing other required fields for a PDU.** - schema: schema: type: object properties: diff --git a/api/server-server/query.yaml b/api/server-server/query.yaml index dc14724c..29826b32 100644 --- a/api/server-server/query.yaml +++ b/api/server-server/query.yaml @@ -81,7 +81,7 @@ paths: servers: type: array description: |- - An array of server names that are likely to hold the given room. This + An array of server names that are likely to hold the given room. This list may or may not include the server answering the query. items: type: string @@ -128,7 +128,7 @@ paths: x-example: "@someone:example.org" - in: query name: field - type: enum + type: string enum: ['displayname', 'avatar_url'] description: |- The field to query. If specified, the server will only return the given field @@ -139,7 +139,7 @@ paths: description: |- The profile for the user. If a ``field`` is specified in the request, only the matching field should be included in the response. If no ``field`` was specified, - the response should include the fields of the user's profile that can be made + the response should include the fields of the user's profile that can be made public, such as the display name and avatar. If the user does not have a particular field set on their profile, the server diff --git a/api/server-server/user_devices.yaml b/api/server-server/user_devices.yaml index 4805deb4..362f9baa 100644 --- a/api/server-server/user_devices.yaml +++ b/api/server-server/user_devices.yaml @@ -42,7 +42,6 @@ paths: description: |- The user ID to retrieve devices for. Must be a user local to the receiving homeserver. - required: true x-example: "@alice:example.org" responses: 200: @@ -82,4 +81,4 @@ paths: description: Optional display name for the device. example: "Alice's Mobile Phone" required: ['device_id', 'keys'] - required: ['user_id', 'stream_id', 'devices'] \ No newline at end of file + required: ['user_id', 'stream_id', 'devices'] diff --git a/api/server-server/version.yaml b/api/server-server/version.yaml index 19975529..929f7b91 100644 --- a/api/server-server/version.yaml +++ b/api/server-server/version.yaml @@ -27,6 +27,7 @@ paths: get: summary: Get the implementation name and version of this homeserver. description: Get the implementation name and version of this homeserver. + operationId: getVersion responses: 200: description: diff --git a/api/server-server/wellknown.yaml b/api/server-server/wellknown.yaml index 75676646..bc390bd5 100644 --- a/api/server-server/wellknown.yaml +++ b/api/server-server/wellknown.yaml @@ -29,6 +29,7 @@ paths: Gets information about the delegated server for server-server communication between Matrix homeservers. Servers should follow 30x redirects, carefully avoiding redirect loops, and use normal X.509 certificate validation. + operationId: getWellKnown responses: 200: description: From 500f3d3bf121f30171422566a46dbbeab3d88608 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 22:28:57 -0600 Subject: [PATCH 65/79] Clarify that the default s2s transport is JSON over HTTP Fixes https://github.com/matrix-org/matrix-doc/issues/1713 --- .../identity_service/newsfragments/2086.clarification | 1 + specification/client_server_api.rst | 3 ++- specification/server_server_api.rst | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 changelogs/identity_service/newsfragments/2086.clarification diff --git a/changelogs/identity_service/newsfragments/2086.clarification b/changelogs/identity_service/newsfragments/2086.clarification new file mode 100644 index 00000000..7016308b --- /dev/null +++ b/changelogs/identity_service/newsfragments/2086.clarification @@ -0,0 +1 @@ +Clarify that the default transport is JSON over HTTP. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..d5b6491f 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -57,6 +57,8 @@ The following other versions are also available, in reverse chronological order: API Standards ------------- +.. TODO: Move a lot of this to a common area for all specs. + .. TODO Need to specify any HMAC or access_token lifetime/ratcheting tricks We need to specify capability negotiation for extensible transports @@ -82,7 +84,6 @@ names in JSON objects passed over the API also follow this convention. ``/createRoom``. A future version of this specification will address the inconsistency. - Any errors which occur at the Matrix API level MUST return a "standard error response". This is a JSON object which looks like: diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index b765e36a..28876e44 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -83,6 +83,17 @@ The following other versions are also available, in reverse chronological order: - `r0.1.1 `_ - `r0.1.0 `_ + +API standards +------------- + +The mandatory baseline for client-server communication in Matrix is exchanging +JSON objects over HTTP APIs. More efficient optional transports will in future +be supported as optional extensions - e.g. a packed binary encoding over +stream-cipher encrypted TCP socket for low-bandwidth/low-roundtrip mobile usage. +For the default HTTP transport, all API calls use a Content-Type of +``application/json``. In addition, all strings MUST be encoded as UTF-8. + Server discovery ---------------- From bbc740197390091f9d9a83fb0a12c79b1a2e620f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:14:33 -0600 Subject: [PATCH 66/79] Clarify when and where CORS headers should be returned Fixes https://github.com/matrix-org/matrix-doc/issues/1736 Fixes https://github.com/matrix-org/matrix-doc/issues/2013 --- .../newsfragments/2089.clarification | 1 + specification/client_server_api.rst | 18 +++++++++++++++--- 2 files changed, 16 insertions(+), 3 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2089.clarification diff --git a/changelogs/client_server/newsfragments/2089.clarification b/changelogs/client_server/newsfragments/2089.clarification new file mode 100644 index 00000000..17405adc --- /dev/null +++ b/changelogs/client_server/newsfragments/2089.clarification @@ -0,0 +1 @@ +Clarify when and where CORS headers should be returned. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index d9342a5b..e38c8ac0 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -242,6 +242,9 @@ recommended. {{versions_cs_http_api}} + +.. _`CORS`: + Web Browser Clients ------------------- @@ -250,9 +253,14 @@ web browser or similar environment. In these cases, the homeserver should respon to pre-flight requests and supply Cross-Origin Resource Sharing (CORS) headers on all requests. -When a client approaches the server with a pre-flight (``OPTIONS``) request, the -server should respond with the CORS headers for that route. The recommended CORS -headers to be returned by servers on all requests are: +Servers MUST expect that clients will approach them with ``OPTIONS`` requests, +allowing clients to discover the CORS headers. All endpoints in this specification s +upport the ``OPTIONS`` method, however the server MUST NOT perform any logic defined +for the endpoints when approached with an ``OPTIONS`` request. + +When a client approaches the server with a request, the server should respond with +the CORS headers for that route. The recommended CORS headers to be returned by +servers on all requests are: .. code:: @@ -296,6 +304,10 @@ In this section, the following terms are used with specific meanings: Well-known URI ~~~~~~~~~~~~~~ +.. Note:: + Servers hosting the ``.well-known`` JSON file SHOULD offer CORS headers, as + per the `CORS`_ section in this specification. + The ``.well-known`` method uses a JSON file at a predetermined location to specify parameter values. The flow for this method is as follows: From 79bbb47d9f200c3d99d3c6d8e2f7456cd66ac94f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Wed, 5 Jun 2019 23:18:04 -0600 Subject: [PATCH 67/79] Clarify when authorization and rate-limiting are not applicable Fixes https://github.com/matrix-org/matrix-doc/issues/1971 --- .../newsfragments/2090.clarification | 1 + .../matrix_templates/templates/http-api.tmpl | 18 +++++++++--------- 2 files changed, 10 insertions(+), 9 deletions(-) create mode 100644 changelogs/client_server/newsfragments/2090.clarification diff --git a/changelogs/client_server/newsfragments/2090.clarification b/changelogs/client_server/newsfragments/2090.clarification new file mode 100644 index 00000000..23ab50f7 --- /dev/null +++ b/changelogs/client_server/newsfragments/2090.clarification @@ -0,0 +1 @@ +Clarify when authorization and rate-limiting are not applicable. diff --git a/scripts/templating/matrix_templates/templates/http-api.tmpl b/scripts/templating/matrix_templates/templates/http-api.tmpl index 0b9207d9..74836045 100644 --- a/scripts/templating/matrix_templates/templates/http-api.tmpl +++ b/scripts/templating/matrix_templates/templates/http-api.tmpl @@ -10,13 +10,13 @@ {{endpoint.desc}} -{{":Rate-limited: Yes." if endpoint.rate_limited else "" }} -{{":Requires auth: Yes." if endpoint.requires_auth else "" }} +{{":Rate-limited: Yes." if endpoint.rate_limited else ":Rate-limited: No." }} +{{":Requires auth: Yes." if endpoint.requires_auth else ":Requires auth: No." }} .. class:: httpheaders - + Request format: - + {% if (endpoint.req_param_by_loc | length) %} {{ tables.split_paramtable(endpoint.req_param_by_loc) }} {% if (endpoint.req_body_tables) %} @@ -33,7 +33,7 @@ {% if endpoint.res_headers is not none -%} .. class:: httpheaders - + Response headers: {{ tables.paramtable(endpoint.res_headers.rows) }} @@ -42,7 +42,7 @@ {% if endpoint.res_tables|length > 0 -%} .. class:: httpheaders - + Response format: {% for table in endpoint.res_tables -%} @@ -54,7 +54,7 @@ {% endif -%} .. class:: httpheaders - + Example request: .. code:: http @@ -64,7 +64,7 @@ {% if endpoint.responses|length > 0 -%} .. class:: httpheaders - + Response{{"s" if endpoint.responses|length > 1 else "" }}: {% endif -%} @@ -78,7 +78,7 @@ {% if res["example"] -%} .. class:: httpheaders - + Example .. code:: json From e644227f4b607b3438cb37b21616a68e7f007645 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 14:13:20 -0600 Subject: [PATCH 68/79] Clarify that the server shouldn't process retries for UIA --- specification/client_server_api.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 318ac08d..4593311d 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -557,9 +557,10 @@ message in the standard format. For example: } If the client has completed all stages of a flow, the homeserver performs the -API call and returns the result as normal. Completed stages cannot be re-tried; -The client must abandon the current session and start over. Homeservers should -treat retries as authentication errors. +API call and returns the result as normal. Completed stages cannot be retried +by clients, therefore servers must return either a 401 response with the completed +stages, or the result of the API call if all stages were completed when a client +retries a stage. Some authentication types may be completed by means other than through the Matrix client, for example, an email confirmation may be completed when the user From 9bf0103ef359202026ffce958c243b898a828f9e Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 14:36:30 -0600 Subject: [PATCH 69/79] Clarify how many PDUs are in a given transaction object Fixes https://github.com/matrix-org/matrix-doc/issues/2093 --- api/server-server/backfill.yaml | 2 +- .../definitions/single_pdu_transaction.yaml | 32 ++++++++++++++++++ .../unlimited_pdu_transaction.yaml | 33 +++++++++++++++++++ api/server-server/events.yaml | 2 +- .../newsfragments/2095.clarification | 1 + 5 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 api/server-server/definitions/single_pdu_transaction.yaml create mode 100644 api/server-server/definitions/unlimited_pdu_transaction.yaml create mode 100644 changelogs/server_server/newsfragments/2095.clarification diff --git a/api/server-server/backfill.yaml b/api/server-server/backfill.yaml index 0da0e234..2ed6298c 100644 --- a/api/server-server/backfill.yaml +++ b/api/server-server/backfill.yaml @@ -64,7 +64,7 @@ paths: A transaction containing the PDUs that preceded the given event(s), including the given event(s), up to the given limit. schema: - $ref: "definitions/transaction.yaml" + $ref: "definitions/unlimited_pdu_transaction.yaml" "/get_missing_events/{roomId}": post: summary: Retrieves events that the sender is missing diff --git a/api/server-server/definitions/single_pdu_transaction.yaml b/api/server-server/definitions/single_pdu_transaction.yaml new file mode 100644 index 00000000..ff682a44 --- /dev/null +++ b/api/server-server/definitions/single_pdu_transaction.yaml @@ -0,0 +1,32 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: + - $ref: "transaction.yaml" +properties: + pdus: + type: array + description: |- + A single PDU. Note that events have a different format depending on the room + version - check the `room version specification`_ for precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ contained in the transaction. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + properties: [] + example: + $ref: "../examples/minimal_pdu.json" +required: ['origin', 'origin_server_ts', 'pdus'] diff --git a/api/server-server/definitions/unlimited_pdu_transaction.yaml b/api/server-server/definitions/unlimited_pdu_transaction.yaml new file mode 100644 index 00000000..0fc31ee4 --- /dev/null +++ b/api/server-server/definitions/unlimited_pdu_transaction.yaml @@ -0,0 +1,33 @@ +# Copyright 2019 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +type: object +allOf: + - $ref: "transaction.yaml" +properties: + pdus: + type: array + description: |- + List of persistent updates to rooms. Note that events have a different format + depending on the room version - check the `room version specification`_ for + precise event formats. + items: + type: object + title: PDU + description: |- + The `PDUs <#pdus>`_ contained in the transaction. The event format varies depending + on the room version - check the `room version specification`_ for precise event formats. + properties: [] + example: + $ref: "../examples/minimal_pdu.json" +required: ['origin', 'origin_server_ts', 'pdus'] diff --git a/api/server-server/events.yaml b/api/server-server/events.yaml index 1f1a802d..1f8ee537 100644 --- a/api/server-server/events.yaml +++ b/api/server-server/events.yaml @@ -156,4 +156,4 @@ paths: 200: description: A transaction containing a single PDU which is the event requested. schema: - $ref: "definitions/transaction.yaml" + $ref: "definitions/single_pdu_transaction.yaml" diff --git a/changelogs/server_server/newsfragments/2095.clarification b/changelogs/server_server/newsfragments/2095.clarification new file mode 100644 index 00000000..66257e17 --- /dev/null +++ b/changelogs/server_server/newsfragments/2095.clarification @@ -0,0 +1 @@ +Clarify how many PDUs are contained in transaction objects for various endpoints. From ae9abe798ef07de9c48f739b50b01e955725c63b Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 23:41:51 -0600 Subject: [PATCH 70/79] Revert signature change for redactable event test The previous signature was calculated on the unredacted event, which means the signature produced was wrong. --- specification/appendices/test_vectors.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 7759fa88..790ed878 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -162,7 +162,7 @@ The event signing algorithm should emit the following signed event: "sender": "@u:domain", "signatures": { "domain": { - "ed25519:1": "4zc79tH2cU6Y+eg4YbbF7KiDOrnwEDjlhTqIKiH4k7L9zD9XCiomD7x9odL9eEwnyy1144QyMBe8O3HK++GHBg" + "ed25519:1": "Wm+VzmOUOz08Ds+0NTWb1d4CZrVsJSikkeRxh6aCcUwu6pNC78FunoD7KNWzqFn241eYHYMGCA5McEiVPdhzBA" } }, "unsigned": { From 7f01346bbad3f0cc31bf2d2fcfde4dd7daa0b2f6 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Thu, 6 Jun 2019 23:50:26 -0600 Subject: [PATCH 71/79] Provide a more complete example of a "minimally-sized event" Using all the required fields of a v1 event. --- specification/appendices/test_vectors.rst | 25 +++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 790ed878..1de458cc 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -91,11 +91,22 @@ Given the following minimally-sized event: .. code:: json { + "room_id": "!x:domain", + "sender": "@a:domain", "event_id": "$0:domain", "origin": "domain", "origin_server_ts": 1000000, "signatures": {}, + "hashes": {}, "type": "X", + "content": {}, + "prev_events": [ + ["$1:domain", "ExampleHash"] + ], + "auth_events": [ + ["$2", "ExampleHash2"] + ], + "depth": 3, "unsigned": { "age_ts": 1000000 } @@ -106,15 +117,25 @@ The event signing algorithm should emit the following signed event: .. code:: json { + "auth_events": [ + ["$2", "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI"] + ], + "content": {}, + "depth": 3, "event_id": "$0:domain", "hashes": { - "sha256": "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI" + "sha256": "6AaJICN1NJURTtaomDYfJlCPMIU+0gtkwg7qzd8FiJM" }, "origin": "domain", "origin_server_ts": 1000000, + "prev_events": [ + ["$1:domain", "onLKD1bGljeBWQhWZ1kaP9SorVmRQNdN5aM2JYU2n/g"] + ], + "room_id": "!x:domain", + "sender": "@a:domain", "signatures": { "domain": { - "ed25519:1": "JV2dlZUASAefSdywnyCxzykHlyr7xkKGK7IRir1cF8eYsnONrCSb+GRn7aXXstr1UHKvzYjRXPx0001+boD1Ag" + "ed25519:1": "51U0wpKYsaNLTQRbha2v5EGO2cVA6pCtnAKEXguu3j3efCLlmq/53vEfWhsk3tY6gnLsV0YM4Lx2NGZkzmV2Ag" } }, "type": "X", From d7858354f26aa3bb8a6cc58e0b1db1d280eee21e Mon Sep 17 00:00:00 2001 From: Jamie McClymont Date: Fri, 7 Jun 2019 20:54:47 +1200 Subject: [PATCH 72/79] Fix 404s in links from room v1 spec --- specification/rooms/v1.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/rooms/v1.rst b/specification/rooms/v1.rst index 1c7a56c4..e8cbf663 100644 --- a/specification/rooms/v1.rst +++ b/specification/rooms/v1.rst @@ -293,5 +293,5 @@ Events in version 1 rooms have the following structure: {{definition_ss_pdu}} -.. _`auth events selection`: ../../server_server/r0.1.1.html#auth-events-selection -.. _`Signing Events`: ../../server_server/r0.1.1.html#signing-events +.. _`auth events selection`: ../server_server/r0.1.1.html#auth-events-selection +.. _`Signing Events`: ../server_server/r0.1.1.html#signing-events From 00fee7463623b38516687d1d4ea5463a5e5e3ec8 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 07:40:52 -0600 Subject: [PATCH 73/79] Update example --- specification/appendices/test_vectors.rst | 22 ++++++---------------- 1 file changed, 6 insertions(+), 16 deletions(-) diff --git a/specification/appendices/test_vectors.rst b/specification/appendices/test_vectors.rst index 1de458cc..05b115db 100644 --- a/specification/appendices/test_vectors.rst +++ b/specification/appendices/test_vectors.rst @@ -93,19 +93,14 @@ Given the following minimally-sized event: { "room_id": "!x:domain", "sender": "@a:domain", - "event_id": "$0:domain", "origin": "domain", "origin_server_ts": 1000000, "signatures": {}, "hashes": {}, "type": "X", "content": {}, - "prev_events": [ - ["$1:domain", "ExampleHash"] - ], - "auth_events": [ - ["$2", "ExampleHash2"] - ], + "prev_events": [], + "auth_events": [], "depth": 3, "unsigned": { "age_ts": 1000000 @@ -117,25 +112,20 @@ The event signing algorithm should emit the following signed event: .. code:: json { - "auth_events": [ - ["$2", "6tJjLpXtggfke8UxFhAKg82QVkJzvKOVOOSjUDK4ZSI"] - ], + "auth_events": [], "content": {}, "depth": 3, - "event_id": "$0:domain", "hashes": { - "sha256": "6AaJICN1NJURTtaomDYfJlCPMIU+0gtkwg7qzd8FiJM" + "sha256": "5jM4wQpv6lnBo7CLIghJuHdW+s2CMBJPUOGOC89ncos" }, "origin": "domain", "origin_server_ts": 1000000, - "prev_events": [ - ["$1:domain", "onLKD1bGljeBWQhWZ1kaP9SorVmRQNdN5aM2JYU2n/g"] - ], + "prev_events": [], "room_id": "!x:domain", "sender": "@a:domain", "signatures": { "domain": { - "ed25519:1": "51U0wpKYsaNLTQRbha2v5EGO2cVA6pCtnAKEXguu3j3efCLlmq/53vEfWhsk3tY6gnLsV0YM4Lx2NGZkzmV2Ag" + "ed25519:1": "KxwGjPSDEtvnFgU00fwFz+l6d2pJM6XBIaMEn81SXPTRl16AqLAYqfIReFGZlHi5KLjAWbOoMszkwsQma+lYAg" } }, "type": "X", From d49c7fb3b02db22a4275bfb0f147b0b4a0477b6d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:01:17 -0600 Subject: [PATCH 74/79] Apply suggestions from code review Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- event-schemas/schema/m.key.verification.accept | 2 +- specification/modules/end_to_end_encryption.rst | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/event-schemas/schema/m.key.verification.accept b/event-schemas/schema/m.key.verification.accept index e52df39e..41c59968 100644 --- a/event-schemas/schema/m.key.verification.accept +++ b/event-schemas/schema/m.key.verification.accept @@ -17,7 +17,7 @@ properties: type: string enum: ["m.sas.v1"] description: |- - The verification method to use. Must be ``m.sas.v1``. + The verification method to use. key_agreement_protocol: type: string description: |- diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 4bd12b71..fb3d2ba5 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -536,15 +536,15 @@ The process between Alice and Bob verifying each other would be: .. |AlicePublicKey| replace:: :math:`K_{A}^{public}` .. |AlicePrivateKey| replace:: :math:`K_{A}^{private}` -.. |AliceCurve25519| replace:: :math:`K_{A}^{private}K_{A}^{public}` +.. |AliceCurve25519| replace:: :math:`K_{A}^{private},K_{A}^{public}` .. |BobPublicKey| replace:: :math:`K_{B}^{public}` .. |BobPrivateKey| replace:: :math:`K_{B}^{private}` -.. |BobCurve25519| replace:: :math:`K_{B}^{private}K_{B}^{public}` +.. |BobCurve25519| replace:: :math:`K_{B}^{private},K_{B}^{public}` .. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` .. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` .. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` -1. Alice and Bob establish a secure connection, likely meeting in-person. "Secure" +1. Alice and Bob establish a secure out-of-band connection, such as meeting in-person or a video call. "Secure" here means that either party cannot be impersonated, not explicit secrecy. #. Alice and Bob communicate which devices they'd like to verify with each other. #. Alice selects Bob's device from the device list and begins verification. @@ -573,9 +573,9 @@ The process between Alice and Bob verifying each other would be: methods are available, clients should allow the users to select a method. #. Alice and Bob compare the strings shown by their devices, and tell their devices if they match or not. -#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device +#. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify. HMAC is defined in RFC 2104, and SHA-256 as the hash function. The key for + to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). The key for the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the @@ -619,7 +619,7 @@ At any point the interactive verfication can go wrong. The following describes w to do when an error happens: * Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` - message must be sent to signify the cancelation. + message must be sent to signify the cancellation. * The verification can time out. Clients should time out a verification that does not complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` which goes unused for 5 minutes after having last sent/received it. The client should @@ -668,8 +668,8 @@ are used in addition to those already specified: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in RFC 5869 and uses the previously -agreed upon hash function for the hash function. The shared secret is supplied +In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) and uses the previously +agreed-upon hash function for the hash function. The shared secret is supplied as the input keying material. No salt is used, and the input parameter is the concatenation of: From 3877896a4cb232ce760f1556d7a1fc6de001b95f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:10:52 -0600 Subject: [PATCH 75/79] Clarify how we now expect verification to be done --- .../modules/end_to_end_encryption.rst | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index fb3d2ba5..fb320dc4 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -384,20 +384,10 @@ man-in-the-middle. This verification process requires an out-of-band channel: there is no way to do it within Matrix without trusting the administrators of the homeservers. -In Matrix, the basic process for device verification is for Alice to verify -that the public Ed25519 signing key she received via ``/keys/query`` for Bob's -device corresponds to the private key in use by Bob's device. For now, it is -recommended that clients provide mechanisms by which the user can see: - -1. The public part of their device's Ed25519 signing key, encoded using - `unpadded Base64`_. - -2. The list of devices in use for each user in a room, along with the public - Ed25519 signing key for each device, again encoded using unpadded Base64. - -Alice can then meet Bob in person, or contact him via some other trusted -medium, and use `SAS Verification`_ or ask him to read out the Ed25519 key -shown on his device, comparing it to the one shown on Alice's device. +In Matrix, verification works by Alice meeting Bob in person, or contact him +via some other trusted medium, and use `SAS Verification`_ to interactively +verify Bob's devices. Alice and Bob may also read aloud their unpadded base64 +encoded Ed25519 public key, as returned by ``/keys/query``. Device verification may reach one of several conclusions. For example: From 5ec1a50b94cca9a92951b7f3139c4ff76ed6c2b2 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:12:09 -0600 Subject: [PATCH 76/79] Linefeeds and other clarifications --- .../modules/end_to_end_encryption.rst | 39 ++++++++++--------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index fb320dc4..0de1328b 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -530,12 +530,12 @@ The process between Alice and Bob verifying each other would be: .. |BobPublicKey| replace:: :math:`K_{B}^{public}` .. |BobPrivateKey| replace:: :math:`K_{B}^{private}` .. |BobCurve25519| replace:: :math:`K_{B}^{private},K_{B}^{public}` -.. |AliceBobCurve25519| replace:: :math:`K_{A}^{private}K_{B}^{public}` .. |BobAliceCurve25519| replace:: :math:`K_{B}^{private}K_{A}^{public}` .. |AliceBobECDH| replace:: :math:`ECDH(K_{A}^{private},K_{B}^{public})` -1. Alice and Bob establish a secure out-of-band connection, such as meeting in-person or a video call. "Secure" - here means that either party cannot be impersonated, not explicit secrecy. +1. Alice and Bob establish a secure out-of-band connection, such as meeting + in-person or a video call. "Secure" here means that either party cannot be + impersonated, not explicit secrecy. #. Alice and Bob communicate which devices they'd like to verify with each other. #. Alice selects Bob's device from the device list and begins verification. #. Alice's client ensures it has a copy of Bob's device key. @@ -565,9 +565,9 @@ The process between Alice and Bob verifying each other would be: they match or not. #. Assuming they match, Alice and Bob's devices calculate the HMAC of their own device keys and a comma-separated sorted list of of the key IDs that they wish the other user - to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). The key for - the HMAC is different for each item and is calculated by generating 32 bytes (256 bits) - using `the key verification HKDF <#SAS-HKDF>`_. + to verify, using SHA-256 as the hash function. HMAC is defined in [RFC 2104](https://tools.ietf.org/html/rfc2104). + The key for the HMAC is different for each item and is calculated by generating + 32 bytes (256 bits) using `the key verification HKDF <#SAS-HKDF>`_. #. Alice's device sends Bob's device a ``m.key.verification.mac`` message containing the MAC of Alice's device keys and the MAC of her key IDs to be verified. Bob's device does the same for Bob's device keys and key IDs concurrently with Alice. @@ -611,12 +611,12 @@ to do when an error happens: * Alice or Bob can cancel the verification at any time. A ``m.key.verification.cancel`` message must be sent to signify the cancellation. * The verification can time out. Clients should time out a verification that does not - complete within 5 minutes. Additionally, clients should expire a ``transaction_id`` - which goes unused for 5 minutes after having last sent/received it. The client should - inform the user that the verification timed out, and send an appropriate ``m.key.verification.cancel`` - message to the other device. -* When the same device attempts to intiate multiple verification attempts, cancel all - attempts with that device. + complete within 10 minutes. Additionally, clients should expire a ``transaction_id`` + which goes unused for 10 minutes after having last sent/received it. The client should + inform the user that the verification timed out, and send an appropriate + ``m.key.verification.cancel`` message to the other device. +* When the same device attempts to intiate multiple verification attempts, the receipient + should cancel all attempts with that device. * When a device receives an unknown ``transaction_id``, it should send an appropriate ``m.key.verfication.cancel`` message to the other device indicating as such. This does not apply for inbound ``m.key.verification.start`` or ``m.key.verification.cancel`` @@ -658,10 +658,10 @@ are used in addition to those already specified: HKDF calculation <<<<<<<<<<<<<<<< -In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) and uses the previously -agreed-upon hash function for the hash function. The shared secret is supplied -as the input keying material. No salt is used, and the input parameter is the -concatenation of: +In all of the SAS methods, HKDF is as defined in [RFC 5869](https://tools.ietf.org/html/rfc5869) +and uses the previously agreed-upon hash function for the hash function. The shared +secret is supplied as the input keying material. No salt is used, and the input +parameter is the concatenation of: * The string ``MATRIX_KEY_VERIFICATION_SAS``. * The Matrix ID of the user who sent the ``m.key.verification.start`` message. @@ -670,8 +670,9 @@ concatenation of: * The Device ID of the device which sent the ``m.key.verification.accept`` message. * The ``transaction_id`` being used. -HKDF is used over the plain shared secret as it results in a harder attack -as well as more uniform data to work with. +.. admonition:: Rationale + HKDF is used over the plain shared secret as it results in a harder attack + as well as more uniform data to work with. For verification of each party's device keys, HKDF is as defined in RFC 5869 and uses SHA-256 as the hash function. The shared secret is supplied as the input keying @@ -697,7 +698,7 @@ The bitwise operations to get the numbers given the 5 bytes :math:`B_{0}, B_{1}, B_{2}, B_{3}, B_{4}` would be: * First: :math:`(B_{0} \ll 5 | B_{1} \gg 3) + 1000` -* Second: :math:`(B_{1} \& 0x7 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` +* Second: :math:`((B_{1} \& 0x7) \ll 10 | B_{2} \ll 2 | B_{3} \gg 6) + 1000` * Third: :math:`((B_{3} \& 0x3F) \ll 7 | B_{4} \gg 1) + 1000` The digits are displayed to the user either with an appropriate separator, From 360ac0b90088a0b9fcb6876fb9bac847fb3f0038 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 08:54:21 -0600 Subject: [PATCH 77/79] Update specification/server_server_api.rst Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- specification/server_server_api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/server_server_api.rst b/specification/server_server_api.rst index 812f0ffc..7acc71e4 100644 --- a/specification/server_server_api.rst +++ b/specification/server_server_api.rst @@ -1100,7 +1100,7 @@ a redacted copy. The signatures expected on an event are: -* The sender's server, unless the invite was created as a result of 3rd party invite. +* The ``sender``'s server, unless the invite was created as a result of 3rd party invite. The sender must already match the 3rd party invite, and the server which actually sends the event may be a different server. * For room versions 1 and 2, the server which created the ``event_id``. Other room From 46747e897ecc0b7c4c06e6488a02b5c208c87d3d Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 7 Jun 2019 10:32:06 -0600 Subject: [PATCH 78/79] ing --- specification/modules/end_to_end_encryption.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/modules/end_to_end_encryption.rst b/specification/modules/end_to_end_encryption.rst index 0de1328b..62881967 100644 --- a/specification/modules/end_to_end_encryption.rst +++ b/specification/modules/end_to_end_encryption.rst @@ -384,7 +384,7 @@ man-in-the-middle. This verification process requires an out-of-band channel: there is no way to do it within Matrix without trusting the administrators of the homeservers. -In Matrix, verification works by Alice meeting Bob in person, or contact him +In Matrix, verification works by Alice meeting Bob in person, or contacting him via some other trusted medium, and use `SAS Verification`_ to interactively verify Bob's devices. Alice and Bob may also read aloud their unpadded base64 encoded Ed25519 public key, as returned by ``/keys/query``. From be568ba9ab8c8f4342220331eb66fe70ae09e5c7 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Fri, 7 Jun 2019 19:10:26 +0100 Subject: [PATCH 79/79] link to the legalified version of MSC1779 --- proposals/1779-open-governance.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/1779-open-governance.md b/proposals/1779-open-governance.md index 0a1cfca4..bfe86eb8 100644 --- a/proposals/1779-open-governance.md +++ b/proposals/1779-open-governance.md @@ -12,6 +12,9 @@ for more context. This obsoletes [MSC1318](https://github.com/matrix-org/matrix-doc/issues/1318). +**This MSC is now formalised in the official Rules of the Matrix.org Foundation, +maintained at https://docs.google.com/document/d/1MhqsuIUxPc7Vf_y8D250mKZlLeQS6E39DPY6Azpc2NY** + ## Introduction Historically the core team of Matrix has been paid to work on it by the same