Merge branch 'master' into matthew/msc1779

.gitignore

@@ -11,3 +11,4 @@
 *.pyc
 *.swp
 _rendered.rst
+/.vscode/

CONTRIBUTING.rst

@@ -76,11 +76,11 @@ Adding to the changelog
 Currently only changes to the client-server API need to end up in a changelog. The
 other APIs are not yet stable and therefore do not have a changelog. Adding to the
 changelog can only be done after you've opened your pull request, so be sure to do
-that first. 
+that first.
 
 The changelog is managed by Towncrier (https://github.com/hawkowl/towncrier) in the
 form of "news fragments". The news fragments for the client-server API are stored
-under ``changelogs/client_server/newsfragments``. 
+under ``changelogs/client_server/newsfragments``.
 
 To create a changelog entry, create a file named in the format ``prNumber.type`` in
 the ``newsfragments`` directory. The ``type`` can be one of the following:
@@ -99,7 +99,7 @@ the ``newsfragments`` directory. The ``type`` can be one of the following:
 * ``deprecation`` - Used when deprecating something
 
 All news fragments must have a brief summary explaining the change in the contents
-of the file.
+of the file. The summary must end in a full stop to be in line with the style guide.
 
 Changes that do not change the spec, such as changes to the build script, formatting,
 CSS, etc should not get a news fragment.

api/client-server/cas_login_ticket.yaml

@@ -1,66 +0,0 @@
-# Copyright 2016 OpenMarket Ltd
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-swagger: '2.0'
-info:
-  title: "Matrix Client-Server CAS Login API"
-  version: "1.0.0"
-host: localhost:8008
-schemes:
-  - https
-  - http
-basePath: /_matrix/client/%CLIENT_MAJOR_VERSION%
-paths:
-  "/login/cas/ticket":
-    get:
-      summary: Receive and validate a CAS login ticket.
-      description: |-
-        Once the CAS server has authenticated the user, it will redirect the
-        browser to this endpoint (assuming |/login/cas/redirect|_ gave it the
-        correct ``service`` parameter).
-
-        The server MUST call ``/proxyValidate`` on the CAS server, to validate
-        the ticket supplied by the browser.
-
-        If validation is successful, the server must generate a Matrix login
-        token. It must then respond with an HTTP redirect to the URI given in
-        the ``redirectUrl`` parameter, adding a ``loginToken`` query parameter
-        giving the generated token.
-
-        If validation is unsuccessful, the server should respond with a ``401
-        Unauthorized`` error, the body of which will be displayed to the user.
-      operationId: loginByCASTicket
-      parameters:
-        - in: query
-          type: string
-          name: redirectUrl
-          description: |-
-            The ``redirectUrl`` originally provided by the client to
-            |/login/cas/redirect|_.
-          required: true
-        - in: query
-          type: string
-          name: ticket
-          description: |-
-            CAS authentication ticket.
-          required: true
-      responses:
-        302:
-          description: A redirect to the Matrix client.
-          headers:
-            Location:
-              type: "string"
-              x-example: |-
-                 https://client.example.com/?q=p&loginToken=secrettoken
-        401:
-          description: The server was unable to validate the CAS ticket.

api/client-server/sso_login_redirect.yaml

@@ -1,4 +1,4 @@
-# Copyright 2016 OpenMarket Ltd
+# Copyright 2019 New Vector Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -13,7 +13,7 @@
 # limitations under the License.
 swagger: '2.0'
 info:
-  title: "Matrix Client-Server CAS Login API"
+  title: "Matrix Client-Server SSO Login API"
   version: "1.0.0"
 host: localhost:8008
 schemes:
@@ -21,34 +21,26 @@ schemes:
   - http
 basePath: /_matrix/client/%CLIENT_MAJOR_VERSION%
 paths:
-  "/login/cas/redirect":
+  "/login/sso/redirect":
     get:
-      summary: Redirect the user's browser to the CAS interface.
+      summary: Redirect the user's browser to the SSO interface.
       description: |-
         A web-based Matrix client should instruct the user's browser to
-        navigate to this endpoint in order to log in via CAS.
+        navigate to this endpoint in order to log in via SSO.
 
-        The server MUST respond with an HTTP redirect to the CAS interface. The
+        The server MUST respond with an HTTP redirect to the SSO interface.
-        URI MUST include a ``service`` parameter giving the path of the
+      operationId: redirectToSSO
-        |/login/cas/ticket|_ endpoint (including the ``redirectUrl`` query
-        parameter).
-
-        For example, if the endpoint is called with
-        ``redirectUrl=https://client.example.com/?q=p``, it might redirect to
-        ``https://cas.example.com/?service=https%3A%2F%2Fserver.example.com%2F_matrix%2Fclient%2F%CLIENT_MAJOR_VERSION%%2Flogin%2Fcas%2Fticket%3FredirectUrl%3Dhttps%253A%252F%252Fclient.example.com%252F%253Fq%253Dp``.
-
-      operationId: redirectToCAS
       parameters:
         - in: query
           type: string
           name: redirectUrl
           description: |-
             URI to which the user will be redirected after the homeserver has
-            authenticated the user with CAS.
+            authenticated the user with SSO.
           required: true
       responses:
         302:
-          description: A redirect to the CAS interface.
+          description: A redirect to the SSO interface.
           headers:
             Location:
               type: "string"

changelogs/client_server/newsfragments/1744.clarification

@@ -1 +1 @@
-Add missing status_msg to m.presence schema
+Add missing status_msg to m.presence schema.

changelogs/client_server/newsfragments/1789.feature

@@ -0,0 +1 @@
+Add a generic SSO login API.

proposals/1704-matrix.to-permalinks.md

@@ -0,0 +1,36 @@
+# matrix.to permalink navigation
+
+Currently Matrix uses matrix.to URIs to reference rooms and other entities in a
+permanent manner. With just a room ID, users can't get into rooms if their server
+is not already aware of the room. This makes permalinks to rooms or events difficult
+as the user won't actually be able to join. A matrix.to link generated using a
+room's alias is not a permanent link due to aliases being transferable.
+
+In lieu of an improved way to reference entities permanently in Matrix, a new parameter
+is to be added to matrix.to URIs to assist clients and servers receiving permanent links
+in joining the room.
+
+For reference, existing permalinks look like this:
+
+```
+https://matrix.to/#/!somewhere:example.org
+https://matrix.to/#/!somewhere:example.org/$something:example.org
+```
+
+By adding a new parameter to the end, receivers can more easily join the room:
+
+```
+https://matrix.to/#/!somewhere:example.org?via=example-1.org&via=example-2.org
+https://matrix.to/#/!somewhere:example.org/$something:example.org?via=example-1.org&via=example-2.org
+```
+
+Clients can pass the servers directly to `/join` in the form of `server_name`
+parameters.
+
+When generating the permalinks, clients should pick servers that have a reasonably
+high chance of being in the room in the distant future. The current recommendation
+is to pick up to 3 unique servers where the first one is that of the user with the
+highest power level in the room, provided that power level is 50 or higher. The other
+2 servers should be the most popular servers in the room based on the number of joined
+users. This same heuristic should apply to the first server if no user meets the power
+level requirements.

proposals/1708-well-known-for-federation.md

@@ -0,0 +1,203 @@
+# MSC1708: .well-known support for server name resolution
+
+Currently, mapping from a server name to a hostname for federation is done via
+`SRV` records. However,
+[MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711) proposes
+requiring valid X.509 certificates on the federation endpoint. It will then be
+necessary for the homeserver to present a certificate which is valid for the
+server name. This presents difficulties for hosted server offerings: BigCorp
+may want to delegate responsibility for running its Matrix homeserver to an
+outside supplier, but it may be difficult for that supplier to obtain a TLS
+certificate for `bigcorp.com` (and BigCorp may be reluctant to let them have
+one).
+
+This MSC proposes to solve this problem by augmenting the current `SRV` record
+with a `.well-known` lookup.
+
+## Proposal
+
+For reference, the current [specification for resolving server
+names](https://matrix.org/docs/spec/server_server/unstable.html#resolving-server-names)
+is as follows:
+
+1. If the hostname is an IP literal, then that IP address should be used,
+   together with the given port number, or 8448 if no port is given.
+
+2. Otherwise, if the port is present, then an IP address is discovered by
+   looking up an AAAA or A record for the hostname, and the specified port is
+   used.
+
+3. If the hostname is not an IP literal and no port is given, the server is
+   discovered by first looking up a `_matrix._tcp` SRV record for the
+   hostname, which may give a hostname (to be looked up using AAAA or A queries)
+   and port.
+
+4. Finally, the server is discovered by looking up an AAAA or A record on the
+   hostname, and taking the default fallback port number of 8448.
+
+We insert the following between Steps 3 and 4.
+
+If the SRV record does not exist, the requesting server should make a `GET`
+request to `https://<server_name>/.well-known/matrix/server`, with normal X.509
+certificate validation, and following 30x redirects (being careful to avoid
+redirect loops). If the request does not return a 200, continue to step 4,
+otherwise:
+
+The response must have a `Content-Type` of `application/json`, and must be
+valid JSON which follows the structure documented below. Otherwise, the
+request is aborted.
+
+If the response is valid, the `m.server` property is parsed as
+`<delegated_server_name>[:<delegated_port>]`, and processed as follows:
+
+* If `<delegated_server_name>` is an IP literal, then that IP address should be
+  used, together with `<delegated_port>`, or 8448 if no port is given. The
+  server should present a valid TLS certificate for `<delegated_server_name>`.
+
+* If `<delegated_server_name>` is not an IP literal, and `<delegated_port>` is
+  present, then an IP address is discovered by looking up an AAAA or A record
+  for `<delegated_server_name>`, and the specified port is used. The server
+  should present a valid TLS certificate for `<delegated_server_name>`.
+
+  (In other words, the federation connection is made to
+  `https://<delegated_server_name>:<delegated_port>`).
+
+* If the hostname is not an IP literal and no port is given, a second SRV
+  record is looked up; this time for `_matrix._tcp.<delegated_server_name>`,
+  which may give yet another hostname (to be looked up using A/AAAA queries)
+  and port. The server must present a TLS cert for the
+  `<delegated_server_name>` from the .well-known.
+
+* If no SRV record is found, the server is discovered by looking up an AAAA
+  or A record on `<delegated_server_name>`, and taking the default fallback
+  port number of 8448.
+
+  (In other words, the federation connection is made to
+  `https://<delegated_server_name>:8448`).
+
+### Structure of the `.well-known` response
+
+The contents of the `.well-known` response should be structured as shown:
+
+```json
+{
+   "m.server": "<server>[:<port>]"
+}
+```
+
+If the response cannot be parsed as JSON, or lacks a valid `m.server` property,
+the request is considered to have failed, and no fallback to port 8448 takes
+place.
+
+The formal grammar for the `m.server` property is the same as that of a [server
+name](https://matrix.org/docs/spec/appendices.html#server-name): it is a
+hostname or IP address, followed by an optional port.
+
+### Caching
+
+Servers should not look up the `.well-known` file for every request, as this
+would impose an unacceptable overhead on both sides. Instead, the results of
+the `.well-known` request should be cached according to the HTTP response
+headers, as per [RFC7234](https://tools.ietf.org/html/rfc7234). If the response
+does not include an explicit expiry time, the requesting server should use a
+sensible default: 24 hours is suggested.
+
+Because there is no way to request a revalidation, it is also recommended that
+requesting servers cap the expiry time. 48 hours is suggested.
+
+A failure to retrieve the `.well-known` file should also be cached, though care
+must be taken that a single 500 error or connection failure should not break
+federation for an extended period. A short cache time of about an hour might be
+appropriate; alternatively, servers might use an exponential backoff.
+
+## Problems
+
+It will take a while for `.well-known` to be supported across the ecosystem;
+until it is, it will be difficult to deploy homeservers which rely on it for
+their routing: if Alice is using a current homeserver implementation, and Bob
+deploys a new implementation which relies on `.well-known` for routing, then
+Alice will be unable to send messages to Bob. (This is the same problem we have with
+[SNI](https://github.com/matrix-org/synapse/issues/1491#issuecomment-415153428).)
+
+The main defence against this seems to be to release support for `.well-known`
+as soon as possible, to maximise uptake in the ecosystem. It is likely that, as
+we approach Matrix 1.0, there will be sufficient other new features (such as
+new Room versions) that upgrading will be necessary anyway.
+
+## Security considerations
+
+The `.well-known` file potentially broadens the attack surface for an attacker
+wishing to intercept federation traffic to a particular server.
+
+## Dismissed alternatives
+
+For future reference, here are the alternative solutions which have been
+considered and dismissed.
+
+### Look up the `.well-known` file before the SRV record
+
+We could make the request for `.well-known` before looking up the `SRV`
+record. On the one hand this is maybe marginally simpler (and avoids the
+overhead of having to make *two* `SRV` lookups in the case that a `.well-known`
+is found. It might also open a future path for using `.well-known` for
+information other than delegation.
+
+Ultimately we decided to include the initial `SRV` lookup so that deployments
+have a mechanism to avoid the `.well-known` overhead in the common case that it
+is not required.
+
+### Subdomain hack
+
+As well as accepting TLS certs for `example.com`, we could also accept them for
+`delegated--matrix.example.com`. This would allow `example.com` to delegate its
+matrix hosting by (a) setting up the SRV record at `_matrix._tcp.example.com`
+and (b) setting up a CNAME at `delegated--matrix.example.com`. The latter would
+enable the delegatee to obtain an acceptable TLS certificate.
+
+This was certainly an interesting idea, but we dismissed it for the following
+reasons:
+
+* There's a security trap for anybody who lets people sign up for subdomains
+  (which is certainly not an uncommon business model): if you can register for
+  delegated--matrix.example.com, you get to intercept all the matrix traffic
+  for example.com.
+
+* Generally it feels quite unintuitive and violates the principle of least
+  surprise.
+
+* The fact that we can't find any prior art for this sets off alarm bells too.
+
+### Rely on DNS/DNSSEC
+
+If we could trust SRV records, we would be able to accept TLS certs for the
+*target* of the SRV record, which avoids this whole problem.
+
+Such trust could come from assuming that plain DNS is "good enough". However,
+DNS cache poisoning attacks are a real thing, and the fact that the designers
+of TLS chose to implement a server-name check specifically to deal with this
+case suggests we would be foolish to make this assumption.
+
+The alternative is to rely on DNSSEC to provide security for SRV records. The
+problem here is simply that DNSSEC is not that widely deployed currently. A
+number of large organisations are actively avoiding enabling it on their
+domains, so requiring DNSSEC would be a direct impediment to the uptake of
+Matrix. Furthermore, if we required DNSSEC-authenticated SRV records for
+domains doing delegation, we would end up with a significant number of
+homeservers unable to talk to such domains, because their local DNS
+infrastructure may not implement DNSSEC.
+
+Finally, if we're expecting servers to present the cert for the *target* of the
+SRV record, then we'll have to change the Host and SNI fields, and that will
+break backwards compat everywhere (and it's hard to see how to mitigate that).
+
+### Stick with perspectives
+
+The final option is to double-down on the Perspectives approach, ie to skip
+[MSC1711](https://github.com/matrix-org/matrix-doc/pull/1711). MSC1711
+discusses the reasons we do not believe this to be a viable option.
+
+## Conclusion
+
+This proposal adds a new mechanism, alongside the existing `SRV` record lookup
+for finding the server responsible for a particular matrix server_name, which
+will allow greater flexibility in deploying homeservers.

proposals/1711-x509-for-federation.md

@@ -0,0 +1,230 @@
+# MSC1711: X.509 certificate verification for federation connections
+
+TLS connections for server-to-server communication currently rely on an
+approach borrowed from the [Perspectives
+project](https://web.archive.org/web/20170702024706/https://perspectives-project.org/)
+to provide certificate verification, rather than the more normal model using
+certificates signed by trusted Certificate Authorities. This document sets out
+the reasons that this has not been a success, and suggests that we should
+instead revert to the CA model.
+
+## Background: the failure of the Perspectives approach
+
+The Perspectives approach replaces the conventional hierarchy of trust provided
+by the Certificate Authority model with a large number of "notary" servers
+distributed around the world. The intention is that the notary servers
+regularly monitor remote servers and observe the certificates they present;
+when making a connection to a new site, a client can correlate the certificate
+it presents with that seen by the notary servers. In theory this makes it very
+hard to mount a Man-in-the-Middle (MitM) attack, because it would require
+intercepting traffic between the target server and a large number of the notary
+servers.
+
+It is notable that the Perspectives project itself appears to have largely been
+abandoned: its website has largely been repurposed, the [Firefox
+extension](https://addons.mozilla.org/en-GB/firefox/addon/perspectives/) does
+not work with modern versions of Firefox, the [mailing
+list](https://groups.google.com/forum/#!forum/perspectives-dev) is inactive,
+and several of the (ten) published notary servers are no longer functional. The
+reasons for this are not entirely clear, though clearly it never gained
+widespread adoption.
+
+When Matrix was originally designed in 2014, the Perspectives project was
+heavily active, and avoiding dependencies on the relatively centralised
+Certificate Authorities was attractive, in accordance with Matrix's design as a
+decentralised protocol. However, this has not been a success in practice.
+
+Matrix was unable to make use of the existing notary servers (largely because
+we wanted to extend the protocol to include signing keys): the intention was
+that, as the Matrix ecosystem grew, public Matrix servers would act as notary
+servers. However, in practice we have ended up in a situation where almost <sup
+id="a1">[1](#f1)</sup> every Matrix homeserver either uses `matrix.org` as the
+sole notary, or does no certificate verification at all. Far from avoiding the
+centralisation of the Certificate Authorities, the entire protocol is therefore
+dependent on a single point of control at `matrix.org` - and because
+`matrix.org` only monitors from a single location, the protection against MitM
+attacks is weak.
+
+It is also clear that the Perspectives approach is poorly-understood. It is a
+common error for homeservers to be deployed behind reverse-proxies which make
+the Perspectives-based approach unreliable. The CA model, for all its flaws, is
+at least commonly used, which makes it easier for administrators to deploy
+(secure) homeservers, and allows server implementations to leverage existing
+libraries.
+
+## Proposal
+
+We propose that Matrix homeservers should be required to present valid TLS
+certificates, signed by a known Certificate Authority, on their federation
+port.
+
+In order to ease transition and give administrators time to switch to a signed
+certificate, we will continue to follow the current, perspectives-based
+approach for servers whose TLS certificates fail validation.
+
+However, this fallback will be strictly time-limited, and Matrix S2S spec r0
+will not accept self-signed certificates, nor will it include the
+`tls_fingerprints` property of the
+[`/_matrix/key/v2`](https://matrix.org/docs/spec/server_server/unstable.html#retrieving-server-keys)
+endpoints. Synapse 1.0 will not accept self-signed certificates by default.
+
+The `matrix.org` team will proactively attempt to reach out to homeserver
+administrators who do not update their certificates in the coming weeks.
+
+The process of determining which CAs are trusted to sign certificates would be
+implementation-specific, though it should almost certainly make use of existing
+operating-system support for maintaining such lists. It might also be useful if
+administrators could override this list, for the purpose of setting up a
+private federation using their own CA.
+
+It would also be useful for administrators to be able to disable the
+certificate checks for a whitelist of domains/netmasks. This would be useful
+for testing, or for networks that provide server verification themselves,
+such as like `.onion` domains on Tor or `fc00::/8` IPs on cjdns.
+
+### Interaction with SRV records
+
+With the use of `SRV` records, it is possible for the hostname of a homeserver
+to be quite different from the matrix domain it is hosting. For example, if
+there were an SRV record at `_matrix._tcp.matrix.org` which pointed to
+`server.example.com`, then any federation requests for `matrix.org` would be
+routed to `server.example.com`. The question arises as to which certificate
+`server.example.com` should present.
+
+In short: the server should present a certificate for the matrix domain
+(`matrix.org` in the above example). This ensures that traffic cannot be
+intercepted by a MitM who can control the DNS response for the `SRV` record
+(perhaps via cache-poisoning or falsifying DNS responses).
+
+This will be in line with the current
+[requirements](https://matrix.org/docs/spec/server_server/unstable.html#resolving-server-names)
+in the Federation API specification for the `Host`, and by implication, the TLS
+Server Name Indication <sup id="a2">[2](#f2)</sup>. It is also consistent with
+the recommendations of
+[RFC6125](https://tools.ietf.org/html/rfc6125#section-6.2.1) and the
+conventions established by the XMPP protocol (per [RFC6120](https://tools.ietf.org/html/rfc6120#section-13.7.2.1).
+
+### Extensions
+
+HTTP-Based Public Key Pinning (HPKP) and
+[Certificate transparency](https://www.certificate-transparency.org) are
+both HTTP extensions which attempt to work around some of the deficiencies in
+the CA model, by making it more obvious if a CA has issued a certificate
+incorrectly.
+
+HPKP has not been particularly successful, and is
+[deprecated](https://developers.google.com/web/updates/2018/04/chrome-67-deps-rems#deprecate_http-based_public_key_pinning)
+in Google Chrome as of April 2018. Certificate transparency, however, is seeing
+widespread adoption from Certificate Authories and HTTP clients.
+
+This proposal sees both technologies as optional techniques which could be
+provided by homeserver implementations. We encourage but do not mandate the use
+of Certificate Transparency.
+
+### Related work
+
+The Perspectives approach is also currently used for exchanging the keys that
+are used by homeservers to sign Matrix events and federation requests (the
+"signing keys"). Problems similar to those covered here also apply to that
+mechanism. This is discussed at [#1685](thttps://github.com/matrix-org/matrix-doc/issues/1685).
+
+## Alternatives
+
+There are well-known problems with the CA model, including a number of
+widely-published incidents in which CAs have issued certificates
+incorrectly. It is therefore important to consider alternatives to the CA
+model.
+
+### Improving support for the Perspectives model
+
+In principle, we could double-down on the Perspectives approach, and make an effort
+to get servers other than `matrix.org` used as notary servers. However, there
+remain significant problems with such an approach:
+
+* Perspectives remain complex to configure correctly. Ideally, administrators
+  need to make conscious choices about which notaries to trust, which is hard
+  to do, especially for newcomers to the ecosystem. (In practice, people use
+  the out-of-the-box configuration, which is why everyone just uses
+  `matrix.org` today).
+
+* A *correct* implementation of Perspectives really needs to take into account
+  more than the latest state seen by the notary servers: some level of history
+  should be taken into account too.
+
+Essentially, whilst we still believe the Perspectives approach has some merit,
+we believe it needs further research before it can be relied upon. We believe
+that the resources of the Matrix ecosystem are better spent elsewhere.
+
+### DANE
+
+DNS-Based Authentication of Named Entities (DANE) can be used as an alternative
+to the CA model. (It is arguably more appropriately used *together* with the CA
+model.)
+
+It is not obvious to the author of this proposal that DANE provides any
+material advantages over the CA model. In particular it replaces the
+centralised trust of the CAs with the centralised trust of the DNS registries.
+
+## Potential issues
+
+Beyond the problems already discussed with the CA model, requiring signed
+certificates comes with a number of downsides.
+
+### More difficult setup
+
+Configuring a working, federating homeserver is a process fraught with
+pitfalls. This proposal adds the requirement to obtain a signed certificate to
+that process. Even with modern intiatives such as Let's Encrypt, this is
+another procedure requiring manual intervention across several moving parts.
+
+On the other hand: obtaining an SSL certificate should be a familiar process to
+anybody capable of hosting a production homeserver (indeed, they should
+probably already have one for the client port). This change also opens the
+possibility of putting the federation port behind a reverse-proxy without the
+need for additional configuration. Hopefully making the certificate usage more
+conventional will offset the overhead of setting up a certificate.
+
+Furthermore, homeserver implementations could provide an implementation of the
+ACME protocol and integration with Let's Encrypt, to make it easier for
+administrators to get started. (This would of course be
+implementation-specific, and administrators who wanted to keep control of the
+certificate creation process would be free to do so).
+
+### Inferior support for IP literals
+
+Whilst it is possible to obtain an SSL cert which is valid for a literal IP
+address, this typically requires purchase of a premium certificate; in
+particular, Let's Encrypt will not issue certificates for IP literals. This may
+make it impractical to run a homeserver which uses an IP literal, rather than a
+DNS name, as its `server_name`.
+
+It has long been the view of the `matrix.org` administrators that IP literals
+are only really suitable for internal testing. Those who wish to use them for
+that purpose could either disable certificate checks inside their network, or
+use their own CA to issue certificates.
+
+### Inferior support for hidden services (`.onion` addresses)
+
+It is currently possible to correctly route traffic to a homeserver on a
+`.onion` domain, provided any remote servers which may need to reach that
+server are configured to route to such addresses via the Tor network. However,
+it can be difficult to get a certificate for a `.onion` domain (again, Let's
+Encrypt do not support them).
+
+The reasons for requiring a signed certificate (or indeed, for using TLS at
+all) are weakened when traffic is routed via the Tor network. Administrators
+using the Tor network could disable certificate checks for `.onion` addresses.
+
+## Conclusion
+
+We believe that requiring homeservers to present an X.509 certificate signed by
+a recognised Certificate Authority will improve security, reduce
+centralisation, and eliminate some common deployment pitfalls.
+
+<a id="f1"/>[1] It's *possible* to set up homeservers to use servers other than
+`matrix.org` as notaries, but only a minority are actually set up this
+way. [↩](#a1)
+
+<a id="f2"/>[2] I've not been able to find an authoritative source on this, but
+most reverse-proxies will reject requests where the SNI and Host headers do not
+match. [↩](#a2)

proposals/1753-capabilities.md

@@ -0,0 +1,117 @@
+# MSC1753: client-server capabilities API
+
+A mechanism is needed for clients to interrogate servers to establish whether
+particular operations can be performed.
+
+For example, users may not be able to change their password if a server is
+configured to authenticate against a separate system, in which case it is
+nonsensical to offer the user such an option.
+
+## Proposal
+
+### `GET /_matrix/client/r0/capabilities`
+
+We will add a new endpoint to the client-server API: `GET
+/_matrix/client/r0/capabilities`. The endpoint will be authenticated as normal
+via an access token.
+
+The server should reply with a list of supported features, as shown:
+
+```json
+{
+    "capabilities": {
+        "m.capability_one": {}
+    }
+}
+```
+
+The keys of the `capabilities` object are capability identifiers. As with
+other identifiers in the Matrix protocol, the `m.` prefix is reserved for
+definition in the Matrix specification; other values can be used within an
+organisation following the Java package naming conventions.
+
+The values of the `capabilities` object will depend on the capability
+identifier, though in general the empty object will suffice.
+
+### Initial capability identifiers
+
+As a starting point, a single capability identifier is proposed:
+`m.change_password`, which should be considered supported if it is possible to
+change the user's password via the `POST /_matrix/client/r0/account/password`
+API.
+
+The value of the `capabilities` object in the response should be the empty
+object.
+
+### Fallback behaviour
+
+Clients will need to be aware of servers which do not support the new endpoint,
+and fall back to their current behaviour if they receive a 404 response.
+
+### Suitable applications
+
+In general, capabilities advertised via this endpoint should depend in some way
+on the state of the user or server - in other words, they will be inherently
+"optional" features in the API.
+
+This endpoint should *not* be used to advertise support for experimental or
+unstable features, which is better done via `/client/versions` (see
+[MSC1497](https://github.com/matrix-org/matrix-doc/issues/1497)).
+
+Examples of features which might reasonably be advertised here include:
+
+ * Whether the server supports user presence.
+
+ * Whether the server supports other optional features. The following could be
+   made optional via this mechanism:
+   * Room directory
+   * URL previews
+
+ * Policy restricitions, such as:
+   * Whether certain types of content are permitted on this server.
+   * The number of rooms you are allowed in.
+   * Configured ratelimits.
+
+Features which might be better advertised elsewhere include:
+
+ * Support for e2e key backups
+   ([MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219)) - list in
+   `/client/versions`.
+
+ * Support for lazy-loading of room members - list in `/client/versions`.
+
+ * Media size limits - list in `/media/r0/config`, because the media server may
+   be a separate process.
+
+ * Optional transports/encodings for the CS API - probably better handled via
+   HTTP headers etc.
+
+ * Variations in room state resolution - this is implied via the room version
+   (which is in the `m.room.create` event).
+
+## Tradeoffs
+
+One alternative would be to provide specific ways of establishing support for
+each operation: for example, a client might send an `GET
+/_matrix/client/r0/account/password` request to see if the user can change
+their password. The concern with this approach is that this could require a
+large number of requests to establish which entries should appear on a menu or
+dialog box.
+
+Another alternative is to provide a generic query mechanism where the client
+can query for specific capabilities it is interested in. However, this adds
+complication and makes it harder to discover capability identifiers.
+
+## Potential issues
+
+None yet identified.
+
+## Security considerations
+
+None yet identified.
+
+## Conclusion
+
+We propose adding a new endpoint to the Client-Server API, which will allow
+clients to query for supported operations so that they can decide whether to
+expose them in their user-interface.

scripts/css/basic.css

@@ -462,6 +462,28 @@ dl.glossary dt {
     font-style: oblique;
 }
 
+/* -- proposals page -------------------------------------------------------- */
+
+#tables-of-tracked-proposals h2 {
+    padding-left: 10px;
+    position: -webkit-sticky;
+    position: sticky;
+}
+
+/* Move sticky headers below header bar on desktop */
+@media all and (min-width:980px) {
+    #tables-of-tracked-proposals h2 {
+        top: 52px;
+    }
+}
+
+/* Sticky headers stick to the top on mobile */
+@media all and (min-width:0px) and (max-width: 980px) {
+    #tables-of-tracked-proposals h2 {
+        top: 0px;
+    }
+}
+
 /* -- code displays --------------------------------------------------------- */
 
 pre {

specification/client_server_api.rst

@@ -1016,9 +1016,14 @@ follows:
   }
 
 As with `token-based`_ interactive login, the ``token`` must encode the
-user id. In the case that the token is not valid, the homeserver must respond
+user ID. In the case that the token is not valid, the homeserver must respond
 with ``403 Forbidden`` and an error code of ``M_FORBIDDEN``.
 
+If the homeserver advertises ``m.login.sso`` as a viable flow, and the client
+supports it, the client should redirect the user to the ``/redirect`` endpoint
+for `Single Sign-On <#sso-client-login>`_. After authentication is complete, the
+client will need to submit a ``/login`` request matching ``m.login.token``.
+
 {{login_cs_http_api}}
 
 {{logout_cs_http_api}}

specification/modules/cas_login.rst

@@ -1,119 +0,0 @@
-.. Copyright 2016 OpenMarket Ltd
-..
-.. Licensed under the Apache License, Version 2.0 (the "License");
-.. you may not use this file except in compliance with the License.
-.. You may obtain a copy of the License at
-..
-..     http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS,
-.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-.. See the License for the specific language governing permissions and
-.. limitations under the License.
-
-CAS-based client login
-======================
-
-.. _module:cas_login:
-
-`Central Authentication Service
-<https://github.com/apereo/cas/blob/master/docs/cas-server-documentation/protocol/CAS-Protocol-Specification.md>`_
-(CAS) is a web-based single sign-on protocol.
-
-An overview of the process, as used in Matrix, is as follows:
-
-1. The Matrix client instructs the user's browser to navigate to the
-   |/login/cas/redirect|_ endpoint on the user's homeserver.
-
-2. The homeserver responds with an HTTP redirect to the CAS user interface,
-   which the browser follows.
-
-3. The CAS system authenticates the user.
-
-4. The CAS server responds to the user's browser with a redirect back to the
-   |/login/cas/ticket|_ endpoint on the homeserver, which the browser
-   follows. A 'ticket' identifier is passed as a query parameter in the
-   redirect.
-
-5. The homeserver receives the ticket ID from the user's browser, and makes a
-   request to the CAS server to validate the ticket.
-
-6. Having validated the ticket, the homeserver responds to the browser with a
-   third HTTP redirect, back to the Matrix client application. A login token
-   is passed as a query parameter in the redirect.
-
-7. The Matrix client receives the login token and passes it to the |/login|_
-   API.
-
-Client behaviour
-----------------
-
-The client starts the process by instructing the browser to navigate to
-|/login/cas/redirect|_ with an appropriate ``redirectUrl``. Once authentication
-is successful, the browser will be redirected to that ``redirectUrl``.
-
-.. TODO-spec
-
-   Should we recommend some sort of CSRF protection here (specifically, we
-   should guard against people accidentally logging in by sending them a link
-   to ``/login/cas/redirect``.
-
-   Maybe we should recommend that the ``redirectUrl`` should contain a CSRF
-   token which the client should then check before sending the login token to
-   ``/login``?
-
-{{cas_login_redirect_cs_http_api}}
-{{cas_login_ticket_cs_http_api}}
-
-Server behaviour
-----------------
-
-The URI for the CAS system to be used should be configured on the server by the
-server administrator.
-
-Handling the redirect endpoint
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-When responding to the ``/login/cas/redirect`` endpoint, the server must
-generate a URI for the CAS login page. The server should take the base CAS URI
-described above, and add a ``service`` query parameter. This parameter MUST be
-the URI of the ``/login/cas/ticket`` endpoint, including the ``redirectUrl``
-query parameter. Because the homeserver may not know its base URI, this may
-also require manual configuration.
-
-.. TODO-spec:
-
-   It might be nice if the server did some validation of the ``redirectUrl``
-   parameter, so that we could check that aren't going to redirect to a non-TLS
-   endpoint, and to give more meaningful errors in the case of
-   faulty/poorly-configured clients.
-
-Handling the authentication endpoint
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-When responding to the ``/login/cas/ticket`` endpoint, the server MUST make a
-request to the CAS server to validate the provided ticket. The server MAY also
-check for certain user attributes in the response. Any required attributes
-should be configured by the server administrator.
-
-Once the ticket has been validated, the server MUST map the CAS ``user_id``
-to a valid `Matrix user identifier <../index.html#user-identifiers>`_. The
-guidance in `Mapping from other character sets
-<../index.html#mapping-from-other-character-sets>`_ may be useful.
-
-If the generated user identifier represents a new user, it should be registered
-as a new user.
-
-Finally, the server should generate a short-term login token. The generated
-token should be a macaroon, suitable for use with the ``m.login.token`` type of
-the |/login|_ API, and `token-based interactive login <#token-based>`_. The
-lifetime of this token SHOULD be limited to around five seconds.
-
-
-.. |/login| replace:: ``/login``
-.. _/login: #post-matrix-client-%CLIENT_MAJOR_VERSION%-login
-.. |/login/cas/redirect| replace:: ``/login/cas/redirect``
-.. _/login/cas/redirect: #get-matrix-client-%CLIENT_MAJOR_VERSION%-login-cas-redirect
-.. |/login/cas/ticket| replace:: ``/login/cas/ticket``
-.. _/login/cas/ticket: #get-matrix-client-%CLIENT_MAJOR_VERSION%-login-cas-ticket

specification/modules/sso_login.rst

@@ -0,0 +1,113 @@
+.. Copyright 2019 New Vector Ltd
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+SSO client login
+================
+
+.. _module:sso_login:
+
+Single Sign-On (SSO) is a generic term which refers to protocols which allow
+users to log into applications via a single web-based authentication portal.
+Examples include "Central Authentication Service" (CAS) and SAML.
+
+An overview of the process, as used in Matrix, is as follows:
+
+1. The Matrix client instructs the user's browser to navigate to the
+   |/login/sso/redirect|_ endpoint on the user's homeserver.
+
+2. The homeserver responds with an HTTP redirect to the SSO user interface,
+   which the browser follows.
+
+3. The SSO system authenticates the user.
+
+4. The SSO server and the homeserver interact to verify the user's identity
+   and other authentication information, potentially using a number of redirects.
+
+5. The browser is directed to the ``redirectUrl`` provided by the client with
+   a ``loginToken`` query parameter for the client to log in with.
+
+.. Note::
+   In the older `r0.4.0 version <https://matrix.org/docs/spec/client_server/r0.4.0.html#cas-based-client-login>`_
+   of this specification it was possible to authenticate via CAS when the server
+   provides a ``m.login.cas`` login flow. This specification deprecates the use
+   of ``m.login.cas`` to instead prefer ``m.login.sso``, which is the same process
+   with the only change being which redirect endpoint to use: for ``m.login.cas``, use
+   ``/cas/redirect`` and for ``m.login.sso`` use ``/sso/redirect`` (described below).
+   The endpoints are otherwise the same.
+
+Client behaviour
+----------------
+
+The client starts the process by instructing the browser to navigate to
+|/login/sso/redirect|_ with an appropriate ``redirectUrl``. Once authentication
+is successful, the browser will be redirected to that ``redirectUrl``.
+
+.. TODO-spec
+
+   Should we recommend some sort of CSRF protection here (specifically, we
+   should guard against people accidentally logging in by sending them a link
+   to ``/login/sso/redirect``.
+
+   Maybe we should recommend that the ``redirectUrl`` should contain a CSRF
+   token which the client should then check before sending the login token to
+   ``/login``?
+
+{{sso_login_redirect_cs_http_api}}
+
+Server behaviour
+----------------
+
+The URI for the SSO system to be used should be configured on the server by the
+server administrator. The server is expected to set up any endpoints required to
+interact with that SSO system. For example, for CAS authentication the homeserver
+should provide a means for the administrator to configure where the CAS server is
+and the REST endpoints which consume the ticket. A good reference for how CAS could
+be implemented is available in the older `r0.4.0 version <https://matrix.org/docs/spec/client_server/r0.4.0.html#cas-based-client-login>`_
+of this specification.
+
+Handling the redirect endpoint
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When responding to the ``/login/sso/redirect`` endpoint, the server must
+generate a URI for the SSO login page with any appropriate parameters.
+
+.. TODO-spec:
+
+   It might be nice if the server did some validation of the ``redirectUrl``
+   parameter, so that we could check that aren't going to redirect to a non-TLS
+   endpoint, and to give more meaningful errors in the case of
+   faulty/poorly-configured clients.
+
+Handling the authentication endpoint
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Once the homeserver has verified the user's identity with the SSO system, it
+MUST map the user ID to a valid `Matrix user identifier <../index.html#user-identifiers>`_.
+The guidance in `Mapping from other character sets
+<../index.html#mapping-from-other-character-sets>`_ may be useful.
+
+If the generated user identifier represents a new user, it should be registered
+as a new user.
+
+Finally, the server should generate a short-term login token. The generated
+token should be a macaroon, suitable for use with the ``m.login.token`` type of
+the |/login|_ API, and `token-based interactive login <#token-based>`_. The
+lifetime of this token SHOULD be limited to around five seconds. This token is
+given to the client via the ``loginToken`` query parameter previously mentioned.
+
+
+.. |/login| replace:: ``/login``
+.. _/login: #post-matrix-client-%CLIENT_MAJOR_VERSION%-login
+.. |/login/sso/redirect| replace:: ``/login/sso/redirect``
+.. _/login/sso/redirect: #get-matrix-client-%CLIENT_MAJOR_VERSION%-login-sso-redirect

specification/proposals_intro.rst

@@ -320,7 +320,7 @@ Lifetime States
 ===============================  =============================  ====================================
 Name                             GitHub Label                   Description
 ===============================  =============================  ====================================
-Proposal Drafting and Feedback   N/A                            A proposal document which is still work-in-progress but is being shared to incorporate feedback
+Proposal Drafting and Feedback   N/A                            A proposal document which is still work-in-progress but is being shared to incorporate feedback. Please prefix your proposal's title with ``[WIP]`` to make it easier for reviewers to skim their notifications list.
 Proposal In Review               proposal-in-review             A proposal document which is now ready and waiting for review by the Spec Core Team and community
 Proposed Final Comment Period    proposed-final-comment-period  Currently awaiting signoff of a majority of team members in order to enter the final comment period
 Final Comment Period             final-comment-period           A proposal document which has reached final comment period either for merge, closure or postponement

specification/targets.yaml

@@ -61,7 +61,7 @@ groups:  # reusable blobs of files when prefixed with 'group:'
     - modules/account_data.rst
     - modules/admin.rst
     - modules/event_context.rst
-    - modules/cas_login.rst
+    - modules/sso_login.rst
     - modules/dm.rst
     - modules/ignore_users.rst
     - modules/stickers.rst