matrix-spec-proposals/proposals/3923-ietf-spec-process.md

MSC3923: Bringing Matrix into the IETF process

The More Instant Messaging Interoperability (MIMI) working group aims to specify the minimal set of mechanisms required to support modern messaging in an interoperable way, and Matrix is an example of how that can be accomplished by acting as a generic and openly specified communications layer. Matrix currently uses a specification process that allows for rapid prototyping, which the ecosystem relies on: it is important to Matrix to maintain this highly dynamic approach to specification while still being able to participate in venues such as IETF for interoperable messaging.

This proposal covers the process approach on the Matrix side to support a venture into the IETF process, largely as it relates to being able to rapidly build/test features without going through a naturally-lengthy IETF review.

Background

Matrix is currently specified at https://spec.matrix.org/v1.6 and split into several coarse domains:

1. The Client-Server API (or "CS API"), covering the communication between a client and server.
2. The Server-Server API (or "Federation API"), covering communication (transport) between servers.
3. The Room Version specifications, which define how servers (and sometimes clients) are expected to behave in a given room. This is the core feature of Matrix: how a room actually works.
4. The Application Service API (or "Appservice API"), which defines an interface for high-traffic bots and bridges to communicate with a homeserver.
5. The Identity Service API, covering how clients and servers interact with an identity server. Identity servers store mappings of third party identifiers to Matrix IDs, and are not part of the authentication or user infrastructure in Matrix: you can use Matrix without ever touching an identity server.
6. The Push Gateway API, which enables mobile app developers to use push notifications in a decentralized environment.
7. Supporting documents such as the introduction page and appendices, largely covering grammar and general descriptions of Matrix.

The intention of splitting Matrix into these domains is to allow for implementations of Matrix's core principles without having to be tied unnecessarily to writing APIs that won't be used. For example, it's entirely reasonable that a purpose-built homeserver only implement some room versions and the federation API without ever attempting to support appservices or the specified Client-Server API. This is often the case when a project is adding support for Matrix: they are most interested in interoperating with other Matrix homeservers and already have their own client-server API to work with.

Note: Historically, a lot of the behaviours a server needs to implement have ended up in Matrix's Client-Server API. It is a goal of the Spec Core Team (SCT) to move the remaining behaviours to a more correct place, therefore reinforcing the original intent mentioned above more concretely. This MSC describes a world where the SCT's goal has been achieved, not the current state. A natural consequence of going through the IETF process is that Matrix's own specification will improve as areas are identified as needing better segmentation.

Proposal

With consideration for how Matrix is split into major domains, only a small portion of Matrix's core needs to be specified for MIMI. Specifically, the areas covered by I-D.ralston-mimi-matrix-framework (MSC3977): a single room version, definitions for what a homeserver, event, room, and user are, loose descriptions for what is needed of a federation transport API, and noting the importance of end-to-end encryption in the messaging sphere. The Client-Server API, Application Service API, Identity Service API, and Push Gateway API are all entirely out of scope because they're simply not needed for MIMI.

Proposing these areas through the IETF process as-is would normally mean that they get transferred to the IETF, using the IETF process instead of MSCs for any future changes after being accepted. This could put a damper on Matrix's ability to experiment with features, though. In order to avoid this damper, we instead use an LTS (Long-Term Stable) approach where the core of Matrix is versioned as an LTS within the IETF process and regular/non-LTS Matrix continues as-is.

Logistically, this means room versions (as the primary area of concern) will get an "LTS stable" designation and associated identifier. Room versions in non-LTS Matrix will continue to get created, though not all of them will end up in the IETF process: when it makes sense to do, such as when a given room version "feels" stable enough, that room version will get proposed to the IETF through the normal Internet-Draft process.

For example, if given room versions 10, 11, 12, and 13 from Matrix, room version 13 might be proposed to the IETF while the others simply aren't. While 13 works through the IETF process (being renamed as I.2 in the proposal), changes might happen to it and other non-LTS room versions get created. Like with Matrix->LTS intent, anything which makes sense to bring to the other process gets raised as such in the traditional ways (an MSC in the case of non-LTS, and an Internet-Draft in the case of IETF).

Room versions typically hold the core of Matrix, however anything which needs to enter the IETF process would do so just the same as room versions: anything which makes sense, and when it makes sense, would go through the IETF process and carry any useful changes back to Matrix as MSCs.

Similarly, it's entirely possible that an IETF Internet-Draft gets raised without an accompanying MSC to change how the LTS version of Matrix works: if that change makes sense to bring over, it would be.

In practice, the Matrix.org Foundation would be the ones ensuring both backwards compatibility between LTS and non-LTS Matrix as well as ferrying changes back and forth as needed. This would fall under the Spec Core Team's remit.

Handling backwards-incompatible changes

Given the IETF and Matrix processes can both modify their copy of the protocol without involving the other process, it's very possible that one doesn't work with the other anymore. This should be quite easy to mitigate, however: because we're using room versions to contain the core protocol, servers intending to support both LTS and non-LTS versions simply implement both room versions and they'll be covered.

Alternatives

Other ideas have been discussed with some members of the Spec Core Team, with the combination of them appearing in the above proposal. Taking elements of the proposal and creating a new proposal around them is feasible, though those approaches do not independently solve the concerns the SCT has. Namely, we'd like to:

• have a compatible version of Matrix specified at the IETF level (specifically for federation)
• be free to experiment without the burden of process, and rapidly respond to shifts in the larger, external, ecosystem as needed
• keep the layers which aren't needed out of the IETF, for sake of implementation effort for Matrix-compatible implementations from the IETF level

... and other points along those same sentiments.

Suggestions for alternative approaches are welcome, though unlike other MSCs, solutions which are more carefully considered than usual are appreciated.

Dependent MSCs

This MSC ends up affecting the future of the following MSCs, though is not dependent itself:

• https://github.com/matrix-org/matrix-spec-proposals/pull/3918
• https://github.com/matrix-org/matrix-spec-proposals/pull/3919
• https://github.com/matrix-org/matrix-spec-proposals/pull/3977
• Future MSCs/I-Ds for Matrix-as-MIMI