archive
/
matrix-spec-proposals
mirror of https://github.com/matrix-org/matrix-spec-proposals
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'matrix-org:main' into main

Browse Source
pull/3659/head
Josh Qou 1 year ago committed by GitHub
parent d3c3ddd917 a3778b3f82
commit f8cd199c25
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
666 changed files with 7315 additions and 56778 deletions
Show Stats Download Patch File Download Diff File

2
.github/FUNDING.yml vendored
Unescape Escape View File

@ -1,2 +0,0 @@
patreon: matrixdotorg
liberapay: matrixdotorg

13
.github/ISSUE_TEMPLATE/clarification.md vendored
Unescape Escape View File

@ -1,13 +0,0 @@
---
name: Clarity problem
about: Report an area of the spec that is unclear.
title: ''
labels: 'clarification'
assignees: ''
---
**Link to problem area**:
**Issue**
What is wrong? How can we improve?

8
.github/ISSUE_TEMPLATE/config.yaml vendored
Unescape Escape View File

@ -1,8 +0,0 @@
blank_issues_enabled: true
contact_links:
- name: Matrix Spec Discussion
url: "https://matrix.to/#/#matrix-spec:matrix.org"
about: Questions about the spec and proposal process can be asked here.
- name: Matrix Security Policy
url: https://www.matrix.org/security-disclosure-policy/
about: Learn more about our security disclosure policy.

13
.github/ISSUE_TEMPLATE/cosmetic-bug.md vendored
Unescape Escape View File

@ -1,13 +0,0 @@
---
name: Cosmetic issue
about: Report an issue with how the spec looks.
title: ''
labels: 'aesthetic'
assignees: ''
---
**Link to problem area**:
**Issue**
What is wrong? What can we do to improve?

12
.github/ISSUE_TEMPLATE/idea.md vendored
Unescape Escape View File

@ -1,12 +0,0 @@
---
name: Spec idea
about: Suggest a future MSC idea.
title: ''
labels: 'improvement'
assignees: ''
---
**Suggestion**
What would you like to see in Matrix? If your idea is vaguely complete enough, we
recommend submitting [an MSC](https://matrix.org/docs/spec/proposals) instead.

16
.github/ISSUE_TEMPLATE/spec-bug.md vendored
Unescape Escape View File

@ -1,16 +0,0 @@
---
name: Documentation error
about: Report an issue with the spec itself (incorrect text).
title: ''
labels: 'spec-bug'
assignees: ''
---
**Link to problem area**:
**Issue**
What is wrong?
**Expected behaviour**
How can the issue be fixed? Links to implementations/documents which prove the spec is wrong are appreciated.

12
.github/PULL_REQUEST_TEMPLATE/ready-proposal.md vendored
Unescape Escape View File

@ -11,9 +11,11 @@ assignees: ''
### Pull Request Checklist
<!-- Please read CONTRIBUTING.rst before submitting your pull request -->
<!-- Please read CONTRIBUTING.md before submitting your pull request -->
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog)
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off)
* [ ] Pull request includes ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above.
* [ ] Pull request title and file name include this PR's number as the MSC number.
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-spec-proposals/blob/master/CONTRIBUTING.md#sign-off)
* [ ] Update the title and file name of your proposal to match this PR's number (after opening).
* [ ] Pull request includes a ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above.
* [ ] Ask in
[#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) to
get feedback on this PR.

16
.github/PULL_REQUEST_TEMPLATE/spec-change.md vendored
Unescape Escape View File

@ -1,16 +0,0 @@
---
name: Spec clarification/not a proposal
about: A change that's not a spec proposal, such as a clarification to the spec itself.
title: ''
labels: ''
assignees: ''
---
### Pull Request Checklist
<!-- Please read CONTRIBUTING.rst before submitting your pull request -->
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog)
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off)
* [ ] Pull request is classified as ['other changes'](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#other-changes)

10
.github/PULL_REQUEST_TEMPLATE/wip-proposal.md vendored
Unescape Escape View File

@ -11,10 +11,10 @@ assignees: ''
### Pull Request Checklist
<!-- Please read CONTRIBUTING.rst before submitting your pull request -->
<!-- Please read CONTRIBUTING.md before submitting your pull request -->
* [ ] Pull request includes a [changelog file](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#adding-to-the-changelog)
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-doc/blob/master/CONTRIBUTING.rst#sign-off)
* [ ] A ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above.
* [ ] Pull request includes a [sign off](https://github.com/matrix-org/matrix-spec-proposals/blob/master/CONTRIBUTING.md#sign-off)
* [ ] Update the title and file name of your proposal to match this PR's number (after opening).
* [ ] Ask in [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) to get this marked as ready for review, once it is ready for review.
* [ ] Pull request includes a ['Rendered' link](https://matrix.org/docs/spec/proposals#process) above.
<!-- Once the proposal is ready for review, ask in [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) to get it marked as such. -->

10
.github/_typos.toml vendored
Unescape Escape View File

@ -1,12 +1,2 @@
[files]
extend-exclude = ["/themes", "/attic", "/data-definitions", "*.css", "package-lock.json"]
[default]
check-filename = true
[default.extend-identifiers]
au1ba7o = "au1ba7o"
[default.extend-words]
Appy = "Appy"
fo = "fo"

175
.github/workflows/main.yml vendored
Unescape Escape View File

@ -1,175 +0,0 @@
name: "Spec"
on:
push:
branches:
- main
tags:
- v*
pull_request:
workflow_dispatch:
jobs:
validate-openapi:
name: "🔎 Validate OpenAPI specifications"
runs-on: ubuntu-latest
steps:
- name: "📥 Source checkout"
uses: actions/checkout@v2
- name: "➕ Setup Node"
uses: actions/setup-node@v2
with:
node-version: '14'
- name: "⚙️ npm"
working-directory: "./scripts"
run: |
npm install
- name: "🔎 Run validator"
working-directory: "./scripts"
run: |
node validator.js -s "../data/api/client-server"
check-examples:
name: "🔎 Check Event schema examples"
runs-on: ubuntu-latest
container: uhoreg/matrix-doc-build
steps:
- name: "📥 Source checkout"
uses: actions/checkout@v2
- name: "🔎 Run validator"
run: |
/env/bin/python scripts/check-event-schema-examples.py
calculate-baseurl:
name: "⚙️ Calculate baseURL for later jobs"
runs-on: ubuntu-latest
outputs:
baseURL: "${{ steps.set-baseurl.outputs.baseURL }}"
steps:
# For PRs, set the baseURL to `/`.
# For releases, set the baseURL to `/$tag` (eg: `/v1.2`).
# Otherwise, set it to `/unstable`.
- name: "⚙️ Calculate baseURL"
id: set-baseurl
# Double brackets on the elif to avoid auto-escaping refs/tags/* because we need
# the asterisk matching behaviour, not the literal string.
run: |
if [ "${GITHUB_EVENT_NAME}" == "pull_request" ]; then
echo ::set-output name=baseURL::/
elif [[ "${GITHUB_REF}" == refs/tags/* ]]; then
echo ::set-output name=baseURL::"/${GITHUB_REF/refs\/tags\//}"
else
echo ::set-output name=baseURL::/unstable
fi
build-openapi:
name: "🐍 Build OpenAPI definitions"
runs-on: ubuntu-latest
container: "python:3.9"
needs: [calculate-baseurl]
steps:
- name: "📥 Source checkout"
uses: actions/checkout@v2
- name: "📦 Asset creation"
run: |
python3 -m venv env && . env/bin/activate
pip install -r scripts/requirements.txt
# The output path matches the final deployment path at spec.matrix.org
scripts/dump-swagger.py \
--base-url "https://spec.matrix.org${{ needs.calculate-baseurl.outputs.baseURL }}" \
-o spec/client-server-api/api.json
tar -czf openapi.tar.gz spec
- name: "📤 Artifact upload"
uses: actions/upload-artifact@v2
with:
name: openapi-artifact
path: openapi.tar.gz
build-spec:
name: "📖 Build the spec"
runs-on: ubuntu-latest
needs: [calculate-baseurl, build-openapi]
steps:
- name: "➕ Setup Node"
uses: actions/setup-node@v2
with:
node-version: '14'
- name: "➕ Setup Hugo"
uses: peaceiris/actions-hugo@c03b5dbed22245418539b65eb9a3b1d5fdd9a0a6
with:
hugo-version: '0.85.0'
extended: true
- name: "📥 Source checkout"
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: "⚙️ npm"
run: |
npm i
npm run get-proposals
- name: "⚙️ hugo"
run: hugo --baseURL "${{ needs.calculate-baseurl.outputs.baseURL }}" -d "spec"
# We manually unpack the spec OpenAPI definition JSON to the website tree
# to make it available to the world in a canonical place:
# https://spec.matrix.org/latest/client-server-api/api.json
# Works for /unstable/ and /v1.1/ as well.
- name: "📥 Spec definition download"
uses: actions/download-artifact@v2
with:
name: openapi-artifact
- name: "📝 Unpack the OpenAPI definitions in the right location"
run: |
tar -xzf openapi.tar.gz
- name: "📦 Tarball creation"
run: tar -czf spec.tar.gz spec
- name: "📤 Artifact upload"
uses: actions/upload-artifact@v2
with:
name: spec-artifact
path: spec.tar.gz
build-historical-spec:
name: "📖 Build the historical backup spec"
runs-on: ubuntu-latest
needs: [build-openapi]
if: ${{ startsWith(github.ref, 'refs/tags/') }}
steps:
- name: "➕ Setup Node"
uses: actions/setup-node@v2
with:
node-version: '14'
- name: "➕ Setup Hugo"
uses: peaceiris/actions-hugo@c03b5dbed22245418539b65eb9a3b1d5fdd9a0a6
with:
hugo-version: '0.85.0'
extended: true
- name: "📥 Source checkout"
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: "⚙️ npm"
run: |
npm i
npm run get-proposals
- name: "⚙️ hugo"
# Create a baseURL like `/v1.2` out of the `v1.2` tag
run: |
echo -e '[params.version]\nstatus="historical"' > historical.toml
hugo --config config.toml,historical.toml --baseURL "/${GITHUB_REF/refs\/tags\//}" -d "spec"
- name: "📥 Spec definition download"
uses: actions/download-artifact@v2
with:
name: openapi-artifact
- name: "📝 Unpack the OpenAPI definitions in the right location"
run: |
tar -xzf openapi.tar.gz
- name: "📦 Tarball creation"
run: tar -czf spec-historical.tar.gz spec
- name: "📤 Artifact upload"
uses: actions/upload-artifact@v2
with:
name: spec-historical-artifact
path: spec-historical.tar.gz

70
.github/workflows/netlify.yaml vendored
Unescape Escape View File

@ -1,70 +0,0 @@
# GHA workflow which publishes previews of spec PRs to netlify.
#
# We keep this in a separate workflow to the main spec build, because it
# requires access to the Netlify secret. By having it run on `workflow_run`, we
# will only use the workflow definition file on the default branch, so we can
# ensure that the secret can't be exfiltrated.
#
name: Upload Preview Build to Netlify
on:
workflow_run:
workflows: [Spec]
types: [completed]
jobs:
build:
runs-on: ubuntu-latest
if: github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'pull_request'
steps:
- name: "dump context data"
run: |
jq . < $GITHUB_EVENT_PATH
- name: "🔍 Read PR number"
id: readctx
# we need to find the PR number that corresponds to the branch, which we do by
# searching the GH API
run: |
head_branch='${{github.event.workflow_run.head_repository.owner.login}}:${{github.event.workflow_run.head_branch}}'
echo "head branch: $head_branch"
pulls_uri="https://api.github.com/repos/${{ github.repository }}/pulls?head=$(jq -Rr '@uri' <<<$head_branch)"
pr_number=$(curl -H 'Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' "$pulls_uri" |
jq -r '.[] | .number')
echo "PR number: $pr_number"
echo "::set-output name=prnumber::$pr_number"
- name: '📥 Download artifact'
uses: dawidd6/action-download-artifact@af92a8455a59214b7b932932f2662fdefbd78126 # v2.15.0
with:
workflow: main.yaml
run_id: ${{ github.event.workflow_run.id }}
name: spec-artifact
- name: "📦 Extract Artifacts"
run: tar -xzvf spec.tar.gz && rm spec.tar.gz
- name: "📤 Deploy to Netlify"
id: netlify
# v1.2.2
uses: nwtgck/actions-netlify@f517512ae75beec8896aa7b027c1c72f01816200
with:
publish-dir: spec
deploy-message: "Deploy from GitHub Actions"
enable-pull-request-comment: false
enable-commit-comment: false
alias: pr${{ steps.readctx.outputs.prnumber }}
env:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
timeout-minutes: 1
- name: "📝 Edit PR Description"
# v1.0.1
uses: Beakyn/gha-comment-pull-request@2167a7aee24f9e61ce76a23039f322e49a990409
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
pull-request-number: ${{ steps.readctx.outputs.prnumber }}
description-message: |
Preview: ${{ steps.netlify.outputs.deploy-url }}

12
.github/workflows/spell-check.yaml vendored
Unescape Escape View File

@ -1,15 +1,19 @@
name: Spell Check
on: [pull_request]
on:
push:
branches:
- main
pull_request:
jobs:
run:
name: Spell Check with Typos
runs-on: ubuntu-latest
steps:
- name: Checkout Actions Repository
- name: Check out repository
uses: actions/checkout@v2
- name: Check spelling of proposals
uses: crate-ci/typos@master
uses: crate-ci/typos@9be36f97fdbe645ee9a12449fb13aca856c2516a
with:
config: ${{github.workspace}}/.github/_typos.toml
config: ${{github.workspace}}/.github/_typos.toml

16
.gitignore vendored
Unescape Escape View File

@ -1,16 +0,0 @@
node_modules
/data/msc
/env*
/resources
/scripts/swagger
/scripts/tmp
/hugo-config.toml
/public
*.pyc
*.swp
_rendered.rst
/.vscode/
/.idea/
/spec/
changelogs/rendered.*
.hugo_build.lock

4
.gitmodules vendored
Unescape Escape View File

@ -1,4 +0,0 @@
[submodule "themes/docsy"]
path = themes/docsy
url = https://github.com/matrix-org/docsy.git
branch = master

87
CONTRIBUTING.md
Unescape Escape View File

@ -0,0 +1,87 @@
# Contributing to `matrix-spec-proposals`
Thank you for taking the time to contribute to Matrix!
This repository is for proposals for changes to the Matrix protocol. The process
for submitting a proposal is described in
[this repository's README](README.md#the-matrix-spec-process) or in further detail at
https://spec.matrix.org/proposals/#process.
## Sign off
We ask that everybody who contributes to this project signs off their
contributions, as explained below.
We follow a simple 'inbound=outbound' model for contributions: the act of
submitting an 'inbound' contribution means that the contributor agrees to
license their contribution under the same terms as the project's overall
'outbound' license - in our case, this is Apache Software License v2 (see
[LICENSE](./LICENSE)).
In order to have a concrete record that your contribution is intentional and
you agree to license it under the same terms as the project's license, we've
adopted the same lightweight approach used by the [Linux
Kernel](https://www.kernel.org/doc/html/latest/process/submitting-patches.html),
[Docker](https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and
many other projects: the [Developer Certificate of
Origin](http://developercertificate.org/) (DCO). This is a simple declaration
that you wrote the contribution or otherwise have the right to contribute it to
Matrix:
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
If you agree to this for your contribution, then all that's needed is to
include the line in your commit or pull request comment:
Signed-off-by: Your Name <your@email.example.org>
...using your real name; unfortunately pseudonyms and anonymous contributions
can't be accepted. Git makes this trivial - just use the -s flag when you do
``git commit``, having first set ``user.name`` and ``user.email`` git configs
(which you should have done anyway :)
### Private sign off
If you would like to provide your legal name privately to the Matrix.org
Foundation (instead of in a public commit or comment), you can do so by emailing
your legal name and a link to the pull request to dco@matrix.org. It helps to
include "sign off" or similar in the subject line. You will then be instructed
further.
Once private sign off is complete, doing so for future contributions will not be required.
[1]: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request

175
CONTRIBUTING.rst
Unescape Escape View File

@ -1,175 +0,0 @@
Contributing to matrix-doc
==========================
Everyone is welcome to contribute to the Matrix specification!
Please ensure that you sign off your contributions. See `Sign off`_ below.
Code style
----------
The documentation style is described at
https://github.com/matrix-org/matrix-doc/blob/master/meta/documentation_style.rst.
Python code within the ``matrix-doc`` project should follow the same style as
synapse, which is documented at
https://github.com/matrix-org/synapse/tree/master/docs/code_style.md.
Matrix-doc workflows
--------------------
Specification changes
~~~~~~~~~~~~~~~~~~~~~
The Matrix specification documents the APIs which Matrix clients and servers use.
For this to be effective, the APIs need to be present and working correctly in a
server before they can be documented in the specification. This process can take
some time to complete.
Changes to the protocol (new endpoints, ideas, etc) need to go through the
`proposals process <https://matrix.org/docs/spec/proposals>`_. Other changes,
such as fixing bugs, typos, or clarifying existing behaviour do not need a proposal.
If you're not sure, visit us at `#matrix-spec:matrix.org`_ and ask.
Other changes
~~~~~~~~~~~~~
The above process is unnecessary for smaller changes, and those which do not
put new requirements on servers. This category of changes includes the
following:
* Changes to the scripts used to generate the specification.
* Addition of features which have been in use in practice for some time, but
have never made it into the spec (including anything with the `spec-omission
<https://github.com/matrix-org/matrix-doc/labels/spec-omission>`_ label).
* Likewise, corrections to the specification, to fix situations where, in
practice, servers and clients behave differently to the specification,
including anything with the `spec-bug
<https://github.com/matrix-org/matrix-doc/labels/spec-bug>`_ label.
(If there is any doubt about whether it is the spec or the implementations
that need fixing, please discuss it with us first in `#matrix-spec:matrix.org`_.)
* Clarifications to the specification which do not change the behaviour of
Matrix servers or clients in a way which might introduce compatibility
problems for existing deployments. This includes anything with the
`clarification <https://github.com/matrix-org/matrix-doc/labels/clarification>`_
label.
For example, areas where the specification is unclear do not require a proposal
to fix. On the other hand, introducing new behaviour is best represented by a
proposal.
* Design or aesthetic changes, such as improving accessibility, colour schemes,
etc. Please check in with us at `#matrix-docs:matrix.org`_ with your proposed
design change before opening a PR so we can work with you on it.
For such changes, please do just open a `pull request`_. If you're not sure if
your change is covered by the above, please visit `#matrix-spec:matrix.org` and
ask.
.. _`pull request`: https://help.github.com/articles/about-pull-requests
.. _`#matrix-spec:matrix.org`: https://matrix.to/#/#matrix-spec:matrix.org
.. _`#matrix-docs:matrix.org`: https://matrix.to/#/#matrix-docs:matrix.org
Adding to the changelog
~~~~~~~~~~~~~~~~~~~~~~~
All API specifications require a changelog entry. Adding to the changelog can only
be done after you've opened your pull request, so be sure to do that first.
The changelog is managed by Towncrier (https://github.com/hawkowl/towncrier) in the
form of "news fragments". The news fragments for the client-server API are stored
under ``changelogs/client_server/newsfragments``.
To create a changelog entry, create a file named in the format ``prNumber.type`` in
the ``newsfragments`` directory. The ``type`` can be one of the following:
* ``new`` - Used when adding new endpoints. Please have the file contents be the
method and route being added, surrounded in markdown code tags. For example: \`POST
/accounts/whoami\`.
* ``feature`` - Used when adding backwards-compatible changes to the API.
* ``clarification`` - Used when an area of the spec is being improved upon and does
not change or introduce any functionality.
* ``breaking`` - Used when the change is not backwards compatible.
* ``deprecation`` - Used when deprecating something.
All news fragments must have a brief summary explaining the change in the
contents of the file. The summary must end in a full stop to be in line with
the style guide and formatting must be done using Markdown.
Changes that do not change the spec, such as changes to the build script, formatting,
CSS, etc should not get a news fragment.
Sign off
--------
We ask that everybody who contributes to their project signs off their
contributions, as explained below.
We follow a simple 'inbound=outbound' model for contributions: the act of
submitting an 'inbound' contribution means that the contributor agrees to
license their contribution under the same terms as the project's overall 'outbound'
license - in our case, this is Apache Software License v2 (see LICENSE).
In order to have a concrete record that your contribution is intentional
and you agree to license it under the same terms as the project's license, we've adopted the
same lightweight approach that the Linux Kernel
(https://www.kernel.org/doc/Documentation/SubmittingPatches), Docker
(https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
projects use: the DCO (Developer Certificate of Origin:
http://developercertificate.org/). This is a simple declaration that you wrote
the contribution or otherwise have the right to contribute it to Matrix::
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
If you agree to this for your contribution, then all that's needed is to
include the line in your commit or pull request comment::
Signed-off-by: Your Name <your@email.example.org>
...using your real name; unfortunately pseudonyms and anonymous contributions
can't be accepted. Git makes this trivial - just use the -s flag when you do
``git commit``, having first set ``user.name`` and ``user.email`` git configs
(which you should have done anyway :)

342
README.md
Unescape Escape View File

@ -1,104 +1,292 @@
# Matrix Specification
# Matrix Specification Proposals
This repository contains the Matrix Specification, rendered at [spec.matrix.org](http://spec.matrix.org/).
This repository contains proposals for changes to the [Matrix
Protocol](http://spec.matrix.org), aka "Matrix Spec Changes" (MSCs). The
[`proposals`](./proposals) directory contains MSCs which have been accepted.
Developers looking to use Matrix should join [#matrix-dev:matrix.org](https://matrix.to/#/#matrix-dev:matrix.org)
on Matrix for help.
See below for instructions for creating new
proposals. See also https://spec.matrix.org/proposals/ for more
information on the MSC process, in particular
https://spec.matrix.org/proposals/#process.
Spec authors and proposal writers are welcome to join [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org).
We welcome contributions! See [CONTRIBUTING.rst](./CONTRIBUTING.rst) for details.
The source of the Matrix specification itself is maintained at
https://github.com/matrix-org/matrix-spec.
## Structure
## The Matrix Spec Process
The Matrix spec is compiled with [Hugo](https://gohugo.io/) (a static site generator) with the following structure:
An MSC is meant to be a **technical document that unambiguously describes a
change to the Matrix Spec**, while also justifying _why_ the change should be
made.
* `/assets`: assets that need postprocessing using [Hugo Pipes](https://gohugo.io/hugo-pipes/introduction/).
For example, Sass files would go here.
The document is used both to judge whether the change should be made as
described *and* by developers to actually implement the changes. This is why
it's important for an MSC to be fully fleshed out in technical detail, as once
merged it's immediately part of the formal spec (even though it still needs to
be transcribed into the actual spec itself).
* `/content`: files that will become pages in the site go here. Typically these are Markdown files with some YAML front
matter indicating, [among other things](https://gohugo.io/content-management/front-matter/), what layout should be
applied to this page. The organization of files under `/content` determines the organization of pages in the built
site.
### What changes need to follow this process?
* `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as
[data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and
parse them. This is also where our Swagger/OpenAPI definitions and schemas are.
In most cases a change to [the Matrix protocol](https://spec.matrix.org) will
require an MSC. Changes that would not require an MSC are typically small and
uncontentious, or are simply clarifications to the spec. Fixing typos in the
spec do not require an MSC. In most cases, removing ambiguities do not either.
The exception may be if implementations in the ecosystem have differing views
on clarifying the ambiguity. In that case, an MSC is typically the best place
to reach consensus.
* `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of
a page: for example, whether it has header, footer, sidebar, and so on.
* `/layouts/partials`: these templates can be called from other templates, so they can be used to factor out
template code that's used in more than one template. An obvious example here is something like a sidebar, where
several different page layouts might all include the sidebar. But also, partial templates can return values: this
means they can be used like functions, that can be called by multiple templates to do some common processing.
* `/layouts/shortcodes`: these templates can be called directly from files in `/content`.
Ultimately, the [Spec Core Team](https://matrix.org/foundation) have the final
say on this, but generally if the change would require updates to a
non-insignificant portion of the Matrix implementation ecosystem or would be
met with contention, an MSC is the best route to take. You can also ask in the
[Matrix Spec](https://matrix.to/#/#matrix-spec:matrix.org) or [Office of the
Spec Core Team](https://matrix.to/#/#sct-office:matrix.org) Matrix rooms for
clarification.
* `/static`: static files which don't need preprocessing. JS or CSS files could live here.
### Summary of the process
* `/themes`: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are
supplied in a `/themes/$theme_name/layouts` directory. You can use a theme but customise it by providing your own
versions of any of the theme layouts in the base `/layouts` directory. That is, if a theme provides
`/themes/$theme_name/layouts/sidebar.html` and you provide `/layouts/sidebar.html`, then your version of the
template will be used.
The MSC process consists of three basic steps:
It also has the following top-level file:
1. **Write up the proposal** in a
[markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#GitHub-flavored-markdown)
document. (There's a [proposal
template](proposals/0000-proposal-template.md), but don't feel bound by it.)
2. **Submit it as a Pull Request** to this repo, marking it as a draft until
it's ready for wider review.
3. **Seek review** from the community. Once people are generally happy with it,
ask the [Spec Core Team](https://matrix.org/foundation) to look at it in
[the Office of the SCT Matrix
room](https://matrix.to/#/#sct-office:matrix.org). When the SCT are happy
with the proposal, and after a successful voting process, your pull request
is merged and the **MSC is now officially accepted** as part of the Matrix
Spec and can be used 🎉
* `config.toml`: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings
defined here are available in templates. All these directories above are configurable via `config.toml` settings.
For simple changes this is really all you need to know. For larger or more
controversial changes, getting an MSC merged can take more time and effort, but
the overall process remains the same.
Additionally, the following directories may be of interest:
Below is various guidance to try and help make the experience smoother.
* `/attic`: Here contains historical sections of specification and legacy drafts for the specification.
* `/changelogs`: Various bits of changelog for the specification areas.
* `/data-definitions`: Bits of structured data consumable by Matrix implementations.
* `/meta`: Documentation relating to the spec's processes that are otherwise untracked (release instructions, etc).
* `/scripts`: Various scripts for generating the spec and validating its contents.
* `/proposals`: Matrix Spec Change (MSC) proposals. See <https://spec.matrix.org/unstable/proposals/>.
### Guidance on the process
## Authoring changes to the spec
#### 1. Writing the proposal
Please read [CONTRIBUTING.rst](./CONTRIBUTING.rst) before authoring a change to the spec. Note that spec authoring takes
place after an MSC has been accepted, not as part of a proposal itself.
Come up with an idea. The idea can be for anything, but the solution (MSC)
needs to benefit the Matrix ecosystem rather than yourself (or your company)
specifically. Sometimes this means that the solution needs to be more generic
than the specific itch that you are trying to scratch.
1. Install the extended version (often the OS default) of Hugo:
<https://gohugo.io/getting-started/installing>. Note that at least Hugo
v0.74 is required.
Remember that an MSC is a formal technical document which will be used by
others in the wider community to judge if the proposal should be accepted *and*
to actually implement the changes in clients and servers. This means that for
an MSC to be accepted it should include justifications and describe the
technical changes unambiguously, including specifying what happens in any and
all edge cases.
Alternatively, use the Docker image at
https://hub.docker.com/r/klakegg/hugo/. (The "extended edition" is required
to process the SCSS.)
2. Run `git submodule update --init --recursive` for good measure.
3. Run `npm i` to install the dependencies. Note that this will require NodeJS to be installed.
4. Run `npm run get-proposals` to seed proposal data. This is merely for populating the content of the "Spec Change Proposals"
page and is not required.
5. Run `hugo serve` (or `docker run --rm -it -v $(pwd):/src -p 1313:1313
klakegg/hugo:ext serve`) to run a local webserver which builds whenever a file
change is detected. If watching doesn't appear to be working for you, try
adding `--disableFastRender` to the commandline.
6. Edit the specification 🙂
There's a [proposal template](proposals/0000-proposal-template.md) under
`docs/0000-proposal.md`, but you don't necessarily need to use it. Covering the
same major points is fine.
* Note: At this stage, you won't have an MSC number, so feel free to use
`0000`, `XXXX`, or whatever other placeholder you feel comfortable with.
We use a highly customized [Docsy](https://www.docsy.dev/) theme for our generated site, which uses Bootstrap and Font
Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in
[#matrix-docs:matrix.org](https://matrix.to/#/#matrix-docs:matrix.org) before opening a PR.
Some tips for MSC writing:
## Building the specification
* Please wrap your lines to 120 characters maximum.
This allows readers to review your markdown without needing to horizontally
scroll back and forth. Many markdown text editors have this a feature.
* If you are referencing an existing endpoint in the spec, or another MSC, it
is very helpful to add a link to them so the reader does not need to search
themselves. Examples:
* "This MSC proposals an alternative to
[MSC3030](https://github.com/matrix-org/matrix-spec-proposals/pull/3030)."
* "A new field will be added to the response body of
[`/_matrix/client/v3/sync`](https://spec.matrix.org/v1.3/client-server-api/#get_matrixclientv3sync)".
* Note: it is best to link to the latest stable version of the spec
(e.g. /v1.3, not /latest) - failing that,
[/unstable](https://spec.matrix.org/unstable/) if the change is not
yet in a released spec version.
* GitHub supports rendering fancy diagrams from text with very little
effort using [Mermaid](https://mermaid-js.github.io/mermaid/#/). See [this
guide](https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/)
for more information.
If for some reason you're not a CI/CD system and want to render a static version of the spec for yourself, follow the above
steps for authoring changes to the specification and instead of `hugo serve` run `hugo -d "spec"` - this will generate the
spec to `/spec`. If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"`
to the `hugo -d "spec"` command.
#### 2. Submitting a Pull Request
For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt`
and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`. To make use of the generated file,
there are a number of options:
1. Open a [Pull
Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request)
to add your proposal document to the [`proposals`](proposals) directory.
Note that this will require a GitHub account.
* [Mark your Pull Request as a
draft](https://github.blog/2019-02-14-introducing-draft-pull-requests/)
for now.
2. The MSC number is the number of the pull request that is automatically
assigned by GitHub. Go back through and edit the document accordingly. Don't
forget the file name itself!
3. Edit the pull request title to fit the format "MSC1234: Your proposal
title".
4. Once your proposal is correctly formatted and ready for review from the
wider ecosystem, [take your Pull Request out of draft
status](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review).
* It can be uploaded from your filesystem to an online editor/viewer such as [on the swagger website](http://editor.swagger.io/).
* You can run a local HTTP server by running `./scripts/swagger-http-server.py`, and then view the documentation via an
online viewer; for example, at <http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json>.
* You can host the swagger UI yourself. See <https://github.com/swagger-api/swagger-ui#how-to-run> for advice on how to
do so.
The Spec Core Team will notice this and apply various labels/status tracking to
your MSC, which will announce it to the wider world.
## Issue tracking
#### 3. Seeking review
Specification issues are tracked on github at <https://github.com/matrix-org/matrix-doc/issues>.
Seek review from the Matrix community. Generally this will happen naturally,
but if you feel that your proposal is lacking review then ask for people's
opinion in the [Matrix Spec room on
Matrix](https://matrix.to/#/#matrix-spec:matrix.org).
See [meta/github-labels.rst](./meta/github-labels.rst) for information on what the labels mean.
Reviews can take many forms, and do not need to be done solely by members of
the Spec Core Team. Getting other people who are familiar with the area of
Matrix you are proposing changes to is a great first step; especially those who
may be implementing these changes in clients and/or homeservers.
While the proposal is a work in progress, it's fine for it to be high level
and hand-wavy in places, but remember that before it can be accepted it needs
to be expanded to fully flesh out all the technical detail and edge cases.
At this stage the proposal should also be implemented as a proof of concept
somewhere to show that it _actually_ works in practice. This can be done on any
client or server and doesn't need to be merged or released.
#### 4. Entering Final Comment Period
After the MSC has been implemented, fully fleshed out, and generally feels
ready for final review, you should ask a member of the Spec Core Team to review it in
the public [Spec Core Team Office room on
Matrix](https://matrix.to/#/#sct-office:matrix.org). Someone from the SCT will
then review it, and if all looks well will propose FCP
to start.
At this point, other members of the SCT will look at the proposal and consider
it for inclusion in the spec.
After enough SCT members have approved the proposal, the MSC will enter
something called _Final Comment Period_. This is a 5 calendar day countdown to
give anyone one last chance to raise any blockers or concerns about the
proposed change. Typically MSCs pass this stage without incident, but it
nevertheless serves as a safeguard.
#### 5. The MSC is accepted
Once FCP has ended and the MSC pull request is merged, the proposed change is
considered officially part of the spec. Congratulations!
Clients and servers can now start using the change, even though at this stage
it still needs to be transcribed into the spec document. This happens over in
https://github.com/matrix-org/matrix-spec/ and you are very welcome to do it
yourself! Otherwise it will be handled by a Spec Core Team member. If you would
like help with writing spec PRs, feel free to join and ask questions in the
[Matrix Spec and Docs Authoring Room on Matrix](https://matrix.to/#/#matrix-docs:matrix.org).
### Other useful information
#### Unstable prefixes
*Unstable* prefixes are the namespaces which are used before an MSC has
completed FCP (see above). While the MSC might propose that a `m.space` or
`/_matrix/client/v1/account/whoami` endpoint should exist, the implementation
cannot use a *stable* identifier such as `/v1/` or `m.space` prior to the MSC
being accepted: it needs unstable prefixes.
Typically for MSCs, one will use `org.matrix.msc0000` (using the real MSC
number once known) as a prefix. For the above examples, this would mean
`org.matrix.msc0000.space` and
`/_matrix/client/unstable/org.matrix.msc0000/account/whoami` to allow for
breaking compatibility changes between edits of the MSC itself, or indeed
another competing MSC that's attempting to add the same identifiers.
#### Room versions
To summarize [the spec](https://spec.matrix.org/latest/rooms/) on room
versions: they are how servers agree upon algorithms in a decentralized world
like ours. Examples of changes that require a new room version include anything that changes:
* The format of the core event structure (such as renaming a top-level field,
or modifying [the redaction
algorithm](https://spec.matrix.org/latest/client-server-api/#redactions)),
therefore altering the [reference
hash](https://spec.matrix.org/latest/server-server-api/#calculating-the-reference-hash-for-an-event)
of an event.
* [The authorisation of
events](https://spec.matrix.org/latest/server-server-api/#authorization-rules)
(such as changes to power levels).
Unstable prefixes (see above) for room versions work the same as they do for
other identifiers; your unstable room version may be called
"org.matrix.msc1234".
In order for the changes to end up in a "real" room version (the ones listed in
the spec), it will need a second MSC which aggregates a bunch of functionality
from various MSCs into a single room version. Typically these sorts of curating
MSCs are written by the Spec Core Team given the complexity in wording, but
you're more than welcome to bring an MSC forward which makes the version real.
For an example of what introducing a new room version-required feature can look
like, see [MSC3667](https://github.com/matrix-org/matrix-doc/pull/3667). For an
example of what making a new "real" room version looks like, see
[MSC3604](https://github.com/matrix-org/matrix-doc/pull/3604).
#### Ownership of MSCs and closing them
If an author decides that they would no longer like to pursue their MSC, they
can either pass ownership of it off to someone else, or close it themselves.
* The author of an MSC can close their MSC at any time before FCP by simply
closing the pull request.
* To appoint another user as an author of the MSC (either to replace the author
entirely or to provide additional help), make a note in the MSC's PR
description by writing the following on its own line:
```
Author: @username
```
where `@username` is a valid GitHub username. Multiple such lines can be
added.
Finally, [give that user access to write to your fork of
matrix-spec-proposals on
GitHub](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-access-to-your-personal-repositories/inviting-collaborators-to-a-personal-repository),
which your PR originates from. This will allow them to change the text of
your MSC.
Similar to accepting an MSC, the Spec Core Team may propose a Final Comment
Period with a disposition of "close". This can happen if the MSC appears
abandoned by its author, or the idea is widely rejected by the community. A
vote and final comment period will still be required for the motion to pass.
Additionally, FCP can be also proposed with a disposition of "postpone". This
may be done for MSCs for which the proposed changes do not make sense for the
current state of the ecosystem, but may make sense further down the road.
## Asking for help
The Matrix community and members of the Spec Core Team are here to help guide
you through the process!
If you'd just like to get initial feedback about an idea that's not fully
fleshed out yet, creating an issue at
https://github.com/matrix-org/matrix-spec/issues is a great place to start. Be
sure to search for any existing issues first to see if someone has already had
the same idea!
A few official rooms exist on Matrix where your questions can be answered, or
feedback on your proposal can be requested:
* [#matrix-spec:matrix.org](https://matrix.to/#/#matrix-spec:matrix.org) -
General chat for MSCs, the spec, and pretty much anything in that sphere.
* [#sct-office:matrix.org](https://matrix.to/#/#sct-office:matrix.org) - Where
the Spec Core Team hangs out and is available. This room is intended to have
extremely high signal and low noise, primarily to ensure that MSCs are not
falling through the cracks. If an MSC requires attention or comment from Spec
Core Team members, bring it up here.
* [#matrix-spec-process:matrix.org](https://matrix.to/#/#matrix-spec-process:matrix.org) - A
room dedicated to [the spec process
itself](https://spec.matrix.org/proposals/#process). If you have any
questions about or suggestions to improve the Matrix Spec process, ask them
here.
* [#matrix-docs:matrix.org](https://matrix.to/#/#matrix-docs:matrix.org) - A
quieter room for discussion of the [formal spec
text](https://spec.matrix.org) and [matrix.org](https://matrix.org) website.

5
assets/icons/logo.svg
Unescape Escape View File

@ -1,5 +0,0 @@
<svg width="75" height="32" xmlns="http://www.w3.org/2000/svg">
<g fill="#000" fill-rule="nonzero">
<path d="M.936.732V31.25H3.13v.732H.095V0h3.034v.732zM9.386 10.407v1.544h.044a4.461 4.461 0 0 1 1.487-1.368c.58-.323 1.245-.485 1.993-.485.72 0 1.377.14 1.972.42.595.279 1.047.771 1.355 1.477.338-.5.796-.941 1.377-1.323.58-.383 1.266-.574 2.06-.574.602 0 1.16.074 1.674.22.514.148.954.383 1.322.707.366.323.653.746.859 1.268.205.522.308 1.15.308 1.887v7.633H20.71v-6.464c0-.383-.015-.743-.044-1.082a2.305 2.305 0 0 0-.242-.882 1.473 1.473 0 0 0-.584-.596c-.257-.146-.606-.22-1.047-.22-.44 0-.796.085-1.068.253-.272.17-.485.39-.639.662a2.654 2.654 0 0 0-.308.927 7.074 7.074 0 0 0-.078 1.048v6.354h-3.128v-6.398c0-.338-.007-.673-.021-1.004a2.825 2.825 0 0 0-.188-.916 1.411 1.411 0 0 0-.55-.673c-.258-.168-.636-.253-1.135-.253a2.33 2.33 0 0 0-.584.1 1.94 1.94 0 0 0-.705.374c-.228.184-.422.449-.584.794-.161.346-.242.798-.242 1.357v6.619H6.434V10.407h2.952zM25.842 12.084a3.751 3.751 0 0 1 1.233-1.17 5.37 5.37 0 0 1 1.685-.629 9.579 9.579 0 0 1 1.884-.187c.573 0 1.153.04 1.74.121.588.081 1.124.24 1.609.475.484.235.88.562 1.19.981.308.42.462.975.462 1.666v5.934c0 .516.03 1.008.088 1.478.058.471.161.824.308 1.06H32.87a4.435 4.435 0 0 1-.22-1.104c-.5.515-1.087.876-1.762 1.081a7.084 7.084 0 0 1-2.071.31c-.544 0-1.05-.067-1.52-.2a3.472 3.472 0 0 1-1.234-.617 2.87 2.87 0 0 1-.826-1.059c-.199-.426-.298-.934-.298-1.522 0-.647.114-1.18.342-1.6.227-.419.52-.753.881-1.004.36-.25.771-.437 1.234-.562.462-.125.929-.224 1.399-.298.47-.073.932-.132 1.387-.176.456-.044.86-.11 1.212-.199.353-.088.631-.217.837-.386.206-.169.301-.415.287-.74 0-.337-.055-.606-.166-.804a1.217 1.217 0 0 0-.44-.464 1.737 1.737 0 0 0-.639-.22 5.292 5.292 0 0 0-.782-.055c-.617 0-1.101.132-1.454.397-.352.264-.558.706-.617 1.323h-3.128c.044-.735.227-1.345.55-1.83zm6.179 4.423a5.095 5.095 0 0 1-.639.165 9.68 9.68 0 0 1-.716.11c-.25.03-.5.067-.749.11a5.616 5.616 0 0 0-.694.177 2.057 2.057 0 0 0-.594.298c-.17.125-.305.284-.408.474-.103.192-.154.434-.154.728 0 .28.051.515.154.706.103.192.242.342.419.453.176.11.381.187.617.231.234.044.477.066.726.066.617 0 1.094-.102 1.432-.309.338-.205.587-.452.75-.739.16-.286.26-.576.297-.87.036-.295.055-.53.055-.707v-1.17a1.4 1.4 0 0 1-.496.277zM43.884 10.407v2.096h-2.291v5.647c0 .53.088.883.264 1.059.176.177.529.265 1.057.265.177 0 .345-.007.507-.022.161-.015.316-.037.463-.066v2.426a7.49 7.49 0 0 1-.882.089 21.67 21.67 0 0 1-.947.022c-.484 0-.944-.034-1.377-.1a3.233 3.233 0 0 1-1.145-.386 2.04 2.04 0 0 1-.782-.816c-.191-.353-.287-.816-.287-1.39v-6.728H36.57v-2.096h1.894v-3.42h3.129v3.42h2.29zM48.355 10.407v2.118h.044a3.907 3.907 0 0 1 1.454-1.754 4.213 4.213 0 0 1 1.036-.497 3.734 3.734 0 0 1 1.145-.176c.206 0 .433.037.683.11v2.912a5.862 5.862 0 0 0-.528-.077 5.566 5.566 0 0 0-.595-.033c-.573 0-1.058.096-1.454.287a2.52 2.52 0 0 0-.958.783 3.143 3.143 0 0 0-.518 1.158 6.32 6.32 0 0 0-.154 1.434v5.14h-3.128V10.407h2.973zM54.039 8.642V6.06h3.128v2.582H54.04zm3.128 1.765v11.405H54.04V10.407h3.128zM58.797 10.407h3.569l2.005 2.978 1.982-2.978h3.459l-3.745 5.339 4.208 6.067h-3.57l-2.378-3.596-2.38 3.596h-3.502l4.097-6.001zM74.094 31.25V.732H71.9V0h3.035v31.982H71.9v-.732z"/>
</g>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 3.1 KiB

41
assets/scss/_variables_project.scss
Unescape Escape View File

@ -1,41 +0,0 @@
/*
Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
$primary: #FFF;
$secondary: #0098D4;
$dark: #333;
$gray-100: #FBFBFB;
$secondary-background: #E5F5FB;
$secondary-lighter-background: #F4FaFC;
$secondary-lightest-background: #FBFDFD;
$warning: #FF6666;
$note: $secondary;
$note-background: $secondary-background;
$warning-background: #FFE0E0;
$table-row-alternate: $secondary-lightest-background;
$table-row-default: $secondary-lighter-background;
/*
Opt to serve fonts locally by overriding web-font-path to be a non-google fonts URL.
This is only possible with our modified docsy theme: https://github.com/matrix-org/docsy
*/
$web-font-path: "../css/fonts/Inter.css";
$google_font_name: "Inter";

428
assets/scss/custom.scss
Unescape Escape View File

@ -1,428 +0,0 @@
/*
Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
/*
Custom SCSS for the Matrix spec
*/
@import "variables_project";
@import "variables";
/* Overrides for the navbar */
.td-navbar {
box-shadow: 0px 0px 8px rgba(179, 179, 179, 0.25);
min-height: 5rem;
.navbar-brand {
font-size: 1.1rem;
.navbar-version {
color: $secondary;
}
}
a {
color: $black;
}
}
/* Styles for the sidebar nav */
.td-sidebar-nav {
scroll-behavior: smooth;
overscroll-behavior: contain;
&>.td-sidebar-nav__section {
margin-top: 1rem;
}
&>.td-sidebar-nav__section > li > a.td-sidebar-link {
font-weight: $font-weight-bold;
font-size: 1.3rem;
}
/* This is to make the width of the items that have sub-items (like room versions)
the same as the width of items that don't (like changelog) */
.pr-md-3, .px-md-3 {
padding-right: 0 !important;
}
a.indent-1 {
padding-left: 1rem !important;
}
a.indent-2 {
padding-left: 2rem;
}
a, a.td-sidebar-link {
color: $gray-800;
font-weight: $font-weight-normal;
padding-top: .2rem;
padding-bottom: .2rem;
display: block;
transition: all 100ms ease-in-out;
&:hover {
background-color: $secondary-lighter-background;
color: $gray-800;
}
&.active, &active:hover {
background-color: $secondary-background;
font-weight: $font-weight-normal;
}
}
}
@media (min-width: 768px) {
@supports (position: sticky) {
.td-sidebar-nav {
/* This overrides calc(100vh - 10rem);, which gives us a blank space at the bottom of the sidebar */
max-height: calc(100vh - 6rem);
}
}
}
/* Customise footer */
footer {
box-shadow: 0px 0px 8px rgba(179, 179, 179, 0.25);
}
/* Auto numbering for headings */
.td-content {
counter-reset: h2;
&> h2 {
counter-reset: h3
}
&> h3 {
counter-reset: h4
}
&> h4 {
counter-reset: h5
}
&> h5 {
counter-reset: h6
}
&> h2:not(.no-numbers):before {
display: inline; visibility: visible; counter-increment: h2; content: counter(h2) ". "
}
&> h3:not(.no-numbers):before {
display: inline; visibility: visible; counter-increment: h3; content: counter(h2) "." counter(h3) ". "
}
&> h4:not(.no-numbers):before {
display: inline; visibility: visible; counter-increment: h4; content: counter(h2) "." counter(h3) "." counter(h4) ". "
}
&> h5:not(.no-numbers):before {
display: inline; visibility: visible; counter-increment: h5; content: counter(h2) "." counter(h3) "." counter(h4) "." counter(h5) ". "
}
&> h6:not(.no-numbers):before {
display: inline; visibility: visible; counter-increment: h6; content: counter(h2) "." counter(h3) "." counter(h4) "." counter(h5) "." counter(h6) ". "
}
}
/* Adjust heading anchors for site header */
.td-content {
&> h2,
&> h3,
&> h4,
&> h5,
&> h6,
.rendered-data h1 {
scroll-margin-top: 5rem;
}
}
/* Styles for the table of contents */
#toc {
padding-top: .5rem;
padding-left: 1.5rem;
ol {
padding-left: 1rem;
counter-reset: section;
list-style-type: none;
}
#TableOfContents {
&>ol>li {
margin-bottom: .5rem;
&>a {
font-weight: $font-weight-bold;
}
}
ol {
padding-left: 0;
}
&>ol>li>a {
padding-left: 1rem;
}
&>ol>li>ol>li>a {
padding-left: 2rem;
}
&>ol>li>ol>li>ol>li>a {
padding-left: 3rem;
}
&>ol>li>ol>li>ol>li>ol>li>a {
padding-left: 4rem;
}
&>ol>li>ol>li>ol>li>ol>li>ol>li>a {
padding-left: 5rem;
}
}
li a:before {
counter-increment: section;
content: counters(section, ".") " ";
}
#toc-title {
font-weight: $font-weight-bold;
font-size: 1.3rem;
}
}
/* Styles for alert boxes */
.alert {
&.note {
&:not(.omit-title):before {
content: "INFO: ";
font-weight: $font-weight-bold;
}
border: 2px solid $note;
border-left-width: 5px;
background: $note-background;
}
&.rationale {
&:not(.omit-title):before {
content: "RATIONALE: ";
font-weight: $font-weight-bold;
}
border: 2px solid $note;
border-left-width: 5px;
background: $note-background;
}
&.warning {
&:not(.omit-title):before {
content: "WARNING: ";
font-weight: $font-weight-bold;
}
border: 2px solid $warning;
border-left-width: 5px;
background: $warning-background;
}
}
/* Styles for sections that are rendered from data, such as HTTP APIs and event schemas */
.rendered-data {
margin: 1rem 0 3rem 0;
details {
summary {
padding: .5rem 0;
list-style-position: outside;
}
}
.deprecated-inline {
&:after {
content: " — DEPRECATED";
color: $warning;
font-weight: $font-weight-bold;
}
}
h1 {
display: inline-block;
font-size: 1.3rem;
.endpoint {
color: $secondary;
}
}
h2 {
font-weight: $font-weight-bold;
font-size: 1.3rem;
margin: 3rem 0 .5rem 0;
}
h3 {
font-weight: $font-weight-bold;
font-size: 1.1rem;
margin: 1.5rem 0 .75rem 0;
}
h2 + table, h3 + table, h3 + div.highlight {
margin-top: 0;
}
hr {
border-bottom: 2px solid $dark;
margin-bottom: 1.5rem;
}
p {
max-width: 80%;
}
p code, table code {
background-color: $white;
}
table {
table-layout: fixed;
width: 100%;
margin: 4rem 0;
caption {
caption-side: top;
color: $dark;
font-size: 1rem;
font-weight: $font-weight-bold;
}
th, td, caption {
padding: 1rem;
}
th {
background-color: $white;
}
caption, tr {
background-color: $table-row-default;
}
tr:nth-child(even) {
background-color: $table-row-alternate;
}
&.basic-info, &.basic-info th, &.basic-info td {
table-layout: fixed;
margin: 1rem 0 .5rem 0;
background-color: $white;
}
&.basic-info th {
width: 15rem;
}
.col-name, .col-type, .col-status {
width: 25%;
}
.col-description {
width: 50%;
}
.col-status-description {
width: 75%;
}
}
pre {
border: 0;
border-left: solid 5px $secondary;
}
.http-api-method {
font-weight: $font-weight-bold;
}
}
/* Miscellaneous custom bits */
/* Update link colours for MAtrix style */
a, a:hover {
color: $secondary;
}
/* This is needed to stop the bottom of the Matrix icon from getting snipped off. */
.td-navbar .navbar-brand svg {
height: 32px;
}
/* Give code samples and pre elements full-width */
.td-content > .highlight, .td-content > pre {
max-width: 100%;
}
/* The default CSS applies a style for blockquotes but only to immediate children
of .td-content. This applies the same style to any blockquotes that descend from
.td-content. */
.td-content blockquote {
padding: 0 0 0 1rem;
margin-bottom: $spacer;
color: $gray-600;
border-left: 6px solid $secondary;
}
/*
Make padding symmetrical (this selector is used in the default styles to apply padding-left: 3rem)
*/
.pl-md-5, .px-md-5 {
padding-right: 3rem;
}
/* Adjust default styles for info banner */
.pageinfo-primary {
max-width: 80%;
margin-left: 0;
border: 0;
border-left: solid 5px $secondary;
background-color: $gray-100;
}
.pageinfo-unstable {
background-image: url('../icons/unstable.png');
background-position: left 1rem center;
background-repeat: no-repeat;
padding-left: 100px;
}
/* Full-width tables */
.td-content > table {
width: 100%;
display: table;
}

11
attic/drafts/ancient_federated_versioning_design_notes.rst
Unescape Escape View File

@ -1,11 +0,0 @@
Versioning is, like, hard for backfilling backwards because of the number of homeservers involved.
The way we solve this is by doing versioning as an acyclic directed graph of PDUs. For backfilling purposes, this is done on a per context basis.
When we send a PDU we include all PDUs that have been received for that context that hasn't been subsequently listed in a later PDU. The trivial case is a simple list of PDUs, e.g. A <- B <- C. However, if two servers send out a PDU at the same to, both B and C would point at A - a later PDU would then list both B and C.
Problems with opaque version strings:
- How do you do clustering without mandating that a cluster can only have one transaction in flight to a given remote homeserver at a time.
If you have multiple transactions sent at once, then you might drop one transaction, receive another with a version that is later than the dropped transaction and which point ARGH WE LOST A TRANSACTION.
- How do you do backfilling? A version string defines a point in a stream w.r.t. a single homeserver, not a point in the context.
We only need to store the ends of the directed graph, we DO NOT need to do the whole one table of nodes and one of edges.

244
attic/drafts/application_services.rst
Unescape Escape View File

@ -1,244 +0,0 @@
.. TODO
Sometimes application services need to create rooms (e.g. when lazy loading
from room aliases). Created rooms need to have a user that created them, so
federation works (as it relies on an entry existing in m.room.member). We should
be able to add metadata to m.room.member to state that this user is an application
service, a virtual user, etc.
Application Services
====================
Overview
========
Application services provide a way of implementing custom serverside functionality
on top of Matrix without the complexity of implementing the full federation API.
By acting as a trusted service logically located behind an existing homeserver,
Application services are decoupled from:
* Signing or validating federated traffic or conversation history
* Validating authorisation constraints on federated traffic
* Managing routing or retry schemes to the rest of the Matrix federation
As such, developers can focus entirely on implementing application logic rather
than being concerned with the details of managing Matrix federation.
Features available to application services include:
* Privileged subscription to any events available to the homeserver
* Synthesising virtual users
* Synthesising virtual rooms
* Injecting message history for virtual rooms
Features not provided by application services include:
* Intercepting and filtering/modifying message or behaviour within a room
(this is a job for a Policy Server, as it requires a single logical focal
point for messages in order to consistently apply the custom business logic)
Example use cases for application services include:
* Exposing existing communication services in Matrix
* Gateways to/from standards-based protocols (SIP, XMPP, IRC, RCS (MSRP), SIMPLE, Lync, etc)
* Gateways to/from closed services (e.g. WhatsApp)
* Gateways could be architected as:
* Act as a virtual client on the non-Matrix network
(e.g. connect as multiple virtual clients to an IRC or XMPP server)
* Act as a server on the non-Matrix network
(e.g. speak s2s XMPP federation, or IRC link protocol)
* Act as an application service on the non-Matrix network
(e.g. link up as IRC services, or an XMPP component)
* Exposing a non-Matrix client interface listener from the AS
(e.g. listen on port 6667 for IRC clients, or port 5222 for XMPP clients)
* Bridging existing APIs into Matrix
* e.g. SMS/MMS aggregator APIs
* Domain-specific APIs such as SABRE
* Integrating more exotic content into Matrix
* e.g. MIDI<->Matrix gateway/bridge
* 3D world <-> Matrix bridge
* Application services:
* Search engines (e.g. elasticsearch search indices)
* Notification systems (e.g. send custom pushes for various hooks)
* VoIP Conference services
* Text-to-speech and Speech-to-text services
* Signal processing
* IVR
* Server-machine translation
* Censorship service
* Multi-User Gaming (Dungeons etc)
* Other "constrained worlds" (e.g. 3D geometry representations)
* applying physics to a 3D world on the serverside
* (applying gravity and friction and air resistance... collision detection)
* domain-specific merge conflict resolution of events
* Payment style transactional usecases with transactional guarantees
Architecture Outline
====================
The application service registers with its host homeserver to offer its services.
In the registration process, the AS provides:
* Credentials to identify itself as an approved application service for that HS
* Details of the namespaces of users and rooms the AS is acting on behalf of and
"subscribing to"
* Namespaces are defined as a list of regexps against which to match room aliases,
room IDs, and user IDs. Regexps give the flexibility to say, sub-domain MSISDN
ranges per AS, whereas a blunt prefix string does not. These namespaces are further
configured by setting whether they are ``exclusive`` or not. An exclusive namespace
prevents entities other than the aforementioned AS from creating/editing/deleting
entries within that namespace. This does not affect the visibility/readability of
entries within that namespace (e.g. it doesn't prevent users joining exclusive
aliases, or ASes from listening to exclusive aliases, but does prevent both users
and ASes from creating/editing/deleting aliases within that namespace).
* There is overlap between selecting events via the csv2 Filter API and subscribing
to events here - perhaps subscription involves passing a filter token into the
registration API.
* A URL base for receiving requests from the HS (as the AS is a server,
implementers expect to receive data via inbound requests rather than
long-poll outbound requests)
On HS handling events to unknown users:
* If the HS receives an event for an unknown user who is in the namespace delegated to
the AS, then the HS queries the AS for the profile of that user. If the AS
confirms the existence of that user (from its perspective), then the HS
creates an account to represent the virtual user.
* The namespace of virtual user accounts should conform to a structure like
``@.irc.freenode.Arathorn:matrix.org``. This lets Matrix users communicate with
foreign users who are not yet mapped into Matrix via 3PID mappings or through
an existing non-virtual Matrix user by trying to talk to them via a gateway.
* The AS can alternatively preprovision virtual users using the existing CS API
rather than lazy-loading them in this manner.
* The AS may want to link the matrix ID of the sender through to their 3PID in
the remote ecosystem. E.g. a message sent from ``@matthew:matrix.org`` may wish
to originate from Arathorn on irc.freenode.net in the case of an IRC bridge.
It's left as an AS implementation detail as to how the user should authorise
the AS to act on its behalf.
On HS handling events to unknown rooms:
* If the HS receives an invite to an unknown room which is in the namespace
delegated to the AS, then the HS queries the AS for the existence of that room.
If the AS confirms its existence (from its perspective), then the HS creates
the room.
* The initial state of the room may be populated by the AS by querying an
initialSync API (probably a subset of the CS initialSync API, to reuse the
same pattern for the equivalent function). As messages have to be signed
from the point of ``m.room.create``, we will not be able to back-populate
arbitrary history for rooms which are lazy-created in this manner, and instead
have to chose the amount of history to be synchronised into the AS as a one-off.
* If exposing arbitrary history is required, then:
* either: the room history must be preemptively provisioned in the HS by the AS via
the CS API (TODO: meaning the CS API needs to support massaged
timestamps), resulting in conversation history being replicated between
the HS and the source store.
* or: the HS must delegate conversation storage entirely to the
AS using a Storage API (not defined here) which allows the existing
conversation store to back the HS, complete with all necessary Matrix
metadata (e.g. hashes, signatures, federation DAG, etc). This obviously
increases the burden of implementing an AS considerably, but is the only
option if the implementer wants to avoid duplicating conversation history
between the external data source and the HS.
On HS handling events to existing users and rooms:
* If the HS receives an event for a user or room that already exists (either
provisioned by the AS or by normal client interactions), then the message
is handled as normal.
* Events in the namespaces of rooms and users that the AS has subscribed to
are pushed to the AS using the same pattern as the federation API (without
any of the encryption or federation metadata). This serves precisely the
same purpose as the CS event stream and has the same data flow semantics
(and indeed an AS implementer could chose to use the CS event stream instead)
* Events are linearised to avoid the AS having to handle the complexity of
linearisation, and because if linearisation is good enough for CS, it
should be good enough for AS. Should the AS require non-linearised events
from Matrix, it should implement the federation API rather than the AS API
instead.
* HS->AS event pushes are retried for reliability with sequence numbers
(or logical timestamping?) to preserve the linearisation order and ensure
a reliable event stream.
* Clustered HSes must linearise just as they do for the CS API. Clustered
ASes must loadbalance the inbound stream across the cluster as required.
On AS relaying events from unknown-to-HS users:
* AS injects the event to the HS using the CS API, irrespective of whether the
target user or room is known to the HS or not. If the HS doesn't recognise
the target it goes through the same lazy-load provisioning as per above.
* The reason for not using a subset of the federation API here is because it
allows AS developers to reuse existing CS SDKs and benefit from the more
meaningful error handling of the CS API. The sending user ID must be
explicitly specified, as it cannot be inferred from the access_token, which
will be the same for all AS requests.
* TODO: or do we maintain a separate ``access_token`` mapping? It seems like
unnecessary overhead for the AS developer; easier to just use a single
privileged ``access_token`` and just track which ``user_id`` is emitting events?
* If the AS is spoofing the identity of a real (not virtual) matrix user,
we should actually let them log themselves in via OAuth2 to give permission
to the AS to act on their behalf.
* We can't auth gatewayed virtual users from 3rd party systems who are being
relayed into Matrix, as the relaying is happening whether the user likes it
or not. Therefore we do need to be able to spoof sender ID for virtual users.
On AS relaying events in unknown-to-HS rooms:
* See above.
On AS publishing aliases for virtual rooms:
* AS uses the normal alias management API to preemptively create/delete public
directory entries for aliases for virtual rooms provided by the AS.
* In order to create these aliases, the underlying room ID must also exist, so
at least the ``m.room.create`` of that room must also be prepopulated. It seems
sensible to prepopulate the required initial state and history of the room to
avoid a two-phase prepopulation process.
On unregistering the AS from the HS:
* An AS must tell the HS when it is going offline in order to stop receiving
requests from the HS. It does this by hitting an API on the HS.
AS Visibility:
* If an AS needs to sniff events in a room in order to operate on them (e.g.
to act as a search engine) but not inject traffic into the room, it should
do so by subscribing to the relevant events without actually joining the room.
* If the AS needs to participate in the room as a virtual user (e.g. an IVR
service, or a bot, or a gatewayed virtual user), it should join the room
normally.
* There are rare instances where an AS may wish to participate in a room
(including inserting messages), but be hidden from the room list - e.g. a
conferencing server focus bot may wish to join many rooms as the focus and
both listen to VoIP setups and inject its own VoIP answers, without ever
being physically seen in the room. In this scenario, the user should set
its presence to 'invisible', a state that HSes should only allow AS-authed
users to set.
E2E Encryption
* The AS obviously has no visibility to E2E encrypted messages, unless it is
explicitly added to an encrypted room and participates in the group chat
itself.
Extensions to CS API
====================
* Ability to assert the identity of the virtual user for all methods.
* Ability to massage timestamps when prepopulating historical state and
messages of virtual rooms (either by overriding ``origin_server_ts`` (preferred) or
adding an ``as_ts`` which we expect clients to honour)
* Ability to delete aliases (including from the directory) as well as create them.

222
attic/drafts/data_flows.rst
Unescape Escape View File

@ -1,222 +0,0 @@
Data flows for use cases
========================
::
<- Data from server to client
-> Data from client to server
Instant Messaging
-----------------
Without storage
~~~~~~~~~~~~~~~
::
Home screen
Data required on load:
<- For each room the user is joined: Name, topic, # members, last message, room ID, aliases
Data required when new message arrives for a room:
<- Room ID, message content, sender (user ID, display name, avatar url)
Data required when someone invites you to a room:
<- Room ID, sender (user ID, display name, avatar url), Room Name, Room Topic
Data required when you leave a room on another device:
<- Room ID
Data required when you join a room on another device:
<- Name, topic, # members, last message, room ID, aliases
Data required when your profile info changes on another device:
<- new profile info e.g. avatar, display name, etc.
Creating a room
-> Invitee list of user IDs, public/private, name of room, alias of room, topic of room
<- Room ID
Joining a room (and dumped into chat screen on success)
-> Room ID / Room alias
<- Room ID, Room aliases (plural), Name, topic, member list (f.e. member: user ID,
avatar, presence, display name, power level, whether they are typing), enough
messages to fill screen (and whether there are more)
Chat Screen
Data required when member name changes:
<- new name, room ID, user ID, when in the context of the room did this occur
Data required when the room name changes:
<- new name, room ID, old room name?
Invite a user:
-> user ID, room ID
<- display name / avatar of user invited (if known)
Kick a user:
-> user ID, room ID
<- what message it came after
Leave a room:
-> room ID
<- what message it came after
Send a message
-> Message content, room ID, message sequencing (eg sending my 1st, 2nd, 3rd msg)
<- actual content sent (if server mods it), what message it comes after (to correctly
display the local echo)
Place a call (receive a call is just reverse)
<- turn servers
-> SDP offer
-> Ice candidates (1 by 1; trickling)
<- SDP answer
<- Ice candidates
Scrolling back (infinite scrolling)
-> Identifier for the earliest message, # requested messages
<- requested messages (f.e change in display name, what the old name was), whether
there are more.
With storage
~~~~~~~~~~~~
::
Home Screen
On Load
-> Identifier which tells the server the client's current state (which rooms it is aware
of, which messages it has, what display names for users, etc..)
<- A delta from the client's current state to the current state on the server (e.g. the
new rooms, the *latest* message if different, the changed display names, the new
invites, etc). f.e Room: Whether the cache of the room that you have has been replaced
with this new state.
Pre-load optimisation (not essential for this screen)
-> Number of desired messages f.e room to cache
<- f.e Room: the delta OR the entire state
Bug Tracking
------------
::
Landing Page
On Load
<- Issues assigned to me, Issues I'm watching, Recent activity on other issues includes
comments, list of projects
Search for an issue (assume text)
-> Search string
<- List of paginated issues
Request page 2:
-> Page number requested
<- Page of paginated issues
Issue Page
On Load
-> Issue ID and Project ID (equiv to Room)
<- Issue contents e.g. priority, resolution state, etc. All comments e.g. user ID,
comment text, timestamp. Entire issue history e.g. changes in priority
Post a comment
-> Issue ID, comment content, Project ID (equiv to Room)
<- actual content sent (if modded), what comment it comes after
Set issue priority
-> Issue ID, Project ID, desired priority
<- What action in the history it came after
Someone else sets issue priority
<- Issue ID, Project ID, new priority, where in the history
Mapping model use cases to matrix models (Room, Message, etc)
=============================================================
To think about:
- Do we want to support the idea of forking off new rooms from existing ones? This
and forums could benefit from it.
Bug tracking UI
---------------
::
Projects => Rooms
Issues => Message Events
Comments => Message Events (relates_to key)
Projects:
- Unlikely that there will be 100,000s of issues, so having to pull in all the issues for a project is okay.
- Permissions are usually per project and this Just Works.
- New issues come in automatically and Just Work.
- Can have read-only members
Issues:
- Don't really want 1 Room per Issue, else you can have thousands of Rooms PER PROJECT, hence choice for
Issues as Messages. Don't need to join a room for each issue.
- Idea of issue owner is clear (sender of the message)
- Updating issues requires an additional event similar to comments (with ``relates_to``)? Could possibly
be state events? Don't really want all the history if say the priority was changed 1000 times, just want
the current state of the key.
Comments:
- Additional event with ``relates_to`` key.
Forum
-----
::
Forum => Room (with pointers to Board Rooms)
Boards => Room (with pointers to Thread Rooms)
Threads => Room
Messages => Message Events
Forum:
- Contains 10s of Boards.
- Contains special Message Events which point to different rooms f.e Board.
Boards:
- Contains 100s of Threads.
- Contains special Message Events which point to different rooms f.e. Thread.
Threads:
- Contains 100s of Messages.
Can't do this nicely with the current Federation API because you have loads of
Rooms and what does posting a message look like? Creating a thread is done by..?
The user who is posting cannot create the thread because otherwise they would be
the room creator and have ultimate privileges. So it has to be created by a bot
of some kind which ties into auth (Application services?). To follow a board,
you need a bot to join the Board Room and then watch it for changes...
Fundamental problem with forums is that there is only 1 PDU graph per room and
you either have to pull in lots of graphs separately or one graph and filter it
separately to get to the desired sub set of data. You have to subscribe into a
lot of graphs if you subscribe to a board... If you have the entire board...
good luck scrollbacking a particular thread.
Google+ Community
-----------------
::
Community => Room (with pointers to Category Rooms)
Category => Room
Post => Message Events
Comment => Message Events (relates_to key)
Community:
- Contains 10s of categories.
- Contains special Message Events which point to different rooms f.e Category.
- Moderators of the community are mods in this room. They are in charge of making
new categories and the subsequent rooms. Can get a bit funky if a mod creates a
category room without the same permissions as the community room... but another
mod can always delete the pointer to the buggy category room and make a new one.
- Do we want to support the idea of forking off new rooms from existing ones? This
and forums could benefit from it.
Category:
- Contains 1000s of posts.
- Same permissions as the community room. How to enforce? Fork off the community
room?
Posts:
- Contains 10s of comments.
This is similar to forums but you can more reasonably say "screw it, pull in the
entire community of posts."

141
attic/drafts/erik-model.rst
Unescape Escape View File

@ -1,141 +0,0 @@
This is a standalone description of the data architecture of Synapse. There is a
lot of overlap with the current specification, so it has been split out here for
posterity. Hopefully all the important bits have been merged into the relevant
places in the main spec.
Model
-----
Overview
~~~~~~~~
Matrix is used to reliably distribute data between sets of `users`.
Users are associated with one of many matrix `servers`. These distribute,
receive and store data on behalf of its registered users. Servers can be run on
any host accessible from the internet.
When a user wishes to send data to users on different servers the local server
will distribute the data to each remote server. These will in turn distribute
to their local users involved.
A user sends and receives data using one or more authenticated `clients`
connected to his server. Clients may persist data locally or request it when
required from the server.
Events
~~~~~~
An event is a collection of data (the `payload`) and metadata to be distributed
across servers and is the primary data unit in Matrix. Events are extensible
so that clients and servers can add extra arbitrary fields to both the payload
or metadata.
Events are distributed to interested servers upon creation. Historical events
may be requested from servers; servers are not required to produce all
or any events requested.
All events have a metadata `type` field that is used by client and servers to
determine how the payload should be processed and used. There are a number of
types reserved by the protocol for particular uses, but otherwise types may be
defined by applications, clients or servers for their own purposes.
.. TODO : Namespacing of new types.
Graph
+++++
Each event has a list of zero or more `parent` events. These relations form
directed acyclic graphs of events called `event graphs`. Every event graph has
a single root event, and each event graph forms the basis of the history of a
matrix room.
Event graphs give a partial ordering of events, i.e. given two events one may
be considered to have come before the other if one is an ancestor of the other.
Since two events may be on separate branches, not all events can be compared in
this manner.
Every event has a metadata `depth` field that is a positive integer that is
strictly greater than the depths of any of its parents. The root event should
have a depth of 1.
[Note: if one event is before another, then it must have a strictly smaller
depth]
Integrity
+++++++++
.. TODO: Specify the precise subset of essential fields
Portions of events will be signed by one or more servers or clients. The parent
relations, type, depth and payload (as well as other metadata fields that will
be specified) must be signed by the originating server. [Note: Thus, once an
event is distributed and referenced by later events, they effectively become
immutable].
The payload may also be encrypted by clients, except in the case where the
payload needs to be interpreted by the servers. A list of event types that
cannot have an encrypted payload are given later.
State
~~~~~
Event graphs may have meta information associated with them, called `state`.
State can be updated over time by servers or clients, subject to
authorisation.
The state of a graph is split into `sections` that can be atomically updated
independently of each other.
State is stored within the graph itself, and can be computed by looking at the
graph in its entirety. We define the state at a given event to be the state of
the sub graph of all events "before" and including that event.
Some sections of the state may determine behaviour of the protocol, including
authorisation and distribution. These sections must not be encrypted.
State Events
++++++++++++
`State events` are events that update a section of state data for a room. These
state events hold all the same properties of events, and are part of the event
graph. The payload of the event is the replacement value for the particular
section of state being updated.
State events must also include a `state_key` metadata field. The pair of fields
type and state_key uniquely defines the section of state that is to be updated.
State Resolution
++++++++++++++++
A given state section may have multiple state events associated with it in a
given graph. A consistent method of selecting which state event takes
precedence is therefore required.
This is done by taking the latest state events, i.e. the set of events that are
either incomparable or after every other event in the graph. A state resolution
algorithm is then applied to this set to select the single event that takes
precedence.
The state resolution algorithm must be transitive and not depend on server
state, as it must consistently select the same event irrespective of the server
or the order the events were received in.
State Dictionary
++++++++++++++++
The state dictionary is the mapping from sections of state to the state events
which set the section to its current value. The state dictionary, like the
state itself, depends on the events currently in the graph and so is updated
with each new event received.
Since the sections of the state are defined by the pair of strings from the
type and state_key of the events that update them, the state dictionary can be
defined as a mapping from the pair (type, state_key) to a state event with
those values in the graph.
Deleting State
++++++++++++++
State sections may also be deleted, i.e. removed from the state dictionary. The
state events will still be present in the event graph.
This is done by sending a special state event indicating that the given entry
should be removed from the dictionary. These events follow the same rules for
state resolution, with the added requirement that it loses all conflicts.
[Note: This is required to make the algorithm transitive.]

253
attic/drafts/erikj_federation.rst
Unescape Escape View File

@ -1,253 +0,0 @@
Federation
==========
.. sectnum::
.. contents:: Table of Contents
Auth chain
~~~~~~~~~~
The *auth chain* for an event is the recursive list of auth events and the auth
chain for those auth events.
.. Note:: The auth chain for an event gives all the information a server needs
to accept an event. However, being given an auth chain for an event
that appears valid does not mean that the event might not later be
rejected. For example if we discover that the sender had been banned
between the join event listed in the auth events and the event being
authed.
**TODO**: Clean the above explanations up a bit.
Auth chain resolution
~~~~~~~~~~~~~~~~~~~~~
If an auth check fails, or if we get told something we accepted should have
been rejected, we need to try and determine who is right.
If two servers disagree about the validity of the auth events, both should
inform the other of what they think the current auth chain is. If either are
missing auth events that they know are valid (through authorization and state
resolution) they process the missing events as usual.
If either side notice that the other has accepted an auth events we think
should be rejected (for reasons *not* in their auth chain), that server should
inform the other with suitable proof.
The proofs can be:
- An *event chain* that shows an auth event is *not* an ancestor of the event.
This can be done by giving the full ancestor chains up to the depth of the
invalid auth event.
- Given an event (and event chain?) showing that authorization had been revoked.
If a server discovers it cannot prove the other side is wrong, then it accepts
that the other is correct; i.e. we always accept that the other side is correct
unless we can prove otherwise.
Constructing a new event
------------------------
**TODO**
When constructing a new event, the server should insert the following fields:
- ``prev_events``: The list of event ids of what the server believes are the
current leaf nodes of the event graph (i.e., nodes that have been received
but are yet to be referenced by another event).
- ``depth``: An integer one greater than the maximum depth of the event's
previous events.
- ``auth_events``: The list of event ids that authorizes this event. This
should be a subset of the current state.
- ``origin_server_ts``: The time the server created the event.
- ``origin``: The name of the server.
Signing and Hashes
~~~~~~~~~~~~~~~~~~
**TODO**
Validation
----------
**TODO**
Domain specific string
A string of the form ``<prefix><localpart>:<domain>``, where <prefix> is a
single character, ``<localpart>`` is an arbitrary string that does not
include a colon, and `<domain>` is a valid server name.
``room_id``
A domain specific string with prefix ``!`` that is static across all events
in a graph and uniquely identifies it. The ``domain`` should be that of the
homeserver that created the room (i.e., the server that generated the
first ``m.room.create`` event).
``sender``
The entity that logically sent the event. This is usually a user id, but
can also be a server name.
User Id
A domain specific string with prefix ``@`` representing a user account. The
``domain`` is the homeserver of the user and is the server used to contact
the user.
Joining a room
--------------
If a user requests to join a room that the server is already in (i.e. the a
user on that server has already joined the room) then the server can simply
generate a join event and send it as normal.
If the server is not already in the room it needs to will need to join via
another server that is already in the room. This is done as a two step process.
First, the local server requests from the remote server a skeleton of a join
event. The remote does this as the local server does not have the event graph
to use to fill out the ``prev_events`` key in the new event. Critically, the
remote server does not process the event it responded with.
Once the local server has this event, it fills it out with any extra data and
signs it. Once ready the local server sends this event to a remote server
(which could be the same or different from the first remote server), this
remote server then processes the event and distributes to all the other
participating servers in that room. The local server is told about the
current state and complete auth chain for the join event. The local server
can then process the join event itself.
.. Note::
Finding which server to use to join any particular room is not specified.
Inviting a user
---------------
To invite a remote user to a room we need their homeserver to sign the invite
event. This is done by sending the event to the remote server, which then signs
the event, before distributing the invite to other servers.
Handling incoming events
------------------------
When a server receives an event, it should:
#. Check if it knows about the room. If it doesn't, then it should get the
current state and auth events to determine whether the server *should* be in
the room. If so continue, if not drop or reject the event
#. If the server already knew about the room, check the prev events to see if
it is missing any events. If it is, request them. Servers should limit how
far back they will walk the event graph for missing events.
#. If the server does not have all the prev events, then it should request the
current state and auth events from a server.
Failures
--------
A server can notify a remote server about something it thinks it has done
wrong using the failures mechanism. For example, the remote accepted an event
the local think it shouldn't have.
A failure has a severity level depending on the action taken by the local
server. These levels are:
``FATAL``
The local server could not parse the event, for example due to a missing
required field.
``ERROR``
The local server *could* parse the event, but it was rejected. For example,
the event may have failed an authorization check.
``WARN``
The local server accepted the event, but something was unexpected about it.
For example, the event may have referenced another event the local server
thought should be rejected.
A failure also includes several other fields:
``code``
A numeric code (to be defined later) indicating a particular type of
failure.
``reason``
A short string indicating what was wrong, for diagnosis purposes on the
remote server.
``affected``
The event id of the event this failure is responding to. For example, if
an accepted event referenced a rejected event, this would point to the
accepted one.
``source``
The event id of the event that was the source of this unexpected behaviour.
For example, if an accepted event referenced a rejected event, this would
point to the rejected one.
Appendix
========
**TODO**
Example event:
.. code::
{
"auth_events": [
[
"$14187571482fLeia:localhost:8480",
{
"sha256": "kiZUclzzPetHfy0rVoYKnYXnIv5VxH8a4996zVl8xbw"
}
],
[
"$14187571480odWTd:localhost:8480",
{
"sha256": "GqtndjviW9yPGaZ6EJfzuqVCRg5Lhoyo4YYv1NFP7fw"
}
],
[
"$14205549830rrMar:localhost:8480",
{
"sha256": "gZmL23QdWjNOmghEZU6YjqgHHrf2fxarKO2z5ZTbkig"
}
]
],
"content": {
"body": "Test!",
"msgtype": "m.text"
},
"depth": 250,
"event_id": "$14207181140uTFlx:localhost:8480",
"hashes": {
"sha256": "k1nuafFdFvZXzhb5NeTE0Q2Jkqu3E8zkh3uH3mqwIxc"
},
"origin": "localhost:8480",
"origin_server_ts": 1420718114694,
"prev_events": [
[
"$142071809077XNNkP:localhost:8480",
{
"sha256": "xOnU1b+4LOVz5qih0dkNFrdMgUcf35fKx9sdl/gqhjY"
}
]
],
"room_id": "!dwZDafgDEFTtpPKpLy:localhost:8480",
"sender": "@bob:localhost:8480",
"signatures": {
"localhost:8480": {
"ed25519:auto": "Nzd3D+emFBJJ4LCTzQEZaKO0Sa3sSTR1fGpu8OWXYn+7XUqke9Q1jYUewrEfxb3lPxlYWm/GztVUJizLz1K5Aw"
}
},
"type": "m.room.message",
"unsigned": {
"age": 500
}
}

146
attic/drafts/flows_and_auth.rst
Unescape Escape View File

@ -1,146 +0,0 @@
Failures
--------
A server may encouter an error when trying to process an event received from a
remote server. In these cases the server may send a `failure` to the remote.
A `failure` references both the event that it was trying to process and the
event that triggered the processing. For example, a failure may be emitted if
one of the parents of the received events was not authorized.
A failure also includes a `severity` field that indicates what action was taken
by the server. There are three valid values:
* `Fatal`: The server failed to parse the event. The event is dropped by the
server as well as all descendants.
* `Error`: The server rejected the event, for example due to authorization.
That event is dropped, but descendants may be accepted.
* `Warn`: The server accepted all events, but believes the remote did
something wrong. For example, references an event the local server believes
is unauthorized.
Data Flows
----------
Invite
++++++
To invite a remote user to an existing room a server distributes an invitiation
event signed by the remote invitee's server (allowing other servers in the room
to be sure that the invitee's server had seen the invite.)
To get the remote server's signature on the event it is sent in a special
request to the remote server, which then responds with the signed invite (if it
accepted it as valid.) The remote server may respond with an error if the user
does not exist.
Join
++++
If a server is already in the room it can simply emit a join event for the user
joining.
If the server is not currently in the room it needs to join via a remote server
in the room, therefore to join a room a server must have have both the room id
and a list of remote servers that may be in the room.
To join via a remote server the local server first requests a valid join event
for the room from the remote server. The local then fills it out, signs it, and
then sends the final join event to the remote server for distribution. The
remore responds to this second request with the current state of the room at
the join request and the auth chain for that state.
Authorization
-------------
The authorization for an event depends solely on the current state at that
event. If a policy server is configured for the room, then the authorization
for the event is the signature of the policy server.
The state events that affect whether an event is authorized are called
`protocol events`, and are:
* `m.room.create`
* `m.room.power_levels`
* `m.room.member`
* `m.room.join_rules`
All events *must* list all the protocol events that grant them their
authorization. All origin servers *must* serve up on request the full graph of
protocol events for all events it has sent. The graph of protocol events is the
connected directed acyclic graph with events as nodes and the list of protocol
events their edges.
Join
++++
A user may join a room if:
* The join rule is "public".
* The join rule is "invite" and the user has been invited by a user that has
already joined.
* The user is in the `may_join` list.
Invite
++++++
A user may invite another user if the join rule is either "public" or "invite"
and the user has joined the room.
Creation
++++++++
A `m.room.create` must be the first event in the room.
Ban, Kick and Redaction
+++++++++++++++++++++++
To ban or kick another user in the room, or to redact an event, then the user
must have a power level of at least that specificied in the
`m.room.power_level` event for kick, ban and redactions.
Other Events
++++++++++++
A user may send an event if all the following hold true:
* The user is in the room.
* If the type of the event is listed in the `m.room.power_levels`, then the
user must have at least that power level. Otherwise, the user must have a
power level of at least `events_default` or `state_default`, depending on
if the event is a message or state event respectively.
Unauthorized Events
-------------------
An unauthorized event should not be accepted into the event graph, i.e. new
events should not reference any unauthorized events. There are situations where
this can happen and so it is not considered an error to include an unauthorized
event in the event graph. It is an error for events to refer unauthorized
events in their `auth_events` section and will in turn be considered
unauthorized.
A server may choose to store only the redacted form of an unauthorized event if
it is included in the event graph.
A server may emit a warning to a remote server if it references an event it
considers unauthorized.
State and Authorization Querying
--------------------------------
A local server may become aware that it and a remote server's view of the
current state are inconsistent. The local server may then send its current
state to the remote, which responds with its view of the current state. Both
servers should then recompute the local state. If they are conforming
implementations then they will reach the same conclusions.

1186
attic/drafts/general_api.rst
View File

File diff suppressed because it is too large Load Diff

133
attic/drafts/human-id-rules.rst
Unescape Escape View File

@ -1,133 +0,0 @@
Abstract
========
This document outlines a proposed format for human-readable IDs within Matrix.
For status see https://github.com/matrix-org/matrix-doc/pull/3/files
Background
----------
UTF-8 is the dominant character encoding for Unicode on the web. However,
using Unicode as the character set for human-readable IDs is troublesome. There
are many different characters which appear identical to each other, but would
produce different IDs. In addition, there are non-printable characters which
cannot be rendered by the end-user. This creates an opportunity for
phishing/spoofing of IDs, commonly known as a homograph attack.
Web browsers encountered this problem when International Domain Names were
introduced. A variety of checks were put in place in order to protect users. If
an address failed the check, the raw punycode would be displayed to
disambiguate the address.
The human-readable IDs in Matrix are Room Aliases and User IDs.
Room aliases look like ``#localpart:domain``. These aliases point to opaque
non human-readable room IDs. These pointers can change to point at a different
room ID at any time. User IDs look like ``@localpart:domain``. These represent
actual end-users (there is no indirection).
Proposal
========
User IDs and Room Aliases MUST be Unicode as UTF-8. Checks are performed on
these IDs by homeservers to protect users from phishing/spoofing attacks.
These checks are:
User ID Localparts:
- MUST NOT contain a ``:`` or start with a ``@`` or ``.``
- MUST NOT contain one of the 107 blacklisted characters on this list:
http://kb.mozillazine.org/Network.IDN.blacklist_chars
- After stripping " 0-9, +, -, [, ], _, and the space character it MUST NOT
contain characters from >1 language, defined by the `exemplar characters`_
on http://cldr.unicode.org/
.. _exemplar characters: http://cldr.unicode.org/translation/characters#TOC-Exemplar-Characters
Room Alias Localparts:
- MUST NOT contain a ``:``
- MUST NOT contain one of the 107 blacklisted characters on this list:
http://kb.mozillazine.org/Network.IDN.blacklist_chars
- After stripping " 0-9, +, -, [, ], _, and the space character it MUST NOT
contain characters from >1 language, defined by the `exemplar characters`_
on http://cldr.unicode.org/
.. _exemplar characters: http://cldr.unicode.org/translation/characters#TOC-Exemplar-Characters
In the event of a failed user ID check, well behaved homeservers MUST:
- Rewrite user IDs in the offending events to be punycode with an additional ``@``
prefix **before** delivering them to clients. There are no guarantees for
consistency between homeserver ID checking implementations. As a result, user
IDs MUST be sent in their *original* form over federation. This can be done in
a stateless manner as the punycode form has no information loss.
In the event of a failed room alias check, well behaved homeservers MUST:
- Send an HTTP status code 400 with an ``errcode`` of ``M_FAILED_HUMAN_ID_CHECK``
to the client if the client is attempting to *create* this alias.
- Send an HTTP status code 400 with an ``errcode`` of ``M_FAILED_HUMAN_ID_CHECK``
to the client if the client is attempting to *join* a room via this alias.
Examples::
@ebаy:example.org (Cyrillic 'a', everything else English)
@@xn--eby-7cd:example.org (Punycode with additional '@')
Homeservers SHOULD NOT allow two user IDs that differ only by case. This
SHOULD be applied based on the capitalisation rules in the CLDR dataset:
http://cldr.unicode.org/
This check SHOULD be applied when the user ID is created, in order to prevent
registration with the same name and different capitalisations, e.g.
``@foo:bar`` vs ``@Foo:bar`` vs ``@FOO:bar``. Homeservers MAY canonicalise
the user ID to be completely lower-case if desired.
Rationale
=========
Each ID is split into segments (localpart/domain) around the ``:``. For
this reason, ``:`` is a reserved character and cannot be a localpart character.
The 107 blacklisted characters are used to prevent non-printable characters and
spaces from being used. The decision to ban characters from more than 1 language
matches the behaviour of `Google Chrome for IDN handling`_. This is to protect
against common homograph attacks such as ebаy.com (Cyrillic "a", rest is
English). This would always result in a failed check. Even with this though
there are limitations. For example, сахар is entirely Cyrillic, whereas caxap is
entirely Latin.
.. _Google Chrome for IDN handling: https://www.chromium.org/developers/design-documents/idn-in-google-chrome
User ID localparts cannot start with ``@`` so that a namespace of localparts
beginning with ``@`` can be created. This namespace is used for user IDs which
fail the ID checks. A failed ID could look like ``@@xn--c1yn36f:example.org``.
If a user ID fails the check, the user ID on the event is renamed. This doesn't
require extra work for clients, and users will see an odd user ID rather than a
spoofed name. Renaming is done in order to protect users of a given HS, so if a
malicious HS doesn't rename their IDs, it doesn't affect any other HS.
Room aliases cannot be rewritten as punycode and sent to the HS the alias is
referring to as the HS will not necessarily understand the rewritten alias.
Other rejected solutions for failed checks
------------------------------------------
- Additional key: Informational key on the event attached by HS to say "unsafe
ID". Problem: clients can just ignore it, and since it will appear only very
rarely, easy to forget when implementing clients.
- Require client handshake: Forces clients to implement
a check, else they cannot communicate with the misleading ID. However, this
is extra overhead in both client implementations and round-trips.
- Reject event: Outright rejection of the ID at the point of creation /
receiving event. Point of creation rejection is preferable to avoid the ID
entering the system in the first place. However, malicious HSes can just
allow the ID. Hence, other homeservers must reject them if they see them in
events. Client never sees the problem ID, provided the HS is correctly
implemented. However, it is difficult to ensure that ALL HSes will come to the
same conclusion (given the CLDR dataset does come out with new versions).
Outstanding Problems
====================
Capitalisation
--------------
The capitalisation rules outlined above are nice but do not fully resolve issues
where ``@alice:example.com`` tries to speak with ``@bob:example.org`` using
``@Bob:example.org``. It is up to ``example.org`` to map ``Bob`` to ``bob`` in
a sensible way.

149
attic/drafts/model/presence.rst
Unescape Escape View File

@ -1,149 +0,0 @@
API Efficiency
==============
A simple implementation of presence messaging has the ability to cause a large
amount of Internet traffic relating to presence updates. In order to minimise
the impact of such a feature, the following observations can be made:
* There is no point in a homeserver polling status for peers in a user's
presence list if the user has no clients connected that care about it.
* It is highly likely that most presence subscriptions will be symmetric - a
given user watching another is likely to in turn be watched by that user.
* It is likely that most subscription pairings will be between users who share
at least one Room in common, and so their homeservers are actively
exchanging message PDUs or transactions relating to that Room.
* Presence update messages do not need realtime guarantees. It is acceptable to
delay delivery of updates for some small amount of time (10 seconds to a
minute).
The general model of presence information is that of a HS registering its
interest in receiving presence status updates from other HSes, which then
promise to send them when required. Rather than actively polling for the
current state all the time, HSes can rely on their relative stability to only
push updates when required.
A homeserver should not rely on the longterm validity of this presence
information, however, as this would not cover such cases as a user's server
crashing and thus failing to inform their peers that users it used to host are
no longer available online. Therefore, each promise of future updates should
carry with a timeout value (whether explicit in the message, or implicit as some
defined default in the protocol), after which the receiving HS should consider
the information potentially stale and request it again.
However, because of the likelihood that two homeservers are exchanging messages
relating to chat traffic in a room common to both of them, the ongoing receipt
of these messages can be taken by each server as an implicit notification that
the sending server is still up and running, and therefore that no status changes
have happened; because if they had the server would have sent them. A second,
larger timeout should be applied to this implicit inference however, to protect
against implementation bugs or other reasons that the presence state cache may
become invalid; eventually the HS should re-enquire the current state of users
and update them with its own.
The following workflows can therefore be used to handle presence updates:
1 When a user first appears online their HS sends a message to each other HS
containing at least one user to be watched; each message carrying both a
notification of the sender's new online status, and a request to obtain and
watch the target users' presence information. This message implicitly
promises the sending HS will now push updates to the target HSes.
2 The target HSes then respond a single message each, containing the current
status of the requested user(s). These messages too implicitly promise the
target HSes will themselves push updates to the sending HS.
As these messages arrive at the sending user's HS they can be pushed to the
user's client(s), possibly batched again to ensure not too many small
messages which add extra protocol overheads.
At this point, all the user's clients now have the current presence status
information for this moment in time, and have promised to send each other
updates in future.
3 The HS maintains two watchdog timers per peer HS it is exchanging presence
information with. The first timer should have a relatively small expiry
(perhaps 1 minute), and the second timer should have a much longer time
(perhaps 1 hour).
4 Any time any kind of message is received from a peer HS, the short-term
presence timer associated with it is reset.
5 Whenever either of these timers expires, an HS should push a status reminder
to the target HS whose timer has now expired, and request again from that
server the status of the subscribed users.
6 On receipt of one of these presence status reminders, an HS can reset both
of its presence watchdog timers.
To avoid bursts of traffic, implementations should attempt to stagger the expiry
of the longer-term watchdog timers for different peer HSes.
When individual users actively change their status (either by explicit requests
from clients, or inferred changes due to idle timers or client timeouts), the HS
should batch up any status changes for some reasonable amount of time (10
seconds to a minute). This allows for reduced protocol overheads in the case of
multiple messages needing to be sent to the same peer HS; as is the likely
scenario in many cases, such as a given human user having multiple user
accounts.
API Requirements
================
The data model presented here puts the following requirements on the APIs:
Client-Server
-------------
Requests that a client can make to its homeserver
* get/set current presence state
Basic enumeration + ability to set a custom piece of text
* report per-device idle time
After some (configurable?) idle time the device should send a single message
to set the idle duration. The HS can then infer a "start of idle" instant and
use that to keep the device idleness up to date. At some later point the
device can cancel this idleness.
* report per-device type
Inform the server that this device is a "mobile" device, or perhaps some
other to-be-defined category of reduced capability that could be presented to
other users.
* start/stop presence polling for my presence list
It is likely that these messages could be implicitly inferred by other
messages, though having explicit control is always useful.
* get my presence list
[implicit poll start?]
It is possible that the HS doesn't yet have current presence information when
the client requests this. There should be a "don't know" type too.
* add/remove a user to my presence list
Server-Server
-------------
Requests that homeservers make to others
* request permission to add a user to presence list
* allow/deny a request to add to a presence list
* perform a combined presence state push and subscription request
For each sending user ID, the message contains their new status.
For each receiving user ID, the message should contain an indication on
whether the sending server is also interested in receiving status from that
user; either as an immediate update response now, or as a promise to send
future updates.
Server to Client
----------------
[[TODO(paul): There also needs to be some way for a user's HS to push status
updates of the presence list to clients, but the general server-client event
model currently lacks a space to do that.]]

231
attic/drafts/model/profiles.rst
Unescape Escape View File

@ -1,231 +0,0 @@
========
Profiles
========
A description of Synapse user profile metadata support.
Overview
========
Internally within Synapse users are referred to by an opaque ID, which consists
of some opaque localpart combined with the domain name of their homeserver.
Obviously this does not yield a very nice user experience; users would like to
see readable names for other users that are in some way meaningful to them.
Additionally, users like to be able to publish "profile" details to inform other
users of other information about them.
It is also conceivable that since we are attempting to provide a
worldwide-applicable messaging system, that users may wish to present different
subsets of information in their profile to different other people, from a
privacy and permissions perspective.
A Profile consists of a display name, an (optional?) avatar picture, and a set
of other metadata fields that the user may wish to publish (email address, phone
numbers, website URLs, etc...). We put no requirements on the display name other
than it being a valid Unicode string. Since it is likely that users will end up
having multiple accounts (perhaps by necessity of being hosted in multiple
places, perhaps by choice of wanting multiple distinct identifies), it would be
useful that a metadata field type exists that can refer to another Synapse User
ID, so that clients and HSes can make use of this information.
Metadata Fields
---------------
[[TODO(paul): Likely this list is incomplete; more fields can be defined as we
think of them. At the very least, any sort of supported ID for the 3rd Party ID
servers should be accounted for here.]]
* Synapse Directory Server username(s)
* Email address
* Phone number - classify "home"/"work"/"mobile"/custom?
* Twitter/Facebook/Google+/... social networks
* Location - keep this deliberately vague to allow people to choose how
granular it is
* "Bio" information - date of birth, etc...
* Synapse User ID of another account
* Web URL
* Freeform description text
Visibility Permissions
======================
A homeserver implementation could offer the ability to set permissions on
limited visibility of those fields. When another user requests access to the
target user's profile, their own identity should form part of that request. The
HS implementation can then decide which fields to make available to the
requestor.
A particular detail of implementation could allow the user to create one or more
ACLs; where each list is granted permission to see a given set of non-public
fields (compare to Google+ Circles) and contains a set of other people allowed
to use it. By giving these ACLs strong identities within the HS, they can be
referenced in communications with it, granting other users who encounter these
the "ACL Token" to use the details in that ACL.
If we further allow an ACL Token to be present on Room join requests or stored
by 3PID servers, then users of these ACLs gain the extra convenience of not
having to manually curate people in the access list; anyone in the room or with
knowledge of the 3rd Party ID is automatically granted access. Every HS and
client implementation would have to be aware of the existence of these ACL
Token, and include them in requests if present, but not every HS implementation
needs to actually provide the full permissions model. This can be used as a
distinguishing feature among competing implementations. However, servers MUST
NOT serve profile information from a cache if there is a chance that its limited
understanding could lead to information leakage.
Client Concerns of Multiple Accounts
====================================
Because a given person may want to have multiple Synapse User accounts, client
implementations should allow the use of multiple accounts simultaneously
(especially in the field of mobile phone clients, which generally don't support
running distinct instances of the same application). Where features like address
books, presence lists or rooms are presented, the client UI should remember to
make distinct with user account is in use for each.
Directory Servers
=================
Directory Servers can provide a forward mapping from human-readable names to
User IDs. These can provide a service similar to giving domain-namespaced names
for Rooms; in this case they can provide a way for a user to reference their
User ID in some external form (e.g. that can be printed on a business card).
The format for Synapse user name will consist of a localpart specific to the
directory server, and the domain name of that directory server:
@localname:some.domain.name
The localname is separated from the domain name using a colon, so as to ensure
the localname can still contain periods, as users may want this for similarity
to email addresses or the like, which typically can contain them. The format is
also visually quite distinct from email addresses, phone numbers, etc... so
hopefully reasonably "self-describing" when written on e.g. a business card
without surrounding context.
[[TODO(paul): we might have to think about this one - too close to email?
Twitter? Also it suggests a format scheme for room names of
#localname:domain.name, which I quite like]]
Directory server administrators should be able to make some kind of policy
decision on how these are allocated. Servers within some "closed" domain (such
as company-specific ones) may wish to verify the validity of a mapping using
their own internal mechanisms; "public" naming servers can operate on a FCFS
basis. There are overlapping concerns here with the idea of the 3rd party
identity servers as well, though in this specific case we are creating a new
namespace to allocate names into.
It would also be nice from a user experience perspective if the profile that a
given name links to can also declare that name as part of its metadata.
Furthermore as a security and consistency perspective it would be nice if each
end (the directory server and the user's homeserver) check the validity of the
mapping in some way. This needs investigation from a security perspective to
ensure against spoofing.
One such model may be that the user starts by declaring their intent to use a
given user name link to their homeserver, which then contacts the directory
service. At some point later (maybe immediately for "public open FCFS servers",
maybe after some kind of human intervention for verification) the DS decides to
honour this link, and includes it in its served output. It should also tell the
HS of this fact, so that the HS can present this as fact when requested for the
profile information. For efficiency, it may further wish to provide the HS with
a cryptographically-signed certificate as proof, so the HS serving the profile
can provide that too when asked, avoiding requesting HSes from constantly having
to contact the DS to verify this mapping. (Note: This is similar to the security
model often applied in DNS to verify PTR <-> A bidirectional mappings).
Identity Servers
================
The identity servers should support the concept of pointing a 3PID being able to
store an ACL Token as well as the main User ID. It is however, beyond scope to
do any kind of verification that any third-party IDs that the profile is
claiming match up to the 3PID mappings.
User Interface and Expectations Concerns
========================================
Given the weak "security" of some parts of this model as compared to what users
might expect, some care should be taken on how it is presented to users,
specifically in the naming or other wording of user interface components.
Most notably mere knowledge of an ACL Pointer is enough to read the information
stored in it. It is possible that Home or Identity Servers could leak this
information, allowing others to see it. This is a security-vs-convenience
balancing choice on behalf of the user who would choose, or not, to make use of
such a feature to publish their information.
Additionally, unless some form of strong end-to-end user-based encryption is
used, a user of ACLs for information privacy has to trust other homeservers not
to lie about the identify of the user requesting access to the Profile.
API Requirements
================
The data model presented here puts the following requirements on the APIs:
Client-Server
-------------
Requests that a client can make to its homeserver
* get/set my Display Name
This should return/take a simple "text/plain" field
* get/set my Avatar URL
The avatar image data itself is not stored by this API; we'll just store a
URL to let the clients fetch it. Optionally HSes could integrate this with
their generic content attacmhent storage service, allowing a user to set
upload their profile Avatar and update the URL to point to it.
* get/add/remove my metadata fields
Also we need to actually define types of metadata
* get another user's Display Name / Avatar / metadata fields
TODO(paul): At some later stage we should consider the API for:
* get/set ACL permissions on my metadata fields
* manage my ACL tokens
Server-Server
-------------
Requests that homeservers make to others
* get a user's Display Name / Avatar
* get a user's full profile - name/avatar + MD fields
This request must allow for specifying the User ID of the requesting user,
for permissions purposes. It also needs to take into account any ACL Tokens
the requestor has.
* push a change of Display Name to observers (overlaps with the presence API)
Room Event PDU Types
--------------------
Events that are pushed from homeservers to other homeservers or clients.
* user Display Name change
* user Avatar change
[[TODO(paul): should the avatar image itself be stored in all the room
histories? maybe this event should just be a hint to clients that they should
re-fetch the avatar image]]

68
attic/drafts/model/protocol_examples.rst
Unescape Escape View File

@ -1,68 +0,0 @@
PUT /send/abc/ HTTP/1.1
Host: ...
Content-Length: ...
Content-Type: application/json
.. code :: javascript
{
"origin": "localhost:5000",
"pdus": [
{
"content": {},
"context": "tng",
"depth": 12,
"is_state": false,
"origin": "localhost:5000",
"pdu_id": 1404381396854,
"pdu_type": "feedback",
"prev_pdus": [
[
"1404381395883",
"localhost:6000"
]
],
"ts": 1404381427581
}
],
"prev_ids": [
"1404381396852"
],
"ts": 1404381427823
}
HTTP/1.1 200 OK
======================================
GET /pull/-1/ HTTP/1.1
Host: ...
Content-Length: 0
HTTP/1.1 200 OK
Content-Length: ...
Content-Type: application/json
.. code :: javascript
{
origin: ...,
prev_ids: ...,
data: [
{
data_id: ...,
prev_pdus: [...],
depth: ...,
ts: ...,
context: ...,
origin: ...,
content: {
...
}
},
...,
]
}

113
attic/drafts/model/room-join-workflow.rst
Unescape Escape View File

@ -1,113 +0,0 @@
==================
Room Join Workflow
==================
An outline of the workflows required when a user joins a room.
Discovery
=========
To join a room, a user has to discover the room by some mechanism in order to
obtain the (opaque) Room ID and a candidate list of likely homeservers that
contain it.
Sending an Invitation
---------------------
The most direct way a user discovers the existence of a room is from a
invitation from some other user who is a member of that room.
The inviter's HS sets the membership status of the invitee to "invited" in the
"m.members" state key by sending a state update PDU. The HS then broadcasts this
PDU among the existing members in the usual way. An invitation message is also
sent to the invited user, containing the Room ID and the PDU ID of this
invitation state change and potentially a list of some other homeservers to use
to accept the invite. The user's client can then choose to display it in some
way to alert the user.
[[TODO(paul): At present, no API has been designed or described to actually send
that invite to the invited user. Likely it will be some facet of the larger
user-user API required for presence, profile management, etc...]]
Directory Service
-----------------
Alternatively, the user may discover the channel via a directory service; either
by performing a name lookup, or some kind of browse or search acitivty. However
this is performed, the end result is that the user's homeserver requests the
Room ID and candidate list from the directory service.
[[TODO(paul): At present, no API has been designed or described for this
directory service]]
Joining
=======
Once the ID and homeservers are obtained, the user can then actually join the
room.
Accepting an Invite
-------------------
If a user has received and accepted an invitation to join a room, the invitee's
homeserver can now send an invite acceptance message to a chosen candidate
server from the list given in the invitation, citing also the PDU ID of the
invitation as "proof" of their invite. (This is required as due to late message
propagation it could be the case that the acceptance is received before the
invite by some servers). If this message is allowed by the candidate server, it
generates a new PDU that updates the invitee's membership status to "joined",
referring back to the acceptance PDU, and broadcasts that as a state change in
the usual way. The newly-invited user is now a full member of the room, and
state propagation proceeds as usual.
Joining a Public Room
---------------------
If a user has discovered the existence of a room they wish to join but does not
have an active invitation, they can request to join it directly by sending a
join message to a candidate server on the list provided by the directory
service. As this list may be out of date, the HS should be prepared to retry
other candidates if the chosen one is no longer aware of the room, because it
has no users as members in it.
Once a candidate server that is aware of the room has been found, it can
broadcast an update PDU to add the member into the "m.members" key setting their
state directly to "joined" (i.e. bypassing the two-phase invite semantics),
remembering to include the new user's HS in that list.
Knocking on a Semi-Public Room
------------------------------
If a user requests to join a room but the join mode of the room is "knock", the
join is not immediately allowed. Instead, if the user wishes to proceed, they
can instead post a "knock" message, which informs other members of the room that
the would-be joiner wishes to become a member and sets their membership value to
"knocked". If any of them wish to accept this, they can then send an invitation
in the usual way described above. Knowing that the user has already knocked and
expressed an interest in joining, the invited user's homeserver should
immediately accept that invitation on the user's behalf, and go on to join the
room in the usual way.
[[NOTE(Erik): Though this may confuse users who expect 'X has joined' to
actually be a user initiated action, i.e. they may expect that 'X' is actually
looking at synapse right now?]]
[[NOTE(paul): Yes, a fair point maybe we should suggest HSes don't do that, and
just offer an invite to the user as normal]]
Private and Non-Existent Rooms
------------------------------
If a user requests to join a room but the room is either unknown by the home
server receiving the request, or is known by the join mode is "invite" and the
user has not been invited, the server must respond that the room does not exist.
This is to prevent leaking information about the existence and identity of
private rooms.
Outstanding Questions
=====================
* Do invitations or knocks time out and expire at some point? If so when? Time
is hard in distributed systems.

276
attic/drafts/model/rooms.rst
Unescape Escape View File

@ -1,276 +0,0 @@
===========
Rooms Model
===========
A description of the general data model used to implement Rooms, and the
user-level visible effects and implications.
Overview
========
"Rooms" in Synapse are shared messaging channels over which all the participant
users can exchange messages. Rooms have an opaque persistent identify, a
globally-replicated set of state (consisting principly of a membership set of
users, and other management and miscellaneous metadata), and a message history.
Room Identity and Naming
========================
Rooms can be arbitrarily created by any user on any homeserver; at which point
the homeserver will sign the message that creates the channel, and the
fingerprint of this signature becomes the strong persistent identify of the
room. This now identifies the room to any homeserver in the network regardless
of its original origin. This allows the identify of the room to outlive any
particular server. Subject to appropriate permissions [to be discussed later],
any current member of a room can invite others to join it, can post messages
that become part of its history, and can change the persistent state of the room
(including its current set of permissions).
Homeservers can provide a directory service, allowing a lookup from a
convenient human-readable form of room label to a room ID. This mapping is
scoped to the particular homeserver domain and so simply represents that server
administrator's opinion of what room should take that label; it does not have to
be globally replicated and does not form part of the stored state of that room.
This room name takes the form
#localname:some.domain.name
for similarity and consistency with user names on directories.
To join a room (and therefore to be allowed to inspect past history, post new
messages to it, and read its state), a user must become aware of the room's
fingerprint ID. There are two mechanisms to allow this:
* An invite message from someone else in the room
* A referral from a room directory service
As room IDs are opaque and ephemeral, they can serve as a mechanism to create
"ad-hoc" rooms deliberately unnamed, for small group-chats or even private
one-to-one message exchange.
Stored State and Permissions
============================
Every room has a globally-replicated set of stored state. This state is a set of
key/value or key/subkey/value pairs. The value of every (sub)key is a
JSON-representable object. The main key of a piece of stored state establishes
its meaning; some keys store sub-keys to allow a sub-structure within them [more
detail below]. Some keys have special meaning to Synapse, as they relate to
management details of the room itself, storing such details as user membership,
and permissions of users to alter the state of the room itself. Other keys may
store information to present to users, which the system does not directly rely
on. The key space itself is namespaced, allowing 3rd party extensions, subject
to suitable permission.
Permission management is based on the concept of "power-levels". Every user
within a room has an integer assigned, being their "power-level" within that
room. Along with its actual data value, each key (or subkey) also stores the
minimum power-level a user must have in order to write to that key, the
power-level of the last user who actually did write to it, and the PDU ID of
that state change.
To be accepted as valid, a change must NOT:
* Be made by a user having a power-level lower than required to write to the
state key
* Alter the required power-level for that state key to a value higher than the
user has
* Increase that user's own power-level
* Grant any other user a power-level higher than the level of the user making
the change
[[TODO(paul): consider if relaxations should be allowed; e.g. is the current
outright-winner allowed to raise their own level, to allow for "inflation"?]]
Room State Keys
===============
[[TODO(paul): if this list gets too big it might become necessary to move it
into its own doc]]
The following keys have special semantics or meaning to Synapse itself:
m.member (has subkeys)
Stores a sub-key for every Synapse User ID which is currently a member of
this room. Its value gives the membership type ("knocked", "invited",
"joined").
m.power_levels
Stores a mapping from Synapse User IDs to their power-level in the room. If
they are not present in this mapping, the default applies.
The reason to store this as a single value rather than a value with subkeys
is that updates to it are atomic; allowing a number of colliding-edit
problems to be avoided.
m.default_level
Gives the default power-level for members of the room that do not have one
specified in their membership key.
m.invite_level
If set, gives the minimum power-level required for members to invite others
to join, or to accept knock requests from non-members requesting access. If
absent, then invites are not allowed. An invitation involves setting their
membership type to "invited", in addition to sending the invite message.
m.join_rules
Encodes the rules on how non-members can join the room. Has the following
possibilities:
- "public" - a non-member can join the room directly
- "knock" - a non-member cannot join the room, but can post a single "knock"
message requesting access, which existing members may approve or deny
- "invite" - non-members cannot join the room without an invite from an
existing member
- "private" - nobody who is not in the 'may_join' list or already a member
may join by any mechanism
In any of the first three modes, existing members with sufficient permission
can send invites to non-members if allowed by the "m.invite_level" key. A
"private" room is not allowed to have the "m.invite_level" set.
A client may use the value of this key to hint at the user interface
expectations to provide; in particular, a private chat with one other use
might warrant specific handling in the client.
m.may_join
A list of User IDs that are always allowed to join the room, regardless of any
of the prevailing join rules and invite levels. These apply even to private
rooms. These are stored in a single list with normal update-powerlevel
permissions applied; users cannot arbitrarily remove themselves from the list.
m.add_state_level
The power-level required for a user to be able to add new state keys.
m.public_history
If set and true, anyone can request the history of the room, without needing
to be a member of the room.
m.archive_servers
For "public" rooms with public history, gives a list of homeservers that
should be included in message distribution to the room, even if no users on
that server are present. These ensure that a public room can still persist
even if no users are currently members of it. This list should be consulted by
the dirctory servers as the candidate list they respond with.
The following keys are provided by Synapse for user benefit, but their value is
not otherwise used by Synapse.
m.name
Stores a short human-readable name for the room, such that clients can display
to a user to assist in identifying which room is which.
This name specifically is not the strong ID used by the message transport
system to refer to the room, because it may be changed from time to time.
m.topic
Stores the current human-readable topic
Room Creation Templates
=======================
A client (or maybe homeserver?) could offer a few templates for the creation of
new rooms. For example, for a simple private one-to-one chat the channel could
assign the creator a power-level of 1, requiring a level of 1 to invite, and
needing an invite before members can join. An invite is then sent to the other
party, and if accepted and the other user joins, the creator's power-level can
now be reduced to 0. This now leaves a room with two participants in it being
unable to add more.
Rooms that Continue History
===========================
An option that could be considered for room creation, is that when a new room is
created the creator could specify a PDU ID into an existing room, as the history
continuation point. This would be stored as an extra piece of meta-data on the
initial PDU of the room's creation. (It does not appear in the normal previous
PDU linkage).
This would allow users in rooms to "fork" a room, if it is considered that the
conversations in the room no longer fit its original purpose, and wish to
diverge. Existing permissions on the original room would continue to apply of
course, for viewing that history. If both rooms are considered "public" we might
also want to define a message to post into the original room to represent this
fork point, and give a reference to the new room.
User Direct Message Rooms
=========================
There is no need to build a mechanism for directly sending messages between
users, because a room can handle this ability. To allow direct user-to-user chat
messaging we simply need to be able to create rooms with specific set of
permissions to allow this direct messaging.
Between any given pair of user IDs that wish to exchange private messages, there
will exist a single shared Room, created lazily by either side. These rooms will
need a certain amount of special handling in both homeservers and display on
clients, but as much as possible should be treated by the lower layers of code
the same as other rooms.
Specially, a client would likely offer a special menu choice associated with
another user (in room member lists, presence list, etc..) as "direct chat". That
would perform all the necessary steps to create the private chat room. Receiving
clients should display these in a special way too as the room name is not
important; instead it should distinguish them on the Display Name of the other
party.
Homeservers will need a client-API option to request setting up a new user-user
chat room, which will then need special handling within the server. It will
create a new room with the following
m.member: the proposing user
m.join_rules: "private"
m.may_join: both users
m.power_levels: empty
m.default_level: 0
m.add_state_level: 0
m.public_history: False
Having created the room, it can send an invite message to the other user in the
normal way - the room permissions state that no users can be set to the invited
state, but because they're in the may_join list then they'd be allowed to join
anyway.
In this arrangement there is now a room with both users may join but neither has
the power to invite any others. Both users now have the confidence that (at
least within the messaging system itself) their messages remain private and
cannot later be provably leaked to a third party. They can freely set the topic
or name if they choose and add or edit any other state of the room. The update
powerlevel of each of these fixed properties should be 1, to lock out the users
from being able to alter them.
Anti-Glare
==========
There exists the possibility of a race condition if two users who have no chat
history with each other simultaneously create a room and invite the other to it.
This is called a "glare" situation. There are two possible ideas for how to
resolve this:
* Each homeserver should persist the mapping of (user ID pair) to room ID, so
that duplicate requests can be suppressed. On receipt of a room creation
request that the HS thinks there already exists a room for, the invitation to
join can be rejected if:
- a) the HS believes the sending user is already a member of the room (and
maybe their HS has forgotten this fact), or
- b) the proposed room has a lexicographically-higher ID than the existing
room (to resolve true race condition conflicts)
* The room ID for a private 1:1 chat has a special form, determined by
concatenting the User IDs of both members in a deterministic order, such that
it doesn't matter which side creates it first; the HSes can just ignore
(or merge?) received PDUs that create the room twice.

108
attic/drafts/model/third-party-id.rst
Unescape Escape View File

@ -1,108 +0,0 @@
======================
Third Party Identities
======================
A description of how email addresses, mobile phone numbers and other third
party identifiers can be used to authenticate and discover users in Matrix.
Overview
========
New users need to authenticate their account. An email or SMS text message can
be a convenient form of authentication. Users already have email addresses
and phone numbers for contacts in their address book. They want to communicate
with those contacts in Matrix without manually exchanging a Matrix User ID with
them.
Third Party IDs
---------------
[[TODO(markjh): Describe the format of a 3PID]]
Third Party ID Associations
---------------------------
An Associaton is a binding between a Matrix User ID and a Third Party ID (3PID).
Each 3PID can be associated with one Matrix User ID at a time.
[[TODO(markjh): JSON format of the association.]]
Verification
------------
An Assocation must be verified by a trusted Verification Server. Email
addresses and phone numbers can be verified by sending a token to the address
which a client can supply to the verifier to confirm ownership.
An email Verification Server may be capable of verifying all email 3PIDs or may
be restricted to verifying addresses for a particular domain. A phone number
Verification Server may be capable of verifying all phone numbers or may be
restricted to verifying numbers for a given country or phone prefix.
Verification Servers fulfil a similar role to Certificate Authorities in PKI so
a similar level of vetting should be required before clients trust their
signatures.
A Verification Server may wish to check for existing Associations for a 3PID
before creating a new Association.
Discovery
---------
Users can discover Associations using a trusted Identity Server. Each
Association will be signed by the Identity Server. An Identity Server may store
the entire space of Associations or may delegate to other Identity Servers when
looking up Associations.
Each Association returned from an Identity Server must be signed by a
Verification Server. Clients should check these signatures.
Identity Servers fulfil a similar role to DNS servers.
Privacy
-------
A User may publish the association between their phone number and Matrix User ID
on the Identity Server without publishing the number in their Profile hosted on
their homeserver.
Identity Servers should refrain from publishing reverse mappings and should
take steps, such as rate limiting, to prevent attackers enumerating the space of
mappings.
Federation
==========
Delegation
----------
Verification Servers could delegate signing to another server by issuing
certificate to that server allowing it to verify and sign a subset of 3PID on
its behalf. It would be necessary to provide a language for describing which
subset of 3PIDs that server had authority to validate. Alternatively it could
delegate the verification step to another server but sign the resulting
association itself.
The 3PID space will have a heirachical structure like DNS so Identity Servers
can delegate lookups to other servers. An Identity Server should be prepared
to host or delegate any valid association within the subset of the 3PIDs it is
resonsible for.
Multiple Root Verification Servers
----------------------------------
There can be multiple root Verification Servers and an Association could be
signed by multiple servers if different clients trust different subsets of
the verification servers.
Multiple Root Identity Servers
------------------------------
There can be be multiple root Identity Servers. Clients will add each
Association to all root Identity Servers.
[[TODO(markjh): Describe how clients find the list of root Identity Servers]]

27
attic/drafts/object_model.rst
Unescape Escape View File

@ -1,27 +0,0 @@
..TODO
What are the start & end tokens doing here?!
::
+---------------+
| Room |
| "Room-ID" |
| {State} | +----------------------+
| Name------|-------->| Event m.room.name |
| Topic | | "Name" |
| [Aliases] | +----------------------+ +-------------+
| [Members]-|---+ +----------------------+ <----| Start Token |
| [Messages] | | | Event m.room.member | +-------------+
| | | | +---->| "invite/join/ban" |
+---------------+ | "User-ID" |
| | +----------------------+
| | +----------------------+
| | Message | Event m.room.message |
| +-------------->| {content} |<--+
| +----------------------+ |
| Comment +----------------------+ |
+------------------>| Event m.room.message | |
| {content} | |
| "relates-to"-------|---+ +-------------+
+----------------------+ <----| End Token |
+-------------+

48
attic/drafts/pstn_gatewaying.txt
Unescape Escape View File

@ -1,48 +0,0 @@
Gatewaying to the PSTN via Matrix Application Services
======================================================
Matrix Application Services (AS) provides a way for PSTN users to interact
with Matrix via an AS acting as a gateway. Each PSTN user is represented as a
virtual user on a specific homeserver maintained by the AS. Typically the AS
is provisioned on a well-known AS-supplier HS (e.g. matrix.openmarket.com) or
is a service provisioned on the user's local HS.
In either scenario, the AS maintains virtual users of form
@.tel.e164:homeserver. These are lazily created (as per the AS spec) when
matrix users try to contact a user id of form @.tel.*:homeserver, or when the
AS needs to inject traffic into the HS on behalf of the PSTN user. The reason
for these being a visible virtual user rather than an invisible user or an
invisible sniffing AS is because they do represent real physical 3rd party
endpoints in the PSTN, and need to be able to send return messages.
Communication with an actual PSTN user happens in a normal Matrix room, which
for 1:1 matrix<->pstn contact will typically store all conversation history
with that user. On first contact, the matrix user invites the virtual user
into the room (or vice versa). In the event of switching to another AS-enabled
HS, the matrix user would kick the old AS and invite the new one. In the event
of needing loadbalancing between two SMS gateways (for instance), the user
would set visibility flags (TODO: specify per-message ACLs, or use crypto to
only sign messages so they're visible to certain other users?) to adjust which
virtual AS users could see which messages in the room.
For group chat, one or more AS virtual users may be invited to a group chat,
where-upon they will relay all the traffic in that group chat through to their
PSTN counterpart (and vice versa). This behaviour requires no additional
functionality beyond that required to support 1:1 chat.
When contacting a user, Matrix clients should check whether a given E.164
number is already mapped to a real Matrix user by querying the identity
servers (or subscribing to identity updates for a given E.164 number. TODO: ID
server subscriptions). If the E.164 number has a validated mapping in the ID
server to a Matrix ID, then this target ID should be used instead of
contacting the virtual user.
It's likely that PSTN gateway ASes will need to charge the end-user for use of
the gateway. The AS must therefore track credit per matrix ID it interacts
with, and stop gatewaying as desired once credit is exhausted. The task of
extracting credit from the end-user and adding it to the AS is not covered by
the Matrix specification.
For SMS routing, options are:
* Terminate traffic only (from a shared shortcode originator)
* Two-way traffic via a VMN. To save allocating huge numbers of VMNs to Matrix users, the VMN can be allocated from a pool such that each {caller,callee} tuple is unique (but the caller number will only work from that specific callee).

76
attic/drafts/reputation_thoughts.rst
Unescape Escape View File

@ -1,76 +0,0 @@
(a stream of incoherent consciousness from matthew which should go somewhere)
Invite-graph based reputation data:
* Users need a reputation score to issue invites or join public rooms.
* A user can have many reputation scores in different audiences (and perhaps a global average?)
* A room (degenerate case: user) can align itself with a given audience in order to consume the reputation data for that audience.
* The people that a user invites inherits a proportion of their reputation.
* If your reputation in an audience is ever reduced, it similarly reduces the reputation you have ever conveyed to anyone else (which propagates through the invite graph).
* Users increase reputation by:
* Inviting someone.
* Upvoting their messages in a room (i.e. for the suitability of that audience)
* Users decrease reputation by:
* Blocking them.
* Downvoting their messages in a room (i.e. for the suitability of that audience)
Need to ensure the accounts are of a decent quality - making it harder to create sockpuppet accounts and associating them with real people is more important than the actual reputation problem.
Build a war game simulation to test?
Problems:
* How are audiences defined? Just a given unique set of users? Which then makes inheriting reputation easy between audiences - if the overlap is significant, the chances are the reputation rules are the same.
* But is it possible to have the same set of users in two different rooms have different rules for reputation? Probably yes, as the potential audience may include future invitees or indeed the general public, so history visibility rules should probably also contribute to this. But given privacy rules can change over time, each room should effectively define its own audience. So in the end, an audience === a room.
* Create a large network of fake users, and go and have them all vote up each other's score for a given audience.
* This can be solved if the root inviter is penalised, which then destroys all the reputation they conveyed to their graph.
Could Reputation == Power Level (!?!?!)
Inheritence semantics for reputation between different audiences is hard.
* You should base the reputation of a stranger on their reputation in other communities that you or your communities have some overlap with.
* Do you consider 2nd hand reputation data at all from private rooms? Or do you look only at the public reputation data?
How do you do these calculations in a byzantine world?
How do you do these calculations whilst preserving privacy?
* Only consider reputation data from rooms you are actually in?
* Store reputation data in room state?
* Have a function (HS? client? AS? spider?) that aggregates reputation data (and proves that the aggregation is accurate, almost like blockchain mining?)?
* Or have a separate reputation global db seperate from room state that people contribute metrics into (which gathers the aggregate data into a single place, and makes it easier to query reputation data for strangers)
How do you avoid backstabbing? (People maliciously ganging up on someone to downvote them)?
How do you avoid a voting war? (Community fragments; different factions turn up and try to downvote the other)?
* This is effectively two different audiences emerging in a single room.
* Perhaps this means we should model audiences separately from rooms.
* Perhaps audiences are literally ACL groups? And eventually, one might change the ACLs of a room to eject one of the groups?
* Or do you just synthesise audiences based on cliques of people who support each other? The act of upvoting someone is effectively aligning yourself as being part of the same audience?
So:
* Gather all public upvote/downvotes/invites/blocks in a global DB.
* Partition this into audiences based on who votes on who. Stuff which is read and not complained about could provide a small implicit approval? Although this makes it easy to flood content to boost your reputation, so bad idea.
* Partitioning algorithm could be quite subtle.
* You could end up with lots of small audiences (including invalid ones), and it's fairly unclear how they get aggregated into a single view. How should you treat a stranger who you have no audience-overlap with at all? Treat them as effectively having zero reputation from your perspective?
Problem:
* If the douchebag who invites spammers never says anything, how do you go vote on their reputation? Should there be some kind of back-propagation? Or is there explicitly a "this person invited a douchebag" downvote? Or hang on - how can they ever get reputation in the first place to invite their sockpuppets if they don't say anything (beyond the initial invite)?
* What if users simply don't talk in public? Is it right that we prevent them issuing invites just because they stick to private rooms? What about inviting people into those private rooms? I guess the point is that if these are public invites, then they need to have some kind of public reputation, or rely on out-of-band private invitation to establish trust?
* Are we rewarding people who don't change their habits? There's no time component considered here, and we punish people's entire history of invites and rep if they misbehave. The only way to escape is to create a new identity atm. Is this a feature or a bug?
* How does this handle people's accounts getting 0wn3d and doing things which wipe out their reputation? => This is always a risk; ignore it.
* Do you need a particular level of reputation to be able to vote on people?
Summary?
* Partition the global population into multiple overlapping clusters called 'audiences' based on mutual(?) upvote/downvote relationships in public rooms.
* Clusters of the same people but in different rooms could be modelled as separate (but overlapping) clusters.
* Each audience builds up a reputation score for the global population, blending in damped scores from overlapping audiences.
* Anyone can upvote/downvote, but the votes will not contribute to your personal opinion unless the voter overlaps with your audience's scoresheet.
* A room could adopt a given audience (that of the moderators'?) for considering the reputation of who can join, invite people, etc.
* A user uses their own 'audience of one' scoresheet to put a threshold on filtering out contact from other users (invites, messages, etc).
* Their personal scoresheet is presumably a blend of all the audiences they are already in.
* The act of inviting someone gives them some reputation, within your audiences, proportional to your own. Similarly blocking reduces reputation.
* If you are downvoted, it retrospectively reduces the weight of all of your upvote/downvotes (at least for audiences that the downvoter's opinion contributes to). Similarly for upvoting.
* This penalisation process is transitive.
Do we even need the penalisation stuff if audience partitioning works?

76
attic/drafts/url_previews.md
Unescape Escape View File

@ -1,76 +0,0 @@
URL Previews
============
Design notes on a URL previewing service for Matrix:
Options are:
1. Have an AS which listens for URLs, downloads them, and inserts an event that describes their metadata.
* Pros:
* Decouples the implementation entirely from Synapse.
* Uses existing Matrix events & content repo to store the metadata.
* Cons:
* Which AS should provide this service for a room, and why should you trust it?
* Doesn't work well with E2E; you'd have to cut the AS into every room
* the AS would end up subscribing to every room anyway.
2. Have a generic preview API (nothing to do with Matrix) that provides a previewing service:
* Pros:
* Simple and flexible; can be used by any clients at any point
* Cons:
* If each HS provides one of these independently, all the HSes in a room may needlessly DoS the target URI
* We need somewhere to store the URL metadata rather than just using Matrix itself
* We can't piggyback on matrix to distribute the metadata between HSes.
3. Make the synapse of the sending user responsible for spidering the URL and inserting an event asynchronously which describes the metadata.
* Pros:
* Works transparently for all clients
* Piggy-backs nicely on using Matrix for distributing the metadata.
* No confusion as to which AS
* Cons:
* Doesn't work with E2E
* We might want to decouple the implementation of the spider from the HS, given spider behaviour can be quite complicated and evolve much more rapidly than the HS. It's more like a bot than a core part of the server.
4. Make the sending client use the preview API and insert the event itself when successful.
* Pros:
* Works well with E2E
* No custom server functionality
* Lets the client customise the preview that they send (like on FB)
* Cons:
* Entirely specific to the sending client, whereas it'd be nice if /any/ URL was correctly previewed if clients support it.
5. Have the option of specifying a shared (centralised) previewing service used by a room, to avoid all the different HSes in the room DoSing the target.
Best solution is probably a combination of both 2 and 4.
* Sending clients do their best to create and send a preview at the point of sending the message, perhaps delaying the message until the preview is computed? (This also lets the user validate the preview before sending)
* Receiving clients have the option of going and creating their own preview if one doesn't arrive soon enough (or if the original sender didn't create one)
This is a bit magical though in that the preview could come from two entirely different sources - the sending HS or your local one. However, this can always be exposed to users: "Generate your own URL previews if none are available?"
This is tantamount also to senders calculating their own thumbnails for sending in advance of the main content - we are trusting the sender not to lie about the content in the thumbnail. Whereas currently thumbnails are calculated by the receiving homeserver to avoid this attack.
However, this kind of phishing attack does exist whether we let senders pick their thumbnails or not, in that a malicious sender can send normal text messages around the attachment claiming it to be legitimate. We could rely on (future) reputation/abuse management to punish users who phish (be it with bogus metadata or bogus descriptions). Bogus metadata is particularly bad though, especially if it's avoidable.
As a first cut, let's do #2 and have the receiver hit the API to calculate its own previews (as it does currently for image thumbnails). We can then extend/optimise this to option 4 as a special extra if needed.
API
---
```
GET /_matrix/media/r0/preview_url?url=http://wherever.com
200 OK
{
"og:type" : "article"
"og:url" : "https://twitter.com/matrixdotorg/status/684074366691356672"
"og:title" : "Matrix on Twitter"
"og:image" : "https://pbs.twimg.com/profile_images/500400952029888512/yI0qtFi7_400x400.png"
"og:description" : "“Synapse 0.12 is out! Lots of polishing, performance &amp;amp; bugfixes: /sync API, /r0 prefix, fulltext search, 3PID invites https://t.co/5alhXLLEGP”"
"og:site_name" : "Twitter"
}
```
* Downloads the URL
* If HTML, just stores it in RAM and parses it for OG meta tags
* Download any media OG meta tags to the media repo, and refer to them in the OG via mxc:// URIs.
* If a media filetype we know we can thumbnail: store it on disk, and hand it to the thumbnailer. Generate OG meta tags from the thumbnailer contents.
* Otherwise, don't bother downloading further.

317
attic/drafts/use_cases.rst
Unescape Escape View File

@ -1,317 +0,0 @@
General UI/UX requirements:
===========================
- Live updates
- No flicker:
* Sending message (local echo)
* Receiving images (encoding w/h)
* Scrollback
* Resolving display names (from user ID)
- Fast startup times
- Fast "opening room" times (esp. when clicking through from a notification)
- Low latency file transfer.
Use cases
---------
- #1: Lightweight IM client (no perm storage) - e.g. Web client
- #2: Bug tracking software
- #3: Forum
- #4: Google + style communities
- #5: Email style threading
- #6: Multi-column threaded IM
- #7: Mobile IM client (perm storage)
- #8: MIDI client
- #9: Animatrix client
- #10: Unity object trees
- #11: Social Network ("Walls", PMs, groups)
- #12: Minecraft-clone
- #13: Global 'Like' widget, which links through to a room.
#1 Web client UI
================
Model::
Rooms ----< Messages
- name - type (call/image)
- topic
Home Screen
What's visible:
- Recent chats ordered by timestamp of latest event (with # users)
- Your own display name, user ID and avatar url
- A searchable list of public rooms (with # users and alias + room name + room topic)
What you can do:
- Create a room (public/private, with alias)
- Join a room from alias
- Message a user (with user ID)
- Leave a recent room
- Open a room
- Open a chat history link.
- Search for a public room.
Chat Screen
What's visible:
- Enough scrollback to fill a "screen full" of content.
- Each message: timestamp, user ID, display name at the time the message was
sent, avatar URL at the time the message was sent, whether it was a bing message
or not.
- User list: for each user: presence, current avatar url in the room, current
display name in the room, power level, ordered by when they were last speaking.
- Recents list: (same as Home Screen)
- Room name
- Room topic
- Typing notifications
- Desktop/Push Notifications for messages
What you can do:
- Invite a user
- Kick a user
- Ban/Unban a user
- Leave the room
- Send a message (image/text/emote)
- Change someone's power level
- Change your own display name
- Accept an incoming call
- Make an outgoing call
- Get older messages by scrolling up (scrollback)
- Redact a message
- Resend a message which was not sent
Message sending:
- Immediate local echo
- Queue up messages which haven't been sent yet
- Reordering local echo to where it actually happened
VoIP:
- One entry in your display for a call (which may contain duration, type, status)
- Glare resolution
Scrollback:
- Display in reverse chronological order by the originating server's timestamp
- Terminates at the start of the room (which then makes it impossible to request
more scrollback)
Local storage:
- Driven by desire for fast startup times and minimal network traffic
- Display messages from storage and from the network without any gaps in messages.
- Persist scrollback if possible: Scrollback from storage first then from the
network.
Notifications:
- Receive notifications for rooms you're interested in (explicitly or from a default)
- Maybe per device.
- Maybe depending on presence (e.g. idle)
- Maybe depending on message volume
- Maybe depending on room config options.
Message contents:
- images
- video
- rich text
- audio
- arbitrary files
- location
- vcards (potentially)
Chat History Screen
What's visible:
- The linked message and enough scrollback to fill a "screen full" of content.
- Each message: timestamp, user ID, display name at the time the message was
sent, avatar URL at the time the message was sent, whether it was a bing message
or not.
- The historical user list. *TODO: Is this taken at the linked message, or at
wherever the user has scrolled to?*
What you can do:
- Get older messages by scrolling up (scrollback)
- Get newer messages by scrolling down
Public Room Search Screen
What's visible:
- The current search text.
- The homeserver being searched (defaults to the HS the client is connected to).
- The results of the current search with enough results to fill the screen
with # users and alias + room name + room topic.
What you can do:
- Change what you are searching for.
- Change the server that's being searched.
- Scroll down to get more search results.
User screen
What's visible:
- Display name
- Avatar
- User ID
What you can do:
- Start a chat with the user
#2 Bug tracking UI
==================
Model::
Projects ----< Issues ---< Comments
- key - summary - user
- name - ID - message
SYN SYN-52 Fix it nooow!
Landing page
What's visible:
- Issues assigned to me
- Issues I'm watching
- Recent activity on other issues (not refined to me)
- List of projects
What you can do:
- View an issue
- Create an issue
- Sort issues
- View a user
- View a project
- Search for issues (by name, time, priority, description contents, reporter, etc...)
Issue page
What's visible:
- Summary of issue
- Issue key
- Project affected
- Description
- Comments
- Priority, labels, type, purpose, etc..
- Reporter/assignee
- Creation and last updated times
- History of issue changes
What you can do:
- Comment on issue
- Change issue info (labels, type, purpose, etc..)
- Open/Close/Resolve the issue
- Edit the issue
- Watch/Unwatch the issue
#3 Forum UI
===========
Model::
Forum ----< Boards ----< Threads ----< Messages
- Matrix - Dev - HALP! - please halp!
Main page
What's visible:
- Categories (containing boards)
- Boards (with names and # posts and tagline and latest post)
What you can do:
- View a board
- View the latest message on a board
Board page
What's visible:
- Threads (titles, OP, latest post date+author, # replies, # upvotes, whether
the OP contains an image or hyperlink (small icon on title))
- Whether the thread is answered (with link to the answer)
- Pagination for posts within a thread (1,2,3,4,5...10)
- Pagination for threads within a board
- List of threads in chronological order
- Stickied threads
What you can do:
- View a user
- View a thread on a particular page
- View the latest message on a thread
- View older threads (pagination)
- Search the board
Thread page
What's visible:
- Messages in chronological order
- For each message: author, timestamp, # posts by author, avatar, registration
date, status message, message contents, # views of message
What you can do:
- Upvote the message
- Flag the message for a mod
- Reply to the message
- Subscribe to thread or message's RSS feed
- Go to previous/next thread
#4 Google+ community
====================
Model::
Community -----< Categories ----< Posts ---< Comments
Kerbal SP Mods, Help Text Text
(no title!)
Communities page
What's visible:
- List of communities
- For each community: # users, # posts, group pic, title
What you can do:
- Join a community
- View a community
Community Page
What's visible:
- Title, pic
- List of categories
- List of members with avatars (+ total #)
- Most recent posts with comments (most recent comment if >1)
What you can do:
- Join the group
- Post a post (with voting and options)
- Report abuse
- View member
- Expand comments
- Infinite scrolling
- Add a comment to a post
- Share a post
- +1 a post
#5 Email style threading
========================
Chat Screen
What's visible:
- Enough scrollback to fill a "screen full" of content.
- Threads:
- Initially will only display the timestamp and user ID of the *first*
message. But can expand to show the entire tree.
- Tree of messages indicating which message is a reply to which.
- Ordered by the arbitrary field (timestamp of oldest message in thread;
newest message in thread; sender id; sender display name; etc)
- Each message: timestamp, user ID, display name at the time of the message
- Room name
- Room topic
- Typing notifications
- Desktop/Push Notifications for messages
What you can do:
- Send a message in reply to another message:
- Immediate local echo, may cause messages to re-order
- Messages that haven't reached the server are queued.
- Thread is displayed where it should be in the thread order once the
message is sent.
- Start a new thread by sending a message.
#6 Multi-threaded IM
====================
Chat Screen
What's visible:
- A multi-column grid of threads from a number of chatrooms
Each concurrent thread is displayed in a different column.
The columns start and end as threads split and rejoin the main conversation
The messages for each thread are ordered by how recent they are::
Room #1 Room # 2 Room # 2
+------------+ +----------------+ Side thread.
| * Message1 | | * Root | +--------------+
| * Message2 | | * A1 -> Root | | * B1 -> Root |
+------------+ | * A2 -> A1 | | * B2 -> B1 |
| * M -> A2, B2 | +--------------+
+----------------+
- Typing notifications. Displayed within the correct thread/column.
What you can do:
- Send a message into a particular thread/column.
- Move an *existing* message into a new thread creating a new column
- Move an existing message into an existing thread, causing the threads to
reconverge (i.e. provide a route from the sidebar back into the existing
thread). This does not imply terminating the thread, which can continue
independently of the merge.

288
attic/drafts/websockets.rst
Unescape Escape View File

@ -1,288 +0,0 @@
WebSockets API
==============
Introduction
------------
This document is a proposal for a WebSockets-based client-server API. It is not
intended to replace the REST API, but rather to complement it and provide an
alternative interface for certain operations.
The primary goal is to offer a more efficient interface than the REST API: by
using a bidirectional protocol such as WebSockets we can avoid the overheads
involved in long-polling (SSL negotiation, HTTP headers, etc). In doing so we
will reduce the latency between server and client by allowing the server to
send events as soon as they arrive, rather than having to wait for a poll from
the client.
Note: This proposal got continued in a google document you can find here:
https://docs.google.com/document/d/104ClehFBgqLQbf4s-AKX2ijr8sOAxcizfcRs_atsB0g
Handshake
---------
1. Instead of calling ``/sync``, the client makes a websocket request to
``/_matrix/client/rN/stream``, passing the query parameters ``access_token``
and ``since``, and optionally ``filter`` - all of which have the same
meaning as for ``/sync``.
* The client sets the ``Sec-WebSocket-Protocol`` to ``m.json``. (Servers may
offer alternative encodings; at present only the JSON encoding is
specified but in future we will specify alternative encodings.)
#. The server returns the websocket handshake; the socket is then connected.
If the server does not return a valid websocket handshake, this indicates that
the server or an intermediate proxy does not support WebSockets. In this case,
the client should fall back to polling the ``/sync`` REST endpoint.
Example
~~~~~~~
Client request:
.. code:: http
GET /_matrix/client/v2_alpha/stream?access_token=123456&since=s72594_4483_1934 HTTP/1.1
Host: matrix.org
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Protocol: m.json
Sec-WebSocket-Version: 13
Origin: https://matrix.org
Server response:
.. code:: http
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: m.json
Update Notifications
--------------------
Once the socket is connected, the server begins streaming updates over the
websocket. The server sends Update notifications about new messages or state
changes. To make it easy for clients to parse, Update notifications have the
same structure as the response to ``/sync``: an object with the following
members:
============= ========== ===================================================
Key Type Description
============= ========== ===================================================
next_batch string The batch token to supply in the ``since`` param of
the next /sync request. This is not required for
streaming of events over the WebSocket, but is
provided so that clients can reconnect if the
socket is disconnected.
presence Presence The updates to the presence status of other users.
rooms Rooms Updates to rooms.
============= ========== ===================================================
Example
~~~~~~~
Message from the server:
.. code:: json
{
"next_batch": "s72595_4483_1934",
"presence": {
"events": []
},
"rooms": {
"join": {},
"invite": {},
"leave": {}
}
}
Client-initiated operations
---------------------------
The client can perform certain operations by sending a websocket message to
the server. Such a "Request" message should be a JSON-encoded object with
the following members:
============= ========== ===================================================
Key Type Description
============= ========== ===================================================
id string A unique identifier for this request
method string Specifies the name of the operation to be
performed; see below for available operations
param object The parameters for the requested operation.
============= ========== ===================================================
The server responds to a client Request with a Response message. This is a
JSON-encoded object with the following members:
============= ========== ===================================================
Key Type Description
============= ========== ===================================================
id string The same as the value in the corresponding Request
object. The presence of the ``id`` field
distinguishes a Response message from an Update
notification.
result object On success, the results of the request.
error object On error, an object giving the resons for the
error. This has the same structure as the "standard
error response" for the Matrix API: an object with
the fields ``errcode`` and ``error``.
============= ========== ===================================================
Request methods
~~~~~~~~~~~~~~~
It is not intended that all operations which are available via the REST API
will be available via the WebSockets API, but a few simple, common operations
will be exposed. The initial operations will be as follows.
``ping``
^^^^^^^^
This is a no-op which clients may use to keep their connection alive.
The request ``params`` and the response ``result`` should be empty.
``send``
^^^^^^^^
Send a message event to a room. The parameters are as follows:
============= ========== ===================================================
Parameter Type Description
============= ========== ===================================================
room_id string **Required.** The room to send the event to
event_type string **Required.** The type of event to send.
content object **Required.** The content of the event.
============= ========== ===================================================
The result is as follows:
============= ========== ===================================================
Key Type Description
============= ========== ===================================================
event_id string A unique identifier for the event.
============= ========== ===================================================
The ``id`` from the Request message is used as the transaction ID by the
server.
``state``
^^^^^^^^^
Update the state on a room.
============= ========== ===================================================
Parameter Type Description
============= ========== ===================================================
room_id string **Required.** The room to set the state in
event_type string **Required.** The type of event to send.
state_key string **Required.** The state_key for the state to send.
content object **Required.** The content of the event.
============= ========== ===================================================
The result is as follows:
============= ========== ===================================================
Key Type Description
============= ========== ===================================================
event_id string A unique identifier for the event.
============= ========== ===================================================
Example
~~~~~~~
Client request:
.. code:: json
{
"id": "12345",
"method": "send",
"params": {
"room_id": "!d41d8cd:matrix.org",
"event_type": "m.room.message",
"content": {
"msgtype": "m.text",
"body": "hello"
}
}
}
Server response:
.. code:: json
{
"id": "12345",
"result": {
"event_id": "$66697273743031:matrix.org"
}
}
Alternative server response, in case of error:
.. code:: json
{
"id": "12345",
"error": {
"errcode": "M_MISSING_PARAM",
"error": "Missing parameter: event_type"
}
}
Rationale
---------
Alternatives to WebSockets include HTTP/2, CoAP, and simply rolling our own
protocol over raw TCP sockets. However, the need to implement browser-based
clients essentially reduces our choice to WebSockets. HTTP/2 streams will
probably provide an interesting alternative in the future, but current browsers
do not appear to give javascript applications low-level access to the protocol.
Concerning the continued use of the JSON encoding: we prefer to focus on the
transition to WebSockets initially. Replacing JSON with a compact
representation such as CBOR, MessagePack, or even just compressed JSON will be
a likely extension for the future. The support for negotiation of subprotocols
within WebSockets should make this a simple transition once time permits.
The number of methods available for client requests is deliberately limited, as
each method requires code to be written to map it onto the equivalent REST
implementation. Some REST methods - for instance, user registration and login -
would be pointless to expose via WebSockets. It is likely, however, that we
will increate the number of methods available via the WebSockets API as it
becomes clear which would be most useful.
Open questions
--------------
Throttling
~~~~~~~~~~
At least in v2 sync, clients are inherently self-throttling - if they do not
poll quickly enough, events will be dropped from the next result. This proposal
raises the possibility that events will be produced more quickly than they can
be sent to the client; backlogs will build up on the server and/or in the
intermediate network, which will not only lead to high latency on events being
delivered, but will lead to responses to client requests also being delayed.
We may need to implement some sort of throttling mechanism by which the server
can start to drop events. The difficulty is in knowing when to start dropping
events. A few ideas:
* Use websocket pings to measure the RTT; if it starts to increase, start
dropping events. But this requires knowledge of the base RTT, and a useful
model of what constitutes an excessive increase.
* Have the client acknowledge each batch of events, and use a window to ensure
the number of outstanding batches is limited. This is annoying as it requires
the client to have to acknowledge batches - and it's not clear what the right
window size is: we want a big window for long fat networks (think of mobile
clients), but a small one for one with lower latency.
* Start dropping events if the server's TCP buffer starts filling up. This has
the advantage of delegating the congestion-detection to TCP (which already
has a number of algorithms to deal with it, to greater or lesser
effectiveness), but relies on homeservers being hosted on OSes which use
sensible TCP congestion-avoidance algorithms, and more critically, an ability
to read the fill level of the TCP send buffer.

343
attic/v1_registration_login.rst
Unescape Escape View File

@ -1,343 +0,0 @@
Registration and Login
----------------------
Clients must register with a homeserver in order to use Matrix. After
registering, the client will be given an access token which must be used in ALL
requests to that homeserver as a query parameter 'access_token'.
If the client has already registered, they need to be able to login to their
account. The homeserver may provide many different ways of logging in, such as
user/password auth, login via a social network (OAuth2), login by confirming a
token sent to their email address, etc. This specification does not define how
homeservers should authorise their users who want to login to their existing
accounts, but instead defines the standard interface which implementations
should follow so that ANY client can login to ANY homeserver. Clients login
using the |login|_ API. Clients register using the |register|_ API.
Registration follows the same general procedure as login, but the path requests
are sent to and the details contained in them are different.
In both registration and login cases, the process takes the form of one or more
stages, where at each stage the client submits a set of data for a given stage
type and awaits a response from the server, which will either be a final
success or a request to perform an additional stage. This exchange continues
until the final success.
In order to determine up-front what the server's requirements are, the client
can request from the server a complete description of all of its acceptable
flows of the registration or login process. It can then inspect the list of
returned flows looking for one for which it believes it can complete all of the
required stages, and perform it. As each homeserver may have different ways of
logging in, the client needs to know how they should login. All distinct login
stages MUST have a corresponding ``type``. A ``type`` is a namespaced string
which details the mechanism for logging in.
A client may be able to login via multiple valid login flows, and should choose
a single flow when logging in. A flow is a series of login stages. The home
server MUST respond with all the valid login flows when requested by a simple
``GET`` request directly to the ``/login`` or ``/register`` paths::
{
"flows": [
{
"type": "<login type1a>",
"stages": [ "<login type 1a>", "<login type 1b>" ]
},
{
"type": "<login type2a>",
"stages": [ "<login type 2a>", "<login type 2b>" ]
},
{
"type": "<login type3>"
}
]
}
The client can now select which flow it wishes to use, and begin making
``POST`` requests to the ``/login`` or ``/register`` paths with JSON body
content containing the name of the stage as the ``type`` key, along with
whatever additional parameters are required for that login or registration type
(see below). After the flow is completed, the client's fully-qualified user
ID and a new access token MUST be returned::
{
"user_id": "@user:matrix.org",
"access_token": "abcdef0123456789"
}
The ``user_id`` key is particularly useful if the homeserver wishes to support
localpart entry of usernames (e.g. "user" rather than "@user:matrix.org"), as
the client may not be able to determine its ``user_id`` in this case.
If the flow has multiple stages to it, the homeserver may wish to create a
session to store context between requests. If a homeserver responds with a
``session`` key to a request, clients MUST submit it in subsequent requests
until the flow is completed::
{
"session": "<session id>"
}
This specification defines the following login types:
- ``m.login.password``
- ``m.login.oauth2``
- ``m.login.email.code``
- ``m.login.email.url``
- ``m.login.email.identity``
Password-based
~~~~~~~~~~~~~~
:Type:
``m.login.password``
:Description:
Login is supported via a username and password.
To respond to this type, reply with::
{
"type": "m.login.password",
"user": "<user_id or user localpart>",
"password": "<password>"
}
The homeserver MUST respond with either new credentials, the next stage of the
login process, or a standard error response.
Captcha-based
~~~~~~~~~~~~~
:Type:
``m.login.recaptcha``
:Description:
Login is supported by responding to a captcha (in the case of the Synapse
implementation, Google's Recaptcha library is used).
To respond to this type, reply with::
{
"type": "m.login.recaptcha",
"challenge": "<challenge token>",
"response": "<user-entered text>"
}
.. NOTE::
In Synapse, the Recaptcha parameters can be obtained in Javascript by calling:
Recaptcha.get_challenge();
Recaptcha.get_response();
The homeserver MUST respond with either new credentials, the next stage of the
login process, or a standard error response.
OAuth2-based
~~~~~~~~~~~~
:Type:
``m.login.oauth2``
:Description:
Login is supported via OAuth2 URLs. This login consists of multiple requests.
To respond to this type, reply with::
{
"type": "m.login.oauth2",
"user": "<user_id or user localpart>"
}
The server MUST respond with::
{
"uri": <Authorization Request URI OR service selection URI>
}
The homeserver acts as a 'confidential' client for the purposes of OAuth2. If
the uri is a ``sevice selection URI``, it MUST point to a webpage which prompts
the user to choose which service to authorize with. On selection of a service,
this MUST link through to an ``Authorization Request URI``. If there is only 1
service which the homeserver accepts when logging in, this indirection can be
skipped and the "uri" key can be the ``Authorization Request URI``.
The client then visits the ``Authorization Request URI``, which then shows the
OAuth2 Allow/Deny prompt. Hitting 'Allow' returns the ``redirect URI`` with the
auth code. Homeservers can choose any path for the ``redirect URI``. The
client should visit the ``redirect URI``, which will then finish the OAuth2
login process, granting the homeserver an access token for the chosen service.
When the homeserver gets this access token, it verifies that the cilent has
authorised with the 3rd party, and can now complete the login. The OAuth2
``redirect URI`` (with auth code) MUST respond with either new credentials, the
next stage of the login process, or a standard error response.
For example, if a homeserver accepts OAuth2 from Google, it would return the
Authorization Request URI for Google::
{
"uri": "https://accounts.google.com/o/oauth2/auth?response_type=code&
client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos"
}
The client then visits this URI and authorizes the homeserver. The client then
visits the REDIRECT_URI with the auth code= query parameter which returns::
{
"user_id": "@user:matrix.org",
"access_token": "0123456789abcdef"
}
Email-based (code)
~~~~~~~~~~~~~~~~~~
:Type:
``m.login.email.code``
:Description:
Login is supported by typing in a code which is sent in an email. This login
consists of multiple requests.
To respond to this type, reply with::
{
"type": "m.login.email.code",
"user": "<user_id or user localpart>",
"email": "<email address>"
}
After validating the email address, the homeserver MUST send an email
containing an authentication code and return::
{
"type": "m.login.email.code",
"session": "<session id>"
}
The second request in this login stage involves sending this authentication
code::
{
"type": "m.login.email.code",
"session": "<session id>",
"code": "<code in email sent>"
}
The homeserver MUST respond to this with either new credentials, the next
stage of the login process, or a standard error response.
Email-based (url)
~~~~~~~~~~~~~~~~~
:Type:
``m.login.email.url``
:Description:
Login is supported by clicking on a URL in an email. This login consists of
multiple requests.
To respond to this type, reply with::
{
"type": "m.login.email.url",
"user": "<user_id or user localpart>",
"email": "<email address>"
}
After validating the email address, the homeserver MUST send an email
containing an authentication URL and return::
{
"type": "m.login.email.url",
"session": "<session id>"
}
The email contains a URL which must be clicked. After it has been clicked, the
client should perform another request::
{
"type": "m.login.email.url",
"session": "<session id>"
}
The homeserver MUST respond to this with either new credentials, the next
stage of the login process, or a standard error response.
A common client implementation will be to periodically poll until the link is
clicked. If the link has not been visited yet, a standard error response with
an errcode of ``M_LOGIN_EMAIL_URL_NOT_YET`` should be returned.
Email-based (identity server)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:Type:
``m.login.email.identity``
:Description:
Login is supported by authorising an email address with an identity server.
Prior to submitting this, the client should authenticate with an identity
server. After authenticating, the session information should be submitted to
the homeserver.
To respond to this type, reply with::
{
"type": "m.login.email.identity",
"threepidCreds": [
{
"sid": "<identity server session id>",
"clientSecret": "<identity server client secret>",
"idServer": "<url of identity server authed with, e.g. 'matrix.org:8090'>"
}
]
}
N-Factor Authentication
~~~~~~~~~~~~~~~~~~~~~~~
Multiple login stages can be combined to create N-factor authentication during
login.
This can be achieved by responding with the ``next`` login type on completion
of a previous login stage::
{
"next": "<next login type>"
}
If a homeserver implements N-factor authentication, it MUST respond with all
``stages`` when initially queried for their login requirements::
{
"type": "<1st login type>",
"stages": [ <1st login type>, <2nd login type>, ... , <Nth login type> ]
}
This can be represented conceptually as::
_______________________
| Login Stage 1 |
| type: "<login type1>" |
| ___________________ |
| |_Request_1_________| | <-- Returns "session" key which is used throughout.
| ___________________ |
| |_Request_2_________| | <-- Returns a "next" value of "login type2"
|_______________________|
|
|
_________V_____________
| Login Stage 2 |
| type: "<login type2>" |
| ___________________ |
| |_Request_1_________| |
| ___________________ |
| |_Request_2_________| |
| ___________________ |
| |_Request_3_________| | <-- Returns a "next" value of "login type3"
|_______________________|
|
|
_________V_____________
| Login Stage 3 |
| type: "<login type3>" |
| ___________________ |
| |_Request_1_________| | <-- Returns user credentials
|_______________________|
Fallback
~~~~~~~~
Clients cannot be expected to be able to know how to process every single login
type. If a client determines it does not know how to handle a given login type,
it should request a login fallback page::
GET matrix/client/api/v1/login/fallback
This MUST return an HTML page which can perform the entire login process.

15
changelogs/README.md
Unescape Escape View File

@ -1,15 +0,0 @@
# Changelogs
[Towncrier](https://github.com/hawkowl/towncrier) is used to manage the changelog and
keep it up to date. Because of this, updating a changelog is really easy.
## Generating the changelog
Please see the [release docs](../meta/releasing.md) for more information.
## Creating a new changelog
There are a few places you'll have to update:
* `/layouts/shortcodes/changelog/changelog-changes.html` to account for the new changelog.
* `/scripts/generate-changelog.sh` to render the changelog for releases.
* Supporting documentation such as the contributing guidelines.

1
changelogs/appendices/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/appendices/newsfragments/3171.feature
Unescape Escape View File

@ -1 +0,0 @@
Describe "Common Namespaced Identifier Grammar" as per [MSC2758](https://github.com/matrix-org/matrix-doc/pull/2758).

1
changelogs/appendices/newsfragments/3608.feature
Unescape Escape View File

@ -1 +0,0 @@
Describe the `matrix:` URI scheme as per [MSC2312](https://github.com/matrix-org/matrix-doc/pull/2312).

1
changelogs/application_service/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/client_server/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/client_server/newsfragments/3324.feature
Unescape Escape View File

@ -1 +0,0 @@
Extend `/_matrix/client/r0/login` to accept a `m.login.appservice`, as per [MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778).

1
changelogs/client_server/newsfragments/3387.feature
Unescape Escape View File

@ -1 +0,0 @@
Add support for `restricted` rooms as per [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083), [MSC3289](https://github.com/matrix-org/matrix-doc/pull/3289), and [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375).

1
changelogs/client_server/newsfragments/3481.clarification
Unescape Escape View File

@ -1 +0,0 @@
Make `AesHmacSha2KeyDescription` consistent with `KeyDescription` in marking `name` as optional.

1
changelogs/client_server/newsfragments/3482.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3492.clarification
Unescape Escape View File

@ -1 +0,0 @@
Explicitly mention RFC5870 in the definition of `m.location` events

1
changelogs/client_server/newsfragments/3495.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3509.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3524.breaking
Unescape Escape View File

@ -1 +0,0 @@
The `prev_content` field is now returned inside the `unsigned` property of events, rather than at the top level, as per [MSC3442](https://github.com/matrix-org/matrix-doc/pull/3442).

1
changelogs/client_server/newsfragments/3530.clarification
Unescape Escape View File

@ -1 +0,0 @@
Add `403 M_FORBIDDEN` error code to `/profile/{userId}` as per [MSC3550](https://github.com/matrix-org/matrix-doc/pull/3550).

1
changelogs/client_server/newsfragments/3535.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3543.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify the description of the `/sync` API, including a fix to an ASCII art diagram.

1
changelogs/client_server/newsfragments/3562.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify that `base_url` in client `well_known` may or may not include trailing slash.

1
changelogs/client_server/newsfragments/3573.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify which signature to check when decrypting `m.olm.v1.curve25519-aes-sha2` messages.

1
changelogs/client_server/newsfragments/3591.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3601.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3605.feature.1
Unescape Escape View File

@ -1 +0,0 @@
Add `is_guest` to `/account/whoami` as per [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069).

1
changelogs/client_server/newsfragments/3605.feature.2
Unescape Escape View File

@ -1 +0,0 @@
Expand guest access to sending any room event and state event as per [MSC3419](https://github.com/matrix-org/matrix-doc/pull/3419).

1
changelogs/client_server/newsfragments/3606.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify what "Stripped State" is and what purpose it serves, as per [MSC3173](https://github.com/matrix-org/matrix-doc/pull/3173).

1
changelogs/client_server/newsfragments/3610.feature
Unescape Escape View File

@ -1 +0,0 @@
Add Spaces and room types as per [MSC1772](https://github.com/matrix-org/matrix-doc/pull/1772) and [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

1
changelogs/client_server/newsfragments/3610.new
Unescape Escape View File

@ -1 +0,0 @@
Add the Space Hierarchy API (`GET /_matrix/client/v1/rooms/{roomId}/hierarchy`) as per [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

1
changelogs/client_server/newsfragments/3611.clarification
Unescape Escape View File

@ -1 +0,0 @@
Fix various typos throughout the specification.

1
changelogs/client_server/newsfragments/3614.feature
Unescape Escape View File

@ -1 +0,0 @@
Add new `m.set_displayname`, `m.set_avatar_url`, and `m.3pid_changes` capabilities as per [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283).

1
changelogs/client_server/newsfragments/3615.feature
Unescape Escape View File

@ -1 +0,0 @@
Add support for fallback keys (optional keys used once one-time keys run out), as per [MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732).

1
changelogs/client_server/newsfragments/3616.feature
Unescape Escape View File

@ -1 +0,0 @@
Add token-authenticated registration support as per [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

1
changelogs/client_server/newsfragments/3616.new
Unescape Escape View File

@ -1 +0,0 @@
Add `/_matrix/client/v1/register/m.login.registration_token/validity` as per [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).

1
changelogs/client_server/newsfragments/3624.breaking
Unescape Escape View File

@ -1 +0,0 @@
The `aliases` property from the chunks returned by `/publicRooms`, as per [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432).

1
changelogs/client_server/newsfragments/3636.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify how to interpret missing one-time key counts.

1
changelogs/client_server/newsfragments/3652.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify that group mentions are no longer in the specification.

16
changelogs/header.md
Unescape Escape View File

@ -1,16 +0,0 @@
<!--
This is a header file for the generated changelog.
Variables:
VERSION = Replaced by the version number (eg: v1.2)
DATE = Replaced by the date (eg: April 01, 2021)
-->
## VERSION
<table class="release-info">
<tr><th>Git commit</th><td><a href="https://github.com/matrix-org/matrix-doc/tree/VERSION">https://github.com/matrix-org/matrix-doc/tree/VERSION</a></td>
<tr><th>Release date</th><td>DATE</td>
</table>
<!-- Intentionally blank line to ensure headers work in the concatenated changelog -->

1
changelogs/identity_service/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/identity_service/newsfragments/3610.feature
Unescape Escape View File

@ -1 +0,0 @@
Add the `room_type` to stored invites as per [MSC3288](https://github.com/matrix-org/matrix-doc/pull/3288).

26
changelogs/legacy/application_service.rst
Unescape Escape View File

@ -1,26 +0,0 @@
r0.1.2
======
Spec Clarifications
-------------------
- Clearer wording for the legacy routes section. (`#2160 <https://github.com/matrix-org/matrix-doc/issues/2160>`_)
r0.1.1
======
Spec Clarifications
-------------------
- Change examples to use example.org instead of a real domain. (`#1650 <https://github.com/matrix-org/matrix-doc/issues/1650>`_)
- Add missing definition for how appservices verify requests came from a homeserver. (`#2037 <https://github.com/matrix-org/matrix-doc/issues/2037>`_)
r0.1.0
======
This is the first release of the Application Service specification. It
includes support for application services being able to interact with
homeservers and bridge third party networks, such as IRC, over to Matrix
in a standard and accessible way.

575
changelogs/legacy/client_server.rst
Unescape Escape View File

@ -1,575 +0,0 @@
r0.6.1
======
New Endpoints
-------------
- Added ``/rooms/{roomId}/aliases`` for retrieving local aliases for a room. (`#2562 <https://github.com/matrix-org/matrix-doc/issues/2562>`_)
Backwards Compatible Changes
----------------------------
- Added data structures for defining moderation policies in rooms per `MSC2313 <https://github.com/matrix-org/matrix-doc/pull/2313>`_. (`#2434 <https://github.com/matrix-org/matrix-doc/issues/2434>`_)
- Optionally invalidate other access tokens during password modification per `MSC2457 <https://github.com/matrix-org/matrix-doc/pull/2457>`_. (`#2523 <https://github.com/matrix-org/matrix-doc/issues/2523>`_)
- Add User-Interactive Authentication for SSO-backed homeserver per `MSC2454 <https://github.com/matrix-org/matrix-doc/pull/2454>`_. (`#2532 <https://github.com/matrix-org/matrix-doc/issues/2532>`_)
- Add soft-logout support per `MSC1466 <https://github.com/matrix-org/matrix-doc/issues/1466>`_. (`#2546 <https://github.com/matrix-org/matrix-doc/issues/2546>`_)
- Replaced legacy room alias handling with a more sustainable solution per `MSC2432 <https://github.com/matrix-org/matrix-doc/pull/2432>`_. (`#2562 <https://github.com/matrix-org/matrix-doc/issues/2562>`_)
Spec Clarifications
-------------------
- List available enum values for the room versions capability. (`#2245 <https://github.com/matrix-org/matrix-doc/issues/2245>`_)
- Fix various spelling errors throughout the specification. (`#2351 <https://github.com/matrix-org/matrix-doc/issues/2351>`_, `#2415 <https://github.com/matrix-org/matrix-doc/issues/2415>`_, `#2453 <https://github.com/matrix-org/matrix-doc/issues/2453>`_, `#2524 <https://github.com/matrix-org/matrix-doc/issues/2524>`_, `#2553 <https://github.com/matrix-org/matrix-doc/issues/2553>`_, `#2569 <https://github.com/matrix-org/matrix-doc/issues/2569>`_)
- Minor clarifications to token-based User-Interactive Authentication. (`#2369 <https://github.com/matrix-org/matrix-doc/issues/2369>`_)
- Minor clarification for what the user directory searches. (`#2381 <https://github.com/matrix-org/matrix-doc/issues/2381>`_)
- Fix key export format example to match the specification. (`#2430 <https://github.com/matrix-org/matrix-doc/issues/2430>`_)
- Clarify the IV data type for encrypted files. (`#2492 <https://github.com/matrix-org/matrix-doc/issues/2492>`_)
- Fix the ``.m.rule.contains_user_name`` default push rule to set the highlight tweak. (`#2519 <https://github.com/matrix-org/matrix-doc/issues/2519>`_)
- Clarify that an ``event_id`` is returned when sending events. (`#2525 <https://github.com/matrix-org/matrix-doc/issues/2525>`_)
- Fix some numbers in the specification to match their explanation text. (`#2554 <https://github.com/matrix-org/matrix-doc/issues/2554>`_)
- Move redaction algorithm into the room version specifications. (`#2563 <https://github.com/matrix-org/matrix-doc/issues/2563>`_)
- Clarify signature object structures for encryption. (`#2566 <https://github.com/matrix-org/matrix-doc/issues/2566>`_)
- Clarify which events are created as part of ``/createRoom``. (`#2571 <https://github.com/matrix-org/matrix-doc/issues/2571>`_)
- Remove claims that the homeserver is exclusively responsible for profile information in membership events. (`#2574 <https://github.com/matrix-org/matrix-doc/issues/2574>`_)
r0.6.0
======
Breaking Changes
----------------
- Add ``id_access_token`` as a required request parameter to a few endpoints which require an ``id_server`` parameter as part of `MSC2140 <https://github.com/matrix-org/matrix-doc/pull/2140>`_. (`#2255 <https://github.com/matrix-org/matrix-doc/issues/2255>`_)
New Endpoints
-------------
- Add ``POST /account/3pid/unbind`` for removing a 3PID from an identity server. (`#2282 <https://github.com/matrix-org/matrix-doc/issues/2282>`_)
Backwards Compatible Changes
----------------------------
- Add ``M_USER_DEACTIVATED`` error code. (`#2234 <https://github.com/matrix-org/matrix-doc/issues/2234>`_)
- Remove ``bind_msisdn`` and ``bind_email`` from ``/register`` now that the identity server's bind endpoint requires authentication. (`#2279 <https://github.com/matrix-org/matrix-doc/issues/2279>`_)
- Add ``m.identity_server`` account data for tracking the user's preferred identity server. (`#2281 <https://github.com/matrix-org/matrix-doc/issues/2281>`_)
- Deprecate ``id_server`` and make it optional in several places. (`#2310 <https://github.com/matrix-org/matrix-doc/issues/2310>`_)
Spec Clarifications
-------------------
- Add missing format fields to ``m.room.message$m.notice`` schema. (`#2125 <https://github.com/matrix-org/matrix-doc/issues/2125>`_)
- Remove "required" designation from the ``url`` field of certain ``m.room.message`` msgtypes. (`#2129 <https://github.com/matrix-org/matrix-doc/issues/2129>`_)
- Fix various typos throughout the specification. (`#2131 <https://github.com/matrix-org/matrix-doc/issues/2131>`_, `#2136 <https://github.com/matrix-org/matrix-doc/issues/2136>`_, `#2148 <https://github.com/matrix-org/matrix-doc/issues/2148>`_, `#2215 <https://github.com/matrix-org/matrix-doc/issues/2215>`_)
- Clarify the distinction between ``m.key.verification.start`` and its ``m.sas.v1`` variant. (`#2132 <https://github.com/matrix-org/matrix-doc/issues/2132>`_)
- Fix link to Olm signing specification. (`#2133 <https://github.com/matrix-org/matrix-doc/issues/2133>`_)
- Clarify the conditions for the ``.m.rule.room_one_to_one`` push rule. (`#2152 <https://github.com/matrix-org/matrix-doc/issues/2152>`_)
- Clarify the encryption algorithms supported by the device of the device keys example. (`#2157 <https://github.com/matrix-org/matrix-doc/issues/2157>`_)
- Clarify that ``/rooms/:roomId/event/:eventId`` returns a Matrix error. (`#2204 <https://github.com/matrix-org/matrix-doc/issues/2204>`_)
- Add a missing ``state_key`` check on ``.m.rule.tombstone``. (`#2223 <https://github.com/matrix-org/matrix-doc/issues/2223>`_)
- Fix the ``m.room_key_request`` ``action`` value, setting it from ``cancel_request`` to ``request_cancellation``. (`#2247 <https://github.com/matrix-org/matrix-doc/issues/2247>`_)
- Clarify that the ``submit_url`` field is without authentication. (`#2341 <https://github.com/matrix-org/matrix-doc/issues/2341>`_)
- Clarify the expected phone number format. (`#2342 <https://github.com/matrix-org/matrix-doc/issues/2342>`_)
- Clarify that clients should consider not requesting URL previews in encrypted rooms. (`#2343 <https://github.com/matrix-org/matrix-doc/issues/2343>`_)
- Add missing information on how filters are meant to work with ``/context``. (`#2344 <https://github.com/matrix-org/matrix-doc/issues/2344>`_)
- Clarify what the keys are for rooms in ``/sync``. (`#2345 <https://github.com/matrix-org/matrix-doc/issues/2345>`_)
r0.5.0
======
Breaking Changes
----------------
- Add a new ``submit_url`` field to the responses of ``/requestToken`` which older clients will not be able to handle correctly. (`#2101 <https://github.com/matrix-org/matrix-doc/issues/2101>`_)
Deprecations
------------
- Remove references to presence lists. (`#1817 <https://github.com/matrix-org/matrix-doc/issues/1817>`_)
New Endpoints
-------------
- ``GET /account_data`` routes. (`#1873 <https://github.com/matrix-org/matrix-doc/issues/1873>`_)
Backwards Compatible Changes
----------------------------
- Add megolm session export format. (`#1701 <https://github.com/matrix-org/matrix-doc/issues/1701>`_)
- Add support for advertising experimental features to clients. (`#1786 <https://github.com/matrix-org/matrix-doc/issues/1786>`_)
- Add a generic SSO login API. (`#1789 <https://github.com/matrix-org/matrix-doc/issues/1789>`_)
- Add a mechanism for servers to redirect clients to an alternative homeserver after logging in. (`#1790 <https://github.com/matrix-org/matrix-doc/issues/1790>`_)
- Add room version upgrades. (`#1791 <https://github.com/matrix-org/matrix-doc/issues/1791>`_, `#1875 <https://github.com/matrix-org/matrix-doc/issues/1875>`_)
- Support optional features by having clients query for capabilities. (`#1829 <https://github.com/matrix-org/matrix-doc/issues/1829>`_, `#1879 <https://github.com/matrix-org/matrix-doc/issues/1879>`_)
- Add ``M_RESOURCE_LIMIT_EXCEEDED`` as an error code for when homeservers exceed limits imposed on them. (`#1874 <https://github.com/matrix-org/matrix-doc/issues/1874>`_)
- Emit ``M_UNSUPPORTED_ROOM_VERSION`` error codes where applicable on ``/createRoom`` and ``/invite`` APIs. (`#1908 <https://github.com/matrix-org/matrix-doc/issues/1908>`_)
- Add a ``.m.rule.tombstone`` default push rule for room upgrade notifications. (`#2020 <https://github.com/matrix-org/matrix-doc/issues/2020>`_)
- Add support for sending server notices to clients. (`#2026 <https://github.com/matrix-org/matrix-doc/issues/2026>`_)
- Add MSISDN (phone number) support to User-Interactive Authentication. (`#2030 <https://github.com/matrix-org/matrix-doc/issues/2030>`_)
- Add the option to lazy-load room members for increased client performance. (`#2035 <https://github.com/matrix-org/matrix-doc/issues/2035>`_)
- Add ``id_server`` to ``/deactivate`` and ``/3pid/delete`` endpoints to unbind from a specific identity server. (`#2046 <https://github.com/matrix-org/matrix-doc/issues/2046>`_)
- Add support for Olm sessions becoming un-stuck. (`#2059 <https://github.com/matrix-org/matrix-doc/issues/2059>`_)
- Add interactive device verification, including a common framework for device verification. (`#2072 <https://github.com/matrix-org/matrix-doc/issues/2072>`_)
Spec Clarifications
-------------------
- Change examples to use example.org instead of a real domain. (`#1650 <https://github.com/matrix-org/matrix-doc/issues/1650>`_)
- Clarify that ``state_default`` in ``m.room.power_levels`` always defaults to 50. (`#1656 <https://github.com/matrix-org/matrix-doc/issues/1656>`_)
- Add missing ``status_msg`` to ``m.presence`` schema. (`#1744 <https://github.com/matrix-org/matrix-doc/issues/1744>`_)
- Fix various spelling mistakes throughout the specification. (`#1838 <https://github.com/matrix-org/matrix-doc/issues/1838>`_, `#1853 <https://github.com/matrix-org/matrix-doc/issues/1853>`_, `#1860 <https://github.com/matrix-org/matrix-doc/issues/1860>`_, `#1933 <https://github.com/matrix-org/matrix-doc/issues/1933>`_, `#1969 <https://github.com/matrix-org/matrix-doc/issues/1969>`_, `#1988 <https://github.com/matrix-org/matrix-doc/issues/1988>`_, `#1989 <https://github.com/matrix-org/matrix-doc/issues/1989>`_, `#1991 <https://github.com/matrix-org/matrix-doc/issues/1991>`_, `#1992 <https://github.com/matrix-org/matrix-doc/issues/1992>`_)
- Add the missing ``m.push_rules`` event schema. (`#1889 <https://github.com/matrix-org/matrix-doc/issues/1889>`_)
- Clarify how modern day local echo is meant to be solved by clients. (`#1891 <https://github.com/matrix-org/matrix-doc/issues/1891>`_)
- Clarify that ``width`` and ``height`` are required parameters on ``/_matrix/media/r0/thumbnail/{serverName}/{mediaId}``. (`#1975 <https://github.com/matrix-org/matrix-doc/issues/1975>`_)
- Clarify how ``m.login.dummy`` can be used to disambiguate login flows. (`#1999 <https://github.com/matrix-org/matrix-doc/issues/1999>`_)
- Remove ``prev_content`` from the redaction algorithm's essential keys list. (`#2016 <https://github.com/matrix-org/matrix-doc/issues/2016>`_)
- Fix the ``third_party_signed`` definitions for the join APIs. (`#2025 <https://github.com/matrix-org/matrix-doc/issues/2025>`_)
- Clarify why User Interactive Auth is used on password changes and how access tokens are handled. (`#2027 <https://github.com/matrix-org/matrix-doc/issues/2027>`_)
- Clarify that devices are deleted upon logout. (`#2028 <https://github.com/matrix-org/matrix-doc/issues/2028>`_)
- Add ``M_NOT_FOUND`` error definition for deleting room aliases. (`#2029 <https://github.com/matrix-org/matrix-doc/issues/2029>`_)
- Add missing ``reason`` to ``m.call.hangup``. (`#2031 <https://github.com/matrix-org/matrix-doc/issues/2031>`_)
- Clarify how redactions affect room state. (`#2032 <https://github.com/matrix-org/matrix-doc/issues/2032>`_)
- Clarify that ``FAIL_ERROR`` in autodiscovery is not limited to just homeservers. (`#2036 <https://github.com/matrix-org/matrix-doc/issues/2036>`_)
- Fix example ``Content-Type`` for ``/media/upload`` request. (`#2041 <https://github.com/matrix-org/matrix-doc/issues/2041>`_)
- Clarify that login flows are meant to be completed in order. (`#2042 <https://github.com/matrix-org/matrix-doc/issues/2042>`_)
- Clarify that clients should not send read receipts for their own messages. (`#2043 <https://github.com/matrix-org/matrix-doc/issues/2043>`_)
- Use consistent examples of events throughout the specification. (`#2051 <https://github.com/matrix-org/matrix-doc/issues/2051>`_)
- Clarify which push rule condition kinds exist. (`#2052 <https://github.com/matrix-org/matrix-doc/issues/2052>`_)
- Clarify the required fields on ``m.file`` (and similar) messages. (`#2053 <https://github.com/matrix-org/matrix-doc/issues/2053>`_)
- Clarify that User-Interactive Authentication stages cannot be attempted more than once. (`#2054 <https://github.com/matrix-org/matrix-doc/issues/2054>`_)
- Clarify which parameters apply in what scenarios on ``/register``. (`#2055 <https://github.com/matrix-org/matrix-doc/issues/2055>`_)
- Clarify how to interpret changes of ``membership`` over time. (`#2056 <https://github.com/matrix-org/matrix-doc/issues/2056>`_)
- Clarify exactly what invite_room_state consists of. (`#2067 <https://github.com/matrix-org/matrix-doc/issues/2067>`_)
- Clarify how the content repository works, and what it is used for. (`#2068 <https://github.com/matrix-org/matrix-doc/issues/2068>`_)
- Clarify the order events in chunk are returned in for ``/messages``. (`#2069 <https://github.com/matrix-org/matrix-doc/issues/2069>`_)
- Clarify the key object definition for the key management API. (`#2083 <https://github.com/matrix-org/matrix-doc/issues/2083>`_)
- Reorganize information about events into a common section. (`#2087 <https://github.com/matrix-org/matrix-doc/issues/2087>`_)
- De-duplicate ``/state/<event_type>`` endpoints, clarifying that the ``<state_key>`` is optional. (`#2088 <https://github.com/matrix-org/matrix-doc/issues/2088>`_)
- Clarify when and where CORS headers should be returned. (`#2089 <https://github.com/matrix-org/matrix-doc/issues/2089>`_)
- Clarify when authorization and rate-limiting are not applicable. (`#2090 <https://github.com/matrix-org/matrix-doc/issues/2090>`_)
- Clarify that ``/register`` must produce valid Matrix User IDs. (`#2091 <https://github.com/matrix-org/matrix-doc/issues/2091>`_)
- Clarify how ``unread_notifications`` is calculated. (`#2097 <https://github.com/matrix-org/matrix-doc/issues/2097>`_)
- Clarify what a "module" is and update feature profiles for clients. (`#2098 <https://github.com/matrix-org/matrix-doc/issues/2098>`_)
r0.4.0
======
New Endpoints
-------------
- ``POST /user_directory/search`` (`#1096 <https://github.com/matrix-org/matrix-doc/issues/1096>`_)
- ``GET /rooms/{roomId}/event/{eventId}`` (`#1110 <https://github.com/matrix-org/matrix-doc/issues/1110>`_)
- ``POST /delete_devices`` (`#1239 <https://github.com/matrix-org/matrix-doc/issues/1239>`_)
- ``GET /thirdparty/*`` Endpoints (`#1353 <https://github.com/matrix-org/matrix-doc/issues/1353>`_)
- ``POST /account/3pid/msisdn/requestToken``, ``POST /register/msisdn/requestToken``, and ``POST /account/password/msisdn/requestToken`` (`#1507 <https://github.com/matrix-org/matrix-doc/issues/1507>`_)
- ``POST /account/3pid/delete`` (`#1567 <https://github.com/matrix-org/matrix-doc/issues/1567>`_)
- ``POST /rooms/{roomId}/read_markers`` (`#1635 <https://github.com/matrix-org/matrix-doc/issues/1635>`_)
Backwards Compatible Changes
----------------------------
- Add more presence options to the ``set_presence`` parameter of ``/sync``. (Thanks @mujx!) (`#780 <https://github.com/matrix-org/matrix-doc/issues/780>`_)
- Add ``token`` parameter to the ``/keys/query`` endpoint (`#1104 <https://github.com/matrix-org/matrix-doc/issues/1104>`_)
- Add the room visibility options for the room directory (`#1141 <https://github.com/matrix-org/matrix-doc/issues/1141>`_)
- Add spec for ignoring users (`#1142 <https://github.com/matrix-org/matrix-doc/issues/1142>`_)
- Add the ``/register/available`` endpoint for username availability (`#1151 <https://github.com/matrix-org/matrix-doc/issues/1151>`_)
- Add sticker messages (`#1158 <https://github.com/matrix-org/matrix-doc/issues/1158>`_)
- Specify how to control the power level required for ``@room`` (`#1176 <https://github.com/matrix-org/matrix-doc/issues/1176>`_)
- Document ``/logout/all`` endpoint (`#1263 <https://github.com/matrix-org/matrix-doc/issues/1263>`_)
- Add report content API (`#1264 <https://github.com/matrix-org/matrix-doc/issues/1264>`_)
- Add ``allow_remote`` to the content repo to avoid routing loops (`#1265 <https://github.com/matrix-org/matrix-doc/issues/1265>`_)
- Document `highlights` field in /search response (`#1274 <https://github.com/matrix-org/matrix-doc/issues/1274>`_)
- End-to-end encryption for group chats:
* Olm and Megolm messaging algorithms.
* ``m.room.encrypted``, ``m.room.encryption``, ``m.room_key`` events.
* Device verification process.
* ``device_one_time_keys_count`` sync parameter.
* ``device_lists:left`` sync parameter. (`#1284 <https://github.com/matrix-org/matrix-doc/issues/1284>`_)
- Add ``.well-known`` server discovery method (`#1359 <https://github.com/matrix-org/matrix-doc/issues/1359>`_)
- Document the GET version of ``/login`` (`#1361 <https://github.com/matrix-org/matrix-doc/issues/1361>`_)
- Document the ``server_name`` parameter on ``/join/{roomIdOrAlias}`` (`#1364 <https://github.com/matrix-org/matrix-doc/issues/1364>`_)
- Document the CORS/preflight headers (`#1365 <https://github.com/matrix-org/matrix-doc/issues/1365>`_)
- Add new user identifier object for logging in (`#1390 <https://github.com/matrix-org/matrix-doc/issues/1390>`_)
- Document message formats on ``m.text`` and ``m.emote`` messages (`#1397 <https://github.com/matrix-org/matrix-doc/issues/1397>`_)
- Encrypt file attachments (`#1420 <https://github.com/matrix-org/matrix-doc/issues/1420>`_)
- Share room decryption keys between devices (`#1465 <https://github.com/matrix-org/matrix-doc/issues/1465>`_)
- Document and improve client interaction with pushers. (`#1506 <https://github.com/matrix-org/matrix-doc/issues/1506>`_)
- Add support for Room Versions. (`#1516 <https://github.com/matrix-org/matrix-doc/issues/1516>`_)
- Guests can now call /context and /event to fetch events (`#1542 <https://github.com/matrix-org/matrix-doc/issues/1542>`_)
- Add a common standard for user, room, and group mentions in messages. (`#1547 <https://github.com/matrix-org/matrix-doc/issues/1547>`_)
- Add server ACLs as an option for controlling federation in a room. (`#1550 <https://github.com/matrix-org/matrix-doc/issues/1550>`_)
- Add new push rules for encrypted events and ``@room`` notifications. (`#1551 <https://github.com/matrix-org/matrix-doc/issues/1551>`_)
- Add third party network room directories, as provided by application services. (`#1554 <https://github.com/matrix-org/matrix-doc/issues/1554>`_)
- Document the ``validated_at`` and ``added_at`` fields on ``GET /acount/3pid``. (`#1567 <https://github.com/matrix-org/matrix-doc/issues/1567>`_)
- Add an ``inhibit_login`` registration option. (`#1589 <https://github.com/matrix-org/matrix-doc/issues/1589>`_)
- Recommend that servers set a Content Security Policy for the content repository. (`#1600 <https://github.com/matrix-org/matrix-doc/issues/1600>`_)
- Add "rich replies" - a way for users to better represent the conversation thread they are referencing in their messages. (`#1617 <https://github.com/matrix-org/matrix-doc/issues/1617>`_)
- Add support for read markers. (`#1635 <https://github.com/matrix-org/matrix-doc/issues/1635>`_)
Spec Clarifications
-------------------
- Mark ``home_server`` return field for ``/login`` and ``/register`` endpoints as deprecated (`#1097 <https://github.com/matrix-org/matrix-doc/issues/1097>`_)
- Fix response format of ``/keys/changes`` endpoint (`#1106 <https://github.com/matrix-org/matrix-doc/issues/1106>`_)
- Clarify default values for some fields on the ``/search`` API (`#1109 <https://github.com/matrix-org/matrix-doc/issues/1109>`_)
- Fix the representation of ``m.presence`` events (`#1137 <https://github.com/matrix-org/matrix-doc/issues/1137>`_)
- Clarify that ``m.tag`` ordering is done with numbers, not strings (`#1139 <https://github.com/matrix-org/matrix-doc/issues/1139>`_)
- Clarify that ``/account/whoami`` should consider application services (`#1152 <https://github.com/matrix-org/matrix-doc/issues/1152>`_)
- Update ``ImageInfo`` and ``ThumbnailInfo`` dimension schema descriptions to clarify that they relate to intended display size, as opposed to the intrinsic size of the image file. (`#1158 <https://github.com/matrix-org/matrix-doc/issues/1158>`_)
- Mark ``GET /rooms/{roomId}/members`` as requiring authentication (`#1245 <https://github.com/matrix-org/matrix-doc/issues/1245>`_)
- Clarify ``changed`` field behaviour in device tracking process (`#1284 <https://github.com/matrix-org/matrix-doc/issues/1284>`_)
- Describe ``StateEvent`` for ``/createRoom`` (`#1329 <https://github.com/matrix-org/matrix-doc/issues/1329>`_)
- Describe how the ``reason`` is handled for kicks/bans (`#1362 <https://github.com/matrix-org/matrix-doc/issues/1362>`_)
- Mark ``GET /presence/{userId}/status`` as requiring authentication (`#1371 <https://github.com/matrix-org/matrix-doc/issues/1371>`_)
- Describe the rate limit error response schema (`#1373 <https://github.com/matrix-org/matrix-doc/issues/1373>`_)
- Clarify that clients must leave rooms before forgetting them (`#1378 <https://github.com/matrix-org/matrix-doc/issues/1378>`_)
- Document guest access in ``/createRoom`` presets (`#1379 <https://github.com/matrix-org/matrix-doc/issues/1379>`_)
- Define what a ``RoomEvent`` is on ``/rooms/{roomId}/messages`` (`#1380 <https://github.com/matrix-org/matrix-doc/issues/1380>`_)
- Clarify the request and result types on ``/search`` (`#1381 <https://github.com/matrix-org/matrix-doc/issues/1381>`_)
- Clarify some of the properties on the search result (`#1400 <https://github.com/matrix-org/matrix-doc/issues/1400>`_)
- Clarify how access tokens are meant to be supplied to the homeserver. (`#1517 <https://github.com/matrix-org/matrix-doc/issues/1517>`_)
- Document additional parameters on the ``/createRoom`` API. (`#1518 <https://github.com/matrix-org/matrix-doc/issues/1518>`_)
- Clarify that new push rules should be enabled by default, and that unrecognised conditions should not match. (`#1551 <https://github.com/matrix-org/matrix-doc/issues/1551>`_)
- Update all event examples to be accurate representations of their associated events. (`#1558 <https://github.com/matrix-org/matrix-doc/issues/1558>`_)
- Clarify the supported HTML features for room messages. (`#1562 <https://github.com/matrix-org/matrix-doc/issues/1562>`_)
- Move the ``invite_room_state`` definition under ``unsigned`` where it actually resides. (`#1568 <https://github.com/matrix-org/matrix-doc/issues/1568>`_)
- Clarify the homeserver's behaviour for searching users. (`#1569 <https://github.com/matrix-org/matrix-doc/issues/1569>`_)
- Clarify the object structures and defaults for Filters. (`#1570 <https://github.com/matrix-org/matrix-doc/issues/1570>`_)
- Clarify instances of ``type: number`` in the swagger/OpenAPI schema definitions. (`#1571 <https://github.com/matrix-org/matrix-doc/issues/1571>`_)
- Clarify that left rooms also have account data in ``/sync``. (`#1572 <https://github.com/matrix-org/matrix-doc/issues/1572>`_)
- Clarify the event fields used in the ``/sync`` response. (`#1573 <https://github.com/matrix-org/matrix-doc/issues/1573>`_)
- Fix naming of the body field in ``PUT /directory/room``. (`#1574 <https://github.com/matrix-org/matrix-doc/issues/1574>`_)
- Clarify the filter object schema used in room searching. (`#1577 <https://github.com/matrix-org/matrix-doc/issues/1577>`_)
- Document the 403 error for sending state events. (`#1590 <https://github.com/matrix-org/matrix-doc/issues/1590>`_)
- specify how to handle multiple olm sessions with the same device (`#1596 <https://github.com/matrix-org/matrix-doc/issues/1596>`_)
- Add the other keys that redactions are expected to preserve. (`#1602 <https://github.com/matrix-org/matrix-doc/issues/1602>`_)
- Clarify that clients should not be generating invalid HTML for formatted events. (`#1605 <https://github.com/matrix-org/matrix-doc/issues/1605>`_)
- Clarify the room tag structure (thanks @KitsuneRal!) (`#1606 <https://github.com/matrix-org/matrix-doc/issues/1606>`_)
- Add a note that clients may use the transaction ID to avoid flickering when doing local echo. (`#1619 <https://github.com/matrix-org/matrix-doc/issues/1619>`_)
- Include the request and response structures for the various ``/requestToken`` endpoints. (`#1636 <https://github.com/matrix-org/matrix-doc/issues/1636>`_)
- Clarify the available error codes, and when to prefer the HTTP status code over the ``errcode``. (`#1637 <https://github.com/matrix-org/matrix-doc/issues/1637>`_)
- Clarify and generalise the language used for describing pagination. (`#1642 <https://github.com/matrix-org/matrix-doc/issues/1642>`_)
r0.3.0
======
- Breaking changes:
- Change the rule kind of ``.m.rule.contains_display_name`` from
``underride`` to ``override``. This works with all known clients
which support push rules, but any other clients implementing
the push rules API should be aware of this change. This
makes it simple to mute rooms correctly in the API
(`#373 <https://github.com/matrix-org/matrix-doc/pull/373>`_).
- Remove ``/tokenrefresh`` from the API
(`#395 <https://github.com/matrix-org/matrix-doc/pull/395>`_).
- Remove requirement that tokens used in token-based login be macaroons
(`#395 <https://github.com/matrix-org/matrix-doc/pull/395>`_).
- Move ``thumbnail_url`` and ``thumbnail_info`` members of json objects
for ``m.room.message`` events with msgtypes ``m.image``, ``m.file``
and ``m.location``, inside the ``info`` member, to match ``m.video``
events
(`#723 <https://github.com/matrix-org/matrix-doc/pull/723>`_).
- Changes to the API which will be backwards-compatible for clients:
- Add ``filename`` parameter to ``POST /_matrix/media/r0/upload``
(`#364 <https://github.com/matrix-org/matrix-doc/pull/364>`_).
- Document CAS-based client login and the use of ``m.login.token`` in
``/login`` (`#367 <https://github.com/matrix-org/matrix-doc/pull/367>`_).
- Make ``origin_server_ts`` a mandatory field of room events
(`#379 <https://github.com/matrix-org/matrix-doc/pull/370>`_).
- Add top-level ``account_data`` key to the responses to ``GET /sync`` and
``GET /initialSync``
(`#380 <https://github.com/matrix-org/matrix-doc/pull/380>`_).
- Add ``is_direct`` flag to ``POST /createRoom`` and invite member event.
Add 'Direct Messaging' module
(`#389 <https://github.com/matrix-org/matrix-doc/pull/389>`_).
- Add ``contains_url`` option to ``RoomEventFilter``
(`#390 <https://github.com/matrix-org/matrix-doc/pull/390>`_).
- Add ``filter`` optional query param to ``/messages``
(`#390 <https://github.com/matrix-org/matrix-doc/pull/390>`_).
- Add 'Send-to-Device messaging' module
(`#386 <https://github.com/matrix-org/matrix-doc/pull/386>`_).
- Add 'Device management' module
(`#402 <https://github.com/matrix-org/matrix-doc/pull/402>`_).
- Require that User-Interactive auth fallback pages call
``window.postMessage`` to notify apps of completion
(`#398 <https://github.com/matrix-org/matrix-doc/pull/398>`_).
- Add pagination and filter support to ``/publicRooms``. Change response to
omit fields rather than return ``null``. Add estimate of total number of
rooms in list.
(`#388 <https://github.com/matrix-org/matrix-doc/pull/388>`_).
- Allow guest accounts to use a number of endpoints which are required for
end-to-end encryption.
(`#751 <https://github.com/matrix-org/matrix-doc/pull/751>`_).
- Add key distribution APIs, for use with end-to-end encryption.
(`#894 <https://github.com/matrix-org/matrix-doc/pull/894>`_).
- Add ``m.room.pinned_events`` state event for rooms.
(`#1007 <https://github.com/matrix-org/matrix-doc/pull/1007>`_).
- Add mention of ability to send Access Token via an Authorization Header.
- Add ``guest_can_join`` parameter to ``POST /createRoom``
(`#1093 <https://github.com/matrix-org/matrix-doc/pull/1093>`_).
- New endpoints:
- ``GET /joined_rooms``
(`#999 <https://github.com/matrix-org/matrix-doc/pull/999>`_).
- ``GET /rooms/{roomId}/joined_members``
(`#999 <https://github.com/matrix-org/matrix-doc/pull/999>`_).
- ``GET /account/whoami``
(`#1063 <https://github.com/matrix-org/matrix-doc/pull/1063>`_).
- ``GET /media/{version}/preview_url``
(`#1064 <https://github.com/matrix-org/matrix-doc/pull/1064>`_).
- Spec clarifications:
- Add endpoints and logic for invites and third-party invites to the federation
spec and update the JSON of the request sent by the identity server upon 3PID
binding
(`#997 <https://github.com/matrix-org/matrix-doc/pull/997>`_)
- Fix "membership" property on third-party invite upgrade example
(`#995 <https://github.com/matrix-org/matrix-doc/pull/995>`_)
- Fix response format and 404 example for room alias lookup
(`#960 <https://github.com/matrix-org/matrix-doc/pull/960>`_)
- Fix examples of ``m.room.member`` event and room state change,
and added a clarification on the membership event sent upon profile update
(`#950 <https://github.com/matrix-org/matrix-doc/pull/950>`_).
- Spell out the way that state is handled by ``POST /createRoom``
(`#362 <https://github.com/matrix-org/matrix-doc/pull/362>`_).
- Clarify the fields which are applicable to different types of push rule
(`#365 <https://github.com/matrix-org/matrix-doc/pull/365>`_).
- A number of clarifications to authentication
(`#371 <https://github.com/matrix-org/matrix-doc/pull/371>`_).
- Correct references to ``user_id`` which should have been ``sender``
(`#376 <https://github.com/matrix-org/matrix-doc/pull/376>`_).
- Correct inconsistent specification of ``redacted_because`` fields and their
values (`#378 <https://github.com/matrix-org/matrix-doc/pull/378>`_).
- Mark required fields in response objects as such
(`#394 <https://github.com/matrix-org/matrix-doc/pull/394>`_).
- Make ``m.notice`` description a bit harder in its phrasing to try to
dissuade the same issues that occurred with IRC
(`#750 <https://github.com/matrix-org/matrix-doc/pull/750>`_).
- ``GET /user/{userId}/filter/{filterId}`` requires authentication
(`#1003 <https://github.com/matrix-org/matrix-doc/pull/1003>`_).
- Add some clarifying notes on the behaviour of rooms with no
``m.room.power_levels`` event
(`#1026 <https://github.com/matrix-org/matrix-doc/pull/1026>`_).
- Clarify the relationship between ``username`` and ``user_id`` in the
``/register`` API
(`#1032 <https://github.com/matrix-org/matrix-doc/pull/1032>`_).
- Clarify rate limiting and security for content repository.
(`#1064 <https://github.com/matrix-org/matrix-doc/pull/1064>`_).
r0.2.0
======
- Spec clarifications:
- Room aliases (`#337 <https://github.com/matrix-org/matrix-doc/pull/337>`_):
- Make it clear that ``GET /directory/room/{roomAlias}`` must work for
federated aliases.
- ``GET /directory/room/{roomAlias}`` cannot return a 409; the ``PUT``
endpoint can, however.
- Power levels:
- Clarify the defaults to be used for various fields of the
``m.room.power_levels`` event
(`#286 <https://github.com/matrix-org/matrix-doc/pull/286>`_,
`#341 <https://github.com/matrix-org/matrix-doc/pull/341>`_).
- Add suggestions for mapping of names to power levels
(`#336 <https://github.com/matrix-org/matrix-doc/pull/336>`_).
- Clarify the room naming algorithm in certain edge cases
(`#351 <https://github.com/matrix-org/matrix-doc/pull/351>`_).
- Remove outdated references to the pre-r0 ``/events`` API, and clarify the
section on syncing
(`#352 <https://github.com/matrix-org/matrix-doc/pull/352>`_).
- Changes to the API which will be backwards-compatible for clients:
- New endpoints:
- ``POST /register/email/requestToken``
(`#343 <https://github.com/matrix-org/matrix-doc/pull/343>`_).
- ``POST /account/3pid/email/requestToken``
(`#346 <https://github.com/matrix-org/matrix-doc/pull/346>`_).
- ``POST /account/password/email/requestToken``
(`#346 <https://github.com/matrix-org/matrix-doc/pull/346>`_).
- ``POST /account/deactivate``
(`#361 <https://github.com/matrix-org/matrix-doc/pull/361>`_).
- Updates to the Presence module
(`#278 <https://github.com/matrix-org/matrix-doc/pull/278>`_,
`#342 <https://github.com/matrix-org/matrix-doc/pull/342>`_):
- Remove unused ``free_for_chat`` presence state
- Add ``currently_active`` flag to the ``m.presence`` event and the ``GET
/presence/{userId}/status`` response.
- Make idle timeout the responsibility of the server
- Remove requirements on servers to propagate profile information via
``m.presence`` events.
- Add new predefined push rules
(`#274 <https://github.com/matrix-org/matrix-doc/pull/274>`_,
`#340 <https://github.com/matrix-org/matrix-doc/pull/340/files>`_).
- ``/sync`` should always return a ``prev_batch`` token
(`#345 <https://github.com/matrix-org/matrix-doc/pull/345>`_).
- add ``to`` parameter to ``GET /rooms/{roomId}/messages`` API
(`#348 <https://github.com/matrix-org/matrix-doc/pull/348>`_).
r0.1.0
======
This release includes the following changes since r0.0.1:
- Breaking changes to the API [#]_:
- ``POST /rooms/{roomId}/join`` no longer permits use of a room alias instead
of a room id. (``POST /join/{roomIdOrAlias}`` continues to allow either.)
- ``POST /account/3pid``: correct the name of the ``three_pid_creds``
parameter
- The "Push Rules" module no longer supports device-specific rules:
- ``GET /pushrules`` no longer returns a ``device`` property
- ``device/{profile_tag}`` is no longer a valid ``scope`` for push rules
- ``profile_tag`` is no longer a valid kind of condition on push rules.
(Device-specific push rules will be reintroduced in the future; in the
meantime, their specification has been moved to a `draft branch`__.)
__ https://matrix.org/speculator/spec/drafts%2Freinstate_device_push_rules/
- Changes to the API which will be backwards-compatible for clients:
- New endpoints:
- ``POST /logout``
- ``POST /rooms/{roomId}/unban``
- ``POST /rooms/{roomId}/kick``
- ``GET /pushers``
- ``GET /pushrules/{scope}/{kind}/{ruleId}/enabled``
(previously ``PUT``-only)
- ``GET`` and ``PUT /pushrules/{scope}/{kind}/{ruleId}/actions``
- Add ``third_party_signed`` parameter to ``POST /rooms/{roomId}/join``
- Add ``M_INVALID_USERNAME`` as valid response to ``POST /register``
- Add ``unread_notifications`` field to ``GET /sync`` response
- Add optional ``invite`` property to ``m.room.power_levels`` state event
- Add optional ``public_key`` and ``public_keys`` to
``m.room.third_party_invite`` state event
- Password-based ``/login`` may now use a third-party identifier instead of
a matrix user id.
- Spec clarifications
- Make the state diagram for room membership explicit
- Note that a user may not be invited to a room while banned
- Clarify the expected order of events in the response to
``GET /rooms/{roomId}/context/{eventId}``, as well as correcting the
example for that API
- Clarify the behaviour of the "Room History Visibility" module; in
particular, the behaviour of the ``shared`` history visibility, and how
events at visibility boundaries should be handled
- Separate the "Room Previews" module from "Guest access"
- Reword the description of the ``profile_tag`` property in
``PUT /pushers/set``, and note that it is not mandatory.
.. [#] Our `versioning policy <../index.html#specification-versions>`_ would
strictly require that a breaking change be denoted by a new major
specification version. However we are not aware of any clients which
rely on the old behaviour here, nor server implementations which offer
it, so we have chosen to retain the r0 designation on this occasion.
r0.0.1
======
This release includes the following changes since r0.0.0:
- API changes:
- Added new ``/versions`` API
- ``/createRoom`` takes an optional ``invite_3pid`` parameter
- ``/publicRooms`` returns an ``avatar_url`` result
- The following APIs are now deprecated:
- ``/initialSync``
- ``/events``
- ``/events/:eventId``
- ``/rooms/:roomId/initialSync``
- Spec clarifications
- Document inter-version compatibility
- Document the parameters to the ``/user/:userId/filter`` API
- Document the ``next_batch`` parameter on ``/search``
- Document the membership states on ``m.room.member`` events
- Minor clarifications/corrections to:
- Guest access module
- Search module
- ``/login`` API
- ``/rooms/:roomId/send/:eventType/:txnId`` API
- ``/rooms/:roomId/context/:eventId`` API
r0.0.0
======
This is the first release of the client-server specification. It is largely a dump of what has currently been implemented, and there are several inconsistencies.
An upcoming minor release will deprecate many of these inconsistencies, and they will be removed in the next major release.
Since the draft stage, the following major changes have been made:
- /api/v1 and /v2_alpha path segments have been replaced with the major version of the release (i.e. 'r0').
- Some POST versions of APIs with both POST and PUT have been removed.
- The specification has been split into one specification per API. This is the client-server API. The server-server API can be found documented separately.
- All APIs are now documented using Swagger
- The following modules have been added:
- Content repository
- Instant messaging
- Push notification
- History visibility
- Search
- Invites based on third party identifiers
- Room tagging
- Guest access
- Client config
- The following APIs were added:
- ``/sync``
- ``/publicRooms``
- ``/rooms/{roomId}/forget``
- ``/admin/whois``
- ``/rooms/{roomId}/redact``
- ``/user/{userId}/filter``
- The following APIs have been significantly modified:
- Invitations now contain partial room state
- Invitations can now be rejected
- ``/directory``
- The following events have been added:
- ``m.room.avatar``
- Example signed json is included for reference
- Commentary on display name calculation was added

54
changelogs/legacy/identity_service.rst
Unescape Escape View File

@ -1,54 +0,0 @@
r0.3.0
======
New Endpoints
-------------
- Add ``/account``, ``/account/register``, and ``/account/logout`` to authenticate with the identity server. (`#2255 <https://github.com/matrix-org/matrix-doc/issues/2255>`_)
- Add endpoints for accepting and handling terms of service. (`#2258 <https://github.com/matrix-org/matrix-doc/issues/2258>`_)
- Add ``/hash_details`` and a new ``/lookup`` endpoint for performing hashed association lookups. (`#2287 <https://github.com/matrix-org/matrix-doc/issues/2287>`_)
Backwards Compatible Changes
----------------------------
- Deprecate the v1 API in favour of an authenticated v2 API. (`#2254 <https://github.com/matrix-org/matrix-doc/issues/2254>`_)
r0.2.1
======
Spec Clarifications
-------------------
- Remove incorrect ``id_server`` parameter from ``/requestToken`` endpoints. (`#2124 <https://github.com/matrix-org/matrix-doc/issues/2124>`_)
- Clarify that identity servers can return 403 for unbind requests. (`#2126 <https://github.com/matrix-org/matrix-doc/issues/2126>`_)
r0.2.0
======
New Endpoints
-------------
- Add ``/3pid/unbind`` for removing 3PIDs. (`#2046 <https://github.com/matrix-org/matrix-doc/issues/2046>`_)
Spec Clarifications
-------------------
- Fix various spelling mistakes throughout the specification. (`#1853 <https://github.com/matrix-org/matrix-doc/issues/1853>`_)
- Fix route for ``/3pid/bind``. (`#1967 <https://github.com/matrix-org/matrix-doc/issues/1967>`_)
- Add missing aesthetic parameters to ``/store-invite``. (`#2049 <https://github.com/matrix-org/matrix-doc/issues/2049>`_)
- Clarify what the client should receive upon sending an identical email validation request multiple times. (`#2057 <https://github.com/matrix-org/matrix-doc/issues/2057>`_)
- Clarify that the default transport is JSON over HTTP. (`#2086 <https://github.com/matrix-org/matrix-doc/issues/2086>`_)
r0.1.0
======
This is the first release of the Identity Service API. With this API, clients and
homeservers can store bindings between third party identifiers such as email addresses
and phone numbers, associating them with Matrix user IDs. Additionally, identity
servers offer the ability to invite third party users to Matrix rooms by storing
the invite until the identifier is bound.

15
changelogs/legacy/push_gateway.rst
Unescape Escape View File

@ -1,15 +0,0 @@
r0.1.1
======
Spec Clarifications
-------------------
- Fix ``event_id`` field in push request body. (`#2151 <https://github.com/matrix-org/matrix-doc/issues/2151>`_)
r0.1.0
======
The first release of the Push Gateway specification. This release contains
a single endpoint, ``/notify``, that pushers may use to send push notifications
to clients.

74
changelogs/legacy/server_server.rst
Unescape Escape View File

@ -1,74 +0,0 @@
r0.1.4
======
New Endpoints
-------------
- Add new ``POST /publicRooms`` endpoint for filtering the room directory. (`#2305 <https://github.com/matrix-org/matrix-doc/issues/2305>`_)
- Add new v2 ``/send_join`` and ``/send_leave`` endpoints per `MSC1802 <https://github.com/matrix-org/matrix-doc/pull/1802>`_. (`#2547 <https://github.com/matrix-org/matrix-doc/issues/2547>`_)
Removed Endpoints
-----------------
- Remove the unused ``query_auth`` API per `MSC2451 <https://github.com/matrix-org/matrix-doc/pull/2451>`_. (`#2470 <https://github.com/matrix-org/matrix-doc/issues/2470>`_)
Spec Clarifications
-------------------
- Move auth event selection to a more obvious location. (`#2392 <https://github.com/matrix-org/matrix-doc/issues/2392>`_)
- Fix typo in Request Authentication python example. (`#2510 <https://github.com/matrix-org/matrix-doc/issues/2510>`_)
- Clarify which fields are required on the key server endpoints. (`#2527 <https://github.com/matrix-org/matrix-doc/issues/2527>`_)
- Clarify the limits of ``prev_events`` and ``auth_events`` for PDUs. (`#2538 <https://github.com/matrix-org/matrix-doc/issues/2538>`_)
- Clarify which events are targeted by backfill. (`#2559 <https://github.com/matrix-org/matrix-doc/issues/2559>`_)
- Fix the response format of the ``/send`` endpoint. (`#2560 <https://github.com/matrix-org/matrix-doc/issues/2560>`_)
- Clarify signature object structures for encryption. (`#2566 <https://github.com/matrix-org/matrix-doc/issues/2566>`_)
- Clarify the server names to use when signing requests. (`#2570 <https://github.com/matrix-org/matrix-doc/issues/2570>`_)
- Clarify the state/auth chain requirements for ``/send_join``. (`#2575 <https://github.com/matrix-org/matrix-doc/issues/2575>`_)
- Fix various spelling errors throughout the specification. (`#2577 <https://github.com/matrix-org/matrix-doc/issues/2577>`_)
r0.1.3
======
Spec Clarifications
-------------------
- Clarify the encryption algorithms supported by the device of the user keys query example. (`#2157 <https://github.com/matrix-org/matrix-doc/issues/2157>`_)
- Clarify the purpose of reference hashes. (`#2159 <https://github.com/matrix-org/matrix-doc/issues/2159>`_)
r0.1.2
======
Spec Clarifications
-------------------
- Change examples to use example.org instead of a real domain. (`#1650 <https://github.com/matrix-org/matrix-doc/issues/1650>`_)
- Fix the ``access_token`` parameter in the open_id endpoint. (`#1906 <https://github.com/matrix-org/matrix-doc/issues/1906>`_)
- Fix various spelling mistakes throughout the specification. (`#1991 <https://github.com/matrix-org/matrix-doc/issues/1991>`_)
- Clarify exactly what invite_room_state consists of. (`#2067 <https://github.com/matrix-org/matrix-doc/issues/2067>`_)
- Clarify how ``valid_until_ts`` behaves with respect to room version. (`#2080 <https://github.com/matrix-org/matrix-doc/issues/2080>`_)
- Clarify which servers are supposed to sign events. (`#2081 <https://github.com/matrix-org/matrix-doc/issues/2081>`_)
- Clarify the key object definition for the key management API. (`#2083 <https://github.com/matrix-org/matrix-doc/issues/2083>`_)
- Clarify how many PDUs are contained in transaction objects for various endpoints. (`#2095 <https://github.com/matrix-org/matrix-doc/issues/2095>`_)
- Clarify that the trailing slash is optional on ``/keys/*`` endpoints when no key ID is requested. (`#2097 <https://github.com/matrix-org/matrix-doc/issues/2097>`_)
r0.1.1
======
Spec Clarifications
-------------------
- Remove legacy references to TLS fingerprints. (`#1844 <https://github.com/matrix-org/matrix-doc/issues/1844>`_)
- Clarify that servers should not fail to contact servers if ``/.well-known`` fails. (`#1855 <https://github.com/matrix-org/matrix-doc/issues/1855>`_)
r0.1.0
======
This is the first release of the Server Server (Federation) specification.
It includes support for homeservers being able to interact with other
homeservers in a decentralized and standard way.

1
changelogs/push_gateway/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

39
changelogs/pyproject.toml
Unescape Escape View File

@ -1,39 +0,0 @@
[tool.towncrier]
version = "unused"
filename = "../rendered.md"
issue_format = "[#{issue}](https://github.com/matrix-org/matrix-doc/issues/{issue})"
title_format = "### {name}" # Matches rendered spec, even if awkward
underlines = " " # 3 spaces intentionally to hide RST headings
# Note: The names below have the <strong> tag built-in so the rendered spec *and* the generated
# changelog can benefit from sane headings.
[[tool.towncrier.type]]
directory = "breaking"
name = "<strong>Breaking Changes</strong>"
showcontent = true
[[tool.towncrier.type]]
directory = "deprecation"
name = "<strong>Deprecations</strong>"
showcontent = true
[[tool.towncrier.type]]
directory = "new"
name = "<strong>New Endpoints</strong>"
showcontent = true
[[tool.towncrier.type]]
directory = "removal"
name = "<strong>Removed Endpoints</strong>"
showcontent = true
[[tool.towncrier.type]]
directory = "feature"
name = "<strong>Backwards Compatible Changes</strong>"
showcontent = true
[[tool.towncrier.type]]
directory = "clarification"
name = "<strong>Spec Clarifications</strong>"
showcontent = true

1
changelogs/room_versions/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/room_versions/newsfragments/3387.feature.1
Unescape Escape View File

@ -1 +0,0 @@
Add Room Version 8 as per [MSC3289](https://github.com/matrix-org/matrix-doc/pull/3289).

1
changelogs/room_versions/newsfragments/3387.feature.2
Unescape Escape View File

@ -1 +0,0 @@
Add Room Version 9 as per [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375).

1
changelogs/room_versions/newsfragments/3501.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarifications to sections on event IDs and event formats.

1
changelogs/room_versions/newsfragments/3522.clarification
Unescape Escape View File

@ -1 +0,0 @@
Remove a number of fields which were incorrectly shown to form part of the `unsigned` data of a Federation PDU.

1
changelogs/server_server/newsfragments/.gitignore vendored
Unescape Escape View File

@ -1 +0,0 @@
!.gitignore

1
changelogs/server_server/newsfragments/3387.feature
Unescape Escape View File

@ -1 +0,0 @@
Add support for `restricted` rooms as per [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083), [MSC3289](https://github.com/matrix-org/matrix-doc/pull/3289), and [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375).

1
changelogs/server_server/newsfragments/3583.clarification
Unescape Escape View File

@ -1 +0,0 @@
Clarify that `GET /_matrix/federation/v1/event_auth/{roomId}/{eventId}` does *not* return the auth chain for the full state of the room.

1
changelogs/server_server/newsfragments/3610.new
Unescape Escape View File

@ -1 +0,0 @@
Add the Space Hierarchy API (`GET /_matrix/federation/v1/hierarchy/{roomId}`) as per [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

84
config.toml
Unescape Escape View File

@ -1,84 +0,0 @@
baseURL = "/"
title = "Matrix Specification"
# Prepends absolute URLs with the baseURL. Useful when hosting on non-root
# paths, such as /unstable.
canonifyURLs = true
enableRobotsTXT = true
# Hugo allows theme composition (and inheritance). The precedence is from left to right.
theme = ["docsy"]
disableKinds = ["taxonomy", "taxonomyTerm"]
[languages]
[languages.en]
title = "Matrix Specification"
description = "Home of the Matrix specification for decentralised communication"
languageName ="English"
# Weight used for sorting.
weight = 1
[markup]
[markup.goldmark]
[markup.goldmark.renderer]
# Enables us to render raw HTML
unsafe = true
[markup.highlight]
# See a complete list of available styles at https://xyproto.github.io/splash/docs/all.html
style = "tango"
# Everything below this are Site Params
[params]
copyright = "The Matrix.org Foundation CIC"
privacy_policy = "https://matrix.org/legal/privacy-notice"
[params.version]
# must be one of "unstable", "current", "historical"
# this is used to decide whether to show a banner pointing to the current release
status = "unstable"
# A URL pointing to the latest, stable release of the spec. To be shown in the unstable version warning banner.
current_version_url = "https://spec.matrix.org/latest"
# The following is used when status = "stable", and is displayed in various UI elements on a released version
# of the spec. CI will set these values here automatically when a release git tag (i.e `v1.5`) is created.
#major = "1"
#minor = "1"
#release_date = "November 09, 2021"
# User interface configuration
[params.ui]
# Set to true to disable the About link in the site footer
footer_about_disable = false
# Collapse HTTP API and event <details> elements
rendered_data_collapsed = false
[params.links]
# End user relevant links. These will show up on left side of footer and in the community page if you have one.
# [[params.links.user]]
# name = "User mailing list"
# url = "https://example.org/mail"
# icon = "fa fa-envelope"
# desc = "Discussion and help from your fellow users"
# Developer relevant links. These will show up on right side of footer and in the community page if you have one.
[[params.links.developer]]
name = "GitHub"
url = "https://github.com/matrix-org"
icon = "fab fa-github"
desc = "Matrix on GitHub"
[[params.links.developer]]
name = "GitLab"
url = "https://gitlab.matrix.org/matrix-org"
icon = "fab fa-gitlab"
desc = "Matrix on GitLab"
[[params.links.developer]]
name = "YouTube"
url = "https://www.youtube.com/channel/UCVFkW-chclhuyYRbmmfwt6w"
icon = "fab fa-youtube"
desc = "Matrix YouTube channel"
[[params.links.developer]]
name = "Twitter"
url = "https://twitter.com/matrixdotorg"
icon = "fab fa-twitter"
desc = "Matrix on Twitter"

506
content/_index.md
Unescape Escape View File

@ -1,506 +0,0 @@
---
title: "Matrix Specification"
type: docs
weight: 10
---
Matrix defines a set of open APIs for decentralised communication,
suitable for securely publishing, persisting and subscribing to data
over a global open federation of servers with no single point of
control. Uses include Instant Messaging (IM), Voice over IP (VoIP)
signalling, Internet of Things (IoT) communication, and bridging
together existing communication silos - providing the basis of a new
open real-time communication ecosystem.
To propose a change to the Matrix Spec, see the explanations at
[Proposals for Spec Changes to Matrix](/proposals).
## Matrix APIs
The specification consists of the following parts:
* [Client-Server API](/client-server-api)
* [Server-Server API](/server-server-api)
* [Application Service API](/application-service-api)
* [Identity Service API](/identity-service-api)
* [Push Gateway API](/push-gateway-api)
* [Room Versions](/rooms)
* [Appendices](/appendices)
Additionally, this introduction page contains the key baseline
information required to understand the specific APIs, including the
section the [overall architecture](#architecture).
The [Matrix Client-Server API Swagger
Viewer](https://matrix.org/docs/api/client-server/) is useful for
browsing the Client-Server API.
## Introduction to the Matrix APIs
Matrix is a set of open APIs for open-federated Instant Messaging (IM),
Voice over IP (VoIP) and Internet of Things (IoT) communication,
designed to create and support a new global real-time communication
ecosystem. The intention is to provide an open decentralised pubsub
layer for the internet for securely persisting and
publishing/subscribing JSON objects. This specification is the ongoing
result of standardising the APIs used by the various components of the
Matrix ecosystem to communicate with one another.
The principles that Matrix attempts to follow are:
- Pragmatic Web-friendly APIs (i.e. JSON over REST)
- Keep It Simple & Stupid
- provide a simple architecture with minimal third-party
dependencies.
- Fully open:
- Fully open federation - anyone should be able to participate in
the global Matrix network
- Fully open standard - publicly documented standard with no IP or
patent licensing encumbrances
- Fully open source reference implementation - liberally-licensed
example implementations with no IP or patent licensing
encumbrances
- Empowering the end-user
- The user should be able to choose the server and clients they
use
- The user should be able to control how private their
communication is
- The user should know precisely where their data is stored
- Fully decentralised - no single points of control over conversations
or the network as a whole
- Learning from history to avoid repeating it
- Trying to take the best aspects of XMPP, SIP, IRC, SMTP, IMAP
and NNTP whilst trying to avoid their failings
The functionality that Matrix provides includes:
- Creation and management of fully distributed chat rooms with no
single points of control or failure
- Eventually-consistent cryptographically secure synchronisation of
room state across a global open network of federated servers and
services
- Sending and receiving extensible messages in a room with (optional)
end-to-end encryption
- Extensible user management (inviting, joining, leaving, kicking,
banning) mediated by a power-level based user privilege system.
- Extensible room state management (room naming, aliasing, topics,
bans)
- Extensible user profile management (avatars, display names, etc)
- Managing user accounts (registration, login, logout)
- Use of 3rd Party IDs (3PIDs) such as email addresses, phone numbers,
Facebook accounts to authenticate, identify and discover users on
Matrix.
- Trusted federation of identity servers for:
- Publishing user public keys for PKI
- Mapping of 3PIDs to Matrix IDs
The end goal of Matrix is to be a ubiquitous messaging layer for
synchronising arbitrary data between sets of people, devices and
services - be that for instant messages, VoIP call setups, or any other
objects that need to be reliably and persistently pushed from A to B in
an interoperable and federated manner.
### Spec Change Proposals
To propose a change to the Matrix Spec, see the explanations at
[Proposals for Spec Changes to Matrix](/proposals).
## Architecture
Matrix defines APIs for synchronising extensible JSON objects known as
"events" between compatible clients, servers and services. Clients are
typically messaging/VoIP applications or IoT devices/hubs and
communicate by synchronising communication history with their
"homeserver" using the "Client-Server API". Each homeserver stores the
communication history and account information for all of its clients,
and shares data with the wider Matrix ecosystem by synchronising
communication history with other homeservers and their clients.
Clients typically communicate with each other by emitting events in the
context of a virtual "room". Room data is replicated across *all of the
homeservers* whose users are participating in a given room. As such, *no
single homeserver has control or ownership over a given room*.
Homeservers model communication history as a partially ordered graph of
events known as the room's "event graph", which is synchronised with
eventual consistency between the participating servers using the
"Server-Server API". This process of synchronising shared conversation
history between homeservers run by different parties is called
"Federation". Matrix optimises for the Availability and Partitioned
properties of CAP theorem at the expense of Consistency.
For example, for client A to send a message to client B, client A
performs an HTTP PUT of the required JSON event on its homeserver (HS)
using the client-server API. A's HS appends this event to its copy of
the room's event graph, signing the message in the context of the graph
for integrity. A's HS then replicates the message to B's HS by
performing an HTTP PUT using the server-server API. B's HS authenticates
the request, validates the event's signature, authorises the event's
contents and then adds it to its copy of the room's event graph. Client
B then receives the message from his homeserver via a long-lived GET
request.
How data flows between clients:
```
{ Matrix client A } { Matrix client B }
^ | ^ |
| events | Client-Server API | events |
| V | V
+------------------+ +------------------+
| |---------( HTTPS )--------->| |
| homeserver | | homeserver |
| |<--------( HTTPS )----------| |
+------------------+ Server-Server API +------------------+
History Synchronisation
(Federation)
```
### Users
Each client is associated with a user account, which is identified in
Matrix using a unique "user ID". This ID is namespaced to the homeserver
which allocated the account and has the form:
@localpart:domain
See ['Identifier Grammar' in the
appendices](/appendices#identifier-grammar) for full details of the
structure of user IDs.
### Devices
The Matrix specification has a particular meaning for the term "device".
As a user, I might have several devices: a desktop client, some web
browsers, an Android device, an iPhone, etc. They broadly relate to a
real device in the physical world, but you might have several browsers
on a physical device, or several Matrix client applications on a mobile
device, each of which would be its own device.
Devices are used primarily to manage the keys used for end-to-end
encryption (each device gets its own copy of the decryption keys), but
they also help users manage their access - for instance, by revoking
access to particular devices.
When a user first uses a client, it registers itself as a new device.
The longevity of devices might depend on the type of client. A web
client will probably drop all of its state on logout, and create a new
device every time you log in, to ensure that cryptography keys are not
leaked to a new user. In a mobile client, it might be acceptable to
reuse the device if a login session expires, provided the user is the
same.
Devices are identified by a `device_id`, which is unique within the
scope of a given user.
A user may assign a human-readable display name to a device, to help
them manage their devices.
### Events
All data exchanged over Matrix is expressed as an "event". Typically
each client action (e.g. sending a message) correlates with exactly one
event. Each event has a `type` which is used to differentiate different
kinds of data. `type` values MUST be uniquely globally namespaced
following Java's [package naming
conventions](https://en.wikipedia.org/wiki/Java_package#Package_naming_conventions),
e.g. `com.example.myapp.event`. The special top-level namespace `m.` is
reserved for events defined in the Matrix specification - for instance
`m.room.message` is the event type for instant messages. Events are
usually sent in the context of a "Room".
{{% boxes/warning %}}
Event bodies are considered untrusted data. This means that any application using
Matrix must validate that the event body is of the expected shape/schema
before using the contents verbatim.
**It is not safe to assume that an event body will have all the expected
fields of the expected types.**
See [MSC2801](https://github.com/matrix-org/matrix-doc/pull/2801) for more
detail on why this assumption is unsafe.
{{% /boxes/warning %}}
### Event Graphs
Events exchanged in the context of a room are stored in a directed
acyclic graph (DAG) called an "event graph". The partial ordering of
this graph gives the chronological ordering of events within the room.
Each event in the graph has a list of zero or more "parent" events,
which refer to any preceding events which have no chronological
successor from the perspective of the homeserver which created the
event.
Typically an event has a single parent: the most recent message in the
room at the point it was sent. However, homeservers may legitimately
race with each other when sending messages, resulting in a single event
having multiple successors. The next event added to the graph thus will
have multiple parents. Every event graph has a single root event with no
parent.
To order and ease chronological comparison between the events within the
graph, homeservers maintain a `depth` metadata field on each event. An
event's `depth` is a positive integer that is strictly greater than the
depths of any of its parents. The root event should have a depth of 1.
Thus if one event is before another, then it must have a strictly
smaller depth.
### Room structure
A room is a conceptual place where users can send and receive events.
Events are sent to a room, and all participants in that room with
sufficient access will receive the event. Rooms are uniquely identified
internally via "Room IDs", which have the form:
!opaque_id:domain
There is exactly one room ID for each room. Whilst the room ID does
contain a domain, it is simply for globally namespacing room IDs. The
room does NOT reside on the domain specified.
See ['Identifier Grammar' in the
appendices](/appendices#identifier-grammar) for full details of the
structure of a room ID.
The following conceptual diagram shows an `m.room.message` event being
sent to the room `!qporfwt:matrix.org`:
{ @alice:matrix.org } { @bob:example.org }
| ^
| |
[HTTP POST] [HTTP GET]
Room ID: !qporfwt:matrix.org Room ID: !qporfwt:matrix.org
Event type: m.room.message Event type: m.room.message
Content: { JSON object } Content: { JSON object }
| |
V |
+------------------+ +------------------+
| homeserver | | homeserver |
| matrix.org | | example.org |
+------------------+ +------------------+
| ^
| [HTTP PUT] |
| Room ID: !qporfwt:matrix.org |
| Event type: m.room.message |
| Content: { JSON object } |
`-------> Pointer to the preceding message ------`
PKI signature from matrix.org
Transaction-layer metadata
PKI Authorization header
....................................
| Shared Data |
| State: |
| Room ID: !qporfwt:matrix.org |
| Servers: matrix.org, example.org |
| Members: |
| - @alice:matrix.org |
| - @bob:example.org |
| Messages: |
| - @alice:matrix.org |
| Content: { JSON object } |
|....................................|
Federation maintains *shared data structures* per-room between multiple
homeservers. The data is split into `message events` and `state events`.
Message events:
These describe transient 'one-off' activity in a room such as
instant messages, VoIP call setups, file transfers, etc. They generally
describe communication activity.
State events:
These describe updates to a given piece of persistent information
('state') related to a room, such as the room's name, topic, membership,
participating servers, etc. State is modelled as a lookup table of
key/value pairs per room, with each key being a tuple of `state_key` and
`event type`. Each state event updates the value of a given key.
The state of the room at a given point is calculated by considering all
events preceding and including a given event in the graph. Where events
describe the same state, a merge conflict algorithm is applied. The
state resolution algorithm is transitive and does not depend on server
state, as it must consistently select the same event irrespective of the
server or the order the events were received in. Events are signed by
the originating server (the signature includes the parent relations,
type, depth and payload hash) and are pushed over federation to the
participating servers in a room, currently using full mesh topology.
Servers may also request backfill of events over federation from the
other servers participating in a room.
{{% boxes/note %}}
Events are not limited to the types defined in this specification. New
or custom event types can be created on a whim using the Java package
naming convention. For example, a `com.example.game.score` event can be
sent by clients and other clients would receive it through Matrix,
assuming the client has access to the `com.example` namespace.
{{% /boxes/note %}}
#### Room Aliases
Each room can also have multiple "Room Aliases", which look like:
#room_alias:domain
See ['Identifier Grammar' in the
appendices](/appendices#identifier-grammar) for full details of the
structure of a room alias.
A room alias "points" to a room ID and is the human-readable label by
which rooms are publicised and discovered. The room ID the alias is
pointing to can be obtained by visiting the domain specified. Note that
the mapping from a room alias to a room ID is not fixed, and may change
over time to point to a different room ID. For this reason, Clients
SHOULD resolve the room alias to a room ID once and then use that ID on
subsequent requests.
When resolving a room alias the server will also respond with a list of
servers that are in the room that can be used to join via.
HTTP GET
#matrix:example.org !aaabaa:matrix.org
| ^
| |
_______V____________________|____
| example.org |
| Mappings: |
| #matrix >> !aaabaa:matrix.org |
| #golf >> !wfeiofh:sport.com |
| #bike >> !4rguxf:matrix.org |
|________________________________|
### Identity
Users in Matrix are identified via their Matrix user ID. However,
existing 3rd party ID namespaces can also be used in order to identify
Matrix users. A Matrix "Identity" describes both the user ID and any
other existing IDs from third party namespaces *linked* to their
account. Matrix users can *link* third-party IDs (3PIDs) such as email
addresses, social network accounts and phone numbers to their user ID.
Linking 3PIDs creates a mapping from a 3PID to a user ID. This mapping
can then be used by Matrix users in order to discover the user IDs of
their contacts. In order to ensure that the mapping from 3PID to user ID
is genuine, a globally federated cluster of trusted "identity servers"
(IS) are used to verify the 3PID and persist and replicate the mappings.
Usage of an IS is not required in order for a client application to be
part of the Matrix ecosystem. However, without one clients will not be
able to look up user IDs using 3PIDs.
### Profiles
Users may publish arbitrary key/value data associated with their account
- such as a human-readable display name, a profile photo URL, contact
information (email address, phone numbers, website URLs etc).
### Private User Data
Users may also store arbitrary private key/value data in their account -
such as client preferences, or server configuration settings which lack
any other dedicated API. The API is symmetrical to managing Profile
data.
## Common concepts
Various things are common throughout all of the Matrix APIs. They are
documented here.
### Namespacing
Namespacing helps prevent conflicts between multiple applications and
the specification itself. Where namespacing is used, `m.` prefixes are
used by the specification to indicate that the field is controlled by
the specification. Custom or non-specified namespaces used in the wild
MUST use the Java package naming convention to prevent conflicts.
As an example, event types defined in the specification are namespaced
under the special `m.` prefix, however any client can send a custom
event type, such as `com.example.game.score` (assuming the client has
rights to the `com.example` namespace) without needing to put the event
into the `m.` namespace.
### Timestamps
Unless otherwise stated, timestamps are measured as milliseconds since
the Unix epoch. Throughout the specification this may be referred to as
POSIX, Unix, or just "time in milliseconds".
## Specification Versions
Matrix as a whole is released under a single specification number in the
form `vX.Y`.
* A change to `X` reflects a breaking or substantially invasive change.
When exactly to increment this number is left to the Spec Core Team,
however it is intended for changes such as moving away from JSON,
altering the signing algorithm, or when a large number of `Y` changes
feel deserving of a major version increase.
* A change to `Y` represents a backwards compatible or "managed" backwards
compatible change to the specification, usually in the form of features.
Additionally, the spec version may have arbitrary metadata applied to it
when followed by a `-`. For example, `v1.1-alpha`. Usage of this is not
strictly specified but is intended for usage of pre-release builds of the
specification.
Note that while `v1.2` is meant to be backwards compatible with `v1.1`, it
is not guaranteed that future versions will be fully backwards compatible
with `v1.1`. For example, if `/test` were to be introduced in `v1.1` and
deprecated in `v1.2`, then it can be removed in `v1.3`. More information
about this is described in the [deprecation policy](#deprecation-policy)
below.
### Endpoint versioning
All API endpoints within the specification are versioned individually.
This means that `/v3/sync` (for example) can get deprecated in favour
of `/v4/sync` without affecting `/v3/profile` at all. A server supporting
`/v4/sync` would keep serving `/v3/profile` as it always has.
When an MSC proposes a breaking change to an endpoint it should also
deprecate the existing endpoint. For some endpoints this might be implicit,
such as `/v4/sync` being introduced (deprecating `/v3/sync`), however
for more nuanced examples the MSC should deprecate the endpoint explicitly.
### Deprecation policy
An MSC is required to transition something from stable (the default) to
deprecated. Once something has been deprecated for suitably long enough
(usually 1 version), it is eligible for removal from the specification
with another MSC.
Implementations of Matrix are required to implement deprecated functionality
of the specification, though when the functionality is later removed then
the implementation is welcome to drop support (if they don't advertise
support for a version which includes deprecated functionality). For
example, if `/test` were deprecated in `v1.2` and removed in `v1.3`, then
an implementation which wants to advertise support for `v1.2` would have
to implement `/test`, even if the implementation also advertises support
for `v1.3`. If that implementation *only* advertises support for `v1.3`
then it would not be required to implement `/test`.
### Legacy versioning
Prior to this system, the different APIs of Matrix were versioned individually.
This is no longer possible with the new specification versioning approach.
For historical reference, the APIs were versioned as `rX.Y.Z` where `X`
roughly represents a breaking change, `Y` a backwards-compatible change, and
`Z` a patch or insignificant alteration to the API.
`v1.0` of Matrix was released on June 10th, 2019 with the following API
versions:
| API/Specification | Version |
|-------------------------|---------|
| Client-Server API | r0.5.0 |
| Server-Server API | r0.1.2 |
| Application Service API | r0.1.1 |
| Identity Service API | r0.1.1 |
| Push Gateway API | r0.1.0 |
| Room Version | v5 |
## License
The Matrix specification is licensed under the [Apache License, Version
2.0](http://www.apache.org/licenses/LICENSE-2.0).

1249
content/appendices.md
View File

File diff suppressed because it is too large Load Diff

397
content/application-service-api.md
Unescape Escape View File

@ -1,397 +0,0 @@
---
title: "Application Service API"
weight: 30
type: docs
---
The Matrix client-server API and server-server APIs provide the means to
implement a consistent self-contained federated messaging fabric.
However, they provide limited means of implementing custom server-side
behaviour in Matrix (e.g. gateways, filters, extensible hooks etc). The
Application Service API (AS API) defines a standard API to allow such
extensible functionality to be implemented irrespective of the
underlying homeserver implementation.
## Application Services
Application services are passive and can only observe events from
homeserver. They can inject events into rooms they are participating in.
They cannot prevent events from being sent, nor can they modify the
content of the event being sent. In order to observe events from a
homeserver, the homeserver needs to be configured to pass certain types
of traffic to the application service. This is achieved by manually
configuring the homeserver with information about the application
service.
### Registration
{{% boxes/note %}}
Previously, application services could register with a homeserver via
HTTP APIs. This was removed as it was seen as a security risk. A
compromised application service could re-register for a global `*` regex
and sniff *all* traffic on the homeserver. To protect against this,
application services now have to register via configuration files which
are linked to the homeserver configuration file. The addition of
configuration files allows homeserver admins to sanity check the
registration for suspicious regex strings.
{{% /boxes/note %}}
Application services register "namespaces" of user IDs, room aliases and
room IDs. These namespaces are represented as regular expressions. An
application service is said to be "interested" in a given event if one
of the IDs in the event match the regular expression provided by the
application service, such as the room having an alias or ID in the
relevant namespaces. Similarly, the application service is said to be
interested in a given event if one of the application service's
namespaced users is the target of the event, or is a joined member of
the room where the event occurred.
An application service can also state whether they should be the only
ones who can manage a specified namespace. This is referred to as an
"exclusive" namespace. An exclusive namespace prevents humans and other
application services from creating/deleting entities in that namespace.
Typically, exclusive namespaces are used when the rooms represent real
rooms on another service (e.g. IRC). Non-exclusive namespaces are used
when the application service is merely augmenting the room itself (e.g.
providing logging or searching facilities). Namespaces are represented
by POSIX extended regular expressions and look like:
users:
- exclusive: true
regex: "@_irc_bridge_.*"
Application services may define the following namespaces (with none
being explicitly required):
| Name | Description |
|----------|------------------------------------------------------------|
| users | Events which are sent from certain users. |
| aliases | Events which are sent in rooms with certain room aliases. |
| rooms | Events which are sent in rooms with certain room IDs. |
Each individual namespace MUST declare the following fields:
| Name | Description |
|------------|------------------------------------------------------------------------------------------------------------------------------------|
| exclusive | **Required** A true or false value stating whether this application service has exclusive access to events within this namespace. |
| regex | **Required** A regular expression defining which values this namespace includes. |
Exclusive user and alias namespaces should begin with an underscore
after the sigil to avoid collisions with other users on the homeserver.
Application services should additionally attempt to identify the service
they represent in the reserved namespace. For example, `@_irc_.*` would
be a good namespace to register for an application service which deals
with IRC.
The registration is represented by a series of key-value pairs, which
this specification will present as YAML. See below for the possible
options along with their explanation:
| Name | Description |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| id | **Required** A unique, user-defined ID of the application service which will never change. |
| url | **Required** The URL for the application service. May include a path after the domain name. Optionally set to null if no traffic is required. |
| as_token | **Required** A unique token for application services to use to authenticate requests to Homeservers. |
| hs_token | **Required** A unique token for Homeservers to use to authenticate requests to application services. |
| sender_localpart | **Required** The localpart of the user associated with the application service. |
| namespaces | **Required** A list of `users`, `aliases` and `rooms` namespaces that the application service controls. |
| rate_limited | Whether requests from masqueraded users are rate-limited. The sender is excluded. |
| protocols | The external protocols which the application service provides (e.g. IRC). |
An example registration file for an IRC-bridging application service is
below:
id: "IRC Bridge"
url: "http://127.0.0.1:1234"
as_token: "30c05ae90a248a4188e620216fa72e349803310ec83e2a77b34fe90be6081f46"
hs_token: "312df522183efd404ec1cd22d2ffa4bbc76a8c1ccf541dd692eef281356bb74e"
sender_localpart: "_irc_bot" # Will result in @_irc_bot:example.org
namespaces:
users:
- exclusive: true
regex: "@_irc_bridge_.*"
aliases:
- exclusive: false
regex: "#_irc_bridge_.*"
rooms: []
{{% boxes/warning %}}
If the homeserver in question has multiple application services, each
`as_token` and `id` MUST be unique per application service as these are
used to identify the application service. The homeserver MUST enforce
this.
{{% /boxes/warning %}}
### Homeserver -&gt; Application Service API
#### Authorization
Homeservers MUST include a query parameter named `access_token`
containing the `hs_token` from the application service's registration
when making requests to the application service. Application services
MUST verify the provided `access_token` matches their known `hs_token`,
failing the request with an `M_FORBIDDEN` error if it does not match.
#### Legacy routes
Previous drafts of the application service specification had a mix of
endpoints that have been used in the wild for a significant amount of
time. The application service specification now defines a version on all
endpoints to be more compatible with the rest of the Matrix
specification and the future.
Homeservers should attempt to use the specified endpoints first when
communicating with application services. However, if the application
service receives an HTTP status code that does not indicate success
(i.e.: 404, 500, 501, etc) then the homeserver should fall back to the
older endpoints for the application service.
The older endpoints have the exact same request body and response
format, they just belong at a different path. The equivalent path for
each is as follows:
- `/_matrix/app/v1/transactions/{txnId}` should fall back to
`/transactions/{txnId}`
- `/_matrix/app/v1/users/{userId}` should fall back to
`/users/{userId}`
- `/_matrix/app/v1/rooms/{roomAlias}` should fall back to
`/rooms/{roomAlias}`
- `/_matrix/app/v1/thirdparty/protocol/{protocol}` should fall back to
`/_matrix/app/unstable/thirdparty/protocol/{protocol}`
- `/_matrix/app/v1/thirdparty/user/{user}` should fall back to
`/_matrix/app/unstable/thirdparty/user/{user}`
- `/_matrix/app/v1/thirdparty/location/{location}` should fall back to
`/_matrix/app/unstable/thirdparty/location/{location}`
- `/_matrix/app/v1/thirdparty/user` should fall back to
`/_matrix/app/unstable/thirdparty/user`
- `/_matrix/app/v1/thirdparty/location` should fall back to
`/_matrix/app/unstable/thirdparty/location`
Homeservers should periodically try again for the newer endpoints
because the application service may have been updated.
#### Pushing events
The application service API provides a transaction API for sending a
list of events. Each list of events includes a transaction ID, which
works as follows:
```
Typical
HS ---> AS : Homeserver sends events with transaction ID T.
<--- : Application Service sends back 200 OK.
```
```
AS ACK Lost
HS ---> AS : Homeserver sends events with transaction ID T.
<-/- : AS 200 OK is lost.
HS ---> AS : Homeserver retries with the same transaction ID of T.
<--- : Application Service sends back 200 OK. If the AS had processed these
events already, it can NO-OP this request (and it knows if it is the
same events based on the transaction ID).
```
The events sent to the application service should be linearised, as if
they were from the event stream. The homeserver MUST maintain a queue of
transactions to send to the application service. If the application
service cannot be reached, the homeserver SHOULD backoff exponentially
until the application service is reachable again. As application
services cannot *modify* the events in any way, these requests can be
made without blocking other aspects of the homeserver. Homeservers MUST
NOT alter (e.g. add more) events they were going to send within that
transaction ID on retries, as the application service may have already
processed the events.
{{% http-api spec="application-service" api="transactions" %}}
#### Querying
The application service API includes two querying APIs: for room aliases
and for user IDs. The application service SHOULD create the queried
entity if it desires. During this process, the application service is
blocking the homeserver until the entity is created and configured. If
the homeserver does not receive a response to this request, the
homeserver should retry several times before timing out. This should
result in an HTTP status 408 "Request Timeout" on the client which
initiated this request (e.g. to join a room alias).
{{% boxes/rationale %}}
Blocking the homeserver and expecting the application service to create
the entity using the client-server API is simpler and more flexible than
alternative methods such as returning an initial sync style JSON blob
and get the HS to provision the room/user. This also meant that there
didn't need to be a "backchannel" to inform the application service
about information about the entity such as room ID to room alias
mappings.
{{% /boxes/rationale %}}
{{% http-api spec="application-service" api="query_user" %}}
{{% http-api spec="application-service" api="query_room" %}}
#### Third party networks
Application services may declare which protocols they support via their
registration configuration for the homeserver. These networks are
generally for third party services such as IRC that the application
service is managing. Application services may populate a Matrix room
directory for their registered protocols, as defined in the
Client-Server API Extensions.
Each protocol may have several "locations" (also known as "third party
locations" or "3PLs"). A location within a protocol is a place in the
third party network, such as an IRC channel. Users of the third party
network may also be represented by the application service.
Locations and users can be searched by fields defined by the application
service, such as by display name or other attribute. When clients
request the homeserver to search in a particular "network" (protocol),
the search fields will be passed along to the application service for
filtering.
{{% http-api spec="application-service" api="protocols" %}}
### Client-Server API Extensions
Application services can use a more powerful version of the
client-server API by identifying itself as an application service to the
homeserver.
Endpoints defined in this section MUST be supported by homeservers in
the client-server API as accessible only by application services.
#### Identity assertion
The client-server API infers the user ID from the `access_token`
provided in every request. To avoid the application service from having
to keep track of each user's access token, the application service
should identify itself to the Client-Server API by providing its
`as_token` for the `access_token` alongside the user the application
service would like to masquerade as.
Inputs:
- Application service token (`as_token`)
- User ID in the AS namespace to act as.
Notes:
- This applies to all aspects of the Client-Server API, except for
Account Management.
- The `as_token` is inserted into `access_token` which is usually
where the client token is, such as via the query string or
`Authorization` header. This is done on purpose to allow application
services to reuse client SDKs.
- The `access_token` should be supplied through the `Authorization`
header where possible to prevent the token appearing in HTTP request
logs by accident.
The application service may specify the virtual user to act as through
use of a `user_id` query string parameter on the request. The user
specified in the query string must be covered by one of the application
service's `user` namespaces. If the parameter is missing, the homeserver
is to assume the application service intends to act as the user implied
by the `sender_localpart` property of the registration.
An example request would be:
GET /_matrix/client/v3/account/whoami?user_id=@_irc_user:example.org
Authorization: Bearer YourApplicationServiceTokenHere
#### Timestamp massaging
Previous drafts of the Application Service API permitted application
services to alter the timestamp of their sent events by providing a `ts`
query parameter when sending an event. This API has been excluded from
the first release due to design concerns, however some servers may still
support the feature. Please visit [issue
\#1585](https://github.com/matrix-org/matrix-doc/issues/1585) for more
information.
#### Server admin style permissions
The homeserver needs to give the application service *full control* over
its namespace, both for users and for room aliases. This means that the
AS should be able to manage any users and room alias in its namespace. No additional API
changes need to be made in order for control of room aliases to be
granted to the AS.
Creation of users needs API changes in order to:
- Work around captchas.
- Have a 'passwordless' user.
This involves bypassing the registration flows entirely. This is
achieved by including the `as_token` on a `/register` request, along
with a login type of `m.login.application_service` to set the desired
user ID without a password.
POST /_matrix/client/v3/register
Authorization: Bearer YourApplicationServiceTokenHere
Content:
{
type: "m.login.application_service",
username: "_irc_example"
}
Similarly, logging in as users needs API changes in order to allow the AS to
log in without needing the user's password. This is achieved by including the
`as_token` on a `/login` request, along with a login type of
`m.login.application_service`:
{{% added-in v="1.2" %}}
POST /_matrix/client/%CLIENT_MAJOR_VERSION%/login
Authorization: Bearer YourApplicationServiceTokenHere
Content:
{
type: "m.login.application_service",
"identifier": {
"type": "m.id.user",
"user": "_irc_example"
}
}
Application services which attempt to create users or aliases *outside*
of their defined namespaces, or log in as users outside of their defined
namespaces will receive an error code `M_EXCLUSIVE`.
Similarly, normal users who attempt to create users or aliases *inside*
an application service-defined namespace will receive the same
`M_EXCLUSIVE` error code, but only if the application service has
defined the namespace as `exclusive`.
#### Using `/sync` and `/events`
Application services wishing to use `/sync` or `/events` from the
Client-Server API MUST do so with a virtual user (provide a `user_id`
via the query string). It is expected that the application service use
the transactions pushed to it to handle events rather than syncing with
the user implied by `sender_localpart`.
#### Application service room directories
Application services can maintain their own room directories for their
defined third party protocols. These room directories may be accessed by
clients through additional parameters on the `/publicRooms`
client-server endpoint.
{{% http-api spec="client-server" api="appservice_room_directory" %}}
### Referencing messages from a third party network
Application services should include an `external_url` in the `content`
of events it emits to indicate where the message came from. This
typically applies to application services that bridge other networks
into Matrix, such as IRC, where an HTTP URL may be available to
reference.
Clients should provide users with a way to access the `external_url` if
it is present. Clients should additionally ensure the URL has a scheme
of `https` or `http` before making use of it.
The presence of an `external_url` on an event does not necessarily mean
the event was sent from an application service. Clients should be wary
of the URL contained within, as it may not be a legitimate reference to
the event's source.

50
content/changelog.md
Unescape Escape View File

@ -1,50 +0,0 @@
---
title: Changelog
type: docs
weight: 1000
---
{{% changelog/changelog-description %}}
{{% changelog/changelog-changes %}}
<!-- DO NOT REMOVE OR CHANGE - Release script puts next release here -->
{{% changelog/changelog-rendered p="changelogs/v1.1.md" %}}
<h2 id="historical-versions" class="no-numbers">Historical versions</h2>
Before version 1.1, versioning was applied at the level of individual API specifications. This section includes links to these versions of the APIs.
* **Client-Server API**
- [r0.6.1](https://matrix.org/docs/spec/client_server/r0.6.1.html)
- [r0.6.0](https://matrix.org/docs/spec/client_server/r0.6.0.html)
- [r0.5.0](https://matrix.org/docs/spec/client_server/r0.5.0.html)
- [r0.4.0](https://matrix.org/docs/spec/client_server/r0.4.0.html)
- [r0.3.0](https://matrix.org/docs/spec/client_server/r0.3.0.html)
- [r0.2.0](https://matrix.org/docs/spec/client_server/r0.2.0.html)
- [r0.1.0](https://matrix.org/docs/spec/client_server/r0.1.0.html)
- [r0.0.1](https://matrix.org/docs/spec/r0.0.1/client_server.html)
- [r0.0.0](https://matrix.org/docs/spec/r0.0.0/client_server.html)
- [Legacy](https://matrix.org/docs/spec/legacy/#client-server-api):
The last draft before the spec was formally released in version
r0.0.0.
* **Server-Server API**
- [r0.1.4](https://matrix.org/docs/spec/server_server/r0.1.4.html)
- [r0.1.3](https://matrix.org/docs/spec/server_server/r0.1.3.html)
- [r0.1.2](https://matrix.org/docs/spec/server_server/r0.1.2.html)
- [r0.1.1](https://matrix.org/docs/spec/server_server/r0.1.1.html)
- [r0.1.0](https://matrix.org/docs/spec/server_server/r0.1.0.html)
* **Application Service API**
- [r0.1.1](https://matrix.org/docs/spec/application_service/r0.1.1.html)
- [r0.1.0](https://matrix.org/docs/spec/application_service/r0.1.0.html)
* **Identity Service API**
- [r0.3.0](https://matrix.org/docs/spec/identity_service/r0.3.0.html)
- [r0.2.1](https://matrix.org/docs/spec/identity_service/r0.2.1.html)
- [r0.2.0](https://matrix.org/docs/spec/identity_service/r0.2.0.html)
- [r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html)
* **Push Gateway API**
- [r0.1.0](https://matrix.org/docs/spec/push_gateway/r0.1.0.html)

Some files were not shown because too many files have changed in this diff Show More

Write Preview
Loading…
Cancel
Save