Add a room version specification

The "Room Specification" (or "Room Version Specification") is the specification that defines which room versions do what and are intended to be documents which speak the truth about how rooms operate under the hood.

The approach taken here is a bit different than other specifications. For starters, the specification is versioned in this project instead of relying on the matrix.org repository to track compiled HTML. This is done for a couple reasons, the first being we're still developing the v1 specification while concurrently making a v2 spec and the second being trying to reduce the reliance on matrix.org's repository for specifications.

Because the room spec is built into versions, some changes needed to be made. The `targets.yaml` now has a special syntax for indicating what version something is at, and the changelog generator can handle rendering different versions of the same changelog (as parsed from the RST). Some additional work has been put in to the changelog parsing to allow us to reference the v1 room spec as "v1" without having to sacrifice clarity in the changelog headings.

Finally, this moves the state resolution algorithms into the versioned
spec as a result of MSC1759 (https://github.com/matrix-org/matrix-doc/pull/1759).

Note: this does not introduce the concept of versioned schemas (tabs) that I was previously working with. There's currently no use for them, so they are shelved elsewhere.
pull/977/head
Travis Ralston 6 years ago
parent 8f1291a3e7
commit ffe577371d

@ -0,0 +1,11 @@
.. version: v2
.. Note: We set the version as "version 2" so that we can maintain a specific version
.. variable in the changelog. We already know the next version is going to be v2, so
.. this makes it easier to copy/paste unstable.rst to v2.rst
Room version 1
==============
.. version: v1
This is the first iteration of rooms in Matrix.

@ -0,0 +1,30 @@
[tool.towncrier]
filename = "../rooms.rst"
directory = "newsfragments"
issue_format = "`#{issue} <https://github.com/matrix-org/matrix-doc/issues/{issue}>`_"
title_format = "{version}"
[[tool.towncrier.type]]
directory = "breaking"
name = "Breaking Changes"
showcontent = true
[[tool.towncrier.type]]
directory = "deprecation"
name = "Deprecations"
showcontent = true
[[tool.towncrier.type]]
directory = "new"
name = "New Endpoints"
showcontent = true
[[tool.towncrier.type]]
directory = "feature"
name = "Backwards Compatible Changes"
showcontent = true
[[tool.towncrier.type]]
directory = "clarification"
name = "Spec Clarifications"
showcontent = true

@ -0,0 +1,38 @@
# How to release Room Version 2
Room versions are a bit special for the release given they have been
introduced at v2 rather than ever getting a v1 release. Additionally,
room versions have different release requirements due to the in-project
versioning of documents rather than relying on matrix.org to maintain
old generated output of specifications.
As of writing, the project is structured to support 3 logical versions
for rooms: v1, v2, and unstable. Unstable is currently pointed at v2
in order to aid development. After v2 is released, unstable may wish
to be left as an independent version or similarly be pointed at a "v3"
document.
Due to room versions being versioned in-project, the generated output
from a release is not to be sent off to matrix-doc/matrix.org. Instead,
in `gendoc.py` the default value for `--room_version` should be set to
the current release (`v2`, for example) so the index renders the right
edition in the table.
After editing `gendoc.py`, the changelog should be generated according
to the towncrier instructions. You may need to fix the `.. version: v2`
comment located in the `rooms.rst` changelog to be just underneath the
title instead of at the end of the section.
The `targets.yaml` file needs to be set up to point unstable to the
right set of files. Ensure that `unstable.rst` is referenced instead
of `v2.rst`, and that `unstable.rst` has appropriate contents.
Finally, in the `intro.rst` for room versions, re-add unstable to the
list of available versions. It is currently commented out to avoid
confusing people, so it should be possible to uncomment it and put it
back into the list.
From there, the standard process of using a release branch, tagging it,
and announcing it to the world should be followed. If required, the
various other APIs should be updated to better support the new room
version.

@ -457,7 +457,7 @@ def main(targets, dest_dir, keep_intermediates, substitutions):
rst_file = os.path.join(tmp_dir, "spec_%s.rst" % (target_name,))
if version_label:
d = os.path.join(dest_dir, target_name)
d = os.path.join(dest_dir, target_name.split('@')[0])
if not os.path.exists(d):
os.mkdir(d)
html_file = os.path.join(d, "%s.html" % version_label)
@ -529,6 +529,10 @@ if __name__ == '__main__':
"--identity_release", "-i", action="store", default="unstable",
help="The identity service release tag to generate, e.g. r1.2"
)
parser.add_argument(
"--room_version", "-r", action="store", default="unstable",
help="The current room version to advertise, e.g. v2"
)
parser.add_argument(
"--list_targets", action="store_true",
help="Do not update the specification. Instead print a list of targets.",
@ -555,6 +559,7 @@ if __name__ == '__main__':
"%APPSERVICE_RELEASE_LABEL%": args.appservice_release,
"%IDENTITY_RELEASE_LABEL%": args.identity_release,
"%PUSH_GATEWAY_RELEASE_LABEL%": args.push_gateway_release,
"%CURRENT_ROOM_VERSION%": args.room_version,
}
exit (main(args.target or ["all"], args.dest, args.nodelete, substitutions))

@ -17,8 +17,11 @@ from batesian.sections import Sections
import inspect
import json
import os
import logging
logger = logging.getLogger(__name__)
class MatrixSections(Sections):
# pass through git ver so it'll be dropped in the input file
@ -28,26 +31,19 @@ class MatrixSections(Sections):
def render_git_rev(self):
return self.units.get("git_version")["revision"]
def render_client_server_changelog(self):
changelogs = self.units.get("changelogs")
return changelogs["client_server"]
# TODO: We should make this a generic variable instead of having to add functions all the time.
def render_push_gateway_changelog(self):
changelogs = self.units.get("changelogs")
return changelogs["push_gateway"]
def render_identity_service_changelog(self):
changelogs = self.units.get("changelogs")
return changelogs["identity_service"]
def render_server_server_changelog(self):
changelogs = self.units.get("changelogs")
return changelogs["server_server"]
def render_application_service_changelog(self):
def render_changelogs(self):
rendered = {}
changelogs = self.units.get("changelogs")
return changelogs["application_service"]
for spec, versioned in changelogs.items():
spec_var = "%s_changelog" % spec
logger.info("Rendering changelog for spec: %s" % spec)
for version, changelog in versioned.items():
version_var = "%s_%s" % (spec_var, version)
logger.info("Rendering changelog for %s" % version_var)
rendered[version_var] = changelog
if version == "unstable":
rendered[spec_var] = changelog
return rendered
def _render_events(self, filterFn, sortFn):
template = self.env.get_template("events.tmpl")

@ -757,6 +757,7 @@ class MatrixUnits(Units):
is_ver = substitutions.get("%IDENTITY_RELEASE_LABEL%", "unstable")
as_ver = substitutions.get("%APPSERVICE_RELEASE_LABEL%", "unstable")
push_gw_ver = substitutions.get("%PUSH_GATEWAY_RELEASE_LABEL%", "unstable")
room_ver = substitutions.get("%CURRENT_ROOM_VERSION%", "unstable")
# we abuse the typetable to return this info to the templates
return TypeTable(rows=[
@ -780,6 +781,10 @@ class MatrixUnits(Units):
"`Push Gateway API <push_gateway/"+push_gw_ver+".html>`_",
push_gw_ver,
"Push notifications for Matrix events",
), TypeTableRow(
"`Rooms <rooms/"+room_ver+".html>`_",
room_ver,
"Specification for behaviour of rooms, such as event formats",
),
])
@ -906,11 +911,26 @@ class MatrixUnits(Units):
def load_changelogs(self):
changelogs = {}
# Changelog generation is a bit complicated. We rely on towncrier to
# generate the unstable/current changelog, but otherwise use the RST
# edition to record historical changelogs. This is done by prepending
# the towncrier output to the RST in memory, then parsing the RST by
# hand. We parse the entire changelog to create a changelog for each
# version which may be of use in some APIs.
# Map specific headers to specific keys that'll be used eventually
# in variables. Things not listed here will get lowercased and formatted
# such that characters not [a-z0-9] will be replaced with an underscore.
keyword_versions = {
"Unreleased Changes": "unstable"
}
# Only generate changelogs for things that have an RST document
for f in os.listdir(CHANGELOG_DIR):
if not f.endswith(".rst"):
continue
path = os.path.join(CHANGELOG_DIR, f)
name = f[:-4]
name = f[:-4] # take off ".rst"
# If there's a directory with the same name, we'll try to generate
# a towncrier changelog and prepend it to the general changelog.
@ -959,15 +979,39 @@ class MatrixUnits(Units):
prev_line = line
else: # have title, get body (stop on next title or EOF)
if re.match("^[=]{3,}$", line.strip()):
# we added the title in the previous iteration, pop it
# then bail out.
changelog_lines.pop()
break
# we hit another title, so pop the last line of
# the changelog and record the changelog
new_title = changelog_lines.pop()
if name not in changelogs:
changelogs[name] = {}
if title_part in keyword_versions:
title_part = keyword_versions[title_part]
title_part = title_part.strip().replace("^[a-zA-Z0-9]", "_").lower()
changelog = "".join(changelog_lines)
changelogs[name][title_part] = changelog
# reset for the next version
changelog_lines = []
title_part = new_title.strip()
continue
# Don't generate subheadings (we'll keep the title though)
if re.match("^[-]{3,}$", line.strip()):
continue
if line.strip().startswith(".. version: "):
# The changelog is directing us to use a different title
# for the changelog.
title_part = line.strip()[len(".. version: "):]
continue
if line.strip().startswith(".. "):
continue # skip comments
changelog_lines.append(" " + line + '\n')
changelogs[name] = "".join(changelog_lines)
if len(changelog_lines) > 0 and title_part is not None:
if name not in changelogs:
changelogs[name] = {}
if title_part in keyword_versions:
title_part = keyword_versions[title_part]
changelog = "".join(changelog_lines)
changelogs[name][title_part.replace("^[a-zA-Z0-9]", "_").lower()] = changelog
return changelogs

@ -16,6 +16,12 @@
Identifier Grammar
------------------
Some identifiers are specific to given room versions, please see the
`room specification`_ for more information.
.. _`room specification`: ../rooms/latest.html
Server Name
~~~~~~~~~~~
@ -78,38 +84,6 @@ Some recommendations for a choice of server name follow:
* The length of the complete server name should not exceed 230 characters.
* Server names should not use upper-case characters.
Room Versions
~~~~~~~~~~~~~
Room versions are used to change properties of rooms that may not be compatible
with other servers. For example, changing the rules for event authorization would
cause older servers to potentially end up in a split-brain situation due to them
not understanding the new rules.
A room version is defined as a string of characters which MUST NOT exceed 32
codepoints in length. Room versions MUST NOT be empty and SHOULD contain only
the characters ``a-z``, ``0-9``, ``.``, and ``-``.
Room versions are not intended to be parsed and should be treated as opaque
identifiers. Room versions consisting only of the characters ``0-9`` and ``.``
are reserved for future versions of the Matrix protocol.
The complete grammar for a legal room version is::
room_version = 1*room_version_char
room_version_char = DIGIT
/ %x61-7A ; a-z
/ "-" / "."
Examples of valid room versions are:
* ``1`` (would be reserved by the Matrix protocol)
* ``1.2`` (would be reserved by the Matrix protocol)
* ``1.2-beta``
* ``com.example.version``
Common Identifier Format
~~~~~~~~~~~~~~~~~~~~~~~~

@ -0,0 +1,70 @@
.. Copyright 2018 New Vector Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
Room Specification
==================
.. contents:: Table of Contents
.. sectnum::
Rooms are central to how Matrix operates, and have strict rules for what
is allowed to be contained within them. Rooms can also have various
algorithms that handle different tasks, such as what to do when two or
more events collide in the underlying DAG. To allow rooms to be improved
upon through new algorithms or rules, "room versions" are employed to
manage a set of expectations for each room.
Room version grammar
--------------------
Room versions are used to change properties of rooms that may not be compatible
with other servers. For example, changing the rules for event authorization would
cause older servers to potentially end up in a split-brain situation due to them
not understanding the new rules.
A room version is defined as a string of characters which MUST NOT exceed 32
codepoints in length. Room versions MUST NOT be empty and SHOULD contain only
the characters ``a-z``, ``0-9``, ``.``, and ``-``.
Room versions are not intended to be parsed and should be treated as opaque
identifiers. Room versions consisting only of the characters ``0-9`` and ``.``
are reserved for future versions of the Matrix protocol.
The complete grammar for a legal room version is::
room_version = 1*room_version_char
room_version_char = DIGIT
/ %x61-7A ; a-z
/ "-" / "."
Examples of valid room versions are:
* ``1`` (would be reserved by the Matrix protocol)
* ``1.2`` (would be reserved by the Matrix protocol)
* ``1.2-beta``
* ``com.example.version``
Other room versions
-------------------
The available room versions are:
* `Version 1 <v1.html>`_ - The current version of most rooms.
* `Version 2 <v2.html>`_ - Currently in development.
.. Note: the 'unstable' version is commented out pending a real release of rooms v2
.. See meta/releasing-rooms-v2.md
.. * `Unstable <unstable.html>`_ - The upcoming version of the room specification.

@ -0,0 +1,54 @@
.. Copyright 2018 New Vector Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
.. DEV NOTE: This is stubbed as a template and not actually used anywhere.
.. See v2.rst for the "unstable" room version, which is currently under
.. development.
..
.. See meta/releasing-rooms-v2.md
.. Note: This document appended to the end of the intro, so this next line
.. appears under "Other Room Versions".
.. Warning::
This is the specification for unreleased changes to rooms. The stability
of rooms using this specification cannot be guaranteed.
Changelog
---------
.. topic:: unstable
{{rooms_changelog_unstable}}
This version of the specification is generated from
`matrix-doc <https://github.com/matrix-org/matrix-doc>`_ as of Git commit
`{{git_version}} <https://github.com/matrix-org/matrix-doc/tree/{{git_rev}}>`_.
For the full historical changelog, see
https://github.com/matrix-org/matrix-doc/blob/master/changelogs/rooms.rst
Some Module
-----------
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In sit amet
eros turpis. Quisque commodo diam vel massa ultrices, vel egestas eros
dignissim. Sed sit amet lacus eget metus auctor malesuada at ut odio.
In turpis leo, viverra et mi porttitor, condimentum bibendum dolor.
.. {-{versioned_test_definition}-}

@ -0,0 +1,112 @@
.. Copyright 2018 New Vector Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
.. Note: This document appended to the end of the intro, so this next line
.. appears under "Other Room Versions".
This is the specification for **room version 1** (``"1"``).
Changelog
---------
.. topic:: Room version 1
{{rooms_changelog_v1}}
This version of the specification is generated from
`matrix-doc <https://github.com/matrix-org/matrix-doc>`_ as of Git commit
`{{git_version}} <https://github.com/matrix-org/matrix-doc/tree/{{git_rev}}>`_.
For the full historical changelog, see
https://github.com/matrix-org/matrix-doc/blob/master/changelogs/rooms.rst
Server implementation components
--------------------------------
.. WARNING::
The information contained in this section is strictly for server implementors.
Applications which use the Client-Server API are generally unaffected by the
details contained here, and can safely ignore their presence.
The algorithms defined here should only apply to version 1 rooms. Other algorithms
may be used by other room versions, and as such servers should be aware of which
version room they are dealing with prior to executing a given algorithm.
.. WARNING::
Although room version 1 is the most popular room version, it is known to have
undesirable effects. Servers implementing support for room version 1 should be
aware that restrictions should be generally relaxed and be aware that inconsistencies
may occur until room version 2 is ready and adopted.
State resolution
~~~~~~~~~~~~~~~~
.. WARNING::
This section documents the state resolution algorithm as implemented by
Synapse as of December 2017 (and therefore the de-facto Matrix protocol).
However, this algorithm is known to have some problems.
The room state :math:`S'(E)` after an event :math:`E` is defined in terms of
the room state :math:`S(E)` before :math:`E`, and depends on whether
:math:`E` is a state event or a message event:
* If :math:`E` is a message event, then :math:`S'(E) = S(E)`.
* If :math:`E` is a state event, then :math:`S'(E)` is :math:`S(E)`, except
that its entry corresponding to :math:`E`'s ``event_type`` and ``state_key``
is replaced by :math:`E`'s ``event_id``.
The room state :math:`S(E)` before :math:`E` is the *resolution* of the set of
states :math:`\{ S'(E'), S'(E''), … \}` consisting of the states after each of
:math:`E`'s ``prev_event``\s :math:`\{ E', E'', … \}`.
The *resolution* of a set of states is defined as follows. The resolved state
is built up in a number of passes; here we use :math:`R` to refer to the
results of the resolution so far.
* Start by setting :math:`R` to the union of the states to be resolved,
excluding any *conflicting* events.
* First we resolve conflicts between ``m.room.power_levels`` events. If there
is no conflict, this step is skipped, otherwise:
* Assemble all the ``m.room.power_levels`` events from the states to
be resolved into a list.
* Sort the list by ascending ``depth`` then descending ``sha1(event_id)``.
* Add the first event in the list to :math:`R`.
* For each subsequent event in the list, check that the event would be
allowed by the `authorization rules`_ for a room in state :math:`R`. If the
event would be allowed, then update :math:`R` with the event and continue
with the next event in the list. If it would not be allowed, stop and
continue below with ``m.room.join_rules`` events.
* Repeat the above process for conflicts between ``m.room.join_rules`` events.
* Repeat the above process for conflicts between ``m.room.member`` events.
* No other events affect the authorization rules, so for all other conflicts,
just pick the event with the highest depth and lowest ``sha1(event_id)`` that
passes authentication in :math:`R` and add it to :math:`R`.
A *conflict* occurs between states where those states have different
``event_ids`` for the same ``(state_type, state_key)``. The events thus
affected are said to be *conflicting* events.
.. _`authorization rules`: ../server_server/unstable.html#authorization-rules

@ -0,0 +1,181 @@
.. Copyright 2018 New Vector Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
.. Note: This document appended to the end of the intro, so this next line
.. appears under "Other Room Versions".
This is the specification for **room version 2** (``"2"``).
.. Warning::
Room version 2 is under development and cannot be relied on in production
environments.
Changelog
---------
.. topic:: Room version 2
{{rooms_changelog_v2}}
This version of the specification is generated from
`matrix-doc <https://github.com/matrix-org/matrix-doc>`_ as of Git commit
`{{git_version}} <https://github.com/matrix-org/matrix-doc/tree/{{git_rev}}>`_.
For the full historical changelog, see
https://github.com/matrix-org/matrix-doc/blob/master/changelogs/rooms.rst
Server implementation components
--------------------------------
.. WARNING::
The information contained in this section is strictly for server implementors.
Applications which use the Client-Server API are generally unaffected by the
details contained here, and can safely ignore their presence.
The algorithms defined here should only apply to version 2 rooms. Other algorithms
may be used by other room versions, and as such servers should be aware of which
version room they are dealing with prior to executing a given algorithm.
State resolution
~~~~~~~~~~~~~~~~
The room state :math:`S'(E)` after an event :math:`E` is defined in terms of
the room state :math:`S(E)` before :math:`E`, and depends on whether
:math:`E` is a state event or a message event:
* If :math:`E` is a message event, then :math:`S'(E) = S(E)`.
* If :math:`E` is a state event, then :math:`S'(E)` is :math:`S(E)`, except
that its entry corresponding to :math:`E`'s ``event_type`` and ``state_key``
is replaced by :math:`E`'s ``event_id``.
The room state :math:`S(E)` before :math:`E` is the *resolution* of the set of
states :math:`\{ S'(E_1), S'(E_2), … \}` consisting of the states after each of
:math:`E`'s ``prev_event``\s :math:`\{ E_1, E_2, … \}`, where the resolution of
a set of states is given in the algorithm below.
Definitions
+++++++++++
The state resolution algorithm for version 2 rooms uses the following
definitions, given the set of room states :math:`\{ S_1, S_2, \ldots \}`:
Power events
A *power event* is a state event with type ``m.room.power_levels`` or
``m.room.join_rules``, or a state event with type ``m.room.member`` where the
``membership`` is ``leave`` or ``ban`` and the ``sender`` does not match the
``state_key``. The idea behind this is that power events are events that have
may remove someone's ability to do something in the room.
Unconflicted state map and conflicted state set
The *unconflicted state map* is the state where the value of each key exists
and is the same in each state :math:`S_i`. The *conflicted state set* is the
set of all other state events. Note that the unconflicted state map only has
one event per ``(event_type, state_key)``, whereas the conflicted state set
may have multiple events.
Auth difference
The *auth difference* is calculated by first calculating the full auth chain
for each state :math:`S_i`, that is the union of the auth chains for each
event in :math:`S_i`, and then taking every event that doesn't appear in
every auth chain. If :math:`C_i` is the full auth chain of :math:`S_i`, then
the auth difference is :math:`\cup C_i - \cap C_i`.
Full conflicted set
The *full conflicted set* is the union of the conflicted state set and the
auth difference.
Reverse topological power ordering
The *reverse topological power ordering* of a set of events is the
lexicographically smallest topological ordering based on the DAG formed by
auth events. The reverse topological power ordering is ordered from earliest
event to latest. For comparing two topological orderings to determine which
is the lexicographically smallest, the following comparison relation on
events is used: for events :math:`x` and :math:`y`, :math:`x<y` if
1. :math:`x`'s sender has *greater* power level than :math:`y`'s sender,
when looking at their respective ``auth_event``\s; or
2. the senders have the same power level, but :math:`x`'s
``origin_server_ts`` is *less* than :math:`y`'s ``origin_server_ts``; or
3. the senders have the same power level and the events have the same
``origin_server_ts``, but :math:`x`'s ``event_id`` is *less* than
:math:`y`'s ``event_id``.
The reverse topological power ordering can be found by sorting the events
using Kahn's algorithm for topological sorting, and at each step selecting,
among all the candidate vertices, the smallest vertex using the above
comparison relation.
Mainline ordering
Given an ``m.room.power_levels`` event :math:`P`, the *mainline of* :math:`P`
is the list of events generated by starting with :math:`P` and recursively
taking the ``m.room.power_levels`` events from the ``auth_events``, ordered
such that :math:`P` is last. Given another event :math:`e`, the *closest
mainline event to* :math:`e` is the first event encountered in the mainline
when iteratively descending through the ``m.room.power_levels`` events in the
``auth_events`` starting at :math:`e`. If no mainline event is encountered
when iteratively descending through the ``m.room.power_levels`` events, then
the closest mainline event to :math:`e` can be considered to be a dummy event
that is before any other event in the mainline of :math:`P` for the purposes
of condition 1 below.
The *mainline ordering based on* :math:`P` of a set of events is the
ordering, from smallest to largest, using the following comparision relation
on events: for events :math:`x` and :math:`y`, :math:`x<y` if
1. the closest mainline event to :math:`x` appears *before* the closest
mainline event to :math:`y`; or
2. the closest mainline events are the same, but :math:`x`\'s
``origin_server_ts`` is *less* than :math:`y`\'s ``origin_server_ts``; or
3. the closest mainline events are the same and the events have the same
``origin_server_ts``, but :math:`x`\'s ``event_id`` is *less* than
:math:`y`\'s ``event_id``.
Iterative auth checks
The *iterative auth checks algorithm* takes as input an initial room state
and a sorted list of state events, and constructs a new room state by
iterating through the event list and applying the state event to the room
state if the state event is allowed by the `authorization rules`_. If the
state event is not allowed by the authorization rules, then the event is
ignored. If a ``(event_type, state_key)`` key that is required for checking
the authorization rules is not present in the state, then the appropriate
state event from the event's ``auth_events`` is used.
Algorithm
+++++++++
The *resolution* of a set of states is obtained as follows:
1. Take all *power events* and any events in their auth chains, recursively,
that appear in the *full conflicted set* and order them by the *reverse
topological power ordering*.
2. Apply the *iterative auth checks algorithm* on the *unconflicted state map*
and the list of events from the previous step to get a partially resolved
state.
3. Take all remaining events that weren't picked in step 1 and order them by
the mainline ordering based on the power level in the partially resolved
state obtained in step 2.
4. Apply the *iterative auth checks algorithm* on the partial resolved
state and the list of events from the previous step.
5. Update the result by replacing any event with the event with the same key
from the *unconflicted state map*, if such an event exists, to get the final
resolved state.
.. _`authorization rules`: ../server_server/unstable.html#authorization-rules

@ -752,191 +752,9 @@ is at the top)::
Suppose E3 and E4 are both ``m.room.name`` events which set the name of the
room. What should the name of the room be at E5?
Servers should follow one of the following recursively-defined algorithms,
depending on the room version, to determine the room state at a given point on
the DAG.
State resolution algorithm for version 2 rooms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The room state :math:`S'(E)` after an event :math:`E` is defined in terms of
the room state :math:`S(E)` before :math:`E`, and depends on whether
:math:`E` is a state event or a message event:
* If :math:`E` is a message event, then :math:`S'(E) = S(E)`.
* If :math:`E` is a state event, then :math:`S'(E)` is :math:`S(E)`, except
that its entry corresponding to :math:`E`'s ``event_type`` and ``state_key``
is replaced by :math:`E`'s ``event_id``.
The room state :math:`S(E)` before :math:`E` is the *resolution* of the set of
states :math:`\{ S'(E_1), S'(E_2), … \}` consisting of the states after each of
:math:`E`'s ``prev_event``\s :math:`\{ E_1, E_2, … \}`, where the resolution of
a set of states is given in the algorithm below.
Definitions
+++++++++++
The state resolution algorithm for version 2 rooms uses the following
definitions, given the set of room states :math:`\{ S_1, S_2, \ldots \}`:
Power events
A *power event* is a state event with type ``m.room.power_levels`` or
``m.room.join_rules``, or a state event with type ``m.room.member`` where the
``membership`` is ``leave`` or ``ban`` and the ``sender`` does not match the
``state_key``. The idea behind this is that power events are events that have
may remove someone's ability to do something in the room.
Unconflicted state map and conflicted state set
The *unconflicted state map* is the state where the value of each key exists
and is the same in each state :math:`S_i`. The *conflicted state set* is the
set of all other state events. Note that the unconflicted state map only has
one event per ``(event_type, state_key)``, whereas the conflicted state set
may have multiple events.
Auth difference
The *auth difference* is calculated by first calculating the full auth chain
for each state :math:`S_i`, that is the union of the auth chains for each
event in :math:`S_i`, and then taking every event that doesn't appear in
every auth chain. If :math:`C_i` is the full auth chain of :math:`S_i`, then
the auth difference is :math:`\cup C_i - \cap C_i`.
Full conflicted set
The *full conflicted set* is the union of the conflicted state set and the
auth difference.
Reverse topological power ordering
The *reverse topological power ordering* of a set of events is the
lexicographically smallest topological ordering based on the DAG formed by
auth events. The reverse topological power ordering is ordered from earliest
event to latest. For comparing two topological orderings to determine which
is the lexicographically smallest, the following comparison relation on
events is used: for events :math:`x` and :math:`y`, :math:`x<y` if
1. :math:`x`'s sender has *greater* power level than :math:`y`'s sender,
when looking at their respective ``auth_event``\s; or
2. the senders have the same power level, but :math:`x`'s
``origin_server_ts`` is *less* than :math:`y`'s ``origin_server_ts``; or
3. the senders have the same power level and the events have the same
``origin_server_ts``, but :math:`x`'s ``event_id`` is *less* than
:math:`y`'s ``event_id``.
The reverse topological power ordering can be found by sorting the events
using Kahn's algorithm for topological sorting, and at each step selecting,
among all the candidate vertices, the smallest vertex using the above
comparison relation.
Mainline ordering
Given an ``m.room.power_levels`` event :math:`P`, the *mainline of* :math:`P`
is the list of events generated by starting with :math:`P` and recursively
taking the ``m.room.power_levels`` events from the ``auth_events``, ordered
such that :math:`P` is last. Given another event :math:`e`, the *closest
mainline event to* :math:`e` is the first event encountered in the mainline
when iteratively descending through the ``m.room.power_levels`` events in the
``auth_events`` starting at :math:`e`. If no mainline event is encountered
when iteratively descending through the ``m.room.power_levels`` events, then
the closest mainline event to :math:`e` can be considered to be a dummy event
that is before any other event in the mainline of :math:`P` for the purposes
of condition 1 below.
The *mainline ordering based on* :math:`P` of a set of events is the
ordering, from smallest to largest, using the following comparision relation
on events: for events :math:`x` and :math:`y`, :math:`x<y` if
1. the closest mainline event to :math:`x` appears *before* the closest
mainline event to :math:`y`; or
2. the closest mainline events are the same, but :math:`x`\'s
``origin_server_ts`` is *less* than :math:`y`\'s ``origin_server_ts``; or
3. the closest mainline events are the same and the events have the same
``origin_server_ts``, but :math:`x`\'s ``event_id`` is *less* than
:math:`y`\'s ``event_id``.
Iterative auth checks
The *iterative auth checks algorithm* takes as input an initial room state
and a sorted list of state events, and constructs a new room state by
iterating through the event list and applying the state event to the room
state if the state event is allowed by the `authorization rules`_. If the
state event is not allowed by the authorization rules, then the event is
ignored. If a ``(event_type, state_key)`` key that is required for checking
the authorization rules is not present in the state, then the appropriate
state event from the event's ``auth_events`` is used.
Algorithm
+++++++++
The *resolution* of a set of states is obtained as follows:
1. Take all *power events* and any events in their auth chains, recursively,
that appear in the *full conflicted set* and order them by the *reverse
topological power ordering*.
2. Apply the *iterative auth checks algorithm* on the *unconflicted state map*
and the list of events from the previous step to get a partially resolved
state.
3. Take all remaining events that weren't picked in step 1 and order them by
the mainline ordering based on the power level in the partially resolved
state obtained in step 2.
4. Apply the *iterative auth checks algorithm* on the partial resolved
state and the list of events from the previous step.
5. Update the result by replacing any event with the event with the same key
from the *unconflicted state map*, if such an event exists, to get the final
resolved state.
State resolution algorithm for version 1 rooms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. WARNING::
This section documents the state resolution algorithm as implemented by
Synapse as of December 2017 (and therefore the de-facto Matrix protocol).
However, this algorithm is known to have some problems.
The room state :math:`S'(E)` after an event :math:`E` is defined in terms of
the room state :math:`S(E)` before :math:`E`, and depends on whether
:math:`E` is a state event or a message event:
* If :math:`E` is a message event, then :math:`S'(E) = S(E)`.
* If :math:`E` is a state event, then :math:`S'(E)` is :math:`S(E)`, except
that its entry corresponding to :math:`E`'s ``event_type`` and ``state_key``
is replaced by :math:`E`'s ``event_id``.
The room state :math:`S(E)` before :math:`E` is the *resolution* of the set of
states :math:`\{ S'(E'), S'(E''), … \}` consisting of the states after each of
:math:`E`'s ``prev_event``\s :math:`\{ E', E'', … \}`.
The *resolution* of a set of states is defined as follows. The resolved state
is built up in a number of passes; here we use :math:`R` to refer to the
results of the resolution so far.
* Start by setting :math:`R` to the union of the states to be resolved,
excluding any *conflicting* events.
* First we resolve conflicts between ``m.room.power_levels`` events. If there
is no conflict, this step is skipped, otherwise:
* Assemble all the ``m.room.power_levels`` events from the states to
be resolved into a list.
* Sort the list by ascending ``depth`` then descending ``sha1(event_id)``.
* Add the first event in the list to :math:`R`.
* For each subsequent event in the list, check that the event would be
allowed by the `authorization rules`_ for a room in state :math:`R`. If the
event would be allowed, then update :math:`R` with the event and continue
with the next event in the list. If it would not be allowed, stop and
continue below with ``m.room.join_rules`` events.
* Repeat the above process for conflicts between ``m.room.join_rules`` events.
* Repeat the above process for conflicts between ``m.room.member`` events.
* No other events affect the authorization rules, so for all other conflicts,
just pick the event with the highest depth and lowest ``sha1(event_id)`` that
passes authentication in :math:`R` and add it to :math:`R`.
A *conflict* occurs between states where those states have different
``event_ids`` for the same ``(state_type, state_key)``. The events thus
affected are said to be *conflicting* events.
Servers must use the appropriate recursively-defined algorithm as required
by the room version. For a description of each room version's algorithm, please
see the `room version specification`_ .
Backfilling and retrieving missing events
@ -1500,3 +1318,4 @@ Example code
.. _`Checking for a signature`: ../appendices.html#checking-for-a-signature
.. _`Device Management module`: ../client_server/%CLIENT_RELEASE_LABEL%.html#device-management
.. _`End-to-End Encryption module`: ../client_server/%CLIENT_RELEASE_LABEL%.html#end-to-end-encryption
.. _`room version specification`: ../rooms/latest.html

@ -26,6 +26,35 @@ targets:
files:
- push_gateway.rst
version_label: "%PUSH_GATEWAY_RELEASE_LABEL%"
rooms:
files:
- rooms/intro.rst
# TODO: Switch this back to unstable.rst after releasing rooms v2
# We temporarily do this so that "unstable" points to the in-dev
# version, however we may also want to hardlink to v2 in places and
# thus we maintain a v2 version of the same doc.
#
# See meta/releasing-rooms-v2.md
#- rooms/unstable.rst
- rooms/v2.rst
version_label: unstable
rooms@v1: # this is translated to be rooms/v1.html
files:
- rooms/intro.rst
- rooms/v1.rst
version_label: v1
rooms@v2: # this is translated to be rooms/v2.html
files:
- rooms/intro.rst
- rooms/v2.rst
version_label: v2
rooms@latest: # this is translated to be rooms/latest.html
files:
- rooms/intro.rst
- rooms/v1.rst # TODO: Find a better way to link to the v2 spec
version_label: latest
appendices:
files:
- appendices.rst

Loading…
Cancel
Save