Merge remote-tracking branch 'remotes/origin/master'

pull/977/head
Kitsune Ral 7 years ago
commit 66268260c7

1
.gitignore vendored

@ -1,6 +1,7 @@
/api/node_modules
/assets
/assets.tar.gz
/env
/scripts/gen
/scripts/continuserv/continuserv
/scripts/speculator/speculator

@ -1,13 +1,22 @@
Contributing to matrix-doc
==========================
Everyone is welcome to contribute to the ``matrix-doc`` project, provided that they
are willing to license their contributions under the same license as the
project itself. We follow a simple 'inbound=outbound' model for contributions:
the act of submitting an 'inbound' contribution means that the contributor
agrees to license the code under the same terms as the project's overall
'outbound' license - in our case, this is Apache Software License
v2 (see LICENSE).
Everyone is welcome to contribute to the Matrix specification!
Please ensure that you sign off your contributions. See `Sign off`_ below.
Code style
----------
The documentation style is described at
https://github.com/matrix-org/matrix-doc/blob/master/meta/documentation_style.rst.
Python code within the ``matrix-doc`` project should follow the same style as
synapse, which is documented at
https://github.com/matrix-org/synapse/tree/master/docs/code_style.rst.
Matrix-doc workflows
--------------------
Specification changes
~~~~~~~~~~~~~~~~~~~~~
@ -29,8 +38,8 @@ workflow:
The Matrix Core Team's preferred tool for such discussion documents is
`Google Docs <https://docs.google.com>`_ thanks to its support for comment
threads. Works in progress are kept in a folder at
https://drive.google.com/drive/folders/0B4wHq8qP86r2ck15MHEwMmlNVUk.
threads. Works in progress are kept in the `Matrix Design drafts folder
<https://drive.google.com/drive/folders/0B4wHq8qP86r2ck15MHEwMmlNVUk>`_.
2. Seek feedback on the proposal. `#matrix-dev:matrix.org
<http://matrix.to/#/#matrix-dev:matrix.org>`_ is a good place to reach the
@ -59,43 +68,46 @@ The above process is unnecessary for smaller changes, and those which do not
put new requirements on servers. This category of changes includes the
following:
* changes to supporting documentation
* Changes to the scripts used to generate the specification.
* changes to the scripts used to generate the specification
* Addition of features which have been in use in practice for some time, but
have never made it into the spec (including anything with the `spec-omission
<https://github.com/matrix-org/matrix-doc/labels/spec-omission>`_ label).
* clarifications to the specification which do not change the behaviour of
Matrix servers or clients in a way which might introduce compatibility
problems for existing deployments. For example, recommendations for UI
behaviour do not require a proposal document. On the other hand, changes to
event contents would be best discussed in a proposal document even though no
changes would be necessary to server implementations.
* Likewise, corrections to the specification, to fix situations where, in
practice, servers and clients behave differently to the specification,
including anything with the `spec-bug
<https://github.com/matrix-org/matrix-doc/labels/spec-bug>`_ label.
For such changes, please do just open a `pull request`_.
(If there is any doubt about whether it is the spec or the implementations
that need fixing, please discuss it with us first in `#matrix-dev:matrix.org
<http://matrix.to/#/#matrix-dev:matrix.org>`_.)
* Clarifications to the specification which do not change the behaviour of
Matrix servers or clients in a way which might introduce compatibility
problems for existing deployments. This includes anything with the
`clarification <https://github.com/matrix-org/matrix-doc/labels/clarification>`_
label.
Pull requests
~~~~~~~~~~~~~
.. _pull request: `Pull requests`_
The preferred and easiest way to contribute changes to the ``matrix-doc`` project
is to fork it on github, and then create a pull request to ask us to pull your
changes into our repo (https://help.github.com/articles/using-pull-requests/).
For example, recommendations for UI behaviour do not require a proposal
document. On the other hand, changes to event contents would be best
discussed in a proposal document even though no changes would be necessary to
server implementations.
(Note that, unlike most of the other matrix.org projects, pull requests for
matrix-doc should be based on the ``master`` branch.)
For such changes, please do just open a `pull request`_.
Code style
~~~~~~~~~~
.. _pull request: https://help.github.com/articles/about-pull-requests
The documentation style is described at
https://github.com/matrix-org/matrix-doc/blob/master/meta/documentation_style.rst.
Sign off
--------
Python code within the ``matrix-doc`` project should follow the same style as
synapse, which is documented at
https://github.com/matrix-org/synapse/tree/master/docs/code_style.rst.
We ask that everybody who contributes to their project signs off their
contributions, as explained below.
Sign off
~~~~~~~~
We follow a simple 'inbound=outbound' model for contributions: the act of
submitting an 'inbound' contribution means that the contributor agrees to
license their contribution under the same terms as the project's overall 'outbound'
license - in our case, this is Apache Software License v2 (see LICENSE).
In order to have a concrete record that your contribution is intentional
and you agree to license it under the same terms as the project's license, we've adopted the

@ -1,39 +1,194 @@
This repository contains the documentation for Matrix.
This repository contains the Matrix specification.
Structure
=========
If you want to ask more about the specification, join us on
`#matrix-dev:matrix.org <http://matrix.to/#/#matrix-dev:matrix.org>`_.
We welcome contributions to the spec! See the notes below on `Building the
specification`_, and `<CONTRIBUTING.rst>`_ to get started making contributions.
Note that the Matrix Project lists, which were previously kept in this
repository, are now in https://github.com/matrix-org/matrix.org.
- ``api`` : Contains the HTTP API specification.
- ``attic``: Contains historical sections of specification for reference
Structure of this repository
============================
- ``api`` : `OpenAPI`_ (swagger) specifications for the the HTTP APIs.
- ``attic``: historical sections of specification for reference
purposes.
- ``changelogs``: Contains change logs for the various parts of the
- ``changelogs``: change logs for the various parts of the
specification.
- ``drafts``: Previously, contained documents which were under discussion for
future incusion into the specification and/or supporting documentation. This
is now historical, as we use separate discussion documents (see
`<CONTRIBUTING.rst>`_).
- ``event-schemas``: Contains the `JSON Schema`_ for all Matrix events
- ``event-schemas``: the `JSON Schema`_ for all Matrix events
contained in the specification, along with example JSON files.
- ``meta``: Contains documents outlining the processes involved when writing
- ``meta``: documents outlining the processes involved when writing
documents, e.g. documentation style, guidelines.
- ``scripts``: Contains scripts to generate formatted versions of the
- ``scripts``: scripts to generate formatted versions of the
documentation, typically HTML.
- ``specification``: Contains the specification split up into sections.
- ``supporting-docs``: Contains additional documents which explain design
decisions, examples, use cases, etc.
- ``templating``: Contains the templates and templating system used to
generate the spec.
- ``specification``: the specification split up into sections.
Contributing
============
.. _OpenAPI: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md
.. _JSON Schema: http://json-schema.org/
Known issues with the specification are represented as JIRA issues at
`<https://matrix.org/jira/browse/SPEC>`_.
Building the specification
==========================
If you want to ask more about the specification, join us on
`#matrix-dev:matrix.org <http://matrix.to/#/#matrix-dev:matrix.org>`_.
The Matrix Spec is generated by a set of scripts, from the RST documents, API
specs and event schemas in this repository.
If you would like to contribute to the specification or supporting
documentation, see `<CONTRIBUTING.rst>`_.
Preparation
-----------
.. _JSON Schema: http://json-schema.org/
To use the scripts, it is best to create a Python virtualenv as follows::
virtualenv env
env/bin/pip install -r scripts/requirements.txt
(Benjamin Synders has contributed a script for `Nix`_ users, which can be
invoked with ``nix-shell scripts/contrib/shell.nix``.)
.. TODO: Possibly we need some libs installed; should record what they are.
.. _`Nix`: https://nixos.org/nix/
Generating the specification
----------------------------
To rebuild the specification, use ``scripts/gendoc.py``::
source env/bin/activate
./scripts/gendoc.py
The above will write the rendered version of the specification to
``scripts/gen``. To view it, point your browser at ``scripts/gen/index.html``.
Generating the OpenAPI (Swagger) specs
--------------------------------------
`Swagger`_ is a framework for representing RESTful APIs. We use it to generate
interactive documentation for our APIs.
Before the Swagger docs can be used in the Swagger UI (or other tool expecting
a Swagger specs, they must be combined into a single json file. This can be
done as follows::
source env/bin/activate
./dump-swagger.py
By default, ``dump-swagger`` will write to ``scripts/swagger/api-docs.json``.
To make use of the generated file, there are a number of options:
* It can be uploaded from your filesystem to an online editor/viewer such as
http://editor.swagger.io/
* You can run a local HTTP server by running
``./scripts/swagger-http-server.py``, and then view the documentation via an
online viewer; for example, at
http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json
* You can host the swagger UI yourself. See
https://github.com/swagger-api/swagger-ui#how-to-run for advice on how to do
so.
.. _`Swagger`: http://swagger.io/
Continuserv
-----------
Continuserv is a script which will rebuild the specification every time a file
is changed, and will serve it to a browser over HTTP. It is intended for use by
specification authors, so that they can quickly see the effects of their
changes.
It is written in Go, so you will need the ``go`` compiler installed on your
computer. You will also need to install fsnotify by running::
go get gopkg.in/fsnotify.v1
Then, create a virtualenv as described above under `Preparation`_,
and::
source env/bin/activate
go run ./scripts/continuserv/main.go
You will then be able to view the generated spec by visiting
http://localhost:8000/index.html.
Issue tracking
==============
Issues with the Matrix specification are tracked in `GitHub
<https://github.com/matrix-org/matrix-doc/issues>`_.
The following labels are used to help categorize issues:
`spec-omission <https://github.com/matrix-org/matrix-doc/labels/spec-omission>`_
--------------------------------------------------------------------------------
Things which have been implemented but not currently specified. These may range
from entire API endpoints, to particular options or return parameters.
Issues with this label will have been implemented in `Synapse
<https://github.com/matrix-org/synapse>`_. Normally there will be a design
document in Google Docs or similar which describes the feature.
Examples:
* `Spec PUT /directory/list <https://github.com/matrix-org/matrix-doc/issues/417>`_
* `Unspec'd server_name request param for /join/{roomIdOrAlias}
<https://github.com/matrix-org/matrix-doc/issues/904>`_
`clarification <https://github.com/matrix-org/matrix-doc/labels/clarification>`_
--------------------------------------------------------------------------------
An area where the spec could do with being more explicit.
Examples:
* `Spec the implicit limit on /syncs
<https://github.com/matrix-org/matrix-doc/issues/708>`_
* `Clarify the meaning of the currently_active flags in presence events
<https://github.com/matrix-org/matrix-doc/issues/686>`_
`spec-bug <https://github.com/matrix-org/matrix-doc/labels/spec-bug>`_
----------------------------------------------------------------------
Something which is in the spec, but is wrong.
Note: this is *not* for things that are badly designed or don't work well
(for which see 'improvement' or 'feature') - it is for places where the
spec doesn't match reality.
Examples:
* `swagger is wrong for directory PUT
<https://github.com/matrix-org/matrix-doc/issues/933>`_
* `receipts section still refers to initialSync
<https://github.com/matrix-org/matrix-doc/issues/695>`_
`improvement <https://github.com/matrix-org/matrix-doc/labels/improvement>`_
----------------------------------------------------------------------------
A suggestion for a relaatively simple improvement to the protocol.
Examples:
* `We need a 'remove 3PID' API so that users can remove mappings
<https://github.com/matrix-org/matrix-doc/issues/620>`_
* `We should mandate that /publicRooms requires an access_token
<https://github.com/matrix-org/matrix-doc/issues/612>`_
`feature <https://github.com/matrix-org/matrix-doc/labels/feature>`_
--------------------------------------------------------------------
A suggestion for a significant extension to the matrix protocol which
needs considerable consideration before implementation.
Examples:
* `Peer-to-peer Matrix <https://github.com/matrix-org/matrix-doc/issues/710>`_
* `Specify a means for clients to "edit" previous messages
<https://github.com/matrix-org/matrix-doc/issues/682>`_

@ -1,3 +1,2 @@
This directory contains swagger-compatible representations of our APIs. See
scripts/README.md for details on how to make use of them.
the main README.rst for details on how to make use of them.

@ -37,6 +37,9 @@ paths:
the new room, including checking power levels for each event. It MUST
apply the events implied by the request in the following order:
0. A default ``m.room.power_levels`` event, giving the room creator
(and not other members) permission to send state events.
1. Events set by ``presets``.
2. Events listed in ``initial_state``, in the order that they are

@ -12,6 +12,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
properties:
event_id:
description: The ID of this event, if applicable.
type: string
content:
description: The content of this event. The fields in this object will vary depending
on the type of event.

@ -149,5 +149,7 @@ paths:
type: object
allOf:
- $ref: "definitions/sync_filter.yaml"
404:
description: "Unknown filter."
tags:
- Room participation

@ -45,6 +45,16 @@ paths:
If the client does not supply a ``device_id``, the server must
auto-generate one.
The server SHOULD register an account with a User ID based on the
``username`` provided, if any. Note that the grammar of Matrix User ID
localparts is restricted, so the server MUST either map the provided
``username`` onto a ``user_id`` in a logical manner, or reject
``username``\s which do not comply to the grammar, with
``M_INVALID_USERNAME``.
Matrix clients MUST NOT assume that localpart of the registered
``user_id`` matches the provided ``username``.
The returned access token must be associated with the ``device_id``
supplied by the client or generated by the server. The server may
invalidate any access token previously associated with that device. See
@ -87,7 +97,7 @@ paths:
username:
type: string
description: |-
The local part of the desired Matrix ID. If omitted,
The basis for the localpart of the desired Matrix ID. If omitted,
the homeserver MUST generate a Matrix ID local part.
example: cheeky_monkey
password:
@ -122,7 +132,11 @@ paths:
properties:
user_id:
type: string
description: The fully-qualified Matrix ID that has been registered.
description: |-
The fully-qualified Matrix user ID (MXID) that has been registered.
Any user ID returned by this API must conform to the grammar given in the
`Matrix specification <https://matrix.org/docs/spec/appendices.html#user-identifiers>`_.
access_token:
type: string
description: |-

@ -89,8 +89,12 @@ paths:
filter:
type: object
title: Filter
# Within the C-S spec document, `filter`_ is picked up
# as a link to the filtering section. In OpenAPI 3.0,
# we could use the link feature, but we're still on 2.0
# for now :/
description: |-
This takes a `filter <https://matrix.org/docs/spec/%CLIENT_RELEASE_LABEL%/client_server.html#filtering>`_.
This takes a `filter`_.
order_by:
title: "Ordering"
type: string

@ -48,6 +48,9 @@
(`#751 <https://github.com/matrix-org/matrix-doc/pull/751>`_).
- Add key distribution APIs, for use with end-to-end encryption.
(`#894 <https://github.com/matrix-org/matrix-doc/pull/894>`_).
- Add ``m.room.pinned_events`` state event for rooms.
(`#1007 <https://github.com/matrix-org/matrix-doc/pull/1007>`_).
- Add mention of ability to send Access Token via an Authorization Header.
- New endpoints:
@ -62,11 +65,11 @@
- Add endpoints and logic for invites and third-party invites to the federation
spec and update the JSON of the request sent by the identity server upon 3PID
binding
(`#997 <https://github.com/matrix-org/matrix-doc/pull/997>`)
(`#997 <https://github.com/matrix-org/matrix-doc/pull/997>`_)
- Fix "membership" property on third-party invite upgrade example
(`#995 <https://github.com/matrix-org/matrix-doc/pull/995>`)
(`#995 <https://github.com/matrix-org/matrix-doc/pull/995>`_)
- Fix response format and 404 example for room alias lookup
(`#960 <https://github.com/matrix-org/matrix-doc/pull/960>`)
(`#960 <https://github.com/matrix-org/matrix-doc/pull/960>`_)
- Fix examples of ``m.room.member`` event and room state change,
and added a clarification on the membership event sent upon profile update
(`#950 <https://github.com/matrix-org/matrix-doc/pull/950>`_).
@ -87,6 +90,12 @@
(`#750 <https://github.com/matrix-org/matrix-doc/pull/750>`_).
- ``GET /user/{userId}/filter/{filterId}`` requires authentication
(`#1003 <https://github.com/matrix-org/matrix-doc/pull/1003>`_).
- Add some clarifying notes on the behaviour of rooms with no
``m.room.power_levels`` event
(`#1026 <https://github.com/matrix-org/matrix-doc/pull/1026>`_).
- Clarify the relationship between ``username`` and ``user_id`` in the
``/register`` API
(`#1032 <https://github.com/matrix-org/matrix-doc/pull/1032>`_).
r0.2.0
======

@ -3,96 +3,6 @@ Federation
.. sectnum::
.. contents:: Table of Contents
Authorization
-------------
When receiving new events from remote servers, or creating new events, a server
must know whether that event is allowed by the authorization rules. These rules
depend solely on the state at that event. The types of state events that affect
authorization are:
- ``m.room.create``
- ``m.room.member``
- ``m.room.join_rules``
- ``m.room.power_levels``
Servers should not create new events that reference unauthorized events.
However, any event that does reference an unauthorized event is not itself
automatically considered unauthorized.
Unauthorized events that appear in the event graph do *not* have any effect on
the state of the graph.
.. Note:: This is in contrast to redacted events which can still affect the
state of the graph. For example, a redacted *"join"* event will still
result in the user being considered joined.
Rules
~~~~~
The following are the rules to determine if an event is authorized (this does
include validation).
**TODO**: What signatures do we expect?
1. If type is ``m.room.create`` allow if and only if it has no prev events.
#. If type is ``m.room.member``:
a. If ``membership`` is ``join``:
i. If the previous event is an ``m.room.create``, the depth is 1 and
the ``state_key`` is the creator, then allow.
#. If the ``state_key`` does not match ``sender`` key, reject.
#. If the current state has ``membership`` set to ``join``.
#. If the ``sender`` is in the ``m.room.may_join`` list. [Not currently
implemented]
#. If the ``join_rules`` is:
- ``public``: allow.
- ``invite``: allow if the current state has ``membership`` set to
``invite``
- ``knock``: **TODO**.
- ``private``: Reject.
#. Reject
#. If ``membership`` is ``invite`` then allow if ``sender`` is in room,
otherwise reject.
#. If ``membership`` is ``leave``:
i. If ``sender`` matches ``state_key`` allow.
#. If ``sender``'s power level is greater than the the ``kick_level``
given in the current ``m.room.power_levels`` state (defaults to 50),
and the ``state_key``'s power level is less than or equal to the
``sender``'s power level, then allow.
#. Reject.
#. If ``membership`` is ``ban``:
i. **TODO**.
#. Reject.
#. Reject the event if the event type's required power level is less that the
``sender``'s power level.
#. If the ``sender`` is not in the room, reject.
#. If the type is ``m.room.power_levels``:
a. **TODO**.
#. Allow.
Definitions
~~~~~~~~~~~
Required Power Level
A given event type has an associated *required power level*. This is given
by the current ``m.room.power_levels`` event, it is either listed explicitly
in the ``events`` section or given by either ``state_default`` or
``events_default`` depending on if the event type is a state event or not.
Auth events
~~~~~~~~~~~

@ -1,52 +0,0 @@
State Resolution
================
This section describes why we need state resolution and how it works.
Motivation
-----------
We want to be able to associate some shared state with rooms, e.g. a room name
or members list. This is done by having a current state dictionary that maps
from the pair event type and state key to an event.
However, since the servers involved in the room are distributed we need to be
able to handle the case when two (or more) servers try and update the state at
the same time. This is done via the state resolution algorithm.
State Tree
------------
State events contain a reference to the state it is trying to replace. These
relations form a tree where the current state is one of the leaf nodes.
Note that state events are events, and so are part of the PDU graph. Thus we
can be sure that (modulo the internet being particularly broken) we will see
all state events eventually.
Algorithm requirements
----------------------
We want the algorithm to have the following properties:
- Since we aren't guaranteed what order we receive state events in, except that
we see parents before children, the state resolution algorithm must not depend
on the order and must always come to the same result.
- If we receive a state event whose parent is the current state, then the
algorithm will select it.
- The algorithm does not depend on internal state, ensuring all servers should
come to the same decision.
These three properties mean it is enough to keep track of the current state and
compare it with any new proposed state, rather than having to keep track of all
the leafs of the tree and recomputing across the entire state tree.
Current Implementation
----------------------
The current implementation works as follows: Upon receipt of a newly proposed
state change we first find the common ancestor. Then we take the maximum
across each branch of the users' power levels, if one is higher then it is
selected as the current state. Otherwise, we check if one chain is longer than
the other, if so we choose that one. If that also fails, then we concatenate
all the pdu ids and take a SHA1 hash and compare them to select a common
ancestor.

@ -0,0 +1,12 @@
{
"age": 242352,
"content": {
"pinned": ["$someevent:localhost"]
},
"state_key": "",
"origin_server_ts": 1431961217939,
"event_id": "$WLGTSEFSEF:localhost",
"type": "m.room.pinned_events",
"room_id": "!Cuyf34gef24t:localhost",
"sender": "@example:localhost"
}

@ -3,11 +3,12 @@ properties:
content:
description: The fields in this object will vary depending on the type of event.
When interacting with the REST API, this is the HTTP body.
title: EventContent
type: object
type:
description: The type of event. This SHOULD be namespaced similar to Java package
naming conventions e.g. 'com.example.subdomain.event.type'
type: string
required:
- type
title: Event
type: object

@ -4,17 +4,17 @@ description: In addition to the Event fields, Room Events have the following add
fields.
properties:
event_id:
description: Required. The globally unique event identifier.
description: The globally unique event identifier.
type: string
room_id:
description: Required. The ID of the room associated with this event.
description: The ID of the room associated with this event.
type: string
sender:
description: Required. Contains the fully-qualified ID of the user who *sent*
description: Contains the fully-qualified ID of the user who *sent*
this event.
type: string
origin_server_ts:
description: Required. Timestamp in milliseconds on originating homeserver
description: Timestamp in milliseconds on originating homeserver
when this event was sent.
type: number
unsigned:

@ -0,0 +1,25 @@
---
allOf:
- $ref: core-event-schema/state_event.yaml
description: This event is used to "pin" particular events in a room for other participants to review later. The order of the pinned events is guaranteed and based upon the order supplied in the event. Clients should be aware that the current user may not be able to see some of the events pinned due to visibility settings in the room. Clients are responsible for determining if a particular event in the pinned list is displayable, and have the option to not display it if it cannot be pinned in the client.
properties:
content:
properties:
pinned:
description: An ordered list of event IDs to pin.
items:
type: string
type: array
required:
- pinned
type: object
state_key:
description: A zero-length string.
pattern: '^$'
type: string
type:
enum:
- m.room.pinned_events
type: string
title: Pinned events in a room
type: object

@ -29,6 +29,17 @@ description: |-
to 50 if they are not specified in the ``m.room.power_levels`` event, or if
the room contains no ``m.room.power_levels`` event.
.. NOTE::
As noted above, in the absence of an ``m.room.power_levels`` event, the
``state_default`` is 0, and all users are considered to have power level 0.
That means that **any** member of the room can send an
``m.room.power_levels`` event, changing the permissions in the room.
Server implementations should therefore ensure that each room has an
``m.room.power_levels`` event as soon as it is created. See also the
documentation of the ``/createRoom`` API.
properties:
content:
properties:

@ -4,12 +4,7 @@ set -ex
virtualenv env
. env/bin/activate
pip install \
docutils \
pygments \
Jinja2 \
jsonschema \
PyYAML
pip install -r scripts/requirements.txt
# do sanity checks on the examples and swagger
(cd event-schemas/ && ./check_examples.py)

@ -1,45 +0,0 @@
Generating the HTML for the specification
=========================================
Requirements:
- docutils (for converting RST to HTML)
- Jinja2 (for templating)
- PyYAML (for reading YAML files)
Nix[2] users can enter an environment with the appropriate tools and
dependencies available by invoking `nix-shell contrib/shell.nix` in this
directory.
To generate the complete specification along with supporting documentation, run:
python gendoc.py
The output of this will be inside the "scripts/gen" folder.
Matrix.org only ("gen" folder has matrix.org tweaked pages):
./matrix-org-gendoc.sh /path/to/matrix.org/includes/nav.html
Generating the Swagger documentation
====================================
Swagger[1] is a framework for representing RESTful APIs. We use it to generate
interactive documentation for our APIs.
Swagger UI reads a JSON description of the API. To generate this file from the
YAML files in the `api` folder, run:
./dump-swagger.py
By default, `dump-swagger` will write to `scripts/swagger/api-docs.json`.
To make use of the generated file, there are a number of options:
* It can be uploaded from your filesystem to an online editor/viewer such as
http://editor.swagger.io/
* You can run a local HTTP server by running `./swagger-http-server.py`, and
then view the documentation via an online viewer; for example, at
http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json
* You can host the swagger UI yourself:
* download the latest release from https://github.com/swagger-api/swagger-ui
* copy the contents of the 'dist' directory alongside `api-docs.json`
* View the UI via your browser at http://\<hostname>?url=api-docs.json
[1] http://swagger.io/
[2] https://nixos.org/nix/

@ -52,7 +52,7 @@ func main() {
walker := makeWalker(dir, w)
paths := []string{"api", "changelogs", "event-schemas", "scripts",
"specification", "templating"}
"specification"}
for _, p := range paths {
filepath.Walk(path.Join(dir, p), walker)
@ -175,7 +175,7 @@ func generate(dir string) (map[string][]byte, error) {
// cheekily dump the swagger docs into the gen directory so that it is
// easy to serve
cmd = exec.Command("python", "dump-swagger.py", "gen/api-docs.json")
cmd = exec.Command("python", "dump-swagger.py", "-o", "gen/api-docs.json")
cmd.Dir = path.Join(dir, "scripts")
cmd.Stderr = &b
if err := cmd.Run(); err != nil {

@ -19,7 +19,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import errno
import json
import logging
@ -30,23 +30,31 @@ import yaml
scripts_dir = os.path.dirname(os.path.abspath(__file__))
templating_dir = os.path.join(os.path.dirname(scripts_dir), "templating")
templating_dir = os.path.join(scripts_dir, "templating")
api_dir = os.path.join(os.path.dirname(scripts_dir), "api")
sys.path.insert(0, templating_dir)
from matrix_templates import units
if len(sys.argv) > 3:
sys.stderr.write("usage: %s [output_file] [client_release_label]\n" % (sys.argv[0],))
sys.exit(1)
if len(sys.argv) > 1:
output_file = os.path.abspath(sys.argv[1])
else:
output_file = os.path.join(scripts_dir, "swagger", "api-docs.json")
release_label = sys.argv[2] if len(sys.argv) > 2 else "unstable"
parser = argparse.ArgumentParser(
"dump-swagger.py - assemble the Swagger specs into a single JSON file"
)
parser.add_argument(
"--client_release", "-c", metavar="LABEL",
default="unstable",
help="""The client-server release version to gneerate for. Default:
%(default)s""",
)
parser.add_argument(
"-o", "--output",
default=os.path.join(scripts_dir, "swagger", "api-docs.json"),
help="File to write the output to. Default: %(default)s"
)
args = parser.parse_args()
output_file = os.path.abspath(args.output)
release_label = args.client_release
major_version = release_label
match = re.match("^(r\d+)(\.\d+)*$", major_version)

@ -273,7 +273,7 @@ def addAnchors(path):
def run_through_template(input_files, set_verbose, substitutions):
args = [
'python', 'build.py',
'python', script_dir+'/templating/build.py',
"-o", tmp_dir,
"-i", "matrix_templates",
]
@ -288,10 +288,7 @@ def run_through_template(input_files, set_verbose, substitutions):
log("EXEC: %s" % " ".join(args))
log(" ==== build.py output ==== ")
subprocess.check_call(
args,
cwd=os.path.join(docs_dir, "templating"),
)
subprocess.check_call(args)
"""
Extract and resolve groups for the given target in the given targets listing.

@ -9,8 +9,8 @@ mkdir -p assets
# generate the spec docs
./scripts/gendoc.py -d assets/spec
# also need the supporting-docs, which become the jekyll posts.
cp -rT supporting-docs assets/jekyll-posts
# and the swagger
./scripts/dump-swagger.py -o assets/spec/client_server/unstable.json
# create a tarball of the assets. Exclude the spec index for now, as
# we want to leave it pointing at the release versions of the specs.

@ -0,0 +1,8 @@
# no doubt older versions would be fine for many of these but these were
# current at the time of writing
docutils >= 0.14
pygments >= 2.2.0
Jinja2 >= 2.9.6
jsonschema >= 2.6.0
PyYAML >= 3.12

@ -159,7 +159,7 @@ func generate(dir string) error {
// cheekily dump the swagger docs into the gen directory so they can be
// served by serveSpec
cmd = exec.Command("python", "dump-swagger.py", "gen/api-docs.json")
cmd = exec.Command("python", "dump-swagger.py", "-o", "gen/api-docs.json")
cmd.Dir = path.Join(dir, "scripts")
cmd.Stderr = &b
if err := cmd.Run(); err != nil {

@ -12,11 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""Parent class for writing units."""
from . import AccessKeyStore
import inspect
import json
import os
import subprocess
class Units(object):
@ -57,9 +54,6 @@ class Units(object):
unit_dict[unit_key] = func(self.substitutions)
else:
unit_dict[unit_key] = func()
self.log("Generated unit '%s' : %s" % (
unit_key, json.dumps(unit_dict[unit_key])[:50].replace(
"\n",""
)
))
self.log("Generated unit '%s'" % unit_key)
return unit_dict

@ -59,10 +59,12 @@ import importlib
import json
import logging
import os
import re
import sys
from textwrap import TextWrapper
from matrix_templates.units import TypeTableRow
def create_from_template(template, sections):
return template.render(sections)
@ -117,15 +119,23 @@ def main(input_module, files=None, out_dir=None, verbose=False, substitutions={}
Given a list of rows, returns a list giving the maximum length of the
values in each column.
:param list[dict[str, str]] input: a list of rows. Each row should be a
dict with the keys given in ``keys``.
:param list[TypeTableRow|dict[str,str]] input:
a list of rows
:param list[str] keys: the keys corresponding to the table columns
:param list[int] defaults: for each column, the default column width.
:param int default_width: if ``defaults`` is shorter than ``keys``, this
will be used as a fallback
"""
def getrowattribute(row, k):
# the row may be a dict (particularly the title row, which is
# generated by the template
if not isinstance(row, TypeTableRow):
return row[k]
return getattr(row, k)
def colwidth(key, default):
return reduce(max, (len(row[key]) for row in input),
rowwidths = (len(getrowattribute(row, key)) for row in input)
return reduce(max, rowwidths,
default if default is not None else default_width)
results = map(colwidth, keys, defaults)

@ -0,0 +1,13 @@
{% import 'tables.tmpl' as tables -%}
{{common_event.title}} Fields
{{(7 + common_event.title | length) * title_kind}}
{{common_event.desc}}
{% for table in common_event.tables %}
{{"``"+table.title+"``" if table.title else "" }}
{{ tables.paramtable(table.rows, ["Key", "Type", "Description"]) }}
{% endfor %}

@ -9,7 +9,8 @@
{% endif -%}
{{event.desc | wrap(80)}}
{{event.desc}}
{% for table in event.content_fields %}
{{"``"+table.title+"``" if table.title else "" }}

@ -27,10 +27,10 @@ Request format:
`No parameters`
{% endif %}
{% if endpoint.res_headers|length > 0 -%}
{% if endpoint.res_headers is not none -%}
Response headers:
{{ tables.paramtable(endpoint.res_headers) }}
{{ tables.paramtable(endpoint.res_headers.rows) }}
{% endif -%}
{% if endpoint.res_tables|length > 0 -%}

@ -6,8 +6,7 @@
{#
# write a table for a list of parameters.
#
# 'rows' is the list of parameters. Each row should have the keys
# 'key', 'type', and 'desc'.
# 'rows' is the list of parameters. Each row should be a TypeTableRow.
#}
{% macro paramtable(rows, titles=["Parameter", "Type", "Description"]) -%}
{{ split_paramtable({None: rows}, titles) }}
@ -21,11 +20,11 @@
# As a special case, if a key of 'rows_by_loc' is 'None', no title row is
# written for that location. This is used by the standard 'paramtable' macro.
#}
{% macro split_paramtable(rows_by_loc,
{% macro split_paramtable(rows_by_loc,
titles=["Parameter", "Type", "Description"]) -%}
{% set rowkeys = ['key', 'type', 'desc'] %}
{% set titlerow = {'key': titles[0], 'type': titles[1], 'desc': titles[2]} %}
{% set rowkeys = ['key', 'title', 'desc'] %}
{% set titlerow = {'key': titles[0], 'title': titles[1], 'desc': titles[2]} %}
{# We need the rows flattened into a single list. Abuse the 'sum' filter to
# join arrays instead of add numbers. -#}
@ -34,7 +33,7 @@
{# Figure out the widths of the columns. The last column is always 50 characters
# wide; the others default to 10, but stretch if there is wider text in the
# column. -#}
{% set fieldwidths = (([titlerow] + flatrows) |
{% set fieldwidths = (([titlerow] + flatrows) |
fieldwidths(rowkeys[0:-1], [10, 10])) + [50] -%}
{{ tableheader(fieldwidths) }}
@ -57,7 +56,7 @@
{#
# Write a table header row, for the given column widths
# Write a table header row, for the given column widths
#}
{% macro tableheader(widths) -%}
{% for arg in widths -%}
@ -67,7 +66,7 @@
{#
# Write a normal table row. Each of 'widths' and 'keys' should be sequences
# Write a normal table row. Each of 'widths' and 'keys' should be sequences
# of the same length; 'widths' defines the column widths, and 'keys' the
# attributes of 'row' to look up for values to put in the columns.
#}
@ -81,7 +80,7 @@
{# the last column needs wrapping and indenting (by the sum of the widths of
the preceding columns, plus the number of preceding columns (for the
separators)) -#}
{{ value | wrap(widths[loop.index0]) |
{{ value | wrap(widths[loop.index0]) |
indent_block(widths[0:-1]|sum + loop.index0) -}}
{% endif -%}
{% endfor -%}
@ -90,7 +89,7 @@
{#
{#
# write a tablespan row. This is a single value which spans the entire table.
#}
{% macro tablespan(widths, value) -%}

@ -25,23 +25,27 @@ from collections import OrderedDict
import logging
import json
import os
import os.path
import re
import subprocess
import sys
import urllib
import yaml
matrix_doc_dir=reduce(lambda acc,_: os.path.dirname(acc),
range(1, 5), os.path.abspath(__file__))
HTTP_APIS = {
"../api/application-service": "as",
"../api/client-server": "cs",
"../api/identity": "is",
"../api/push-gateway": "push",
os.path.join(matrix_doc_dir, "api/application-service"): "as",
os.path.join(matrix_doc_dir, "api/client-server"): "cs",
os.path.join(matrix_doc_dir, "api/identity"): "is",
os.path.join(matrix_doc_dir, "api/push-gateway"): "push",
}
EVENT_EXAMPLES = "../event-schemas/examples"
EVENT_SCHEMA = "../event-schemas/schema"
CORE_EVENT_SCHEMA = "../event-schemas/schema/core-event-schema"
CHANGELOG_DIR = "../changelogs"
TARGETS = "../specification/targets.yaml"
EVENT_EXAMPLES = os.path.join(matrix_doc_dir, "event-schemas/examples")
EVENT_SCHEMA = os.path.join(matrix_doc_dir, "event-schemas/schema")
CORE_EVENT_SCHEMA = os.path.join(matrix_doc_dir, "event-schemas/schema/core-event-schema")
CHANGELOG_DIR = os.path.join(matrix_doc_dir, "changelogs")
TARGETS = os.path.join(matrix_doc_dir, "specification/targets.yaml")
ROOM_EVENT = "core-event-schema/room_event.yaml"
STATE_EVENT = "core-event-schema/state_event.yaml"
@ -62,6 +66,50 @@ OrderedLoader.add_constructor(
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
construct_mapping)
class TypeTable(object):
"""Describes a table documenting an object type
Attributes:
title(str|None): Title of the table - normally the object type
desc(str|None): description of the object
rows(list[TypeTableRow]): the rows in the table
"""
def __init__(self, title=None, desc=None, rows=[]):
self.title=title
self.desc=desc
self._rows = []
for row in rows:
self.add_row(row)
def add_row(self, row):
if not isinstance(row, TypeTableRow):
raise ValueError("Can only add TypeTableRows to TypeTable")
self._rows.append(row)
def __getattr__(self, item):
if item == 'rows':
return list(self._rows)
return super(TypeTable, self).__getattr__(item)
def __repr__(self):
return "TypeTable[%s, rows=%s]" % (self.title, self._rows)
class TypeTableRow(object):
"""Describes an object field defined in the json schema
"""
def __init__(self, key, title, desc, required=False):
self.key = key
self.title = title
self.desc = desc
self.required = required
def __repr__(self):
return "TypeTableRow[%s: %s]" % (self.key, self.desc)
def resolve_references(path, schema):
if isinstance(schema, dict):
# do $ref first
@ -99,10 +147,16 @@ def inherit_parents(obj):
# iterate through the parents first, and then overwrite with the settings
# from the child.
for p in map(inherit_parents, parents) + [obj]:
for key in ('title', 'type', 'required', 'description'):
# child blats out type, title and description
for key in ('type', 'title', 'description'):
if p.get(key):
result[key] = p[key]
# other fields get merged
for key in ('required', ):
if p.get(key):
result.setdefault(key, []).extend(p[key])
for key in ('properties', 'additionalProperties', 'patternProperties'):
if p.get(key):
result.setdefault(key, OrderedDict()).update(p[key])
@ -111,6 +165,21 @@ def inherit_parents(obj):
def get_json_schema_object_fields(obj, enforce_title=False):
"""Parse a JSON schema object definition
Args:
obj(dict): definition from the JSON schema file. $refs should already
have been resolved.
enforce_title (bool): if True, and the definition has no "title",
the 'title' result will be set to 'NO_TITLE' (otherwise it will be
set to None)
Returns:
dict: with the following fields:
- title (str): title (normally the type name) for the object
- tables (list[TypeTable]): list of the tables for the type
definition
"""
# Algorithm:
# f.e. property => add field info (if field is object then recurse)
if obj.get("type") != "object":
@ -131,7 +200,7 @@ def get_json_schema_object_fields(obj, enforce_title=False):
key_type = additionalProps.get("x-pattern", "string")
res = process_data_type(additionalProps)
return {
"type": "{%s: %s}" % (key_type, res["type"]),
"title": "{%s: %s}" % (key_type, res["title"]),
"tables": res["tables"],
}
@ -151,7 +220,7 @@ def get_json_schema_object_fields(obj, enforce_title=False):
# doing all the keys.
if not props:
return {
"type": obj_title if obj_title else 'object',
"title": obj_title if obj_title else 'object',
"tables": [],
}
@ -171,12 +240,12 @@ def get_json_schema_object_fields(obj, enforce_title=False):
required = key_name in required_keys
res = process_data_type(props[key_name], required)
first_table_rows.append({
"key": key_name,
"type": res["type"],
"required": required,
"desc": res["desc"],
})
first_table_rows.append(TypeTableRow(
key=key_name,
title=res["title"],
required=required,
desc=res["desc"],
))
tables.extend(res["tables"])
logger.debug("Done property %s" % key_name)
@ -187,19 +256,19 @@ def get_json_schema_object_fields(obj, enforce_title=False):
# we don't lose information about where the error occurred.
raise e2, None, sys.exc_info()[2]
tables.insert(0, {
"title": obj_title,
"rows": first_table_rows,
})
tables.insert(0, TypeTable(title=obj_title, rows=first_table_rows))
for table in tables:
assert isinstance(table, TypeTable)
return {
"type": obj_title,
"title": obj_title,
"tables": tables,
}
# process a data type definition. returns a dictionary with the keys:
# type: stringified type name
# title: stringified type name
# desc: description
# enum_desc: description of permissible enum fields
# is_object: true if the data type is an object
@ -217,19 +286,22 @@ def process_data_type(prop, required=False, enforce_title=True):
prop,
enforce_title=enforce_title,
)
prop_type = res["type"]
prop_title = res["title"]
tables = res["tables"]
is_object = True
elif prop_type == "array":
nested = process_data_type(prop["items"])
prop_type = "[%s]" % nested["type"]
prop_title = "[%s]" % nested["title"]
tables = nested["tables"]
enum_desc = nested["enum_desc"]
else:
prop_title = prop_type
if prop.get("enum"):
if len(prop["enum"]) > 1:
prop_type = "enum"
prop_title = "enum"
enum_desc = (
"One of: %s" % json.dumps(prop["enum"])
)
@ -238,15 +310,17 @@ def process_data_type(prop, required=False, enforce_title=True):
"Must be '%s'." % prop["enum"][0]
)
if isinstance(prop_type, list):
prop_type = " or ".join(prop_type)
if isinstance(prop_title, list):
prop_title = " or ".join(prop_title)
rq = "**Required.**" if required else None
desc = " ".join(x for x in [rq, prop.get("description"), enum_desc] if x)
for table in tables:
assert isinstance(table, TypeTable)
return {
"type": prop_type,
"title": prop_title,
"desc": desc,
"enum_desc": enum_desc,
"is_object": is_object,
@ -263,13 +337,10 @@ def deduplicate_tables(tables):
titles = set()
filtered = []
for table in reversed(tables):
if table.get("no-table"):
continue
if table.get("title") in titles:
if table.title in titles:
continue
titles.add(table.get("title"))
titles.add(table.title)
filtered.append(table)
filtered.reverse()
@ -286,14 +357,10 @@ def get_tables_for_response(schema):
# make up the first table, with just the 'body' row in, unless the response
# is an object, in which case there's little point in having one.
if not pv["is_object"]:
tables = [{
"title": None,
"rows": [{
"key": "<body>",
"type": pv["type"],
"desc": pv["desc"],
}]
}] + tables
first_table_row = TypeTableRow(
key="<body>", title=pv["title"], desc=pv["desc"],
)
tables.insert(0, TypeTable(None, rows=[first_table_row]))
logger.debug("response: %r" % tables)
@ -383,9 +450,9 @@ class MatrixUnits(Units):
endpoints.append(endpoint)
except Exception as e:
raise Exception(
"Error handling endpoint %s %s: %s" % (method, path, e),
)
logger.error("Error handling endpoint %s %s: %s",
method, path, e)
raise
return {
"base": api.get("basePath").rstrip("/"),
"group": group_name,
@ -404,7 +471,7 @@ class MatrixUnits(Units):
"rate_limited": 429 in endpoint_swagger.get("responses", {}),
"req_param_by_loc": {},
"req_body_tables": [],
"res_headers": [],
"res_headers": None,
"res_tables": [],
"responses": [],
"example": {
@ -440,11 +507,9 @@ class MatrixUnits(Units):
" One of: %s" % json.dumps(param.get("enum"))
)
endpoint["req_param_by_loc"].setdefault(param_loc, []).append({
"key": param_name,
"type": val_type,
"desc": desc
})
endpoint["req_param_by_loc"].setdefault(param_loc, []).append(
TypeTableRow(key=param_name, title=val_type, desc=desc),
)
example = get_example_for_param(param)
if example is None:
@ -484,14 +549,12 @@ class MatrixUnits(Units):
good_response["schema"]
)
if "headers" in good_response:
headers = []
for (header_name, header) in good_response[
"headers"].iteritems():
headers.append({
"key": header_name,
"type": header["type"],
"desc": header["description"],
})
headers = TypeTable()
for (header_name, header) in good_response["headers"].iteritems():
headers.add_row(
TypeTableRow(key=header_name, title=header["type"],
desc=header["description"]),
)
endpoint["res_headers"] = headers
query_string = "" if len(
example_query_params) == 0 else "?" + urllib.urlencode(
@ -531,7 +594,7 @@ class MatrixUnits(Units):
# put the top-level parameters into 'req_param_by_loc', and the others
# into 'req_body_tables'
body_params = endpoint_data['req_param_by_loc'].setdefault("JSON body",[])
body_params.extend(req_body_tables[0]["rows"])
body_params.extend(req_body_tables[0].rows)
body_tables = req_body_tables[1:]
endpoint_data['req_body_tables'].extend(body_tables)
@ -565,70 +628,64 @@ class MatrixUnits(Units):
return apis
def load_common_event_fields(self):
"""Parse the core event schema files
Returns:
dict: with the following properties:
"title": Event title (from the 'title' field of the schema)
"desc": desc
"tables": list[TypeTable]
"""
path = CORE_EVENT_SCHEMA
event_types = {}
for (root, dirs, files) in os.walk(path):
for filename in files:
if not filename.endswith(".yaml"):
continue
for filename in os.listdir(path):
if not filename.endswith(".yaml"):
continue
filepath = os.path.join(path, filename)
event_type = filename[:-5] # strip the ".yaml"
logger.info("Reading event schema: %s" % filepath)
with open(filepath) as f:
event_schema = yaml.load(f, OrderedLoader)
event_type = filename[:-5] # strip the ".yaml"
filepath = os.path.join(root, filename)
with open(filepath) as f:
try:
event_info = yaml.load(f, OrderedLoader)
except Exception as e:
raise ValueError(
"Error reading file %r" % (filepath,), e
)
if "event" not in event_type:
continue # filter ImageInfo and co
table = {
"title": event_info["title"],
"desc": event_info["description"],
"rows": []
}
for prop in sorted(event_info["properties"]):
row = {
"key": prop,
"type": event_info["properties"][prop]["type"],
"desc": event_info["properties"][prop].get("description","")
}
table["rows"].append(row)
event_types[event_type] = table
schema_info = process_data_type(
event_schema,
enforce_title=True,
)
event_types[event_type] = schema_info
return event_types
def load_apis(self, substitutions):
cs_ver = substitutions.get("%CLIENT_RELEASE_LABEL%", "unstable")
fed_ver = substitutions.get("%SERVER_RELEASE_LABEL%", "unstable")
return {
"rows": [{
"key": "`Client-Server API <client_server/"+cs_ver+".html>`_",
"type": cs_ver,
"desc": "Interaction between clients and servers",
}, {
"key": "`Server-Server API <server_server/"+fed_ver+".html>`_",
"type": fed_ver,
"desc": "Federation between servers",
}, {
"key": "`Application Service API <application_service/unstable.html>`_",
"type": "unstable",
"desc": "Privileged server plugins",
}, {
"key": "`Identity Service API <identity_service/unstable.html>`_",
"type": "unstable",
"desc": "Mapping of third party IDs to Matrix IDs",
}, {
"key": "`Push Gateway API <push_gateway/unstable.html>`_",
"type": "unstable",
"desc": "Push notifications for Matrix events",
}]
}
# we abuse the typetable to return this info to the templates
return TypeTable(rows=[
TypeTableRow(
"`Client-Server API <client_server/"+cs_ver+".html>`_",
cs_ver,
"Interaction between clients and servers",
), TypeTableRow(
"`Server-Server API <server_server/"+fed_ver+".html>`_",
fed_ver,
"Federation between servers",
), TypeTableRow(
"`Application Service API <application_service/unstable.html>`_",
"unstable",
"Privileged server plugins",
), TypeTableRow(
"`Identity Service API <identity_service/unstable.html>`_",
"unstable",
"Mapping of third party IDs to Matrix IDs",
), TypeTableRow(
"`Push Gateway API <push_gateway/unstable.html>`_",
"unstable",
"Push notifications for Matrix events",
),
])
def load_event_examples(self):
path = EVENT_EXAMPLES
@ -673,24 +730,23 @@ class MatrixUnits(Units):
json_schema = yaml.load(f, OrderedLoader)
schema = {
# one of "Message Event" or "State Event"
"typeof": "",
"typeof_info": "",
# event type, eg "m.room.member". Note *not* the type of the
# event object (which should always be 'object').
"type": None,
"title": None,
"desc": None,
"msgtype": None,
"content_fields": [
# {
# title: "<title> key"
# rows: [
# { key: <key_name>, type: <string>,
# desc: <desc>, required: <bool> }
# ]
# }
# <TypeTable>
]
}
# add typeof
# before we resolve the references, see if the first reference is to
# the message event or state event schemas, and add typeof info if so.
base_defs = {
ROOM_EVENT: "Message Event",
STATE_EVENT: "State Event"

@ -0,0 +1,225 @@
.. Copyright 2016 Openmarket Ltd.
.. Copyright 2017 New Vector Ltd.
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
Identifier Grammar
------------------
Server Name
~~~~~~~~~~~
A homeserver is uniquely identified by its server name. This value is used in a
number of identifiers, as described below.
The server name represents the address at which the homeserver in question can
be reached by other homeservers. The complete grammar is::
server_name = dns_name [ ":" port]
dns_name = host
port = *DIGIT
where ``host`` is as defined by `RFC3986, section 3.2.2
<https://tools.ietf.org/html/rfc3986#section-3.2.2>`_.
Examples of valid server names are:
* ``matrix.org``
* ``matrix.org:8888``
* ``1.2.3.4`` (IPv4 literal)
* ``1.2.3.4:1234`` (IPv4 literal with explicit port)
* ``[1234:5678::abcd]`` (IPv6 literal)
* ``[1234:5678::abcd]:5678`` (IPv6 literal with explicit port)
Common Identifier Format
~~~~~~~~~~~~~~~~~~~~~~~~
The Matrix protocol uses a common format to assign unique identifiers to a
number of entities, including users, events and rooms. Each identifier takes
the form::
&localpart:domain
where ``&`` represents a 'sigil' character; ``domain`` is the `server name`_ of
the homeserver which allocated the identifier, and ``localpart`` is an
identifier allocated by that homeserver.
The sigil characters are as follows:
* ``@``: User ID
* ``!``: Room ID
* ``$``: Event ID
* ``#``: Room alias
The precise grammar defining the allowable format of an identifier depends on
the type of identifier.
User Identifiers
++++++++++++++++
Users within Matrix are uniquely identified by their Matrix user ID. The user
ID is namespaced to the homeserver which allocated the account and has the
form::
@localpart:domain
The ``localpart`` of a user ID is an opaque identifier for that user. It MUST
NOT be empty, and MUST contain only the characters ``a-z``, ``0-9``, ``.``,
``_``, ``=``, ``-``, and ``/``.
The ``domain`` of a user ID is the `server name`_ of the homeserver which
allocated the account.
The length of a user ID, including the ``@`` sigil and the domain, MUST NOT
exceed 255 characters.
The complete grammar for a legal user ID is::
user_id = "@" user_id_localpart ":" server_name
user_id_localpart = 1*user_id_char
user_id_char = DIGIT
/ %x61-7A ; a-z
/ "-" / "." / "=" / "_" / "/"
.. admonition:: Rationale
A number of factors were considered when defining the allowable characters
for a user ID.
Firstly, we chose to exclude characters outside the basic US-ASCII character
set. User IDs are primarily intended for use as an identifier at the protocol
level, and their use as a human-readable handle is of secondary
benefit. Furthermore, they are useful as a last-resort differentiator between
users with similar display names. Allowing the full unicode character set
would make very difficult for a human to distinguish two similar user IDs. The
limited character set used has the advantage that even a user unfamiliar with
the Latin alphabet should be able to distinguish similar user IDs manually, if
somewhat laboriously.
We chose to disallow upper-case characters because we do not consider it
valid to have two user IDs which differ only in case: indeed it should be
possible to reach ``@user:matrix.org`` as ``@USER:matrix.org``. However,
user IDs are necessarily used in a number of situations which are inherently
case-sensitive (notably in the ``state_key`` of ``m.room.member``
events). Forbidding upper-case characters (and requiring homeservers to
downcase usernames when creating user IDs for new users) is a relatively simple
way to ensure that ``@USER:matrix.org`` cannot refer to a different user to
``@user:matrix.org``.
Finally, we decided to restrict the allowable punctuation to a very basic set
to reduce the possibility of conflicts with special characters in various
situations. For example, "*" is used as a wildcard in some APIs (notably the
filter API), so it cannot be a legal user ID character.
The length restriction is derived from the limit on the length of the
``sender`` key on events; since the user ID appears in every event sent by the
user, it is limited to ensure that the user ID does not dominate over the actual
content of the events.
Matrix user IDs are sometimes informally referred to as MXIDs.
Historical User IDs
<<<<<<<<<<<<<<<<<<<
Older versions of this specification were more tolerant of the characters
permitted in user ID localparts. There are currently active users whose user
IDs do not conform to the permitted character set, and a number of rooms whose
history includes events with a ``sender`` which does not conform. In order to
handle these rooms successfully, clients and servers MUST accept user IDs with
localparts from the expanded character set::
extended_user_id_char = %x21-7E
Mapping from other character sets
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
In certain circumstances it will be desirable to map from a wider character set
onto the limited character set allowed in a user ID localpart. Examples include
a homeserver creating a user ID for a new user based on the username passed to
``/register``, or a bridge mapping user ids from another protocol.
.. TODO-spec
We need to better define the mechanism by which homeservers can allow users
to have non-Latin login credentials. The general idea is for clients to pass
the non-Latin in the ``username`` field to ``/register`` and ``/login``, and
the HS then maps it onto the MXID space when turning it into the
fully-qualified ``user_id`` which is returned to the client and used in
events.
Implementations are free to do this mapping however they choose. Since the user
ID is opaque except to the implementation which created it, the only
requirement is that the implemention can perform the mapping
consistently. However, we suggest the following algorithm:
1. Encode character strings as UTF-8.
2. Convert the bytes ``A-Z`` to lower-case.
* In the case where a bridge must be able to distinguish two different users
with ids which differ only by case, escape upper-case characters by
prefixing with ``_`` before downcasing. For example, ``A`` becomes
``_a``. Escape a real ``_`` with a second ``_``.
3. Encode any remaining bytes outside the allowed character set, as well as
``=``, as their hexadecimal value, prefixed with ``=``. For example, ``#``
becomes ``=23``; ``á`` becomes ``=c3=a1``.
.. admonition:: Rationale
The suggested mapping is an attempt to preserve human-readability of simple
ASCII identifiers (unlike, for example, base-32), whilst still allowing
representation of *any* character (unlike punycode, which provides no way to
encode ASCII punctuation).
Room IDs and Event IDs
++++++++++++++++++++++
A room has exactly one room ID. A room ID has the format::
!opaque_id:domain
An event has exactly one event ID. An event ID has the format::
$opaque_id:domain
The ``domain`` of a room/event ID is the `server name`_ of the homeserver which
created the room/event. The domain is used only for namespacing to avoid the
risk of clashes of identifiers between different homeservers. There is no
implication that the room or event in question is still available at the
corresponding homeserver.
Event IDs and Room IDs are case-sensitive. They are not meant to be human
readable.
.. TODO-spec
What is the grammar for the opaque part? https://matrix.org/jira/browse/SPEC-389
Room Aliases
++++++++++++
A room may have zero or more aliases. A room alias has the format::
#room_alias:domain
The ``domain`` of a room alias is the `server name`_ of the homeserver which
created the alias. Other servers may contact this homeserver to look up the
alias.
Room aliases MUST NOT exceed 255 bytes (including the ``#`` sigil and the
domain).
.. TODO-spec
- Need to specify precise grammar for Room Aliases. https://matrix.org/jira/browse/SPEC-391

@ -86,6 +86,155 @@ insignificant whitespace, fractions, exponents and redundant character escapes
/ %x75.30.30.30 (%x30-37 / %x62 / %x65-66) ; u000X
/ %x75.30.30.31 (%x30-39 / %x61-66) ; u001X
Examples
++++++++
To assist in the development of compatible implementations, the following test
values may be useful for verifying the canonical transformation code.
Given the following JSON object:
.. code:: json
{}
The following canonical JSON should be produced:
.. code:: json
{}
Given the following JSON object:
.. code:: json
{
"one": 1,
"two": "Two"
}
The following canonical JSON should be produced:
.. code:: json
{"one":1,"two":"Two"}
Given the following JSON object:
.. code:: json
{
"b": "2",
"a": "1"
}
The following canonical JSON should be produced:
.. code:: json
{"a":"1","b":"2"}
Given the following JSON object:
.. code:: json
{"b":"2","a":"1"}
The following canonical JSON should be produced:
.. code:: json
{"a":"1","b":"2"}
Given the following JSON object:
.. code:: json
{
"auth": {
"success": true,
"mxid": "@john.doe:example.com",
"profile": {
"display_name": "John Doe",
"three_pids": [
{
"medium": "email",
"address": "john.doe@example.org"
},
{
"medium": "msisdn",
"address": "123456789"
}
]
}
}
}
The following canonical JSON should be produced:
.. code:: json
{"auth":{"mxid":"@john.doe:example.com","profile":{"display_name":"John Doe","three_pids":[{"address":"john.doe@example.org","medium":"email"},{"address":"123456789","medium":"msisdn"}]},"success":true}}
Given the following JSON object:
.. code:: json
{
"a": "日本語"
}
The following canonical JSON should be produced:
.. code:: json
{"a":"日本語"}
Given the following JSON object:
.. code:: json
{
"本": 2,
"日": 1
}
The following canonical JSON should be produced:
.. code:: json
{"日":1,"本":2}
Given the following JSON object:
.. code:: json
{
"a": "\u65E5"
}
The following canonical JSON should be produced:
.. code:: json
{"a":"日"}
Given the following JSON object:
.. code:: json
{
"a": null
}
The following canonical JSON should be produced:
.. code:: json
{"a":null}
Signing Details
~~~~~~~~~~~~~~~

@ -45,7 +45,7 @@ Other versions of this specification
The following other versions are also available, in reverse chronological order:
- `HEAD <https://matrix.org/speculator/spec/HEAD/client_server/unstable.html>`_: Includes all changes since the latest versioned release.
- `HEAD <https://matrix.org/docs/spec/client_server/unstable.html>`_: Includes all changes since the latest versioned release.
- `r0.2.0 <https://matrix.org/docs/spec/client_server/r0.2.0.html>`_
- `r0.1.0 <https://matrix.org/docs/spec/client_server/r0.1.0.html>`_
- `r0.0.1 <https://matrix.org/docs/spec/r0.0.1/client_server.html>`_
@ -171,8 +171,8 @@ Client Authentication
Most API endpoints require the user to identify themselves by presenting
previously obtained credentials in the form of an ``access_token`` query
parameter. An access token is typically obtained via the `Login`_ or
`Registration`_ processes.
parameter or through an Authorization Header of ``Bearer $access_token``.
An access token is typically obtained via the `Login`_ or `Registration`_ processes.
When credentials are required but missing or invalid, the HTTP call will
return with a status of 401 and the error code, ``M_MISSING_TOKEN`` or
@ -880,6 +880,8 @@ Where $streamtoken is an opaque token which can be used in another query to
get the next set of results. The "start" and "end" keys can only be omitted if
the complete dataset is provided in "chunk".
.. _`filter`:
Filtering
---------

@ -32,8 +32,11 @@ server-server API.
Size limits
-----------
The total size of any event MUST NOT exceed 65 KB. There are additional
restrictions on sizes per key:
The complete event MUST NOT be larger than 65535 bytes, when formatted as a
`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_,
including any signatures, and encoded as `Canonical JSON`_.
There are additional restrictions on sizes per key:
- ``sender`` MUST NOT exceed 255 bytes (including domain).
- ``room_id`` MUST NOT exceed 255 bytes.
@ -67,3 +70,4 @@ prefixed with ``m.``
{{m_room_redaction_event}}
.. _`Canonical JSON`: ../appendices.html#canonical-json

@ -27,17 +27,17 @@ Voice over IP (VoIP) signalling, Internet of Things (IoT) communication, and bri
together existing communication silos - providing the basis of a new open real-time
communication ecosystem.
`Introduction to Matrix <intro.html>`_ provides a full introduction to Matrix and the spec.
Matrix APIs
-----------
The following APIs are documented in this specification:
The specification consists of the following parts:
`Introduction to Matrix <intro.html>`_ provides a full introduction to Matrix and the spec.
{{apis}}
`Appendices <appendices.html>`_ with supplemental information not specific to
one of the above APIs are also available.
The `Appendices <appendices.html>`_ contain supplemental information not specific to
one of the above APIs.
Specification Versions
----------------------

@ -157,9 +157,8 @@ allocated the account and has the form::
@localpart:domain
See the `Identifier Grammar`_ section for full details of the structure of
user IDs.
See the `appendices <appendices.html#identifier-grammar>`_ for full details of
the structure of user IDs.
Devices
~~~~~~~
@ -242,8 +241,8 @@ There is exactly one room ID for each room. Whilst the room ID does contain a
domain, it is simply for globally namespacing room IDs. The room does NOT
reside on the domain specified.
See the `Identifier Grammar`_ section for full details of the structure of
a room ID.
See the `appendices <appendices.html#identifier-grammar>`_ for full details of
the structure of a room ID.
The following conceptual diagram shows an
``m.room.message`` event being sent to the room ``!qporfwt:matrix.org``::
@ -318,8 +317,8 @@ Each room can also have multiple "Room Aliases", which look like::
#room_alias:domain
See the `Identifier Grammar`_ section for full details of the structure of
a room alias.
See the `appendices <appendices.html#identifier-grammar>`_ for full details of
the structure of a room alias.
A room alias "points" to a room ID and is the human-readable label by which
rooms are publicised and discovered. The room ID the alias is pointing to can
@ -387,221 +386,6 @@ dedicated API. The API is symmetrical to managing Profile data.
Would it really be overengineered to use the same API for both profile &
private user data, but with different ACLs?
Identifier Grammar
------------------
Server Name
~~~~~~~~~~~
A homeserver is uniquely identified by its server name. This value is used in a
number of identifiers, as described below.
The server name represents the address at which the homeserver in question can
be reached by other homeservers. The complete grammar is::
server_name = dns_name [ ":" port]
dns_name = host
port = *DIGIT
where ``host`` is as defined by `RFC3986, section 3.2.2
<https://tools.ietf.org/html/rfc3986#section-3.2.2>`_.
Examples of valid server names are:
* ``matrix.org``
* ``matrix.org:8888``
* ``1.2.3.4`` (IPv4 literal)
* ``1.2.3.4:1234`` (IPv4 literal with explicit port)
* ``[1234:5678::abcd]`` (IPv6 literal)
* ``[1234:5678::abcd]:5678`` (IPv6 literal with explicit port)
Common Identifier Format
~~~~~~~~~~~~~~~~~~~~~~~~
The Matrix protocol uses a common format to assign unique identifiers to a
number of entities, including users, events and rooms. Each identifier takes
the form::
&localpart:domain
where ``&`` represents a 'sigil' character; ``domain`` is the `server name`_ of
the homeserver which allocated the identifier, and ``localpart`` is an
identifier allocated by that homeserver.
The sigil characters are as follows:
* ``@``: User ID
* ``!``: Room ID
* ``$``: Event ID
* ``#``: Room alias
The precise grammar defining the allowable format of an identifier depends on
the type of identifier.
User Identifiers
++++++++++++++++
Users within Matrix are uniquely identified by their Matrix user ID. The user
ID is namespaced to the homeserver which allocated the account and has the
form::
@localpart:domain
The ``localpart`` of a user ID is an opaque identifier for that user. It MUST
NOT be empty, and MUST contain only the characters ``a-z``, ``0-9``, ``.``,
``_``, ``=``, and ``-``.
The ``domain`` of a user ID is the `server name`_ of the homeserver which
allocated the account.
The length of a user ID, including the ``@`` sigil and the domain, MUST NOT
exceed 255 characters.
The complete grammar for a legal user ID is::
user_id = "@" user_id_localpart ":" server_name
user_id_localpart = 1*user_id_char
user_id_char = DIGIT
/ %x61-7A ; a-z
/ "-" / "." / "=" / "_"
.. admonition:: Rationale
A number of factors were considered when defining the allowable characters
for a user ID.
Firstly, we chose to exclude characters outside the basic US-ASCII character
set. User IDs are primarily intended for use as an identifier at the protocol
level, and their use as a human-readable handle is of secondary
benefit. Furthermore, they are useful as a last-resort differentiator between
users with similar display names. Allowing the full unicode character set
would make very difficult for a human to distinguish two similar user IDs. The
limited character set used has the advantage that even a user unfamiliar with
the Latin alphabet should be able to distinguish similar user IDs manually, if
somewhat laboriously.
We chose to disallow upper-case characters because we do not consider it
valid to have two user IDs which differ only in case: indeed it should be
possible to reach ``@user:matrix.org`` as ``@USER:matrix.org``. However,
user IDs are necessarily used in a number of situations which are inherently
case-sensitive (notably in the ``state_key`` of ``m.room.member``
events). Forbidding upper-case characters (and requiring homeservers to
downcase usernames when creating user IDs for new users) is a relatively simple
way to ensure that ``@USER:matrix.org`` cannot refer to a different user to
``@user:matrix.org``.
Finally, we decided to restrict the allowable punctuation to a very basic set
to ensure that the identifier can be used as-is in as wide a number of
situations as possible, without requiring escaping. For instance, allowing
"%" or "/" would make it harder to use a user ID in a URI. "*" is used as a
wildcard in some APIs (notably the filter API), so it also cannot be a legal
user ID character.
The length restriction is derived from the limit on the length of the
``sender`` key on events; since the user ID appears in every event sent by the
user, it is limited to ensure that the user ID does not dominate over the actual
content of the events.
Matrix user IDs are sometimes informally referred to as MXIDs.
Historical User IDs
<<<<<<<<<<<<<<<<<<<
Older versions of this specification were more tolerant of the characters
permitted in user ID localparts. There are currently active users whose user
IDs do not conform to the permitted character set, and a number of rooms whose
history includes events with a ``sender`` which does not conform. In order to
handle these rooms successfully, clients and servers MUST accept user IDs with
localparts from the expanded character set::
extended_user_id_char = %x21-7E
Mapping from other character sets
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
In certain circumstances it will be desirable to map from a wider character set
onto the limited character set allowed in a user ID localpart. Examples include
a homeserver creating a user ID for a new user based on the username passed to
``/register``, or a bridge mapping user ids from another protocol.
.. TODO-spec
We need to better define the mechanism by which homeservers can allow users
to have non-Latin login credentials. The general idea is for clients to pass
the non-Latin in the ``username`` field to ``/register`` and ``/login``, and
the HS then maps it onto the MXID space when turning it into the
fully-qualified ``user_id`` which is returned to the client and used in
events.
Implementations are free to do this mapping however they choose. Since the user
ID is opaque except to the implementation which created it, the only
requirement is that the implemention can perform the mapping
consistently. However, we suggest the following algorithm:
1. Encode character strings as UTF-8.
2. Convert the bytes ``A-Z`` to lower-case.
* In the case where a bridge must be able to distinguish two different users
with ids which differ only by case, escape upper-case characters by
prefixing with ``_`` before downcasing. For example, ``A`` becomes
``_a``. Escape a real ``_`` with a second ``_``.
3. Encode any remaining bytes outside the allowed character set, as well as
``=``, as their hexadecimal value, prefixed with ``=``. For example, ``#``
becomes ``=23``; ``á`` becomes ``=c3=a1``.
.. admonition:: Rationale
The suggested mapping is an attempt to preserve human-readability of simple
ASCII identifiers (unlike, for example, base-32), whilst still allowing
representation of *any* character (unlike punycode, which provides no way to
encode ASCII punctuation).
Room IDs and Event IDs
++++++++++++++++++++++
A room has exactly one room ID. A room ID has the format::
!opaque_id:domain
An event has exactly one event ID. An event ID has the format::
$opaque_id:domain
The ``domain`` of a room/event ID is the `server name`_ of the homeserver which
created the room/event. The domain is used only for namespacing to avoid the
risk of clashes of identifiers between different homeservers. There is no
implication that the room or event in question is still available at the
corresponding homeserver.
Event IDs and Room IDs are case-sensitive. They are not meant to be human
readable.
.. TODO-spec
What is the grammar for the opaque part? https://matrix.org/jira/browse/SPEC-389
Room Aliases
++++++++++++
A room may have zero or more aliases. A room alias has the format::
#room_alias:domain
The ``domain`` of a room alias is the `server name`_ of the homeserver which
created the alias. Other servers may contact this homeserver to look up the
alias.
Room aliases MUST NOT exceed 255 bytes (including the ``#`` sigil and the
domain).
.. TODO-spec
- Need to specify precise grammar for Room Aliases. https://matrix.org/jira/browse/SPEC-391
License
-------

@ -46,6 +46,8 @@ Usage of this event is discouraged for several reasons:
{{m_room_avatar_event}}
{{m_room_pinned_events_event}}
m.room.message msgtypes
~~~~~~~~~~~~~~~~~~~~~~~

@ -1,4 +1,5 @@
.. Copyright 2016 OpenMarket Ltd
.. Copyright 2017 New Vector Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
@ -442,24 +443,173 @@ keys exist to support this:
``state_key`` String Combined with the ``pdu_type`` this
identifies the which part of the room
state is updated
``required_power_level`` Integer The required power level needed to
replace this update.
``prev_state_id`` String The PDU id of the update this replaces.
``prev_state_origin`` String The homeserver of the update this
replaces.
``user_id`` String The user updating the state.
======================== ============ =========================================
.. code:: json
Authorization of PDUs
~~~~~~~~~~~~~~~~~~~~~
{...,
"is_state":true,
"state_key":TODO-doc
"required_power_level":TODO-doc
"prev_state_id":TODO-doc
"prev_state_origin":TODO-doc
}
Whenever a server receives an event from a remote server, the receiving server
must check that the event is allowed by the authorization rules. These rules
depend on the state of the room at that event.
Definitions
+++++++++++
Required Power Level
A given event type has an associated *required power level*. This is given by
the current ``m.room.power_levels`` event. The event type is either listed
explicitly in the ``events`` section or given by either ``state_default`` or
``events_default`` depending on if the event is a state event or not.
Invite Level, Kick Level, Ban Level, Redact Level
The levels given by the ``invite``, ``kick``, ``ban``, and ``redact``
properties in the current ``m.room.power_levels`` state. Each defaults to 50
if unspecified.
Target User
For an ``m.room.member`` state event, the user given by the ``state_key`` of
the event.
Rules
+++++
The rules governing whether an event is authorized depend solely on the
state of the room at the point in the room graph at which the new event is to
be inserted. The types of state events that affect authorization are:
- ``m.room.create``
- ``m.room.member``
- ``m.room.join_rules``
- ``m.room.power_levels``
Servers should not create new events that reference unauthorized events.
However, any event that does reference an unauthorized event is not itself
automatically considered unauthorized.
Unauthorized events that appear in the event graph do *not* have any effect on
the state of the room.
.. Note:: This is in contrast to redacted events which can still affect the
state of the room. For example, a redacted ``join`` event will still
result in the user being considered joined.
1. If type is ``m.room.create``, allow if and only if has depth 0 and it has no
previous events - *i.e.* it is the first event in the room.
#. If type is ``m.room.member``:
a. If ``membership`` is ``join``:
i. If the previous event in the room graph is an ``m.room.create``, the
depth is 1 and the ``state_key`` is the creator, allow.
#. If the ``sender`` does not match ``state_key``, reject.
#. If the user's current membership state is ``invite`` or ``join``,
allow.
#. If the ``join_rule`` is ``public``, allow.
#. Otherwise, reject.
#. If ``membership`` is ``invite``:
i. If the ``sender``'s current membership state is not ``joined``, reject.
#. If *target user*'s current membership state is ``join`` or ``ban``,
reject.
#. If the ``sender``'s power level is greater than or equal to the *invite
level*, allow.
#. Otherwise, reject.
#. If ``membership`` is ``leave``:
i. If the ``sender`` matches ``state_key``, allow if and only if that user's
current membership state is ``invite`` or ``join``.
#. If the ``sender``'s current membership state is not ``joined``, reject.
#. If the *target user*'s current membership state is ``ban``, and the
``sender``'s power level is less than the *ban level*, reject.
#. If the ``sender``'s power level is greater than or equal to the *kick
level*, and the *target user*'s power level is less than the
``sender``'s power level, allow.
#. Otherwise, reject.
#. If ``membership`` is ``ban``:
i. If the ``sender``'s current membership state is not ``joined``, reject.
#. If the ``sender``'s power level is greater than or equal to the *ban
level*, and the *target user*'s power level is less than the
``sender``'s power level, allow.
#. Otherwise, reject.
#. Otherwise, the membership is unknown. Reject.
#. If the ``sender``'s current membership state is not ``joined``, reject.
#. If the event type's *required power level* is greater than the ``sender``'s power
level, reject.
#. If type is ``m.room.power_levels``:
a. For each of the keys ``users_default``, ``events_default``,
``state_default``, ``ban``, ``redact``, ``kick``, ``invite``, as well as
each entry being changed under the ``events`` or ``users`` keys:
i. If the current value is higher than the ``sender``'s current power level,
reject.
#. If the new value is higher than the ``sender``'s current power level,
reject.
#. For each entry being changed under the ``users`` key, other than the
``sender``'s own entry:
i. If the current value is equal to the ``sender``'s current power level,
reject.
#. Otherwise, allow.
#. If type is ``m.room.redact``:
#. If the ``sender``'s power level is greater than or equal to the *redact
level*, allow.
#. If the ``sender`` of the event being redacted is the same as the
``sender`` of the ``m.room.redact``, allow.
#. Otherwise, reject.
#. Otherwise, allow.
.. NOTE::
Some consequences of these rules:
* Unless you are a member of the room, the only permitted operations (apart
from the intial create/join) are: joining a public room; accepting or
rejecting an invitation to a room.
* To unban somebody, you must have power level greater than or equal to both
the kick *and* ban levels, *and* greater than the target user's power
level.
.. TODO-spec
I think there is some magic about 3pid invites too.
EDUs
----

@ -34,6 +34,7 @@ targets:
- appendices.rst
- appendices/base64.rst
- appendices/signing_json.rst
- appendices/identifier_grammar.rst
- appendices/threat_model.rst
- appendices/test_vectors.rst
groups: # reusable blobs of files when prefixed with 'group:'

@ -1,129 +0,0 @@
Application Services
====================
This file contains examples of some application service
IRC Bridge
----------
Pre-conditions:
- Server admin stores the AS token "T_a" on the homeserver.
- Homeserver has a token "T_h".
- Homeserver has the domain "hsdomain.com"
1. Application service registration
::
AS -> HS: Registers itself with the homeserver
POST /register
{
url: "https://someapp.com/matrix",
as_token: "T_a",
namespaces: {
users: [
{
"exclusive": true,
"regex": "@irc\.freenode\.net/.*"
}
],
aliases: [
{
"exclusive": true,
"regex": "#irc\.freenode\.net/.*"
}
]
}
}
Returns 200 OK:
{
hs_token: "T_h"
}
2. IRC user "Bob" says "hello?" on "#matrix" at timestamp 1421416883133:
::
- AS stores message as potential scrollback.
- Nothing happens as no Matrix users are in the room.
3. Matrix user "@alice:hsdomain.com" wants to join "#matrix":
::
User -> HS: Request to join "#irc.freenode.net/#matrix:hsdomain.com"
HS -> AS: Room Query "#irc.freenode.net/#matrix:hsdomain.com"
GET /rooms/%23irc.freenode.net%2F%23matrix%3Ahsdomain.com?access_token=T_h
[Starts blocking]
AS -> HS: Creates room. Gets room ID "!aasaasasa:hsdomain.com".
AS -> HS: Sets room name to "#matrix".
AS -> HS: Sends message as ""@irc.freenode.net/Bob:hsdomain.com"
PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message
?access_token=T_a
&user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com
&ts=1421416883133
{
body: "hello?"
msgtype: "m.text"
}
HS -> AS: User Query "@irc.freenode.net/Bob:hsdomain.com"
GET /users/%40irc.freenode.net%2FBob%3Ahsdomain.com?access_token=T_h
[Starts blocking]
AS -> HS: Creates user using CS API extension.
POST /register?access_token=T_a
{
type: "m.login.application_service",
user: "irc.freenode.net/Bob"
}
AS -> HS: Set user display name to "Bob".
[Finishes blocking]
[Finished blocking]
- HS sends room information back to client.
4. @alice:hsdomain.com says "hi!" in this room:
::
User -> HS: Send message "hi!" in room !aasaasasa:hsdomain.com
- HS sends message.
- HS sees the room ID is in the AS namespace and pushes it to the AS.
HS -> AS: Push event
PUT /transactions/1?access_token=T_h
{
events: [
{
content: {
body: "hi!",
msgtype: "m.text"
},
origin_server_ts: <generated by hs>,
user_id: "@alice:hsdomain.com",
room_id: "!aasaasasa:hsdomain.com",
type: "m.room.message"
}
]
}
- AS passes this through to IRC.
5. IRC user "Bob" says "what's up?" on "#matrix" at timestamp 1421418084816:
::
IRC -> AS: "what's up?"
AS -> HS: Send message via CS API extension
PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message
?access_token=T_a
&user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com
&ts=1421418084816
{
body: "what's up?"
msgtype: "m.text"
}
- HS modifies the user_id and origin_server_ts on the event and sends it.

@ -1,141 +0,0 @@
---
layout: post
title: Getting involved
categories: guides
---
# How can I get involved?
Matrix is an ecosystem consisting of several apps written by lots of people. We at Matrix.org have written one server and a few clients, and people in the community have also written several clients, servers, and Application Services. We are collecting [a list of all known Matrix-apps](https://matrix.org/blog/try-matrix-now/).
|
You have a few options when it comes to getting involved: if you just want to use Matrix, you can [register an account on a public server using a public webclient](#reg). If you have a virtual private server (VPS) or similar, you might want to [run a server and/or client yourself](#run). If you want to look under the hood, you can [checkout the code and modify it - or write your own client or server](#checkout). Or you can write an [Application Service](#as), for example a bridge to an existing ecosystem.
|
We very much welcome [contributions](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst) to any of our projects, which you can find in our [github space](https://github.com/matrix-org/).
|
<a class="anchor" id="reg"></a>
## Access Matrix via a public webclient/server
The easiest thing to do if you want to just have a play, is to use the [Riot.im
webclient](https://riot.im). You can use it as a guest, or register for an
account. For details of other clients, see
[https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now).
<a class="anchor" id="run"></a>
## Run a server and/or client yourself
You can clone our open source projects and run clients and servers yourself. Here is how:
### Running your own client:
You can run your own Matrix client; there are [numerous clients
available](https://matrix.org/blog/try-matrix-now/). You can easily [run your
own copy](https://github.com/vector-im/vector-web#getting-started) of the
Riot.im web client.
### Running your own homeserver:
One of the core features of Matrix is that anyone can run a homeserver and join the federated network on equal terms (there is no hierarchy). If you want to set up your own homeserver, please see the relevant docs of the server you want to run. If you want to run Matrix.org's reference homeserver, please consult the [readme of the Synapse project](https://github.com/matrix-org/synapse/blob/master/README.rst).
|
Note that Synapse comes with a bundled Matrix.org webclient - but you can tell it to use your [development checkout snapshot instead](https://github.com/matrix-org/matrix-angular-sdk#matrix-angular-sdk) (or to not run a webclient at all via the "web_client: false" config option).
|
<a class="anchor" id="checkout"></a>
## Checkout our code - or write your own
As described above, you can clone our code and [run a server and/or client yourself](#run). Infact, all the code that we at Matrix.org write is available from [our github](http://github.com/matrix-org) - and other servers and clients may also be open sourced - see [our list of all known Matrix-apps](https://matrix.org/blog/try-matrix-now/).
|
You can also implement your own client or server - after all, Matrix is at its core "just" a specification of a protocol.
|
### Write your own client:
The [client-server API
spec](http://matrix.org/docs/spec/client_server/latest.html) describes what API
calls are available to clients, and there is a [HOWTO
guide](http://matrix.org/docs/guides/client-server.html) which provides an
introduction to using the API along with some common operations. A quick
step-by-step guide would include:
|
1. Get a user either by registering your user in an existing client or running the [new-user script](https://github.com/matrix-org/synapse/blob/master/scripts/register_new_matrix_user) if you are running your own Synapse homeserver.
2. Assuming the homeserver you are using allows logins by password, log in via the login API:
```
curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/login"
```
3. If successful, this returns the following, including an `access_token`:
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd",
"home_server": "localhost",
"user_id": "@example:localhost"
}
4. This ``access_token`` will be used for authentication for the rest of your API calls. Potentially the next step you want is to make a call to the sync API and get the last few events from each room your user is in:
```
curl -XGET "http://localhost:8008/_matrix/client/r0/sync?access_token=YOUR_ACCESS_TOKEN"
```
5. The above will return something like this:
{
"next_batch": "s72595_4483_1934",
"rooms": {
"join": {
"!726s6s6q:example.com": {
"state": {
"events": [
...
]
},
"timeline": {
"events": [
...
]
}
},
...
}
}
}
You can then use the "next_batch" token to start listening for new events like so:
```
curl -XGET "http://localhost:8008/_matrix/client/r0/sync?access_token=YOUR_ACCESS_TOKEN&since=s72595_4483_1934"
```
6. This request will block waiting for an incoming event, timing out after several seconds if there is no event. This ensures that you only get new events. Now you have the initial room states, and a stream of events - a good client should be able to process all these events and present them to the user. And potentially you might want to add functionality to generate events as well (such as messages from the user, for example)!
### Write your own server:
We are still working on the server-server spec, so the best thing to do if you are interested in writing a server, is to come talk to us in [#matrix:matrix.org](https://matrix.to/#/#matrix:matrix.org).
If you are interested in how federation works, please see the [Server-Server API spec](http://matrix.org/docs/spec/server_server/unstable.html).
|
<a class="anchor" id="as"></a>
## Write an Application Service:
Information about Application services and how they can be used can be found in the [Application services](./application_services.html) document! (This is based on Kegan's excellent blog posts, but now lives here so it can be kept up-to-date!)

@ -1,691 +0,0 @@
---
layout: post
title: FAQ
date: 2015-08-19 16:58:07
categories: guides
---
<link href="/docs/css/faq.css" type="text/css" rel="stylesheet" />
# FAQ
{:.no_toc}
Categories
----------
{:.no_toc}
[General](#general)
[Quick Start](#quick-start)
[Standard](#standard)
[Servers](#servers)
[Clients](#clients)
|
FAQ Content
-----------
{:.no_toc}
* TOC
{:toc .toc}
### General
##### What is Matrix?
Matrix is an open standard for interoperable, decentralised,
real-time communication over IP. It can be used to power Instant
Messaging, VoIP/WebRTC signalling, Internet of Things communication - or anywhere
you need a standard HTTP API for publishing and subscribing to
data whilst tracking the conversation history.
|
Matrix defines the standard, and provides open source reference implementations
of Matrix-compatible Servers, Clients, Client SDKs and Application Services
to help you create new communication solutions or extend the capabilities
and reach of existing ones.
##### What is Matrix's Mission?
Matrix's initial goal is to fix the problem of fragmented IP communications:
letting users message and call each other without having to care what app
the other user is on - making it as easy as sending an email.
|
The longer term goal is for Matrix to act as a generic HTTP messaging and data
synchronisation system for the whole web - allowing people, services and devices
to easily communicate with each other, empowering users to own and control their
data and select the services and vendors they want to use.
##### What does Matrix provide?
Matrix provides:
- [Open Standard](/docs/spec) HTTP APIs for transferring JSON messages (e.g. instant messages, WebRTC signalling), including:
- [Client\<-\>Server API](/docs/spec#client-server-api-v1) - defines how Matrix compatible clients communicate with Matrix homeservers.
- [Server\<-\>Server API](/docs/spec#federation-api) - defines how Matrix homeservers exchange messages and synchronise history with each other.
- [Application Service API](/docs/spec/#application-service-api) - defines how to extend the functionality of Matrix with 'integrations' and bridge to other networks.
- [Modules](/docs/spec/#modules) - specifies features that must be implemented by particular classes of clients.
- Open source reference implementations of:
- Clients (Web (React), iOS, Android)
- Client SDKs (Javascript, Web (React), iOS, Android)
- Homeservers (Synapse)
- Application Services (bridges to IRC, Slack, Skype, Lync and more...)
- The actual ecosystem and community of everyone running Matrix servers and services
- Loads of 3rd party contributions of clients, SDKs, servers and services.
You can find the full list of Matrix enabled projects at [https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now).
##### What does this mean for users?
The aim is to provide an analogous ecosystem to email - one where you
can communicate with pretty much anyone, without caring what app or
server they are using, using whichever app & server you chose to use,
and use a neutral identity system like an e-mail address or phone
number to discover people to talk to.
##### What kind of company is Matrix.org?
Matrix.org is an open initiative which acts as a neutral and independent custodian
of the Matrix standard. As of Sept 2017 we are finally in the process of incorporating
it as a dedicated non-profit entity (most likely a limited by guarantee UK private company
called the Matrix.org Foundation).
##### Who is funding Matrix.org?
Matrix.org is currently (Sept 2017) funded by the community, through a
combination of community support (via
[Patreon](https://patreon.com/matrixdotorg),
[Liberapay](https://liberapay.com/matrixdotorg), Bitcoin and Ethereum),
corporate sponsorship, and grant funding. Current Elliptic-level supporters on
Patreon and corporate sponsors can be found on our [supporters
page](https://matrix.org/blog/supporters). If you would like to support the core
Matrix team as a member of the community, please visit our
[Patreon](https://patreon.com/matrixdotorg) or
[Liberapay](https://liberapay.com/matrixdotorg) pages, or you can send us
Bitcoin at 1LxowEgsquZ3UPZ68wHf8v2MDZw82dVmAE or Ethereum at ETH
0xA5f9a4f9E024F6D727f7afdA9257e22329A97485. If you would like to sponsor the
team as a corporation, or are interested in paying for prioritised or custom
development, please [get in touch](mailto:matthew@matrix.org).
For the first three years of Matrix's development (2014-2017), most of the core
contributors worked for [Amdocs](https://www.amdocs.com), who paid for them to
work fulltime on Matrix. In July 2017, Amdocs considered the project to be
sufficiently successful that it could now self-support and so stopped funding.
The majority of the core team is now employed by New Vector, an independent company
set up to hire the team and support Matrix's development. Other contributors
are funded by their own employers or donate their own time to the project.
##### Who is building Matrix?
The core team is ~12 people with extensive experience in building custom
VoIP and Messaging apps for mobile network operators. Most of us work for New Vector,
but there are an increasing number of contributors from other companies and
folks all over the internet.
##### Why are you called Matrix?
We are called Matrix because we provide a structure in which all
communication can be matrixed together.
|
No, it's nothing to do with the film (although you could go and build virtual
worlds on top of Matrix if you wanted :)
##### Why have you released this as open source?
We believe that any open standard defining interoperable communication
needs to be justified, demonstrated and validated with transparent open
source implementations. For Matrix to achieve its mission of making all
communications services interoperable we believe it needs to be truly
open, giving people access to take all the code we produce and to use
and build on top of it.
##### What do you mean by open?
Matrix is an open standard, meaning that we have freely published the
details for how to communicate interoperably using the Matrix set of
HTTP APIs. We encourage anyone and everyone to use the APIs and build
their own projects which implement them and so benefit from
interoperability with the rest of the Matrix ecosystem. We also
ensure the standard is not encumbered by any known patent licensing
requirements.
|
Matrix is also open source, meaning that we have released the source
code of the reference servers, clients and services to the public domain
under the [Apache Licence v2](http://www.apache.org/licenses/LICENSE-2.0.html), to
encourage anyone and everyone to run their own servers and clients, and
enhance them and contribute their enhancements as they see fit.
##### What does federated mean?
Federation allows separate deployments of a communication service to
communicate with each other - for instance a mail server run by Google
federates with a mail server run by Microsoft when you send email from
@gmail.com to @hotmail.com.
|
Federation is different to interoperability, as interoperable clients
may simply be running on the same deployment - whereas in federation the
deployments themselves are exchanging data in a compatible manner.
|
Matrix provides open federation - meaning that anyone on the internet
can join into the Matrix ecosystem by deploying their own server.
##### How is this like e-mail?
When email was first set up in the early 80s, companies like Compuserve
and AT&T and Sprint set up isolated email communities which only allowed
you to exchange mail with users on the same system.  If you got your
email from one service and your friend from another, then you couldn't
message each other.  This is basically the situation we're in today with
VoIP and IM.
##### Why has no-one done this before?
There have been several attempts before including SIP, XMPP and RCS.
 All of these have had some level of success, but many different
technological/usability/economic factors have ended up limiting their
success. Unfortunately, we've not ended up in a world where everyone
has a SIP URI or Jabber ID on their business card, or a phone that
actually uses RCS.
##### What is the difference between Matrix and IRC?
We love IRC.  In fact, as of today the core Matrix team still uses it as
our primary communication tool. Between us we've written IRCds, IRC bots
and admined dreamforge, UnrealIRCd, epona, ircservices and several
others. That said, it has some limitations that Matrix seeks to improve
on:
- Text only
- No history
- No multiple-device support
- No presence support
- Fragmented identity model
- No open federation
- No standard APIs, just a rather limited TCP line protocol
- Non-standardised federation protocol
- No built-in end-to-end encryption
- Disruptive net-splits
- Non-extensible
[IRCv3](http://ircv3.net) exists and is addressing some of these issues;
this is great news and we wish them well. It's almost a contradiction
in terms to get competitive between openly interoperable communication
projects - we look forward to increasing the richness of Matrix\<-\>IRC
bridges as the project progresses.
##### What is the difference between Matrix and XMPP?
The Matrix team used XMPP (Openfire, ejabberd, spectrum, asmack,
XMPPFramework) for IM before starting to experiment with open HTTP APIs
as an alternative in around 2012. The main issues with XMPP that
drove us in this direction **as of 2012** were:
- Not particularly web-friendly - you can't easily speak XMPP from a
web browser. *N.B. Nowadays you have options like XMPP-FTW and
Stanza.io that help loads with letting browsers talk XMPP*
- Single logical server per MUC is a single point of control and
availability. *MUCs can be distributed over multiple physical
servers, but they still sit behind a single logical JID and domain.
FMUC improves this with a similar approach to Matrix, but as of Oct
2015 there are no open source implementations. The MIX XMPP extension
also aims to address this limitation*.
- History synchronisation is very much a second class citizen feature
- Bridging to other protocols and defragmenting existing communication
apps and networks is very much a second class citizen feature
- Stanzas aren't framed or reliably delivered without extensions. *See
[wiki.xmpp.org](http://wiki.xmpp.org/web/Myths#Myth_Four:_XMPP_is_unreliable_without_a_bunch_of_extensions.)
for an XMPP take on this*
- Multiple device support is limited. *Carbons and MAM aim to resolve this*
- Baseline feature set is so minimal that fragmentation of features
between clients and servers is common, especially as interoperability
profiles for features have fallen behind (as of July 2015)
- No strong identity system (i.e. no standard E2E PKI, unless you
count X.509 certs, which [are
questionable](http://www.thoughtcrime.org/blog/ssl-and-the-future-of-authenticity/))
- Not particularly well designed for mobile use cases: push;
bandwidth-efficient transports. *Since the time of writing a [Push
XEP has appeared](http://xmpp.org/extensions/xep-0357.html), and
[wiki.xmpp.org](http://wiki.xmpp.org/web/Myths#Myth_Three:_It.27s_too_bandwidth-inefficient_for_mobile.)
states that XMPP is usable over a 9600bps + 30s latency link.*
This said, the whole area of XMPP vs Matrix is quite subjective.
Rather than fighting over which open interoperable communication
standard works the best, we should just collaborate and bridge everything
together. The more federation and interoperability the better.
|
We think of Matrix and XMPP as being quite different; at its core
Matrix can be thought of as an eventually consistent global JSON db with
an HTTP API and pubsub semantics - whilst XMPP can be thought of as a
message passing protocol. You can use them both to build chat systems;
you can use them both to build pubsub systems; each comes with different
tradeoffs. Matrix has a deliberately extensive 'kitchen sink' baseline
of functionality; XMPP has a deliberately minimal baseline set of
functionality. If XMPP does what you need it to do, then we're genuinely
happy for you :) Meanwhile, rather than competing, an XMPP Bridge like
[Skaverat's xmpptrix beta](https://github.com/SkaveRat/xmpptrix) or
[jfred's matrix-xmpp-bridge](https://github.com/jfrederickson/matrix-xmpp-bridge)
or Matrix.org's own [purple-matrix](https://github.com/matrix-org/purple-matrix/)
has potential to let both environments coexist and make the most of each
other's benefits.
##### What is the difference between Matrix and PSYC?
PSYC is a open federated messaging protocol loosely inspired by IRC.  In
version 1 it was a standalone protocol, and in version 2 it is being
reutilised as a messaging layer on top of GNUnet.  We honestly don't
know that much about it, beyond trying to use psycd as an XMPP\<-\>IRC
bridge in 2010. Matrix differentiates primarily by providing simple HTTP
APIs rather than the more exotic compact line protocol in PSYC v1 or the
comprehensive GNUnet stack in v2, and Matrix focuses more on decentralised
conversation history rather than just decentralised chat servers.
On the other hand, Matrix doesn't provide the metadata protection
guarantees that GNUnet/PSYC aims for.
|
See [http://about.psyc.eu/Matrix](http://about.psyc.eu/Matrix) for
PSYC's views on Matrix.
##### What is the difference between Matrix and Tox?
Tox.chat looks to be a very cool clone of Skype - a fully decentralised
peer-to-peer network.  Matrix is deliberately not a 'pure' peer-to-peer
system; instead each user has a well-defined homeserver which stores
his data and that he can depend upon.  Matrix provides HTTP APIs;
Tox.chat provides C APIs. As of October 2015 Tox doesn't seem to have an
answer yet for decentralised conversation history storage.
##### How does Matrix compare with something like Trillian or Pidgin?
Trillian and Pidgin and similar aggregating IM clients merge all your IM
activity into a single app.  However, your history and
identity is still fragmented across the networks.  People can't find you
easily, and your history is fragmented (other than on the device
where the client runs).   And rather than being able to chose the right
app for the job when communicating with people, you are pushed towards
relying on a specific aggregation app.
Matrix lets you get the best of both worlds by linking to all the
different networks (XMPP, AIM, ICQ, Lync, Skype etc) on the serverside,
using bridges which can be run by anyone. Matrix then provides a simple
standard HTTP API to access any of these networks, and lets you choose
whichever client you prefer (either as a 'native' Matrix client or using
a non-Matrix client from one of the networks which has been bridged in).
##### What Matrix compliant apps are there?
Quite a few, ranging from the glossy mass-market to the geeky command-line. There's even an emacs macro. Check out [https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now) for the current
list of Matrix enabled projects.
##### What bridges to other networks are available?
The number of 'bridges' which integrate existing communication networks into
Matrix are growing on a daily basis - both written by the Matrix core team
and contributed by the wider community. The full list can be seen at
[https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now), but the core ones as of Oct 2015 include:
* [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) - an increasingly comprehensive Matrix\<-\>IRC bridge
* [matrix-appservice-verto](https://github.com/matrix-org/matrix-appservice-verto) - links from Matrix to FreeSWITCH via the Verto protocol
* [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) - a basic bridge to Slack
* [node-purple](https://github.com/matrix-org/node-purple) - lets you access any of the 20+ protocols supported by
[libpurple](https://developer.pidgin.im/wiki/WhatIsLibpurple), including
Skype, Lync, XMPP, etc)
* [matrix-appservice-bridge](https://github.com/matrix-org/matrix-appservice-bridge) - a general NodeJS framework for writing bridges
Writing new bridges is incredibly fun and easy - see the [matrix-appservice-bridge HOWTO](https://github.com/matrix-org/matrix-appservice-bridge/blob/master/HOWTO.md)
for an example of how to write a fully functional Slack bridge in less than 100 lines of code!
##### Why do you think existing apps will ever join this officially?
We firmly believe it is what is right for the consumer. As people begin
to use interoperable communications tools, service providers will see the
benefit and compete on quality of service, security and features rather
than relying on locking people into their walled garden. We believe as
soon as users see the availability and benefits of interoperable
services they will demand it.
##### Why aren't you doing this through the IETF? or W3C? or 3GPP?
We do recognise the advantages of working with existing standards
bodies. We have been focused on writing code and getting it out, and the standard has been evolving rapidly since initial release in September 2014.
Once the standard has matured sufficiently it may well be appropriate to work with an official
standard body to maintain it going forwards.
### Quick Start
##### How do I get an account and get started?
The quickest way is to pick a client from [https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now) and sign up.
Please note that you can point clients to access any homeserver - you don't have to use matrix.org,
although as of day 1, matrix.org is the only communal homeserver
available.
##### What can I actually do with this?
A typical client provides a simple chatroom interface to Matrix -
letting the user interact with users and rooms anywhere within the
Matrix federation.  Text and image messages are supported, and basic
voice-only VoIP calling via WebRTC is supported in one-to-one rooms.
(As of October 2015, experimental multi-way calling is also available
on Riot.im).
##### How do I connect my homeserver to the public Matrix network?
See
[http://github.com/matrix-org/synapse](http://github.com/matrix-org/synapse)
for details
##### How do I Matrix-enable my existing app?
If your app doesn't have any communication capability already, you'll want
to use one of the Matrix client SDKs to add it in. These come in different
levels of sophistication - ranging from a simple HTTP API wrapper (like matrix-js-sdk, matrix-ios-sdk or matrix-android-sdk)
through to reusable UI components (like matrix-react-sdk and matrix-ios-kit). Pick
the one for your platform, or a 3rd party one if none of the above work for you,
and get plugging it in. You'll probably also want to read the [Client-Server API
HOWTO](http://matrix.org/docs/howtos/client-server.html) too.
If you already have communication infrastructure set up (XMPP, custom HTTP, or whatever),
then you'll want to run a bridge to expose it to the wider Matrix ecosystem.
See [matrix-appservice-bridge HOWTO](https://github.com/matrix-org/matrix-appservice-bridge/blob/master/HOWTO.md) for a
guide of how to write bridges using the matrix-appservice-bridge framework, or co-opt one
from the list at [https://matrix.org/blog/try-matrix-now](https://matrix.org/blog/try-matrix-now).
[Application Service API](/docs/spec/#application-service-api) gives the details of the API
that bridges have to implement.
##### How can I write a client on Matrix?
See the [Client-Server API
HOWTO](http://matrix.org/docs/howtos/client-server.html) and the [API
docs](/docs/api) and [the Spec](/docs/spec) for all the details you need
to write a client.
##### How can I help out with this?
Come say hi on [\#matrix:matrix.org](https://matrix.to/#/#matrix:matrix.org)! Install synapse and tell us how you get on. Critique the spec.  Write
clients. Write bridges! Run bridges! Nose around in [Jira](https://matrix.org/jira) and
send us some pull requests on github to fix some bugs or add some features! You could even
try to write a homeserver (but be warned, Matrix's architecture makes homeservers orders of
magnitude harder than clients or bridges.)
See [CONTRIBUTING.rst](http://github.com/matrix-org/synapse/tree/master/CONTRIBUTING.rst) for
full details on how to contribute to the project. All are welcome!
##### Where can I get support?
[\#matrix:matrix.org](https://matrix.to/#/#matrix:matrix.org) aka \#matrix on irc.freenode.is your best bet.
##### How do I register custom matrix event types?
We're not yet managing a registry of custom matrix event types.  If you
have any particularly good ones you want to tell the world about, please
let us know on #matrix-dev:matrix.org.
##### How mature is this?
We started working on Matrix in July 2014, and opened it to the
public in September 2014. We got all the core features in place in December 2014
and entered beta, and since then have been iterating away on the beta refining the
architecture and APIs, fixing bugs and scalability, and adding new features, clients,
bridges etc.
As of October 2015 (synapse 0.10) it's good for serious experimentation and
non-production services and can absolutely be used in the real world. However, we're
still in beta and we'll want to freeze the spec and implement clustering and other
nice features before we really declare it ready for production.
### Standard
##### What is a client?
Users in Matrix use one or more clients to communicate. This could be any combination of a web client, a command line client, a mobile client - or embedded clients built into existing apps. It could even be a piece of hardware (e.g. a drone) that is Matrix enabled.
##### Can I use Matrix without installing a Matrix client?
Sure. An ever increasing number of protocols are being bridged into Matrix, so if you use something like IRC on Freenode you may well be indirectly benefiting from Matrix, as others may be connected into the IRC channel via Matrix.
##### What is a home server?
A user's clients connect to a single homeserver, which stores the communication history and account information for that user, and shares data with the wider Matrix ecosystem by synchronising communication history with other homeservers.
##### What is a MXID?
Matrix user IDs (MXID) are unique user IDs. They are in the format ```@username:homeserver.tld``` (this format is used to avoid confusing them with email addresses). They are intended to be fairly hidden (although right now they are not) - instead you will find and identify other users via 3PIDs.
##### What is a 3PID?
Third-party IDs (3PIDs) are IDs from other systems or contexts, such as email addresses, social network accounts and phone numbers.
##### What is an identity server?
Users in Matrix are identified internally via their matrix user ID (MXID). However, existing 3rd party ID (3PID) namespaces such as email addresses or phone numbers should be used publically to identify Matrix users, at least for invitation purposes. A Matrix "Identity" describes both the user ID and any other existing IDs from third party namespaces linked to their account.
|
Matrix users can link third-party IDs (3PIDs) to their user ID. Linking 3PIDs creates a mapping from a 3PID to a user ID. This mapping can then be used by Matrix users in order to discover the MXIDs of their contacts.
|
In order to ensure that the mapping from 3PID to user ID is genuine, a globally federated cluster of trusted "Identity Servers" (IS) are used to verify the 3PID and persist and replicate the mappings.
Usage of an IS is not required in order for a client application to be part of the Matrix ecosystem. However, without one clients will not be able to look up user IDs using 3PIDs.
|
The precise architecture of identity servers is currently in flux and subject to change as we work to fully decentralise them.
##### Where do my conversations get stored?
Each homeserver stores the communication history and account information for all of its clients, and shares data with the wider Matrix ecosystem by synchronising communication history with other homeservers and their clients. Clients typically communicate with each other by emitting events in the context of a virtual room. Room data is replicated across all of the homeservers *whose users are participating in a given room*.
##### What are redactions?
Since events are extensible it is possible for malicious users and/or servers to add keys that are, for example offensive or illegal. Since some events cannot be simply deleted (e.g. membership events) we instead 'redact' events, essentially stripping the event of all keys that are not required by the protocol. Redacting an event cannot be undone, allowing server owners to also delete the offending content from the databases.
##### How do you do VoIP calls on Matrix?
Voice (and video) over Matrix uses the WebRTC 1.0 standard to transfer call media (i.e. the actual voice and video traffic). Matrix is used to signal the establishment and termination of the call by sending call events, like any other event.
##### Are VoIP calls encrypted?
WebRTC encrypts the media that's being sent. The signalling events that set up (and end) the call are encrypted if the room they were sent in has enabled encryption.
##### Do I need a TURN server?
VoIP calls should work if both parties are on public networks. However, in practice one (or both) devices are often behind NAT, and so having a [TURN](https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT) server is important to help set up the call.
See [this guide](https://github.com/matrix-org/synapse/blob/master/docs/turn-howto.rst) for setting up a TURN server with Synapse.
##### Can I log into other homeservers with my username and password?
Currently, no. We are looking at options for decentralising or migrating user accounts between multiple servers, and might add this feature at a later stage.
##### Why Apache Licence?
The Apache Licence is a permissive licence. We want the Matrix protocol itself to be free and open, but people are free to create both free and commercial apps and services that uses the protocol. In our opinion, any Matrix-service only enhances the Matrix ecosystem.
##### Can I write a Matrix homeserver?
Yes. Matrix is just a spec, so implementations of the spec are very welcome! It should be noted that as of October 2015, changes are still being made to the spec, so if you want to write a Matrix homeserver, it is strongly recommended that you chat to the Matrix.org devs in [\#matrix:matrix.org](https://matrix.to/#/#matrix:matrix.org) first! You can also read about the [Federation API here](https://matrix.org/docs/spec/server_server/unstable.html).
##### How secure is this?
Server-server traffic is mandatorily TLS from the outset. Server-client traffic mandates transport layer encryption other than for tinkering. Servers maintain a public/private key pair, and sign the integrity of all messages in the context of the historical conversation, preventing tampering. Server keys are distributed using a [Perspectives](https://perspectives-project.org/)-style system.
End-to-end encryption is now available in the various [Riot.im](https://Riot.im) builds! This allows you to encrypt both 1:1 and group chats to protect user data stored on servers, using the [Olm](https://matrix.org/git/olm) cryptographic ratchet implementation. Read more on the [blog post](https://matrix.org/blog/2016/11/21/matrixs-olm-end-to-end-encryption-security-assessment-released-and-implemented-cross-platform-on-riot-at-last/) that announced the feature!
Privacy of metadata is not currently protected from server administrators - a malicious homeserver administrator can see who is talking to who and when, but not what is being said (once E2E encryption is enabled). See [this presentation from Jardin Entropique](http://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20Entropique.pdf) for a more comprehensive discussion of privacy in Matrix.
##### What is Perspectives?
Rather than relying on Certificate Authorities (CAs) as in traditional SSL, a [Perspectives](https://perspectives-project.org/)-style system uses a more decentralized model for verifying keys. Perspectives uses notary servers to verify that the same key is seen across the network, making a man-in-the-middle attack much harder since an attacker must insert itself into multiple places. For federation in Matrix, each Home Server acts as a notary. When one Home Server connects to another Home Server that uses a key that it doesn't recognize, it contacts other Home Servers to ensure that they all see the same key from that Home Server.
##### Why HTTP? Doesn't HTTP suck? Why don't you use websockets/CoAP/HTTP2/etc?
HTTP is indeed not the most efficient transport, but it is ubiquitous, very well understood and has numerous implementations on almost every platform and language. It also has a simple upgrade path to HTTP/2, which is relatively bandwidth and round-trip efficient.
It has thus been chosen as the mandatory baseline of the exchange, but it is still entirely possible to use more fancy protocols for communication between clients and server (see for example this [websocket transport draft](https://github.com/matrix-org/matrix-doc/blob/master/drafts/websockets.rst)), and it's also possible in the future that negotiation of more efficient protocols will be added for the federation between servers, with HTTP+JSON remaining as the compability baseline.
### Servers
##### What is Synapse?
Synapse is a reference "homeserver" implementation of Matrix from the core development team at matrix.org, written in Python 2/Twisted. It is intended to showcase the concept of Matrix and let folks see the spec in the context of a codebase and let you run your own homeserver and generally help bootstrap the ecosystem.
##### How do I join the global Matrix federation?
You can download and run one of the available Matrix servers - please see [this guide](http://matrix.org/docs/guides/getting_involved.html#run) for details!
##### What ports do I have to open up to join the global Matrix federation?
We recommend servers use port 8448 for server\<-\>server HTTPS traffic. Look at ["Setting up Federation"](https://github.com/matrix-org/synapse#setting-up-federation) in the Synapse readme file for details.
Client\<-\>Server traffic can talk directly to Synapse via port 8448, but as by default Synapse creates a self-signed TLS certificate this can cause problems for clients which can't easily trust self-signed certificates (e.g. most web browsers). Instead, you can proxy access to Synapse's HTTP listener on port 8008 via an existing HTTPS proxy with a valid certificate (e.g. an nginx listening on port 443), or you can point Synapse at a valid X.509 signed TLS certificate. In future, Synapse will probably use letsencrypt to autogenerate valid certificates rather than self-signed ones during installation, simplifying this process enormously.
You can also put Synapse entirely behind an existing TLS load balancer and not expose port 8448 at all. In this situation, Synapse will need to be configured to share the same *public* TLS certificate as the load balancer (as Synapse uses the public certificate for identity in other areas too, and it has to match the certificate that other servers see when they connect).
##### How do I run my own homeserver?
Follow the instructions for the homeserver you want to run. If you want to run Synapse, the reference homeserver from Matrix.org, follow [these instructions](https://github.com/matrix-org/synapse#synapse-installation).
##### Can I run my own identity server?
Yes - the reference implementation is
[sydent](https://github.com/matrix-org/sydent) and you can run your own ID server cluster that tracks 3rd party to Matrix ID mappings. This won't be very useful right now, though, and we don't recommend it.
If you want your server to participate in the global replicated Matrix ID
service then please get in touch with us. Meanwhile, we are looking at
ways of decentralising the 'official' Matrix identity service so that
identity servers are 100% decentralised and can openly federate with
each other. **N.B. that you can use Matrix without ever using the
identity service - it exists only to map 3rd party IDs (e.g. email
addresses) to matrix IDs to aid user discovery**.
##### What are Synapse's platform requirements?
Synapse will use as much RAM as you give it in order to cache conversations in RAM to avoid hitting the database. For small deployments (<50 active users) around 512MB of RAM is probably okay. You can configure the amount of RAM used by synapse with the event_cache_size config parameter - the more events in the cache, the more RAM required. Synapse itself requires relatively little diskspace other than for logging (which as of October 2015 is quite verbose for debugging purposes), but as it caches the content of all the file attachments (images, videos etc) viewed by its users, you may need to size storage appropriately. Synapse is currently effectively single threaded, and will never use more than 1 core.
|
For better performance, one should back Synapse with a Postgres database rather than the default SQLite - see [https://github.com/matrix-org/synapse/tree/master/README.rst#using-postgresql](https://github.com/matrix-org/synapse/tree/master/README.rst#using-postgresql) for details.
##### Why is Synapse in Python/Twisted?
This is because both provide a mature and well known event-driven async IO framework for writing serverside code. Whilst this has been okay for our initial experimentation and proof of concept, it's likely that future homeserver work will be written in a more strongly typed language (e.g. Go).
##### Why aren't you using an ORM layer like SqlAlchemy in Synapse?
Synapse is *very* database dependent (as of Oct 2015; this is improving in the near future however), and we like having the flexibility to sculpt our own queries.
##### Will Synapse share my chat data with other servers in the federation?
Data is only shared between servers of participating users of a room. If all users in a room are on your server, no data is shared with other servers.
##### Why can't I rename my homeserver?
Currently, the homeserver name is assumed never to change. This means that if you rename your server, other servers will think it's a different server.
Perhaps in the future we will add an API for changing the homeserver name, but for now this is not supported.
### Clients
##### Where can I find a mobile app?
Riot is available for Android and iOS.
The iOS version can be downloaded from the [Apple store](https://itunes.apple.com/us/app/vector.im/id1083446067).
The Android version can be downloaded from the [Google Play store](https://play.google.com/store/apps/details?id=im.vector.alpha) or [F-Droid](https://f-droid.org/repository/browse/?fdid=im.vector.alpha). If you are not sure which one to choose, install Riot from the [Google Play store](https://play.google.com/store/apps/details?id=im.vector.alpha).
For the Android app, you can also install the latest development version
built by [Jenkins](http://matrix.org/jenkins/job/VectorAndroidDevelop). Use it at your own risk and only if you know what you are doing.
##### I installed Riot via F-Droid, why is it draining my battery?
The F-Droid release of Riot does not use [Google Cloud Messaging](https://developers.google.com/cloud-messaging/). This allows users that do not have or want Google Services installed to use Riot.
The drawback is that Riot has to pull for new messages, which can drain your battery. To counter this, you can change the delay between polls in the settings. Higher delay means better battery life (but may delay receiving messages). You can also disable the background sync entirely (which means that you won't get any notifications at all).
If you don't mind using Google Services, you might be better off installing the [Google Play store](https://play.google.com/store/apps/details?id=im.vector.alpha) version.
##### Where can I find a web app?
You can use [Riot.im](https://Riot.im) - a glossy web client written on top of [matrix-react-sdk](https://github.com/matrix-org/matrix-react-sdk).
You can also run Vector, the code that Riot.im uses, on your own server. It's a static web application, just download the [last release](https://github.com/vector-im/vector-web/) and unpack it.
##### Where can I find a desktop client?
You can use the desktop build of [Riot.im](https://riot.im/desktop.html).
There are also other desktop clients - check the list of clients on [matrix.org](http://matrix.org/docs/projects/try-matrix-now.html#clients).
##### Why can't end-to-end encryption be turned off?
When encryption is enabled in a room, a flag is set in the room state, so that
all clients know to encrypt any messages they send. The room state stores
information about the room like the topic, the avatar, and the membership list.
Imagine if encryption could be turned off the same way as it is turned
on. Anyone with admin rights in the room could clear the flag and then messages
would start being transmitted unencrypted. It would be very easy for a user to
miss the change in configuration, and accidentally send a sensitive message
without encryption.
Worse yet, anyone with sysadmin access to a server could also clear the flag
(remember that the main reason for using e2e encryption is that we don't trust
the sysadmins), and could then easily read any sensitive content which was
sent.
The solution we have taken for now is to make clients ignore any requests to
disable encryption. We might experiment with ways to improve this in the future
- for instance, by alerting the user next time they try to send a message in
the room if encryption has been disabled.
##### Why isn't end-to-end encryption enabled by default?
We are deliberately keeping E2E opt-in during the beta as there is a small risk of undecryptable messages, and we dont want to lull folks into a false sense of security. As soon as we are out of beta, we will turn E2E on for any room with private history by default. Another consideration is to give other clients a chance to catch up with E2E support before it's used by default.
|
### QUESTIONS TO BE ANSWERED!
This FAQ is a constant work in progress - patches and pull requests are *very* welcome to help us improve it. Some of the frequent questions where we need to write an answer include:
* How do I change the TLS key of my server?
* How do I maintain my synapse's DB (e.g. prune old conversations)?
* How do I maintain my synapse's content repository (e.g. prune old content)?
* Why is the spec so big, especially relative to the XMPP baseline spec?
* How do I contribute to the spec?
* What is the privacy policy on Matrix.org?
* How precisely does E2E work?
* How does Matrix actually work architecturally?
* What IOT use cases are there for Matrix?
* Why is are the Matrix reference implementations written in so many different languages?
* How does push work?
* What's on the roadmap?
* How can I use Matrix to talk on Freenode or other IRC networks?
* Where can I learn more about Matrix? (link to PDFs of other presentations etc)
* Why is synapse so resource intensive immediately after federating for the first time?
* \[your question goes here...\]
|
Any other questions? Please contact us in
[\#matrix:matrix.org](https://matrix.to/#/#matrix:matrix.org).

@ -1,146 +0,0 @@
---
layout: post
title: Application services
categories: guides
---
# Application services
Application services are distinct modules which which sit alongside a homeserver providing arbitrary extensible functionality decoupled from the homeserver implementation. Just like the rest of Matrix, they communicate via HTTP using JSON. Application services function in a very similar way to traditional clients, but they are given much more power than a normal client. They can reserve entire namespaces of room aliases and user IDs for their own purposes. They can silently monitor events in rooms, or any events directed at any user ID. This power allows application services to have extremely useful abilities which overall enhance the end user experience.
|
One of the main use cases for application services is protocol bridges. Our Matrix server on Matrix.org links in to various IRC channels and networks. This functionality was initially implemented as a simple bot which resided as a user on the Matrix rooms we wanted to link to freenode channels (#matrix, #matrix-dev, #openwebrtc and #vuc etc). There was nothing special about this bot; it is just treated as a client. However, as we started to rely on it more and more though, we realised that there were things that were impossible for simple client-side bots to do by themselves - for example, the bot could not reserve the virtual user IDs it wanted to create, and could not lazily bridge arbitrary IRC rooms on-the-fly - and this spurred the development of Application Services.
|
### Some of the features of the IRC application service we have since implemented include:
- Specific channel-to-matrix room bridging : This is what the original IRC bot did. You can specify specific channels and specific room IDs, and messages will be bridged.
- Dynamic channel-to-matrix room bridging : This allows Matrix users to join any channel on an IRC network, rather than being forced to use one of the specific channels configured.
- Two-way PM support : IRC users can PM the virtual "M-" users and private Matrix rooms will be created. Likewise, Matrix users can invite the virtual "@irc_Nick:domain" user IDs to a room and a PM to the IRC nick will be made.
- IRC nick changing support: Matrix users are no longer forced to use "M-" nicks and can change them by sending "!nick" messages directly to the bridge.
- Ident support: This allows usernames to be authenticated for virtual IRC clients, which means IRC bans can be targeted at the Matrix user rather than the entire application service.
|
### The use of the Application Services API means:
- The bot can reserve user IDs. This prevents humans from registering for @irc_... user IDs which would then clash with the operation of the bot.
- The bot can reserve room aliases. This prevents humans from register for #irc_... aliases which would then clash with the operation of the bot.
- The bot can trivially manage hundreds of users. Events are pushed to the application service directly. If you tried to do this as a client-side bot, you would need one event stream connection per virtual user.
- The bot can lazily create rooms on demand. This means Matrix users can join non-existent room aliases and have the application service quickly track an IRC channel and create a room with that alias, allowing the join request to succeed.
|
### Implementation details:
- Written in Node.js, designed to be run using <code>forever</code>.
- Built on the generic <a href="http://github.com/matrix-org/matrix-appservice-node">matrix-appservice-node</a> framework.
- Supports sending metrics in statsd format.
- Uses matrix-appservice-node to provide a standardised interface when writing application services, rather than an explicit web framework (though under the hood matrix-appservice-node is using Express).
|
At present, the IRC application service is in beta, and is being run on #matrix and #matrix-dev. If you want to give it a go, <a title="Matrix-IRC Application Service" href="https://github.com/matrix-org/matrix-appservice-irc">check it out on Github</a>!
|
# What Application services can do for you
Application services have enormous potential for creating new and exciting ways to transform and enhance the core Matrix protocol. For example, you could aggregate information from multiple rooms into a summary room, or create throwaway virtual user accounts to proxy messages for a fixed user ID on-the-fly. As you may expect, all of this power assumes a high degree of trust between application services and homeservers. Only homeserver admins can allow an application service to link up with their homeserver, and the application service is in no way federated to other homeservers. You can think of application services as additional logic on the homeserver itself, without messing around with the book-keeping that homeservers have to do. This makes adding useful functionality very easy.
|
### Example
The application service (AS) API itself uses webhooks to communicate from the homeserver to the AS:
- Room Alias Query API : The homeserver hits a URL on your application server to see if a room alias exists.
- User Query API : The homeserver hits a URL on your application server to see if a user ID exists.
- Push API : The homeserver hits a URL on your application server to notify you of new events for your users and rooms.
A very basic application service may want to log all messages in rooms which have an alias starting with "#logged_" (side note: logging won't work if these rooms are using end-to-end encryption).
Here's an example of a very basic application service using Python (with Flask and Requests) which logs room activity:
# app_service.py:
import json, requests  # we will use this later
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.route("/transactions/&lt;transaction&gt;", methods=["PUT"])
def on_receive_events(transaction):
events = request.get_json()["events"]
for event in events:
print "User: %s Room: %s" % (event["user_id"], event["room_id"])
print "Event Type: %s" % event["type"]
print "Content: %s" % event["content"]
return jsonify({})
if __name__ == "__main__":
app.run()
Set your new application service running on port 5000 with:
python app_service.py
The homeserver needs to know that the application service exists before it will send requests to it. This is done via a registration YAML file which is specified in Synapse's main config file e.g. <code>homeserver.yaml</code>. The server admin needs to add the application service registration configuration file as an entry to this file.
# homeserver.yaml
app_service_config_files:
- "/path/to/appservice/registration.yaml"
NB: Note the "-" at the start; this indicates a list element. The registration file <code>registration.yaml</code> should look like:
# registration.yaml
# An ID which is unique across all application services on your homeserver. This should never be changed once set.
id: "something-good"
# this is the base URL of the application service
url: "http://localhost:5000"
# This is the token that the AS should use as its access_token when using the Client-Server API
# This can be anything you want.
as_token: wfghWEGh3wgWHEf3478sHFWE
# This is the token that the HS will use when sending requests to the AS.
# This can be anything you want.
hs_token: ugw8243igya57aaABGFfgeyu
# this is the local part of the desired user ID for this AS (in this case @logging:localhost)
sender_localpart: logging
namespaces:
users: []
rooms: []
aliases:
- exclusive: false
regex: "#logged_.*"
**You will need to restart the homeserver after editing the config file before it will take effect.**
|
To test everything is working correctly, go ahead and explicitly create a room with the alias "#logged_test:localhost" and send a message into the room: the HS will relay the message to the AS by PUTing to /transactions/&lt;tid&gt; and you should see your AS print the event on the terminal. This will monitor any room which has an alias prefix of "#logged_", but it won't lazily create room aliases if they don't already exist. This means it will only log messages in the room you created before: #logged_test:localhost. Try joining the room "#logged_test2:localhost" without creating it, and it will fail. Let's fix that and add in lazy room creation:
@app.route("/rooms/&lt;alias&gt;")
def query_alias(alias):
alias_localpart = alias.split(":")[0][1:]
requests.post(
# NB: "TOKEN" is the as_token referred to in registration.yaml
"http://localhost:8008/_matrix/client/api/v1/createRoom?access_token=TOKEN",
json.dumps({
"room_alias_name": alias_localpart
}),
headers={"Content-Type":"application/json"}
)
return jsonify({})
This makes the application service lazily create a room with the requested alias whenever the HS queries the AS for the existence of that alias (when users try to join that room), allowing any room with the alias prefix #logged_ to be sent to the AS. Now try joining the room "#logged_test2:localhost" and it will work as you'd expect.  You can see that if this were a real bridge, the AS would have checked for the existence of #logged_test2 in the remote network, and then lazily-created it in Matrix as required.
|
Application services are powerful components which extend the functionality of homeservers, but they are limited. They can only ever function in a "passive" way. For example, you cannot implement an application service which censors swear words in rooms, because there is no way to prevent the event from being sent. Aside from the fact that censoring will not work when using end-to-end encryption, all federated homeservers would also need to reject the event in order to stop developing an inconsistent event graph. To "actively" monitor events, another component called a "Policy Server" is required, which is beyond the scope of this post.  Also, Application Services can result in a performance bottleneck, as all events on the homeserver must be ordered and sent to the registered application services.  If you are bridging huge amounts of traffic, you may be better off having your bridge directly talk the Server-Server federation API rather than the simpler Application Service API.
I hope this demonstrates how easy it is to create an application service, along with a few ideas of the kinds of things you can do with them. Obvious uses include build protocol bridges, search engines, invisible bots, etc. For more information on the AS HTTP API, check out the new <a href="http://matrix.org/docs/spec/#application-service-api">Application Service API</a> section in the spec, or the raw drafts and spec in <a href="https://github.com/matrix-org/matrix-doc/" target="_blank">https://github.com/matrix-org/matrix-doc/</a>.

@ -1,18 +0,0 @@
---
layout: default
categories: guides
---
<div class="home">
<h1>Guides</h1>
<p>Here is a collection of guides that might help you get involved with Matrix.</p>
<p>First, there is the <a href="./getting_involved.html" title="Getting Involved">Getting Involved</a> guide, which explains various ways of getting started with Matrix, and the <a href="./faq.html" title="FAQ">FAQ</a>, which tries to answer all your questions relating to Matrix.</p>
<p>The <a href="/docs/guides/client-server.html" title="Client-Server API">Client-Server API</a> guide explains in detail how to use the CS API, which is useful if you want to write a client (or modify an existing one) - or if you're just interested in how it works "under the hood".</p>
<p>If you were using the old v1 CS API, there is also the <a href="/docs/guides/client-server-migrating-from-v1.html">v1 migration guide</a> which justs lists the changes from v1 to r0.</p>
<p><a href="./lets-encrypt.html">Let's Encrypt Matrix</a> explains how to use Let's Encrypt's certificates with your Synapse installation. This guide was written by William A Stevens.</p>
<p>The <a href="./application_services.html" title="Application services">Application services</a> guide introduces and explains Application services, and what they can be used for.</p>
<p><a href="./types-of-bridging.html">Types of Bridging</a> should be read by all bridge developers to ensure everyone has the same mental map of terminology when implementing bridges.</p>
<p>The <a href="./e2e_implementation.html">End-to-end Encryption Implementation Guide</a> is intended for client developers who wish to add support for end-to-end encryption.</p>
</div>

@ -1,85 +0,0 @@
---
layout: post
version: v1.0
title: Code of Conduct
categories: guides
---
<link href="/docs/css/faq.css" type="text/css" rel="stylesheet" />
# Matrix Code of Conduct
This code of conduct outlines our expectations for participants within the Matrix community, as well as steps for reporting unacceptable behaviour. We are committed to providing a welcoming and inspiring community for all, and expect our code of conduct to be honoured. Anyone who violates this code of conduct may be banned from the community.
This applies to conversation in the #matrix* rooms (#matrix:matrix.org, #matrix-dev:matrix.org, #matrix-spam:matrix.org) and commits and comments relating to any project in the [matrix-org](https://github.com/matrix-org) github space.
Our open source community strives to:
* **Be friendly and patient.**
* **Be welcoming**: We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, colour, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.
* **Be considerate**: Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions. Remember that we're a world-wide community, so you might not be communicating in someone else's primary language.
* **Be respectful**: Not all of us will agree all the time, but disagreement is no excuse for poor behaviour and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. Its important to remember that a community where people feel uncomfortable or threatened is not a productive one.
* **Be careful in the words that we choose**: Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behaviour aren't acceptable.
* **Try to understand why we disagree**: Disagreements, both social and technical, happen all the time. It is important that we resolve disagreements and differing views constructively. Remember that were different. The strength of our community comes from its diversity, people from a wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesnt mean that theyre wrong. Dont forget that it is human to err and blaming each other doesnt get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.
|
## Definitions
Harassment includes, but is not limited to:
- Offensive comments related to gender, gender identity and expression, sexual orientation, disability, mental illness, neuro(a)typicality, physical appearance, body size, race, age, regional discrimination, political or religious affiliation
- Unwelcome comments regarding a persons lifestyle choices and practices, including those related to food, health, parenting, drugs, and employment
- Deliberate misgendering. This includes deadnaming or persistently using a pronoun that does not correctly reflect a person's gender identity. You must address people by the name they give you when not addressing them by their username or handle
- Physical contact and simulated physical contact (eg, textual descriptions like “*hug*” or “*backrub*”) without consent or after a request to stop
- Threats of violence, both physical and psychological
- Incitement of violence towards any individual, including encouraging a person to commit suicide or to engage in self-harm
- Deliberate intimidation
- Stalking or following
- Harassing photography or recording, including logging online activity for harassment purposes
- Sustained disruption of discussion
- Unwelcome sexual attention, including gratuitous or off-topic sexual images or behaviour
- Pattern of inappropriate social contact, such as requesting/assuming inappropriate levels of intimacy with others
- Continued one-on-one communication after requests to cease
- Deliberate “outing” of any aspect of a persons identity without their consent except as necessary to protect others from intentional abuse
- Publication of non-harassing private communication
|
We will not act on complaints regarding:
- Good faith and non-malicious conduct whose object is to ameliorate the conditions of disadvantaged individuals or groups including those that are disadvantaged because of race, national or ethnic origin, colour, religion, sex, age or mental or physical disability.
- Reasonable communication of boundaries, such as “leave me alone,” “go away,” or “Im not discussing this with you”
- Refusal to explain or debate social justice concepts
- Communicating in a tone you dont find congenial
- Criticizing racist, sexist, cissexist, or otherwise oppressive behaviour or assumptions
|
### Diversity Statement
We encourage everyone to participate and are committed to building a community for all. Although we will fail at times, we seek to treat everyone both as fairly and equally as possible. Whenever a participant has made a mistake, we expect them to take responsibility for it. If someone has been harmed or offended, it is our responsibility to listen carefully and respectfully, and do our best to right the wrong.
Although this list cannot be exhaustive, we explicitly honour diversity in age, gender, gender identity or expression, culture, ethnicity, language, national origin, political beliefs, profession, race, religion, sexual orientation, socioeconomic status, and technical ability. We will not tolerate discrimination based on any of the protected
characteristics above, including participants with disabilities.
|
### Reporting Issues
If you experience or witness unacceptable behaviour — or have any other concerns — please report it by contacting us via abuse@matrix.org. All reports will be handled with discretion. In your report please include:
- Your contact information.
- Names (usernames and nicks, real names, and/or pseudonyms) of any individuals involved. If there are additional witnesses, please
include them as well. Your account of what occurred, and if you believe the incident is ongoing.
- The date and time of the incident (or start of incident).
- Any additional information that may be helpful.
After filing a report, a representative will contact you personally, review the incident, follow up with any additional questions, and make a decision as to how to respond. If the person who is harassing you is part of the response team, they will recuse themselves from handling your incident. If the complaint originates from a member of the response team, it will be handled by a different member of the response team. We will respect confidentiality requests for the purpose of protecting victims of abuse.
|
### Attribution & Acknowledgements
This Code of Conduct is based on the [TODO Group](https://twitter.com/todogroup)'s [Open Code of Conduct template](https://github.com/todogroup/opencodeofconduct), but with some modifications.

@ -1,37 +0,0 @@
---
layout: post
title: Let's Encrypt Matrix
categories: guides
---
====================
Let's Encrypt Matrix
====================
Let's Encrypt is a free Certificate Authority that makes it easy to secure your server's internet traffic. This makes it really easy to secure your Matrix homeserver, and this guide will explain exactly how you do this. Guide written by William A Stevens - thanks!
0: Prerequisites
================
* Install Synapse_.
* Install (or Download) certbot_
1: Get certificates
===================
When executing the Let's Encrypt client, it will ask for the domain name of your server, and your email address. The domain list can include multiple names and should include any domain you want to access the server from.
Also, the certificates will be in a folder under /etc/letsencrypt (see below) and owned by root.
::
# certbot certonly --standalone
A note about renewal
--------------------
These certificates will expire in 3 months. To renew certificates, run ```certbot renew```. It is recommended to create a cronjob, which attempts renewal twice a day. Depending on your distribution, that could be already configured.
2: Install Certificates
=======================
At the top of your homeserver.yaml there should be two keys, ```tls_certificate_path``` and ```tls_private_key_path```. These should be changed so that instead of pointing to the default keys, they now point to the Let's Encrypt keys. ```tls_certificate_path``` should point to ```/etc/letsencrypt/live/(your domain name)/fullchain.pem```. ```tls_private_key_path``` should point to ```/etc/letsencrypt/live/(your domain name)/privkey.pem```. ```tls_dh_params_path``` can stay the same as before.
.. _Synapse: https://github.com/matrix-org/synapse/blob/master/README.rst#synapse-installation
.. _certbot: https://certbot.eff.org/

@ -1,114 +0,0 @@
---
layout: post
title: Migrating from Client Server API v1
categories: guides
---
Migrating from client-server API v1
===================================
This guide assists developers of API clients that target the ``v1`` version of
the API to migrate their code to the later ``r0``. It does not aim to introduce
new concepts that were added in ``r0`` except where these replace things that
were removed since ``v1``.
Updated Version In Path
=======================
The new version of the API is ``r0``; this should be used in paths where
``v1`` used to appear. Additionally, the ``/api`` path component has now been
removed. API endpoint paths are now::
POST /_matrix/client/r0/register
GET /_matrix/client/r0/login
etc...
New Registration and Login Endpoints
====================================
The ``r0`` version of the ``/register`` and ``/login`` endpoints is different
to the ``v1`` version. See the updated API documentation for details on how the
new API works. In brief, the changes are that the new version returns extra
information in the form of the ``params`` object, and that a sequence of
multiple calls may be statefully chained together by the ``session`` parameter.
Additionally, whereas in ``v1`` the client performed a ``GET`` request to
discover the list of supported flows for ``/register``, in ``r0`` this is done
by sending a ``POST`` request with an empty data body. The ``/login`` endpoint
continues to use the ``GET`` method as before.
Deprecated Endpoints
====================
The following endpoints are now deprecated and replaced by the ``/sync`` API::
/initialSync
/events
/rooms/:roomId/initialSync
The new ``/sync`` API takes an optional ``since`` parameter to distinguish the
initial sync from subsequent updates for more events.
The return value takes a different structure to that from the previous
``/initialSync`` API. For full details see the API documentation, but the
following summary may be useful to compare with ``v1``:
* ``/initialSync`` returned a ``state`` key containing the most recent state
in the room, whereas the new ``/sync`` API's ``state`` corresponds to the
room state at the start of the returned timeline. This makes it easier for
clients to represent state changes that occur within the region of returned
timeline.
* In ``/events``, if more events occurred since the ``since`` token than the
``limit`` parameter allowed, then events from the start of this range were
returned and the client had to perform another fetch to incrementally obtain
more of them. In the ``/sync`` API the result always contains the most
recent events, creating a gap if this would be more events than the
requested limit. If this occurs then the client can use the ``prev_batch``
token as a reference to obtaining more.
* The ``state`` contained in the response to a ``/sync`` request that has a
``since`` parameter will contain only keys that have changed since the
basis given in the ``since`` parameter, rather than containing a full set
values.
The ``/initialSync`` API allowed a parameter called ``limit`` to limit the
number of events returned. To apply this limit to the new ``/sync`` API, you
can supply an ad-hoc filter::
GET .../sync?filter={"room":{"timeline":{"limit:$limit}}}
There is no direct replacement for the per-room ``/rooms/:roomId/initialSync``
endpoint, but the behaviour can be recreated by applying an ad-hoc filter using
the ``filter`` parameter to ``/sync`` that selects only the required room ID::
GET .../sync?filter={"room":{"rooms":[$room_id]}}
However, the way that the new ``/sync`` API works should remove any need to do
this kind of query, in the situations where the ``v1`` API needed it.
Specifically, on joining a new room the initial information about that room is
sent in the next ``/sync`` batch, so it should not be necessary to query that
one room specially.
The following endpoint is deprecated and has no direct replacement::
/events/:eventId
However, if the client knows the room ID of the room that the event is in, it
can use the ``/rooms/:roomId/context/:eventId`` request to fetch the event
itself. By giving the ``limit`` parameter of ``0`` the client can save using
extra bandwidth by actually returning additional context events around the
requested one.
Removed POST Endpoint
=====================
The room message sending API endpoint in ``v1`` accepted both ``PUT`` and
``POST`` methods, where the client could specify a message ID in the ``PUT``
path for de-duplication purposes, or have the server allocate one during
``POST``. In ``r0`` this latter form no longer exists. Clients will now have
to generate these IDs locally.
The following URLs have therefore been removed::
POST .../rooms/:roomId/send/:messageType

@ -1,394 +0,0 @@
---
layout: post
title: Client Server API
categories: guides
---
.. TODO kegan
Room config (specifically: message history,
public rooms).
How to use the client-server API
================================
.. NOTE::
The git version of this document is ``{% project_version %}``
This guide focuses on how the client-server APIs *provided by the reference
homeserver* can be used. Since this is specific to a homeserver
implementation, there may be variations in relation to registering/logging in
which are not covered in extensive detail in this guide.
If you haven't already, get a homeserver up and running on
``https://localhost:8448``.
Accounts
========
Before you can send and receive messages, you must **register** for an account.
If you already have an account, you must **login** into it.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/register_login
Registration
------------
The aim of registration is to get a user ID and access token which you will need
when accessing other APIs::
curl -XPOST -d '{"username":"example", "password":"wordpass", "auth": {"type":"m.login.dummy"}}' "https://localhost:8448/_matrix/client/r0/register"
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.AqdSzFmFYrLrTmteXc",
"home_server": "localhost",
"user_id": "@example:localhost"
}
NB: If a ``user`` is not specified, one will be randomly generated for you.
If you do not specify a ``password``, you will be unable to login to the account
if you forget the ``access_token``.
Implementation note: The matrix specification does not enforce how users
register with a server. It just specifies the URL path and absolute minimum
keys. The reference homeserver uses a username/password to authenticate user,
but other homeservers may use different methods. This is why you need to
specify the ``type`` of method.
Login
-----
The aim when logging in is to get an access token for your existing user ID::
curl -XGET "https://localhost:8448/_matrix/client/r0/login"
{
"flows": [
{
"type": "m.login.password"
}
]
}
curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "https://localhost:8448/_matrix/client/r0/login"
{
"access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd",
"home_server": "localhost",
"user_id": "@example:localhost"
}
Implementation note: Different homeservers may implement different methods for
logging in to an existing account. In order to check that you know how to login
to this homeserver, you must perform a ``GET`` first and make sure you
recognise the login type. If you do not know how to login, you can
``GET /login/fallback`` which will return a basic webpage which you can use to
login. The reference homeserver implementation support username/password login,
but other homeservers may support different login methods (e.g. OAuth2).
Communicating
=============
In order to communicate with another user, you must **create a room** with that
user and **send a message** to that room.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/create_room_send_msg
Creating a room
---------------
If you want to send a message to someone, you have to be in a room with them. To
create a room::
curl -XPOST -d '{"room_alias_name":"tutorial"}' "https://localhost:8448/_matrix/client/r0/createRoom?access_token=YOUR_ACCESS_TOKEN"
{
"room_alias": "#tutorial:localhost",
"room_id": "!asfLdzLnOdGRkdPZWu:localhost"
}
The "room alias" is a human-readable string which can be shared with other users
so they can join a room, rather than the room ID which is a randomly generated
string. You can have multiple room aliases per room.
.. TODO(kegan)
How to add/remove aliases from an existing room.
Sending messages
----------------
You can now send messages to this room::
curl -XPOST -d '{"msgtype":"m.text", "body":"hello"}' "https://localhost:8448/_matrix/client/r0/rooms/%21asfLdzLnOdGRkdPZWu:localhost/send/m.room.message?access_token=YOUR_ACCESS_TOKEN"
{
"event_id": "YUwRidLecu"
}
The event ID returned is a unique ID which identifies this message.
NB: There are no limitations to the types of messages which can be exchanged.
The only requirement is that ``"msgtype"`` is specified. The Matrix
specification outlines the following standard types: ``m.text``, ``m.image``,
``m.audio``, ``m.video``, ``m.location``, ``m.emote``. See the specification for
more information on these types.
Users and rooms
===============
Each room can be configured to allow or disallow certain rules. In particular,
these rules may specify if you require an **invitation** from someone already in
the room in order to **join the room**. In addition, you may also be able to
join a room **via a room alias** if one was set up.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/room_memberships
Inviting a user to a room
-------------------------
You can directly invite a user to a room like so::
curl -XPOST -d '{"user_id":"@myfriend:localhost"}' "https://localhost:8448/_matrix/client/r0/rooms/%21asfLdzLnOdGRkdPZWu:localhost/invite?access_token=YOUR_ACCESS_TOKEN"
This informs ``@myfriend:localhost`` of the room ID
``!CvcvRuDYDzTOzfKKgh:localhost`` and allows them to join the room.
Joining a room via an invite
----------------------------
If you receive an invite, you can join the room::
curl -XPOST -d '{}' "https://localhost:8448/_matrix/client/r0/rooms/%21asfLdzLnOdGRkdPZWu:localhost/join?access_token=YOUR_ACCESS_TOKEN"
NB: Only the person invited (``@myfriend:localhost``) can change the membership
state to ``"join"``. Repeatedly joining a room does nothing.
Joining a room via an alias
---------------------------
Alternatively, if you know the room alias for this room and the room config
allows it, you can directly join a room via the alias::
curl -XPOST -d '{}' "https://localhost:8448/_matrix/client/r0/join/%21asfLdzLnOdGRkdPZWu:localhost?access_token=YOUR_ACCESS_TOKEN"
{
"room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
}
You will need to use the room ID when sending messages, not the room alias.
NB: If the room is configured to be an invite-only room, you will still require
an invite in order to join the room even though you know the room alias. As a
result, it is more common to see a room alias in relation to a public room,
which do not require invitations.
Getting events
==============
An event is some interesting piece of data that a client may be interested in.
It can be a message in a room, a room invite, etc. There are many different ways
of getting events, depending on what the client already knows.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/event_stream
Getting all state
-----------------
If the client doesn't know any information on the rooms the user is
invited/joined on, they can get all the user's state for all rooms::
curl -XGET "https://localhost:8448/_matrix/client/r0/sync?access_token=YOUR_ACCESS_TOKEN"
{
"account_data": {
"events": [
{
...
}
]
},
"next_batch": "s9_3_0_1_1_1",
"presence": {
"events": [
{
"content": {
"currently_active": true,
"last_active_ago": 19,
"presence": "online"
},
"sender": "@example:localhost",
"type": "m.presence"
}
]
},
"rooms": {
"invite": {},
"join": {
"!asfLdzLnOdGRkdPZWu:localhost": {
"account_data": {
"events": []
},
"ephemeral": {
"events": []
},
"state": {
"events": []
},
"timeline": {
"events": [
{
"content": {
"creator": "@example:localhost"
},
"event_id": "$14606534990LhqHt:localhost",
"origin_server_ts": 1460653499699,
"sender": "@example:localhost",
"state_key": "",
"type": "m.room.create",
"unsigned": {
"age": 239192
}
},
{
"content": {
"avatar_url": null,
"displayname": null,
"membership": "join"
},
"event_id": "$14606534991nsZKk:localhost",
"membership": "join",
"origin_server_ts": 1460653499727,
"sender": "@example:localhost",
"state_key": "@example:localhost",
"type": "m.room.member",
"unsigned": {
"age": 239164
}
},
...
],
"limited": false,
"prev_batch": "s9_3_0_1_1_1"
},
"unread_notifications": {}
}
},
"leave": {}
}
}
This returns all the room information the user is invited/joined on, as well as
all of the presences relevant for these rooms. This can be a LOT of data. You
may just want the most recent event for each room. This can be achieved by
applying a filter that asks for a limit of 1 timeline event per room::
curl --globoff -XGET "https://localhost:8448/_matrix/client/r0/sync?filter={'room':{'timeline':{'limit':1}}}&access_token=YOUR_ACCESS_TOKEN"
{
...
"rooms": {
"invite": {},
"join": {
"!asfLdzLnOdGRkdPZWu:localhost": {
...
"timeline": {
"events": [
{
"content": {
"body": "hello",
"msgtype": "m.text"
},
"event_id": "$14606535757KCGXo:localhost",
"origin_server_ts": 1460653575105,
"sender": "@example:localhost",
"type": "m.room.message",
"unsigned": {
"age": 800348
}
}
],
"limited": true,
"prev_batch": "t8-8_7_0_1_1_1"
},
"unread_notifications": {}
}
},
"leave": {}
}
}
(additionally we have to ask ``curl`` not to try to interpret any ``{}``
characters in the URL, which is what the ``--globoff`` option is for)
Getting live state
------------------
In the response to this ``sync`` request the server includes a token that can
be used to obtain updates since this point under the object key ``next_batch``.
To use this token, specify its value as the ``since`` parameter to another
``/sync`` request.::
curl -XGET "https://localhost:8448/_matrix/client/r0/sync?since=s9_7_0_1_1_1&access_token=YOUR_ACCESS_TOKEN"
{
"account_data": {
"events": []
},
"next_batch": "s9_9_0_1_1_1",
"presence": {
"events": [
{
"content": {
"currently_active": true,
"last_active_ago": 12,
"presence": "online"
},
"sender": "@example:localhost",
"type": "m.presence"
}
]
},
"rooms": {
"invite": {},
"join": {},
"leave": {}
}
}
By default this request will not wait in the server, always returning a value
even if nothing interesting happened. However, by applying the ``timeout``
query parameter, which gives a duration in miliseconds, we can ask the server
to wait for up to that amount of time before it returns. If no interesting
events have happened since then, the response will be relatively empty.::
curl -XGET "https://localhost:8448/_matrix/client/r0/sync?since=s9_13_0_1_1_1&access_token=YOUR_ACCESS_TOKEN"
{
"account_data": {
"events": []
},
"next_batch": "s9_13_0_1_1_1",
"presence": {
"events": []
},
"rooms": {
"invite": {},
"join": {},
"leave": {}
}
}
Example application
-------------------
The following example demonstrates registration and login, live event streaming,
creating and joining rooms, sending messages, getting member lists and getting
historical messages for a room. This covers most functionality of a messaging
application.
.. NOTE::
`Try out the fiddle`__
.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/matrix-doc/tree/master/supporting-docs/howtos/jsfiddles/example_app

@ -1,782 +0,0 @@
---
layout: post
title: End-to-End Encryption implementation guide
categories: guides
---
Implementing End-to-End Encryption in Matrix clients
====================================================
This guide is intended for authors of Matrix clients who wish to add
support for end-to-end encryption. It is highly recommended that readers
be familiar with the Matrix protocol and the use of access tokens before
proceeding.
.. contents::
The libolm library
------------------
End-to-end encryption in Matrix is based on the Olm and Megolm
cryptographic ratchets. The recommended starting point for any client
authors is with the `libolm <http://matrix.org/git/olm>`__ library,
which contains implementations of all of the cryptographic primitives
required. The library itself is written in C/C++, but is architected in
a way which makes it easy to write wrappers for higher-level languages.
Devices
-------
We have a particular meaning for “device”. As a user, I might have
several devices (a desktop client, some web browsers, an Android device,
an iPhone, etc). When I first use a client, it should register itself as
a new device. If I log out and log in again as a different user, the
client must register as a new device. Critically, the client must create
a new set of keys (see below) for each “device”.
The longevity of devices will depend on the client. In the web client,
we create a new device every single time you log in. In a mobile client,
it might be acceptable to reuse the device if a login session expires,
**provided** the user is the same. **Never** share keys between
different users.
Devices are identified by their ``device_id`` (which is unique within
the scope of a given user). By default, the ``/login`` and ``/register``
endpoints will auto-generate a ``device_id`` and return it in the
response; a client is also free to generate its own ``device_id`` or, as
above, reuse a device, in which case the client should pass the
``device_id`` in the request body.
The lifetime of devices and ``access_token``\ s are closely related. In
the simple case where a new device is created each time you log in,
there is a one-to-one mapping between a ``device_id`` and an
``access_token``. If a client reuses a ``device_id`` when logging
in, there will be several ``access_token``\ s associated with a
given ``device_id`` - but still, we would expect only one of these to be
active at once (though we do not currently enforce that in Synapse).
Keys used in End-to-End encryption
----------------------------------
There are a number of keys involved in encrypted communication: a
summary of them follows.
Ed25519 fingerprint key pair
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ed25519 is a public-key cryptographic system for signing messages. In
Matrix, each device has an Ed25519 key pair which serves to identify
that device. The private part of the key pair should never leave the
device, but the public part is published to the Matrix network.
Curve25519 identity key pair
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Curve25519 is a public-key cryptographic system which can be used to
establish a shared secret. In Matrix, each device has a long-lived
Curve25519 identity key which is used to establish Olm sessions with
that device. Again, the private key should never leave the device, but
the public part is signed with the Ed25519 fingerprint key and published
to the network.
Theoretically we should rotate the Curve25519 identity key from time to
time, but we haven't implemented this yet.
Curve25519 one-time keys
~~~~~~~~~~~~~~~~~~~~~~~~
As well as the identity key, each device creates a number of Curve25519
key pairs which are also used to establish Olm sessions, but can only be
used once. Once again, the private part remains on the device.
At startup, Alice creates a number of one-time key pairs, and publishes
them to her homeserver. If Bob wants to establish an Olm session with
Alice, he needs to claim one of Alices one-time keys, and creates a new
one of his own. Those two keys, along with Alices and Bobs identity
keys, are used in establishing an Olm session between Alice and Bob.
Megolm encryption keys
~~~~~~~~~~~~~~~~~~~~~~
The Megolm key is used to encrypt group messages (in fact it is used to
derive an AES-256 key, and an HMAC-SHA-256 key). It is initialised with
random data. Each time a message is sent, a hash calculation is done on
the Megolm key to derive the key for the next message. It is therefore
possible to share the current state of the Megolm key with a user,
allowing them to decrypt future messages but not past messages.
Ed25519 Megolm signing key pair
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When a sender creates a Megolm session, he also creates another Ed25519
signing key pair. This is used to sign messages sent via that Megolm
session, to authenticate the sender. Once again, the private part of the
key remains on the device. The public part is shared with other devices
in the room alongside the encryption key.
Creating and registering device keys
------------------------------------
This process only happens once, when a device first starts.
It must create the Ed25519 fingerprint key pair and the Curve25519
identity key pair. This is done by calling ``olm_create_account`` in
libolm. The (base64-encoded) keys are retrieved by calling
``olm_account_identity_keys``. The account should be stored for future
use.
It should then publish these keys to the homeserver. To do this, it
should construct a JSON object as follows:
.. code:: json
{
"algorithms": ["m.olm.v1.curve25519-aes-sha2", "m.megolm.v1.aes-sha2"],
"device_id": "<device_id>",
"keys": {
"curve25519:<device_id>": "<curve25519_key>",
"ed25519:<device_id>": "<ed25519_key>"
},
"user_id: <user_id>"
}
The object should be formatted as `Canonical
JSON <http://matrix.org/docs/spec/server_server/unstable.html#canonical-json>`__,
then signed with ``olm_account_sign``; the signature should be added to
the JSON as ``signatures.<user_id>.ed25519:<device_id>``.
The signed JSON is then uploaded via
``POST /_matrix/client/unstable/keys/upload``.
Creating and registering one-time keys
--------------------------------------
At first start, and at regular intervals
thereafter\ [#]_, the client should check how
many one-time keys the homeserver has stored for it, and, if necessary,
generate and upload some more.
.. [#] Every 10 minutes is suggested.
The number of one-time keys currently stored is returned by
``POST /_matrix/client/unstable/keys/upload``. (Post an empty JSON object
``{}`` if you dont want to upload the device keys.)
The maximum number of active keys supported by libolm is returned by
``olm_account_max_number_of_one_time_keys``. The client should try to
maintain about half this number on the homeserver.
To generate new one-time keys:
* Call ``olm_account_generate_one_time_keys`` to generate new keys.
* Call ``olm_account_one_time_keys`` to retrieve the unpublished keys. This
returns a JSON-formatted object with the single property ``curve25519``,
which is itself an object mapping key id to base64-encoded Curve25519
key. For example:
.. code:: json
{
"curve25519": {
"AAAAAA": "wo76WcYtb0Vk/pBOdmduiGJ0wIEjW4IBMbbQn7aSnTo",
"AAAAAB": "LRvjo46L1X2vx69sS9QNFD29HWulxrmW11Up5AfAjgU"
}
}
* Each key should be signed with the account key. To do this:
* Construct a JSON object as follows:
.. code:: json
{
"key": "<curve25519_key>"
}
* Call ``olm_account_sign`` to calculate the signature.
* Add the signature should be added to the JSON as
``signatures.<user_id>.ed25519:<device_id>``.
* The complete key object should now look like:
.. code:: json
{
"key": "wo76WcYtb0Vk/pBOdmduiGJ0wIEjW4IBMbbQn7aSnTo",
"signatures": {
"@alice:example.com": {
"ed25519:JLAFKJWSCS": "dSO80A01XiigH3uBiDVx/EjzaoycHcjq9lfQX0uWsqxl2giMIiSPR8a4d291W1ihKJL/a+myXS367WT6NAIcBA"
}
}
}
* Aggregate all the signed one-time keys into a single JSON object as follows:
.. code:: json
{
"one_time_keys": {
"signed_curve25519:<key_id>": {
"key": "<curve25519_key>",
"signatures": {
"<user_id>": {
"ed25519:<device_id>": "<signature>"
}
}
},
"signed_curve25519:<key_id>": {
...
},
...
}
}
* Upload the object via ``POST /_matrix/client/unstable/keys/upload``.
* Call ``olm_account_mark_keys_as_published`` to tell the olm library not to
return the same keys from a future call to ``olm_account_one_time_keys``.
Configuring a room to use encryption
------------------------------------
To enable encryption in a room, a client should send a state event of
type ``m.room.encryption``, and content ``{ "algorithm":
"m.megolm.v1.aes-sha2" }``.
.. |m.room.encryption| replace:: ``m.room.encryption``
.. _`m.room.encryption`:
Handling an ``m.room.encryption`` state event
---------------------------------------------
When a client receives an ``m.room.encryption`` event as above, it
should set a flag to indicate that messages sent in the room should be
encrypted.
This flag should **not** be cleared if a later ``m.room.encryption``
event changes the configuration. This is to avoid a situation where a
MITM can simply ask participants to disable encryption. In short: once
encryption is enabled in a room, it can never be disabled.
The event should contain an ``algorithm`` property which defines which
encryption algorithm should be used for encryption. Currently only
``m.megolm.v1-aes-sha2`` is permitted here.
The event may also include other settings for how messages sent in the room
should be encrypted (for example, ``rotation_period_ms`` to define how often
the session should be replaced).
Handling an ``m.room.encrypted`` event
--------------------------------------
Encrypted events have a type of ``m.room.encrypted``. They have a
content property ``algorithm`` which gives the encryption algorithm in
use, as well as other properties specific to the algorithm.
The encrypted payload is a JSON object with the properties ``type``
(giving the decrypted event type), and ``content`` (giving the decrypted
content). Depending on the algorithm in use, the payload may contain
additional keys.
There are currently two defined algorithms:
``m.olm.v1.curve25519-aes-sha2``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Encrypted events using this algorithm should have a ``sender_key`` and a
``ciphertext`` property.
The ``sender_key`` property of the event content gives the Curve25519
identity key of the sender. Clients should maintain a list of known Olm
sessions for each device they speak to; it is recommended to index them
by Curve25519 identity key.
Olm messages are encrypted separately for each recipient device.
``ciphertext`` is an object mapping from the Curve25519 identity key for
the recipient device. The receiving client should, of course, look for
its own identity key in this object. (If it isn't listed, the message
wasn't sent for it, and the client can't decrypt it; it should show an
error instead, or similar).
This should result in an object with the properties ``type`` and
``body``. Messages of type '0' are 'prekey' messages which are used to
establish a new Olm session between two devices; type '1' are normal
messages which are used once a message has been received on the session.
When a message (of either type) is received, a client should first
attempt to decrypt it with each of the known sessions for that sender.
There are two steps to this:
- If (and only if) ``type==0``, the client should call
``olm_matches_inbound_session`` with the session and ``body``. This
returns a flag indicating whether the message was encrypted using
that session.
- The client calls ``olm_decrypt``, with the session, ``type``, and
``body``. If this is successful, it returns the plaintext of the
event.
If the client was unable to decrypt the message using any known sessions
(or if there are no known sessions yet), **and** the message had type 0,
**and** ``olm_matches_inbound_session`` wasn't true for any existing
sessions, then the client can try establishing a new session. This is
done as follows:
- Call ``olm_create_inbound_session_from`` using the olm account, and
the ``sender_key`` and ``body`` of the message.
- If the session was established successfully:
- call ``olm_remove_one_time_keys`` to ensure that the same
one-time-key cannot be reused.
- Call ``olm_decrypt`` with the new session
- Store the session for future use
At the end of this, the client will hopefully have successfully
decrypted the payload.
As well as the ``type`` and ``content`` properties, the payload should
contain a number of other properties. Each of these should be checked as
follows [#]_.
``sender``
The user ID of the sender. The client should check that this matches the
``sender`` in the event.
``recipient``
The user ID of the recipient. The client should check that this matches the
local user ID.
``keys``
an object with a property ``ed25519``, The client should check that the
value of this property matches the sender's fingerprint key when `marking
the event as verified`_\ .
``recipient_keys``
an object with a property ``ed25519``. The client should check that the
value of this property matches its own fingerprint key.
.. [#] These tests prevent an attacker publishing someone else's curve25519
keys as their own and subsequently claiming to have sent messages which they
didn't.
``m.megolm.v1.aes-sha2``
~~~~~~~~~~~~~~~~~~~~~~~~
Encrypted events using this algorithm should have ``sender_key``,
``session_id`` and ``ciphertext`` content properties. If the
``room_id``, ``sender_key`` and ``session_id`` correspond to a known
Megolm session (see `below`__), the ciphertext can be
decrypted by passing the ciphertext into ``olm_group_decrypt``.
__ `m.room_key`_
In order to avoid replay attacks a client should remember the megolm
``message_index`` returned by ``olm_group_decrypt`` of each event they decrypt
for each session. If the client decrypts an event with the same
``message_index`` as one that it has already received using that session then
it should treat the message as invalid.
The client should check that the sender's fingerprint key matches the
``keys.ed25519`` property of the event which established the Megolm session
when `marking the event as verified`_.
.. _`m.room_key`:
Handling an ``m.room_key`` event
--------------------------------
These events contain key data to allow decryption of other messages.
They are sent to specific devices, so they appear in the ``to_device``
section of the response to ``GET /_matrix/client/r0/sync``. They will
also be encrypted, so will need decrypting as above before they can be
seen. (These events are generated by other clients - see `starting a megolm
session`_).
The event content will contain an 'algorithm' property, indicating the
encryption algorithm the key data is to be used for. Currently, this
will always be ``m.megolm.v1.aes-sha2``.
Room key events for Megolm will also have ``room_id``, ``session_id``, and
``session_key`` keys. They are used to establish a Megolm session. The
``room_id`` identifies which room the session will be used in. The ``room_id``,
together with the ``sender_key`` of the ``room_key`` event before it was
decrypted, and the ``session_id``, uniquely identify a Megolm session. If they
do not represent a known session, the client should start a new inbound Megolm
session by calling ``olm_init_inbound_group_session`` with the ``session_key``.
The client should remember the value of the keys property of the payload
of the encrypted ``m.room_key`` event and store it with the inbound
session. This is used as above when marking the event as verified.
.. _`download the device list`:
Downloading the device list for users in the room
-------------------------------------------------
Before an encrypted message can be sent, it is necessary to retrieve the
list of devices for each user in the room. This can be done proactively,
or deferred until the first message is sent. The information is also
required to allow users to `verify or block devices`__.
__ `blocking`_
The client should build a JSON query object as follows:
.. code:: json
{
"device_keys": {
"<user_id>": {},
[...]
}
}
Each member in the room should be included in the query. This is then
sent via ``POST /_matrix/client/unstable/keys/query.``
The result includes, for each listed user id, a map from device ID to an
object containing information on the device, as follows:
.. code:: json
{
"algorithms": [...],
"device_id": "<device_id>",
"keys": {
"curve25519:<device_id>": "<curve25519_key>",
"ed25519:<device_id>": "<ed25519_key>"
},
"signatures": {
"<userId>": {
"ed25519:<device_id>": "<signature>"
},
},
"unsigned": {
"device_display_name": "<display name>"
},
"user_id: <user_id>"
}
The client should first check the signature on this object. To do this,
it should remove the ``signatures`` and ``unsigned`` properties, format
the remainder as Canonical JSON, and pass the result into
``olm_ed25519_verify``, using the Ed25519 key for the ``key`` parameter,
and the corresponding signature for the ``signature`` parameter. If the
signature check fails, no further processing should be done on the
device.
The client must also check that the ``user_id`` and ``device_id`` fields in the
object match those in the top-level map [#]_.
The client should check if the ``user_id``/``device_id`` correspond to a device
it had seen previously. If it did, the client **must** check that the Ed25519
key hasn't changed. Again, if it has changed, no further processing should be
done on the device.
Otherwise the client stores the information about this device.
.. [#] This prevents a malicious or compromised homeserver replacing the keys
for the device with those of another.
Sending an encrypted event
--------------------------
When sending a message in a room `configured to use encryption`__, a client
first checks to see if it has an active outbound Megolm session. If not, it
first `creates one as per below`__. If an outbound session exists, it should
check if it is time to `rotate`__ it, and create a new one if so.
__ `Configuring a room to use encryption`_
__ `Starting a Megolm session`_
__ `Rotating Megolm sessions`_
The client then builds an encryption payload as follows:
.. code:: json
{
"type": "<event type>",
"content": "<event content>",
"room_id": "<id of destination room>"
}
and calls ``olm_group_encrypt`` to encrypt the payload. This is then packaged
into event content as follows:
.. code:: json
{
"algorithm": "m.megolm.v1.aes-sha2",
"sender_key": "<our curve25519 device key>",
"ciphertext": "<encrypted payload>",
"session_id": "<outbound group session id>",
"device_id": "<our device ID>"
}
Finally, the encrypted event is sent to the room with ``POST
/_matrix/client/r0/rooms/<room_id>/send/m.room.encrypted/<txn_id>``.
Starting a Megolm session
~~~~~~~~~~~~~~~~~~~~~~~~~
When a message is first sent in an encrypted room, the client should
start a new outbound Megolm session. This should **not** be done
proactively, to avoid proliferation of unnecessary Megolm sessions.
To create the session, the client should call
``olm_init_outbound_group_session``, and store the details of the
outbound session for future use.
The client should then call ``olm_outbound_group_session_id`` to get the
unique ID of the new session, and ``olm_outbound_group_session_key`` to
retrieve the current ratchet key and index. It should store these
details as an inbound session, just as it would when `receiving them via
an m.room_key event`__.
__ `m.room_key`_
The client must then share the keys for this session with each device in the
room. It must therefore `download the device list`_ if it hasn't already done
so, and for each device in the room which has not been `blocked`__, the client
should:
__ `blocking`_
* Build a content object as follows:
.. code:: json
{
"algorithm": "m.megolm.v1.aes-sha2",
"room_id": "<id of destination room>",
"session_id": "<session id>",
"session_key": "<session_key>"
}
- Encrypt the content as an ``m.room_key`` event using Olm, as below.
Once all of the key-sharing event contents have been assembled, the
events should be sent to the corresponding devices via
``PUT /_matrix/client/unstable/sendToDevice/m.room.encrypted/<txnId>``.
Rotating Megolm sessions
~~~~~~~~~~~~~~~~~~~~~~~~
Megolm sessions may not be reused indefinitely.
The number of messages which can be sent before a session should be rotated is
given by the ``rotation_period_msgs`` property of the |m.room.encryption|_
event, or ``100`` if that property isn't present.
Similarly, the maximum age of a megolm session is given, in milliseconds, by
the ``rotation_period_ms`` property of the ``m.room.encryption``
event. ``604800000`` (a week) is the recommended default here.
Once either the message limit or time limit have been reached, the client
should start a new session before sending any more messages.
Encrypting an event with Olm
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Olm is not used for encrypting room events, as it requires a separate
copy of the ciphertext for each device, and because the receiving device
can only decrypt received messages once. However, it is used for
encrypting key-sharing events for Megolm.
When encrypting an event using Olm, the client should:
- Build an encryption payload as follows:
.. code:: json
{
"type": "<event type>",
"content": "<event content>",
"sender": "<our user ID>",
"sender_device": "<our device ID>",
"keys": {
"ed25519": "<our ed25519 fingerprint key>"
},
"recipient": "<recipient user ID>",
"recipient_keys": {
"ed25519": "<recipient's ed25519 fingerprint key>"
},
}
- Check if it has an existing Olm session; if it does not, `start a new
one`__. If it has several (as may happen due to
races when establishing sessions), it should use the one with the
first session_id when sorted by their ASCII codepoints (ie, 'A'
would be before 'Z', which would be before 'a').
__ `Starting an Olm session`_
- Encrypt the payload by calling ``olm_encrypt``.
- Package the payload into event content as follows:
.. code:: json
{
"algorithm": "m.olm.v1.curve25519-aes-sha2",
"sender_key": "<our curve25519 identity key>",
"ciphertext": "<encrypted payload>"
}
Starting an Olm session
~~~~~~~~~~~~~~~~~~~~~~~
To start a new Olm session with another device, a client must first
claim one of the other device's one-time keys. To do this, it should
create a query object as follows:
.. code:: json
{
"<user id>": {
"<device_id>": "signed_curve25519",
...
},
...
}
and send this via ``POST /_matrix/client/unstable/keys/claim``. Claims
for multiple devices should be aggregated into a single request.
This will return a result as follows:
.. code:: json
{
"one_time_keys": {
"<user id>": {
"<device_id>": {
"signed_curve25519:<key_id>": {
"key": "<curve25519_key>",
"signatures": {
"<user_id>": {
"ed25519:<device_id>": "<signature>"
}
}
},
},
...
},
...
}
}
The client should first check the signatures on the signed key objects. As with
checking the signatures on the device keys, it should remove the ``signatures``
and (if present) ``unsigned`` properties, format the remainder as Canonical
JSON, and pass the result into ``olm_ed25519_verify``, using the Ed25519 device
key for the ``key`` parameter.
Provided the key object passes verification, the client should then pass the
key, along with the Curve25519 Identity key for the remote device, into
``olm_create_outbound_session``.
Handling membership changes
---------------------------
The client should monitor rooms which are configured to use encryption for
membership changes.
When a member leaves a room, the client should invalidate any active outbound
Megolm session, to ensure that a new session is used next time the user sends a
message.
When a new member joins a room, the client should first `download the device
list`_ for the new member, if it doesn't already have it.
After giving the user an opportunity to `block`__ any suspicious devices, the
client should share the keys for the outbound Megolm session with all the new
member's devices. This is done in the same way as `creating a new session`__,
except that there is no need to start a new Megolm session: due to the design
of the Megolm ratchet, the new user will only be able to decrypt messages
starting from the current state. The recommended method is to maintain a list
of members who are waiting for the session keys, and share them when the user
next sends a message.
__ `blocking`_
__ `Starting a Megolm session`_
Sending New Device announcements
--------------------------------
When a user logs in on a new device, it is necessary to make sure that
other devices in any rooms with encryption enabled are aware of the new
device. This is done as follows.
Once the initial call to the ``/sync`` API completes, the client should
iterate through each room where encryption is enabled. For each user
(including the client's own user), it should build a content object as
follows:
.. code:: json
{
"device_id": "<our device ID>",
"rooms": ["<shared room id 1>", "<room id 2>", ... ]
}
Once all of these have been constructed, they should be sent to all of the
relevant user's devices (using the wildcard ``*`` in place of the
``device_id``) via ``PUT
/_matrix/client/unstable/sendToDevice/m.new_device/<txnId>.``
Handling an ``m.new_device`` event
----------------------------------
As with ``m.room_key`` events, these will appear in the ``to_device``
section of the ``/sync`` response.
The client should `download the device list`_ of the sender, to get the details
of the new device.
The event content will contain a ``rooms`` property, as well as the
``device_id`` of the new device. For each room in the list, the client
should check if encryption is enabled, and if the sender of the event is
a member of that room. If so, the client should share the keys for the
outbound Megolm session with the new device, in the same way as
`handling a new user in the room`__.
__ `Handling membership changes`_
.. _`blocking`:
Blocking / Verifying devices
----------------------------
It should be possible for a user to mark each device belonging to
another user as 'Blocked' or 'Verified'.
When a user chooses to block a device, this means that no further
encrypted messages should be shared with that device. In short, it
should be excluded when sharing room keys when `starting a new Megolm
session <#_p5d1esx6gkrc>`__. Any active outbound Megolm sessions whose
keys have been shared with the device should also be invalidated so that
no further messages are sent over them.
Verifying a device involves ensuring that the device belongs to the
claimed user. Currently this must be done by showing the user the
Ed25519 fingerprint key for the device, and prompting the user to verify
out-of-band that it matches the key shown on the other user's device.
.. _`marking the event as verified`:
Marking events as 'verified'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Once a device has been verified, it is possible to verify that events
have been sent from a particular device. See the section on `Handling an
m.room.encrypted event`_ for notes on how to do this
for each algorithm. Events sent from a verified device can be decorated
in the UI to show that they have been sent from a verified device.

@ -1,79 +0,0 @@
---
layout: post
version: v1.0
title: Types of Bridging
categories: guides
---
<link href="/docs/css/faq.css" type="text/css" rel="stylesheet" />
# Types of bridging
Bridges come in many flavours, and we need consistent terminology within the Matrix community to ensure everyone (users, developers, core team) is on the same page. This post is primarily intended for bridge developers to refer to when building bridges.
*The most recent version of this document is [here](https://matrix.org/docs/guides/types-of-bridging.html) ([source](https://github.com/matrix-org/matrix-doc/blob/master/supporting-docs/guides/2017-03-11-types-of-bridging.md)) but were also posting it as a blog post for visibility.*
## Types of rooms
### Portal rooms
Bridges can register themselves as controlling chunks of room aliases namespace, letting Matrix users join remote rooms transparently if they /join #freenode_#wherever:matrix.org or similar. The resulting Matrix room is typically automatically bridged to the single target remote room. Access control for Matrix users is typically managed by the remote networks side of the room. This is called a portal room, and is useful for jumping into remote rooms without any configuration needed whatsoever - using Matrix as a bouncer for the remote network.
### Plumbed rooms
Alternatively, an existing Matrix room can be can plumbed into one or more specific remote rooms by configuring a bridge (which can be run by anyone). For instance, #matrix:matrix.org is plumbed into #matrix on Freenode, matrixdotorg/#matrix on Slack, etc. Access control for Matrix users is necessarily managed by the Matrix side of the room. This is useful for using Matrix to link together different communities.
Migrating rooms between a portal & plumbed room is currently a bit of a mess, as theres not yet a way for users to remove portal rooms once theyre created, so you can end up with a mix of portal & plumbed users bridged into a room, which looks weird from both the Matrix and non-Matrix viewpoints. https://github.com/matrix-org/matrix-appservice-irc/issues/387 tracks this.
## Types of bridges (simplest first):
### Bridgebot-based bridges
The simplest way to exchange messages with a remote network is to have the bridge log into the network using one or more predefined users called bridge bots - typically called MatrixBridge or MatrixBridge[123] etc. These relay traffic on behalf of the users on the other side, but its a terrible experience as all the metadata about the messages and senders is lost. This is how the [telematrix](https://github.com/SijmenSchoon/telematrix) matrix<->telegram bridge currently works.
### Bot-API (aka Virtual user) based bridges
Some remote systems support the idea of injecting messages from fake or virtual users, which can be used to represent the Matrix-side users as unique entities in the remote network. For instance, Slacks inbound webhooks lets remote bots be created on demand, letting Matrix users be shown cosmetically correctly in the timeline as virtual users. However, the resulting virtual users arent real users on the remote system, so dont have presence/profile and cant be tab-completed or direct-messaged etc. They also have no way to receive typing notifs or other richer info which may not be available via bot APIs. This is how the current [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge works.
### Simple puppeted bridge
This is a richer form of bridging, where the bridge logs into the remote service as if it were a real 3rd party client for that service. As a result, the Matrix user has to already have a valid account on the remote system. In exchange, the Matrix user puppets their remote user, such that other users on the remote system arent even aware they are speaking to a user via Matrix. The full semantics of the remote system are available to the bridge to expose into Matrix. However, the bridge has to handle the authentication process to log the user into the remote bridge.
This is essentially how the current [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge works (if you configure it to log into the remote IRC network as your real IRC nickname). [matrix-appservice-gitter](https://github.com/matrix-org/matrix-appservice-gitter) is being extended to support both puppeted and bridgebot-based operation. Its how the experimental [matrix-appservice-tg](https://github.com/matrix-org/matrix-appservice-tg) bridge works.
Going forwards were aiming for all bridges to be at least simple puppeted, if not double-puppeted.
### Double-puppeted bridge
A simple puppeted bridge allows the Matrix user to control their account on their remote network. However, ideally this puppeting should work in both directions, so if the user logs into (say) their native telegram client and starts conversations, sends messages etc, these should be reflected back into Matrix as if the user had done them there. This requires the bridge to be able to puppet the Matrix side of the bridge on behalf of the user.
This is the holy-grail of bridging because both the Matrix account and the third party account are accurately represented on their respective networks, with all user metadata intact. This is in contrast to a relaybot which would appear as a separate user from whom it represents.
Several obstacles exist to the proper implementation of double-puppeted bridges. On the Matrix side, we need an elegant way of having the bridge auth with Matrix as the matrix user (which requires some kind of scoped access_token delegation). On the third-party network, unique problems exist depending on the limitations of that particular network network. For example, many third party networks will lack the ability to represent other Matrix users than the one being puppeted (see hybrid relaybot).
[matrix-puppet-bridge](https://github.com/matrix-hacks/matrix-puppet-bridge) is a community project that tries to facilitate development of double-puppeted bridges, having done so, without a bridgebot feature, for [several networks](https://github.com/matrix-hacks/matrix-puppet-bridge#examples). A downside to their approach is the assumption that an individual will run the bridge on their own homeserver, thus working around the problem of sharing auth credentials on a shared homeserver.
### Hybrid Relaybot Puppet Bridge
This type of bridge is a combination single or double puppet bridge which tries to solve the problem of representing other users by means of the bridgebot technique. [matrix-appservice-gitter](https://github.com/matrix-org/matrix-appservice-gitter) works in this way.
### Server-to-server bridging
Some remote protocols (IRC, XMPP, SIP, SMTP, NNTP, GnuSocial etc) support federation - either open or closed. The most elegant way of bridging to these protocols would be to have the bridge participate in the federation as a server, directly bridging the entire namespace into Matrix.
Were not aware of anyone whos done this yet.
### One-way bridging
One-way bridging is rare, but can be used to represent a bridge that is bridging from the remote system into matrix. This is common when the remote system does not permit message posting, or is simply not capable of handling posting outside their system. The users bridged from the remote system often appear as virtual users in matrix, as is the case with [matrix-appservice-instagram](https://github.com/turt2live/matrix-appservice-instagram).
### Sidecar bridge
Finally: the types of bridging described above assume that you are synchronising the conversation history of the remote system into Matrix, so it may be decentralised and exposed to multiple users within the wider Matrix network.
This can cause problems where the remote system may have arbitrarily complicated permissions (ACLs) controlling access to the history, which will then need to be correctly synchronised with Matrixs ACL model, without introducing security issues such as races. We already see some problems with this on the IRC bridge, where history visibility for +i and +k channels have to be carefully synchronised with the Matrix rooms.
You can also hit problems with other network-specific features not yet having equivalent representation in the Matrix protocol (e.g. ephemeral messages, or op-only messages - although arguably thats a type of ACL).
One solution could be to support an entirely different architecture of bridging, where the Matrix client-server API is mapped directly to the remote service, meaning that ACL decisions are delegated to the remote service, and conversations are not exposed into the wider Matrix. This is effectively using the bridge purely as a 3rd party client for the network (similar to Bitlbee). The bridge is only available to a single user, and conversations cannot be shared with other Matrix users as they arent actually Matrix rooms. (Another solution could be to use Active Policy Servers at last as a way of centralising and delegating ACLs for a room)
This is essentially an entirely different product to the rest of Matrix, and whilst it could be a solution for some particularly painful ACL problems, were focusing on non-sidecar bridges for now.

@ -1,11 +0,0 @@
---
layout: project
title: Try Matrix Now!
categories: howtos
---
<html>
<head>
<meta http-equiv="refresh" content="0; url=../guides/client-server.html" />
</head>
</html>

@ -1,17 +0,0 @@
.loggedin {
visibility: hidden;
}
p {
font-family: monospace;
}
table
{
border-spacing:5px;
}
th,td
{
padding:5px;
}

@ -1,30 +0,0 @@
<div>
<p>This room creation / message sending demo requires a homeserver to be running on http://localhost:8008</p>
</div>
<form class="loginForm">
<input type="text" id="userLogin" placeholder="Username"></input>
<input type="password" id="passwordLogin" placeholder="Password"></input>
<input type="button" class="login" value="Login"></input>
</form>
<div class="loggedin">
<form class="createRoomForm">
<input type="text" id="roomAlias" placeholder="Room alias (optional)"></input>
<input type="button" class="createRoom" value="Create Room"></input>
</form>
<form class="sendMessageForm">
<input type="text" id="roomId" placeholder="Room ID"></input>
<input type="text" id="messageBody" placeholder="Message body"></input>
<input type="button" class="sendMessage" value="Send Message"></input>
</form>
<table id="rooms">
<tbody>
<tr>
<th>Room ID</th>
<th>My state</th>
<th>Room Alias</th>
<th>Latest message</th>
</tr>
</tbody>
</table>
</div>

@ -1,113 +0,0 @@
var accountInfo = {};
var showLoggedIn = function(data) {
accountInfo = data;
getCurrentRoomList();
$(".loggedin").css({visibility: "visible"});
};
$('.login').live('click', function() {
var user = $("#userLogin").val();
var password = $("#passwordLogin").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/login",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
showLoggedIn(data);
},
error: function(err) {
var errMsg = "To try this, you need a homeserver running!";
var errJson = $.parseJSON(err.responseText);
if (errJson) {
errMsg = JSON.stringify(errJson);
}
alert(errMsg);
}
});
});
var getCurrentRoomList = function() {
var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
$.getJSON(url, function(data) {
var rooms = data.rooms;
for (var i=0; i<rooms.length; ++i) {
rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;
addRoom(rooms[i]);
}
}).fail(function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
});
};
$('.createRoom').live('click', function() {
var roomAlias = $("#roomAlias").val();
var data = {};
if (roomAlias.length > 0) {
data.room_alias_name = roomAlias;
}
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
data.membership = "join"; // you are automatically joined into every room you make.
data.latest_message = "";
addRoom(data);
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});
var addRoom = function(data) {
row = "<tr>" +
"<td>"+data.room_id+"</td>" +
"<td>"+data.membership+"</td>" +
"<td>"+data.room_alias+"</td>" +
"<td>"+data.latest_message+"</td>" +
"</tr>";
$("#rooms").append(row);
};
$('.sendMessage').live('click', function() {
var roomId = $("#roomId").val();
var body = $("#messageBody").val();
var msgId = $.now();
if (roomId.length === 0 || body.length === 0) {
return;
}
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomid", encodeURIComponent(roomId));
var data = {
msgtype: "m.text",
body: body
};
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
$("#messageBody").val("");
// wipe the table and reload it. Using the event stream would be the best
// solution but that is out of scope of this fiddle.
$("#rooms").find("tr:gt(0)").remove();
getCurrentRoomList();
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});

@ -1,17 +0,0 @@
.loggedin {
visibility: hidden;
}
p {
font-family: monospace;
}
table
{
border-spacing:5px;
}
th,td
{
padding:5px;
}

@ -1,23 +0,0 @@
<div>
<p>This event stream demo requires a homeserver to be running on http://localhost:8008</p>
</div>
<form class="loginForm">
<input type="text" id="userLogin" placeholder="Username"></input>
<input type="password" id="passwordLogin" placeholder="Password"></input>
<input type="button" class="login" value="Login"></input>
</form>
<div class="loggedin">
<form class="sendMessageForm">
<input type="button" class="sendMessage" value="Send random message"></input>
</form>
<p id="streamErrorText"></p>
<table id="rooms">
<tbody>
<tr>
<th>Room ID</th>
<th>Latest message</th>
</tr>
</tbody>
</table>
</div>

@ -1,145 +0,0 @@
var accountInfo = {};
var eventStreamInfo = {
from: "END"
};
var roomInfo = [];
var longpollEventStream = function() {
var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$from", eventStreamInfo.from);
$.getJSON(url, function(data) {
eventStreamInfo.from = data.end;
var hasNewLatestMessage = false;
for (var i=0; i<data.chunk.length; ++i) {
if (data.chunk[i].type === "m.room.message") {
for (var j=0; j<roomInfo.length; ++j) {
if (roomInfo[j].room_id === data.chunk[i].room_id) {
roomInfo[j].latest_message = data.chunk[i].content.body;
hasNewLatestMessage = true;
}
}
}
}
if (hasNewLatestMessage) {
setRooms(roomInfo);
}
$("#streamErrorText").text("");
longpollEventStream();
}).fail(function(err) {
$("#streamErrorText").text("Event stream error: "+JSON.stringify($.parseJSON(err.responseText)));
setTimeout(longpollEventStream, 5000);
});
};
var showLoggedIn = function(data) {
accountInfo = data;
longpollEventStream();
getCurrentRoomList();
$(".loggedin").css({visibility: "visible"});
};
$('.login').live('click', function() {
var user = $("#userLogin").val();
var password = $("#passwordLogin").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/login",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
$("#rooms").find("tr:gt(0)").remove();
showLoggedIn(data);
},
error: function(err) {
var errMsg = "To try this, you need a homeserver running!";
var errJson = $.parseJSON(err.responseText);
if (errJson) {
errMsg = JSON.stringify(errJson);
}
alert(errMsg);
}
});
});
var getCurrentRoomList = function() {
$("#roomId").val("");
var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
$.getJSON(url, function(data) {
var rooms = data.rooms;
for (var i=0; i<rooms.length; ++i) {
if ("messages" in rooms[i]) {
rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;
}
}
roomInfo = rooms;
setRooms(roomInfo);
}).fail(function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
});
};
$('.sendMessage').live('click', function() {
if (roomInfo.length === 0) {
alert("There is no room to send a message to!");
return;
}
var index = Math.floor(Math.random() * roomInfo.length);
sendMessage(roomInfo[index].room_id);
});
var sendMessage = function(roomId) {
var body = "jsfiddle message @" + $.now();
if (roomId.length === 0) {
return;
}
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomid", encodeURIComponent(roomId));
var data = {
msgtype: "m.text",
body: body
};
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
$("#messageBody").val("");
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
};
var setRooms = function(roomList) {
// wipe existing entries
$("#rooms").find("tr:gt(0)").remove();
var rows = "";
for (var i=0; i<roomList.length; ++i) {
row = "<tr>" +
"<td>"+roomList[i].room_id+"</td>" +
"<td>"+roomList[i].latest_message+"</td>" +
"</tr>";
rows += row;
}
$("#rooms").append(rows);
};

@ -1,43 +0,0 @@
.roomListDashboard, .roomContents, .sendMessageForm {
visibility: hidden;
}
.roomList {
background-color: #909090;
}
.messageWrapper {
background-color: #EEEEEE;
height: 400px;
overflow: scroll;
}
.membersWrapper {
background-color: #EEEEEE;
height: 200px;
width: 50%;
overflow: scroll;
}
.textEntry {
width: 100%
}
p {
font-family: monospace;
}
table
{
border-spacing:5px;
}
th,td
{
padding:5px;
}
.roomList tr:not(:first-child):hover {
background-color: orange;
cursor: pointer;
}

@ -1,7 +0,0 @@
name: Example Matrix Client
description: Includes login, live event streaming, creating rooms, sending messages and viewing member lists.
authors:
- matrix.org
resources:
- http://matrix.org
normalize_css: no

@ -1,56 +0,0 @@
<div class="signUp">
<p>Matrix example application: Requires a local homeserver running at http://localhost:8008</p>
<form class="registrationForm">
<p>No account? Register:</p>
<input type="text" id="userReg" placeholder="Username"></input>
<input type="password" id="passwordReg" placeholder="Password"></input>
<input type="button" class="register" value="Register"></input>
</form>
<form class="loginForm">
<p>Got an account? Login:</p>
<input type="text" id="userLogin" placeholder="Username"></input>
<input type="password" id="passwordLogin" placeholder="Password"></input>
<input type="button" class="login" value="Login"></input>
</form>
</div>
<div class="roomListDashboard">
<form class="createRoomForm">
<input type="text" id="roomAlias" placeholder="Room alias"></input>
<input type="button" class="createRoom" value="Create Room"></input>
</form>
<table id="rooms" class="roomList">
<tbody>
<tr>
<th>Room</th>
<th>My state</th>
<th>Latest message</th>
</tr>
</tbody>
</table>
</div>
<div class="roomContents">
<p id="roomName">Select a room</p>
<div class="messageWrapper">
<table id="messages">
<tbody>
</tbody>
</table>
</div>
<form class="sendMessageForm">
<input type="text" class="textEntry" id="body" placeholder="Enter text here..." onkeydown="javascript:if (event.keyCode == 13) document.getElementById('sendMsg').focus()"></input>
<input type="button" class="sendMessage" id="sendMsg" value="Send"></input>
</form>
</div>
<div>
<p>Member list:</p>
<div class="membersWrapper">
<table id="members">
<tbody>
</tbody>
</table>
</div>
</div>

@ -1,327 +0,0 @@
var accountInfo = {};
var eventStreamInfo = {
from: "END"
};
var roomInfo = [];
var memberInfo = [];
var viewingRoomId;
// ************** Event Streaming **************
var longpollEventStream = function() {
var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$from", eventStreamInfo.from);
$.getJSON(url, function(data) {
eventStreamInfo.from = data.end;
var hasNewLatestMessage = false;
var updatedMemberList = false;
var i=0;
var j=0;
for (i=0; i<data.chunk.length; ++i) {
if (data.chunk[i].type === "m.room.message") {
console.log("Got new message: " + JSON.stringify(data.chunk[i]));
if (viewingRoomId === data.chunk[i].room_id) {
addMessage(data.chunk[i]);
}
for (j=0; j<roomInfo.length; ++j) {
if (roomInfo[j].room_id === data.chunk[i].room_id) {
roomInfo[j].latest_message = data.chunk[i].content.body;
hasNewLatestMessage = true;
}
}
}
else if (data.chunk[i].type === "m.room.member") {
if (viewingRoomId === data.chunk[i].room_id) {
console.log("Got new member: " + JSON.stringify(data.chunk[i]));
addMessage(data.chunk[i]);
for (j=0; j<memberInfo.length; ++j) {
if (memberInfo[j].state_key === data.chunk[i].state_key) {
memberInfo[j] = data.chunk[i];
updatedMemberList = true;
break;
}
}
if (!updatedMemberList) {
memberInfo.push(data.chunk[i]);
updatedMemberList = true;
}
}
if (data.chunk[i].state_key === accountInfo.user_id) {
getCurrentRoomList(); // update our join/invite list
}
}
else {
console.log("Discarding: " + JSON.stringify(data.chunk[i]));
}
}
if (hasNewLatestMessage) {
setRooms(roomInfo);
}
if (updatedMemberList) {
$("#members").empty();
for (i=0; i<memberInfo.length; ++i) {
addMember(memberInfo[i]);
}
}
longpollEventStream();
}).fail(function(err) {
setTimeout(longpollEventStream, 5000);
});
};
// ************** Registration and Login **************
var onLoggedIn = function(data) {
accountInfo = data;
longpollEventStream();
getCurrentRoomList();
$(".roomListDashboard").css({visibility: "visible"});
$(".roomContents").css({visibility: "visible"});
$(".signUp").css({display: "none"});
};
$('.login').live('click', function() {
var user = $("#userLogin").val();
var password = $("#passwordLogin").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/login",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
onLoggedIn(data);
},
error: function(err) {
alert("Unable to login: is the homeserver running?");
}
});
});
$('.register').live('click', function() {
var user = $("#userReg").val();
var password = $("#passwordReg").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/register",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
onLoggedIn(data);
},
error: function(err) {
var msg = "Is the homeserver running?";
var errJson = $.parseJSON(err.responseText);
if (errJson !== null) {
msg = errJson.error;
}
alert("Unable to register: "+msg);
}
});
});
// ************** Creating a room ******************
$('.createRoom').live('click', function() {
var roomAlias = $("#roomAlias").val();
var data = {};
if (roomAlias.length > 0) {
data.room_alias_name = roomAlias;
}
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(response) {
$("#roomAlias").val("");
response.membership = "join"; // you are automatically joined into every room you make.
response.latest_message = "";
roomInfo.push(response);
setRooms(roomInfo);
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});
// ************** Getting current state **************
var getCurrentRoomList = function() {
var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
$.getJSON(url, function(data) {
var rooms = data.rooms;
for (var i=0; i<rooms.length; ++i) {
if ("messages" in rooms[i]) {
rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;
}
}
roomInfo = rooms;
setRooms(roomInfo);
}).fail(function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
});
};
var loadRoomContent = function(roomId) {
console.log("loadRoomContent " + roomId);
viewingRoomId = roomId;
$("#roomName").text("Room: "+roomId);
$(".sendMessageForm").css({visibility: "visible"});
getMessages(roomId);
getMemberList(roomId);
};
var getMessages = function(roomId) {
$("#messages").empty();
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" +
encodeURIComponent(roomId) + "/messages?access_token=" + accountInfo.access_token + "&from=END&dir=b&limit=10";
$.getJSON(url, function(data) {
for (var i=data.chunk.length-1; i>=0; --i) {
addMessage(data.chunk[i]);
}
});
};
var getMemberList = function(roomId) {
$("#members").empty();
memberInfo = [];
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" +
encodeURIComponent(roomId) + "/members?access_token=" + accountInfo.access_token;
$.getJSON(url, function(data) {
for (var i=0; i<data.chunk.length; ++i) {
memberInfo.push(data.chunk[i]);
addMember(data.chunk[i]);
}
});
};
// ************** Sending messages **************
$('.sendMessage').live('click', function() {
if (viewingRoomId === undefined) {
alert("There is no room to send a message to!");
return;
}
var body = $("#body").val();
sendMessage(viewingRoomId, body);
});
var sendMessage = function(roomId, body) {
var msgId = $.now();
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomid", encodeURIComponent(roomId));
var data = {
msgtype: "m.text",
body: body
};
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
$("#body").val("");
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
};
// ************** Navigation and DOM manipulation **************
var setRooms = function(roomList) {
// wipe existing entries
$("#rooms").find("tr:gt(0)").remove();
var rows = "";
for (var i=0; i<roomList.length; ++i) {
row = "<tr>" +
"<td>"+roomList[i].room_id+"</td>" +
"<td>"+roomList[i].membership+"</td>" +
"<td>"+roomList[i].latest_message+"</td>" +
"</tr>";
rows += row;
}
$("#rooms").append(rows);
$('#rooms').find("tr").click(function(){
var roomId = $(this).find('td:eq(0)').text();
var membership = $(this).find('td:eq(1)').text();
if (membership !== "join") {
console.log("Joining room " + roomId);
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/join?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomid", encodeURIComponent(roomId));
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({membership: "join"}),
dataType: "json",
success: function(data) {
loadRoomContent(roomId);
getCurrentRoomList();
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
}
else {
loadRoomContent(roomId);
}
});
};
var addMessage = function(data) {
var msg = data.content.body;
if (data.type === "m.room.member") {
if (data.content.membership === undefined) {
return;
}
if (data.content.membership === "invite") {
msg = "<em>invited " + data.state_key + " to the room</em>";
}
else if (data.content.membership === "join") {
msg = "<em>joined the room</em>";
}
else if (data.content.membership === "leave") {
msg = "<em>left the room</em>";
}
else if (data.content.membership === "ban") {
msg = "<em>was banned from the room</em>";
}
}
if (msg === undefined) {
return;
}
var row = "<tr>" +
"<td>"+data.user_id+"</td>" +
"<td>"+msg+"</td>" +
"</tr>";
$("#messages").append(row);
};
var addMember = function(data) {
var row = "<tr>" +
"<td>"+data.state_key+"</td>" +
"<td>"+data.content.membership+"</td>" +
"</tr>";
$("#members").append(row);
};

@ -1,7 +0,0 @@
.loggedin {
visibility: hidden;
}
p {
font-family: monospace;
}

@ -1,20 +0,0 @@
<div>
<p>This registration/login demo requires a homeserver to be running on http://localhost:8008</p>
</div>
<form class="registrationForm">
<input type="text" id="user" placeholder="Username"></input>
<input type="password" id="password" placeholder="Password"></input>
<input type="button" class="register" value="Register"></input>
</form>
<form class="loginForm">
<input type="text" id="userLogin" placeholder="Username"></input>
<input type="password" id="passwordLogin" placeholder="Password"></input>
<input type="button" class="login" value="Login"></input>
</form>
<div class="loggedin">
<p id="welcomeText"></p>
<input type="button" class="testToken" value="Test token"></input>
<input type="button" class="logout" value="Logout"></input>
<p id="imSyncText"></p>
</div>

@ -1,79 +0,0 @@
var accountInfo = {};
var showLoggedIn = function(data) {
accountInfo = data;
$(".loggedin").css({visibility: "visible"});
$("#welcomeText").text("Welcome " + accountInfo.user_id+". Your access token is: " +
accountInfo.access_token);
};
$('.register').live('click', function() {
var user = $("#user").val();
var password = $("#password").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/register",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
showLoggedIn(data);
},
error: function(err) {
var errMsg = "To try this, you need a homeserver running!";
var errJson = $.parseJSON(err.responseText);
if (errJson) {
errMsg = JSON.stringify(errJson);
}
alert(errMsg);
}
});
});
var login = function(user, password) {
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/login",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
showLoggedIn(data);
},
error: function(err) {
var errMsg = "To try this, you need a homeserver running!";
var errJson = $.parseJSON(err.responseText);
if (errJson) {
errMsg = JSON.stringify(errJson);
}
alert(errMsg);
}
});
};
$('.login').live('click', function() {
var user = $("#userLogin").val();
var password = $("#passwordLogin").val();
$.getJSON("http://localhost:8008/_matrix/client/api/v1/login", function(data) {
if (data.flows[0].type !== "m.login.password") {
alert("I don't know how to login with this type: " + data.type);
return;
}
login(user, password);
});
});
$('.logout').live('click', function() {
accountInfo = {};
$("#imSyncText").text("");
$(".loggedin").css({visibility: "hidden"});
});
$('.testToken').live('click', function() {
var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
$.getJSON(url, function(data) {
$("#imSyncText").text(JSON.stringify(data, undefined, 2));
}).fail(function(err) {
$("#imSyncText").text(JSON.stringify($.parseJSON(err.responseText)));
});
});

@ -1,17 +0,0 @@
.loggedin {
visibility: hidden;
}
p {
font-family: monospace;
}
table
{
border-spacing:5px;
}
th,td
{
padding:5px;
}

@ -1,37 +0,0 @@
<div>
<p>This room membership demo requires a homeserver to be running on http://localhost:8008</p>
</div>
<form class="loginForm">
<input type="text" id="userLogin" placeholder="Username"></input>
<input type="password" id="passwordLogin" placeholder="Password"></input>
<input type="button" class="login" value="Login"></input>
</form>
<div class="loggedin">
<form class="createRoomForm">
<input type="button" class="createRoom" value="Create Room"></input>
</form>
<form class="changeMembershipForm">
<input type="text" id="roomId" placeholder="Room ID"></input>
<input type="text" id="targetUser" placeholder="Target User ID"></input>
<select id="membership">
<option value="invite">invite</option>
<option value="join">join</option>
<option value="leave">leave</option>
</select>
<input type="button" class="changeMembership" value="Change Membership"></input>
</form>
<form class="joinAliasForm">
<input type="text" id="roomAlias" placeholder="Room Alias (#name:domain)"></input>
<input type="button" class="joinAlias" value="Join via Alias"></input>
</form>
<table id="rooms">
<tbody>
<tr>
<th>Room ID</th>
<th>My state</th>
<th>Room Alias</th>
</tr>
</tbody>
</table>
</div>

@ -1,141 +0,0 @@
var accountInfo = {};
var showLoggedIn = function(data) {
accountInfo = data;
getCurrentRoomList();
$(".loggedin").css({visibility: "visible"});
$("#membership").change(function() {
if ($("#membership").val() === "invite") {
$("#targetUser").css({visibility: "visible"});
}
else {
$("#targetUser").css({visibility: "hidden"});
}
});
};
$('.login').live('click', function() {
var user = $("#userLogin").val();
var password = $("#passwordLogin").val();
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/login",
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
dataType: "json",
success: function(data) {
$("#rooms").find("tr:gt(0)").remove();
showLoggedIn(data);
},
error: function(err) {
var errMsg = "To try this, you need a homeserver running!";
var errJson = $.parseJSON(err.responseText);
if (errJson) {
errMsg = JSON.stringify(errJson);
}
alert(errMsg);
}
});
});
var getCurrentRoomList = function() {
$("#roomId").val("");
// wipe the table and reload it. Using the event stream would be the best
// solution but that is out of scope of this fiddle.
$("#rooms").find("tr:gt(0)").remove();
var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
$.getJSON(url, function(data) {
var rooms = data.rooms;
for (var i=0; i<rooms.length; ++i) {
addRoom(rooms[i]);
}
}).fail(function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
});
};
$('.createRoom').live('click', function() {
var data = {};
$.ajax({
url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
data.membership = "join"; // you are automatically joined into every room you make.
data.latest_message = "";
addRoom(data);
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});
var addRoom = function(data) {
row = "<tr>" +
"<td>"+data.room_id+"</td>" +
"<td>"+data.membership+"</td>" +
"<td>"+data.room_alias+"</td>" +
"</tr>";
$("#rooms").append(row);
};
$('.changeMembership').live('click', function() {
var roomId = $("#roomId").val();
var member = $("#targetUser").val();
var membership = $("#membership").val();
if (roomId.length === 0) {
return;
}
var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/$membership?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomid", encodeURIComponent(roomId));
url = url.replace("$membership", membership);
var data = {};
if (membership === "invite") {
data = {
user_id: member
};
}
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify(data),
dataType: "json",
success: function(data) {
getCurrentRoomList();
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});
$('.joinAlias').live('click', function() {
var roomAlias = $("#roomAlias").val();
var url = "http://localhost:8008/_matrix/client/api/v1/join/$roomalias?access_token=$token";
url = url.replace("$token", accountInfo.access_token);
url = url.replace("$roomalias", encodeURIComponent(roomAlias));
$.ajax({
url: url,
type: "POST",
contentType: "application/json; charset=utf-8",
data: JSON.stringify({}),
dataType: "json",
success: function(data) {
getCurrentRoomList();
},
error: function(err) {
alert(JSON.stringify($.parseJSON(err.responseText)));
}
});
});

@ -1,20 +0,0 @@
---
layout: project
title: Riot
categories: projects client
thumbnail: /docs/projects/images/riot-web-small.png
author: Riot.im
description: Riot is a glossy web client with an emphasis on performance and usability
maturity: Released
---
![screenshot](/docs/projects/images/riot-web-large.png "{{ page.title }}")
# {{ page.title }}
[Riot](https://riot.im) is a glossy Matrix client built on top of [matrix-react-sdk](http://matrix.org/docs/projects/sdk/matrix.org-react-sdk.html) with an emphasis on performance and usability.
You can use it at [https://riot.im/app](https://riot.im/app), read more at [https://riot.im](https://riot.im) and get the source from [github](https://github.com/vector-im/vector-web)!
There is also a desktop version, which is available at [riot.im](https://riot.im/desktop.html). Taw has created RPM package builds for Fedora, CentOS, and Red Hat Enterprise Linux which are available via [GitHub](https://github.com/taw00/riot-rpm/) and [FedoraCorp](https://copr.fedorainfracloud.org/coprs/taw/Riot/).
Josué Tille has contributed a [Yunohost app](https://github.com/Josue-T/riot_ynh).

@ -1,10 +0,0 @@
---
layout: project
title: Try Matrix Now!
---
<html>
<head>
<meta http-equiv="refresh" content="0; url=./riot.html" />
</head>
</html>

@ -1,32 +0,0 @@
---
layout: project
title: Synapse
categories: projects server
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/Screen-Shot-2015-04-29-at-00.28.25-400x284.png
author: Matrix.org team
maturity: Late beta
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/Screen-Shot-2015-04-29-at-00.28.25-1080x579.png "{{ page.title }}")
# {{ page.title }}
Matrix.orgs reference server Synapse: [https://github.com/matrix-org/synapse](https://github.com/matrix-org/synapse)
Apt repo: [https://matrix.org/packages/debian/](https://matrix.org/packages/debian/)
Dockerfile from Silvio Fricke: [https://registry.hub.docker.com/u/silviof/docker-matrix/](https://registry.hub.docker.com/u/silviof/docker-matrix/) also hosted on [https://github.com/matrix-org/docker-matrix](https://github.com/matrix-org/docker-matrix)
ArchLinux package from Ivan Shapovalov: [https://www.archlinux.org/packages/community/any/matrix-synapse/](https://www.archlinux.org/packages/community/any/matrix-synapse/)
There is a FreeBSD package on [freshports.org](http://www.freshports.org/net/py-matrix-synapse/).
Martin Giess has created an auto-deployment process with vagrant/ansible, tested with VirtualBox/AWS/DigitalOcean see [https://github.com/EMnify/matrix-synapse-auto-deploy](https://github.com/EMnify/matrix-synapse-auto-deploy) for details.
Synapse is also on the [Open Build Service](https://obs.infoserver.lv/project/show/matrix-synapse).
Synapse is available for the [Nix package manager](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix).
You can get Synapse on [Yunohost](https://github.com/YunoHost-Apps/synapse_ynh)
There is also a [handy spreadsheet](http://matrix.org/docs/projects/other/hdd-space-calc-for-synapse.html) to calculate HDD space for your Synapse instance.

@ -1,11 +0,0 @@
---
layout: project
title: NEB (N. E. Bot)
categories: projects other
description: Our dear Matrix Bot
author: Kegsay
maturity: Late beta
---
# {{ page.title }}
Kegsay's general-purpose Python Matrix Bot framework ([github](https://github.com/Kegsay/Matrix-NEB))

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org Android SDK
categories: projects sdk
author: Matrix.org team
maturity: Late beta
---
# {{ page.title }}
Matrix.org's Android SDK ([github](https://github.com/matrix-org/matrix-android-sdk))

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org iOS SDK
categories: projects sdk
author: Matrix.org team
maturity: Late beta
---
# {{ page.title }}
Matrix.org's iOS SDK ([github](https://github.com/matrix-org/matrix-ios-sdk))

@ -1,13 +0,0 @@
---
layout: project
title: Pallium
categories: projects server
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/pallium.png
author: KoFish
maturity: Alpha
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/pallium.png "{{ page.title }}")
# {{ page.title }}
KoFishs homeserver written in Go Pallium ([github](https://github.com/KoFish/pallium))

@ -1,12 +0,0 @@
---
layout: project
title: Matrix.org AngularJS SDK
categories: projects sdk
author: Matrix.org team
maturity: DEPRECATED
---
# {{ page.title }}
AngularJS services from Matrix.org for building communication apps in Angular based on Matrix: [https://github.com/matrix-org/matrix-angular-sdk](https://github.com/matrix-org/matrix-angular-sdk)
These are no longer being actively maintained.

@ -1,11 +0,0 @@
---
layout: project
title: Perl Matrix-IRC Bridge
categories: projects other
description: The first Matrix/IRC bridge
author: Tom Molesworth / Paul Evans
maturity: Beta
---
# {{ page.title }}
The first Matrix/IRC bridge, originally written by Tom Molesworth using Paul Evans' Net::Async::Matrix Perl client SDK and then later maintained by Paul Evans. Nowadays superceded by the [matrix-appservice-irc Node IRC bridge AS](../as/irc-bridge.html).

@ -1,11 +0,0 @@
---
layout: project
title: ivar2 Matrix/IRC Bot
categories: projects other
description: IRC bot with native Matrix support
author: haste / Tor
maturity: Beta
---
# {{ page.title }}
ivar2 is a very comprehensive IRC bot written in Lua which has native support for Matrix! [https://github.com/torhve/ivar2](https://github.com/torhve/ivar2)

@ -1,11 +0,0 @@
---
layout: project
title: Try Matrix Now!
categories: projects
---
<html>
<head>
<meta http-equiv="refresh" content="0; url=./try-matrix-now.html" />
</head>
</html>

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org Python SDK
categories: projects sdk
author: Matrix.org team
maturity: Alpha
---
# {{ page.title }}
Matrix.org's Python SDK ([github](https://github.com/matrix-org/matrix-python-sdk))

@ -1,13 +0,0 @@
---
layout: project
title: OpenMarket's SMS Gateway
categories: projects as
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/omlogo-400x284.jpg
author: OpenMarket
maturity: Late beta
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/omlogo.jpg "{{ page.title }}")
# {{ page.title }}
OpenMarkets SMS Gateway is an Application Service that enables Matrix-users to send SMS. [More info here](http://matrix.org/blog/2015/02/26/welcoming-the-openmarket-matrix-gateway/).

@ -1,18 +0,0 @@
---
layout: project
title: WeeChat script
categories: projects client
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/Screen-Shot-2015-04-22-at-21.43.01-400x284.png
description: For fans of command line interfaces, a nice Matrix script for WeeChat, an IRSSI like CLI
author: Tor
maturity: Late beta
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/Screen-Shot-2015-04-22-at-21.43.01.png "{{ page.title }}")
# {{ page.title }}
[WeeChat](http://weechat.org/) is a super powerful CLI chat client that is extensible in many languages to allow for new protocols like Matrix. @torhve contributed this great WeeChat script for Matrix which is getting a good feedback from more geeky users.
To use it, you will need to sign up for a Matrix account on a server using the [Console Web](./matrix-console.html) or [Vector](./vector.html) clients, as WeeChat doesnt support registration yet.
To run it, grab the code from [@torhve's github repo](https://github.com/torhve/weechat-matrix-protocol-script).

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org AS Node SDK
categories: projects as
author: Matrix.org team
maturity: Early beta
---
# {{ page.title }}
A framework from Matrix.org for building Application Services in Node.js/Express: [https://github.com/matrix-org/matrix-appservice-node](https://github.com/matrix-org/matrix-appservice-node)

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org JS SDK
categories: projects sdk
author: Matrix.org team
maturity: Early beta
---
# {{ page.title }}
Matrix.org's JS SDK ([github](https://github.com/matrix-org/matrix-js-sdk))

@ -1,14 +0,0 @@
---
layout: project
title: Matrix.org IRC Bridge
categories: projects as
description:
author: Matrix.org team
maturity: Early beta
---
# {{ page.title }}
An application service gateway from Matrix.org for bridging between IRC networks and Matrix, written using [matrix-appservice-node](http://matrix.org/blog/project/matrix-appservice-node/). You can get the [code on github](https://github.com/matrix-org/matrix-appservice-irc).
Supports dynamically bridging IRC channels on demand; synchronised user-lists, and other goodness.

@ -1,10 +0,0 @@
---
layout: project
title: redpill IRC bridge
categories: projects as
author: Tjgillies
maturity: Alpha
---
# {{ page.title }}
Kodo's IRC<->matrix GW written in Ruby ([github](https://github.com/tjgillies/redpill))

@ -1,13 +0,0 @@
---
layout: project
title: jSynapse
categories: projects server
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/jsynapse2.png
author: Swarmcom
maturity: Alpha
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/jsynapse2.png "{{ page.title }}")
# {{ page.title }}
Swarmcoms Java homeserver jSynapse ([github](https://github.com/swarmcom/jSynapse/))

@ -1,10 +0,0 @@
---
layout: project
title: gomatrix IRC bridge
categories: projects as
author: Tor
maturity: Alpha
---
# {{ page.title }}
Tor's IRC<->Matrix bridge written in the Go language ([github](https://github.com/torhve/gomirc))

@ -1,10 +0,0 @@
---
layout: project
title: Matrix.org MatrixKit (iOS)
categories: projects sdk
author: Matrix.org team
maturity: Late beta
---
# {{ page.title }}
Matrix.org's reusable UI interfaces for iOS ([github](https://github.com/matrix-org/matrix-ios-kit))

@ -1,16 +0,0 @@
---
layout: project
title: Riot iOS
categories: projects client
thumbnail: /docs/projects/images/vector-iOS-small.png
description: Riot is a glossy client with an emphasis on performance and usability
author: Riot.im
maturity: Released
---
![screenshot](/docs/projects/images/vector-iOS-large.png "{{ page.title }}")
# {{ page.title }}
The iOS version of the [Riot](https://matrix.org/docs/projects/client/riot.html) web client. Riot is a glossy client with focus on performance and usability.
The code is available from [github](https://github.com/vector-im/vector-ios), and the app is available from the Apple [app store](https://itunes.apple.com/gb/app/vector.im/id1083446067?mt=8).

@ -1,10 +0,0 @@
---
layout: project
title: Try Matrix Now!
---
<html>
<head>
<meta http-equiv="refresh" content="0; url=./riot-ios.html" />
</head>
</html>

@ -1,15 +0,0 @@
---
layout: project
title: Project Clearwater / Matrix Gateway
categories: projects as
thumbnail: https://matrix.org/blog/wp-content/uploads/2015/04/clearwter-400x284.jpg
author: Matt Williams
maturity: Alpha
---
![screenshot](https://matrix.org/blog/wp-content/uploads/2015/04/clearwter.jpg "{{ page.title }}")
![demo](https://matrix.org/blog/wp-content/uploads/2015/04/projectclearwatermatrixgateway1.jpg "Matt presenting his hack at TADHack London")
# {{ page.title }}
Matt Williams IMS->Matrix gateway ([github](https://github.com/matt-williams/sprout/tree/matrix))

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save