Merge branch 'master' of github.com:matrix-org/matrix-doc

pull/977/head
Oddvar Lovaas 9 years ago
commit 10f3e7e84f

3
.gitignore vendored

@ -1,5 +1,8 @@
scripts/gen
scripts/continuserv/continuserv
scripts/speculator/speculator
templating/out
*.pyc
supporting-docs/_site
supporting-docs/.sass-cache
api/node_modules

@ -6,6 +6,34 @@
.. in Jenkins. Comments like this are ignored by both RST and the templating
.. system. Add the newest release notes beneath this comment.
Specification changes in v0.2.0 (2015-10-02)
============================================
This update fundamentally restructures the specification. The specification has
been split into more digestible "modules" which each describe a particular
function (e.g. typing). This was done in order make the specification easier to
maintain and help define which modules are mandatory for certain types
of clients. Types of clients along with the mandatory modules can be found in a
new "Feature Profiles" section. This update also begins to aggressively
standardise on using Swagger and JSON Schema to document HTTP endpoints and
Events respectively. It also introduces a number of new concepts to Matrix.
Additions:
- New section: Feature Profiles.
- New section: Receipts.
- New section: Room history visibility.
- New event: ``m.receipt``.
- New event: ``m.room.canonical_alias``
- New event: ``m.room.history_visibility``
- New keys: ``/createRoom`` - allows room "presets" using ``preset`` and
``initial_state`` keys.
- New endpoint: ``/tokenrefresh`` - Related to refreshing access tokens.
Modifications:
- Convert most of the older HTTP APIs to Swagger documentation.
- Convert most of the older event formats to JSON Schema.
- Move selected client-server sections to be "Modules".
Specification changes in v0.1.0 (2015-06-01)
============================================
- First numbered release.

@ -0,0 +1,118 @@
#! /usr/bin/env python
import sys
import json
import os
def import_error(module, package, debian, error):
sys.stderr.write((
"Error importing %(module)s: %(error)r\n"
"To install %(module)s run:\n"
" pip install %(package)s\n"
"or on Debian run:\n"
" sudo apt-get install python-%(debian)s\n"
) % locals())
if __name__ == '__main__':
sys.exit(1)
try:
import jsonschema
except ImportError as e:
import_error("jsonschema", "jsonschema", "jsonschema", e)
raise
try:
import yaml
except ImportError as e:
import_error("yaml", "PyYAML", "yaml", e)
raise
def check_parameter(filepath, request, parameter):
schema = parameter.get("schema")
example = None
try:
example_json = schema.get('example')
if example_json and not schema.get("format") == "byte":
example = json.loads(example_json)
except Exception as e:
raise ValueError("Error parsing JSON example request for %r" % (
request
), e)
fileurl = "file://" + os.path.abspath(filepath)
if example and schema:
try:
print ("Checking request schema for: %r %r" % (
filepath, request
))
# Setting the 'id' tells jsonschema where the file is so that it
# can correctly resolve relative $ref references in the schema
schema['id'] = fileurl
jsonschema.validate(example, schema)
except Exception as e:
raise ValueError("Error validating JSON schema for %r %r" % (
request, code
), e)
def check_response(filepath, request, code, response):
example = None
try:
example_json = response.get('examples', {}).get('application/json')
if example_json:
example = json.loads(example_json)
except Exception as e:
raise ValueError("Error parsing JSON example response for %r %r" % (
request, code
), e)
schema = response.get('schema')
fileurl = "file://" + os.path.abspath(filepath)
if example and schema:
try:
print ("Checking response schema for: %r %r %r" % (
filepath, request, code
))
# Setting the 'id' tells jsonschema where the file is so that it
# can correctly resolve relative $ref references in the schema
schema['id'] = fileurl
jsonschema.validate(example, schema)
except Exception as e:
raise ValueError("Error validating JSON schema for %r %r" % (
request, code
), e)
def check_swagger_file(filepath):
with open(filepath) as f:
swagger = yaml.load(f)
for path, path_api in swagger.get('paths', {}).items():
for method, request_api in path_api.items():
request = "%s %s" % (method.upper(), path)
for parameter in request_api.get('parameters', ()):
if parameter['in'] == 'body':
check_parameter(filepath, request, parameter)
try:
responses = request_api['responses']
except KeyError:
raise ValueError("No responses for %r" % (request,))
for code, response in responses.items():
check_response(filepath, request, code, response)
if __name__ == '__main__':
paths = sys.argv[1:]
if not paths:
paths = []
for (root, dirs, files) in os.walk(os.curdir):
for filename in files:
if filename.endswith(".yaml"):
paths.append(os.path.join(root, filename))
for path in paths:
try:
check_swagger_file(path)
except Exception as e:
raise ValueError("Error checking file %r" % (path,), e)

@ -15,16 +15,22 @@ paths:
summary: Upload some content to the content repository.
produces: ["application/json"]
parameters:
- in: header
name: Content-Type
type: string
description: The content type of the file being uploaded
x-example: "Content-Type: audio/mpeg"
- in: body
name: content
name: "<content>"
description: The content to be uploaded.
required: true
schema:
type: string
example: "<bytes>"
format: byte
responses:
200:
description: Information about the uploaded content.
description: The MXC URI for the uploaded content.
schema:
type: object
required: ["content_uri"]
@ -32,6 +38,11 @@ paths:
content_uri:
type: string
description: "The MXC URI to the uploaded content."
examples:
"application/json": |-
{
"content_uri": "mxc://example.com/AQwafuaFswefuhsfAFAgsw"
}
"/download/{serverName}/{mediaId}":
get:
summary: "Download content from the content repository."
@ -40,18 +51,27 @@ paths:
- in: path
type: string
name: serverName
x-example: matrix.org
required: true
description: |
The server name from the ``mxc://`` URI (the authoritory component)
- in: path
type: string
name: mediaId
x-example: ascERGshawAWawugaAcauga
required: true
description: |
The media ID from the ``mxc://`` URI (the path component)
responses:
200:
description: "The content downloaded."
description: "The content that was previously uploaded."
headers:
Content-Type:
description: "The content type of the file that was previously uploaded."
type: "string"
Content-Disposition:
description: "The name of the file that was previously uploaded, if set."
type: "string"
schema:
type: file
"/thumbnail/{serverName}/{mediaId}":
@ -63,30 +83,44 @@ paths:
type: string
name: serverName
required: true
x-example: matrix.org
description: |
The server name from the ``mxc://`` URI (the authoritory component)
- in: path
type: string
name: mediaId
x-example: ascERGshawAWawugaAcauga
required: true
description: |
The media ID from the ``mxc://`` URI (the path component)
- in: query
type: integer
x-example: 64
name: width
description: The desired width of the thumbnail.
description: |-
The *desired* width of the thumbnail. The actual thumbnail may not
match the size specified.
- in: query
type: integer
x-example: 64
name: height
description: The desired height of the thumbnail.
description: |-
The *desired* height of the thumbnail. The actual thumbnail may not
match the size specified.
- in: query
type: string
enum: ["crop", "scale"]
name: method
x-example: "scale"
description: The desired resizing method.
responses:
200:
description: "A thumbnail of the requested content."
headers:
Content-Type:
description: "The content type of the thumbnail."
type: "string"
enum: ["image/jpeg", "image/png"]
schema:
type: file

@ -0,0 +1 @@
v1-event-schema/core-event-schema

@ -1,7 +0,0 @@
type: object
description: A Matrix Event
properties:
event_id:
type: string
description: An event ID.
required: ["event_id"]

@ -1,9 +0,0 @@
type: object
description: A Matrix Room Event
properties:
event_id:
type: string
description: An event ID.
room_id:
type: string
required: ["event_id", "room_id"]

@ -1,11 +0,0 @@
type: object
description: A Matrix State Event
properties:
event_id:
type: string
description: An event ID.
room_id:
type: string
state_key:
type: string
required: ["event_id", "room_id", "state_key"]

@ -0,0 +1,147 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v1 Registration and Login API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/api/v1
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/login":
post:
summary: Authenticates the user.
description: |-
Authenticates the user by password, and issues an access token they can
use to authorize themself in subsequent requests.
security:
- accessToken: []
parameters:
- in: body
name: body
schema:
type: object
example: |-
{
"username": "cheeky_monkey",
"password": "ilovebananas"
}
properties:
username:
type: string
description: The fully qualified user ID or just local part of the user ID, to log in.
password:
type: string
description: The user's password.
required: ["username", "password"]
responses:
200:
description: The user has been authenticated.
examples:
application/json: |-
{
"user_id": "@cheeky_monkey:matrix.org",
"access_token": "abc123",
"home_server": "matrix.org"
}
schema:
type: object
properties:
user_id:
type: string
description: The fully-qualified Matrix ID that has been registered.
access_token:
type: string
description: |-
An access token for the account.
This access token can then be used to authorize other requests.
The access token may expire at some point, and if so, it SHOULD come with a ``refresh_token``.
There is no specific error message to indicate that a request has failed because
an access token has expired; instead, if a client has reason to believe its
access token is valid, and it receives an auth error, they should attempt to
refresh for a new token on failure, and retry the request with the new token.
refresh_token:
type: string
# TODO: Work out how to linkify /tokenrefresh
description: |-
(optional) A ``refresh_token`` may be exchanged for a new ``access_token`` using the /tokenrefresh API endpoint.
home_server:
type: string
description: The hostname of the Home Server on which the account has been registered.
403:
description: |-
The login attempt failed. For example, the password may have been incorrect.
examples:
application/json: |-
{"errcode": "M_FORBIDDEN"}
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"
"/tokenrefresh":
post:
summary: Exchanges a refresh token for an access token.
description: |-
Exchanges a refresh token for a new access token.
This is intended to be used if the access token has expired.
security:
- accessToken: []
parameters:
- in: body
name: body
schema:
type: object
example: |-
{
"refresh_token": "a1b2c3"
}
properties:
refresh_token:
type: string
description: The refresh token which was issued by the server.
required: ["refresh_token"]
responses:
200:
description: |-
The refresh token was accepted, and a new access token has been issued.
The passed refresh token is no longer valid and cannot be used.
A new refresh token will have been returned unless some policy does
not allow the user to continue to renew their session.
examples:
application/json: |-
{
"access_token": "bearwithme123",
"refresh_token": "exchangewithme987"
}
schema:
type: object
properties:
access_token:
type: string
description: |-
An access token for the account.
This access token can then be used to authorize other requests.
The access token may expire at some point, and if so, it SHOULD come with a ``refresh_token``.
refresh_token:
type: string
description: (optional) A ``refresh_token`` may be exchanged for a new ``access_token`` using the TODO Linkify /tokenrefresh API endpoint.
403:
description: |-
The exchange attempt failed. For example, the refresh token may have already been used.
examples:
application/json: |-
{"errcode": "M_FORBIDDEN"}
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"

@ -0,0 +1,128 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v1 Room Membership API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/api/v1
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/rooms/{roomId}/join":
post:
summary: Start the requesting user participating in a particular room.
description: |-
This API starts a user participating in a particular room, if that user
is allowed to participate in that room. After this call, the client is
allowed to see all current state events in the room, and all subsequent
events associated with the room until the user leaves the room.
After a user has joined a room, the room will appear as an entry in the
response of the |initialSync| API.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room identifier or room alias to join.
required: true
x-example: "#monkeys:matrix.org"
responses:
200:
description: |-
The room has been joined.
The joined room ID must be returned in the ``room_id`` field.
examples:
application/json: |-
{"room_id": "!d41d8cd:matrix.org"}
schema:
type: object
403:
description: |-
You do not have permission to join the room. A meaningful ``errcode`` and description error text will be returned. Example reasons for rejection are:
- The room is invite-only and the user was not invited.
- The user has been banned from the room.
examples:
application/json: |-
{"errcode": "M_FORBIDDEN", "error": "You are not invited to this room."}
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"
x-alias:
canonical-link: "post-matrix-client-api-v1-rooms-roomid-join"
aliases:
- /join/{roomId}
"/rooms/{roomId}/invite":
post:
summary: Invite a user to participate in a particular room.
description: |-
This API invites a user to participate in a particular room.
They do not start participating in the room until they actually join the
room.
This serves two purposes; firstly, to notify the user that the room
exists (and that their presence is requested). Secondly, some rooms can
only be joined if a user is invited to join it; sending the invite gives
that user permission to join the room.
Only users currently in a particular room can invite other users to
join that room.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room identifier (not alias) to which to invite the user.
required: true
x-example: "!d41d8cd:matrix.org"
- in: body
name: user_id
required: true
schema:
type: object
example: |-
{
"user_id": "@cheeky_monkey:matrix.org"
}
properties:
user_id:
type: string
description: The fully qualified user ID of the invitee.
required: ["user_id"]
responses:
200:
description: The user has been invited to join the room.
examples:
application/json: |-
{}
schema:
type: object # empty json object
403:
description: |-
You do not have permission to invite the user to the room. A meaningful ``errcode`` and description error text will be returned. Example reasons for rejections are:
- The invitee has been banned from the room.
- The invitee is already a member of the room.
- The inviter is not currently in the room.
- The inviter's power level is insufficient to invite users to the room.
examples:
application/json: |-
{"errcode": "M_FORBIDDEN", "error": "@cheeky_monkey:matrix.org is banned from the room"}
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"

@ -101,7 +101,7 @@ paths:
The length of time in milliseconds since an action was performed
by this user.
status_msg:
type: string
type: [string, "null"]
description: The state message for this user if one was set.
404:
description: |-
@ -185,7 +185,7 @@ paths:
"last_active_ago": 395,
"presence": "offline",
"user_id": "@alice:matrix.org"
}
},
"type": "m.presence"
},
{
@ -195,7 +195,7 @@ paths:
"last_active_ago": 16874,
"presence": "online",
"user_id": "@marisa:matrix.org"
}
},
"type": "m.presence"
}
]
@ -205,5 +205,4 @@ paths:
type: object
title: PresenceEvent
allOf:
- "$ref": "definitions/event.yaml"
- "$ref": "core-event-schema/event.json"

@ -0,0 +1,448 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v1 Rooms API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/api/v1
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/rooms/{roomId}/state/{eventType}/{stateKey}":
get:
summary: Get the state identified by the type and key.
description: |-
Looks up the contents of a state event in a room. If the user is
joined to the room then the state is taken from the current
state of the room. If the user has left the room then the state is
taken from the state of the room when they left.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room to look up the state in.
required: true
x-example: "!636q39766251:example.com"
- in: path
type: string
name: eventType
description: The type of state to look up.
required: true
x-example: "m.room.name"
- in: path
type: string
name: stateKey
description: The key of the state to look up. Defaults to the empty string.
required: true
x-example: ""
responses:
200:
description: The content of the state event.
examples:
application/json: |-
{"name": "Example room name"}
schema:
type: object
404:
description: The room has no state with the given type or key.
403:
description: >
You aren't a member of the room and weren't previously a
member of the room.
"/rooms/{roomId}/state":
get:
summary: Get all state events in the current state of a room.
description: |-
Get the state events for the current state of a room.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room to look up the state for.
required: true
x-example: "!636q39766251:example.com"
responses:
200:
description: The current state of the room
examples:
application/json: |-
[
{
"age": 7148266897,
"content": {
"join_rule": "public"
},
"event_id": "$14259997323TLwtb:example.com",
"origin_server_ts": 1425999732392,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.join_rules",
"user_id": "@alice:example.com"
},
{
"age": 6547561012,
"content": {
"avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto",
"displayname": null,
"membership": "join"
},
"event_id": "$1426600438280zExKY:example.com",
"membership": "join",
"origin_server_ts": 1426600438277,
"room_id": "!636q39766251:example.com",
"state_key": "@alice:example.com",
"type": "m.room.member",
"user_id": "@alice:example.com"
},
{
"age": 7148267200,
"content": {
"creator": "@alice:example.com"
},
"event_id": "$14259997320KhbwJ:example.com",
"origin_server_ts": 1425999732089,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.create",
"user_id": "@alice:example.com"
},
{
"age": 1622568720,
"content": {
"avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto",
"displayname": "Bob",
"membership": "join"
},
"event_id": "$1431525430134MxlLX:example.com",
"origin_server_ts": 1431525430569,
"replaces_state": "$142652023736BSXcM:example.com",
"room_id": "!636q39766251:example.com",
"state_key": "@bob:example.com",
"type": "m.room.member",
"user_id": "@bob:example.com"
},
{
"age": 7148267004,
"content": {
"ban": 50,
"events": {
"m.room.name": 100,
"m.room.power_levels": 100
},
"events_default": 0,
"kick": 50,
"redact": 50,
"state_default": 50,
"users": {
"@alice:example.com": 100
},
"users_default": 0
},
"event_id": "$14259997322mqfaq:example.com",
"origin_server_ts": 1425999732285,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.power_levels",
"user_id": "@alice:example.com"
}
]
schema:
type: array
title: RoomState
description: |-
If the user is a member of the room this will be the
current state of the room as a list of events. If the user
has left the room then this will be the state of the room
when they left as a list of events.
items:
title: StateEvent
type: object
allOf:
- "$ref": "core-event-schema/state_event.json"
403:
description: >
You aren't a member of the room and weren't previously a
member of the room.
"/rooms/{roomId}/initialSync":
get:
summary: Snapshot the current state of a room and its most recent messages.
description: |-
Get a copy of the current state and the most recent messages in a room.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room to get the data.
required: true
x-example: "!636q39766251:example.com"
responses:
200:
description: The current state of the room
examples:
application/json: |-
{
"membership": "join",
"messages": {
"chunk": [
{
"age": 343513403,
"content": {
"body": "foo",
"msgtype": "m.text"
},
"event_id": "$14328044851tzTJS:example.com",
"origin_server_ts": 1432804485886,
"room_id": "!636q39766251:example.com",
"type": "m.room.message",
"user_id": "@alice:example.com"
},
{
"age": 343511809,
"content": {
"body": "bar",
"msgtype": "m.text"
},
"event_id": "$14328044872spjFg:example.com",
"origin_server_ts": 1432804487480,
"room_id": "!636q39766251:example.com",
"type": "m.room.message",
"user_id": "@bob:example.com"
}
],
"end": "s3456_9_0",
"start": "t44-3453_9_0"
},
"room_id": "!636q39766251:example.com",
"state": [
{
"age": 7148266897,
"content": {
"join_rule": "public"
},
"event_id": "$14259997323TLwtb:example.com",
"origin_server_ts": 1425999732392,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.join_rules",
"user_id": "@alice:example.com"
},
{
"age": 6547561012,
"content": {
"avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto",
"displayname": null,
"membership": "join"
},
"event_id": "$1426600438280zExKY:example.com",
"membership": "join",
"origin_server_ts": 1426600438277,
"room_id": "!636q39766251:example.com",
"state_key": "@alice:example.com",
"type": "m.room.member",
"user_id": "@alice:example.com"
},
{
"age": 7148267200,
"content": {
"creator": "@alice:example.com"
},
"event_id": "$14259997320KhbwJ:example.com",
"origin_server_ts": 1425999732089,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.create",
"user_id": "@alice:example.com"
},
{
"age": 1622568720,
"content": {
"avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto",
"displayname": "Bob",
"membership": "join"
},
"event_id": "$1431525430134MxlLX:example.com",
"origin_server_ts": 1431525430569,
"replaces_state": "$142652023736BSXcM:example.com",
"room_id": "!636q39766251:example.com",
"state_key": "@bob:example.com",
"type": "m.room.member",
"user_id": "@bob:example.com"
},
{
"age": 7148267004,
"content": {
"ban": 50,
"events": {
"m.room.name": 100,
"m.room.power_levels": 100
},
"events_default": 0,
"kick": 50,
"redact": 50,
"state_default": 50,
"users": {
"@alice:example.com": 100
},
"users_default": 0
},
"event_id": "$14259997322mqfaq:example.com",
"origin_server_ts": 1425999732285,
"room_id": "!636q39766251:example.com",
"state_key": "",
"type": "m.room.power_levels",
"user_id": "@alice:example.com"
}
],
"visibility": "private"
}
schema:
title: RoomInfo
type: object
properties:
room_id:
type: string
description: "The ID of this room."
membership:
type: string
description: "The user's membership state in this room."
enum: ["invite", "join", "leave", "ban"]
messages:
type: object
title: PaginationChunk
description: "The pagination chunk for this room."
properties:
start:
type: string
description: |-
A token which correlates to the first value in ``chunk``.
Used for pagination.
end:
type: string
description: |-
A token which correlates to the last value in ``chunk``.
Used for pagination.
chunk:
type: array
description: |-
If the user is a member of the room this will be a
list of the most recent messages for this room. If
the user has left the room this will be the
messages that preceeded them leaving. This array
will consist of at most ``limit`` elements.
items:
type: object
title: RoomEvent
allOf:
- "$ref": "core-event-schema/room_event.json"
required: ["start", "end", "chunk"]
state:
type: array
description: |-
If the user is a member of the room this will be the
current state of the room as a list of events. If the
user has left the room this will be the state of the
room when they left it.
items:
title: StateEvent
type: object
allOf:
- "$ref": "core-event-schema/state_event.json"
visibility:
type: string
enum: ["private", "public"]
description: |-
Whether this room is visible to the ``/publicRooms`` API
or not."
required: ["room_id", "membership"]
403:
description: >
You aren't a member of the room and weren't previously a
member of the room.
"/rooms/{roomId}/members":
get:
summary: Get the m.room.member events for the room.
description:
Get the list of members for this room.
parameters:
- in: path
type: string
name: roomId
description: The room to get the member events for.
required: true
x-example: "!636q39766251:example.com"
responses:
200:
description: |-
A list of members of the room. If you are joined to the room then
this will be the current members of the room. If you have left te
room then this will be the members of the room when you left.
examples:
application/json: |-
{
"chunk": [
{
"age": 6547561012,
"content": {
"avatar_url": "mxc://example.com/fzysBrHpPEeTGANCVLXWXNMI#auto",
"displayname": null,
"membership": "join"
},
"event_id": "$1426600438280zExKY:example.com",
"membership": "join",
"origin_server_ts": 1426600438277,
"room_id": "!636q39766251:example.com",
"state_key": "@alice:example.com",
"type": "m.room.member",
"user_id": "@alice:example.com"
},
{
"age": 1622568720,
"content": {
"avatar_url": "mxc://example.com/GCmhgzMPRjqgpODLsNQzVuHZ#auto",
"displayname": "Bob",
"membership": "join"
},
"event_id": "$1431525430134MxlLX:example.com",
"origin_server_ts": 1431525430569,
"replaces_state": "$142652023736BSXcM:example.com",
"room_id": "!636q39766251:example.com",
"state_key": "@bob:example.com",
"type": "m.room.member",
"user_id": "@bob:example.com"
}
]
}
schema:
type: object
properties:
chunk:
type: array
items:
title: MemberEvent
type: object
allOf:
- "$ref": "v1-event-schema/m.room.member"
403:
description: >
You aren't a member of the room and weren't previously a
member of the room.

@ -82,7 +82,7 @@ paths:
type: object
title: RoomEvent
allOf:
- "$ref": "definitions/room_event.yaml"
- "$ref": "core-event-schema/room_event.json"
400:
description: "Bad pagination ``from`` parameter."
"/initialSync":
@ -253,7 +253,7 @@ paths:
type: object
title: Event
allOf:
- "$ref": "definitions/event.yaml"
- "$ref": "core-event-schema/event.json"
rooms:
type: array
items:
@ -267,6 +267,12 @@ paths:
type: string
description: "The user's membership state in this room."
enum: ["invite", "join", "leave", "ban"]
invite:
type: object
title: "InviteEvent"
description: "The invite event if ``membership`` is ``invite``"
allOf:
- "$ref": "v1-event-schema/m.room.member"
messages:
type: object
title: PaginationChunk
@ -285,24 +291,29 @@ paths:
chunk:
type: array
description: |-
A list of the most recent messages for this room. This
array will consist of at most ``limit`` elements.
If the user is a member of the room this will be a
list of the most recent messages for this room. If
the user has left the room this will be the
messages that preceeded them leaving. This array
will consist of at most ``limit`` elements.
items:
type: object
title: RoomEvent
allOf:
- "$ref": "definitions/room_event.yaml"
- "$ref": "core-event-schema/room_event.json"
required: ["start", "end", "chunk"]
state:
type: array
description: |-
A list of state events representing the current state
of the room.
If the user is a member of the room this will be the
current state of the room as a list of events. If the
user has left the room this will be the state of the
room when they left it.
items:
title: StateEvent
type: object
allOf:
- "$ref": "definitions/state_event.yaml"
- "$ref": "core-event-schema/state_event.json"
visibility:
type: string
enum: ["private", "public"]
@ -338,13 +349,13 @@ paths:
"body": "Hello world!",
"msgtype": "m.text"
},
"room_id:" "!wfgy43Sg4a:matrix.org",
"room_id:": "!wfgy43Sg4a:matrix.org",
"user_id": "@bob:matrix.org",
"event_id": "$asfDuShaf7Gafaw:matrix.org",
"type": "m.room.message"
}
schema:
allOf:
- "$ref": "definitions/event.yaml"
- "$ref": "core-event-schema/event.json"
404:
description: The event was not found or you do not have permission to read this event.
description: The event was not found or you do not have permission to read this event.

@ -0,0 +1,77 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v1 Typing API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/api/v1
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/rooms/{roomId}/typing/{userId}":
put:
summary: Informs the server that the user has started or stopped typing.
description: |-
This tells the server that the user is typing for the next N
milliseconds where N is the value specified in the ``timeout`` key.
Alternatively, if ``typing`` is ``false``, it tells the server that the
user has stopped typing.
security:
- accessToken: []
parameters:
- in: path
type: string
name: userId
description: The user who has started to type.
required: true
x-example: "@alice:example.com"
- in: path
type: string
name: roomId
description: The room in which the user is typing.
required: true
x-example: "!wefh3sfukhs:example.com"
- in: body
name: typingState
description: The current typing state.
required: true
schema:
type: object
example: |-
{
"typing": true,
"timeout": 30000
}
properties:
typing:
type: boolean
description: |-
Whether the user is typing or not. If ``false``, the ``timeout``
key can be omitted.
timeout:
type: integer
description: The length of time in milliseconds to mark this user as typing.
required: ["typing"]
responses:
200:
description: The new typing state was set.
examples:
application/json: |-
{}
schema:
type: object # empty json object
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"

@ -0,0 +1 @@
../../../event-schemas/schema/v1

@ -0,0 +1,68 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v1 Voice over IP API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/api/v1
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/turnServer":
get:
summary: Obtain TURN server credentials.
description: |-
This API provides credentials for the client to use when initiating
calls.
security:
- accessToken: []
responses:
200:
description: The TURN server credentials.
examples:
application/json: |-
{
"username":"1443779631:@user:example.com",
"password":"JlKfBy1QwLrO20385QyAtEyIv0=",
"uris":[
"turn:turn.example.com:3478?transport=udp",
"turn:10.20.30.40:3478?transport=tcp",
"turns:10.20.30.40:443?transport=tcp"
],
"ttl":86400
}
schema:
type: object
properties:
username:
type: string
description: |-
The username to use.
password:
type: string
description: |-
The password to use.
uris:
type: array
items:
type: string
description: A list of TURN URIs
ttl:
type: integer
description: The time-to-live in seconds
required: ["username", "password", "uris", "ttl"]
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"

@ -0,0 +1,12 @@
{
"type": "object",
"properties": {
"events": {
"type": "array",
"description": "List of events",
"items": {
"type": "object"
}
}
}
}

@ -0,0 +1,42 @@
{
"type": "object",
"properties": {
"limit": {
"type": "integer",
"description":
"The maximum number of events to return."
},
"types": {
"type": "array",
"description":
"A list of event types to include. If this list is absent then all event types are included. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
},
"not_types": {
"type": "array",
"description":
"A list of event types to exclude. If this list is absent then no event types are excluded. A matching type will be excluded even if it is listed in the 'types' filter. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
},
"senders": {
"type": "array",
"description":
"A list of senders IDs to include. If this list is absent then all senders are included. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
},
"not_senders": {
"type": "array",
"description":
"A list of sender IDs to exclude. If this list is absent then no senders are excluded. A matching sender will be excluded even if it is listed in the 'senders' filter. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
}
}
}

@ -0,0 +1,12 @@
{
"type": "object",
"properties": {
"events": {
"type": "array",
"description": "List of event ids",
"items": {
"type": "string"
}
}
}
}

@ -0,0 +1,22 @@
{
"type": "object",
"allOf": [{"$ref": "definitions/event_filter.json"}],
"properties": {
"rooms": {
"type": "array",
"description":
"A list of room IDs to include. If this list is absent then all rooms are included. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
},
"not_rooms": {
"type": "array",
"description":
"A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the 'rooms' filter. A '*' can be used as a wildcard to match any sequence of characters.",
"items": {
"type": "string"
}
}
}
}

@ -0,0 +1,44 @@
{
"type": "object",
"properties": {
"room": {
"type": "object",
"properties": {
"state": {
"description":
"The state events to include for rooms.",
"allOf": [{"$ref": "definitions/room_event_filter.json"}]
},
"timeline": {
"description":
"The message and state update events to include for rooms.",
"allOf": [{"$ref": "definitions/room_event_filter.json"}]
},
"ephemeral": {
"description":
"The events that aren't recorded in the room history, e.g. typing and receipts, to include for rooms.",
"allOf": [{"$ref": "definitions/room_event_filter.json"}]
}
}
},
"presence": {
"description":
"The presence updates to include.",
"allOf": [{"$ref": "definitions/event_filter.json"}]
},
"event_format": {
"description":
"The format to use for events. 'client' will return the events in a format suitable for clients. 'federation' will return the raw event as receieved over federation. The default is 'client'.",
"type": "string",
"enum": ["client", "federation"]
},
"event_fields": {
"type": "array",
"description":
"List of event fields to include. If this list is absent then all fields are included. The entries may include '.' charaters to indicate sub-fields. So ['content.body'] will include the 'body' field of the 'content' object. A literal '.' character in a field name may be escaped using a '\\'. A server may include more fields than were requested.",
"items": {
"type": "string"
}
}
}
}

@ -0,0 +1,14 @@
{
"type": "object",
"allOf": [{"$ref":"definitions/room_event_batch.json"}],
"properties": {
"limited": {
"type": "boolean",
"description": "Whether there are more events on the server"
},
"prev_batch": {
"type": "string",
"description": "If the batch was limited then this is a token that can be supplied to the server to retrieve more events"
}
}
}

@ -0,0 +1,139 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v2 filter API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
basePath: /_matrix/client/v2_alpha
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/user/{userId}/filter":
post:
summary: Upload a new filter.
description: |-
Uploads a new filter definition to the homeserver.
Returns a filter ID that may be used in /sync requests to
retrict which events are returned to the client.
security:
- accessToken: []
parameters:
- in: path
type: string
name: userId
required: true
description:
The id of the user uploading the filter. The access token must be
authorized to make requests for this user id.
x-example: "@alice:example.com"
- in: body
name: filter
required: true
description: The filter to upload.
schema:
type: object
allOf:
- $ref: "definitions/sync_filter.json"
example: |-
{
"room": {
"state": {
"types": ["m.room.*"],
"not_rooms": ["!726s6s6q:example.com"]
},
"timeline": {
"limit": 10,
"types": ["m.room.message"],
"not_rooms": ["!726s6s6q:example.com"],
"not_senders": ["@spam:example.com"]
},
"emphemeral": {
"types": ["m.receipt", "m.typing"],
"not_rooms": ["!726s6s6q:example.com"],
"not_senders": ["@spam:example.com"]
}
},
"presence": {
"types": ["m.presence"],
"not_senders": ["@alice:example.com"]
},
"event_format": "client",
"event_fields": ["type", "content", "sender"]
}
responses:
200:
description: The filter was created.
examples:
application/json: |-
{
"filter_id": "66696p746572"
}
schema:
type: object
properties:
filter_id:
type: string
description: |-
The ID of the filter that was created.
"/user/{userId}/filter/{filterId}":
get:
summary: Download a filter
parameters:
- in: path
name: userId
type: string
description: |-
The user ID to download a filter for.
x-example: "@alice:example.com"
required: true
- in: path
name: filterId
type: string
description: |-
The filter ID to download.
x-example: "66696p746572"
required: true
responses:
200:
description: |-
"The filter defintion"
examples:
application/json: |-
{
"room": {
"state": {
"types": ["m.room.*"],
"not_rooms": ["!726s6s6q:example.com"]
},
"timeline": {
"limit": 10,
"types": ["m.room.message"],
"not_rooms": ["!726s6s6q:example.com"],
"not_senders": ["@spam:example.com"]
},
"emphemeral": {
"types": ["m.receipt", "m.typing"],
"not_rooms": ["!726s6s6q:example.com"],
"not_senders": ["@spam:example.com"]
}
},
"presence": {
"types": ["m.presence"],
"not_senders": ["@alice:example.com"]
},
"event_format": "client",
"event_fields": ["type", "content", "sender"]
}
schema:
type: object
allOf:
- $ref: "definitions/sync_filter.json"

@ -0,0 +1,68 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v2 Receipts API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
- http
basePath: /_matrix/client/v2_alpha
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/rooms/{roomId}/receipt/{receiptType}/{eventId}":
post:
summary: Send a receipt for the given event ID.
description: |-
This API updates the marker for the given receipt type to the event ID
specified.
security:
- accessToken: []
parameters:
- in: path
type: string
name: roomId
description: The room in which to send the event.
required: true
x-example: "!wefuh21ffskfuh345:example.com"
- in: path
type: string
name: receiptType
description: The type of receipt to send.
required: true
x-example: "m.read"
enum: ["m.read"]
- in: path
type: string
name: eventId
description: The event ID to acknowledge up to.
required: true
x-example: "$1924376522eioj:example.com"
- in: body
description: |-
Extra receipt information to attach to ``content`` if any. The
server will automatically set the ``ts`` field.
schema:
type: object
example: |-
{}
responses:
200:
description: The receipt was sent.
examples:
application/json: |-
{}
schema:
type: object # empty json object
429:
description: This request was rate-limited.
schema:
"$ref": "definitions/error.yaml"

@ -0,0 +1,257 @@
swagger: '2.0'
info:
title: "Matrix Client-Server v2 sync API"
version: "1.0.0"
host: localhost:8008
schemes:
- https
basePath: /_matrix/client/v2_alpha
consumes:
- application/json
produces:
- application/json
securityDefinitions:
accessToken:
type: apiKey
description: The user_id or application service access_token
name: access_token
in: query
paths:
"/sync":
get:
summary: Synchronise the client's state and receive new messages.
description: |-
Synchronise the client's state with the latest state on the server.
Client's use this API when they first log in to get an initial snapshot
of the state on the server, and then continue to call this API to get
incremental deltas to the state, and to receive new messages.
security:
- accessToken: []
parameters:
- in: query
name: filter
type: string
description: |-
The ID of a filter created using the filter API.
x-example: "66696p746572"
- in: query
name: since
type: string
description: |-
A point in time to continue a sync from.
x-example: "s72594_4483_1934"
- in: query
name: set_presence
type: string
enum: ["offline"]
description: |-
Controls whether the client is automatically marked as online by
polling this API. If this parameter is omitted then the client is
automatically marked as online when it uses this API. Otherwise if
the parameter is set to "offline" then the client is not marked as
being online when it uses this API.
x-example: "offline"
- in: query
name: timeout
type: integer
description: |-
The maximum time to poll in milliseconds before returning this
request.
x-example: 30000
responses:
200:
description:
The initial snapshot or delta for the client to use to update their
state.
schema:
type: object
properties:
next_batch:
type: string
description: |-
The batch token to supply in the ``since`` param of the next
``/sync`` request.
rooms:
type: object
description: |-
Updates to rooms.
properties:
joined:
type: object
additionalProperties:
type: object
properties:
event_map:
type: object
description: |-
A map from event ID to events for this room. The
events are referenced from the ``timeline`` and
``state`` keys for this room.
additionalProperties:
description: An event object.
type: object
state:
description: |-
The state updates for the room.
allOf:
- $ref: "definitions/room_event_batch.json"
timeline:
description: |-
The timeline of messages and state changes in the
room.
allOf:
- $ref: "definitions/timeline_batch.json"
ephemeral:
description: |-
The ephemeral events in the room that aren't
recorded in the timeline or state of the room.
e.g. typing.
allOf:
- $ref: "definitions/event_batch.json"
invited:
type: object
description: |-
The rooms that the user has been invited to.
additionalProperties:
type: object
properties:
invite_state:
description: |-
The state of a room that the user has been invited
to. These state events may only have the `sender``,
``type``, ``state_key`` and ``content`` keys
present. These events do not replace any state that
the client already has for the room, for example if
the client has archived the room. Instead the
client should keep two separate copies of the
state: the one from the ``invite_state`` and one
from the archived ``state``. If the client joins
the room then the current state will be given as a
delta against the archived ``state`` not the
``invite_state``.
allOf:
- $ref: "definitions/event_batch.json"
archived:
type: object
description: |-
The rooms that the user has left or been banned from. The
entries in the room_map will lack an ``ephemeral`` key.
additionalProperties:
type: object
properties:
event_map:
type: object
description: |-
A map from event ID to events for this room. The
events are referenced from the ``timeline`` and
``state`` keys for this room.
additionalProperties:
description: An event object.
type: object
state:
description: |-
The state updates for the room up to the point when
the user left.
allOf:
- $ref: "definitions/room_event_batch.json"
timeline:
description: |-
The timeline of messages and state changes in the
room up to the point when the user left.
allOf:
- $ref: "definitions/timeline_batch.json"
presence:
description: |-
The updates to the presence status of other users.
allOf:
- $ref: "definitions/event_batch.json"
examples:
application/json: |-
{
"next_batch": "s72595_4483_1934",
"presence": {
"events": [
{
"sender": "@alice:example.com",
"type": "m.presence",
"content": {"presence": "online"}
}
]
},
"rooms": {
"joined": {
"!726s6s6q:example.com": {
"event_map": {
"$66697273743031:example.com": {
"sender": "@alice:example.com",
"type": "m.room.member",
"state_key": "@alice:example.com",
"content": {"membership": "join"},
"origin_server_ts": 1417731086795
},
"$7365636s6r6432:example.com": {
"sender": "@bob:example.com",
"type": "m.room.member",
"state_key": "@bob:example.com",
"content": {"membership": "join"},
"origin_server_ts": 1417731086795
},
"$74686972643033:example.com": {
"sender": "@alice:example.com",
"type": "m.room.message",
"unsigned": {"age": "124524", "txn_id": "1234"},
"content": {
"body": "I am a fish",
"msgtype": "m.text"
},
"origin_server_ts": 1417731086797
}
},
"state": {
"events": [
"$66697273743031:example.com",
"$7365636s6r6432:example.com"
]
},
"timeline": {
"events": [
"$7365636s6r6432:example.com",
"$74686972643033:example.com"
],
"limited": true,
"prev_batch": "t34-23535_0_0"
},
"ephemeral": {
"events": [
{
"room_id": "!726s6s6q:example.com",
"type": "m.typing",
"content": {"user_ids": ["@alice:example.com"]}
}
]
}
}
},
"invited": {
"!696r7674:example.com": {
"invite_state": {
"events": [
{
"sender": "@alice:example.com",
"type": "m.room.name",
"state_key": "",
"content": {"name": "My Room Name"}
},
{
"sender": "@alice:example.com",
"type": "m.room.member",
"state_key": "@bob:example.com",
"content": {"membership": "invite"}
}
]
}
}
},
"archived": {}
}
}

@ -10,6 +10,6 @@
"license": "ISC",
"dependencies": {
"nopt": "^3.0.2",
"swagger-parser": "^2.4.1"
"swagger-parser": "^3.2.1"
}
}

@ -26,11 +26,10 @@ if (!opts.schema) {
}
var errFn = function(err, api, metadata) {
var errFn = function(err, api) {
if (!err) {
return;
}
console.log(metadata);
console.error(err);
process.exit(1);
};
@ -44,12 +43,14 @@ if (isDir) {
process.exit(1);
}
files.forEach(function(f) {
if (f.indexOf(".yaml") > 0) {
parser.parse(path.join(opts.schema, f), function(err, api, metadata) {
var suffix = ".yaml";
if (f.indexOf(suffix, f.length - suffix.length) > 0) {
parser.validate(path.join(opts.schema, f), function(err, api, metadata) {
if (!err) {
console.log("%s is valid.", f);
}
else {
console.error("%s is not valid.", f);
errFn(err, api, metadata);
}
});
@ -58,12 +59,12 @@ if (isDir) {
});
}
else{
parser.parse(opts.schema, function(err, api, metadata) {
parser.validate(opts.schema, function(err, api) {
if (!err) {
console.log("%s is valid", opts.schema);
}
else {
errFn(err, api, metadata);
errFn(err, api);
}
});
};

@ -3,10 +3,10 @@ Address book repository
.. NOTE::
This section is a work in progress.
Do we even need it? Clients can use out-of-band addressbook servers for now;
this should definitely not be core.
.. TODO-spec
Do we even need it? Clients can use out-of-band addressbook servers for now;
this should definitely not be core.
- format: POST(?) wodges of json, some possible processing, then return wodges of json on GET.
- processing may remove dupes, merge contacts, pepper with extra info (e.g. matrix-ability of
contacts), etc.

@ -0,0 +1,34 @@
Macaroon Caveats
================
`Macaroons`_ are issued by Matrix servers as authorization tokens. Macaroons may be restricted by adding caveats to them.
.. _Macaroons: http://theory.stanford.edu/~ataly/Papers/macaroons.pdf
Caveats can only be used for reducing the scope of a token, never for increasing it. Servers are required to reject any macroon with a caveat that they do not understand.
Some caveats are specified in this specification, and must be understood by all servers. The use of non-standard caveats is allowed.
All caveats must take the form:
`key` `operator` `value`
where `key` is a non-empty string drawn from the character set [A-Za-z0-9_]
`operator` is a non-empty string which does not contain whitespace
`value` is a non-empty string
And these are joined by single space characters.
Specified caveats:
+-------------+--------------------------------------------------+------------------------------------------------------------------------------------------------+
| Caveat name | Description | Legal Values |
+-------------+--------------------------------------------------+------------------------------------------------------------------------------------------------+
| gen | Generation of the macaroon caveat spec. | 1 |
| user_id | ID of the user for which this macaroon is valid. | Pure equality check. Operator must be =. |
| type | The purpose of this macaroon. | access - used to authorize any action except token refresh |
| refresh - only used to authorize a token refresh |
| time | Time before/after which this macaroon is valid. | A POSIX timestamp in milliseconds (in UTC). |
| Operator < means the macaroon is valid before the timestamp, as interpreted by the server. |
| Operator > means the macaroon is valid after the timestamp, as interpreted by the server. |
| Operator == means the macaroon is valid at exactly the timestamp, as interpreted by the server.|
| Note that exact equality of time is largely meaningless. |
+-------------+--------------------------------------------------+------------------------------------------------------------------------------------------------+

@ -0,0 +1,76 @@
#! /usr/bin/env python
import sys
import json
import os
import traceback
def import_error(module, package, debian, error):
sys.stderr.write((
"Error importing %(module)s: %(error)r\n"
"To install %(module)s run:\n"
" pip install %(package)s\n"
"or on Debian run:\n"
" sudo apt-get install python-%(debian)s\n"
) % locals())
if __name__ == '__main__':
sys.exit(1)
try:
import jsonschema
except ImportError as e:
import_error("jsonschema", "jsonschema", "jsonschema", e)
raise
try:
import yaml
except ImportError as e:
import_error("yaml", "PyYAML", "yaml", e)
raise
def check_example_file(examplepath, schemapath):
with open(examplepath) as f:
example = yaml.load(f)
with open(schemapath) as f:
schema = yaml.load(f)
fileurl = "file://" + os.path.abspath(schemapath)
print ("Checking schema for: %r %r" % (examplepath, schemapath))
# Setting the 'id' tells jsonschema where the file is so that it
# can correctly resolve relative $ref references in the schema
schema['id'] = fileurl
try:
jsonschema.validate(example, schema)
except Exception as e:
raise ValueError("Error validating JSON schema for %r %r" % (
examplepath, schemapath
), e)
def check_example_dir(exampledir, schemadir):
errors = []
for root, dirs, files in os.walk(exampledir):
for filename in files:
if filename.startswith("."):
# Skip over any vim .swp files.
continue
examplepath = os.path.join(root, filename)
schemapath = examplepath.replace(exampledir, schemadir)
try:
check_example_file(examplepath, schemapath)
except Exception as e:
errors.append(sys.exc_info())
for (exc_type, exc_value, exc_trace) in errors:
traceback.print_exception(exc_type, exc_value, exc_trace)
if errors:
raise ValueError("Error validating examples")
if __name__ == '__main__':
try:
check_example_dir("examples", "schema")
except:
sys.exit(1)

@ -0,0 +1,13 @@
{
"type": "m.receipt",
"room_id": "!KpjVgQyZpzBwvMBsnT:matrix.org",
"content": {
"$1435641916114394fHBLK:matrix.org": {
"m.read": {
"@rikj:jki.re": {
"ts": 1436451550453
}
}
}
}
}

@ -0,0 +1,7 @@
{
"type": "m.typing",
"room_id": "!z0mnsuiwhifuhwwfw:matrix.org",
"content": {
"user_ids": ["@alice:matrix.org", "@bob:example.com"]
}
}

@ -1,88 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"event": {
"title": "Event",
"description": "The basic set of fields all events must have.",
"type": "object",
"properties": {
"event_id": {
"type": "string",
"description": "The globally unique event identifier."
},
"user_id": {
"type": "string",
"description": "Contains the fully-qualified ID of the user who *sent* this event."
},
"content": {
"type": "object",
"description": "The fields in this object will vary depending on the type of event. When interacting with the REST API, this is the HTTP body."
},
"type": {
"type": "string",
"description": "The type of event. This SHOULD be namespaced similar to Java package naming conventions e.g. 'com.example.subdomain.event.type'"
}
},
"required": ["event_id", "user_id", "content", "type"]
},
"room_event": {
"type": "object",
"title": "Room Event",
"description": "In addition to the Event fields, Room Events MUST have the following additional field.",
"allOf":[{
"$ref": "#/definitions/event"
}],
"properties": {
"room_id": {
"type": "string",
"description": "The ID of the room associated with this event."
}
},
"required": ["room_id"]
},
"state_event": {
"type": "object",
"title": "State Event",
"description": "In addition to the Room Event fields, State Events have the following additional fields.",
"allOf":[{
"$ref": "#/definitions/room_event"
}],
"properties": {
"state_key": {
"type": "string",
"description": "A unique key which defines the overwriting semantics for this piece of room state. This value is often a zero-length string. The presence of this key makes this event a State Event."
},
"prev_content": {
"type": "object",
"description": "Optional. The previous ``content`` for this event. If there is no previous content, this key will be missing."
}
},
"required": ["state_key"]
},
"msgtype_infos": {
"image_info": {
"type": "object",
"title": "ImageInfo",
"description": "Metadata about an image.",
"properties": {
"size": {
"type": "integer",
"description": "Size of the image in bytes."
},
"w": {
"type": "integer",
"description": "The width of the image in pixels."
},
"h": {
"type": "integer",
"description": "The height of the image in pixels."
},
"mimetype": {
"type": "string",
"description": "The mimetype of the image, e.g. ``image/jpeg``."
}
}
}
}
}
}

@ -0,0 +1,15 @@
{
"type": "object",
"title": "Event",
"description": "The basic set of fields all events must have.",
"properties": {
"content": {
"type": "object",
"description": "The fields in this object will vary depending on the type of event. When interacting with the REST API, this is the HTTP body."
},
"type": {
"type": "string",
"description": "The type of event. This SHOULD be namespaced similar to Java package naming conventions e.g. 'com.example.subdomain.event.type'"
}
}
}

@ -0,0 +1,23 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "ImageInfo",
"description": "Metadata about an image.",
"properties": {
"size": {
"type": "integer",
"description": "Size of the image in bytes."
},
"w": {
"type": "integer",
"description": "The width of the image in pixels."
},
"h": {
"type": "integer",
"description": "The height of the image in pixels."
},
"mimetype": {
"type": "string",
"description": "The mimetype of the image, e.g. ``image/jpeg``."
}
}
}

@ -0,0 +1,23 @@
{
"type": "object",
"title": "Room Event",
"description": "In addition to the Event fields, Room Events MUST have the following additional field.",
"allOf":[{
"$ref": "core-event-schema/event.json"
}],
"properties": {
"room_id": {
"type": "string",
"description": "The ID of the room associated with this event."
},
"event_id": {
"type": "string",
"description": "The globally unique event identifier."
},
"user_id": {
"type": "string",
"description": "Contains the fully-qualified ID of the user who *sent* this event."
}
},
"required": ["room_id"]
}

@ -0,0 +1,19 @@
{
"type": "object",
"title": "State Event",
"description": "In addition to the Room Event fields, State Events have the following additional fields.",
"allOf":[{
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"state_key": {
"type": "string",
"description": "A unique key which defines the overwriting semantics for this piece of room state. This value is often a zero-length string. The presence of this key makes this event a State Event."
},
"prev_content": {
"type": "object",
"description": "Optional. The previous ``content`` for this event. If there is no previous content, this key will be missing."
}
},
"required": ["state_key"]
}

@ -1,9 +1,8 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"description": "This event is sent by the callee when they wish to answer the call.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,9 +1,8 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"description": "This event is sent by callers after sending an invite and by the callee after answering. Its purpose is to give the other party additional ICE candidates to try using to communicate.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,9 +1,8 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"description": "Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,9 +1,8 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"description": "This event is sent by the caller when they wish to establish a call.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,5 +1,4 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Presence Event",
"description": "Informs the client of a user's presence state change.",

@ -0,0 +1,48 @@
{
"type": "object",
"title": "Receipt Event",
"description": "Informs the client of new receipts.",
"properties": {
"content": {
"type": "object",
"patternProperties": {
"^\\$": {
"type": "object",
"x-pattern": "$EVENT_ID",
"title": "Receipts",
"description": "The mapping of event ID to a collection of receipts for this event ID. The event ID is the ID of the event being acknowledged and *not* an ID for the receipt itself.",
"properties": {
"m.read": {
"type": "object",
"title": "Users",
"description": "A collection of users who have sent ``m.read`` receipts for this event.",
"patternProperties": {
"^@": {
"type": "object",
"title": "Receipt",
"description": "The mapping of user ID to receipt. The user ID is the entity who sent this receipt.",
"x-pattern": "$USER_ID",
"properties": {
"ts": {
"type": "number",
"description": "The timestamp the receipt was sent at."
}
}
}
}
}
}
}
},
"additionalProperties": false
},
"type": {
"type": "string",
"enum": ["m.receipt"]
},
"room_id": {
"type": "string"
}
},
"required": ["room_id", "type", "content"]
}

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Informs the room about what room aliases it has been given.",
"description": "This event is sent by a homeserver directly to inform of changes to the list of aliases it knows about for that room. The ``state_key`` for this event is set to the homeserver which owns the room alias. The entire set of known aliases for the room is the union of all the ``m.room.aliases`` events, one for each homeserver. Clients **should** check the validity of any room alias given in this list before presenting it to the user as trusted fact. The lists given by this event should be considered simply as advice on which aliases might exist, for which the client can perform the lookup to confirm whether it receives the correct room ID.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Informs the room as to which alias is the canonical one.",
"description": "This event is used to inform the room about which alias should be considered the canonical one. This could be for display purposes or as suggestion to users which alias to use to advertise the room.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "The first event in the room.",
"description": "This is the first event in a room and cannot be changed. It acts as the root of all other events.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {
@ -13,6 +12,10 @@
"creator": {
"type": "string",
"description": "The ``user_id`` of the room creator. This is set by the homeserver."
},
"m.federate": {
"type": "boolean",
"description": "Whether users on other servers can join this room. Defaults to ``true`` if key does not exist."
}
},
"required": ["creator"]

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Controls visibility of history.",
"description": "This event controls whether a member of a room can see the events that happened in a room from before they joined.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Describes how users are allowed to join the room.",
"description": "A room may be ``public`` meaning anyone can join the room without any prior action. Alternatively, it can be ``invite`` meaning that a user who wishes to join the room must first receive an invite to the room from someone already inside of the room. Currently, ``knock`` and ``private`` are reserved keywords which are not implemented.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "The current membership state of a user in the room.",
"description": "Adjusts the membership state for a user in a room. It is preferable to use the membership APIs (``/rooms/<room id>/invite`` etc) when performing membership actions rather than adjusting the state directly as there are a restricted set of valid transformations. For example, user A cannot force user B to join a room, and trying to force this state change directly will fail.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {
@ -20,7 +19,7 @@
"description": "The avatar URL for this user, if any. This is added by the homeserver."
},
"displayname": {
"type": "string",
"type": ["string", "null"],
"description": "The display name for this user, if any. This is added by the homeserver."
}
},
@ -33,6 +32,26 @@
"type": {
"type": "string",
"enum": ["m.room.member"]
},
"invite_room_state": {
"type": "array",
"description": "A subset of the state of the room at the time of the invite, if ``membership`` is ``invite``",
"items": {
"type": "object",
"title": "StateEvent",
"description": "A stripped down state event, with only the ``type``, ``state_key`` and ``content`` keys.",
"properties": {
"type": {
"type": "string"
},
"state_key": {
"type": "string"
},
"content": {
"type": "object"
}
}
}
}
}
}

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Message",
"description": "This event is used when sending messages in a room. Messages are not limited to be text. The ``msgtype`` key outlines the type of message, e.g. text, audio, image, video, etc. The ``body`` key is text and MUST be used with every kind of ``msgtype`` as a fallback mechanism for when a client cannot render a message.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "AudioMessage",
"description": "This message represents a single audio clip.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "EmoteMessage",
"description": "This message is similar to ``m.text`` except that the sender is 'performing' the action contained in the ``body`` key, similar to ``/me`` in IRC. This message should be prefixed by the name of the sender. This message could also be represented in a different colour to distinguish it from regular ``m.text`` messages.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "FileMessage",
"description": "This message represents a generic file.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {
@ -50,7 +49,7 @@
"title": "ImageInfo",
"description": "Metadata about the image referred to in ``thumbnail_url``.",
"allOf": [{
"$ref": "core#/definitions/msgtype_infos/image_info"
"$ref": "core-event-schema/msgtype_infos/image_info.json"
}]
}
},

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "ImageMessage",
"description": "This message represents a single image and an optional thumbnail.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {
@ -31,7 +30,7 @@
"title": "ImageInfo",
"description": "Metadata about the image referred to in ``thumbnail_url``.",
"allOf": [{
"$ref": "core#/definitions/msgtype_infos/image_info"
"$ref": "core-event-schema/msgtype_infos/image_info.json"
}]
},
"info": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "LocationMessage",
"description": "This message represents a real-world location.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {
@ -30,7 +29,7 @@
"type": "object",
"title": "ImageInfo",
"allOf": [{
"$ref": "core#/definitions/msgtype_infos/image_info"
"$ref": "core-event-schema/msgtype_infos/image_info.json"
}]
}
},

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "NoticeMessage",
"description": "A m.notice message should be considered similar to a plain m.text message except that clients should visually distinguish it in some way. It is intended to be used by automated clients, such as bots, bridges, and other entities, rather than humans. Additionally, such automated agents which watch a room for messages and respond to them ought to ignore m.notice messages. This helps to prevent infinite-loop situations where two automated clients continuously exchange messages, as each responds to the other.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "TextMessage",
"description": "This message is the most basic message and is used to represent text.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "VideoMessage",
"description": "This message represents a single video clip.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {
@ -55,7 +54,7 @@
"type": "object",
"title": "ImageInfo",
"allOf": [{
"$ref": "core#/definitions/msgtype_infos/image_info"
"$ref": "core-event-schema/msgtype_infos/image_info.json"
}]
}
}

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "MessageFeedback",
"description": "Feedback events are events sent to acknowledge a message in some way. There are two supported acknowledgements: ``delivered`` (sent when the event has been received) and ``read`` (sent when the event has been observed by the end-user). The ``target_event_id`` should reference the ``m.room.message`` event being acknowledged. N.B. not implemented in Synapse, and superceded in v2 CS API by the ``relates_to`` event field.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "RoomName",
"description": "A room has an opaque room ID which is not human-friendly to read. A room alias is human-friendly, but not all rooms have room aliases. The room name is a human-friendly string designed to be displayed to the end-user. The room name is not unique, as multiple rooms can have the same room name set. The room name can also be set when creating a room using ``/createRoom`` with the ``name`` key.",
"type": "object",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Defines the power levels (privileges) of users in the room.",
"description": "This event specifies the minimum level a user must have in order to perform a certain action. It also specifies the levels of each user in the room. If a ``user_id`` is in the ``users`` list, then that ``user_id`` has the associated power level. Otherwise they have the default level ``users_default``. If ``users_default`` is not supplied, it is assumed to be 0. The level required to send a certain event is governed by ``events``, ``state_default`` and ``events_default``. If an event type is specified in ``events``, then the user must have at least the level specified in order to send that event. If the event type is not supplied, it defaults to ``events_default`` for Message Events and ``state_default`` for State Events.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Redaction",
"description": "Events can be redacted by either room or server admins. Redacting an event means that all keys not required by the protocol are stripped off, allowing admins to remove offensive or illegal content that may have been attached to any event. This cannot be undone, allowing server owners to physically delete the offending data. There is also a concept of a moderator hiding a message event, which can be undone, but cannot be applied to state events. The event that has been redacted is specified in the ``redacts`` event level key.",
"allOf": [{
"$ref": "core#/definitions/room_event"
"$ref": "core-event-schema/room_event.json"
}],
"properties": {
"content": {

@ -1,10 +1,9 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Topic",
"description": "A topic is a short message detailing what is currently being discussed in the room. It can also be used as a way to display extra information about the room, which may not be suitable for the room name. The room topic can also be set when creating a room using ``/createRoom`` with the ``topic`` key.",
"allOf": [{
"$ref": "core#/definitions/state_event"
"$ref": "core-event-schema/state_event.json"
}],
"properties": {
"content": {

@ -0,0 +1,28 @@
{
"type": "object",
"title": "Typing Event",
"description": "Informs the client of the list of users currently typing.",
"properties": {
"content": {
"type": "object",
"properties": {
"user_ids": {
"type": "array",
"items": {
"type": "string"
},
"description": "The list of user IDs typing in this room, if any."
}
},
"required": ["user_ids"]
},
"type": {
"type": "string",
"enum": ["m.typing"]
},
"room_id": {
"type": "string"
}
},
"required": ["type", "room_id", "content"]
}

@ -0,0 +1,9 @@
#! /bin/bash
set -ex
(cd event-schemas/ && ./check_examples.py)
(cd api && ./check_examples.py)
(cd scripts && ./gendoc.py)
(cd api && npm install && node validator.js -s "client-server/v1" && node validator.js -s "client-server/v2_alpha")
(cd event-schemas/ && ./check.sh)

@ -0,0 +1,6 @@
pre.code .comment, code .comment { color: green }
pre.code .keyword, code .keyword { color: darkred; font-weight: bold }
pre.code .name.builtin, code .name.builtin { color: darkred; font-weight: bold }
pre.code .literal.number, code .literal.number { color: blue }
pre.code .name.tag, code .name.tag { color: darkgreen }
pre.code .literal.string, code .literal.string { color: darkblue }

@ -0,0 +1,6 @@
continuserv proactively re-generates the spec on filesystem changes, and serves it over HTTP.
To run it, you must install the `go` tool. You will also need to install fsnotify by running:
`go get gopkg.in/fsnotify.v1`
You can then run continuserv by running:
`go run main.go`

@ -0,0 +1,161 @@
// continuserv proactively re-generates the spec on filesystem changes, and serves it over HTTP.
// It will always serve the most recent version of the spec, and may block an HTTP request until regeneration is finished.
// It does not currently pre-empt stale generations, but will block until they are complete.
package main
import (
"bytes"
"flag"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"time"
fsnotify "gopkg.in/fsnotify.v1"
)
var (
port = flag.Int("port", 8000, "Port on which to serve HTTP")
toServe atomic.Value // Always contains valid []byte to serve. May be stale unless wg is zero.
wg sync.WaitGroup // Indicates how many updates are pending.
mu sync.Mutex // Prevent multiple updates in parallel.
)
func main() {
flag.Parse()
w, err := fsnotify.NewWatcher()
if err != nil {
log.Fatalf("Error making watcher: %v", err)
}
dir, err := os.Getwd()
if err != nil {
log.Fatalf("Error getting wd: %v", err)
}
for ; !exists(path.Join(dir, ".git")); dir = path.Dir(dir) {
if dir == "/" {
log.Fatalf("Could not find git root")
}
}
filepath.Walk(dir, makeWalker(w))
wg.Add(1)
populateOnce(dir)
ch := make(chan struct{}, 100) // Buffered to ensure we can multiple-increment wg for pending writes
go doPopulate(ch, dir)
go watchFS(ch, w)
fmt.Printf("Listening on port %d\n", *port)
http.HandleFunc("/", serve)
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), nil))
}
func watchFS(ch chan struct{}, w *fsnotify.Watcher) {
for {
select {
case e := <-w.Events:
if filter(e) {
fmt.Printf("Noticed change to %s, re-generating spec\n", e.Name)
ch <- struct{}{}
}
}
}
}
func makeWalker(w *fsnotify.Watcher) filepath.WalkFunc {
return func(path string, _ os.FileInfo, err error) error {
if err != nil {
log.Fatalf("Error walking: %v", err)
}
if err := w.Add(path); err != nil {
log.Fatalf("Failed to add watch: %v", err)
}
return nil
}
}
// Return true if event should trigger re-population
func filter(e fsnotify.Event) bool {
// vim is *really* noisy about how it writes files
if e.Op != fsnotify.Write {
return false
}
// Avoid some temp files that vim writes
if strings.HasSuffix(e.Name, "~") || strings.HasSuffix(e.Name, ".swp") || strings.HasPrefix(e.Name, ".") {
return false
}
// Ignore the .git directory - It's very noisy
if strings.Contains(e.Name, "/.git/") {
return false
}
// Avoid infinite cycles being caused by writing actual output
if strings.Contains(e.Name, "/tmp/") || strings.Contains(e.Name, "/gen/") {
return false
}
return true
}
func serve(w http.ResponseWriter, req *http.Request) {
wg.Wait()
b := toServe.Load().([]byte)
w.Write(b)
}
func populateOnce(dir string) {
defer wg.Done()
mu.Lock()
defer mu.Unlock()
cmd := exec.Command("python", "gendoc.py")
cmd.Dir = path.Join(dir, "scripts")
var b bytes.Buffer
cmd.Stderr = &b
err := cmd.Run()
if err != nil {
toServe.Store([]byte(fmt.Errorf("error generating spec: %v\nOutput from gendoc:\n%v", err, b.String()).Error()))
return
}
specBytes, err := ioutil.ReadFile(path.Join(dir, "scripts", "gen", "specification.html"))
if err != nil {
toServe.Store([]byte(fmt.Errorf("error reading spec: %v", err).Error()))
return
}
toServe.Store(specBytes)
}
func doPopulate(ch chan struct{}, dir string) {
var pending int
for {
select {
case <-ch:
if pending == 0 {
wg.Add(1)
}
pending++
case <-time.After(10 * time.Millisecond):
if pending > 0 {
pending = 0
populateOnce(dir)
}
}
}
}
func exists(path string) bool {
_, err := os.Stat(path)
return !os.IsNotExist(err)
}

@ -1,22 +1,231 @@
#! /usr/bin/env python
from argparse import ArgumentParser
from docutils.core import publish_file
import copy
import fileinput
import glob
import os
import re
import shutil
import subprocess
import sys
import yaml
os.chdir(os.path.dirname(os.path.abspath(__file__)))
stylesheets = {
"stylesheet_path": ["basic.css", "nature.css"]
"stylesheet_path": ["basic.css", "nature.css", "codehighlight.css"]
}
def glob_spec_to(out_file_name):
with open(out_file_name, "wb") as outfile:
for f in sorted(glob.glob("../specification/*.rst")):
with open(f, "rb") as infile:
outfile.write(infile.read())
VERBOSE = False
"""
Read a RST file and replace titles with a different title level if required.
Args:
filename: The name of the file being read (for debugging)
file_stream: The open file stream to read from.
title_level: The integer which determines the offset to *start* from.
title_styles: An array of characters detailing the right title styles to use
e.g. ["=", "-", "~", "+"]
Returns:
string: The file contents with titles adjusted.
Example:
Assume title_styles = ["=", "-", "~", "+"], title_level = 1, and the file
when read line-by-line encounters the titles "===", "---", "---", "===", "---".
This function will bump every title encountered down a sub-heading e.g.
"=" to "-" and "-" to "~" because title_level = 1, so the output would be
"---", "~~~", "~~~", "---", "~~~". There is no bumping "up" a title level.
"""
def load_with_adjusted_titles(filename, file_stream, title_level, title_styles):
rst_lines = []
title_chars = "".join(title_styles)
title_regex = re.compile("^[" + re.escape(title_chars) + "]{3,}$")
prev_line_title_level = 0 # We expect the file to start with '=' titles
file_offset = None
prev_non_title_line = None
for i, line in enumerate(file_stream, 1):
# ignore anything which isn't a title (e.g. '===============')
if not title_regex.match(line):
rst_lines.append(line)
prev_non_title_line = line
continue
# The title underline must match at a minimum the length of the title
if len(prev_non_title_line) > len(line):
rst_lines.append(line)
prev_non_title_line = line
continue
line_title_style = line[0]
line_title_level = title_styles.index(line_title_style)
# Not all files will start with "===" and we should be flexible enough
# to allow that. The first title we encounter sets the "file offset"
# which is added to the title_level desired.
if file_offset is None:
file_offset = line_title_level
if file_offset != 0:
logv((" WARNING: %s starts with a title style of '%s' but '%s' " +
"is preferable.") % (filename, line_title_style, title_styles[0]))
# Sanity checks: Make sure that this file is obeying the title levels
# specified and bail if it isn't.
# The file is allowed to go 1 deeper or any number shallower
if prev_line_title_level - line_title_level < -1:
raise Exception(
("File '%s' line '%s' has a title " +
"style '%s' which doesn't match one of the " +
"allowed title styles of %s because the " +
"title level before this line was '%s'") %
(filename, (i + 1), line_title_style, title_styles,
title_styles[prev_line_title_level])
)
prev_line_title_level = line_title_level
adjusted_level = (
title_level + line_title_level - file_offset
)
# Sanity check: Make sure we can bump down the title and we aren't at the
# lowest level already
if adjusted_level >= len(title_styles):
raise Exception(
("Files '%s' line '%s' has a sub-title level too low and it " +
"cannot be adjusted to fit. You can add another level to the " +
"'title_styles' key in targets.yaml to fix this.") %
(filename, (i + 1))
)
if adjusted_level == line_title_level:
# no changes required
rst_lines.append(line)
continue
# Adjusting line levels
logv(
"File: %s Adjusting %s to %s because file_offset=%s title_offset=%s" %
(filename, line_title_style, title_styles[adjusted_level],
file_offset, title_level)
)
rst_lines.append(line.replace(
line_title_style,
title_styles[adjusted_level]
))
return "".join(rst_lines)
def get_rst(file_info, title_level, title_styles, spec_dir, adjust_titles):
# string are file paths to RST blobs
if isinstance(file_info, basestring):
log("%s %s" % (">" * (1 + title_level), file_info))
with open(os.path.join(spec_dir, file_info), "r") as f:
rst = None
if adjust_titles:
rst = load_with_adjusted_titles(
file_info, f, title_level, title_styles
)
else:
rst = f.read()
if rst[-2:] != "\n\n":
raise Exception(
("File %s should end with TWO new-line characters to ensure " +
"file concatenation works correctly.") % (file_info,)
)
return rst
# dicts look like {0: filepath, 1: filepath} where the key is the title level
elif isinstance(file_info, dict):
levels = sorted(file_info.keys())
rst = []
for l in levels:
rst.append(get_rst(file_info[l], l, title_styles, spec_dir, adjust_titles))
return "".join(rst)
# lists are multiple file paths e.g. [filepath, filepath]
elif isinstance(file_info, list):
rst = []
for f in file_info:
rst.append(get_rst(f, title_level, title_styles, spec_dir, adjust_titles))
return "".join(rst)
raise Exception(
"The following 'file' entry in this target isn't a string, list or dict. " +
"It really really should be. Entry: %s" % (file_info,)
)
def build_spec(target, out_filename):
with open(out_filename, "wb") as outfile:
for file_info in target["files"]:
section = get_rst(
file_info=file_info,
title_level=0,
title_styles=target["title_styles"],
spec_dir="../specification/",
adjust_titles=True
)
outfile.write(section)
"""
Replaces relative title styles with actual title styles.
The templating system has no idea what the right title style is when it produces
RST because it depends on the build target. As a result, it uses relative title
styles defined in targets.yaml to say "down a level, up a level, same level".
This function replaces these relative titles with actual title styles from the
array in targets.yaml.
"""
def fix_relative_titles(target, filename, out_filename):
title_styles = target["title_styles"]
relative_title_chars = [
target["relative_title_styles"]["subtitle"],
target["relative_title_styles"]["sametitle"],
target["relative_title_styles"]["supertitle"]
]
relative_title_matcher = re.compile(
"^[" + re.escape("".join(relative_title_chars)) + "]{3,}$"
)
title_matcher = re.compile(
"^[" + re.escape("".join(title_styles)) + "]{3,}$"
)
current_title_style = None
with open(filename, "r") as infile:
with open(out_filename, "w") as outfile:
for line in infile.readlines():
if not relative_title_matcher.match(line):
if title_matcher.match(line):
current_title_style = line[0]
outfile.write(line)
continue
line_char = line[0]
replacement_char = None
current_title_level = title_styles.index(current_title_style)
if line_char == target["relative_title_styles"]["subtitle"]:
if (current_title_level + 1) == len(title_styles):
raise Exception(
"Encountered sub-title line style but we can't go " +
"any lower."
)
replacement_char = title_styles[current_title_level + 1]
elif line_char == target["relative_title_styles"]["sametitle"]:
replacement_char = title_styles[current_title_level]
elif line_char == target["relative_title_styles"]["supertitle"]:
if (current_title_level - 1) < 0:
raise Exception(
"Encountered super-title line style but we can't go " +
"any higher."
)
replacement_char = title_styles[current_title_level - 1]
else:
raise Exception(
"Unknown relative line char %s" % (line_char,)
)
outfile.write(
line.replace(line_char, replacement_char)
)
def rst2html(i, o):
@ -31,18 +240,126 @@ def rst2html(i, o):
settings_overrides=stylesheets
)
def run_through_template(input):
null = open(os.devnull, 'w')
subprocess.check_output(
[
'python', 'build.py',
"-i", "matrix_templates",
"-o", "../scripts/tmp",
"../scripts/"+input
],
stderr=null,
cwd="../templating",
)
def run_through_template(input, set_verbose):
tmpfile = './tmp/output'
try:
with open(tmpfile, 'w') as out:
args = [
'python', 'build.py',
"-i", "matrix_templates",
"-o", "../scripts/tmp",
"../scripts/"+input
]
if set_verbose:
args.insert(2, "-v")
log("EXEC: %s" % " ".join(args))
log(" ==== build.py output ==== ")
print subprocess.check_output(
args,
stderr=out,
cwd="../templating"
)
except subprocess.CalledProcessError as e:
with open(tmpfile, 'r') as f:
sys.stderr.write(f.read() + "\n")
raise
"""
Extract and resolve groups for the given target in the given targets listing.
Args:
targets_listing (str): The path to a YAML file containing a list of targets
target_name (str): The name of the target to extract from the listings.
Returns:
dict: Containing "filees" (a list of file paths), "relative_title_styles"
(a dict of relative style keyword to title character) and "title_styles"
(a list of characters which represent the global title style to follow,
with the top section title first, the second section second, and so on.)
"""
def get_build_target(targets_listing, target_name):
build_target = {
"title_styles": [],
"relative_title_styles": {},
"files": []
}
with open(targets_listing, "r") as targ_file:
all_targets = yaml.load(targ_file.read())
build_target["title_styles"] = all_targets["title_styles"]
build_target["relative_title_styles"] = all_targets["relative_title_styles"]
target = all_targets["targets"].get(target_name)
if not target:
raise Exception(
"No target by the name '" + target_name + "' exists in '" +
targets_listing + "'."
)
if not isinstance(target.get("files"), list):
raise Exception(
"Found target but 'files' key is not a list."
)
def get_group(group_id, depth):
group_name = group_id[len("group:"):]
group = all_targets.get("groups", {}).get(group_name)
if not group:
raise Exception(
"Tried to find group '%s' but it doesn't exist." % group_name
)
if not isinstance(group, list):
raise Exception(
"Expected group '%s' to be a list but it isn't." % group_name
)
# deep copy so changes to depths don't contaminate multiple uses of this group
group = copy.deepcopy(group)
# swap relative depths for absolute ones
for i, entry in enumerate(group):
if isinstance(entry, dict):
group[i] = {
(rel_depth + depth): v for (rel_depth, v) in entry.items()
}
return group
resolved_files = []
for file_entry in target["files"]:
# file_entry is a group id
if isinstance(file_entry, basestring) and file_entry.startswith("group:"):
group = get_group(file_entry, 0)
# The group may be resolved to a list of file entries, in which case
# we want to extend the array to insert each of them rather than
# insert the entire list as a single element (which is what append does)
if isinstance(group, list):
resolved_files.extend(group)
else:
resolved_files.append(group)
# file_entry is a dict which has more file entries as values
elif isinstance(file_entry, dict):
resolved_entry = {}
for (depth, entry) in file_entry.iteritems():
if not isinstance(entry, basestring):
raise Exception(
"Double-nested depths are not supported. Entry: %s" % (file_entry,)
)
if entry.startswith("group:"):
resolved_entry[depth] = get_group(entry, depth)
else:
# map across without editing (e.g. normal file path)
resolved_entry[depth] = entry
resolved_files.append(resolved_entry)
continue
# file_entry is just a plain ol' file path
else:
resolved_files.append(file_entry)
build_target["files"] = resolved_files
return build_target
def log(line):
print "gendoc: %s" % line
def logv(line):
if VERBOSE:
print "gendoc:V: %s" % line
def prepare_env():
try:
@ -53,30 +370,49 @@ def prepare_env():
os.makedirs("./tmp")
except OSError:
pass
def cleanup_env():
shutil.rmtree("./tmp")
def main():
def main(target_name, keep_intermediates):
prepare_env()
glob_spec_to("tmp/full_spec.rst")
run_through_template("tmp/full_spec.rst")
log("Building spec [target=%s]" % target_name)
target = get_build_target("../specification/targets.yaml", target_name)
build_spec(target=target, out_filename="tmp/templated_spec.rst")
run_through_template("tmp/templated_spec.rst", VERBOSE)
fix_relative_titles(
target=target, filename="tmp/templated_spec.rst",
out_filename="tmp/full_spec.rst"
)
shutil.copy("../supporting-docs/howtos/client-server.rst", "tmp/howto.rst")
run_through_template("tmp/howto.rst")
run_through_template("tmp/howto.rst", False) # too spammy to mark -v on this
rst2html("tmp/full_spec.rst", "gen/specification.html")
rst2html("tmp/howto.rst", "gen/howtos.html")
cleanup_env()
if not keep_intermediates:
cleanup_env()
if __name__ == '__main__':
if len(sys.argv) > 1:
# we accept no args, so they don't know what they're doing!
print "gendoc.py - Generate the Matrix specification as HTML."
print "Usage:"
print " python gendoc.py"
print ""
print "The specification can then be found in the gen/ folder."
print ""
print "Requirements:"
print " - This script requires Jinja2 and rst2html (docutils)."
sys.exit(0)
main()
parser = ArgumentParser(
"gendoc.py - Generate the Matrix specification as HTML to the gen/ folder."
)
parser.add_argument(
"--nodelete", "-n", action="store_true",
help="Do not delete intermediate files. They will be found in tmp/"
)
parser.add_argument(
"--target", "-t", default="main",
help="Specify the build target to build from specification/targets.yaml"
)
parser.add_argument(
"--verbose", "-v", action="store_true",
help="Turn on verbose mode."
)
args = parser.parse_args()
if not args.target:
parser.print_help()
sys.exit(1)
VERBOSE = args.verbose
main(args.target, args.nodelete)

@ -244,10 +244,6 @@ div.viewcode-block:target {
border-bottom: 1px solid #ac9;
}
p {
margin: 0;
}
ul li dd {
margin-top: 0;
}
@ -282,3 +278,9 @@ td[colspan]:not([colspan="1"]) {
thead {
background: #eeeeee;
}
div.admonition-rationale {
background-color: #efe;
border: 1px solid #ccc;
}

@ -0,0 +1,10 @@
speculator allows you to preview pull requests to the matrix.org specification.
It serves the following HTTP endpoints:
- / lists open pull requests
- /spec/123 which renders the spec as html at pull request 123.
- /diff/rst/123 which gives a diff of the spec's rst at pull request 123.
- /diff/html/123 which gives a diff of the spec's HTML at pull request 123.
To run it, you must install the `go` tool, and run:
`go run main.go`

@ -0,0 +1,564 @@
#!/usr/bin/perl
#
# htmldiff - present a diff marked version of two html documents
#
# Copyright (c) 1998-2006 MACS, Inc.
#
# Copyright (c) 2007 SiSco, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#
# See http://www.themacs.com for more information.
#
# usage: htmldiff [[-c] [-l] [-o] oldversion newversion [output]]
#
# -c - disable metahtml comment processing
# -o - disable outputting of old text
# -l - use navindex to create sequence of diffs
# oldversion - the previous version of the document
# newversion - the newer version of the document
# output - a filename to place the output in. If omitted, the output goes to
# standard output.
#
# if invoked with no options or arguments, operates as a CGI script. It then
# takes the following parameters:
#
# oldfile - the URL of the original file
# newfile - the URL of the new file
# mhtml - a flag to indicate whether it should be aware of MetaHTML comments.
#
# requires GNU diff utility
# also requires the perl modules Getopt::Std
#
# NOTE: The markup created by htmldiff may not validate against the HTML 4.0
# DTD. This is because the algorithm is realtively simple, and there are
# places in the markup content model where the span element is not allowed.
# Htmldiff is NOT aware of these places.
#
# $Source: /u/sources/public/2009/htmldiff/htmldiff.pl,v $
# $Revision: 1.1 $
#
# $Log: htmldiff.pl,v $
# Revision 1.1 2014/01/06 08:04:51 dom
# added copy of htmldiff perl script since aptest.com repo no longer available
#
# Revision 1.5 2008/03/05 13:23:16 ahby
# Fixed a problem with leading whitespace before markup.
#
# Revision 1.4 2007/12/13 13:09:16 ahby
# Updated copyright and license.
#
# Revision 1.3 2007/12/13 12:53:34 ahby
# Changed use of span to ins and del
#
# Revision 1.2 2002/02/13 16:27:23 ahby
# Changed processing model.
# Improved handling of old text and changed styles.
#
# Revision 1.1 2000/07/12 12:20:04 ahby
# Updated to remove empty spans - this fixes validation problems under
# strict.
#
# Revision 1.11 1999/12/08 19:46:45 ahby
# Fixed validation errors introduced by placing markup where it didn't
# belong.
#
# Revision 1.10 1999/10/18 13:42:58 ahby
# Added -o to the usage message.
#
# Revision 1.9 1999/05/04 12:29:11 ahby
# Added an option to turn off the display of old text.
#
# Revision 1.8 1999/04/09 14:37:27 ahby
# Fixed a perl syntax error.
#
# Revision 1.7 1999/04/09 14:35:49 ahby
# Added reference to MACS homepage.
#
# Revision 1.6 1999/04/09 14:35:09 ahby
# Added comment about validity of generated markup.
#
# Revision 1.5 1999/02/22 22:17:54 ahby
# Changed to use stylesheets.
# Changed to rely upon span.
# Changed to work around content model problems.
#
# Revision 1.4 1999/02/08 02:32:22 ahby
# Added a copyright statement.
#
# Revision 1.3 1999/02/08 02:30:40 ahby
# Added header processing.
#
# Revision 1.2 1998/12/10 17:31:31 ahby
# Fixed to escape less-thans in change blocks and to not permit change
# markup within specific elements (like TITLE).
#
# Revision 1.1 1998/11/26 00:09:22 ahby
# Initial revision
#
#
use Getopt::Std;
sub usage {
print STDERR "htmldiff [-c] [-o] oldversion newversion [output]\n";
exit;
}
sub url_encode {
my $str = shift;
$str =~ s/([\x00-\x1f\x7F-\xFF])/
sprintf ('%%%02x', ord ($1))/eg;
return $str;
}
# markit - diff-mark the streams
#
# markit(file1, file2)
#
# markit relies upon GNUdiff to mark up the text.
#
# The markup is encoded using special control sequences:
#
# a block wrapped in control-a is deleted text
# a block wrapped in control-b is old text
# a block wrapped in control-c is new text
#
# The main processing loop attempts to wrap the text blocks in appropriate
# SPANs based upon the type of text that it is.
#
# When the loop encounters a < in the text, it stops the span. Then it outputs
# the element that is defined, then it restarts the span.
sub markit {
my $retval = "";
my($file1) = shift;
my($file2) = shift;
# my $old="<span class=\\\"diff-old-a\\\">deleted text: </span>%c'\012'%c'\001'%c'\012'%<%c'\012'%c'\001'%c'\012'";
my $old="%c'\012'%c'\001'%c'\012'%<%c'\012'%c'\001'%c'\012'";
my $new="%c'\012'%c'\003'%c'\012'%>%c'\012'%c'\003'%c'\012'";
my $unchanged="%=";
my $changed="%c'\012'%c'\001'%c'\012'%<%c'\012'%c'\001'%c'\012'%c'\004'%c'\012'%>%c'\012'%c'\004'%c'\012'";
if ($opt_o) {
$old = "";
$changed = "%c'\012'%c'\004'%c'\012'%>%c'\012'%c'\004'%c'\012'";
}
# my $old="%c'\002'<font color=\\\"purple\\\" size=\\\"-2\\\">deleted text:</font><s>%c'\012'%c'\001'%c'\012'%<%c'\012'%c'\001'%c'\012'</s>%c'\012'%c'\002'";
# my $new="%c'\002'<font color=\\\"purple\\\"><u>%c'\012'%c'\002'%>%c'\002'</u></font>%c'\002'%c'\012'";
# my $unchanged="%=";
# my $changed="%c'\002'<s>%c'\012'%c'\001'%c'\012'%<%c'\012'%c'\001'%c'\012'</s><font color=\\\"purple\\\"><u>%c'\002'%c'\012'%>%c'\012'%c'\002'</u></font>%c'\002'%c'\012'";
my @span;
$span[0]="</span>";
$span[1]="<del class=\"diff-old\">";
$span[2]="<del class=\"diff-old\">";
$span[3]="<ins class=\"diff-new\">";
$span[4]="<ins class=\"diff-chg\">";
my @diffEnd ;
$diffEnd[1] = '</del>';
$diffEnd[2] = '</del>';
$diffEnd[3] = '</ins>';
$diffEnd[4] = '</ins>';
my $diffcounter = 0;
open(FILE, qq(diff -d --old-group-format="$old" --new-group-format="$new" --changed-group-format="$changed" --unchanged-group-format="$unchanged" $file1 $file2 |)) || die("Diff failed: $!");
# system (qq(diff --old-group-format="$old" --new-group-format="$new" --changed-group-format="$changed" --unchanged-group-format="$unchanged" $file1 $file2 > /tmp/output));
my $state = 0;
my $inblock = 0;
my $temp = "";
my $lineCount = 0;
# strategy:
#
# process the output of diff...
#
# a link with control A-D means the start/end of the corresponding ordinal
# state (1-4). Resting state is state 0.
#
# While in a state, accumulate the contents for that state. When exiting the
# state, determine if it is appropriate to emit the contents with markup or
# not (basically, if the accumulated buffer contains only empty lines or lines
# with markup, then we don't want to emit the wrappers. We don't need them.
#
# Note that if there is markup in the "old" block, that markup is silently
# removed. It isn't really that interesting, and it messes up the output
# something fierce.
while (<FILE>) {
my $anchor = $opt_l ? qq[<a tabindex="$diffcounter">] : "" ;
my $anchorEnd = $opt_l ? q[</a>] : "" ;
$lineCount ++;
if ($state == 0) { # if we are resting and we find a marker,
# then we must be entering a block
if (m/^([\001-\004])/) {
$state = ord($1);
$_ = "";
}
# if (m/^\001/) {
# $state = 1;
# s/^/$span[1]/;
# } elsif (m/^\002/) {
# $state = 2;
# s/^/$span[2]/;
# } elsif (m/^\003/) {
# $state = 3;
# s/^/$span[3]/;
# } elsif (m/^\004/) {
# $state = 4;
# s/^/$span[4]/;
# }
} else {
# if we are in "old" state, remove markup
if (($state == 1) || ($state == 2)) {
s/\<.*\>//; # get rid of any old markup
s/\</&lt;/g; # escape any remaining STAG or ETAGs
s/\>/&gt;/g;
}
# if we found another marker, we must be exiting the state
if (m/^([\001-\004])/) {
if ($temp ne "") {
$_ = $span[$state] . $anchor . $temp . $anchorEnd . $diffEnd[$state] . "\n";
$temp = "";
} else {
$_ = "" ;
}
$state = 0;
} elsif (m/^\s*\</) { # otherwise, is this line markup?
# if it is markup AND we haven't seen anything else yet,
# then we will emit the markup
if ($temp eq "") {
$retval .= $_;
$_ = "";
} else { # we wrap it with the state switches and hold it
s/^/$anchorEnd$diffEnd[$state]/;
s/$/$span[$state]$anchor/;
$temp .= $_;
$_ = "";
}
} else {
if (m/.+/) {
$temp .= $_;
$_ = "";
}
}
}
s/\001//g;
s/\002//g;
s/\003//g;
s/\004//g;
if ($_ !~ m/^$/) {
$retval .= $_;
}
$diffcounter++;
}
close FILE;
$retval =~ s/$span[1]\n+$diffEnd[1]//g;
$retval =~ s/$span[2]\n+$diffEnd[2]//g;
$retval =~ s/$span[3]\n+$diffEnd[3]//g;
$retval =~ s/$span[4]\n+$diffEnd[4]//g;
$retval =~ s/$span[1]\n*$//g;
$retval =~ s/$span[2]\n*$//g;
$retval =~ s/$span[3]\n*$//g;
$retval =~ s/$span[4]\n*$//g;
return $retval;
}
sub splitit {
my $filename = shift;
my $headertmp = shift;
my $inheader=0;
my $preformatted=0;
my $inelement=0;
my $retval = "";
my $styles = q(<style type='text/css'>
.diff-old-a {
font-size: smaller;
color: red;
}
.diff-new { background-color: yellow; }
.diff-chg { background-color: lime; }
.diff-new:before,
.diff-new:after
{ content: "\2191" }
.diff-chg:before, .diff-chg:after
{ content: "\2195" }
.diff-old { text-decoration: line-through; background-color: #FBB; }
.diff-old:before,
.diff-old:after
{ content: "\2193" }
:focus { border: thin red solid}
</style>
);
if ($opt_t) {
$styles .= q(
<script type="text/javascript">
<!--
function setOldDisplay() {
for ( var s = 0; s < document.styleSheets.length; s++ ) {
var css = document.styleSheets[s];
var mydata ;
try { mydata = css.cssRules ;
if ( ! mydata ) mydata = css.rules;
for ( var r = 0; r < mydata.length; r++ ) {
if ( mydata[r].selectorText == '.diff-old' ) {
mydata[r].style.display = ( mydata[r].style.display == '' ) ? 'none'
: '';
return;
}
}
} catch(e) {} ;
}
}
-->
</script>
);
}
if ($stripheader) {
open(HEADER, ">$headertmp");
}
my $incomment = 0;
my $inhead = 1;
open(FILE, $filename) || die("File $filename cannot be opened: $!");
while (<FILE>) {
if ($inhead == 1) {
if (m/\<\/head/i) {
print HEADER $styles;
}
if (m/\<body/i) {
$inhead = 0;
print HEADER;
if ($opt_t) {
print HEADER q(
<form action=""><input type="button" onclick="setOldDisplay()" value="Show/Hide Old Content" /></form>
);
}
close HEADER;
} else {
print HEADER;
}
} else {
if ($incomment) {
if (m;-->;) {
$incomment = 0;
s/.*-->//;
} else {
next;
}
}
if (m;<!--;) {
while (m;<!--.*-->;) {
s/<!--.*?-->//;
}
if (m;<!--; ) {
$incomment = 1;
s/<!--.*//;
}
}
if (m/\<pre/i) {
$preformatted = 1;
}
if (m/\<\/pre\>/i) {
$preformatted = 0;
}
if ($preformatted) {
$retval .= $_;
} elsif ($mhtmlcomments && /^;;;/) {
$retval .= $_;
} else {
my @list = split(' ');
foreach $element (@list) {
if ($element =~ m/\<H[1-6]/i) {
# $inheader = 1;
}
if ($inheader == 0) {
$element =~ s/</\n</g;
$element =~ s/^\n//;
$element =~ s/>/>\n/g;
$element =~ s/\n$//;
$element =~ s/>\n([.,:!]+)/>$1/g;
}
if ($element =~ m/\<\/H[1-6]\>/i) {
$inheader = 0;
}
$retval .= "$element";
$inelement += ($element =~ s/</&lt;/g);
$inelement -= ($element =~ s/>/&gt;/g);
if ($inelement < 0) {
$inelement = 0;
}
if (($inelement == 0) && ($inheader == 0)) {
$retval .= "\n";
} else {
$retval .= " ";
}
}
undef @list;
}
}
}
$retval .= "\n";
close FILE;
return $retval;
}
$mhtmlcomments = 1;
sub cli {
getopts("clto") || usage();
if ($opt_c) {$mhtmlcomments = 0;}
if (@ARGV < 2) { usage(); }
$file1 = $ARGV[0];
$file2 = $ARGV[1];
$file3 = $ARGV[2];
$tmp = splitit($file1, $headertmp1);
open (FILE, ">$tmp1");
print FILE $tmp;
close FILE;
$tmp = splitit($file2, $headertmp2);
open (FILE, ">$tmp2");
print FILE $tmp;
close FILE;
$output = "";
if ($stripheader) {
open(FILE, $headertmp2);
while (<FILE>) {
$output .= $_;
}
close(FILE);
}
$output .= markit($tmp1, $tmp2);
if ($file3) {
open(FILE, ">$file3");
print FILE $output;
close FILE;
} else {
print $output;
}
}
sub cgi {
# use LWP::UserAgent;
# use CGI;
my $query = new CGI;
my $url1 = $query->param("oldfile");
my $url2 = $query->param("newfile");
my $mhtml = $query->param("mhtml");
my $file1 = "/tmp/htdcgi1.$$";
my $file2 = "/tmp/htdcgi2.$$";
my $ua = new LWP::UserAgent;
$ua->agent("MACS, Inc. HTMLdiff/0.9 " . $ua->agent);
# Create a request
my $req1 = new HTTP::Request GET => $url1;
my $res1 = $ua->request($req1, $file1);
if ($res1->is_error) {
print $res1->error_as_HTML();
print "<p>The URL $url1 could not be found. Please check it and try again.</p>";
return;
}
my $req2 = new HTTP::Request GET => $url2;
my $res2 = $ua->request($req2, $file2);
if ($res2->is_error) {
print $res2->error_as_HTML();
print "<p>The URL $url2 could not be found. Please check it and try again.</p>";
return;
}
$split1 = splitit($file1, $headertmp1);
open (FILE, ">$tmp1");
print FILE $split1;
close FILE;
$split2 = splitit($file2, $headertmp2);
open (FILE, ">$tmp2");
print FILE $split2;
close FILE;
$output = "";
if ($stripheader) {
open(FILE, $headertmp2);
while (<FILE>) {
$output .= $_;
}
close(FILE);
}
$output .= markit($tmp1, $tmp2);
my $base=$res2->base;
if ($base !~ /\/$/) {
$base =~ s/[^\/]*$//;
}
if ( $output !~ /<base/i ) {
$output =~ s/<head>/<head>\n<base href="$base">/i ||
$output =~ s/<html>/<html>\n<base href="$base">/i ;
}
print $query->header(-type=>'text/html',-nph=>1);
print $output;
unlink $file1;
unlink $file2;
}
$tmp1="/tmp/htdtmp1.$$";
$headertmp1="/tmp/htdhtmp1.$$";
$tmp2="/tmp/htdtmp2.$$";
$headertmp2="/tmp/htdhtmp2.$$";
$stripheader = 1;
if (@ARGV == 0) {
cgi(); # if no arguments, we must be operating as a cgi script
} else {
cli(); # if there are arguments, then we are operating as a CLI
}
unlink $tmp1;
unlink $headertmp1;
unlink $tmp2;
unlink $headertmp2;

@ -0,0 +1,374 @@
// speculator allows you to preview pull requests to the matrix.org specification.
// It serves the following HTTP endpoints:
// - / lists open pull requests
// - /spec/123 which renders the spec as html at pull request 123.
// - /diff/rst/123 which gives a diff of the spec's rst at pull request 123.
// - /diff/html/123 which gives a diff of the spec's HTML at pull request 123.
// It is currently woefully inefficient, and there is a lot of low hanging fruit for improvement.
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"io/ioutil"
"log"
"math/rand"
"net/http"
"net/url"
"os"
"os/exec"
"path"
"strconv"
"strings"
"syscall"
"time"
)
type PullRequest struct {
Number int
Base Commit
Head Commit
Title string
User User
HTMLURL string `json:"html_url"`
}
type Commit struct {
SHA string
Repo RequestRepo
}
type RequestRepo struct {
CloneURL string `json:"clone_url"`
}
type User struct {
Login string
HTMLURL string `json:"html_url"`
}
var (
port = flag.Int("port", 9000, "Port on which to listen for HTTP")
allowedMembers map[string]bool
)
func (u *User) IsTrusted() bool {
return allowedMembers[u.Login]
}
const (
pullsPrefix = "https://api.github.com/repos/matrix-org/matrix-doc/pulls"
matrixDocCloneURL = "https://github.com/matrix-org/matrix-doc.git"
)
func gitClone(url string, shared bool) (string, error) {
directory := path.Join("/tmp/matrix-doc", strconv.FormatInt(rand.Int63(), 10))
cmd := exec.Command("git", "clone", url, directory)
if shared {
cmd.Args = append(cmd.Args, "--shared")
}
err := cmd.Run()
if err != nil {
return "", fmt.Errorf("error cloning repo: %v", err)
}
return directory, nil
}
func gitCheckout(path, sha string) error {
return runGitCommand(path, []string{"checkout", sha})
}
func gitFetchAndMerge(path string) error {
if err := runGitCommand(path, []string{"fetch"}); err != nil {
return err
}
return runGitCommand(path, []string{"merge"})
}
func runGitCommand(path string, args []string) error {
cmd := exec.Command("git", args...)
cmd.Dir = path
err := cmd.Run()
if err != nil {
return fmt.Errorf("error running %q: %v", strings.Join(cmd.Args, " "), err)
}
return nil
}
func lookupPullRequest(url url.URL, pathPrefix string) (*PullRequest, error) {
if !strings.HasPrefix(url.Path, pathPrefix+"/") {
return nil, fmt.Errorf("invalid path passed: %s expect %s/123", url.Path, pathPrefix)
}
prNumber := url.Path[len(pathPrefix)+1:]
if strings.Contains(prNumber, "/") {
return nil, fmt.Errorf("invalid path passed: %s expect %s/123", url.Path, pathPrefix)
}
resp, err := http.Get(fmt.Sprintf("%s/%s", pullsPrefix, prNumber))
defer resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("error getting pulls: %v", err)
}
dec := json.NewDecoder(resp.Body)
var pr PullRequest
if err := dec.Decode(&pr); err != nil {
return nil, fmt.Errorf("error decoding pulls: %v", err)
}
return &pr, nil
}
func generate(dir string) error {
cmd := exec.Command("python", "gendoc.py", "--nodelete")
cmd.Dir = path.Join(dir, "scripts")
var b bytes.Buffer
cmd.Stderr = &b
err := cmd.Run()
if err != nil {
return fmt.Errorf("error generating spec: %v\nOutput from gendoc:\n%v", err, b.String())
}
return nil
}
func writeError(w http.ResponseWriter, code int, err error) {
w.WriteHeader(code)
io.WriteString(w, fmt.Sprintf("%v\n", err))
}
type server struct {
matrixDocCloneURL string
}
// generateAt generates spec from repo at sha.
// Returns the path where the generation was done.
func (s *server) generateAt(sha string) (dst string, err error) {
err = gitFetchAndMerge(s.matrixDocCloneURL)
if err != nil {
return
}
dst, err = gitClone(s.matrixDocCloneURL, true)
if err != nil {
return
}
if err = gitCheckout(dst, sha); err != nil {
return
}
err = generate(dst)
return
}
func (s *server) serveSpec(w http.ResponseWriter, req *http.Request) {
var sha string
if strings.ToLower(req.URL.Path) == "/spec/head" {
sha = "HEAD"
} else {
pr, err := lookupPullRequest(*req.URL, "/spec")
if err != nil {
writeError(w, 400, err)
return
}
// We're going to run whatever Python is specified in the pull request, which
// may do bad things, so only trust people we trust.
if err := checkAuth(pr); err != nil {
writeError(w, 403, err)
return
}
sha = pr.Head.SHA
}
dst, err := s.generateAt(sha)
defer os.RemoveAll(dst)
if err != nil {
writeError(w, 500, err)
return
}
b, err := ioutil.ReadFile(path.Join(dst, "scripts/gen/specification.html"))
if err != nil {
writeError(w, 500, fmt.Errorf("Error reading spec: %v", err))
return
}
w.Write(b)
}
func checkAuth(pr *PullRequest) error {
if !pr.User.IsTrusted() {
return fmt.Errorf("%q is not a trusted pull requester", pr.User.Login)
}
return nil
}
func (s *server) serveRSTDiff(w http.ResponseWriter, req *http.Request) {
pr, err := lookupPullRequest(*req.URL, "/diff/rst")
if err != nil {
writeError(w, 400, err)
return
}
// We're going to run whatever Python is specified in the pull request, which
// may do bad things, so only trust people we trust.
if err := checkAuth(pr); err != nil {
writeError(w, 403, err)
return
}
base, err := s.generateAt(pr.Base.SHA)
defer os.RemoveAll(base)
if err != nil {
writeError(w, 500, err)
return
}
head, err := s.generateAt(pr.Head.SHA)
defer os.RemoveAll(head)
if err != nil {
writeError(w, 500, err)
return
}
diffCmd := exec.Command("diff", "-u", path.Join(base, "scripts", "tmp", "full_spec.rst"), path.Join(head, "scripts", "tmp", "full_spec.rst"))
var diff bytes.Buffer
diffCmd.Stdout = &diff
if err := ignoreExitCodeOne(diffCmd.Run()); err != nil {
writeError(w, 500, fmt.Errorf("error running diff: %v", err))
return
}
w.Write(diff.Bytes())
}
func (s *server) serveHTMLDiff(w http.ResponseWriter, req *http.Request) {
pr, err := lookupPullRequest(*req.URL, "/diff/html")
if err != nil {
writeError(w, 400, err)
return
}
// We're going to run whatever Python is specified in the pull request, which
// may do bad things, so only trust people we trust.
if err := checkAuth(pr); err != nil {
writeError(w, 403, err)
return
}
base, err := s.generateAt(pr.Base.SHA)
defer os.RemoveAll(base)
if err != nil {
writeError(w, 500, err)
return
}
head, err := s.generateAt(pr.Head.SHA)
defer os.RemoveAll(head)
if err != nil {
writeError(w, 500, err)
return
}
htmlDiffer, err := findHTMLDiffer()
if err != nil {
writeError(w, 500, fmt.Errorf("could not find HTML differ"))
return
}
cmd := exec.Command(htmlDiffer, path.Join(base, "scripts", "gen", "specification.html"), path.Join(head, "scripts", "gen", "specification.html"))
var b bytes.Buffer
cmd.Stdout = &b
if err := cmd.Run(); err != nil {
writeError(w, 500, fmt.Errorf("error running HTML differ: %v", err))
return
}
w.Write(b.Bytes())
}
func findHTMLDiffer() (string, error) {
wd, err := os.Getwd()
if err != nil {
return "", err
}
differ := path.Join(wd, "htmldiff.pl")
if _, err := os.Stat(differ); err == nil {
return differ, nil
}
return "", fmt.Errorf("unable to find htmldiff.pl")
}
func listPulls(w http.ResponseWriter, req *http.Request) {
resp, err := http.Get(pullsPrefix)
if err != nil {
writeError(w, 500, err)
return
}
defer resp.Body.Close()
dec := json.NewDecoder(resp.Body)
var pulls []PullRequest
if err := dec.Decode(&pulls); err != nil {
writeError(w, 500, err)
return
}
if len(pulls) == 0 {
io.WriteString(w, "No pull requests found")
return
}
s := "<body><ul>"
for _, pull := range pulls {
s += fmt.Sprintf(`<li>%d: <a href="%s">%s</a>: <a href="%s">%s</a>: <a href="spec/%d">spec</a> <a href="diff/html/%d">spec diff</a> <a href="diff/rst/%d">rst diff</a></li>`,
pull.Number, pull.User.HTMLURL, pull.User.Login, pull.HTMLURL, pull.Title, pull.Number, pull.Number, pull.Number)
}
s += `</ul><div><a href="spec/head">View the spec at head</a></div></body>`
io.WriteString(w, s)
}
func ignoreExitCodeOne(err error) error {
if err == nil {
return err
}
if exiterr, ok := err.(*exec.ExitError); ok {
if status, ok := exiterr.Sys().(syscall.WaitStatus); ok {
if status.ExitStatus() == 1 {
return nil
}
}
}
return err
}
func main() {
flag.Parse()
// It would be great to read this from github, but there's no convenient way to do so.
// Most of these memberships are "private", so would require some kind of auth.
allowedMembers = map[string]bool{
"dbkr": true,
"erikjohnston": true,
"illicitonion": true,
"Kegsay": true,
"NegativeMjark": true,
}
rand.Seed(time.Now().Unix())
masterCloneDir, err := gitClone(matrixDocCloneURL, false)
if err != nil {
log.Fatal(err)
}
s := server{masterCloneDir}
http.HandleFunc("/spec/", s.serveSpec)
http.HandleFunc("/diff/rst/", s.serveRSTDiff)
http.HandleFunc("/diff/html/", s.serveHTMLDiff)
http.HandleFunc("/healthz", serveText("ok"))
http.HandleFunc("/", listPulls)
fmt.Printf("Listening on port %d\n", *port)
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), nil))
}
func serveText(s string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, s)
}
}

@ -1,8 +1,8 @@
Signing Events
==============
--------------
Canonical JSON
--------------
~~~~~~~~~~~~~~
Matrix events are represented using JSON objects. If we want to sign JSON
events we need to encode the JSON as a binary string. Unfortunately the same
@ -30,7 +30,7 @@ using this representation.
value,
# Encode code-points outside of ASCII as UTF-8 rather than \u escapes
ensure_ascii=False,
# Remove unecessary white space.
# Remove unnecessary white space.
separators=(',',':'),
# Sort the keys of dictionaries.
sort_keys=True,
@ -38,7 +38,7 @@ using this representation.
).encode("UTF-8")
Grammar
~~~~~~~
+++++++
Adapted from the grammar in http://tools.ietf.org/html/rfc7159 removing
insignificant whitespace, fractions, exponents and redundant character escapes
@ -69,14 +69,14 @@ insignificant whitespace, fractions, exponents and redundant character escapes
/ %x75.30.30.31 (%x30-39 / %x61-66) ; u001X
Signing JSON
------------
~~~~~~~~~~~~
We can now sign a JSON object by encoding it as a sequence of bytes, computing
the signature for that sequence and then adding the signature to the original
JSON object.
Signing Details
~~~~~~~~~~~~~~~
+++++++++++++++
JSON is signed by encoding the JSON object without ``signatures`` or keys grouped
as ``unsigned``, using the canonical encoding described above. The JSON bytes are then signed using the
@ -133,7 +133,7 @@ and additional signatures.
return json_object
Checking for a Signature
~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++
To check if an entity has signed a JSON object a server does the following
@ -151,7 +151,7 @@ To check if an entity has signed a JSON object a server does the following
the check fails. Otherwise the check succeeds.
Signing Events
--------------
~~~~~~~~~~~~~~
Signing events is a more complicated process since servers can choose to redact
non-essential parts of an event. Before signing the event it is encoded as

@ -0,0 +1,38 @@
Events
======
All communication in Matrix is expressed in the form of data objects called
Events. These are the fundamental building blocks common to the client-server,
server-server and application-service APIs, and are described below.
{{common_event_fields}}
{{common_room_event_fields}}
{{common_state_event_fields}}
Room Events
-----------
.. NOTE::
This section is a work in progress.
This specification outlines several standard event types, all of which are
prefixed with ``m.``
{{m_room_aliases_event}}
{{m_room_canonical_alias_event}}
{{m_room_create_event}}
{{m_room_history_visibility_event}}
{{m_room_join_rules_event}}
{{m_room_member_event}}
{{m_room_power_levels_event}}
{{m_room_redaction_event}}

@ -0,0 +1,93 @@
Feature Profiles
================
.. sect:feature-profiles:
Matrix supports many different kinds of clients: from embedded IoT devices to
desktop clients. Not all clients can provide the same feature sets as other
clients e.g. due to lack of physical hardware such as not having a screen.
Clients can fall into one of several profiles and each profile contains a set
of features that the client MUST support. This section details a set of
"feature profiles". Clients are expected to implement a profile in its entirety
in order for it to be classified as that profile.
Summary
-------
===================================== ========== ========== ========== ========== ==========
Module / Profile Web Mobile Desktop CLI Embedded
===================================== ========== ========== ========== ========== ==========
`Instant Messaging`_ Required Required Required Required Optional
`Presence`_ Required Required Required Required Optional
`Push Notifications`_ Optional Required Optional Optional Optional
`Receipts`_ Required Required Required Required Optional
`Typing Notifications`_ Required Required Required Required Optional
`VoIP`_ Required Required Required Optional Optional
`Content Repository`_ Required Required Required Optional Optional
`Managing History Visibility`_ Required Required Required Required Optional
`End-to-End Encryption`_ Optional Optional Optional Optional Optional
===================================== ========== ========== ========== ========== ==========
*Please see each module for more details on what clients need to implement.*
.. _End-to-End Encryption: `module:e2e`_
.. _Instant Messaging: `module:im`_
.. _Presence: `module:presence`_
.. _Push Notifications: `module:push`_
.. _Receipts: `module:receipts`_
.. _Typing Notifications: `module:typing`_
.. _VoIP: `module:voip`_
.. _Content Repository: `module:content`_
.. _Managing History Visibility: `module:history-visibility`_
Clients
-------
Stand-alone web (``Web``)
~~~~~~~~~~~~~~~~~~~~~~~~~
This is a web page which heavily uses Matrix for communication. Single-page web
apps would be classified as a stand-alone web client, as would multi-page web
apps which use Matrix on nearly every page.
Mobile (``Mobile``)
~~~~~~~~~~~~~~~~~~~
This is a Matrix client specifically designed for consumption on mobile devices.
This is typically a mobile app but need not be so provided the feature set can
be reached (e.g. if a mobile site could display push notifications it could be
classified as a mobile client).
Desktop (``Desktop``)
~~~~~~~~~~~~~~~~~~~~~
This is a native GUI application which can run in its own environment outside a
browser.
Command Line Interface (``CLI``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a client which is used via a text-based terminal.
Embedded (``Embedded``)
~~~~~~~~~~~~~~~~~~~~~~~
This is a client which is embedded into another application or an embedded
device.
Application
+++++++++++
This is a Matrix client which is embedded in another website, e.g. using
iframes. These embedded clients are typically for a single purpose
related to the website in question, and are not intended to be fully-fledged
communication apps.
Device
++++++
This is a client which is typically running on an embedded device such as a
kettle, fridge or car. These clients tend to perform a few operations and run
in a resource constrained environment. Like embedded applications, they are
not intended to be fully-fledged communication systems.

@ -24,7 +24,6 @@ Introduction
The Matrix specification is still evolving: the APIs are not yet frozen
and this document is in places a work in progress or stale. We have made every
effort to clearly flag areas which are still being finalised.
We're publishing it at this point because it's complete enough to be more than
useful and provide a canonical reference to how Matrix is evolving. Our end
goal is to mirror WHATWG's `Living Standard
@ -34,10 +33,9 @@ Matrix is a set of open APIs for open-federated Instant Messaging (IM), Voice
over IP (VoIP) and Internet of Things (IoT) communication, designed to create
and support a new global real-time communication ecosystem. The intention is to
provide an open decentralised pubsub layer for the internet for securely
persisting and publishing/subscribing JSON objects.
This specification is the ongoing result of standardising the APIs used by the
various components of the Matrix ecosystem to communicate with one another.
persisting and publishing/subscribing JSON objects. This specification is the
ongoing result of standardising the APIs used by the various components of the
Matrix ecosystem to communicate with one another.
The principles that Matrix attempts to follow are:
@ -79,7 +77,7 @@ The functionality that Matrix provides includes:
- Extensible user management (inviting, joining, leaving, kicking, banning)
mediated by a power-level based user privilege system.
- Extensible room state management (room naming, aliasing, topics, bans)
- Extensible user profile management (avatars, displaynames, etc)
- Extensible user profile management (avatars, display names, etc)
- Managing user accounts (registration, login, logout)
- Use of 3rd Party IDs (3PIDs) such as email addresses, phone numbers,
Facebook accounts to authenticate, identify and discover users on Matrix.
@ -91,7 +89,7 @@ The functionality that Matrix provides includes:
The end goal of Matrix is to be a ubiquitous messaging layer for synchronising
arbitrary data between sets of people, devices and services - be that for
instant messages, VoIP call setups, or any other objects that need to be
reliably and persistently pushed from A to B in an interoperable and federated
reliably and persistently pushed from A to B in an inter-operable and federated
manner.
Overview
@ -171,20 +169,23 @@ All data exchanged over Matrix is expressed as an "event". Typically each client
action (e.g. sending a message) correlates with exactly one event. Each event
has a ``type`` which is used to differentiate different kinds of data. ``type``
values MUST be uniquely globally namespaced following Java's `package naming
conventions
<http://docs.oracle.com/javase/specs/jls/se5.0/html/packages.html#7.7>`, e.g.
conventions`_, e.g.
``com.example.myapp.event``. The special top-level namespace ``m.`` is reserved
for events defined in the Matrix specification - for instance ``m.room.message``
is the event type for instant messages. Events are usually sent in the context
of a "Room".
.. _package naming conventions: https://en.wikipedia.org/wiki/Java_package#Package_naming_conventions
Event Graphs
~~~~~~~~~~~~
.. _sect:event-graph:
Events exchanged in the context of a room are stored in a directed acyclic graph
(DAG) called an ``event graph``. The partial ordering of this graph gives the
chronological ordering of events within the room. Each event in the graph has a
list of zero or more ``parent`` events, which refer to any preceeding events
list of zero or more ``parent`` events, which refer to any preceding events
which have no chronological successor from the perspective of the homeserver
which created the event.
@ -213,10 +214,8 @@ which have the form::
There is exactly one room ID for each room. Whilst the room ID does contain a
domain, it is simply for globally namespacing room IDs. The room does NOT
reside on the domain specified. Room IDs are not meant to be human readable.
They are case-sensitive.
The following conceptual diagram shows an ``m.room.message`` event being sent to
the room ``!qporfwt:matrix.org``::
They are case-sensitive. The following conceptual diagram shows an
``m.room.message`` event being sent to the room ``!qporfwt:matrix.org``::
{ @alice:matrix.org } { @bob:domain.com }
| ^
@ -257,28 +256,28 @@ the room ``!qporfwt:matrix.org``::
Federation maintains *shared data structures* per-room between multiple home
servers. The data is split into ``message events`` and ``state events``.
``Message events`` describe transient 'once-off' activity in a room such as an
instant messages, VoIP call setups, file transfers, etc. They generally describe
communication activity.
Message events:
These describe transient 'once-off' activity in a room such as an
instant messages, VoIP call setups, file transfers, etc. They generally
describe communication activity.
``State events`` describe updates to a given piece of persistent information
('state') related to a room, such as the room's name, topic, membership,
participating servers, etc. State is modelled as a lookup table of key/value
pairs per room, with each key being a tuple of ``state_key`` and ``event type``.
Each state event updates the value of a given key.
State events:
These describe updates to a given piece of persistent information
('state') related to a room, such as the room's name, topic, membership,
participating servers, etc. State is modelled as a lookup table of key/value
pairs per room, with each key being a tuple of ``state_key`` and ``event type``.
Each state event updates the value of a given key.
The state of the room at a given point is calculated by considering all events
preceding and including a given event in the graph. Where events describe the
same state, a merge conflict algorithm is applied. The state resolution
algorithm is transitive and does not depend on server state, as it must
consistently select the same event irrespective of the server or the order the
events were received in.
Events are signed by the originating server (the signature includes the parent
relations, type, depth and payload hash) and are pushed over federation to the
participating servers in a room, currently using full mesh topology. Servers may
also request backfill of events over federation from the other servers
participating in a room.
events were received in. Events are signed by the originating server (the
signature includes the parent relations, type, depth and payload hash) and are
pushed over federation to the participating servers in a room, currently using
full mesh topology. Servers may also request backfill of events over federation
from the other servers participating in a room.
Room Aliases
@ -323,12 +322,10 @@ Users in Matrix are identified via their matrix user ID (MXID). However,
existing 3rd party ID namespaces can also be used in order to identify Matrix
users. A Matrix "Identity" describes both the user ID and any other existing IDs
from third party namespaces *linked* to their account.
Matrix users can *link* third-party IDs (3PIDs) such as email addresses, social
network accounts and phone numbers to their user ID. Linking 3PIDs creates a
mapping from a 3PID to a user ID. This mapping can then be used by Matrix
users in order to discover the MXIDs of their contacts.
In order to ensure that the mapping from 3PID to user ID is genuine, a globally
federated cluster of trusted "Identity Servers" (IS) are used to verify the 3PID
and persist and replicate the mappings.
@ -337,55 +334,13 @@ Usage of an IS is not required in order for a client application to be part of
the Matrix ecosystem. However, without one clients will not be able to look up
user IDs using 3PIDs.
Presence
~~~~~~~~
Each user has the concept of presence information. This encodes:
* Whether the user is currently online
* How recently the user was last active (as seen by the server)
* Whether a given client considers the user to be currently idle
* Arbitrary information about the user's current status (e.g. "in a meeting").
This information is collated from both per-device (online; idle; last_active) and
per-user (status) data, aggregated by the user's homeserver and transmitted as
an ``m.presence`` event. This is one of the few events which are sent *outside
the context of a room*. Presence events are sent to all users who subscribe to
this user's presence through a presence list or by sharing membership of a room.
.. TODO
How do we let users hide their presence information?
.. TODO
The last_active specifics should be moved to the detailed presence event section
Last activity is tracked by the server maintaining a timestamp of the last time
it saw a pro-active event from the user. Any event which could be triggered by a
human using the application is considered pro-active (e.g. sending an event to a
room). An example of a non-proactive client activity would be a client setting
'idle' presence status, or polling for events. This timestamp is presented via a
key called ``last_active_ago``, which gives the relative number of milliseconds
since the message is generated/emitted that the user was last seen active.
N.B. in v1 API, status/online/idle state are muxed into a single 'presence' field on the m.presence event.
Presence Lists
~~~~~~~~~~~~~~
Each user's home server stores a "presence list". This stores a list of user IDs
whose presence the user wants to follow.
To be added to this list, the user being added must be invited by the list owner
and accept the invitation. Once accepted, both user's HSes track the
subscription.
Profiles
~~~~~~~~
Users may publish arbitrary key/value data associated with their account - such
as a human readable ``display name``, a profile photo URL, contact information
(email address, phone nubers, website URLs etc).
(email address, phone numbers, website URLs etc).
In Client-Server API v2, profile data is typed using namespaced keys for
interoperability, much like events - e.g. ``m.profile.display_name``.
@ -408,6 +363,10 @@ dedicated API. The API is symmetrical to managing Profile data.
API Standards
-------------
.. TODO
Need to specify any HMAC or access_token lifetime/ratcheting tricks
We need to specify capability negotiation for extensible transports
The mandatory baseline for communication in Matrix is exchanging JSON objects
over HTTP APIs. HTTPS is mandated as the baseline for server-server
(federation) communication. HTTPS is recommended for client-server
@ -415,20 +374,11 @@ communication, although HTTP may be supported as a fallback to support basic
HTTP clients. More efficient optional transports for client-server
communication will in future be supported as optional extensions - e.g. a
packed binary encoding over stream-cipher encrypted TCP socket for
low-bandwidth/low-roundtrip mobile usage.
.. TODO
We need to specify capability negotiation for extensible transports
For the default HTTP transport, all API calls use a Content-Type of
``application/json``. In addition, all strings MUST be encoded as UTF-8.
Clients are authenticated using opaque ``access_token`` strings (see
`Client Authentication`_ for details), passed as a query string parameter on
all requests.
.. TODO
Need to specify any HMAC or access_token lifetime/ratcheting tricks
low-bandwidth/low-roundtrip mobile usage. For the default HTTP transport, all
API calls use a Content-Type of ``application/json``. In addition, all strings
MUST be encoded as UTF-8. Clients are authenticated using opaque
``access_token`` strings (see `Client Authentication`_ for details), passed as a
query string parameter on all requests.
Any errors which occur at the Matrix API level MUST return a "standard error
response". This is a JSON object which looks like::
@ -442,7 +392,7 @@ The ``error`` string will be a human-readable error message, usually a sentence
explaining what went wrong. The ``errcode`` string will be a unique string
which can be used to handle an error message e.g. ``M_FORBIDDEN``. These error
codes should have their namespace first in ALL CAPS, followed by a single _ to
ease seperating the namespace from the error code.. For example, if there was a
ease separating the namespace from the error code. For example, if there was a
custom namespace ``com.mydomain.here``, and a
``FORBIDDEN`` code, the error code should look like
``COM.MYDOMAIN.HERE_FORBIDDEN``. There may be additional keys depending on the

@ -1,5 +1,5 @@
Client-Server API v1
====================
Client-Server API
=================
Overview
--------
@ -8,11 +8,11 @@ The client-server API provides a simple lightweight API to let clients send
messages, control rooms and synchronise conversation history. It is designed to
support both lightweight clients which store no state and lazy-load data from
the server as required - as well as heavyweight clients which maintain a full
local peristent copy of server state.
local persistent copy of server state.
This mostly describes v1 of the Client-Server API as featured in the original September
2014 launch of Matrix, apart from user-interactive authentication where it is
encouraged to move to V2, therefore this is the version documented here.
encouraged to move to v2, therefore this is the version documented here.
Version 2 is currently in development (as of Jan-March 2015) as an incremental
but backwards-incompatible refinement of Version 1 and will be released
shortly.
@ -154,7 +154,7 @@ Matrix client, for example, an email confirmation may be completed when the user
clicks on the link in the email. In this case, the client retries the request
with an auth dict containing only the session key. The response to this will be
the same as if the client were attempting to complete an auth state normally,
ie. the request will either complete or request auth, with the presence or
i.e. the request will either complete or request auth, with the presence or
absence of that login stage type in the 'completed' array indicating whether
that stage is complete.
@ -197,6 +197,7 @@ This specification defines the following login types:
- ``m.login.recaptcha``
- ``m.login.oauth2``
- ``m.login.email.identity``
- ``m.login.token``
- ``m.login.dummy``
Password-based
@ -204,7 +205,7 @@ Password-based
:Type:
``m.login.password``
:Description:
The client submits a username and secret password, both sent in plaintext.
The client submits a username and secret password, both sent in plain-text.
To respond to this type, reply with an auth dict as follows::
@ -228,6 +229,37 @@ To respond to this type, reply with an auth dict as follows::
"response": "<captcha response>"
}
Token-based
~~~~~~~~~~~
:Type:
``m.login.token``
:Description:
The client submits a username and token.
To respond to this type, reply with an auth dict as follows::
{
"type": "m.login.token",
"user": "<user_id or user localpart>",
"token": "<token>",
"txn_id": "<client generated nonce>"
}
The ``nonce`` should be a random string generated by the client for the
request. The same ``nonce`` should be used if retrying the request.
There are many ways a client may receive a ``token``, including via an email or
from an existing logged in device.
The ``txn_id`` may be used by the server to disallow other devices from using
the token, thus providing "single use" tokens while still allowing the device
to retry the request. This would be done by tying the token to the ``txn_id``
server side, as well as potentially invalidating the token completely once the
device has successfully logged in (e.g. when we receive a request from the
newly provisioned access_token).
The ``token`` must be a macaroon.
OAuth2-based
~~~~~~~~~~~~
:Type:
@ -247,10 +279,10 @@ service which the home server accepts when logging in, this indirection can be
skipped and the "uri" key can be the ``Authorization Request URI``.
The client then visits the ``Authorization Request URI``, which then shows the
OAuth2 Allow/Deny prompt. Hitting 'Allow' returns the [XXX: redirects to the?]``redirect URI`` with the
auth code. Home servers can choose any path for the ``redirect URI``. Once the
OAuth flow has completed, the client retries the request with the session only,
as above.
OAuth2 Allow/Deny prompt. Hitting 'Allow' redirects to the ``redirect URI`` with
the auth code. Home servers can choose any path for the ``redirect URI``. Once
the OAuth flow has completed, the client retries the request with the session
only, as above.
Email-based (identity server)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -308,7 +340,7 @@ Where ``stage type`` is the type name of the stage it is attempting and
``session id`` is the ID of the session given by the home server.
This MUST return an HTML page which can perform this authentication stage. This
page must attempt to call the Javascript function ``window.onAuthDone`` when
page must attempt to call the JavaScript function ``window.onAuthDone`` when
the authentication has been completed.
Pagination
@ -373,7 +405,7 @@ now show page 3 (rooms R11 -> 15)::
Returns: R11,R12,R13,R14,R15
Note that tokens are treated in an *exclusive*, not inclusive, manner. The end
token from the intial request was '9' which corresponded to R10. When the 2nd
token from the initial request was '9' which corresponded to R10. When the 2nd
request was made, R10 did not appear again, even though from=9 was specified. If
you know the token, you already have the data.
@ -395,6 +427,8 @@ the complete dataset is provided in "chunk".
Events
------
.. _sect:events:
Overview
~~~~~~~~
@ -425,9 +459,9 @@ You can visualise the range of events being returned as::
| |
start: '1-2-3' end: 'a-b-c'
Now, to receive future events in realtime on the eventstream, you simply GET
Now, to receive future events in real-time on the eventstream, you simply GET
$PREFIX/events with a ``from`` parameter of 'a-b-c': in other words passing in the
``end`` token returned by initialsync. The request blocks until new events are
``end`` token returned by initial sync. The request blocks until new events are
available or until your specified timeout elapses, and then returns a
new paginatable chunk of events alongside new start and end parameters::
@ -467,7 +501,7 @@ event stream. When the request returns, an ``end`` token is included in the
response. This token can be used in the next request to continue where the
last request left off.
All events must be deduplicated based on their event ID.
All events must be de-duplicated based on their event ID.
.. TODO
is deduplication actually a hard requirement in CS v2?
@ -493,7 +527,7 @@ Room events are split into two categories:
:Message events:
These are events which describe transient "once-off" activity in a room:
typically communication such as sending an instant messaage or setting up a
typically communication such as sending an instant message or setting up a
VoIP call. These used to be called 'non-state' events.
This specification outlines several events, all with the event type prefix
@ -631,49 +665,7 @@ Getting events for a room
There are several APIs provided to ``GET`` events for a room:
``/rooms/<room id>/state/<event type>/<state key>``
Description:
Get the state event identified.
Response format:
A JSON object representing the state event **content**.
Example:
``/rooms/!room:domain.com/state/m.room.name`` returns ``{ "name": "Room name" }``
|/rooms/<room_id>/state|_
Description:
Get all state events for a room.
Response format:
``[ { state event }, { state event }, ... ]``
Example:
TODO-doc
|/rooms/<room_id>/members|_
Description:
Get all ``m.room.member`` state events.
Response format:
``{ "start": "<token>", "end": "<token>", "chunk": [ { m.room.member event }, ... ] }``
Example:
TODO-doc
|/rooms/<room_id>/messages|_
Description:
Get all events from the room's timeline. This API supports
pagination using ``from`` and ``to`` query parameters, coupled with the
``start`` and ``end`` tokens from an |initialSync|_ API.
Response format:
``{ "start": "<token>", "end": "<token>" }``
Example:
TODO-doc
|/rooms/<room_id>/initialSync|_
Description:
Get all relevant events for a room. This includes state events, paginated
non-state events and presence events.
Response format:
`` { TODO-doc } ``
Example:
TODO-doc
{{rooms_http_api}}
Redactions
~~~~~~~~~~
@ -682,13 +674,10 @@ to add keys that are, for example offensive or illegal. Since some events
cannot be simply deleted, e.g. membership events, we instead 'redact' events.
This involves removing all keys from an event that are not required by the
protocol. This stripped down event is thereafter returned anytime a client or
remote server requests it.
Events that have been redacted include a ``redacted_because`` key whose value
is the event that caused it to be redacted, which may include a reason.
Redacting an event cannot be undone, allowing server owners to delete the
offending content from the databases.
remote server requests it. Redacting an event cannot be undone, allowing server
owners to delete the offending content from the databases. Events that have been
redacted include a ``redacted_because`` key whose value is the event that caused
it to be redacted, which may include a reason.
.. TODO
Currently, only room admins can redact events by sending a ``m.room.redaction``
@ -719,12 +708,10 @@ one of the following event types:
.. TODO
Need to update m.room.power_levels to reflect new power levels formatting
The redaction event should be added under the key ``redacted_because``.
When a client receives a redaction event it should change the redacted event
The redaction event should be added under the key ``redacted_because``. When a
client receives a redaction event it should change the redacted event
in the same way a server does.
Rooms
-----
@ -792,13 +779,65 @@ options which can be set when creating a room:
This will tell the server to invite everyone in the list to the newly
created room.
``creation_content``
Type:
Object
Optional:
Yes
Value:
Extra keys to be added to the content of the ``m.room.create``. The server
will clober the following keys: ``creator``. Future versions of this
spec may allow the server to clobber other keys if required.
Description:
Allows clients to add keys to the content of ``m.room.create``.
``preset``
Type:
String
Optional:
Yes
Value:
``private_chat``, ``trusted_private_chat`` or ``public_chat``
Description:
Convenience parameter for setting various default state events based on a
preset.
Three presets are defined:
- ``private_chat``: Sets the ``join_rules`` to ``invite`` and
``history_visibility`` to ``shared``
- ``trusted_private_chat``: Set the ``join_rules`` to ``invite``,
``history_visibility`` to ``shared`` and gives all invitees the same
power level as the creator.
- ``public_chat``: Sets the ``join_rules`` to ``public`` and
``history_visibility`` to ``shared``
``initial_state``
Type:
List
Optional:
Yes
Value:
A list of state events to set in the new room.
Description:
Allows the user to override the default state events set in the new room.
The expected format of the state events are an object with ``type``,
``state_key`` and ``content`` keys set.
Takes precedence over events set by ``presets``, but gets overriden by
``name`` and ``topic`` keys.
Example::
{
"visibility": "public",
"preset": "public_chat",
"room_alias_name": "thepub",
"name": "The Grand Duke Pub",
"topic": "All about happy hour"
"topic": "All about happy hour",
"creation_content": {
"m.federate": false
}
}
The home server will create a ``m.room.create`` event when the room is created,
@ -863,72 +902,33 @@ Permissions
Permissions for rooms are done via the concept of power levels - to do any
action in a room a user must have a suitable power level. Power levels are
stored as state events in a given room.
The power levels required for operations and the power levels for users are
defined in ``m.room.power_levels``, where both a default and specific users'
power levels can be set.
stored as state events in a given room. The power levels required for operations
and the power levels for users are defined in ``m.room.power_levels``, where
both a default and specific users' power levels can be set.
By default all users have a power level of 0, other than the room creator whose
power level defaults to 100. Users can grant other users increased power levels
up to their own power level. For example, user A with a power level of 50 could
increase the power level of user B to a maximum of level 50. Power levels for
users are tracked per-room even if the user is not present in the room.
The keys contained in ``m.room.power_levels`` determine the levels required for
certain operations such as kicking, banning and sending state events. See
`m.room.power_levels`_ for more information.
Joining rooms
~~~~~~~~~~~~~
.. TODO-doc What does the home server have to do to join a user to a room?
- See SPEC-30.
Users need to join a room in order to send and receive events in that room. A
user can join a room by making a request to |/join/<room_alias_or_id>|_ with::
{}
Alternatively, a user can make a request to |/rooms/<room_id>/join|_ with the
same request content. This is only provided for symmetry with the other
membership APIs: ``/rooms/<room id>/invite`` and ``/rooms/<room id>/leave``. If
a room alias was specified, it will be automatically resolved to a room ID,
which will then be joined. The room ID that was joined will be returned in
response::
-------------
Users need to be a member of a room in order to send and receive events in that
room. There are several states in which a user may be, in relation to a room:
{
"room_id": "!roomid:domain"
}
The membership state for the joining user can also be modified directly to be
``join`` by sending the following request to
``/rooms/<room id>/state/m.room.member/<url encoded user id>``::
{
"membership": "join"
}
- Unrelated (the user cannot send or receive events in the room)
- Invited (the user has been invited to participate in the room, but is not
yet participating)
- Joined (the user can send and receive events in the room)
- Banned (the user is not allowed to join the room)
See the `Room events`_ section for more information on ``m.room.member``.
After the user has joined a room, they will receive subsequent events in that
room. This room will now appear as an entry in the |initialSync|_ API.
Some rooms enforce that a user is *invited* to a room before they can join that
room. Other rooms will allow anyone to join the room even if they have not
received an invite.
Inviting users
~~~~~~~~~~~~~~
.. TODO-doc Invite-join dance
- Outline invite join dance. What is it? Why is it required? How does it work?
- What does the home server have to do?
The purpose of inviting users to a room is to notify them that the room exists
so they can choose to become a member of that room. Some rooms require that all
users who join a room are previously invited to it (an "invite-only" room).
Whether a given room is an "invite-only" room is determined by the room config
key ``m.room.join_rules``. It can have one of the following values:
Some rooms require that users be invited to it before they can join; others
allow anyone to join. Whether a given room is an "invite-only" room is
determined by the room config key ``m.room.join_rules``. It can have one of the
following values:
``public``
This room is free for anyone to join without an invite.
@ -936,26 +936,7 @@ key ``m.room.join_rules``. It can have one of the following values:
``invite``
This room can only be joined if you were invited.
Only users who have a membership state of ``join`` in a room can invite new
users to said room. The person being invited must not be in the ``join`` state
in the room. The fully-qualified user ID must be specified when inviting a
user, as the user may reside on a different home server. To invite a user, send
the following request to |/rooms/<room_id>/invite|_, which will manage the
entire invitation process::
{
"user_id": "<user id to invite>"
}
Alternatively, the membership state for this user in this room can be modified
directly by sending the following request to
``/rooms/<room id>/state/m.room.member/<url encoded user id>``::
{
"membership": "invite"
}
See the `Room events`_ section for more information on ``m.room.member``.
{{membership_http_api}}
Leaving rooms
~~~~~~~~~~~~~
@ -985,11 +966,8 @@ directly by sending the following request to
"membership": "leave"
}
See the `Room events`_ section for more information on ``m.room.member``.
Once a user has left a room, that room will no longer appear on the
|initialSync|_ API.
See the `Room events`_ section for more information on ``m.room.member``. Once a
user has left a room, that room will no longer appear on the |initialSync|_ API.
If all members in a room leave, that room becomes eligible for deletion.
Banning users in a room
@ -1027,7 +1005,7 @@ Registering for a user account is done using the request::
POST $V2PREFIX/register
This API endpoint uses the User-Interactive Authentication API.
This API endoint does not require an access token.
This API endpoint does not require an access token.
The body of the POST request is a JSON object containing:
@ -1064,32 +1042,7 @@ was registered whilst the client was performing authentication.
Old V1 API docs: |register|_
Login
~~~~~
This section refers to API Version 1.
API docs: |login|_
Obtaining an access token for an existing user account is done using the
request::
POST $PREFIX/login
The body of the POST request is a JSON object containing:
username
The full qualified or local part of the Matrix ID to log in with.
password
The password for the account.
On success, this returns a JSON object with keys:
user_id
The fully-qualified Matrix ID that has been registered.
access_token
An access token for the new account.
home_server
The hostname of the Home Server on which the account has been registered.
{{login_http_api}}
Changing Password
~~~~~~~~~~~~~~~~~
@ -1140,7 +1093,7 @@ The third party identifier credentials object comprises:
id_server
The colon-separated hostname and port of the Identity Server used to
authenticate the third party identifer. If the port is the default, it and the
authenticate the third party identifier. If the port is the default, it and the
colon should be omitted.
sid
The session ID given by the Identity Server
@ -1177,18 +1130,33 @@ medium
address
The textual address of the 3pid, eg. the email address
Presence
--------
.. TODO-spec
- Define how users receive presence invites, and how they accept/decline them
{{presence_http_api}}
Profiles
--------
{{profile_http_api}}
Events on Change of Profile Information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because the profile display name and avatar information are likely to be used in
many places of a client's display, changes to these fields cause an automatic
propagation event to occur, informing likely-interested parties of the new
values. This change is conveyed using two separate mechanisms:
- a ``m.room.member`` event is sent to every room the user is a member of,
to update the ``displayname`` and ``avatar_url``.
- a ``m.presence`` presence status update is sent, again containing the new
values of the ``displayname`` and ``avatar_url`` keys, in addition to the
required ``presence`` key containing the current presence state of the user.
Both of these should be done automatically by the home server when a user
successfully changes their display name or avatar URL fields.
Additionally, when home servers emit room membership events for their own
users, they should include the display name and avatar URL fields in these
events so that clients already have these details to hand, and do not have to
perform extra round trips to query it.
Security
--------

@ -0,0 +1,3 @@
Modules
=======

@ -1,157 +0,0 @@
Events
======
All communication in Matrix is expressed in the form of data objects called
Events. These are the fundamental building blocks common to the client-server,
server-server and application-service APIs, and are described below.
{{common_event_fields}}
{{common_room_event_fields}}
{{common_state_event_fields}}
Room Events
-----------
.. NOTE::
This section is a work in progress.
This specification outlines several standard event types, all of which are
prefixed with ``m.``
{{room_events}}
m.room.message msgtypes
~~~~~~~~~~~~~~~~~~~~~~~
.. TODO-spec
How a client should handle unknown message types.
Each `m.room.message`_ MUST have a ``msgtype`` key which identifies the type
of message being sent. Each type has their own required and optional keys, as
outlined below.
{{msgtype_events}}
Presence Events
~~~~~~~~~~~~~~~
{{presence_events}}
Each user has the concept of presence information. This encodes the
"availability" of that user, suitable for display on other user's clients.
This is transmitted as an ``m.presence`` event and is one of the few events
which are sent *outside the context of a room*. The basic piece of presence
information is represented by the ``presence`` key, which is an enum of one
of the following:
- ``online`` : The default state when the user is connected to an event
stream.
- ``unavailable`` : The user is not reachable at this time.
- ``offline`` : The user is not connected to an event stream.
- ``free_for_chat`` : The user is generally willing to receive messages
moreso than default.
- ``hidden`` : Behaves as offline, but allows the user to see the client
state anyway and generally interact with client features. (Not yet
implemented in synapse).
In addition, the server maintains a timestamp of the last time it saw a
pro-active event from the user; either sending a message to a room, or
changing presence state from a lower to a higher level of availability
(thus: changing state from ``unavailable`` to ``online`` counts as a
proactive event, whereas in the other direction it will not). This timestamp
is presented via a key called ``last_active_ago``, which gives the relative
number of milliseconds since the message is generated/emitted that the user
was last seen active.
Events on Change of Profile Information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because the profile displayname and avatar information are likely to be used in
many places of a client's display, changes to these fields cause an automatic
propagation event to occur, informing likely-interested parties of the new
values. This change is conveyed using two separate mechanisms:
- a ``m.room.member`` event is sent to every room the user is a member of,
to update the ``displayname`` and ``avatar_url``.
- a ``m.presence`` presence status update is sent, again containing the new values of the
``displayname`` and ``avatar_url`` keys, in addition to the required
``presence`` key containing the current presence state of the user.
Both of these should be done automatically by the home server when a user
successfully changes their displayname or avatar URL fields.
Additionally, when home servers emit room membership events for their own
users, they should include the displayname and avatar URL fields in these
events so that clients already have these details to hand, and do not have to
perform extra roundtrips to query it.
Voice over IP
-------------
Matrix can also be used to set up VoIP calls. This is part of the core
specification, although is at a relatively early stage. Voice (and video) over
Matrix is built on the WebRTC 1.0 standard.
Call events are sent to a room, like any other event. This means that clients
must only send call events to rooms with exactly two participants as currently
the WebRTC standard is based around two-party communication.
{{voip_events}}
Message Exchange
~~~~~~~~~~~~~~~~
A call is set up with messages exchanged as follows:
::
Caller Callee
m.call.invite ----------->
m.call.candidate -------->
[more candidates events]
User answers call
<------ m.call.answer
[...]
<------ m.call.hangup
Or a rejected call:
::
Caller Callee
m.call.invite ----------->
m.call.candidate -------->
[more candidates events]
User rejects call
<------- m.call.hangup
Calls are negotiated according to the WebRTC specification.
Glare
~~~~~
This specification aims to address the problem of two users calling each other
at roughly the same time and their invites crossing on the wire. It is a far
better experience for the users if their calls are connected if it is clear
that their intention is to set up a call with one another.
In Matrix, calls are to rooms rather than users (even if those rooms may only
contain one other user) so we consider calls which are to the same room.
The rules for dealing with such a situation are as follows:
- If an invite to a room is received whilst the client is preparing to send an
invite to the same room, the client should cancel its outgoing call and
instead automatically accept the incoming call on behalf of the user.
- If an invite to a room is received after the client has sent an invite to
the same room and is waiting for a response, the client should perform a
lexicographical comparison of the call IDs of the two calls and use the
lesser of the two calls, aborting the greater. If the incoming call is the
lesser, the client should accept this call on behalf of the user.
The call setup should appear seamless to the user as if they had simply placed
a call and the other party had accepted. Thusly, any media stream that had been
setup for use on a call should be transferred and used for the call that
replaces it.

@ -4,11 +4,9 @@ Application Service API
The Matrix client-server API and server-server APIs provide the means to
implement a consistent self-contained federated messaging fabric. However, they
provide limited means of implementing custom server-side behaviour in Matrix
(e.g. gateways, filters, extensible hooks etc).
The Application Service API defines a standard API to allow such extensible
functionality to be implemented irrespective of the underlying homeserver
implementation.
(e.g. gateways, filters, extensible hooks etc). The Application Service API
defines a standard API to allow such extensible functionality to be implemented
irrespective of the underlying homeserver implementation.
.. TODO-spec
Add in Client-Server services? Overview of bots? Seems weird to be in the spec
@ -18,12 +16,10 @@ Passive Application Services
----------------------------
"Passive" application services can only observe events from a given home server.
They cannot prevent events from being sent, nor can they modify the content of
the event being sent.
In order to observe events from a homeserver, the homeserver needs to be
configured to pass certain types of traffic to the application service. This
is achieved by manually configuring the homeserver with information about the
AS..
the event being sent. In order to observe events from a homeserver, the
homeserver needs to be configured to pass certain types of traffic to the
application service. This is achieved by manually configuring the homeserver
with information about the AS.
.. NOTE::
Previously, application services could register with a homeserver via HTTP
@ -66,13 +62,13 @@ An example HS configuration required to pass traffic to the AS is:
application service is merely augmenting the room itself (e.g. providing
logging or searching facilities).
- Namespaces are represented by POSIX extended regular expressions,
e.g.:
e.g:
.. code-block:: yaml
users:
- exclusive: true
regex: @irc.freenode.net/.*
regex: @irc.freenode.net_.*
Home Server -> Application Service API
@ -326,7 +322,7 @@ but only if the application service has defined the namespace as ``exclusive``.
ID conventions
~~~~~~~~~~~~~~
.. NOTE::
.. TODO-spec
- Giving HSes the freedom to namespace still feels like the Right Thing here.
- Exposing a public API provides the consistency which was the main complaint
against namespacing.
@ -345,7 +341,7 @@ types, including:
- MSISDNs (``tel``)
- Email addresses (``mailto``)
- IRC nicks (``irc`` - https://tools.ietf.org/html/draft-butcher-irc-url-04)
- XMPP (xep-0032)
- XMPP (XEP-0032)
- SIP URIs (RFC 3261)
As a result, virtual user IDs SHOULD relate to their URI counterpart. This
@ -401,21 +397,3 @@ client from which the event originated. For instance, this could contain the
message-ID for emails/nntp posts, or a link to a blog comment when gatewaying
blog comment traffic in & out of matrix
Active Application Services
----------------------------
.. TODO-spec
API that provides hooks into the server so that you can intercept and
manipulate events, and/or insert virtual users & rooms into the server.
Policy Servers
==============
.. NOTE::
This section is a work in progress.
.. TODO-spec
We should mention them in the Architecture section at least: how they fit
into the picture.
Enforcing policies
------------------

@ -2,10 +2,9 @@ Federation API
==============
Matrix home servers use the Federation APIs (also known as server-server APIs)
to communicate with each other.
Home servers use these APIs to push messages to each other in real-time, to
request historic messages from each other, and to query profile and presence
information about users on each other's servers.
to communicate with each other. Home servers use these APIs to push messages to
each other in real-time, to request historic messages from each other, and to
query profile and presence information about users on each other's servers.
The APIs are implemented using HTTPS GETs and PUTs between each of the
servers. These HTTPS requests are strongly authenticated using public key
@ -21,7 +20,7 @@ Persisted Data Units (PDUs):
context.
Like email, it is the responsibility of the originating server of a PDU
to deliver that event to its recepient servers. However PDUs are signed
to deliver that event to its recipient servers. However PDUs are signed
using the originating server's public key so that it is possible to
deliver them through third-party servers.
@ -60,13 +59,11 @@ and an optional TLS port.
.. **
If the port is present then the server is discovered by looking up an AAAA or
A record for the DNS name and connecting to the specified TLS port.
If the port is absent then the server is discovered by looking up a
``_matrix._tcp`` SRV record for the DNS name. If this record does not exist
then the server is discovered by looking up an AAAA or A record on the DNS
name and taking the default fallback port number of 8448.
A record for the DNS name and connecting to the specified TLS port. If the port
is absent then the server is discovered by looking up a ``_matrix._tcp`` SRV
record for the DNS name. If this record does not exist then the server is
discovered by looking up an AAAA or A record on the DNS name and taking the
default fallback port number of 8448.
Home servers may use SRV records to load balance requests between multiple TLS
endpoints or to failover to another endpoint if an endpoint fails.
@ -84,18 +81,19 @@ directly or by querying an intermediate notary server using a
response with their own key. A server may query multiple notary servers to
ensure that they all report the same public keys.
This approach is borrowed from the Perspectives Project
(http://perspectives-project.org/), but modified to include the NACL keys and to
use JSON instead of XML. It has the advantage of avoiding a single trust-root
since each server is free to pick which notary servers they trust and can
corroborate the keys returned by a given notary server by querying other
servers.
This approach is borrowed from the `Perspectives Project`_, but modified to
include the NACL keys and to use JSON instead of XML. It has the advantage of
avoiding a single trust-root since each server is free to pick which notary
servers they trust and can corroborate the keys returned by a given notary
server by querying other servers.
.. _Perspectives Project: http://perspectives-project.org/
Publishing Keys
_______________
^^^^^^^^^^^^^^^
Home servers publish the allowed TLS fingerprints and signing keys in a JSON
object at ``/_matrix/key/v2/server/${key_id}``. The response contains a list of
object at ``/_matrix/key/v2/server/{key_id}``. The response contains a list of
``verify_keys`` that are valid for signing federation requests made by the
server and for signing events. It contains a list of ``old_verify_keys``
which are only valid for signing events. Finally the response contains a list
@ -178,7 +176,7 @@ events sent by that server can still be checked.
}
Querying Keys Through Another Server
____________________________________
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Servers may offer a query API ``_matrix/key/v2/query/`` for getting the keys
for another server. This API can be used to GET at list of JSON objects for a
@ -510,7 +508,7 @@ To backfill events on a given context::
Retrieves a sliding-window history of previous PDUs that occurred on the given
context. Starting from the PDU ID(s) given in the "v" argument, the PDUs that
preceeded it are retrieved, up to a total number given by the "limit" argument.
preceded it are retrieved, up to a total number given by the "limit" argument.
These are then returned in a new Transaction containing all of the PDUs.
@ -554,9 +552,7 @@ Every HTTP request made by a homeserver is authenticated using public key
digital signatures. The request method, target and body are signed by wrapping
them in a JSON object and signing it using the JSON signing algorithm. The
resulting signatures are added as an Authorization header with an auth scheme
of X-Matrix.
Note that the target field should include the full path starting with
of X-Matrix. Note that the target field should include the full path starting with
``/_matrix/...``, including the ``?`` and any query parameters if present, but
should not include the leading ``https:``, nor the destination server's
hostname.
@ -656,12 +652,12 @@ State Conflict Resolution
- How does this work with deleting current state
- How do we reject invalid federation traffic?
[[TODO(paul): At this point we should probably have a long description of how
State management works, with descriptions of clobbering rules, power levels, etc
etc... But some of that detail is rather up-in-the-air, on the whiteboard, and
so on. This part needs refining. And writing in its own document as the details
relate to the server/system as a whole, not specifically to server-server
federation.]]
[[TODO(paul): At this point we should probably have a long description of how
State management works, with descriptions of clobbering rules, power levels, etc
etc... But some of that detail is rather up-in-the-air, on the whiteboard, and
so on. This part needs refining. And writing in its own document as the details
relate to the server/system as a whole, not specifically to server-server
federation.]]
Presence
--------
@ -677,8 +673,8 @@ Performing a presence update and poll subscription request::
Each should be an object with the following keys:
user_id: string containing a User ID
presence: "offline"|"unavailable"|"online"|"free_for_chat"
status_msg: (optional) string of freeform text
last_active_ago: miliseconds since the last activity by the user
status_msg: (optional) string of free-form text
last_active_ago: milliseconds since the last activity by the user
poll: (optional): list of strings giving User IDs
@ -696,7 +692,7 @@ removed until explicitly requested by a later ``unpoll``.
On receipt of a message containing a non-empty ``poll`` list, the receiving
server should immediately send the sending server a presence update EDU of its
own, containing in a ``push`` list the current state of every user that was in
the orginal EDU's ``poll`` list.
the original EDU's ``poll`` list.
Sending a presence invite::
@ -721,7 +717,7 @@ Rejecting a presence invite::
Content keys - as for m.presence_invite
.. TODO-doc
- Explain the timing-based roundtrip reduction mechanism for presence
- Explain the timing-based round-trip reduction mechanism for presence
messages
- Explain the zero-byte presence inference logic
See also: docs/client-server/model/presence
@ -742,8 +738,8 @@ Querying profile information::
field: (optional) string giving a field name
Returns: JSON object containing the following keys:
displayname: string of freeform text
avatar_url: string containing an http-scheme URL
displayname: string of free-form text
avatar_url: string containing an HTTP-scheme URL
If the query contains the optional ``field`` key, it should give the name of a
result field. If such is present, then the result should contain only a field
@ -769,3 +765,4 @@ Querying directory information::
The list of join candidates is a list of server names that are likely to hold
the given room; these are servers that the requesting server may wish to try
joining with. This list may or may not include the server answering the query.

@ -1,76 +0,0 @@
Content repository
==================
HTTP API
--------
Uploads are POSTed to a resource which returns a token which is used to GET
the download. Uploads are POSTed to the sender's local homeserver, but are
downloaded from the recipient's local homeserver, which must thus first transfer
the content from the origin homeserver using the same API (unless the origin
and destination homeservers are the same). The upload/download API is::
=> POST /_matrix/media/v1/upload HTTP/1.1
Content-Type: <media-type>
<media>
<= HTTP/1.1 200 OK
Content-Type: application/json
{ "content-uri": "mxc://<server-name>/<media-id>" }
=> GET /_matrix/media/v1/download/<server-name>/<media-id> HTTP/1.1
<= HTTP/1.1 200 OK
Content-Type: <media-type>
Content-Disposition: attachment;filename=<upload-filename>
<media>
Clients can get thumbnails by supplying a desired width and height and
thumbnailing method::
=> GET /_matrix/media/v1/thumbnail/<server_name>
/<media-id>?width=<w>&height=<h>&method=<m> HTTP/1.1
<= HTTP/1.1 200 OK
Content-Type: image/jpeg or image/png
<thumbnail>
The thumbnail methods are "crop" and "scale". "scale" trys to return an
image where either the width or the height is smaller than the requested
size. The client should then scale and letterbox the image if it needs to
fit within a given rectangle. "crop" trys to return an image where the
width and height are close to the requested size and the aspect matches
the requested size. The client should scale the image if it needs to fit
within a given rectangle.
Homeservers may generate thumbnails for content uploaded to remote
homeservers themselves or may rely on the remote homeserver to thumbnail
the content. Homeservers may return thumbnails of a different size to that
requested. However homeservers should provide exact matches where reasonable.
Homeservers must never upscale images.
Security
--------
Clients may try to upload very large files. Homeservers should not store files
that are too large and should not serve them to clients.
Clients may try to upload very large images. Homeservers should not attempt to
generate thumbnails for images that are too large.
Remote homeservers may host very large files or images. Homeserver should not
proxy or thumbnail large files or images from remote homeservers.
Clients may try to upload a large number of files. Homeservers should limit the
number and total size of media that can be uploaded by clients.
Clients may try to access a large number of remote files through a homeserver.
Homeservers should restrict the number and size of remote files that it caches.
Clients or remote homeservers may try to upload malicious files targeting
vulnerabilities in either the homeserver thumbnailing or the client decoders.

@ -1,58 +0,0 @@
Typing Notifications
====================
Client APIs
-----------
To set "I am typing for the next N msec"::
PUT .../rooms/<room_id>/typing/<user_id>
Content: { "typing": true, "timeout": N }
# timeout is in msec; I suggest no more than 20 or 30 seconds
This should be re-sent by the client to continue informing the server the user
is still typing; I suggest a safety margin of 5 seconds before the expected
timeout runs out. Just keep declaring a new timeout, it will replace the old
one.
To set "I am no longer typing"::
PUT ../rooms/<room_id>/typing/<user_id>
Content: { "typing": false }
Client Events
-------------
All room members will receive an event on the event stream::
{
"type": "m.typing",
"room_id": "!room-id-here:matrix.org",
"content": {
"user_ids": ["list of", "every user", "who is", "currently typing"]
}
}
The client must use this list to *REPLACE* its knowledge of every user who is
currently typing. The reason for this is that the server DOES NOT remember
users who are not currently typing, as that list gets big quickly. The client
should mark as not typing, any user ID who is not in that list.
Server APIs
-----------
Servers will emit EDUs in the following form::
{
"type": "m.typing",
"content": {
"room_id": "!room-id-here:matrix.org",
"user_id": "@user-id-here:matrix.org",
"typing": true/false,
}
}
Server EDUs don't (currently) contain timing information; it is up to
originating HSes to ensure they eventually send "stop" notifications.
((This will eventually need addressing, as part of the wider typing/presence
timer addition work))

@ -0,0 +1,8 @@
Identity Servers
================
.. NOTE::
This section is a work in progress.
.. TODO-doc Dave
- 3PIDs and identity server, functions

@ -128,11 +128,3 @@ Threat: Disclosure to Servers Within Chatroom
An attacker could take control of a server within a chatroom to expose message
contents or metadata for messages in that room.
Identity Servers
================
.. NOTE::
This section is a work in progress.
.. TODO-doc Dave
- 3PIDs and identity server, functions

@ -0,0 +1,56 @@
Module Heading
==============
.. _module:short-name:
A short summary of the module. What features does this module provide? An anchor
should be specified at the top of the module using the format ``module:name``.
Complicated modules may wish to have architecture diagrams or event flows
(e.g. VoIP call flows) here. Custom subsections can be included but they should
be used *sparingly* to reduce the risk of putting client or server behaviour
information in these custom sections.
Events
------
List the new event types introduced by this module, if any. If there are no
new events, this section can be omitted. Event types should be done as
subsections. This section is intended to document the "common shared event
structure" between client and server. Deviations from this shared structure
should be documented in the relevant behaviour section.
``m.example.event.type``
~~~~~~~~~~~~~~~~~~~~~~~~
There should be JSON Schema docs for this event. Once there is JSON schema,
there will be a template variable with dots in the event type replaced with
underscores and the suffix ``_event``. You can insert a template like so:
{{m_example_event_type_event}}
Client behaviour
----------------
List any new HTTP endpoints. These endpoints should be documented using Swagger.
Once there is Swagger, there will be a template variable based on the name of
the YAML file with the suffix ``_http_api``. You can insert a template for
swagger docs like so:
{{name-of-yaml-file-without-file-ext_http_api}}
List the steps the client needs to take to
correctly process this module. List what data structures the client should be
storing in order to aid implementation.
Server behaviour
----------------
Does the server need to handle any of the new events in a special way (e.g.
typing timeouts, presence). Advice on how to persist events and/or requests are
recommended to aid implementation. Federation-specific logic should be included
here.
Security considerations
-----------------------
This includes privacy leaks: for example leaking presence info. How do
misbehaving clients or servers impact this module? This section should always be
included, if only to say "we've thought about it but there isn't anything to do
here".

@ -0,0 +1,74 @@
Content repository
==================
.. _module:content:
This module allows users to upload content to their homeserver which is
retrievable from other homeservers. Its' purpose is to allow users to share
attachments in a room. Content locations are represented as Matrix Content (MXC)
URIs. They look like::
mxc://<server-name>/<media-id>
<server-name> : The name of the homeserver where this content originated, e.g. matrix.org
<media-id> : An opaque ID which identifies the content.
Uploads are POSTed to a resource on the user's local homeserver which returns a
token which is used to GET the download. Content is downloaded from the
recipient's local homeserver, which must first transfer the content from the
origin homeserver using the same API (unless the origin and destination
homeservers are the same).
Client behaviour
----------------
Clients can upload and download content using the following HTTP APIs.
{{content_repo_http_api}}
Thumbnails
~~~~~~~~~~
The thumbnail methods are "crop" and "scale". "scale" tries to return an
image where either the width or the height is smaller than the requested
size. The client should then scale and letterbox the image if it needs to
fit within a given rectangle. "crop" tries to return an image where the
width and height are close to the requested size and the aspect matches
the requested size. The client should scale the image if it needs to fit
within a given rectangle.
Server behaviour
----------------
Homeservers may generate thumbnails for content uploaded to remote
homeservers themselves or may rely on the remote homeserver to thumbnail
the content. Homeservers may return thumbnails of a different size to that
requested. However homeservers should provide exact matches where reasonable.
Homeservers must never upscale images.
Security considerations
-----------------------
The HTTP GET endpoint does not require any authentication. Knowing the URL of
the content is sufficient to retrieve the content, even if the entity isn't in
the room.
Homeservers have additional concerns:
- Clients may try to upload very large files. Homeservers should not store files
that are too large and should not serve them to clients.
- Clients may try to upload very large images. Homeservers should not attempt to
generate thumbnails for images that are too large.
- Remote homeservers may host very large files or images. Homeservers should not
proxy or thumbnail large files or images from remote homeservers.
- Clients may try to upload a large number of files. Homeservers should limit the
number and total size of media that can be uploaded by clients.
- Clients may try to access a large number of remote files through a homeserver.
Homeservers should restrict the number and size of remote files that it caches.
- Clients or remote homeservers may try to upload malicious files targeting
vulnerabilities in either the homeserver thumbnailing or the client decoders.

@ -1,6 +1,8 @@
End-to-End Encryption
=====================
.. _module:e2e:
.. TODO-doc
- Why is this needed.
- Overview of process

@ -1,5 +1,7 @@
Room History Visibility
=======================
-----------------------
.. _module:history-visibility:
Whether a member of a room can see the events that happened in a room from
before they joined the room is controlled by the ``history_visibility`` key

@ -0,0 +1,29 @@
Instant Messaging
=================
.. _module:im:
Events
------
{{m_room_message_event}}
{{m_room_message_feedback_event}}
{{m_room_name_event}}
{{m_room_topic_event}}
m.room.message msgtypes
-----------------------
.. TODO-spec
How a client should handle unknown message types.
Each `m.room.message`_ MUST have a ``msgtype`` key which identifies the type
of message being sent. Each type has their own required and optional keys, as
outlined below.
{{msgtype_events}}

@ -0,0 +1,112 @@
Presence
========
.. _module:presence:
Each user has the concept of presence information. This encodes:
* Whether the user is currently online
* How recently the user was last active (as seen by the server)
* Whether a given client considers the user to be currently idle
* Arbitrary information about the user's current status (e.g. "in a meeting").
This information is collated from both per-device (``online``, ``idle``,
``last_active``) and per-user (status) data, aggregated by the user's homeserver
and transmitted as an ``m.presence`` event. This is one of the few events which
are sent *outside the context of a room*. Presence events are sent to all users
who subscribe to this user's presence through a presence list or by sharing
membership of a room.
A presence list is a list of user IDs whose presence the user wants to follow.
To be added to this list, the user being added must be invited by the list owner
who must accept the invitation.
User's presence state is represented by the ``presence`` key, which is an enum
of one of the following:
- ``online`` : The default state when the user is connected to an event
stream.
- ``unavailable`` : The user is not reachable at this time e.g. they are
idle.
- ``offline`` : The user is not connected to an event stream or is
explicitly suppressing their profile information from being sent.
- ``free_for_chat`` : The user is generally willing to receive messages
moreso than default.
Events
------
{{presence_events}}
Client behaviour
----------------
Clients can manually set/get their presence/presence list using the HTTP APIs
listed below.
{{presence_http_api}}
Idle timeout
~~~~~~~~~~~~
Clients SHOULD implement an "idle timeout". This is a timer which fires after
a period of inactivity on the client. The definition of inactivity varies
depending on the client. For example, web implementations may determine
inactivity to be not moving the mouse for a certain period of time. When this
timer fires it should set the presence state to ``unavailable``. When the user
becomes active again (e.g. by moving the mouse) the client should set the
presence state to ``online``. A timeout value between 1 and 5 minutes is
recommended.
Server behaviour
----------------
Each user's home server stores a "presence list" per user. Once a user accepts
a presence list, both user's HSes must track the subscription.
Propagating profile information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because the profile display name and avatar information are likely to be used in
many places of a client's display, changes to these fields SHOULD cause an
automatic propagation event to occur, informing likely-interested parties of the
new values. One of these change mechanisms SHOULD be via ``m.presence`` events.
These events should set ``displayname`` and ``avatar_url`` to the new values
along with the presence-specific keys. This SHOULD be done automatically by the
home server when a user successfully changes their display name or avatar URL.
.. admonition:: Rationale
The intention for sending this information in ``m.presence`` is so that any
"user list" can display the *current* name/presence for a user ID outside the
scope of a room e.g. for a user page. This is bundled into a single event for
several reasons. The user's display name can change per room. This
event provides the "canonical" name for the user. In addition, the name is
bundled into a single event for the ease of client implementations. If this
was not done, the client would need to search all rooms for their own
membership event to pull out the display name.
Last active ago
~~~~~~~~~~~~~~~
The server maintains a timestamp of the last time it saw a
pro-active event from the user. A pro-active event may be sending a message to a
room or changing presence state to a higher level of availability. Levels of
availability are defined from low to high as follows:
- ``offline``
- ``unavailable``
- ``online``
- ``free_for_chat``
Based on this list, changing state from ``unavailable`` to ``online`` counts as
a pro-active event, whereas ``online`` to ``unavailable`` does not. This
timestamp is presented via a key called ``last_active_ago`` which gives the
relative number of milliseconds since the pro-active event.
Security considerations
-----------------------
Presence information is shared with all users who share a room with the target
user. In large public rooms this could be undesirable.

@ -70,7 +70,7 @@ Room Rules
Sender
These rules configure notification behaviour for messages from a specific,
named Matrix user ID. The rule_id of Sender rules is always the Matrix user
ID of the user whose messages theyt apply to.
ID of the user whose messages they'd apply to.
Underride
These are identical to override rules, but have a lower priority than content,
room and sender rules.
@ -99,20 +99,17 @@ be redundant. Actions for the highest priority rule and only that rule apply
(for example, a set_tweak action in a lower priority rule will not apply if a
higher priority rule matches, even if that rule does not specify any tweaks).
Rules also have an identifier, rule_id, which is a string. The rule_id is
unique within the kind of rule and scope: rule_ids need not be unique between
rules of the same kind on different devices.
A home server may also have server default rules of each kind and in each scope.
Server default rules are lower priority than user-defined rules in each scope.
Server default rules (and only server default rules) begin with a dot ('.')
character.
In addition, all rules may be enabled or disabled. Disabled rules never match.
Rules also have an identifier, ``rule_id``, which is a string. The ``rule_id``
is unique within the kind of rule and scope: ``rule_ids`` need not be unique
between rules of the same kind on different devices. A home server may also have
server default rules of each kind and in each scope. Server default rules are
lower priority than user-defined rules in each scope. Server default rules (and
only server default rules) begin with a dot ('.') character. In addition, all
rules may be enabled or disabled. Disabled rules never match.
If no rules match an event, the Home Server should not notify for the message
(that is to say, the default action is "dont-notify"). Events that the user sent
themself are never alerted for.
themselves are never alerted for.
Predefined Rules
----------------
@ -128,7 +125,7 @@ with these IDs, their semantics should match those given below:
{
"rule_id": ".m.rule.contains_user_name"
"pattern": "[the lcoal part of the user's Matrix ID]",
"pattern": "[the local part of the user's Matrix ID]",
"actions": [
"notify",
{
@ -220,7 +217,7 @@ with these IDs, their semantics should match those given below:
Push Rules: Actions:
--------------------
All rules have an associated list of 'actions'. An action affects if and how a
notification is delievered for a matching event. This standard defines the
notification is delivered for a matching event. This standard defines the
following actions, although if Home servers wish to support more, they are free
to do so:
@ -241,11 +238,11 @@ set_tweak
Actions that have no parameters are represented as a string. Otherwise, they are
represented as a dictionary with a key equal to their name and other keys as
their parameters, eg. { "set_tweak": "sound", "value": "default" }
their parameters, e.g. ``{ "set_tweak": "sound", "value": "default" }``
Push Rules: Actions: Tweaks
---------------------------
The 'set_tweak' key action is used to add an entry to the 'tweaks' dictionary
The ``set_tweak`` key action is used to add an entry to the 'tweaks' dictionary
that is sent in the notification poke. The following tweaks are defined:
sound
@ -275,7 +272,7 @@ do so:
event_match
This is a glob pattern match on a field of the event. Parameters:
* 'key': The dot-separated field of the event to match, eg. content.body
* 'key': The dot-separated field of the event to match, e.g. content.body
* 'pattern': The glob-style pattern to match against. Patterns with no
special glob characters should be treated as having asterisks
prepended and appended when testing the condition.
@ -295,7 +292,7 @@ room_member_count
'>=' or '<='. A prefix of '<' matches rooms where the member count is
strictly less than the given number and so forth. If no prefix is present,
this matches rooms where the member count is exactly equal to the given
number (ie. the same as '==').
number (i.e. the same as '==').
Room, Sender, User and Content rules do not have conditions in the same way,
but instead have predefined conditions, the behaviour of which can be configured
@ -314,7 +311,7 @@ scope
Either 'global' or 'device/<profile_tag>' to specify global rules or
device rules for the given profile_tag.
kind
The kind of rule, ie. 'override', 'underride', 'sender', 'room', 'content'.
The kind of rule, i.e. 'override', 'underride', 'sender', 'room', 'content'.
rule_id
The identifier for the rule.
@ -330,7 +327,7 @@ after
rule.
All requests to the push rules API also require an access_token as a query
paraemter.
parameter.
The content of the PUT request is a JSON object with a list of actions under the
'actions' key and either conditions (under the 'conditions' key) or the

@ -1,6 +1,8 @@
Push Notifications
==================
.. _module:push:
Overview
--------

@ -1,7 +1,7 @@
HTTP Notification Protocol
--------------------------
This describes the format used by "http" pushers to send notifications of
This describes the format used by "HTTP" pushers to send notifications of
events.
Notifications are sent as HTTP POST requests to the URL configured when the
@ -77,10 +77,10 @@ counts
This is a dictionary of the current number of unacknowledged communications
for the recipient user. Counts whose value is zero are omitted.
unread
The number of unread messages a user has accross all of the rooms they are a
The number of unread messages a user has across all of the rooms they are a
member of.
missed_calls
The number of unacknowledged missed calls a user has accross all rooms of
The number of unacknowledged missed calls a user has across all rooms of
which they are a member.
device
This is an array of devices that the notification should be sent to.
@ -104,13 +104,13 @@ And additional key is defined but only present on member events:
user_is_target
This is true if the user receiving the notification is the subject of a member
event (ie. the state_key of the member event is equal to the user's Matrix
event (i.e. the state_key of the member event is equal to the user's Matrix
ID).
The recipient of an HTTP notification should respond with an HTTP 2xx response
when the notification has been processed. If the endpoint returns an HTTP error
code, the Home Server should retry for a reasonable amount of time with a
reasonable backoff scheme.
reasonable back-off scheme.
The endpoint should return a JSON dictionary as follows::

@ -0,0 +1,84 @@
Receipts
========
.. _module:receipts:
This module adds in support for receipts. These receipts are a form of
acknowledgement of an event. This module defines a single acknowledgement:
``m.read`` which indicates that the user has read up to a given event.
Sending a receipt for each event can result in sending large amounts of traffic
to a homeserver. To prevent this from becoming a problem, receipts are implemented
using "up to" markers. This marker indicates that the acknowledgement applies
to all events "up to and including" the event specified. For example, marking
an event as "read" would indicate that the user had read all events *up to* the
referenced event.
Events
------
Each ``user_id``, ``receipt_type`` pair must be associated with only a
single ``event_id``.
{{m_receipt_event}}
Client behaviour
----------------
In v1 ``/initialSync``, receipts are listed in a separate top level ``receipts``
key. In v2 ``/sync``, receipts are contained in the ``ephemeral`` block for a
room. New receipts that come down the event streams are deltas which update
existing mappings. Clients should replace older receipt acknowledgements based
on ``user_id`` and ``receipt_type`` pairs. For example::
Client receives m.receipt:
user = @alice:example.com
receipt_type = m.read
event_id = $aaa:example.com
Client receives another m.receipt:
user = @alice:example.com
receipt_type = m.read
event_id = $bbb:example.com
The client should replace the older acknowledgement for $aaa:example.com with
this one for $bbb:example.com
Clients should send read receipts when there is some certainty that the event in
question has been **displayed** to the user. Simply receiving an event does not
provide enough certainty that the user has seen the event. The user SHOULD need
to *take some action* such as viewing the room that the event was sent to or
dismissing a notification in order for the event to count as "read".
A client can update the markers for its user by interacting with the following
HTTP APIs.
{{v2_receipts_http_api}}
Server behaviour
----------------
For efficiency, receipts SHOULD be batched into one event per room before
delivering them to clients.
Receipts are sent across federation as EDUs with type ``m.receipt``. The
format of the EDUs are::
{
<room_id>: {
<receipt_type>: {
<user_id>: { <content> }
},
...
},
...
}
These are always sent as deltas to previously sent receipts. Currently only a
single ``<receipt_type>`` should be used: ``m.read``.
Security considerations
-----------------------
As receipts are sent outside the context of the event graph, there are no
integrity checks performed on the contents of ``m.receipt`` events.

@ -0,0 +1,66 @@
Typing Notifications
====================
.. _module:typing:
Users may wish to be informed when another user is typing in a room. This can be
achieved using typing notifications. These are ephemeral events scoped to a
``room_id``. This means they do not form part of the `Event Graph`_ but still
have a ``room_id`` key.
.. _Event Graph: `sect:event-graph`_
Events
------
{{m_typing_event}}
Client behaviour
----------------
When a client receives an ``m.typing`` event, it MUST use the user ID list to
**REPLACE** its knowledge of every user who is currently typing. The reason for
this is that the server *does not remember* users who are not currently typing
as that list gets big quickly. The client should mark as not typing any user ID
who is not in that list.
It is recommended that clients store a ``boolean`` indicating whether the user
is typing or not. Whilst this value is ``true`` a timer should fire periodically
every N seconds to send a typing HTTP request. The value of N is recommended to
be no more than 20-30 seconds. This request should be re-sent by the client to
continue informing the server the user is still typing. As subsequent
requests will replace older requests, a safety margin of 5 seconds before the
expected timeout runs out is recommended. When the user stops typing, the
state change of the ``boolean`` to ``false`` should trigger another HTTP request
to inform the server that the user has stopped typing.
{{typing_http_api}}
Server behaviour
----------------
Servers MUST emit typing EDUs in a different form to ``m.typing`` events which
are shown to clients. This form looks like::
{
"type": "m.typing",
"content": {
"room_id": "!room-id-here:matrix.org",
"user_id": "@user-id-here:matrix.org",
"typing": true/false
}
}
This does not contain timing information so it is up to originating homeservers
to ensure they eventually send "stop" notifications.
.. TODO
((This will eventually need addressing, as part of the wider typing/presence
timer addition work))
Security considerations
-----------------------
Clients may not wish to inform everyone in a room that they are typing and
instead only specific users in the room.

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save