matrix-spec-proposals/README.rst

This repository contains the Matrix specification.

If you want to ask more about the specification, join us on
`#matrix-dev:matrix.org <http://matrix.to/#/#matrix-dev:matrix.org>`_.

We welcome contributions to the spec! See the notes below on `Building the
specification`_, and `<CONTRIBUTING.rst>`_ to get started making contributions.

Note that the Matrix Project lists, which were previously kept in this
repository, are now in https://github.com/matrix-org/matrix.org.

Structure of this repository
============================

- ``api`` : `OpenAPI`_ (swagger) specifications for the the HTTP APIs.
- ``attic``: historical sections of specification for reference
  purposes.
- ``changelogs``: change logs for the various parts of the
  specification.
- ``drafts``: Previously, contained documents which were under discussion for
  future incusion into the specification and/or supporting documentation. This
  is now historical, as we use separate discussion documents (see
  `<CONTRIBUTING.rst>`_).
- ``event-schemas``: the `JSON Schema`_ for all Matrix events
  contained in the specification, along with example JSON files.
- ``meta``: documents outlining the processes involved when writing
  documents, e.g. documentation style, guidelines.
- ``scripts``: scripts to generate formatted versions of the
  documentation, typically HTML.
- ``specification``: the specification split up into sections.

.. _OpenAPI: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md
.. _JSON Schema: http://json-schema.org/

Building the specification
==========================

The Matrix Spec is generated by a set of scripts, from the RST documents, API
specs and event schemas in this repository.

Preparation
-----------

To use the scripts, it is best to create a Python virtualenv as follows::

  virtualenv env
  env/bin/pip install -r scripts/requirements.txt

(Benjamin Synders has contributed a script for `Nix`_ users, which can be
invoked with ``nix-shell scripts/contrib/shell.nix``.)

.. TODO: Possibly we need some libs installed; should record what they are.

.. _`Nix`: https://nixos.org/nix/

Generating the specification
----------------------------

To rebuild the specification, use ``scripts/gendoc.py``::

  source env/bin/activate
  ./scripts/gendoc.py

The above will write the rendered version of the specification to
``scripts/gen``. To view it, point your browser at ``scripts/gen/index.html``.

Generating the OpenAPI (Swagger) specs
--------------------------------------

`Swagger`_ is a framework for representing RESTful APIs. We use it to generate
interactive documentation for our APIs.

Before the Swagger docs can be used in the Swagger UI (or other tool expecting
a Swagger specs, they must be combined into a single json file. This can be
done as follows::

  source env/bin/activate
  ./dump-swagger.py

By default, ``dump-swagger`` will write to ``scripts/swagger/api-docs.json``.

To make use of the generated file, there are a number of options:

* It can be uploaded from your filesystem to an online editor/viewer such as
  http://editor.swagger.io/
* You can run a local HTTP server by running
  ``./scripts/swagger-http-server.py``, and then view the documentation via an
  online viewer; for example, at
  http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json
* You can host the swagger UI yourself. See
  https://github.com/swagger-api/swagger-ui#how-to-run for advice on how to do
  so.

.. _`Swagger`: http://swagger.io/

Continuserv
-----------

Continuserv is a script which will rebuild the specification every time a file
is changed, and will serve it to a browser over HTTP. It is intended for use by
specification authors, so that they can quickly see the effects of their
changes.

It is written in Go, so you will need the ``go`` compiler installed on your
computer. You will also need to install fsnotify by running::

  go get gopkg.in/fsnotify.v1

Then, create a virtualenv as described above under `Preparation`_,
and::

  source env/bin/activate
  go run ./scripts/continuserv/main.go

You will then be able to view the generated spec by visiting
http://localhost:8000/index.html.

Issue tracking
==============

Issues with the Matrix specification are tracked in `GitHub
<https://github.com/matrix-org/matrix-doc/issues>`_.

The following labels are used to help categorize issues:

`spec-omission <https://github.com/matrix-org/matrix-doc/labels/spec-omission>`_
--------------------------------------------------------------------------------

Things which have been implemented but not currently specified. These may range
from entire API endpoints, to particular options or return parameters.

Issues with this label will have been implemented in `Synapse
<https://github.com/matrix-org/synapse>`_. Normally there will be a design
document in Google Docs or similar which describes the feature.

Examples:

* `Spec PUT /directory/list <https://github.com/matrix-org/matrix-doc/issues/417>`_
* `Unspec'd server_name request param for /join/{roomIdOrAlias}
  <https://github.com/matrix-org/matrix-doc/issues/904>`_

`clarification <https://github.com/matrix-org/matrix-doc/labels/clarification>`_
--------------------------------------------------------------------------------

An area where the spec could do with being more explicit.

Examples:

* `Spec the implicit limit on /syncs
  <https://github.com/matrix-org/matrix-doc/issues/708>`_

* `Clarify the meaning of the currently_active flags in presence events
  <https://github.com/matrix-org/matrix-doc/issues/686>`_

`spec-bug <https://github.com/matrix-org/matrix-doc/labels/spec-bug>`_
----------------------------------------------------------------------

Something which is in the spec, but is wrong.

Note: this is *not* for things that are badly designed or don't work well
(for which see 'improvement' or 'feature') - it is for places where the
spec doesn't match reality.

Examples:

* `swagger is wrong for directory PUT
  <https://github.com/matrix-org/matrix-doc/issues/933>`_

* `receipts section still refers to initialSync
  <https://github.com/matrix-org/matrix-doc/issues/695>`_

`improvement <https://github.com/matrix-org/matrix-doc/labels/improvement>`_
----------------------------------------------------------------------------

A suggestion for a relaatively simple improvement to the protocol.

Examples:

* `We need a 'remove 3PID' API so that users can remove mappings
  <https://github.com/matrix-org/matrix-doc/issues/620>`_
* `We should mandate that /publicRooms requires an access_token
  <https://github.com/matrix-org/matrix-doc/issues/612>`_

`feature <https://github.com/matrix-org/matrix-doc/labels/feature>`_
--------------------------------------------------------------------

A suggestion for a significant extension to the matrix protocol which
needs considerable consideration before implementation.

Examples:

* `Peer-to-peer Matrix <https://github.com/matrix-org/matrix-doc/issues/710>`_
* `Specify a means for clients to "edit" previous messages
  <https://github.com/matrix-org/matrix-doc/issues/682>`_