diff --git a/.gitignore b/.gitignore index 5245c82b..cce5848f 100644 --- a/.gitignore +++ b/.gitignore @@ -18,3 +18,4 @@ _rendered.rst /.vscode/ /.idea/ +/spec/ \ No newline at end of file diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 2d26e8a8..b9c66805 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -29,9 +29,7 @@ some time to complete. Changes to the protocol (new endpoints, ideas, etc) need to go through the `proposals process `_. Other changes, such as fixing bugs, typos, or clarifying existing behaviour do not need a proposal. -If you're not sure, visit us at `#matrix-spec:matrix.org`_ -and ask. - +If you're not sure, visit us at `#matrix-spec:matrix.org`_ and ask. Other changes ~~~~~~~~~~~~~ @@ -64,12 +62,17 @@ following: to fix. On the other hand, introducing new behaviour is best represented by a proposal. +* Design or aesthetic changes, such as improving accessibility, colour schemes, + etc. Please check in with us at `#matrix-docs:matrix.org`_ with your proposed + design change before opening a PR so we can work with you on it. + For such changes, please do just open a `pull request`_. If you're not sure if your change is covered by the above, please visit `#matrix-spec:matrix.org` and ask. .. _`pull request`: https://help.github.com/articles/about-pull-requests .. _`#matrix-spec:matrix.org`: https://matrix.to/#/#matrix-spec:matrix.org +.. _`#matrix-docs:matrix.org`: https://matrix.to/#/#matrix-docs:matrix.org Adding to the changelog @@ -100,8 +103,7 @@ the ``newsfragments`` directory. The ``type`` can be one of the following: All news fragments must have a brief summary explaining the change in the contents of the file. The summary must end in a full stop to be in line with -the style guide and and formatting must be done using `Restructured Text -`_. +the style guide and and formatting must be done using Markdown. Changes that do not change the spec, such as changes to the build script, formatting, CSS, etc should not get a news fragment. diff --git a/README.md b/README.md new file mode 100644 index 00000000..a9d8dd9b --- /dev/null +++ b/README.md @@ -0,0 +1,97 @@ +# Matrix Specification + +This repository contains the Matrix Specification, rendered at [spec.matrix.org](http://spec.matrix.org/). + +Developers looking to use Matrix should join [#matrix-dev:matrix.org](http://matrix.to/#/#matrix-dev:matrix.org) +on Matrix for help. + +Spec authors and proposal writers are welcome to join [#matrix-spec:matrix.org](http://matrix.to/#/#matrix-spec:matrix.org). We welcome contributions! See [CONTRIBUTING.rst](./CONTRIBUTING.rst) for details. + +## Structure + +The Matrix spec is compiled with [Hugo](https://gohugo.io/) (a static site generator) with the following structure: + +* `/assets`: assets that need postprocessing using [Hugo Pipes](https://gohugo.io/hugo-pipes/introduction/). + For example, Sass files would go here. + +* `/content`: files that will become pages in the site go here. Typically these are Markdown files with some YAML front + matter indicating, [among other things](https://gohugo.io/content-management/front-matter/), what layout should be + applied to this page. The organization of files under `/content` determines the organization of pages in the built + site. + +* `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as + [data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and + parse them. This is also where our + +* `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of + a page: for example, whether it has header, footer, sidebar, and so on. + * `/layouts/partials`: these templates can be called from other templates, so they can be used to factor out + template code that's used in more than one template. An obvious example here is something like a sidebar, where + several different page layouts might all include the sidebar. But also, partial templates can return values: this + means they can be used like functions, that can be called by multiple templates to do some common processing. + * `/layouts/shortcodes`: these templates can be called directly from files in `/content`. + +* `/static`: static files which don't need preprocessing. JS or CSS files could live here. + +* `/themes`: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are + supplied in a `/themes/$theme_name/layouts` directory. You can use a theme but customise it by providing your own + versions of any of the them layouts in the base `/layouts` directory. That is, if a theme provides + `/themes/$theme_name/layouts/sidebar.html` and you provide `/layouts/sidebar.html`, then your version of this + template will be used. + +It also has the following top-level file: + +* `config.toml`: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings + defined here are available in templates. All these directories above are configurable via `config.toml` settings. + +Additionally, the following directories may be of interest: + +* `/attic`: Here contains historical sections of specification and legacy drafts for the specification. +* `/changelogs`: Various bits of changelog for the specification areas. +* `/event-schemas`: [JSON Schema](http://json-schema.org/) definitions for the spec. +* `/data-definitions`: Bits of structured data consumable by Matrix implementations. +* `/meta`: Documentation relating to the spec's processes that are otherwise untracked (release instructions, etc). +* `/scripts`: Various scripts for generating the spec. +* `/proposals`: Matrix Spec Change (MSC) proposals. See . +* `/api`: [OpenAPI](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md) / Swagger definitions for + the spec. + +## Authoring changes to the spec + +Please read [CONTRIBUTING.rst](./CONTRIBUTING.rst) before authoring a change to the spec. Note that spec authoring takes +place after an MSC has been accepted, not as part of a proposal itself. + +1. Install the extended version (often the OS default) of Hugo: +2. Run `git submodule update --init --recursive` for good measure. +3. Run `npm i` to install the dependencies. Note that this will require NodeJS to be installed. +4. Run `npm run get-proposals` to seed the proposals data. This is not required. +5. Run `hugo serve` to run a local webserver which builds whenever a file change is detected. If watching doesn't appear + to be working for you, try `hugo serve --disableFastRender` instead. +6. Edit the specification 🙂 + +We use a highly customized [Docsy](https://www.docsy.dev/) theme for our generated site, which uses Bootstrap and Font +Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in +[#matrix-docs:matrix.org](https://matrix.to/#/#matrix-docs:matrix.org) before opening a PR. + +## Building the specification + +If for some reason you're not a CI/CD system and want to render the spec yourself, follow the above steps for authoring +changes to the specification and instead of `hugo serve` run `hugo -d "spec"` - this will generate the spec to `/spec`. +If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"` to +the `hugo -d "spec"` command. + +For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt` and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`. +To make use of the generated file, there are a number of options: + +* It can be uploaded from your filesystem to an online editor/viewer such as + http://editor.swagger.io/ +* You can run a local HTTP server by running `./scripts/swagger-http-server.py`, and then view the documentation via an + online viewer; for example, at +* You can host the swagger UI yourself. See for advice on how to + do so. + +## Issue tracking + +Specification issues are tracked on github at . + +See [meta/github-labels.rst](./meta/github-labels.rst) for information on what the labels mean. diff --git a/README.rst b/README.rst deleted file mode 100644 index 61c27f15..00000000 --- a/README.rst +++ /dev/null @@ -1,143 +0,0 @@ -This repository contains the Matrix specification. - -If you want to ask more about the specification, join us on -`#matrix-dev:matrix.org `_. - -We welcome contributions to the spec! See the notes below on `Building the -specification`_, and ``_ to get started making contributions. - -Note that the Matrix Project lists, which were previously kept in this -repository, are now in https://github.com/matrix-org/matrix.org. - -Structure of this repository -============================ - -- ``api`` : `OpenAPI`_ (swagger) specifications for the the HTTP APIs. -- ``attic``: historical sections of specification for reference - purposes. -- ``changelogs``: change logs for the various parts of the - specification. -- ``drafts``: Previously, contained documents which were under discussion for - future incusion into the specification and/or supporting documentation. This - is now historical, as we use separate discussion documents (see - ``_). -- ``event-schemas``: the `JSON Schema`_ for all Matrix events - contained in the specification, along with example JSON files. -- ``meta``: documents outlining the processes involved when writing - documents, e.g. documentation style, guidelines. -- ``scripts``: scripts to generate formatted versions of the - documentation, typically HTML. -- ``specification``: the specification split up into sections. - -.. _OpenAPI: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md -.. _JSON Schema: http://json-schema.org/ - -Building the specification -========================== - -The Matrix Spec is generated by a set of scripts, from the RST documents, API -specs and event schemas in this repository. - -Preparation ------------ - -To use the scripts, it is best to create a Python 3.4+ virtualenv as follows:: - - virtualenv -p python3 env - env/bin/pip install -r scripts/requirements.txt - -(Benjamin Saunders has contributed a script for `Nix`_ users, which can be -invoked with ``nix-shell scripts/contrib/shell.nix``.) - -.. TODO: Possibly we need some libs installed; should record what they are. - -.. _`Nix`: https://nixos.org/nix/ - -Generating the specification ----------------------------- - -To rebuild the specification, use ``scripts/gendoc.py``:: - - source env/bin/activate - ./scripts/gendoc.py - -The above will write the rendered version of the specification to -``scripts/gen``. To view it, point your browser at ``scripts/gen/index.html``. - -Windows users -~~~~~~~~~~~~~ -The ``source`` program does not exist on Windows, so instead run one of the -``activate`` files in ``.\env\Scripts\`` to activate the virtual environment. - -If you're on Windows Vista or higher, be sure that the "Symbolic Links" -option was selected when installing Git prior to cloning this repository. If -you're still seeing errors about files not being found it is likely because -the symlink at ``api/client-server/definitions/event-schemas`` looks like a -file. To correct the problem, open an Administrative/Elevated Command Prompt in your -cloned matrix-doc directory and run the following:: - - cd api\client-server\definitions - del event-schemas - mklink /D event-schemas "..\..\..\event-schemas" - -This will delete the file and replace it with a symlink. Git should not detect -this as a change, and you should be able to go back to building the project. - -Generating the OpenAPI (Swagger) specs --------------------------------------- - -`Swagger`_ is a framework for representing RESTful APIs. We use it to generate -interactive documentation for our APIs. - -Before the Swagger docs can be used in the Swagger UI (or other tool expecting -a Swagger specs, they must be combined into a single json file. This can be -done as follows:: - - source env/bin/activate - ./scripts/dump-swagger.py - -By default, ``dump-swagger`` will write to ``scripts/swagger/api-docs.json``. - -To make use of the generated file, there are a number of options: - -* It can be uploaded from your filesystem to an online editor/viewer such as - http://editor.swagger.io/ -* You can run a local HTTP server by running - ``./scripts/swagger-http-server.py``, and then view the documentation via an - online viewer; for example, at - http://petstore.swagger.io/?url=http://localhost:8000/api-docs.json -* You can host the swagger UI yourself. See - https://github.com/swagger-api/swagger-ui#how-to-run for advice on how to do - so. - -.. _`Swagger`: http://swagger.io/ - -Continuserv ------------ - -Continuserv is a script which will rebuild the specification every time a file -is changed, and will serve it to a browser over HTTP. It is intended for use by -specification authors, so that they can quickly see the effects of their -changes. - -It is written in Go, so you will need the ``go`` compiler installed on your -computer. You will also need to install fsnotify by running:: - - go get gopkg.in/fsnotify/fsnotify.v1 - -Then, create a virtualenv as described above under `Preparation`_, -and:: - - source env/bin/activate - go run ./scripts/continuserv/main.go - -You will then be able to view the generated spec by visiting -http://localhost:8000/index.html. - -Issue tracking -============== - -Issues with the Matrix specification are tracked in `GitHub -`_. - -See `meta/github-labels.rst `_ for notes on what the labels mean. diff --git a/meta/documentation_style.rst b/meta/documentation_style.rst index e3d363c4..5528f73e 100644 --- a/meta/documentation_style.rst +++ b/meta/documentation_style.rst @@ -8,52 +8,36 @@ in. Format ------ -Documentation is written either in github-flavored markdown or RST. +Documentation is written either in github-flavored markdown. Sections -------- -RST support lots of different punctuation characters for underlines on sections. -Content in the specification MUST use the same characters in order for the -complete specification to be merged correctly. These characters are: - -- ``=`` -- ``-`` -- ``~`` -- ``+`` -- ``^`` -- \ ````` -- ``@`` -- ``:`` - -If you find yourself using ``^`` or beyond, you should rethink your document -layout if possible. +Markdown supports headings through the `#` prefix on text. Please avoid heavily +nested titles (h6, or 6 `#` characters) and instead re-evaluate the document structure. Correct capitalisation for long section names ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Headings should start with a capital letter, and use lower-case otherwise. - +Headings should start with a capital letter, and use lower-case otherwise. This +document is an example of what we mean. TODOs ----- -Any RST file in this repository may make it onto ``matrix.org``. We do not want -``TODO`` markers visible there. For internal comments, notes, TODOs, use standard -RST comments like so:: - - .. TODO-Bob - There is something to do here. This will not be rendered by something like - rst2html.py so it is safe to put internal comments here. - -You SHOULD put your username with the TODO so we know who to ask about it. +Any file in this repository might make it onto the matrix.org site, and as such +we do not want ``TODO`` markers visible there. For internal comments, notes, TODOs, +etc please use standard markdown comments (``). Please +include your name in the TODO comment so we know who to ask about it in the future. Line widths ----------- -We use 80 characters for line widths. This is a guideline and can be flouted IF +We use 80 characters for line widths. This is a guideline and can be ignored IF AND ONLY IF it makes reading more legible. Use common sense. +For proposals, please use 120 characters as a guide. + Stylistic notes ---------------