This PR attempts to update the CI of matrix-doc to build [the new spec redesign](https://github.com/matrix-org/matrix-doc/issues/2906). It does so by additionally building the new spec in parallel to the old. The plan is to continue to host the old spec at https://matrix.org/docs/spec, while the new spec will be at https://spec.matrix.org. Eventually we will retire the old version of the spec, and have the old URL redirect to the new one.
In detail, this PR:
* Adds a new step to CircleCI to build the new spec with `hugo`. This step uses alpine, grabs some dependencies, and then builds the HTML.
* We needed to hand some specific options to hugo for CircleCI in order to continue allowing CircleCI to host temporary builds of the spec after each CI run. This required changing some assumptions related to relative paths.
* CircleCI's artifacts hosting is also quite limited. Specifically it will not automatically resolve `/some/path` to `/some/path/index.html`, which our hugo theme relied on. Fixes were implemented for this, but we may want to consider switching away from CircleCI artifacts as a host, and using something like [netlify](https://www.netlify.com/) instead.
* Modifies the existing Buildkite pipeline step to build both the new spec in a separate step. It additionally modifies the old spec to be built with alpine. (Separate out into another PR)
* We'd like to separate out the deployment of matrix.org from the new spec. Therefore a new step, with a separate artifact build (`spec.tar.gz`). We will eventually remove the old step and the matrix.org build trigger.
* Modifies `pyproject.toml` to update the config of [giles](https://github.com/OpenAstronomy/baldrick/blob/master/baldrick/plugins/circleci_artifacts.py), which is what creates the "docs", "swagger" links in the CI steps for matrix-docs PRs.
* A new step was added for the new spec. The old spec was renamed to "legacy".
Historical note: this was originally a series of several commits, spread out
over several weeks. They have been squashed together to make `git annotate`
work properly.
The original commits were:
* 91ab3934 <Will> 2021-01-25 21:16:42 -0800 Add raw API end event schemas into /data directory
* aae22f47 <Will> 2021-01-25 21:33:06 -0800 Remove non-data files
* 1092d4ca <Will> 2021-01-26 20:41:33 -0800 Add data-compatiuble extension (.yaml) to all data files that currently omit one
* 21060109 <Will> 2021-01-26 20:57:28 -0800 Remove symlink to event-schemas, and update openAPI schema paths accordingly
* 4f633845 <Travis Ralston> 2021-04-12 21:54:54 -0600 Fix event schema examples too
* 301c7b2f <Will> 2021-02-05 10:15:42 -0800 Restore docs describing OpenAPI extensions that we use
check-swagger-sources also had a bug which caused it to fail while validating the directory
structure of /data/api. This was fixed by @KitsuneRal - thank you!
* add a base file
* Fix directory name
* Added translation using Weblate (English)
* Translated using Weblate (English)
Currently translated at 1.6% (1 of 64 strings)
Translation: matrix-doc/SAS Emoji v1
Translate-URL: https://translate.riot.im/projects/matrix-doc/sas-emoji-v1/en_EN/
* add english files
* delete english files
* Added translation using Weblate (English)
* Added translation using Weblate (English)
* Do manual translations
* Deleted translation using Weblate (English)
* Deleted translation using Weblate (English)
* Add a script to update the definitions with the translations
* update i18n
* Add a note to the spec about translations
* changelog
* Ensure translations end with json
MSC: https://github.com/matrix-org/matrix-doc/pull/2209
The changes are slightly difficult to word without dumping the text in and playing a game of spot the difference, so we now use our pre-existing pygments support to render a representation of the difference. The difference is shown in markdown-like format instead of RST for ease of understanding. It's also not rendered HTML for largely complexity reasons.
Part of MSC2140
Convert status codes to strings if there is a string status code. Fixes a build error when we mix 4xx and 403 in the same definition. We also have to correct stringified numbers to pass the build.
Affects the title and the table of contents. We can't realistically alter just the table of contents, but the table of contents is generated from this header.
Fixes https://github.com/matrix-org/matrix-doc/issues/1800
Currently the *m.key.verification.start* event appears twice with the
exact same title, in the "Key verification framework" section and the
"Short Authentication (SAS) verification" section. It's not immediately
clear that the first occurrence describes the format of the event in
general terms and that the second occurrence describes the fields when
the *m.sas.v1* verification method is being used. This is a similar
relationship to the *m.room.message* event and its various *msgtype*
variants.
This commit does three things:
* It tweaks the generation of the documentation to change the title
of the second occurrence of *m.key.verification.start* to
distinguish it from the first.
* It updates the language in the description of the two versions of the
event to better describe the relationship between the two.
* It adds the optional `next_method` field to the schema of the
*m.sas.v1* variant, as specified in the general form of
*m.key.verification.start*.
Signed-off-by: Jimmy Cuadra <jimmy@jimmycuadra.com>
Fixes https://github.com/matrix-org/matrix-doc/issues/1166
Fixes https://github.com/matrix-org/matrix-doc/issues/1527
Fixes https://github.com/matrix-org/matrix-doc/issues/1827
Note: In order to fix the "state events have the following fields: [no words]" bug (1827) we need to resolve references on common event types. When doing this we ultimately end up with more fields than may be required to explain the section, however this commit alters the section descriptions to just say "these fields" instead of "these additional fields".
This is also preferable over trying to get the inheritance reversed in the common event types, as the `/sync` endpoint has a high amount of reliance on partial events definitions.
# is reserved by the swagger validator as a way to include partial content from a JSON object (eg: "#/path" would include {"test": true} from the object {"path":{"test":true}}). Instead of trying to convince the validator that it is wrong, we'll just use a different character.
Note that our rendering tools do not care about #-style references to objects. It's still somewhat worth changing the character though.
As per [MSC1452](https://github.com/matrix-org/matrix-doc/issues/1452)
Fixes https://github.com/matrix-org/matrix-doc/issues/1254
Although MSC1452 focuses on just the warnings part of the server notices, the base for notices has not been established in the spec. This commit adds the needed support to be able to handle notices.
No intentional divergences from the proposal are included in this changeset. There are a few additions which are used in practice although not defined in the proposal, such as who is responsible for aesthetics, sending notices, and other misc rules.
We don'e need `{{server_server_changelog_r0.1.0}}` (for example), so don't go through the hassle of generating it. Instead, we'll generate the changelog for the requested versions of each API and put that in place. In the future, we may wish to consider bringing back more complicated variables when/if we start generating released versions of the spec on the fly rather than manually.
Currently if you generate a changelog for r0.1.1 of an API, you'd get "No significant changes" which is wrong. You should get a real changelog for the version.
This is now handled by generating a "preferred" changelog which acts as the default for version variables in the RST. Using a specific version's changelog is still supported for the rare cases where that is desired.
Versions are actually on a scale of recommendations, and are expected to be created as needed. The scale presented here (develop/beta/default/recommended/mandatory) is a more wordy version of what was previously discussed/intended for room versions - the labels aren't final and may be changed.
The "Room Specification" (or "Room Version Specification") is the specification that defines which room versions do what and are intended to be documents which speak the truth about how rooms operate under the hood.
The approach taken here is a bit different than other specifications. For starters, the specification is versioned in this project instead of relying on the matrix.org repository to track compiled HTML. This is done for a couple reasons, the first being we're still developing the v1 specification while concurrently making a v2 spec and the second being trying to reduce the reliance on matrix.org's repository for specifications.
Because the room spec is built into versions, some changes needed to be made. The `targets.yaml` now has a special syntax for indicating what version something is at, and the changelog generator can handle rendering different versions of the same changelog (as parsed from the RST). Some additional work has been put in to the changelog parsing to allow us to reference the v1 room spec as "v1" without having to sacrifice clarity in the changelog headings.
Finally, this moves the state resolution algorithms into the versioned
spec as a result of MSC1759 (https://github.com/matrix-org/matrix-doc/pull/1759).
Note: this does not introduce the concept of versioned schemas (tabs) that I was previously working with. There's currently no use for them, so they are shelved elsewhere.
Travis Ralston