Merge branch 'main' into fix-receipts

.github/workflows/main.yml

@@ -92,17 +92,17 @@ jobs:
             export RELEASE="unstable"
           fi
           # The output path matches the final deployment path at spec.matrix.org
-          scripts/dump-swagger.py \
+          scripts/dump-openapi.py \
             --base-url "https://spec.matrix.org${{ needs.calculate-baseurl.outputs.baseURL }}" \
             --api application-service \
             -r "$RELEASE" \
             -o spec/application-service-api/api.json
-          scripts/dump-swagger.py \
+          scripts/dump-openapi.py \
             --base-url "https://spec.matrix.org${{ needs.calculate-baseurl.outputs.baseURL }}" \
             --api client-server \
             -r "$RELEASE" \
             -o spec/client-server-api/api.json
-          scripts/dump-swagger.py \
+          scripts/dump-openapi.py \
             --base-url "https://spec.matrix.org${{ needs.calculate-baseurl.outputs.baseURL }}" \
             --api push-gateway \
             -r "$RELEASE" \

.github/workflows/release.yaml

@@ -20,6 +20,7 @@ jobs:
         uses: actions/setup-node@v3
         with:
           cache: "yarn"
+          cache-dependency-path: packages/npm/yarn.lock
           registry-url: "https://registry.npmjs.org"
 
       - name: 🔨 Install dependencies

.gitignore

@@ -2,7 +2,7 @@ node_modules
 /data/msc
 /env*
 /resources
-/scripts/swagger
+/scripts/openapi
 /scripts/tmp
 /hugo-config.toml
 /public

README.md

@@ -22,7 +22,7 @@ The Matrix spec is compiled with [Hugo](https://gohugo.io/) (a static site gener
 
 * `/data`: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as
   [data objects](https://gohugo.io/templates/data-templates/), so templates don't need to load them from a file and
-  parse them. This is also where our Swagger/OpenAPI definitions and schemas are.
+  parse them. This is also where our OpenAPI definitions and schemas are.
 
 * `/layouts`: this contains [Hugo templates](https://gohugo.io/templates/). Some templates define the overall layout of
   a page: for example, whether it has header, footer, sidebar, and so on.
@@ -52,7 +52,7 @@ Additionally, the following directories may be of interest:
 * `/data-definitions`: Bits of structured data consumable by Matrix implementations.
 * `/meta`: Documentation relating to the spec's processes that are otherwise untracked (release instructions, etc).
 * `/scripts`: Various scripts for generating the spec and validating its contents.
-* `/packages`: Various packages for shipping spec files like OpenAPI/swagger bindings and data definitions.
+* `/packages`: Various packages for shipping spec files like OpenAPI bindings and data definitions.
 
 ## Authoring changes to the spec
 
@@ -87,13 +87,13 @@ steps for authoring changes to the specification and instead of `hugo serve` run
 spec to `/spec`. If you'd like to serve the spec off a path instead of a domain root (eg: `/unstable`), add `--baseURL "/unstable"`
 to the `hugo -d "spec"` command.
 
-For building the swagger definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt`
-and finally `python ./scripts/dump-swagger.py` to generate it to `./scripts/swagger/api-docs.json`. To make use of the generated file,
+For building the OpenAPI definitions, create a python3 virtualenv and activate it. Then run `pip install -r ./scripts/requirements.txt`
+and finally `python ./scripts/dump-openapi.py` to generate it to `./scripts/openapi/api-docs.json`. To make use of the generated file,
 there are a number of options:
 
-* You can open `./scripts/swagger-preview.html` in your browser, and then open the file by clicking on `Local JSON File`.
-* You can run a local HTTP server by running `./scripts/swagger-http-server.py`, and then view the documentation by
-  opening `./scripts/swagger-preview.html` in your browser.
+* You can open `./scripts/openapi-preview.html` in your browser, and then open the file by clicking on `Local JSON File`.
+* You can run a local HTTP server by running `./scripts/openapi-http-server.py`, and then view the documentation by
+  opening `./scripts/openapi-preview.html` in your browser.
 
 ## Issue tracking
 

changelogs/client_server/newsfragments/1638.feature

@@ -0,0 +1 @@
+Add a note to the `/publicRooms` API that the server name is case sensitive. 

changelogs/client_server/newsfragments/1639.clarification

@@ -0,0 +1 @@
+Clarify that an `m.room.name` event with an absent `name` field is not expected behavior.

changelogs/internal/newsfragments/1633.clarification

@@ -0,0 +1 @@
+Replace all mentions of Swagger by OpenAPI.

changelogs/internal/newsfragments/1635.clarification

@@ -0,0 +1 @@
+Fix schema of `m.mentions` object.

changelogs/internal/newsfragments/1648.bugfix

@@ -0,0 +1 @@
+Fix github action workflow responsible for releasing of @matrix-org/spec package.

changelogs/server_server/newsfragments/1636.clarification

@@ -0,0 +1 @@
+Fix schema of `m.receipt` EDU.

content/client-server-api/modules/mentions.md

@@ -13,6 +13,20 @@ the event to reference the entity being mentioned.
 
 {{% definition path="api/client-server/definitions/m.mentions" %}}
 
+An event's content will then look like this:
+
+```json
+{
+    "body": "Hello Alice!",
+    "msgtype": "m.text",
+    "format": "org.matrix.custom.html",
+    "formatted_body": "Hello <a href='https://matrix.to/#/@alice:example.org'>Alice</a>!",
+    "m.mentions": {
+        "user_ids": ["@alice:example.org"]
+    }
+}
+```
+
 Additionally, see the [`.m.rule.is_user_mention`](#_m_rule_is_user_mention) and
 [`.m.rule.is_room_mention`](#_m_rule_is_room_mention) push rules.
 Users should not add their own Matrix ID to the `m.mentions` property as outgoing

data/api/client-server/definitions/m.mentions.yaml

@@ -18,17 +18,13 @@ description: |-
   Describes whether the event mentions other users or the room. This is contained
   within the event's `content` alongside other fields for the relevant event type.
 example: {
-    "body": "Hello Alice!",
-    "msgtype": "m.text",
-    "format": "org.matrix.custom.html",
-    "formatted_body": "Hello <a href='https://matrix.to/#/@alice:example.org'>Alice</a>!",
-    "m.mentions": {
-        "user_ids": ["@alice:example.org"]
-    }
+    "user_ids": ["@alice:example.org"]
 }
 properties:
   user_ids:
-    type: string[]
+    type: array
+    items:
+      type: string
     description: A list of Matrix IDs of mentioned users.
   room:
     type: boolean

data/api/client-server/keys.yaml

@@ -40,7 +40,7 @@ paths:
                 one_time_keys:
                   # $ref: "definitions/one_time_keys.yaml"
                   # XXX: We can't define an actual object here, so we have to hope
-                  # that people will look at the swagger source or can figure it out
+                  # that people will look at the OpenAPI source or can figure it out
                   # from the other endpoints/example.
                   type: object
                   title: OneTimeKeys

data/api/client-server/list_public_rooms.yaml

@@ -154,7 +154,7 @@ paths:
           name: server
           description: |-
             The server to fetch the public room lists from. Defaults to the
-            local server.
+            local server. Case sensitive.
           schema:
             type: string
       responses:
@@ -181,7 +181,7 @@ paths:
           name: server
           description: |-
             The server to fetch the public room lists from. Defaults to the
-            local server.
+            local server. Case sensitive.
           schema:
             type: string
       requestBody:

data/api/server-server/definitions/event-schemas/m.receipt.yaml

@@ -41,41 +41,45 @@ allOf:
             # on. At that point, m.read can become optional (maybe).
             "m.read":
               type: object
-              description: Read receipts for users in the room.
-              title: User Read Receipt
-              properties:
-                event_ids:
-                  type: array
-                  description: |-
-                    The extremity event IDs that the user has read up to.
-                  minItems: 1
-                  maxItems: 1
-                  items:
-                    type: string
-                  example: ['$read_this_event:matrix.org']
-                data:
-                  type: object
-                  description: Metadata for the read receipt.
-                  title: Read Receipt Metadata
-                  properties:
-                    ts:
-                      type: integer
-                      format: int64
-                      description: |-
-                        A POSIX timestamp in milliseconds for when the user read
-                        the event specified in the read receipt.
-                      example: 1533358089009
-                    thread_id:
-                        type: string
-                        x-addedInMatrixVersion: "1.4"
+              description: |-
+                Read receipts for users in the room. The string key is the user
+                ID the receipt belongs to.
+              additionalProperties:
+                type: object
+                title: User Read Receipt
+                properties:
+                  event_ids:
+                    type: array
+                    description: |-
+                      The extremity event IDs that the user has read up to.
+                    minItems: 1
+                    maxItems: 1
+                    items:
+                      type: string
+                    example: ['$read_this_event:matrix.org']
+                  data:
+                    type: object
+                    description: Metadata for the read receipt.
+                    title: Read Receipt Metadata
+                    properties:
+                      ts:
+                        type: integer
+                        format: int64
                         description: |-
-                            The root thread event's ID (or `main`) for which
-                            thread this receipt is intended to be under. If
-                            not specified, the read receipt is *unthreaded*
-                            (default).
-                        example: "$threadroot"
-                  required: ['ts']
-              required: ['event_ids', 'data']
+                          A POSIX timestamp in milliseconds for when the user read
+                          the event specified in the read receipt.
+                        example: 1533358089009
+                      thread_id:
+                          type: string
+                          x-addedInMatrixVersion: "1.4"
+                          description: |-
+                              The root thread event's ID (or `main`) for which
+                              thread this receipt is intended to be under. If
+                              not specified, the read receipt is *unthreaded*
+                              (default).
+                          example: "$threadroot"
+                    required: ['ts']
+                required: ['event_ids', 'data']
           required: ['m.read']
         example: {
           "!some_room:example.org": {

data/event-schemas/schema/m.room.name.yaml

@@ -7,9 +7,8 @@ description: |-
     is a human-friendly string designed to be displayed to the end-user. The
     room name is not unique, as multiple rooms can have the same room name set.
 
-    A room with an `m.room.name` event with an absent, null, or empty
-    `name` field should be treated the same as a room with no `m.room.name`
-    event.
+    If a room has an `m.room.name` event with an absent, null, or empty `name`
+    field, it should be treated the same as a room with no `m.room.name` event.
 
     An event of this type is automatically created when creating a room using
     `/createRoom` with the `name` key.

layouts/partials/json-schema/resolve-allof.html

@@ -1,6 +1,6 @@
 {{/*
 
-  Resolves the `allOf` keyword (https://swagger.io/specification/v2/#composition-and-inheritance-polymorphism),
+  Resolves the `allOf` keyword (https://spec.openapis.org/oas/v3.1.0#composition-and-inheritance-polymorphism),
   given a JSON schema object.
 
   `allOf` is used to support a kind of inheritance for JSON schema objects.

layouts/partials/openapi/render-api.html

@@ -2,7 +2,7 @@
 
   Render an HTTP API, given:
 
-  * `api_data`: the OpenAPI/Swagger data
+  * `api_data`: the OpenAPI data
   * `base_url`: the base URL: that is, the part we glue onto the front
   of each value in `paths` to get a complete URL.
   * `path`: the directory under /data where we found this API definition.

layouts/partials/openapi/render-operation.html

@@ -4,7 +4,7 @@
 
   * `method`: the method, e.g. GET, PUT
   * `endpoint`: the endpoint
-  * `operation_data`: the OpenAPI/Swagger data for the operation
+  * `operation_data`: the OpenAPI data for the operation
   * `path`: the path where this definition was found, to enable us to resolve "$ref"
 
   This template renders the operation as a `<section>` containing:

layouts/partials/openapi/render-parameters.html

@@ -2,7 +2,7 @@
 
   Render the parameters of a given type, given:
 
-  * `parameters`: OpenAPI/Swagger data specifying the parameters
+  * `parameters`: OpenAPI data specifying the parameters
   * `type`: the type of parameters to render: "header, ""path", "query"
   * `caption`: caption to use for the table
 

layouts/partials/openapi/render-request.html

@@ -2,8 +2,8 @@
 
   Render the request part of a single HTTP API operation, given:
 
-  * `parameters`: OpenAPI/Swagger data specifying the parameters
-  * `request_body`: OpenAPI/Swagger data specifying the request body
+  * `parameters`: OpenAPI data specifying the parameters
+  * `request_body`: OpenAPI data specifying the request body
   * `path`: the path where this definition was found, to enable us to resolve "$ref"
   * `anchor_base`: a prefix to add to the HTML anchors generated for each object
 

layouts/partials/openapi/render-responses.html

@@ -2,7 +2,7 @@
 
   Render the response part of a single HTTP API operation, given:
 
-  * `responses`: OpenAPI/Swagger data specifying the responses
+  * `responses`: OpenAPI data specifying the responses
   * `path`: the path where this definition was found, to enable us to resolve "$ref"
   * `anchor_base`: a prefix to add to the HTML anchors generated for each object
 

layouts/shortcodes/http-api.html

@@ -1,12 +1,12 @@
 {{/*
 
-  This template is used to render an HTTP API, given an OpenAPI/Swagger definition.
+  This template is used to render an HTTP API, given an OpenAPI definition.
 
   It expects to be passed two parameters:
 
   * a `spec` parameter identifying the spec, which must be the name of
   a directory under /data/api
-  * an `api` parameter, identifying an OpenAPI/Swagger definition,
+  * an `api` parameter, identifying an OpenAPI definition,
   which is the name of a schema file under "data/api/$spec".
   The file extension is omitted. For example:
 

openapi_extensions.md

@@ -1,6 +1,6 @@
 # OpenAPI Extensions
 
-For some functionality that is not directly provided by the OpenAPI v2
+For some functionality that is not directly provided by the OpenAPI v3.1
 specification, some extensions have been added that are to be consistent
 across the specification. The defined extensions are listed below. Extensions
 should not break parsers, however if extra functionality is required, aware
@@ -12,56 +12,9 @@ To ease API design and management, the API definition is split across several
 files. Each of these files is self-contained valid OpenAPI.
 
 There is no single root file in the source tree as OpenAPI requires; this file
-can be generated by `dump-swagger.py`. The script does not convert
-the extensions described further in this document (`oneOf` and parameter
-exploding) so there can be minor interoperability issues with tooling that
-expects compliant Swagger.
-
-## Extensible Query Parameters
-
-<!-- TODO: Remove and change instances to 'explode' after OpenAPI/Swagger v3 update -->
-
-If a unknown amount of query parameters can be added to a request, the `name`
-must be `fields...`, with the trailing ellipses representing the possibility
-of more fields.
-
-Example:
-
-```
-  - in: query
-    name: fields...
-    type: string
-```
-
-## Using oneOf to provide type alternatives
-
-<!-- TODO: Remove this section after upgrading to OpenAPI v3 -->
-
-`oneOf` (available in JSON Schema and Swagger/OpenAPI v3 but not in v2)
-is used in cases when a simpler type specification as a list of types
-doesn't work, as in the following example:
-```
-  properties:
-    old: # compliant with old Swagger
-      type:
-        - string
-        - object # Cannot specify a schema here
-    new: # uses oneOf extension
-      oneOf:
-        - type: string
-        - type: object
-          title: CustomSchemaForTheWin
-          properties:
-            ...
-```
-
-## OpenAPI 3's "2xx" format for response codes
-
-<!-- TODO: Remove this section after upgrading to OpenAPI v3 -->
-
-In some cases, the schema will have HTTP response code definitions like
-`2xx`, `3xx`, and `4xx`. These indicate that a response code within those
-ranges (`2xx` = `200` to `299`) is valid for the schema.
+can be generated by `dump-openapi.py`. The script does not convert
+the extensions described further in this document so there can be minor
+interoperability issues with tooling that expects compliant OpenAPI.
 
 ## Custom `x-addedInMatrixVersion` key
 

scripts/check-openapi-sources.py

@@ -87,11 +87,11 @@ def check_response(filepath, request, code, response):
             ), e)
 
 
-def check_swagger_file(filepath):
+def check_openapi_file(filepath):
     with open(filepath) as f:
-        swagger = yaml.safe_load(f)
+        openapi = yaml.safe_load(f)
 
-    for path, path_api in swagger.get('paths', {}).items():
+    for path, path_api in openapi.get('paths', {}).items():
 
         for method, request_api in path_api.items():
             request = "%s %s" % (method.upper(), path)
@@ -169,7 +169,7 @@ if __name__ == '__main__':
     # Get the directory that this script is residing in
     script_directory = os.path.dirname(os.path.realpath(__file__))
 
-    # Resolve the directory containing the swagger sources,
+    # Resolve the directory containing the OpenAPI sources,
     # relative to the script path
     source_files_directory = os.path.realpath(os.path.join(script_directory, "../data"))
 
@@ -182,6 +182,6 @@ if __name__ == '__main__':
             path = os.path.join(root, filename)
 
             try:
-                check_swagger_file(path)
+                check_openapi_file(path)
             except Exception as e:
                 raise ValueError("Error checking file %s" % (path,), e)

scripts/dump-openapi.py

@@ -1,9 +1,8 @@
 #!/usr/bin/env python3
 
-# dump-swagger reads all of the swagger API docs used in spec generation and
-# outputs a JSON file which merges them all, for use as input to a swagger UI
+# dump-openapi reads all of the OpenAPI docs used in spec generation and
+# outputs a JSON file which merges them all, for use as input to an OpenAPI
 # viewer.
-# See https://github.com/swagger-api/swagger-ui for details of swagger-ui.
 
 # Copyright 2016 OpenMarket Ltd
 #
@@ -85,7 +84,7 @@ def edit_links(node, base_url):
             edit_links(item, base_url)
 
 parser = argparse.ArgumentParser(
-    "dump-swagger.py - assemble the Swagger specs into a single JSON file"
+    "dump-openapi.py - assemble the OpenAPI specs into a single JSON file"
 )
 parser.add_argument(
     "--base-url", "-b",
@@ -114,7 +113,7 @@ parser.add_argument(
 )
 parser.add_argument(
     "-o", "--output",
-    default=os.path.join(scripts_dir, "swagger", "api-docs.json"),
+    default=os.path.join(scripts_dir, "openapi", "api-docs.json"),
     help="File to write the output to. Default: %(default)s"
 )
 args = parser.parse_args()

scripts/openapi-http-server.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
-# Runs an HTTP server on localhost:8000 which will serve the generated swagger
-# JSON so that it can be viewed in an online swagger UI.
+# Runs an HTTP server on localhost:8000 which will serve the generated OpenAPI
+# JSON so that it can be viewed in an online OpenAPI viewer.
 
 # Copyright 2016 OpenMarket Ltd
 #
@@ -41,13 +41,13 @@ if __name__ == '__main__':
         help='TCP port to listen on (default: %(default)s)',
     )
     parser.add_argument(
-        'swagger_dir', nargs='?',
-        default=os.path.join(scripts_dir, 'swagger'),
+        'openapi_dir', nargs='?',
+        default=os.path.join(scripts_dir, 'openapi'),
         help='directory to serve (default: %(default)s)',
     )
     args = parser.parse_args()
 
-    os.chdir(args.swagger_dir)
+    os.chdir(args.openapi_dir)
 
     httpd = socketserver.TCPServer(("localhost", args.port),
                                    MyHTTPRequestHandler)