Merge pull request #1555 from turt2live/travis/as/cleanup

Application service spec cleanup; Security definitions; r0 prep

api/application-service/definitions/security.yaml

@@ -0,0 +1,18 @@
+# Copyright 2018 New Vector Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+homeserverAccessToken:
+  type: apiKey
+  description: The ``hs_token`` provided by the application service's registration.
+  name: access_token
+  in: query

api/application-service/protocols.yaml

@@ -24,6 +24,8 @@ consumes:
   - application/json
 produces:
   - application/json
+securityDefinitions:
+  $ref: definitions/security.yaml
 paths:
   "/_matrix/app/unstable/thirdparty/protocol/{protocol}":
     get:
@@ -33,6 +35,8 @@ paths:
         with specific information about the various third party networks that
         an application service supports.
       operationId: getProtocolMetadata
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: protocol
@@ -80,6 +84,8 @@ paths:
         User ID linked to a user on the third party network, given a set of
         user parameters.
       operationId: queryUserByProtocol
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: protocol
@@ -131,6 +137,8 @@ paths:
       description: |-
         Retrieve a list of Matrix portal rooms that lead to the matched third party location.
       operationId: queryLocationByProtocol
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: protocol
@@ -183,6 +191,8 @@ paths:
         Retrieve an array of third party network locations from a Matrix room
         alias.
       operationId: queryLocationByAlias
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: query
           name: alias
@@ -227,6 +237,8 @@ paths:
       description: |-
         Retrieve an array of third party users from a Matrix User ID.
       operationId: queryUserByID
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: query 
           name: userid

api/application-service/query_room.yaml

@@ -25,6 +25,8 @@ consumes:
   - application/json
 produces:
   - application/json
+securityDefinitions:
+  $ref: definitions/security.yaml
 paths:
   "/rooms/{roomAlias}":
     get:
@@ -36,6 +38,8 @@ paths:
         homeserver will send this request when it receives a request to join a
         room alias within the application service's namespace.
       operationId: queryRoomByAlias
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: roomAlias

api/application-service/query_user.yaml

@@ -25,6 +25,8 @@ consumes:
   - application/json
 produces:
   - application/json
+securityDefinitions:
+  $ref: definitions/security.yaml
 paths:
   "/users/{userId}":
     get:
@@ -36,6 +38,8 @@ paths:
         send this request when it receives an event for an unknown user ID in
         the application service's namespace, such as a room invite.
       operationId: queryUserById
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: userId

api/application-service/transactions.yaml

@@ -23,6 +23,8 @@ schemes:
 basePath: "/"
 produces:
   - application/json
+securityDefinitions:
+  $ref: definitions/security.yaml
 paths:
   "/transactions/{txnId}":
     put:
@@ -35,6 +37,8 @@ paths:
         from message events via the presence of a ``state_key``, rather than
         via the event type.
       operationId: sendTransaction
+      security:
+        - homeserverAccessToken: []
       parameters:
         - in: path
           name: txnId

changelogs/application_service/newsfragments/.gitignore

@@ -0,0 +1 @@
+!.gitignore

changelogs/application_service/pyproject.toml

@@ -0,0 +1,30 @@
+[tool.towncrier]
+    filename = "../application_service.rst"
+    directory = "newsfragments"
+    issue_format = "`#{issue} <https://github.com/matrix-org/matrix-doc/issues/{issue}>`_"
+    title_format = "{version}"
+    
+    [[tool.towncrier.type]]
+        directory = "breaking"
+        name = "Breaking Changes"
+        showcontent = true
+    
+    [[tool.towncrier.type]]
+        directory = "deprecation"
+        name = "Deprecations"
+        showcontent = true
+    
+    [[tool.towncrier.type]]
+        directory = "new"
+        name = "New Endpoints"
+        showcontent = true
+    
+    [[tool.towncrier.type]]
+        directory = "feature"
+        name = "Backwards Compatible Changes"
+        showcontent = true
+    
+    [[tool.towncrier.type]]
+        directory = "clarification"
+        name = "Spec Clarifications"
+        showcontent = true

scripts/gendoc.py

@@ -518,6 +518,10 @@ if __name__ == '__main__':
         "--server_release", "-s", action="store", default="unstable",
         help="The server-server release tag to generate, e.g. r1.2"
     )
+    parser.add_argument(
+        "--appservice_release", "-a", action="store", default="unstable",
+        help="The appservice release tag to generate, e.g. r1.2"
+    )
     parser.add_argument(
         "--push_gateway_release", "-p", action="store", default="unstable",
         help="The push gateway release tag to generate, e.g. r1.2"
@@ -546,6 +550,7 @@ if __name__ == '__main__':
         "%CLIENT_MAJOR_VERSION%": "r0",
         "%SERVER_RELEASE_LABEL%": args.server_release,
         "%SERVER_MAJOR_VERSION%": extract_major(args.server_release),
+        "%APPSERVICE_RELEASE_LABEL%": args.appservice_release,
         "%PUSH_GATEWAY_RELEASE_LABEL%": args.push_gateway_release,
     }
 

scripts/templating/matrix_templates/sections.py

@@ -37,6 +37,10 @@ class MatrixSections(Sections):
         changelogs = self.units.get("changelogs")
         return changelogs["push_gateway"]
 
+    def render_application_service_changelog(self):
+        changelogs = self.units.get("changelogs")
+        return changelogs["application_service"]
+
     def _render_events(self, filterFn, sortFn):
         template = self.env.get_template("events.tmpl")
         examples = self.units.get("event_examples")

scripts/templating/matrix_templates/units.py

@@ -754,6 +754,7 @@ class MatrixUnits(Units):
     def load_apis(self, substitutions):
         cs_ver = substitutions.get("%CLIENT_RELEASE_LABEL%", "unstable")
         fed_ver = substitutions.get("%SERVER_RELEASE_LABEL%", "unstable")
+        as_ver = substitutions.get("%APPSERVICE_RELEASE_LABEL%", "unstable")
         push_gw_ver = substitutions.get("%PUSH_GATEWAY_RELEASE_LABEL%", "unstable")
 
         # we abuse the typetable to return this info to the templates
@@ -767,8 +768,8 @@ class MatrixUnits(Units):
                 fed_ver,
                 "Federation between servers",
             ), TypeTableRow(
-                "`Application Service API <application_service/unstable.html>`_",
-                "unstable",
+                "`Application Service API <application_service/"+as_ver+".html>`_",
+                as_ver,
                 "Privileged server plugins",
             ), TypeTableRow(
                 "`Identity Service API <identity_service/unstable.html>`_",

specification/application_service_api.rst

@@ -30,22 +30,37 @@ irrespective of the underlying homeserver implementation.
 .. contents:: Table of Contents
 .. sectnum::
 
-Specification version
----------------------
+Changelog
+---------
+
+
+.. topic:: Version: unstable
+{{application_service_changelog}}
 
 This version of the specification is generated from
 `matrix-doc <https://github.com/matrix-org/matrix-doc>`_ as of Git commit
 `{{git_version}} <https://github.com/matrix-org/matrix-doc/tree/{{git_rev}}>`_.
 
+For the full historical changelog, see
+https://github.com/matrix-org/matrix-doc/blob/master/changelogs/application_service.rst
+
+Other versions of this specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following other versions are also available, in reverse chronological order:
+
+- `HEAD <https://matrix.org/docs/spec/application_service/unstable.html>`_: Includes all changes since the latest versioned release.
+
+
 Application Services
 --------------------
-Application services are passive and can only observe events from a given
-homeserver (HS). They can inject events into rooms they are participating in.
+Application services are passive and can only observe events from homeserver.
+They can inject events into rooms they are participating in.
 They cannot prevent events from being sent, nor can they modify the content of
 the event being sent. In order to observe events from a homeserver, the
 homeserver needs to be configured to pass certain types of traffic to the
 application service.  This is achieved by manually configuring the homeserver
-with information about the application service (AS).
+with information about the application service.
 
 Registration
 ~~~~~~~~~~~~
@@ -179,24 +194,24 @@ events. Each list of events includes a transaction ID, which works as follows:
 
  Typical
  HS ---> AS : Homeserver sends events with transaction ID T.
-    <---    : AS sends back 200 OK.
+    <---    : Application Service sends back 200 OK.
 
  AS ACK Lost
  HS ---> AS : Homeserver sends events with transaction ID T.
     <-/-    : AS 200 OK is lost.
  HS ---> AS : Homeserver retries with the same transaction ID of T.
-    <---    : AS sends back 200 OK. If the AS had processed these events
-              already, it can NO-OP this request (and it knows if it is the same
-              events based on the transaction ID).
+    <---    : Application Service sends back 200 OK. If the AS had processed these
+              events already, it can NO-OP this request (and it knows if it is the
+              same events based on the transaction ID).
 
 The events sent to the application service should be linearised, as if they were
 from the event stream. The homeserver MUST maintain a queue of transactions to
-send to the AS. If the application service cannot be reached, the homeserver
-SHOULD backoff exponentially until the application service is reachable again.
+send to the application service. If the application service cannot be reached, the
+homeserver SHOULD backoff exponentially until the application service is reachable again.
 As application services cannot *modify* the events in any way, these requests can
 be made without blocking other aspects of the homeserver. Homeservers MUST NOT
 alter (e.g. add more) events they were going to send within that transaction ID
-on retries, as the AS may have already processed the events.
+on retries, as the application service may have already processed the events.
 
 {{transactions_as_http_api}}
 
@@ -337,7 +352,7 @@ users needs API changes in order to:
 - Have a 'passwordless' user.
 
 This involves bypassing the registration flows entirely. This is achieved by
-including the AS token on a ``/register`` request, along with a login type of
+including the ``as_token`` on a ``/register`` request, along with a login type of
 ``m.login.application_service`` to set the desired user ID without a password.
 
 ::
@@ -377,8 +392,6 @@ additional parameters on the ``/publicRooms`` client-server endpoint.
 Event fields
 ~~~~~~~~~~~~
 
-.. TODO-TravisR: Fix this section to be a general "3rd party networks" section
-
 We recommend that any events that originated from a remote network should
 include an ``external_url`` field in their content to provide a way for Matrix
 clients to link into the 'native' client from which the event originated.

specification/targets.yaml

@@ -13,7 +13,7 @@ targets:
   application_service:
     files:
       - application_service_api.rst
-    version_label: unstable
+    version_label: "%APPSERVICE_RELEASE_LABEL%"
   server_server:
     files:
       - server_server_api.rst