Improve documentation for pushers and push gateways

This fixes a number of formatting issues alongside a few documentation problems:
* The push gateway can actually expect less parameters than previously advertised. This is for user privacy.
* Introduction of the `m.email` pusher for email-capable homeservers.
* Fields not being flagged as required on some endpoints.
* Document the `event_id_only` format

Note: this does not attempt to document push rules, just pushers.

Fixes https://github.com/matrix-org/matrix-doc/issues/1374
Fixes https://github.com/matrix-org/matrix-doc/issues/1087

api/client-server/pusher.yaml

@@ -1,4 +1,5 @@
 # Copyright 2016 OpenMarket Ltd
+# Copyright 2018 New Vector Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -31,13 +32,13 @@ paths:
     get:
       summary: Gets the current pushers for the authenticated user
       description: |-
-          Gets all currently active pushers for the authenticated user
+          Gets all currently active pushers for the authenticated user.
       operationId: getPushers
       security:
         - accessToken: []
       responses:
         200:
-          description: The pushers for this user
+          description: The pushers for this user.
           examples:
             application/json: {
               "pushers": [
@@ -70,7 +71,7 @@ paths:
                     pushkey:
                       type: string
                       description: |-
-                        This is a unique identifier for this pusher. See `/set` for
+                        This is a unique identifier for this pusher. See ``/set`` for
                         more detail.
                         Max length, 512 bytes.
                     kind:
@@ -115,6 +116,19 @@ paths:
                           description: |-
                             Required if ``kind`` is ``http``. The URL to use to send
                             notifications to.
+                        format:
+                          type: string
+                          description: |-
+                            The format to use when sending notifications to the Push
+                            Gateway.
+                  required:
+                    - pushkey
+                    - app_id
+                    - kind
+                    - app_display_name
+                    - device_display_name
+                    - lang
+                    - data
       tags:
         - Push notifications
   "/pushers/set":
@@ -130,7 +144,7 @@ paths:
       parameters:
         - in: body
           name: pusher
-          description: The pusher information
+          description: The pusher information.
           required: true
           schema:
             type: object
@@ -143,7 +157,8 @@ paths:
               "app_id": "com.example.app.ios",
               "pushkey": "APA91bHPRgkF3JUikC4ENAHEeMrd41Zxv3hVZjC9KtT8OvPVGJ-hQMRKRrZuJAEcl7B338qju59zJMjw2DELjzEvxwYv7hH5Ynpc1ODQ0aT4U4OFEeco8ohsN5PjL1iC2dNtk2BAokeMCg2ZXKqpc8FXKmhX94kIxQ",
               "data": {
-                  "url": "https://push-gateway.location.here"
+                "url": "https://push-gateway.location.here/_matrix/push/v1/notify",
+                "format": "event_id_only"
               },
               "append": false
             }
@@ -157,11 +172,15 @@ paths:
                   for APNS or the Registration ID for GCM. If your notification
                   client has no such concept, use any unique identifier.
                   Max length, 512 bytes.
+
+                  If the ``kind`` is ``"email"``, this is the email address to
+                  send notifications to.
               kind:
                 type: string
                 description: |-
                   The kind of pusher to configure. ``"http"`` makes a pusher that
-                  sends HTTP pokes. ``null`` deletes the pusher.
+                  sends HTTP pokes. ``"email"`` makes a pusher that emails the
+                  user with unread notifications. ``null`` deletes the pusher.
               app_id:
                 type: string
                 description: |-
@@ -169,6 +188,8 @@ paths:
                   It is recommended that this end with the platform, such that
                   different platform versions get different app identifiers.
                   Max length, 64 chars.
+
+                  If the ``kind`` is ``"email"``, this is ``"m.email"``.
               app_display_name:
                 type: string
                 description: |-
@@ -188,7 +209,7 @@ paths:
                 type: string
                 description: |-
                   The preferred language for receiving notifications (e.g. 'en'
-                  or 'en-US')
+                  or 'en-US').
               data:
                 type: object
                 description: |-
@@ -202,6 +223,15 @@ paths:
                     description: |-
                       Required if ``kind`` is ``http``. The URL to use to send
                       notifications to.
+                  format:
+                    type: string
+                    description: |-
+                      The format to send notifications in to Push Gateways. If
+                      unset or unrecognized, the homeserver should populate as
+                      many fields as it can in the push notification. If set to
+                      ``"event_id_only"``, the homeserver should only send the
+                      absolute minimum amount of information possible to the push
+                      gateway by not including optional fields.
               append:
                 type: boolean
                 description: |-
@@ -216,10 +246,10 @@ paths:
         200:
           description: The pusher was set.
           examples:
-            application/json: {
+            application/json: {}
-              }
           schema:
-            type: object  # empty json object
+            type: object
+            description: An empty object.
         400:
           description: One or more of the pusher values were invalid.
           examples:

api/push-gateway/push_notifier.yaml

@@ -1,4 +1,5 @@
 # Copyright 2016 OpenMarket Ltd
+# Copyright 2018 New Vector Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,7 +20,7 @@ host: localhost:8008
 schemes:
   - https
   - http
-basePath: /_matrix/push/%CLIENT_MAJOR_VERSION%
+basePath: /_matrix/push/v1
 consumes:
   - application/json
 produces:
@@ -38,14 +39,14 @@ paths:
 
         Notifications about a particular event will normally cause the user to be
         alerted in some way. It is therefore necessary to perform duplicate
-        suppression for such notifications using the `event_id` field to avoid
+        suppression for such notifications using the ``event_id`` field to avoid
         retries of this HTTP API causing duplicate alerts. The operation of
         updating counts of unread notifications should be idempotent and
         therefore do not require duplicate suppression.
 
-        Notifications are sent to the URL configured when the pusher is
+        Notifications are sent to the URL configured when the pusher is created.
-        created. This means that the HTTP path may be different depending on the
+        This means that the HTTP path may be different depending on the push
-        push gateway.
+        gateway.
       operationId: notify
       parameters:
         - in: body
@@ -111,14 +112,10 @@ paths:
                     type: string
                     description: |-
                       The type of the event as in the event's ``type`` field.
-                      Required if the notification relates to a specific
-                      Matrix event.
                   sender:
                     type: string
                     description: |-
                       The sender of the event as in the corresponding event field.
-                      Required if the notification relates to a specific
-                      Matrix event.
                   sender_display_name:
                     type: string
                     description: |-
@@ -149,14 +146,15 @@ paths:
                     title: EventContent
                     description: |-
                       The ``content`` field from the event, if present. If the
-                      event had no content field, this field is omitted.
+                      event had no content field or the pusher wishes to not include
+                      it, this field is omitted.
                   counts:
                     type: object
                     title: Counts
                     description: |-
                       This is a dictionary of the current number of unacknowledged
                       communications for the recipient user. Counts whose value is
-                      zero are omitted.
+                      zero should be omitted.
                     properties:
                       unread:
                         type: integer
@@ -180,10 +178,10 @@ paths:
                         app_id:
                           type: string
                           description: |-
-                            The app_id given when the pusher was created.
+                            The ``app_id`` given when the pusher was created.
                         pushkey:
                           type: string
-                          description: The pushkey given when the pusher was created.
+                          description: The ``pushkey`` given when the pusher was created.
                         pushkey_ts:
                           type: integer
                           description: |-
@@ -202,6 +200,7 @@ paths:
                           description: |-
                             A dictionary of customisations made to the way this
                             notification is to be presented. These are added by push rules.
+                      required: ['app_id', 'pushkey']
       responses:
         200:
           description: A list of rejected push keys.
@@ -222,7 +221,8 @@ paths:
                   pushkeys and remove the associated pushers. It may not
                   necessarily be the notification in the request that failed:
                   it could be that a previous notification to the same pushkey
-                  failed.
+                  failed. May be empty.
                 items:
                   type: string
-                  description: A pushkey
+                  description: A pushkey that has been rejected.
+            required: ['rejected']

specification/modules/push.rst

@@ -65,6 +65,9 @@ APNS or Google's GCM. This happens as follows:
    notifications.
 5. The Push Provider sends the notification to the device.
 
+Homeservers may optionally support email notifications or other push kinds
+of push, identified by the ``kind`` field of the pusher configuration.
+
 Definitions for terms used in this section are below:
 
 Push Provider