Merge pull request #241 from matrix-org/daniel/kegandrafts

Delete old drafts This information is now either in the spec, or complete lies
9 years ago · 40b76c0364
parent 70a6bef580 17716c6c64
commit 40b76c0364
5 changed files with 0 additions and 780 deletions
--- a/drafts/address-book-repo.rst
+++ b/drafts/address-book-repo.rst
@ -1,14 +0,0 @@
 Address book repository
 =======================
 .. NOTE::
  This section is a work in progress.
 .. TODO-spec
  Do we even need it?  Clients can use out-of-band addressbook servers for now;
  this should definitely not be core.
  - format: POST(?) wodges of json, some possible processing, then return wodges of json on GET.
  - processing may remove dupes, merge contacts, pepper with extra info (e.g. matrix-ability of
    contacts), etc.
  - Standard json format for contacts? Piggy back off vcards?
--- a/drafts/as-http-api.rst
+++ b/drafts/as-http-api.rst
@ -1,574 +0,0 @@
 .. TODO
  Sometimes application services need to create rooms (e.g. when lazy loading 
  from room aliases). Created rooms need to have a user that created them, so 
  federation works (as it relies on an entry existing in m.room.member). We 
  should be able to add metadata to m.room.member to state that this user is an 
  application service, a virtual user, etc.
 Application Services HTTP API
 =============================
 .. contents:: Table of Contents
 .. sectnum::
 Application Service -> Homeserver
 ----------------------------------
 This contains homeserver APIs which are used by the application service.
 Registration API ``[Draft]``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 This API registers the application service with its host homeserver to offer its
 services.
 Inputs:
 - Credentials (e.g. some kind of string token)
 - Namespace[users]
 - Namespace[room aliases]
 - URL base to receive inbound comms
 Output:
 - The credentials the HS will use to query the AS with in return. (e.g. some 
   kind of string token)
 Side effects:
 - The HS will start delivering events to the URL base specified if this 200s.
 API called when:
 - The application service wants to register with a brand new homeserver.
 Notes:
 - An application service can state whether they should be the only ones who 
   can manage a specified namespace. This is referred to as an "exclusive" 
   namespace. An exclusive namespace prevents humans and other application 
   services from creating/deleting entities in that namespace. Typically,
   exclusive namespaces are used when the rooms represent real rooms on
   another service (e.g. IRC). Non-exclusive namespaces are used when the
   application service is merely augmenting the room itself (e.g. providing
   logging or searching facilities).
 - Namespaces are represented by POSIX extended regular expressions in JSON. 
   They look like::
     users: [
       {
         "exclusive": true,
         "regex": "@irc\.freenode\.net/.*"
       }
     ]
 ::
 POST /register
 Request format
 {
   url: "https://my.application.service.com/matrix/",
   as_token: "some_AS_token",
   namespaces: {
     users: [
       {
         "exclusive": true,
         "regex": "@irc\.freenode\.net/.*"
       }
     ],
     aliases: [
       {
         "exclusive": true,
         "regex": "#irc\.freenode\.net/.*"
       }
     ],
     rooms: [
       {
         "exclusive": true,
         "regex": "!irc\.freenode\.net/.*"
       }
     ]
   }
 }
 Returns:
   200 : Registration accepted.
   400 : Namespaces do not conform to regex
   401 : Credentials need to be supplied.
   403 : AS credentials rejected.
   200 OK response format
   {
     hs_token: "string"
   }
 Unregister API ``[Draft]``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 This API unregisters a previously registered AS from the homeserver.
 Inputs:
 - AS token
 Output:
 - None.
 Side effects:
 - The HS will stop delivering events to the URL base specified for this AS if 
   this 200s.
 API called when:
 - The application service wants to stop receiving all events from the HS.
 ::
  POST /unregister
  Request format
  {
    as_token: "string"
  }
 Homeserver -> Application Service
 ----------------------------------
 This contains application service APIs which are used by the homeserver.
 User Query ``[Draft]``
 ~~~~~~~~~~~~~~~~~~~~~~
 This API is called by the HS to query the existence of a user on the Application
 Service's namespace.
 Inputs:
 - User ID
 - HS Credentials
 Output:
 - Whether the user exists.
 Side effects:
 - User is created on the HS by the AS via CS APIs during the processing of this request.
 API called when:
 - HS receives an event for an unknown user ID in the AS's namespace, e.g. an
   invite event to a room.
 Notes:
 - When the AS receives this request, if the user exists, it must create the user via
   the CS API.
 - It can also set arbitrary information about the user (e.g. display name, join rooms, etc)
   using the CS API.
 - When this setup is complete, the AS should respond to the HS request. This means the AS 
   blocks the HS until the user is created.
 - This is deemed more flexible than alternative methods (e.g. returning a JSON blob with the
   user's display name and get the HS to provision the user).
 Retry notes:
 - The homeserver cannot respond to the client's request until the response to
   this API is obtained from the AS.
 - Recommended that homeservers try a few times then time out, returning a
   408 Request Timeout to the client.
 ::
 GET /users/$user_id?access_token=$hs_token
 Returns:
   200 : User is recognised.
   404 : User not found.
   401 : Credentials need to be supplied.
   403 : HS credentials rejected.
   200 OK response format
   {}
 Room Alias Query ``[Draft]``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 This API is called by the HS to query the existence of a room alias on the 
 Application Service's namespace.
 Inputs:
 - Room alias
 - HS Credentials
 Output:
 - Whether the room exists.
 Side effects:
 - Room is created on the HS by the AS via CS APIs during the processing of 
   this request.
 API called when:
 - HS receives an event to join a room alias in the AS's namespace.
 Notes:
 - When the AS receives this request, if the room exists, it must create the room via
   the CS API.
 - It can also set arbitrary information about the room (e.g. name, topic, etc)
   using the CS API.
 - It can send messages as other users in order to populate scrollback.
 - When this setup is complete, the AS should respond to the HS request. This means the AS 
   blocks the HS until the room is created and configured.
 - This is deemed more flexible than alternative methods (e.g. returning an initial sync
   style JSON blob and get the HS to provision the room). It also means that the AS knows
   the room ID -> alias mapping.
 Retry notes:
 - The homeserver cannot respond to the client's request until the response to
   this API is obtained from the AS.
 - Recommended that homeservers try a few times then time out, returning a
   408 Request Timeout to the client.
 ::
 GET /rooms/$room_alias?access_token=$hs_token
 Returns:
   200 : Room is recognised.
   404 : Room not found.
   401 : Credentials need to be supplied.
   403 : HS credentials rejected.
   200 OK response format
   {}
 Pushing ``[Draft]``
 ~~~~~~~~~~~~~~~~~~~
 This API is called by the HS when the HS wants to push an event (or batch of 
 events) to the AS.
 Inputs:
 - HS Credentials
 - Event(s) to give to the AS
 - HS-generated transaction ID
 Output:
 - None. 
 Data flows:
 ::
 Typical
 HS ---> AS : Homeserver sends events with transaction ID T.
    <---    : AS sends back 200 OK.
 AS ACK Lost
 HS ---> AS : Homeserver sends events with transaction ID T.
    <-/-    : AS 200 OK is lost.
 HS ---> AS : Homeserver retries with the same transaction ID of T.
    <---    : AS sends back 200 OK. If the AS had processed these events 
              already, it can NO-OP this request (and it knows if it is the same
              events based on the transacton ID).
 Retry notes:
 - If the HS fails to pass on the events to the AS, it must retry the request.
 - Since ASes by definition cannot alter the traffic being passed to it (unlike
   say, a Policy Server), these requests can be done in parallel to general HS
   processing; the HS doesn't need to block whilst doing this.
 - Homeservers should use exponential backoff as their retry algorithm.
 - Homeservers MUST NOT alter (e.g. add more) events they were going to
   send within that transaction ID on retries, as the AS may have already 
   processed the events.
 Ordering notes:
 - The events sent to the AS should be linearised, as they are from the event
   stream.
 - The homeserver will need to maintain a queue of transactions to send to 
   the AS.
 ::
  PUT /transactions/$transaction_id?access_token=$hs_token
  Request format
  {
    events: [
      ...
    ]
  }
 Client-Server v2 API Extensions
 -------------------------------
 Identity assertion
 ~~~~~~~~~~~~~~~~~~
 The client-server API infers the user ID from the ``access_token`` provided in 
 every request. It would be an annoying amount of book-keeping to maintain tokens
 for every virtual user. It would be preferable if the application service could
 use the CS API with its own ``as_token`` instead, and specify the virtual user
 they wish to be acting on behalf of. For real users, this would require 
 additional permissions granting the AS permission to masquerade as a matrix user.
 Inputs:
 - Application service token (``access_token``)
 Either:
   - User ID in the AS namespace to act as.
 Or:
   - OAuth2 token of real user (which may end up being an access token) 
 Notes:
 - This will apply on all aspects of the CS API, except for Account Management.
 - The ``as_token`` is inserted into ``access_token`` which is usually where the
   client token is. This is done on purpose to allow application services to 
   reuse client SDKs.
 ::
 /path?access_token=$token&user_id=$userid
 Query Parameters:
   access_token: The application service token
   user_id: The desired user ID to act as.
 /path?access_token=$token&user_token=$token
 Query Parameters:
   access_token: The application service token
   user_token: The token granted to the AS by the real user
 Timestamp massaging
 ~~~~~~~~~~~~~~~~~~~
 The application service may want to inject events at a certain time (reflecting
 the time on the network they are tracking e.g. irc, xmpp). Application services
 need to be able to adjust the ``origin_server_ts`` value to do this.
 Inputs:
 - Application service token (``as_token``)
 - Desired timestamp
 Notes:
 - This will only apply when sending events.
 ::
 /path?access_token=$token&ts=$timestamp
 Query Parameters added to the send event APIs only:
   access_token: The application service token
   ts: The desired timestamp
 Server admin style permissions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The homeserver needs to give the application service *full control* over its
 namespace, both for users and for room aliases. This means that the AS should
 be able to create/edit/delete any room alias in its namespace, as well as
 create/delete any user in its namespace. No additional API changes need to be
 made in order for control of room aliases to be granted to the AS. Creation of
 users needs API changes in order to:
 - Work around captchas.
 - Have a 'passwordless' user.
 This involves bypassing the registration flows entirely. This is achieved by
 including the AS token on a ``/register`` request, along with a login type of
 ``m.login.application_service`` to set the desired user ID without a password.
 ::
  /register?access_token=$as_token
  Content:
  {
    type: "m.login.application_service",
    user: "<desired user localpart in AS namespace>"
  }
 Application services which attempt to create users or aliases *outside* of
 their defined namespaces will receive an error code ``M_EXCLUSIVE``. Similarly,
 normal users who attempt to create users or alises *inside* an application
 service-defined namespace will receive the same ``M_EXCLUSIVE`` error code.
 ID conventions ``[Draft]``
 --------------------------
 .. NOTE::
  - Giving HSes the freedom to namespace still feels like the Right Thing here.
  - Exposing a public API provides the consistency which was the main complaint
    against namespacing.
  - This may have knock-on effects for the AS registration API. E.g. why don't
    we let ASes specify the *URI* regex they want?
 This concerns the well-defined conventions for mapping 3P network IDs to matrix
 IDs, which we expect clients to be able to do by themselves.
 User IDs
 ~~~~~~~~
 Matrix users may wish to directly contact a virtual user, e.g. to send an email.
 The URI format is a well-structured way to represent a number of different ID
 types, including:
 - MSISDNs (``tel``)
 - Email addresses (``mailto``)
 - IRC nicks (``irc`` - https://tools.ietf.org/html/draft-butcher-irc-url-04)
 - XMPP (xep-0032)
 - SIP URIs (RFC 3261)
 As a result, virtual user IDs SHOULD relate to their URI counterpart. This
 mapping from URI to user ID can be expressed in a number of ways:
 - Expose a C-S API on the HS which takes URIs and responds with user IDs.
 - Munge the URI with the user ID.
 Exposing an API would allow HSes to internally map user IDs however they like,
 at the cost of an extra round trip (of which the response can be cached).
 Munging the URI would allow clients to apply the mapping locally, but would force
 user X on service Y to always map to the same munged user ID. Considering the
 exposed API could just be applying this munging, there is more flexibility if
 an API is exposed. 
 ::
  GET /_matrix/app/v1/user?uri=$url_encoded_uri
  Returns 200 OK:
  {
    user_id: <complete user ID on local HS>
  }
 Room Aliases
 ~~~~~~~~~~~~
 We may want to expose some 3P network rooms so Matrix users can join them directly,
 e.g. IRC rooms. We don't want to expose every 3P network room though, e.g. mailto,
 tel. Rooms which are publicly accessible (e.g. IRC rooms) can be exposed as an alias by
 the application service. Private rooms (e.g. sending an email to someone) should not
 be exposed in this way, but should instead operate using normal invite/join semantics.
 Therefore, the ID conventions discussed below are only valid for public rooms which 
 expose room aliases.
 Matrix users may wish to join XMPP rooms (e.g. using XEP-0045) or IRC rooms. In both
 cases, these rooms can be expressed as URIs. For consistency, these "room" URIs 
 SHOULD be mapped in the same way as "user" URIs.
 ::
  GET /_matrix/app/v1/alias?uri=$url_encoded_uri
  Returns 200 OK:
  {
    alias: <complete room alias on local HS>
  }
 Event fields
 ~~~~~~~~~~~~
 We recommend that any gatewayed events should include an ``external_url`` field
 in their content to provide a way for Matrix clients to link into the 'native'
 client from which the event originated. For instance, this could contain the
 message-ID for emails/nntp posts, or a link to a blog comment when gatewaying
 blog comment traffic in & out of matrix
 Examples
 --------
 .. NOTE::
  - User/Alias namespaces are subject to change depending on ID conventions.
 IRC
 ~~~
 Pre-conditions:
  - Server admin stores the AS token "T_a" on the homeserver.
  - Homeserver has a token "T_h".
  - Homeserver has the domain "hsdomain.com"
 1. Application service registration
 ::
  AS -> HS: Registers itself with the homeserver
  POST /register 
  {
   url: "https://someapp.com/matrix",
   as_token: "T_a",
   namespaces: {
     users: [
       {
         "exclusive": true,
         "regex": "@irc\.freenode\.net/.*"
       }
     ],
     aliases: [
       {
         "exclusive": true,
         "regex": "#irc\.freenode\.net/.*"
       }
     ]
   }
  }
  Returns 200 OK:
  {
    hs_token: "T_h"
  }
 2. IRC user "Bob" says "hello?" on "#matrix" at timestamp 1421416883133:
 ::  
  - AS stores message as potential scrollback.
  - Nothing happens as no Matrix users are in the room.
 3. Matrix user "@alice:hsdomain.com" wants to join "#matrix":
 ::
  User -> HS: Request to join "#irc.freenode.net/#matrix:hsdomain.com"
  HS -> AS: Room Query "#irc.freenode.net/#matrix:hsdomain.com"
  GET /rooms/%23irc.freenode.net%2F%23matrix%3Ahsdomain.com?access_token=T_h
  [Starts blocking]
    AS -> HS: Creates room. Gets room ID "!aasaasasa:hsdomain.com".
    AS -> HS: Sets room name to "#matrix".
    AS -> HS: Sends message as ""@irc.freenode.net/Bob:hsdomain.com"
      PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message
                      ?access_token=T_a
                      &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com
                      &ts=1421416883133
      {
        body: "hello?"
        msgtype: "m.text"
      }
    HS -> AS: User Query "@irc.freenode.net/Bob:hsdomain.com"
      GET /users/%40irc.freenode.net%2FBob%3Ahsdomain.com?access_token=T_h
      [Starts blocking]
        AS -> HS: Creates user using CS API extension.
          POST /register?access_token=T_a
          {
            type: "m.login.application_service",
            user: "irc.freenode.net/Bob"
          }
        AS -> HS: Set user display name to "Bob".
      [Finishes blocking]
  [Finished blocking]
  - HS sends room information back to client.
 4. @alice:hsdomain.com says "hi!" in this room:
 ::
  User -> HS: Send message "hi!" in room !aasaasasa:hsdomain.com
  - HS sends message.
  - HS sees the room ID is in the AS namespace and pushes it to the AS.
  HS -> AS: Push event
  PUT /transactions/1?access_token=T_h
  {
    events: [
      {
        content: {
          body: "hi!",
          msgtype: "m.text"
        },
        origin_server_ts: <generated by hs>,
        user_id: "@alice:hsdomain.com",
        room_id: "!aasaasasa:hsdomain.com",
        type: "m.room.message"
      }
    ]
  }
  - AS passes this through to IRC.
 5. IRC user "Bob" says "what's up?" on "#matrix" at timestamp 1421418084816:
 ::
  IRC -> AS: "what's up?"
  AS -> HS: Send message via CS API extension
  PUT /rooms/%21aasaasasa%3Ahsdomain.com/send/m.room.message
                  ?access_token=T_a
                  &user_id=%40irc.freenode.net%2FBob%3Ahsdomain.com
                  &ts=1421418084816
  {
    body: "what's up?"
    msgtype: "m.text"
  }
  - HS modifies the user_id and origin_server_ts on the event and sends it.
--- a/drafts/definitions.rst
+++ b/drafts/definitions.rst
@ -1,55 +0,0 @@
 Definitions
 ===========
 (a relatively recent (Oct 2014) new list of definitions from Erik)
 # *Event* -- A JSON object that represents a piece of information to be
 distributed to the the room. The object includes a payload and metadata,
 including a ``type`` used to indicate what the payload is for and how to process
 them. It also includes one or more references to previous events.
 # *Event graph* -- Events and their references to previous events form a
 directed acyclic graph. All events must be a descendant of the first event in a
 room, except for a few special circumstances.
 # *State event* -- A state event is an event that has a non-null string valued
 `state_key` field. It may also include a ``prev_state`` key referencing exactly
 one state event with the same type and state key, in the same event graph.
 # *State tree* -- A state tree is a tree formed by a collection of state events
 that have the same type and state key (all in the same event graph.
 # *State resolution algorithm* -- An algorithm that takes a state tree as input
 and selects a single leaf node.
 # *Current state event* -- The leaf node of a given state tree that has been
 selected by the state resolution algorithm.
 # *Room state* / *state dictionary* / *current state* -- A mapping of the pair
 (event type, state key) to the current state event for that pair.
 # *Room* -- An event graph and its associated state dictionary. An event is in
 the room if it is part of the event graph.
 # *Topological ordering* -- The partial ordering that can be extracted from the
 event graph due to it being a DAG.
 (The state definitions are purposely slightly ill-defined, since if we allow
 deleting events we might end up with multiple state trees for a given event
 type and state key pair.)
 Federation specific
 -------------------
 # *(Persistent data unit) PDU* -- An encoding of an event for distribution of
 the server to server protocol.
 # *(Ephemeral data unit) EDU* -- A piece of information that is sent between
 servers and doesn't encode an event.
 Client specific
 ---------------
 # *Child events* -- Events that reference a single event in the same room
 independently of the event graph.
 # *Collapsed events* -- Events that have all child events that reference it
 included in the JSON object.
--- a/drafts/design_workflow.rst
+++ b/drafts/design_workflow.rst
@ -1,14 +0,0 @@
 Matrix Spec Design Workflow
 ===========================
 1. Write use cases
 2. Design data flows for use cases
 3. Design generic API (factoring out commonalities where possible)
 4. Design transport-specific API with justifications
 5. Formalise transport-specific API as swagger or similar
 6. Evolve the generic API design doc and transport-specific API into the actual spec.
--- a/drafts/event_schema.yaml
+++ b/drafts/event_schema.yaml
@ -1,123 +0,0 @@
 # JSON Schema for Matrix events. http://json-schema.org/
 title: "Matrix Event"
 type: object
 properties:
  auth_events:
    description: The events needed to authenticate this event
    $ref: "#/definitions/event_reference_list"
  content:
    type: object
    description: The body of the event.
  depth:
    type: integer
    description: A number one greater than that of any preceeding event.
    minimum: 0
  event_id:
    $ref: "#/definitions/event_id"
  hashes:
    $ref: "#/definitions/hashes"
  origin:
    $ref: "#/definitions/server_id"
  origin_server_ts:
    type: integer
    description: Posix timestamp on the originating server
    minimum: 0
  prev_events:
    description: The event(s) this event came after.
    $ref: "#/definitions/event_reference_list"
  prev_state:
    description: The state event(s) this event replaces.
    $ref: "#/definitions/event_reference_list"
  room_id:
    $ref: "#/definitions/room_id"
  sender:
    oneOf:
    - $ref: "#/definitions/user_id"
    - $ref: "#/definitions/server_id"
  state_key:
    type: string
  type:
    type: string
  unsigned:
    type: object
 required:
 - auth_events
 - content
 - depth
 - event_id
 - hashes
 - origin
 - origin_server_ts
 - prev_events
 - room_id
 - sender
 - type
 dependencies:
  state_key:
  - prev_state
  prev_state:
  - state_key
 definitions:
  server_id:
    type: string
    description: Identifies a server.
    pattern: "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  event_id:
    type: string
    description: Identifies an event.
    pattern: "^\\$[^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  room_id:
    type: string
    description: Identifies a room.
    pattern: "^\\![^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  user_id:
    type: string
    description: Identifies a user.
    pattern: "^\\@[^:]+:\
             (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\
             ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\
             (:[0-9]*[0-9])?$"
  base64:
    type: string
    description: Base64 string without padding.
    pattern: "^[a-zA-Z0-9/+]*$"
  hashes:
    type: object
    description: Hashes
    minProperties: 1
    additionalProperties:
      $ref: "#/definitions/base64"
  signatures:
    type: object
    description: Signatures
    additionalProperties:
      type: object
      minProperties: 1
      additionalProperties:
        $ref: "#/definitions/base64"
  event_reference:
    type: array
    minItems: 2
    maxItems: 2
    items:
    - type: string
      description: Event id of the referenced event.
      $ref: "#/definitions/event_id"
    - type: object
      description: Reference hashes of the referenced event.
      $ref: "#/definitions/hashes"
  event_reference_list:
    type: array
    items:
      $ref: "#/definitions/event_reference"