diff --git a/drafts/client-server-v2.rst b/drafts/client-server-v2.rst deleted file mode 100644 index e8036094..00000000 --- a/drafts/client-server-v2.rst +++ /dev/null @@ -1,297 +0,0 @@ -API Changes Summary -=================== -- Split up requesting tokens from delivering the data. This allows you to config many different tokens depending on the - client. In v1, you could only hit /initialSync and get everything, even if that isn't what you wanted. - -- Introduction of an 'event filtering token'. This filters the event types returned from streams/pagination. Clients may - have several of these in use at any moment, depending on the endpoint they are hitting. - -- All APIs which return events can optionally take a streaming token parameter. The server uses this to manage how events - are sent to the client (either in response to the API call or via the stream, eg but not both for duplicate info reduction). - This does mean each device would have a different streaming token. - -- New API: Scrollback API. This is designed to be used when you click on a room and want to display something. It is used - in conjunction with a max # events limit. If you have some previous data, then you supply your streaming token. - If there are > max events between that event ID and now, it returns a brand new page of events and a new - pagination token. If there are < max events, it just returns them incrementally. This supports both heavy/lightweight - clients. This can use the event filtering token to allow only 'displayable' events to be shown. - -- New API: Pagination Overview API: This is designed for cases like forum threads, which tell you how many pages there are - and allow you to jump around. This API returns an array of pagination tokens which represent each page. You tell it how - many events per page. This can use the event filtering token to allow only 'displayable' events to be shown. - -Resolves issues: ----------------- -- You can't get events for a single room only as the event stream is global. FIX: You specify via the stream token. -- You can't filter between "data" events (e.g. m.room.message) for display, and "metadata" events (e.g. power level changes). - FIX: You specify via the event filter token. -- There are race conditions when getting events via room initial sync and from the event stream. FIX: optional streaming - token param allows intelligent suppression -- You can't tell if an event you PUT is the same event when it comes down the event stream. FIX: optional streaming token - param allows intelligent suppression] -- How do you obtain partial room state / handle large room state? FIX: You specify via the event filter token. -- How do you sensibly do incremental updates? FIX: You give it a streaming token to return incremental updates. - -Outstanding Issues ------------------- -- Duplication of events in /initialSync is sub-optimal - -Issues not addressed --------------------- -These issues are more implementation specific (HTTP/JSON) and therefore haven't been addressed by this data model: - -- Naming of endpoints / keys isn't great -- Can't set power levels incrementally. -- State event PUTs are not consistent with other APIs. - -These issues are added features which have not been addressed: - -- Accessing federation level events (prev_pdus, signing keys, etc) -- How do you reject an invite? -- How do you delete state? -- Paginating on global initial sync e.g. 10 most recently active rooms. -- How do you determine the capabilities of a given HS? -- Requesting context (read: events around) around an arbitrary event (which may have been 6 months ago) - -These issues are federation-related which have not been addressed: - -- Pagination can take a while for backfill. FIX: Add flag to say server_local vs backfill_yes_please? Given the client - is best suited to say how long they are willing to wait. -- Sending events may need to be multi-stage e.g. for signing. FIX: Extra 'Action API' added. Shouldn't be too invasive. -- Handle rejection of events after the fact. e.g. HS later finds out that it shouldn't have accepted an event. - TODO: Clarifiy if 'rejection' === redaction. - -These issues relate to events themselves which have not been addressed: - -- Distinguish between *room* EDUs (e.g. typing) and PDUs -- Event timestamps (ISO8601?) - - -Meta APIs (API calls used to configure other APIs) -================================================== - -Generating an event filtering token ------------------------------------ -Args: - | Event Type Filter <[String]> (e.g. ["m.*", "org.matrix.custom.*", "my.specific.event.type"]) -Response: - | Event Filter Token -Use Cases: - | Picking out "displayable" events when paginating. - | Reducing the amount of unhandled event types being sent to the client wasting bandwidth - | Control whether presence is sent to the client (useless if they don't display it on the client!) - -Generating a streaming token ----------------------------- -Args: - | Stream Config -> - | Room ID Filter <[String]> (e.g. ["!asd:foo.bar", "!dsf:foo.bar", ...] - | User ID Filter <[String]> (e.g. ["@friend:foo.bar", "@boss:foo.bar", ...] or ["*"]) - e.g. control which user presence to get updates for -Response: - | Token -Use Cases: - | Lightweight monitor-only-this-room-please - | Heavyweight ALL THE THINGS - | Middle of the road (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases) - - -Action APIs (performs some sort of action) -========================================== - -Create Room ------------ -Args: - | Creation Config -> Join rules , Visibility - | Response Config -> Events/filters/etc - | Invitees <[String]> -Response: - | Room ID - | Response (Optional) -Use Cases: - | Create 1:1 PM room. - | Create new private group chat - | Create new public group chat with alias - | Create new "forum thread" - -Send Message ------------- -Args: - | Room ID - | Event Content - | Event Type - | State Key (Optional) -Response: - | ??? ACK ??? -Use Cases: - | Sending message to a room. - | Sending generic events to a room. - | Sending state events to a room. - | Send message in response to another message (commenting) - -Joining a room --------------- -Args: - | Invite Event ID(?sufficient?) OR Room Alias : This is how you accept an invite. - | Response Config -> Events/filters/etc -Response: - | Room ID - | Response (Optional) -Use Cases: - | Joining a room from an invite - | Joining a room from a room alias - - -Invite/Leave/Kick/Ban ---------------------- -Args: - | Room ID - | User ID - | Reason/Invitation Text (Optional) -Response: - | ? ACK ? - - -Syncing APIs -============ - -Scrollback (aka I clicked a room and now want to display something) -------------------------------------------------------------------- -Args: - | Room ID - | Max # Message Events - | Message Event Filter Token (allows just 'displayable' events) - | *Current* State Event Filter Token (get member list, etc) - | Streaming Token -Response: - | Events <[Object]> - | Incremental - True if the events are incremental from the streaming token provided. If false, there is > Max # events between NOW and the token provided. - | Pagination Token - The start token for the earliest message if not incremental. -Use Cases: - | Open a room and display messages (if no perm storage, supply no stream token to get the latest X events) - | Open a room and get incremental (supply stream token and get either incremental messages or a new fresh lot depending on amount of events) - -Syncing (aka I want live data) ------------------------------- -NB: Does NOT provide any sort of 'catchup' service. This keeps the API simpler, and prevents potential attacks where people are dumb/maliciously request from ancient streaming tokens which then return 100000s of events, slowing down the HS. Alternatively, we could expire streaming tokens after a given time (but that doesn't help if 10000s of events come down really quickly). The general idea is to block all forms of historical data behind max events limits. - -Args: - | Streaming Token - | Event Filtering Token (optional) -Response: - | ??? EVENT STREAM DATA ??? - | Updated Streaming Token -Use Cases: - | Getting events as they happen. - - -Pagination (aka The user is infinite scrolling in a room) ---------------------------------------------------------- -Getting messages: - -Args: - | Pagination Token - | Event Filter Token - | Room ID - | Max # Events -Response: - | Events [] - | New Pagination Token -Use Cases: - | Infinite scrolling - -Requesting overview of pagination: - -Args: - | Event Filter Token - | Room ID - | Max # Events per page -Response: - | Pagination Tokens<[String]> - A snapshot of all the events *at that point* with the tokens you need to feed in to get each page. E.g. to get the 1st page, use token[0] into the "Getting messages" API. -Use Cases: - | Forum threads (page X of Y) - Allows jumping around. - -Initial Sync (aka I just booted up and want to know what is going on) ---------------------------------------------------------------------- -Args: - | Message Events Event Filter Token - the filter applied to message events f.e room - | *Current* State Event Filter Token - the filter applied to state events f.e room. Can specify nothing to not get ANY state events. - | Max # events per room - can be 0 - | Streaming Token - A streaming token if you have one, to return incremental results -Response: - | Events per room [] - Up to max # events per room. NB: Still get duplicates for state/message events! - | Pagination token per room (if applicable) [] - Rooms which have > max events returns a fresh batch of events (See "Scrollback") -Use Cases: - | Populate recent activity completely from fresh. - | Populate recent activity incrementally from a token. - | Populate name/topic but NOT the name/topic changes when paginating (so m.room.name in state filter but not message filter) - - -Examples -======== - -Some examples of how these APIs could be used (emphasis on syncing since that is the main problem). - -#1 SYWEB recreation -------------------- -The aim of this is to reproduce the same data as v1, as a sanity check that this API is functional. - -- Login. POST streaming token (users["*"], rooms["*"]). Store token. -- GET /initialSync max=30 with token. Returns all rooms (because rooms["*"]) and all state, and all presence (because users["*"]), - and 30 messages. All event types returned since I didn't supply any event filter tokens. Since the streaming token - hasn't ever been used (I just made one), it returns the most recent 30 messages f.e room. This is semantically the same - as v1's /initialSync. -- GET /eventStream with streaming token. Starts blocking. -- Click on room !foo:bar. Start infinite scrolling. -- GET /paginate. Pagination token from initial sync. Get events and store new pagination token. - -#2 Less buggy SYWEB recreation ------------------------------- -The aim of this is to leverage the new APIs to fix some bugs. - -- Login. POST streaming token (users["*"], rooms["*"]). Store stream token. -- POST event filter token (["m.room.message", "m.room.topic", "m.room.name", "m.room.member"]). Store as Message Event filter token. -- POST event filter token (["m.room.name", "m.room.topic", "m.room.member"]). Store as Current State Event filter token. -- GET /initialSync max=1 with all tokens. Returns all rooms (rooms["*"]), with name/topic/members (NOT all state), with - max 1 m.room.message/topic/name/member (truly honouring max=1), with presence (users["*"]). -- GET /eventStream with stream token. Blocks. -- Click on room !foo:bar. Start infinite scrolling. -- GET /paginate with Message Event filter token. Returns only m.room.message/name/topic/member events. - -#3 Mobile client (permanent storage) ------------------------------------- -The aim of this is to use the new APIs to get incremental syncing working. - -Initially: - -- Login. POST streaming token (users["*"], rooms["*"]). Store as stream token. -- POST event filter token (["m.room.message"]). Store as Message Event filter token. -- POST event filter token (["m.*"]). Store as Current State Event filter token. -- GET /initialSync max=30 (we want a page worth of material) with all tokens. Returns all rooms (rooms["*"]), - with all m.* current state, with max 1 m.room.message, with presence (users["*"]). -- GET /eventStream with stream token. Blocks. -- Get some new events, new stream token. Quit app. - -Subsequently: - -- GET /initialSync max=30 with all tokens. Because the stream token has been used before, it tries to get the diff between - then and now, with the filter tokens specified. If it finds > 30 events for a given room, it returns a brand new page - for that room. If it finds < 30 events, it returns those events. Any new rooms are also returned. Returns a new stream token. -- GET /eventStream with new stream token. Blocks. - -#4 Lightweight client (super lazy loading, no permanent storage) ----------------------------------------------------------------- -The aim of this is to have a working app with the smallest amount of data transfer. Event filter tokens MAY be reused -if the lightweight client persists them, reducing round-trips. - -- POST streaming token (rooms["*"] only, no presence). Store as streaming token. -- POST event filter token (["m.room.message"]). Store message event filter token. -- POST event filter token (["m.room.name"]). Store as current state event filter token. -- POST event filter token (["m.room.message", "m.room.name", "m.room.member"]). Store as eventStream filter token. -- GET /initialSync max=1 with all tokens. Returns all rooms (rooms["*"]), with 1 m.room.message, no presence, and just - the current m.room.name if a room has it. -- Click on room !foo:bar. -- POST streaming token (rooms["!foo:bar"]), store as foo:bar token. -- GET /eventStream with foo:bar token AND eventStream token. This will get new messages (m.room.message) and room name - changes (m.room.name). It will also get me new room invites (m.room.member) and join/leave/kick/ban events (m.room.member), - all JUST FOR THE ROOM !foo:bar. - diff --git a/drafts/client_server_use_cases.log b/drafts/client_server_use_cases.log deleted file mode 100644 index df6b0c6f..00000000 --- a/drafts/client_server_use_cases.log +++ /dev/null @@ -1,303 +0,0 @@ -#1: Lightweight IM client (no perm storage) -#2: Mobile IM client (perm storage) -#3: MIDI client -#4: Animatrix client -#5: Unity object trees -#6: Forum -#7: Social Network ("Walls", PMs, groups) -#8: Minecraft-clone -#9: Bug Tracking Software -#10: Global 'Like' widget, which links through to a room. - -================ - -#1: Lightweight IM client (no perm storage) -------------------------------------------- -Description: - An IM client (think web client) with no way of persisting data beyond - a session (the instance a person is using the app). -Features: - Recent activity, Room screen (member list, etc), User page, just like - the web client. -Actions: - - Send a one-to-one message to someone. - - Accept an invite. - - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases) - - Populate scrollback if click on room - - Populate member list if click on room + get presence updates for them - - Populate room name / topic if click on room - - Create an empty room. - - Join a room from an alias. - - ---- - -Action: - Send a one-to-one message to someone. -How: - Enter their username and hit Message. Taken to room page with invited user. - History displays that I've invited someone / joined the room. Enter a message - and hit send. Message appears in window. - -:: - - Client Server - -- @user:domain --> - <--- room ID, ACK-- - <-historical msgs-- - -- msg,room ID ---> - <--- ACK ---------- - ---- - -Action: - Accept an invite. -How: - Get list of invites. Click one of them to 'accept' it. May or may not want - room content. - -:: - - Client Server - ---- req invites -> - <--- [inv,inv] ---- - ---- accept inv --> - <--- ACK ---------- - <--- room content-- (optional) - ---- - -Action: - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases) -How: - Request joined rooms with latest message and room name. Request rooms invited to. Possibly extra info like # joined members. - -:: - - Client Server - ---- req sync ----> - <---joined rooms--- {msg,name,alias,#members?} - <---invited rooms-- {name,alias} - ---- - -Action: - Populate scrollback if click on room. -How: - Request scrollback for room. - -:: - - Client Server - ---- room id -----> - <--- scrollback --- - ---- - -Action: - Populate member list if click on room + get presence updates for them. -How: - Click on room. Member list with names/presence/pics appears. May not want - pic. - -:: - - Client Server - ---- req mem list -> - <--- members ------- {name,pic,presence} - - monitor presence-> - ... - <- presence change-- - <- presence change-- - ... - -- stop presence ---> - ---- - -Action: - Populate room name / topic if click on room. -How: - Click on room. Room name and topic with aliases appears. May not want topic - (eg screen size). - -:: - - Client Server - ---- req room info-> - <--- room info ----- {name,topic,aliases} - ---- - -Action: - Create an empty room. -How: - Type in room config (desired name, public/private, etc). Hit Create. Room is - created. Possibly get room info. - -:: - - Client Server - ---- mkroom{config}-> - <--ACK{room_id}------ - <-- room info ------- (optional) - ---- - -Action: - Join a room from an alias. -How: - Type in alias. Hit Join. Room is joined. Possibly get room info. - -:: - - Client Server - -- join{alias} -----> - <--ACK{room_id}------ - <--room info--------- (optional) - - -=========================== - -#2: Mobile IM client (perm storage) ------------------------------------ -Description: - An IM client (think android/ios) which persists data on a database. -Features: - Recent activity, Room screen (member list, etc), User page, just like - the web client. -Actions: - - Send a one-to-one message to someone. - - Accept a stored invite. - - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases) - - Populate scrollback if click on room - - Populate member list if click on room + get presence updates for them - - Populate room name / topic if click on room - - Create an empty room. - - Join a room from an alias. - - ---- - -Action: - Send a one-to-one message to someone (single room). -How: - Enter their username and hit Message. Taken to room page with invited user if no room exists, - else takes to existing room. History displays that I've invited someone or scrollback. Enter - a message and hit send. Message appears in window. - -:: - - Client Server - -- @user:domain --> - <--- room ID, ACK-- - <-historical msgs-- (optional; not if existing room) - -- msg,room ID ---> - <--- ACK ---------- - ---- - -Action: - Accept a stored invite. -How: - Send invite to server. Get room content (or NO-OP if already joined). - -:: - - Client Server - ---- accept inv --> - <--- ACK ---------- - <--- room content-- (optional) - ---- - -Action: - Populate recent activity (all rooms joined with latest message + room names/aliases, rooms invited to + room names/aliases) - incrementally. -How: - Request recent activity diff. Get updated msg/name/#members for changed values only. - -:: - - Client Server - - req sync{token}-> - <---diff{rooms}---- {msg,name,alias,#members?} - ---- - -Action: - Populate scrollback if click on room. -How: - Request scrollback for room. Either a diff or a page of scrollback - depending on cached data. - -:: - - Client Server - -room id{latest event}-> {max msgs} - <--- scrollback -------- {fresh/incremental flag} - ---- - -Action: - Populate member list if click on room + get presence updates for them. -How: - Click on room. Member list with names/presence/pics appears. May not want - pic. - -:: - - Client Server - ---- req mem list -> - <--- members ------- {name,pic,presence} - - monitor presence-> - ... - <- presence change-- - <- presence change-- - ... - -- stop presence ---> - ---- - -Action: - Populate room name / topic if click on room. -How: - Click on room. Room name and topic with aliases appears. May not want topic - (eg screen size). Display cached info until updated. - -:: - - Client Server - ---- req room info-> - <--- room info ----- {name,topic,aliases} - ---- - -Action: - Create an empty room. -How: - Type in room config (desired name, public/private, etc). Hit Create. Room is - created. Possibly get room info. - -:: - - Client Server - ---- mkroom{config}-> - <--ACK{room_id}------ - <-- room info ------- (optional) - ---- - -Action: - Join a room from an alias. -How: - Type in alias. Hit Join. Room is joined. Possibly get room info. - -:: - - Client Server - -- join{alias} -----> - <--ACK{room_id}------ - <--room info--------- (optional) - - - diff --git a/drafts/data_flows.rst b/drafts/data_flows.rst new file mode 100644 index 00000000..4e8cd09a --- /dev/null +++ b/drafts/data_flows.rst @@ -0,0 +1,222 @@ +Data flows for use cases +======================== + +:: + + <- Data from server to client + -> Data from client to server + +Instant Messaging +----------------- + +Without storage +~~~~~~~~~~~~~~~ + +:: + + Home screen + Data required on load: + <- For each room the user is joined: Name, topic, # members, last message, room ID, aliases + Data required when new message arrives for a room: + <- Room ID, message content, sender (user ID, display name, avatar url) + Data required when someone invites you to a room: + <- Room ID, sender (user ID, display name, avatar url), Room Name, Room Topic + Data required when you leave a room on another device: + <- Room ID + Data required when you join a room on another device: + <- Name, topic, # members, last message, room ID, aliases + Data required when your profile info changes on another device: + <- new profile info e.g. avatar, display name, etc. + + Creating a room + -> Invitee list of user IDs, public/private, name of room, alias of room, topic of room + <- Room ID + + Joining a room (and dumped into chat screen on success) + -> Room ID / Room alias + <- Room ID, Room aliases (plural), Name, topic, member list (f.e. member: user ID, + avatar, presence, display name, power level, whether they are typing), enough + messages to fill screen (and whether there are more) + + Chat Screen + Data required when member name changes: + <- new name, room ID, user ID, when in the context of the room did this occur + Data required when the room name changes: + <- new name, room ID, old room name? + Invite a user: + -> user ID, room ID + <- display name / avatar of user invited (if known) + Kick a user: + -> user ID, room ID + <- what message it came after + Leave a room: + -> room ID + <- what message it came after + + Send a message + -> Message content, room ID, message sequencing (eg sending my 1st, 2nd, 3rd msg) + <- actual content sent (if server mods it), what message it comes after (to correctly + display the local echo) + + Place a call (receive a call is just reverse) + <- turn servers + -> SDP offer + -> Ice candidates (1 by 1; trickling) + <- SDP answer + <- Ice candidates + + Scrolling back (infinite scrolling) + -> Identifier for the earliest message, # requested messages + <- requested messages (f.e change in display name, what the old name was), whether + there are more. + + +With storage +~~~~~~~~~~~~ +:: + + Home Screen + On Load + -> Identifier which tells the server the client's current state (which rooms it is aware + of, which messages it has, what display names for users, etc..) + <- A delta from the client's current state to the current state on the server (e.g. the + new rooms, the *latest* message if different, the changed display names, the new + invites, etc). f.e Room: Whether the cache of the room that you have has been replaced + with this new state. + + Pre-load optimisation (not essential for this screen) + -> Number of desired messages f.e room to cache + <- f.e Room: the delta OR the entire state + + +Bug Tracking +------------ +:: + + Landing Page + On Load + <- Issues assigned to me, Issues I'm watching, Recent activity on other issues includes + comments, list of projects + + Search for an issue (assume text) + -> Search string + <- List of paginated issues + Request page 2: + -> Page number requested + <- Page of paginated issues + + Issue Page + On Load + -> Issue ID and Project ID (equiv to Room) + <- Issue contents e.g. priority, resolution state, etc. All comments e.g. user ID, + comment text, timestamp. Entire issue history e.g. changes in priority + + Post a comment + -> Issue ID, comment content, Project ID (equiv to Room) + <- actual content sent (if modded), what comment it comes after + + Set issue priority + -> Issue ID, Project ID, desired priority + <- What action in the history it came after + + Someone else sets issue priority + <- Issue ID, Project ID, new priority, where in the history + + +Mapping model use cases to matrix models (Room, Message, etc) +============================================================= + +To think about: + - Do we want to support the idea of forking off new rooms from existing ones? This + and forums could benefit from it. + +Bug tracking UI +--------------- +:: + + Projects => Rooms + Issues => Message Events + Comments => Message Events (relates_to key) + +Projects: + - Unlikely that there will be 100,000s of issues, so having to pull in all the issues for a project is okay. + - Permissions are usually per project and this Just Works. + - New issues come in automatically and Just Work. + - Can have read-only members + +Issues: + - Don't really want 1 Room per Issue, else you can have thousands of Rooms PER PROJECT, hence choice for + Issues as Messages. Don't need to join a room for each issue. + - Idea of issue owner is clear (sender of the message) + - Updating issues requires an additional event similar to comments (with ``relates_to``)? Could possibly + be state events? Don't really want all the history if say the priority was changed 1000 times, just want + the current state of the key. + +Comments: + - Additional event with ``relates_to`` key. + + +Forum +----- +:: + + Forum => Room (with pointers to Board Rooms) + Boards => Room (with pointers to Thread Rooms) + Threads => Room + Messages => Message Events + +Forum: + - Contains 10s of Boards. + - Contains special Message Events which point to different rooms f.e Board. + +Boards: + - Contains 100s of Threads. + - Contains special Message Events which point to different rooms f.e. Thread. + +Threads: + - Contains 100s of Messages. + +Can't do this nicely with the current Federation API because you have loads of +Rooms and what does posting a message look like? Creating a thread is done by..? +The user who is posting cannot create the thread because otherwise they would be +the room creator and have ultimate privileges. So it has to be created by a bot +of some kind which ties into auth (Application services?). To follow a board, +you need a bot to join the Board Room and then watch it for changes... + +Fundamental problem with forums is that there is only 1 PDU graph per room and +you either have to pull in lots of graphs separately or one graph and filter it +separately to get to the desired sub set of data. You have to subscribe into a +lot of graphs if you subscribe to a board... If you have the entire board... +good luck scrollbacking a particular thread. + + +Google+ Community +----------------- +:: + + Community => Room (with pointers to Category Rooms) + Category => Room + Post => Message Events + Comment => Message Events (relates_to key) + +Community: + - Contains 10s of categories. + - Contains special Message Events which point to different rooms f.e Category. + - Moderators of the community are mods in this room. They are in charge of making + new categories and the subsequent rooms. Can get a bit funky if a mod creates a + category room without the same permissions as the community room... but another + mod can always delete the pointer to the buggy category room and make a new one. + - Do we want to support the idea of forking off new rooms from existing ones? This + and forums could benefit from it. + +Category: + - Contains 1000s of posts. + - Same permissions as the community room. How to enforce? Fork off the community + room? + +Posts: + - Contains 10s of comments. + +This is similar to forums but you can more reasonably say "screw it, pull in the +entire community of posts." + diff --git a/drafts/design_workflow.rst b/drafts/design_workflow.rst new file mode 100644 index 00000000..32a37655 --- /dev/null +++ b/drafts/design_workflow.rst @@ -0,0 +1,14 @@ +Matrix Spec Design Workflow +=========================== + +1. Write use cases + +2. Design data flows for use cases + +3. Design generic API (factoring out commonalities where possible) + +4. Design transport-specific API with justifications + +5. Formalise transport-specific API as swagger or similar + +6. Evolve the generic API design doc and transport-specific API into the actual spec. \ No newline at end of file diff --git a/drafts/erikj_federation.rst b/drafts/erikj_federation.rst new file mode 100644 index 00000000..0644f271 --- /dev/null +++ b/drafts/erikj_federation.rst @@ -0,0 +1,408 @@ +Federation +========== +.. sectnum:: +.. contents:: Table of Contents + +Authorization +------------- + +When receiving new events from remote servers, or creating new events, a server +must know whether that event is allowed by the authorization rules. These rules +depend solely on the state at that event. The types of state events that affect +authorization are: + +- ``m.room.create`` +- ``m.room.member`` +- ``m.room.join_rules`` +- ``m.room.power_levels`` + +Servers should not create new events that reference unauthorized events. +However, any event that does reference an unauthorized event is not itself +automatically considered unauthorized. + +Unauthorized events that appear in the event graph do *not* have any effect on +the state of the graph. + +.. Note:: This is in contrast to redacted events which can still affect the + state of the graph. For example, a redacted *"join"* event will still + result in the user being considered joined. + + +Rules +~~~~~ + +The following are the rules to determine if an event is authorized (this does +include validation). + +**TODO**: What signatures do we expect? + +1. If type is ``m.room.create`` allow if and only if it has no prev events. +#. If type is ``m.room.member``: + + a. If ``membership`` is ``join``: + + i. If the previous event is an ``m.room.create``, the depth is 1 and + the ``state_key`` is the creator, then allow. + #. If the ``state_key`` does not match ``sender`` key, reject. + #. If the current state has ``membership`` set to ``join``. + #. If the ``sender`` is in the ``m.room.may_join`` list. [Not currently + implemented] + #. If the ``join_rules`` is: + + - ``public``: allow. + - ``invite``: allow if the current state has ``membership`` set to + ``invite`` + - ``knock``: **TODO**. + - ``private``: Reject. + + #. Reject + + #. If ``membership`` is ``invite`` then allow if ``sender`` is in room, + otherwise reject. + #. If ``membership`` is ``leave``: + + i. If ``sender`` matches ``state_key`` allow. + #. If ``sender``'s power level is greater than the the ``kick_level`` + given in the current ``m.room.power_levels`` state (defaults to 50), + and the ``state_key``'s power level is less than or equal to the + ``sender``'s power level, then allow. + #. Reject. + + #. If ``membership`` is ``ban``: + + i. **TODO**. + + #. Reject. + +#. Reject the event if the event type's required power level is less that the + ``sender``'s power level. +#. If the ``sender`` is not in the room, reject. +#. If the type is ``m.room.power_levels``: + + a. **TODO**. + +#. Allow. + + +Definitions +~~~~~~~~~~~ + +Required Power Level + A given event type has an associated *required power level*. This is given + by the current ``m.room.power_levels`` event, it is either listed explicitly + in the ``events`` section or given by either ``state_default`` or + ``events_default`` depending on if the event type is a state event or not. + + +Auth events +~~~~~~~~~~~ + +The auth events of an event are the set of events used by the authorization +algorithm to accept the event. These should be a subset of the current state. + +A server is required to store the complete chain of auth events for all events +it serves to remote servers. + +All auth events have type: + + - ``m.room.create`` + - ``m.room.power_levels`` + - ``m.room.member`` + +.. todo + We probably should probably give a lower band of how long auth events + should be kept around for. + +Auth chain +~~~~~~~~~~ + +The *auth chain* for an event is the recursive list of auth events and the auth +chain for those auth events. + +.. Note:: The auth chain for an event gives all the information a server needs + to accept an event. However, being given an auth chain for an event + that appears valid does not mean that the event might not later be + rejected. For example if we discover that the sender had been banned + between the join event listed in the auth events and the event being + authed. + +**TODO**: Clean the above explanations up a bit. + + +Auth chain resolution +~~~~~~~~~~~~~~~~~~~~~ + +If an auth check fails, or if we get told something we accepted should have +been rejected, we need to try and determine who is right. + +If two servers disagree about the validity of the auth events, both should +inform the other of what they think the current auth chain is. If either are +missing auth events that they know are valid (through authorization and state +resolution) they process the missing events as usual. + +If either side notice that the other has accepted an auth events we think +should be rejected (for reasons *not* in their auth chain), that server should +inform the other with suitable proof. + +The proofs can be: + +- An *event chain* that shows an auth event is *not* an ancestor of the event. + This can be done by giving the full ancestor chains up to the depth of the + invalid auth event. +- Given an event (and event chain?) showing that authorization had been revoked. + +If a server discovers it cannot prove the other side is wrong, then it accepts +that the other is correct; i.e. we always accept that the other side is correct +unless we can prove otherwise. + + + +State Resolution +---------------- + + **TODO** + +When two branches in the event graph merge, the state of those branches might +differ, so a *state resolution* algorithm must be used to determine the current +state of the resultant merge. + +The properties of the state resolution algorithm are: + +- Must only depend on the event graph, and not local server state. +- When two state events are comparable, the descendant one should be picked. +- Must not require the full event graph. + +The following algorithm satisfies these requirements; given two or more events, +pick the one with the greatest: + +#. Depth. +#. Hash of event_id. + + +This works except in the case of auth events, where we need to mitigate against +the attack where servers artificially netsplit to avoid bans or power level +changes. + +We want the following rules to apply: + +#. If power levels have been changed on two different branches use the rules + above, ensuring that the one picked is a valid change from the one not picked. +#. Similarly handle membership changes (e.g. bans, kicks, etc.) +#. Any state merged must be allowed by the newly merged auth events. If none of + the candidate events for a given state are allowed, we pick the last event + given by the ordering above (i.e. we pick one with the least depth). + + + +State Conflict Resolution +------------------------- + +If a server discovers that it disagrees with another about the current state, +it can follow the same process outlined in *Auth chain resolution* to resolve +these conflicts. + +Constructing a new event +------------------------ + + **TODO** + +When constructing a new event, the server should insert the following fields: + +- ``prev_events``: The list of event ids of what the server believes are the + current leaf nodes of the event graph (i.e., nodes that have been received + but are yet to be referenced by another event). +- ``depth``: An integer one greater than the maximum depth of the event's + previous events. +- ``auth_events``: The list of event ids that authorizes this event. This + should be a subset of the current state. +- ``origin_server_ts``: The time the server created the event. +- ``origin``: The name of the server. + + +Signing and Hashes +~~~~~~~~~~~~~~~~~~ + + **TODO** + +Validation +---------- + + **TODO** + +Domain specific string + A string of the form ``:``, where is a + single character, ```` is an arbitrary string that does not + include a colon, and `` is a valid server name. + +``room_id`` + A domain specific string with prefix ``!`` that is static across all events + in a graph and uniquely identifies it. The ``domain`` should be that of the + home server that created the room (i.e., the server that generated the + first ``m.room.create`` event). + +``sender`` + The entity that logically sent the event. This is usually a user id, but + can also be a server name. + +User Id + A domain specific string with prefix ``@`` representing a user account. The + ``domain`` is the home server of the user and is the server used to contact + the user. + +Joining a room +-------------- + +If a user requests to join a room that the server is already in (i.e. the a +user on that server has already joined the room) then the server can simply +generate a join event and send it as normal. + +If the server is not already in the room it needs to will need to join via +another server that is already in the room. This is done as a two step process. + +First, the local server requests from the remote server a skeleton of a join +event. The remote does this as the local server does not have the event graph +to use to fill out the ``prev_events`` key in the new event. Critically, the +remote server does not process the event it responded with. + +Once the local server has this event, it fills it out with any extra data and +signs it. Once ready the local server sends this event to a remote server +(which could be the same or different from the first remote server), this +remote server then processes the event and distributes to all the other +participating servers in that room. The local server is told about the +current state and complete auth chain for the join event. The local server +can then process the join event itself. + + +.. Note:: + Finding which server to use to join any particular room is not specified. + + +Inviting a user +--------------- + +To invite a remote user to a room we need their home server to sign the invite +event. This is done by sending the event to the remote server, which then signs +the event, before distributing the invite to other servers. + + +Handling incoming events +------------------------ + +When a server receives an event, it should: + +#. Check if it knows about the room. If it doesn't, then it should get the + current state and auth events to determine whether the server *should* be in + the room. If so continue, if not drop or reject the event +#. If the server already knew about the room, check the prev events to see if + it is missing any events. If it is, request them. Servers should limit how + far back they will walk the event graph for missing events. +#. If the server does not have all the prev events, then it should request the + current state and auth events from a server. + + +Failures +-------- + +A server can notify a remote server about something it thinks it has done +wrong using the failures mechanism. For example, the remote accepted an event +the local think it shouldn't have. + +A failure has a severity level depending on the action taken by the local +server. These levels are: + +``FATAL`` + The local server could not parse the event, for example due to a missing + required field. + +``ERROR`` + The local server *could* parse the event, but it was rejected. For example, + the event may have failed an authorization check. + +``WARN`` + The local server accepted the event, but something was unexpected about it. + For example, the event may have referenced another event the local server + thought should be rejected. + +A failure also includes several other fields: + +``code`` + A numeric code (to be defined later) indicating a particular type of + failure. + +``reason`` + A short string indicating what was wrong, for diagnosis purposes on the + remote server. + +``affected`` + The event id of the event this failure is responding to. For example, if + an accepted event referenced a rejected event, this would point to the + accepted one. + +``source`` + The event id of the event that was the source of this unexpected behaviour. + For example, if an accepted event referenced a rejected event, this would + point to the rejected one. + + +Appendix +======== + + **TODO** + +Example event: + +.. code:: + + { + "auth_events": [ + [ + "$14187571482fLeia:localhost:8480", + { + "sha256": "kiZUclzzPetHfy0rVoYKnYXnIv5VxH8a4996zVl8xbw" + } + ], + [ + "$14187571480odWTd:localhost:8480", + { + "sha256": "GqtndjviW9yPGaZ6EJfzuqVCRg5Lhoyo4YYv1NFP7fw" + } + ], + [ + "$14205549830rrMar:localhost:8480", + { + "sha256": "gZmL23QdWjNOmghEZU6YjqgHHrf2fxarKO2z5ZTbkig" + } + ] + ], + "content": { + "body": "Test!", + "msgtype": "m.text" + }, + "depth": 250, + "event_id": "$14207181140uTFlx:localhost:8480", + "hashes": { + "sha256": "k1nuafFdFvZXzhb5NeTE0Q2Jkqu3E8zkh3uH3mqwIxc" + }, + "origin": "localhost:8480", + "origin_server_ts": 1420718114694, + "prev_events": [ + [ + "$142071809077XNNkP:localhost:8480", + { + "sha256": "xOnU1b+4LOVz5qih0dkNFrdMgUcf35fKx9sdl/gqhjY" + } + ] + ], + "room_id": "!dwZDafgDEFTtpPKpLy:localhost:8480", + "sender": "@bob:localhost:8480", + "signatures": { + "localhost:8480": { + "ed25519:auto": "Nzd3D+emFBJJ4LCTzQEZaKO0Sa3sSTR1fGpu8OWXYn+7XUqke9Q1jYUewrEfxb3lPxlYWm/GztVUJizLz1K5Aw" + } + }, + "type": "m.room.message", + "unsigned": { + "age": 500 + } + } + diff --git a/drafts/event_schema.yaml b/drafts/event_schema.yaml new file mode 100644 index 00000000..2a6e4ed4 --- /dev/null +++ b/drafts/event_schema.yaml @@ -0,0 +1,123 @@ +# JSON Schema for Matrix events. http://json-schema.org/ +title: "Matrix Event" +type: object +properties: + auth_events: + description: The events needed to authenticate this event + $ref: "#/definitions/event_reference_list" + content: + type: object + description: The body of the event. + depth: + type: integer + description: A number one greater than that of any preceeding event. + minimum: 0 + event_id: + $ref: "#/definitions/event_id" + hashes: + $ref: "#/definitions/hashes" + origin: + $ref: "#/definitions/server_id" + origin_server_ts: + type: integer + description: Posix timestamp on the originating server + minimum: 0 + prev_events: + description: The event(s) this event came after. + $ref: "#/definitions/event_reference_list" + prev_state: + description: The state event(s) this event replaces. + $ref: "#/definitions/event_reference_list" + room_id: + $ref: "#/definitions/room_id" + sender: + oneOf: + - $ref: "#/definitions/user_id" + - $ref: "#/definitions/server_id" + state_key: + type: string + type: + type: string + unsigned: + type: object + +required: +- auth_events +- content +- depth +- event_id +- hashes +- origin +- origin_server_ts +- prev_events +- room_id +- sender +- type + +dependencies: + state_key: + - prev_state + prev_state: + - state_key + +definitions: + server_id: + type: string + description: Identifies a server. + pattern: "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\ + ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\ + (:[0-9]*[0-9])?$" + event_id: + type: string + description: Identifies an event. + pattern: "^\\$[^:]+:\ + (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\ + ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\ + (:[0-9]*[0-9])?$" + room_id: + type: string + description: Identifies a room. + pattern: "^\\![^:]+:\ + (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\ + ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\ + (:[0-9]*[0-9])?$" + user_id: + type: string + description: Identifies a user. + pattern: "^\\@[^:]+:\ + (([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*\ + ([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\ + (:[0-9]*[0-9])?$" + base64: + type: string + description: Base64 string without padding. + pattern: "^[a-zA-Z0-9/+]*$" + hashes: + type: object + description: Hashes + minProperties: 1 + additionalProperties: + $ref: "#/definitions/base64" + signatures: + type: object + description: Signatures + additionalProperties: + type: object + minProperties: 1 + additionalProperties: + $ref: "#/definitions/base64" + event_reference: + type: array + minItems: 2 + maxItems: 2 + items: + - type: string + description: Event id of the referenced event. + $ref: "#/definitions/event_id" + - type: object + description: Reference hashes of the referenced event. + $ref: "#/definitions/hashes" + event_reference_list: + type: array + items: + $ref: "#/definitions/event_reference" diff --git a/drafts/general_api.rst b/drafts/general_api.rst new file mode 100644 index 00000000..2b42b8b0 --- /dev/null +++ b/drafts/general_api.rst @@ -0,0 +1,1169 @@ +Table of Contents +================= + +.. contents:: Table of Contents +.. sectnum:: + +Summary of changes from v1 +========================== + +Version 2.0 +----------- +- Event filtering (type/room/users, federation-style events) +- Incremental initial syncing +- Deleting state +- Race conditions on event stream / actions +- Out-of-order events +- Published room API: support searching remote HSes. +- Account management: specifically the concept of devices so push works. +- Multiple devices + - Presence status unioning: is partially specced (needs more eyes). + - Syncing data between multiple devices: is specced. + - TODO: Push for offline devices. + +Lower priority +~~~~~~~~~~~~~~ +- Capabilities +- Editing/updating messages (updates key) +- Room alias API + +Version 2.1 +----------- +- Comments (relates_to key) +- Contextual messages (view messages around an arbitrary message) +- Rejecting invites +- State event pagination (e.g. from initial sync) +- Initial sync pagination (e.g. X most recent rooms) + +Out of scope +------------ +- Searching messages + +Version 2 API +============= + +Legend: + - ``[TODO]``: API is not in this document yet. + - ``[ONGOING]``: API is proposed but needs more work. There are known issues + to be addressed. + - ``[Draft]``: API is proposed and has no outstanding issues to be addressed, + but needs more feedback. + - ``[Final]``: The API has no outstanding issues. + +This contains the formal proposal for Matrix Client-Server API v2. This API +would completely replace v1. It is a general API, not specific to any particular +protocol e.g. HTTP. The following APIs will remain unchanged from v1: + +- Content repository API + +This version will change the path prefix for HTTP: + - Version 1: ``/_matrix/client/api/v1`` + - Version 2: ``/_matrix/client/v2`` + +Note the lack of the ``api`` segment. This is for consistency between other +home server path prefixes. + +Terminology: + - ``Chunk token`` : An opaque string which can be used to return another chunk + of results. For example, the initial sync API and scrollback/contextual + windowing APIs. If the total size of the data set is unknown, it should + return a chunk token to navigate it. + - ``Filter token`` : An opaque string representing the inputs originally given + to the filter API. + - ``Pagination token`` : An opaque string used for pagination requests. For + example, the published room list API. The size of the data set is known (e.g. + because a snapshot of it was taken) and you can do "Page X of Y" style + navigation. + + +Filter API ``[Draft]`` +------------------------ + +Inputs: + - Which event types (incl wildcards) + - Which room IDs + - Which user IDs (for profile/presence) + - Whether you want federation-style event JSON + - Whether you want coalesced ``updates`` events + - Whether you want coalesced ``relates_to`` events (and the max # to coalesce, + and the relationship types, and the sort order) + - limit= param? (XXX: probably not; this should be done in the query itself) + - Which keys to return for events? e.g. no ``origin_server_ts`` if you don't + show timestamps (n.b. encrypted keys can't be filtered out) +Outputs: + - An opaque token which represents the inputs, the "filter token". +Notes: + - The token may expire, in which case you would need to request another one. + - The token could be as simple as a concatenation of the requested filters with + a delimiter between them. + - Omitting the token on APIs results in ALL THE THINGS coming down. + - Clients should remember which token they need to use for which API. + - It should be possible to define negative filters (e.g. not presence) + - HTTP note: If the filter API is a separate endpoint, then you could easily + allow APIs which use filtering to ALSO specifiy query parameters to tweak the + filter. + +Global initial sync API ``[Draft]`` +------------------------------------- +.. NOTE:: + - The output to this should also have something like: + For each room the user is invited to: + - The invite event + - Other state info (e.g. room name, topic, join_rules to know if pubilc) + - # members? + + so clients know more information about the room other than the user_id of the + inviter, timestamp and the room ID. + + v2.1: + - Will need some form of state event pagination like we have for message + events to handle large amounts of state events for a room. Need to think of + the consequences of this: you may not get a ``m.room.member`` for someone's + message and so cannot display their display name / avatar. Do we want to + provide pagination on an event type basis? + - Handle paginating initial sync results themselves (e.g. 10 most recent + rooms) + +Inputs: + - A way of identifying the user (e.g. access token, user ID, etc) + - Filter to apply (e.g. a single room ID for a 'room initial sync') + - Chunk token (for incremental deltas) +Outputs: + - For each room the user is joined: + - Requested state events + - # members + - max of limit= message events + - room ID +Notes: + - If a chunk token is applied, you will get a delta relative to the last request + performed with that streaming token rather than all the rooms. +Compacting notes: + - Fixes the problem with the same event appearing in both the ``messages`` and + ``state`` keys. Represent as something like:: + + { + events: { event_id: Event, ... }, + messages: [ event_id, event_id, ...], + state: [ event_id, event_id, ...], + } +Duplicate content notes: + - For non-compacted state events, duplicate state events in the ``messages`` + key need to have a ``prev_content`` to correctly display the state change + text. This is not required for ``state`` key events, which just represent + the *current* state and as such do not need a ``prev_content``. Compacted + state events will need to specify the ``prev_content``. +What data flows does it address: + - Home screen: data required on load. + - XXX: would this also be used for initially loading room history screens too? + +Event Stream API ``[Draft]`` +---------------------------- +Inputs: + - Position in the stream (chunk token) + - Filter to apply: which event types, which room IDs, whether to get + out-of-order events, which users to get presence/profile updates for + - User ID + - Device ID +Outputs: + - 0-N events the client hasn't seen. NB: Deleted state events will be missing a + ``content`` key. Deleted message events are ``m.room.redaction`` events. + - New position in the stream. (chunk token) +State Events Ordering Notes: + - Home servers may receive state events over federation that are superceded by + state events previously sent to the client. The home server *cannot* send + these events to the client else they would end up erroneously clobbering the + superceding state event. + - As a result, the home server reserves the right to omit sending state events + which are known to be superceded already. + - This may result in missed *state* events. However, the state of the room will + always be eventually consistent. +Message Events Ordering Notes: + - Home servers may receive message events over federation that happened a long + time ago. The client may or may not be interested in these message events. + - For clients which do not store scrollback for a room (they discard events + after processing them), this is not a problem as they only care about the + recent messages. + - For clients which do persist scrollback for a room, they need to know about + the message event and where to insert it so that scrollback remains + consistent and doesn't omit messages. + - Clients can specify an input parameter stating that they wish to receive + these out-of-order events. + - The event, when it comes down the stream, will indicate which event it comes + after. +Rejected events: + - A home server may find out via federation that it should not have accepted + an event (e.g. to send a message/state event in a room). For example, it may + send an event to another home server and receive an auth event stating + that the event should not have been sent. + - If this happens, the home server will send a ``m.room.redaction`` for the + event in question. This will be a local server event (not shared with other + servers). + - If the event was a state event, it will synthesise a new state event to + correct the client's room state. This will be a local server event (not + shared with other servers). + - In practice, clients don't need any extra special handling for this. +Unknown rooms: + - You could receive events for rooms you are unaware of (e.g. you didn't do an + initial sync, or your HS lost its database and is told from another HS that + they are in this room). How do you handle this? + - The simplest option would be to redo the initial sync with a filter on the + room ID you're unaware of. This would retrieve the room state so you can + display the room. +What data flows does it address: + - Home Screen: Data required when new message arrives for a room + - Home Screen: Data required when someone invites you to a room + - Home Screen: Data required when you leave a room on another device + - Home Screen: Data required when you join a room on another device + - Home Screen: Data required when your profile info changes on another device + - Chat Screen: Data required when member name changes + - Chat Screen: Data required when the room name changes + - Chat Screen: Data required when a new message arrives + +Room Creation API ``[Draft]`` +----------------------------- +Inputs: + - Invitee list of user IDs, published/not, state events to set on creation + e.g. name of room, alias of room, topic of room +Output: + - Room ID +Notes: + - This is a special case of joining a room. See the notes on joining a room. +What data flows does it address: + - Home Screen: Creating a room + +Joining API ``[Draft]`` +----------------------- +Inputs: + - Room ID (with list of servers to join from) / room alias / invite event ID + - Optional filter (which events to return, whether the returned events should + come down the event stream) +Outputs: + - Room ID, the returned state events from the filter e.g. Room aliases + (plural), Name, topic, member list (f.e. member: user ID, avatar, presence, + display name, power level, whether they are typing), enough messages to fill + screen (and whether there are more) +Notes: + - How do you return room information? In response to the join, or from the + event stream? + - The events returned need to be filterable. Different clients for the same + user may want different information (e.g. the client performing the join may + jump to the chat screen and therefore want some messages, whereas the client + not performing the join just needs to be aware of the new room). + - As a result, the join response should return events *instead of* to the + event stream, depending on the client. +Mapping messages to the event stream: + - Once you join a room, you will start getting message events for it. How do + you know when you started getting events for this room? You need to know so + you can provide a token when scrolling back. You cannot currently infer this + from the join event itself, as individual events do not have tokens (only + chunks do). + - This token can be provided as a separate server-generated event, or an + annotation on the join event itself. + - We propose that a server-generated event is sent down the event stream to all + clients, rather than annotating the join event. The server-generated event + works nicely for Application Services where an entity subscribes to a room + without a join event. + - This will look like an event for the room, but have a special + "server-generated" event type e.g. ``m.homeserver.scrollback`` with a + ``token`` containing the start token for the room. +What data flows does it address: + - Home Screen: Joining a room + +Room History +------------ +This concerns APIs which return historical events for a room. There are several +common parameters. + +Inputs: + - Room ID + - Max number of events to return + - Filter to apply. +Outputs: + - Requested events + - Chunk token to use to request more events. + + +Scrollback API ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. NOTE:: + - Pagination: Would be nice to have "and X more". It will probably be + Google-style estimates given we can't know the exact number over federation, + but as a purely informational display thing it would be nice. + +Additional Inputs: + - flag to say if the home server should do a backfill over federation +Additional Outputs: + - whether there are more events on the local HS / over federation. +What data flows does it address: + - Chat Screen: Scrolling back (infinite scrolling) + +Contextual windowing API ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This refers to showing a "window" of message events around a given message +event. The window provides the "context" for the given message event. + +Additional Inputs: + - Event ID of the message to get the surrounding context for (this specifies + the room to get messages in). + - Whether to get results before / after / around (mid point) this event +Additional Outputs: + - Start / End chunk tokens to go either way (not just one token) + - Current room state at the end of the chunk as per initial sync. + +Room Alias API ``[Draft]`` +-------------------------- +This provides mechanisms for creating and removing room aliases for a room on a +home server. Typically, any user in a room can make an alias for that room. The +alias creator (or anyone in the room?) can delete that alias. Server admins can +also delete any alias on their server. + +Mapping a room alias to a room: + +Inputs: + - Room Alias +Output: + - Room ID + - List of home servers to join via. + +Mapping a room to an alias: + +Inputs: + - Room ID + - Desired room alias localpart + - User ID (for auth) +Output: + - Room alias +Notes: + - The home server may add restrictions e.g. the user must be in the room. + +Deleting a mapping: + +Inputs: + - Room alias + - User ID (for auth) +Output: + - None. + + +Published room list API ``[Draft]`` +----------------------------------- +This provides mechanisms for searching for published rooms on a home server. + +Inputs: + - Search text (e.g. room alias/name/topic to search on) + - Home server to search on (this may just be the URL hit for HTTP) + - Any existing pagination token, can be missing if this is the first hit. + - Limit for pagination +Output: + - Pagination token + - Total number of rooms + - Which 'page' of results this response represents + - A list of rooms with: + - # users + - A set of 'publishable' room state events, presumably ``m.room.name``, + ``m.room.topic`` and ``m.room.aliases``. This cannot be user-configured + since the user is not in the room. +Notes: + - This API would be hit again for the next page of results, with the pagination + token provided from the previous hit. + - We should probably provide "and X more" estimates for the number of + pagination results. This can be calculated by providing the total number of + rooms e.g. '100' and the page e.g. '3' coupled with the limit parameter (aka + the number of results per page) specified e.g. '10'. + - In order to prevent the dataset from changing underneath the client whilst + they paginate, a request without a pagination token should take a "snapshot" + of the underlying data which is then paginated on, rather than the database + which is a moving target as other clients add new published rooms. + + +User Profile API ``[Draft]`` +---------------------------- +Every user on a home server has a profile. This profile is effectively a +key-value store scoped to a user ID. It can include an ``avatar_url``, +``displayname`` and other metadata. Updates to a profile should propagate to +other interested users. + +Setting display name (strings): + +Inputs: + - User ID + - New display name +Output: + - None. +Notes: + - This is a generic problem, so should probably not be special cased for + display names. E.g. having an arbitrary key-value store here. + +Setting avatar url (blob data): + +Inputs: + - User ID + - New avatar url / file blob? +Output: + - None. +Notes: + - We may want to provide file uploading on this API for convenience. + +Retrieving profile information: + +Inputs: + - User ID + - Which keys to retrieve +Output: + - The key/values specified. + +Propagation ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~ +The goals of propagation are: + +- Profile updates should propagate to all rooms the user is in so + rooms can display change events. Due to this, profile propagation + HAS to be in the event graph for the room, in order to place it in + the right position. +- We should support different kinds of profiles for different rooms. +- Propagation should avoid flicker between joining a room and getting + profile information. + +In v1, this was achieved by sending ``displayname`` and ``avatar_url`` +keys inside the ``content`` of an ``m.room.member`` event. This event +type was chosen in order to prevent flicker on the client, as all the +information came down in one lump. + +This had a number of problems associated with it: + +- It conflated profile info and membership info, simply to avoid client + flicker. +- Name/avatar changes created more ``m.room.member`` events which meant + they needed to be included in the auth chains for federation. This + created long auth chains which is suboptimal since home servers need + to store the auth chains forever. + +These problems can be resolved by creating an ``m.room.member.profile`` +event which contains profile information. This reduces the number of +``m.room.member`` events over federation, since profile changes are of +a different event type. This also prevents conflation of profile changes +and membership changes. + +However, this introduces its own set of problems, namely flicker. The +client would receive the ``m.room.member`` event first, followed by +the ``m.room.member.profile`` event, which could cause a flicker. In +addition, federation may not send both events in a single transaction, +resulting in missing information on the receiving home server. + +For federation, these problems can be resolved by sending the +``m.room.member`` event as they are in v1 (with ``displayname`` and +``avatar_url`` in the ``content``). These keys need to be signed so +they cannot be in the ``unsigned`` part of the event. The receiving home +server will then extract these keys and create a server-generated +``m.room.member.profile`` event. To avoid confusion with duplicate +information, the ``avatar_url`` and ``displayname`` keys should be +removed from the ``m.room.member`` event by the receiving home server. +When a client requests these events (either from the event stream +or from an initial sync), the server will send the generated +``m.room.member.profile`` event under the ``unsigned.profile`` key of the +``m.room.member`` event. Subsequent profile updates are just sent as +``m.room.member.profile`` events. + +For clients, profile information is now *entirely* represented in +``m.room.member.profile`` events. To avoid flicker, this event is +combined with the ``m.room.member`` event under an ``unsigned.profile`` +key. + +:: + + Case #1: @user:domain "User" joins a room + + HS --> HS: + { + content: { + displayname: "User", + membership: "join" + }, + type: "m.room.member", + [...] + } + + Receiving HS transformation: + { + content: { + + membership: "join" + }, + type: "m.room.member", + [...] + } + + Receiving HS creates new server-generated event: + { + content: { + displayname: "User" + }, + type: "m.room.member.profile", + [...] + } + + Client sees: (e.g. from event stream / initial sync) + { + content: { + membership: "join" + }, + type: "m.room.member", + unsigned: { + profile: { + content: { + displayname: "User" + }, + type: "m.room.member.profile", + [...] + } + } + [...] + } + +:: + + Case #2: @user:domain "User" updates their display name to "User2" + (they are already in the room) + + HS --> HS: + { + content: { + displayname: "User2" + }, + prev_content: { + displayname: "User" + }, + type: "m.room.member.profile", + [...] + } + + Client sees: + { + content: { + displayname: "User2" + }, + prev_content: { + displayname: "User" + }, + type: "m.room.member.profile", + [...] + } + +The removal of the ``displayname`` and ``avatar_url`` keys from ``m.room.member`` +can only be done if the client trusts their HS, as it will break the sending HS's +signature. Requesting the "raw" federation event will have to return these keys. + +Account Management API ``[Draft]`` +---------------------------------- +The registration and login APIs in v2 do not support specifying device IDs. In v2, +this will become *mandatory* when sending your initial request. Access tokens will +be scoped per device, so using the same device ID twice when logging in will +clobber the old access token. + +In terms of additional APIs, users may wish to delete their account, revoke access +tokens, manage their devices, etc. This is achieved using an account management API. + +Deleting an account: + +Inputs: + - User ID to delete + - Auth key (e.g. access_token of user, of server admin, etc) +Output: + - None. + +Viewing devices related to this account: + +Inputs: + - User ID + - Auth key (e.g. access_token of user, of server admin, etc) +Output: + - A list of devices (+ last used / access tokens / creation date / device / + user-agent?) + +Removing an access token: + +Inputs: + - User ID + - Auth key (e.g. access_token of user, of server admin, etc) + - Access token to revoke. +Output: + - None. + +Removing a device: + +Inputs: + - User ID + - Auth key (e.g. access_token of user, of server admin, etc) + - Device ID to remove. +Output: + - None. +Notes: + - This revokes all access tokens linked to this device. + +Action APIs +----------- +The following APIs are "action APIs". This is defined to be a request which +alters the state of a room you are already joined to. + +When you perform an action in a room, you immediately want to display the local +echo. The client can receive the response to the action either directly or from +the event stream. The order in which you receive these responses is undefined. +As a result, clients MUST be able to handle all possible orderings:: + + 1 2a 3 + START ----> REQUEST SENT ---> RESPONSE TO REQUEST RECEIVED --------> GOT BOTH + | ^ + | 2b | + +----------> APPEARS IN EVENT STREAM -------------------+ + + 1: Can display local echo at this point. + 2a: The request has been successfully processed and can be displayed as Sent. + 2b/3: The request has been successfully processed and the client knows its + position in the event stream. + +When a client sends a request, they can include an "action ID" so that they can +match up the event in the event stream to the request which they made. This ID +is created by the client. This ID serves as a transaction ID for idempotency +as well as a marker to match the response when it appears in the event stream. +Events for actions performed by a client in that client's event stream will +include the action ID the client submitted when making the request. The +action ID will *not* appear in other client's event streams. + +Action IDs are optional and are only needed by clients that retransmit their +requests or display local echo. An example of a client which may not need the +use of action IDs includes bots which operate using basic request/responses +in a synchronous fashion. + +A client may wish to send multiple actions in parallel. The send event APIs +support sending multiple events in a batch. + +Inviting a user ``[ONGOING]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. NOTE:: + - Clients need to know *why* they are being invited (e.g. a ``reason`` key, + just like for kicks/bans). However, this opens up a spam vector where any + user can send any other user a string. Do we really want to do that? + - It may be useful to send other state information such as the room name, + topic, etc. How is this specified in this request? Does the inviter even + specify this, or is it a room config option which fields are shared? This + has a lot of parallels with the published room API which exposes some state + events. How do we prevent spam attacks via the room name/topic? + - The spam attack vector may be something we're just going to have to deal + with. Ultimately, we need to expose more data about the room. This data is + going to be set by the client. Compromises like "just give the event type" + don't really fix the problem "because.my.event.type.could.be.like.this". + +Inputs: + - User ID + - Room ID + - Action ID (optional) +Outputs: + - Display name / avatar of user invited (if known) +What data flows does it address: + - Chat Screen: Invite a user + +Rejecting an invite ``[Final]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Event ID (to know which invite you're rejecting) +Outputs: + - None. +Notes: + - Giving the event ID rather than user ID/room ID combo because mutliple users + can invite the same user into the same room. + - Rejecting an invite results in the ``m.room.member`` state event being + DELETEd for that user. + +Sending state events ``[Final]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Event type[s] + - State key[s] + - Room ID + - Content[s] +Outputs: + - None. +Notes: + - A batching version of this API needs to be provided. + +Deleting state events ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Event type + - State key + - Room ID +Outputs: + - None. +Notes: + - This is represented on the event stream as an event lacking a ``content`` + key (for symmetry with ``prev_content``) + +Patching power levels ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Room ID + - Key to update + - New value +Outputs: + - None. +Notes: + - This allows a single key on power levels to be updated e.g. specifying + ``kick`` as the key and ``60`` as the value to change the level required to + kick someone. + - The local HS will take the current ``m.room.power_levels`` event and set the + new key before sending it to other HSes *in its full form*. This means HSes + will not need to worry about partial power level events. + +Knocking on a room ``[TODO]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If a room has the right ``join_rule`` e.g. ``knock``, then it should be able +to send a special knock event to ask to join the room. + +Read-up-to markers ``[ONGOING]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. NOTE:: + - Convert to EDUs for markers with periodic PDUs to reduce event graph size? + +Inputs: + - State Event type (``m.room.marker.delivered`` and ``m.room.marker.read``) + - Event ID to mark up to. This is inclusive of the event ID specified. +Outputs: + - None. +Efficiency notes: + - Sending "read up to" markers is preferable to sending receipts for every + message due to scaling problems on the client with one receipt per message. + This results in an ever increasing amount of bandwidth being devoted to + receipts and not messages. + - For individual receipts, each person would need to send at least 1 receipt + for every message, which would give a total number of ``msgs * num_people`` + receipts per room. Assuming that people in a room generally converse at say + a rate of 1 message per unit time, this scales ``n^2`` on the number of + people in the room. + - Sending "read up to" markers in contrast allows people to skip some messages + entirely. By making them state events, each user would clobber their own + marker, keeping the scaling at ``n``. For scrollback, the event filter would + NOT want to retrieve these markers as they will be updated frequently. + - This primarily benefits clients when doing an initial sync. Event graphs + will still have a lot of events, most of them from clobbering these state + events. Some gains can be made by skipping receipts, but it is difficult to + judge whether this would be substantial. +Notes: + - What do you do if you get a marker for an event you don't have? Do you fall + back to some kind of ordering heuristic e.g. ``if origin_server_ts > + latest message``. Do you request that event ID directly from the HS? How do + you fit that in to the message thread if you did so? Would probably have to + fall back to the timestamp heuristic. After all, these markers are only ever + going to be heuristics given they are not acknowledging each message event. + - These markers also allow unread message counts to be kept in sync for multiple + devices. + +Kicking a user ``[Final]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - User ID + - Room ID + - Action ID (optional) +Outputs: + - None. +What data flows does it address: + - Chat Screen: Kick a user + +Leaving a room ``[Final]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Room ID + - A way of identifying the user (user ID, access token) + - Action ID (optional) +Outputs: + - None. +What data flows does it address: + - Chat Screen: Leave a room + +Send a message ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ +Inputs: + - Room ID + - Message contents + - Action ID (optional) + - Whether the full event should be returned, or a compact version + (default=full) +Outputs: + - The actual event sent incl content OR: + - The extra keys added or keys modified e.g. 'content' from a policy server + (if compact=true) +What data flows does it address: + - Chat Screen: Send a Message +E2E Notes: + - For signing: You send the original message to the HS and it will return the + full event JSON which will be sent. This full event is then signed and sent + to the HS again to send the message. +Compact flag notes: + - You need to know information about the event sent, including the event ID, + timestamp, etc. + - Default behaviour will return the entire event JSON sent to make client + implementations simple (just clobber it). + - It sucks to have your own messages echoed back to you in response though. + As a result, you can ask for a compact version which just sends down the + keys which were added, e.g. timestamp and event ID. +Notes: + - A batching version of this API needs to be provided. + +Presence API ``[Draft]`` +------------------------ +The goals of presence are to: + +- Let other users know if someone is "online". +- Let other users know if someone is likely to respond to any messages. +- Let other users know specific status information (e.g. "In a Meeting"). + +"Online" state can be detected by inspecting when the last time the client made +a request to the server. This could be any request, or a specific kind of request. +For connection-orientated protocols, detecting "online" state can be determined by +the state of this connection stream. For HTTP, this can be detected via requests +to the event stream. + +Online state is separate from letting other users know if someone is *likely to +respond* to messages. This introduces the concept of an "idle" flag, which is +set when the user has not done any "interaction" with the app. The definition of +"interaction" varies based on the app, so it is up to the app to set this "idle" +flag. + +Letting users know specific status information can be achieved via the same method +as v1. Status information should be scoped per *user* and not device as determining +a union algorithm between statuses is nonsensical. Passing status information per +device to all other users just redirects the union problem to the client, which +will commonly be presenting this information as an icon alongside the user. + +When a client hits the event stream, the home server can treat the user as +"online". This behaviour should be able to be overridden to avoid flicker +during connection losses when the client is appear offline (e.g. device is +appear offline > goes into a tunnel > server times out > device regains +connection and hits the event stream forcing the device online before the +"appear offline" state can be set). When the client has not hit the event +stream for a certain period of time, the home server can treat the user as +"offline". + +The user should also be able to set their presence via a direct API, without +having to hit the event stream. The home server will set a timer when the +connection ends, after which it will set that device to offline. + +As the idle flag and online state is determined per device, there needs to be a +union algorithm to merge these into a single state and flag per user, which will +be sent to other users. The algorithm is: + +- If any device is online and not idle, the user is online. +- Else if all online devices are idle, the user is idle. +- Else the user is offline (no online devices). + +Changing presence status: + +Inputs: + - User ID + - Presence status (online, away, busy, do not disturb, etc) +Outputs: + - None. + +Setting the idle flag: + +Inputs: + - User ID + - Is idle +Outputs: + - None. + +Extra parameters associated with the event stream: + +Inputs: + - Presence state (online, appear offline) + + +Typing API ``[Final]`` +------------------------ +.. NOTE:: + - Linking the termination of typing events to the message itself, so you don't + need to send two events and don't get flicker? + +The typing API remains unchanged from v1. + +Relates-to pagination API ``[Draft]`` +------------------------------------- +See the "Relates to" section for more info. + +Inputs: + - Event ID + - Chunk token + - limit +Output: + - A chunk of child events + - A new chunk token for earlier child events. + +Capabilities API ``[ONGOING]`` +------------------------------ +.. NOTE:: + - Server capabilities: Keep hashing step for consistency or not? Extra request + if we do. + - Client capabilities: Clients which are offline but can be pushed should have + their capabilities visible. How to manage unregistering them e.g. if they + uninstall the app? Have a set of 'offline' capabilities? + + +How does a client know if the server it is using supports a content repository? +How does a client know if another client has VoIP support? This section outlines +capability publishing for servers, clients and federation. + +Server +~~~~~~ +- List of extensions it supports (e.g. content repo, contact repo, turn servers) + +Inputs: + - User ID (e.g. only @bob can use the content repo) +Output: + - Hash of the capabilities:: + + { + "sha256": "fD876SFrt3sugh23FWEjio3" + } + +This hash is fed into another API: + +Inputs: + - The hash of the capabilities +Output: + - A list of capabilities:: + + { + "custom.feature.v1": {}, + "m.cap.turnserver.v1": {} + } + +Client +~~~~~~ +- e.g. Whether this client supports VoIP + +When a client app is launched, the client needs to provide a capability set. The +server will take the hashes of all the user's connected clients' capability +sets and send the list of hashes as part of presence information +(not necesarily as a ``m.presence`` event, but it should act like presence +events). It is sent as a list instead of a union of hashes because hashes work +best when they don't change. A union of many devices' hashes will change +frequently when devices come on and offline (``max hashes = 2^num_devices``). +In contrast, the size of the list would vary, but the hashes themselves +would remain the same for a given device (``max hashes = num_devices``). Keeping +the hashes the same is the best as that means clients do not need to request +the capabilities for the given hash. + +On first signup, the client will attempt to send the hash and be most likely +refused by the home server as it does not know the full capability set for that +hash. The client will then have to upload the full capability set to the home +server. The client will then be able to send the hash as normal. + +When a client receives a hash, the client will either recognise the hash or +will have to request the capability set from their home server: + +Inputs: + - Hash + - User ID +Output: + - A list of capabilities + +Federation +~~~~~~~~~~ +- e.g. Whether you support backfill, hypothetical search/query/threading APIs +- Same as the server capability API + +VoIP ``[TODO]`` +--------------- +This addresses one-to-one calling with multiple devices. This uses the +``updates`` key to handle signalling. + +Event updates +~~~~~~~~~~~~~ +- Call is placed by caller. Event generated with offer. +- 1-N callees may pick up or reject this offer. +- Callees update the event (with sdp answer if they are accepting the call) +- Caller acknowledges *one* of the callees (either one which picked up or + rejected) by updating the event. +- Callees who weren't chosen then give up (Answered elsewhere, Rejected + elsewhere, etc) +- Update with ICE candidates as they appear. +- ... in call ... +- Send hangup update when hanging up. + +Placing a call +~~~~~~~~~~~~~~ +:: + + caller callee + |-----m.call.invite--->| + | | + |<----m.call.answer----| + | device_id=foo | + | | + |------m.call.ack----->| + | device_id=foo | + | | + |<--m.call.candidate---| + |---m.call.candidate-->| + | | + [...] [...] + | | + |<----m.call.hangup----| + | device_id=foo | + +Expiry +~~~~~~ +- WIP: Of invites +- WIP: Of calls themselves (as they may never send a ``m.call.hangup`` + + +General client changes +---------------------- +These are changes which do not introduce new APIs, but are required for the new +APIs in order to fix certain issues. + +Updates (Events) ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Events may update other events. This is represented by the ``updates`` key. This +is a key which contains the event ID for the event it relates to. Events that +relate to other events are referred to as "Child Events". The event being +related to is referred to as "Parent Events". Child events cannot stand alone as +a separate entity; they require the parent event in order to make sense. + +Bundling updates +++++++++++++++++ +Events that relate to another event should come down inside that event. That is, +the top-level event should come down with all the child events at the same time. +This is called a "bundle" and it is represented as an array of events inside the +top-level event.There are some issues with this however: + +- Scrollback: Should you be told about child events for which you do not know + the parent event? Conclusion: No you shouldn't be told about child events. + You will receive them when you scroll back to the parent event. +- Pagination of child events: You don't necessarily want to have 1000000s of + child events with the parent event. We can't reasonably paginate child events + because we require all the child events in order to display the event + correctly. Comments on a message should be done via another technique, + such as ``relates_to``. +- Do you allow child events to relate to other child events? There is no + technical reason why we cannot nest child events, however we can't think of + any use cases for it. The behaviour would be to get the child events + recursively from the top-level event. + +Main use cases for ``updates``: + - Call signalling (child events are ICE candidates, answer to the offer, and + termination) + - *Local* Delivery/Read receipts : "Local" means they are not shared with other + users on the same home server or via federation but *are* shared between + clients for the same user; useful for push notifications, read count markers, + etc. This is done to avoid the ``n^2`` problem for sending receipts, where + the vast majority of traffic tends towards sending more receipts. + - s/foo/bar/ style message edits + +Clients *always* need to know how to apply the deltas because clients may +receive the events separately down the event stream. Combining event updates +server-side does not make client implementation simpler, as the client still +needs to know how to combine the events. + +Relates to (Events) ``[ONGOING]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. NOTE:: + - Should be able to specify more relationship info other than just the event + type. Forcing that m.room.message A "relates_to" another m.room.message B + means that A is a threaded conversation reply to B is needlessly + restrictive. What if A and B relate to each other by some other metric + (e.g. they are part of a group rather than a thread? or we distinguish + mail-style threading from multithreaded-IM threading for the same set of + messages? etc)? E.g. ``"relates_to" : [["in_reply_to", "$event_id1"], + ["another_type_of_relation", "$event_id2"]]`` + +Events may be in response to other events, e.g. comments. This is represented +by the ``relates_to`` key. This differs from the ``updates`` key as they *do +not update the event itself*, and are *not required* in order to display the +parent event. Crucially, the child events can be paginated, whereas ``updates`` +child events cannot be paginated. + +Bundling relations +++++++++++++++++++ +Child events can be optionally bundled with the parent event, depending on your +display mechanism. The number of child events which can be bundled should be +limited to prevent events becoming too large. This limit should be set by the +client. If the limit is exceeded, then the bundle should also include a +chunk token so that the client can request more child events. + +Main use cases for ``relates_to``: + - Comments on a message. + - Non-local delivery/read receipts : If doing separate receipt events for each + message. + - Meeting invite responses : Yes/No/Maybe for a meeting. + +Like with ``updates``, clients need to know how to apply the deltas because +clients may receive the events separately down the event stream. + +TODO: + - Can a child event reply to multiple parent events? Use case? + - Should a parent event and its children share a thread ID? Does the + originating HS set this ID? Is this thread ID exposed through federation? + e.g. can a HS retrieve all events for a given thread ID from another HS? + + +Example using 'updates' and 'relates_to' +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +- Room with a single message. +- 10 comments are added to the message via ``relates_to``. +- An edit is made to the original message via ``updates``. +- An initial sync on this room with a limit of 3 comments, would return the + message with the update event bundled with it and the most recent 3 comments + and a chunk token to request earlier comments + + .. code :: javascript + + { + content: { body: "I am teh winner!" }, + updated_by: [ + { content: { body: "I am the winner!" }, ... } + ], + replies: { + start: "some_token", + chunk: [ + { content: { body: "8th comment" }, ... }, + { content: { body: "9th comment" }, ... }, + { content: { body: "10th comment" }, ... } + ] + }, + ... + } + +Events (breaking changes; event version 2) ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Prefix the event ``type`` to say if it is a state event, message event or + ephemeral event. Needed because you can't tell the different between message + events and ephemeral ROOM events (e.g. typing). +- State keys need additional restrictions in order to increase flexibility on + state event permissions. State keys prefixed with an ``_`` have no specific + restrictions. 0-length state keys are now represented by just a single ``_``. + State keys prefixed with ``@`` can be modified only by the named user ID *OR* + the room ops. They can have an optional path suffixed to it. State keys that + start with a server name can only be modified by that server name (e.g. + ``some.server.com/some/path`` can only be modified by ``some.server.com``). +- Do we want to specify what restrictions apply to the state key in the event + type? This would allow HSes to enforce this, making life easier for clients + when dealing with custom event types. E.g. ``_custom.event`` would allow + anything in the state key, ``_@custom.event`` would only allow user IDs in + the state key, etc. +- s/user_id/sender/g given that home servers can send events, not just users. + +Server-generated events ``[Draft]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Home servers may want to send events to their local clients or to other home +servers e.g. for server status notifications. + +These events look like regular events but have a server domain name as the +``sender`` and not a user ID. This can be easily detected by clients by the +absence of a starting ``@``. + +Different types of events (e.g. EDUs, room EDUs) are detected in the same way +as normal events (as proposed in the ``Events`` section of this document). + + + diff --git a/drafts/model/profiles.rst b/drafts/model/profiles.rst index f7d6bd56..15f65feb 100644 --- a/drafts/model/profiles.rst +++ b/drafts/model/profiles.rst @@ -198,12 +198,11 @@ Requests that a client can make to its Home Server * get another user's Display Name / Avatar / metadata fields -[[TODO(paul): At some later stage we should consider the API for: +TODO(paul): At some later stage we should consider the API for: * get/set ACL permissions on my metadata fields * manage my ACL tokens -]] Server-Server ------------- diff --git a/drafts/model/protocol_examples.rst b/drafts/model/protocol_examples.rst index 61a599b4..f1d31f34 100644 --- a/drafts/model/protocol_examples.rst +++ b/drafts/model/protocol_examples.rst @@ -3,7 +3,9 @@ Host: ... Content-Length: ... Content-Type: application/json -{ +.. code :: javascript + + { "origin": "localhost:5000", "pdus": [ { @@ -27,7 +29,7 @@ Content-Type: application/json "1404381396852" ], "ts": 1404381427823 -} + } HTTP/1.1 200 OK ... @@ -42,7 +44,9 @@ HTTP/1.1 200 OK Content-Length: ... Content-Type: application/json -{ +.. code :: javascript + + { origin: ..., prev_ids: ..., data: [ @@ -59,6 +63,6 @@ Content-Type: application/json }, ..., ] -} + } diff --git a/drafts/model/rooms.rst b/drafts/model/rooms.rst index 0007e48e..07c5e71f 100644 --- a/drafts/model/rooms.rst +++ b/drafts/model/rooms.rst @@ -125,13 +125,14 @@ m.invite_level m.join_rules Encodes the rules on how non-members can join the room. Has the following possibilities: - "public" - a non-member can join the room directly - "knock" - a non-member cannot join the room, but can post a single "knock" - message requesting access, which existing members may approve or deny - "invite" - non-members cannot join the room without an invite from an - existing member - "private" - nobody who is not in the 'may_join' list or already a member - may join by any mechanism + + - "public" - a non-member can join the room directly + - "knock" - a non-member cannot join the room, but can post a single "knock" + message requesting access, which existing members may approve or deny + - "invite" - non-members cannot join the room without an invite from an + existing member + - "private" - nobody who is not in the 'may_join' list or already a member + may join by any mechanism In any of the first three modes, existing members with sufficient permission can send invites to non-members if allowed by the "m.invite_level" key. A @@ -263,10 +264,11 @@ resolve this: that duplicate requests can be suppressed. On receipt of a room creation request that the HS thinks there already exists a room for, the invitation to join can be rejected if: - a) the HS believes the sending user is already a member of the room (and - maybe their HS has forgotten this fact), or - b) the proposed room has a lexicographically-higher ID than the existing - room (to resolve true race condition conflicts) + + - a) the HS believes the sending user is already a member of the room (and + maybe their HS has forgotten this fact), or + - b) the proposed room has a lexicographically-higher ID than the existing + room (to resolve true race condition conflicts) * The room ID for a private 1:1 chat has a special form, determined by concatenting the User IDs of both members in a deterministic order, such that diff --git a/drafts/object_model.rst b/drafts/object_model.rst new file mode 100644 index 00000000..1ae60b2c --- /dev/null +++ b/drafts/object_model.rst @@ -0,0 +1,28 @@ + + + + +:: + + +---------------+ + | Room | + | "Room-ID" | + | {State} | +----------------------+ + | Name------|-------->| Event m.room.name | + | Topic | | "Name" | + | [Aliases] | +----------------------+ +-------------+ + | [Members]-|---+ +----------------------+ <----| Start Token | + | [Messages] | | | Event m.room.member | +-------------+ + | | | | +---->| "invite/join/ban" | + +---------------+ | "User-ID" | + | | +----------------------+ + | | +----------------------+ + | | Message | Event m.room.message | + | +-------------->| {content} |<--+ + | +----------------------+ | + | Comment +----------------------+ | + +------------------>| Event m.room.message | | + | {content} | | + | "relates-to"-------|---+ +-------------+ + +----------------------+ <----| End Token | + +-------------+ diff --git a/drafts/old-client-server-specification.rst b/drafts/old-client-server-specification.rst deleted file mode 100644 index e06bd676..00000000 --- a/drafts/old-client-server-specification.rst +++ /dev/null @@ -1,1281 +0,0 @@ -======================== -Matrix Client-Server API -======================== - - - -.. WARNING:: - This specification is quite old and obsolete, but not all of the bits which - are still relevant have been merged into matrix-doc/specification. - As such this is still quite a useful trove of info. - This needs to happen asap. - - - - - -The following specification outlines how a client can send and receive data from -a home server. - -[[TODO(kegan): 4/7/14 Grilling -- Mechanism for getting historical state changes (e.g. topic updates) - add - query param flag? -- Generic mechanism for linking first class events (e.g. feedback) with other s - first class events (e.g. messages)? -- Generic mechanism for updating 'stuff about the room' (e.g. favourite coffee) - AND specifying clobbering rules (clobber/add to list/etc)? -- How to ensure a consistent view for clients paginating through room lists? - They aren't really ordered in any way, and if you're paginating - through them, how can you show them a consistent result set? Temporary 'room - list versions' akin to event version? How does that work? -]] - -[[TODO(kegan): -Outstanding problems / missing spec: -- Push -- Typing notifications -]] - -Terminology ------------ -Stream Tokens: -An opaque token used to make further streaming requests. When using any -pagination streaming API, responses will contain a start and end stream token. -When reconnecting to the stream, these tokens can be used to tell the server -where the client got up to in the stream. - -Event ID: -Every event that comes down the event stream or that is returned from the REST -API has an associated event ID (event_id). This ID will be the same between the -REST API and the event stream, so any duplicate events can be clobbered -correctly without knowing anything else about the event. - -Message ID: -The ID of a message sent by a client in a room. Clients send IMs to each other -in rooms. Each IM sent by a client must have a unique message ID which is unique -for that particular client. - -User ID: -The @username:host style ID of the client. When registering for an account, the -client specifies their username. The user_id is this username along with the -home server's unique hostname. When federating between home servers, the user_id -is used to uniquely identify users across multiple home servers. - -Room ID: -The room_id@host style ID for the room. When rooms are created, the client either -specifies or is allocated a room ID. This room ID must be used to send messages -in that room. Like with clients, there may be multiple rooms with the same ID -across multiple home servers. The room_id is used to uniquely identify a room -when federating. - -Global message ID: -The globally unique ID for a message. This ID is formed from the msg_id, the -client's user_id and the room_id. This uniquely identifies any -message. It is represented with '-' as the delimeter between IDs. The -global_msg_id is of the form: room_id-user_id-msg_id - - -REST API and the Event Stream ------------------------------ -Clients send data to the server via a RESTful API. They can receive data via -this API or from an event stream. An event stream is a special path which -streams all events the client may be interested in. This makes it easy to -immediately receive updates from the REST API. All data is represented as JSON. - -Pagination streaming API -======================== -Clients are often interested in very large datasets. The data itself could -be 1000s of messages in a given room, 1000s of rooms in a public room list, or -1000s of events (presence, typing, messages, etc) in the system. It is not -practical to send vast quantities of data to the client every time they -request a list of public rooms for example. There needs to be a way to show a -subset of this data, and apply various filters to it. This is what the pagination -streaming API is. This API defines standard request/response parameters which -can be used when navigating this stream of data. - -Pagination Request Query Parameters ------------------------------------ -Clients may wish to paginate results from the event stream, or other sources of -information where the amount of information may be a problem, -e.g. in a room with 10,000s messages. The pagination query parameters provide a -way to navigate a 'window' around a large set of data. These -parameters are only valid for GET requests. - - S e r v e r - s i d e d a t a - |-------------------------------------------------| -START ^ ^ END - |_______________| - | - Client-extraction - -'START' and 'END' are magic token values which specify the start and end of the -dataset respectively. - -Query parameters: - from : $streamtoken - The opaque token to start streaming from. - to : $streamtoken - The opaque token to end streaming at. Typically, - clients will not know the item of data to end at, so this will usually be - START or END. - limit : integer - An integer representing the maximum number of items to - return. - -For example, the event stream has events E1 -> E15. The client wants the last 5 -events and doesn't know any previous events: - -S E -|-E1-E2-E3-E4-E5-E6-E7-E8-E9-E10-E11-E12-E13-E14-E15-| -| | | -| _____| | -|__________________ | ___________________| - | | | - GET /events?to=START&limit=5&from=END - Returns: - E15,E14,E13,E12,E11 - - -Another example: a public room list has rooms R1 -> R17. The client is showing 5 -rooms at a time on screen, and is on page 2. They want to -now show page 3 (rooms R11 -> 15): - -S E -| 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | stream token -|-R1-R2-R3-R4-R5-R6-R7-R8-R9-R10-R11-R12-R13-R14-R15-R16-R17| room - |____________| |________________| - | | - Currently | - viewing | - | - GET /rooms/list?from=9&to=END&limit=5 - Returns: R11,R12,R13,R14,R15 - -Note that tokens are treated in an *exclusive*, not inclusive, manner. The end -token from the intial request was '9' which corresponded to R10. When the 2nd -request was made, R10 did not appear again, even though from=9 was specified. If -you know the token, you already have the data. - -Pagination Response -------------------- -Responses to pagination requests MUST follow the format: -{ - "chunk": [ ... , Responses , ... ], - "start" : $streamtoken, - "end" : $streamtoken -} -Where $streamtoken is an opaque token which can be used in another query to -get the next set of results. The "start" and "end" keys can only be omitted if -the complete dataset is provided in "chunk". - -If the client wants earlier results, they should use from=$start_streamtoken, -to=START. Likewise, if the client wants later results, they should use -from=$end_streamtoken, to=END. - -Unless specified, the default pagination parameters are from=START, to=END, -without a limit set. This allows you to hit an API like -/events without any query parameters to get everything. - -The Event Stream ----------------- -The event stream returns events using the pagination streaming API. When the -client disconnects for a while and wants to reconnect to the event stream, they -should specify from=$end_streamtoken. This lets the server know where in the -event stream the client is. These tokens are completely opaque, and the client -cannot infer anything from them. - - GET /events?from=$LAST_STREAM_TOKEN - REST Path: /events - Returns (success): A JSON array of Event Data. - Returns (failure): An Error Response - -LAST_STREAM_TOKEN is the last stream token obtained from the event stream. If the -client is connecting for the first time and does not know any stream tokens, -they can use "START" to request all events from the start. For more information -on this, see "Pagination Request Query Parameters". - -The event stream supports shortpoll and longpoll with the "timeout" query -parameter. This parameter specifies the number of milliseconds the server should -hold onto the connection waiting for incoming events. If no events occur in this -period, the connection will be closed and an empty chunk will be returned. To -use shortpoll, specify "timeout=0". - -Event Data ----------- -This is a JSON object which looks like: -{ - "event_id" : $EVENT_ID, - "type" : $EVENT_TYPE, - $URL_ARGS, - "content" : { - $EVENT_CONTENT - } -} - -EVENT_ID - An ID identifying this event. This is so duplicate events can be suppressed on - the client. - -EVENT_TYPE - The namespaced event type (m.*) - -URL_ARGS - Path specific data from the REST API. - -EVENT_CONTENT - The event content, matching the REST content PUT previously. - -Events are differentiated via the event type "type" key. This is the type of -event being received. This can be expanded upon by using different namespaces. -Every event MUST have a 'type' key. - -Most events will have a corresponding REST URL. This URL will generally have -data in it to represent the resource being modified, -e.g. /rooms/$room_id. The event data will contain extra top-level keys to expose -this information to clients listening on an event -stream. The event content maps directly to the contents submitted via the REST -API. - -For example: - Event Type: m.example.room.members - REST Path: /examples/room/$room_id/members/$user_id - REST Content: { "membership" : "invited" } - -is represented in the event stream as: - -{ - "event_id" : "e_some_event_id", - "type" : "m.example.room.members", - "room_id" : $room_id, - "user_id" : $user_id, - "content" : { - "membership" : "invited" - } -} - -As convention, the URL variable "$varname" will map directly onto the name -of the JSON key "varname". - -Error Responses ---------------- -If the client sends an invalid request, the server MAY respond with an error -response. This is of the form: -{ - "error" : "string", - "errcode" : "string" -} -The 'error' string will be a human-readable error message, usually a sentence -explaining what went wrong. - -The 'errcode' string will be a unique string which can be used to handle an -error message e.g. "M_FORBIDDEN". These error codes should have their namespace -first in ALL CAPS, followed by a single _. For example, if there was a custom -namespace com.mydomain.here, and a "FORBIDDEN" code, the error code should look -like "COM.MYDOMAIN.HERE_FORBIDDEN". There may be additional keys depending on -the error, but the keys 'error' and 'errcode' will always be present. - -Some standard error codes are below: - -M_FORBIDDEN: -Forbidden access, e.g. joining a room without permission, failed login. - -M_UNKNOWN_TOKEN: -The access token specified was not recognised. - -M_BAD_JSON: -Request contained valid JSON, but it was malformed in some way, e.g. missing -required keys, invalid values for keys. - -M_NOT_JSON: -Request did not contain valid JSON. - -M_NOT_FOUND: -No resource was found for this request. - -Some requests have unique error codes: - -M_USER_IN_USE: -Encountered when trying to register a user ID which has been taken. - -M_ROOM_IN_USE: -Encountered when trying to create a room which has been taken. - -M_BAD_PAGINATION: -Encountered when specifying bad pagination values to a Pagination Streaming API. - - -======== -REST API -======== - -All content must be application/json. Some keys are required, while others are -optional. Unless otherwise specified, -all HTTP PUT/POST/DELETEs will return a 200 OK with an empty response body on -success, and a 4xx/5xx with an optional Error Response on failure. When sending -data, if there are no keys to send, an empty JSON object should be sent. - -All POST/PUT/GET/DELETE requests MUST have an 'access_token' query parameter to -allow the server to authenticate the client. All -POST requests MUST be submitted as application/json. - -All paths MUST be namespaced by the version of the API being used. This should -be: - -/_matrix/client/api/v1 - -All REST paths in this section MUST be prefixed with this. E.g. - REST Path: /rooms/$room_id - Absolute Path: /_matrix/client/api/v1/rooms/$room_id - -Registration -============ -Clients must register with the server in order to use the service. After -registering, the client will be given an -access token which must be used in ALL requests as a query parameter -'access_token'. - -Registering for an account --------------------------- - POST /register - With: A JSON object containing the key "user_id" which contains the desired - user_id, or an empty JSON object to have the server allocate a user_id - automatically. - Returns (success): 200 OK with a JSON object: - { - "user_id" : "string [user_id]", - "access_token" : "string" - } - Returns (failure): An Error Response. M_USER_IN_USE if the user ID is taken. - - -Unregistering an account ------------------------- - POST /unregister - With query parameters: access_token=$ACCESS_TOKEN - Returns (success): 200 OK - Returns (failure): An Error Response. - - -Logging in to an existing account -================================= -If the client has already registered, they need to be able to login to their -account. The home server may provide many different ways of logging in, such -as user/password auth, login via a social network (OAuth), login by confirming -a token sent to their email address, etc. This section does NOT define how home -servers should authorise their users who want to login to their existing -accounts. This section defines the standard interface which implementations -should follow so that ANY client can login to ANY home server. - -The login process breaks down into the following: - 1: Get login process info. - 2: Submit the login stage credentials. - 3: Get access token or be told the next stage in the login process and repeat - step 2. - -Getting login process info: - GET /login - Returns (success): 200 OK with LoginInfo. - Returns (failure): An Error Response. - -Submitting the login stage credentials: - POST /login - With: LoginSubmission - Returns (success): 200 OK with LoginResult - Returns (failure): An Error Response - -Where LoginInfo is a JSON object which MUST have a "type" key which denotes the -login type. If there are multiple login stages, this object MUST also contain a -"stages" key, which has a JSON array of login types denoting all the steps in -order to login, including the first stage which is in "type". This allows the -client to make an informed decision as to whether or not they can natively -handle the entire login process, or whether they should fallback (see below). - -Where LoginSubmission is a JSON object which MUST have a "type" key. - -Where LoginResult is a JSON object which MUST have either a "next" key OR an -"access_token" key, depending if the login process is over or not. This object -MUST have a "session" key if multiple POSTs need to be sent to /login. - -Fallback --------- -If the client does NOT know how to handle the given type, they should: - GET /login/fallback -This MUST return an HTML page which can perform the entire login process. - -Password-based --------------- -Type: "m.login.password" -LoginSubmission: -{ - "type": "m.login.password", - "user": , - "password": -} - -Example: -Assume you are @bob:matrix.org and you wish to login on another mobile device. -First, you GET /login which returns: -{ - "type": "m.login.password" -} -Your client knows how to handle this, so your client prompts the user to enter -their username and password. This is then submitted: -{ - "type": "m.login.password", - "user": "@bob:matrix.org", - "password": "monkey" -} -The server checks this, finds it is valid, and returns: -{ - "access_token": "abcdef0123456789" -} -The server may optionally return "user_id" to confirm or change the user's ID. -This is particularly useful if the home server wishes to support localpart entry -of usernames (e.g. "bob" rather than "@bob:matrix.org"). - -OAuth2-based ------------- -Type: "m.login.oauth2" -This is a multi-stage login. - -LoginSubmission: -{ - "type": "m.login.oauth2", - "user": -} -Returns: -{ - "uri": -} - -The home server acts as a 'confidential' Client for the purposes of OAuth2. - -If the uri is a "sevice selection uri", it is a simple page which prompts the -user to choose which service to authorize with. On selection of a service, they -link through to Authorization Request URIs. If there is only 1 service which the -home server accepts when logging in, this indirection can be skipped and the -"uri" key can be the Authorization Request URI. - -The client visits the Authorization Request URI, which then shows the OAuth2 -Allow/Deny prompt. Hitting 'Allow' returns the redirect URI with the auth code. -Home servers can choose any path for the redirect URI. The client should visit -the redirect URI, which will then finish the OAuth2 login process, granting the -home server an access token for the chosen service. When the home server gets -this access token, it knows that the cilent has authed with the 3rd party, and -so can return a LoginResult. - -The OAuth redirect URI (with auth code) MUST return a LoginResult. - -Example: -Assume you are @bob:matrix.org and you wish to login on another mobile device. -First, you GET /login which returns: -{ - "type": "m.login.oauth2" -} -Your client knows how to handle this, so your client prompts the user to enter -their username. This is then submitted: -{ - "type": "m.login.oauth2", - "user": "@bob:matrix.org" -} -The server only accepts auth from Google, so returns the Authorization Request -URI for Google: -{ - "uri": "https://accounts.google.com/o/oauth2/auth?response_type=code& - client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos" -} -The client then visits this URI and authorizes the home server. The client then -visits the REDIRECT_URI with the auth code= query parameter which returns: -{ - "access_token": "0123456789abcdef" -} - -Email-based (code) ------------------- -Type: "m.login.email.code" -This is a multi-stage login. - -First LoginSubmission: -{ - "type": "m.login.email.code", - "user": - "email": -} -Returns: -{ - "session": -} - -The email contains a code which must be sent in the next LoginSubmission: -{ - "type": "m.login.email.code", - "session": , - "code": -} -Returns: -{ - "access_token": -} - -Example: -Assume you are @bob:matrix.org and you wish to login on another mobile device. -First, you GET /login which returns: -{ - "type": "m.login.email.code" -} -Your client knows how to handle this, so your client prompts the user to enter -their email address. This is then submitted: -{ - "type": "m.login.email.code", - "user": "@bob:matrix.org", - "email": "bob@mydomain.com" -} -The server confirms that bob@mydomain.com is linked to @bob:matrix.org, then -sends an email to this address and returns: -{ - "session": "ewuigf7462" -} -The client's screen changes to a code submission page. The email arrives and it -says something to the effect of "please enter 2348623 into the app". This is -the submitted along with the session: -{ - "type": "m.login.email.code", - "session": "ewuigf7462", - "code": "2348623" -} -The server accepts this and returns: -{ - "access_token": "abcdef0123456789" -} - -Email-based (url) ------------------ -Type: "m.login.email.url" -This is a multi-stage login. - -First LoginSubmission: -{ - "type": "m.login.email.url", - "user": - "email": -} -Returns: -{ - "session": -} - -The email contains a URL which must be clicked. After it has been clicked, the -client should perform a request: -{ - "type": "m.login.email.code", - "session": -} -Returns: -{ - "access_token": -} - -Example: -Assume you are @bob:matrix.org and you wish to login on another mobile device. -First, you GET /login which returns: -{ - "type": "m.login.email.url" -} -Your client knows how to handle this, so your client prompts the user to enter -their email address. This is then submitted: -{ - "type": "m.login.email.url", - "user": "@bob:matrix.org", - "email": "bob@mydomain.com" -} -The server confirms that bob@mydomain.com is linked to @bob:matrix.org, then -sends an email to this address and returns: -{ - "session": "ewuigf7462" -} -The client then starts polling the server with the following: -{ - "type": "m.login.email.url", - "session": "ewuigf7462" -} -(Alternatively, the server could send the device a push notification when the -email has been validated). The email arrives and it contains a URL to click on. -The user clicks on the which completes the login process with the server. The -next time the client polls, it returns: -{ - "access_token": "abcdef0123456789" -} - -N-Factor auth -------------- -Multiple login stages can be combined with the "next" key in the LoginResult. - -Example: -A server demands an email.code then password auth before logging in. First, the -client performs a GET /login which returns: -{ - "type": "m.login.email.code", - "stages": ["m.login.email.code", "m.login.password"] -} -The client performs the email login (See "Email-based (code)"), but instead of -returning an access_token, it returns: -{ - "next": "m.login.password" -} -The client then presents a user/password screen and the login continues until -this is complete (See "Password-based"), which then returns the "access_token". - -Rooms -===== -A room is a conceptual place where users can send and receive messages. Rooms -can be created, joined and left. Messages are sent -to a room, and all participants in that room will receive the message. Rooms are -uniquely identified via the room_id. - -Creating a room (with a room ID) --------------------------------- - Event Type: m.room.create [TODO(kegan): Do we generate events for this?] - REST Path: /rooms/$room_id - Valid methods: PUT - Required keys: None. - Optional keys: - visibility : [public|private] - Set whether this room shows up in the public - room list. - Returns: - On Failure: MAY return a suggested alternative room ID if this room ID is - taken. - { - suggested_room_id : $new_room_id - error : "Room already in use." - errcode : "M_ROOM_IN_USE" - } - - -Creating a room (without a room ID) ------------------------------------ - Event Type: m.room.create [TODO(kegan): Do we generate events for this?] - REST Path: /rooms - Valid methods: POST - Required keys: None. - Optional keys: - visibility : [public|private] - Set whether this room shows up in the public - room list. - Returns: - On Success: The allocated room ID. Additional information about the room - such as the visibility MAY be included as extra keys in this response. - { - room_id : $room_id - } - -Setting the topic for a room ----------------------------- - Event Type: m.room.topic - REST Path: /rooms/$room_id/topic - Valid methods: GET/PUT - Required keys: - topic : $topicname - Set the topic to $topicname in room $room_id. - - -See a list of public rooms --------------------------- - REST Path: /public/rooms?pagination_query_parameters - Valid methods: GET - This API can use pagination query parameters. - Returns: - { - "chunk" : JSON array of RoomInfo JSON objects - Required. - "start" : "string (start token)" - See Pagination Response. - "end" : "string (end token)" - See Pagination Response. - "total" : integer - Optional. The total number of rooms. - } - -RoomInfo: Information about a single room. - Servers MUST send the key: room_id - Servers MAY send the keys: topic, num_members - { - "room_id" : "string", - "topic" : "string", - "num_members" : integer - } - -Room Members -============ - -Invite/Joining/Leaving a room ------------------------------ - Event Type: m.room.member - REST Path: /rooms/$room_id/members/$user_id/state - Valid methods: PUT/GET/DELETE - Required keys: - membership : [join|invite] - The membership state of $user_id in room - $room_id. - Optional keys: - displayname, - avatar_url : String fields from the member user's profile - state, - status_msg, - mtime_age : Presence information - - These optional keys provide extra information that the client is likely to - be interested in so it doesn't have to perform an additional profile or - presence information fetch. - -Where: - join - Indicate you ($user_id) are joining the room $room_id. - invite - Indicate that $user_id has been invited to room $room_id. - -User $user_id can leave room $room_id by DELETEing this path. - -Checking the user list of a room --------------------------------- - REST Path: /rooms/$room_id/members/list - This API can use pagination query parameters. - Valid methods: GET - Returns: - A pagination response with chunk data as m.room.member events. - -Messages -======== -Users send messages to other users in rooms. These messages may be text, images, -video, etc. Clients may also want to acknowledge messages by sending feedback, -in the form of delivery/read receipts. - -Server-attached keys --------------------- -The server MAY attach additional keys to messages and feedback. If a client -submits keys with the same name, they will be clobbered by -the server. - -Required keys: -from : "string [user_id]" - The user_id of the user who sent the message/feedback. - -Optional keys: -hsob_ts : integer - A timestamp (ms resolution) representing when the message/feedback got to the - sender's home server ("home server outbound timestamp"). - -hsib_ts : integer - A timestamp (ms resolution) representing when the - message/feedback got to the receiver's home server ("home server inbound - timestamp"). This may be the same as hsob_ts if the sender/receiver are on the - same home server. - -Sending messages ----------------- - Event Type: m.room.message - REST Path: /rooms/$room_id/messages/$from/$msg_id - Valid methods: GET/PUT - URL parameters: - $from : user_id - The sender's user_id. This value will be clobbered by the - server before sending. - Required keys: - msgtype: [m.text|m.emote|m.image|m.audio|m.video|m.location|m.file] - - The type of message. Not to be confused with the Event 'type'. - Optional keys: - sender_ts : integer - A timestamp (ms resolution) representing the - wall-clock time when the message was sent from the client. - Reserved keys: - body : "string" - The human readable string for compatibility with clients - which cannot process a given msgtype. This key is optional, but - if it is included, it MUST be human readable text - describing the message. See individual msgtypes for more - info on what this means in practice. - -Each msgtype may have required fields of their own. - -msgtype: m.text ----------------- -Required keys: - body : "string" - The body of the message. -Optional keys: - None. - -msgtype: m.emote ------------------ -Required keys: - body : "string" - *tries to come up with a witty explanation*. -Optional keys: - None. - -msgtype: m.image ------------------ -Required keys: - url : "string" - The URL to the image. -Optional keys: - body : "string" - info : JSON object (ImageInfo) - The image info for image - referred to in 'url'. - thumbnail_url : "string" - The URL to the thumbnail. - thumbnail_info : JSON object (ImageInfo) - The image info for the image - referred to in 'thumbnail_url'. - -ImageInfo: Information about an image. -{ - "size" : integer (size of image in bytes), - "w" : integer (width of image in pixels), - "h" : integer (height of image in pixels), - "mimetype" : "string (e.g. image/jpeg)" -} - -Interpretation of 'body' key: The alt text of the image, or some kind of content -description for accessibility e.g. "image attachment". - -msgtype: m.audio ------------------ -Required keys: - url : "string" - The URL to the audio. -Optional keys: - info : JSON object (AudioInfo) - The audio info for the audio referred to in - 'url'. - -AudioInfo: Information about a piece of audio. -{ - "mimetype" : "string (e.g. audio/aac)", - "size" : integer (size of audio in bytes), - "duration" : integer (duration of audio in milliseconds) -} - -Interpretation of 'body' key: A description of the audio e.g. "Bee Gees - -Stayin' Alive", or some kind of content description for accessibility e.g. -"audio attachment". - -msgtype: m.video ------------------ -Required keys: - url : "string" - The URL to the video. -Optional keys: - info : JSON object (VideoInfo) - The video info for the video referred to in - 'url'. - -VideoInfo: Information about a video. -{ - "mimetype" : "string (e.g. video/mp4)", - "size" : integer (size of video in bytes), - "duration" : integer (duration of video in milliseconds), - "w" : integer (width of video in pixels), - "h" : integer (height of video in pixels), - "thumbnail_url" : "string (URL to image)", - "thumbanil_info" : JSON object (ImageInfo) -} - -Interpretation of 'body' key: A description of the video e.g. "Gangnam style", -or some kind of content description for accessibility e.g. "video attachment". - -msgtype: m.location --------------------- -Required keys: - geo_uri : "string" - The geo URI representing the location. -Optional keys: - thumbnail_url : "string" - The URL to a thumnail of the location being - represented. - thumbnail_info : JSON object (ImageInfo) - The image info for the image - referred to in 'thumbnail_url'. - -Interpretation of 'body' key: A description of the location e.g. "Big Ben, -London, UK", or some kind of content description for accessibility e.g. -"location attachment". - - -Sending feedback ----------------- -When you receive a message, you may want to send delivery receipt to let the -sender know that the message arrived. You may also want to send a read receipt -when the user has read the message. These receipts are collectively known as -'feedback'. - - Event Type: m.room.message.feedback - REST Path: /rooms/$room_id/messages/$msgfrom/$msg_id/feedback/$from/$feedback - Valid methods: GET/PUT - URL parameters: - $msgfrom - The sender of the message's user_id. - $from : user_id - The sender of the feedback's user_id. This value will be - clobbered by the server before sending. - $feedback : [d|r] - Specify if this is a [d]elivery or [r]ead receipt. - Required keys: - None. - Optional keys: - sender_ts : integer - A timestamp (ms resolution) representing the - wall-clock time when the receipt was sent from the client. - -Receiving messages (bulk/pagination) ------------------------------------- - Event Type: m.room.message - REST Path: /rooms/$room_id/messages/list - Valid methods: GET - Query Parameters: - feedback : [true|false] - Specify if feedback should be bundled with each - message. - This API can use pagination query parameters. - Returns: - A JSON array of Event Data in "chunk" (see Pagination Response). If the - "feedback" parameter was set, the Event Data will also contain a "feedback" - key which contains a JSON array of feedback, with each element as Event Data - with compressed feedback for this message. - -Event Data with compressed feedback is a special type of feedback with -contextual keys removed. It is designed to limit the amount of redundant data -being sent for feedback. This removes the type, event_id, room ID, -message sender ID and message ID keys. - - ORIGINAL (via event streaming) -{ - "event_id":"e1247632487", - "type":"m.room.message.feedback", - "from":"string [user_id]", - "feedback":"string [d|r]", - "room_id":"$room_id", - "msg_id":"$msg_id", - "msgfrom":"$msgfromid", - "content":{ - "sender_ts":139880943 - } -} - - COMPRESSED (via /messages/list) -{ - "from":"string [user_id]", - "feedback":"string [d|r]", - "content":{ - "sender_ts":139880943 - } -} - -When you join a room $room_id, you may want the last 10 messages with feedback. -This is represented as: - GET - /rooms/$room_id/messages/list?from=END&to=START&limit=10&feedback=true - -You may want to get 10 messages even earlier than that without feedback. If the -start stream token from the previous request was stok_019173, this request would -be: - GET - /rooms/$room_id/messages/list?from=stok_019173&to=START&limit=10& - feedback=false - -NOTE: Care must be taken when using this API in conjunction with event - streaming. It is possible that this will return a message which will - then come down the event stream, resulting in a duplicate message. Clients - should clobber based on the global message ID, or event ID. - - -Get current state for all rooms (aka IM Initial Sync API) -------------------------------- - REST Path: /im/sync - Valid methods: GET - This API can use pagination query parameters. Pagination is applied on a per - *room* basis. E.g. limit=1 means "get 1 message for each room" and not "get 1 - room's messages". If there is no limit, all messages for all rooms will be - returned. - If you want 1 room's messages, see "Receiving messages (bulk/pagination)". - Additional query parameters: - feedback: [true] - Bundles feedback with messages. - Returns: - An array of RoomStateInfo. - -RoomStateInfo: A snapshot of information about a single room. - { - "room_id" : "string", - "membership" : "string [join|invite]", - "messages" : { - "start": "string", - "end": "string", - "chunk": - m.room.message pagination stream events (with feedback if specified), - this is the same as "Receiving messages (bulk/pagination)". - } - } -The "membership" key is the calling user's membership state in the given -"room_id". The "messages" key may be omitted if the "membership" value is -"invite". Additional keys may be added to the top-level object, such as: - "topic" : "string" - The topic for the room in question. - "room_image_url" : "string" - The URL of the room image if specified. - "num_members" : integer - The number of members in the room. - - -Profiles -======== - -Getting/Setting your own displayname ------------------------------------- - REST Path: /profile/$user_id/displayname - Valid methods: GET/PUT - Required keys: - displayname : The displayname text - -Getting/Setting your own avatar image URL ------------------------------------------ -The homeserver does not currently store the avatar image itself, but offers -storage for the user to specify a web URL that points at the required image, -leaving it up to clients to fetch it themselves. - REST Path: /profile/$user_id/avatar_url - Valid methods: GET/PUT - Required keys: - avatar_url : The URL path to the required image - -Getting other user's profile information ----------------------------------------- -Either of the above REST methods may be used to fetch other user's profile -information by the client, either on other local users on the same homeserver or -for users from other servers entirely. - - -Presence -======== - -In the following messages, the presence state is a presence string as described in -the main specification document. - -Getting/Setting your own presence state ---------------------------------------- - REST Path: /presence/$user_id/status - Valid methods: GET/PUT - Required keys: - presence : - The user's new presence state - Optional keys: - status_msg : text string provided by the user to explain their status - -Fetching your presence list ---------------------------- - REST Path: /presence_list/$user_id - Valid methods: GET/(post) - Returns: - An array of presence list entries. Each entry is an object with the - following keys: - { - "user_id" : string giving the observed user's ID - "presence" : int giving their status - "status_msg" : optional text string - "displayname" : optional text string from the user's profile - "avatar_url" : optional text string from the user's profile - } - -Maintaining your presence list ------------------------------- - REST Path: /presence_list/$user_id - Valid methods: POST/(get) - With: A JSON object optionally containing either of the following keys: - "invite" : a list of strings giving user IDs to invite for presence - subscription - "drop" : a list of strings giving user IDs to remove from your presence - list - -Receiving presence update events --------------------------------- - Event Type: m.presence - Keys of the event's content are the same as those returned by the presence - list. - -Examples -======== - -The following example is the story of "bob", who signs up at "sy.org" and joins -the public room "room_beta@sy.org". They get the 2 most recent -messages (with feedback) in that room and then send a message in that room. - -For context, here is the complete chat log for room_beta@sy.org: - -Room: "Hello world" (room_beta@sy.org) -Members: (2) alice@randomhost.org, friend_of_alice@randomhost.org -Messages: - alice@randomhost.org : hi friend! - [friend_of_alice@randomhost.org DELIVERED] - alice@randomhost.org : you're my only friend - [friend_of_alice@randomhost.org DELIVERED] - alice@randomhost.org : afk - [friend_of_alice@randomhost.org DELIVERED] - [ bob@sy.org joins ] - bob@sy.org : Hi everyone - [ alice@randomhost.org changes the topic to "FRIENDS ONLY" ] - alice@randomhost.org : Hello!!!! - alice@randomhost.org : Let's go to another room - alice@randomhost.org : You're not my friend - [ alice@randomhost.org invites bob@sy.org to the room - commoners@randomhost.org] - - -REGISTER FOR AN ACCOUNT -POST: /register -Content: {} -Returns: { "user_id" : "bob@sy.org" , "access_token" : "abcdef0123456789" } - -GET PUBLIC ROOM LIST -GET: /rooms/list?access_token=abcdef0123456789 -Returns: -{ - "total":3, - "chunk": - [ - { "room_id":"room_alpha@sy.org", "topic":"I am a fish" }, - { "room_id":"room_beta@sy.org", "topic":"Hello world" }, - { "room_id":"room_xyz@sy.org", "topic":"Goodbye cruel world" } - ] -} - -JOIN ROOM room_beta@sy.org -PUT -/rooms/room_beta%40sy.org/members/bob%40sy.org/state? - access_token=abcdef0123456789 -Content: { "membership" : "join" } -Returns: 200 OK - -GET LATEST 2 MESSAGES WITH FEEDBACK -GET -/rooms/room_beta%40sy.org/messages/list?from=END&to=START&limit=2& - feedback=true&access_token=abcdef0123456789 -Returns: -{ - "chunk": - [ - { - "event_id":"01948374", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"avefifu", - "from":"alice@randomhost.org", - "hs_ts":139985736, - "content":{ - "msgtype":"m.text", - "body":"afk" - } - "feedback": [ - { - "from":"friend_of_alice@randomhost.org", - "feedback":"d", - "hs_ts":139985850, - "content":{ - "sender_ts":139985843 - } - } - ] - }, - { - "event_id":"028dfe8373", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"afhgfff", - "from":"alice@randomhost.org", - "hs_ts":139970006, - "content":{ - "msgtype":"m.text", - "body":"you're my only friend" - } - "feedback": [ - { - "from":"friend_of_alice@randomhost.org", - "feedback":"d", - "hs_ts":139970144, - "content":{ - "sender_ts":139970122 - } - } - ] - }, - ], - "start": "stok_04823947", - "end": "etok_1426425" -} - -SEND MESSAGE IN ROOM -PUT -/rooms/room_beta%40sy.org/messages/bob%40sy.org/m0001? - access_token=abcdef0123456789 -Content: { "msgtype" : "text" , "body" : "Hi everyone" } -Returns: 200 OK - - -Checking the event stream for this user: -GET: /events?from=START&access_token=abcdef0123456789 -Returns: -{ - "chunk": - [ - { - "event_id":"e10f3d2b", - "type":"m.room.member", - "room_id":"room_beta@sy.org", - "user_id":"bob@sy.org", - "content":{ - "membership":"join" - } - }, - { - "event_id":"1b352d32", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"m0001", - "from":"bob@sy.org", - "hs_ts":140193857, - "content":{ - "msgtype":"m.text", - "body":"Hi everyone" - } - } - ], - "start": "stok_9348635", - "end": "etok_1984723" -} - -Client disconnects for a while and the topic is updated in this room, 3 new -messages arrive whilst offline, and bob is invited to another room. - -GET /events?from=etok_1984723&access_token=abcdef0123456789 -Returns: -{ - "chunk": - [ - { - "event_id":"feee0294", - "type":"m.room.topic", - "room_id":"room_beta@sy.org", - "from":"alice@randomhost.org", - "content":{ - "topic":"FRIENDS ONLY", - } - }, - { - "event_id":"a028bd9e", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"z839409", - "from":"alice@randomhost.org", - "hs_ts":140195000, - "content":{ - "msgtype":"m.text", - "body":"Hello!!!" - } - }, - { - "event_id":"49372d9e", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"z839410", - "from":"alice@randomhost.org", - "hs_ts":140196000, - "content":{ - "msgtype":"m.text", - "body":"Let's go to another room" - } - }, - { - "event_id":"10abdd01", - "type":"m.room.message", - "room_id":"room_beta@sy.org", - "msg_id":"z839411", - "from":"alice@randomhost.org", - "hs_ts":140197000, - "content":{ - "msgtype":"m.text", - "body":"You're not my friend" - } - }, - { - "event_id":"0018453d", - "type":"m.room.member", - "room_id":"commoners@randomhost.org", - "from":"alice@randomhost.org", - "user_id":"bob@sy.org", - "content":{ - "membership":"invite" - } - }, - ], - "start": "stok_0184288", - "end": "etok_1348723" -} diff --git a/drafts/state_resolution.rst b/drafts/state_resolution.rst index fec290dd..3c570b8e 100644 --- a/drafts/state_resolution.rst +++ b/drafts/state_resolution.rst @@ -27,6 +27,7 @@ all state events eventually. Algorithm requirements ---------------------- We want the algorithm to have the following properties: + - Since we aren't guaranteed what order we receive state events in, except that we see parents before children, the state resolution algorithm must not depend on the order and must always come to the same result. diff --git a/drafts/typing_notifications.rst b/drafts/typing_notifications.rst index 3d3d23e0..048eba98 100644 --- a/drafts/typing_notifications.rst +++ b/drafts/typing_notifications.rst @@ -7,7 +7,7 @@ Client APIs To set "I am typing for the next N msec":: PUT .../rooms/:room_id/typing/:user_id Content: { "typing": true, "timeout": N } - # timeout is in msec; I suggest no more than 20 or 30 seconds + # timeout is in msec; I suggest no more than 20 or 30 seconds This should be re-sent by the client to continue informing the server the user is still typing; I suggest a safety margin of 5 seconds before the expected diff --git a/drafts/use_cases.rst b/drafts/use_cases.rst new file mode 100644 index 00000000..09172d49 --- /dev/null +++ b/drafts/use_cases.rst @@ -0,0 +1,317 @@ +General UI/UX requirements: +=========================== +- Live updates +- No flicker: + * Sending message (local echo) + * Receiving images (encoding w/h) + * Scrollback + * Resolving display names (from user ID) +- Fast startup times +- Fast "opening room" times (esp. when clicking through from a notification) +- Low latency file transfer. + +Use cases +--------- +- #1: Lightweight IM client (no perm storage) - e.g. Web client +- #2: Bug tracking software +- #3: Forum +- #4: Google + style communities +- #5: Email style threading +- #6: Multi-column threaded IM +- #7: Mobile IM client (perm storage) +- #8: MIDI client +- #9: Animatrix client +- #10: Unity object trees +- #11: Social Network ("Walls", PMs, groups) +- #12: Minecraft-clone +- #13: Global 'Like' widget, which links through to a room. + + +#1 Web client UI +================ + +Model:: + + Rooms ----< Messages + - name - type (call/image) + - topic + +Home Screen + What's visible: + - Recent chats ordered by timestamp of latest event (with # users) + - Your own display name, user ID and avatar url + - A searchable list of public rooms (with # users and alias + room name + room topic) + What you can do: + - Create a room (public/private, with alias) + - Join a room from alias + - Message a user (with user ID) + - Leave a recent room + - Open a room + - Open a chat history link. + - Search for a public room. + +Chat Screen + What's visible: + - Enough scrollback to fill a "screen full" of content. + - Each message: timestamp, user ID, display name at the time the message was + sent, avatar URL at the time the message was sent, whether it was a bing message + or not. + - User list: for each user: presence, current avatar url in the room, current + display name in the room, power level, ordered by when they were last speaking. + - Recents list: (same as Home Screen) + - Room name + - Room topic + - Typing notifications + - Desktop/Push Notifications for messages + What you can do: + - Invite a user + - Kick a user + - Ban/Unban a user + - Leave the room + - Send a message (image/text/emote) + - Change someone's power level + - Change your own display name + - Accept an incoming call + - Make an outgoing call + - Get older messages by scrolling up (scrollback) + - Redact a message + - Resend a message which was not sent + Message sending: + - Immediate local echo + - Queue up messages which haven't been sent yet + - Reordering local echo to where it actually happened + VoIP: + - One entry in your display for a call (which may contain duration, type, status) + - Glare resolution + Scrollback: + - Display in reverse chronological order by the originating server's timestamp + - Terminates at the start of the room (which then makes it impossible to request + more scrollback) + Local storage: + - Driven by desire for fast startup times and minimal network traffic + - Display messages from storage and from the network without any gaps in messages. + - Persist scrollback if possible: Scrollback from storage first then from the + network. + Notifications: + - Receive notifications for rooms you're interested in (explicitly or from a default) + - Maybe per device. + - Maybe depending on presence (e.g. idle) + - Maybe depending on message volume + - Maybe depending on room config options. + Message contents: + - images + - video + - rich text + - audio + - arbitrary files + - location + - vcards (potentially) + +Chat History Screen + What's visible: + - The linked message and enough scrollback to fill a "screen full" of content. + - Each message: timestamp, user ID, display name at the time the message was + sent, avatar URL at the time the message was sent, whether it was a bing message + or not. + - The historical user list. *TODO: Is this taken at the linked message, or at + wherever the user has scrolled to?* + What you can do: + - Get older messages by scrolling up (scrollback) + - Get newer messages by scrolling down + +Public Room Search Screen + What's visible: + - The current search text. + - The homeserver being searched (defaults to the HS the client is connected to). + - The results of the current search with enough results to fill the screen + with # users and alias + room name + room topic. + What you can do: + - Change what you are searching for. + - Change the server that's being searched. + - Scroll down to get more search results. + +User screen + What's visible: + - Display name + - Avatar + - User ID + What you can do: + - Start a chat with the user + + +#2 Bug tracking UI +================== + +Model:: + + Projects ----< Issues ---< Comments + - key - summary - user + - name - ID - message + SYN SYN-52 Fix it nooow! + +Landing page + What's visible: + - Issues assigned to me + - Issues I'm watching + - Recent activity on other issues (not refined to me) + - List of projects + What you can do: + - View an issue + - Create an issue + - Sort issues + - View a user + - View a project + - Search for issues (by name, time, priority, description contents, reporter, etc...) + +Issue page + What's visible: + - Summary of issue + - Issue key + - Project affected + - Description + - Comments + - Priority, labels, type, purpose, etc.. + - Reporter/assignee + - Creation and last updated times + - History of issue changes + What you can do: + - Comment on issue + - Change issue info (labels, type, purpose, etc..) + - Open/Close/Resolve the issue + - Edit the issue + - Watch/Unwatch the issue + + +#3 Forum UI +=========== + +Model:: + + Forum ----< Boards ----< Threads ----< Messages + - Matrix - Dev - HALP! - please halp! + +Main page + What's visible: + - Categories (containing boards) + - Boards (with names and # posts and tagline and latest post) + What you can do: + - View a board + - View the latest message on a board + +Board page + What's visible: + - Threads (titles, OP, latest post date+author, # replies, # upvotes, whether + the OP contains an image or hyperlink (small icon on title)) + - Whether the thread is answered (with link to the answer) + - Pagination for posts within a thread (1,2,3,4,5...10) + - Pagination for threads within a board + - List of threads in chronological order + - Stickied threads + What you can do: + - View a user + - View a thread on a particular page + - View the latest message on a thread + - View older threads (pagination) + - Search the board + +Thread page + What's visible: + - Messages in chronological order + - For each message: author, timestamp, # posts by author, avatar, registration + date, status message, message contents, # views of message + What you can do: + - Upvote the message + - Flag the message for a mod + - Reply to the message + - Subscribe to thread or message's RSS feed + - Go to previous/next thread + + +#4 Google+ community +==================== + +Model:: + + Community -----< Categories ----< Posts ---< Comments + Kerbal SP Mods, Help Text Text + (no title!) + +Communities page + What's visible: + - List of communities + - For each community: # users, # posts, group pic, title + What you can do: + - Join a community + - View a community + +Community Page + What's visible: + - Title, pic + - List of categories + - List of members with avatars (+ total #) + - Most recent posts with comments (most recent comment if >1) + What you can do: + - Join the group + - Post a post (with voting and options) + - Report abuse + - View member + - Expand comments + - Infinite scrolling + - Add a comment to a post + - Share a post + - +1 a post + +#5 Email style threading +======================== + +Chat Screen + What's visible: + - Enough scrollback to fill a "screen full" of content. + - Threads: + + - Initially will only display the timestamp and user ID of the *first* + message. But can expand to show the entire tree. + - Tree of messages indicating which message is a reply to which. + - Ordered by the arbitrary field (timestamp of oldest message in thread; + newest message in thread; sender id; sender display name; etc) + - Each message: timestamp, user ID, display name at the time of the message + - Room name + - Room topic + - Typing notifications + - Desktop/Push Notifications for messages + What you can do: + - Send a message in reply to another message: + + - Immediate local echo, may cause messages to re-order + - Messages that haven't reached the server are queued. + - Thread is displayed where it should be in the thread order once the + message is sent. + - Start a new thread by sending a message. + +#6 Multi-threaded IM +==================== + +Chat Screen + What's visible: + - A multi-column grid of threads from a number of chatrooms + Each concurrent thread is displayed in a different column. + The columns start and end as threads split and rejoin the main conversation + The messages for each thread are ordered by how recent they are:: + + Room #1 Room # 2 Room # 2 + +------------+ +----------------+ Side thread. + | * Message1 | | * Root | +--------------+ + | * Message2 | | * A1 -> Root | | * B1 -> Root | + +------------+ | * A2 -> A1 | | * B2 -> B1 | + | * M -> A2, B2 | +--------------+ + +----------------+ + + - Typing notifications. Displayed within the correct thread/column. + + What you can do: + - Send a message into a particular thread/column. + - Move an *existing* message into a new thread creating a new column + - Move an existing message into an existing thread, causing the threads to + reconverge (i.e. provide a route from the sidebar back into the existing + thread). This does not imply terminating the thread, which can continue + independently of the merge. diff --git a/jsfiddles/create_room_send_msg/demo.css b/jsfiddles/create_room_send_msg/demo.css new file mode 100644 index 00000000..48a55f37 --- /dev/null +++ b/jsfiddles/create_room_send_msg/demo.css @@ -0,0 +1,17 @@ +.loggedin { + visibility: hidden; +} + +p { + font-family: monospace; +} + +table +{ + border-spacing:5px; +} + +th,td +{ + padding:5px; +} diff --git a/jsfiddles/create_room_send_msg/demo.html b/jsfiddles/create_room_send_msg/demo.html new file mode 100644 index 00000000..088ff7ac --- /dev/null +++ b/jsfiddles/create_room_send_msg/demo.html @@ -0,0 +1,30 @@ +
+

This room creation / message sending demo requires a home server to be running on http://localhost:8008

+
+
+ + + +
+
+
+ + +
+
+ + + +
+ + + + + + + + + +
Room IDMy stateRoom AliasLatest message
+
+ diff --git a/jsfiddles/create_room_send_msg/demo.js b/jsfiddles/create_room_send_msg/demo.js new file mode 100644 index 00000000..9c346e2f --- /dev/null +++ b/jsfiddles/create_room_send_msg/demo.js @@ -0,0 +1,113 @@ +var accountInfo = {}; + +var showLoggedIn = function(data) { + accountInfo = data; + getCurrentRoomList(); + $(".loggedin").css({visibility: "visible"}); +}; + +$('.login').live('click', function() { + var user = $("#userLogin").val(); + var password = $("#passwordLogin").val(); + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/login", + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({ user: user, password: password, type: "m.login.password" }), + dataType: "json", + success: function(data) { + showLoggedIn(data); + }, + error: function(err) { + var errMsg = "To try this, you need a home server running!"; + var errJson = $.parseJSON(err.responseText); + if (errJson) { + errMsg = JSON.stringify(errJson); + } + alert(errMsg); + } + }); +}); + +var getCurrentRoomList = function() { + var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1"; + $.getJSON(url, function(data) { + var rooms = data.rooms; + for (var i=0; i 0) { + data.room_alias_name = roomAlias; + } + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify(data), + dataType: "json", + success: function(data) { + data.membership = "join"; // you are automatically joined into every room you make. + data.latest_message = ""; + addRoom(data); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); +}); + +var addRoom = function(data) { + row = "" + + ""+data.room_id+"" + + ""+data.membership+"" + + ""+data.room_alias+"" + + ""+data.latest_message+"" + + ""; + $("#rooms").append(row); +}; + +$('.sendMessage').live('click', function() { + var roomId = $("#roomId").val(); + var body = $("#messageBody").val(); + var msgId = $.now(); + + if (roomId.length === 0 || body.length === 0) { + return; + } + + var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$roomid", encodeURIComponent(roomId)); + + var data = { + msgtype: "m.text", + body: body + }; + + $.ajax({ + url: url, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify(data), + dataType: "json", + success: function(data) { + $("#messageBody").val(""); + // wipe the table and reload it. Using the event stream would be the best + // solution but that is out of scope of this fiddle. + $("#rooms").find("tr:gt(0)").remove(); + getCurrentRoomList(); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); +}); diff --git a/jsfiddles/event_stream/demo.css b/jsfiddles/event_stream/demo.css new file mode 100644 index 00000000..48a55f37 --- /dev/null +++ b/jsfiddles/event_stream/demo.css @@ -0,0 +1,17 @@ +.loggedin { + visibility: hidden; +} + +p { + font-family: monospace; +} + +table +{ + border-spacing:5px; +} + +th,td +{ + padding:5px; +} diff --git a/jsfiddles/event_stream/demo.html b/jsfiddles/event_stream/demo.html new file mode 100644 index 00000000..7657780d --- /dev/null +++ b/jsfiddles/event_stream/demo.html @@ -0,0 +1,23 @@ +
+

This event stream demo requires a home server to be running on http://localhost:8008

+
+
+ + + +
+
+
+ +
+

+ + + + + + + +
Room IDLatest message
+
+ diff --git a/jsfiddles/event_stream/demo.js b/jsfiddles/event_stream/demo.js new file mode 100644 index 00000000..acba8391 --- /dev/null +++ b/jsfiddles/event_stream/demo.js @@ -0,0 +1,145 @@ +var accountInfo = {}; + +var eventStreamInfo = { + from: "END" +}; + +var roomInfo = []; + +var longpollEventStream = function() { + var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$from", eventStreamInfo.from); + + $.getJSON(url, function(data) { + eventStreamInfo.from = data.end; + + var hasNewLatestMessage = false; + for (var i=0; i"+roomList[i].room_id+"" + + ""+roomList[i].latest_message+"" + + ""; + rows += row; + } + + $("#rooms").append(rows); +}; + diff --git a/jsfiddles/example_app/demo.css b/jsfiddles/example_app/demo.css new file mode 100644 index 00000000..4c1e157c --- /dev/null +++ b/jsfiddles/example_app/demo.css @@ -0,0 +1,43 @@ +.roomListDashboard, .roomContents, .sendMessageForm { + visibility: hidden; +} + +.roomList { + background-color: #909090; +} + +.messageWrapper { + background-color: #EEEEEE; + height: 400px; + overflow: scroll; +} + +.membersWrapper { + background-color: #EEEEEE; + height: 200px; + width: 50%; + overflow: scroll; +} + +.textEntry { + width: 100% +} + +p { + font-family: monospace; +} + +table +{ + border-spacing:5px; +} + +th,td +{ + padding:5px; +} + +.roomList tr:not(:first-child):hover { + background-color: orange; + cursor: pointer; +} diff --git a/jsfiddles/example_app/demo.details b/jsfiddles/example_app/demo.details new file mode 100644 index 00000000..3f96d3e7 --- /dev/null +++ b/jsfiddles/example_app/demo.details @@ -0,0 +1,7 @@ + name: Example Matrix Client + description: Includes login, live event streaming, creating rooms, sending messages and viewing member lists. + authors: + - matrix.org + resources: + - http://matrix.org + normalize_css: no \ No newline at end of file diff --git a/jsfiddles/example_app/demo.html b/jsfiddles/example_app/demo.html new file mode 100644 index 00000000..7a9dffdd --- /dev/null +++ b/jsfiddles/example_app/demo.html @@ -0,0 +1,56 @@ + + +
+
+ + +
+ + + + + + + + +
RoomMy stateLatest message
+
+ +
+

Select a room

+
+ + + +
+
+
+ + +
+
+ +
+

Member list:

+
+ + + +
+
+
+ diff --git a/jsfiddles/example_app/demo.js b/jsfiddles/example_app/demo.js new file mode 100644 index 00000000..13c9c2b3 --- /dev/null +++ b/jsfiddles/example_app/demo.js @@ -0,0 +1,327 @@ +var accountInfo = {}; + +var eventStreamInfo = { + from: "END" +}; + +var roomInfo = []; +var memberInfo = []; +var viewingRoomId; + +// ************** Event Streaming ************** +var longpollEventStream = function() { + var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$from", eventStreamInfo.from); + + $.getJSON(url, function(data) { + eventStreamInfo.from = data.end; + + var hasNewLatestMessage = false; + var updatedMemberList = false; + var i=0; + var j=0; + for (i=0; i 0) { + data.room_alias_name = roomAlias; + } + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify(data), + dataType: "json", + success: function(response) { + $("#roomAlias").val(""); + response.membership = "join"; // you are automatically joined into every room you make. + response.latest_message = ""; + + roomInfo.push(response); + setRooms(roomInfo); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); +}); + +// ************** Getting current state ************** +var getCurrentRoomList = function() { + var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1"; + $.getJSON(url, function(data) { + var rooms = data.rooms; + for (var i=0; i=0; --i) { + addMessage(data.chunk[i]); + } + }); +}; + +var getMemberList = function(roomId) { + $("#members").empty(); + memberInfo = []; + var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" + + encodeURIComponent(roomId) + "/members?access_token=" + accountInfo.access_token; + $.getJSON(url, function(data) { + for (var i=0; i"+roomList[i].room_id+"" + + ""+roomList[i].membership+"" + + ""+roomList[i].latest_message+"" + + ""; + rows += row; + } + + $("#rooms").append(rows); + + $('#rooms').find("tr").click(function(){ + var roomId = $(this).find('td:eq(0)').text(); + var membership = $(this).find('td:eq(1)').text(); + if (membership !== "join") { + console.log("Joining room " + roomId); + var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/join?access_token=$token"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$roomid", encodeURIComponent(roomId)); + $.ajax({ + url: url, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({membership: "join"}), + dataType: "json", + success: function(data) { + loadRoomContent(roomId); + getCurrentRoomList(); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); + } + else { + loadRoomContent(roomId); + } + }); +}; + +var addMessage = function(data) { + + var msg = data.content.body; + if (data.type === "m.room.member") { + if (data.content.membership === undefined) { + return; + } + if (data.content.membership === "invite") { + msg = "invited " + data.state_key + " to the room"; + } + else if (data.content.membership === "join") { + msg = "joined the room"; + } + else if (data.content.membership === "leave") { + msg = "left the room"; + } + else if (data.content.membership === "ban") { + msg = "was banned from the room"; + } + } + if (msg === undefined) { + return; + } + + var row = "" + + ""+data.user_id+"" + + ""+msg+"" + + ""; + $("#messages").append(row); +}; + +var addMember = function(data) { + var row = "" + + ""+data.state_key+"" + + ""+data.content.membership+"" + + ""; + $("#members").append(row); +}; + diff --git a/jsfiddles/register_login/demo.css b/jsfiddles/register_login/demo.css new file mode 100644 index 00000000..11781c25 --- /dev/null +++ b/jsfiddles/register_login/demo.css @@ -0,0 +1,7 @@ +.loggedin { + visibility: hidden; +} + +p { + font-family: monospace; +} diff --git a/jsfiddles/register_login/demo.html b/jsfiddles/register_login/demo.html new file mode 100644 index 00000000..fcac453a --- /dev/null +++ b/jsfiddles/register_login/demo.html @@ -0,0 +1,20 @@ +
+

This registration/login demo requires a home server to be running on http://localhost:8008

+
+
+ + + +
+
+ + + +
+
+

+ + +

+
+ diff --git a/jsfiddles/register_login/demo.js b/jsfiddles/register_login/demo.js new file mode 100644 index 00000000..2e6957b6 --- /dev/null +++ b/jsfiddles/register_login/demo.js @@ -0,0 +1,79 @@ +var accountInfo = {}; + +var showLoggedIn = function(data) { + accountInfo = data; + $(".loggedin").css({visibility: "visible"}); + $("#welcomeText").text("Welcome " + accountInfo.user_id+". Your access token is: " + + accountInfo.access_token); +}; + +$('.register').live('click', function() { + var user = $("#user").val(); + var password = $("#password").val(); + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/register", + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({ user: user, password: password, type: "m.login.password" }), + dataType: "json", + success: function(data) { + showLoggedIn(data); + }, + error: function(err) { + var errMsg = "To try this, you need a home server running!"; + var errJson = $.parseJSON(err.responseText); + if (errJson) { + errMsg = JSON.stringify(errJson); + } + alert(errMsg); + } + }); +}); + +var login = function(user, password) { + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/login", + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({ user: user, password: password, type: "m.login.password" }), + dataType: "json", + success: function(data) { + showLoggedIn(data); + }, + error: function(err) { + var errMsg = "To try this, you need a home server running!"; + var errJson = $.parseJSON(err.responseText); + if (errJson) { + errMsg = JSON.stringify(errJson); + } + alert(errMsg); + } + }); +}; + +$('.login').live('click', function() { + var user = $("#userLogin").val(); + var password = $("#passwordLogin").val(); + $.getJSON("http://localhost:8008/_matrix/client/api/v1/login", function(data) { + if (data.flows[0].type !== "m.login.password") { + alert("I don't know how to login with this type: " + data.type); + return; + } + login(user, password); + }); +}); + +$('.logout').live('click', function() { + accountInfo = {}; + $("#imSyncText").text(""); + $(".loggedin").css({visibility: "hidden"}); +}); + +$('.testToken').live('click', function() { + var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1"; + $.getJSON(url, function(data) { + $("#imSyncText").text(JSON.stringify(data, undefined, 2)); + }).fail(function(err) { + $("#imSyncText").text(JSON.stringify($.parseJSON(err.responseText))); + }); +}); diff --git a/jsfiddles/room_memberships/demo.css b/jsfiddles/room_memberships/demo.css new file mode 100644 index 00000000..48a55f37 --- /dev/null +++ b/jsfiddles/room_memberships/demo.css @@ -0,0 +1,17 @@ +.loggedin { + visibility: hidden; +} + +p { + font-family: monospace; +} + +table +{ + border-spacing:5px; +} + +th,td +{ + padding:5px; +} diff --git a/jsfiddles/room_memberships/demo.html b/jsfiddles/room_memberships/demo.html new file mode 100644 index 00000000..e6f39df5 --- /dev/null +++ b/jsfiddles/room_memberships/demo.html @@ -0,0 +1,37 @@ +
+

This room membership demo requires a home server to be running on http://localhost:8008

+
+
+ + + +
+
+
+ +
+
+ + + + +
+
+ + +
+ + + + + + + + +
Room IDMy stateRoom Alias
+
+ diff --git a/jsfiddles/room_memberships/demo.js b/jsfiddles/room_memberships/demo.js new file mode 100644 index 00000000..8a7b1aa8 --- /dev/null +++ b/jsfiddles/room_memberships/demo.js @@ -0,0 +1,141 @@ +var accountInfo = {}; + +var showLoggedIn = function(data) { + accountInfo = data; + getCurrentRoomList(); + $(".loggedin").css({visibility: "visible"}); + $("#membership").change(function() { + if ($("#membership").val() === "invite") { + $("#targetUser").css({visibility: "visible"}); + } + else { + $("#targetUser").css({visibility: "hidden"}); + } +}); +}; + +$('.login').live('click', function() { + var user = $("#userLogin").val(); + var password = $("#passwordLogin").val(); + $.ajax({ + url: "http://localhost:8008/_matrix/client/api/v1/login", + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({ user: user, password: password, type: "m.login.password" }), + dataType: "json", + success: function(data) { + $("#rooms").find("tr:gt(0)").remove(); + showLoggedIn(data); + }, + error: function(err) { + var errMsg = "To try this, you need a home server running!"; + var errJson = $.parseJSON(err.responseText); + if (errJson) { + errMsg = JSON.stringify(errJson); + } + alert(errMsg); + } + }); +}); + +var getCurrentRoomList = function() { + $("#roomId").val(""); + // wipe the table and reload it. Using the event stream would be the best + // solution but that is out of scope of this fiddle. + $("#rooms").find("tr:gt(0)").remove(); + + var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1"; + $.getJSON(url, function(data) { + var rooms = data.rooms; + for (var i=0; i"+data.room_id+"" + + ""+data.membership+"" + + ""+data.room_alias+"" + + ""; + $("#rooms").append(row); +}; + +$('.changeMembership').live('click', function() { + var roomId = $("#roomId").val(); + var member = $("#targetUser").val(); + var membership = $("#membership").val(); + + if (roomId.length === 0) { + return; + } + + var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/$membership?access_token=$token"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$roomid", encodeURIComponent(roomId)); + url = url.replace("$membership", membership); + + var data = {}; + + if (membership === "invite") { + data = { + user_id: member + }; + } + + $.ajax({ + url: url, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify(data), + dataType: "json", + success: function(data) { + getCurrentRoomList(); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); +}); + +$('.joinAlias').live('click', function() { + var roomAlias = $("#roomAlias").val(); + var url = "http://localhost:8008/_matrix/client/api/v1/join/$roomalias?access_token=$token"; + url = url.replace("$token", accountInfo.access_token); + url = url.replace("$roomalias", encodeURIComponent(roomAlias)); + $.ajax({ + url: url, + type: "POST", + contentType: "application/json; charset=utf-8", + data: JSON.stringify({}), + dataType: "json", + success: function(data) { + getCurrentRoomList(); + }, + error: function(err) { + alert(JSON.stringify($.parseJSON(err.responseText))); + } + }); +}); diff --git a/scripts/gendoc.sh b/scripts/gendoc.sh index 38143939..4a92f681 100755 --- a/scripts/gendoc.sh +++ b/scripts/gendoc.sh @@ -18,6 +18,6 @@ perl -pi -e 's###' $MATRIXDO perl -MFile::Slurp -pi -e 'BEGIN { $nav = read_file("'$MATRIXDOTORG'/includes/nav.html") } s##
[matrix]
#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html -perl -pi -e 's##
#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html +perl -pi -e 's###' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html -scp -r $MATRIXDOTORG/docs matrix@ldc-prd-matrix-001:/sites/matrix \ No newline at end of file +scp -r $MATRIXDOTORG/docs matrix@ldc-prd-matrix-001:/sites/matrix diff --git a/specification/00_basis.rst b/specification/00_basis.rst index 8dceac1f..77aec367 100644 --- a/specification/00_basis.rst +++ b/specification/00_basis.rst @@ -134,7 +134,7 @@ federate with other HSes. It is typically responsible for multiple clients. more home servers. Events -++++++ +~~~~~~ Data in Matrix is encapsulated in an "event". An event is an action within the system. Typically each action (e.g. sending a message) correlates with exactly @@ -148,7 +148,7 @@ is the event type for instant messages. Events are usually sent in the context of a "Room". Event Graphs -++++++++++++ +~~~~~~~~~~~~ Each event has a list of zero or more `parent` events. These relations form directed acyclic graphs of events called `event graphs`. Every event graph has a single root event, and each event graph forms the @@ -247,7 +247,7 @@ participating in a room. Room Aliases -~~~~~~~~~~~~ +++++++++++++ Each room can also have multiple "Room Aliases", which looks like:: @@ -282,7 +282,7 @@ that are in the room that can be used to join via. |________________________________| Identity -~~~~~~~~ +++++++++ Users in Matrix are identified via their matrix user ID (MXID). However, existing 3rd party ID namespaces can also be used in order to identify Matrix @@ -303,7 +303,7 @@ the Matrix ecosystem. However, without one clients will not be able to look up user IDs using 3PIDs. Presence -~~~~~~~~ +++++++++ Each user has the concept of presence information. This encodes the "availability" of that user, suitable for display on other user's clients. This @@ -343,7 +343,7 @@ the other direction it will not). This timestamp is presented via a key called message is generated/emitted that the user was last seen active. Presence List -+++++++++++++ +~~~~~~~~~~~~~ Each user's home server stores a "presence list". This stores a list of user IDs whose presence the user wants to follow. @@ -353,7 +353,7 @@ and accept the invitation. Once accepted, both user's HSes track the subscription. Presence and Permissions -++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~ For a viewing user to be allowed to see the presence information of a target user, either: @@ -367,7 +367,7 @@ presence information in a user list for a room. Profiles -~~~~~~~~ +++++++++ .. TODO-spec - Metadata extensibility diff --git a/specification/10_events.rst b/specification/10_events.rst index aa9f99b1..20dccb6b 100644 --- a/specification/10_events.rst +++ b/specification/10_events.rst @@ -330,6 +330,22 @@ outlined below: Example: ``{ "msgtype": "m.emote", "body": "tries to come up with a witty explanation" }`` +``m.notice`` + Required keys: + - ``body`` : "string" - The body of the message. + Optional keys: + None. + Example: + ``{ "msgype": "m.notice", "body": "some kind of automated announcement" }`` + + A ``m.notice`` message should be considered similar to a plain ``m.text`` + message except that clients should visually distinguish it in some way. It is + intended to be used by automated clients, such as bots, bridges, and other + entities, rather than humans. Additionally, such automated agents which watch + a room for messages and respond to them ought to ignore ``m.notice`` messages. + This helps to prevent infinite-loop situations where two automated clients + continuously exchange messages, as each responds to the other. + ``m.image`` Required keys: - ``url`` : "string" - The URL to the image. @@ -428,7 +444,7 @@ outlined below: "mimetype" : "string (e.g. image/jpeg)", } -.. TODO-spec:: +.. TODO-spec Make the definitions "inherit" from FileInfo where necessary... diff --git a/specification/11_event_signing.rst b/specification/11_event_signing.rst index 9d08f996..68f9c178 100644 --- a/specification/11_event_signing.rst +++ b/specification/11_event_signing.rst @@ -38,7 +38,7 @@ using this representation. ).encode("UTF-8") Grammar -+++++++ +~~~~~~~ Adapted from the grammar in http://tools.ietf.org/html/rfc7159 removing insignificant whitespace, fractions, exponents and redundant character escapes @@ -76,7 +76,7 @@ the signature for that sequence and then adding the signature to the original JSON object. Signing Details -+++++++++++++++ +~~~~~~~~~~~~~~~ JSON is signed by encoding the JSON object without ``signatures`` or keys grouped as ``unsigned``, using the canonical encoding described above. The JSON bytes are then signed using the @@ -133,7 +133,7 @@ and additional signatures. return json_object Checking for a Signature -++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~ To check if an entity has signed a JSON object a server does the following