Flesh out IM module

specification/0-intro.rst

@@ -434,6 +434,8 @@ Some requests have unique error codes:
 :``M_LOGIN_EMAIL_URL_NOT_YET``:
   Encountered when polling for an email link which has not been clicked yet.
 
+.. _sect:txn_ids:
+
 The C-S API typically uses ``HTTP POST`` to submit requests. This means these
 requests are not idempotent. The C-S API also allows ``HTTP PUT`` to make
 requests idempotent. In order to use a ``PUT``, paths should be suffixed with

specification/modules/instant_messaging.rst

@@ -53,7 +53,14 @@ display the fallback plain text ``body`` key instead.
 Client behaviour
 ----------------
 
-Events which have attachments (e.g. ``m.image``, ``m.file``) are advised to be
+Clients SHOULD verify the structure of incoming events to ensure that the
+expected keys exist and that they are of the right type. Clients can discard
+malformed events or display a placeholder message to the user. Redacted
+``m.room.message`` events MUST be removed from the client. This can either be
+replaced with placeholder text (e.g. "[REDACTED]") or the redacted message can
+be removed entirely from the messages view.
+
+Events which have attachments (e.g. ``m.image``, ``m.file``) SHOULD be
 uploaded using the `content repository module`_ where available. The
 resulting ``mxc://`` URI can then be used in the ``url`` key. The
 attachment SHOULD be uploaded *prior* to sending the event in order to stop a
@@ -63,27 +70,87 @@ race condition where the recipient receives a link to a non-existent attachment.
 
 Recommendations when sending messages
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Advise using idempotent PUTs to send messages (and why)
-- Retries (exp backoff, giveup eventually allowing manual retry)
-- Queueing (bucket per room)
 
-Implementing local echo
-~~~~~~~~~~~~~~~~~~~~~~~
-- Local echo (document bug with races) - sending state. Pairing returned event ID.
+Clients can send messages using ``POST`` or ``PUT`` requests. Clients SHOULD use
+``PUT`` requests with `transaction IDs`_ to make requests idempotent. This
+ensures that messages are sent exactly once even under poor network conditions.
+Clients SHOULD retry requests using an exponential-backoff algorithm for a
+certain amount of time T. It is recommended that T is no longer than 5 minutes.
+After this time, the client should stop retrying and mark the message as "unsent".
+Users should be able to manually resend unsent messages.
+
+Users may type several messages at once and send them all in quick succession.
+Clients SHOULD preserve the order in which they were sent by the user. This
+means that clients should wait for the response to the previous request before
+sending the next request. This can lead to head-of-line blocking. In order to
+reduce the impact of head-of-line blocking, clients should use a queue per room
+rather than a global queue, as ordering is only relevant within a single room
+rather than between rooms.
+
+.. _`transaction IDs`: `sect:txn_ids`_
+
+Local echo
+~~~~~~~~~~
+
+Messages SHOULD appear immediately in the message view when a user presses the
+"send" button. This should occur even if the message is still sending. This is
+referred to as "local echo". Clients SHOULD implement "local echo" of messages.
+ 
+Clients need to be able to pair up the "remote echo" from the server with the
+"local echo" to prevent duplicate messages being displayed. Ideally this pairing
+would occur transparently to the user: the UI would not flicker as it transitions
+from local to remote. Flickering cannot be fully avoided in version 1 of the
+client-server API. Two scenarios need to be considered:
+
+- The client sends a message and the remote echo arrives on the event stream
+  *after* the request to send the message completes.
+- The client sends a message and the remote echo arrives on the event stream
+  *before* the request to send the message completes.
+
+In the first scenario, the client will receive an event ID when the request to
+send the message completes. This ID can be used to identify the duplicate event
+when it arrives on the event stream. However, in the second scenario, the event
+arrives before the client has obtained an event ID. This makes it impossible to
+identify it as a duplicate event. This results in the client displaying the
+message twice for a fraction of a second before the the original request to send
+the message completes. Once it completes, the client can take remedial actions
+to remove the duplicate event by looking for duplicate event IDs. Version 2 of
+the client-server API resolves this by attaching the transaction ID of the
+sending request to the event itself.
+
 
 Displaying membership information with messages
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- Member name linking (incl. pagination aka historical display names)
+Clients may wish to show the display name and avatar URL of the room member who
+sent a message. This can be achieved by inspecting the ``m.room.member`` event
+for that user ID.
+
+When a user paginates the message history, clients may wish to show the
+**historical** display name and avatar URL for a room member. This is possible
+because older ``m.room.member`` events are returned when paginating. This can
+be implemented efficiently by keeping two sets of room state: old and current.
+As new events arrive and/or the user paginates back in time, these two sets of
+state diverge from each other. New events update the current state and paginated
+events update the old state. When paginated events are processed sequentially,
+the old state represents the state of the room *at the time the event was sent*.
+This can then be used to set the historical display name and avatar URL.
 
 Server behaviour
 ----------------
 
-- SHOULD enforce the body/msgtype keys are present (can 400 them)
+Homeservers SHOULD enforce that ``m.room.message`` events have textual ``body``
+and ``msgtype`` keys by 400ing the request to send a message.
 
 Security considerations
 -----------------------
 
-- Not encrypted, link to E2E module.
-- XSS: Should sanitise ALL KEYS before injecting as unsafe HTML (name/topic/body/etc)
+Messages sent using this module are not encrypted. Encryption can be layered
+over the top of this module: where the plaintext format is an ``m.room.message``
+conforming to this module. This can be achieved using the `E2E module`_.
+
+Clients should sanitise **all keys** for unsafe HTML to prevent Cross-Site
+Scripting (XSS) attacks. This includes room names and topics.
+
+.. _`E2E module`: `module:e2e`_