Events ====== All communication in Matrix is expressed in the form of data objects called Events. These are the fundamental building blocks common to the client-server, server-server and application-service APIs, and are described below. {{common_event_fields}} {{common_room_event_fields}} {{common_state_event_fields}} Room Events ----------- .. NOTE:: This section is a work in progress. This specification outlines several standard event types, all of which are prefixed with ``m.`` {{room_events}} m.room.message msgtypes ~~~~~~~~~~~~~~~~~~~~~~~ .. TODO-spec How a client should handle unknown message types. .. TODO-spec We've forgotten m.file... .. TODO-spec It's really confusing that the m. prefix is used both for event types and for msgtypes. We should namespace them differently somehow. Each ``m.room.message`` MUST have a ``msgtype`` key which identifies the type of message being sent. Each type has their own required and optional keys, as outlined below: ``m.text`` Required keys: - ``body`` : "string" - The body of the message. Optional keys: None. Example: ``{ "msgtype": "m.text", "body": "I am a fish" }`` ``m.emote`` Required keys: - ``body`` : "string" - The emote action to perform. Optional keys: None. Example: ``{ "msgtype": "m.emote", "body": "tries to come up with a witty explanation" }`` ``m.notice`` Required keys: - ``body`` : "string" - The body of the message. Optional keys: None. Example: ``{ "msgype": "m.notice", "body": "some kind of automated announcement" }`` A ``m.notice`` message should be considered similar to a plain ``m.text`` message except that clients should visually distinguish it in some way. It is intended to be used by automated clients, such as bots, bridges, and other entities, rather than humans. Additionally, such automated agents which watch a room for messages and respond to them ought to ignore ``m.notice`` messages. This helps to prevent infinite-loop situations where two automated clients continuously exchange messages, as each responds to the other. ``m.image`` Required keys: - ``url`` : "string" - The URL to the image. Optional keys: - ``info`` : "string" - info : JSON object (ImageInfo) - The image info for image referred to in ``url``. - ``thumbnail_url`` : "string" - The URL to the thumbnail. - ``thumbnail_info`` : JSON object (ImageInfo) - The image info for the image referred to in ``thumbnail_url``. - ``body`` : "string" - The alt text of the image, or some kind of content description for accessibility e.g. "image attachment". ImageInfo: Information about an image:: { "size" : integer (size of image in bytes), "w" : integer (width of image in pixels), "h" : integer (height of image in pixels), "mimetype" : "string (e.g. image/jpeg)", } ``m.audio`` Required keys: - ``url`` : "string" - The URL to the audio. Optional keys: - ``info`` : JSON object (AudioInfo) - The audio info for the audio referred to in ``url``. - ``body`` : "string" - A description of the audio e.g. "Bee Gees - Stayin' Alive", or some kind of content description for accessibility e.g. "audio attachment". AudioInfo: Information about a piece of audio:: { "mimetype" : "string (e.g. audio/aac)", "size" : integer (size of audio in bytes), "duration" : integer (duration of audio in milliseconds), } ``m.video`` Required keys: - ``url`` : "string" - The URL to the video. Optional keys: - ``info`` : JSON object (VideoInfo) - The video info for the video referred to in ``url``. - ``body`` : "string" - A description of the video e.g. "Gangnam style", or some kind of content description for accessibility e.g. "video attachment". VideoInfo: Information about a video:: { "mimetype" : "string (e.g. video/mp4)", "size" : integer (size of video in bytes), "duration" : integer (duration of video in milliseconds), "w" : integer (width of video in pixels), "h" : integer (height of video in pixels), "thumbnail_url" : "string (URL to image)", "thumbanil_info" : JSON object (ImageInfo) } ``m.location`` Required keys: - ``geo_uri`` : "string" - The geo URI representing the location. Optional keys: - ``thumbnail_url`` : "string" - The URL to a thumnail of the location being represented. - ``thumbnail_info`` : JSON object (ImageInfo) - The image info for the image referred to in ``thumbnail_url``. - ``body`` : "string" - A description of the location e.g. "Big Ben, London, UK", or some kind of content description for accessibility e.g. "location attachment". ``m.file`` Required keys: - ``url`` : "string" - The URL for the file - ``filename`` : "string" - The original filename of the uploaded file Optional keys: - ``info`` : JSON object (FileInfo) - The info for the file referred to in ``url``. - ``thumbnail_url`` : "string" - The URL to a thumnail of the location being represented. - ``thumbnail_info`` : JSON object (ImageInfo) - The image info for the image referred to in ``thumbnail_url``. - ``body`` : "string" - A human-readable description of the file for backwards compatibility. This is recommended to be the filename of the original upload. FileInfo: Information about a file:: { "size" : integer (size of file in bytes), "mimetype" : "string (e.g. image/jpeg)", } .. TODO-spec Make the definitions "inherit" from FileInfo where necessary... Presence Events ~~~~~~~~~~~~~~~ ``m.presence`` Summary: Informs you of a user's presence state changes. Type: Presence event JSON format:: { "displayname": "utf-8 string", "avatar_url": "url", "presence": "enum [ online|unavailable|offline|free_for_chat|hidden ]", "last_active_ago": "milliseconds" } Example:: { "displayname": "Matthew", "avatar_url": "mxc://domain/id", "presence": "online", "last_active_ago": 10000 } Description: Each user has the concept of presence information. This encodes the "availability" of that user, suitable for display on other user's clients. This is transmitted as an ``m.presence`` event and is one of the few events which are sent *outside the context of a room*. The basic piece of presence information is represented by the ``presence`` key, which is an enum of one of the following: - ``online`` : The default state when the user is connected to an event stream. - ``unavailable`` : The user is not reachable at this time. - ``offline`` : The user is not connected to an event stream. - ``free_for_chat`` : The user is generally willing to receive messages moreso than default. - ``hidden`` : Behaves as offline, but allows the user to see the client state anyway and generally interact with client features. (Not yet implemented in synapse). In addition, the server maintains a timestamp of the last time it saw a pro-active event from the user; either sending a message to a room, or changing presence state from a lower to a higher level of availability (thus: changing state from ``unavailable`` to ``online`` counts as a proactive event, whereas in the other direction it will not). This timestamp is presented via a key called ``last_active_ago``, which gives the relative number of milliseconds since the message is generated/emitted that the user was last seen active. Events on Change of Profile Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Because the profile displayname and avatar information are likely to be used in many places of a client's display, changes to these fields cause an automatic propagation event to occur, informing likely-interested parties of the new values. This change is conveyed using two separate mechanisms: - a ``m.room.member`` event is sent to every room the user is a member of, to update the ``displayname`` and ``avatar_url``. - a ``m.presence`` presence status update is sent, again containing the new values of the ``displayname`` and ``avatar_url`` keys, in addition to the required ``presence`` key containing the current presence state of the user. Both of these should be done automatically by the home server when a user successfully changes their displayname or avatar URL fields. Additionally, when home servers emit room membership events for their own users, they should include the displayname and avatar URL fields in these events so that clients already have these details to hand, and do not have to perform extra roundtrips to query it. Voice over IP ------------- Matrix can also be used to set up VoIP calls. This is part of the core specification, although is at a relatively early stage. Voice (and video) over Matrix is built on the WebRTC 1.0 standard. Call events are sent to a room, like any other event. This means that clients must only send call events to rooms with exactly two participants as currently the WebRTC standard is based around two-party communication. Events ~~~~~~ ``m.call.invite`` This event is sent by the caller when they wish to establish a call. Required keys: - ``call_id`` : "string" - A unique identifier for the call - ``offer`` : "offer object" - The session description - ``version`` : "integer" - The version of the VoIP specification this message adheres to. This specification is version 0. - ``lifetime`` : "integer" - The time in milliseconds that the invite is valid for. Once the invite age exceeds this value, clients should discard it. They should also no longer show the call as awaiting an answer in the UI. Optional keys: None. Example:: { "version" : 0, "call_id": "12345", "offer": { "type" : "offer", "sdp" : "v=0\r\no=- 6584580628695956864 2 IN IP4 127.0.0.1[...]" } } ``Offer Object`` Required keys: - ``type`` : "string" - The type of session description, in this case 'offer' - ``sdp`` : "string" - The SDP text of the session description ``m.call.candidates`` This event is sent by callers after sending an invite and by the callee after answering. Its purpose is to give the other party additional ICE candidates to try using to communicate. Required keys: - ``call_id`` : "string" - The ID of the call this event relates to - ``version`` : "integer" - The version of the VoIP specification this messages adheres to. his specification is version 0. - ``candidates`` : "array of candidate objects" - Array of object describing the candidates. ``Candidate Object`` Required Keys: - ``sdpMid`` : "string" - The SDP media type this candidate is intended for. - ``sdpMLineIndex`` : "integer" - The index of the SDP 'm' line this candidate is intended for - ``candidate`` : "string" - The SDP 'a' line of the candidate ``m.call.answer`` Required keys: - ``call_id`` : "string" - The ID of the call this event relates to - ``version`` : "integer" - The version of the VoIP specification this messages - ``answer`` : "answer object" - Object giving the SDP answer ``Answer Object`` Required keys: - ``type`` : "string" - The type of session description. 'answer' in this case. - ``sdp`` : "string" - The SDP text of the session description ``m.call.hangup`` Sent by either party to signal their termination of the call. This can be sent either once the call has has been established or before to abort the call. Required keys: - ``call_id`` : "string" - The ID of the call this event relates to - ``version`` : "integer" - The version of the VoIP specification this messages Message Exchange ~~~~~~~~~~~~~~~~ A call is set up with messages exchanged as follows: :: Caller Callee m.call.invite -----------> m.call.candidate --------> [more candidates events] User answers call <------ m.call.answer [...] <------ m.call.hangup Or a rejected call: :: Caller Callee m.call.invite -----------> m.call.candidate --------> [more candidates events] User rejects call <------- m.call.hangup Calls are negotiated according to the WebRTC specification. Glare ~~~~~ This specification aims to address the problem of two users calling each other at roughly the same time and their invites crossing on the wire. It is a far better experience for the users if their calls are connected if it is clear that their intention is to set up a call with one another. In Matrix, calls are to rooms rather than users (even if those rooms may only contain one other user) so we consider calls which are to the same room. The rules for dealing with such a situation are as follows: - If an invite to a room is received whilst the client is preparing to send an invite to the same room, the client should cancel its outgoing call and instead automatically accept the incoming call on behalf of the user. - If an invite to a room is received after the client has sent an invite to the same room and is waiting for a response, the client should perform a lexicographical comparison of the call IDs of the two calls and use the lesser of the two calls, aborting the greater. If the incoming call is the lesser, the client should accept this call on behalf of the user. The call setup should appear seamless to the user as if they had simply placed a call and the other party had accepted. Thusly, any media stream that had been setup for use on a call should be transferred and used for the call that replaces it.