archive
/
matrix-spec-proposals
mirror of https://github.com/matrix-org/matrix-spec-proposals
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge aeadae81e2 into ecf996389f

Browse Source
pull/2762/merge
Travis Ralston 2 months ago committed by GitHub
parent ecf996389f aeadae81e2
commit 4cce50b39e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 367 additions and 0 deletions
Show Stats Download Patch File Download Diff File

367
proposals/2762-widget-event-receiving.md
Unescape Escape View File

@ -0,0 +1,367 @@
# MSC2762: Allowing widgets to send/receive events
[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) originally specified a Widget API
which supports widgets being able to receive specified events from the client, and for widgets to
be able to send more than stickers.
Sticker support is already specified for widgets, though support for text and image events has been
excluded from the initial specification, as has MSC1236's event receiving support. These components
have been excluded from the specification due to lack of documentation and lack of reference
implementation to influence the spec writing process.
This proposal aims to bring the functionality originally proposed by MSC1236 into the widget
specification with the accuracy and implementation validation required by modern MSCs. Additionally,
this MSC explores options for widgets being able to see events/state for rooms in which they aren't
operating directly. An example usecase of this is a calendar system built on top of Matrix where a
calendar view might belong to a room but needs information from "calendar event rooms". The widget
would therefore need to query state from these other rooms.
## Prerequisite background
Widgets are relatively new to Matrix and so the terminology and behaviour might not be known to all
readers. This section should clarify the components of widgets that are applicable to this MSC without
going on a deep dive into widgets in general.
Widgets are embedded HTML/JS/CSS applications in a client which use the `postMessage` API to talk
to the client. This communication allows widgets to provide enhanced functionality such as sticker
pickers (when applied to a user) or performance dashboards (in rooms).
One of the first things that happens over this communication channel is a "capabilities negotiation"
where the client asks the widget what permissions it wants, and the widget replies with its ideal
set. The client then either decides or asks the user if the permissions requested are okay.
All communication over the channel is done in a simple request/response flow, using actions to
describe the request. For the capabilities negotiation, this would be the client sending the widget
a request with an `action` of `capabilities`, and the widget would respond to that request with a
response object.
The channel in which communication occurs is called a "session", where the session is "established"
after the capabilities negotiation. Sessions can only be terminated by the client.
The Widget API is split into two parts: `toWidget` (client->widget) and `fromWidget` (widget->client).
They are differentiated by where the request originates.
For a bit of background, stickers are gated by an `m.sticker` capability and have a `m.sticker`
action on the `fromWidget` API. If the widget was granted the capability and sent a valid request
to the client, the client would send an `m.sticker` event to the currently viewed room as the
user. This is all a bit confusing due to the naming of all the identifiers, but the principle
is that there's prior art for sending events from widgets.
## Proposal (sending events from widgets)
As mentioned above in the prerequisite background, sticker messages can currently be sent over the
Widget API but other events are not possible. To facilitate sending other event types to the room,
some new capabilities are introduced to allow clients to easily differentiate between custom
capabilities and custom event types (using the `m.sticker` convention could be confusing between a
capability of `com.example.event` and an event type of the same name).
The new capabilities are:
* `m.send.event:<event type>` (eg: `m.send.event:m.room.message`) - Used for sending room messages of
a given type.
* `m.send.state_event:<event type>` (eg: `m.send.state_event:m.room.topic`) - Used for sending state
events of a given type.
Being able to send other kinds of events (EDUs, account data, etc) is not currently proposed.
Clients SHOULD automatically deny `m.send.event` and `m.send.state_event` capability requests for
known event types which do not match the descriptor. For example, `m.send.event:m.room.topic` should
be denied, as should `m.send.state_event:m.room.message`.
As with capabilities negotiation already, the user SHOULD be prompted to approve these capabilities
if the widget requests them.
State events can have their capabilities requested against specific state keys as well, helping the
client limit its exposure to the room's history. This is done by appending a `#` and the state key
the capability should be against. For example, `m.send.state_event:m.room.name#` will represent an
`m.room.name` state event with an empty state key whereas `m.send.state_event:m.room.name#test` will
be an `m.room.name` state event still, though with `test` as the state key. Clients should only split
on the first `#`, so `m.room.name##test` becomes an event type of `m.room.name` and state key of `#test`.
To get around an issue where widgets would not be able to request an event type with `#` in it (because
it'll be seen as a state key), widgets can use a `\` character to escape the `#`. For example,
`org.example.\#test#hello` would be parsed as an event type of `org.example.#test` with state key `hello`.
Clients should be careful to parse `\\#` as `\#` (single escape).
`m.room.message` is the only non-state event which also makes use of this `#` system, though targeting
the `msgtype` of a `m.room.message` event instead. All the same rules apply as they do to state events,
except instead to `msgtype`. This ensures that widgets cannot interfere with encryption verification.
It is expected that most widgets looking to use this functionality will request the following:
* `m.send.event:m.room.message#m.notice`
* `m.send.event:m.room.message#m.text`
* `m.send.event:m.room.message#m.emote`
Other non-state event types with `#` in them do not get parsed in any special way, and do not need escaping.
To actually send the event, widgets would use a new `fromWidget` request with action `send_event`
which takes the following shape:
```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"state_key": "",
"type": "m.room.topic",
"content": {
"topic": "Hello world!"
}
}
}
```
Under `data`, the `state_key` is omitted if the widget is not sending a state event. The other
properties of `data` are required.
The client is responsible for encrypting the event before sending, if required by the room. The widget
should not need to be made aware of encryption or have to encrypt events.
The widget can add an additional `room_id` property to the `data` object if it would like to target
a specific room. This requires that the widget be approved for sending to that room, which is dicussed
later in this document.
If the widget did not get approved for the capability/capabilities required to send the event, the
client MUST send an error response (as required currently by the capabilities system for widgets). If
the widget has permission to send to the room, defaulting to whichever room the user is currently
viewing, the client MUST try to send the event to that room.
The client SHOULD NOT modify the `type`, `state_key`, or `content` of the request unless required for
encryption. The widget is responsible for producing valid events - the client MUST pass through any
errors, such as permission errors, to the widget using the standard error response in the Widget API.
For added clarity, the client picks either the `/send` or `/state` endpoint to use on the homeserver
depending on the presence of a `state_key` in the request data. The client then forms a request using
the `type`, `state_key`, and `content` by matching those against the endpoint's parameters, after
encryption if required.
If the event is successfully sent by the client, the client sends the following response:
```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"state_key": "",
"type": "m.room.topic",
"content": {
"topic": "Hello world!"
}
},
"response": {
"room_id": "!room:example.org",
"event_id": "$example"
}
}
```
*Note: Widget API responses are a clone of the request with an added `response` field.*
Both fields of the `response` are required and represent the room ID in which the event was sent,
and the event ID of that event.
With this new approach, the `m.sticker` capability and associated action are deprecated in favour of
this MSC. If this proposal is able to land in the specification before the widgets spec has a first
release, the `m.sticker` approach described in the prerequisite background section is not to be
included in the release (existing clients may still support it for legacy purposes).
### Special case: Redactions
Due to the `redacts` key being at the top level, [at least for now](https://github.com/matrix-org/matrix-doc/pull/2174),
clients should interpret a `redacts` in the content for `m.room.redaction` events as needing to call
the [`/redact` endpoint](https://matrix.org/docs/spec/client_server/r0.6.1#put-matrix-client-r0-rooms-roomid-redact-eventid-txnid)
on behalf of the widget.
## Proposal (receiving events in a widget)
In addition to being able to send events into the room, some widgets have an interest in reacting
to particular events that appear in the room. Using a similar approach to the sending of events,
a new capability matching `m.receive.event:<event type>` and `m.receive.state_event:<event type>`
are introduced, with the same formatting requirements as the `m.send.event` and `m.send.state_event`
capabilities above (ie: `m.receive.event:m.room.message#m.text`).
For each event type requested and approved, the client sends a `toWidget` request with action `event`
is sent to the widget with the `data` being the event itself. For example:
```json
{
"api": "toWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"type": "m.room.topic",
"sender": "@alice:example.org",
"event_id": "$example",
"room_id": "!room:example.org",
"state_key": "",
"origin_server_ts": 1574383781154,
"content": {
"topic": "Hello world!"
},
"unsigned": {
"age": 12345
}
}
}
```
The widget acknowledges receipt of this request with an empty `response` object.
The client SHOULD only send events which were received by the client *after* the session has been
established with the widget (after the widget's capabilities are negotiated). Clients are expected
to apply the same semantics as the send event capabilities: widgets don't receive `m.emote` msgtypes
unless they asked for it (and were approved), and they receive *decrypted* events.
Note that the client should also be sending the widget any events in rooms where the widget is permitted
to receive events from. The exact details of these permissions are covered later in this document.
Widgets can also read the events they were approved to receive on demand with the following `fromWidget`
API action:
```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "read_events",
"data": {
"state_key": "",
"type": "m.room.topic",
"limit": 25
}
}
```
When a `state_key` is present, the client will respond with state events matching that state key. If
`state_key` is instead a boolean `true`, the client will respond with state events of the given type
with any state key. For clarity, `"state_key": "@alice:example.org"` would return the state event with
the specified state key (there can only be one or zero), while `"state_key": true` would return any
state events of the type, regardless of state key.
To support the ability to read particular msgtypes, the widget can specify a `msgtype` in place of the
`state_key` for `m.room.message` requests.
The `type` is simply the event type to go searching for.
The `limit` is the number of events the widget is looking for. The client can arbitrarily decide to
return less than this limit, though should never return more than the limit. For example, a client
may decide that for privacy reasons a widget can only ever see the last 5 room messages - even though
the widget requested 25, it will only ever get 5 maximum back. When `limit` is not present it is
assumed that the widget wants as many events as the client will give it. When negative, the client
can reject the request with an error.
There is no recommended maximum `limit`, though clients will want to consider local limitations in
being able to send events. Web clients, for example, may be more able to send *every* event it knows
about. The default assumption is that the client will send over as much as possible as an upper limit.
The client is not required to backfill (use the `/messages` endpoint) to get more events for the
widget, and is able to return less than the requested amount of events. When returning state events,
the client should always return the current state event (in the client's view) rather than the history
of an event. For example, `{"type":"m.room.topic", "state_key": "", "limit": 5}` should return zero
or one topic events, not 5, even if the topic has changed more than once.
An optional `room_ids` property may also be added to the `data` object by the widget, indicating which
room(s) to listen for events in. This is either an array of room IDs, undefined, or the special string
`"*"` to denote "any room in which the widget has permission for reading that event" (covered later).
When undefined, the client should send events sent in the user's currently viewed room only.
The client's response would look like so (note that because of how Widget API actions work, the request
itself is repeated in the response - the actual response from the client is held within the `response`
object):
```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "read_events",
"data": {
"state_key": "",
"type": "m.room.topic",
"limit": 25
},
"response": {
"events": [
{
"type": "m.room.topic",
"sender": "@alice:example.org",
"event_id": "$example",
"room_id": "!room:example.org",
"state_key": "",
"origin_server_ts": 1574383781154,
"content": {
"topic": "Hello world!"
},
"unsigned": {
"age": 12345
}
}
]
}
}
```
The `events` array is simply the array of events requested. When no matching events are found, this
array must be defined but can be empty.
## Proposal (accessing other rooms)
As mentioned earlier in this MSC, widgets are typically limited to the room in which the user is currently
viewing - they cannot typically reach out into other rooms or see what other rooms are out there. This
has limitations on certain kinds of widgets which rely on room structures to store data outside of a
single canonical room, however.
To complement the send/receive event capabilities, a single capability is introduced to access the timelines
of other rooms: `m.timeline:<Room ID>`. The `<Room ID>` can either be an actual room ID, or a `*` to denote
all joined or invited rooms the client is able to see, current and future. The widget can limit its exposure
by simply requesting highly scoped send/receive capabilities to accompany the timeline capability.
Do note that a widget does not need to request capabilities for all rooms if it only ever interacts with the
user's currently viewed room. Widgets such as stickerpickers will not need to request timeline capabilities
because they'll always send events to the user's currently viewed room, and the client will let them do that
without special room timeline permissions.
There is no Widget API action exposed for listing the user's invited/joined rooms: the widget can request
permission to read/receive the `m.room.create` state event of rooms and query that way. Clients should be
aware of this trick and describe the situation appropriately to users.
## Alternatives
Widgets could be powered by a bot or some sort of backend which allows them to filter the room state
and timeline themselves, however this can be a large amount of infrastructure for a widget to maintain
and the user experience is not as great. The client already has most of the information a widget would
need, and trying to interact through a bot would generally mean slower response times or technical
challenges on the part of the widget developer.
## Security considerations
Because the widget can implicitly decrypt room history, it is absolutely imperative that clients
prompt for permission to use these capabilities even though the capabilities negotation does not
require this to be done. Clients which approve the capabilities proposed by this MSC without
asking the user first are strongly frowned upon. There are very few use cases where not asking for
the user's permission is valid.
This MSC allows widgets to arbitrarily read history from a room without the user necessarily knowing.
Clients should apply strict limits to the number of events they are willing to provide to widgets
and ensure that users are prompted to explicitly approve the permissions requested, like in MSC2762.
Clients may also wish to consider putting iconography next to room messages when a widget reads them.
This MSC allows widgets to arbitrarily access/modify history in, at worst, all of the user's rooms.
Clients should apply strict limits or checks to ensure the user understands what the widget is trying
to do and isn't unreasonably accessing the user's account. For example, a large warning saying that
a room-based widget is trying to access messages in *all rooms* might be suitable. Another approach
might be to simply limit the number of rooms a widget can access, requiring the widget to know what
room IDs it specifically wants (ie: denying the `*` request on behalf of the user).
## Unstable prefix
While this MSC is not present in the spec, clients and widgets should:
* Use `org.matrix.msc2762.` in place of `m.` in all new identifiers of this MSC.
* Only call/support the `action`s if an API version of `org.matrix.msc2762` is advertised.
Write Preview
Loading…
Cancel
Save