From 8cc34dfc550bf36ef7a71ac0da08a1ad8fd0046e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:24:06 +0100 Subject: [PATCH 01/77] Simplified sliding sync --- proposals/4186-simplified-sliding-sync.md | 345 ++++++++++++++++++++++ 1 file changed, 345 insertions(+) create mode 100644 proposals/4186-simplified-sliding-sync.md diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md new file mode 100644 index 000000000..41b4382e2 --- /dev/null +++ b/proposals/4186-simplified-sliding-sync.md @@ -0,0 +1,345 @@ +# MSC4186: Simplified Sliding Sync + +The current `/sync` endpoint scales badly as the number of rooms on an account increases. It scales badly because all +rooms are returned to the client, incremental syncs are unbounded and slow down based on how long the user has been +offline, and clients cannot opt-out of a large amount of extraneous data such as receipts. On large accounts with +thousands of rooms, the initial sync operation can take tens of minutes to perform. This significantly delays the +initial login to Matrix clients, and also makes incremental sync very heavy when resuming after any significant pause in +usage. + +Note: this is a “simplified” version of the sliding sync API proposed in +[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575), based on paring back that API based +on real world use cases and usages. + + +# Goals + +This improved `/sync` mechanism has a number of goals: + +- Sync time should be independent of the number of rooms you are in. +- Time from launch to confident usability should be as low as possible. +- Time from login on existing accounts to usability should be as low as possible. +- Bandwidth should be minimized. +- Support lazy-loading of things like read receipts (and avoid sending unnecessary data to the client) +- Support informing the client when room state changes from under it, due to state resolution. +- Clients should be able to work correctly without ever syncing in the full set of rooms they’re in. +- Don’t incremental sync rooms you don’t care about. +- Servers should not need to store all past since tokens. If a since token has been discarded we should gracefully + degrade to initial sync. + +These goals shaped the design of this proposal. + + +# Proposal + +The core differences between sync v2 and simplified sliding sync are: + +- The server initially only sends the most recent N rooms to the client (where N is specified by the lcient), which then + can paginate in older rooms in subsequent requests +- The client can configure which information the server will return for different sets of rooms (e.g. a smaller timeline + limit for older rooms). +- The client can filter what rooms it is interested in +- The client can maintain multiple sync loops (with some caveats) + +The basic operation is similar between sync v2 and simplified sliding sync: both use long-polling with tokens to fetch +updates from the server. I.e., the basic operation of both APIs is to do an “initial” request and then repeatedly call +the API supplying the token returned in the previous response in the subsequent “incremental” request. + + +## Lists and room subscriptions + +The core component of a sliding sync request is “lists”, which specify what information to return about which rooms. +Each list specifies some filters on rooms (e.g. ignore spaces), the range of filtered rooms to select (e.g. the most +recent 20 filtered rooms), and the config for the data to return for those rooms (e.g. the required state, timeline +limit, etc). The order of rooms is always done based on when the server received the most recent event for the room. + +The client can also specify config for specific rooms if it has their room ID, these are known as room subscriptions. + +Multiple lists and subscriptions can be specified in a request. If a room matches multiple lists/subscriptions then the +config is “combined” to be the superset of all configs (e.g. take the maximum timeline limit). See below for the exact +algorithm. + +The server tracks what data has been sent to the client in which rooms. If a room matches a list or subscription that +hasn’t been sent down before, then the server will respond with the full metadata about the room indicated by `initial: +true`. If a room stops matching a list (i.e. it falls out of range) then no further updates will be sent until it starts +matching a list again, at which point the missing updates (limited by the `timeline_limit`) will be sent down. However, +as clients are now expected to paginate all rooms in the room list in the background (in order to correctly order and +search them), the act of a room falling out of range is a temporary edge-case. + + +## Pagination + +Pagination is achieved by the client increasing the ranges of one (or more) lists. + +For example an initial request might have a list called `all_rooms` specifying a range of `0..20` in the initial +request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client +may change the range to `0..100`, at which point the server will respond with the top 100 rooms that either a) weren’t +sent down in the first request, or b) have updates since the first request. + +Clients can increase and decrease the ranges as they see fit. A common approach would be to start with a small window +and grow that until the range covers all the rooms. After some threshold of the app being offline it may reduce the +range back down and incrementally grow it again. This allows for ensuring that a limited amount of data is requested at +once, to improve response times. + + +## Connections + +Clients can have multiple “connections” (i.e. sync loops) with the server, so long as each connection has a different +`conn_id` set in the request. + +Clients must only have a single request in-flight at any time per connection (clients can have multiple connections by +specifying a unique `conn_id`). If a client needs to send another request before receiving a response to an in-flight +request (e.g. for retries or to change parameters) the client *must* cancel the in-flight request and *not* process any +response it receives for it. + +In particular, a client must use the returned `pos` value in a response as the `since` param in exactly one request that +the client will process the response for. Clients must be careful to ensure that when processing a response any new +requests use the new `pos`, and any in-flight requests using an old `pos` are canceled. + +The server cannot assume that a client has received a response until it receives a new request with the `since` token +set to the `pos` it returned in the response. The server must ensure that any per-connection state it tracks correctly +handles receiving multiple requests with the same `since` token (e.g. the client retries the request or decides to +cancel and resend a request with different parameters). + +A server may decide to “expire” connections, either to free resources or because the server thinks it would be faster +for the client to start from scratch (e.g. because there are many updates to send down). This is done by responding with +a 400 HTTP status and an error code of `M_UNKNOWN_POS`. + + +## List configuration + +**TODO**, these are the same as in [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): + +- Required state format +- The filters +- Lazy loading of members +- Combining room config + + +## Room config changes + +When a room comes in and out of different lists or subscriptions, the effective `timeline_limit` and `required_state` +parameters may change. This section outlines how the server should handle these cases. + +If the `timeline_limit` *increases* then the server *may* choose to send down more historic data. This is to support the +ability to get more history for certain rooms, e.g. when subscribing to the currently visible rooms in the list to +precache their history. This is done by setting `unstable_expanded_timeline` to true and sending down the last N events +(this may include events that have already been sent down). The server may choose not to do this if it believes it has +already sent down the appropriate number of events. + +If new entries are added to `required_state` then the server must send down matching current state events. + + +## Extensions + +We anticipate that as more features land in Matrix, different kinds of data will also want to be synced to clients. Sync +v2 did not have any first-class support to opt-in to new data. Sliding Sync does have support for this via "extensions". +Extensions also allow this proposal to be broken up into more manageable sections. Extensions are requested by the +client in a dedicated extensions block. + +In an effort to reduce the size of this proposal, extensions will be done in separate MSCs. There will be extensions +for: + +- To Device Messaging \- MSC3885 +- End-to-End Encryption \- MSC3884 +- Typing Notifications \- MSC3961 +- Receipts \- MSC3960 +- Presence \- presence in sync v2: spec +- Account Data \- account\_data in sync v2: MSC3959 +- Threads + +**TODO** explain how these interact with the room lists, this is the same as in +[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) + +## Request format + +```javascript +{ + "conn_id": "", // Client chosen ID of the connection, c.f. "Connections" + + // The set of room lists + "lists": { + // An arbitrary string which the client is using to refer to this list for this connection. + "": { + + // Sliding window ranges, c.f. Lists and room subscriptions + "ranges": [[0, 10]], + + // Filters to apply to the list. + "filters": { + // Flag which only returns rooms present (or not) in the DM section of account data. + // If unset, both DM rooms and non-DM rooms are returned. If false, only non-DM rooms + // are returned. If true, only DM rooms are returned. + "is_dm": true|false|null, + + // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, + // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms + // are returned. If true, only encrypted rooms are returned. + "is_encrypted": true|false|null, + + // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, + // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms + // are returned. If true, only encrypted rooms are returned. + "is_invite": true|false|null, + + // If specified, only rooms where the `m.room.create` event has a `type` matching one + // of the strings in this array will be returned. If this field is unset, all rooms are + // returned regardless of type. This can be used to get the initial set of spaces for an account. + // For rooms which do not have a room type, use 'null' to include them. + "room_types": [ ... ], + + // Same as "room_types" but inverted. This can be used to filter out spaces from the room list. + // If a type is in both room_types and not_room_types, then not_room_types wins and they are + // not included in the result. + "not_room_types": [ ... ], + }, + + // The maximum number of timeline events to return per response. + "timeline_limit": 10, + + // Required state for each room returned. An array of event type and state key tuples. + // Elements in this array are ORd together to produce the final set of state events + // to return. One unique exception is when you request all state events via ["*", "*"]. When used, + // all state events are returned by default, and additional entries FILTER OUT the returned set + // of state events. These additional entries cannot use '*' themselves. + // For example, ["*", "*"], ["m.room.member", "@alice:example.com"] will _exclude_ every m.room.member + // event _except_ for @alice:example.com, and include every other state event. + // In addition, ["*", "*"], ["m.space.child", "*"] is an error, the m.space.child filter is not + // required as it would have been returned anyway. + "required_state": [ ... ], + } + }, + + // The set of room subscriptions + "room_subscriptions": { + // The key is the room to subscribe to. + "!foo:example.com": { + // These have the same meaning as in `lists` section + "timeline_limit": 10, + "required_state": [ ... ], + } + }, + + // c.f. "Extensions" + "extensions": { + } +} +``` + + +## Response format + +```javascript +{ + // The position to use as the `since` token in the next sliding sync request. + // c.f. Connections. + "pos": "", + + // Information about the lists supplied in the request. + "lists": { + // Matches the list name supplied by the client in the request + "" { + // The total number of rooms that match the list's filter. + "count": 1234, + } + }, + + // Aggregated rooms from lists and room subscriptions. There will be one entry per room, even if + // the room appears in multiple lists and/or room subscriptions. + "rooms": { + "!foo:example.com": { + // The room name, if one exists. Only sent initially and when it changes. + "name": str|null, + // The room avatar, if one exists. Only sent initially and when it changes. + "avatar_url": str|null, + // The "heroes" for the room, if there is no room name. Only sent initially and when it changes. + "heroes": [ + {"user_id":"@alice:example.com","displayname":"Alice","avatar_url":"mxc://..."}, + ], + + // Flag which is set when this is the first time the server is sending this data on this connection. + // When set the client must replace any stored metadata for the room with the new data. In + // particular, the state must be replaced with the state in `required_state`. + "initial": true|null, + + // Same as in sync v2. Indicates whether there are more events to fetch than those in the timeline. + "limited:" true|null, + // Indicates if we have "expanded" the timeline due to the timeline_limit changing, c.f. Room config + // changes above. + "unstable_expanded_timeline": true|null, + // The list of events, sorted least to most recent. + "timeline": [ ... ], + // The current state of the room as a list of events + "required_state": [ ... ], + // The number of timeline events which have just occurred and are not historical. + // The last N events are 'live' and should be treated as such. + "num_live": 1, + // Same as sync v2, passed to `/messages` to fetch more past events. + "prev_batch": "...", + + // For invites this is the stripped state of the room at the time of invite + "invite_state": [ .. ], + + // For knocks this is the stripped state of the room at time of knock + "knock_state": [ .. ], + + // Whether the room is a DM room. + "is_dm": true|null, + + // An opaque integer that can be used to sort the rooms by "Bump Stamp" + "bump_stamp": 1, + + // These are the same as sync v2. + "joined_count": 1, + "invited_count": 1, + "notification_count": 1, + "highlight_count": 1, + } + }, + + "extensions": { + }, +} +``` + + +# Alternatives / changes + +There are a number of potential changes that we could make. + +## Pagination + +In practice, having the client specify the ranges to use for the lists is often sub-optimal. The client generally wants +to have the sync request return as quickly as possible, but it doesn't know how much data the server has to return and +so whether to increase or decrease the range. + +An alternative is for the client to specify a `page_size`, where the server sends down at most `page_size` number of +rooms. If there are more rooms to send to the client (beyond `page_size`), then the client can request to "paginate" in +these missed updates in subsequent updates. + +Since this would require client side changes, this should be explored in a separate MSC. + +## Timeline event trickling + +If the `timeline_limit` is increased then the server will send down historic data (c.f. "Room config changes"), which +allows the clients to easily preload more history in recent rooms. + +This mechanism is fiddly to implement, and ends up resending down events that we have previously sent to the client. + +A simpler alternative is to use `/messages` to fetch the history. This has two main problems: 1) clients generally want +to preload history for multiple rooms at once, and 2) `/messages` can be slow if it tries to backfill over federation. + +We could implement a bulk `/messages` endpoint, where the client would specify multiple rooms and `prev_batch` tokens. +We can also add a flag to disable attempting to backfill over pagination (to match the behaviour of the sync timeline). + +## `required_state` response format + +The format of returned state in `required_state` is a list of events. This does now allow the server to indicate if a +"state reset" has happened which removed an entry from the state entirely (rather than it being replaced with another +event). + +This is particularly problematic if the user gets "state reset" out of the room, where the server has no mechanism to +indicate to the client that the user has effectively left the room (the server has no leave event to return). + +We may want to allow special entries in the `required_state` list of the form +`{"type": .., "state_key": .., content: null}` to indicate that the state entry has been removed. From 456db3b9d3b936457c9e780e439e4e6cd30f28a9 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:52:28 +0100 Subject: [PATCH 02/77] Add example usage --- proposals/4186-simplified-sliding-sync.md | 27 +++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 41b4382e2..9c6d7379d 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -302,6 +302,33 @@ for: } ``` +# Example usage + +This section gives an example of how a client can use this API (roughly based on how Element X currently uses the API). + +When the app starts up it configures a single list with a range of `[0, 19]` (to get the top 20 rooms) and a +`timeline_limit` of 1. This returns quickly with the top 20 rooms (or just the changes in the top 20 rooms if a token +was specified). + +The client then increases the range (in the next request) to `[0, 99]`, which will return the next 80 rooms. The server +may sort the rooms differently than they are returned by the server (e.g. they may ignore reactions for sorting +purposes). + +The client can use room subscriptions, with a `timeline_limit` of 20, to preload history for the top rooms. This means +that if the user clicks on one of the top rooms the app can immediately display a screens worth of history. (An +alternative would be to have a second list with a static range of `[0, 19]` and a `timeline_limit` of 20. The downside) + +The client can keep increasing the list range in increments to pull in the full list of rooms. The client uses the +returned `count` for the list to know when to stop expanding the list. + +The client *may* decided to reduce the range back to `[0, 19]` (and then subsequently incrementally expand the range), +this can be done. + +When the client is expecting a fast response (e.g. while expanding the lists), it should set the `timeout` parameter to +0 to ensure the server doesn't block waiting for new data. This can easily happen if the app starts and sends the first +request with a `since` parameter, if the client shows a spinner but doesn't set a timeout then the request may take a +long time to return (if there were no updates to return). + # Alternatives / changes From 6fe4ba72da515f3e473be42a57a7a32f03e303f8 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 30 Aug 2024 10:59:21 +0100 Subject: [PATCH 03/77] Add security and unstable prefix --- proposals/4186-simplified-sliding-sync.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 9c6d7379d..275512a7c 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -370,3 +370,15 @@ indicate to the client that the user has effectively left the room (the server h We may want to allow special entries in the `required_state` list of the form `{"type": .., "state_key": .., content: null}` to indicate that the state entry has been removed. + + +# Security considerations + +Care must be taken, as with sync v2, to ensure that only the data that the user is authorized to see is returned in the +response. + + +# Unstable prefix + +The unstable URL for simplified sliding sync is `/org.matrix.simplified_msc3575/sync`. The flag in `/versions` is +`org.matrix.simplified_msc3575`. From a5dc74bc409900c23cabf689857c9398e1b78ad0 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 8 Jan 2025 14:40:19 +0000 Subject: [PATCH 04/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Johannes Marbach --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 275512a7c..4f7c729d5 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -34,7 +34,7 @@ These goals shaped the design of this proposal. The core differences between sync v2 and simplified sliding sync are: -- The server initially only sends the most recent N rooms to the client (where N is specified by the lcient), which then +- The server initially only sends the most recent N rooms to the client (where N is specified by the client), which then can paginate in older rooms in subsequent requests - The client can configure which information the server will return for different sets of rooms (e.g. a smaller timeline limit for older rooms). From 4c47844771bb160a4efefe4bd063efd33902df73 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 8 Jan 2025 15:21:11 +0000 Subject: [PATCH 05/77] Some clarifications --- proposals/4186-simplified-sliding-sync.md | 30 ++++++++++++++--------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 4f7c729d5..a2896889f 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -17,7 +17,7 @@ on real world use cases and usages. This improved `/sync` mechanism has a number of goals: - Sync time should be independent of the number of rooms you are in. -- Time from launch to confident usability should be as low as possible. +- Time from opening of the app (when already logged in) to confident usability should be as low as possible. - Time from login on existing accounts to usability should be as low as possible. - Bandwidth should be minimized. - Support lazy-loading of things like read receipts (and avoid sending unnecessary data to the client) @@ -40,6 +40,8 @@ The core differences between sync v2 and simplified sliding sync are: limit for older rooms). - The client can filter what rooms it is interested in - The client can maintain multiple sync loops (with some caveats) + - This is useful for e.g. iOS clients which have a separate process to deal with notifications, as well as allowing + the app to split handling of things like encryption entirely from room data. The basic operation is similar between sync v2 and simplified sliding sync: both use long-polling with tokens to fetch updates from the server. I.e., the basic operation of both APIs is to do an “initial” request and then repeatedly call @@ -89,8 +91,8 @@ Clients can have multiple “connections” (i.e. sync loops) with the server, s Clients must only have a single request in-flight at any time per connection (clients can have multiple connections by specifying a unique `conn_id`). If a client needs to send another request before receiving a response to an in-flight -request (e.g. for retries or to change parameters) the client *must* cancel the in-flight request and *not* process any -response it receives for it. +request (e.g. for retries or to change parameters) the client *must* cancel the in-flight request (at the HTTP level) +and *not* process any response it receives for it. In particular, a client must use the returned `pos` value in a response as the `since` param in exactly one request that the client will process the response for. Clients must be careful to ensure that when processing a response any new @@ -177,9 +179,9 @@ for: // are returned. If true, only encrypted rooms are returned. "is_encrypted": true|false|null, - // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, - // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms - // are returned. If true, only encrypted rooms are returned. + // Flag which only returns rooms the user is currently invited to. If unset, both invited + // and joined rooms are returned. If false, no invited rooms are returned. If true, only + // invited rooms are returned. "is_invite": true|false|null, // If specified, only rooms where the `m.room.create` event has a `type` matching one @@ -239,7 +241,8 @@ for: "lists": { // Matches the list name supplied by the client in the request "" { - // The total number of rooms that match the list's filter. + // The total number of rooms that match the list's filter. Note that rooms can be in + // multiple lists, so may be double counted. "count": 1234, } }, @@ -248,7 +251,8 @@ for: // the room appears in multiple lists and/or room subscriptions. "rooms": { "!foo:example.com": { - // The room name, if one exists. Only sent initially and when it changes. + // The room name (as specified by any `m.room.name` event), if one exists. Only sent initially + // and when it changes. "name": str|null, // The room avatar, if one exists. Only sent initially and when it changes. "avatar_url": str|null, @@ -269,7 +273,8 @@ for: "unstable_expanded_timeline": true|null, // The list of events, sorted least to most recent. "timeline": [ ... ], - // The current state of the room as a list of events + // The current state of the room as a list of events. This is the full state if `initial` + // state is set, otherwise it is a delta from the previous sync. "required_state": [ ... ], // The number of timeline events which have just occurred and are not historical. // The last N events are 'live' and should be treated as such. @@ -312,11 +317,14 @@ was specified). The client then increases the range (in the next request) to `[0, 99]`, which will return the next 80 rooms. The server may sort the rooms differently than they are returned by the server (e.g. they may ignore reactions for sorting -purposes). +purposes). Note: the range here matches 100 rooms, however we only send the 80 rooms that we didn't send down in the +previous request. The client can use room subscriptions, with a `timeline_limit` of 20, to preload history for the top rooms. This means that if the user clicks on one of the top rooms the app can immediately display a screens worth of history. (An -alternative would be to have a second list with a static range of `[0, 19]` and a `timeline_limit` of 20. The downside) +alternative would be to have a second list with a static range of `[0, 19]` and a `timeline_limit` of 20. The downside +is that the clients may use a different order for the room list and so always fetching extra events for the top 20 rooms +may return more data than required.) The client can keep increasing the list range in increments to pull in the full list of rooms. The client uses the returned `count` for the list to know when to stop expanding the list. From 3fc851ad02dd878f4c744da11827b44284e385e8 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:08:05 +0100 Subject: [PATCH 06/77] Rewrite MSC to include full API description This is a big rewrite of the MSC that includes a full API definition. --- proposals/4186-simplified-sliding-sync.md | 676 +++++++++++++--------- 1 file changed, 417 insertions(+), 259 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index a2896889f..69554339b 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -7,14 +7,24 @@ thousands of rooms, the initial sync operation can take tens of minutes to perfo initial login to Matrix clients, and also makes incremental sync very heavy when resuming after any significant pause in usage. -Note: this is a “simplified” version of the sliding sync API proposed in -[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575), based on paring back that API based -on real world use cases and usages. +> [!Note] +> This is a “simplified” version of the sliding sync API proposed in +> [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575), based on paring back that API based on real +> world use cases and usages. +## High level overview -# Goals +From a high level, sliding sync works by the client specifying a set of rules to match rooms (via "lists" and +"subscriptions"). The server will use the rules to match relevant rooms to the user, and return any rooms that have had +changes since the previous time the room was returned to the client. If no rooms have changes, the server will block +waiting for a change (aka long-polling, like sync v2). -This improved `/sync` mechanism has a number of goals: +By judicious use of lists and subscriptions the client can control exactly what data is returned when, and help ensure +that it doesn't request "too much" data at once (thus avoiding slow responses). + +## Goals + +During initial design, the following goals were taken into account: - Sync time should be independent of the number of rooms you are in. - Time from opening of the app (when already logged in) to confident usability should be as low as possible. @@ -27,287 +37,422 @@ This improved `/sync` mechanism has a number of goals: - Servers should not need to store all past since tokens. If a since token has been discarded we should gracefully degrade to initial sync. -These goals shaped the design of this proposal. - # Proposal -The core differences between sync v2 and simplified sliding sync are: +We add a new `POST` `/sync` API to replace the existing `GET` `/sync` API. Clients use the new API in a similar manner +to the old one; the client repeatedly calls `/sync` which will return with any new data. -- The server initially only sends the most recent N rooms to the client (where N is specified by the client), which then - can paginate in older rooms in subsequent requests -- The client can configure which information the server will return for different sets of rooms (e.g. a smaller timeline - limit for older rooms). -- The client can filter what rooms it is interested in -- The client can maintain multiple sync loops (with some caveats) - - This is useful for e.g. iOS clients which have a separate process to deal with notifications, as well as allowing - the app to split handling of things like encryption entirely from room data. +See the API section below for the request and response schemas. -The basic operation is similar between sync v2 and simplified sliding sync: both use long-polling with tokens to fetch -updates from the server. I.e., the basic operation of both APIs is to do an “initial” request and then repeatedly call -the API supplying the token returned in the previous response in the subsequent “incremental” request. +## Connections +A "connection" (aka "sync loop") is a long-running set of sync requests where each new request uses the `pos` from the +previous request. Clients can have multiple connections with the server, so long as each connection has a different +`conn_id` set in the request. -## Lists and room subscriptions +Clients must only have a single active request in-flight at any time per connection. If a client needs to send another +request before receiving a response to an in-flight request (e.g. for retries or to change parameters) the client *must* +cancel the in-flight request (at the HTTP level) and *not* process any response it receives for it. Clients MAY change +the request body if they cancel a request and send a new one with the same `pos`. -The core component of a sliding sync request is “lists”, which specify what information to return about which rooms. -Each list specifies some filters on rooms (e.g. ignore spaces), the range of filtered rooms to select (e.g. the most -recent 20 filtered rooms), and the config for the data to return for those rooms (e.g. the required state, timeline -limit, etc). The order of rooms is always done based on when the server received the most recent event for the room. +In particular, a client must use the returned `pos` value from the last *processed* response as the `pos` parameter in +the next request. A `pos` from an older response MUST NOT be used again. Doing so may either result in missed data or +result in a fatal error response from the server. -The client can also specify config for specific rooms if it has their room ID, these are known as room subscriptions. +The server cannot assume that a client has received a response until it receives a new request with the `pos` token set +to the `pos` it returned in the response. The server must ensure that any per-connection state it tracks correctly +handles receiving multiple requests with the latest `pos` token (e.g. the client retries the request or decides to cancel +and resend a request with different parameters). Once a server has seen a request with a given `pos`, the server may +clean up any per-connection state from before that `pos`. -Multiple lists and subscriptions can be specified in a request. If a room matches multiple lists/subscriptions then the -config is “combined” to be the superset of all configs (e.g. take the maximum timeline limit). See below for the exact -algorithm. +A server may decide to "expire" connections, either to free resources or because the server thinks it would be faster +for the client to start from scratch (e.g. because there are many updates to send down). This is done by responding with +a 400 HTTP status and an error code of `M_UNKNOWN_POS`. -The server tracks what data has been sent to the client in which rooms. If a room matches a list or subscription that -hasn’t been sent down before, then the server will respond with the full metadata about the room indicated by `initial: -true`. If a room stops matching a list (i.e. it falls out of range) then no further updates will be sent until it starts -matching a list again, at which point the missing updates (limited by the `timeline_limit`) will be sent down. However, -as clients are now expected to paginate all rooms in the room list in the background (in order to correctly order and -search them), the act of a room falling out of range is a temporary edge-case. +## Lists and subscriptions -## Pagination +Clients specify a set of rules that the server uses to determine which rooms the client is interested in receiving. A +room is returned in the response if all of the following conditions hold: +1. The user has permission to view the room (or some subset of information about it). +1. The room matches at least one of the rules in the request. +1. The room has either never been sent to the client on that connection before, or there have been updates to the room + since the last time it was sent to the client. -Pagination is achieved by the client increasing the ranges of one (or more) lists. +This MSC specifies two types of rule: a "list" and a "subscription". -For example an initial request might have a list called `all_rooms` specifying a range of `0..20` in the initial -request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client -may change the range to `0..100`, at which point the server will respond with the top 100 rooms that either a) weren’t -sent down in the first request, or b) have updates since the first request. +### Lists -Clients can increase and decrease the ranges as they see fit. A common approach would be to start with a small window -and grow that until the range covers all the rooms. After some threshold of the app being offline it may reduce the -range back down and incrementally grow it again. This allows for ensuring that a limited amount of data is requested at -once, to improve response times. +Lists are the primary way of specifying the rooms the client is interested in. They operate against a sequence of rooms +for the user that the server maintains. This server list is, for each user, the set of rooms that the user either +joined, been invited, knocked on, left, or been kicked or banned from, sorted by recent activity (see below for exact +ordering semantics). +Rooms that the user has been in but left are only included if the room was previously sent to the client in that +connection. Rooms the user has been kicked or banned from will always be included. We do not include rooms the user has +left to save bandwidth and general efficiency (as the user knows they've left), but we still include kicked and banned +rooms as a) this should be uncommon, and b) the user may not have seen that they've been kicked/banned from the room +otherwise. -## Connections +A "list" is then a set of filters (e.g. only match invites, or DM rooms, etc) plus a "range" that indexes into the +*filtered* list of rooms. For example, a common list config would be no filters (i.e. all rooms) plus the range +`[0,19]`, which would cause the server to return the top 20 rooms (by activity). -Clients can have multiple “connections” (i.e. sync loops) with the server, so long as each connection has a different -`conn_id` set in the request. +Specifically, a room matches a given list if after filtering the server maintained list of rooms by the list's filters, +the room's index in the filtered list is within the list's range. -Clients must only have a single request in-flight at any time per connection (clients can have multiple connections by -specifying a unique `conn_id`). If a client needs to send another request before receiving a response to an in-flight -request (e.g. for retries or to change parameters) the client *must* cancel the in-flight request (at the HTTP level) -and *not* process any response it receives for it. +#### Activity ordering -In particular, a client must use the returned `pos` value in a response as the `since` param in exactly one request that -the client will process the response for. Clients must be careful to ensure that when processing a response any new -requests use the new `pos`, and any in-flight requests using an old `pos` are canceled. +Rooms are ordered by last activity, based on when the last event in the room was received by the server. The exact +ordering is determined by the server implementation. -The server cannot assume that a client has received a response until it receives a new request with the `since` token -set to the `pos` it returned in the response. The server must ensure that any per-connection state it tracks correctly -handles receiving multiple requests with the same `since` token (e.g. the client retries the request or decides to -cancel and resend a request with different parameters). +> [!Important] +> Rooms are ***not*** ordered by "`bump_stamp`", a field returned in the room response. -A server may decide to “expire” connections, either to free resources or because the server thinks it would be faster -for the client to start from scratch (e.g. because there are many updates to send down). This is done by responding with -a 400 HTTP status and an error code of `M_UNKNOWN_POS`. +### Subscriptions + +Subscriptions are simply the room ID to match. This is useful if e.g. the user has opened the room and the client always wants to get the latest data for that room. + +The server MUST ensure that user has permission to see any information the server returns. However, the user need not be +in the room, e.g.clients can specify room IDs for world-readable rooms and they would be returned. + +> [!Note] +> Synapse currently does not support subscribing to rooms that the user is not part of. + +### Room config + +Each rule also specifies a "room config", used to configure what data to return for a room that matches the rule. The +"room config" has two fields: the "timeline limit" and "required state". The "timeline limit" specifies the maximum +number of events to return in the timeline section of a room, and the "required state" specifies what state to return. + +## Room results + +A room is returned in the response if it matches at least one rule, and there is new data to return (if the room has +previously been sent to the client). + +See the API section below for exactly what is returned. A subset may be returned if the user does not have permission to +view the room, e.g. if they are invited but not yet joined to the room. + +The server MUST not send any room information down that the user does not have permission to see. Specifically, the +server should only return rooms the user: is or has previously been joined to, is invited to, or has knocked on. An +exception is for rooms that are world readable and the user has subscribed to. + +The number of events in the timeline and what state is returned depends on the "room config" specified in the rules that +the room matches from the request. If a given room matches multiple rules and therefore multiple room configs, then the +room configs are combined (to be a superset) based on the rules below. The clients can also change room configs between +requests, see below for the semantics. + +The data returned in the response is only the data that has changed, e.g. if the room name hasn't changed then the +`name` field will only be sent down the first sync with the room in it and not subsequently. Clients can detect if the +data returned is the full response or a delta based on the `initial` flag. + + +### Combining room configs + +If a room matches multiple rules, and therefore multiple room configs, then the room configs must be combined into one +before being applied. -## List configuration +The fields are combined by: +- Timeline limit — take the maximum timeline limit across all room configs. +- Required state — take the superset of required, i.e. if a state event would be returned by any room config it is returned by the combined room config. -**TODO**, these are the same as in [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): -- Required state format -- The filters -- Lazy loading of members -- Combining room config +### Changing room configs +When a room matches one or more rules (i.e. is eligible to be returned in the sync response) that has previously been +returned to the client, the server checks whether the combined room config is different than when the room was last +eligible to be returned. If the new config is a superset of the previous config, then the server handles the config +differently. -## Room config changes +#### Timeline events -When a room comes in and out of different lists or subscriptions, the effective `timeline_limit` and `required_state` -parameters may change. This section outlines how the server should handle these cases. +Normally the timeline events returned are only the events that have been received since the last time the room was sent +to the client (i.e. only new events). However, if the `timeline_limit` has increased (to say `N`) the server SHOULD +ignore this and send down the latest `N` events, even if some of those events have previously been sent. The server MAY +ignore behaviour if the server knows it has previously sent down all of the latest `N` events. -If the `timeline_limit` *increases* then the server *may* choose to send down more historic data. This is to support the -ability to get more history for certain rooms, e.g. when subscribing to the currently visible rooms in the list to -precache their history. This is done by setting `unstable_expanded_timeline` to true and sending down the last N events -(this may include events that have already been sent down). The server may choose not to do this if it believes it has -already sent down the appropriate number of events. +> [!IMPORTANT] +> The server should return rooms that have expanded timelines immediately, rather than waiting for the next update to +the room. -If new entries are added to `required_state` then the server must send down matching current state events. +This behaviour is useful to reduce bandwidth in various cases. For example, a client may specify a list with range +`[0,99]` and a timeline limit of 10, plus a list with range `[100, ]` and timeline limit `1`. This would cause the +server to return the most recent 10 events for rooms with recent activity, but only 1 event for older rooms (that the +user is unlikely to visit). If an older room that we previously only returned with one timeline event receives a new +event, we'll end up sending it down with not just the new event but the previous 10 events as well (despite having sent +the second to last event previously). If the room then drops below the threshold (and so has timeline limit of 1), and +then receives another update, the server MAY remember that it has already sent the previous 10 events and only return +the latest one. + +#### Required state + +Required state expansion works in a similar way. If a room has an expanded `required_state` then the server checks if +the room has any state that matches the new, but not the old, `required_state`. If so then that state is included in the +response. The server MAY chose not to send that state if the client has previously seen that state. + +> [!Note] +> Synapse currently does not return rooms with expanded state immediately, instead waits for the next update. ## Extensions -We anticipate that as more features land in Matrix, different kinds of data will also want to be synced to clients. Sync -v2 did not have any first-class support to opt-in to new data. Sliding Sync does have support for this via "extensions". -Extensions also allow this proposal to be broken up into more manageable sections. Extensions are requested by the -client in a dedicated extensions block. - -In an effort to reduce the size of this proposal, extensions will be done in separate MSCs. There will be extensions -for: - -- To Device Messaging \- MSC3885 -- End-to-End Encryption \- MSC3884 -- Typing Notifications \- MSC3961 -- Receipts \- MSC3960 -- Presence \- presence in sync v2: spec -- Account Data \- account\_data in sync v2: MSC3959 -- Threads - -**TODO** explain how these interact with the room lists, this is the same as in -[MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) - -## Request format - -```javascript -{ - "conn_id": "", // Client chosen ID of the connection, c.f. "Connections" - - // The set of room lists - "lists": { - // An arbitrary string which the client is using to refer to this list for this connection. - "": { - - // Sliding window ranges, c.f. Lists and room subscriptions - "ranges": [[0, 10]], - - // Filters to apply to the list. - "filters": { - // Flag which only returns rooms present (or not) in the DM section of account data. - // If unset, both DM rooms and non-DM rooms are returned. If false, only non-DM rooms - // are returned. If true, only DM rooms are returned. - "is_dm": true|false|null, - - // Flag which only returns rooms which have an `m.room.encryption` state event. If unset, - // both encrypted and unencrypted rooms are returned. If false, only unencrypted rooms - // are returned. If true, only encrypted rooms are returned. - "is_encrypted": true|false|null, - - // Flag which only returns rooms the user is currently invited to. If unset, both invited - // and joined rooms are returned. If false, no invited rooms are returned. If true, only - // invited rooms are returned. - "is_invite": true|false|null, - - // If specified, only rooms where the `m.room.create` event has a `type` matching one - // of the strings in this array will be returned. If this field is unset, all rooms are - // returned regardless of type. This can be used to get the initial set of spaces for an account. - // For rooms which do not have a room type, use 'null' to include them. - "room_types": [ ... ], - - // Same as "room_types" but inverted. This can be used to filter out spaces from the room list. - // If a type is in both room_types and not_room_types, then not_room_types wins and they are - // not included in the result. - "not_room_types": [ ... ], - }, - - // The maximum number of timeline events to return per response. - "timeline_limit": 10, - - // Required state for each room returned. An array of event type and state key tuples. - // Elements in this array are ORd together to produce the final set of state events - // to return. One unique exception is when you request all state events via ["*", "*"]. When used, - // all state events are returned by default, and additional entries FILTER OUT the returned set - // of state events. These additional entries cannot use '*' themselves. - // For example, ["*", "*"], ["m.room.member", "@alice:example.com"] will _exclude_ every m.room.member - // event _except_ for @alice:example.com, and include every other state event. - // In addition, ["*", "*"], ["m.space.child", "*"] is an error, the m.space.child filter is not - // required as it would have been returned anyway. - "required_state": [ ... ], - } - }, - - // The set of room subscriptions - "room_subscriptions": { - // The key is the room to subscribe to. - "!foo:example.com": { - // These have the same meaning as in `lists` section - "timeline_limit": 10, - "required_state": [ ... ], - } - }, +For requesting data other than room events (such as account data or typing notifications), clients can use "extensions". +These are split out into separate sections to a) make it easier for clients to specify just what they need, and b) to make it easier to extend in the future. - // c.f. "Extensions" - "extensions": { - } -} -``` +Example extensions, which will be specified elsewhere: +- To Device Messaging +- End-to-End Encryption +- Typing Notifications +- Receipts +- Presence +- Account Data -## Response format +# API -```javascript -{ - // The position to use as the `since` token in the next sliding sync request. - // c.f. Connections. - "pos": "", +The endpoint is a `POST` request with a JSON body to `/org.matrix.simplified_msc3575/sync`. - // Information about the lists supplied in the request. - "lists": { - // Matches the list name supplied by the client in the request - "" { - // The total number of rooms that match the list's filter. Note that rooms can be in - // multiple lists, so may be double counted. - "count": 1234, - } - }, - - // Aggregated rooms from lists and room subscriptions. There will be one entry per room, even if - // the room appears in multiple lists and/or room subscriptions. - "rooms": { - "!foo:example.com": { - // The room name (as specified by any `m.room.name` event), if one exists. Only sent initially - // and when it changes. - "name": str|null, - // The room avatar, if one exists. Only sent initially and when it changes. - "avatar_url": str|null, - // The "heroes" for the room, if there is no room name. Only sent initially and when it changes. - "heroes": [ - {"user_id":"@alice:example.com","displayname":"Alice","avatar_url":"mxc://..."}, - ], - - // Flag which is set when this is the first time the server is sending this data on this connection. - // When set the client must replace any stored metadata for the room with the new data. In - // particular, the state must be replaced with the state in `required_state`. - "initial": true|null, - - // Same as in sync v2. Indicates whether there are more events to fetch than those in the timeline. - "limited:" true|null, - // Indicates if we have "expanded" the timeline due to the timeline_limit changing, c.f. Room config - // changes above. - "unstable_expanded_timeline": true|null, - // The list of events, sorted least to most recent. - "timeline": [ ... ], - // The current state of the room as a list of events. This is the full state if `initial` - // state is set, otherwise it is a delta from the previous sync. - "required_state": [ ... ], - // The number of timeline events which have just occurred and are not historical. - // The last N events are 'live' and should be treated as such. - "num_live": 1, - // Same as sync v2, passed to `/messages` to fetch more past events. - "prev_batch": "...", - - // For invites this is the stripped state of the room at the time of invite - "invite_state": [ .. ], - - // For knocks this is the stripped state of the room at time of knock - "knock_state": [ .. ], - - // Whether the room is a DM room. - "is_dm": true|null, - - // An opaque integer that can be used to sort the rooms by "Bump Stamp" - "bump_stamp": 1, - - // These are the same as sync v2. - "joined_count": 1, - "invited_count": 1, - "notification_count": 1, - "highlight_count": 1, +## Request URL parameters + +| Name | Type | Required | Comment | +| - | - | - | - | +| `pos` | string | No | Omitted if this is the first request of a connection. Otherwise, the `pos` token from the previous call to `/sync` | +| `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | + + +## Request Body + +### Top-level + +| Name | Type | Required | Comment | +| - | - | - | - | +| `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given conn_id (empty or not). | +| `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). Max lists: 100. The list keys should be arbitrary strings which the client is using to refer to the list. Max length: 64 bytes. | +| `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | +| `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | + + +### `SyncListConfig` + +| Name | Type | Required | Comment | +| - | - | - | - | +| `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | +| `required_state` | [RequiredStateTuple] | Yes | Required state for each room returned. | +| `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*. (This is a list of 2-tuples.) | +| `filters` | `RoomFilter` | No | Filters to apply to the list before sorting. | + +### `RoomSubscription` + +| Name | Type | Required | Comment | +| - | - | - | - | +| `timeline_limit` | `int` | Yes | Same as in `SyncListConfig` | +| `required_state` | [RequiredStateTuple] | Yes | Same as in `SyncListConfig` | + +### `RequiredStateTuple` + +An array of event type and state key tuples. Elements in this array are ORd together to produce the final set of state +events to return. + +For example, the following would return the create event and current power levels: + +```jsonc + { + "required_state": [ + ["m.room.create", ""], + ["m.room.power_levels", ""] + ] } - }, +``` + + +#### Special values + +There are a number of special values that the client can use in place of either the event type or state key: + +1. `"*"` — Can be used in either the type or state key position to match any type or state key. +1. `"$LAZY"` — Can be used in the state key position when the type is `m.room.member`. This triggers the lazy loading + membership behaviour (as in sync v2), described in the next sections. +1. `"$ME"` — Can be used in the state key position to match the user's full user ID. This is merely a convenience for + clients, and has the same semantics as if the client used the full user ID (e.g. `"@user:example.com"`). + +#### Wildcards + +The wildcard (`"*"`) can be used in either the type or state key position, and matches any type or state key. The client +can use `["*", "*"]` to request all state. + +There is a special case when `["*", "*"]` is used with additional entries: the additional entries are FILTERED OUT the +of the returned set of state events. These additional entries cannot use `*` themselves. For example, `["*", "*"], +["m.room.member", "@alice:example.com"]` will *exclude* every `m.room.member` event *except* for `@alice:example.com`, +and include every other state event. In addition, `["*", "*"], ["m.space.child", "*"]` is an error, the `m.space.child` +filter is not required as it would have been returned anyway. + +> [!Note] +> Synapse doesn't currently support filtering out state, and simply returns all state. + +#### Lazy loaded memberships + +Room members can be lazily-loaded by using the special `$LAZY` state key (`["m.room.member", "$LAZY"]`). Typically, when +you view a room, you want to retrieve all state events except for m.room.member events which you want to lazily load. To +get this behaviour, clients can send the following: - "extensions": { - }, -} +```jsonc + { + "required_state": [ + // activate lazy loading + ["m.room.member", "$LAZY"], + // request all state events _except_ for m.room.member events which are lazily loaded + ["*", "*"] + ] + } ``` -# Example usage +This is (almost) the same as lazy loaded memberships in sync v2. When specified, the server will return the membership +events for: +1. All the senders of events in `timeline_events` unless previously returned. This ensures that the client can render + all the timeline events without having to fetch more events from the server. +2. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to + cache the membership list without requiring the server to send all membership updates for large gaps. + + +#### Combining `required_state` + +When combining room configs with different `required_state` fields the result must be the superset of the different sets +of `required_state`. There are two approaches server-side for handling this: a) keep the `required_state` sets +separates and return any state that matches any set, or b) merge the sets together, however care must be taken to +correctly account for wildcards, i.e. one cannot simply join the lists together. + + +### `RoomFilter` + +| Name | Type | Required | Comment | +| - | - | - | - | +| `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data. If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | +| `spaces` | `` | No | Filter the room based on the space they belong to according to `m.space.child` state events. If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | +| `is_encrypted` | `[string]` | No | Flag which only returns rooms which have an `m.room.encryption` state event. If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | +| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to. If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | +| `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned. If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | +| `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted. This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | +| `room_name_like` | `string` | No | Filter the room name. Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'. | +| `tags` | `[string]` | No | Filter the room based on its room tags. If multiple tags are present, a room can have any one of the listed tags (OR'd). | +| `not_tags` | `[string]` | No | Filter the room based on its room tags. Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | + + +## Response Body + + +### Top-level + +| Name | Type | Required | Comment | +| - | - | - | - | +| `pos` | `string` | Yes | The next position token in the sliding window to request. | +| `lists` | `{string: SyncListResult}` | No | A map of list key to list results. | +| `rooms` | `{string: RoomResult}` | No | A map of room ID to room results. | +| `extensions` | `{string: ExtensionResult}` | No | A map of extension key to extension results. Different extensions have different result formats. | + + +### `SyncListResult` + +| Name | Type | Required | Comment | +| - | - | - | - | +| `count` | `int` | Yes | The total number of entries in the list. | + + +### `RoomResult` + +Not all fields will be returned depending on the user's membership in the room. + +All fields in this section may be omitted. When `initial` is set to `false` then an omitted filed means that the field +remains unchanged from its previous value. When `initial` is set to `true` then an omitted field means that the field is +not set, e.g. if `initial` is true and `name` is not set then the room has no name. Care must be taken by clients to +ensure that if they previously saw that the room had a name, then it MUST be cleared if it receives a room response with +`initial: true` and no `name` field. + +#### Joined/left/banned + +When a user is or has been in the room, the following field are returned: + +| Name | Type | Required | Comment | +| - | - | - | - | +| `name` | `string` | No | Room name or calculated room name. | +| `avatar` | `string` | No | Room avatar | +| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. | +| `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). | +| `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | +| `unstable_expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | +| `required_state` | `[Event]` | No | Changes in the current state of the room. | +| `timeline_events` | `[Event]` | No | Latest events in the room. May not include all events because e.g. there were more events than the configured `timeline_limit`, see `limited` field. If `limited` is true then we include bundle aggregations for the event, as per sync v2. The last event is the most recent. | +| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received of one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, or a timestamp, etc. | +| `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | +| `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events. Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room. Absence means `false` | +| `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such. This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | +| `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as sync `v2 m.joined_member_count`) | +| `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | +| `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2) | +| `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2) | + + +> [!Note] +> Synapse always returns 0 for `notification_count` and `highlight_count` + + +#### Invite + +For rooms the user is invited to, the client only gets the stripped state events: + +| Name | Type | Comment | +| - | - | - | +| `invite_state` | `[StrippedState]` | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` in sync v2. | +| `bump_stamp` | `int` | Same as for joined/left/banned rooms. | + + +> [!Note] +> Synapse currently may inadvertently return extra fields from the previous section. + +#### Knock + +For rooms the user has knocked, the client only gets the stripped state events: + +| Name | Type | Comment | +| - | - | - | +| `knock_state` | `[StrippedState]` | Stripped state events (for rooms where the user has knocked). Same as `rooms.invite.$room_id.knock_state` in sync v2. | +| `bump_stamp` | `int` | Same as for joined/left/banned rooms. | + +> [!Note] +> This hasn't been implemented in Synapse. + + +#### `StrippedHero` type + +The `StrippedHero` has the following fields: + + +| Name | Type | Required | Comment | +| - | - | - | - | +| `user_id` | string | Yes | The user ID of the hero. | +| `displayname` | string | No | The display name of the user from the membership event, if set | +| `avatar_url` | string | No | The avatar url from the membership event, if set | + +# Common patterns + +Below are some potentially common patterns that clients may wish to use. + +## Paginating room list + +Pagination of the room list is achieved by the client increasing the ranges of one (or more) lists. + +For example an initial request might have a list called `all_rooms` specifying a range of `[0,19]` in the initial +request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client +may change the range to `[0,99]`, at which point the server will respond with the top 100 rooms that either a) weren’t +sent down in the first request, or b) have updates since the first request. + +Clients can increase and decrease the ranges as they see fit. A common approach would be to start with a small window +and grow that until the range covers all the rooms. After some threshold of the app being offline it may reduce the +range back down and incrementally grow it again. This allows for ensuring that a limited amount of data is requested at +once, to improve response times. + +## Example usage This section gives an example of how a client can use this API (roughly based on how Element X currently uses the API). @@ -337,10 +482,7 @@ When the client is expecting a fast response (e.g. while expanding the lists), i request with a `since` parameter, if the client shows a spinner but doesn't set a timeout then the request may take a long time to return (if there were no updates to return). - -# Alternatives / changes - -There are a number of potential changes that we could make. +# Alternatives ## Pagination @@ -352,11 +494,12 @@ An alternative is for the client to specify a `page_size`, where the server send rooms. If there are more rooms to send to the client (beyond `page_size`), then the client can request to "paginate" in these missed updates in subsequent updates. -Since this would require client side changes, this should be explored in a separate MSC. +Since this may require substantial client side changes compared with the current implementation, this should be explored +separately. ## Timeline event trickling -If the `timeline_limit` is increased then the server will send down historic data (c.f. "Room config changes"), which +If the `timeline_limit` is increased then the server will send down historic data (c.f. "Changing room configs"), which allows the clients to easily preload more history in recent rooms. This mechanism is fiddly to implement, and ends up resending down events that we have previously sent to the client. @@ -367,18 +510,6 @@ to preload history for multiple rooms at once, and 2) `/messages` can be slow if We could implement a bulk `/messages` endpoint, where the client would specify multiple rooms and `prev_batch` tokens. We can also add a flag to disable attempting to backfill over pagination (to match the behaviour of the sync timeline). -## `required_state` response format - -The format of returned state in `required_state` is a list of events. This does now allow the server to indicate if a -"state reset" has happened which removed an entry from the state entirely (rather than it being replaced with another -event). - -This is particularly problematic if the user gets "state reset" out of the room, where the server has no mechanism to -indicate to the client that the user has effectively left the room (the server has no leave event to return). - -We may want to allow special entries in the `required_state` list of the form -`{"type": .., "state_key": .., content: null}` to indicate that the state entry has been removed. - # Security considerations @@ -390,3 +521,30 @@ response. The unstable URL for simplified sliding sync is `/org.matrix.simplified_msc3575/sync`. The flag in `/versions` is `org.matrix.simplified_msc3575`. + + +# Appendix + +## Open questions + +1. In the response should we specify which lists a room is part of? +1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? +1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? +1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or + remove the functionality and replace it with a bulk `/messages` endpoint (for multiple rooms). See "Timeline event + trickling" section. +1. The request `required_state` field is a bit confusing and uses special strings (like `"*"` and `"$LAZY"`). +1. Duplication between room response `heroes` and `required_state` when specifying `["m.room.member", "$LAZY"]`, e.g. + should we drop `heroes` if we are returning membership events? +1. Should the room response include the user's current membership? The client can always request it via + `required_state`, but that may be wasted if the client doesn't need any other information from the member event. +1. Should we remove the `highlight_count` and `notification_count` fields, given clients increasingly must calculate + this themselves, and Synapse currently doesn't implement it nor does Element X use it. +1. Should we support subscribing to rooms the user is not a member of, i.e. peeking for world readable rooms. +1. Should we support timeline filtering? + +## TODOs + +1. We should implement `set_presence` as per sync v2 +1. If we're keeping the notification counts we should add `unread_thread_notifications` +1. We should add `knock_servers` as per MSC4233, if that lands. From a1bd6bfb48d9af2ef67c09f04942fb32f4e66ab1 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:11:32 +0100 Subject: [PATCH 07/77] Use full URLs --- proposals/4186-simplified-sliding-sync.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 69554339b..d9738eccb 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -212,7 +212,7 @@ Example extensions, which will be specified elsewhere: # API -The endpoint is a `POST` request with a JSON body to `/org.matrix.simplified_msc3575/sync`. +The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/org.matrix.simplified_msc3575/sync`. ## Request URL parameters @@ -519,8 +519,8 @@ response. # Unstable prefix -The unstable URL for simplified sliding sync is `/org.matrix.simplified_msc3575/sync`. The flag in `/versions` is -`org.matrix.simplified_msc3575`. +The unstable URL for simplified sliding sync is `/_matrix/client/unstable/org.matrix.simplified_msc3575/sync`. The flag +in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. # Appendix From 839eb35c65943ea252026e92fe425003b5af7015 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:14:16 +0100 Subject: [PATCH 08/77] Clarify $LAZY --- proposals/4186-simplified-sliding-sync.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index d9738eccb..8bb8045f1 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -312,7 +312,8 @@ This is (almost) the same as lazy loaded memberships in sync v2. When specified, events for: 1. All the senders of events in `timeline_events` unless previously returned. This ensures that the client can render all the timeline events without having to fetch more events from the server. -2. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to +1. The target (i.e. `state_key`) of all membership events in `timeline_events`. +1. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to cache the membership list without requiring the server to send all membership updates for large gaps. From a9141ab17a1d0bbe4b0e6e4eeab1740cf7590282 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:19:09 +0100 Subject: [PATCH 09/77] Note that bump stamps can decrease --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 8bb8045f1..1cb17adcb 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -384,7 +384,7 @@ When a user is or has been in the room, the following field are returned: | `unstable_expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event]` | No | Changes in the current state of the room. | | `timeline_events` | `[Event]` | No | Latest events in the room. May not include all events because e.g. there were more events than the configured `timeline_limit`, see `limited` field. If `limited` is true then we include bundle aggregations for the event, as per sync v2. The last event is the most recent. | -| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received of one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, or a timestamp, etc. | +| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received of one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, or a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | | `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events. Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room. Absence means `false` | | `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such. This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | From dbf593ad0f8a8a95fb8b734a290fb1d66bcf7b0d Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:19:19 +0100 Subject: [PATCH 10/77] Add an open question for pre-flight CORS --- proposals/4186-simplified-sliding-sync.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 1cb17adcb..b9e0f0983 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -543,6 +543,8 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. this themselves, and Synapse currently doesn't implement it nor does Element X use it. 1. Should we support subscribing to rooms the user is not a member of, i.e. peeking for world readable rooms. 1. Should we support timeline filtering? +1. Should we move `pos`, and other URL params, into the request body? This would allow web clients to cache the CORS + requests, rather than having to pre-flight each request. ## TODOs From a325def59f7cba5f7c04a40692780bc6a17b5199 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 9 Sep 2025 10:52:53 +0100 Subject: [PATCH 11/77] Clarifications --- proposals/4186-simplified-sliding-sync.md | 128 +++++++++++++++++----- 1 file changed, 103 insertions(+), 25 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index b9e0f0983..e8675409f 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -15,8 +15,8 @@ usage. ## High level overview From a high level, sliding sync works by the client specifying a set of rules to match rooms (via "lists" and -"subscriptions"). The server will use the rules to match relevant rooms to the user, and return any rooms that have had -changes since the previous time the room was returned to the client. If no rooms have changes, the server will block +"subscriptions"). The server will use these rules to match relevant rooms to the user, and return any rooms that have +had changes since the previous time the room was returned to the client. If no rooms have changes, the server will block waiting for a change (aka long-polling, like sync v2). By judicious use of lists and subscriptions the client can control exactly what data is returned when, and help ensure @@ -47,6 +47,9 @@ See the API section below for the request and response schemas. ## Connections +Each request may include a `pos` token, and each response includes a `pos` token that can be used for subsequent +requests. + A "connection" (aka "sync loop") is a long-running set of sync requests where each new request uses the `pos` from the previous request. Clients can have multiple connections with the server, so long as each connection has a different `conn_id` set in the request. @@ -113,7 +116,9 @@ ordering is determined by the server implementation. ### Subscriptions -Subscriptions are simply the room ID to match. This is useful if e.g. the user has opened the room and the client always wants to get the latest data for that room. +Subscriptions are a rule that simply matches against a specified room ID, i.e. they allow the client to specify that a +given room should always be returned (if there are updates). This is useful if e.g. the user has opened the room and the +client always wants to get the latest data for that room. The server MUST ensure that user has permission to see any information the server returns. However, the user need not be in the room, e.g.clients can specify room IDs for world-readable rooms and they would be returned. @@ -154,24 +159,25 @@ data returned is the full response or a delta based on the `initial` flag. If a room matches multiple rules, and therefore multiple room configs, then the room configs must be combined into one before being applied. -The fields are combined by: +The fields are combined by taking the "superset", i.e.: - Timeline limit — take the maximum timeline limit across all room configs. -- Required state — take the superset of required, i.e. if a state event would be returned by any room config it is returned by the combined room config. +- Required state — take the superset of the required state fields, i.e. if a state event would be returned by any room + config it is returned by the combined room config. ### Changing room configs When a room matches one or more rules (i.e. is eligible to be returned in the sync response) that has previously been returned to the client, the server checks whether the combined room config is different than when the room was last -eligible to be returned. If the new config is a superset of the previous config, then the server handles the config -differently. +eligible to be returned. If the new config has fields that are a superset of the previous config, then the server +handles the config differently. #### Timeline events Normally the timeline events returned are only the events that have been received since the last time the room was sent to the client (i.e. only new events). However, if the `timeline_limit` has increased (to say `N`) the server SHOULD ignore this and send down the latest `N` events, even if some of those events have previously been sent. The server MAY -ignore behaviour if the server knows it has previously sent down all of the latest `N` events. +ignore this behaviour if the server knows it has previously sent down all of the latest `N` events. > [!IMPORTANT] > The server should return rooms that have expanded timelines immediately, rather than waiting for the next update to @@ -201,7 +207,7 @@ response. The server MAY chose not to send that state if the client has previous For requesting data other than room events (such as account data or typing notifications), clients can use "extensions". These are split out into separate sections to a) make it easier for clients to specify just what they need, and b) to make it easier to extend in the future. -Example extensions, which will be specified elsewhere: +Examples of extensions, which will be specified elsewhere, are: - To Device Messaging - End-to-End Encryption - Typing Notifications @@ -329,15 +335,45 @@ correctly account for wildcards, i.e. one cannot simply join the lists together. | Name | Type | Required | Comment | | - | - | - | - | -| `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data. If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | -| `spaces` | `` | No | Filter the room based on the space they belong to according to `m.space.child` state events. If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | -| `is_encrypted` | `[string]` | No | Flag which only returns rooms which have an `m.room.encryption` state event. If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | -| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to. If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | -| `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned. If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | -| `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted. This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | -| `room_name_like` | `string` | No | Filter the room name. Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'. | -| `tags` | `[string]` | No | Filter the room based on its room tags. If multiple tags are present, a room can have any one of the listed tags (OR'd). | -| `not_tags` | `[string]` | No | Filter the room based on its room tags. Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | +| `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | +| `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | +| `is_encrypted` | `[string]` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | +| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | +| `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | +| `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | +| `room_name_like` | `string` | No | Filter the room name.

Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'. | +| `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | +| `not_tags` | `[string]` | No | Filter the room based on its room tags.

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | + +### Example request + +```jsonc +{ + "conn_id": "main", + + // Sliding Window API + "lists": { + "foo-list": { + "ranges": [ [0, 99] ], + "required_state": [ + ["m.room.create", ""], + ["m.room.member", "$LAZY"], + ], + "timeline_limit": 10, + "filters": { + "is_dm": true + }, + } + }, + // Room Subscriptions API + "room_subscriptions": { + "!sub1:bar": { + "required_state": [ ["*","*"] ], + "timeline_limit": 10, + } + } +} +``` ## Response Body @@ -383,11 +419,11 @@ When a user is or has been in the room, the following field are returned: | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `unstable_expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event]` | No | Changes in the current state of the room. | -| `timeline_events` | `[Event]` | No | Latest events in the room. May not include all events because e.g. there were more events than the configured `timeline_limit`, see `limited` field. If `limited` is true then we include bundle aggregations for the event, as per sync v2. The last event is the most recent. | -| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received of one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, or a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | +| `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | +| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received is one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | -| `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events. Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room. Absence means `false` | -| `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such. This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | +| `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events.

Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room.

Absence means `false` | +| `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | | `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as sync `v2 m.joined_member_count`) | | `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | | `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2) | @@ -400,7 +436,7 @@ When a user is or has been in the room, the following field are returned: #### Invite -For rooms the user is invited to, the client only gets the stripped state events: +For rooms the user is invited to, the client only gets the stripped state events and `bump_stamp`: | Name | Type | Comment | | - | - | - | @@ -413,7 +449,7 @@ For rooms the user is invited to, the client only gets the stripped state events #### Knock -For rooms the user has knocked, the client only gets the stripped state events: +For rooms the user has knocked, the client only gets the stripped state events and `bump_stamp`: | Name | Type | Comment | | - | - | - | @@ -424,7 +460,7 @@ For rooms the user has knocked, the client only gets the stripped state events: > This hasn't been implemented in Synapse. -#### `StrippedHero` type +### `StrippedHero` type The `StrippedHero` has the following fields: @@ -435,6 +471,48 @@ The `StrippedHero` has the following fields: | `displayname` | string | No | The display name of the user from the membership event, if set | | `avatar_url` | string | No | The avatar url from the membership event, if set | + +### Example response + +```jsonc +{ + "pos": "s58_224_0_13_10_1_1_16_0_1", + "lists": { + "foo-list": { + "count": 1337, + } + }, + "rooms": { + "!sub1:bar": { + "name": "Alice and Bob", + "avatar": "mxc://...", + "initial": true, + "required_state": [ + { + "sender":"@alice:example.com", + "type":"m.room.create", + "state_key":"", + "content": {} + }, + ... + ], + "timeline": [ + {"sender":"@alice:example.com","type":"m.room.message", "content":{"body":"B"}}, + ... + ], + "prev_batch": "t111_222_333", + "joined_count": 41, + "invited_count": 1, + "notification_count": 1, + "highlight_count": 0, + "num_live": 2 + }, + ... + }, + "extensions": {} +} +``` + # Common patterns Below are some potentially common patterns that clients may wish to use. From 992007aaae73b99e47e6b47853080d3d92738caa Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 09:42:37 +0100 Subject: [PATCH 12/77] Add support for `set_presence` --- proposals/4186-simplified-sliding-sync.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index e8675409f..953b58262 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -226,6 +226,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | - | - | - | - | | `pos` | string | No | Omitted if this is the first request of a connection. Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | +| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | ## Request Body @@ -626,6 +627,12 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. ## TODOs -1. We should implement `set_presence` as per sync v2 1. If we're keeping the notification counts we should add `unread_thread_notifications` 1. We should add `knock_servers` as per MSC4233, if that lands. + + +## Changelog + +Changes from the initial implementation of simplified sliding sync. + +1. Add `set_presence` URL param. From 165aaffd51fd57ddd0e7dc2eed759d564a96596b Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 11:33:39 +0100 Subject: [PATCH 13/77] Rename invite_state to stripped_state --- proposals/4186-simplified-sliding-sync.md | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 953b58262..24059901d 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -435,30 +435,21 @@ When a user is or has been in the room, the following field are returned: > Synapse always returns 0 for `notification_count` and `highlight_count` -#### Invite +#### Invite/knock -For rooms the user is invited to, the client only gets the stripped state events and `bump_stamp`: +For rooms the user is invited to or has knocked on, the client only gets the stripped state events and `bump_stamp`: | Name | Type | Comment | | - | - | - | -| `invite_state` | `[StrippedState]` | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` in sync v2. | +| `stripped_state` | `[StrippedState]` | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | | `bump_stamp` | `int` | Same as for joined/left/banned rooms. | > [!Note] > Synapse currently may inadvertently return extra fields from the previous section. -#### Knock - -For rooms the user has knocked, the client only gets the stripped state events and `bump_stamp`: - -| Name | Type | Comment | -| - | - | - | -| `knock_state` | `[StrippedState]` | Stripped state events (for rooms where the user has knocked). Same as `rooms.invite.$room_id.knock_state` in sync v2. | -| `bump_stamp` | `int` | Same as for joined/left/banned rooms. | - > [!Note] -> This hasn't been implemented in Synapse. +> Knock support hasn't been implemented in Synapse. ### `StrippedHero` type @@ -608,7 +599,7 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. ## Open questions 1. In the response should we specify which lists a room is part of? -1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? +1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? 1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? 1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or remove the functionality and replace it with a bulk `/messages` endpoint (for multiple rooms). See "Timeline event @@ -636,3 +627,4 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. Changes from the initial implementation of simplified sliding sync. 1. Add `set_presence` URL param. +2. Rename `invite_state` to `stripped_state` From 495963fecd26ca7176575a380a485da2375dffcc Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 11:39:26 +0100 Subject: [PATCH 14/77] Handle state deletion --- proposals/4186-simplified-sliding-sync.md | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 24059901d..ceab67bda 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -419,7 +419,7 @@ When a user is or has been in the room, the following field are returned: | `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `unstable_expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | -| `required_state` | `[Event]` | No | Changes in the current state of the room. | +| `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | | `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | | `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received is one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | @@ -464,6 +464,16 @@ The `StrippedHero` has the following fields: | `avatar_url` | string | No | The avatar url from the membership event, if set | +### `StateStub` type + +The `StateStub` is used in `required_state` to indicate that a piece of state has been deleted. + +| Name | Type | Required | Comment | +| - | - | - | - | +| `type` | string | Yes | The `type` of the state entry that was deleted | +| `state_key` | string | Yes | The `state_key` of the state entry that was deleted | + + ### Example response ```jsonc @@ -600,7 +610,7 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. 1. In the response should we specify which lists a room is part of? 1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? -1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? +1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? 1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or remove the functionality and replace it with a bulk `/messages` endpoint (for multiple rooms). See "Timeline event trickling" section. @@ -628,3 +638,4 @@ Changes from the initial implementation of simplified sliding sync. 1. Add `set_presence` URL param. 2. Rename `invite_state` to `stripped_state` +3. When state is deleted we return a stub `{"type: "..", "state_key": ".." }` in `required_state`. From c6cd26e4acc36a7431991dbbc2865e9fa5851e2b Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 11:40:45 +0100 Subject: [PATCH 15/77] Rename unstable_expanded_timeline --- proposals/4186-simplified-sliding-sync.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index ceab67bda..16b040b0c 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -179,6 +179,8 @@ to the client (i.e. only new events). However, if the `timeline_limit` has incre ignore this and send down the latest `N` events, even if some of those events have previously been sent. The server MAY ignore this behaviour if the server knows it has previously sent down all of the latest `N` events. +If the server does send down extra events, it MUST set the `expanded_timeline` to `true`. + > [!IMPORTANT] > The server should return rooms that have expanded timelines immediately, rather than waiting for the next update to the room. @@ -418,7 +420,7 @@ When a user is or has been in the room, the following field are returned: | `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. | | `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | -| `unstable_expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | +| `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | | `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | | `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received is one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | @@ -611,9 +613,9 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. 1. In the response should we specify which lists a room is part of? 1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? 1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? -1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or +1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or remove the functionality and replace it with a bulk `/messages` endpoint (for multiple rooms). See "Timeline event - trickling" section. + trickling" section. 1. The request `required_state` field is a bit confusing and uses special strings (like `"*"` and `"$LAZY"`). 1. Duplication between room response `heroes` and `required_state` when specifying `["m.room.member", "$LAZY"]`, e.g. should we drop `heroes` if we are returning membership events? @@ -639,3 +641,4 @@ Changes from the initial implementation of simplified sliding sync. 1. Add `set_presence` URL param. 2. Rename `invite_state` to `stripped_state` 3. When state is deleted we return a stub `{"type: "..", "state_key": ".." }` in `required_state`. +4. Rename `unstable_expanded_timeline` to `expanded_timeline` From 486efe299bf687eafd3ff22599ff4723cc90f751 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 11:43:34 +0100 Subject: [PATCH 16/77] Add new lists field --- proposals/4186-simplified-sliding-sync.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 16b040b0c..71264c7cf 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -431,6 +431,7 @@ When a user is or has been in the room, the following field are returned: | `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | | `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2) | | `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2) | +| `lists` | `[string]` | No | The name of the lists that match this room. The field is omitted if it doesn't match any list and is included only due to a subscription. | > [!Note] @@ -441,10 +442,11 @@ When a user is or has been in the room, the following field are returned: For rooms the user is invited to or has knocked on, the client only gets the stripped state events and `bump_stamp`: -| Name | Type | Comment | -| - | - | - | -| `stripped_state` | `[StrippedState]` | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | -| `bump_stamp` | `int` | Same as for joined/left/banned rooms. | +| Name | Type | Required | Comment | +| - | - | - | - | +| `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | +| `bump_stamp` | `int` | Yes | Same as for joined/left/banned rooms. | +| `lists` | `[string]` | No | Same as for joined/left/banned rooms. | > [!Note] @@ -610,7 +612,7 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. ## Open questions -1. In the response should we specify which lists a room is part of? +1. In the response should we specify which lists a room is part of? 1. Should `knock_state` and `invite_state` use the same name in the room response, e.g. `stripped_state`? 1. In the room response how do we inform clients that a piece of state was deleted (rather than added/updated)? 1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or @@ -642,3 +644,4 @@ Changes from the initial implementation of simplified sliding sync. 2. Rename `invite_state` to `stripped_state` 3. When state is deleted we return a stub `{"type: "..", "state_key": ".." }` in `required_state`. 4. Rename `unstable_expanded_timeline` to `expanded_timeline` +5. Add `lists` to room response From 97feb73da01f4accc7b49a329a430315a5380808 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 15:12:14 +0100 Subject: [PATCH 17/77] Add `membership` field in room response --- proposals/4186-simplified-sliding-sync.md | 26 +++++++++++++++-------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 71264c7cf..7cc7a8031 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -409,9 +409,19 @@ not set, e.g. if `initial` is true and `name` is not set then the room has no na ensure that if they previously saw that the room had a name, then it MUST be cleared if it receives a room response with `initial: true` and no `name` field. +#### Common fields + +These fields are always included no matter the membership. + +| Name | Type | Required | Comment | +| - | - | - | - | +| `bump_stamp` | `int` | Yes | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received is one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | +| `membership` | `string` | No | The current membership of the user, or omitted if user not in room (for peeking). | +| `lists` | `[string]` | No | The name of the lists that match this room. The field is omitted if it doesn't match any list and is included only due to a subscription. | + #### Joined/left/banned -When a user is or has been in the room, the following field are returned: +When a user is or has been in the room, the following field are also returned: | Name | Type | Required | Comment | | - | - | - | - | @@ -423,7 +433,6 @@ When a user is or has been in the room, the following field are returned: | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | | `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | -| `bump_stamp` | `int` | No | An integer that can be used to sort rooms based on the last "proper" activity in the room. Greater means more recent.

"Proper" activity is defined as an event being received is one of the following types: `m.room.create`, `m.room.message`, `m.room.encrypted`, `m.sticker`, `m.call.invite`, `m.poll.start`, `m.beacon_info`.

For rooms that the user is not currently joined to, this instead represents when the relevant membership happened, e.g. when the user left the room.

The exact value of `bump_stamp` is opaque to the client, a server may use e.g. an auto-incrementing integer, a timestamp, etc.

The `bump_stamp` may decrease in subsequent responses, if e.g. an event was redacted/removed (or purged in cases of retention policies). | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | | `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events.

Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room.

Absence means `false` | | `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | @@ -440,14 +449,11 @@ When a user is or has been in the room, the following field are returned: #### Invite/knock -For rooms the user is invited to or has knocked on, the client only gets the stripped state events and `bump_stamp`: +For rooms the user is invited to or has knocked on, the client also gets the stripped state event: | Name | Type | Required | Comment | | - | - | - | - | | `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | -| `bump_stamp` | `int` | Yes | Same as for joined/left/banned rooms. | -| `lists` | `[string]` | No | Same as for joined/left/banned rooms. | - > [!Note] > Synapse currently may inadvertently return extra fields from the previous section. @@ -511,7 +517,8 @@ The `StateStub` is used in `required_state` to indicate that a piece of state ha "invited_count": 1, "notification_count": 1, "highlight_count": 0, - "num_live": 2 + "num_live": 2, + "membership": "join" }, ... }, @@ -621,8 +628,8 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. 1. The request `required_state` field is a bit confusing and uses special strings (like `"*"` and `"$LAZY"`). 1. Duplication between room response `heroes` and `required_state` when specifying `["m.room.member", "$LAZY"]`, e.g. should we drop `heroes` if we are returning membership events? -1. Should the room response include the user's current membership? The client can always request it via - `required_state`, but that may be wasted if the client doesn't need any other information from the member event. +1. Should the room response include the user's current membership? The client can always request it via + `required_state`, but that may be wasted if the client doesn't need any other information from the member event. 1. Should we remove the `highlight_count` and `notification_count` fields, given clients increasingly must calculate this themselves, and Synapse currently doesn't implement it nor does Element X use it. 1. Should we support subscribing to rooms the user is not a member of, i.e. peeking for world readable rooms. @@ -645,3 +652,4 @@ Changes from the initial implementation of simplified sliding sync. 3. When state is deleted we return a stub `{"type: "..", "state_key": ".." }` in `required_state`. 4. Rename `unstable_expanded_timeline` to `expanded_timeline` 5. Add `lists` to room response +6. Add `membership` field to room response. From 1786eeb7fd055b8757f51cb8c19904dd5b293891 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 11 Sep 2025 15:44:49 +0100 Subject: [PATCH 18/77] Move unread threads to thread extension --- proposals/4186-simplified-sliding-sync.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 7cc7a8031..c112793df 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -639,7 +639,8 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. ## TODOs -1. If we're keeping the notification counts we should add `unread_thread_notifications` +1. If we're keeping the notification counts we should add `unread_thread_notifications` + - This should exist in the thread extension 1. We should add `knock_servers` as per MSC4233, if that lands. From d156cb97fd8f731d4cdfa86a56618c6285899677 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 12 Sep 2025 10:06:21 +0100 Subject: [PATCH 19/77] Change the required_state request format --- proposals/4186-simplified-sliding-sync.md | 123 +++++++++++----------- 1 file changed, 62 insertions(+), 61 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index c112793df..98c64370d 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -248,7 +248,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | -| `required_state` | [RequiredStateTuple] | Yes | Required state for each room returned. | +| `required_state` | [RequiredStateRequest] | Yes | Required state for each room returned. | | `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*. (This is a list of 2-tuples.) | | `filters` | `RoomFilter` | No | Filters to apply to the list before sorting. | @@ -257,65 +257,33 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | | `timeline_limit` | `int` | Yes | Same as in `SyncListConfig` | -| `required_state` | [RequiredStateTuple] | Yes | Same as in `SyncListConfig` | +| `required_state` | [RequiredStateRequest] | Yes | Same as in `SyncListConfig` | -### `RequiredStateTuple` +### `RequiredStateRequest` -An array of event type and state key tuples. Elements in this array are ORd together to produce the final set of state -events to return. +Describes the set of state that the server should return for the room. -For example, the following would return the create event and current power levels: - -```jsonc - { - "required_state": [ - ["m.room.create", ""], - ["m.room.power_levels", ""] - ] - } -``` - - -#### Special values - -There are a number of special values that the client can use in place of either the event type or state key: +| Name | Type | Required | Comment | +| - | - | - | - | +| `include` | `[RequiredStateElement]` | Yes | The set of state to return (unless filtered out by `exclude`). | +| `exclude` | `[RequiredStateElement]` | No | The set of state to filter out before returning, if any. | +| `lazy_members` | `bool` | No | Whether to enable lazy loaded membership behaviour. Defaults to false. | -1. `"*"` — Can be used in either the type or state key position to match any type or state key. -1. `"$LAZY"` — Can be used in the state key position when the type is `m.room.member`. This triggers the lazy loading - membership behaviour (as in sync v2), described in the next sections. -1. `"$ME"` — Can be used in the state key position to match the user's full user ID. This is merely a convenience for - clients, and has the same semantics as if the client used the full user ID (e.g. `"@user:example.com"`). -#### Wildcards +#### `RequiredStateElement` -The wildcard (`"*"`) can be used in either the type or state key position, and matches any type or state key. The client -can use `["*", "*"]` to request all state. +| Name | Type | Required | Comment | +| - | - | - | - | +| `type` | `string` | No | The event type to match. If omitted then matches all types. | +| `state_key` | `string` | No | The event state key to match. If omitted then matches all state keys. | -There is a special case when `["*", "*"]` is used with additional entries: the additional entries are FILTERED OUT the -of the returned set of state events. These additional entries cannot use `*` themselves. For example, `["*", "*"], -["m.room.member", "@alice:example.com"]` will *exclude* every `m.room.member` event *except* for `@alice:example.com`, -and include every other state event. In addition, `["*", "*"], ["m.space.child", "*"]` is an error, the `m.space.child` -filter is not required as it would have been returned anyway. -> [!Note] -> Synapse doesn't currently support filtering out state, and simply returns all state. #### Lazy loaded memberships -Room members can be lazily-loaded by using the special `$LAZY` state key (`["m.room.member", "$LAZY"]`). Typically, when -you view a room, you want to retrieve all state events except for m.room.member events which you want to lazily load. To -get this behaviour, clients can send the following: - -```jsonc - { - "required_state": [ - // activate lazy loading - ["m.room.member", "$LAZY"], - // request all state events _except_ for m.room.member events which are lazily loaded - ["*", "*"] - ] - } -``` +Room members can be lazily-loaded by using the `lazy_members` flag is set. Typically, when you view a room, you want to +retrieve all state events except for m.room.member events which you want to lazily load. To get this behaviour, clients +can send the following: This is (almost) the same as lazy loaded memberships in sync v2. When specified, the server will return the membership events for: @@ -326,12 +294,41 @@ events for: cache the membership list without requiring the server to send all membership updates for large gaps. +Memberships returned to the client due to `lazy_members` are *not* filtered by `exclude`. + + #### Combining `required_state` -When combining room configs with different `required_state` fields the result must be the superset of the different sets -of `required_state`. There are two approaches server-side for handling this: a) keep the `required_state` sets -separates and return any state that matches any set, or b) merge the sets together, however care must be taken to -correctly account for wildcards, i.e. one cannot simply join the lists together. +When combining room configs with different `required_state` fields the result must be the superset of them all. There +are two approaches server-side for handling this: a) keep the `required_state` separate and return any state that +matches any of them, or b) merge the fields together, however care must be taken to correctly account for wildcards. + +#### Examples + +Simple example that returns the create event and power levels: + +```jsonc + { + "required_state": { + "include": [ + {"type": "m.room.create", "state_key": ""}, + {"type": "m.room.power_levels", "state_key": ""}, + ], + } + } +``` + +An example that returns all the state except the create event: + +```jsonc + { + "required_state": { + "include": [{}], // An empty object matches everything + "exclude": [{"type": "m.room.create", "state_key": ""}] + } + } +``` + ### `RoomFilter` @@ -358,10 +355,12 @@ correctly account for wildcards, i.e. one cannot simply join the lists together. "lists": { "foo-list": { "ranges": [ [0, 99] ], - "required_state": [ - ["m.room.create", ""], - ["m.room.member", "$LAZY"], - ], + "required_state": { + "include": [ + {"type": "m.room.create", "state_key": ""}, + ], + "lazy_members": true, + }, "timeline_limit": 10, "filters": { "is_dm": true @@ -371,7 +370,7 @@ correctly account for wildcards, i.e. one cannot simply join the lists together. // Room Subscriptions API "room_subscriptions": { "!sub1:bar": { - "required_state": [ ["*","*"] ], + "required_state": { "include": [{}] }, "timeline_limit": 10, } } @@ -625,9 +624,9 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. 1. We need to decide what to do with `unstable_expanded_timeline`. We can either rename it to `expanded_timeline`, or remove the functionality and replace it with a bulk `/messages` endpoint (for multiple rooms). See "Timeline event trickling" section. -1. The request `required_state` field is a bit confusing and uses special strings (like `"*"` and `"$LAZY"`). -1. Duplication between room response `heroes` and `required_state` when specifying `["m.room.member", "$LAZY"]`, e.g. - should we drop `heroes` if we are returning membership events? +1. The request `required_state` field is a bit confusing and uses special strings (like `"*"` and `"$LAZY"`). +1. Duplication between room response `heroes` and `required_state` when specifying `lazy_members`, e.g. should we drop + `heroes` if we are returning membership events? 1. Should the room response include the user's current membership? The client can always request it via `required_state`, but that may be wasted if the client doesn't need any other information from the member event. 1. Should we remove the `highlight_count` and `notification_count` fields, given clients increasingly must calculate @@ -636,6 +635,7 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. 1. Should we support timeline filtering? 1. Should we move `pos`, and other URL params, into the request body? This would allow web clients to cache the CORS requests, rather than having to pre-flight each request. +1. How do we make it so that the clients don't have to send up the same body each time? ## TODOs @@ -654,3 +654,4 @@ Changes from the initial implementation of simplified sliding sync. 4. Rename `unstable_expanded_timeline` to `expanded_timeline` 5. Add `lists` to room response 6. Add `membership` field to room response. +7. Change the format of `required_state` in the request. From de0fe5559a9f73b4ef246cd8b9921e0b6b72bc69 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 12 Sep 2025 10:08:43 +0100 Subject: [PATCH 20/77] Move URL params to request body --- proposals/4186-simplified-sliding-sync.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 98c64370d..7c26ec230 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -222,15 +222,6 @@ Examples of extensions, which will be specified elsewhere, are: The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/org.matrix.simplified_msc3575/sync`. -## Request URL parameters - -| Name | Type | Required | Comment | -| - | - | - | - | -| `pos` | string | No | Omitted if this is the first request of a connection. Otherwise, the `pos` token from the previous call to `/sync` | -| `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | -| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | - - ## Request Body ### Top-level @@ -238,6 +229,9 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | | `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given conn_id (empty or not). | +| `pos` | string | No | Omitted if this is the first request of a connection. Otherwise, the `pos` token from the previous call to `/sync` | +| `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | +| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | | `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). Max lists: 100. The list keys should be arbitrary strings which the client is using to refer to the list. Max length: 64 bytes. | | `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | | `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | @@ -570,8 +564,8 @@ this can be done. When the client is expecting a fast response (e.g. while expanding the lists), it should set the `timeout` parameter to 0 to ensure the server doesn't block waiting for new data. This can easily happen if the app starts and sends the first -request with a `since` parameter, if the client shows a spinner but doesn't set a timeout then the request may take a -long time to return (if there were no updates to return). +request with a `pos` parameter, if the client shows a spinner but doesn't set a timeout then the request may take a long +time to return (if there were no updates to return). # Alternatives @@ -633,8 +627,8 @@ in `/_matrix/client/versions` is `org.matrix.simplified_msc3575`. this themselves, and Synapse currently doesn't implement it nor does Element X use it. 1. Should we support subscribing to rooms the user is not a member of, i.e. peeking for world readable rooms. 1. Should we support timeline filtering? -1. Should we move `pos`, and other URL params, into the request body? This would allow web clients to cache the CORS - requests, rather than having to pre-flight each request. +1. Should we move `pos`, and other URL params, into the request body? This would allow web clients to cache the CORS + requests, rather than having to pre-flight each request. 1. How do we make it so that the clients don't have to send up the same body each time? ## TODOs @@ -655,3 +649,5 @@ Changes from the initial implementation of simplified sliding sync. 5. Add `lists` to room response 6. Add `membership` field to room response. 7. Change the format of `required_state` in the request. +8. Move URL parameters to the request body. This is so that on web every request doesn't need to be pre-flighted for + CORS. From b7b363e7f66f03bf65c757899aa8c828f52dd72a Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 22 Sep 2025 10:49:22 +0100 Subject: [PATCH 21/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Patrick Cloke --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 7c26ec230..a90d16cbc 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -331,7 +331,7 @@ An example that returns all the state except the create event: | - | - | - | - | | `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | | `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | -| `is_encrypted` | `[string]` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | +| `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | | `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | From 2d2890d18f8e24747ad75156d60092c709d557ab Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 11:47:15 +0100 Subject: [PATCH 22/77] Apply suggestions from code review Co-authored-by: Patrick Cloke --- proposals/4186-simplified-sliding-sync.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index a90d16cbc..e18acb663 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -116,7 +116,7 @@ ordering is determined by the server implementation. ### Subscriptions -Subscriptions are a rule that simply matches against a specified room ID, i.e. they allow the client to specify that a +Subscriptions are a rule that matches against a specified room ID, i.e. they allow the client to specify that a given room should always be returned (if there are updates). This is useful if e.g. the user has opened the room and the client always wants to get the latest data for that room. @@ -161,7 +161,7 @@ before being applied. The fields are combined by taking the "superset", i.e.: - Timeline limit — take the maximum timeline limit across all room configs. -- Required state — take the superset of the required state fields, i.e. if a state event would be returned by any room +- Required state — take the union of the required state fields, i.e. if a state event would be returned by any room config it is returned by the combined room config. @@ -209,7 +209,7 @@ response. The server MAY chose not to send that state if the client has previous For requesting data other than room events (such as account data or typing notifications), clients can use "extensions". These are split out into separate sections to a) make it easier for clients to specify just what they need, and b) to make it easier to extend in the future. -Examples of extensions, which will be specified elsewhere, are: +Examples of extensions, which will be specified in future MSCs, are: - To Device Messaging - End-to-End Encryption - Typing Notifications @@ -276,7 +276,7 @@ Describes the set of state that the server should return for the room. #### Lazy loaded memberships Room members can be lazily-loaded by using the `lazy_members` flag is set. Typically, when you view a room, you want to -retrieve all state events except for m.room.member events which you want to lazily load. To get this behaviour, clients +retrieve all state events except for `m.room.member` events which you want to lazily load. To get this behaviour, clients can send the following: This is (almost) the same as lazy loaded memberships in sync v2. When specified, the server will return the membership @@ -396,7 +396,7 @@ An example that returns all the state except the create event: Not all fields will be returned depending on the user's membership in the room. -All fields in this section may be omitted. When `initial` is set to `false` then an omitted filed means that the field +All fields in this section may be omitted. When `initial` is set to `false` then an omitted field means that the field remains unchanged from its previous value. When `initial` is set to `true` then an omitted field means that the field is not set, e.g. if `initial` is true and `name` is not set then the room has no name. Care must be taken by clients to ensure that if they previously saw that the room had a name, then it MUST be cleared if it receives a room response with @@ -442,7 +442,7 @@ When a user is or has been in the room, the following field are also returned: #### Invite/knock -For rooms the user is invited to or has knocked on, the client also gets the stripped state event: +For rooms the user is invited to or has knocked on, the client also gets the stripped state events: | Name | Type | Required | Comment | | - | - | - | - | From b405735f1b469e2a2a359741a7cd8e1fe019157e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:16:00 +0100 Subject: [PATCH 23/77] 0-indexed --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index e18acb663..997279856 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -243,7 +243,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | - | - | - | - | | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | | `required_state` | [RequiredStateRequest] | Yes | Required state for each room returned. | -| `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*. (This is a list of 2-tuples.) | +| `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a list of 2-tuples.) | | `filters` | `RoomFilter` | No | Filters to apply to the list before sorting. | ### `RoomSubscription` From 888e070a08d5cc7b56b1e399f7d608e04bc7e5db Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:16:24 +0100 Subject: [PATCH 24/77] s/RoomFilter/SlidingRoomFilter/ --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 997279856..eb5cf3532 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -244,7 +244,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | | `required_state` | [RequiredStateRequest] | Yes | Required state for each room returned. | | `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a list of 2-tuples.) | -| `filters` | `RoomFilter` | No | Filters to apply to the list before sorting. | +| `filters` | `SlidingRoomFilter` | No | Filters to apply to the list before sorting. | ### `RoomSubscription` @@ -325,7 +325,7 @@ An example that returns all the state except the create event: -### `RoomFilter` +### `SlidingRoomFilter` | Name | Type | Required | Comment | | - | - | - | - | From d6d5eddc2d400f32a6deac0a9a53065df682be9e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:19:13 +0100 Subject: [PATCH 25/77] Make RequiredStateRequest.include optional --- proposals/4186-simplified-sliding-sync.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index eb5cf3532..5f45a484f 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -242,7 +242,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | -| `required_state` | [RequiredStateRequest] | Yes | Required state for each room returned. | +| `required_state` | `RequiredStateRequest` | Yes | Required state for each room returned. | | `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a list of 2-tuples.) | | `filters` | `SlidingRoomFilter` | No | Filters to apply to the list before sorting. | @@ -251,7 +251,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | | `timeline_limit` | `int` | Yes | Same as in `SyncListConfig` | -| `required_state` | [RequiredStateRequest] | Yes | Same as in `SyncListConfig` | +| `required_state` | `RequiredStateRequest` | Yes | Same as in `SyncListConfig` | ### `RequiredStateRequest` @@ -259,7 +259,7 @@ Describes the set of state that the server should return for the room. | Name | Type | Required | Comment | | - | - | - | - | -| `include` | `[RequiredStateElement]` | Yes | The set of state to return (unless filtered out by `exclude`). | +| `include` | `[RequiredStateElement]` | No | The set of state to return (unless filtered out by `exclude`), if any. | | `exclude` | `[RequiredStateElement]` | No | The set of state to filter out before returning, if any. | | `lazy_members` | `bool` | No | Whether to enable lazy loaded membership behaviour. Defaults to false. | From 5b5e82b0ba84d727d1ce0350a0aa1a61e61895fb Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:20:29 +0100 Subject: [PATCH 26/77] Link to lazy-loaded membership --- proposals/4186-simplified-sliding-sync.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 5f45a484f..d24da673b 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -279,8 +279,9 @@ Room members can be lazily-loaded by using the `lazy_members` flag is set. Typic retrieve all state events except for `m.room.member` events which you want to lazily load. To get this behaviour, clients can send the following: -This is (almost) the same as lazy loaded memberships in sync v2. When specified, the server will return the membership -events for: +This is (almost) the same as [lazy loaded +memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in sync v2. When specified, the +server will return the membership events for: 1. All the senders of events in `timeline_events` unless previously returned. This ensures that the client can render all the timeline events without having to fetch more events from the server. 1. The target (i.e. `state_key`) of all membership events in `timeline_events`. From a2d36847978d4b8f8d56a5cd37037aac2070da71 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:25:22 +0100 Subject: [PATCH 27/77] Link to tagging --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index d24da673b..93ec4b811 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -338,7 +338,7 @@ An example that returns all the state except the create event: | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | | `room_name_like` | `string` | No | Filter the room name.

Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'. | | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | -| `not_tags` | `[string]` | No | Filter the room based on its room tags.

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | +| `not_tags` | `[string]` | No | Filter the room based on its [room tags](https://spec.matrix.org/v1.16/client-server-api/#room-tagging).

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | ### Example request From 4a38bb9e49dced284f602067441dd0ba45c1a259 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:28:41 +0100 Subject: [PATCH 28/77] is_dm being omitted implies not a DM room --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 93ec4b811..42fcaafbf 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -422,7 +422,7 @@ When a user is or has been in the room, the following field are also returned: | `name` | `string` | No | Room name or calculated room name. | | `avatar` | `string` | No | Room avatar | | `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. | -| `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). | +| `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). If absent the room is not a DM room. | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | From 1465e25e2cee62766f37a74e8b53a211f9e999df Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:31:59 +0100 Subject: [PATCH 29/77] Define 'just occured' --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 42fcaafbf..cdbeecc9f 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -429,7 +429,7 @@ When a user is or has been in the room, the following field are also returned: | `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | | `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events.

Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room.

Absence means `false` | -| `num_live` | `int` | No | The number of timeline events which have just occurred and are not historical. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | +| `num_live` | `int` | No | The number of timeline events which have "just occurred" and are not historical, i.e. that have happened since the previous sync request. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | | `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as sync `v2 m.joined_member_count`) | | `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | | `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2) | From 0fbd845188e1df8b518470b5d657f9fc79ae9b06 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:39:14 +0100 Subject: [PATCH 30/77] Note about resource consumption --- proposals/4186-simplified-sliding-sync.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index cdbeecc9f..719b94054 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -602,6 +602,9 @@ We can also add a flag to disable attempting to backfill over pagination (to mat Care must be taken, as with sync v2, to ensure that only the data that the user is authorized to see is returned in the response. +Servers SHOULD limit the amount of data that they store per-user to guard against resource exhaustion, e.g. limiting the +number of connections a device can have active. + # Unstable prefix From ce6f0f3165bd3fddacd88de1b4eb992ca89658a1 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:39:40 +0100 Subject: [PATCH 31/77] Note about resource consumption --- proposals/4186-simplified-sliding-sync.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 719b94054..b23b70d21 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -602,8 +602,9 @@ We can also add a flag to disable attempting to backfill over pagination (to mat Care must be taken, as with sync v2, to ensure that only the data that the user is authorized to see is returned in the response. -Servers SHOULD limit the amount of data that they store per-user to guard against resource exhaustion, e.g. limiting the -number of connections a device can have active. +Servers SHOULD limit the amount of data that they store per-user to guard against resource consumption, e.g. limiting +the number of connections a device can have active. This protects against malicious clients creating large numbers of +connections that get persisted to the database. # Unstable prefix From 985e71f8d4ccc38d90ddf3bfed0be9e8ba35610a Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 24 Sep 2025 14:42:06 +0100 Subject: [PATCH 32/77] Add note that servers can expire sync connections if response is too large --- proposals/4186-simplified-sliding-sync.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index b23b70d21..52250d9d9 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -606,6 +606,9 @@ Servers SHOULD limit the amount of data that they store per-user to guard agains the number of connections a device can have active. This protects against malicious clients creating large numbers of connections that get persisted to the database. +Servers MAY decide to expire the sync connection if the generated response on an incremental request is likely very +large or expensive to compute. + # Unstable prefix From 492db052d3e34449ee703dae31949ec887ad761e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:20:31 +0100 Subject: [PATCH 33/77] Remove ability to peek via room subscriptions --- proposals/4186-simplified-sliding-sync.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 52250d9d9..8b09d81db 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -120,11 +120,12 @@ Subscriptions are a rule that matches against a specified room ID, i.e. they all given room should always be returned (if there are updates). This is useful if e.g. the user has opened the room and the client always wants to get the latest data for that room. -The server MUST ensure that user has permission to see any information the server returns. However, the user need not be -in the room, e.g.clients can specify room IDs for world-readable rooms and they would be returned. +The server MUST ensure that user has permission to see any information the server returns. Currently, the user must +either be in the room, or be invited/knocked to the room. Otherwise, the room will not be returned in the response. > [!Note] -> Synapse currently does not support subscribing to rooms that the user is not part of. +> A future MSC may relax this requirement to allow peeking into world-readable rooms. + ### Room config From 1058ad7653dc5dc5b3ac25d47eeed0d32e2d72f1 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:51:04 +0100 Subject: [PATCH 34/77] Change 'ranges' to 'range' --- proposals/4186-simplified-sliding-sync.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 8b09d81db..eb4bd736b 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -244,7 +244,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | - | - | - | - | | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | | `required_state` | `RequiredStateRequest` | Yes | Required state for each room returned. | -| `ranges` | `[[int, int]]` | No | Sliding window ranges. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a list of 2-tuples.) | +| `range` | `[int, int]` | No | Sliding window range. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a 2-tuple.) | | `filters` | `SlidingRoomFilter` | No | Filters to apply to the list before sorting. | ### `RoomSubscription` @@ -350,7 +350,7 @@ An example that returns all the state except the create event: // Sliding Window API "lists": { "foo-list": { - "ranges": [ [0, 99] ], + "range": [0, 99], "required_state": { "include": [ {"type": "m.room.create", "state_key": ""}, @@ -527,14 +527,14 @@ Below are some potentially common patterns that clients may wish to use. ## Paginating room list -Pagination of the room list is achieved by the client increasing the ranges of one (or more) lists. +Pagination of the room list is achieved by the client increasing the range of one (or more) lists. For example an initial request might have a list called `all_rooms` specifying a range of `[0,19]` in the initial request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client may change the range to `[0,99]`, at which point the server will respond with the top 100 rooms that either a) weren’t sent down in the first request, or b) have updates since the first request. -Clients can increase and decrease the ranges as they see fit. A common approach would be to start with a small window +Clients can increase and decrease the range as they see fit. A common approach would be to start with a small window and grow that until the range covers all the rooms. After some threshold of the app being offline it may reduce the range back down and incrementally grow it again. This allows for ensuring that a limited amount of data is requested at once, to improve response times. @@ -573,7 +573,7 @@ time to return (if there were no updates to return). ## Pagination -In practice, having the client specify the ranges to use for the lists is often sub-optimal. The client generally wants +In practice, having the client specify the range to use for the lists is often sub-optimal. The client generally wants to have the sync request return as quickly as possible, but it doesn't know how much data the server has to return and so whether to increase or decrease the range. @@ -660,3 +660,4 @@ Changes from the initial implementation of simplified sliding sync. 7. Change the format of `required_state` in the request. 8. Move URL parameters to the request body. This is so that on web every request doesn't need to be pre-flighted for CORS. +9. Convert `ranges` to `range` in `SyncListConfig` in the request. From bade6bb540807b0d74008d7e6ef0050c09b169ab Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:52:27 +0100 Subject: [PATCH 35/77] Move room configs section for clarity --- proposals/4186-simplified-sliding-sync.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index eb4bd736b..c567e8f27 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -85,6 +85,10 @@ room is returned in the response if all of the following conditions hold: This MSC specifies two types of rule: a "list" and a "subscription". +Each rule also specifies a "room config", used to configure what data to return for a room that matches the rule. The +"room config" has two fields: the "timeline limit" and "required state". The "timeline limit" specifies the maximum +number of events to return in the timeline section of a room, and the "required state" specifies what state to return. + ### Lists Lists are the primary way of specifying the rooms the client is interested in. They operate against a sequence of rooms @@ -127,12 +131,6 @@ either be in the room, or be invited/knocked to the room. Otherwise, the room wi > A future MSC may relax this requirement to allow peeking into world-readable rooms. -### Room config - -Each rule also specifies a "room config", used to configure what data to return for a room that matches the rule. The -"room config" has two fields: the "timeline limit" and "required state". The "timeline limit" specifies the maximum -number of events to return in the timeline section of a room, and the "required state" specifies what state to return. - ## Room results A room is returned in the response if it matches at least one rule, and there is new data to return (if the room has From 474e681a491ef5655533bc3ac821a9c41abffda4 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:55:49 +0100 Subject: [PATCH 36/77] Allow 'room_name_like' to be implementation dependent --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index c567e8f27..731fcb7e2 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -335,7 +335,7 @@ An example that returns all the state except the create event: | `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | -| `room_name_like` | `string` | No | Filter the room name.

Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'. | +| `room_name_like` | `string` | No | Filter the room name.

Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'.

The exact matching semantics is left up to the implementation, for example a server may user full-text search rather than simple matching. | | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | | `not_tags` | `[string]` | No | Filter the room based on its [room tags](https://spec.matrix.org/v1.16/client-server-api/#room-tagging).

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | From 6e96ddbdf32311a17046d323535bd7f6ab4a0483 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:57:20 +0100 Subject: [PATCH 37/77] Clarify notifications are unthreaded --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 731fcb7e2..82983a768 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -431,8 +431,8 @@ When a user is or has been in the room, the following field are also returned: | `num_live` | `int` | No | The number of timeline events which have "just occurred" and are not historical, i.e. that have happened since the previous sync request. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | | `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as sync `v2 m.joined_member_count`) | | `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | -| `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2) | -| `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2) | +| `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2).

Does not included threaded notifications, which are returned in an extension. | +| `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2)

Does not included threaded notifications, which are returned in an extension. | | `lists` | `[string]` | No | The name of the lists that match this room. The field is omitted if it doesn't match any list and is included only due to a subscription. | From 2db894088ff516f4dbd6729f53e7cfb9ac92023c Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 14:59:55 +0100 Subject: [PATCH 38/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Patrick Cloke --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 82983a768..89d6e0f6a 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -529,7 +529,7 @@ Pagination of the room list is achieved by the client increasing the range of on For example an initial request might have a list called `all_rooms` specifying a range of `[0,19]` in the initial request, and the server will respond with the top 20 rooms (by most recently updated). On the second request the client -may change the range to `[0,99]`, at which point the server will respond with the top 100 rooms that either a) weren’t +may change the range to `[0,99]`, at which point the server will use the top 100 rooms and respond with the ones that either a) weren’t sent down in the first request, or b) have updates since the first request. Clients can increase and decrease the range as they see fit. A common approach would be to start with a small window From cc759db391987b285507ff5812dc1dce7f9557e8 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:21:49 +0100 Subject: [PATCH 39/77] Remove misleading sentence --- proposals/4186-simplified-sliding-sync.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 89d6e0f6a..b6dafd0e1 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -545,10 +545,8 @@ When the app starts up it configures a single list with a range of `[0, 19]` (to `timeline_limit` of 1. This returns quickly with the top 20 rooms (or just the changes in the top 20 rooms if a token was specified). -The client then increases the range (in the next request) to `[0, 99]`, which will return the next 80 rooms. The server -may sort the rooms differently than they are returned by the server (e.g. they may ignore reactions for sorting -purposes). Note: the range here matches 100 rooms, however we only send the 80 rooms that we didn't send down in the -previous request. +The client then increases the range (in the next request) to `[0, 99]`, which will return the next 80 rooms. Note: the +range here matches 100 rooms, however we only send the 80 rooms that we didn't send down in the previous request. The client can use room subscriptions, with a `timeline_limit` of 20, to preload history for the top rooms. This means that if the user clicks on one of the top rooms the app can immediately display a screens worth of history. (An From 98307d1da4a17832ef736e3c0db15fd79f65f9fe Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:30:07 +0100 Subject: [PATCH 40/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Eric Eastwood --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index b6dafd0e1..39b8405ae 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -185,7 +185,7 @@ If the server does send down extra events, it MUST set the `expanded_timeline` t the room. This behaviour is useful to reduce bandwidth in various cases. For example, a client may specify a list with range -`[0,99]` and a timeline limit of 10, plus a list with range `[100, ]` and timeline limit `1`. This would cause the +`[0,99]` and a `timeline_limit` of 10, plus a list with range `[100, ]` and `timeline_limit` of `1`. This would cause the server to return the most recent 10 events for rooms with recent activity, but only 1 event for older rooms (that the user is unlikely to visit). If an older room that we previously only returned with one timeline event receives a new event, we'll end up sending it down with not just the new event but the previous 10 events as well (despite having sent From ca38ae8bdb00f46dddcbf1376429a6347ae80b0f Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:34:31 +0100 Subject: [PATCH 41/77] Apply suggestions from code review Co-authored-by: Eric Eastwood --- proposals/4186-simplified-sliding-sync.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 39b8405ae..c6a8ef7d6 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -30,7 +30,7 @@ During initial design, the following goals were taken into account: - Time from opening of the app (when already logged in) to confident usability should be as low as possible. - Time from login on existing accounts to usability should be as low as possible. - Bandwidth should be minimized. -- Support lazy-loading of things like read receipts (and avoid sending unnecessary data to the client) +- Support lazy-loading of things like membership and read receipts (and avoid sending unnecessary data to the client) - Support informing the client when room state changes from under it, due to state resolution. - Clients should be able to work correctly without ever syncing in the full set of rooms they’re in. - Don’t incremental sync rooms you don’t care about. @@ -98,7 +98,7 @@ ordering semantics). Rooms that the user has been in but left are only included if the room was previously sent to the client in that connection. Rooms the user has been kicked or banned from will always be included. We do not include rooms the user has -left to save bandwidth and general efficiency (as the user knows they've left), but we still include kicked and banned +left themselves to save bandwidth and general efficiency (as the user knows they've left), but we still include kicked and banned rooms as a) this should be uncommon, and b) the user may not have seen that they've been kicked/banned from the room otherwise. @@ -227,8 +227,8 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | Name | Type | Required | Comment | | - | - | - | - | -| `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given conn_id (empty or not). | -| `pos` | string | No | Omitted if this is the first request of a connection. Otherwise, the `pos` token from the previous call to `/sync` | +| `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given `conn_id` (empty or not). | +| `pos` | string | No | Omitted if this is the first request of a connection (initial sync). Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | | `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | | `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). Max lists: 100. The list keys should be arbitrary strings which the client is using to refer to the list. Max length: 64 bytes. | @@ -243,7 +243,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | `timeline_limit` | `int` | Yes | The maximum number of timeline events to return per response. The server may cap this number. | | `required_state` | `RequiredStateRequest` | Yes | Required state for each room returned. | | `range` | `[int, int]` | No | Sliding window range. If this field is missing, no sliding window is used and all rooms are returned in this list. Integers are *inclusive*, and are 0-indexed. (This is a 2-tuple.) | -| `filters` | `SlidingRoomFilter` | No | Filters to apply to the list before sorting. | +| `filters` | `SlidingRoomFilter` | No | Filters to apply to the list. | ### `RoomSubscription` @@ -260,7 +260,7 @@ Describes the set of state that the server should return for the room. | - | - | - | - | | `include` | `[RequiredStateElement]` | No | The set of state to return (unless filtered out by `exclude`), if any. | | `exclude` | `[RequiredStateElement]` | No | The set of state to filter out before returning, if any. | -| `lazy_members` | `bool` | No | Whether to enable lazy loaded membership behaviour. Defaults to false. | +| `lazy_members` | `bool` | No | Whether to enable lazy loaded membership behaviour. Defaults to `false`. | #### `RequiredStateElement` @@ -281,7 +281,7 @@ can send the following: This is (almost) the same as [lazy loaded memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in sync v2. When specified, the server will return the membership events for: -1. All the senders of events in `timeline_events` unless previously returned. This ensures that the client can render +1. All the `senders` of events in `timeline_events` unless previously returned. This ensures that the client can render all the timeline events without having to fetch more events from the server. 1. The target (i.e. `state_key`) of all membership events in `timeline_events`. 1. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to @@ -331,7 +331,7 @@ An example that returns all the state except the create event: | - | - | - | - | | `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | | `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | -| `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `False`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | +| `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `false`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | | `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | @@ -398,8 +398,8 @@ Not all fields will be returned depending on the user's membership in the room. All fields in this section may be omitted. When `initial` is set to `false` then an omitted field means that the field remains unchanged from its previous value. When `initial` is set to `true` then an omitted field means that the field is -not set, e.g. if `initial` is true and `name` is not set then the room has no name. Care must be taken by clients to -ensure that if they previously saw that the room had a name, then it MUST be cleared if it receives a room response with +not set, e.g. if `initial` is `true` and `name` is not set then the room has no name. Care must be taken by clients to +ensure that if they previously saw that the room had a `name`, then it MUST be cleared if it receives a room response with `initial: true` and no `name` field. #### Common fields From e5b047aa8c2111d11a658c83c061721598fc660e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:42:23 +0100 Subject: [PATCH 42/77] Explain why we don't include more fields in invited rooms --- proposals/4186-simplified-sliding-sync.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index c6a8ef7d6..6da219b12 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -448,6 +448,10 @@ For rooms the user is invited to or has knocked on, the client also gets the str | - | - | - | - | | `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | +The reason these can't be included is because we don't have any of that information for remote invites and the user +isn't participating in the room yet so we shouldn't leak anything to them. We can only rely on the information in the +invite event. + > [!Note] > Synapse currently may inadvertently return extra fields from the previous section. From a48eb1ba1b25e0dcaba0e585a8ab8ec34a61d576 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:58:02 +0100 Subject: [PATCH 43/77] Remove 'room_name_like' as not implemented or used --- proposals/4186-simplified-sliding-sync.md | 1 - 1 file changed, 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 6da219b12..69085da66 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -335,7 +335,6 @@ An example that returns all the state except the create event: | `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | -| `room_name_like` | `string` | No | Filter the room name.

Case-insensitive partial matching e.g 'foo' matches 'abFooab'. The term 'like' is inspired by SQL 'LIKE', and the text here is similar to '%foo%'.

The exact matching semantics is left up to the implementation, for example a server may user full-text search rather than simple matching. | | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | | `not_tags` | `[string]` | No | Filter the room based on its [room tags](https://spec.matrix.org/v1.16/client-server-api/#room-tagging).

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | From 25479be35204934e0b4d2ed36d062f5ba751988b Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 15:59:43 +0100 Subject: [PATCH 44/77] Review comment --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 69085da66..76803aa84 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -86,8 +86,8 @@ room is returned in the response if all of the following conditions hold: This MSC specifies two types of rule: a "list" and a "subscription". Each rule also specifies a "room config", used to configure what data to return for a room that matches the rule. The -"room config" has two fields: the "timeline limit" and "required state". The "timeline limit" specifies the maximum -number of events to return in the timeline section of a room, and the "required state" specifies what state to return. +"room config" has two fields: the `timeline_limit` and `required_state`. The `timeline_limit` specifies the maximum +number of events to return in the timeline section of a room, and the `required_state` specifies what state to return. ### Lists From ac356a762d8d91a4239d4e758b643fdfb98a689c Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 16:15:37 +0100 Subject: [PATCH 45/77] Point to existing sync v2 heroes spec --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 76803aa84..e31573a4c 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -419,7 +419,7 @@ When a user is or has been in the room, the following field are also returned: | - | - | - | - | | `name` | `string` | No | Room name or calculated room name. | | `avatar` | `string` | No | Room avatar | -| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. | +| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. c.f. `m.heroes` section in the sync v2 [specification](https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3sync_response-200_roomsummary) | | `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). If absent the room is not a DM room. | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | From 31d18c18a0e45776254f6a4ff1ee31f591eb2795 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 17:15:01 +0100 Subject: [PATCH 46/77] Reword heroes section --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index e31573a4c..ec6a6f72b 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -419,7 +419,7 @@ When a user is or has been in the room, the following field are also returned: | - | - | - | - | | `name` | `string` | No | Room name or calculated room name. | | `avatar` | `string` | No | Room avatar | -| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. c.f. `m.heroes` section in the sync v2 [specification](https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3sync_response-200_roomsummary) | +| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. The same as the `m.heroes` section in the sync v2 [specification](https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3sync_response-200_roomsummary) | | `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). If absent the room is not a DM room. | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | From bbc82d2149919f6ceb7ed2c2d0e39ebb1e690034 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 17:15:15 +0100 Subject: [PATCH 47/77] Note in lazy loading section state resolution --- proposals/4186-simplified-sliding-sync.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index ec6a6f72b..7b77d543a 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -285,7 +285,9 @@ server will return the membership events for: all the timeline events without having to fetch more events from the server. 1. The target (i.e. `state_key`) of all membership events in `timeline_events`. 1. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to - cache the membership list without requiring the server to send all membership updates for large gaps. + cache the membership list without requiring the server to send all membership updates for large gaps. Note that + clients can't rely on seeing membership changes in the `timeline` section to keep the current state up-to-date, due + to state resolution. Memberships returned to the client due to `lazy_members` are *not* filtered by `exclude`. From 49bbe41005046b17272c9dabded828669621be02 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 17:17:45 +0100 Subject: [PATCH 48/77] Note why caching members is useful --- proposals/4186-simplified-sliding-sync.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 7b77d543a..ae16c6458 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -285,9 +285,10 @@ server will return the membership events for: all the timeline events without having to fetch more events from the server. 1. The target (i.e. `state_key`) of all membership events in `timeline_events`. 1. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to - cache the membership list without requiring the server to send all membership updates for large gaps. Note that - clients can't rely on seeing membership changes in the `timeline` section to keep the current state up-to-date, due - to state resolution. + cache the membership list without requiring the server to send all membership updates for large gaps. Caching is + useful as it reduces the frequency that clients have to fetch the full membership list from the server, which needs + to happen e.g. to send an encrypted message to the room. Note that clients can't rely on seeing membership changes in + the `timeline` section to keep the current state up-to-date, due to state resolution. Memberships returned to the client due to `lazy_members` are *not* filtered by `exclude`. From c4bf57d20e754830d89adf05411fc1724d04e8b3 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 25 Sep 2025 17:27:32 +0100 Subject: [PATCH 49/77] Mention `expanded_timeline` --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index ae16c6458..4b41dd9e9 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -588,8 +588,8 @@ separately. ## Timeline event trickling -If the `timeline_limit` is increased then the server will send down historic data (c.f. "Changing room configs"), which -allows the clients to easily preload more history in recent rooms. +If the `timeline_limit` is increased then the server will send down historic data (c.f. "Changing room configs") with +`expanded_timeline` set, which allows the clients to easily preload more history in recent rooms. This mechanism is fiddly to implement, and ends up resending down events that we have previously sent to the client. From 74e8e76d56e7c402217cb7e2923e07c053f10611 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 26 Sep 2025 10:30:05 +0100 Subject: [PATCH 50/77] Make the 'lists' request field sticky --- proposals/4186-simplified-sliding-sync.md | 24 +++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 4b41dd9e9..4a38c3600 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -64,7 +64,7 @@ the next request. A `pos` from an older response MUST NOT be used again. Doing s result in a fatal error response from the server. The server cannot assume that a client has received a response until it receives a new request with the `pos` token set -to the `pos` it returned in the response. The server must ensure that any per-connection state it tracks correctly +to the `pos` it returned in the response. The server MUST ensure that any per-connection state it tracks correctly handles receiving multiple requests with the latest `pos` token (e.g. the client retries the request or decides to cancel and resend a request with different parameters). Once a server has seen a request with a given `pos`, the server may clean up any per-connection state from before that `pos`. @@ -231,7 +231,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | `pos` | string | No | Omitted if this is the first request of a connection (initial sync). Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | | `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | -| `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). Max lists: 100. The list keys should be arbitrary strings which the client is using to refer to the list. Max length: 64 bytes. | +| `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). The list keys should be arbitrary strings which the client is using to refer to the list.

If omitted, the server will reuse the last `lists` field specified in the connection, if any. See "Sticky `lists` field" section below.

Max lists: 100.
Max list name length: 64 bytes. | | `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | | `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | @@ -341,6 +341,25 @@ An example that returns all the state except the create event: | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | | `not_tags` | `[string]` | No | Filter the room based on its [room tags](https://spec.matrix.org/v1.16/client-server-api/#room-tagging).

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | + +### Sticky `lists` field + +Typically, most sliding sync requests from a client will contain the same `lists` config each time, which can be quite +large. To improve efficiency, clients MAY omit the `lists` field in requests if it the same as the previous request. To update the config clients simply send the full updated `lists` field in the next request. + +Servers MUST track the latest `lists` fields that are used in the connection, and use it if the client omits the `lists` +field in subsequent sections. + +Care must be taken server-side to handle "cancelled"/"lost" requests (c.f. "Connections" section above), in the same way +as other per-connection state. As discussed in the "Connections" section, a client MAY send multiple requests reusing +the latest `pos` token, but with different `lists` fields. In this case the server MUST: + 1. retain the tracked `lists` field of the original request; and + 1. track each new `lists` field from the requests. + +Once the client uses a `pos` from one of the requests, the server can delete all the tracked `lists` from the other +requests. + + ### Example request ```jsonc @@ -663,3 +682,4 @@ Changes from the initial implementation of simplified sliding sync. 8. Move URL parameters to the request body. This is so that on web every request doesn't need to be pre-flighted for CORS. 9. Convert `ranges` to `range` in `SyncListConfig` in the request. +10. Make the `lists` request field "sticky". From 2cf4783058929c38ab471a37e76b340e1670328e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 29 Sep 2025 09:53:16 +0100 Subject: [PATCH 51/77] Clear up situation with rejected invites --- proposals/4186-simplified-sliding-sync.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 4a38c3600..bdef84c38 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -140,8 +140,10 @@ See the API section below for exactly what is returned. A subset may be returned view the room, e.g. if they are invited but not yet joined to the room. The server MUST not send any room information down that the user does not have permission to see. Specifically, the -server should only return rooms the user: is or has previously been joined to, is invited to, or has knocked on. An -exception is for rooms that are world readable and the user has subscribed to. +server should only return rooms the user: is or has previously been joined to, is invited to (or rejected an invite to), +or has knocked on (or had a knock rejected). + +In future MSCs, an exception *may* be added for rooms that are world readable and the user has subscribed to. The number of events in the timeline and what state is returned depends on the "room config" specified in the rules that the room matches from the request. If a given room matches multiple rules and therefore multiple room configs, then the @@ -433,7 +435,7 @@ These fields are always included no matter the membership. | `membership` | `string` | No | The current membership of the user, or omitted if user not in room (for peeking). | | `lists` | `[string]` | No | The name of the lists that match this room. The field is omitted if it doesn't match any list and is included only due to a subscription. | -#### Joined/left/banned +#### Currently or previously joined rooms When a user is or has been in the room, the following field are also returned: @@ -461,17 +463,18 @@ When a user is or has been in the room, the following field are also returned: > Synapse always returns 0 for `notification_count` and `highlight_count` -#### Invite/knock +#### Invite/knock/rejections -For rooms the user is invited to or has knocked on, the client also gets the stripped state events: +For rooms the user has not been joined to the client also gets the stripped state events. This is commonly the case for +invites or knocks, but can also be for when the user has rejected an invite. | Name | Type | Required | Comment | | - | - | - | - | | `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | -The reason these can't be included is because we don't have any of that information for remote invites and the user -isn't participating in the room yet so we shouldn't leak anything to them. We can only rely on the information in the -invite event. +The reason the full fields from the previous section can't be included is because we don't have any of that information +for remote invites and the user isn't participating in the room yet so we shouldn't leak anything to them. We can only +rely on the information in the invite/knock/etc event. > [!Note] > Synapse currently may inadvertently return extra fields from the previous section. From f5f6f834f66cf3459fe0eeb60c2edf4dfdd3b083 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 29 Sep 2025 09:56:37 +0100 Subject: [PATCH 52/77] Add note about 'last activity' being server dependent --- proposals/4186-simplified-sliding-sync.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index bdef84c38..12b81ed72 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -112,7 +112,9 @@ the room's index in the filtered list is within the list's range. #### Activity ordering Rooms are ordered by last activity, based on when the last event in the room was received by the server. The exact -ordering is determined by the server implementation. +ordering is determined by the server implementation. (Typically, this would be essentially based on when the "server +received" the last event in the room, however the preside definition depends on the server architecture, especially for +servers that are "distributed"). > [!Important] > Rooms are ***not*** ordered by "`bump_stamp`", a field returned in the room response. From 2e8be0ab556e099634b6fc1303dda5d6e0f5e3d7 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 1 Oct 2025 11:38:47 +0100 Subject: [PATCH 53/77] Remove sticky lists --- proposals/4186-simplified-sliding-sync.md | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 12b81ed72..7be520f77 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -235,7 +235,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/o | `pos` | string | No | Omitted if this is the first request of a connection (initial sync). Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | | `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | -| `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). The list keys should be arbitrary strings which the client is using to refer to the list.

If omitted, the server will reuse the last `lists` field specified in the connection, if any. See "Sticky `lists` field" section below.

Max lists: 100.
Max list name length: 64 bytes. | +| `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). The list keys should be arbitrary strings which the client is using to refer to the list.

Max lists: 100.
Max list name length: 64 bytes. | | `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | | `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | @@ -346,24 +346,6 @@ An example that returns all the state except the create event: | `not_tags` | `[string]` | No | Filter the room based on its [room tags](https://spec.matrix.org/v1.16/client-server-api/#room-tagging).

Takes priority over `tags`. For example, a room with tags A and B with filters `tags: [A]` `not_tags: [B]` would NOT be included because `not_tags` takes priority over `tags`. This filter is useful if your rooms list does NOT include the list of favourite rooms again. | -### Sticky `lists` field - -Typically, most sliding sync requests from a client will contain the same `lists` config each time, which can be quite -large. To improve efficiency, clients MAY omit the `lists` field in requests if it the same as the previous request. To update the config clients simply send the full updated `lists` field in the next request. - -Servers MUST track the latest `lists` fields that are used in the connection, and use it if the client omits the `lists` -field in subsequent sections. - -Care must be taken server-side to handle "cancelled"/"lost" requests (c.f. "Connections" section above), in the same way -as other per-connection state. As discussed in the "Connections" section, a client MAY send multiple requests reusing -the latest `pos` token, but with different `lists` fields. In this case the server MUST: - 1. retain the tracked `lists` field of the original request; and - 1. track each new `lists` field from the requests. - -Once the client uses a `pos` from one of the requests, the server can delete all the tracked `lists` from the other -requests. - - ### Example request ```jsonc @@ -687,4 +669,4 @@ Changes from the initial implementation of simplified sliding sync. 8. Move URL parameters to the request body. This is so that on web every request doesn't need to be pre-flighted for CORS. 9. Convert `ranges` to `range` in `SyncListConfig` in the request. -10. Make the `lists` request field "sticky". +10. Make the `lists` request field "sticky". From ef197a4c32b5c9131a27f41b0a984ab59878d4da Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 3 Oct 2025 10:26:52 +0100 Subject: [PATCH 54/77] Add double initial sync as an alternative --- proposals/4186-simplified-sliding-sync.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 7be520f77..1164f3a3f 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -605,6 +605,10 @@ to preload history for multiple rooms at once, and 2) `/messages` can be slow if We could implement a bulk `/messages` endpoint, where the client would specify multiple rooms and `prev_batch` tokens. We can also add a flag to disable attempting to backfill over pagination (to match the behaviour of the sync timeline). +Another alternative is to do one initial sync with a low `timeline_limit` and then another with the higher +`timeline_limit`. This is conceptually simple, though has the main downside of potentially duplicating a lot of data +(such as state). The approach also doesn't support the use case for having two lists so that rooms that bubble to the +top of the room list automatically get expanded timelines. # Security considerations From 5bcef01a5bf5d53b89e0ada0e65c4e66c0325eae Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 7 Oct 2025 15:17:12 +0100 Subject: [PATCH 55/77] Document the stable path --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 1164f3a3f..848772532 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -223,7 +223,7 @@ Examples of extensions, which will be specified in future MSCs, are: # API -The endpoint is a `POST` request with a JSON body to `/_matrix/client/unstable/org.matrix.simplified_msc3575/sync`. +The endpoint is a `POST` request with a JSON body to `/_matrix/client/v4/sync`. ## Request Body From 5bfe61c0aadbe4316136f6b8f3e1a16248f3383d Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 7 Oct 2025 15:45:15 +0100 Subject: [PATCH 56/77] Clarify list order --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 848772532..da2ca0b9b 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -432,7 +432,7 @@ When a user is or has been in the room, the following field are also returned: | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | -| `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event is the most recent. | +| `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event in the list is the most recent. | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | | `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events.

Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room.

Absence means `false` | | `num_live` | `int` | No | The number of timeline events which have "just occurred" and are not historical, i.e. that have happened since the previous sync request. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | From 3462215dc58a1c6483104aded76790cab6deb9c9 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Wed, 8 Oct 2025 11:41:07 +0100 Subject: [PATCH 57/77] Remove duplicate lists entry --- proposals/4186-simplified-sliding-sync.md | 1 - 1 file changed, 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index da2ca0b9b..a59bfcca6 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -440,7 +440,6 @@ When a user is or has been in the room, the following field are also returned: | `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | | `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2).

Does not included threaded notifications, which are returned in an extension. | | `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2)

Does not included threaded notifications, which are returned in an extension. | -| `lists` | `[string]` | No | The name of the lists that match this room. The field is omitted if it doesn't match any list and is included only due to a subscription. | > [!Note] From f90fcf558c75cb5c50eae96c67757383137080b4 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 10 Oct 2025 10:11:51 +0100 Subject: [PATCH 58/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Hubert Chathi --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index a59bfcca6..711ed054a 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -113,7 +113,7 @@ the room's index in the filtered list is within the list's range. Rooms are ordered by last activity, based on when the last event in the room was received by the server. The exact ordering is determined by the server implementation. (Typically, this would be essentially based on when the "server -received" the last event in the room, however the preside definition depends on the server architecture, especially for +received" the last event in the room, however the precise definition depends on the server architecture, especially for servers that are "distributed"). > [!Important] From 64964706c7fc0283234f755a3c4cdbe193a447ce Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 10 Oct 2025 10:12:03 +0100 Subject: [PATCH 59/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Hubert Chathi --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 711ed054a..328196b90 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -278,7 +278,7 @@ Describes the set of state that the server should return for the room. #### Lazy loaded memberships -Room members can be lazily-loaded by using the `lazy_members` flag is set. Typically, when you view a room, you want to +Room members can be lazily-loaded by using the `lazy_members` flag. Typically, when you view a room, you want to retrieve all state events except for `m.room.member` events which you want to lazily load. To get this behaviour, clients can send the following: From b255bf3f4c4bf9cd9a7d20a8b74c612275da59e4 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 10 Oct 2025 10:15:09 +0100 Subject: [PATCH 60/77] Clarify lazyloading --- proposals/4186-simplified-sliding-sync.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 328196b90..0953ea6e3 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -285,9 +285,10 @@ can send the following: This is (almost) the same as [lazy loaded memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in sync v2. When specified, the server will return the membership events for: -1. All the `senders` of events in `timeline_events` unless previously returned. This ensures that the client can render - all the timeline events without having to fetch more events from the server. -1. The target (i.e. `state_key`) of all membership events in `timeline_events`. +1. All the `senders` of events in `timeline_events`, excluding membership events that were previously returned. This + ensures that the client can render all the timeline events without having to fetch more events from the server. +1. The target (i.e. `state_key`) of all membership events in `timeline_events`, excluding membership events previously + returned. 1. All membership updates since the last sync when `limited` is false (i.e. non-gappy syncs). This allows the client to cache the membership list without requiring the server to send all membership updates for large gaps. Caching is useful as it reduces the frequency that clients have to fetch the full membership list from the server, which needs From de0a6a4491e1fc0b1423ed44c7d2cfbc4d82aae9 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:29:36 +0000 Subject: [PATCH 61/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 0953ea6e3..66eaf13e5 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -30,10 +30,10 @@ During initial design, the following goals were taken into account: - Time from opening of the app (when already logged in) to confident usability should be as low as possible. - Time from login on existing accounts to usability should be as low as possible. - Bandwidth should be minimized. -- Support lazy-loading of things like membership and read receipts (and avoid sending unnecessary data to the client) -- Support informing the client when room state changes from under it, due to state resolution. +- The API should support lazy-loading of things like membership and read receipts (and avoid sending unnecessary data to the client). +- The API should support informing the client when room state changes from under it, due to state resolution. - Clients should be able to work correctly without ever syncing in the full set of rooms they’re in. -- Don’t incremental sync rooms you don’t care about. +- Incremental sync responses should only include rooms that the user cares about. - Servers should not need to store all past since tokens. If a since token has been discarded we should gracefully degrade to initial sync. From 380bbf9833ba067b6d593e70d86bccb35b1c4add Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:30:16 +0000 Subject: [PATCH 62/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 66eaf13e5..12af7e89a 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -106,7 +106,7 @@ A "list" is then a set of filters (e.g. only match invites, or DM rooms, etc) pl *filtered* list of rooms. For example, a common list config would be no filters (i.e. all rooms) plus the range `[0,19]`, which would cause the server to return the top 20 rooms (by activity). -Specifically, a room matches a given list if after filtering the server maintained list of rooms by the list's filters, +Specifically, a room matches a given list if after filtering the server-maintained list of rooms by the list's filters, the room's index in the filtered list is within the list's range. #### Activity ordering From f53bae5b9887b87646b4cdf62019ef489d5efe1d Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:30:42 +0000 Subject: [PATCH 63/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 12af7e89a..ed72139d7 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -112,8 +112,8 @@ the room's index in the filtered list is within the list's range. #### Activity ordering Rooms are ordered by last activity, based on when the last event in the room was received by the server. The exact -ordering is determined by the server implementation. (Typically, this would be essentially based on when the "server -received" the last event in the room, however the precise definition depends on the server architecture, especially for +ordering is determined by the server implementation. (Typically, this would be essentially based on when the server +received the last event in the room, however the precise definition depends on the server architecture, especially for servers that are "distributed"). > [!Important] From 88568ac29f1c69f7c17c455e1a6ad3112fa92738 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:34:06 +0000 Subject: [PATCH 64/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index ed72139d7..a9208f41c 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -122,7 +122,7 @@ servers that are "distributed"). ### Subscriptions -Subscriptions are a rule that matches against a specified room ID, i.e. they allow the client to specify that a +A subscription is a rule that matches against a specified room ID, i.e. they allow the client to specify that a given room should always be returned (if there are updates). This is useful if e.g. the user has opened the room and the client always wants to get the latest data for that room. From e9a03fe1fa94bcffc258b2347f77cbde9bc30fcf Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:36:15 +0000 Subject: [PATCH 65/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index a9208f41c..807e1b595 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -135,8 +135,8 @@ either be in the room, or be invited/knocked to the room. Otherwise, the room wi ## Room results -A room is returned in the response if it matches at least one rule, and there is new data to return (if the room has -previously been sent to the client). +A room is returned in the response if it matches at least one rule, and either there is new data to return, or the room has +not previously been sent to the client. See the API section below for exactly what is returned. A subset may be returned if the user does not have permission to view the room, e.g. if they are invited but not yet joined to the room. From fb3eaca01a72be9210dd8899db9407e532ebdd97 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:36:47 +0000 Subject: [PATCH 66/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 807e1b595..7a586e26a 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -141,7 +141,7 @@ not previously been sent to the client. See the API section below for exactly what is returned. A subset may be returned if the user does not have permission to view the room, e.g. if they are invited but not yet joined to the room. -The server MUST not send any room information down that the user does not have permission to see. Specifically, the +The server MUST NOT send any room information down that the user does not have permission to see. Specifically, the server should only return rooms the user: is or has previously been joined to, is invited to (or rejected an invite to), or has knocked on (or had a knock rejected). From 5fd796926aab9bb8f42f906875fb3f6e278c0125 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:40:18 +0000 Subject: [PATCH 67/77] at least one field that is a superset of --- proposals/4186-simplified-sliding-sync.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 7a586e26a..cd510330d 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -172,8 +172,8 @@ The fields are combined by taking the "superset", i.e.: When a room matches one or more rules (i.e. is eligible to be returned in the sync response) that has previously been returned to the client, the server checks whether the combined room config is different than when the room was last -eligible to be returned. If the new config has fields that are a superset of the previous config, then the server -handles the config differently. +eligible to be returned. If the new config has at least one field that is a superset of the previous config, then the +server handles the config differently. #### Timeline events From 505fbf8f5d0a898b8b9c738364070bc42650cc31 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 13:59:04 +0000 Subject: [PATCH 68/77] Clarify timeline expansion --- proposals/4186-simplified-sliding-sync.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index cd510330d..da37e5c61 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -178,9 +178,15 @@ server handles the config differently. #### Timeline events Normally the timeline events returned are only the events that have been received since the last time the room was sent -to the client (i.e. only new events). However, if the `timeline_limit` has increased (to say `N`) the server SHOULD -ignore this and send down the latest `N` events, even if some of those events have previously been sent. The server MAY -ignore this behaviour if the server knows it has previously sent down all of the latest `N` events. +to the client (i.e. only new events). However, if the `timeline_limit` has increased (to say `N`) and the server has not +previously sent down all of the latest `N` events on the connection, the server SHOULD send down the latest `N` events +even if *some* of the events have previously been sent. If the server does not know if it has sent down all events it +SHOULD send down the latest `N` events. + +For example, say the latest events in the room are `A`, `B`, `C` and `D` (from earliest to latest), and the client has +previously seen `B`, `C` and `D` with a `timeline_limit` of 1. If the client increases the `timeline_limit` to 4 then +the server SHOULD return `A`, `B`, `C` and `D`, but if the client increases it instead to 3 then server does not need to +return any events as it knows the client already saw `B`, `C` and `D`. If the server does send down extra events, it MUST set the `expanded_timeline` to `true`. From 83a0e3f3431ec82b0619b3f7a9b7a0854a4b6c3c Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:05:02 +0000 Subject: [PATCH 69/77] Clarify example of timeline expansion --- proposals/4186-simplified-sliding-sync.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index da37e5c61..969aaa3a5 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -195,13 +195,13 @@ If the server does send down extra events, it MUST set the `expanded_timeline` t the room. This behaviour is useful to reduce bandwidth in various cases. For example, a client may specify a list with range -`[0,99]` and a `timeline_limit` of 10, plus a list with range `[100, ]` and `timeline_limit` of `1`. This would cause the -server to return the most recent 10 events for rooms with recent activity, but only 1 event for older rooms (that the -user is unlikely to visit). If an older room that we previously only returned with one timeline event receives a new -event, we'll end up sending it down with not just the new event but the previous 10 events as well (despite having sent -the second to last event previously). If the room then drops below the threshold (and so has timeline limit of 1), and -then receives another update, the server MAY remember that it has already sent the previous 10 events and only return -the latest one. +`[0,99]` and a `timeline_limit` of 10, plus a list with range `[100, ]` and `timeline_limit` of `1`. This would +cause the server to return the most recent 10 events for rooms with recent activity, but only 1 event for older rooms +(that the user is unlikely to visit). If an older room that we previously only returned with one timeline event receives +a new event, we'll end up sending it down with not just the new event but the previous 10 events as well (despite having +sent the second to last event previously). If the room then drops below the threshold (and so has `timeline_limit` of +1), and then receives another update (and so the `timeline_limit` increases back to 10), the server MAY choose to +remember that it has already sent the previous 10 events and only return the latest event. #### Required state From e3e46b16dff5fccdd1e7aa1df93b7c14e376d11e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:10:25 +0000 Subject: [PATCH 70/77] Unknown presence values result in a 400 --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 969aaa3a5..cf82a011c 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -240,7 +240,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/v4/sync`. | `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given `conn_id` (empty or not). | | `pos` | string | No | Omitted if this is the first request of a connection (initial sync). Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | -| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle. | +| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle.

An unknown value will result in a 400 error response with code `M_INVALID_PARAM`. | | `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). The list keys should be arbitrary strings which the client is using to refer to the list.

Max lists: 100.
Max list name length: 64 bytes. | | `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | | `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | From b42d2fe7a86c96a40adc0534c611f7c36d384648 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:26:46 +0000 Subject: [PATCH 71/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index cf82a011c..59b5ac041 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -346,7 +346,7 @@ An example that returns all the state except the create event: | `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | | `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | | `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `false`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | -| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `False`, no invited rooms are returned. If `True`, only invited rooms are returned. | +| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `false`, no invited rooms are returned. If `true`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | From 684a8ae9e668f6bf520dfecf0b44ec018d393dd3 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:35:28 +0000 Subject: [PATCH 72/77] Update proposals/4186-simplified-sliding-sync.md Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 59b5ac041..60fa208b8 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -343,7 +343,7 @@ An example that returns all the state except the create event: | Name | Type | Required | Comment | | - | - | - | - | -| `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the DM section of account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | +| `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the `m.direct` entry in account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | | `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | | `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `false`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | | `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `false`, no invited rooms are returned. If `true`, only invited rooms are returned. | From 27cd7eaa2e828d7452c83d4e8d1e8b8d62fa3a01 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:38:04 +0000 Subject: [PATCH 73/77] Replace references to sync v2 with /v3/sync --- proposals/4186-simplified-sliding-sync.md | 26 +++++++++++------------ 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 60fa208b8..68f1e7c62 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -17,7 +17,7 @@ usage. From a high level, sliding sync works by the client specifying a set of rules to match rooms (via "lists" and "subscriptions"). The server will use these rules to match relevant rooms to the user, and return any rooms that have had changes since the previous time the room was returned to the client. If no rooms have changes, the server will block -waiting for a change (aka long-polling, like sync v2). +waiting for a change (aka long-polling, like `/v3/sync`). By judicious use of lists and subscriptions the client can control exactly what data is returned when, and help ensure that it doesn't request "too much" data at once (thus avoiding slow responses). @@ -240,7 +240,7 @@ The endpoint is a `POST` request with a JSON body to `/_matrix/client/v4/sync`. | `conn_id` | `string` | No | An optional string to identify this connection to the server. Only one sliding sync connection is allowed per given `conn_id` (empty or not). | | `pos` | string | No | Omitted if this is the first request of a connection (initial sync). Otherwise, the `pos` token from the previous call to `/sync` | | `timeout` | int | No | How long to wait for new events in milliseconds. If omitted the response is always returned immediately, even if there are no changes. Ignored when no `pos` is set. | -| `set_presence` | string | No | Same as in sync v2, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle.

An unknown value will result in a 400 error response with code `M_INVALID_PARAM`. | +| `set_presence` | string | No | Same as in `/v3/sync`, controls whether the client is automatically marked as online by polling this API.

If this parameter is omitted then the client is automatically marked as online when it uses this API. Otherwise if the parameter is set to “offline” then the client is not marked as being online when it uses this API. When set to “unavailable”, the client is marked as being idle.

An unknown value will result in a 400 error response with code `M_INVALID_PARAM`. | | `lists` | `{string: SyncListConfig}` | No | Sliding window API. A map of list key to list information (`SyncListConfig`). The list keys should be arbitrary strings which the client is using to refer to the list.

Max lists: 100.
Max list name length: 64 bytes. | | `room_subscriptions` | `{string: RoomSubscription}` | No | A map of room ID to room subscription information. Used to subscribe to a specific room. Sometimes clients know exactly which room they want to get information about e.g by following a permalink or by refreshing a webapp currently viewing a specific room. The sliding window API alone is insufficient for this use case because there's no way to say "please track this room explicitly". | | `extensions` | `{string: ExtensionConfig}` | No | A map of extension key to extension config. Different extensions have different configuration formats. | @@ -289,8 +289,8 @@ retrieve all state events except for `m.room.member` events which you want to la can send the following: This is (almost) the same as [lazy loaded -memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in sync v2. When specified, the -server will return the membership events for: +memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in `/v3/sync`. When specified, +the server will return the membership events for: 1. All the `senders` of events in `timeline_events`, excluding membership events that were previously returned. This ensures that the client can render all the timeline events without having to fetch more events from the server. 1. The target (i.e. `state_key`) of all membership events in `timeline_events`, excluding membership events previously @@ -434,19 +434,19 @@ When a user is or has been in the room, the following field are also returned: | - | - | - | - | | `name` | `string` | No | Room name or calculated room name. | | `avatar` | `string` | No | Room avatar | -| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. The same as the `m.heroes` section in the sync v2 [specification](https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3sync_response-200_roomsummary) | +| `heroes` | `[StrippedHero]` | No | A truncated list of users in the room that can be used to calculate the room name. Will first include joined users, then invited users, and then finally left users. The same as the `m.heroes` section in the `/v3/sync` [specification](https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3sync_response-200_roomsummary) | | `is_dm` | `bool` | No | Flag to specify whether the room is a direct-message room (according to account data). If absent the room is not a DM room. | | `initial` | `bool` | No | Flag which is set when this is the first time the server is sending this data on this connection, or if the client should replace all room data with what is returned. Clients can use this flag to replace or update their local state. The absence of this flag means `false`. | | `expanded_timeline` | `bool` | No | Flag which is set if we're returning more historic events due to the timeline limit having increased. See "Changing room configs" section. | | `required_state` | `[Event\|StateStub]` | No | Changes in the current state of the room.

To handle state being deleted, the list may include a `StateStub` type (c.f. schema below) that only has `type` and `state_key` fields. The presence or absence of `content` field can be used to differentiate between the two cases. | -| `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per sync v2.

The last event in the list is the most recent. | +| `timeline_events` | `[Event]` | No | The latest events in the room. May not include all events if e.g. there were more events than the configured `timeline_limit`, c.f. the `limited` field.

If `limited` is true then we include bundle aggregations for the event, as per `/v3/sync`.

The last event in the list is the most recent. | | `prev_batch` | `string` | No | A token that can be passed as a start parameter to the `/rooms//messages` API to retrieve earlier messages. | | `limited` | `bool` | No | True if there are more events since the previous sync than were included in the `timeline_events` field, or that the client should paginate to fetch more events.

Note that server may return fewer than the requested number of events and still set `limited` to true, e.g. because there is a gap in the history the server has for the room.

Absence means `false` | | `num_live` | `int` | No | The number of timeline events which have "just occurred" and are not historical, i.e. that have happened since the previous sync request. The last `N` events are 'live' and should be treated as such.

This is mostly useful to e.g. determine whether a given `@mention` event should make a noise or not. Clients cannot rely solely on the absence of `initial: true` to determine live events because if a room not in the sliding window bumps into the window because of an `@mention` it will have `initial: true` yet contain a single live event (with potentially other old events in the timeline). | -| `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as sync `v2 m.joined_member_count`) | -| `invited_count` | `int` | No | The number of users with membership of invite. (same as sync v2 `m.invited_member_count`) | -| `notification_count` | `int` | No | The total number of unread notifications for this room. (same as sync v2).

Does not included threaded notifications, which are returned in an extension. | -| `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as sync v2)

Does not included threaded notifications, which are returned in an extension. | +| `joined_count` | `int` | No | The number of users with membership of join, including the client's own user ID. (same as `/v3/sync` `m.joined_member_count`) | +| `invited_count` | `int` | No | The number of users with membership of invite. (same as `/v3/sync` `m.invited_member_count`) | +| `notification_count` | `int` | No | The total number of unread notifications for this room. (same as `/v3/sync`).

Does not included threaded notifications, which are returned in an extension. | +| `highlight_count` | `int` | No | The number of unread notifications for this room with the highlight flag set. (same as `/v3/sync`)

Does not included threaded notifications, which are returned in an extension. | > [!Note] @@ -460,7 +460,7 @@ invites or knocks, but can also be for when the user has rejected an invite. | Name | Type | Required | Comment | | - | - | - | - | -| `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in sync v2. | +| `stripped_state` | `[StrippedState]` | Yes | Stripped state events (for rooms where the user is invited). Same as `rooms.invite.$room_id.invite_state` for invites in `/v3/sync`. | The reason the full fields from the previous section can't be included is because we don't have any of that information for remote invites and the user isn't participating in the room yet so we shouldn't leak anything to them. We can only @@ -618,8 +618,8 @@ top of the room list automatically get expanded timelines. # Security considerations -Care must be taken, as with sync v2, to ensure that only the data that the user is authorized to see is returned in the -response. +Care must be taken, as with `/v3/sync`, to ensure that only the data that the user is authorized to see is returned in +the response. Servers SHOULD limit the amount of data that they store per-user to guard against resource consumption, e.g. limiting the number of connections a device can have active. This protects against malicious clients creating large numbers of From 0c39b4b2abfb358e96a4b4ac05630b7c6989ac26 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:39:14 +0000 Subject: [PATCH 74/77] Add note about matching against state_key --- proposals/4186-simplified-sliding-sync.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 68f1e7c62..3b3781e58 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -278,7 +278,7 @@ Describes the set of state that the server should return for the room. | Name | Type | Required | Comment | | - | - | - | - | | `type` | `string` | No | The event type to match. If omitted then matches all types. | -| `state_key` | `string` | No | The event state key to match. If omitted then matches all state keys. | +| `state_key` | `string` | No | The event state key to match. If omitted then matches all state keys.

Note: it is possible to match a specific state key, for all event types, by specifying `state_key` but leaving `type` unset. | From 4fa86c90bf53d21dd2d869542dfbda3c86cb5e4f Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 27 Nov 2025 14:41:03 +0000 Subject: [PATCH 75/77] Add back missing example --- proposals/4186-simplified-sliding-sync.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 3b3781e58..47f06ac10 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -288,6 +288,15 @@ Room members can be lazily-loaded by using the `lazy_members` flag. Typically, w retrieve all state events except for `m.room.member` events which you want to lazily load. To get this behaviour, clients can send the following: +```jsonc + { + "required_state": { + "include": [{}], // An empty object matches everything + "lazy_members": true + } + } +``` + This is (almost) the same as [lazy loaded memberships](https://spec.matrix.org/v1.16/client-server-api/#lazy-loading-room-members) in `/v3/sync`. When specified, the server will return the membership events for: From 69dc79812dfacb1d376bab99d627731610deee21 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 28 Nov 2025 10:23:16 +0000 Subject: [PATCH 76/77] Rename to is_invited --- proposals/4186-simplified-sliding-sync.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 47f06ac10..1fcac6103 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -355,7 +355,7 @@ An example that returns all the state except the create event: | `is_dm` | `bool` | No | Flag which only returns rooms present (or not) in the `m.direct` entry in account data.

If unset, both DM rooms and non-DM rooms are returned. If False, only non-DM rooms are returned. If True, only DM rooms are returned. | | `spaces` | `[string]` | No | Filter the room based on the space they belong to according to `m.space.child` state events.

If multiple spaces are present, a room can be part of any one of the listed spaces (OR'd). The server will inspect the `m.space.child` state events for the JOINED space room IDs given. Servers MUST NOT navigate subspaces. It is up to the client to give a complete list of spaces to navigate. Only rooms directly mentioned as `m.space.child` events in these spaces will be returned. Unknown spaces or spaces the user is not joined to will be ignored. | | `is_encrypted` | `bool` | No | Flag which only returns rooms which have an `m.room.encryption` state event.

If unset, both encrypted and unencrypted rooms are returned. If `false`, only unencrypted rooms are returned. If `True`, only encrypted rooms are returned. | -| `is_invite` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `false`, no invited rooms are returned. If `true`, only invited rooms are returned. | +| `is_invited` | `bool` | No | Flag which only returns rooms the user is currently invited to.

If unset, both invited and joined rooms are returned. If `false`, no invited rooms are returned. If `true`, only invited rooms are returned. | | `room_types` | `[string \| null]` | No | If specified, only rooms where the `m.room.create` event has a `type` matching one of the strings in this array will be returned.

If this field is unset, all rooms are returned regardless of type. This can be used to get the initial set of spaces for an account. For rooms which do not have a room type, use `null` to include them. | | `not_room_types` | `[string \| null]` | No | Same as `room_types` but inverted.

This can be used to filter out spaces from the room list. If a type is in both `room_types` and `not_room_types`, then `not_room_types` wins and they are not included in the result. | | `tags` | `[string]` | No | Filter the room based on its room tags.

If multiple tags are present, a room can have any one of the listed tags (OR'd). | @@ -689,3 +689,4 @@ Changes from the initial implementation of simplified sliding sync. CORS. 9. Convert `ranges` to `range` in `SyncListConfig` in the request. 10. Make the `lists` request field "sticky". +11. Rename `is_invite` to `is_invited`. From 564839aae1632129f350306827a60b5d57f7c379 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 28 Nov 2025 10:30:42 +0000 Subject: [PATCH 77/77] Remove spurious receipts claim --- proposals/4186-simplified-sliding-sync.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/proposals/4186-simplified-sliding-sync.md b/proposals/4186-simplified-sliding-sync.md index 1fcac6103..e28e21ba2 100644 --- a/proposals/4186-simplified-sliding-sync.md +++ b/proposals/4186-simplified-sliding-sync.md @@ -1,11 +1,10 @@ # MSC4186: Simplified Sliding Sync The current `/sync` endpoint scales badly as the number of rooms on an account increases. It scales badly because all -rooms are returned to the client, incremental syncs are unbounded and slow down based on how long the user has been -offline, and clients cannot opt-out of a large amount of extraneous data such as receipts. On large accounts with -thousands of rooms, the initial sync operation can take tens of minutes to perform. This significantly delays the -initial login to Matrix clients, and also makes incremental sync very heavy when resuming after any significant pause in -usage. +rooms are returned to the client and incremental syncs are unbounded and slow down based on how long the user has been +offline. On large accounts with thousands of rooms, the initial sync operation can take tens of minutes to perform. This +significantly delays the initial login to Matrix clients, and also makes incremental sync very heavy when resuming after +any significant pause in usage. > [!Note] > This is a “simplified” version of the sliding sync API proposed in