From ad61e4f6bef5f7eff138c0c45d451627efbbc235 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 16 Sep 2025 12:42:25 +0200 Subject: [PATCH 01/12] MSC4356: Recently used emoji Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 65 ++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 proposals/4356-recent-emoji.md diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md new file mode 100644 index 000000000..bc36440b6 --- /dev/null +++ b/proposals/4356-recent-emoji.md @@ -0,0 +1,65 @@ +# MSC4356: Recently used emoji + +Like other chat platforms, Matrix supports emoji as a way to visually express ideas or emotions. In +practice, most people use a limited set of emoji only. Since emoji are commonly used as a quick way +to react to something, it is desirable for clients to offer users shortcuts to their favorite emoji. +Some emoji picker libraries support this feature by locally tracking emoji usage. This doesn't work +well in a multi-device environment, however, because such history cannot easily be shared between +clients. + +This proposal introduces a way for clients to maintain a shared storage of recently used emoji to +enable emoji suggestions across clients. + +## Proposal + +A new global account data event `m.recent_emoji` is introduced. In `content`, it contains a single +property `recent_emoji` that is an array where each element is itself an array. The first element in +this nested array is the emoji, the second element is a counter for how often it was used. The outer +`recent_emoji` array is ordered descendingly by last usage time. + +``` json5 +{ + "type": "m.recent_emoji", + "content": { + "recent_emoji": [ + [ "😅", 7 ], // Most recently used, 7 times overall + [ "👍", 84 ], // Second most recently used, 84 times overall + ... + } +} +``` + +When an emoji is used in a message or an annotation, the sending client moves (or adds) it to the +beginning of the `recent_emoji` array and increments (or initializes) its counter. + +As new emoji are being used, clients SHOULD limit the length of the `recent_emoji` array by dropping +elements from the end. A RECOMMENDED maximum length is 100 emoji. + +Clients MAY freely customise the logic for generating recommendations from the stored emoji. As an +example, they could select the 24 first (= most recently used) emoji and stably sort them by their +counters (so that more recently used emoji are ordered first on ties). + +## Potential issues + +Clients could choose wildly different ways to generate recommendations from the shared storage +leading to significantly different UX across clients. + +## Alternatives + +Further metadata such as the concrete access time or the room could be tracked together with emoji. +It is unclear, however, if this would lead to materially better suggestions, however. + +## Security considerations + +This proposal doesn't mandate encrypting the `m.recent_emoji` account data event. Since emoji are +most commonly used in annotations which are not encrypted, servers could already track and abuse +this information today, however. + +## Unstable prefix + +While this MSC is not considered stable, `m.recent_emoji` should be referred to as +`io.element.recent_emoji`. + +## Dependencies + +None. From 6e1cf39a015d025a51ae95a987606ba0118a85aa Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 17 Sep 2025 08:21:08 +0200 Subject: [PATCH 02/12] Add note about the absence of further counter reset mechanisms --- proposals/4356-recent-emoji.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index bc36440b6..4206c3e0e 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -33,7 +33,9 @@ When an emoji is used in a message or an annotation, the sending client moves (o beginning of the `recent_emoji` array and increments (or initializes) its counter. As new emoji are being used, clients SHOULD limit the length of the `recent_emoji` array by dropping -elements from the end. A RECOMMENDED maximum length is 100 emoji. +elements from the end. A RECOMMENDED maximum length is 100 emoji. Apart from this, no other +mechanism for resetting counters is mandated. [RFC7159] suggests an upper boundary of 2^53-1 for +interoperable integers which seems sufficiently large for all practical purposes. Clients MAY freely customise the logic for generating recommendations from the stored emoji. As an example, they could select the 24 first (= most recently used) emoji and stably sort them by their @@ -63,3 +65,5 @@ While this MSC is not considered stable, `m.recent_emoji` should be referred to ## Dependencies None. + + [RFC7159]: https://datatracker.ietf.org/doc/html/rfc7159 From 1ff30d4b24cc1cac8a4f2ee1d0c5fdc32ecf6c46 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 17 Sep 2025 16:06:02 +0200 Subject: [PATCH 03/12] Make proposal compatible with MSC4027 Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 4206c3e0e..7eaaae4e8 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -32,6 +32,11 @@ this nested array is the emoji, the second element is a counter for how often it When an emoji is used in a message or an annotation, the sending client moves (or adds) it to the beginning of the `recent_emoji` array and increments (or initializes) its counter. +When an image is sent as an inline image or in a reaction (using [MSC4027]), the `mxc://` URI of the +image MAY be used as the "emoji" in this event. Clients which do not support such use of images MUST +tolerate the existence of `mxc://` entries, e.g. by ignoring the entries when deciding what to +display to the user, while still preserving them when modifying the list. + As new emoji are being used, clients SHOULD limit the length of the `recent_emoji` array by dropping elements from the end. A RECOMMENDED maximum length is 100 emoji. Apart from this, no other mechanism for resetting counters is mandated. [RFC7159] suggests an upper boundary of 2^53-1 for @@ -66,4 +71,5 @@ While this MSC is not considered stable, `m.recent_emoji` should be referred to None. + [MSC4027]: https://github.com/matrix-org/matrix-spec-proposals/pull/4027 [RFC7159]: https://datatracker.ietf.org/doc/html/rfc7159 From 58d45ad44a7f8e6f4d1e63b7b03b314014ce5b56 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 17 Sep 2025 16:11:08 +0200 Subject: [PATCH 04/12] Specify explicit upper limit of 2^53-1 Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 7eaaae4e8..0b5e9495d 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -14,8 +14,8 @@ enable emoji suggestions across clients. A new global account data event `m.recent_emoji` is introduced. In `content`, it contains a single property `recent_emoji` that is an array where each element is itself an array. The first element in -this nested array is the emoji, the second element is a counter for how often it was used. The outer -`recent_emoji` array is ordered descendingly by last usage time. +this nested array is the emoji, the second element is a counter (\<= 2^53-1) for how often it was +used. The outer `recent_emoji` array is ordered descendingly by last usage time. ``` json5 { @@ -39,8 +39,8 @@ display to the user, while still preserving them when modifying the list. As new emoji are being used, clients SHOULD limit the length of the `recent_emoji` array by dropping elements from the end. A RECOMMENDED maximum length is 100 emoji. Apart from this, no other -mechanism for resetting counters is mandated. [RFC7159] suggests an upper boundary of 2^53-1 for -interoperable integers which seems sufficiently large for all practical purposes. +mechanism for resetting counters is mandated as the upper boundary of 2^53-1 seems sufficiently +large for all practical purposes. Clients MAY freely customise the logic for generating recommendations from the stored emoji. As an example, they could select the 24 first (= most recently used) emoji and stably sort them by their @@ -72,4 +72,3 @@ While this MSC is not considered stable, `m.recent_emoji` should be referred to None. [MSC4027]: https://github.com/matrix-org/matrix-spec-proposals/pull/4027 - [RFC7159]: https://datatracker.ietf.org/doc/html/rfc7159 From d67a3f874441c52d856d22f84c1af4ed7341217c Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 20 Oct 2025 14:43:38 +0200 Subject: [PATCH 05/12] Remove duplicate "however" Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/4356-recent-emoji.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 0b5e9495d..378ace8a4 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -54,7 +54,7 @@ leading to significantly different UX across clients. ## Alternatives Further metadata such as the concrete access time or the room could be tracked together with emoji. -It is unclear, however, if this would lead to materially better suggestions, however. +It is unclear, however, if this would lead to materially better suggestions. ## Security considerations From 594df055385ad132fd4d3305fda828ac14bca00d Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 20 Oct 2025 15:05:40 +0200 Subject: [PATCH 06/12] Add guards against counters being too small or too large Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 378ace8a4..2473c89ad 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -62,6 +62,13 @@ This proposal doesn't mandate encrypting the `m.recent_emoji` account data event most commonly used in annotations which are not encrypted, servers could already track and abuse this information today, however. +Malicious or buggy clients could cause undefined behavior on other clients by setting emoji counters +to negative values or the maximum allowed value. To prevent this, clients SHOULD drop any emojis +with a count below 0 from the list. When observing a count of 2^53-1 for an emoji, clients SHOULD +normalise the entire set of emoji by dividing all counts by two and rounding up. To prevent race +conditions, both of these changes SHOULD only be applied when the client is updating the account +data event due to local emoji usage. + ## Unstable prefix While this MSC is not considered stable, `m.recent_emoji` should be referred to as From 7855bdd51b4a8a3c890557060427c63b5e9c0a46 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 13 Nov 2025 08:18:22 +0100 Subject: [PATCH 07/12] Explain why the time of last use is not tracked Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 2473c89ad..ae34c3db6 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -54,7 +54,10 @@ leading to significantly different UX across clients. ## Alternatives Further metadata such as the concrete access time or the room could be tracked together with emoji. -It is unclear, however, if this would lead to materially better suggestions. +It is unclear, however, if this would lead to materially better suggestions. A last-used timestamp +could also be used to cull emoji that haven't been used in a very long time. Given that implementations +are already encouraged to limit the maximum number of tracked emoji, this doesn't appear necessary, +however. ## Security considerations From 82a600940008d4353ed0b1a3052659575990c475 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 25 Nov 2025 10:50:55 +0100 Subject: [PATCH 08/12] Switch from nested array to an array of objects Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index ae34c3db6..01ed45990 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -13,17 +13,28 @@ enable emoji suggestions across clients. ## Proposal A new global account data event `m.recent_emoji` is introduced. In `content`, it contains a single -property `recent_emoji` that is an array where each element is itself an array. The first element in -this nested array is the emoji, the second element is a counter (\<= 2^53-1) for how often it was -used. The outer `recent_emoji` array is ordered descendingly by last usage time. +property `recent_emoji` that is an array where each element is an object with the following properties: + +- `emoji` (string, required): The Unicode emoji as string. +- `total` (number, required): The number of times the emoji has been used (\<= 2^53-1). + +The outer `recent_emoji` array is ordered descendingly by last usage time. ``` json5 { "type": "m.recent_emoji", "content": { "recent_emoji": [ - [ "😅", 7 ], // Most recently used, 7 times overall - [ "👍", 84 ], // Second most recently used, 84 times overall + // Most recently used, 7 times overall + { + "emoji": "😅", + "total": 7 + }, + // Second most recently used, 84 times overall + { + "emoji": "👍", + "total": 84 + }, ... } } @@ -59,6 +70,12 @@ could also be used to cull emoji that haven't been used in a very long time. Giv are already encouraged to limit the maximum number of tracked emoji, this doesn't appear necessary, however. +Rather than an array of objects, a nested array could be used. This approach has been used in early +implementations of this proposal under the `io.element.recent_emoji` account data event. While this +results in a more compact storage layout, it doesn't lend itself well to future extension. Additionally, +given request compression and the common expectable length of the array, performance benefits should +be negligible. + ## Security considerations This proposal doesn't mandate encrypting the `m.recent_emoji` account data event. Since emoji are @@ -75,7 +92,7 @@ data event due to local emoji usage. ## Unstable prefix While this MSC is not considered stable, `m.recent_emoji` should be referred to as -`io.element.recent_emoji`. +`io.github.Johennes.msc4356.recent_emoji`. ## Dependencies From cb46b7197883cb6cd88df5ba92cb655d81f130c2 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 25 Nov 2025 10:59:50 +0100 Subject: [PATCH 09/12] Clarify why no recommendation algorithm is prescribed Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 01ed45990..023f062ce 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -60,7 +60,8 @@ counters (so that more recently used emoji are ordered first on ties). ## Potential issues Clients could choose wildly different ways to generate recommendations from the shared storage -leading to significantly different UX across clients. +leading to significantly different UX across clients. A "correct" algorithm is not currently known, +however, and will require future UI exploration in client implementations. ## Alternatives From 403d4d01ccd7b9083b8b42ebd46f40bbde6f4383 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 25 Nov 2025 11:02:24 +0100 Subject: [PATCH 10/12] Relax emoji usage requirements Signed-off-by: Johannes Marbach --- proposals/4356-recent-emoji.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 023f062ce..a495fdd4f 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -40,8 +40,9 @@ The outer `recent_emoji` array is ordered descendingly by last usage time. } ``` -When an emoji is used in a message or an annotation, the sending client moves (or adds) it to the -beginning of the `recent_emoji` array and increments (or initializes) its counter. +When an emoji is used, the sending client moves (or adds) it to the beginning of the `recent_emoji` +array and increments (or initializes) its counter. What exactly constitutes emoji usage is left as +an implementation detail for clients. Obvious choices are sending emoji in messages or annotations. When an image is sent as an inline image or in a reaction (using [MSC4027]), the `mxc://` URI of the image MAY be used as the "emoji" in this event. Clients which do not support such use of images MUST From a42630f28068909c92a31e6932b87f5fdf9adb71 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 25 Nov 2025 14:27:59 +0100 Subject: [PATCH 11/12] Use lowercase in prefix. Co-authored-by: Tulir Asokan --- proposals/4356-recent-emoji.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index a495fdd4f..3fc013871 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -94,7 +94,7 @@ data event due to local emoji usage. ## Unstable prefix While this MSC is not considered stable, `m.recent_emoji` should be referred to as -`io.github.Johennes.msc4356.recent_emoji`. +`io.github.johennes.msc4356.recent_emoji`. ## Dependencies From 2bfd09e6aa7757767d1bb67a129b9ea53dcb1b25 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 2 Dec 2025 20:55:50 +0100 Subject: [PATCH 12/12] Broaden scope of "invalid" emoji entries --- proposals/4356-recent-emoji.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4356-recent-emoji.md b/proposals/4356-recent-emoji.md index 3fc013871..da916206e 100644 --- a/proposals/4356-recent-emoji.md +++ b/proposals/4356-recent-emoji.md @@ -86,7 +86,7 @@ this information today, however. Malicious or buggy clients could cause undefined behavior on other clients by setting emoji counters to negative values or the maximum allowed value. To prevent this, clients SHOULD drop any emojis -with a count below 0 from the list. When observing a count of 2^53-1 for an emoji, clients SHOULD +with invalid entries from the list. When observing a count of 2^53-1 for an emoji, clients SHOULD normalise the entire set of emoji by dividing all counts by two and rounding up. To prevent race conditions, both of these changes SHOULD only be applied when the client is updating the account data event due to local emoji usage.