matrix-spec-proposals/proposals/4356-recent-emoji.md

MSC4356: Recently used emoji

Like other chat platforms, Matrix supports emoji as a way to visually express ideas or emotions. In practice, most people use a limited set of emoji only. Since emoji are commonly used as a quick way to react to something, it is desirable for clients to offer users shortcuts to their favorite emoji. Some emoji picker libraries support this feature by locally tracking emoji usage. This doesn't work well in a multi-device environment, however, because such history cannot easily be shared between clients.

This proposal introduces a way for clients to maintain a shared storage of recently used emoji to enable emoji suggestions across clients.

Proposal

A new global account data event m.recent_emoji is introduced. In content, it contains a single property recent_emoji that is an array where each element is an object with the following properties:

• emoji (string, required): The Unicode emoji as string.
• total (number, required): The number of times the emoji has been used (<= 2^53-1).

The outer recent_emoji array is ordered descendingly by last usage time.

{
  "type": "m.recent_emoji",
  "content": {
    "recent_emoji": [
      // Most recently used, 7 times overall
      {
        "emoji": "😅",
        "total": 7
      },
      // Second most recently used, 84 times overall
      {
        "emoji": "👍",
        "total": 84
      },
      ...
  }
}

When an emoji is used, the sending client moves (or adds) it to the beginning of the recent_emoji array and increments (or initializes) its counter. What exactly constitutes emoji usage is left as an implementation detail for clients. Obvious choices are sending emoji in messages or annotations.

When an image is sent as an inline image or in a reaction (using MSC4027), the mxc:// URI of the image MAY be used as the "emoji" in this event. Clients which do not support such use of images MUST tolerate the existence of mxc:// entries, e.g. by ignoring the entries when deciding what to display to the user, while still preserving them when modifying the list.

As new emoji are being used, clients SHOULD limit the length of the recent_emoji array by dropping elements from the end. A RECOMMENDED maximum length is 100 emoji. Apart from this, no other mechanism for resetting counters is mandated as the upper boundary of 2^53-1 seems sufficiently large for all practical purposes.

Clients MAY freely customise the logic for generating recommendations from the stored emoji. As an example, they could select the 24 first (= most recently used) emoji and stably sort them by their counters (so that more recently used emoji are ordered first on ties).

Potential issues

Clients could choose wildly different ways to generate recommendations from the shared storage leading to significantly different UX across clients. A "correct" algorithm is not currently known, however, and will require future UI exploration in client implementations.

Alternatives

Further metadata such as the concrete access time or the room could be tracked together with emoji. It is unclear, however, if this would lead to materially better suggestions. A last-used timestamp could also be used to cull emoji that haven't been used in a very long time. Given that implementations are already encouraged to limit the maximum number of tracked emoji, this doesn't appear necessary, however.

Rather than an array of objects, a nested array could be used. This approach has been used in early implementations of this proposal under the io.element.recent_emoji account data event. While this results in a more compact storage layout, it doesn't lend itself well to future extension. Additionally, given request compression and the common expectable length of the array, performance benefits should be negligible.

Security considerations

This proposal doesn't mandate encrypting the m.recent_emoji account data event. Since emoji are most commonly used in annotations which are not encrypted, servers could already track and abuse this information today, however.

Malicious or buggy clients could cause undefined behavior on other clients by setting emoji counters to negative values or the maximum allowed value. To prevent this, clients SHOULD drop any emojis with invalid entries from the list. When observing a count of 2^53-1 for an emoji, clients SHOULD normalise the entire set of emoji by dividing all counts by two and rounding up. To prevent race conditions, both of these changes SHOULD only be applied when the client is updating the account data event due to local emoji usage.

Unstable prefix

While this MSC is not considered stable, m.recent_emoji should be referred to as io.github.johennes.msc4356.recent_emoji.

Dependencies

None.