edits

proposals/4225-one-time-key-ordering.md

@@ -1,70 +1,23 @@
 # MSC4225: Specification of an order in which one-time-keys should be issued
 
-## Background
+The specification for the
-
-End-to-end encryption in Matrix relies on individual devices sharing
-[Megolm](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/megolm.md)
-message keys via [to-device
-messages](https://spec.matrix.org/v1.12/client-server-api/#send-to-device-messaging)
-which are themselves encrypted using the
-[Olm](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/olm.md)
-ratchet.
-
-Suppose Alice has wishes to send an Olm-encrypted message to Bob (with whom she
-has not previously established an Olm session). When Bob logged in, his device
-will have created a long-term identity key, as well as a number (typically 50)
-of one-time keys. Each of these keys has a private part, which Bob's device
-retains locally, and a public part, which Bob's device publishes to his
-homeserver via the
-[`/keys/upload`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysupload) endpoint.
-
-Now, to establish an Olm session, Alice needs to claim one of Bob's one-time
-keys. She does this by calling the
 [`/keys/claim`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysclaim)
-endpoint. Together with Bob's identity key, this allows Alice to encrypt a
+endpoint does not specify any particular order in which one-time keys (OTKs)
-message that only Bob will be able to decrypt.
+should be returned, leading to the possibility that old keys can remain on the
-
+server a long time. Clients may discard the private parts of such old keys,
-Over time, Bob's supply of one-time keys will be depleted. The `/sync` endpoint
+leading to [unable-to-decrypt
-informs clients how many one-time keys remain unclaimed on the server, so that
+errors](https://github.com/element-hq/element-meta/issues/2356).
-it can generate new ones when required.
 
-See [One-time and fallback
+See the Appendix below for a more detailed discussion of the problem.
-keys](https://spec.matrix.org/v1.12/client-server-api/#one-time-and-fallback-keys)
-which specifies much of this behaviour.
-
-## Problem
-
-Clearly, a device must retain the private part of each one-time keys until that
-key is used to establish an Olm session. However, for a number of reasons,
-ranging from network errors to malicious activity, it is possible for a claimed
-one-time key never to be used to establish an Olm session.
-
-This presents a problem: there is a limit to the number of private one-time
-keys that a client can retain. Over time, as keys are repeatedly claimed,
-replaced with newly-generated keys, but not actually used, the client must
-start to discard older keys.
-
-Unfortunately, the Matrix specification does not currently specify any order
-in which keys should be returned by `/keys/claim`. This means that homeservers
-can legitimately issue one-time keys effectively at random.
-
-Over time, then, it is easy to get into a situation where the server still
-holds some very old one-time keys, for which the client has discarded the
-private parts.
-
-Suppose, when Alice claims one of Bob's one-time keys, she is issued one whose
-private part Bob has discarded. Then, Bob will be unable to decrypt the Olm
-message from Alice, and (assuming the Olm message contained Megolm keys), will
-be unable to decrypt any room messages that Alice sends.
-
-This problem is discussed at https://github.com/element-hq/element-meta/issues/2356.
 
 ## Proposal
 
 [`/_matrix/client/v3/keys/claim`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysclaim)
 and [`/_matrix/federation/v1/user/keys/claim`](https://spec.matrix.org/v1.12/server-server-api/#post_matrixfederationv1userkeysclaim)
 should return one-time keys in the order that they were uploaded via
-[`/keys/upload`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysupload).
+[`/keys/upload`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysupload). All
+keys uploaded in a given call to `/keys/upload` are considered equivalent in
+this regard; no ordering is specified within them.
 
 This means that the server will retain only the most recently-uploaded one-time
 keys, therefore significantly reducing the chance that clients will discard
@@ -76,7 +29,8 @@ This proposal is not a complete solution to the problem of premature discarding
 of one-time keys: even if the server issues a recent one-time key, it is still
 possible for a to-device message to be delayed so long that the recipient has
 discarded the private part of the one-time key. It is, however, a significant
-improvement.
+improvement. A possible solution is for clients that expect to be used in
+conditions of poor connectivity to keep old OTKs for longer.
 
 There are other ways in which the server and client can get out of sync with
 respect to one-time keys, including by a [database
@@ -85,9 +39,27 @@ implementation defects.
 
 ## Alternatives
 
-* just keep OTKs forever. (We do with megolm keys)
+1. Mandate that clients keep the private parts of all uploaded but unused OTKs
-* synchronization mechanism.
+   indefinitely.
-* rewrite the crypto stack.
+
+   Given that each one-time key is only 32 bytes (plus the ID, say 8 bytes), it
+   would certainly be possible for a client implementation to hold a very large
+   number of private OTKs without significant concerns about resource usage. In
+   particular, a OTK is much smaller than a Megolm key, and there is no limit
+   to the number of Megolm keys that a client has to retain.
+
+   In short though, this just seems inefficient, compared to specifying that
+   OTKs should be issued in upload order.
+
+2. [MSC4162](https://github.com/matrix-org/matrix-spec-proposals/pull/4162)
+   proposes a mechanism by which a client can inform the server that it is
+   discarding certiain OTKs, so that the server can also remove the public
+   keys. This seems a heavier-weight solution to the problem.
+
+3. A more invasive alternative would be to design an encryption stack which
+   does not rely on one-time keys. Setting aside whether the security
+   properties of such a protocol would be sufficient, this is considered well
+   out-of-scope for this proposal.
 
 ## Security considerations
 
@@ -107,3 +79,58 @@ None required.
 ## Dependencies
 
 None.
+
+## Appendix: detailed explanation of the failure mode
+
+### Backgroynd
+
+End-to-end encryption in Matrix relies on individual devices sharing
+[Megolm](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/megolm.md)
+message keys via [to-device
+messages](https://spec.matrix.org/v1.12/client-server-api/#send-to-device-messaging)
+which are themselves encrypted using the
+[Olm](https://gitlab.matrix.org/matrix-org/olm/blob/master/docs/olm.md)
+ratchet.
+
+Suppose Alice wishes to send an Olm-encrypted message to Bob (with whom she
+has not previously established an Olm session). When Bob logged in, his device
+will have created a long-term identity key, as well as a number (typically 50)
+of one-time keys. Each of these keys has a private part, which Bob's device
+retains locally, and a public part, which Bob's device publishes to his
+homeserver via the
+[`/keys/upload`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysupload) endpoint.
+
+Now, to establish an Olm session, Alice needs to claim one of Bob's one-time
+keys. She does this by calling the
+[`/keys/claim`](https://spec.matrix.org/v1.12/client-server-api/#post_matrixclientv3keysclaim)
+endpoint. Together with Bob's identity key, this allows Alice to encrypt a
+message that only Bob will be able to decrypt.
+
+Over time, Bob's supply of one-time keys will be depleted. The `/sync` endpoint
+informs clients how many one-time keys remain unclaimed on the server, so that
+it can generate new ones when required.
+
+See [One-time and fallback
+keys](https://spec.matrix.org/v1.12/client-server-api/#one-time-and-fallback-keys)
+which specifies much of this behaviour.
+
+Clearly, a device must retain the private part of each one-time key until that
+key is used to establish an Olm session. However, for a number of reasons,
+ranging from network errors to malicious activity, it is possible for a claimed
+one-time key never to be used to establish an Olm session.
+
+This presents a problem: there is a limit to the number of private one-time
+keys that a client can retain. Over time, as keys are repeatedly claimed,
+replaced with newly-generated keys, but not actually used, the client must
+start to discard older keys.
+
+Unfortunately, the Matrix specification does not currently specify any order in
+which keys should be returned by `/keys/claim`. This means that homeservers can
+legitimately issue one-time keys effectively at random. Over time, then, it is
+easy to get into a situation where the server still holds some very old
+one-time keys, for which the client has discarded the private parts.
+
+Suppose, when Alice claims one of Bob's one-time keys, she is issued one whose
+private part Bob has discarded. Then, Bob will be unable to decrypt the Olm
+message from Alice, and (assuming the Olm message contained Megolm keys), will
+be unable to decrypt any room messages that Alice sends.