From 63c6d030fdbcee336e7df03e89d96a63c10fa205 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 27 Mar 2019 21:48:12 -0400 Subject: [PATCH 01/18] draft of secure server-side storage --- proposals/xxxx-secure_server-side_storage.md | 144 +++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 proposals/xxxx-secure_server-side_storage.md diff --git a/proposals/xxxx-secure_server-side_storage.md b/proposals/xxxx-secure_server-side_storage.md new file mode 100644 index 000000000..603fe64e0 --- /dev/null +++ b/proposals/xxxx-secure_server-side_storage.md @@ -0,0 +1,144 @@ +# Secure Server-side Storage + +Some features may require clients to store encrypted data on the server so that +it can be shared securely between clients. For example, key backups (MSC-1219) +can store the decryption key for the backups on the server, or cross-signing +(MSC-1756) can store the signing keys. This proposal presents a standardized +way of storing such data. + +## Proposal + +A user can have multiple keys used for encrypting data. This allows the user +to selectively decrypt data. For example, the user could have one key that can +decrypt everything, and another key that can only decrypt their user-signing +key for cross-signing. Each key has an ID, and a discription of the key is +stored in the user's `account_data` using the `type` `m.secure_storage.key.[key +ID]`. The contents of the account data for the key will include an `algorithm` +property, which indicates the encryption algorithm used, as well as a `name` +property, which is a human-readable name. Other properties depend on the +encryption algorithm, and are described below. + +Encrypted data can be stored using the `account_data` API. The `type` for the +`account_data` is defined by the feature that uses the data. For example, +decryption keys for key backups could be stored under the type +`m.megolm_backup.v1.recovery_key`, or the self-signing key for cross-signing +could be stored under the type `m.signing_key.self_signing`. + +Data will be stored using using the following format: + +```json +{ + "encrypted": { + [key ID]: { + "ciphertext": "base64+encoded+encrypted+data", + "mac": "base64+encoded+mac" + } + } +} +``` + +The `encrypted` property is map from key ID to an object. The algorithm for +the given key defines how the other properties are interpreted, though it's +expected that most encryption schemes would have `ciphertext` and `mac` +properties, where the `ciphertext` property is the unpadded base64-encoded +ciphertext, and the `mac` is used to ensure the integrity of the data. + +FIXME: the key format was chosen so that existing backups could be easily +migrated by just copying the configuration from the backup config to the key +description. However, we need a way of signalling that the key for the backup +is the same as the key for decrypting the other bits. Maybe a special flag in +the account data? Or special case backups somehow, say, have clients inspect +the backup's `auth_data` to see of the key config is the same? + +### Encryption algorithms + +#### `m.secure_storage.v1.curve25519-aes-sha2` + +The data is encrypted and MACed as follows: + +1. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral + key and the backup's public key to generate a shared secret. The public + half of the ephemeral key, encoded using base64, becomes the `ephemeral` + property. +2. Using the shared secret, generate 80 bytes by performing an HKDF using + SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string + as the info. The first 32 bytes are used as the AES key, the next 32 bytes + are used as the MAC key, and the last 16 bytes are used as the AES + initialization vector. +4. Encrypt the data using AES-CBC-256 with PKCS#7 padding. This encrypted + data, encoded using base64, becomes the `ciphertext` property. +5. Pass the raw encrypted data (prior to base64 encoding) through HMAC-SHA-256 + using the MAC key generated above. The first 8 bytes of the resulting MAC + are base64-encoded, and become the `mac` property. + +(The key HKDF, AES, and HMAC steps are the same as what are used for encryption +in olm and megolm.) + +FIXME: add an example of `m.secure_storage.key.*`, and of encrypted data. + +##### Keys + +When a user is given a raw key for `m.secure_storage.v1.curve25519-aes-sha2`, +it will be encoded as follows (this is the same as what is proposed in MSC1703): + +* prepend the two bytes 0x8b and 0x01 to the key +* compute a parity byte by XORing all bytes of the resulting string, and append + the parity byte to the string +* base58-encode the resulting byte string with the alphabet + '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'. +* format the resulting ASCII string into groups of 4 characters separated by + spaces. + +When decoding a raw key, the process should be reversed, with the exception +that whitespace is insignificant in the user's ASCII input. + +##### Passphrase + +A user may wish to use a chosen passphrase rather than a randomly generated +key. In this case, information on how to generate the key from a passphrase +will be stored in the `passphrase` property of the `m.secure_storage.key.[key +ID]` account-data: + +```json +{ + "passphrase": { + "algorithm": "m.pbkdf2", + "salt": "MmMsAlty", + "rounds": 100000 + }, + ... +} +``` + +###### `m.pbkdf2` + +The key is generated using PBKDF2 using the salt given in the `salt` +parameter, and the number of rounds given in the `rounds` parameter. + +## Tradeoffs + +Rather than encrypting data on the server using a static key, clients can +exchange data by sending to_device messages encrypted using Olm. This allows +clients to share data securely without requiring the user to enter keys or +passphrases. However, users who have only one device and lose it will still +need a way to, for example, recover their key backup, so we must provide a way +for the data to be stored on the server. + +Currently, only a public/private key mechanism is defined. It may be useful to +also define a secret key mechanism. + +## Potential issues + +Keeping all the data and keys in account data means that it may clutter up the +`/sync`. However, clients can filter out the data that they are not interested +in. One possibility for addressing this would be to add a flag to the account +data to indicate whether it should come down the `/sync` or not. + +## Security considerations + +Yes. + +## Conclusion + +This proposal presents a common way for bits of encrypted data to be stored on +a user's homeserver for use by various features. From 979827bad3ba422b01edaaa1bc49d54e0d0c5f1b Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 27 Mar 2019 21:57:13 -0400 Subject: [PATCH 02/18] rename to match MSC number --- ..._server-side_storage.md => 1946-secure_server-side_storage.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{xxxx-secure_server-side_storage.md => 1946-secure_server-side_storage.md} (100%) diff --git a/proposals/xxxx-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md similarity index 100% rename from proposals/xxxx-secure_server-side_storage.md rename to proposals/1946-secure_server-side_storage.md From da5ce919f2b363e2a660632a8d971dd42f90afb0 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 31 May 2019 12:16:59 -0400 Subject: [PATCH 03/18] add sharing secrets, and a bunch of cleanups --- proposals/1946-secure_server-side_storage.md | 104 ++++++++++++++----- 1 file changed, 78 insertions(+), 26 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 603fe64e0..c44cd57eb 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -1,28 +1,39 @@ -# Secure Server-side Storage +# Secure Secret Storage and Sharing Some features may require clients to store encrypted data on the server so that -it can be shared securely between clients. For example, key backups (MSC-1219) +it can be shared securely between clients. Clients may also wish to securely +send such data directly to each other. For example, key backups (MSC-1219) can store the decryption key for the backups on the server, or cross-signing (MSC-1756) can store the signing keys. This proposal presents a standardized way of storing such data. ## Proposal -A user can have multiple keys used for encrypting data. This allows the user -to selectively decrypt data. For example, the user could have one key that can +Secrets are data that clients need to use and thate are sent through or stored +on the server, but should not be visible to server operators. Secrets are +plain strings -- if clients need to use more complicated data, it must be +encoded as a string. + +### Storage + +If secret data is stored on the server, it must be encrypted in order to +prevent homeserver administrators from being able to read it. A user can have +multiple keys used for encrypting data. This allows the user to selectively +decrypt data on clients. For example, the user could have one key that can decrypt everything, and another key that can only decrypt their user-signing key for cross-signing. Each key has an ID, and a discription of the key is -stored in the user's `account_data` using the `type` `m.secure_storage.key.[key +stored in the user's `account_data` using the `type` `m.secret_storage.key.[key ID]`. The contents of the account data for the key will include an `algorithm` property, which indicates the encryption algorithm used, as well as a `name` -property, which is a human-readable name. Other properties depend on the -encryption algorithm, and are described below. +property, which is a human-readable name. The contents will be signed as +signed JSON using the user's master cross-signing key. Other properties depend +on the encryption algorithm, and are described below. Encrypted data can be stored using the `account_data` API. The `type` for the `account_data` is defined by the feature that uses the data. For example, decryption keys for key backups could be stored under the type `m.megolm_backup.v1.recovery_key`, or the self-signing key for cross-signing -could be stored under the type `m.signing_key.self_signing`. +could be stored under the type `m.cross_signing.self_signing`. Data will be stored using using the following format: @@ -50,16 +61,18 @@ is the same as the key for decrypting the other bits. Maybe a special flag in the account data? Or special case backups somehow, say, have clients inspect the backup's `auth_data` to see of the key config is the same? -### Encryption algorithms +#### Encryption algorithms -#### `m.secure_storage.v1.curve25519-aes-sha2` +##### `m.secret_storage.v1.curve25519-aes-sha2` + +The public key is stored in the `pubkey` property of the `m.secret_storage.key.[key +ID]` `account_data`. The data is encrypted and MACed as follows: 1. Generate an ephemeral curve25519 key, and perform an ECDH with the ephemeral - key and the backup's public key to generate a shared secret. The public - half of the ephemeral key, encoded using base64, becomes the `ephemeral` - property. + key and the public key to generate a shared secret. The public half of the + ephemeral key, encoded using base64, becomes the `ephemeral` property. 2. Using the shared secret, generate 80 bytes by performing an HKDF using SHA-256 as the hash, with a salt of 32 bytes of 0, and with the empty string as the info. The first 32 bytes are used as the AES key, the next 32 bytes @@ -74,11 +87,11 @@ The data is encrypted and MACed as follows: (The key HKDF, AES, and HMAC steps are the same as what are used for encryption in olm and megolm.) -FIXME: add an example of `m.secure_storage.key.*`, and of encrypted data. +FIXME: add an example of `m.secret_storage.key.*`, and of encrypted data. -##### Keys +###### Keys -When a user is given a raw key for `m.secure_storage.v1.curve25519-aes-sha2`, +When a user is given a raw key for `m.secret_storage.v1.curve25519-aes-sha2`, it will be encoded as follows (this is the same as what is proposed in MSC1703): * prepend the two bytes 0x8b and 0x01 to the key @@ -92,11 +105,11 @@ it will be encoded as follows (this is the same as what is proposed in MSC1703): When decoding a raw key, the process should be reversed, with the exception that whitespace is insignificant in the user's ASCII input. -##### Passphrase +###### Passphrase A user may wish to use a chosen passphrase rather than a randomly generated key. In this case, information on how to generate the key from a passphrase -will be stored in the `passphrase` property of the `m.secure_storage.key.[key +will be stored in the `passphrase` property of the `m.secret_storage.key.[key ID]` account-data: ```json @@ -110,19 +123,58 @@ ID]` account-data: } ``` -###### `m.pbkdf2` +**`m.pbkdf2`** The key is generated using PBKDF2 using the salt given in the `salt` parameter, and the number of rounds given in the `rounds` parameter. -## Tradeoffs +### Sharing + +Rather than (or in addition to) storing secrets on the server encrypted by a +shared key, devices can send secrets to each other, encrypted using olm. + +To request a secret, a client sends a `m.secret.request` event with `action` +set to `request` to other devices, and `name` set to the name of the secret +that it wishes to retrieve. A device that wishes to share the secret will +reply with a `m.secret.share` event, encrypted using olm. When the original +client obtains the secret, it sends a `m.secret.request` event with `action` +set to `cancel_request` to all devices other than the one that it received the +secret from. + +Clients SHOULD ensure that they only share secrets with other devices that are +allowed to see them. For example, clients SHOULD only share secrets with devices +that are verified and MAY prompt the user to confirm sharing the secret. + +If a feature allows secrets to be stored or shared, then for consistency it +SHOULD use the same name for both the `account_data` `type` and the `name` in +the `m.secret.request`. -Rather than encrypting data on the server using a static key, clients can -exchange data by sending to_device messages encrypted using Olm. This allows -clients to share data securely without requiring the user to enter keys or -passphrases. However, users who have only one device and lose it will still -need a way to, for example, recover their key backup, so we must provide a way -for the data to be stored on the server. +#### Event definitions + +##### `m.secret.request` + +Sent by a client to request a secret from another device. It is sent as an +unencrypted to-device event. + +- `name`: (string) Required if `action` is `request`. The name of the secret + that is being requested. +- `action`: (enum) Required. One of ["request", "cancel_request"]. +- `requesting_device_id`: (string) Required. ID of the device requesting the + secret. +- `request_id`: (string) Required. A random string uniquely identifying the + request for a secret. If the secret is requested multiple times, it should be + reused. It should also reused in order to cancel a request. + +##### `m.secret.share` + +Sent by a client to share a secret with another device, in response to an +`m.secret.request` event. Typically it is encrypted as an `m.room.encrypted` +event, then sent as a to-device event. + +- `request_id`: (string) Required. The ID of the request that this a response to. +- `secret`: (string) Required. The contents of the secret. + +## Tradeoffs Currently, only a public/private key mechanism is defined. It may be useful to also define a secret key mechanism. From fee2ebf682c7c83cb807e3e991bf91fe2dac8b88 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sun, 2 Jun 2019 16:09:47 -0400 Subject: [PATCH 04/18] fix typo Co-Authored-By: Shamil K --- proposals/1946-secure_server-side_storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index c44cd57eb..8487d6d45 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -9,7 +9,7 @@ way of storing such data. ## Proposal -Secrets are data that clients need to use and thate are sent through or stored +Secrets are data that clients need to use and that are sent through or stored on the server, but should not be visible to server operators. Secrets are plain strings -- if clients need to use more complicated data, it must be encoded as a string. From 3aaf181db24dd0d722c51fc079d90cbcdff30383 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 2 Jul 2019 14:03:35 -0400 Subject: [PATCH 05/18] rename some things and add clarification --- proposals/1946-secure_server-side_storage.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 8487d6d45..dff7a5c70 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -66,7 +66,7 @@ the backup's `auth_data` to see of the key config is the same? ##### `m.secret_storage.v1.curve25519-aes-sha2` The public key is stored in the `pubkey` property of the `m.secret_storage.key.[key -ID]` `account_data`. +ID]` `account_data` as a base64-encoded string. The data is encrypted and MACed as follows: @@ -117,7 +117,7 @@ ID]` account-data: "passphrase": { "algorithm": "m.pbkdf2", "salt": "MmMsAlty", - "rounds": 100000 + "iterations": 100000 }, ... } @@ -125,8 +125,8 @@ ID]` account-data: **`m.pbkdf2`** -The key is generated using PBKDF2 using the salt given in the `salt` -parameter, and the number of rounds given in the `rounds` parameter. +The key is generated using PBKDF2 using the salt given in the `salt` parameter, +and the number of iterations given in the `iterations` parameter. ### Sharing @@ -136,7 +136,7 @@ shared key, devices can send secrets to each other, encrypted using olm. To request a secret, a client sends a `m.secret.request` event with `action` set to `request` to other devices, and `name` set to the name of the secret that it wishes to retrieve. A device that wishes to share the secret will -reply with a `m.secret.share` event, encrypted using olm. When the original +reply with a `m.secret.send` event, encrypted using olm. When the original client obtains the secret, it sends a `m.secret.request` event with `action` set to `cancel_request` to all devices other than the one that it received the secret from. @@ -165,7 +165,7 @@ unencrypted to-device event. request for a secret. If the secret is requested multiple times, it should be reused. It should also reused in order to cancel a request. -##### `m.secret.share` +##### `m.secret.send` Sent by a client to share a secret with another device, in response to an `m.secret.request` event. Typically it is encrypted as an `m.room.encrypted` From 395d40314bcbaf39646c03bdada1e7a59efe3ea4 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 31 Jul 2019 16:20:39 -0400 Subject: [PATCH 06/18] fix typos and make valid JSON --- proposals/1946-secure_server-side_storage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index dff7a5c70..7fb426cec 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -21,7 +21,7 @@ prevent homeserver administrators from being able to read it. A user can have multiple keys used for encrypting data. This allows the user to selectively decrypt data on clients. For example, the user could have one key that can decrypt everything, and another key that can only decrypt their user-signing -key for cross-signing. Each key has an ID, and a discription of the key is +key for cross-signing. Each key has an ID, and a description of the key is stored in the user's `account_data` using the `type` `m.secret_storage.key.[key ID]`. The contents of the account data for the key will include an `algorithm` property, which indicates the encryption algorithm used, as well as a `name` @@ -35,12 +35,12 @@ decryption keys for key backups could be stored under the type `m.megolm_backup.v1.recovery_key`, or the self-signing key for cross-signing could be stored under the type `m.cross_signing.self_signing`. -Data will be stored using using the following format: +Data will be stored using the following format: ```json { "encrypted": { - [key ID]: { + "key_id": { "ciphertext": "base64+encoded+encrypted+data", "mac": "base64+encoded+mac" } From d47e13c6d94392583e3d17b18f72ef71250754c5 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 31 Jul 2019 16:20:55 -0400 Subject: [PATCH 07/18] this FIXME will be addressed in the key backup MSC --- proposals/1946-secure_server-side_storage.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 7fb426cec..c0c828f27 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -54,13 +54,6 @@ expected that most encryption schemes would have `ciphertext` and `mac` properties, where the `ciphertext` property is the unpadded base64-encoded ciphertext, and the `mac` is used to ensure the integrity of the data. -FIXME: the key format was chosen so that existing backups could be easily -migrated by just copying the configuration from the backup config to the key -description. However, we need a way of signalling that the key for the backup -is the same as the key for decrypting the other bits. Maybe a special flag in -the account data? Or special case backups somehow, say, have clients inspect -the backup's `auth_data` to see of the key config is the same? - #### Encryption algorithms ##### `m.secret_storage.v1.curve25519-aes-sha2` From bd9efcdf53b9fb8ecf55db6f36db951080d8c715 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 31 Jul 2019 16:22:24 -0400 Subject: [PATCH 08/18] add some information and an example --- proposals/1946-secure_server-side_storage.md | 36 ++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index c0c828f27..2f269f23b 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -29,6 +29,11 @@ property, which is a human-readable name. The contents will be signed as signed JSON using the user's master cross-signing key. Other properties depend on the encryption algorithm, and are described below. +If a key has the `name` property set to `m.default`, then this key is marked as +the default key for the account. The default key is the one that all secrets +will be encrypted with, and the clients will try to use to decrypt data with, +unless the user specifies otherwise. Only one key can be marked as the default. + Encrypted data can be stored using the `account_data` API. The `type` for the `account_data` is defined by the feature that uses the data. For example, decryption keys for key backups could be stored under the type @@ -80,7 +85,29 @@ The data is encrypted and MACed as follows: (The key HKDF, AES, and HMAC steps are the same as what are used for encryption in olm and megolm.) -FIXME: add an example of `m.secret_storage.key.*`, and of encrypted data. +For example, a key using this algorithm could look like: + +```json +{ + "name": "m.default", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + "pubkey": "base64+public+key" +} +``` + +and data encrypted using this algorithm could look like this: + +```json +{ + "encrypted": { + "key_id": { + "ciphertext": "base64+encoded+encrypted+data", + "ephemeral": "base64+ephemeral+key", + "mac": "base64+encoded+mac" + } + } +} +``` ###### Keys @@ -181,7 +208,12 @@ data to indicate whether it should come down the `/sync` or not. ## Security considerations -Yes. +By storing information encrypted on the server, this allows the server operator +to read the information if they manage to get hold of the decryption keys. +In particular, if the key is based on a passphrase and the passphrase can be +guessed, then the secrets could be compromised. In order to help protect the +secrets, clients should provide feedback to the user when their chosen +passphrase is considered weak. ## Conclusion From 4e2fe124d21a8290b7b2fa01c0200cbc9514dc4f Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Sun, 25 Aug 2019 18:22:23 -0700 Subject: [PATCH 09/18] wording fixes/clarifications --- proposals/1946-secure_server-side_storage.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 2f269f23b..0e6fccb71 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -11,8 +11,8 @@ way of storing such data. Secrets are data that clients need to use and that are sent through or stored on the server, but should not be visible to server operators. Secrets are -plain strings -- if clients need to use more complicated data, it must be -encoded as a string. +plain strings -- if clients need to use more complicated data, they must be +encoded as a string, such as by encoding as JSON. ### Storage @@ -29,10 +29,11 @@ property, which is a human-readable name. The contents will be signed as signed JSON using the user's master cross-signing key. Other properties depend on the encryption algorithm, and are described below. -If a key has the `name` property set to `m.default`, then this key is marked as +If a key has the `name` property set to `m.default`, then this key is treated as the default key for the account. The default key is the one that all secrets -will be encrypted with, and the clients will try to use to decrypt data with, -unless the user specifies otherwise. Only one key can be marked as the default. +will be encrypted with, and that clients will try to use to decrypt data with, +unless the user specifies otherwise. Only one key can be marked as the default +at a time. Encrypted data can be stored using the `account_data` API. The `type` for the `account_data` is defined by the feature that uses the data. For example, @@ -85,7 +86,8 @@ The data is encrypted and MACed as follows: (The key HKDF, AES, and HMAC steps are the same as what are used for encryption in olm and megolm.) -For example, a key using this algorithm could look like: +For example, the `m.secret_storage.key.[key ID]` for a key using this algorithm +could look like: ```json { @@ -159,7 +161,8 @@ that it wishes to retrieve. A device that wishes to share the secret will reply with a `m.secret.send` event, encrypted using olm. When the original client obtains the secret, it sends a `m.secret.request` event with `action` set to `cancel_request` to all devices other than the one that it received the -secret from. +secret from. Clients should ignore `m.secret.send` events received from +devices that it did not send an `m.secret.request` event to. Clients SHOULD ensure that they only share secrets with other devices that are allowed to see them. For example, clients SHOULD only share secrets with devices @@ -213,7 +216,8 @@ to read the information if they manage to get hold of the decryption keys. In particular, if the key is based on a passphrase and the passphrase can be guessed, then the secrets could be compromised. In order to help protect the secrets, clients should provide feedback to the user when their chosen -passphrase is considered weak. +passphrase is considered weak, and may also wish to prevent the user from +reusing their login password. ## Conclusion From cafe49d36d6152789ac23202af914267a48ca011 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 27 Aug 2019 16:13:18 -0700 Subject: [PATCH 10/18] some clarifications --- proposals/1946-secure_server-side_storage.md | 77 +++++++++++++------- 1 file changed, 51 insertions(+), 26 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 0e6fccb71..05c200863 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -2,10 +2,11 @@ Some features may require clients to store encrypted data on the server so that it can be shared securely between clients. Clients may also wish to securely -send such data directly to each other. For example, key backups (MSC-1219) -can store the decryption key for the backups on the server, or cross-signing -(MSC-1756) can store the signing keys. This proposal presents a standardized -way of storing such data. +send such data directly to each other. For example, key backups +([MSC1219](https://github.com/matrix-org/matrix-doc/issues/1219)) can store the +decryption key for the backups on the server, or cross-signing +([MSC1756](https://github.com/matrix-org/matrix-doc/pull/1756)) can store the +signing keys. This proposal presents a standardized way of storing such data. ## Proposal @@ -21,13 +22,29 @@ prevent homeserver administrators from being able to read it. A user can have multiple keys used for encrypting data. This allows the user to selectively decrypt data on clients. For example, the user could have one key that can decrypt everything, and another key that can only decrypt their user-signing -key for cross-signing. Each key has an ID, and a description of the key is -stored in the user's `account_data` using the `type` `m.secret_storage.key.[key -ID]`. The contents of the account data for the key will include an `algorithm` -property, which indicates the encryption algorithm used, as well as a `name` -property, which is a human-readable name. The contents will be signed as -signed JSON using the user's master cross-signing key. Other properties depend -on the encryption algorithm, and are described below. +key for cross-signing. + +Key descriptions and secret data are both stored in the user's `account_data`. + +Each key has an ID, and the description of the key is stored in the +`account_data` using the `type` `m.secret_storage.key.[key ID]`. The contents +of the account data for the key will include an `algorithm` property, which +indicates the encryption algorithm used, as well as a `name` property, which is +a human-readable name. The contents will be signed as signed JSON using the +user's master cross-signing key. Other properties depend on the encryption +algorithm, and are described below. + +Example: + +A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` + +```json +{ + "name": "Some key", + "algorihm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` If a key has the `name` property set to `m.default`, then this key is treated as the default key for the account. The default key is the one that all secrets @@ -35,31 +52,39 @@ will be encrypted with, and that clients will try to use to decrypt data with, unless the user specifies otherwise. Only one key can be marked as the default at a time. -Encrypted data can be stored using the `account_data` API. The `type` for the -`account_data` is defined by the feature that uses the data. For example, -decryption keys for key backups could be stored under the type -`m.megolm_backup.v1.recovery_key`, or the self-signing key for cross-signing -could be stored under the type `m.cross_signing.self_signing`. +Encrypted data is stored in the `account_data` using the `type` defined by the +feature that uses the data. For example, decryption keys for key backups could +be stored under the type `m.megolm_backup.v1.recovery_key`, or the self-signing +key for cross-signing could be stored under the type +`m.cross_signing.self_signing`. + +The `account_data` will have an `encrypted` property that is a map from key ID +to an object. The algorithm from the `m.secret_storage.key.[key ID]` data for +the given key defines how the other properties are interpreted, though it's +expected that most encryption schemes would have `ciphertext` and `mac` +properties, where the `ciphertext` property is the unpadded base64-encoded +ciphertext, and the `mac` is used to ensure the integrity of the data. + +Example: -Data will be stored using the following format: +Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: ```json { "encrypted": { - "key_id": { + "key_id_1": { "ciphertext": "base64+encoded+encrypted+data", - "mac": "base64+encoded+mac" + "mac": "base64+encoded+mac", + // ... other properties according to algorithm property in + // m.secret_storage.key.key_id_1 + }, + "key_id_2": { + // ... } } } ``` -The `encrypted` property is map from key ID to an object. The algorithm for -the given key defines how the other properties are interpreted, though it's -expected that most encryption schemes would have `ciphertext` and `mac` -properties, where the `ciphertext` property is the unpadded base64-encoded -ciphertext, and the `mac` is used to ensure the integrity of the data. - #### Encryption algorithms ##### `m.secret_storage.v1.curve25519-aes-sha2` @@ -181,7 +206,7 @@ unencrypted to-device event. - `name`: (string) Required if `action` is `request`. The name of the secret that is being requested. -- `action`: (enum) Required. One of ["request", "cancel_request"]. +- `action`: (enum) Required. One of ["request", "request_cancellation"]. - `requesting_device_id`: (string) Required. ID of the device requesting the secret. - `request_id`: (string) Required. A random string uniquely identifying the From 9aade7291a79f90f4a4d2fad8d214a5350b57381 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 27 Aug 2019 16:21:00 -0700 Subject: [PATCH 11/18] make it agree with what we actually did with key requests --- proposals/1946-secure_server-side_storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 05c200863..0ccde3110 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -185,7 +185,7 @@ set to `request` to other devices, and `name` set to the name of the secret that it wishes to retrieve. A device that wishes to share the secret will reply with a `m.secret.send` event, encrypted using olm. When the original client obtains the secret, it sends a `m.secret.request` event with `action` -set to `cancel_request` to all devices other than the one that it received the +set to `request_cancellation` to all devices other than the one that it received the secret from. Clients should ignore `m.secret.send` events received from devices that it did not send an `m.secret.request` event to. From 9dfca61ae790e15b7a924955cc93518b4655142f Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 23 Oct 2019 16:13:17 -0400 Subject: [PATCH 12/18] Apply suggestions from code review Co-Authored-By: David Baker Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-Authored-By: Erik Johnston --- proposals/1946-secure_server-side_storage.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 0ccde3110..2f6460c66 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -41,7 +41,7 @@ A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` ```json { "name": "Some key", - "algorihm": "m.secret_storage.v1.curve25519-aes-sha2", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", // ... other properties according to algorithm } ``` @@ -52,7 +52,7 @@ will be encrypted with, and that clients will try to use to decrypt data with, unless the user specifies otherwise. Only one key can be marked as the default at a time. -Encrypted data is stored in the `account_data` using the `type` defined by the +Encrypted data is stored in `account_data` using the `type` defined by the feature that uses the data. For example, decryption keys for key backups could be stored under the type `m.megolm_backup.v1.recovery_key`, or the self-signing key for cross-signing could be stored under the type @@ -180,7 +180,7 @@ and the number of iterations given in the `iterations` parameter. Rather than (or in addition to) storing secrets on the server encrypted by a shared key, devices can send secrets to each other, encrypted using olm. -To request a secret, a client sends a `m.secret.request` event with `action` +To request a secret, a client sends a `m.secret.request` device event with `action` set to `request` to other devices, and `name` set to the name of the secret that it wishes to retrieve. A device that wishes to share the secret will reply with a `m.secret.send` event, encrypted using olm. When the original @@ -229,8 +229,8 @@ also define a secret key mechanism. ## Potential issues -Keeping all the data and keys in account data means that it may clutter up the -`/sync`. However, clients can filter out the data that they are not interested +Keeping all the data and keys in account data means that it may clutter up +`/sync` requests. However, clients can filter out the data that they are not interested in. One possibility for addressing this would be to add a flag to the account data to indicate whether it should come down the `/sync` or not. From b27f6985b2850ceb93ea29112b445ccb821cc82e Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Wed, 23 Oct 2019 17:32:36 -0400 Subject: [PATCH 13/18] more clarifications --- proposals/1946-secure_server-side_storage.md | 55 +++++++++++--------- 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 2f6460c66..49f115da5 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -24,10 +24,12 @@ decrypt data on clients. For example, the user could have one key that can decrypt everything, and another key that can only decrypt their user-signing key for cross-signing. -Key descriptions and secret data are both stored in the user's `account_data`. +Key descriptions and secret data are both stored in the user's account_data. -Each key has an ID, and the description of the key is stored in the -`account_data` using the `type` `m.secret_storage.key.[key ID]`. The contents +#### Key storage + +Each key has an ID, and the description of the key is stored in the user's +account_data using the event type `m.secret_storage.key.[key ID]`. The contents of the account data for the key will include an `algorithm` property, which indicates the encryption algorithm used, as well as a `name` property, which is a human-readable name. The contents will be signed as signed JSON using the @@ -52,13 +54,15 @@ will be encrypted with, and that clients will try to use to decrypt data with, unless the user specifies otherwise. Only one key can be marked as the default at a time. -Encrypted data is stored in `account_data` using the `type` defined by the -feature that uses the data. For example, decryption keys for key backups could -be stored under the type `m.megolm_backup.v1.recovery_key`, or the self-signing -key for cross-signing could be stored under the type +#### Secret storage + +Encrypted data is stored in the user's account_data using the event type +defined by the feature that uses the data. For example, decryption keys for +key backups could be stored under the type `m.megolm_backup.v1.recovery_key`, +or the self-signing key for cross-signing could be stored under the type `m.cross_signing.self_signing`. -The `account_data` will have an `encrypted` property that is a map from key ID +The account_data will have an `encrypted` property that is a map from key ID to an object. The algorithm from the `m.secret_storage.key.[key ID]` data for the given key defines how the other properties are interpreted, though it's expected that most encryption schemes would have `ciphertext` and `mac` @@ -71,17 +75,17 @@ Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: ```json { - "encrypted": { - "key_id_1": { - "ciphertext": "base64+encoded+encrypted+data", - "mac": "base64+encoded+mac", - // ... other properties according to algorithm property in - // m.secret_storage.key.key_id_1 - }, - "key_id_2": { - // ... - } + "encrypted": { + "key_id_1": { + "ciphertext": "base64+encoded+encrypted+data", + "mac": "base64+encoded+mac", + // ... other properties according to algorithm property in + // m.secret_storage.key.key_id_1 + }, + "key_id_2": { + // ... } + } } ``` @@ -90,7 +94,7 @@ Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: ##### `m.secret_storage.v1.curve25519-aes-sha2` The public key is stored in the `pubkey` property of the `m.secret_storage.key.[key -ID]` `account_data` as a base64-encoded string. +ID]` account_data as a base64-encoded string. The data is encrypted and MACed as follows: @@ -189,12 +193,13 @@ set to `request_cancellation` to all devices other than the one that it received secret from. Clients should ignore `m.secret.send` events received from devices that it did not send an `m.secret.request` event to. -Clients SHOULD ensure that they only share secrets with other devices that are -allowed to see them. For example, clients SHOULD only share secrets with devices -that are verified and MAY prompt the user to confirm sharing the secret. +Clients MUST ensure that they only share secrets with other devices that are +allowed to see them. For example, clients SHOULD only share secrets with +devices that are verified and MAY prompt the user to confirm sharing the +secret. If a feature allows secrets to be stored or shared, then for consistency it -SHOULD use the same name for both the `account_data` `type` and the `name` in +SHOULD use the same name for both the account_data event type and the `name` in the `m.secret.request`. #### Event definitions @@ -216,8 +221,8 @@ unencrypted to-device event. ##### `m.secret.send` Sent by a client to share a secret with another device, in response to an -`m.secret.request` event. Typically it is encrypted as an `m.room.encrypted` -event, then sent as a to-device event. +`m.secret.request` event. It MUST be encrypted as an `m.room.encrypted` event, +then sent as a to-device event. - `request_id`: (string) Required. The ID of the request that this a response to. - `secret`: (string) Required. The contents of the secret. From 86eccc3bc8d5c1748a607a2c79067c1dae1b6923 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Fri, 25 Oct 2019 15:00:23 -0400 Subject: [PATCH 14/18] change how we mark the default key, and make sure clients trust keys --- proposals/1946-secure_server-side_storage.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 49f115da5..a2dfb3817 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -48,11 +48,17 @@ A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` } ``` -If a key has the `name` property set to `m.default`, then this key is treated as -the default key for the account. The default key is the one that all secrets -will be encrypted with, and that clients will try to use to decrypt data with, -unless the user specifies otherwise. Only one key can be marked as the default -at a time. +A key can be marked as the "default" key by setting the user's account_data +with event type `m.secret_storage.default_key` to the ID of the key. The +default key will be used to encrypet all secrets that the user would expect to +be available on all their clients. Unless the user specifies otherwise, +clients will try to use the default key to decrypt secrets. + +Clients MUST ensure that the key is trusted before using it to encrypt secrets. +One way to do that is to have the client that creates the key sign the key +description (as signed JSON) using the user's master cross-signing key. +Another way to do that is to prompt the user to enter the passphrase and ensure +that the generated private key correponds to the public key. #### Secret storage From 4fadace9c823112122b407db7281e486e347b048 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Mon, 28 Oct 2019 12:48:59 -0400 Subject: [PATCH 15/18] Update proposals/1946-secure_server-side_storage.md Co-Authored-By: David Baker --- proposals/1946-secure_server-side_storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index a2dfb3817..832144d1d 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -50,7 +50,7 @@ A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` A key can be marked as the "default" key by setting the user's account_data with event type `m.secret_storage.default_key` to the ID of the key. The -default key will be used to encrypet all secrets that the user would expect to +default key will be used to encrypt all secrets that the user would expect to be available on all their clients. Unless the user specifies otherwise, clients will try to use the default key to decrypt secrets. From 5cc5908dd5ca6362314b14637d4fe1a03e6cb3e7 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Thu, 7 Nov 2019 18:45:29 -0500 Subject: [PATCH 16/18] Update proposals/1946-secure_server-side_storage.md Co-Authored-By: Matthew Hodgson --- proposals/1946-secure_server-side_storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 832144d1d..4c5f9b4dd 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -201,7 +201,7 @@ devices that it did not send an `m.secret.request` event to. Clients MUST ensure that they only share secrets with other devices that are allowed to see them. For example, clients SHOULD only share secrets with -devices that are verified and MAY prompt the user to confirm sharing the +the user’s own devices that are verified and MAY prompt the user to confirm sharing the secret. If a feature allows secrets to be stored or shared, then for consistency it From e8ce135a411e5fa2715f9d1cbd57f0fc232bc98f Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Thu, 7 Nov 2019 19:00:33 -0500 Subject: [PATCH 17/18] add clarification and examples --- proposals/1946-secure_server-side_storage.md | 29 ++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index 4c5f9b4dd..aa4fd8f91 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -57,8 +57,9 @@ clients will try to use the default key to decrypt secrets. Clients MUST ensure that the key is trusted before using it to encrypt secrets. One way to do that is to have the client that creates the key sign the key description (as signed JSON) using the user's master cross-signing key. -Another way to do that is to prompt the user to enter the passphrase and ensure -that the generated private key correponds to the public key. +Another way to do that is to prompt the user to enter the passphrase used to +generate the encryption key and ensure that the generated private key +corresponds to the public key. #### Secret storage @@ -79,6 +80,8 @@ Example: Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: +`org.example.some.secret`: + ```json { "encrypted": { @@ -95,6 +98,28 @@ Some secret is encrypted using keys with ID `key_id_1` and `key_id_2`: } ``` +and the key descriptions for the keys would be: + +`m.secret_storage.key.key_id_1`: + +```json +{ + "name": "Some key", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` + +`m.secret_storage.key.key_id_2`: + +```json +{ + "name": "Some other key", + "algorithm": "m.secret_storage.v1.curve25519-aes-sha2", + // ... other properties according to algorithm +} +``` + #### Encryption algorithms ##### `m.secret_storage.v1.curve25519-aes-sha2` From fc793557f8d7e0a71ff671fb97b86d4529910e43 Mon Sep 17 00:00:00 2001 From: Hubert Chathi Date: Tue, 12 Nov 2019 10:48:38 -0500 Subject: [PATCH 18/18] make the default key event an object --- proposals/1946-secure_server-side_storage.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/1946-secure_server-side_storage.md b/proposals/1946-secure_server-side_storage.md index aa4fd8f91..fd907e535 100644 --- a/proposals/1946-secure_server-side_storage.md +++ b/proposals/1946-secure_server-side_storage.md @@ -49,10 +49,11 @@ A key with ID `abcdefg` is stored in `m.secret_storage.key.abcdefg` ``` A key can be marked as the "default" key by setting the user's account_data -with event type `m.secret_storage.default_key` to the ID of the key. The -default key will be used to encrypt all secrets that the user would expect to -be available on all their clients. Unless the user specifies otherwise, -clients will try to use the default key to decrypt secrets. +with event type `m.secret_storage.default_key` to an object that has the ID of +the key as its `key` property. The default key will be used to encrypt all +secrets that the user would expect to be available on all their clients. +Unless the user specifies otherwise, clients will try to use the default key to +decrypt secrets. Clients MUST ensure that the key is trusted before using it to encrypt secrets. One way to do that is to have the client that creates the key sign the key