Merge pull request #12112 from amenonsen/vault-stdio

Implement cat-like filtering behaviour for encrypt/decrypt

docs/man/man1/ansible-vault.1

@@ -1,13 +1,13 @@
 '\" t
 .\"     Title: ansible-vault
 .\"    Author: [see the "AUTHOR" section]
-.\" Generator: DocBook XSL Stylesheets v1.78.1 <http://docbook.sf.net/>
+.\" Generator: DocBook XSL Stylesheets v1.76.1 <http://docbook.sf.net/>
-.\"      Date: 07/28/2015
+.\"      Date: 08/27/2015
 .\"    Manual: System administration commands
 .\"    Source: Ansible 2.0.0
 .\"  Language: English
 .\"
-.TH "ANSIBLE\-VAULT" "1" "07/28/2015" "Ansible 2\&.0\&.0" "System administration commands"
+.TH "ANSIBLE\-VAULT" "1" "08/27/2015" "Ansible 2\&.0\&.0" "System administration commands"
 .\" -----------------------------------------------------------------
 .\" * Define some portability stuff
 .\" -----------------------------------------------------------------
@@ -80,19 +80,35 @@ The \fBedit\fR sub\-command is used to modify a file which was previously encryp
 This command will decrypt the file to a temporary file and allow you to edit the file, saving it back when done and removing the temporary file\&.
 .SH "REKEY"
 .sp
-*$ ansible\-vault rekey [options] FILE_1 [FILE_2, \&..., FILE_N]
+\fB$ ansible\-vault rekey [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
 .sp
 The \fBrekey\fR command is used to change the password on a vault\-encrypted files\&. This command can update multiple files at once, and will prompt for both the old and new passwords before modifying any data\&.
 .SH "ENCRYPT"
 .sp
-*$ ansible\-vault encrypt [options] FILE_1 [FILE_2, \&..., FILE_N]
+\fB$ ansible\-vault encrypt [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
 .sp
 The \fBencrypt\fR sub\-command is used to encrypt pre\-existing data files\&. As with the \fBrekey\fR command, you can specify multiple files in one command\&.
+.sp
+Starting with version 2\&.0, the \fBencrypt\fR command accepts an \fB\-\-output FILENAME\fR option to determine where encrypted output is stored\&. With this option, input is read from the (at most one) filename given on the command line; if no input file is given, input is read from stdin\&. Either the input or the output file may be given as \fI\-\fR for stdin and stdout respectively\&. If neither input nor output file is given, the command acts as a filter, reading plaintext from stdin and writing it to stdout\&.
+.sp
+Thus any of the following invocations can be used:
+.sp
+\fB$ ansible\-vault encrypt\fR
+.sp
+\fB$ ansible\-vault encrypt \-\-output OUTFILE\fR
+.sp
+\fB$ ansible\-vault encrypt INFILE \-\-output OUTFILE\fR
+.sp
+\fB$ echo secret|ansible\-vault encrypt \-\-output OUTFILE\fR
+.sp
+Reading from stdin and writing only encrypted output is a good way to prevent sensitive data from ever hitting disk (either interactively or from a script)\&.
 .SH "DECRYPT"
 .sp
-*$ ansible\-vault decrypt [options] FILE_1 [FILE_2, \&..., FILE_N]
+\fB$ ansible\-vault decrypt [options] FILE_1 [FILE_2, \&..., FILE_N]\fR
 .sp
 The \fBdecrypt\fR sub\-command is used to remove all encryption from data files\&. The files will be stored as plain\-text YAML once again, so be sure that you do not run this command on data files with active passwords or other sensitive data\&. In most cases, users will want to use the \fBedit\fR sub\-command to modify the files securely\&.
+.sp
+As with \fBencrypt\fR, the \fBdecrypt\fR subcommand also accepts the \fB\-\-output FILENAME\fR option to specify where plaintext output is stored, and stdin/stdout is handled as described above\&.
 .SH "AUTHOR"
 .sp
 Ansible was originally written by Michael DeHaan\&. See the AUTHORS file for a complete list of contributors\&.

docs/man/man1/ansible-vault.1.asciidoc.in

@@ -84,7 +84,7 @@ file, saving it back when done and removing the temporary file.
 REKEY
 -----
 
-*$ ansible-vault rekey [options] FILE_1 [FILE_2, ..., FILE_N]
+*$ ansible-vault rekey [options] FILE_1 [FILE_2, ..., FILE_N]*
 
 The *rekey* command is used to change the password on a vault-encrypted files.
 This command can update multiple files at once, and will prompt for both the
@@ -93,21 +93,45 @@ old and new passwords before modifying any data.
 ENCRYPT
 -------
 
-*$ ansible-vault encrypt [options] FILE_1 [FILE_2, ..., FILE_N]
+*$ ansible-vault encrypt [options] FILE_1 [FILE_2, ..., FILE_N]*
 
 The *encrypt* sub-command is used to encrypt pre-existing data files. As with the
 *rekey* command, you can specify multiple files in one command.
 
+Starting with version 2.0, the *encrypt* command accepts an *--output FILENAME*
+option to determine where encrypted output is stored. With this option, input is
+read from the (at most one) filename given on the command line; if no input file
+is given, input is read from stdin. Either the input or the output file may be
+given as '-' for stdin and stdout respectively. If neither input nor output file
+is given, the command acts as a filter, reading plaintext from stdin and writing
+it to stdout.
+
+Thus any of the following invocations can be used:
+
+*$ ansible-vault encrypt*
+
+*$ ansible-vault encrypt --output OUTFILE*
+
+*$ ansible-vault encrypt INFILE --output OUTFILE*
+
+*$ echo secret|ansible-vault encrypt --output OUTFILE*
+
+Reading from stdin and writing only encrypted output is a good way to prevent
+sensitive data from ever hitting disk (either interactively or from a script).
+
 DECRYPT
 -------
 
-*$ ansible-vault decrypt [options] FILE_1 [FILE_2, ..., FILE_N]
+*$ ansible-vault decrypt [options] FILE_1 [FILE_2, ..., FILE_N]*
 
 The *decrypt* sub-command is used to remove all encryption from data files. The files
 will be stored as plain-text YAML once again, so be sure that you do not run this
 command on data files with active passwords or other sensitive data. In most cases, 
 users will want to use the *edit* sub-command to modify the files securely.
 
+As with *encrypt*, the *decrypt* subcommand also accepts the *--output FILENAME*
+option to specify where plaintext output is stored, and stdin/stdout is handled
+as described above.
 
 AUTHOR
 ------

lib/ansible/cli/__init__.py

@@ -260,8 +260,10 @@ class CLI(object):
                 dest='vault_password_file', help="vault password file", action="callback",
                 callback=CLI.expand_tilde, type=str)
             parser.add_option('--new-vault-password-file',
-			    dest='new_vault_password_file', help="new vault password file for rekey", action="callback",
+                dest='new_vault_password_file', help="new vault password file for rekey", action="callback",
-			    callback=CLI.expand_tilde, type=str)
+                callback=CLI.expand_tilde, type=str)
+            parser.add_option('--output', default=None, dest='output_file',
+                help='output file name for encrypt or decrypt; use - for stdout')
 
 
         if subset_opts:

lib/ansible/cli/vault.py

@@ -63,8 +63,21 @@ class VaultCLI(CLI):
         self.options, self.args = self.parser.parse_args()
         self.display.verbosity = self.options.verbosity
 
-        if len(self.args) == 0:
+        can_output = ['encrypt', 'decrypt']
-            raise AnsibleOptionsError("Vault requires at least one filename as a parameter")
+
+        if self.action not in can_output:
+            if self.options.output_file:
+                raise AnsibleOptionsError("The --output option can be used only with ansible-vault %s" % '/'.join(can_output))
+            if len(self.args) == 0:
+                raise AnsibleOptionsError("Vault requires at least one filename as a parameter")
+        else:
+            # This restriction should remain in place until it's possible to
+            # load multiple YAML records from a single file, or it's too easy
+            # to create an encrypted file that can't be read back in. But in
+            # the meanwhile, "cat a b c|ansible-vault encrypt --output x" is
+            # a workaround.
+            if self.options.output_file and len(self.args) > 1:
+                raise AnsibleOptionsError("At most one input file may be used with the --output option")
 
     def run(self):
 
@@ -87,19 +100,34 @@ class VaultCLI(CLI):
 
         self.execute()
 
-    def execute_create(self):
+    def execute_encrypt(self):
 
-        if len(self.args) > 1:
+        if len(self.args) == 0 and sys.stdin.isatty():
-            raise AnsibleOptionsError("ansible-vault create can take only one filename argument")
+            self.display.display("Reading plaintext input from stdin", stderr=True)
 
-        self.editor.create_file(self.args[0])
+        for f in self.args or ['-']:
+            self.editor.encrypt_file(f, output_file=self.options.output_file)
+
+        if sys.stdout.isatty():
+            self.display.display("Encryption successful", stderr=True)
 
     def execute_decrypt(self):
 
-        for f in self.args:
+        if len(self.args) == 0 and sys.stdin.isatty():
-            self.editor.decrypt_file(f)
+            self.display.display("Reading ciphertext input from stdin", stderr=True)
+
+        for f in self.args or ['-']:
+            self.editor.decrypt_file(f, output_file=self.options.output_file)
+
+        if sys.stdout.isatty():
+            self.display.display("Decryption successful", stderr=True)
 
-        self.display.display("Decryption successful", stderr=True)
+    def execute_create(self):
+
+        if len(self.args) > 1:
+            raise AnsibleOptionsError("ansible-vault create can take only one filename argument")
+
+        self.editor.create_file(self.args[0])
 
     def execute_edit(self):
         for f in self.args:
@@ -110,13 +138,6 @@ class VaultCLI(CLI):
         for f in self.args:
             self.editor.view_file(f)
 
-    def execute_encrypt(self):
-
-        for f in self.args:
-            self.editor.encrypt_file(f)
-
-        self.display.display("Encryption successful", stderr=True)
-
     def execute_rekey(self):
         for f in self.args:
             if not (os.path.isfile(f)):

lib/ansible/parsing/vault/__init__.py

@@ -20,6 +20,7 @@ __metaclass__ = type
 import os
 import shlex
 import shutil
+import sys
 import tempfile
 from io import BytesIO
 from subprocess import call
@@ -130,7 +131,7 @@ class VaultLib:
         b_data = to_bytes(data, errors='strict', encoding='utf-8')
 
         if self.is_encrypted(b_data):
-            raise AnsibleError("data is already encrypted")
+            raise AnsibleError("input is already encrypted")
 
         if not self.cipher_name or self.cipher_name not in CIPHER_WRITE_WHITELIST:
             self.cipher_name = u"AES256"
@@ -162,7 +163,7 @@ class VaultLib:
             raise AnsibleError("A vault password must be specified to decrypt data")
 
         if not self.is_encrypted(b_data):
-            raise AnsibleError("data is not encrypted")
+            raise AnsibleError("input is not encrypted")
 
         # clean out header
         b_data = self._split_header(b_data)
@@ -227,7 +228,7 @@ class VaultLib:
 class VaultEditor:
 
     def __init__(self, password):
-        self.password = password
+        self.vault = VaultLib(password)
 
     def _edit_file_helper(self, filename, existing_data=None, force_save=False):
         # make sure the umask is set to a sane value
@@ -248,11 +249,8 @@ class VaultEditor:
             os.remove(tmp_path)
             return
 
-        # create new vault
-        this_vault = VaultLib(self.password)
-
         # encrypt new data and write out to tmp
-        enc_data = this_vault.encrypt(tmpdata)
+        enc_data = self.vault.encrypt(tmpdata)
         self.write_data(enc_data, tmp_path)
 
         # shuffle tmp file into place
@@ -261,109 +259,94 @@ class VaultEditor:
         # and restore umask
         os.umask(old_umask)
 
-    def create_file(self, filename):
+    def encrypt_file(self, filename, output_file=None):
-        """ create a new encrypted file """
 
         check_prereqs()
 
-        if os.path.isfile(filename):
+        plaintext = self.read_data(filename)
-            raise AnsibleError("%s exists, please use 'edit' instead" % filename)
+        ciphertext = self.vault.encrypt(plaintext)
+        self.write_data(ciphertext, output_file or filename)
 
-        # Let the user specify contents and save file
+    def decrypt_file(self, filename, output_file=None):
-        self._edit_file_helper(filename)
+
+        check_prereqs()
 
-    def decrypt_file(self, filename):
+        ciphertext = self.read_data(filename)
+        plaintext = self.vault.decrypt(ciphertext)
+        self.write_data(plaintext, output_file or filename)
+
+    def create_file(self, filename):
+        """ create a new encrypted file """
 
         check_prereqs()
 
-        if not os.path.isfile(filename):
+        # FIXME: If we can raise an error here, we can probably just make it
-            raise AnsibleError("%s does not exist" % filename)
+        # behave like edit instead.
+        if os.path.isfile(filename):
+            raise AnsibleError("%s exists, please use 'edit' instead" % filename)
 
-        tmpdata = self.read_data(filename)
+        self._edit_file_helper(filename)
-        this_vault = VaultLib(self.password)
-        if this_vault.is_encrypted(tmpdata):
-            dec_data = this_vault.decrypt(tmpdata)
-            if dec_data is None:
-                raise AnsibleError("Decryption failed")
-            else:
-                self.write_data(dec_data, filename)
-        else:
-            raise AnsibleError("%s is not encrypted" % filename)
 
     def edit_file(self, filename):
 
         check_prereqs()
 
-        # decrypt to tmpfile
+        ciphertext = self.read_data(filename)
-        tmpdata = self.read_data(filename)
+        plaintext = self.vault.decrypt(ciphertext)
-        this_vault = VaultLib(self.password)
-        dec_data = this_vault.decrypt(tmpdata)
 
-        # let the user edit the data and save
+        if self.vault.cipher_name not in CIPHER_WRITE_WHITELIST:
-        if this_vault.cipher_name not in CIPHER_WRITE_WHITELIST:
             # we want to get rid of files encrypted with the AES cipher
-            self._edit_file_helper(filename, existing_data=dec_data, force_save=True)
+            self._edit_file_helper(filename, existing_data=plaintext, force_save=True)
         else:
-            self._edit_file_helper(filename, existing_data=dec_data, force_save=False)
+            self._edit_file_helper(filename, existing_data=plaintext, force_save=False)
 
     def view_file(self, filename):
 
         check_prereqs()
 
-        # decrypt to tmpfile
+        # FIXME: Why write this to a temporary file at all? It would be safer
-        tmpdata = self.read_data(filename)
+        # to feed it to the PAGER on stdin.
-        this_vault = VaultLib(self.password)
-        dec_data = this_vault.decrypt(tmpdata)
         _, tmp_path = tempfile.mkstemp()
-        self.write_data(dec_data, tmp_path)
+        ciphertext = self.read_data(filename)
+        plaintext = self.vault.decrypt(ciphertext)
+        self.write_data(plaintext, tmp_path)
 
         # drop the user into pager on the tmp file
         call(self._pager_shell_command(tmp_path))
         os.remove(tmp_path)
 
-    def encrypt_file(self, filename):
-
-        check_prereqs()
-
-        if not os.path.isfile(filename):
-            raise AnsibleError("%s does not exist" % filename)
-
-        tmpdata = self.read_data(filename)
-        this_vault = VaultLib(self.password)
-        if not this_vault.is_encrypted(tmpdata):
-            enc_data = this_vault.encrypt(tmpdata)
-            self.write_data(enc_data, filename)
-        else:
-            raise AnsibleError("%s is already encrypted" % filename)
-
     def rekey_file(self, filename, new_password):
 
         check_prereqs()
 
-        # decrypt
+        ciphertext = self.read_data(filename)
-        tmpdata = self.read_data(filename)
+        plaintext = self.vault.decrypt(ciphertext)
-        this_vault = VaultLib(self.password)
-        dec_data = this_vault.decrypt(tmpdata)
 
-        # create new vault
         new_vault = VaultLib(new_password)
-
+        new_ciphertext = new_vault.encrypt(plaintext)
-        # re-encrypt data and re-write file
+        self.write_data(new_ciphertext, filename)
-        enc_data = new_vault.encrypt(dec_data)
-        self.write_data(enc_data, filename)
 
     def read_data(self, filename):
-        f = open(filename, "rb")
+        try:
-        tmpdata = f.read()
+            if filename == '-':
-        f.close()
+                data = sys.stdin.read()
-        return tmpdata
+            else:
+                with open(filename, "rb") as fh:
+                    data = fh.read()
+        except Exception as e:
+            raise AnsibleError(str(e))
+
+        return data
 
     def write_data(self, data, filename):
-        if os.path.isfile(filename):
+        bytes = to_bytes(data, errors='strict')
-            os.remove(filename)
+        if filename == '-':
-        f = open(filename, "wb")
+            sys.stdout.write(bytes)
-        f.write(to_bytes(data, errors='strict'))
+        else:
-        f.close()
+            if os.path.isfile(filename):
+                os.remove(filename)
+            with open(filename, "wb") as fh:
+                fh.write(bytes)
 
     def shuffle_files(self, src, dest):
         # overwrite dest with src