Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* add 'file options' for become and connection pass
* implemented getting passwords from file or script
* added config entry
* fixed env var name and noted executable behaviuor
Co-authored-by: Rick Elrod <rick@elrod.me>
Change:
- After we output the newline in help text, we also added a space. This
caused either a space to show up before the shell prompt (in shells
like bash), or " %" and a newline before the next prompt on shells
like zsh that automatically force commands to end with a newline.
- This change removes the extra space, so that peace can be restored in
the minds of many.
- This only showed up when running commands with no arguments or invalid
arguments.
Test Plan:
- Ran `ansible`, `ansible -h`, `ansible-playbook`, `ansible-galaxy`, and
`ansible-galaxy -h`.
Tickets:
- Introduced in #69458
Signed-off-by: Rick Elrod <rick@elrod.me>
* Remove space before comma in '... bla , use ...'
* 'why' is inserted in the middle of a sentence, between two commas.
* Make deprecations from base.yml show source ansible-core.
* Add changelog fragment.
* Improve some more 'why's.
* Add PR URL to fragment.
* Fix tty_ify bugs and refactor
* Move tty_ify() and supporting attributes to the DocCLI class as that's
the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
website docs many years ago.
* Update test/units/cli/test_doc.py
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`
Test Plan:
- New integration and unit tests
Tickets:
- Fixes#69619
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow to specify collection_name separately for deprecation.
* Use new functionality in Ansible.
* Use new functionality in tests.
* Update tagging/untagging functions.
* Update pylint deprecated sanity test.
* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).
* Improve version validation. Re-add version_added validation.
* Make sure collection names are added to return docs before schema validation.
* Extra checks to avoid crashes on bad data.
* Make C# module utils code work, and update/extend tests.
* Add changelog fragment.
* Stop extracting collection name from potentially tagged versions/dates.
* Simplify C# code.
* Update Windows modules docs.
* Forgot semicolons.
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.
* Make tagging/untagging functions more flexible.
* Tag all versions in doc fragments.
* Tag all deprecation versions issued by code.
* Make Display.deprecated() understand tagged versions.
* Extend validation to enforce tagged version numbers.
* Tag versions in tests.
* Lint and fix test.
* Mention collection name in collection loader's deprecation/removal messages.
* Fix error IDs.
* Handle tagged dates in Display.deprecated().
* Also require that removed_at_date and deprecated_aliases.date are tagged.
* Also automatically tag/untag removed_at_date; fix sanity module removal version check.
* Improve error message when invalid version number is used (like '2.14' in collections).
* Allow to deprecate module by date in documentation.
* Make sure deprecation date/version match between module docs and meta/runtime.yml.
* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.
* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.
* Linting.
* Allow to deprecate plugin options by date.
* Add changelog fragment for deprecation by date (also covers #68177).
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
* Add warning when running devel. Addresses #67362
* Add config entry to disable devel warning, so it doesn't impact CI
* Update warning about using devel
* fix default collection resolution in adhoc
* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names
* Set ANSIBLE_PLAYBOOK_DIR in integration tests.
* Fix config conflict in ansible integration test.
* add adhoc default collection test
* text-ify warning string
* ansible-galaxy tidy up arg parse with better validation
* Add support back in for -v before sub aprser
* Added deprecation warning for manually parsed verbosity
* Add custom action class for version info
* Use args from CLI as prog for ArgumentParser object
* Make prog a required parameter of create_base_parser() and update all uses to pass in the newly required parameter.
* Add unit test for checking ansible --version
* Update other related unit tests
* Start of migration to argparse
* various fixes and improvements
* Linting fixes
* Test fixes
* Fix vault_password_files
* Add PrependAction for argparse
* A bunch of additional tweak/fixes
* Fix ansible-config tests
* Fix man page generation
* linting fix
* More adhoc pattern fixes
* Add changelog fragment
* Add support for argcomplete
* Enable argcomplete global completion
* Rename PrependAction to PrependListAction to better describe what it does
* Add documentation for installing and configuring argcomplete
* Address rebase issues
* Fix display encoding for vault
* Fix line length
* Address rebase issues
* Handle rebase issues
* Use mutually exclusive group instead of handling manually
* Fix rebase issues
* Address rebase issue
* Update version added for argcomplete support
* -e must be given a value
* ci_complete
Ensure inventory plugin loading rel to play
fixes#51033
* clarify paths
* now adding dirs funciton in loader
* better warnings
* each cli should handle adding dirs depending on context
* [WIP] become plugins
Move from hardcoded method to plugins for ease of use, expansion and overrides
- load into connection as it is going to be the main consumer
- play_context will also use to keep backwards compat API
- ensure shell is used to construct commands when needed
- migrate settings remove from base config in favor of plugin specific configs
- cleanup ansible-doc
- add become plugin docs
- remove deprecated sudo/su code and keywords
- adjust become options for cli
- set plugin options from context
- ensure config defs are avaialbe before instance
- refactored getting the shell plugin, fixed tests
- changed into regex as they were string matching, which does not work with random string generation
- explicitly set flags for play context tests
- moved plugin loading up front
- now loads for basedir also
- allow pyc/o for non m modules
- fixes to tests and some plugins
- migrate to play objects fro play_context
- simiplify gathering
- added utf8 headers
- moved option setting
- add fail msg to dzdo
- use tuple for multiple options on fail/missing
- fix relative plugin paths
- shift from play context to play
- all tasks already inherit this from play directly
- remove obsolete 'set play'
- correct environment handling
- add wrap_exe option to pfexec
- fix runas to noop
- fixed setting play context
- added password configs
- removed required false
- remove from doc building till they are ready
future development:
- deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems
* cleanup
remove callers to removed func
removed --sudo cli doc refs
remove runas become_exe
ensure keyerorr on plugin
also fix backwards compat, missing method is attributeerror, not ansible error
get remote_user consistently
ignore missing system_tmpdirs on plugin load
correct config precedence
add deprecation
fix networking imports
backwards compat for plugins using BECOME_METHODS
* Port become_plugins to context.CLIARGS
This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
instead
* Refactor make_become_commands as asked for by alikins
* Typo in comment fix
* Stop loading values from the cli in more than one place
Both play and play_context were saving default values from the cli
arguments directly. This changes things so that the default values are
loaded into the play and then play_context takes them from there.
* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH
As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that
should be done all at one time rather than piecemeal.
* One to throw away
This is a set of hacks to get setting FieldAttribute defaults to command
line args to work. It's not fully done yet.
After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.
What we want to be able to do ideally is something like this:
class Base(FieldAttributeBase):
_check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])
class Play(Base):
# lambda so that we have a chance to parse the command line args
# before we get here. In the future we might be able to restructure
# this so that the cli parsing code runs before these classes are
# defined.
class Task(Base):
pass
And still have a playbook like this function:
---
- hosts:
tasks:
- command: whoami
check_mode: True
(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).
There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now. The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)
* Revert "One to throw away"
This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.
* Set FieldAttr defaults directly from CLIARGS
* Remove dead code
* Move timeout directly to PlayContext, it's never needed on Play
* just for backwards compat, add a static version of BECOME_METHODS to constants
* Make the become attr on the connection public, since it's used outside of the connection
* Logic fix
* Nuke connection testing if it supports specific become methods
* Remove unused vars
* Address rebase issues
* Fix path encoding issue
* Remove unused import
* Various cleanups
* Restore network_cli check in _low_level_execute_command
* type improvements for cliargs_deferred_get and swap shallowcopy to default to False
* minor cleanups
* Allow the su plugin to work, since it doesn't define a prompt the same way
* Fix up ksu become plugin
* Only set prompt if build_become_command was called
* Add helper to assist connection plugins in knowing they need to wait for a prompt
* Fix tests and code expectations
* Doc updates
* Various additional minor cleanups
* Make doas functional
* Don't change connection signature, load become plugin from TaskExecutor
* Remove unused imports
* Add comment about setting the become plugin on the playcontext
* Fix up tests for recent changes
* Support 'Password:' natively for the doas plugin
* Make default prompts raw
* wording cleanups. ci_complete
* Remove unrelated changes
* Address spelling mistake
* Restore removed test, and udpate to use new functionality
* Add changelog fragment
* Don't hard fail in set_attributes_from_cli on missing CLI keys
* Remove unrelated change to loader
* Remove internal deprecated FieldAttributes now
* Emit deprecation warnings now
* move extravars and option vars loading into VM
also safedir setting, all these are intrinsic to VM
avoid uneeded and inefectual shallow copy
remove setters/getters as VM now does most of the work in init
updated and added tests
* feedback + fixes
* keep extra_vars property for vars_prompt
* pass values not objects
* Note: Python2 is not as intelligent at detecting false import loops as
Python3. context_objects.py cannot be added to cli/arguments because it
would set up an import loop between cli/__init__.py,
cli/arguments/context_objects.py, and context.py on Python2.
ci_complete
* Mark methods which are really functions as staticmethod
* Fix calls to other staticmethods to use the subclass rather than the
base class so that any inheritance overriding will be honored.
* Remove unnecessary logic and dead code
* Fix a typo in a docstring of how to implement subclass init_parser()
methods
* Call superclass's post_process_args in ansible-doc
* Fix copyright comment according to suggested practice
The goal of breaking apart the base_parser() function is to get rid of
a bunch of conditionals and parameters in the code and, instead, make
code look like simple composition.
When splitting, a choice had to be made as to whether this would operate
by side effect (modifying a passed in parser) or side effect-free
(returning a new parser everytime).
Making a version that's side-effect-free appears to be fighting with the
optparse API (it wants to work by creating a parser object, configuring
the object, and then parsing the arguments with it) so instead, make it
clear that our helper functions are modifying the passed in parser by
(1) not returning the parser and (2) changing the function names to be
more clear that it is operating by side-effect.
Also move all of the generic optparse code, along with the argument
context classes, into a new subdirectory.
* Once cli args are parsed, they're constant. So, save the parsed args
into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
* Got rid of the private self._options attribute
* Use context.CLIARGS in the individual callback plugins instead.
* Also output positional arguments in default and unixy plugins
* Code has been simplified since we're now dealing with a dict rather
than Optparse.Value
If there were fatal bugs in this portion of config, they would never be displayed
because config would fail to load and then every^U (Exageration... only
half of every) other part of the code which depended on config to be
loaded would fail before we ever got around to a section of code that
would process UNABLE.
Remove the try except from here so that we are able to debug this code
NOTE:
1. Use unsafe decorator but not builtin escape wrapper in jinja2
since ansible will try parse ssh password twice, the builtin
escape wrapper will be removed during the first parse.
2. Use class AnsibleUnsafeText but not '!unsafe' syntax since
passwords are not loaded by YAML env, '!unsafe' syntax doesn't
work for them.
Add the 'localhost_warning' configuration option. When set to 'false',
this will prevent Ansible from issuing a warning when the inventory is
empty and it is using an implicit inventory with only 'localhost'.
Closes#17086
Enforce that there can be only one --new-vault-id or
--new-vault-password-file and use this instead of
--encrypt-vault-id
* Add a config option for default vault encrypt id
`ansible-vault` is the only cli command which knows how to handle the
rekey options `--new-vault-id` and `--new-vault-password-file`. No
point in exposing those rekey options to any of the other ansible
commands.
On a practical level I think this matters most in ensuring that
`--help` doesn't produce any false/unhelpful output.
* Fix vault --ask-vault-pass with no tty
2.4.0 added a check for isatty() that would skip setting up interactive
vault password prompts if not running on a tty.
But... getpass.getpass() will fallback to reading from stdin if
it gets that far without a tty. Since 2.4.0 skipped the interactive
prompts / getpass.getpass() in that case, it would never get a chance
to fall back to stdin.
So if 'echo $VAULT_PASSWORD| ansible-playbook --ask-vault-pass site.yml'
was ran without a tty (ie, from a jenkins job or via the vagrant
ansible provisioner) the 2.4 behavior was different than 2.3. 2.4
would never read the password from stdin, resulting in a vault password
error like:
ERROR! Attempting to decrypt but no vault secrets found
Fix is just to always call the interactive password prompts based
on getpass.getpass() on --ask-vault-pass or --vault-id @prompt and
let getpass sort it out.
* up test_prompt_no_tty to expect prompt with no tty
We do call the PromptSecret class if there is no tty, but
we are back to expecting it to read from stdin in that case.
* Fix logic for when to auto-prompt vault pass
If --ask-vault-pass is used, then pretty much always
prompt.
If it is not used, then prompt if there are no other
vault ids provided and 'auto_prompt==True'.
Fixes vagrant bug https://github.com/hashicorp/vagrant/issues/9033Fixes#30993
Matt Martz