Commit Graph

467 Commits (24fb567c4002255d4bc8359710f7bc4628c9bc31)

Author SHA1 Message Date
Brian Coca 8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
7 years ago
Ganesh Nalawade 5ec7f40196 Fix timeout issue in ansible-connection (#24556)
Fixes #24520 ansible-connection needs
to wait on timeout value of play-context
instead of ssh default timeout
7 years ago
Toshio Kuratomi 84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
7 years ago
Toshio Kuratomi d834412ead Fix for persistent connection plugin on Python3 (#24431)
Fix for persistent connection plugin on Python3.  Note that fixes are also needed to each terminal plugin.  This PR only fixes the ios terminal (as proof that this approach is workable.)  Future PRs can address the other terminal types.

* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin.  A pty can't have control characters.  So we have to send ascii only.  That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent.  So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat.  Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals

Fixes #24355
7 years ago
Peter Sprygada 53c52cf65f updates log messages in ansible-connection (#23076) 7 years ago
Peter Sprygada 77ce83fe22 fixes ansible-connection working directory (#22964)
When ansible-connection forks the process, it changed the working
directory to /.  This patch will prevent ansible-connection from
changing the working directory in the forked process.
7 years ago
Peter Sprygada 4a8f2dde20 fixes exception raised from ansible-connection logging (#22836)
updates ansible-connection logging functions to use display instead of
log.
7 years ago
Peter Sprygada e20ed8bc0c replaces logging with display for network connection plugins (#22819)
* updates network_cli and netconf connection plugins
* updates ansible-connection
7 years ago
Peter Sprygada ed7cace425 implements python logging for network connection plugin (#22817)
* enables logging for network_cli and paramiko
* enables logging for ansible-connection
* enabled logging for netconf connection
7 years ago
Peter Sprygada eed240797a adds connection established message to ansible-connection (#22492)
This will now print a log message that displays how long the ssh
connection took to establish to the remote device.  If the connection
established time is exceed it will also print a message to the log file
that the connection attempts have exceed.

Updates default connection retries to 30

fixes #22319
7 years ago
Toshio Kuratomi 9ff03e6c1e Handle downstream version additions (#22428)
Some downstreams want to ship multiple versions of ansible (Either to
have multiple ansible versions or to have a version that uses python3.X
and a version that uses python2.x).  When they do this, they append
a version number to the cli scripts in /usr/bin.  This patch will remove
those version numbers before trying to find the ansible python module to
import for this commandline
7 years ago
Peter Sprygada 6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 7 years ago
Peter Sprygada e05b2b56f2 capture AnsibleConnectionFailure messages in ansible-connection (#21612)
Send the messages to the log file
7 years ago
Peter Sprygada e77964f9da clean up pylint errors in ansible-connection (#21571) 7 years ago
Peter Sprygada eaba067de9 fixes type that prevents socket clean up from being called (#21568) 7 years ago
Peter Sprygada 3ff2c471b2 fixes Bad file descriptor backtrace raised by ansible-connection (#21526)
This change will redirect stdout and stderr either to the log file
configured by log_path or to /dev/null if no log_path is specified.

fixes #21400
7 years ago
Peter Sprygada 138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
8 years ago
Brian Coca 942ed146c3 errors to stderr 8 years ago
Peter Sprygada b4d75f41f5 one more bug fix for ansible-connection (#20707)
The connection instance will now return a meaningful error if the
connection property returns False
8 years ago
Peter Sprygada d668d53112 checks connected property to validate connection in Server (#20695)
fixes #20693
8 years ago
Peter Sprygada 6673673042 more logging and exception handling in ansible-connection (#20619)
adds more logging to handle display being called from plugins.  Also
rearranges some of the exception handling to better catch exceptions and
log to local syslog
8 years ago
Peter Sprygada 312328e22c adds more logging to ansible-connection (#20298)
* Server() instance will now add the entire traceback to syslog if it fail
  during init
8 years ago
Peter Sprygada 3f949358c7 adds some more logging to ansible-connection (#20205) 8 years ago
TaoBeier 6ec0369c26 fix indent (#20071) 8 years ago
Peter Sprygada 6fe9a5e40c functional updates to ansible-connection (#18574)
* sends the serialized play_context into an already established connection
* hooks the alarm_handler() method in the connection plugin if it exists
* added configuration options for connect interval and retries
* adds syslog logging to Server() instance

This update will send the updated play_context back into an already
established connection in case privilege escalation / descalation activities
need to be performed.  This change will also hook the alarm_handler() method
in the connection instance (if available) and call it in case of a
sigalarm raised.

This update adds two new configuration options

* PERSISTENT_CONNECT_INTERVAL - time to wait in between connection attempts
* PERSISTENT_CONNECT_RETRIES - max number of retries
8 years ago
James Cammarata 336f8d7233 Removing q debugging cruft from ansible-connection 8 years ago
James Cammarata 85064807af Removing test script for ansible-connection 8 years ago
James Cammarata 26ec2ecfce Adding a persistent connection utility 8 years ago
Brian Coca 18d3e0533f always log unexpected exceptions 8 years ago
Toshio Kuratomi bf29961947 Fixes to the controller text model (#17527)
* Fixes to the controller text model

* Change command line args to text type
* Make display replace undecodable bytes with replacement chars.  This
  is only a problem on pyhton3 where surrogates can enter into the msg
  but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml

* Fix up run_cmd to change its parameters to byte string at appropriate times.
8 years ago
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
8 years ago
James Cammarata 37f721f315 Remove debug lock 8 years ago
Michael Scherer f0023610e8 Fix bin/ansible to not make a double traceback on python3 (#15972) 8 years ago
Toshio Kuratomi dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
8 years ago
Brian Coca 0eb2844cc6 moved display's debug lock to cli as global 8 years ago
Brian Coca e74ab3ecdd draft 1st release of ansible-console
porting @dominis 's ansible-shell tool from 1.9 and integrating it into ansible
added verbosity control
made more resilitent to several errors
added highlight color, to configurable colors
more resilient on exception and interruptions
prompt coloring, goes red and changes to # when using become = true and root
become setting is now explicit and not a toggle
8 years ago
Toshio Kuratomi f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
9 years ago
James Cammarata 120b9a7ac6 Changing the way workers are forked 9 years ago
Toshio Kuratomi 318bfbb207 Migrate cli and dependencies to use global display 9 years ago
Brian Coca 10fd717a44 narrow the scope of import error and make sure non - aliases to ansible get their own error 9 years ago
Brian Coca d0a22a7b45 made cli code more flexible to add new command line programs w/o need for updating bin/ansible 9 years ago
Brian Coca 56a2412dc2 now correctly traps unimplemented symlinks as subprograms 9 years ago
Toshio Kuratomi 7c10c16251 Revert "capture some install issues and give out a slightly nicer error"
This reverts commit e6b1dc45e1.

This is not ready yet and it's causing failures in some environments
(travis)
9 years ago
Brian Coca e6b1dc45e1 capture some install issues and give out a slightly nicer error 9 years ago
Toshio Kuratomi f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 9 years ago
James Cammarata de792ba3c2 Improve handling of unicode errors
Fixes #12669
9 years ago
Brian Coca a5e6845c8b Revert "Allow exceptions to pass through the program"
breaks handling constants.py errors gracefully

This reverts commit 59f96d713e.
9 years ago
Will Thames 59f96d713e Allow exceptions to pass through the program
This is useful when using `ipython --pdb -- $(which ansible-playbook)
...` for debugging.

Also show traceback when `ANSIBLE_DEBUG` is on
9 years ago
Brian Coca e49575ff56 added wrap_text to 'last resort' as it fails if there is an error in callbacks 9 years ago
James Cammarata 469aa0c26a Don't wrap text on most errors 9 years ago