Commit Graph

618 Commits (f384fc33d06b1abb66d316943ef9bb7e7cd3d162)

Author SHA1 Message Date
Steven Robertson ba222744af
Merge branch 'master' into add0.2.10+0.3.0changelog 4 years ago
MarkusTeufelberger 8d3026b109
Add ansible.legacy.setup to be fixed on py3.5 4 years ago
Steven Robertson f489478127 code cleanup + adds 0.2.10 + 0.3.0 changelog 4 years ago
Steven Robertson 2510f1a2c2 fix py3.5.1-3.5.3 setup import error for Ansible 2.10 4 years ago
Steven Robertson 5a0da02e6c code review changes, using when statements and adding trailing comma 4 years ago
Steven Robertson 196a476270 🎉 no more warnings, only load specific collection subdirs instead of top-level collection path (ie no ansible_collections/google, only ansible_collections/google/cloud, etc) 4 years ago
Steven Robertson 741e99f698 ansible 2.10 no longer has a at the end of the error msg... 🤦 4 years ago
Steven Robertson 95c43ec9fc fixed issue of switching between mitogen and non-mitogen strategies 4 years ago
Steven Robertson e30e84334e remove synchronize fail test for azure 4 years ago
Steven Robertson 139b9560bc print didn't work because verbosity, throw valueerror to see 4 years ago
Steven Robertson 9bd35adcfb more debugging, synchronize is being weird on azure 4 years ago
Steven Robertson ff8a276186 turn off failing Ansible-only tests for now, also raising errors to see what Azure is gonna do with collections 4 years ago
Steven Robertson f757dbcb82 removed duplicate install and added debug dump of collection loading to see what tests are doing 4 years ago
Steven Robertson 41a13ebce2 hopefully this also fails the same way 4 years ago
Steven Robertson 5aedb5f157 add missing collections 🤦 4 years ago
Steven Robertson 9857dfea5c verify collection is working as expected 4 years ago
Steven Robertson a40c28d93f can't replicate but think it's because synchronize is now a collection 4 years ago
Steven Robertson 49dd8eee1a figure out what synchronize is now 4 years ago
Steven Robertson c6d42212dd add some debugging info, was able to run the failed synchronize test locally just fine using test framework, not sure what's going on 4 years ago
Steven Robertson f91cbf4d00 test cleanup and trying to replicate synchronize fails 4 years ago
Steven Robertson f994807d74 ansible 2.10 renamed ping module to reflect legacy builtin collection 4 years ago
Steven Robertson 03438271bb able to remove the hack on ActionBase after all 4 years ago
Steven Robertson 5215646c8a code cleanup 4 years ago
Steven Robertson 0b421e0d3c able to run docker_container installed via 'ansible-galaxy collection install community.general' 4 years ago
Steven Robertson 45797a0d34 able to send collections paths to master with very little change to core Mitogen, need to ensure imports work properly now though 4 years ago
Steven Robertson 6e51f1a184 found a way to load collections without affecting mitogen core code 4 years ago
Steven Robertson 8d3ee26079 code cleanup 4 years ago
Steven Robertson ca94751f15 remove hack 4 years ago
Steven Robertson f1bdc39047 added note about breaking backwards compat 4 years ago
Steven Robertson 1d13df718a connection_loader.get isn't called anymore, it's connection_loader.get_with_context now 4 years ago
Steven Robertson 6ba08097b6 more notes, strategy plugin is being called but Mitogen's method overrides aren't being triggered 4 years ago
Steven Robertson ca4e8116b7 TODO: turns out ansible 2.10 doesn't run Mitogen like it used to; was running old ansible version before because ansible-base didn't override everything. Did a fresh uninstall of ansible and installed 2.10.0 and Mitogen's connection monkeypatching isn't working 4 years ago
Steven Robertson 1bd4b8afcd much thinking needs to be done regarding how to handle not requiring sshpass... 4 years ago
Steven Robertson 6ac9168d55 need to get around sshpass check here somehow: https://github.com/ansible/ansible/blob/v2.10.0/lib/ansible/plugins/connection/ssh.py#L577 4 years ago
Steven Robertson 583f540889 added comments 4 years ago
Steven Robertson e34cf8667f
Merge branch 'master' into collectionsSupport 4 years ago
Steven Robertson 81076c9da8 fixes setup module relative import fail on some pythons 4 years ago
Steven Robertson 376d8d0fab remove old hacks; ansible_collections is available at time of invoker but not later 4 years ago
Steven Robertson 955e77c5db WIP: able to load subdirs but now need to treat them as submodules properly 4 years ago
Steven Robertson ab55d05267 all in on ansible 2.10+ for collections support 4 years ago
Steven Robertson c92a9ace41 bump max ansible version 4 years ago
Steven Robertson ac1e72eec4 Merge branch 'master' into collectionsSupport 4 years ago
Steven Robertson e632310fc4 no tmpdir to remove for old ansible versions in fetch command 5 years ago
Steven Robertson ddc1eebec8 able to load collection but the mitogen master proc was unable to serve it 5 years ago
Steven Robertson e99d63f4fc able to load collections but they don't do anything because no actionmodulemixin, need to tweak how this works 5 years ago
Steven Robertson acde13f9d6 handles a 'wait_for_connection' call right after a task caused a shutdown 5 years ago
Steven Robertson 5bf327649f add missing / to python paths 5 years ago
Steven Robertson 389cee8c70 fix assertion of python_path, it should be always what 'discovered_interpreter' is 5 years ago
Steven Robertson c959ce010b created temp action obj for 'meta: reset_connection' python interpreter discovery 5 years ago
Steven Robertson 5b8c9da769 _execute_meta doesn't have an Action nooooo this breaks 'meta: reset_connection' because interpreter_discovery needs an Action 5 years ago
Steven Robertson 33e7d0804f fixed tests, and removed the action hack code 5 years ago
Steven Robertson 7411057ba8 adding in python3 as a possible_pythons option for rhel8 5 years ago
Steven Robertson 89d87445c8 able to pass tests running interpreter detection in rhel8 container 5 years ago
Steven Robertson 7d6d76e444 trying to get docker rhel8 working on a Mac host 5 years ago
Steven Robertson 5e0fb902b2 fix lambda args for fallback 5 years ago
Steven Robertson 91785f8862 fix another bad import 5 years ago
Steven Robertson ee310c1a68 remove_internal_keys returns None, so need different way to check fallback 5 years ago
Steven Robertson 722cf0ea8d handle ansible 2.3.3 remove_internal_keys 5 years ago
Steven Robertson cf1c3aac1c fix typo 5 years ago
Steven Robertson ba350aa6cf assigned interpreter vars to the wrong class 5 years ago
Steven Robertson d74da84f62 the clean module was added later 5 years ago
Steven Robertson 496d96c01a need to convert to str because of unicode_literals 5 years ago
Steven Robertson aa0ad5ce8f accidentally broke discovery 5 years ago
Steven Robertson 160dbd86b4 keeping track of interpreter discovery vars in case of ansible < 2.8 5 years ago
Steven Robertson f10fb7d486 more test cases pass 5 years ago
Steven Robertson e3eebb98d0 ImportError instead of newer ModuleNotFoundError 5 years ago
Steven Robertson 2d3d9cb659 fallback to old-style python interpreter default if running ansible < 2.8.0 5 years ago
Steven Robertson ff2fa60184 fixed typo 5 years ago
Steven Robertson 0b57332d75 all test cases for ansible_python_interpreter on OSX pass now 5 years ago
Steven Robertson 81694d07ab fix typo 5 years ago
Steven Robertson 8630f6190c fixed propagating discovered interpreter to future tasks; sys.executable still wrong though 5 years ago
Steven Robertson af1e22d4e1 able to propagate ansible_python_interpreter fact but still doesn't save between task runs 5 years ago
Steven Robertson 119e1c24aa code cleanup 5 years ago
Steven Robertson cec692dc17 faking pipelining so discover_interpreter can successfully exit 5 years ago
Steven Robertson a6d2b2cd90 doesn't error when declaring 'ansible_python_interpreter: auto' but fact isn't being cached 5 years ago
Steven Robertson 19ba2417c8 now able to get the action object, but recursion error in low_level_exec_command 5 years ago
Steven Robertson 806651e112 working on ansible discovery; need to fix passing ActionModuleMixin reference still 5 years ago
Steven Robertson d64adb15d7 reverted autolinted code 5 years ago
Steven Robertson e8f3154cab Merge branch 'master' into complexAnsiblePythonInterpreterArg 5 years ago
David Wilson d6329f3446 Merge devel/290 @ 79b979ec8544ef5d8620c64068d4a42fabf50415 5 years ago
Steven Robertson 4669c8774f handles templating ansible_python_interpreter values 5 years ago
Steven Robertson cc11864b7d code cleanup 5 years ago
Steven Robertson 24b170311a able to get to 'sudo: source not found' after preventing escape of && so python connects 5 years ago
David Wilson be4f1bdb50 issue #646: add extra logging to assertions and start_child() 5 years ago
David Wilson efd82dd35a issue #633: various task_vars fixes
- take host_vars from task_vars too
- make missing task_vars a hard error
- update tests to provide stub task_vars
5 years ago
David Wilson fc09b81949 issue #633: handle meta: reset_connection when become is active
- don't create a new connection during reset if no existing connection
  exists
- strip off last hop in connection stack if PlayContext.become is True.
- log a debug message if reset cannot find an existing connection
5 years ago
David Wilson b6d1df749c issue #633: take inventory_hostname from task_vars
It used to be set by on_action_run() from task_vars, but this doesn't
work for meta: reset_connection. That meant MITOGEN_CPU_COUNT>1 would
pick the wrong mux to reset the connection on.
5 years ago
David Wilson f4cf67f0bd issue #615: remove meaningless test
It has been dead code since at least 2015
5 years ago
David Wilson e02be89879 issue #625: ignore SIGINT within MuxProcess
Without this, MuxProcess will start dying too early, before Ansible /
TaskQueueManager.cleanup() has a chance to wait on worker processes.
That would allow WorkerProcess to see ECONNREFUSED from the MuxProcess
socket much more easily.
5 years ago
David Wilson 67759371f9 issue #615: ensure 4GB max_message_size is configured for task workers.
This 4GB limit was already set for MuxProcess and inherited by all
descendents including the context running on the target host, but it was
not applied to the WorkerProcess router.

That explains why the error from the ticket is being raised by the
router within the WorkerProcess rather than the router on the original
target.
5 years ago
David Wilson 151b490890 issue #615: fetch_file() might be called with AnsibleUnicode. 5 years ago
David Wilson 03d2bc6c59 issue #615: redirect 'fetch' action to 'mitogen_fetch'. 5 years ago
David Wilson 52c8ed7715 issue #615: extricate slurp brainwrong from mitogen_fetch 5 years ago
David Wilson 069285a588 issue #615: ansible: import Ansible fetch.py action plug-in
From ansible/ansible#9773a1f2896a914d237cb9926e3b5cdc0f004d1a
5 years ago
David Wilson 8dfb3966df issue #558, #582: preserve remote tmpdir if caller did not supply one
The undocumented 'tmp' parameter controls whether _execute_module()
would delete anything on 2.3, so mimic that. This means
_execute_remove_stat() calls will not blow away the temp directory,
which broke the unarchive plugin.
5 years ago
David Wilson 3b63da670f Fix up another handful of LGTM errors. 5 years ago
David Wilson f78a5f08c6 issue #605: ansible: share a sem_t instead of a pthread_mutex_t
The previous version quite reliably causes worker deadlocks within 10
minutes running:

    # 100 times:
    - import_playbook: integration/async/runner_one_job.yml
    # 100 times:
    - import_playbook: integration/module_utils/adjacent_to_playbook.yml

via .ci/soak/mitogen.sh with PLAYBOOK= set to the above playbook.

Attaching to the worker with gdb reveals it in an instruction
immediately following a futex() call, which likely returned EINTR due to
attaching gdb. Examining the pthread_mutex_t state reveals it to be
completely unlocked.

pthread_mutex_t on Linux should have zero trouble living in shmem, so
it's not clear how this deadlock is happening. Meanwhile POSIX
semaphores are explicitly designed for cross-process use and have a
completely different internal implementation, so try those instead. 1
hour of soaking reveals no deadlock.

This is about avoiding managing a lockable temporary file on disk to
contain our counter, and somehow communicating a reference to it into
subprocesses (despite the subprocess module closing inherited fds, etc),
somehow deleting it reliably at exit, and somehow avoiding concurrent
Ansible runs stepping on the same file. For now ctypes is still less
pain.

A final possibility would be to abandon a shared counter and instead
pick a CPU based on the hash of e.g. the new child's process ID. That
would likely balance equally well, and might be worth exploring when
making this code work on BSD.
5 years ago
David Wilson 5af6c9b26f issue #615: use FileService for target->controll file transfers 5 years ago
David Wilson 6f12980611 [linear2] merge fallout: re-enable _send_module_forwards(). 5 years ago
David Wilson 5298e87548 Split out and make readable more log messages across both packages 5 years ago