Commit Graph

3391 Commits (11a61acb32156fea37ed970d96f082f74d3e4358)
 

Author SHA1 Message Date
David Wilson 8cc20856a8 issue #106: support custom new-style modules + 'reexec by default'
Rather than assume any structure about the Python code:

* Delete the exit_json/fail_json monkeypatches.
* Patch SystemExit rather than a magic monkeypatch-thrown exception
* Setup fake cStringIO stdin, stdout, stderr and return those along with
  SystemExit exit status
* Setup _ANSIBLE_ARGS as we used to, since we still want to override
  that with '{}' to prevent accidental import hangs, but also provide
  the same string via sys.stdin.
* Compile the module bytecode once and re-execute it for every
  invocation. May change this back again later, once some benchmarks are
  done.
* Remove the fixups stuff for now, it's handled by ^ above.

Should support any "somewhat new style" Python module, including those
that just give up and dump stuff to stdout directly.
7 years ago
David Wilson a954d54644 issue #106: support old-style too. 7 years ago
David Wilson 16b64392e2 issue #106: support WANT_JSON modules. 7 years ago
David Wilson df6daaf3c4 issue #106: working/semantically compatible binary support. 7 years ago
David Wilson 23366b4580 issue #106: import binary modules. 7 years ago
David Wilson 0dd5e04eae issue #106: partially working BinaryRunner/Planner.
Refactor planner.py to look a lot more like runner.py. This 'structural
cutpaste' looks messy -- probably we can simplify this code, even though
it's pretty simple already.
7 years ago
David Wilson 1a040cf5c0 issue #106: get FileService working. 7 years ago
David Wilson 6aac37e157 issue #106: allow any context to contact FileService.
Also fix privilege check for register command.
7 years ago
David Wilson 8fffb34752 issue #106: helpers.get_file(), command logging.
* Add helpers.get_file() that calls back up into FileService as
  necessary. This is a stopgap measure.

* Add logging to exec_args() to simplify debugging of binary runners.
7 years ago
David Wilson dbaca05ac8 issue #106: Runner module docstring 7 years ago
David Wilson f6d436783c issue #106: add Service.__repr__, reply to bad calls
* Don't hang callers that fail validate_args(), instead tell them their
  message was rejected.
* Add Service.repr for nicer logging.
7 years ago
David Wilson 8f175bf7a8 issue #106: _unpickle_context() did not allow nameless contexts.
These are generated by any child calling .context_by_id() without
previously knowing the context's name, such as Contexts set in
ExternalContext.master and ExternalContext.parent.
7 years ago
David Wilson 17bfb596d0 issue #106: mitogen.service missing from modules list. 7 years ago
David Wilson 504032e6e8 issue #106: has_parent_authority should accept own context ID.
When a stream (such as unix.connect()) has its auth_id set to the
current context's, we should allow those requests too, since the request
is working with the privilege of the current context.
7 years ago
David Wilson 43e4f5009a issue #106: remove 2 needless Invocation attributes. 7 years ago
David Wilson 2470f486e1 issue 106: ansible: make the context name available
For use later to track/deduplicate streaming uploads to targets.
7 years ago
David Wilson de27fb3a28 issue #174: test all io_op() logic. 7 years ago
David Wilson c891ab078a issue #106: working old-style native module execution
Still abusing Python import mechanism, but one big step closer to
eliminating that.
7 years ago
David Wilson 34a37a0ba5 issue #106: ansible: rename and significant pad out runners.py
Aiming to have working NativeRunner and BinaryRunner to begin with.
7 years ago
David Wilson 98c15942f7 issue #177: fix bizarre syntax error in last commit. 7 years ago
David Wilson 17b94c56f4 issue #177: import reproduction. 7 years ago
David Wilson cf25437019 issue #177: populate Shell.tempdir global on creating a tempdir.
It looks a lot like multiple calls to _make_tmp_path() will result in
multiple temporary directories on the remote machine, only the last of
which will be cleaned up.

We must be bug-for-bug compatible for now, so ignore the problem in the
meantime.
7 years ago
David Wilson 6eed3aa1fa issue #177: fetch and cache HOME value during connection setup.
This ensures only 1 roundtrip is required for every invocation of
_remote_expand_user().
7 years ago
dw 772e229b7d
Merge pull request #180 from moreati/procedure-033-03
Switch to using unit2 test runner
7 years ago
Alex Willmer 0896e95e2c Set strict mode in test script
Exits with an error if a command is not found, any undefined variable is
used, or a command in a pipeline returns an error.

Should make Travis detect failed tests.
7 years ago
Alex Willmer f92d88c241 travis: Cache wheels and other pip artifacts
Should speedup builds
7 years ago
Alex Willmer 94a082177d tests: Add coverage as a dev requirement 7 years ago
Alex Willmer b5848e7116 Ignore compiled Python files 7 years ago
Alex Willmer 5e66f6c4fa Ignore hidden directory containing tox environments 7 years ago
Alex Willmer dc60f05a40 tests: Switch to unit2 test runner, with coverage
This means test files are imported as modules, not run as scripts. THey
can still be run individually if so desired. Test coverage is measured,
and an html report generated in htmlcov/. Test cases are automativally
discovered, so they need not be listed twice. An overall
passed/failed/skipped summary is printed, rather than for each file.

Arguments passed to ./test are passed on to unit2. For instance

    ./test -v

will print each test name as it is run.
7 years ago
Alex Willmer 7b8fef5284 tests: Make the tests directory an importable package
Required for test discovery by e.g. unit2, pytest
7 years ago
Alex Willmer a22294dda9 call_function_test: Fix assumption that we run as a script 7 years ago
Alex Willmer 0dcaeb21a2 master_test: Don't assume __file__ points to source code
When run under a test runner the unit tests are imported as modules.
This triggers .pyc generation, after which __file__ resolves to the .pyc
file.
7 years ago
Alex Willmer 841c2b13a1 fakessh_test: Apply timeout decorators to rsync tests
timeoutcontext.timeout uses SIGALRM, hence it will only work on Unix
like operating systems.
7 years ago
Alex Willmer 48623763d6 minimize_source: Implement reindentation 7 years ago
Alex Willmer a6e7b26c25 tests: Correct path to module_finder_test 7 years ago
Alex Willmer 556ee2aec6 minimize_source: Handling indentation
In the case that string(s) are replaced, any indent or dedent tags must
be re-inserted after the replacemnts.
7 years ago
Alex Willmer 36c5ff1d05 minimize_source: Make comments test invariant on hashbang/coding handling
strip_comments() currently ignores comments on lines 1 and 2, in order
to preserve lines such as

The comments test had normal comments on those lines, hence it was
failing.
7 years ago
Alex Willmer 3bf61bb311 tests: Clarify which minimize_source samples are the reference 7 years ago
Alex Willmer a1e9b9e8db Issue #160: Reimplement minimize_source as token filters
Benefits:

- More correct than re.sub()
- Better handling of trailing whitespace
- Recognises doc-strings regardless of quoting style

Limitations:

- Still not entirely correct
  - Creates a syntax error when function/class body is only a docstring
  - Doesn't handle indented docstrings yet
- Slower by 50x - 8-10 ms vs 0.2 ms for re.sub()
  - Not much scope for improving this, tokenize is 100% pure Python
- Complex state machine, harder to understand
- Higher line count in parent.py
- Untested with Mitogen parent on Python 2.x and child on Python 2.x+y

No change

- Only requires Python stdlib modules
7 years ago
Alex Willmer 35ae4e4227 Issue #160: Add minimize_source testcases 7 years ago
David Wilson 3dc90b7618 issue #106: import skeletal planner module. 7 years ago
David Wilson 28cd17cf56 issue #106: import skeletal new executor. 7 years ago
Alex Willmer f1e6fe007b preamble_size: Print compressed size and percentage 7 years ago
David Wilson 9067a7b173 ansible: Move setLevel() bits together. 7 years ago
David Wilson 6958b8ff09 docs: More getting started. 7 years ago
David Wilson 76ac49dbdc examples: more comments. 7 years ago
David Wilson 9eccfb4972 examples: add top-level doc 7 years ago
David Wilson a68e833463 examples: add a ton of comments to mitop.py. 7 years ago
dw f440e88f2f
Merge pull request #175 from wezm/become-pass
Use become_pass for sudo password
7 years ago