af0eab071e
Prepare v0.3.38
1 day ago
9d70e83221
mitogen: Consolidate `range` & `xrange` polyfills -> `mitogen.core.range`
2 days ago
2403d4570b
mitogen: Format Message src & dst of as <context>:<handle>
...
The notation is inspired by host:port notation in a URL. In Mitogen context id
and handles fill roughly the same role as IP and port in TCP or UDP.
1 week ago
ba6e612ec4
mitogen: Fix docstring typo
1 week ago
4d372cb160
mitogen: Last few `except ImportError` -> `if sys.version_info ...`
2 weeks ago
0447950acd
mitogen: Replace uses mitogen.core.{PY24,PY3} with sys.version_info
...
Helps static analysis by type checkers, LSPs, etc.
2 weeks ago
52ec693ef7
mitogen: Provide mitogen.core exception fallbacks based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
08a3f271f3
mitogen: Provide mitogen.core.{all,any} based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
9bede962b3
mitogen: Provide mitogen.core.threading* based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
55bd5fd7a4
mitogen: Provide mitogen.core.next based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
d1ee8c788f
mitogen: Configure mitogen.core.str_partition etc. based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
67264ed174
mitogen: Configure mitogen.core.now based on sys.version_info
...
Simplify work of static type checkers, LSPs, etc.
2 weeks ago
94b2f5d8d6
mitogen: Guard importlib imports with sys.version_info
...
Reduce number of doomed mitogen.core.Importer GET_MODULE requests from e.g.
Python 2.x contexts. Simplify work of static type checkers, LSPs, etc.
2 weeks ago
cd58b7eac1
mitogen: Avoid import of linecache on Python >= 2.5
...
It's only needed for a workaround on Python 2.4.
2 weeks ago
384d37f630
mitogen: Use built-in _codecs, eliminate direct import of encodings package
...
The package will still get imported indirectly by _something_, but every
little helps.
2 weeks ago
5011263bf0
mitogen: Use built-in _codecs module to encode latin1
...
Streamlines initilialzation slightly. Next commit will do the same for utf-8.
2 weeks ago
1c5a1a3d72
Begin 0.3.38dev
2 weeks ago
abe3671c7b
Prepare v0.3.37
2 weeks ago
0e2da7deb5
mitogen: Fix AttributeError in mitogen.profiler
2 weeks ago
b7eddf2cdb
mitogen: Add initial support for importlib ResourceReader
...
The new classes are modelled closely on their existing Module* counterparts.
For now I've duplicated the code, once it's bedded in I may refactor it. I
didn't replicate the FORWARD_MODULE plumbing, it didn't seem to be necessary
and may be dead code.
3 weeks ago
1d62a51810
Begin 0.3.37dev
1 month ago
4111224161
Prepare v0.3.36
1 month ago
8807cd53be
mitogen: first_stage: Break the while loop in case of EOF
...
The current implementation can cause an infinite loop, leading to a process that
hangs and consumes 100% CPU. This occurs because the EOF condition is not
handled properly, resulting in repeated select(...) and read(...) calls.
The fix is to properly handle the EOF condition and break out of the loop when
it occurs.
-SSH command size: 822
+SSH command size: 838
Preamble (mitogen.core + econtext) size: 18226 (17.80KiB)
-mitogen.parent 99062 96.7KiB 51235 50.0KiB 51.7% 12936 12.6KiB 13.1%
+mitogen.parent 99240 96.9KiB 51244 50.0KiB 51.6% 12956 12.7KiB 13.1%
Fixes: https://github.com/mitogen-hq/mitogen/issues/1348
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
fdaf09c4d6
mitogen/parent: Fix typo
...
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
606a21fb27
mitogen: Add first line comments to _first_stage()
2 months ago
733f4bca81
mitogen: Allow line comments in first stage, strip them.
2 months ago
ccaaf4b7fe
mitogen: Clarify blacklisted ModuleNotFoundError message
...
Previous phrasing was misleading - it implied a given module was explicitly on
the blacklist, even if it was due to a restrictive whitelist and the blacklist
was empty.
Arguably the blacklist/whitelist semantics are themselves misleading. A
redesign is tempting.
2 months ago
9ce6a43329
Begin 0.3.36dev
2 months ago
4af6a75278
Prepare v0.3.35
2 months ago
19938ec05a
Begin 0.3.35.dev
2 months ago
8e6a93dd0f
Prepare v0.3.34
2 months ago
83b6cdb616
ansible_mitogen: Speedup startup by not sending `__main__`
...
On my laptop his reduces the time to execute `ansible -mping ...` by approx
300 ms with `strategy=mitogen_linear`.
Until this commit Mitogen was unnecessarily sending large chunks of Ansible
from the controller to targets, due to `__main__` being identified as a
related module of `ansible.module_utils.basic`, and resolving to something
within `ansible.cli...`.
On Ansible target hosts executing any Ansible Module `__main__` is imported by
`ansible.module_utils.basic` as part of Ansible's module delivery mechanism.
When `mitogen.master.ModuleResponder` (on the controller) processes the
request for `ansible.module_utils.basic` from the target, it scans
`ansible.module_utils.basic` for related imports and finds `__main__`. However
`__main__` on the controller is not the same module as `__main__` on the
target. On the controller it is a module in `ansible.cli...` that implements
one of the ansible commands (e.g. `ansible`, `ansible-playbook`).
2 months ago
f191f050bf
mitogen: Log why a module is sent or not sent by ModuleResponder
...
This should not change the logic
2 months ago
c9eb6e54e2
Begin 0.3.34dev
2 months ago
4a442f503e
Prepare v0.3.33
2 months ago
61a7fa1fee
Begin 0.3.33dev
2 months ago
28ea4780db
Prepare v0.3.32
2 months ago
83c5ab1900
mitogen: Send first stage parameters as argv (796 bytes -> 822)
...
Benefit: The base64 lump is now static for a given Mitogen version, and the
variable parts are more visible. This will make debugging, auditting, and
allow-listing a bit easier.
Potential benefit: generate the base64 once, at build time or startup. Rather
than once per connection.
Cost: Bootstrap command is 26 bytes longer.
```
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 822
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 98746 96.4KiB 51215 50.0KiB 51.9% 12922 12.6KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
3 months ago
3b7a75dfaf
mitogen: Send first stage as argv (786 bytes -> 796 bytes)
...
This saves one layer of quoting/quote escaping in the bootstrap command and a
string interpolation per connection. The cost is an increasing the bootstrap
command by 10 bytes. I like the tradeoff. I could be convinced to revert it.
```console
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 796
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99181 96.9KiB 51384 50.2KiB 51.8% 12956 12.7KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
3 months ago
191abd492a
mitogen: Compress first stage without header or checksum (790 bytes -> 786)
...
```console
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 786
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99166 96.8KiB 51375 50.2KiB 51.8% 12957 12.7KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
Confirmed Python 2.4 supports this use of zlib.compressobj, despite lack of
mention in https://docs.python.org/2.4/lib/module-zlib.html
```pycon
Python 2.4.6 (#2 , Apr 29 2018, 11:16:24)
[GCC 7.3.0] on linux4
Type "help", "copyright", "credits" or "license" for more information.
>>> import zlib
>>> c=zlib.compressobj(zlib.Z_BEST_COMPRESSION,zlib.DEFLATED,-zlib.MAX_WBITS)
>>> c.compress('qwertyuiop') + c.flush()
'+,O-*\xa9,\xcd\xcc/\x00\x00'
```
3 months ago
408946adbe
mitogen: Golf 8 bytes from bootstrap first stage (798 -> 790)
...
Before
```
SSH command size: 798
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99020 96.7KiB 51247 50.0KiB 51.8% 12910 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
After
```
SSH command size: 790
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99020 96.7KiB 51247 50.0KiB 51.8% 12903 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
3 months ago
e4e82f53a1
Begin 0.3.32dev
3 months ago
77b7a31949
Prepare v0.3.31
3 months ago
c72acfd966
Begin v0.3.31.dev
3 months ago
1e90ff25ee
Prepare v0.3.30
3 months ago
24745183ed
master: Fix LogForwarder in case an own LogRecordFactory is used
...
Since Python 3.2 the log record factory can be changed by using
`logging.setLogRecordFactory` [1]. Therefore use `logging.makeLogRecord` as
recommended in the documentation:
"LogRecord instances are created automatically by the Logger every time
something is logged, and can be created manually via makeLogRecord() (for
example, from a pickled event received over the wire)." [2]
This fixes the test case
`log_handler_test.LogRecordFactoryTest.test_logrecordfactory`.
[1] https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory
[2] https://docs.python.org/3/library/logging.html#logrecord-objects
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
3 months ago
85f0c33dc5
Correct mitogen.imports.* licenses
3 months ago
7d5f63ccbf
Cleanup unused and missing imports
3 months ago
6071fb58c9
Begin 0.3.30dev
4 months ago
e670bf0ebd
Prepare v0.3.29
4 months ago
Alex Willmer