9a2e600317
chore: Convert bug template to a form
2 weeks ago
2cc507a6de
chore: Remove Mitogen 0.2 issue template
...
No longer used
2 weeks ago
db63dd1def
Merge pull request #1359 from moreati/issue1260
...
CI: Remove integration of retired lgtm.com
3 weeks ago
cab024a6fc
CI: Remove integration of retired lgtm.com
...
Company was aquired by Github in 2019. Service was switched off in Dec 2022,
replaced by GitHub code scanning. Fixes #1260
See
- https://github.blog/news-insights/product-news/the-next-step-for-lgtm-com-github-code-scanning/
3 weeks ago
aea028f175
Merge pull request #1357 from moreati/issue1218
...
ansible_mitogen: Remove maximum Ansible version check
3 weeks ago
df890459c5
ansible_mitogen: Remove maximum Ansible version check
...
fixes #1218
3 weeks ago
847f34c17d
Merge pull request #1243 from moreati/boot-cmd--argv
...
mitogen: Pass first stage, preamble length, and context name in argv
3 weeks ago
83c5ab1900
mitogen: Send first stage parameters as argv (796 bytes -> 822)
...
Benefit: The base64 lump is now static for a given Mitogen version, and the
variable parts are more visible. This will make debugging, auditting, and
allow-listing a bit easier.
Potential benefit: generate the base64 once, at build time or startup. Rather
than once per connection.
Cost: Bootstrap command is 26 bytes longer.
```
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 822
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 98746 96.4KiB 51215 50.0KiB 51.9% 12922 12.6KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
4 weeks ago
3b7a75dfaf
mitogen: Send first stage as argv (786 bytes -> 796 bytes)
...
This saves one layer of quoting/quote escaping in the bootstrap command and a
string interpolation per connection. The cost is an increasing the bootstrap
command by 10 bytes. I like the tradeoff. I could be convinced to revert it.
```console
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 796
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99181 96.9KiB 51384 50.2KiB 51.8% 12956 12.7KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
4 weeks ago
191abd492a
mitogen: Compress first stage without header or checksum (790 bytes -> 786)
...
```console
➜ mitogen git:(boot-cmd--argv) ✗ ./preamble_size.py
SSH command size: 786
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99166 96.8KiB 51375 50.2KiB 51.8% 12957 12.7KiB 13.1%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
Confirmed Python 2.4 supports this use of zlib.compressobj, despite lack of
mention in https://docs.python.org/2.4/lib/module-zlib.html
```pycon
Python 2.4.6 (#2 , Apr 29 2018, 11:16:24)
[GCC 7.3.0] on linux4
Type "help", "copyright", "credits" or "license" for more information.
>>> import zlib
>>> c=zlib.compressobj(zlib.Z_BEST_COMPRESSION,zlib.DEFLATED,-zlib.MAX_WBITS)
>>> c.compress('qwertyuiop') + c.flush()
'+,O-*\xa9,\xcd\xcc/\x00\x00'
```
4 weeks ago
408946adbe
mitogen: Golf 8 bytes from bootstrap first stage (798 -> 790)
...
Before
```
SSH command size: 798
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99020 96.7KiB 51247 50.0KiB 51.8% 12910 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
After
```
SSH command size: 790
Preamble (mitogen.core + econtext) size: 18230 (17.80KiB)
Original Minimized Compressed
mitogen.core 152237 148.7KiB 68453 66.8KiB 45.0% 18130 17.7KiB 11.9%
mitogen.parent 99020 96.7KiB 51247 50.0KiB 51.8% 12903 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10847 10.6KiB 6913 6.8KiB 63.7% 2102 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15753 15.4KiB 8135 7.9KiB 51.6% 2672 2.6KiB 17.0%
mitogen.master 52891 51.7KiB 27586 26.9KiB 52.2% 7129 7.0KiB 13.5%
```
4 weeks ago
fdb5c62532
Merge pull request #1353 from moreati/prepare-v0.3.31
...
Prepare v0.3.31
4 weeks ago
e4e82f53a1
Begin 0.3.32dev
4 weeks ago
77b7a31949
Prepare v0.3.31
4 weeks ago
69a5cdce1b
Merge pull request #1352 from moreati/issue1350
...
ansible_mitogen: Fix ModuleNotFoundError: No module named 'ansible_mitogen'
4 weeks ago
85069b28cd
ansible_mitogen: Fix ModuleNotFoundError: No module named 'ansible_mitogen'
...
Loading the ansible_mitogen Ansible plugins apparently doesn't follow the same
rules as importing a Python module. So sys.path manipulations in __init__.py
weren't fired when Ansible tried to load the plugins from a /custom/path that
wasn't already on sys.path.
This wasn't picked up by the test because CI always installs Mitogen as a
Python package (in a virtual env).
This reverses 6145508312
4 weeks ago
2305446ab8
Merge pull request #1346 from moreati/prepare-v0.3.30
...
Prepare v0.3.30
1 month ago
c72acfd966
Begin v0.3.31.dev
1 month ago
1e90ff25ee
Prepare v0.3.30
1 month ago
48243724a0
Merge pull request #1341 from mhartmay/logforwarder-fix
...
master: Fix LogForwarder in case an own LogRecordFactory is used
1 month ago
24745183ed
master: Fix LogForwarder in case an own LogRecordFactory is used
...
Since Python 3.2 the log record factory can be changed by using
`logging.setLogRecordFactory` [1]. Therefore use `logging.makeLogRecord` as
recommended in the documentation:
"LogRecord instances are created automatically by the Logger every time
something is logged, and can be created manually via makeLogRecord() (for
example, from a pickled event received over the wire)." [2]
This fixes the test case
`log_handler_test.LogRecordFactoryTest.test_logrecordfactory`.
[1] https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory
[2] https://docs.python.org/3/library/logging.html#logrecord-objects
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
dad28e8b4a
tests: Add a test case that verifies behavior when the log record factory is modified
...
The test currently fails with the following error:
$ PYTHONPATH=$(pwd)/tests:$PYTHONPATH python3 -m unittest -v log_handler_test
...
test_logrecordfactory (log_handler_test.LogRecordFactoryTest.test_logrecordfactory) ... --- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.12/logging/__init__.py", line 464, in format
return self._format(record)
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 460, in _format
return self._fmt % values
~~~~~~~~~~^~~~~~~~
KeyError: 'custom_attribute'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.12/logging/__init__.py", line 1160, in emit
msg = self.format(record)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
return fmt.format(record)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
return fmt.format(record)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 706, in format
s = self.formatMessage(record)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 675, in formatMessage
return self._style.format(record)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 466, in format
raise ValueError('Formatting field not found in record: %s' % e)
ValueError: Formatting field not found in record: 'custom_attribute'
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
3c648f7df8
Merge pull request #1345 from moreati/oi-mate-wheres-your-loicense
...
Fix and formalise license metadata
1 month ago
01baec8347
Declare license as SPDX identifier in metadata
...
Fixes warning seen during packaging operations
```
➜ mitogen git:(master) ✗ uv build --sdist
Building source distribution...
...
!!
********************************************************************************
Please consider removing the following classifiers in favor of a SPDX license expression:
License :: OSI Approved :: BSD License
See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************
!!
self._finalize_license_expression()
running egg_info
...
```
1 month ago
85f0c33dc5
Correct mitogen.imports.* licenses
1 month ago
8f66aa5fcd
Merge pull request #1248 from moreati/sys.path
...
ansible_mitogen: De-duplicate sys.path manipulations by Ansible plugins
1 month ago
6145508312
ansible_mitogen: De-duplicate sys.path manipulations by Ansible plugins
1 month ago
9701424be0
Merge pull request #1266 from moreati/imports-cleanup
...
Imports cleanup
1 month ago
7d5f63ccbf
Cleanup unused and missing imports
1 month ago
3338a651a6
Merge pull request #1339 from moreati/prepare-v0.3.29
...
Prepare v0.3.29
3 months ago
6071fb58c9
Begin 0.3.30dev
3 months ago
e670bf0ebd
Prepare v0.3.29
3 months ago
f6451bf795
Merge pull request #1287 from moreati/issue1242-py3.14
...
Python 3.14 support
3 months ago
090952a987
Python 3.14 support
3 months ago
d27275ad46
ci: Set global max failed logins on macOS
3 months ago
1b00ca2581
tests: Bump dependency versions
3 months ago
f4f646a00a
Merge pull request #1337 from moreati/prepare-v0.3.28
...
Prepare v0.3.28
3 months ago
b03c1f3d87
Begin 0.3.29dev
3 months ago
9f9b37d1ad
Prepare v0.3.28
3 months ago
f6902dd05d
Merge pull request #1336 from Nihlus/freeipa-fixes
...
Add FreeIPA client modules to the always-fork list
3 months ago
2736f38c4b
docs: Changelog for FreeIPA client modules -> ALWAYS_FORK_MODULES
3 months ago
59d5d74abd
Add FreeIPA client modules to the always-fork list.
3 months ago
36569792bc
Merge pull request #1307 from moreati/issue1306-investigate
...
mitogen: Fix non-blocking IO errors in first stage of bootstrap
3 months ago
85d6046f2f
mitogen: Fix non-blocking IO errors in first stage of bootstrap
...
When /etc/sudoers has log_output (or similar) enabled the process spawned by
`ctx.sudo()` via `mitogen.parent.Connection.start_child()` receives a stdin
that is in non-blocking mode. The immediate symptom is that `os.openfd(0,
...).read(n)` sometimes returns `None`, causing the first stage to raise an
unhandled TypeError.
The fix (for now) is to use `select.select()` in a while loop to read stdin.
This increases the command size slightly, but I think it's a reasonable
tradeoff until/unless the cause is more fully understood.
All CI tests are now run with sudoers log_output enabled, in order to catch
regressions. `first_stage_test.CommandLineTest` has been amended, because it
relied on implementation details of the bootstrap process that are no longer
true.
Before
```
SSH command size: 755
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.core 152218 148.7KiB 68437 66.8KiB 45.0% 18124 17.7KiB 11.9%
mitogen.parent 98853 96.5KiB 51103 49.9KiB 51.7% 12881 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
```
After
```
SSH command size: 798
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.core 152218 148.7KiB 68437 66.8KiB 45.0% 18124 17.7KiB 11.9%
mitogen.parent 98944 96.6KiB 51180 50.0KiB 51.7% 12910 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
```
3 months ago
c508bfb58b
tests: Check stdio is blocking in sudo contexts
...
refs #712
4 months ago
76f6eb741d
tests: Count bytes written in stdio_test.StdIOTest
...
This is mainly for peace of mind. With all this non-blocking IO investigation
I'm getting a bit paranoid wrt file objects.
refs #712
4 months ago
3dfaf83ce7
preamble_size: Fix variability of command & preamble(?) size
...
Previously the command size could very depanding on the current username, hostname, and process pid.
Before
```
SSH command size: 759
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
...
```
After
SSH command size: 755
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
...
```
4 months ago
936b08dd08
preamble_size: Include mitogen.core and clarify bootstrap size
...
After:
SSH command size: 759
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.core 152218 148.7KiB 68437 66.8KiB 45.0% 18124 17.7KiB 11.9%
mitogen.parent 98853 96.5KiB 51103 49.9KiB 51.7% 12881 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
4 months ago
30d8a38a3b
preamble_size: Consolidate table formatting, align columns better
...
Before
./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.parent 98853 96.5KiB 51103 49.9KiB 51.7% 12881 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
After:
SSH command size: 759
Bootstrap (mitogen.core) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.parent 98853 96.5KiB 51103 49.9KiB 51.7% 12881 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
4 months ago
e4e2c6caaf
CI: Move sudo test users defaults into /etc/sudoers.d
...
Prep for reusing it in non-Ansible tests
4 months ago
Alex Willmer