Commit Graph

3980 Commits (b411da83120383db95d62a07fa9c86e868d44d19)
 

Author SHA1 Message Date
Alex Willmer b411da8312
Merge pull request #1354 from moreati/release-v0.3.31
Release v0.3.31
4 weeks ago
Alex Willmer e572922d9d Merge commit '77b7a31' into release-v0.3.31 4 weeks ago
Alex Willmer 77b7a31949 Prepare v0.3.31 4 weeks ago
Alex Willmer 69a5cdce1b
Merge pull request #1352 from moreati/issue1350
ansible_mitogen: Fix ModuleNotFoundError: No module named 'ansible_mitogen'
4 weeks ago
Alex Willmer 85069b28cd ansible_mitogen: Fix ModuleNotFoundError: No module named 'ansible_mitogen'
Loading the ansible_mitogen Ansible plugins apparently doesn't follow the same
rules as importing a Python module. So sys.path manipulations in __init__.py
weren't fired when Ansible tried to load the plugins from a /custom/path that
wasn't already on sys.path.

This wasn't picked up by the test because CI always installs Mitogen as a
Python package (in a virtual env).

This reverses 6145508312.
4 weeks ago
Alex Willmer 81e11d5456
Merge pull request #1347 from moreati/release-v0.3.30
Release v0.3.30
1 month ago
Alex Willmer 2305446ab8
Merge pull request #1346 from moreati/prepare-v0.3.30
Prepare v0.3.30
1 month ago
Alex Willmer 263884070d Merge commit '1e90ff2' into release-v0.3.30 1 month ago
Alex Willmer c72acfd966 Begin v0.3.31.dev 1 month ago
Alex Willmer 1e90ff25ee Prepare v0.3.30 1 month ago
Alex Willmer 48243724a0
Merge pull request #1341 from mhartmay/logforwarder-fix
master: Fix LogForwarder in case an own LogRecordFactory is used
1 month ago
Marc Hartmayer 24745183ed master: Fix LogForwarder in case an own LogRecordFactory is used
Since Python 3.2 the log record factory can be changed by using
`logging.setLogRecordFactory` [1]. Therefore use `logging.makeLogRecord` as
recommended in the documentation:

"LogRecord instances are created automatically by the Logger every time
something is logged, and can be created manually via makeLogRecord() (for
example, from a pickled event received over the wire)." [2]

This fixes the test case
`log_handler_test.LogRecordFactoryTest.test_logrecordfactory`.

[1] https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory
[2] https://docs.python.org/3/library/logging.html#logrecord-objects

Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
Marc Hartmayer dad28e8b4a tests: Add a test case that verifies behavior when the log record factory is modified
The test currently fails with the following error:

  $ PYTHONPATH=$(pwd)/tests:$PYTHONPATH python3 -m unittest -v log_handler_test
  ...
  test_logrecordfactory (log_handler_test.LogRecordFactoryTest.test_logrecordfactory) ... --- Logging error ---
  Traceback (most recent call last):
    File "/usr/lib/python3.12/logging/__init__.py", line 464, in format
      return self._format(record)
             ^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 460, in _format
      return self._fmt % values
             ~~~~~~~~~~^~~~~~~~
  KeyError: 'custom_attribute'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/usr/lib/python3.12/logging/__init__.py", line 1160, in emit
      msg = self.format(record)
            ^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
      return fmt.format(record)
             ^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
      return fmt.format(record)
             ^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 706, in format
      s = self.formatMessage(record)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 675, in formatMessage
      return self._style.format(record)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/logging/__init__.py", line 466, in format
      raise ValueError('Formatting field not found in record: %s' % e)
  ValueError: Formatting field not found in record: 'custom_attribute'

Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
1 month ago
Alex Willmer 3c648f7df8
Merge pull request #1345 from moreati/oi-mate-wheres-your-loicense
Fix and formalise license metadata
1 month ago
Alex Willmer 01baec8347 Declare license as SPDX identifier in metadata
Fixes warning seen during packaging operations

```
➜  mitogen git:(master) ✗ uv build --sdist
Building source distribution...
...
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: BSD License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
running egg_info
...
```
1 month ago
Alex Willmer 85f0c33dc5 Correct mitogen.imports.* licenses 1 month ago
Alex Willmer 8f66aa5fcd
Merge pull request #1248 from moreati/sys.path
ansible_mitogen: De-duplicate sys.path manipulations by Ansible plugins
1 month ago
Alex Willmer 6145508312 ansible_mitogen: De-duplicate sys.path manipulations by Ansible plugins 1 month ago
Alex Willmer 9701424be0
Merge pull request #1266 from moreati/imports-cleanup
Imports cleanup
1 month ago
Alex Willmer 7d5f63ccbf Cleanup unused and missing imports 1 month ago
Alex Willmer c01f430957
Merge pull request #1340 from moreati/release-v0.3.29
Release v0.3.29
3 months ago
Alex Willmer 3338a651a6
Merge pull request #1339 from moreati/prepare-v0.3.29
Prepare v0.3.29
3 months ago
Alex Willmer c0dd5c6966 Merge commit 'e670bf0' into stable 3 months ago
Alex Willmer 6071fb58c9 Begin 0.3.30dev 3 months ago
Alex Willmer e670bf0ebd Prepare v0.3.29 3 months ago
Alex Willmer f6451bf795
Merge pull request #1287 from moreati/issue1242-py3.14
Python 3.14 support
3 months ago
Alex Willmer 090952a987 Python 3.14 support 3 months ago
Alex Willmer d27275ad46 ci: Set global max failed logins on macOS 3 months ago
Alex Willmer 1b00ca2581 tests: Bump dependency versions 3 months ago
Alex Willmer ce0c596700
Merge pull request #1338 from moreati/release-v0.3.28
Release v0.3.28
3 months ago
Alex Willmer f4f646a00a
Merge pull request #1337 from moreati/prepare-v0.3.28
Prepare v0.3.28
3 months ago
Alex Willmer 2791abe17a Merge commit '9f9b37d' into release-v0.3.28 3 months ago
Alex Willmer b03c1f3d87 Begin 0.3.29dev 3 months ago
Alex Willmer 9f9b37d1ad Prepare v0.3.28 3 months ago
Alex Willmer f6902dd05d
Merge pull request #1336 from Nihlus/freeipa-fixes
Add FreeIPA client modules to the always-fork list
3 months ago
Alex Willmer 2736f38c4b docs: Changelog for FreeIPA client modules -> ALWAYS_FORK_MODULES 3 months ago
Jarl Gullberg 59d5d74abd
Add FreeIPA client modules to the always-fork list. 3 months ago
Alex Willmer 36569792bc
Merge pull request #1307 from moreati/issue1306-investigate
mitogen: Fix non-blocking IO errors in first stage of bootstrap
3 months ago
Alex Willmer 85d6046f2f mitogen: Fix non-blocking IO errors in first stage of bootstrap
When /etc/sudoers has log_output (or similar) enabled the process spawned by
`ctx.sudo()` via `mitogen.parent.Connection.start_child()` receives a stdin
that is in non-blocking mode. The immediate symptom is that `os.openfd(0,
...).read(n)` sometimes returns `None`, causing the first stage to raise an
unhandled TypeError.

The fix (for now) is to use `select.select()` in a while loop to read stdin.
This increases the command size slightly, but I think it's a reasonable
tradeoff until/unless the cause is more fully understood.

All CI tests are now run with sudoers log_output enabled, in order to catch
regressions. `first_stage_test.CommandLineTest` has been amended, because it
relied on implementation details of the bootstrap process that are no longer
true.

Before
```
SSH command size: 755
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)

                        Original           Minimized           Compressed
mitogen.core         152218 148.7KiB  68437 66.8KiB 45.0%  18124 17.7KiB 11.9%
mitogen.parent        98853  96.5KiB  51103 49.9KiB 51.7%  12881 12.6KiB 13.0%
mitogen.fork           8445   8.2KiB   4139  4.0KiB 49.0%   1652  1.6KiB 19.6%
mitogen.ssh           10827  10.6KiB   6893  6.7KiB 63.7%   2099  2.0KiB 19.4%
mitogen.sudo          12089  11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select        12325  12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB  7.8%
mitogen.service       41581  40.6KiB  22398 21.9KiB 53.9%   5847  5.7KiB 14.1%
mitogen.fakessh       15767  15.4KiB   8149  8.0KiB 51.7%   2676  2.6KiB 17.0%
mitogen.master        55317  54.0KiB  28846 28.2KiB 52.1%   7528  7.4KiB 13.6%
```

After
```
SSH command size: 798
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)

                        Original           Minimized           Compressed
mitogen.core         152218 148.7KiB  68437 66.8KiB 45.0%  18124 17.7KiB 11.9%
mitogen.parent        98944  96.6KiB  51180 50.0KiB 51.7%  12910 12.6KiB 13.0%
mitogen.fork           8445   8.2KiB   4139  4.0KiB 49.0%   1652  1.6KiB 19.6%
mitogen.ssh           10827  10.6KiB   6893  6.7KiB 63.7%   2099  2.0KiB 19.4%
mitogen.sudo          12089  11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select        12325  12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB  7.8%
mitogen.service       41581  40.6KiB  22398 21.9KiB 53.9%   5847  5.7KiB 14.1%
mitogen.fakessh       15767  15.4KiB   8149  8.0KiB 51.7%   2676  2.6KiB 17.0%
mitogen.master        55317  54.0KiB  28846 28.2KiB 52.1%   7528  7.4KiB 13.6%
```
4 months ago
Alex Willmer c508bfb58b tests: Check stdio is blocking in sudo contexts
refs #712
4 months ago
Alex Willmer 76f6eb741d tests: Count bytes written in stdio_test.StdIOTest
This is mainly for peace of mind. With all this non-blocking IO investigation
I'm getting a bit paranoid wrt file objects.

refs #712
4 months ago
Alex Willmer 3dfaf83ce7 preamble_size: Fix variability of command & preamble(?) size
Previously the command size could very depanding on the current username, hostname, and process pid.

Before
```
SSH command size: 759
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
...
```

After
SSH command size: 755
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
...
```
4 months ago
Alex Willmer 936b08dd08 preamble_size: Include mitogen.core and clarify bootstrap size
After:
SSH command size: 759
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)

                        Original           Minimized           Compressed
mitogen.core         152218 148.7KiB  68437 66.8KiB 45.0%  18124 17.7KiB 11.9%
mitogen.parent        98853  96.5KiB  51103 49.9KiB 51.7%  12881 12.6KiB 13.0%
mitogen.fork           8445   8.2KiB   4139  4.0KiB 49.0%   1652  1.6KiB 19.6%
mitogen.ssh           10827  10.6KiB   6893  6.7KiB 63.7%   2099  2.0KiB 19.4%
mitogen.sudo          12089  11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select        12325  12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB  7.8%
mitogen.service       41581  40.6KiB  22398 21.9KiB 53.9%   5847  5.7KiB 14.1%
mitogen.fakessh       15767  15.4KiB   8149  8.0KiB 51.7%   2676  2.6KiB 17.0%
mitogen.master        55317  54.0KiB  28846 28.2KiB 52.1%   7528  7.4KiB 13.6%
4 months ago
Alex Willmer 30d8a38a3b preamble_size: Consolidate table formatting, align columns better
Before
./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 18227 (17.80KiB)

                              Original          Minimized           Compressed
mitogen.parent            98853 96.5KiB  51103 49.9KiB 51.7%  12881 12.6KiB 13.0%
mitogen.fork               8445  8.2KiB   4139  4.0KiB 49.0%   1652  1.6KiB 19.6%
mitogen.ssh               10827 10.6KiB   6893  6.7KiB 63.7%   2099  2.0KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41581 40.6KiB  22398 21.9KiB 53.9%   5847  5.7KiB 14.1%
mitogen.fakessh           15767 15.4KiB   8149  8.0KiB 51.7%   2676  2.6KiB 17.0%
mitogen.master            55317 54.0KiB  28846 28.2KiB 52.1%   7528  7.4KiB 13.6%

After:
SSH command size: 759
Bootstrap (mitogen.core) size: 18227 (17.80KiB)

                        Original           Minimized           Compressed
mitogen.parent        98853  96.5KiB  51103 49.9KiB 51.7%  12881 12.6KiB 13.0%
mitogen.fork           8445   8.2KiB   4139  4.0KiB 49.0%   1652  1.6KiB 19.6%
mitogen.ssh           10827  10.6KiB   6893  6.7KiB 63.7%   2099  2.0KiB 19.4%
mitogen.sudo          12089  11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select        12325  12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB  7.8%
mitogen.service       41581  40.6KiB  22398 21.9KiB 53.9%   5847  5.7KiB 14.1%
mitogen.fakessh       15767  15.4KiB   8149  8.0KiB 51.7%   2676  2.6KiB 17.0%
mitogen.master        55317  54.0KiB  28846 28.2KiB 52.1%   7528  7.4KiB 13.6%
4 months ago
Alex Willmer e4e2c6caaf CI: Move sudo test users defaults into /etc/sudoers.d
Prep for reusing it in non-Ansible tests
4 months ago
Alex Willmer 5abdde1117 CI: Report sudo version on Ansible targets 4 months ago
Alex Willmer dc7fae973b CI: Fix ci_lib and test_lib have_<cmd>() when <cmd> exits abnormally
We were not raising CalledProcessError when exit status != 0.
4 months ago
Alex Willmer ced63f96d9
Merge pull request #1332 from moreati/release-v0.3.27
Release v0.3.27
4 months ago
Alex Willmer 885c6de65e
Merge pull request #1331 from moreati/prep-v0.3.27
Prep v0.3.27
4 months ago
Alex Willmer cbf47967b1 Merge commit '154331e455a3153252e44fe41a9e0b3d08ccf477' into release-v0.3.27 4 months ago