85f0c33dc5
Correct mitogen.imports.* licenses
3 months ago
7d5f63ccbf
Cleanup unused and missing imports
3 months ago
6071fb58c9
Begin 0.3.30dev
4 months ago
e670bf0ebd
Prepare v0.3.29
4 months ago
090952a987
Python 3.14 support
4 months ago
b03c1f3d87
Begin 0.3.29dev
4 months ago
9f9b37d1ad
Prepare v0.3.28
4 months ago
85d6046f2f
mitogen: Fix non-blocking IO errors in first stage of bootstrap
...
When /etc/sudoers has log_output (or similar) enabled the process spawned by
`ctx.sudo()` via `mitogen.parent.Connection.start_child()` receives a stdin
that is in non-blocking mode. The immediate symptom is that `os.openfd(0,
...).read(n)` sometimes returns `None`, causing the first stage to raise an
unhandled TypeError.
The fix (for now) is to use `select.select()` in a while loop to read stdin.
This increases the command size slightly, but I think it's a reasonable
tradeoff until/unless the cause is more fully understood.
All CI tests are now run with sudoers log_output enabled, in order to catch
regressions. `first_stage_test.CommandLineTest` has been amended, because it
relied on implementation details of the bootstrap process that are no longer
true.
Before
```
SSH command size: 755
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.core 152218 148.7KiB 68437 66.8KiB 45.0% 18124 17.7KiB 11.9%
mitogen.parent 98853 96.5KiB 51103 49.9KiB 51.7% 12881 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
```
After
```
SSH command size: 798
Preamble (mitogen.core + econtext) size: 18227 (17.80KiB)
Original Minimized Compressed
mitogen.core 152218 148.7KiB 68437 66.8KiB 45.0% 18124 17.7KiB 11.9%
mitogen.parent 98944 96.6KiB 51180 50.0KiB 51.7% 12910 12.6KiB 13.0%
mitogen.fork 8445 8.2KiB 4139 4.0KiB 49.0% 1652 1.6KiB 19.6%
mitogen.ssh 10827 10.6KiB 6893 6.7KiB 63.7% 2099 2.0KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41581 40.6KiB 22398 21.9KiB 53.9% 5847 5.7KiB 14.1%
mitogen.fakessh 15767 15.4KiB 8149 8.0KiB 51.7% 2676 2.6KiB 17.0%
mitogen.master 55317 54.0KiB 28846 28.2KiB 52.1% 7528 7.4KiB 13.6%
```
5 months ago
07d1078010
Begin v0.3.28dev
5 months ago
154331e455
Prepare v0.3.27
5 months ago
0e5f47f145
mitogen: Refactor scan_code_imports() as mitogen.imports.codeobj_imports()
...
This replaces `mitogen.master.scan_code_imports()` with
`mitogen.imports.codeobj_imports()`. The Python 3.x implementation now uses
`str.find()`, relying on Python >= 3.6 "widecode" format. Behaviour and
semantics should be unchanged. Now implementations are approx
- 1.5 x faster on Python 2.x
- 2 - 3 x faster on Python 3.x
Before
```console
$ ./tests/bench/scan_code
scan_code_imports python2.7 100 loops, best of 3: 3.19 msec per loop
scan_code_imports python3.9 500 loops, best of 5: 685 usec per loop
scan_code_imports python3.10 500 loops, best of 5: 727 usec per loop
scan_code_imports python3.11 500 loops, best of 5: 601 usec per loop
scan_code_imports python3.12 500 loops, best of 5: 609 usec per loop
scan_code_imports python3.13 500 loops, best of 5: 586 usec per loop
```
After
```console
codeobj_imports python2.7 1000 loops, best of 3: 1.98 msec per loop
codeobj_imports python3.9 1000 loops, best of 5: 302 usec per loop
codeobj_imports python3.10 1000 loops, best of 5: 297 usec per loop
codeobj_imports python3.11 1000 loops, best of 5: 243 usec per loop
codeobj_imports python3.12 1000 loops, best of 5: 278 usec per loop
codeobj_imports python3.13 1000 loops, best of 5: 259 usec per loop
```
```console
$ uname -a
Darwin kintha 24.6.0 Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:29 PDT
2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6000 arm64
```
5 months ago
7fb7567809
Begin v0.3.27dev
6 months ago
5908936f8c
Prepare v0.3.26
6 months ago
17bee70dc2
mitogen: Fix BlockingIOError & EAGAIN in subprocess stdio
...
Mitogen was leaving the stdout and stderr of subprocesses in non-blocking
mode. When Python code ran in the remote process created by Mitogen calls such
as `print(long_string)` or `os.stout.write(bigger_than_the_buffer)` sometimes
raised `BlockingIOError`, or similar.
This change
- Removes code in `mitogen.core.Side` that set blocking/non-blocking mode
- Adds blocking/non-blocking control to `os.mitogen.pipe()` and a new
function `mitogen.core.socketpair()`
- Replaces `mitogen.core.set_block` and `mitogen.core.set_nonblock`
with `mitogen.core.set_blocking`, mirroring `os.set_blocking`
- Updates call sites as appropriate
- Adds tests for new functions and arguments
- Adds a regression test for subprocess stdio blocking/non-blocking
fixes #712
6 months ago
fde2dda87e
mitogen: Simplify `mitogen.parent.Reaper._calc_delay()` calculation
6 months ago
7f84874755
Begin 0.3.26dev
6 months ago
dfafa1430e
Prepare v0.3.25
6 months ago
65a81121c5
Begin 0.3.25dev
8 months ago
229fd67e97
Prepare v0.3.24
8 months ago
566a4c1e3c
mitogen: log child file objects rather than descriptors
...
The repr() of file objects is more self descriptive, and includes the fd.
8 months ago
6cec613daa
mitogen: Only close stdio file descriptors that were open at process startup
...
File descriptors 0, 1, and 2 are usually stdin, stdout, stderr; but not
always. If a process is started without one of these then the first descriptor
allocated by the process opening a file or socket will be allocated an fd <=
STDERR_FILENO. This isn't common, but it does occur, e.g. Windows GUI apps
started without being connected to a console, controller side plugins run
under Ansible 12 (ansible-core 2.19).
In such cases the corresponding sys attribute (e.g. sys.stderr) will be None.
refs #1258
See also
- https://docs.python.org/3/library/sys.html#sys.__stdin__
- https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_12.html#porting-guide-for-v12-0-0a1
- https://github.com/ansible/ansible/pull/82770
- https://github.com/python/typeshed/issues/11778
- https://gist.github.com/moreati/034fef45f73d809d9411a8a63eca34d6
8 months ago
49fea37879
mitogen: Use pty.STD*_FILENO constants
...
Makes it more obvious and easier to find where stdin, stdout, and stderr file
descriptors are being interacted with.
8 months ago
8faae13d7e
mitogen: Improve readability of stderr initialisation in create_child()
...
There should be no behaviour change.
8 months ago
c146682e2e
Begin 0.3.24dev
9 months ago
8e25944c94
Prepare v0.3.23
9 months ago
927fb172d8
mitogen: Log skipped termios attributes
...
refs #1121
12 months ago
5f42da36f3
mitogen: Deduplicate cfmakeraw() flags
...
refs #1121
12 months ago
657e40b982
Begin 0.3.23dev
12 months ago
ae703b97a7
Prepare v0.3.22
12 months ago
67219c309a
mitogen: Fix unclosed file in first stage
1 year ago
6fcb7aae96
mitogen: Replace uses of deprecated `pkgutil.find_loader()`
...
fixes #1111
1 year ago
5c76941d1e
Begin 0.3.22dev
1 year ago
8b7354cb3a
Prepare v0.3.21
1 year ago
3ee6a0ff93
Begin v0.3.21dev
1 year ago
f1f8ad11ee
Prepare v0.3.20
1 year ago
ffb1709630
Begin v0.3.20dev
1 year ago
24ba734ff9
Prepare v0.3.19
1 year ago
1cd7ea18d3
Begin v0.3.19dev
1 year ago
d85d9a25ee
Prepare v0.3.18
1 year ago
d2db3c3840
Begin v0.3.18dev
1 year ago
6cf6f69751
Prepare v0.3.17
1 year ago
757527635d
Begin v0.3.17dev
1 year ago
d28dd09e23
Prepare v0.3.16
1 year ago
26c4c33ad3
Begin 0.3.16dev
1 year ago
7634e2c469
Prepare v0.3.15
1 year ago
d35ca3e4af
Begin 0.3.15.dev
1 year ago
c4ca015266
Prepare v0.3.14
1 year ago
47e25eb8c5
Begin 0.3.14 development
1 year ago
8dec038941
Prepare v0.3.13
1 year ago
1773c9aba6
trivia: Fix trailing whitespace
1 year ago
Alex Willmer