Commit Graph

402 Commits (49eae23f920be56178a6cedbceb977e0de432f3b)

Author SHA1 Message Date
David Wilson cecef992b0 issue #218: core: add Secret and Blob types. 7 years ago
David Wilson 42cc009b60 service: don't sleep on empty Select during shutdown.
Avoids a select error during random CTRL+C.
7 years ago
David Wilson 7f1060f54a issue #186: initial version of subtree detachment. 7 years ago
David Wilson 8bd34e1e28 ansible: gracefully report connection timeouts as StreamError. 7 years ago
David Wilson c0e8b3d60a ssh: error wording was inaccurate. 7 years ago
David Wilson 8fc1eac6ae utils: combine MITOGEN_LOG_LEVEL and MITOGEN_LOG_IO.
Saves lots of readline fiddling.
7 years ago
David Wilson 4d1c6d2101 issue #127: ssh: reasonable solution to host key checking.
Ideally it would be possible to specify a callback function, but this is
not possible for proxied connections. So simply provide the 3 most
useful modes, defaulting to the most secure.

Closes #127. Closes #134.
7 years ago
David Wilson 92a2565507 issue #241: child main thread does not gracefully handle CTRL+C
In Ansible, depending on when CTRL+C is triggered, if it occurs after
the connection multiplexer process has forked, and after it has in turn
forked the "connection: local" context and its corresponding "clean fork
parent", since all the broker processes still belong to Ansible's
terminal foreground process group, they are all capable of receiving
SIGINT in response to CTRL+C being pressed on that terminal.

This papers over the problem. Really we want those KeyboardInterrupts to
be logged, to call setsid() frmo the connection multiplexer process to
isolate it from the terminal foreground process group. That way its only
indication of top-level process shutdown is using the graceful
disconnect mechanism that already exists in process.py::worker_main().
7 years ago
David Wilson 3322eaef45 Basic "su" method. 7 years ago
David Wilson 79346d96db core: Allow dead messages to be delivered regardless of policy 7 years ago
David Wilson f5d22a3ca1 core: support deleting handlers, make Receiver.close() unregister 7 years ago
David Wilson 5e6e56f0c5 issue #212: service: make call_async kwargs optional. 7 years ago
David Wilson afe983d6c9 issue #212: service: support no_reply decorator. 7 years ago
David Wilson bf6c2fa97c issue #212: service: more concise repr 7 years ago
David Wilson ff7fb00569 parent: return latch to wait() caller to allow graceful timeout 7 years ago
David Wilson c0ced6d04a core: fix monster fork FD leak
_sockets only refers to the idle sockets list, it doesn't refer to every
socket currently in use by a Latch, for example, the 2*16 used by e.g.
Ansible's sleeping service pool.
7 years ago
David Wilson 7316c08237 core: fix _tls_init() race.
The GIL could be lost between the check for an empty list and popping a
socket off the list. Previously _tls_init (per its name) used per-thread
storage, hence the bug.
7 years ago
David Wilson fdac4ecce8 setns: typos 7 years ago
David Wilson e93ac2f3a7 debug: implement some basic helpers to debugger. 7 years ago
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
7 years ago
David Wilson 947d35649c parent: note exception machine's hostname.
For dumb situations where user (i.e. me) is trying to fix a problem in
the wrong place.
7 years ago
David Wilson e8b4c4e683 issue #223: implement setns connection type
machinectl does not support any sensible form of pipe to the child
process, so it is necessary to bypass it when talking to a systemd
container (see systemd/systemd#8850).

This can also form the basis for issue #223, where the post-fork
namespace switching dance required to connect to the Pythonless
container will be the same.
7 years ago
David Wilson 3196b6e7f7 Add FreeBSD jail support. 7 years ago
David Wilson b3d352c601 Add lxc container support. 7 years ago
David Wilson 1be03eb458 parent: change create_child interface.
To allow for additional arguments.
7 years ago
David Wilson 1fc7df5be5 Move canonical library version to __init__.py. 7 years ago
David Wilson 6fb3a76e68 master: annotate LogForwarder messages.
mitogen/master.py:
    Annotate forwarded log entries with their original source, logger
    name, and message.

ansible:
    mark stderr in red with -vvv

    Tempting to make this appaer 100% of the time, but some crappy
    bashrcs may cause lots of junk to be printed.
7 years ago
David Wilson 9d0949eb99 docker: fixes & add username parameter. 7 years ago
David Wilson e63ae4768e core: support Receiver.get(thread_dead=False)
For tests.
7 years ago
David Wilson 509ef14121 service: add an on_shutdown() callback. 7 years ago
David Wilson 4c5e13bf87 core: add Stream.pending_bytes() accessor. 7 years ago
David Wilson 7c88e4d013 Move _DEAD into header, autogenerate dead messages
This change blocks off 2 common scenarios where a race condition is
upgraded to a hang, when the library could internally do better.

* Since we don't know whether the receiver of a `reply_to` is expecting
  a raw or pickled message, and since in the case of a raw reply, there
  is no way to signal "dead" to the receiver, override the reply_to
  field to explicitly mark a message as dead using a special handle.

  This replaces the serialized _DEAD sentinel value with a slightly
  neater interface, in the form of the reserved IS_DEAD handle, and
  enables an important subsequent change: when a context cannot route a
  message, it can send a generic 'dead' reply back towards the message
  source, ensuring any sleeping thread is woken with ChannelError.

  The use of this field could potentially be extended later on if
  additional flags are needed, but for now this seems to suffice.

* Teach Router._invoke() to reply with a dead message when it receives a
  message for an invalid local handle.

* Teach Router._async_route() to reply with a dead message when it
  receives an unroutable message.
7 years ago
David Wilson e56608ab91 parent: don't wait for SIGTERM to complete. 7 years ago
David Wilson cbe6be449e issue #201: parent: log a warning and work around race for now. 7 years ago
David Wilson 9fe14e841c parent: reap the child process if connection fails
For example if no response is received in :attr:`connect_timeout`
seconds, the child would be left running.
7 years ago
David Wilson 296683b130 ansible: always display Mitogen errors and warnings.
They're no longer buried in -vvv output.
7 years ago
David Wilson 65fb6ff9fe issue #195: comment out stack pruning code 7 years ago
David Wilson 810f557514 issue #195: MITOGEN_DUMP_THREAD_STACKS=1 7 years ago
David Wilson f06ae05734 issue #195: add extra logging around FileService and get_file(). 7 years ago
David Wilson 46a311165e issue #148: parent: prevent race in iter_read()
There is no guarantee on the ordering select() returns file descriptors.
So if, e.g. in the case of sudo_nonexistent.yml, sudo prints an error
to a single FD before exitting, there was previously no gurantee
iter_read() would read off the error before failing due to detecting
disconnect on any FD.

Now instead we keep reading while any non-disconnected FD exists.
7 years ago
David Wilson e43c6c531b Mostly implement hybrid TTY/socket mode for sudo and SSH.
Presently there is still no mechanism to add :attr:`tty_stream` to the
multiplexer after connection is successful, but for now it's not
expected that anything will be logged to it anyway.

Closes #148.
7 years ago
David Wilson c6284e00e9 Use subprocess to start child processes; closes #185. 7 years ago
David Wilson bdc76c8231 parent: do not attempt to reap child twice. 7 years ago
David Wilson 54452c4015 service: fix UnboundLocalError. 7 years ago
David Wilson 202ce0f641 Prevent construction of unicode Message.data
And fix one case of it in parent.py.
7 years ago
David Wilson c83b113600 service: log better warning when CallError raised. 7 years ago
David Wilson ce6fb05d87 tests: 'fix' responder test.
Needs a complete rewrite, but this will do for now.
7 years ago
David Wilson f9eb66e76e _py_filename() must handle None too. 7 years ago
David Wilson 34a1e3337f Fix get_module_via_sys_modules when running under unit2. 7 years ago
David Wilson cc980569a3 issue #159: initial context LRU implementation
Now Connection.close() *must* be called in the worker, to ensure the
reference count for a context drops correctly.

Remove 'discriminator' for now, I'm not using it for testing any more
and it complicated this code.

This code is a car crash, it needs rewritten again. Ideally some/most of
this behaviour could live on services.DeduplicatingService somehow, but
I couldn't come up with a sensible design.
7 years ago