You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
mitogen/docs/services.rst

131 lines
3.6 KiB
ReStructuredText

.. currentmodule:: mitogen.service
Service Framework
=================
Mitogen includes a simple framework for implementing services exposed to other
contexts, with some built-in subclasses to capture common designs. This is a
work in progress, and new functionality will be added as common usage patterns
emerge.
Overview
--------
Service
* User-supplied class with explicitly exposed methods.
* May be auto-imported/constructed in a child from a parent simply by calling it
* Identified in calls by its canonical name (e.g. mypkg.mymod.MyClass) by
default, but may use any naming scheme the configured activator understands.
* Children receive refusals if the class is not already activated by a aprent
* Has an associated Select instance which may be dynamically loaded with
receivers over time, on_message_received() invoked if any receiver becomes
ready.
Invoker
* Abstracts mechanism for calling a service method and verifying permissions.
* Built-in 'service.Invoker': concurrent execution of all methods on the thread pool.
* Built-in 'service.SerializedInvoker': serialization of all calls on a single
thread borrowed from the pool while any request is pending.
* Built-in 'service.DeduplicatingInvoker': requests are aggregated by distinct
(method, kwargs) key, only one such method ever executes, return value is
cached and broadcast to all request waiters. Waiters do not block additional
pool threads.
Activator
* Abstracts mechanism for activating a service and verifying activation
permission.
* Built-in activator looks for service by fully.qualified.ClassName using
Python import mechanism, and only permits parents to trigger activation.
Pool
* Manages a fixed-size thread pool, mapping of service name to Invoker, and an
aggregate Select over every activate service's Selects.
* Constructed automatically in children in response to the first
CALL_SERVICE message sent to them by a parent.
* Must be constructed manually in parent context.
* Has close() and add() methods.
Example
-------
.. code-block:: python
import mitogen
import mitogen.service
class FileService(mitogen.service.Service):
"""
Simple file server, for demonstration purposes only! Use of this in
real code would be a security vulnerability as it would permit children
to read arbitrary files from the master's disk.
"""
handle = 500
required_args = {
'path': str
}
def dispatch(self, args, msg):
with open(args['path'], 'r') as fp:
return fp.read()
def download_file(context, path):
s = mitogen.service.call(context, FileService.handle, {
'path': path
})
with open(path, 'w') as fp:
fp.write(s)
@mitogen.core.takes_econtext
def download_some_files(paths, econtext):
for path in paths:
download_file(econtext.master, path)
@mitogen.main()
def main(router):
pool = mitogen.service.Pool(router, size=1, services=[
FileService(router),
])
remote = router.ssh(hostname='k3')
remote.call(download_some_files, [
'/etc/passwd',
'/etc/hosts',
])
pool.stop()
Reference
---------
ansible: enable forking when requested and for async jobs. Closes #105. References #155. mitogen/service.py: Refactor services to support individually exposed methods with different security policies for each method. - @mitogen.service.expose() to expose a method and set its policy - @mitogen.service.arg_spec() to validate input. - Require basic service message format to be a tuple of `(method, kwargs)`, where kwargs is always a dict. - Update DeduplicatingService to match the new scheme. ansible_mitogen/connection.py: - Rename 'method' to 'method_name' to disambiguate it from the service.call()'s method= argument. ansible_mitogen/planner.py: - Generate an ID for every job, sync or not, and fetch job results from JobResultService rather than via the initiating function call's return value. - Planner subclasses now get to select whether their Runner should run in a forked process. The base implementation requests this if the 'mitogen_isolation_mode=fork' task variable is present. ansible_mitogen/runner.py: Teach runners to deliver their result via JobResultService executing in their indirect parent mux process. ansible_mitogen/plugins/actions/mitogen_async_status.py: Split the implementation up into methods, and more compatibly emulate Ansible's existing output. ansible_mitogen/process.py: Mux processes now host JobResultService. ansible_mitogen/services.py: Update existing services to the new mitogen.service scheme, and implement JobResultService: * listen() method for synchronous jobs. planner.invoke() registers a Sender with the service prior to invoking the job, then sleeps waiting for the service to write the job result to the corresponding Receiver. * Non-blocking get() method for implementing mitogen_async_status action. * Child-accessible push() method for delivering task results. ansible_mitogen/target.py: New helpers for spawning a virginal subprocess on startup, from which asynchronous and mitogen_task_isolation=fork jobs are forked. Necessary to avoid a task inheriting potentially polluted/monkey-patched parent environment, since remaining jobs continue to run in the original child process. docs/ansible.rst: Add/merge/remove some behaviours/risks. tests/ansible/integration: New tests for forking/async.
7 years ago
.. autoclass:: mitogen.service.Policy
.. autoclass:: mitogen.service.AllowParents
.. autoclass:: mitogen.service.AllowAny
.. autofunction:: mitogen.service.arg_spec
.. autofunction:: mitogen.service.expose
.. autofunction:: mitogen.service.Service
.. autoclass:: mitogen.service.Service
:members:
.. autoclass:: mitogen.service.DeduplicatingService
:members:
.. autoclass:: mitogen.service.Pool
:members: