docs: document mitogen.core.CHUNK_SIZE.

docs/api.rst

@@ -87,10 +87,10 @@ Message Class
 
 .. class:: Message
 
-    Messages are the fundamental unit of communication, comprising the fields
-    from in the :ref:`stream-protocol` header, an optional reference to the
-    receiving :class:`mitogen.core.Router` for ingress messages, and helper
-    methods for deserialization and generating replies.
+    Messages are the fundamental unit of communication, comprising fields from
+    the :ref:`stream-protocol` header, an optional reference to the receiving
+    :class:`mitogen.core.Router` for ingress messages, and helper methods for
+    deserialization and generating replies.
 
     .. attribute:: router
 

docs/internals.rst

@@ -8,6 +8,13 @@ Internal API Reference
     signals
 
 
+Constants
+=========
+
+.. currentmodule:: mitogen.core
+.. autodata:: CHUNK_SIZE
+
+
 Latch Class
 ===========
 
@@ -92,10 +99,6 @@ Side Class
         wrapping the underlying :py:func:`os.write` call with :py:func:`io_op`
         to trap common disconnection connditions.
 
-        :py:meth:`read` always behaves as if it is writing to a regular UNIX
-        file; socket, pipe, and TTY disconnection errors are masked and result
-        in a 0-sized write.
-
         :returns:
             Number of bytes written, or :data:`None` if disconnection was
             detected.

mitogen/core.py

@@ -116,7 +116,34 @@ AnyTextType = (BytesType, UnicodeType)
 if sys.version_info < (2, 5):
     next = lambda it: it.next()
 
+#: Default size for calls to :meth:`Side.read` or :meth:`Side.write`, and the
+#: size of buffers configured by :func:`mitogen.parent.create_socketpair`. This
+#: value has many performance implications, 128KiB seems to be a sweet spot.
+#:
+#: * When set low, large messages cause many :class:`Broker` IO loop
+#:   iterations, burning CPU and reducing throughput.
+#: * When set high, excessive RAM is reserved by the OS for socket buffers (2x
+#:   per child), and an identically sized temporary userspace buffer is
+#:   allocated on each read that requires zeroing, and over a particular size
+#:   may require two system calls to allocate/deallocate.
+#:
+#: Care must be taken to ensure the underlying kernel object and receiving
+#: program support the desired size. For example,
+#:
+#: * Most UNIXes have TTYs with fixed 2KiB-4KiB buffers, making them unsuitable
+#:   for efficient IO.
+#: * Different UNIXes have varying presets for pipes, which may not be
+#:   configurable. On recent Linux the default pipe buffer size is 64KiB, but
+#:   under memory pressure may be as low as 4KiB for unprivileged processes.
+#: * When communication is via an intermediary process, its internal buffers
+#:   effect the speed OS buffers will drain. For example OpenSSH uses 64KiB
+#:   reads.
+#:
+#: An ideal :class:`Message` has a size that is a multiple of
+#: :data:`CHUNK_SIZE` inclusive of headers, to avoid wasting IO loop iterations
+#: writing small trailer chunks.
 CHUNK_SIZE = 131072
+
 _tls = threading.local()