core: set O_NONBLOCK on every side. Closes #33

The last time I tested set_nonblock() as a fix for the rsync hang, I
used F_SETFD rather than F_SETFL, which resulted in no error, but also
did not set O_NONBLOCK. Turns out missing O_NONBLOCK was the problem.

The rsync hang was due to every context blocking in os.write() waiting
for either a parent or child buffer to empty, which was exacerbated by
rsync's own pipelining, that allows writes from both sides to proceed
even while reads aren't progressing. The hang was due to os.write() on a
blocking fd blocking until buffer space is available to complete the
write. Partial writes are only supported when O_NONBLOCK is enabled.
pull/35/head
David Wilson 7 years ago
parent 16950a1620
commit 68f3918072

@ -29,7 +29,7 @@ FORWARD_LOG = 102
ADD_ROUTE = 103
ALLOCATE_ID = 104
CHUNK_SIZE = 4096 # TODO: this was 16384, but that triggers an unfixed hang.
CHUNK_SIZE = 16384
if __name__ == 'mitogen.core':
@ -125,6 +125,11 @@ def set_cloexec(fd):
fcntl.fcntl(fd, fcntl.F_SETFD, flags | fcntl.FD_CLOEXEC)
def set_nonblock(fd):
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
def io_op(func, *args):
try:
return func(*args), False
@ -461,6 +466,8 @@ class Side(object):
#: active reader set to defer shutdown until the side is disconnected.
self.keep_alive = keep_alive
set_nonblock(fd)
def __repr__(self):
return '<Side of %r fd %s>' % (self.stream, self.fd)

Loading…
Cancel
Save