Commit Graph

3264 Commits (e67e4b83b4801c797d9e2080714e78b7b24765d0)
 

Author SHA1 Message Date
David Wilson 6f7941d616 scripts: import affin.sh 5 years ago
David Wilson 558ebfa914 ci: update to Ansible 2.8.3 5 years ago
David Wilson 28b4d63e49 tests: terraform tweaks 5 years ago
David Wilson 3b000c7d15 unix: include more IO in the try/except for connection failure 5 years ago
David Wilson 5ed0b93669 tests: update gcloud.py to match terraform config 5 years ago
David Wilson 4dfbe82e76 tests: hide ugly error during Ansible tests 5 years ago
David Wilson de2e1ec184 tests/ansible/gcloud: terraform conf for load testing 5 years ago
David Wilson 108015aa22 ansible: gracefully handle failure to connect to MuxProcess
It's possible to hit an ugly exception during early CTRL+C
5 years ago
David Wilson a9d3fdf6b7 ansible: fix affinity tests for 5ae45f6612390bbc888b65964fb5c218feed1679 5 years ago
David Wilson bf1f3682aa ansible: pin per-CPU muxes to their corresponding CPU
This slightly breaks the old scheme, in that CPU 1 may now end up with a
mux and the top-level process pinned to it.
5 years ago
David Wilson dc9f4e89e6 ansible: reap mux processes on shut down
Previously we exitted without calling waitpid(), which meant the
top-level process struct rusage did not reflect the resource usage
consumed by the multiplexer processes.

Existing benchmarks are made using perf so this never created a problem,
but it could be confusing to others using the "time" command, and also
allows logging the final exit status of the process.
5 years ago
David Wilson adbad76925 Merge remote-tracking branch 'origin/543-darwin-ansible-ci'
* origin/543-darwin-ansible-ci:
  issue #543: install virtualenv for Azure
  issue #543: dumb fix for file vs. stat :(
  issue #543: disable host key checking
  issue #543: create ~/.ssh if it doesn't exist
  issue #543: Hide Mitogen test users from gdm
  issue #543: skip test that's hard to do on Mac
  issue #543: use key from Git, newer ssh-keygen unsupported by Paramiko
  image_prep: ensure Mac users can SSH without manual intervention
  issue #543: make localhost_ansible_tests run locally
  issue #543: add Ansible job to Azure matrix
  issue #543: localhost_ansible scripts.
5 years ago
David Wilson 57db3a36e1 issue #543: install virtualenv for Azure 5 years ago
David Wilson ebb4a7ca6a issue #543: dumb fix for file vs. stat :( 5 years ago
David Wilson f3915b5f40 issue #543: disable host key checking 5 years ago
David Wilson edb745f434 issue #543: create ~/.ssh if it doesn't exist 5 years ago
David Wilson 0741876392 issue #543: Hide Mitogen test users from gdm 5 years ago
David Wilson 17d0e1b315 issue #543: skip test that's hard to do on Mac 5 years ago
David Wilson 92de2abeea issue #543: use key from Git, newer ssh-keygen unsupported by Paramiko 5 years ago
David Wilson 0e55bb3eb7 image_prep: ensure Mac users can SSH without manual intervention 5 years ago
David Wilson 501cfca82b issue #543: make localhost_ansible_tests run locally 5 years ago
David Wilson 57203aef53 issue #543: add Ansible job to Azure matrix 5 years ago
David Wilson 4ced885619 issue #543: localhost_ansible scripts. 5 years ago
David Wilson c7e2b03d1c Merge remote-tracking branch 'origin/travis-reorder'
* origin/travis-reorder:
  ci: try moving slower 2.8.0 jobs together
  ci: roughly bin-pack Travis jobs longest->shortest
5 years ago
David Wilson 95e7bca518 ci: try moving slower 2.8.0 jobs together 5 years ago
David Wilson 7bf5ce4498 ci: roughly bin-pack Travis jobs longest->shortest 5 years ago
David Wilson 83b33a8fb1 Merge remote-tracking branch 'origin/linear2'
* origin/linear2:
  [linear2] fix another test relying on Connection.parent
  [linear2] more merge fallout, fix Connection._mitogen_reset(mode=)
  [linear2] update mitogen_get_stack for new _build_stack() return value
  [linear2] fix mitogen_shutdown_all service context access
  docs: changelog concision / additions
  add 363 to changelog
  docs: update Changelog
  docs: note fd usage has halved
  docs: more stream-refactor work
  docs: update Changelog for stream-refactor.
  docs: Add lineinfile bug to changelog.
  [linear2] fix MuxProcess test fixture and some merge fallout
  service: avoid taking another lock in the usual case
  service: don't acquire lock when pool already initialized
  profiler: marginal improvements
  core: ensure 'exit' signal fires even on Broker crash.
  core: wake Waker outside of lock.
  core: wake Latch outside of lock.
  core: remove old blocking call guard, it's in the wrong place
  Make setting affinity optional.
  ansible: abstract worker process model.
  [stream-refactor] parent: fix crash on graceful shutdown
  parent: tidy up create_socketpair()
  core: more concise Side.repr.
5 years ago
David Wilson 115145555e [linear2] fix another test relying on Connection.parent 5 years ago
David Wilson 136dee1fb4 [linear2] more merge fallout, fix Connection._mitogen_reset(mode=) 5 years ago
David Wilson a9755d4ad0 [linear2] update mitogen_get_stack for new _build_stack() return value 5 years ago
David Wilson a39169ff16 [linear2] fix mitogen_shutdown_all service context access 5 years ago
David Wilson 9e1faa7927 docs: changelog concision / additions 5 years ago
David Wilson c0357d394f add 363 to changelog 5 years ago
David Wilson 8f25114d1a docs: update Changelog 5 years ago
David Wilson de65790a74 docs: note fd usage has halved 5 years ago
David Wilson ded00d25c9 docs: more stream-refactor work 5 years ago
David Wilson 4cd32cbcba docs: update Changelog for stream-refactor. 5 years ago
David Wilson d35bca3f15 docs: Add lineinfile bug to changelog. 5 years ago
David Wilson 1fca0b7a94 [linear2] fix MuxProcess test fixture and some merge fallout 5 years ago
David Wilson e93762b3db service: avoid taking another lock in the usual case 5 years ago
David Wilson 50bfe4c746 service: don't acquire lock when pool already initialized 5 years ago
David Wilson f4709b1dc2 profiler: marginal improvements 5 years ago
David Wilson 3b585b841e core: ensure 'exit' signal fires even on Broker crash. 5 years ago
David Wilson d6faff06c1 core: wake Waker outside of lock.
Given:

- Broker asleep in poll()
- thread B calling Latch.put()

Previously,

- B takes lock,
- B wakes socket by dropping GIL and writing to it
- Broker wakes from poll(), acquires GIL only to find Latch._lock is held
- Broker drops GIL, sleeps on futex() for _lock
- B wakes, acquires GIL, releases _lock
- Broker wakes from futex(), acquires lock

Now,

- B takes lock, updates state, releases lock
- B wakes socket by droppping GIL and writing to it
- Broker wakes from poll(), acquires GIL and _lock
- Everyone lives happily ever after.
5 years ago
David Wilson 807cbef9ca core: wake Latch outside of lock.
Given:

- thread A asleep in Latch._get_sleep()
- thread B calling Latch.put()

Previously,

- B takes lock,
- B wakes socket by dropping GIL and writing to it
- A wakes from poll(), acquires GIL only to find Latch._lock is held
- A drops GIL, sleeps on futex() for _lock
- B wakes, acquires GIL, releases _lock
- A wakes from futex(), acquires lock

Now,

- B takes lock, updates state, releases lock
- B wakes socket by droppping GIL and writing to it
- A wakes from poll(), acquires GIL and _lock
- Everyone lives happily ever after.
5 years ago
David Wilson 7e51a93231 core: remove old blocking call guard, it's in the wrong place
It should have been in Receiver.get(). Placing it here prevents
*_async() method calls from broker thread.
5 years ago
David Wilson 0f63ca4c68 Make setting affinity optional. 5 years ago
David Wilson 9035884c77 ansible: abstract worker process model.
Move all details of broker/router setup out of connection.py, instead
deferring it to a WorkerModel class exported by process.py via
get_worker_model(). The running strategy can override the configured
worker model via _get_worker_model().

ClassicWorkerModel is installed by default, which implements the
extension's existing process model.

Add optional support for the third party setproctitle module, so
children have pretty names in ps output.

Add optional support for per-CPU multiplexers to classic runs.
5 years ago
David Wilson 6b8a7cbcc4 [stream-refactor] parent: fix crash on graceful shutdown
Now it's possible for stream.protocol to not refer to MitogenProtocol,
move the signal handler to a MitogenProtocol subclass instead.

Fixes a crash where CTRL+C during child bootstrap would print
AttributeError.
5 years ago
David Wilson 2ccdeeeb87 parent: tidy up create_socketpair() 5 years ago