You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
tailscale/ipn
Joe Tsai 3f27087e9d
taildrop: switch hashing to be streaming based (#9861)
While the previous logic was correct, it did not perform well.
Resuming is a dance between the client and server, where
1. the client requests hashes for a partial file,
2. the server then computes those hashes,
3. the client computes hashes locally and compares them.
4. goto 1 while the partial file still has data

While step 2 is running, the client is sitting idle.
While step 3 is running, the server is sitting idle.

By streaming over the block hash immediately after the server
computes it, the client can start checking the hash,
while the server works on the next hash (in a pipelined manner).
This performs dramatically better and also uses less memory
as we don't need to hold a list of hashes, but only need to
handle one hash at a time.

There are two detriments to this approach:
* The HTTP API relies on a JSON stream,
  which is not a standard REST-like pattern.
  However, since we implement both client and server,
  this is fine.
* While the stream is on-going, we hold an open file handle
  on the server side while the file is being hashed.
  On really slow streams, this could hold a file open forever.

Updates tailscale/corp#14772

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
12 months ago
..
conffile ipn/{conffile,ipnlocal}: start booting tailscaled from a config file w/ auth key 12 months ago
ipnauth all: update copyright and license headers 2 years ago
ipnlocal taildrop: switch hashing to be streaming based (#9861) 12 months ago
ipnserver logpolicy, various: allow overriding log function 1 year ago
ipnstate client/web: restrict full management client behind browser sessions 12 months ago
localapi taildrop: switch hashing to be streaming based (#9861) 12 months ago
policy ipn: prefer allow/denylist terminology 2 years ago
store cmd/k8s-operator,ipn/store/kubestore: patch secrets instead of updating 1 year ago
backend.go cmd/tailscale,ipn/ipnlocal: print debug component names 1 year ago
conf.go ipn/{conffile,ipnlocal}: start booting tailscaled from a config file w/ auth key 12 months ago
doc.go all: update copyright and license headers 2 years ago
fake_test.go all: update copyright and license headers 2 years ago
ipn_clone.go ipn: add PostureChecks to Prefs 12 months ago
ipn_test.go net/packet: split off checksum munging into different pkg 12 months ago
ipn_view.go ipn: add PostureChecks to Prefs 12 months ago
prefs.go ipn: add PostureChecks to Prefs 12 months ago
prefs_test.go ipn: add PostureChecks to Prefs 12 months ago
serve.go ipn: use NodeCapMap in CheckFunnel 1 year ago
serve_test.go ipn: use NodeCapMap in CheckFunnel 1 year ago
store.go ipn: avoid useless no-op WriteState calls 1 year ago
store_test.go ipn: avoid useless no-op WriteState calls 1 year ago