· engineering

One setting made our tunnel downloads 4× faster

A real test from a laptop 186 ms from our server showed a 5 MB file taking 14.8 seconds through a tunnel. Here is why, and what changed.

By InstaWormhole team, Engineering

Before calling the tunnel path ready, we ran the most ordinary test there is: serve a page from a laptop, open it through its public instawormhole.site address over the internet, and download a 5 MB file.

The page came back intact, missing pages returned 404, and the tunnel address carried its no-indexing and HTTPS-only headers. The download did not look right: 14.8 seconds for 5 MB. Sending the same file straight from that laptop to our server took 4.9 seconds, so the tunnel was about three times slower than the connection under it.

Why it was slow

Every request through a tunnel travels as a stream inside one connection between your computer's connector and our relay. Each stream has a flow-control window: the most data that can be in flight before the receiver says "send more". That window is what keeps a slow visitor from making the relay buffer an unlimited amount of your app's output.

A window also sets a speed limit. One stream can move at most one window per round trip. We started with a deliberately cautious 64 KiB window. The laptop in this test was 186 ms from our server, and 64 KiB every 186 ms is about 350 KB/s. We measured 340 KB/s.

What we changed

Each stream's window is now 512 KiB. Memory on the relay does not grow, because every connection still has a 1 MiB limit across all of its streams. A slow visitor still pushes back on your app as before. One fast download can now use more of that allowance.

Before the change went live it passed our release checks: the relay tests against the real connector, the tests that prove a stalled visitor cannot make the relay buffer without limit, and a three-minute load run of 775,488 requests with no failures.

The result

The same 5 MB download now takes 3.1 to 3.7 seconds from the same laptop, about four times faster. You don't need to do anything: the relay sets the window when your connector connects, so existing connectors pick it up the next time they reconnect.

The further you are from our server, the more this helps, because round-trip time is exactly what the old window multiplied.

To try it, run your app locally and start a tunnel to its port:

npx @instawormhole/connector 8000