TCP Deadlock

Let's imagine an echo server implementation, where the server simply takes the request input stream, copies the bytes to the response output stream.

Let's say a client starts sending bytes to the server, and the server starts writing the bytes out as soon as they come in, it's pretty nice that the server doesn't need to load the whole request first, which means the client can see the response quickly if it wishes to.

Except the client is still busy streaming the rest of bytes out to the server, so it's not ready to read the response yet, even though the response data has already started to come in.

Meanwhile the server is still writing bytes out, all that has to go somewhere. First they arrive at the client's TCP receive buffer, managed by the operating system, once that's full, the client's side of the TCP connection will tell the server side to stop sending, the server side TCP listens and stops sending data out the wire, anymore bytes written by the server will get placed onto the sending TCP buffer and stay there, once that's full, any attempt to write will block, and since the server is blocked, it won't read anything from the input anymore, so that direction of the buffers will get filled up too, and the client's attempt to write will also block, now we have a deadlock - the client is waiting for the server to free up the buffer, the server can't do that because it's waiting for the client to free up the buffer.

If you were monitoring your TCP Zero Windows you would see something like this, simplified:

% tcpdump 'tcp[14:2] = 0'
... server > client: ... win 0 ...
... client > server: ... win 0 ...
... server > client: ... win 0 ...
... client > server: ... win 0 ...
... server > client: ... win 0 ...

Where 'win 0' is one side telling the other they have no room for anymore data.

So it turns out this simple echo server will deadlock given a large enough request. So to avoid deadlock it will have to consume the whole request first before sending the response, which will require placing a restriction on the size of the request. Unless you have full control of the client code and implement it with reading/writing in parallel.

In fact this problem isn't specific to TCP, any program with similar IO structure will experience deadlock. For example, a parent process starts a child process and sends data to it while the child process is output data at the same time.