Sockets

Ephemeral sockets

We may not spend much time thinking about the number of sockets on our server, but that’s another limit we can run into when traffic becomes heavy. Every active request corresponds to an open socket. The operating system assigns inbound connections to an “ephemeral” port that represents the receiving side of the connection. If we look at the TCP packet format, we’ll see that a port number is 16 bits long. It can only go up to 65,535. Different OSs use different port ranges for ephemeral sockets, but the IANA recommended range is 49,152 to 65535. That gives our server the ability to have at most 16,383 connections open. But our machine is probably dedicated to our service rather than handling, say, user logins. So we can stretch that range to ports 1,024–65,535, for a maximum of 64,511 connections.

Huge connections

Now we’ll tell you that some servers are handling more than a million concurrent connections. Some people are pushing toward ten million connections on a single machine. If there are only 64,511 ports available for connections, how can a server have a million connections? The secret is virtual IP addresses. The operating system binds additional IP addresses to the same network interface. Each IP address has its own range of port numbers, so we would need a total of 16 IP addresses to handle that many connections.

Virtual IP address

Virtual IP addresses are not a trivial thing to tackle. Our application will probably need some changes to listen on multiple IP addresses and handle connections across them all without starving any of the listen queues. A million connections also need a lot of kernel buffers. Plan to spend some time learning about the operating system’s TCP tuning parameters.

Closed sockets

Not only can open sockets be a problem, but the ones we’ve already closed can bite too. After your application code closes a socket, the TCP stack moves it through a couple of terminal states. One of them is the TIME_WAIT state.

Tip to remember: The TIME_WAIT state is a delay period before the socket can be reused for a new connection. It’s there as part of TCP’s defense against bogons.