A limiting factor in the number of connections that can be made per second by the balancer is the availability of 'ephemeral' network ports. These are local port ranges that programs can use when making connections to remote machines, such as the back-ends. When the connection closes, the operating system does not immediately free up a port for further use. You can see how many such connections exist on a machine by running the command 'netstat -a' and searching for lines containing the phrase 'TIME_WAIT'.

These old ports can build up and prevent the balancer making further connections. There are two ways to solve the problem: Increase the range of local ports available, and decrease the length of time that old ports are kept in TIME_WAIT. The following instructions show the commands that should be run in order to increase the limits on ephemeral ports. All typed commands are prefixed by the super-user command prompt '#' - you will need to be root to change these settings.

Please also note that these changes will not persist across reboots, so after testing changes, you will need to add the appropriate commands to a start up script (perhaps $ZEUSHOME/start-zeus) for the changes to be permanent.

Linux

On Linux, the local port ranges are set by the file /proc/sys/net/ipv4/ip_local_port_range. This contains two numbers, the low and high end of the values to use for local ports. Older versions of Linux set these to be 1024 and 4999, respectively. These values are too restrictive. To set a wider range, you can use a command like the following: (run as root)

# echo "1024 61000" > /proc/sys/net/ipv4/ip_local_port_range

The file /proc/sys/net/ipv4/tcp_fin_timeout (if it exists, older kernels do not have this option) contains the time that old connections will be kept in the TIME_WAIT state. Older versions of Linux set this to 180 (seconds). A value of 30 should be fine.

# echo "30" > /proc/sys/net/ipv4/tcp_fin_timeout

FreeBSD / OpenBSD

You can adjust the portranges with the sysctl command (you'll need to be root). E.g.

# sysctl -w net.inet.ip.portrange.last=61000 (FreeBSD)
# sysctl -w net.inet.ip.portlast=61000 (OpenBSD, default settings are fine)

Solaris / HP-UX

The system command 'ndd' can be used to tune the ephemeral port range. By default, the ephemeral port range on HP and Sun machines is quite large, and can be viewed with the following command:

# ndd -get /dev/tcp tcp_smallest_anon_port tcp_largest_anon_port 49152 65535

To increase the range, run the command:

# ndd -set /dev/tcp tcp_smallest_anon_port 8192

Tru64

Tru64 uses the 'traditional' ephemeral port range of 1024-4999, which can be quite limiting. To increase the port range, you can use the 'sysconfig' command:

# sysconfig -r inet ipport_userreserved=65535 ipport_userreserved_min=8192ipport_userreserved: reconfiguredipport_userreserved_min: reconfigured

Content Manager [Administrator] 12 August 2005