This document deals with tuning Zeus Web Server version 4.2 for high performance web serving. These instructions are provided for advanced users who are performing pre-deployment testing of their Zeus Web Server systems and are familiar with load testing concepts and OS/hardware
bottleneck identification as discussed earlier in this section of the FAQ.
If you're looking to achieve maximum performance, then the first thing you can do is disable modules you don't require. Every module has a small overhead. If you don't require functionality such as directory listings, then you'll see a performance gain by disabling it. The only modules required to serve a web request are the map and get modules. Modules can be disabled in the administration server on the module configuration page.
Our log format is fully user-definable, but the NCSA Common Log Format contains some extra optimizations. Alternatively binary log format log files could be used to save on disk writes.
Disable DNS lookups. Although we cache them, you can save the cache lookup and some DNS lookups by disabling DNS lookups for the virtual server you're using.
Tunable Parameters
For most uses, you need not worry about these tunables, the performance gain from any of these tunables is only likely to be a few percent. However, in order to help you get every last iota of performance out of your server, we've provided a set of tunables which should help you boost performance.
Tunables are stored in the $ZEUSHOME/web/global.cfg text file. The following tunables should be placed in this file in the format "tuning!<tunable> <value>", e.g.:
tuning!listen_queue_size 511
tuning!so_rbuff_size 32768
In order for tunables to take effect, you must restart the web server
process. You can do this with:
cd $ZEUSHOME
./restart-zeus
Clustered setup
In a clustered setup, where you have multiple web server machines, each machine has its own tunables that are independent of each other machine. Each machine may be running on different hardware or using a variety of operating systems and the front-end load balancers may distribute requests requiring different types of processing to different back-end machines, hence the flexibility. Simply add the required tunables to each machine's $ZEUSHOME/web/global.cfg file.
TCP tunables
| listen_queue_size |
Size of the TCP listen queue. This is the second argument that
gets passed to the listen() system call.
|
256 |
| so_rbuff_size |
If set, we will set the socket option so_rbuff_size on the
server socket. This will also affect the size of the socket
buffers in all accept()ed sockets. The value is the size
of the socket buffer in bytes.
|
not-set |
| so_wbuff_size |
If set, the size to set the socket write buffer to be. The larger
this is the greater the amount of data swallowed on every
write() request. In theory, bigger buffers should give
better performance.
|
not-set |
| so_nagle_off |
Disable nagle on each request. Note this is only done on the
second (keepalive) HTTP request along any one TCP
connection. While not technically necessary, can offer a
performance boost on 'brain-dead' TCP stacks (which is most of
them!).
|
yes |
| multiple_accept |
Try and do multiple accept() calls when we get a read bit
on the server socket from a poll()/select().
|
yes |
| maxaccept |
The maximum number of accept() calls to perform per server socket
per poll()/select().
|
32 |
| max_connections |
The maximum number of connections to process simultaneously (per
zeus.web process). When this limit is reached, incoming connections
are left on the OS listen queue. For sites serving a lot of dynamic
content (e.g. CGIs or BEA WebLogic), limiting the number of
simultaneously processed requests can prevent overload of these
third-party applications.
Available in ZWS 4.2 and later.
|
Approximately half the available number of file-descriptors |
| clientfirst_optimise |
Try to optimize listening sockets so that connections are not
accept()ed until client data is available to be read. This
allows the initial read on a client socket to succeed every time,
reducing the overhead of connection processing. It is only available
on Linux 2.4 and FreeBSD 4.0 or above.
|
no |
| unique_bind |
If you have multiple child processes and multiple IP addresses,
then have each child bind to a unique IP address, instead of having
every child process connections from each interface. On a
multi-CPU machine with multiple network interfaces, having one
child per network interface can reduce OS contention (as only one
process is ever using it) and give a good performance boost. See
the bindaddr tunable below for how to set a single virtual server
to listen to multiple IP addresses.
|
no |
| bind_any |
Have server sockets that are bound to IPADDR_ANY. This
option is mutually exclusive to unique_bind.
|
yes |
| use_poll |
Use poll() instead of select(). |
yes |
| use_devpoll |
Use the /dev/poll device instead of select() or
poll(). This allows the web server to notify the OS of
active file descriptors only when they change rather than on every
call to poll(), which can improve scalability to large
numbers of active connections. /dev/poll is only available
on some architectures (Solaris 8, HP-UX 11, and some versions of
Linux).
|
no |
| maxfds |
The maximum number of file descriptors to set by
setrlimit(). People using the sendfile() cache
should increase the parameter as far as their OS will allow.
|
16384 |
Cache Tunables
The default cache size is setup for real-world usage, not SPECweb99. For running SPECweb99 you will definitely want to increase the following tunables to ensure no disk IO occurs during a test run.
| cache_files |
The size (number of files) of the web server file cache. |
8011 |
| cache_small_file |
Max size of a 'small' file in bytes. |
4096 |
| cache_large_file |
Min size of a 'large' file in bytes. |
1048576 |
| cache_stat_expire |
Number of seconds to cache the response of a stat() call
for.
|
17 |
| cache_max_bytes |
Maximum number of bytes to reserve for cached files. A value of 0
means no limit (i.e. only the cache_files limit above is applied
and files are discarded with the cache_flush_interval value below).
|
33554432 |
| cache_flush_interval |
Any cached file that hasn't been accessed for longer than this
value in seconds is discarded by the web server, even if the cache
isn't full.
|
120 |
Small files are loaded in and cached in memory allocated by malloc(). Small files are fastest, but less space-efficient. On multiple CPU machines, each zeus.web process will keep a separate cache of small files. Large files are read each time using read(). Other files are read and cached using mmap(). For optimal performance the cache_files option should be a prime number. The optimal values for these settings will depend on the OS, and how much memory you have.
On some operating systems (at time of writing HP-UX 11.00, AIX, Linux 2.2 and above, and FreeBSD) we support the sendfile() API. This gives excellent web server performance. The following tunables affect its behaviour.
| sendfile |
Enable sendfile on platforms that support it. |
yes |
| sendfile_reservefd |
Number of file descriptors to reserve for sendfile() fd
cache. If not set, 10% of all descriptors will be reserved.
|
0 |
| sendfile_minsize |
Size in bytes of the smallest file to be sent with sendfile. |
32768 |
| sendfile_maxsize |
Size in bytes of the biggest file to be sent with sendfile. |
2,147,483,647 |
On recent versions of HP-UX 11, we also support the sendpath() API, which can improve performance further when large numbers of files are being served concurrently. The following tunables affect its behaviour.
| sendpath |
Enable sendpath on platforms that support it. |
yes |
| sendpath_minsize |
Size in bytes of the smallest file to be sent with sendpath. |
32768 |
| sendpath_maxsize |
Size in bytes of the biggest file to be sent with sendpath. |
2,147,483,647 |
HTTP Tunables
| softservers |
Enable 'software' virtual servers. Requires extra processing and
can be turned off if not required.
|
yes |
| keepalive |
Enable HTTP 1.1 'keepalive' connections. Requires extra processing
but disabling will not improve overall performance.
|
yes |
| keepalive_timeout |
If no traffic has been seen on a keepalive connection for this
amount of time (in seconds), then close the connection.
|
15 |
| keepalive_max |
The maximum number of keepalive connections the web server will
allow. Bencharks will want to revise this tunable upward.
|
256 |
SSL tunables
| ssl_keepalive |
Enable SSL 'keepalive' connections. This can substantially increase the performance of a secure site by avoiding the requirement to set up a new session for every request from the same client.
|
yes (as of ZWS 4.1) |
| ssl_diskcache |
When set to 'yes', the web server will use a 2nd level on-disk cache
for storing SSL session information in the directory
$ZEUSHOME/web/ssl_cache. If you are clustering several
machines to host the same SSL website, you will get greatly
enhanced performance if you set this option to yes, and share the
$ZEUSHOME/web/ssl_cache over NFS between the server
machines (e.g. by symlinking the directory to a shared mount
point).
|
no (single CPU machines)/ yes (multi-CPU machines) |
| ssl_diskcache_clean |
The on-disk SSL cache is regularly scanned for expired entries, which
are then removed. This tunable sets the minimum length of time in
seconds between scans.
|
300 |
| ssl_diskcache_expiry |
The length of time in seconds before an entry in the on-disk SSL
cache expires.
|
1800 |
Log management Tunables
| logmanager_cachesize |
The web server keeps a pool of cached open file descriptors to
frequently-accessed log files. If you find your working set of
active logfiles is large, you may be able to get a performance
increase by reserving more file descriptors to the logmanager
cache. Ensure you have enough available file descriptors before
raising this limit too high, see your OS documentation for
information on how to measure and raise this limit.
|
31 |
Kernel Accelerator Tunables
Some operating systems provide HTTP accelerators in their kernels, which use zero-copy techniques to improve the speed of simple static file serving. Zeus Web Server contains specialized support for some accelerators which are not fully transparent.
| accelerator!nca!enabled |
Enable support for NCA (Network Cache Accelerator) on Solaris. The
web server must be listening on port 80.
|
no |
| accelerator!frca!enabled |
Enable support for FRCA (Fast Response Cache Accelerator) on AIX. |
no |
| accelerator!frca!log!enabled |
Ask FRCA to keep an access log of the requests it handles. |
no |
| accelerator!frca!minsize |
Size in bytes of the smallest file to be handled by FRCA. |
32768 |
| accelerator!frca!maxsize |
Size in bytes of the biggest file to be handled by FRCA. |
2,147,483,647 |
Other Tunables
| num_children |
Number of child processes to run. The optimal value is probably 1
per CPU, although you may find benefits with running more or less
depending on the OS and test involved and the number of CPUs on the
system under test.
|
One per CPU |
| timeout |
Timeout for client transfers (seconds), if the server is unable to
communicate with the client for this period of time, the connection
is closed.
|
120 |
| check_symlinks |
When enabled, the server does extra processing ensure every file
returned to the client involves no use of symlinks.
Allowable values are:
| 0 |
no symlink checking |
| 1 |
always deny symlinked files |
| 2 |
allow if owner of symlink = owner of destination |
|
0 |
Per-virtual server settings
These per virtual server settings are not stored in the per-machine global.cfg files, but rather the per-virtual server config files, which are stored on the admin server machine in $ZEUSHOME/webadmin/conf/virtual_servers/sites/<vserver>, where <vserver> is the name of the virtual server you want to tune. After editing these files, you will need to use the adminserver "traffic light" page to toggle the running status of these virtual servers to ensure the new configuration files gets sent over to the web server machine(s).
The following tunables should be placed in these files in the format "<tunable> <value>", e.g.:
bindaddr 10.0.0.1,10.0.0.2,10.0.0.3
| bindaddr |
Comma separated list of IP addresses that the virtual server will
accept connections on. See unique_bind also.
|
blank - uses the first IP address that the DNS name resolves to. |
