crossroads

crossroads.man (109280B)

---

 .TH "Crossroads 1\&.23" "2005, 2006, ff\&."
 .PP
 .SH "Crossroads 1\&.23"
 .SH "Karel Kubat"
 .SH "e-tunity"
 .SH "2005, 2006, ff\&."
 
 .PP 
 
 .SH "1: Introduction"
 Crossroads is a daemon that basically accepts TCP connections
 at preconfigured ports, and given a list of \&'back ends\&'
 distributes each incoming connection to one of the back ends,
 so that a client request is
 served\&. Additionally, crossroads maintains an internal
 administration of the back end connectivity: if a back end isn\&'t
 usable, then the client request is handled using another back
 end\&. Crossroads will then periodically check whether a previously not
 usable back end has come to life yet\&. Also, crossroads can select
 back ends by estimating the load, so that balancing is achieved\&.
 .PP 
 Using this approach, crossroads serves as load balancer and fail over
 utility\&. Crossroads will very likely not be as reliable as
 hardware based balancers, since it always will require a server to
 run on\&. This server, in turn, may become a new Single Point of
 Failure (SPOS)\&.  However, in situations where cost efficiency is an issue,
 crossroads may be a good choice\&. Furthermore, crossroads can be
 deployed in situations where a hardware based balancing already
 exists and augmenting service reliability is needed\&. Or, crossroads may be
 run off a diskless system, which again improves reliability of the
 underlying hardware\&.
 .PP 
 This document describes how to use crossroads, how to configure it
 in order to increase the reliability of your systems, and how to
 compile the program from its sources\&. 
 .PP 
 
 .SH "1\&.1: Obtaining Crossroads"
 
 .PP 
 As quick reference, here are some important URL\&'s for Crossroads:
 .PP 
 .IP o 
 http:/crossroads\&.e-tunity\&.com is the site that serves
 Crossroads\&. You can browse this at leisure
 for documentation, sources, and so on\&.
 .IP 
 .IP o 
 http://freshmeat\&.net/projects/crossr is the
 Freshmeat announcement page\&.
 .IP 
 .IP o 
 svn://svn\&.e-tunity\&.com/crossroads is the SVN
 repository; anonymous reading (fetching) is allowed\&. In order
 to commit changes, mail me for
 credentials\&.
 .PP 
 
 .SH "1\&.2: Copyright and Disclaimer"
 
 .PP 
 Crossroads is distributed as-is, without assumptions of fitness
 or usability\&. You are free to use crossroads to your
 liking\&. It\&'s free, and as with everything that\&'s free: there\&'s
 also no warranty\&.
 .PP 
 You are allowed to make modifications to the source code of
 crossroads, and you are allowed to (re)distribute crossroads, as
 long as you include this text, all sources, and if applicable: all
 your modifications, with each distribution\&.
 .PP 
 While you are allowed to make any and all changes to the sources,
 I would appreciate hearing about them\&. If the changes concern new
 functionality or bugfixes, then I\&'ll include them in a next
 release, stating full credits\&. If you want to seriously contribute (to
 which you are heartily encouraged), then mail me and I\&'ll get you
 access to the Crossroads SVN repository, so that you can update and
 commit as you like\&.
 .PP 
 
 .SH "1\&.3: Terminology"
 
 .PP 
 Throughout this document, the following terms are used: \e (Many
 more meanings of the terms will exist -- yes, I am aware of that\&. I\&'m
 using the terms here in a very strict sense\&.)
 .PP 
 .IP "A client"
 is a process that initiates a network connection
 to get contact with some service\&. 
 .IP "A service"
 or \fBserver process\fP or \fBlistener\fP
 is a central application
 that accepts network connections from clients and sevices
 them\&.
 .IP "Back ends"
 are locations where crossroads looks in
 order to service its clients\&. Crossroads sits \&'in between\&'
 and does its tricks\&. Therefore, as far as the back ends
 are concerned, crossroads behaves like a client\&. As far as
 the true client is concerned, crossroads behaves like the
 service\&. The communication is however transparent: neither
 client nor back end are aware of the middle position of
 crossroads\&.
 .IP "A connection"
 is a network conversation between client and service,
 where data are transferred to and fro\&. As
 far as crossroads is concerned, success means that a
 connection can be established without errors on
 the network level\&. Crossroads isn\&'t aware of service
 pecularities\&. E\&.g\&., when a webserver answers \f(CWHTTP/1\&.0
 500 Server Error\fP then crossroads will see this as a
 succesful connection, though the user behind a browser may
 think otherwise\&.
 .IP "Back end selection algorithms"
 are methods by which
 crossroads determines which back end it will talk to
 next\&. Crossroads has a number of built-in algorithms,
 which may be configured per service\&.
 .IP "Back end states"
 are the statusses of each back end that
 is known to crossroads\&. A back end may be available,
 (temporarily) unavailble or truly down\&. When a back end is
 temporarily unavailable, then crossroads will periodically
 check whether the back end has come to life yet (that is,
 if configured so)\&.
 .IP "A spike"
 is a sudden increase in activity, leading to
 extra load on a given service\&. When crossroads is in
 effect and when the spike occurs in one connection,
 then obviously the spike will also appear at one
 of the back ends\&. However, crossroads will see the spike
 and will make sure that a subsequent request goes to an
 other back end\&. In contrast, when several connections
 arrive simultaneously and cause a spike, then crossroads
 will be able to distribute the connections over several
 back ends, thereby \&'flattening out\&' the increase\&.
 .IP "Load balancing"
 means that incoming client requests are
 distributed over more than just one back end (which wouldn\&'t be the
 case if you wouldn\&'t be running crossroads)\&. Enabling load
 balancing is nothing more than duplicating services over
 more than one back end, and having something (in this
 case: crossroads) distribute the requests, so that per
 back end the load doesn\&'t get too high\&.
 .IP "An HTTP session"
 is a series of separate network connections
 that originate from one browser\&. E\&.g\&., to fill the display
 with text and images, the browser hits a website several times\&.
 An HTTP session may even span several
 screens\&. E\&.g\&., a website registration dialog may involve 3
 screens that when called from the same browser, 
 form a logical group of some sort\&.
 .IP "Headers"
 or \fBheader lines\fP are specific parts of an HTTP
 message\&. Crossroads has directives to add or modify
 headers that are part of the request that a browser sends
 to server, or those that are part of the server\&.
 .IP "Session stickiness"
 means that when a browser starts an
 HTTP dialog, the balancer makes sure that it \&'sticks\&' to
 the same back end (i\&.e\&., subsequent requests from the
 browser are forced to go to the same back end, instead of
 being balanced to other ones)\&.
 .IP "Back end usage"
 is measured by crossroads in order to be
 able to determine back end selection\&. Crossroads stores
 information about the number of active connections, the
 transferred bytes and 
 about the connection duration\&. These numbers can be used to
 estimate which back end is the least used -- and
 therefore, presumably, the best candidate for a new
 request\&.
 .IP "Fail over"
 is almost always used when load balancing is in
 effect\&. The distributor of client requests (crossroads of
 course) can also monitor back ends, so that incase a back
 end is \&'down\&', it is no longer accessed\&.
 .IP "Service downtime"
 normally occurs when a service is
 switched off\&. Downtime is obviously avoided when fail over
 is in effect: a back end can be taken out of service in a
 controlled manner, without any client noticing it\&.
 
 .PP 
 
 .SH "1\&.4: Porting issues for pre-1\&.21 installations"
 
 .PP 
 As of version 1\&.21, the event-hook directives \f(CWonsuccess\fP and
 \f(CWonfailure\fP no longer exists\&.
 .PP 
 .IP o 
 Please replace \f(CWonsuccess\fP by \f(CWonstart\fP;
 .IP o 
 Please replace \f(CWonfailure\fP bu \f(CWonfail\fP;
 .IP o 
 Note that there is a new hook \f(CWonend\fP\&.
 .PP 
 The commands that are run via \f(CWonstart\fP, \f(CWonend\fP or \f(CWonfail\fP
     are subject to format expansion; e\&.g\&., \f(CW%1w\fP is expanded to the
     weight of the first back end, etc\&.\&. See section ?? for details\&.
 .PP 
 
 .SH "1\&.5: Porting issues for pre-0\&.26 installations"
 
 .PP 
 As of version 0\&.26 the syntax of the configuration file has
 changed\&. In particular:
 .PP 
 .IP o 
 The keyword \f(CWmaxconnections\fP is now used instead of
 \f(CWmaxclients\fP;
 .IP o 
 The keyword \f(CWconnectiontimeout\fP is now used instead of
 \f(CWsessiontimeout\fP\&.
 .PP 
 Therefore when converting configuration files to the new syntax,
 the above keywords must be changed\&. (The reason for these changes
 is that 0\&.26 introduces \fIsticky HTTP sessions\fP that span
 multiple TCP connections, and the term
 \fIsession\fP is used strictly in that sense -- and no longer for a
 TCP connection\&.)
 .PP 
 
 .SH "1\&.6: Porting issues for pre-1\&.08 installations"
 
 .PP 
 As of version 1\&.08, the following directives no longer are
 supported:
 .PP 
 .IP o 
 \f(CWinsertstickycookie\fP was replaced by the more generic
 directive \f(CWaddclientheader\fP\&. E\&.g\&., instead of 
 .br 
 \f(CWinsertstickycookie "XRID=100; Path=/";\fP 
 .br 
 the syntax is now 
 .br 
 \f(CWaddclientheader "Set-Cookie: XRID=100; Path=/";\fP
 .IP 
 .IP o 
 \f(CWinsertrealip\fP was replaced by the more generic
 directive \f(CWsetserverheader\fP\&. E\&.g\&., instead of 
 .br 
 \f(CWinsertrealip on;\fP 
 .br 
 the syntax is now 
 .br 
 \f(CWsetserverheader "XR-Real-IP: %r";\fP 
 .br 
 This incidentally also makes it possible to change the header
 name (here: \f(CWXR-Real-IP\fP)\&.
 .PP 
 
 .SH "2: Installation for the impatient"
 For the impatient, here\&'s the very-quick-but-very-superficial recipy
 for getting crossroads up and running:
 .PP 
 .IP o 
 If you don\&'t have SVN or don\&'t want to use it:
 .IP 
 .IP o 
 Obtain the crossroads source archive at
 http://crossroads\&.e-tunity\&.com\&.
 .IP 
 .IP o 
 Change-dir to a \&'sources\&' directory on your system and
 unpack the archive\&.
 .IP 
 .IP o 
 Change-dir into the create directory \f(CWcrossroads/\fP\&.
 .IP 
 .IP o 
 If you have SVN and want to go for the newest snapshot:
 .IP 
 .IP o 
 Get the latest sources and snapshots using SVN from 
 .br 
 \f(CWsvn://svn\&.e-tunity\&.com/crossroads\fP\&.
 .IP 
 .IP o 
 You\&'ll find the newest alpha version under
 \f(CWcrossroads/trunk\fP and the stable versions under
 \f(CWcrossroads/tags\fP,
 e\&.g\&. \f(CWcrossroads/tags/release-1\&.00\fP\&.
 .IP 
 .IP o 
 Choose which you want to use: the latest stable
 release, or the bleeding edge alpha? In the former case,
 change-dir to \f(CWcrossroads/tags/release-\fP\fIX\&.YY\fP, where
 \fIX\&.YY\fP is a release ID\&. In the latter case, change-dir to
 \f(CWcrossroads/trunk\fP\&.
 .IP 
 .IP o 
 Type \f(CWmake install\fP\&. This installs the crossroads
 binary into \f(CW/usr/local/bin/\fP\&. If the compilation doesn\&'t
 work on your system, check \f(CWetc/Makefile\&.def\fP for hints\&.
 .IP 
 .IP o 
 Create a file \f(CW/etc/crossroads\&.conf\fP\&. In it state
 something like:
 .IP 
 .nf 
 service www {
     port 80;
     revivinginterval 15;
     backend one {
         server 10\&.1\&.1\&.100:80;
     }
     backend two {
         server 10\&.1\&.1\&.101:80;
     }
 }
 .fi 
 
 .IP 
 That\&'s off course assuming that you want to balance HTTP on
 port 80 to two back ends at 10\&.1\&.1\&.100 and 10\&.1\&.1\&.101\&.
 .IP 
 .IP o 
 Type \f(CWcrossroads start\fP\&.
 .IP 
 .IP o 
 Surf to the machine where crossroads is running\&. You will
 see the pages served by the back ends 10\&.1\&.1\&.100 or
 10\&.1\&.1\&.101\&.
 .IP 
 .IP o 
 To monitor the status of crossroads, type \f(CWcrossroads
 status\fP\&.
 
 .PP 
 
 .SH "3: Using Crossroads"
 Crossroads is started from the commandline, and highly depends on
 \f(CW/etc/crossroads\&.conf\fP (the default configuration file)\&. It 
 supports a number of flags (e\&.g\&., to overrule the location of the
 configuration file)\&. The actual usage information is always obtained
 by typing \f(CWcrossroads\fP without any arguments\&. Crossroads then
 displays the allowed arguments\&.
 .PP 
 
 .SH "3\&.1: General Commandline Syntax"
 
 .PP 
 This section shows the most basic usage\&. As said above, start
 \f(CWcrossroads\fP without arguments to view the full listing of options\&.
 .PP 
 .IP o 
 \f(CWcrossroads start\fP and \f(CWcrossroads stop\fP are typical
 actions that are run from system startup scripts\&. The
 meaning is self-explanatory\&.
 .IP o 
 \f(CWcrossroads restart\fP is a combination of the former
 two\&. Beware that a restart may cause discontinuity in
 service; it is just a shorthand for typing the \&'stop\&' and
 \&'start\&' actions after one another\&.
 .IP o 
 \f(CWcrossroad status\fP reports on each running
 service\&. Per service, the state of each back end is
 reported\&.
 .IP o 
 \f(CWcrossroads tell\fP \fIservice backend state\fP is a
 command line way of telling crossroads that a given back
 end, of a given service, is in a given state\&. Normally
 crossroads maintains state information itself, but by
 using \f(CWcrossroads tell\fP, a back end can be e\&.g\&. taken
 \&'off line\&' for servicing\&.
 .IP o 
 \f(CWcrossroads configtest\fP tells you whether the
 configuration is syntactially correct\&.
 .IP o 
 \f(CWcrossroads services\fP reports on the configured 
 services\&. In contrast to \f(CWcrossroads status\fP, this
 option only shows what\&'s configured -- not what\&'s up and
 running\&. Therefore, \f(CWcrossroads services\fP doesn\&'t
 report on back end states\&.
 .IP o 
 \f(CWcrossroads sampleconf\fP shows a sample configuration on
 screen\&. A good way of quicky viewing the configuration
 file syntax, or of getting a start for your own
 configuration \f(CW/etc/crossroads\&.conf\fP\&.
 
 .PP 
 
 .SH "3\&.2: Logging-related options"
 
 .PP 
 Two \&'flags\&' of Crossroads are specifically logging-related\&. This
 section elaborates on these flags\&.
 .PP 
 First, there\&'s flag \f(CW-a\fP\&. When present, the start and end of
 activity is logged using statements like
 .PP 
 \fIYYYY-MM-DD HH/MM/SS starting http from 61\&.45\&.32\&.189 to 10\&.1\&.1\&.1\fP
 
 .PP 
 Similarly, there are \&'ending\&' statements\&. Using this flag and 
 scanning your logs for these statements may be helpful in quickly
 determining your system load\&.
 .PP 
 Second, there\&'s flag \f(CW-l\fP\&. This flag selects the \&'facility\&' of
 logging and defaults to \f(CWLOG_DAEMON\fP\&. You can supply a number
 between 0 and 7 to flag \f(CW-l\fP to select \f(CWLOG_LOCAL0\fP to
 \f(CWLOG_LOCAL7\fP\&. This would separate the Crossroads-related logging
 from other streams\&. Here\&'s a very short guide; please read your Unix
 manpages of \f(CWsyslogd\fP for more information\&.
 .PP 
 .IP o 
 First edit \f(CW/etc/syslog\&.conf\fP and add a line:
 .IP 
 .nf 
 local7\&.*   /var/log/crossroads\&.log
 .fi 
 
 .IP 
 That instructs \f(CWsyslogd\fP to send \f(CWLOG_LOCAL7\fP requests to the
 logfile \f(CW/var/log/crossroads\&.log\fP\&.
 .IP 
 .IP o 
 Next, restart \f(CWsyslogd\fP\&. On most Unices that\&'s done by
 issuing \f(CWkillall -1 syslogd\fP\&. (As a side-note, I tried this once
 on an Bull/AIX system, and the box just shut down\&. The \f(CWkillall\fP
 command killed every process\&.\&.\&.)
 .IP 
 .IP o 
 Now start \f(CWcrossroads\fP with the flag \f(CW-l7\fP\&.
 .IP 
 .IP o 
 Finally, monitor \f(CW/var/log/crossroads\&.log\fP for Crossroads\&'
 messages\&.
 .PP 
 
 .SH "3\&.3: Reloading Configurations"
 
 .PP 
 Crossroads doesn\&'t support the reloading of a configuration while
 running (such as other programs, e\&.g\&. Apache do)\&. There are various
 technical reasons for this\&.
 .PP 
 However, external lists of allowed or denied IP addresses can be
 reloaded by sending a signal -1 (\f(CWSIGHUP\fP) to Crossroads\&. See
 section ?? for the details\&.
 .PP 
 
 .SH "4: The configuration"
 The configuration that crossroads uses is normally stored in the file
 \f(CW/etc/crossroads\&.conf\fP\&. This location can be overruled using the
 command line flag \f(CW-c\fP\&.
 .PP 
 This section explains the syntax of the configuration file, and what
 all settings do\&.
 .PP 
 
 .SH "4\&.1: General language elements"
 
 .PP 
 This section describes the general elements of the crossroads
 configuration language\&.
 .PP 
 
 .SH "4\&.1\&.1: Empty lines and comments"
 
 .PP 
 Empty lines are of course allowed in the
 configuration\&. Crossroads recognizes three formats of comment:
 .PP 
 .IP o 
 C-style, between \f(CW/*\fP and \f(CW*/\fP,
 .IP o 
 C++-style, starting with \f(CW//\fP and ending with the end
 of the text line;
 .IP o 
 Shell-style, starting with \f(CW#\fP and ending with the end
 of the text line\&.
 .PP 
 Simply choose your favorite editor and use the comment that \&'looks
 best\&'\&.\e (I favor C or C++ comment\&. My favorite editor \fIemacs\fP
 can be put in \f(CWcmode\fP and nicely highlight what\&'s comment and what\&'s
 not\&. And as a bonus it will auto-indent the configuration!)
 .PP 
 
 .SH "4\&.1\&.2: Keywords, numbers, identifiers, generic strings"
 
 .PP 
 In a configuration file, statements are identified by \fIkeywords\fP,
 such as \f(CWservice\fP, \f(CWverbosity\fP\&. These are reserved words\&.
 .PP 
 Many keywords require an \fIidentifier\fP as the argument\&. E\&.g, a
 service has a unique name, which must start with a letter or
 underscore, followed by zero or more letters, underscores, or
 digits\&. Therefore, in the statement \f(CWservice myservice\fP, the keyword is
 \f(CWservice\fP and the identifier is \f(CWmyservice\fP\&.
 .PP 
 Other keywords require a numeric argument\&. Crossroads knows only
 non-negative integer numbers, as in \f(CWport 8000\fP\&. Here, \f(CWport\fP is
 the keyword and \f(CW8000\fP is the number\&.
 .PP 
 Yet other keywords require \&'generic strings\&', such as hostname
 specifications or system commands\&. Such generic strings contain any
 characters (including white space) up to the terminating statement
 character \f(CW;\fP\&. If a string must contain a semicolon, then it must
 be enclosed in single or double quotes:
 .PP 
 .IP o 
 \f(CWThis is a string;\fP is a string that starts at \f(CWT\fP
 and ends with \f(CWg\fP
 .IP o 
 \f(CW"This is a string";\fP is the same, the double quotes
 are not necessary
 .IP o 
 \f(CW"This is ; a string";\fP has double quotes to protect
 the inner ;
 .PP 
 Finally, an argument can be a \&'boolean\&' value\&. Crossroads knows
 \f(CWtrue\fP, \f(CWfalse\fP, \f(CWyes\fP, \f(CWno\fP, \f(CWon\fP, \f(CWoff\fP\&. The keywords
 \f(CWtrue\fP, \f(CWyes\fP and \f(CWon\fP all mean the same and can be used
 interchangeably; as can the keywords \f(CWfalse\fP, \f(CWno\fP and \f(CWoff\fP\&.
 .PP 
 
 .SH "4\&.2: Service definitions"
 
 .PP 
 Service definitions are blocks in the configuration file that
 state what is for each service\&. A service definition starts with
 \f(CWservice\fP, followed by a unique identifier, and by statements in
 \f(CW{\fP and \f(CW}\fP\&. For example:
 .PP 
 .nf 
 // Definition of service \&'www\&':
 service www {
     \&.\&.\&.
     \&.\&.\&. // statements that define the
     \&.\&.\&. // service named \&'www\&'
     \&.\&.\&.
 }
 .fi 
 
 .PP 
 The configuration file can contain many service blocks, as long as the
 identifying names differ\&. The following list shows possible
 statements\&. Each statement must end with a semicolon, except for the
 \f(CWbackend\fP statement, which has is own block (more on this later)\&.
 .PP 
 
 .SH "4\&.2\&.1: type - Defining the service type"
 .IP "Description:"
 The \f(CWtype\fP statement defines how crossroads handles the stated
 service\&. There are currently two types: \f(CWany\fP and
 \f(CWhttp\fP\&. The type \f(CWany\fP means that crossroads doesn\&'t
 interpret the contents of a TCP stream, but only distributes streams
 over back ends\&. The type \f(CWhttp\fP means that crossroads has to
 analyze what\&'s in the messages, does magical HTTP header tricks, and
 so on -- all to ensure that multiple connections are treated as one
 session, or that the back end is notified of the client\&'s IP
 address\&.
 .IP 
 Unless you really need such special features, use the type \f(CWany\fP (the
 default), even for HTTP protocols\&.
 .IP "Syntax:"
 \f(CWtype\fP \fIspecifier\fP, where \fIspecifier\fP is \f(CWany\fP or
 \f(CWhttp\fP
 .IP "Default:"
 \f(CWany\fP
 
 .PP 
 
 .SH "4\&.2\&.2: port - Specifying the listen port"
 .IP "Description:"
 The \f(CWport\fP statement defines to which TCP port a service
 \&'listens\&'\&. E\&.g\&. \f(CWport 8000\fP says that this service will accept
 connections on port 8000\&.
 .IP "Syntax:"
 \f(CWport\fP \fInumber\fP
 .IP "Default:"
 There is no default\&. This is a required setting\&.
 
 .PP 
 
 .SH "4\&.2\&.3: bindto - Binding to a specific IP address"
 .IP "Description:"
 The \f(CWbindto\fP statement is used in situations where crossroads
 should only listen to the stated port at a given IP address\&. E\&.g\&.,
 \f(CWbindto 127\&.0\&.0\&.1\fP causes crossroads to \&'bind\&' the service only to
 the local IP address\&. Network connections from other hosts won\&'t be
 serviced\&. By default, crossroads binds a service to all presently
 active IP addresses at the invoking host\&.
 .IP "Syntax:"
 \f(CWbindto\fP \fIaddress\fP, where \fIaddress\fP is a numeric IP
 address, such as 127\&.0\&.0\&.1, or the keyword \f(CWany\fP\&.
 .IP "Default:"
 \f(CWany\fP
 
 .PP 
 
 .SH "4\&.2\&.4: verbosity - Controlling debug output"
 .IP "Description:"
 Verbosity statements  come in two forms: \f(CWverbosity on\fP or
 \f(CWverbosity off\fP\&. When \&'on\&', log messages to \f(CW/var/log/messages\fP
 are generated that show what\&'s going on\&.\e (Actually, the
 messages go to \f(CWsyslog(3)\fP, using facility \f(CWLOG_DAEMON\fP and
 priority \f(CWLOG_INFO\fP\&. In most (Linux) cases this will mean: output to
 \f(CW/var/log/messages\fP\&. On Mac OSX the messages go to
 \f(CW/var/log/system\&.log\fP\&.) The keyword \f(CWverbose\fP is an alias for
 \f(CWverbosity\fP\&.
 .IP "Syntax:"
 \f(CWverbosity\fP \fIsetting\fP or \f(CWverbose\fP \fIsetting\fP, where
 \fIsetting\fP is \f(CWtrue\fP, \f(CWyes\fP or \f(CWon\fP to turn 
 verbosity on; or \f(CWfalse\fP, \f(CWno\fP, \f(CWoff\fP to turn it off\&.
 .IP "Default:"
 \f(CWoff\fP
 
 .PP 
 
 .SH "4\&.2\&.5: dispatchmode - How are back ends selected"
 .IP "Description:"
  The dispatch mode controls how crossroads selects a back end from
      a list of active back ends\&. The below text shows the bare
      syntax\&. See section ?? for a textual explanation\&.
 .IP 
 The settings can be:
 .IP 
 .IP o 
 \f(CWdispatchmode roundrobin\fP: Simply the \&'next in line\&' is
 chosen\&. E\&.g, when 3 back ends are active, then the usage
 series is 1, 2, 3, 1, 2, 3, and so on\&.
 .IP 
 Roundrobin dispatching is the default method, when no
 \f(CWdispatchmode\fP statement occurs\&.
 .IP 
 .IP o 
 \f(CWdispatchmode random\fP: Random selection\&. Probably only
 for stress testing, though when used with weights (see below)
 it is a good distributor of new connections too\&.
 .IP 
 .IP o 
 \f(CWdispatchmode bysize [ over\fP \fIconnections\fP \f(CW]\fP:
 The next back end is the one
 that has transferred the least number of bytes\&. This
 selection mechanism assumes that the more bytes, the heavier
 the load\&.
 .IP 
 The modifier \f(CWover\fP \fIconnections\fP is optional\&. (The square
 brackets shown above are not part of the statement but
 indicate optionality\&.)  When given,
 the load is computed as an average of the last stated number of
 connections\&. When this modifier is absent, then the load is
 computed over all connections since startup\&.
 .IP 
 .IP o 
 \f(CWdispatchmode byduration [ over\fP \fIconnections\fP \f(CW]\fP:
 The next back end is the one
 that served connections for the shortest time\&. This mechanism
 assumes that the longer the connection, the heavier the load\&.
 .IP 
 .IP o 
 \f(CWdispatchmode byconnections\fP: The next back end is the one
 with the least active connections\&. This mechanism assumes that
 each connection to a back end represents load\&. It is usable
 for e\&.g\&. database connections\&.
 .IP 
 .IP o 
 \f(CWdispatchmode byorder\fP: The first back end is selected
 every time, unless it\&'s unavailable\&. In that case the second
 is taken, and so on\&.
 .IP 
 .IP o 
 \f(CWdispatchmode externalhandler\fP \fIprogram arguments\fP:
 This is a special mode, where an external program is delegated
 the responsibility to say which back end should be used
 next\&. In this case, Crossroads will call the external program,
 and this will of course be slower than one of the \&'built-in\&'
 dispatch modes\&. However, this is the ultimate escape when
 custom-made dispatch modes are needed\&.
 .IP 
 The dispatch mode that uses an \f(CWexternalhandler\fP is
         discussed separately in section ??\&.
 .IP 
 The selection algorithm is only used when clients are serviced that
 aren\&'t part of a sticky HTTP session\&. This is the case during:
 .IP 
 .IP o 
 all client requests of a service type \f(CWany\fP;
 .IP o 
 new sessions of a service type \f(CWhttp\fP\&.
 .IP 
 When type \f(CWhttp\fP is in effect and a session is underway, then the
 previously used back end is always selected -- regardless of
 dispatching mode\&.
 .IP 
 Your \&'right\&' dispatch mode will depend on the type of service\&. Given
 the fact that crossroads doesn\&'t know (and doesn\&'t care) how to
 estimate load from a network traffic stream, you have to choose an
 appropriate dispatch mode to optimize load balancing\&. In most cases,
 \f(CWroundrobin\fP or \f(CWbyconnections\fP will do the job just fine\&.
 .IP "Syntax:"
 \f(CWdispatchmode\fP \fImode\fP (see above for the modes), optionally
 followed by \f(CWover\fP \fInumber\fP, or when the \fImode\fP is
 \f(CWexternalhandler\fP, followed by \fIprogram\fP\&.
 .IP "Default:"
 \f(CWroundrobin\fP
 
 .PP 
 
 .SH "4\&.2\&.6: revivinginterval - Back end wakeup calls"
 .IP "Description:"
 A reviving interval definition is needed when crossroads
 determines that a back end is temporarily unavailable\&. This will
 happen when: 
 .IP 
 .IP o 
 The back end cannot be reached (network connection
 fails);
 .IP o 
 The network connection to the back end suddenly dies\&.
 .IP 
 An example of the definition is \f(CWrevivinginterval 10\fP\&. When this
 reviving interval is given, crossroads will check each 10 seconds
 whether unavailable back ends have woken up yet\&. A back end is
 considered awake when a network connection to that back end can
 succesfully be established\&.
 .IP "Syntax:"
 \f(CWrevivinginterval\fP \fInumber\fP, where the number is the interval
 in seconds\&.
 .IP "Default:"
 0 (no wakeup calls)
 
 .PP 
 
 .SH "4\&.2\&.7: maxconnections - Limiting concurrent clients at service level"
 .IP "Description:"
 The maximum number of connections is specified using
 \f(CWmaxconnections\fP\&. There is one argument; the number of concurrent
 established connections that may be active within one service\&.
 .IP 
 \&'Throttling\&' the number of connections is a way of preventing Denial of
 Service (DOS) attacks\&. Without a limit, numerous network connections
 may spawn so many server instances, that the service ultimately breaks
 down and becomes unavailable\&.
 .IP "Syntax:"
 \f(CWmaxconnections\fP \fInumber\fP, where the number specifies the
 maximum of concurrent connections to the service\&.
 .IP "Default:"
 0, meaning that all connections will be accepted\&.
 
 .PP 
 
 .SH "4\&.2\&.8: backlog - The TCP Back Log size"
 .IP "Description:"
 The TCP back log size is a number that controls how many
 \&'waiting\&' network connections may be queued, before a client simply
 cannot connect\&. The syntax is e\&.g\&. \f(CWbacklog 5\fP to cause crossroads
 to have 5 waiting connections for 1 active connection\&.
 The backlog queue shouldn\&'t be too
 high, or clients will experience timeouts before they can actually
 connect\&. The queue shouldn\&'t be too small either, because clients
 would be simply rejected\&. Your mileage may vary\&.
 .IP "Syntax:"
 \f(CWbacklog\fP \fInumber\fP
 .IP "Default:"
 0, which takes the operating system\&'s default
 value for socket back log size\&.
 
 .PP 
 
 .SH "4\&.2\&.9: shmkey - Shared Memory Access"
 .IP "Description:"
 Different Crossroads
 invocations must \&'know\&' of each others activity\&. E\&.g, \f(CWcrossroad
 status\fP must be able to get to the actual state information of all
 running services\&. This is internally implemented through shared
 memory, which is reserved using a key\&.
 .IP 
 Normally crossroads will supply a shared memory key, based on the
 service port and bitwise or-ed with a magic number\&. In situations
 where this conflicts with existing keys (of other programs, having
 their own keys), you may supply a chosen value\&.
 .IP 
 The actual key value doesn\&'t matter much, as long as it\&'s unique
 and as long as each invocation of crossroads uses it\&.
 .IP "Syntax:"
 \f(CWshmkey\fP \fInumber\fP
 .IP "Default:"
 0, which means that crossroads will \&'guess\&' its
 own key, based on TCP port and a magic number\&.
 
 .PP 
 
 .SH "4\&.2\&.10: allow* and deny* - Allowing or denying connections"
 .IP "Description:"
 Crossroads can allow or deny
 connections based on the IP address of a client\&. There are four
 directives that are relevant: \f(CWallowfrom\fP, \f(CWallowfile\fP,
 \f(CWdenyfrom\fP and \f(CWdenyfile\fP\&. When using \f(CWallowfrom\fP and
 \f(CWdenyfrom\fP then the IP addresses to allow or deny connections are
 stated in \f(CW/etc/crossroads\&.conf\fP\&.
 .IP 
 When \f(CWallow*\fP directives are used, then all connections are denied
 unless they match the stated allowed IP\&'s\&. When \f(CWdeny*\fP directives
 are used, then all connections are allowed unless they match the
 stated disallowed IP\&'s\&. When denying and allowing is both used,
 then the Crossroads checks the deny list first\&.
 .IP 
 The statements \f(CWallowfrom\fP and \f(CWdenyfrom\fP are followed by a
 list of filter specifications\&. The statements \f(CWallowfile\fP and
 \f(CWdenyfile\fP are followed by a filename; Crossroads will read
 filter specifications from those external files\&. In both cases,
 Crossroads obtains filter specifications and places them in its
 lists of allowed or denied IP addresses\&. The difference between
 specifying filters in \f(CW/etc/crossroads\&.conf\fP or in external
 files, is that Crossroads will reload the external files when it
 receives signal 1 (\f(CWSIGHUP\fP), as in \f(CWkillall -1 crossroads\fP\&.
 .IP 
 The filter specifications must obey the following syntax: it
 consists of up to 
 four numbers ranging from 0 to 255 and separated by a decimal
 sign\&. Optionally a slash follows, with a bitmask which is also a
 decimal number\&.
 .IP 
 This is probably best explained by a few examples:
 .IP 
 .IP o 
 \f(CWallowfrom 10/8;\fP will allow connections from
 \f(CW10\&.*\&.*\&.*\fP (a full Class A network)\&. The mask \f(CW/8\fP means
 that the first 8 bits of the number (ie\&., only the \f(CW10\fP) are
 significant\&. On the last 3 positions of the IP address, all
 numbers are allowed\&. Given this directive, client connections
 from e\&.g\&. 10\&.1\&.1\&.1 and 10\&.2\&.3\&.4 will be allowed\&.
 .IP 
 .IP o 
 \f(CWallowfrom 10\&.3/16;\fP will allow all IP addresses that
 start with \f(CW10\&.3\fP\&.
 .IP 
 .IP o 
 \f(CWallowfrom 10\&.3\&.1/16;\fP is the same as above\&. The third
 byte of the IP address is superfluous because the netmask
 specifies that only the first 16 bits (2 numbers) are taken
 into account\&.
 .IP 
 .IP o 
 \f(CWallowfrom 10\&.3\&.1\&.15;\fP allows traffic from only the
 specified IP address\&. There is no bitmask; all four numbers
 are relevant\&.
 .IP 
 .IP o 
 \f(CWallowfrom 10\&.3\&.1\&.15 10\&.2/16;\fP allows traffic from one
 IP address \f(CW10\&.3\&.1\&.15\fP or from a complete Class B network
 \f(CW10\&.2\&.*\&.*\fP 
 .IP 
 .IP o 
 \f(CWallowfile /tmp/myfile\&.txt;\fP in combination with a file
 \f(CW/tmp/myfile\&.txt\fP, with the contents \f(CW10\&.3\&.1\&.15 10\&.2/16\fP,
 is the same as above\&.
 .IP "Syntax:"
 .IP o 
 \f(CWallowfrom\fP \fIfilter-specificication(s)\fP
 .IP o 
 \f(CWdenyfrom\fP  \fIfilter-specificication(s)\fP
 .IP o 
 \f(CWallowfile\fP  \fIfilename\fP
 .IP o 
 \f(CWdenyfile\fP  \fIfilename\fP
 .IP "Default:"
 In absence of these statements, all client IP\&'s are accepted\&.
 
 .PP 
 
 .SH "4\&.2\&.11: useraccount - Limiting the effective ID of external processes"
 .IP "Description:"
 Using the directive \f(CWuseraccount\fP, the effective user and group
 ID can be restricted\&. This comes into effect when Crossroads runs
 external commands, such as:
 .IP o 
 Hooks for \f(CWonstart\fP, \f(CWonend\fP or \f(CWonfail\fP;
 .IP o 
 External dispatchers, when \f(CWdispatchmode
 externalhandler\fP is in effect\&.
 Once a user name for external commands is specified, Crossroads
 assumes the associated user ID and group ID before running those
 commands\&.
 .IP "Syntax:"
 \f(CWuseraccount\fP \fIusername\fP
 .IP "Default:"
 None; when unspecified, external commands are run with the 
 ID that was in effect when Crossroads was started\&.
 
 .PP 
 
 .SH "4\&.3: Backend definitions"
 
 .PP 
 Inside the service definitions as are described in the previous
 section, \fIbackend definitions\fP must also occur\&. Backend definitions
 are started by the keyword \f(CWbackend\fP, followed by an identifier
 (the back end name) , and statements inside \f(CW{\fP and \f(CW}\fP:
 .PP 
 .nf 
 service myservice {
     \&.\&.\&.
     \&.\&.\&. // statements that define the
     \&.\&.\&. // service named \&'myservice\&'
     \&.\&.\&.
 
     backend mybackend {
         \&.\&.\&.
         \&.\&.\&. // statements that define the
         \&.\&.\&. // backend named \&'mybackend\&'
         \&.\&.\&.
     }
 }
 .fi 
 
 .PP 
 Each service definition must have at least one backend
 definition\&. There may be more (and probably will, if you want
 balancing and fail over) as long as the backend names differ\&.
 The statements in the backend definition blocks are described in the
 following sections\&.
 .PP 
 Some directives (\f(CWstickycookie\fP etc\&.) only have effect when
 Crossroads treats the network traffic as a stream of HTTP messages;
 i\&.e\&., when the service is declared with \f(CWtype http\fP\&. Incase of
 \f(CWtype any\fP, the HTTP-specific directives have no effect\&.
 .PP 
 
 .SH "4\&.3\&.1: server - Specifying the back end address"
 .IP "Description:"
 Each back end must be identified by the network name
 (server name) where it is located\&. For example: \f(CWserver
 10\&.1\&.1\&.23\fP, or \f(CWserver web\&.mydomain\&.org\fP\&. A TCP port specifier
 can follow the server name, as in \f(CWserver web\&.mydomain\&.org:80\fP\&.
 .IP "Syntax:"
 .IP o 
 \f(CWserver\fP \fIservername\fP, where \fIservername\fP is a
 network name or IP address;
 .IP o 
 \f(CWserver\fP \fIservername:port\fP
 .IP "Default:"
 There is no default\&. This is a required setting\&.
 
 .PP 
 
 .SH "4\&.3\&.2: verbosity - Controlling verbosity at the back end level"
 .IP "Description:"
 Similar to \f(CWservice\fP specifications, a
 \f(CWbackend\fP can have its own verbosity (\f(CWon\fP or \f(CWoff\fP)\&. When
 \f(CWon\fP, traffic to and fro this back end is reported\&.
 .IP "Syntax:"
 .IP o 
 \f(CWverbosity\fP \fIsetting\fP, or
 .IP o 
 \f(CWverbose\fP \fIsetting\fP, where \fIsetting\fP is \f(CWtrue\fP,
 \f(CWyes\fP or \f(CWon\fP, or \f(CWfalse\fP, \f(CWno\fP, \f(CWoff\fP to turn it
 off\&.
 .IP "Default:"
 \f(CWoff\fP
 
 .PP 
 
 .SH "4\&.3\&.3: weight - When a back end is more equal than others"
 .IP "Description:"
 To influence how backends are selected, a backend can specify its
 \&'weight\&' in the process\&. The higher the weight, the less likely a
 back end will be chosen\&. The default is 1\&.
 .IP 
 The weighing mechanism only applies to the dispatch modes
 \f(CWrandom\fP, \f(CWbyconnections\fP, \f(CWbysize\fP and \f(CWbyduration\fP\&. 
 The weight is in fact a penalty factor\&. E\&.g\&., if backend A has
 \f(CWweight 2\fP and backend B has \f(CWweight 1\fP, then backend B will
 be selected all the time, until its usage parameter is twice as
 large as the parameter of A\&. Think of it as a \&'sluggishness\&'
 statement\&.
 .IP "Syntax:"
 \f(CWweight\fP \fInumber\fP; the higher the number, the more \&'sluggish\&'
 a back end is
 .IP "Default:"
 1; all back ends have equal weight\&.
 
 .PP 
 
 .SH "4\&.3\&.4: decay - Levelling out activity of a back end"
 .IP "Description:"
 To make sure that a \&'spike\&' of activity doesn\&'t
 influence the perceived load of a back end forever, you may
 specify a certain decay\&. E\&.g, the statement \f(CWdecay 10\fP makes
 sure that the load that crossroads computes for this back end (be
 it in seconds or in bytes) is decreased by 10% each time that
 \fBan other\fP back end is hit\&. Decays are not applied to the count
 of concurrent connections\&.
 .IP 
 This means that when a given back end is hit, then its usage data
 of the transferred bytes and the connection duration are updated
 using the actual number of bytes and actual duration\&. However,
 when a different back end is hit, then the usage data are
 decreased by the specified decay\&. 
 .IP "Syntax:"
 \f(CWdecay\fP \fInumber\fP, where \fInumber\fP is a percentage that
 decreases the back end usage data when other back ends are
 hit\&.
 .IP "Default:"
 0, meaning that no decay is applied to usage statistics\&.
 
 .PP 
 
 .SH "4\&.3\&.5: onstart, onend, onfail - Action Hooks"
 .IP "Description:"
 The three directives \f(CWonstart\fP, \f(CWonend\fP and \f(CWonfail\fP can be
 specified to start system commands (external programs) when a
 connection to a back end starts, fails or ends:
 .IP o 
 \f(CWonstart\fP commands will be run when Crossroads
 successfully connects to a back end, and starts servicing;
 .IP o 
 \f(CWonend\fP commands will be run when a (previously
 established) connection stops;
 .IP o 
 \f(CWonfail\fP commands will be run when Crossroads tries to
 contact a back end to serve a client, but the back end can\&'t
 be reached\&.
 .IP 
 The format is always \f(CWon\fP\fItype\fP \fIcommand\fP\&. The \fIcommand\fP
 is an external program, optionally followed by arguments\&. The
 command is expanded according to the following table:
 .IP 
 .IP o 
 \f(CW%a\fP is the availability of the current back end, when
 a current back end is established;
 .IP o 
 \f(CW%1a\fP is the availability of the first back end (0 when
 unavailable, 1 if available); \f(CW%2a\fP is the availability of
 the second back end, and so on;
 .IP o 
 \f(CW%b\fP is the name of the current back end, when one is
 established; 
 .IP o 
 \f(CW%1b\fP is the name of the first back end, \f(CW%2b\fP of the
 second back end, and so on;
 .IP o 
 \f(CW%e\fP is the count of seconds since start of epoch
 (January 1st 1970 GMT);
 .IP o 
 \f(CW%r\fP is the IP address of the client that requests a
 connection and for whom the external dispatcher should compute
 a back end;
 .IP o 
 \f(CW%s\fP is the name of the current service that the client
 connected to;
 .IP o 
 \f(CW%t\fP is the current local time in ASCII format, in
 \fIYYYY-MM-DD/hhh:mm:ss\fP; 
 .IP o 
 \f(CW%T\fP is the current GMT time in ASCIII format;
 .IP o 
 \f(CW%v\fP is the Crossroads version;
 .IP o 
 Any other chararacter following a \f(CW%\fP sign is taken
 literally; e\&.g\&. \f(CW%z\fP is just a z\&.
 .IP 
 .IP "Syntax:"
 .IP o 
 \f(CWonstart\fP \fIcommandline\fP
 .IP o 
 \f(CWonend\fP \fIcommandline\fP
 .IP o 
 \f(CWonfail\fP \fIcommandline\fP
 .IP o 
 \f(CWonsuccess\fP \fIcommandline\fP
 .IP "Default:"
 There is no default\&. Normally no external programs are run upon
 connection, success or failure of a back end\&.
 
 .PP 
 
 .SH "4\&.3\&.6: trafficlog and throughputlog - Debugging and Performance Aids"
 .IP "Description:"
 Two directives are available
 to log network traffic to files\&. They are \f(CWtrafficlog\fP and
 \f(CWthroughputlog\fP\&.
 .IP 
 The \f(CWtrafficlog\fP statement causes all traffic to be logged in
 hexadecimal format\&. Each line is prefixed by \f(CWB\fP or \f(CWC\fP,
 depending on whether the information was received from the back
 end or from the client\&.
 .IP 
 The \f(CWthroughputlog\fP statement writes shorthand transmissions to
 its log, accompanied by timings\&.
 .IP "Syntax:"
 .IP o 
 \f(CWtrafficlog\fP \fIfilename\fP
 .IP o 
 \f(CWthroughputlog\fP \fIfilename\fP
 .IP "Default:"
 none
 
 .PP 
 
 .SH "4\&.3\&.7: stickycookie - Back end selection with an HTTP cookie"
 .IP "Description:"
 The directive \f(CWstickycookie\fP \fIvalue\fP
 causes Crossroads to unpack clients\&' requests, to check for
 \fIvalue\fP in the cookies\&. When found, the message is routed to the
 back end having the appropriate \f(CWstickycookie\fP directive\&.
 .IP 
 E\&.g\&., consider the following configuration:
 .IP 
 .nf 
 service \&.\&.\&. {
     \&.\&.\&.
     backend one {
         \&.\&.\&.
         stickycookie "BalancerID=first";
     }
     backend two {
         \&.\&.\&.
         stickycookie "BalancerID=second";
     }
 }
 .fi 
 
 .IP 
 When clients\&' messages contain cookies named \f(CWBalancerID\fP with
 the value \f(CWfirst\fP, then such messages are routed to backend
 \f(CWone\fP\&. When the value is \f(CWsecond\fP then they are routed to the
 backend \f(CWtwo\fP\&.
 .IP 
 There are basically to provide such cookies to a browser\&. First, a
 back end can insert such a cookie into the HTTP response\&. E\&.g\&.,
 the webserver of back end \f(CWone\fP might insert a cookie named
 \f(CWBalancerID\fP, having value \f(CWfirst\fP\&.
 Second, Crossroads can insert such cookies using a carefully
 crafted directive \f(CWaddclientheader\fP\&.
 .IP "Syntax:"
 \f(CWstickycookie\fP \fIcookievalue\fP
 .IP "Default:"
 There is no default\&.
 
 .PP 
 
 .SH "4\&.3\&.8: HTTP Header Modification Directives"
 .IP "Description:"
 Crossroads understands the following
 header modification directives: \f(CWaddclientheader\fP,
 \f(CWappendclientheader\fP, \f(CWsetclientheader\fP, \f(CWaddserverheader\fP,
 \f(CWappendserverheader\fP, \f(CWsetserverheader\fP\&.
 .IP 
 The directive names always consist of
 \fIAction\fP\fIDestination\fP\f(CWheader\fP, where:
 .IP 
 .IP o 
 The action is \f(CWadd\fP, \f(CWappend\fP or \f(CWinsert\fP\&.
 .IP 
 .IP o 
 Action \f(CWadd\fP adds a header, even when headers with
 the same name already are present in an HTTP
 message\&. Adding headers is useful for e\&.g\&. \f(CWSet-Cookie\fP
 headers; a message may contain several of such headers\&.
 .IP 
 .IP o 
 Action \f(CWappend\fP adds a header if it isn\&'t present
 yet in an HTTP message\&. If such a header is already
 present, then the value is appended to the pre-existing
 header\&. This is useful for e\&.g\&. \f(CWVia\fP headers\&. Imagine
 an HTTP message with a header \f(CWVia: someproxy\fP\&. Then the
 directive \f(CWappendclientheader "Via: crossroads"\fP will
 rewrite the header to \f(CWVia: someproxy; crossroads\fP\&.
 .IP 
 .IP o 
 Action \f(CWset\fP overwrites headers with the same
 name; or adds a new header if no pre-existing is found\&.
 This is useful for e\&.g\&. \f(CWHost\fP headers\&.
 .IP 
 .IP o 
 The destination is one of \f(CWclient\fP or \f(CWserver\fP\&. When
 the destination is \f(CWserver\fP, then Crossroads will apply such
 directives to HTTP messages that originate from the browser
 and are being forwarded to back ends\&. When the destination is
 \f(CWclient\fP, then Crossroads will apply such directives to
 backend responses that are shuttled to the browser\&.
 .IP 
 The format of the directives is e\&.g\&. \f(CWaddclientheader
 "X-Processed-By: Crossroads"\fP\&. The directives expect one
 argument; a string, consisting of a header name, a colon, and a
 header value\&. As usual, the directive must end with a semicolon\&.
 .IP 
 The header value may contain one of the following formatting
 directives:
 .IP 
 .IP o 
 \f(CW%a\fP is the availability of the current back end, when
 a current back end is established;
 .IP o 
 \f(CW%1a\fP is the availability of the first back end (0 when
 unavailable, 1 if available); \f(CW%2a\fP is the availability of
 the second back end, and so on;
 .IP o 
 \f(CW%b\fP is the name of the current back end, when one is
 established; 
 .IP o 
 \f(CW%1b\fP is the name of the first back end, \f(CW%2b\fP of the
 second back end, and so on;
 .IP o 
 \f(CW%e\fP is the count of seconds since start of epoch
 (January 1st 1970 GMT);
 .IP o 
 \f(CW%r\fP is the IP address of the client that requests a
 connection and for whom the external dispatcher should compute
 a back end;
 .IP o 
 \f(CW%s\fP is the name of the current service that the client
 connected to;
 .IP o 
 \f(CW%t\fP is the current local time in ASCII format, in
 \fIYYYY-MM-DD/hhh:mm:ss\fP; 
 .IP o 
 \f(CW%T\fP is the current GMT time in ASCIII format;
 .IP o 
 \f(CW%v\fP is the Crossroads version;
 .IP o 
 Any other chararacter following a \f(CW%\fP sign is taken
 literally; e\&.g\&. \f(CW%z\fP is just a z\&.
 .IP 
 The following examples show common uses of header modifications\&.
 .IP 
 .IP "Enforcing session stickiness:"
 By combining
 \f(CWstickycookie\fP and \f(CWaddclientheader\fP, HTTP session
 stickiness is enforced\&. Consider the following configuration:
 .IP 
 .nf 
 service \&.\&.\&. {
     \&.\&.\&.
     backend one {
         \&.\&.\&.
         addclientheader "Set-Cookie: BalancerID=first; path=/";
         stickycookie "BalancerID=first";
     }
     backend two {
         \&.\&.\&.
         addclientheader "Set-Cookie: BalancerID=second; path=/";
         stickycookie "BalancerID=second";
     }
 }
 .fi 
 
 .IP 
 The first request of an HTTP session is balanced to either
 backend \f(CWone\fP or \f(CWtwo\fP\&. The server response is enriched
 using \f(CWaddclientheader\fP with an appropriate cookie\&. A
 subsequent request from the same browser now has that cookie
 in place; and is therefore sent to the same back end where the
 its predecessors went\&.
 .IP 
 .IP "Hiding the server software version:"
 Many servers
 (e\&.g\&. Apache) advertize their version, as in \f(CWServer: Apache
 1\&.27\fP\&. This potentially provides information to attackers\&. The
 following configuration hides such information:
 .IP 
 .nf 
 service \&.\&.\&. {
     \&.\&.\&.
     backend one {
         \&.\&.\&.
         setclientheader "Server: WWW-Server";
     }
 }
 .fi 
 
 .IP 
 .IP "Informing the server of the clients\&' IP address:"
 Since
 Crossroads sits \&'in the middle\&' between a client and a back
 end, the back end perceives Crossroads as its client\&. The
 following sends the true clients\&' IP address to the server, in
 a header \f(CWX-Real-IP\fP:
 .IP 
 .nf 
 service \&.\&.\&. {
     \&.\&.\&.
     backend one {
         \&.\&.\&.
         setserverheader "X-Real-IP: %r";
     }
 }
 .fi 
 
 .IP 
 .IP "Keep-Alive Downgrading:"
 The directives
 \f(CWsetclientheader\fP and \f(CWsetserverheader\fP also play a key
 role in downgrading Keep-Alive connections to
 \&'single-shot\&'\&. E\&.g\&., the following configuration makes sure
 that no Keep-Alive connections occur\&.
 .IP 
 .nf 
 service \&.\&.\&. {
     \&.\&.\&.
     backend one {
         \&.\&.\&.
         setserverheader "Connection: close";
         setclientheader "Connection: close";
     }
 }
 .fi 
 .IP "Syntax:"
 .IP o 
 \f(CWaddclientheader\fP \fIHeadername: headervalue\fP to add a
 header in the traffic towards the client, even when another
 header \fIHeadername\fP exists;
 .IP o 
 \f(CWappendclientheader\fP \fIHeadername: headervalue\fP to
 append \fIheadervalue\fP to an existing header \fIHeadername\fP
 in the traffic towards the client,
 or to add the whole header alltogether;
 .IP o 
 \f(CWsetclientheader\fP \fIHeadername: headervalue\fP to
 overwrite an existing header in the traffic towards the
 client, or to add such a header;
 .IP o 
 \f(CWaddserverheader\fP \fIHeadername: headervalue\fP to add a
 header in the traffic towards the server, even when another
 header \fIHeadername\fP exists;
 .IP o 
 \f(CWappendserverheader\fP \fIHeadername: headervalue\fP to
 append \fIheadervalue\fP to an existing header \fIHeadername\fP
 in the traffic towards the server,
 or to add the whole header alltogether;
 .IP o 
 \f(CWsetserverheader\fP \fIHeadername: headervalue\fP to
 overwrite an existing header in the traffic towards the
 server, or to add such a header\&.
 .IP "Default:"
 There is no default\&.
 
 .PP 
 
 .SH "5: Tips, Tricks and Remarks"
 
 The following sections elaborate on the directives as described in
 section ?? to illustrate how crossroads works and to help you
 achieve the "optimal" balancing configuration\&.
 .PP 
 
 .SH "5\&.1: How back ends are selected in load balancing"
 
 .PP 
 In order to tune your load balancing, you\&'ll need to understand how
 crossroads computes usage, how weighing works, and so on\&. In this
 section we\&'ll focus on the dispatching modes \f(CWbysize\fP, \f(CWbyduration\fP
 and \f(CWbyconnections\fP only\&. The other dispatching types are
 self-explanatory\&. 
 .PP 
 
 .SH "5\&.1\&.1: Bysize, byduration or byconnections?"
 
 .PP 
 As stated before, crossroads doesn\&'t know \&'what a service does\&' and
 how to judge whether a given back end is very busy or not\&. You
 must therefore give the right hints:
 .PP 
 .IP o 
 In general, a service which is CPU bound, will be more
 busy when it takes longer to process a request\&. The dispatch
 mode \f(CWbyduration\fP is appropriate here\&.
 .IP 
 .IP o 
 In contrast, a service which is filesystem bound, will be
 more busy when more data are transferred\&. The dispatch mode
 \f(CWbysize\fP is apppropriate\&.
 .IP 
 .IP o 
 The dispatch mode \f(CWbyduration\fP can also be used when
 network latency is an issue\&. E\&.g\&., if your balancer has back
 ends that are geograpically distributed, then \f(CWbyduration\fP
 would be a good way to select best available back ends\&.
 .IP 
 .IP o 
 Furthermore it is noteworthy that \f(CWdispatchmode
 byduration\fP is not usable for interactive processes such as
 SSH logins\&. Idle time of a
 login adds to the duration, while causing (almost) no
 load\&. Mode \f(CWbyduration\fP should only be used for automated
 processes that don\&'t wait for user interaction (e\&.g\&., SOAP
 calls and other HTTP requests)\&.
 .IP 
 .IP o 
 As a last remark, the dispatching mode \f(CWbyconnections\fP can
 be used if you don\&'t have other clues for load
 estimations\&.
 .IP 
 E\&.g\&., consider a database connection\&. What\&'s
 heavier on the back end, time-consuming connections, or connections
 where loads of bytes are transferred? Well, that depends\&. A
 tough \f(CWselect\fP query that joins multiple tables can be very
 heavy on the back end, though the response set can be quite
 small - and hence the number of
 transferred bytes\&. That would suggest
 dispatching by duration\&. However, \f(CWbyduration\fP
 balancing doesn\&'t respresent the true world, when interactive
 connections can occur where users have an idle TCP connection to
 the database:
 this consumes time, but no bytes (see the SSH login example
 above)\&. In this case, the dispatch mode \f(CWbyconnections\fP may be
 your best bet\&.
 .IP 
  
 .SH "5\&.1\&.2: Averaging size and duration"
 
 .PP 
 The configuration statement \f(CWdispatchmode bysize\fP or \f(CWbyduration\fP
 allows an optional modifier \f(CWover\fP \fInumber\fP, where the stated
 number represents a connection count\&. When this modifier is present, then
 crossroads will use a moving average over the last \fIn\fP connections to
 compute duration and size figures\&.
 .PP 
 In the real world you\&'ll always want this modifier\&. E\&.g\&., consider two
 back ends that are running for years now, and one of them is suddenly
 overloaded and very busy (it experiences a \&'spike\&' in activity)\&.
 When the \f(CWover\fP modifier is absent, then
 the sudden load will hardly show up in the usage figures -- it will
 flatten out due to the large usage figures already stored in the years
 of service\&.
 .PP 
 In contrast, when e\&.g\&. \f(CWover 3\fP is in effect, then a sudden load
 does show up -- because it highly contributes to the average of three
 connections\&.
 .PP 
 
 .SH "5\&.1\&.3: Specifying decays"
 
 .PP 
 Decays are also only relevant when crossroads computes the \&'next best
 back end\&' by size (bytes) or duration (seconds)\&. E\&.g\&., imagine two
 back ends A and B, both averaged over say 3 connections\&.
 .PP 
 Now when back end A is suddenly hit by a spike,
 its average would go up accordingly\&. But the back end would never
 again be used, unless B also received a similar spike, because A\&'s
 \&'usage data\&' over its last three connections would forever be larger than
 B\&'s data\&. 
 .PP 
 For that reason, you should in real situations probably always
 specify a decay, so that the backend selection algorithm recovers from
 spikes\&. Note that the usage data of the back end where a decay is
 specified, decay when \fBother\fP back ends are hit\&. The decay parameter
 is like specifying how fast your body regenerates when someone else
 does the work\&.
 .PP 
 The below configuration illustrates this:
 .PP 
 .nf 
 /* Definition of the service */
 service soap {
     /* Local TCP port */
     port 8080;
 
     /* We\&'ll select back ends by the processing
      * duration
      */
     dispatchmode byduration over 3;
 
     /* First back end: */
     backend A {
         /* Back end IP address and port */
         server 10\&.1\&.1\&.1:8080;
 
         /* When this back end is NOT hit because
          * the other one was less busy, then the
          * usage parameters decay 10% per connection
          */
         decay 10;
     }
 
     /* Second back end: */
     backend B {
         server 10\&.1\&.1\&.2:8080;
         decay 10;
     }
 }
 .fi 
 
 .PP 
 
 .SH "5\&.1\&.4: Adjusting the weights"
 
 .PP 
 The back end modifier \f(CWweight\fP is useful in situations where your
 back ends differ in respect to performance\&. E\&.g,\&. your back ends may
 be geographically distributed, and you know that a given back end is
 difficult to reach and often experiences network lag\&.
 .PP 
 Or you may have
 one primary back end, a system with a fast CPU and enough memory, and a
 small fall-back back end, with a slow CPU and short on memory\&. In that
 case you know in advance that the second back end should be used only
 rarely\&. Most requests should go to the big server, up to a certain load\&.
 .PP 
 In such cases you will know in advance that the best performing back ends
 should be selected the most often\&. Here\&'s where the \f(CWweight\fP
 statement comes in: you can simply increase the weight of the back
 ends with the least performance, so that they are selected less
 frequently\&.
 .PP 
 E\&.g\&., consider the following configuration:
 .PP 
 .nf 
 service soap {
     port 8080;
     dispatchmode byduration over 3;
     backend A {
         server 10\&.1\&.1\&.1:8080;
         decay 20;
     }
     backend B {
         server 10\&.1\&.1\&.2:8080;
         weight 2;
         decay 10;
     }
     backend C {
         server 10\&.1\&.1\&.3:8080;
         weight 4;
         decay 5;
     }
 }
 .fi 
 
 .PP 
 This will cause crossroads to select back ends by the processing time,
 averaging over the last three connections\&. However, backend B will kick
 in only when its usage is half of the usage of A (back end B is
 probably only half as fast as A)\&. Backend C will kick in only when its
 usage is a quarter of the usage of A, which is half of the usage of B
 (back end C is probably very weak, and just a fall-back system incase
 both A and B crash)\&. Note also that A\&'s usage data decay much faster
 than B\&'s and C\&'s: we\&'re assuming that this big server recovers quicker
 than its smaller siblings\&.
 .PP 
 
 .SH "5\&.1\&.5: Throttling the number of concurrent connections"
 
 .PP 
 If you suspect that your service may occasionally receive \&'spikes\&' of
 activity\e (which you should always assume), then it might be a
 good idea to protect your service by specifying a maximum number of
 concurrent connections\&. This protection can be specified on two levels:
 .PP 
 .IP "On the service level"
 a statement like \f(CWmaxconnections
 100;\fP states that the service as a whole will never
 service more than 100 concurrent connections\&. This means that
 all your back ends and the crossroads balancer itself
 will be protected from being overloaded\&.
 .IP "On the back end level"
 a statement like \f(CWmaxconnections 10;\fP
 states that this particular back end will never have more
 than 10 concurrent connections; regardless of the overall
 setting on the service level\&. This means that this
 particular back end will be protected from being
 overloaded (regardless of what other back ends may
 experience)\&.
 .PP 
 The \f(CWmaxconnections\fP statement, combined with a back end selection
 algorithm, allows very fine granularity\&. The \f(CWmaxconnections\fP statement
 on the back end level is like a hand brake: even when you specify a
 back end algorithm that would protect a given back end from being used
 too much, a situation may occur where that back end is about to be
 hit\&. A \f(CWmaxconnections\fP statement on the level of that back may then
 protect it\&.
 .PP 
 
 .SH "5\&.2: Using an external program to dispatch"
 
 .PP 
 As mentioned before, Crossroads supports several built-in dispatch
 modes\&. However, you are always free to hook-in your own dispatch mode
 that determines the next back end using your own specific
 algorithm\&. This section explains how to do it\&.
 .PP 
 
 .SH "5\&.2\&.1: Configuring the external handler"
 
 .PP 
 First, the \f(CWdispatchmode\fP statement needs to inform Crossroads that
 an external program will do the job\&. The syntax is: \f(CWdispatchmode
 externalhandler\fP \fIprogram arguments\fP\&. The \fIprogram\fP must point to
 an executable program that will be started by Crossroads\&. The
 specifier \fIarguments\fP can be anything you want; those will be the
 arguments to Crossroads\&. You can however use the following special
 format specifiers:
 .PP 
 .IP o 
 \f(CW%a\fP is the availability of the current back end, when
 a current back end is established;
 .IP o 
 \f(CW%1a\fP is the availability of the first back end (0 when
 unavailable, 1 if available); \f(CW%2a\fP is the availability of
 the second back end, and so on;
 .IP o 
 \f(CW%b\fP is the name of the current back end, when one is
 established; 
 .IP o 
 \f(CW%1b\fP is the name of the first back end, \f(CW%2b\fP of the
 second back end, and so on;
 .IP o 
 \f(CW%e\fP is the count of seconds since start of epoch
 (January 1st 1970 GMT);
 .IP o 
 \f(CW%r\fP is the IP address of the client that requests a
 connection and for whom the external dispatcher should compute
 a back end;
 .IP o 
 \f(CW%s\fP is the name of the current service that the client
 connected to;
 .IP o 
 \f(CW%t\fP is the current local time in ASCII format, in
 \fIYYYY-MM-DD/hhh:mm:ss\fP; 
 .IP o 
 \f(CW%T\fP is the current GMT time in ASCIII format;
 .IP o 
 \f(CW%v\fP is the Crossroads version;
 .IP o 
 Any other chararacter following a \f(CW%\fP sign is taken
 literally; e\&.g\&. \f(CW%z\fP is just a z\&.
 .PP 
 Note that the format specifiers such as \f(CW%b\fP don\&'t make sense in the
 phase in which an external handler is called, since there is no
 current back end yet (the job of the handler is to supply one)\&.
 .PP 
 
 .SH "5\&.2\&.2: Writing the external handler"
 
 .PP 
 The external handler is activated using the arguments that are
 specified in \f(CW/etc/crossroads\&.conf\fP\&. The external handler can do
 whatever it wants, but ultimately, it must write a back end name on
 its \fIstdout\fP\&. Crossroads reads this, and if the back end is
 available, uses that back end for the connection\&.
 .PP 
 
 .SH "5\&.2\&.3: Examples of external handlers"
 
 .PP 
 This section shows some examples of Crossroads configurations
 vs\&. external handlers\&. The sample handlers that are shown here, are
 also included in the Crossroads distribution, under the directory
 \f(CWetc/\fP\&. Also note that the examples shown here are just
 quick-and-dirty Perl scripts, meant to illustrate only\&. Your
 applications may need other external handlers, but you can use the
 shown scripts as a starting point\&.
 .PP 
 .SH "Round-robin dispatching"
 
 .PP 
 This example is trivial in the sense that round-robin dispatching is
 already built into Crossroads, so
 that using an external handler for this purpose only slows down
 Crossroads\&. However, it\&'s a good starting example\&.
 .PP 
 The Crossroads configuration is shown below:
 .PP 
 .nf 
 service test {
     port 8001;
     verbosity on;
     revivinginterval 5;
     
     dispatchmode externalhandler
         /usr/local/src/crossroads/etc/dispatcher-roundrobin
             %1b %1a %2b %2a;
 
     backend testone {
         server localhost:3128;
         verbosity on;
     }
     backend testtwo {
         server locallhost:3128;
         verbosity on;
     }
 }
 .fi 
 
 .PP 
 The relevant \f(CWdispatchmode\fP statement invokes the external program
 \f(CWdispatcher-roundrobin\fP with four arguments: the name of the first
 back end (\f(CWtestone\fP), its availability (0 or 1), the name of the
 second back end (\f(CWtesttwo\fP) and its availability (0 or 1)\&.
 .PP 
 The external handler, which is also included in the Crossroads
 distribution, is shown below\&. It is a Perl script\&.
 .PP 
 .nf 
 #!/usr/bin/perl
 
 use strict;
 
 # Example of a round-robin external dispatcher\&. This is totally
 # superfluous, Crossroads has this on-board; if you use the external
 # program for determining round-robin dispatching, then you\&'ll only
 # slow things down\&. This script is just meant as an example\&.
 
 # Globals / configuration
 # -----------------------
 my $log = \&'/tmp/exthandler\&.log\&';    # Debug log, set to /dev/null to suppress
 my $statefile = \&'/tmp/rr\&.last\&';	    # Where we keep the last used
 
 # Logging
 # -------
 sub msg {
     return if ($log eq \&'/dev/null\&' or $log eq \&'\&');
     open (my $of, ">>$log") or return;
     print $of (scalar(localtime()), \&' \&', @_);
 }
 
 # Read the last used back end
 # ---------------------------
 sub readlast() {
     my $ret;
     
     if (open (my $if, $statefile)) {
         $ret = <$if>;
         chomp ($ret);
         close ($if);
         msg ("Last used back end: $ret\en");
         return ($ret);
     }
     msg ("No last-used back end (yet)\en");
     return (undef);    
 }
 
 # Write back the last used back end, reply to Crossroads and stop
 # ---------------------------------------------------------------
 sub reply ($) {
     my $last = shift;
 
     if (open (my $of, ">$statefile")) {
         print $of ("$last\en");
     }
     print ("$last\en");
     exit (0);
 }
 
 # Main starts here
 # ----------------
 
 # Collect the cmdline arguments\&. We expect pairs of backend-name /
 # backend-availablility, and we\&'ll store only the available ones\&.
 msg ("Dispatch request received\en");
 my @backend;
 for (my $i = 0; $i <= $#ARGV; $i += 2) {
     push (@backend,  $ARGV[$i]) if ($ARGV[$i + 1]);
 }
 msg ("Available back ends: @backend\en");
 
 # Let\&'s see what the last one is\&. If none found, then we return the
 # first available back end\&. Otherwise we need to go thru the list of
 # back ends, and return the next one in line\&.
 my $last = readlast();
 if ($last eq \&'\&') {
     msg ("Returning first available back end $backend[0]\en");
     reply ($backend[0]);
 }
 
 # There **was** a last back end\&.  Try to match it in the list,
 # then return the next-in-line\&.
 for (my $i = 0; $i < $#backend; $i++) {
     if ($last eq $backend[$i]) {
         msg ("Returning next back end ", $backend[$i + 1], "\en");
         reply ($backend[$i + 1]);
     }
 }
 
 # No luck\&.\&. run back to the first one\&.
 msg ("Returning first back end $backend[0]\en");
 reply ($backend[0]);
 .fi 
 
 .PP 
 The working of the script is basically as follows:
 .PP 
 .IP o 
 The argument list is scanned\&. Back ends that are
 available are collected in an array \f(CW@backend\fP\&.
 .IP 
 .IP o 
 The script queries a state file \f(CW/tmp/rr\&.last\fP\&. If a
 back end name occurs there, then the next back end is looked
 up in \f(CW@backend\fP and returned to Crossroads\&. If no last back
 is unknown or can\&'t be matched, then the first available back
 end (first element of \f(CW@backend\fP) is returned to Crossroads\&.
 .IP 
 .IP o 
 Informing Crossroads is done via the subroutine
 \f(CWreply()\fP\&. This code writes the selected back end to file
 \f(CW/tmp/rr\&.last\fP (for future usage) and prints the back end
 name to \fIstdout\fP\&.
 .IP 
 .IP o 
 The script logs its actions to a file
 \f(CW/tmp/exthandler\&.log\fP\&. This log file can be inspected for
 the script\&'s actions\&.
 .PP 
 .SH "Dispatching by the client IP address"
 
 .PP 
 The following example shows a useful real-life situation\&. The
 situation is as follows:
 .PP 
 .IP o 
 Crossroads is used as a single-address point to forward
 Remote Desktop requests to a farm of Windows systems, where
 users can work via remote access;
 .IP 
 .IP o 
 However, users may stop their session, and when they
 re-connect, they expect to be sent to the Windows system that
 they had worked on previously;
 .IP 
 .IP o 
 Client PC\&'s have their distinct IP addresses, which
 distinguishes them\&.
 .IP 
 .IP o 
 Of four windows systems, two are large servers, and two
 are small ones\&. We\&'ll want to assign large servers to clients
 when we have a choice\&.
 .PP 
 The requirements resemble session stickiness in HTTP, except that the remote
 desktop protocol doesn\&'t support stickiness\&. This situation is a
 perfect example of how an external handler can help:
 .PP 
 .IP o 
 A suitable dispatch mode isn\&'t yet available in
 Crossroads, but can be easily coded in an external handler;
 .IP 
 .IP o 
 The potential delay due to the calling of an external
 handler won\&'t even be noticed\&. This is a network service where
 the connection time isn\&'t critical; we\&'d expect only a few
 (albeit lengthy) TCP connections\&.
 .PP 
 The approach to the solution of this problem uses several external
 program hooks:
 .PP 
 .IP o 
 An external dispatcher handler will be responsible for
 suggesting a back end, given a client IP and given the current
 timestamp\&. This handler will consult an internal
 administration to see whether the stated IP address should
 re-use a back end, or to determine which back end is free for usage\&.
 .IP o 
 An external hook \f(CWonstart\fP will be responsible for
 updating the internal administration; i\&.e\&., to flag a back end
 as \&'occupied\&'\&.
 .IP o 
 The external hooks \f(CWonfailure\fP and \f(CWonend\fP will be
 responsible for flagging a back end as \&'free\&' again; i\&.e\&., for
 erasing any previous information that states that the back end
 was occupied\&.
 .PP 
 The Crossroads configuration is shown below\&. Only four Windows back
 ends are shown\&. Each back end is configured on a
 given IP address, port 3389, and is limited to one concurrent connection
 (otherwise a new user might \&'steal\&' a running desktop session)\&.
 .PP 
 .nf 
 service rdp {
     port 3389;
     revivinginterval 5;
 
     /* rdp-helper dispatch IP STAMP \&.\&.\&.  will suggest a back end to use,
      * arguments are for all back ends: name, availability, weight */
     dispatchmode externalhandler
         /usr/local/src/crossroads/etc/rdp-helper dispatch %r %e
             %1b %1a %1w
             %2b %2a %2w
             %3b %3a %3w
             %4b %4a %4w;
             
     backend win1 {
         server 10\&.1\&.1\&.1:3389;
         maxconnections 1;
         /* rdp-helper start IP STAMP BACKEND will log the actual start
          * of a connection;
          * rdp-helper end IP will log the ending of a connection */
         onstart /usr/local/src/crossroads/etc/rdp-helper start %r %e %b;
         onend   /usr/local/src/crossroads/etc/rdp-helper end %r;
         onfail  /usr/local/src/crossroads/etc/rdp-helper end %r;
     }
     backend win2 {
         server 10\&.1\&.1\&.2:3389;
         maxconnections 1;
         onstart /usr/local/src/crossroads/etc/rdp-helper start %r %e %b;
         onend   /usr/local/src/crossroads/etc/rdp-helper end %r;
         onfail  /usr/local/src/crossroads/etc/rdp-helper end %r;
     }
     backend win3 {
         server 10\&.1\&.1\&.3:3389;
         maxconnections 1;
         weight 2;
         onstart /usr/local/src/crossroads/etc/rdp-helper start %r %e %b;
         onend   /usr/local/src/crossroads/etc/rdp-helper end %r;
         onfail  /usr/local/src/crossroads/etc/rdp-helper end %r;
     }
     backend win4 {
         server 10\&.1\&.1\&.4:3389;
         maxconnections 1;
         weight 3;
         onstart /usr/local/src/crossroads/etc/rdp-helper start %r %e %b;
         onend   /usr/local/src/crossroads/etc/rdp-helper end %r;
         onfail  /usr/local/src/crossroads/etc/rdp-helper end %r;
     }
 }
 .fi 
 
 .PP 
 Depending on the dispatcher stage, the exernal handler \f(CWrdp-helper\fP
 is invoked in different ways:
 .PP 
 .IP "During dispatching"
 the helper is called to suggest a back
 end\&. The arguments are an action indicator \f(CWdispatch\fP, the
 client\&'s IP address, the timestamp, and four triplets that
 represent back ends: per back end its name, its availability,
 and its weight\&. The purpose of the helper is to tell
 Crossroads which back end to use\&.
 .IP 
 .IP "During connection start"
 the helper will be invoked to
 inform it of the start of a connection, given a client IP
 address\&.
 .IP 
 .IP "When a connection terminates"
 the helper will be invoked
 to inform it that the connection has ended\&.
 .PP 
 Here\&'s the external handler as Perl script\&. It uses the module
 \f(CWGDBM_File\fP which most likely will not be part of standard Perl
 distributions, but can be added using CPAN\&. (Alternatively, any other
 database module can be used\&.)
 .PP 
 .nf 
 #!/usr/bin/perl
 
 use strict;
 use GDBM_File;
 
 # Global variables and configuration
 # ----------------------------------
 my $log = \&'/tmp/exthandler\&.log\&';    # Debug log, set to /dev/null to suppress
 my $cdb = \&'/tmp/client\&.db\&';         # GDBM database of clients
 my %db;                             # \&.\&. and memory representation of it
 my $timeout = 24*60*60;             # Timeout of a connection in secs
 
 # Logging
 # -------
 sub msg {
     return if ($log eq \&'/dev/null\&' or $log eq \&'\&');
     open (my $of, ">>$log") or return;
     print $of (scalar(localtime()), \&' \&', @_);
     close ($of);
 }
 
 # Reply a back end to the caller and stop processing\&.
 # ---------------------------------------------------
 sub reply ($) {
     my $b = shift;
     msg ("Suggesting $b to Crossroads\&.\en");
     print ("$b\en");
     exit (0);
 }
 
 # Is a value in an array
 # ----------------------
 sub inarray {
     my $val = shift;
     for my $other (@_) {
         return (1) if ($other eq $val);
     }
     return (0);
 }
 
 # A connection is starting
 # ------------------------
 sub start {
     my ($ip, $stamp, $backend) = @_;
     msg ("Logging START of connection for IP $ip on stamp $stamp, ",
          "back end $backend\en");
     $db{$ip} = "$backend:$stamp";
 }
 
 # A connection has ended
 # ----------------------
 sub end {
     my $ip = shift;
     msg ("Logging END of connection for IP $ip\en");
     $db{$ip} = undef;
 }
 
 # Request to determine a back end
 # -------------------------------
 sub dispatch {
     my $ip = shift;
     my $stamp = shift;
 
     msg ("Request to dispatch IP $ip on stamp $stamp\en");
     
     # Read the next arguments\&. They are triplets of
     # backend-name / availability / weight\&. Store if the back end is
     # available\&.
     my (@backends, @weights);
     for (my $i = 0; $i < $#_; $i += 3) {
         if ($_[$i + 1] != 0) {
             push (@backends, $_[$i]);
             push (@weights,  $_[$i + 2]);
             msg ("Candidate back end: $_[$i] with weight ", $_[$i + 2], "\en");
         }
     }
 
     # See if this is a reconnect by a previously seen client IP\&. We\&'ll
     # treat this as a reconnect if the timeout wasn\&'t yet exceeded\&.
     if ($db{$ip} ne \&'\&') {
         my ($last_backend, $last_stamp) = split (/:/, $db{$ip});
         msg ("IP $ip had last connected on $last_stamp to $last_backend\en");
         if ($stamp < $last_stamp + $timeout) {
             msg ("Timeout not yet exceeded, this may be a reconnect\en");
             # We\&'ll allow a reconnect only if the stated last_backend is
             # free (sanity check)\&.
             if (inarray ($last_backend, @backends)) {
                 msg ("Last back end $last_backend is available, ",
                      "letting through\en");
                 reply ($last_backend);
             } else {
                 msg ("Last used back end isn\&'t free, suggesting a new one\en");
             }
         } else {
             msg ("Timeout exceeded, suggesting a new back end\en");
         }
     } else {
         msg ("Np preveious connection data, suggesting a new back end\en");
     }
 
     my $bestweight = -1;
     my $bestbackend;
     for (my $i = 0; $i <= $#weights; $i++) {
         if ($bestweight == -1 or $bestweight > $weights[$i]) {
             $bestweight  = $weights[$i];
             $bestbackend = $backends[$i];
         }
     }
 
     msg ("Best back end: $bestbackend (given weight $bestweight)\en");
     reply ($bestbackend);
 }
 
 # Main starts here
 # ----------------
 msg ("Start of run, attaching GDBM database \&'$cdb\&'\en");
 tie (%db, \&'GDBM_File\&', $cdb, &GDBM_WRCREAT, 0600);
 
 # The first argument must be an action \&'dispatch\&', \&'start\&' or \&'end\&'\&.
 # Depending on the action, we do stuff\&.
 my $action = shift (@ARGV);
 if ($action eq \&'dispatch\&') {
     dispatch (@ARGV);
 } elsif ($action eq \&'start\&') {
     start (@ARGV);
 } elsif ($action eq \&'end\&') {
     end (@ARGV);
 } else {
     print STDERR ("Usage: rdp-helper {dispatch|start|end} args\en");
     exit (1);
 }
 .fi 
 
 .PP 
 
 .SH "5\&.3: HTTP Session Stickiness"
 
 .PP 
 This section focuses on HTTP session stickiness\&. This term refers to
 the ability of a balancer to route a conversation between browser and
 a backend farm always to the same back end\&. In other words: once a
 back end is selected by the balancer, it will remain the back end of
 choice, even for subsequent connections\&.
 .PP 
 
 .SH "5\&.3\&.1: Don\&'t use stickiness!"
 
 .PP 
 The rule of thumb as far as the balancer is concerned, is: \fBDo not
 use HTTP session stickiness unless you really have to\&.\fP Enabling
 session stickiness hampers failover, balancing and performance:
 .PP 
 .IP o 
 Failover is hampered because during the session,
 the balancer has to assign new connections to the same back
 end that was selected at the start of a session\&. If the back
 end suddenly goes \&'down\&', then the session will most likely
 crash\&. (Actually, when a back end becomes unreachable in the
 middle of a session, Crossroads will assign a new back end to
 that session\&. This will most likely result in a malfunction
 of the underlying application\&.)
 .IP o 
 Balancing is hampered because at the start of the session,
 the balancer has selected the next-best back end\&. But during
 the session, that back end may well become overloaded\&. The
 balancer however must continue to send the requests there\&.
 .IP o 
 Performance is hampered because crossroads needs to \&'unpack\&'
 messages as they are passed to and fro\&. That\&'s because
 crossroads needs to check the HTTP headers in the messages
 for persistence cookies\&.
 .PP 
 There is a number of measures that you can take to avoid using session
 stickiness\&. E\&.g\&., session data can be \&'shared\&' between web back
 ends\&. PHP offers functionality to store session data in a database, so
 that all PHP applications have access to these data\&. Application
 servers such as Websphere can be configured to replicate session data
 between nodes\&.
 .PP 
 
 .SH "5\&.3\&.2: But if you must\&.\&."
 
 .PP 
 However, if you \fBmust\fP use session stickiness, then proceed as
 follows:
 .PP 
 .IP o 
 At the level of a \f(CWservice\fP description, set the type to
 \f(CWhttp\fP\&. 
 .IP o 
 At the level of each back end description, configure the
 \f(CWstickycookie\fP and a \f(CWaddclientheader\fP directives\&.
 .PP 
 Once crossroads sees that, it will examine each HTTP message that it
 shuttles between client and back end:
 .PP 
 .IP o 
 If there is no persistence cookie in the HTTP headers of a
 client\&'s request, then the message must be the first one and
 a new session should be established\&. 
 Crossroads selects an appropriate back
 end, sends the message to that back end, catches the reply,
 and inserts a \f(CWSet-Cookie\fP directive\&.
 .IP o 
 If there is a persistence cookie in the HTTP headers of a
 client\&'s request, then the request is part of an already
 established session\&. Crossroads analyzes the cookie and
 forwards the request to the appropriate back end\&.
 .PP 
 Below is a short example of a configuration\&.
 .PP 
 .nf 
 service www {
     port 80;
     type http;
     revivinginterval 15;
     dispatchmode byconnections;
 
     backend one {
         server 10\&.1\&.1\&.100:80;
         stickycookie XRID=100;
         addclientheader "Set-Cookie: XRID=100; Path=/";
     }
 
     backend two {
         server 10\&.1\&.1\&.101:80;
         stickycookie XRID=101;
         addclientheader "Set-Cookie: XRID=101; Path=/";
     }
 }
 .fi 
 
 .PP 
 Note how the cookie names and values in the directives
 \f(CWstickycookie\fP and \f(CWaddclientheader\fP match\&. That is obviously a
 prerequisite for stickiness\&.
 .PP 
 
 .SH "5\&.4: Passing the client\&'s IP address"
 
 .PP 
 Since Crossroads just shuttles bytes to and fro, meta-information of
 network connections is lost\&. As far as the back ends are concerned,
 their connections originate at the Crossroads junction\&.
 For example, standard Apache access logs will show the IP address of
 Crossroads\&. 
 .PP 
 In order to compensate for this, Crossroads can insert a special
 header in HTTP connections, to inform the back end of the original
 client\&'s IP address\&. In order to enable this, the Crossroads
 configuration must state the following:
 .PP 
 .IP o 
 The service type must be \f(CWhttp\fP, and not \f(CWany\fP;
 .IP o 
 In the back end definition, the following statement must
 occur: 
 .br 
 \f(CWaddserverheader "X-Real-IP: %r";\fP 
 .br 
 You are of course free to choose the header name; the here
 used \f(CWX-Real-IP\fP is a common name for this purpose\&.
 .PP 
 After this, HTTP traffic that arrives at the back ends has a new
 header: \f(CWX-Real-IP\fP, holding the client\&'s IP address\&.
 \fBNote that\fP once the type is set to \f(CWhttp\fP, Crossroads\&'
 performance will be hampered -- all passing messages will have to be
 unpacked and analyzed\&.
 .PP 
 
 .SH "5\&.4\&.1: Sample Crossroads configuration"
 
 .PP 
 The below sample configuration shows two HTTP back ends that receive
 the client\&'s IP address:
 .PP 
 .nf 
 
 service www {
     port 80;
     type http;
     revivinginterval 5;
     dispatchmode roundrobin;
 
     backend one {
         server 10\&.1\&.1\&.100:80;
         addserverheader "X-Real-IP: %r";
     }
 
     backend two {
         server 10\&.1\&.1\&.200:80;
         addserverheader "X-Real-IP: %r";
     }
 }
 .fi 
 
 .PP 
 
 .SH "5\&.4\&.2: Sample Apache configuration"
 
 .PP 
 The method by which each back end analyzes the header \f(CWX-Real-IP\fP
 will obviously be different per server implementations\&. However, a
 common method with the Apache webserver is to log the client\&'s IP
 address into the access log\&.
 .PP 
 Often this is accomplished using the log format \f(CWcustom\fP, defined as
 follows:
 .PP 
 .nf 
 LogFormat "%h %l %u %t %D \e"%r\e" %>s %b" common
 CustomLog logs/access_log common
 .fi 
 
 .PP 
 The first line defines the format \f(CWcommon\fP, with the remote host
 specified by \f(CW%h\fP\&. The second line sends access information to a log
 file \f(CWlogs/access_log\fP, using the previously defined format
 \f(CWcommon\fP\&.
 .PP 
 Furtunately, Apache\&'s \f(CWLogFormat\fP allows one to log contents of
 headers\&. By replacing the \f(CW%h\fP with \f(CW%{X-Real-IP}i\fP, the desired
 information is sent to the log\&. Therefore, normally you can simply
 redefine the \f(CWcommon\fP format to 
 .PP 
 .nf 
 LogFormat "%{X-Real-IP}i %l %u %t %D \e"%r\e" %>s %b" common
 .fi 
 
 .PP 
 
 .SH "5\&.5: Debugging network traffic"
 
 .PP 
 Incase the traffic between
 client and backend
 must be debugged, the statement \f(CWtrafficlog\fP \fIfilename\fP can
 be issued\&. This causes the traffic to be dumped in hexadecimal
 format to the stated filename\&.
 .PP 
 Traffic sent by the client is prefixed by a \fBC\fP, traffic sent by
 the back end is prefixed by a \fBB\fP\&. Below is a sample traffic
 dump of a browser trying to get a HTML page\&. The server replies
 that the page was not modified\&.
 .PP 
 .nf 
 C 0000  47 45 54 20 68 74 74 70 3a 2f 2f 77 77 77 2e 63 GET http://www\&.c
 C 0010  73 2e 68 65 6c 73 69 6e 6b 69 2e 66 69 2f 6c 69 s\&.helsinki\&.fi/li
 C 0020  6e 75 78 2f 6c 69 6e 75 78 2d 6b 65 72 6e 65 6c nux/linux-kernel
 C 0030  2f 32 30 30 31 2d 34 37 2f 30 34 31 37 2e 68 74 /2001-47/0417\&.ht
 C 0040  6d 6c 20 48 54 54 50 2f 31 2e 31 0d 0a 43 6f 6e ml HTTP/1\&.1\&.\&.Con
 C 0050  6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a nection: close\&.\&.
 \&.
 \&. etcetera
 \&.
 B 0000  48 54 54 50 2f 31 2e 30 20 33 30 34 20 4e 6f 74 HTTP/1\&.0 304 Not
 B 0010  20 4d 6f 64 69 66 69 65 64 0d 0a 44 61 74 65 3a  Modified\&.\&.Date:
 B 0020  20 54 75 65 2c 20 31 32 20 4a 75 6c 20 32 30 30  Tue, 12 Jul 200
 B 0030  35 20 30 39 3a 34 39 3a 34 37 20 47 4d 54 0d 0a 5 09:49:47 GMT\&.\&.
 B 0040  43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
 B 0050  78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 74 xt/html; charset
 \&.
 \&. etcetera
 \&.
 .fi 
 
 .PP 
 Turning on traffic dumps will \fIsignificantly\fP
 slow down crossroads\&.
 .PP 
 Besides \f(CWtrafficlog\fP, there is also a directive
 \f(CWthroughputlog\fP\&. This directive also takes one argument, a
 filename\&. The file is appended, and the following information is
 logged:
 .PP 
 .IP o 
 The process ID of the crossroads image that serves the
 TCP connection;
 .IP o 
 The time of the request, in seconds and microseconds
 since start of the run;
 .IP o 
 A \fBC\fP when the request originated at the client, or
 \fBB\fP when the request originated at the back end;
 .IP o 
 The first 100 bytes of the request\&.
 .PP 
 As an example, consider the following (the lines are shortened for
 brevity and prefixed by line numbers for clarity):
 .PP 
 .nf 
 
 1 0000594 0\&.000001 C GET http://public\&.e-tunity\&.com/index\&.html\&.\&.\&.
 2 0000594 0\&.173713 B HTTP/1\&.0 200 OK\&.\&.Date: Fri, 18 Nov 2005 0\&.\&.\&.
 3 0000594 0\&.278125 B  width="100" bgcolor="#e0e0e0" valign="to\&.\&.\&.
 4 0000595 0\&.000001 C GET http://public\&.e-tunity\&.com/css/style/\&.\&.\&.
 5 0000594 0\&.944339 B /a></td>\&.\&.  </tr>\&.</table>\&.</td><td class\&.\&.\&.
 6 0000594 0\&.946356 B smallboxdownl">Download</td>\&.\&.  <td class\&.\&.\&.
 7 0000594 0\&.961102 B td><td class="smallboxodd" valign="top"><\&.\&.\&.
 8 0000595 0\&.698215 B HTTP/1\&.0 304 Not Modified\&.\&.Date: Fri, 18 \&.\&.\&.
 .fi 
 
 .PP 
 This tells us that:
 .PP 
 .IP o 
 Line 1:  PID 594 served a request that originated at
 the client\&. The corresponding time is (almost) 0 seconds,
 so this is really the start of the run\&.
 .IP o 
 Line 2: A back end replied 0\&.17 seconds later, and
 0\&.28 seconds later, it was still replying (this is the
 third line, again a \fBB\fP-type transmission)\&.
 .IP o 
 Line 4: PID 595 served a request that originated
 at the client\&. Again, the corresponding time is (almost)
 0 seconds, since this is the first conversation part of
 this connection\&.
 .IP o 
 Lines 5 to 7: This is the continuation of line 2\&. Line 7
 is the last line of the \fBB\fP series (not visible from
 the example, but trust me, it is), so that we may
 conclude that it took the back end 0\&.96 seconds to serve
 the file \f(CWindex\&.html\fP requested in line 1\&.
 .IP o 
 Line 8: This is the answer to the client\&'s request of
 line 4 (you can tell by the process ID  number)\&.
 So the back end took 0\&.68 seconds to confirm that
 the stylesheet requested in line 4 wasn\&'t modified\&.
 .PP 
 It is also worth while remembering that the start time of a \fBC\fP
 request is the time that crossroads sees the activity\&. Any latency
 between the true client and crossroads is obviously not
 included\&. This is illustrated by the below simple ASCII art:
 .PP 
 .nf 
 
 client ---->---->---->--->*crossroads ====>====>====>
                                                      \e    
                                                   back end
                                                      /
 client ----<----<----<---< crossroads ====<====<====<
 
 .fi 
 
 .PP 
 This simple picture shows a typical HTTP request that originates
 at a client, travels to crossroads, and is relayed via the back
 end\&. The \fBC\fP entry in a throughput log is the time when
 crossroads sees the request, indicated by an asterisk\&. The \fBB\fP
 entries are the times that it takes the back end to answer,
 indicated by \f(CW===\fP style lines\&. Therefore, the true roundtrip
 time will be longer than the number of seconds that are logged in
 the throughput log: the latency between client and crossroads
 isn\&'t included in that measurement\&.
 .PP 
 Summarizing, the throughput times of a client-back end connection
 can be analyzed using the directive \f(CWthroughputlog\fP\&. In a
 real-world analysis, you\&'d probably want to write up a script to
 analyze the output and to compute round trip times\&. Such scripts
 are not (yet) included in Crossroads\&.
 .PP 
 
 .SH "5\&.6: Limiting Access to Crossroads by Client IP Address"
 
 .PP 
 
 .SH "5\&.6\&.1: General Examples"
 
 .PP 
 The directives \f(CWallowfrom\fP, \f(CWdenyfrom\fP, \f(CWallowfile\fP and
 \f(CWdenyfile\fP can be used to instruct Crossroads to specifically allow
 access by using a "whitelist" of IP addresses, or to specifically deny
 access by using a "blacklist"\&. E\&.g\&., the following configuration
 allows access to service \f(CWwebproxy\fP only to \fIlocalhost\fP:
 .PP 
 .nf 
 service webproxy {
     port 8000;    
     allowfrom 127\&.0\&.0\&.1;
     backend one {
         \&.
         \&. Back end definitions occur here
         \&.
     }
     \&.
     \&. Other back ends or other service directives
     \&. may occur here
     \&.
 }
 .fi 
 
 .PP 
 In this example there is a "whitelist" having only one entry: IP
 address 127\&.0\&.0\&.1, or \fIlocalhost\fP\&. (Incidentally, the same behaviour
 could be accomplished by stating \fIbindto 127\&.0\&.0\&.1\fP, in which case
 Crossroads would only listen to the local network device\&.)
 .PP 
 In the same vein, the directive \f(CWallowfrom 127\&.0\&.0\&.1 192\&.168\&.1/24\fP
 would allow access to \fIlocalhost\fP and to all IP addresses that start
 with 192\&.168\&.1\&. The specifier \f(CW192\&.168\&.1/24\fP states that there are
 three network bytes (192, 168 and 1), and 24 bits (or 3 bytes) are
 relevant; so that the fourth network byte doesn\&'t matter\&.
 .PP 
 
 .SH "5\&.6\&.2: Using External Files"
 
 .PP 
 The directives \f(CWallowfile\fP and \f(CWdenyfile\fP allow you to specify IP
 addresses in external files\&. The Crossroads configuration states
 e\&.g\&. \f(CWallowfile /tmp/allow\&.txt\fP, and the IP addresses are then in
 \f(CW/tmp/allow\&.txt\fP\&. The format of \f(CW/tmp/allow\&.txt\fP is as follows:
 .PP 
 .IP o 
 The specifications follow again \fIp\&.q\&.r\&.s/mask\fP, where
 p, q, r and s are network bytes which can be left out on the
 right hand side when the mask allows it;
 .IP 
 .IP o 
 The specifications must be separated by white space
 (spaces, tabs or newlines)\&.
 .PP 
 E\&.g\&., the following is a valid example of an external specification
 file:
 .PP 
 .nf 
 127\&.0\&.0\&.1
 192\&.168\&.1/24
 10/8
 .fi 
 
 .PP 
 When external files are in effect, then the signal \f(CWSIGHUP\fP (1)
 causes Crossroads to reload the external file\&. E\&.g\&., while Crossroads
 is running, you may edit \f(CW/tmp/allow\&.txt\fP, and then issue \f(CWkillall
 -1 crossroads\fP\&. The new contents of \f(CW/tmp/allow\&.txt\fP will be
 reloaded\&.
 .PP 
 
 .SH "5\&.6\&.3: Mixing Directives"
 
 .PP 
 Crossroads allows to mix all directives in one service
 description\&. However, some mixes are less meaningful than others\&. It\&'s
 up to you to take this into account\&.
 .PP 
 The following rules apply:
 .PP 
 .IP o 
 Blacklisting and whitelisting can be used together\&. When
 combined, the blacklist will always be interpreted
 first\&. E\&.g\&., consider the following directives:
 .IP 
 .nf 
 allowfrom 192\&.168\&.1/24
 denyfrom  192\&.168\&.1\&.100
 .fi 
 
 .IP 
 Given the fact that the deny list is checked first, client
 192\&.168\&.1\&.100 won\&'t be able to access Crossroads\&. Then the
 allow list will be checked, stating that all clients whose IP
 address starts with 192\&.168\&.1 may connect\&. The effect will be
 that e\&.g\&., client 192\&.168\&.1\&.1 may connect, 192\&.168\&.1\&.2 may
 connect too, 192\&.168\&.1\&.100 will be blocked, and 10\&.1\&.1\&.1 will
 be blocked as well\&.
 .IP 
 Now consider the following directives:
 .IP 
 .nf 
 allowfrom 192\&.168\&.1\&.100 127\&.0\&.0\&.1
 denyfrom  192\&.168\&.1/24
 .fi 
 
 .IP 
 This will first of all deny access to all IP addresses that
 start with 192\&.168\&.1\&. So the rule that allows 192\&.168\&.1\&.100
 won\&'t ever be effective\&. The net result will be that access
 will be granted to 127\&.0\&.0\&.1, and IP addresses that don\&'t
 match 192\&.168\&.1/24\&.
 .IP 
 .IP o 
 Blacklisting or whitelisting can be left out\&.
 A list is considered empty when no appropriate directives
 occur in \f(CW/etc/crossroads\&.conf\fP, or when the directive
 points to an empty or non-existent external file\&.
 .IP 
 .IP o 
 Using \f(CW*from\fP and \f(CW*file\fP statements is allowed, but
 doesn\&'t make sense\&. E\&.g\&., the following configuration sample
 is such a case:
 .IP 
 .nf 
 allowfrom 127\&.0\&.0\&.1 192\&.168\&.1/24
 allowfile /tmp/allow\&.txt
 .fi 
 
 .IP 
 There is a technical reason for this\&. Once Crossroads
 processes the \f(CWallowfile\fP directive, then the whole
 whitelist is cleared (thereby removing the entries 127\&.0\&.0\&.1
 and 192\&.168\&.1/24), and new entries are reloaded from the
 file\&. The net result is that the \f(CWallowfrom\fP specification
 is overruled\&.
 .IP 
 Crossroads doesn\&'t check for such configurations, which are
 syntactially correct, but make no semantic sense\&.
 .PP 
 
 .SH "5\&.7: Configuration examples"
 
 .PP 
 As a general hint, use \f(CWcrossroads sampleconf\fP to view the most
 up-to-date examples of configurations\&. The description below shows a
 few examples too\&.
 .PP 
 
 .SH "5\&.7\&.1: A load balancer for three webserver back ends"
 
 .PP 
 The following configuration example binds crossroads to port 80 of the
 current server, and distributes the load over three back ends\&. This
 configuration shows most of the possible settings\&.
 .PP 
 .nf 
 service www {
     /* We don\&'t need session stickyness\&. */
     type any;
     
     /* Port on which we\&'ll listen in this service: required\&. */
     port 8000;
 
     /* What IP address should this service listen? Default is \&'any\&'\&.
      * Alternatively you can state an explicit IP address, such as
      * 127\&.0\&.0\&.1; that would bind the service only to \&'localhost\&'\&. */
     bindto any;
     
     /* Verbose reporting or not\&. Default is off\&. */    
     verbosity on;
     
     /* Dispatching mode, or: How to select a back end for an incoming
      * request\&. Possible values:
      *   roundrobin: just the next back end in line
      *   random: like roundrobin, but at random to make things more
      *          confusing\&. Probably only good for testing\&.
      *   bysize: The backend that transferred the least nr of bytes
      *          is the next in line\&. As a modifier you can say e\&.g\&.
      *          bysize over 10, meaning that the 10 last connections will
      *          be used to compute the transfer size, instead of all
      *          transfers\&.
      *   byduration: The backend that was active for the shortest time
      *          is the next in line\&. As a modifier you can say e\&.g\&.
      *          byduration of 10 to compute over the last 10 connections\&.
      *   byconnections: The back end with the least active connections
      *          is the next ine line\&.
      *   byorder: The first available back end is always taken\&.
      */
     dispatchmode byduration over 5;
 
     /* Interval at which we\&'ll check whether a temporarily unavailable
      * backend has woken up\&.
      */
     revivinginterval 5;
     
     /* TCP backlog of connections\&. Default is 0 (no backlog, one
      * connection may be active)\&.
      */
     backlog 5;
     
     /* For status reporting: a shared memory key\&. Default is the same
      * as the port number, OR-ed by a magic number\&.
      */
     shmkey 8000;
 
     /* This controls when crossroads should consider a connection as
      * finished even when the TCP sockets weren\&'t closed\&. This is to
      * avoid hanging connections that don\&'t do anything\&. NOTE THAT when
      * crossroads cuts off a connection due to timeout exceed, this is
      * not marked as a failure, but as a success\&. Default is 0: no timeout\&.
      */
     connectiontimeout 300;
 
     /* The max number of allowed client connections\&. When present, connections
      * won\&'t be accepted if the max is about to be exceeded\&. When
      * absent, all connections will be accepted, which might be misused
      * for a DOS attack\&.
      */
     maxconnections 300;
 
     /* Now let\&'s define a couple of back ends\&. Number 1: */
     backend www_backend_1 {
         /* The server and its port, the minimum configuration\&. */
         server httpserver1;
         port 9010;
         /* The \&'decay\&' of usage data of this back end\&. Only relevant
          * when the whole service has \&'dispatchmode bysize\&' or
          * \&'byduration\&'\&. The number is a percentage by which the usage
          * parameter is decreased upon each connection of an other back
          * end\&.
          */
         decay 10;
         
         /* To see what\&'s happening in /var/log/messages: */
         verbosity on;
     }
 
     /* The second one: */
     backend www_backend_2 {
         /* Server and port */
         server httpserver2;
         port 9011;
 
         /* Verbosity of reporting when this back end is active */
         verbosity on;
 
         /* Decay */
         decay 10;
 
         /* This back end is twice as weak as the first one */
         weight 2;
 
         /* Event triggers for system commands upon succesful activation
          * and upon failure\&.
          */
         onsuccess echo \&'success on backend 2\&' | mail root;
         onfailure echo \&'failure on backend 2\&' | mail root;
     }
 
     /* And yet another one\&.\&. this time we will dump the traffic
      * to a trace file\&. Furthermore we don\&'t want more than 10 concurrent
      * connections here\&. Note that there\&'s also a total maxconnections for the
      * whole service\&.
      */
     backend www_backend_3 {
         server httpserver3;
         verbosity on;
         port 9000;
         verbosity on;
         decay 10;
         trafficlog /tmp/backend\&.3\&.log;
         maxconnections 10;
     }
 }
 .fi 
 
 .PP 
 
 .SH "5\&.7\&.2: An HTTP forwarder when travelling"
 
 .PP 
 As another example, here\&'s my \f(CWcrossroads\&.conf\fP that I use on my
 Unix laptop\&. The problem that I face is that I need many HTTP proxy
 configurations (at home, at customers\&' sites and so on) but I\&'m too
 lazy to reconfigure browsers all the time\&.
 .PP 
 Here\&'s how it used to be before crossroads:
 .PP 
 .IP o 
 At home, I would surf through a squid proxy on my local
 machine\&. The browser proxy setting is then
 \f(CWhttp://localhost:3128\fP\&.
 .IP 
 .IP o 
 Sometimes I start up an SSH tunnel to our offices\&. The
 tunnel has a local port 3129, and connects to a squid proxy on
 our e-tunity server\&. Hence, the browser proxy is then
 \f(CWhttp://localhost:3129\fP\&.
 .IP 
 .IP o 
 At a customer\&'s location I need the proxy
 \f(CWhttp://10\&.120\&.34\&.113:8080\fP, because they have configured it
 so\&.
 .IP 
 .IP o 
 And in yet other instances, I use a HTTP diagnostic tool
 Charles
 that sits between browser and website and shows me
 what\&'s happening\&. I run charles on my own machine and it
 listens to port 8888, behaving like a proxy\&. The browser
 configuration for the proxy is then
 \f(CWhttp://localhost:8888\fP\&.
 .PP 
 Here\&'s how it works with a crossroads configuration:    
 .PP 
 .IP o 
 I have configured my browsers to use
 \f(CWhttp://localhost:8080\fP as the proxy\&. For all situations\&.
 .IP 
 .IP o 
 I use the following crossroads configuration, and let
 crossroads figure out which proxy backend works, and which
 doesn\&'t\&. Note two particularities:
 .IP 
 .IP o 
 The statement \f(CWdispatchmode byorder\fP\&. This
 makes sure that once crossroads determines which
 backend works, it will stick to it\&. This usage of
 crossroads doesn\&'t need to balance over more than one
 back end\&.
 .IP 
 .IP o 
 The statement \f(CWbindto 127\&.0\&.0\&.1\fP makes sure
 that requests from other interfaces than loopback
 won\&'t get serviced\&.
 .IP 
 .nf 
 service HttpProxy {
     port 8080;
     bindto 127\&.0\&.0\&.1;
     verbosity on;
     dispatchmode byorder;
     revivinginterval 15;
 
     backend Charles {
         server localhost:8888;
         verbosity on;
     }
 
     backend CustomerProxy {
         server 10\&.120\&.34\&.113:8080;
         verbosity on;
     }
 
     backend SshTunnel {
         server localhost:3129;
     }
 
     backend LocalSquid {
         server localhost:3128;
     }
 }
 .fi 
 
 .PP 
 As a final note, the commandline argument \f(CWtell\fP can be used to
 influence crossroad\&'s own detection mechanism of back end availability
 detection\&. E\&.g\&., if in the above example the back ends \f(CWSshTunnel\fP
 and \f(CWLocalSquid\fP are both active, then \f(CWcrossroads tell httpproxy
 sshtunnel down\fP will \&'take down\&' the back end \f(CWSshTunnel\fP -- and
 will automatically cause crossroads to switch to \f(CWLocalSquid\fP\&.
 .PP 
 
 .SH "5\&.7\&.3: SSH login with enforced idle logout"
 
 .PP 
 The following example shows how crossroads \&'throttles\&' SSH
 logins\&. Connections are accepted on port
 22 (the normal SSH port) and forwarded to the actual SSH daemon
 which is running on port 2222\&.
 .PP 
 Note the usage of the
 \f(CWconnectiontimeout\fP directive\&. This makes sure that users are logged
 out after 10 minutes of inactivity\&. Note also the \f(CWmaxconnections\fP
 setting, this makes sure that no more than 10 concurrent logins occur\&.
 .PP 
 .nf 
 service Ssh {
     port 22;
     backlog 5;
     maxconnections 10;
     connectiontimeout 600;
     backend TrueSshDaemon {
         server localhost:2222;
     }
 }
 .fi 
 
 .PP 
 
 .SH "6: Benchmarking"
 This section shows how crossroads affects the
 transmitting of HTML data when used as an intermediate \&'station\&'
 through which all data travels\&.
 .PP 
 
 .SH "6\&.1: Benchmark 1: Accessing a proxy via crossroads or directly"
 
 .PP 
 The benchmark was run on a system where the following was varied:
 .PP 
 .IP 1\&.
 A website was recursively spidered through a local squid
 proxy\&. The spidering was repeated 10 times, the total was recorded\&.
 .IP 
 .IP 2\&.
 Crossroads was placed in front of the squid proxy, and
 the website was again recursively spidered\&. Again, the
 spidering was repeated 10 times and the total was recorded\&.
 .PP 
 The crossroads configuration of the second alternative is shown below:
 .PP 
 .nf 
 service HttpProxy {
     port 8080;
     verbosity on;
     backend LocalSquid {
         server 127\&.0\&.0\&.1;
         port 3128;
         verbosity on;
     }
 }
 .fi 
 
 .PP 
 
 .SH "6\&.1\&.1: Results"
 
 .PP 
 The results of this test are that crossroads causes a negligible
 delay, if it is statistically relevant at all\&. Without crossroads, the
 timing results are:
 .PP 
 .nf 
 real	0m8\&.146s
 user	0m0\&.130s
 sys	0m0\&.253s
 .fi 
 
 .PP 
 When using crossroads as a middle station, the results are:
 .PP 
 .nf 
 real	0m9\&.481s
 user	0m0\&.141s
 sys	0m0\&.230s
 .fi 
 
 .PP 
 
 .SH "6\&.1\&.2: Discussion"
 
 .PP 
 The above shown results are quite favorable to crossroads\&. However,
 one should know that situations will exist where crossroads leans
 towards the \&'worst case\&' scenario, causing up to 50%
 delay\&.
 .PP 
 E\&.g\&., imagine a test where a \f(CWwget\fP command retrieves a
 HTML document from an Apache server on \f(CWlocalhost\fP\&. Now we have
 (almost) no overhead due to network throttling, hostname lookups and
 so on\&. When this test would be run either with or without crossroads
 in between, then theoretically, crossroads would cause a much larger
 delay, because it has to read from the server, and then write the same
 information to \f(CWwget\fP\&.  Each read/write occurs twice when crossroads
 sits in between\&.
 .PP 
 This worst case scenario will however (fortunately) occur only very
 seldom in the real world:
 .PP 
 .IP o 
 Normally network issues, such as the above mentioned host
 name lookups or throughput restrictions, will add
 significantly to the duration of a request\&. The \&'twice as
 many\&' read/writes caused by crossroads are then relatively
 irrelevant\&.
 .IP 
 .IP o 
 Normally a significant amount of time will be spent in a
 back end, due to processing (e\&.g\&., when calling a servlet on a
 back end)\&. Again, this processing time will weigh much heavier
 than the multiple read/writes\&.
 .PP 
 
 .SH "6\&.2: Benchmark 2: Crossroads versus Linux Virtual Server (LVS)"
 
 .PP 
 LVS is a kernel-based balancer that acts like a masquerading
 firewall: TCP packets that arrive at the balancer are sent to one of
 the configured back ends\&. LVS has the advantage over crossroads that
 there is no stop-and-go in the transmission; in contrast, crossroads
 needs to send data via an internal buffer\&. Crossroads has the
 advantage that it offers instantaneous failover because it tries to
 contact the back end for upon each new TCP connection; in contrast,
 LVS isn\&'t aware of downtime of back ends (unless one implements an
 external heartbeat)\&. Also, crossroads offers more complex balancing
 than LVS\&.
 .PP 
 
 .SH "6\&.2\&.1: Environment"
 
 .PP 
 On the balancer, LVS was run on port 80, its forwarding set up for two
 equally weighted back ends, using \f(CWipvsadm\fP:
 .PP 
 .nf 
 ipvsadm -a -t 192\&.168\&.1\&.250:http -r 10\&.1\&.1\&.100:http -m -w 1
 ipvsadm -a -t 192\&.168\&.1\&.250:http -r 10\&.1\&.1\&.101:http -m -w 1
 .fi 
 
 .PP 
 Crossroads was run on port 81\&. The configuration file is shown below:
 .PP 
 .nf 
 service http {
     port 81;
     dispatchmode roundrobin;
     revivinginterval 5;
     backend one {
         server 10\&.1\&.1\&.100;
         port 80;
     }
     backend two {
         server 10\&.1\&.1\&.101;
         port 80;
     }
 }
 .fi 
 
 .PP 
 
 .SH "6\&.2\&.2: Tests and results"
 
 .PP 
 In the first test, ports 80 and 81 on the balancer were \&'bombed\&' with
 50 concurrent clients, each requesting a small page 50 times\&. The
 following timings where measured:
 .PP 
 .IP o 
 How long it takes to establish a connection;
 .IP o 
 How long it takes to retrieve the page\&.
 .PP 
 The results of this test were:
 .PP 
 .IP o 
 On average, each client took 0\&.12 seconds to connect
 to LVS, and each page was retrieved in 0\&.14 seconds;
 .IP o 
 On average, each client took 0\&.11 seconds to connect to
 crossroads, and each page was retrieved in 0\&.13 seconds\&.
 .PP 
 In this setup there seems to be no difference between the performance
 of LVS and crossroads!
 .PP 
 In a second test, the size of the retrieved page was varied from 2\&.000
 to 2\&.000\&.000 bytes\&. This test was taken to see whether crossroads would
 show performance degradation when transferring larger amounts of data\&.
 .PP 
 For each page size, 30 concurrent clients were started, that retrieved
 the page 50 times\&. Again, the connect times and processing times where
 recorded\&.
 .PP 
 The results of the total time (connect time + retrieval time)
 are shown in the below table:
 .PP 
 .TS 
  tab(~);
 
 
 
 
 
 
 
 
 
 
 
 
 ---
 rrr
 rrr
 rrr
 rrr
 rrr
 ---
 c.
 \fBBytes\fP~\fBLVS timing\fP~\fBCrossroads timing\fP
 2000~0\&.130741688~0\&.12739582
 20000~0\&.490916224~0\&.50376901
 200000~3\&.799440328~4\&.33125273
 2000000~45\&.25090855~45\&.9600728
 .TE 
 
 .PP 
 Again, the results show that crossroads performs just as effectively
 as LVS, even with large data chunks!
 .PP 
 
 .SH "7: Compiling and Installing"
 
 
 .SH "7\&.1: Prerequisites"
 
 .PP 
 The creation of crossroads requires:
 .PP 
 .IP o 
 Standard Unix tools, such as \f(CWsed\fP, \f(CWawk\fP, \f(CWPerl\fP
 (5\&.00 or better);
 .IP 
 .IP o 
 A POSIX-compliant C compiler;
 .IP 
 .IP o 
 Support for SYSV IPC, networking and so on\&.
 
 .PP 
 Basically a Linux or Apple MacOSX box will do nicely\&. To compile and install
 crossroads, follow these steps\&.
 .PP 
 
 .SH "7\&.2: Compiling and installing"
 
 .PP 
 .IP o 
 Obtain the source distribution\&. It can be found on
 http://crossroads\&.e-tunity\&.com\&. The distribution comes as an
 archive \f(CWcrossroads-\fP\fItype\fP\f(CW\&.tar\&.gz\fP, where \fItype\fP is
 \f(CWstable\fP or \f(CWdevel\fP\&.
 .IP 
 .IP o 
 Unpack the archive in a sources directory using \f(CWtar
 xzf crossroads-\fP\fIX\&.YY\fP\f(CW\&.tar\&.gz\fP\&. The contents spill into a
 subdirectory \f(CWcrossroads-\fP\fIX\&.YY/\fP\&.
 .IP 
 .IP o 
 Change-dir into the directory\&.
 .IP 
 .IP o 
 Next, edit \f(CWetc/Makefile\&.def\fP and verify that all
 compilation settings are to your likings\&. The settings are
 explained in the file\&. \fBNote that\fP the default distribution
 of \f(CWMakefile\&.def\fP is suited for Linux or Apple MacOSX
 systems\&. On other Unices, or on non-Unix systems, you must
 particularly pay attention to \f(CWSET_PROC_TITLE_BY\&.\&.\&.\fP\&. When
 in doubt, comment out all \f(CWSET_PROC_TITLE\&.\&.\&.\fP
 settings\&. Crossroads will work nevertheless, but it won\&'t show
 nice titles in \f(CWps\fP listings\&. Also there\&'s a macro
 \f(CWEXTRA_LIBS\fP to add linkage flags (an example for a Solaris
 build is included)\&.
 .IP 
 .IP o 
 Now crossroads is ready for compilation\&. Do a \f(CWmake
 local\fP followed by \f(CWmake install\fP\&. The latter step may have
 to be done by the user \f(CWroot\fP if the \f(CWBINDIR\fP setting of
 \f(CWetc/Makefile\&.def\fP points to a root-owned directory\&.
 .IP 
 .IP o 
 The documentation doesn\&'t install in this process\&. If you
 want to install the documentation, then proceed as follows:
 .IP 
 .IP o 
 Optionally, \f(CWcp doc/crossroads\&.html\fP
 \fIhtmldirectory/\fP; where \fIhtmldirectory\fP is the destination
 directory for your HTML manuals;
 .IP 
 .IP o 
 Optionally, \f(CWcp doc/crossroads\&.pdf\fP
 \fIpdfdirectory/\fP; where \fIpdfdirectory\fP is the
 destination directory for your PDF manuals;
 .IP 
 .IP o 
 Optionally, \f(CWcp doc/crossroads\&.man\fP
 \fImanualdirectory\fP\f(CW/crossroads\&.1\fP, where
 \fImanualdirectory\fP is e\&.g\&. \f(CW/usr/man/man1\fP,
 \f(CW/usr/share/man1\fP, \f(CW/usr/local/man/man1\fP,
 \f(CW/usr/local/share/man1\fP\&. Any possibility is valid, as
 long as \fImanualdirectory\fP is one of the directories
 where manual pages are stored;
 .IP 
 .IP o 
 If your manual page system supports compressed
 manual pages, then you can save some space with
 \f(CWgzip\fP \fImanualdirectory\fP\f(CW/crossroads\&.1\fP\&.
 .IP 
 
 .SH "7\&.3: Configuring crossroads"
 
 .PP 
 Now that the binary is available on your system, you need to create a
 suitable \f(CW/etc/crossroads\&.conf\fP\&. Use this manual or the output of
 \f(CWcrossroads samplconf\fP to get started\&.
 .PP 
 Once you have the configuration ready, start crossroads with
 \f(CWcrossroads start\fP\&. Test the availability of your services and back
 ends\&. Monitor how crossroads is doing with:
 .PP 
 .IP o 
 In one terminal, run the script:
 .nf 
 while [ 1 ] ; do
     tput clear
     crossroads status
     sleep 3
 done
 .fi 
 
 .IP 
 \fBNote\fP that depending on your system you might need
 \f(CWsleep 3s\fP, i\&.e\&., with an \f(CWs\fP appended\&.
 .IP 
 .IP o 
 In another terminal, run:
 .nf 
 while [ 1 ] ; do
     tput clear
     ps ax | grep crossroads | grep -v grep
     sleep 3		    	
 done
 .fi 
 
 .IP 
 \fBNote\fP that depending on your system you might need
 \f(CWps -ef\fP instead of \f(CWps ax\fP\&.
 .IP 
 .IP o 
 In yet another terminal, run \f(CWtail -f
 /var/log/messages\fP (supply the appropriate system log file if
 \f(CW/var/log/messages\fP doesn\&'t work for you)\&.
 .PP 
 Now thoroughly test the availability of your back ends through
 crossroads\&. The status display will show an updated view of which back
 ends are selected and how busy they are\&. The process list will show
 which crossroads daemons are running\&. Finally, the tailing of
 \f(CW/var/log/messages\fP shows what\&'s going on -- especially if you have
 \f(CWverbosity true\fP statements in the configuration\&.
 .PP 
 
 .SH "7\&.4: A boot script"
 
 .PP 
 Finally, you may want to create a boot-time startup script\&. The exact
 procedure depends on the used Unix flavor\&.
 .PP 
 
 .SH "7\&.4\&.1: SysV Style Startup"
 
 .PP 
 On SysV style systems, there\&'s a startup script directory
 \f(CW/etc/init\&.d\fP where bootscripts for all utilities are located\&.
 You may have the \f(CWchkconfig\fP utility to automate the task of
 inserting scripts into the boot sequence, but
 otherwise the steps will resemble the following\&.
 .PP 
 .IP o 
 Create a script \f(CWcrossroads\fP in \f(CW/etc/init\&.d\fP similar to the
 following:
 .IP 
 .nf 
 #!/bin/sh
 /usr/local/bin/crossroads -v $@
 .fi 
 
 .IP 
 The stated directory \f(CW/usr/local/bin\fP must correspond with
 the installation path\&. The flag \f(CW-v\fP causes the startup to
 be more \&'verbose\&'\&. However, once daemonized, the verbosity is
 controlled by the appropriate statements in the configuration\&.
 .IP 
 .IP o 
 Determine your \&'runlevel\&': usually 3 when your system is
 running in text-mode only, or 5 when you are using a graphical
 interface\&. If your runlevel is 3, then:
 .IP 
 .nf 
 root> cd /etc/rc\&.d/rc3\&.d
 root> ln -s /etc/init\&.d/crossroads S99crossroads
 root> ln -s /etc/init\&.d/crossroads K99crossroads
 .fi 
 
 .IP 
 This creates startup (\f(CWS*\fP) and stop (\f(CWK*\fP) links that
 will be run when the system enters or leaves a given runlevel\&.
 .IP 
 If your runlevel is 5, then the right \f(CWcd\fP command is to
 \f(CW/etc/rc\&.d/rc5\&.d\fP\&. Alternatively, you can create the
 symlinks in both runlevel directories\&.
 .PP 
 
 .SH "7\&.4\&.2: BSD Style Startup"
 
 .PP 
 On BSD style systems, daemons are booted directly from \f(CW/etc/rc\fP and
 related scripts\&. Incase you have a file \f(CW/etc/rc\&.local\fP, edit it,
 and add the statement:
 .PP 
 .nf 
 /usr/local/bin/crossroads start
 .fi 
 
 .PP 
 If your BSD system lacks \f(CW/etc/rc\&.local\fP, then you may need to start
 Crossroads from \f(CW/etc/rc\fP\&. Your mileage may vary\&.
 .PP