<!-- $LastChangedRevision$ -->
<!--
- Copyright 2002-2004 The Apache Software Foundation
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
<modulesynopsis metafile="mod_proxy_balancer.xml.meta">
<name>mod_proxy_balancer</name>
-<description><module>mod_proxy</module> extension for
-load balancing </description>
+<description><module>mod_proxy</module> extension for load balancing </description>
<status>Extension</status>
-<sourcefile>proxy_balancer.c</sourcefile>
+<sourcefile>mod_proxy_balancer.c</sourcefile>
<identifier>proxy_balancer_module</identifier>
<compatibility>Available in version 2.1 and later</compatibility>
<code>HTTP</code>, <code>FTP</code> and <code>AJP13</code> protocols
</p>
+ <p>Load balancing scheduler algorithm is provided by not this
+ module but other modules such as:
+ <module>mod_lbmethod_byrequests</module>,
+ <module>mod_lbmethod_bytraffic</module> and
+ <module>mod_lbmethod_bybusyness</module>.
+ </p>
+
<p>Thus, in order to get the ability of load balancing,
- <module>mod_proxy</module> and <module>mod_proxy_balancer</module>
- have to be present in the server.</p>
+ <module>mod_proxy</module>, <module>mod_proxy_balancer</module>
+ and at least one of load balancing scheduler algorithm modules have
+ to be present in the server.</p>
<note type="warning"><title>Warning</title>
<p>Do not enable proxying until you have <a
<section id="scheduler">
<title>Load balancer scheduler algorithm</title>
- <p>The idea behind this scheduler is the following:</p>
+ <p>At present, there are 3 load balancer scheduler algorithms available
+ for use: Request Counting, Weighted Traffic Counting and Pending Request
+ Counting. These are controlled via the <code>lbmethod</code> value of
+ the Balancer definition. See the <directive module="mod_proxy">ProxyPass</directive>
+ directive for more information.</p>
+</section>
- <p><dfn>lbfactor</dfn> is <em>how much we expect this worker
- to work</em>, or <em>the workers's work quota</em>.</p>
+<section id="stickyness">
+ <title>Load balancer stickyness</title>
+ <p>The balancer supports stickyness. When a request is proxied
+ to some back-end, then all following requests from the same user
+ should be proxied to the same back-end. Many load balancers implement
+ this feature via a table that maps client IP addresses to back-ends.
+ This approach is transparent to clients and back-ends, but suffers
+ from some problems: unequal load distribution if clients are themselves
+ hidden behind proxies, stickyness errors when a client uses a dynamic
+ IP address that changes during a session and loss of stickyness, if the
+ mapping table overflows.</p>
+ <p>The module <module>mod_proxy_balancer</module> implements stickyness
+ on top of two alternative means: cookies and URL encoding. Providing the
+ cookie can be either done by the back-end or by the Apache web server
+ itself. The URL encoding is usually done on the back-end.</p>
+</section>
- <p><dfn>lbstatus</dfn> is <em>how urgent this worker has to work
- to fulfill its quota of work</em>.</p>
+<section id="example">
+ <title>Examples of a balancer configuration</title>
+ <p>Before we dive into the technical details, here's an example of
+ how you might use <module>mod_proxy_balancer</module> to provide
+ load balancing between two back-end servers:
+ </p>
- <p>The <dfn>worker</dfn> is a member of the load balancer,
- usually a remote host serving one of the supported protocols.</p>
+ <example>
+ <Proxy balancer://mycluster><br />
+ BalancerMember http://192.168.1.50:80<br />
+ BalancerMember http://192.168.1.51:80<br />
+ </Proxy><br />
+ ProxyPass /test balancer://mycluster
+ </example>
- <p>We distribute each worker's work quota to the worker, and then look
- which of them needs to work most urgently (biggest lbstatus). This
- worker is then selected for work, and its lbstatus reduced by the
- total work quota we distributed to all workers. Thus the sum of all
- lbstatus does not change.(*)</p>
+ <p>Another example of how to provide load balancing with stickyness
+ using <module>mod_headers</module>, even if the back-end server does
+ not set a suitable session cookie:
+ </p>
- <p>If some workers are disabled, the others will
- still be scheduled correctly.</p>
+ <example>
+ Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/"
+ env=BALANCER_ROUTE_CHANGED<br />
+ <Proxy balancer://mycluster><br />
+ BalancerMember http://192.168.1.50:80 route=1<br />
+ BalancerMember http://192.168.1.51:80 route=2<br />
+ ProxySet stickysession=ROUTEID<br />
+ </Proxy><br />
+ ProxyPass /test balancer://mycluster
+ </example>
+</section>
- <example><pre><code>for each worker in workers
- worker lbstatus += worker lbfactor
- total factor += worker lbfactor
- if worker lbstatus > candidate lbstatus
- candidate = worker
+<section id="environment">
+ <title>Exported Environment Variables</title>
+ <p>At present there are 6 environment variables exported:</p>
-candidate lbstatus -= total factor</code></pre>
- </example>
+ <dl>
+ <!-- ============= BALANCER_SESSION_STICKY =============== -->
+ <dt><var><a name="balancer_session_sticky" id="balancer_session_sticky">BALANCER_SESSION_STICKY</a></var></dt>
+ <dd>
+ <p>This is assigned the <var>stickysession</var> value used for the current
+ request. It is the name of the cookie or request parameter used for sticky sessions</p>
+ </dd>
+
+ <!-- ============= BALANCER_SESSION_ROUTE ================ -->
+ <dt><var><a name="balancer_session_route" id="balancer_session_route">BALANCER_SESSION_ROUTE</a></var></dt>
+ <dd>
+ <p>This is assigned the <var>route</var> parsed from the current
+ request.</p>
+ </dd>
+
+ <!-- ============= BALANCER_NAME ========================= -->
+ <dt><var><a name="balancer_name" id="balancer_name">BALANCER_NAME</a></var></dt>
+ <dd>
+ <p>This is assigned the name of the balancer used for the current
+ request. The value is something like <code>balancer://foo</code>.</p>
+ </dd>
+
+ <!-- ============= BALANCER_WORKER_NAME ================== -->
+ <dt><var><a name="balancer_worker_name" id="balancer_worker_name">BALANCER_WORKER_NAME</a></var></dt>
+ <dd>
+ <p>This is assigned the name of the worker used for the current request.
+ The value is something like <code>http://hostA:1234</code>.</p>
+ </dd>
+
+ <!-- ============= BALANCER_WORKER_ROUTE ================= -->
+ <dt><var><a name="balancer_worker_route" id="balancer_worker_route">BALANCER_WORKER_ROUTE</a></var></dt>
+ <dd>
+ <p>This is assigned the <var>route</var> of the worker that will be
+ used for the current request.</p>
+ </dd>
+
+ <!-- ============= BALANCER_ROUTE_CHANGED ================= -->
+ <dt><var><a name="balancer_route_changed" id="balancer_route_changed">BALANCER_ROUTE_CHANGED</a></var></dt>
+ <dd>
+ <p>This is set to 1 if the session route does not match the
+ worker route (BALANCER_SESSION_ROUTE != BALANCER_WORKER_ROUTE) or the
+ session does not yet have an established route. This can be used to
+ determine when/if the client needs to be sent an updated route
+ when sticky sessions are used.</p>
+ </dd>
+ </dl>
- <p>If a balancer is configured as follows:</p>
-
- <table style="data">
- <tr><th>worker</th>
- <th>a</th>
- <th>b</th>
- <th>c</th>
- <th>d</th></tr>
- <tr><th>lbfactor</th>
- <td>25</td>
- <td>25</td>
- <td>25</td>
- <td>25</td></tr>
- <tr><th>lbstatus</th>
- <td>0</td>
- <td>0</td>
- <td>0</td>
- <td>0</td></tr>
- </table>
-
- <p>And <var>b</var> gets disabled, the following schedule is produced:</p>
-
- <table style="data">
- <tr><th>worker</th>
- <th>a</th>
- <th>b</th>
- <th>c</th>
- <th>d</th></tr>
- <tr><th>lbstatus</th>
- <td><em>-50</em></td>
- <td>0</td>
- <td>25</td>
- <td>25</td></tr>
- <tr><th>lbstatus</th>
- <td>-25</td>
- <td>0</td>
- <td><em>-25</em></td>
- <td>50</td></tr>
- <tr><th>lbstatus</th>
- <td>0</td>
- <td>0</td>
- <td>0</td>
- <td><em>0</em></td></tr>
- <tr><td colspan="5">(repeat)</td></tr>
- </table>
-
- <p>That is it schedules: <var>a</var> <var>c</var> <var>d</var>
- <var>a</var> <var>c</var> <var>d</var> <var>a</var> <var>c</var>
- <var>d</var> ...</p>
-
- <p>The following asymmetric configuration works as one would expect:</p>
-
- <table style="data">
- <tr><th>worker</th>
- <th>a</th>
- <th>b</th></tr>
- <tr><th>lbfactor</th>
- <td>70</td>
- <td>30</td></tr>
- <tr><td colspan="2"> </td></tr>
- <tr><th>lbstatus</th>
- <td><em>-30</em></td>
- <td>30</td></tr>
- <tr><th>lbstatus</th>
- <td>40</td>
- <td><em>-40</em></td></tr>
- <tr><th>lbstatus</th>
- <td><em>10</em></td>
- <td>-10</td></tr>
- <tr><th>lbstatus</th>
- <td><em>-20</em></td>
- <td>20</td></tr>
- <tr><th>lbstatus</th>
- <td><em>-50</em></td>
- <td>50</td></tr>
- <tr><th>lbstatus</th>
- <td>20</td>
- <td><em>-20</em></td></tr>
- <tr><th>lbstatus</th>
- <td><em>-10</em></td>
- <td>10</td></tr>
- <tr><th>lbstatus</th>
- <td><em>-40</em></td>
- <td>40</td></tr>
- <tr><th>lbstatus</th>
- <td>30</td>
- <td><em>-30</em></td></tr>
- <tr><th>lbstatus</th>
- <td><em>0</em></td>
- <td>0</td></tr>
- <tr><td colspan="3">(repeat)</td></tr>
- </table>
-
- <p>That is after 10 schedules, the schedule repeats and 7 <var>a</var>
- are selected with 3 <var>b</var> interspersed.</p>
</section>
-<section id="enable">
+<section id="balancer_manager">
<title>Enabling Balancer Manager Support</title>
<p>This module <em>requires</em> the service of
<module>mod_status</module>.
<module>mod_status</module> and <module>mod_proxy_balancer</module>
have to be present in the server.</p>
- <p>To enable load balancer management for browsers from the foo.com
+ <p>To enable load balancer management for browsers from the example.com
domain add this code to your <code>httpd.conf</code>
configuration file</p>
<example>
<br />
Order Deny,Allow<br />
Deny from all<br />
- Allow from .foo.com<br />
+ Allow from .example.com<br />
</Location>
</example>
<code>http://your.server.name/balancer-manager</code></p>
</section>
+<section id="stickyness_implementation">
+ <title>Details on load balancer stickyness</title>
+ <p>When using cookie based stickyness, you need to configure the
+ name of the cookie that contains the information about which back-end
+ to use. This is done via the <var>stickysession</var> attribute added
+ to either <directive module="mod_proxy">ProxyPass</directive> or
+ <directive module="mod_proxy">ProxySet</directive>. The name of
+ the cookie is case-sensitive. The balancer extracts the value of the
+ cookie and looks for a member worker with <var>route</var> equal
+ to that value. The <var>route</var> must also be set in either
+ <directive module="mod_proxy">ProxyPass</directive> or
+ <directive module="mod_proxy">ProxySet</directive>. The cookie can either
+ be set by the back-end, or as shown in the above
+ <a href="#example">example</a> by the Apache web server itself.</p>
+ <p>Some back-ends use a slightly different form of stickyness cookie,
+ for instance Apache Tomcat. Tomcat adds the name of the Tomcat instance
+ to the end of its session id cookie, separated with a dot (<code>.</code>)
+ from the session id. Thus if the Apache web server finds a dot in the value
+ of the stickyness cookie, it only uses the part behind the dot to search
+ for the route. In order to let Tomcat know about its instance name, you
+ need to set the attribute <code>jvmRoute</code> inside the Tomcat
+ configuration file <code>conf/server.xml</code> to the value of the
+ <var>route</var> of the worker that connects to the respective Tomcat.
+ The name of the session cookie used by Tomcat (and more generally by Java
+ web applications based on servlets) is <code>JSESSIONID</code>
+ (upper case) but can be configured to something else.</p>
+ <p>The second way of implementing stickyness is URL encoding.
+ The web server searches for a query parameter in the URL of the request.
+ The name of the parameter is specified again using <var>stickysession</var>.
+ The value of the parameter is used to lookup a member worker with <var>route</var>
+ equal to that value. Since it is not easy to extract and manipulate all
+ URL links contained in responses, generally the work of adding the parameters
+ to each link is done by the back-end generating the content.
+ In some cases it might be feasible doing
+ this via the web server using <module>mod_substitute</module> or
+ <module>mod_sed</module>. This can have negative impact on performance though.</p>
+ <p>The Java standards implement URL encoding slightly different. They use
+ a path info appended to the URL using a semicolon (<code>;</code>)
+ as the separator and add the session id behind. As in the cookie case,
+ Apache Tomcat can include the configured <code>jvmRoute</code> in this path
+ info. To let Apache find this sort of path info, you neet to set
+ <code>scolonpathdelim</code> to <code>On</code> in
+ <directive module="mod_proxy">ProxyPass</directive> or
+ <directive module="mod_proxy">ProxySet</directive>.</p>
+ <p>Finally you can support cookies and URL encoding at the same time, by
+ configuring the name of the cookie and the name of the URL parameter
+ separated by a vertical bar (<code>|</code>) as in the following example:</p>
+ <example>
+ ProxyPass /test balancer://mycluster stickysession=JSESSIONID|jsessionid scolonpathdelim=On
+ <Proxy balancer://mycluster><br />
+ BalancerMember http://192.168.1.50:80 route=node1<br />
+ BalancerMember http://192.168.1.51:80 route=node2<br />
+ </Proxy><br />
+ </example>
+ <p>If the cookie and the request parameter both provide routing information
+ for the same request, the information from the request parameter is used.</p>
+</section>
+
+<section id="stickyness_troubleshooting">
+ <title>Troubleshooting load balancer stickyness</title>
+ <p>If you experience stickyness errors, e.g. users loose their
+ application sessions and need to login again, you first want to
+ check whether this is because the back-ends are sometimes unavailable
+ or whether your configuration is wrong. To find out about possible
+ stability problems with the back-ends, check your Apache error log
+ for proxy error messages.</p>
+ <p>To verify your configuration, first check, whether the stickyness
+ is based on a cookie or on URL encoding. Next step would be logging
+ the appropriate data in the access log by using an enhanced
+ <directive module="mod_log_config">LogFormat</directive>.
+ The following fields are useful:</p>
+ <dl>
+ <dt><code>%{MYCOOKIE}C</code></dt>
+ <dd>The value contained in the cookie with name <code>MYCOOKIE</code>.
+ The name should be the same given in the <var>stickysession</var>
+ attribute.</dd>
+ <dt><code>%{Set-Cookie}o</code></dt>
+ <dd>This logs any cookie set by the back-end. You can track,
+ whether the back-end sets the session cookie you expect, and
+ to which value it is set.</dd>
+ <dt><code>%{BALANCER_SESSION_STICKY}e</code></dt>
+ <dd>The name of the cookie or request parameter used
+ to lookup the routing information.</dd>
+ <dt><code>%{BALANCER_SESSION_ROUTE}e</code></dt>
+ <dd>The route information found in the request.</dd>
+ <dt><code>%{BALANCER_WORKER_ROUTE}e</code></dt>
+ <dd>The route of the worker chosen.</dd>
+ <dt><code>%{BALANCER_ROUTE_CHANGED}e</code></dt>
+ <dd>Set to <code>1</code> if the route in the request
+ is different from the route of the worker, i.e.
+ the request couldn't be handled sticky.</dd>
+ </dl>
+ <p>Common reasons for loss of session are session timeouts,
+ which are usually configurable on the back-end server.</p>
+ <p>The balancer also logs detailed information about handling
+ stickyness to the error log, if the log level is set to
+ <code>debug</code> or higher. This is an easy way to
+ troubleshoot stickyness problems, but the log volume might
+ be to high for production servers under high load.</p>
+</section>
+
+<directivesynopsis>
+<name>BalancerNonce</name>
+<description>Set the nonce used in the balancer-manager application</description>
+<syntax>BalancerNonce Default|None|Set "value"</syntax>
+<default>ProxyStatus Default</default>
+<contextlist><context>server config</context>
+<context>virtual host</context>
+</contextlist>
+<compatibility>Available in version 2.4 and later</compatibility>
+
+<usage>
+ <p>This directive specifies the protective nonce used in the
+ <code>balancer-manager</code> application page.</p>
+ <p>The default is to use an automatically determined UUID-based
+ nonce, to provide for further protection for the page. If set
+ to <code>Set</code>, then the next argument sets the nonce to that
+ value. A setting of <code>None</code> disables all nonce checking.</p>
+
+ <example>
+ BalancerNonce Set "RealGudSharedSecret"
+ </example>
+
+ <note><title>Note</title>
+ <p>In addition to the nonce, the <code>balancer-manager</code> page
+ should be protected via an ACL.</p>
+ </note>
+
+</usage>
+</directivesynopsis>
+
</modulesynopsis>
projects / apache / blobdiff