XML update.

[apache] / docs / manual / mod / event.xml

<?xml version="1.0"?>
<!DOCTYPE modulesynopsis SYSTEM "../style/modulesynopsis.dtd">
<?xml-stylesheet type="text/xsl" href="../style/manual.en.xsl"?>
<!-- $LastChangedRevision$ -->

<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<modulesynopsis metafile="event.xml.meta">
<name>event</name>
<description>A variant of the <module>worker</module> MPM with the goal
of consuming threads only for connections with active processing</description>
<status>MPM</status>
<sourcefile>event.c</sourcefile>
<identifier>mpm_event_module</identifier>

<summary>
    <p>The <module>event</module> Multi-Processing Module (MPM) is,
    as its name implies, an asynchronous, event-based implementation
    designed to allow more requests to be served simultaneously by
    passing off some processing work to the listeners threads, freeing up
    the worker threads to serve new requests.</p>

    <p>To use the <module>event</module> MPM, add
      <code>--with-mpm=event</code> to the <program>configure</program>
      script's arguments when building the <program>httpd</program>.</p>

</summary>

<seealso><a href="worker.html">The worker MPM</a></seealso>

<section id="event-worker-relationship"><title>Relationship with the Worker MPM</title>
<p><module>event</module> is based on the <module>worker</module> MPM, which implements a hybrid
multi-process multi-threaded server. A single control process (the parent) is responsible for launching
child processes. Each child process creates a fixed number of server
threads as specified in the <directive module="mpm_common">ThreadsPerChild</directive> directive, as well
as a listener thread which listens for connections and passes them to a worker thread for processing when they arrive.</p>

<p>Run-time configuration directives are identical to those provided by <module>worker</module>, with the only addition
of the <directive>AsyncRequestWorkerFactor</directive>.</p>

</section>

<section id="how-it-works"><title>How it Works</title>
    <p>This original goal of this MPM was to fix the 'keep alive problem' in HTTP. After a client
    completes the first request, it can keep the connection
    open, sending further requests using the same socket and saving
    significant overhead in creating TCP connections. However,
    Apache HTTP Server traditionally keeps an entire child
    process/thread waiting for data from the client, which brings its own disadvantages.
    To solve this problem, this MPM uses a dedicated listener thread for each process
    along with a pool of worker threads, sharing queues specific for those
    requests in keep-alive mode (or, more simply, "readable"), those in write-
    completion mode, and those in the process of shutting down ("closing").
    An event loop, triggered on the status of the socket's availability,
    adjusts these queues and pushes work to the worker pool.
    </p>

    <p>This new architecture, leveraging non-blocking sockets and modern kernel
       features exposed by <glossary>APR</glossary> (like Linux's epoll),
       no longer requires the <code>mpm-accept</code> <directive module="core">Mutex</directive>
       configured to avoid the thundering herd problem.</p>

    <p>The total amount of connections that a single process/threads block can handle is regulated
        by the <directive>AsyncRequestWorkerFactor</directive> directive.</p>

    <section id="async-connections"><title>Async connections</title>
        <p>Async connections would need a fixed dedicated worker thread with the previous MPMs but not with event.
        The status page of <module>mod_status</module> shows new columns under the Async connections section:</p>
        <dl>
            <dt>Writing</dt>
            <dd>While sending the response to the client, it might happen that the TCP write buffer fills up because the connection is too slow. Usually in this case a <code>write()</code> to the socket returns <code>EWOULDBLOCK</code> or <code>EAGAIN</code>, to become writable again after an idle time. The worker holding the socket might be able to offload the waiting task to the listener thread, that in turn will re-assign it to the first idle worker thread available once an event will be raised for the socket (for example, "the socket is now writable"). Please check the Limitations section for more information.
            </dd>

            <dt>Keep-alive</dt>
            <dd>Keep Alive handling is the most basic improvement from the worker MPM.
            Once a worker thread finishes to flush the response to the client, it can offload the
            socket handling to the listener thread, that in turns will wait for any event from the
            OS, like "the socket is readable". If any new request comes from the client, then the
            listener will forward it to the first worker thread available. Conversely, if the
            <directive module="core">KeepAliveTimeout</directive> occurs then the socket will be
            closed by the listener. In this way the worker threads are not responsible for idle
            sockets and they can be re-used to serve other requests.</dd>

            <dt>Closing</dt>
            <dd>Sometimes the MPM needs to perform a lingering close, namely sending back an early error to the client while it is still transmitting data to httpd.
            Sending the response and then closing the connection immediately is not the correct thing to do since the client (still trying to send the rest of the
            request) would get a connection reset and could not read the httpd's response. The lingering close is time bounded but it can take relatively long
            time, so it's offloaded to a worker thread (including the shutdown hooks and real socket close). From 2.4.28 onward this is also the
            case when connections finally timeout (the listener thread never handles connections besides waiting for and dispatching their events).
            </dd>
        </dl>

        <p>These improvements are valid for both HTTP/HTTPS connections.</p>

        <p>The above connection states are managed by the listener thread via dedicated queues, that up to 2.4.27 were checked every 100ms
        to find which connections hit timeout settings like <directive module="mpm_common">Timeout</directive> and
        <directive module="core">KeepAliveTimeout</directive>. This was a simple and efficient solution, but it presented a downside: the pollset was
        forcing a wake-up of the listener thread even if there was no need (for example because completely idle), wasting resources. From 2.4.28
        these queues are completely managed via an event based logic, not relying anymore on active polling.
        Resource constrained environments, like embedded servers, may benefit from this improvement.</p>

    </section>

    <section id="graceful-close"><title>Graceful process termination and Scoreboard usage</title>
        <p>This mpm showed some scalability bottlenecks in the past leading to the following
        error: "<strong>scoreboard is full, not at MaxRequestWorkers</strong>".
        <directive module="mpm_common">MaxRequestWorkers</directive>
        limits the number of simultaneous requests that will be served at any given time
        and also the number of allowed processes
        (<directive module="mpm_common">MaxRequestWorkers</directive> 
        / <directive module="mpm_common">ThreadsPerChild</directive>), meanwhile
        the Scoreboard is a representation of all the running processes and
        the status of their worker threads. If the scoreboard is full (so all the
        threads have a state that is not idle) but the number of active requests
        served is not <directive module="mpm_common">MaxRequestWorkers</directive>,
        it means that some of them are blocking new requests that could be served
        but that are queued instead (up to the limit imposed by
        <directive module="mpm_common">ListenBacklog</directive>). Most of the times
        the threads are stuck in the Graceful state, namely they are waiting to
        finish their work with a TCP connection to safely terminate and free up a
        scoreboard slot (for example handling long running requests, slow clients
        or connections with keep-alive enabled). Two scenarios are very common:</p>
        <ul>
            <li>During a <a href="../stopping.html#graceful">graceful restart</a>.
            The parent process signals all its children to complete
            their work and terminate, while it reloads the config and forks new
            processes. If the old children keep running for a while before stopping,
            the scoreboard will be partially occupied until their slots are freed.
            </li>
            <li>When the server load goes down in a way that causes httpd to
            stop some processes (for example due to
            <directive module="mpm_common">MaxSpareThreads</directive>).
            This is particularly problematic because when the load increases again,
            httpd will try to start new processes.
            If the pattern repeats, the number of processes can rise quite a bit,
            ending up in a mixture of old processes trying to stop and new ones
            trying to do some work.
            </li>
        </ul>
        <p>From 2.4.24 onward, mpm-event is smarter and it is able to handle
        graceful terminations in a much better way. Some of the improvements are:</p>
        <ul>
            <li>Allow the use of all the scoreboard slots up to
            <directive module="mpm_common">ServerLimit</directive>.
            <directive module="mpm_common">MaxRequestWorkers</directive> and
            <directive module="mpm_common">ThreadsPerChild</directive> are used
            to limit the amount of active processes, meanwhile
            <directive module="mpm_common">ServerLimit</directive> 
            takes also into account the ones doing a graceful
            close to allow extra slots when needed. The idea is to use
            <directive module="mpm_common">ServerLimit</directive> to instruct httpd
            about how many overall processes are tolerated before impacting
            the system resources.
            </li>
            <li>Force gracefully finishing processes to close their
            connections in keep-alive state.</li>
            <li>During graceful shutdown, if there are more running worker threads
            than open connections for a given process, terminate these threads to
            free resources faster (which may be needed for new processes).</li>
            <li>If the scoreboard is full, prevent more processes to finish
            gracefully due to reduced load until old processes have terminated
            (otherwise the situation would get worse once the load increases again).</li>
        </ul>
        <p>The behavior described in the last point is completely observable via
        <module>mod_status</module> in the connection summary table through two new
        columns: "Slot" and "Stopping". The former indicates the PID and
        the latter if the process is stopping or not; the extra state "Yes (old gen)"
        indicates a process still running after a graceful restart.</p>
    </section>

    <section id="limitations"><title>Limitations</title>
        <p>The improved connection handling may not work for certain connection
        filters that have declared themselves as incompatible with event. In these
        cases, this MPM will fall back to the behavior of the
        <module>worker</module> MPM and reserve one worker thread per connection.
        All modules shipped with the server are compatible with the event MPM.</p>

        <p>A similar restriction is currently present for requests involving an
        output filter that needs to read and/or modify the whole response body.
        If the connection to the client blocks while the filter is processing the
        data, and the amount of data produced by the filter is too big to be
        buffered in memory, the thread used for the request is not freed while
        httpd waits until the pending data is sent to the client.<br />
        To illustrate this point we can think about the following two situations:
        serving a static asset (like a CSS file) versus serving content retrieved from
        FCGI/CGI or a proxied server. The former is predictable, namely the event MPM
        has full visibility on the end of the content and it can use events: the worker
        thread serving the response content can flush the first bytes until <code>EWOULDBLOCK</code>
        or <code>EAGAIN</code> is returned, delegating the rest to the listener. This one in turn
        waits for an event on the socket, and delegates the work to flush the rest of the content
        to the first idle worker thread. Meanwhile in the latter example (FCGI/CGI/proxied content)
        the MPM can't predict the end of the response and a worker thread has to finish its work
        before returning the control to the listener. The only alternative is to buffer the
        response in memory, but it wouldn't be the safest option for the sake of the
        server's stability and memory footprint.
        </p>

    </section>

    <section id="background"><title>Background material</title>
        <p>The event model was made possible by the introduction of new APIs into the supported operating systems:</p>
        <ul>
            <li>epoll (Linux) </li>
            <li>kqueue (BSD) </li>
            <li>event ports (Solaris) </li>
        </ul>
        <p>Before these new APIs where made available, the traditional <code>select</code> and <code>poll</code> APIs had to be used.
        Those APIs get slow if used to handle many connections or if the set of connections rate of change is high.
        The new APIs allow to monitor much more connections and they perform way better when the set of connections to monitor changes frequently. So these APIs made it possible to write the event MPM, that scales much better with the typical HTTP pattern of many idle connections.</p>

        <p>The MPM assumes that the underlying <code>apr_pollset</code>
        implementation is reasonably threadsafe. This enables the MPM to
        avoid excessive high level locking, or having to wake up the listener
        thread in order to send it a keep-alive socket. This is currently
        only compatible with KQueue and EPoll.</p>

    </section>

</section>

<section id="requirements"><title>Requirements</title>
    <p>This MPM depends on <glossary>APR</glossary>'s atomic
    compare-and-swap operations for thread synchronization. If you are
    compiling for an x86 target and you don't need to support 386s, or
    you are compiling for a SPARC and you don't need to run on
    pre-UltraSPARC chips, add
    <code>--enable-nonportable-atomics=yes</code> to the
    <program>configure</program> script's arguments. This will cause
    APR to implement atomic operations using efficient opcodes not
    available in older CPUs.</p>

    <p>This MPM does not perform well on older platforms which lack good
    threading, but the requirement for EPoll or KQueue makes this
    moot.</p>

    <ul>

      <li>To use this MPM on FreeBSD, FreeBSD 5.3 or higher is recommended.
      However, it is possible to run this MPM on FreeBSD 5.2.1, if you
      use <code>libkse</code> (see <code>man libmap.conf</code>).</li>

      <li>For NetBSD, at least version 2.0 is recommended.</li>

      <li>For Linux, a 2.6 kernel is recommended. It is also necessary to
      ensure that your version of <code>glibc</code> has been compiled
      with support for EPoll.</li>

    </ul>
</section>

<directivesynopsis location="mpm_common"><name>CoreDumpDirectory</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>EnableExceptionHook</name>
</directivesynopsis>
<directivesynopsis location="mod_unixd"><name>Group</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>Listen</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ListenBacklog</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>SendBufferSize</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>MaxRequestWorkers</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>MaxMemFree</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>MaxConnectionsPerChild</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>MaxSpareThreads</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>MinSpareThreads</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>PidFile</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ScoreBoardFile</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ServerLimit</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>StartServers</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ThreadLimit</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ThreadsPerChild</name>
</directivesynopsis>
<directivesynopsis location="mpm_common"><name>ThreadStackSize</name>
</directivesynopsis>
<directivesynopsis location="mod_unixd"><name>User</name>
</directivesynopsis>

<directivesynopsis>
<name>AsyncRequestWorkerFactor</name>
<description>Limit concurrent connections per process</description>
<syntax>AsyncRequestWorkerFactor <var>factor</var></syntax>
<default>2</default>
<contextlist><context>server config</context> </contextlist>
<compatibility>Available in version 2.3.13 and later</compatibility>

<usage>
    <p>The event MPM handles some connections in an asynchronous way, where
    request worker threads are only allocated for short periods of time as
    needed, and other connections with one request worker thread reserved per
    connection. This can lead to situations where all workers are tied up and
    no worker thread is available to handle new work on established async
    connections.</p>

    <p>To mitigate this problem, the event MPM does two things:</p>
    <ul>
        <li>it limits the number of connections accepted per process, depending on the
            number of idle request workers;</li>
        <li>if all workers are busy, it will
            close connections in keep-alive state even if the keep-alive timeout has
            not expired. This allows the respective clients to reconnect to a
            different process which may still have worker threads available.</li>
    </ul>

    <p>This directive can be used to fine-tune the per-process connection
    limit. A <strong>process</strong> will only accept new connections if the current number of
    connections (not counting connections in the "closing" state) is lower
    than:</p>

    <p class="indent"><strong>
        <directive module="mpm_common">ThreadsPerChild</directive> +
        (<directive>AsyncRequestWorkerFactor</directive> *
        <var>number of idle workers</var>)
    </strong></p>

    <p>An estimation of the maximum concurrent connections across all the processes given
        an average value of idle worker threads can be calculated with:
    </p>


    <p class="indent"><strong>
        (<directive module="mpm_common">ThreadsPerChild</directive> +
        (<directive>AsyncRequestWorkerFactor</directive> *
        <var>number of idle workers</var>)) *
        <directive module="mpm_common">ServerLimit</directive>
    </strong></p>

    <note><title>Example</title>
    <highlight language="config">

ThreadsPerChild = 10
ServerLimit = 4
AsyncRequestWorkerFactor = 2
MaxRequestWorkers = 40

idle_workers = 4 (average for all the processes to keep it simple)

max_connections = (ThreadsPerChild + (AsyncRequestWorkerFactor * idle_workers)) * ServerLimit
                = (10 + (2 * 4)) * 4 = 72

    </highlight>
    </note>

    <p>When all the worker threads are idle, then absolute maximum numbers of concurrent
        connections can be calculared in a simpler way:</p>

    <p class="indent"><strong>
        (<directive>AsyncRequestWorkerFactor</directive> + 1) *
        <directive module="mpm_common">MaxRequestWorkers</directive>
    </strong></p>


    <note><title>Example</title>
    <highlight language="config">

ThreadsPerChild = 10
ServerLimit = 4
MaxRequestWorkers = 40
AsyncRequestWorkerFactor = 2

    </highlight>

    <p>If all the processes have all threads idle then: </p>

    <highlight language="config">idle_workers = 10</highlight>

    <p>We can calculate the absolute maximum numbers of concurrent connections in two ways:</p>

    <highlight language="config">

max_connections = (ThreadsPerChild + (AsyncRequestWorkerFactor * idle_workers)) * ServerLimit
                = (10 + (2 * 10)) * 4 = 120

max_connections = (AsyncRequestWorkerFactor + 1) * MaxRequestWorkers
                = (2 + 1) * 40 = 120

    </highlight>
    </note>

    <p>Tuning <directive>AsyncRequestWorkerFactor</directive> requires knowledge about the traffic handled by httpd in each specific use case, so changing the default value requires extensive testing and data gathering from <module>mod_status</module>.</p>

    <p><directive module="mpm_common">MaxRequestWorkers</directive> was called
    <directive>MaxClients</directive> prior to version 2.3.13. The above value
    shows that the old name did not accurately describe its meaning for the event MPM.</p>

    <p><directive>AsyncRequestWorkerFactor</directive> can take non-integer
    arguments, e.g "1.5".</p>

</usage>

</directivesynopsis>

</modulesynopsis>