granicus.if.org Git - apache/blob - docs/manual/mod/mod_firehose.html.en

   1 <?xml version="1.0" encoding="ISO-8859-1"?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
   4 <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type" />
   5 <!--
   6         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
   7               This file is generated from xml source: DO NOT EDIT
   8         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
   9       -->
  10 <title>mod_firehose - Apache HTTP Server Version 2.5</title>
  11 <link href="../style/css/manual.css" rel="stylesheet" media="all" type="text/css" title="Main stylesheet" />
  12 <link href="../style/css/manual-loose-100pc.css" rel="alternate stylesheet" media="all" type="text/css" title="No Sidebar - Default font size" />
  13 <link href="../style/css/manual-print.css" rel="stylesheet" media="print" type="text/css" /><link rel="stylesheet" type="text/css" href="../style/css/prettify.css" />
  14 <script src="../style/scripts/prettify.min.js" type="text/javascript">
  15 </script>
  16
  17 <link href="../images/favicon.ico" rel="shortcut icon" /></head>
  18 <body>
  19 <div id="page-header">
  20 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p>
  21 <p class="apache">Apache HTTP Server Version 2.5</p>
  22 <img alt="" src="../images/feather.png" /></div>
  23 <div class="up"><a href="./"><img title="&lt;-" alt="&lt;-" src="../images/left.gif" /></a></div>
  24 <div id="path">
  25 <a href="http://www.apache.org/">Apache</a> &gt; <a href="http://httpd.apache.org/">HTTP Server</a> &gt; <a href="http://httpd.apache.org/docs/">Documentation</a> &gt; <a href="../">Version 2.5</a> &gt; <a href="./">Modules</a></div>
  26 <div id="page-content">
  27 <div id="preamble"><h1>Apache Module mod_firehose</h1>
  28 <div class="toplang">
  29 <p><span>Available Languages: </span><a href="../en/mod/mod_firehose.html" title="English">&nbsp;en&nbsp;</a></p>
  30 </div>
  31 <table class="module"><tr><th><a href="module-dict.html#Description">Description:</a></th><td>Multiplexes all I/O to a given file or pipe.</td></tr>
  32 <tr><th><a href="module-dict.html#Status">Status:</a></th><td>Extension</td></tr>
  33 <tr><th><a href="module-dict.html#ModuleIdentifier">Module Identifier:</a></th><td>firehose_module</td></tr>
  34 <tr><th><a href="module-dict.html#SourceFile">Source File:</a></th><td>mod_firehose.c</td></tr></table>
  35 <h3>Summary</h3>
  36
  37     <p><code>mod_firehose</code> provides a mechanism to record data
  38     being passed between the httpd server and the client at the raw
  39     connection level to either a file or a pipe in such a way that the
  40     data can be analysed or played back to the server at a future date.
  41     It can be thought of as "tcpdump for httpd".</p>
  42
  43     <p>Connections are recorded after the SSL has been stripped, and can
  44     be used for forensic debugging.</p>
  45
  46     <p>The <code class="program"><a href="../programs/firehose.html">firehose</a></code> tool can be used to demultiplex
  47     the recorded stream back into individual files for analysis, or
  48     playback using a tool like <code>netcat</code>.</p>
  49
  50     <div class="note"><h3>WARNING</h3>This module IGNORES all request level
  51     mechanisms to keep data private. It is the responsibility of the
  52     administrator to ensure that private data is not inadvertently
  53     exposed using this module.
  54     </div>
  55
  56 </div>
  57 <div id="quickview"><h3>Topics</h3>
  58 <ul id="topics">
  59 <li><img alt="" src="../images/down.gif" /> <a href="#enable">Enabling a Firehose</a></li>
  60 <li><img alt="" src="../images/down.gif" /> <a href="#format">Stream Format</a></li>
  61 </ul><h3 class="directives">Directives</h3>
  62 <ul id="toc">
  63 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseconnectioninput">FirehoseConnectionInput</a></li>
  64 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseconnectionoutput">FirehoseConnectionOutput</a></li>
  65 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseproxyconnectioninput">FirehoseProxyConnectionInput</a></li>
  66 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseproxyconnectionoutput">FirehoseProxyConnectionOutput</a></li>
  67 <li><img alt="" src="../images/down.gif" /> <a href="#firehoserequestinput">FirehoseRequestInput</a></li>
  68 <li><img alt="" src="../images/down.gif" /> <a href="#firehoserequestoutput">FirehoseRequestOutput</a></li>
  69 </ul>
  70 <h3>Bugfix checklist</h3><ul class="seealso"><li><a href="https://www.apache.org/dist/httpd/CHANGES_2.4">httpd changelog</a></li><li><a href="https://bz.apache.org/bugzilla/buglist.cgi?bug_status=__open__&amp;list_id=144532&amp;product=Apache%20httpd-2&amp;query_format=specific&amp;order=changeddate%20DESC%2Cpriority%2Cbug_severity&amp;component=mod_firehose">Known issues</a></li><li><a href="https://bz.apache.org/bugzilla/enter_bug.cgi?product=Apache%20httpd-2&amp;component=mod_firehose">Report a bug</a></li></ul><h3>See also</h3>
  71 <ul class="seealso">
  72 <li><code class="program"><a href="../programs/firehose.html">firehose</a></code></li>
  73 <li><a href="#comments_section">Comments</a></li></ul></div>
  74 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
  75 <div class="section">
  76 <h2><a name="enable" id="enable">Enabling a Firehose</a></h2>
  77
  78
  79     <p>To enable the module, it should be compiled and loaded
  80     in to your running Apache configuration, and the directives below
  81     used to record the data you are interested in.</p>
  82
  83     <p>It is possible to record both incoming and outgoing data to
  84     the same filename if desired, as the direction of flow is recorded
  85     within each fragment.</p>
  86
  87     <p>It is possible to write to both normal files and fifos (pipes).
  88     In the case of fifos, mod_firehose ensures that the packet size is
  89     no larger than PIPE_BUF to ensure writes are atomic.</p>
  90
  91     <p>If a pipe is being used, something must be reading from the pipe
  92     before httpd is started for the pipe to be successfully opened for
  93     write. If the request to open the pipe fails, mod_firehose will
  94     silently stand down and not record anything, and the server will
  95     keep running as normal.</p>
  96
  97     <p>By default, all attempts to write will block the server. If the
  98     webserver has been built against APR v2.0 or later, and an optional
  99     "nonblock" parameter is specified all file writes will be non
 100     blocking, and buffer overflows will cause debugging data to be lost.
 101     In this case it is possible to prioritise the running of the server
 102     over the recording of firehose data.</p>
 103
 104 </div><div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 105 <div class="section">
 106 <h2><a name="format" id="format">Stream Format</a></h2>
 107
 108
 109     <p>The server typically serves multiple connections simultaneously,
 110     and as a result requests and responses need to be multiplexed before
 111     being written to the firehose.</p>
 112
 113     <p>The fragment format is designed as clear text, so that a firehose
 114     can be opened with and inspected by a normal text editor.
 115     Alternatively, the <code class="program"><a href="../programs/firehose.html">firehose</a></code> tool can be used to
 116     demultiplex the firehose back into individual requests or
 117     connections.</p>
 118
 119     <p>The size of the multiplexed fragments is governed by PIPE_BUF,
 120     the maximum size of write the system is prepared to perform
 121     atomically. By keeping the multiplexed fragments below PIPE_BUF in
 122     size, the module guarantees that data from different fragments does
 123     not interleave. The size of PIPE_BUF varies on different operating
 124     systems.</p>
 125
 126     <p>The BNF for the fragment format is as follows:</p>
 127
 128     <pre> stream = 0*(fragment)
 129
 130  fragment = header CRLF body CRLF
 131
 132  header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count
 133
 134  length = &lt;up to 16 byte hex fragment length&gt;
 135  timestamp = &lt;up to 16 byte hex timestamp microseconds since 1970&gt;
 136  request = "&lt;"
 137  response = "&gt;"
 138  uuid = &lt;formatted uuid of the connection&gt;
 139  count = &lt;hex fragment number in the connection&gt;
 140
 141  body = &lt;the binary content of the fragment&gt;
 142
 143  SPC = &lt;a single space&gt;
 144  CRLF = &lt;a carriage return, followed by a line feed&gt;</pre>
 145
 146     <p>All fragments for a connection or a request will share the same
 147     UUID, depending on whether connections or requests are being recorded.
 148     If connections are being recorded, multiple requests may appear within
 149     a connection. A fragment with a zero length indicates the end of the
 150     connection.</p>
 151
 152     <p>Fragments may go missing or be dropped if the process reading the
 153     fragments is too slow. If this happens, gaps will exist in the
 154     connection counter numbering. A warning will be logged in the error
 155     log to indicate the UUID and counter of the dropped fragment, so it
 156     can be confirmed the fragment was dropped.</p>
 157
 158     <p>It is possible that the terminating empty fragment may not appear,
 159     caused by the httpd process crashing, or being terminated ungracefully.
 160     The terminating fragment may be dropped if the process reading the
 161     fragments is not fast enough.</p>
 162
 163 </div>
 164 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 165 <div class="directive-section"><h2><a name="FirehoseConnectionInput" id="FirehoseConnectionInput">FirehoseConnectionInput</a> <a name="firehoseconnectioninput" id="firehoseconnectioninput">Directive</a></h2>
 166 <table class="directive">
 167 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the server on each connection</td></tr>
 168 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 169 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 170 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 171 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 172 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 173 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseConnectionInput is only available in Apache 2.5.0 and
 174 later.</td></tr>
 175 </table>
 176     <p>Capture traffic coming into the server on each connection. Multiple
 177     requests will be captured within the same connection if keepalive is
 178     present.</p>
 179
 180     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseConnectionInput connection-input.firehose</pre>
 181 </div>
 182
 183 </div>
 184 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 185 <div class="directive-section"><h2><a name="FirehoseConnectionOutput" id="FirehoseConnectionOutput">FirehoseConnectionOutput</a> <a name="firehoseconnectionoutput" id="firehoseconnectionoutput">Directive</a></h2>
 186 <table class="directive">
 187 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic going out of the server on each connection</td></tr>
 188 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 189 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 190 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 191 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 192 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 193 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseConnectionOutput is only available in Apache 2.5.0 and
 194 later.</td></tr>
 195 </table>
 196     <p>Capture traffic going out of the server on each connection.
 197     Multiple requests will be captured within the same connection if
 198     keepalive is present.</p>
 199
 200     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseConnectionOutput connection-output.firehose</pre>
 201 </div>
 202
 203 </div>
 204 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 205 <div class="directive-section"><h2><a name="FirehoseProxyConnectionInput" id="FirehoseProxyConnectionInput">FirehoseProxyConnectionInput</a> <a name="firehoseproxyconnectioninput" id="firehoseproxyconnectioninput">Directive</a></h2>
 206 <table class="directive">
 207 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the back of mod_proxy</td></tr>
 208 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseProxyConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 209 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 210 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 211 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 212 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 213 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseProxyConnectionInput is only available in Apache 2.5.0 and
 214 later.</td></tr>
 215 </table>
 216     <p>Capture traffic being received by mod_proxy.</p>
 217
 218     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseProxyConnectionInput proxy-input.firehose</pre>
 219 </div>
 220
 221 </div>
 222 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 223 <div class="directive-section"><h2><a name="FirehoseProxyConnectionOutput" id="FirehoseProxyConnectionOutput">FirehoseProxyConnectionOutput</a> <a name="firehoseproxyconnectionoutput" id="firehoseproxyconnectionoutput">Directive</a></h2>
 224 <table class="directive">
 225 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic sent out from the back of mod_proxy</td></tr>
 226 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseProxyConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 227 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 228 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 229 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 230 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 231 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and
 232 later.</td></tr>
 233 </table>
 234     <p>Capture traffic being sent out by mod_proxy.</p>
 235
 236     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseProxyConnectionOutput proxy-output.firehose</pre>
 237 </div>
 238
 239 </div>
 240 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 241 <div class="directive-section"><h2><a name="FirehoseRequestInput" id="FirehoseRequestInput">FirehoseRequestInput</a> <a name="firehoserequestinput" id="firehoserequestinput">Directive</a></h2>
 242 <table class="directive">
 243 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the server on each request</td></tr>
 244 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseRequestInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 245 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 246 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 247 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 248 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 249 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseRequestInput is only available in Apache 2.5.0 and
 250 later.</td></tr>
 251 </table>
 252     <p>Capture traffic coming into the server on each request. Requests
 253     will be captured separately, regardless of the presence of keepalive.</p>
 254
 255     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseRequestInput request-input.firehose</pre>
 256 </div>
 257
 258 </div>
 259 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 260 <div class="directive-section"><h2><a name="FirehoseRequestOutput" id="FirehoseRequestOutput">FirehoseRequestOutput</a> <a name="firehoserequestoutput" id="firehoserequestoutput">Directive</a></h2>
 261 <table class="directive">
 262 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic going out of the server on each request</td></tr>
 263 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseRequestOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 264 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 265 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 266 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 267 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 268 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseRequestOutput is only available in Apache 2.5.0 and
 269 later.</td></tr>
 270 </table>
 271     <p>Capture traffic going out of the server on each request. Requests
 272     will be captured separately, regardless of the presence of keepalive.</p>
 273
 274     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseRequestOutput request-output.firehose</pre>
 275 </div>
 276
 277 </div>
 278 </div>
 279 <div class="bottomlang">
 280 <p><span>Available Languages: </span><a href="../en/mod/mod_firehose.html" title="English">&nbsp;en&nbsp;</a></p>
 281 </div><div class="top"><a href="#page-header"><img src="../images/up.gif" alt="top" /></a></div><div class="section"><h2><a id="comments_section" name="comments_section">Comments</a></h2><div class="warning"><strong>Notice:</strong><br />This is not a Q&amp;A section. Comments placed here should be pointed towards suggestions on improving the documentation or server, and may be removed again by our moderators if they are either implemented or considered invalid/off-topic. Questions on how to manage the Apache HTTP Server should be directed at either our IRC channel, #httpd, on Freenode, or sent to our <a href="http://httpd.apache.org/lists.html">mailing lists</a>.</div>
 282 <script type="text/javascript"><!--//--><![CDATA[//><!--
 283 var comments_shortname = 'httpd';
 284 var comments_identifier = 'http://httpd.apache.org/docs/trunk/mod/mod_firehose.html';
 285 (function(w, d) {
 286     if (w.location.hostname.toLowerCase() == "httpd.apache.org") {
 287         d.write('<div id="comments_thread"><\/div>');
 288         var s = d.createElement('script');
 289         s.type = 'text/javascript';
 290         s.async = true;
 291         s.src = 'https://comments.apache.org/show_comments.lua?site=' + comments_shortname + '&page=' + comments_identifier;
 292         (d.getElementsByTagName('head')[0] || d.getElementsByTagName('body')[0]).appendChild(s);
 293     }
 294     else {
 295         d.write('<div id="comments_thread">Comments are disabled for this page at the moment.<\/div>');
 296     }
 297 })(window, document);
 298 //--><!]]></script></div><div id="footer">
 299 <p class="apache">Copyright 2017 The Apache Software Foundation.<br />Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.</p>
 300 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p></div><script type="text/javascript"><!--//--><![CDATA[//><!--
 301 if (typeof(prettyPrint) !== 'undefined') {
 302     prettyPrint();
 303 }
 304 //--><!]]></script>
 305 </body></html>