granicus.if.org Git - apache/blob - docs/manual/mod/mod_firehose.html.en

   1 <?xml version="1.0" encoding="ISO-8859-1"?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head><!--
   4         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
   5               This file is generated from xml source: DO NOT EDIT
   6         XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
   7       -->
   8 <title>mod_firehose - Apache HTTP Server Version 2.5</title>
   9 <link href="../style/css/manual.css" rel="stylesheet" media="all" type="text/css" title="Main stylesheet" />
  10 <link href="../style/css/manual-loose-100pc.css" rel="alternate stylesheet" media="all" type="text/css" title="No Sidebar - Default font size" />
  11 <link href="../style/css/manual-print.css" rel="stylesheet" media="print" type="text/css" /><link rel="stylesheet" type="text/css" href="../style/css/prettify.css" />
  12 <script src="../style/scripts/prettify.min.js" type="text/javascript">
  13 </script>
  14
  15 <link href="../images/favicon.ico" rel="shortcut icon" /></head>
  16 <body>
  17 <div id="page-header">
  18 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p>
  19 <p class="apache">Apache HTTP Server Version 2.5</p>
  20 <img alt="" src="../images/feather.gif" /></div>
  21 <div class="up"><a href="./"><img title="&lt;-" alt="&lt;-" src="../images/left.gif" /></a></div>
  22 <div id="path">
  23 <a href="http://www.apache.org/">Apache</a> &gt; <a href="http://httpd.apache.org/">HTTP Server</a> &gt; <a href="http://httpd.apache.org/docs/">Documentation</a> &gt; <a href="../">Version 2.5</a> &gt; <a href="./">Modules</a></div>
  24 <div id="page-content">
  25 <div id="preamble"><h1>Apache Module mod_firehose</h1>
  26 <div class="toplang">
  27 <p><span>Available Languages: </span><a href="../en/mod/mod_firehose.html" title="English">&nbsp;en&nbsp;</a></p>
  28 </div>
  29 <table class="module"><tr><th><a href="module-dict.html#Description">Description:</a></th><td>Multiplexes all I/O to a given file or pipe.</td></tr>
  30 <tr><th><a href="module-dict.html#Status">Status:</a></th><td>Extension</td></tr>
  31 <tr><th><a href="module-dict.html#ModuleIdentifier">Module Identifier:</a></th><td>firehose_module</td></tr>
  32 <tr><th><a href="module-dict.html#SourceFile">Source File:</a></th><td>mod_firehose.c</td></tr></table>
  33 <h3>Summary</h3>
  34
  35     <p><code>mod_firehose</code> provides a mechanism to record data
  36     being passed between the httpd server and the client at the raw
  37     connection level to either a file or a pipe in such a way that the
  38     data can be analysed or played back to the server at a future date.
  39     It can be thought of as "tcpdump for httpd".</p>
  40
  41     <p>Connections are recorded after the SSL has been stripped, and can
  42     be used for forensic debugging.</p>
  43
  44     <p>The <code class="program"><a href="../programs/firehose.html">firehose</a></code> tool can be used to demultiplex
  45     the recorded stream back into individual files for analysis, or
  46     playback using a tool like <code>netcat</code>.</p>
  47
  48     <div class="note"><h3>WARNING</h3>This module IGNORES all request level
  49     mechanisms to keep data private. It is the responsibility of the
  50     administrator to ensure that private data is not inadvertently
  51     exposed using this module.
  52     </div>
  53
  54 </div>
  55 <div id="quickview"><h3 class="directives">Directives</h3>
  56 <ul id="toc">
  57 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseconnectioninput">FirehoseConnectionInput</a></li>
  58 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseconnectionoutput">FirehoseConnectionOutput</a></li>
  59 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseproxyconnectioninput">FirehoseProxyConnectionInput</a></li>
  60 <li><img alt="" src="../images/down.gif" /> <a href="#firehoseproxyconnectionoutput">FirehoseProxyConnectionOutput</a></li>
  61 <li><img alt="" src="../images/down.gif" /> <a href="#firehoserequestinput">FirehoseRequestInput</a></li>
  62 <li><img alt="" src="../images/down.gif" /> <a href="#firehoserequestoutput">FirehoseRequestOutput</a></li>
  63 </ul>
  64 <h3>Topics</h3>
  65 <ul id="topics">
  66 <li><img alt="" src="../images/down.gif" /> <a href="#enable">Enabling a Firehose</a></li>
  67 <li><img alt="" src="../images/down.gif" /> <a href="#format">Stream Format</a></li>
  68 </ul><h3>See also</h3>
  69 <ul class="seealso">
  70 <li><code class="program"><a href="../programs/firehose.html">firehose</a></code></li>
  71 </ul><ul class="seealso"><li><a href="#comments_section">Comments</a></li></ul></div>
  72 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
  73 <div class="section">
  74 <h2><a name="enable" id="enable">Enabling a Firehose</a></h2>
  75
  76
  77     <p>To enable the module, it should be compiled and loaded
  78     in to your running Apache configuration, and the directives below
  79     used to record the data you are interested in.</p>
  80
  81     <p>It is possible to record both incoming and outgoing data to
  82     the same filename if desired, as the direction of flow is recorded
  83     within each fragment.</p>
  84
  85     <p>It is possible to write to both normal files and fifos (pipes).
  86     In the case of fifos, mod_firehose ensures that the packet size is
  87     no larger than PIPE_BUF to ensure writes are atomic.</p>
  88
  89     <p>If a pipe is being used, something must be reading from the pipe
  90     before httpd is started for the pipe to be successfully opened for
  91     write. If the request to open the pipe fails, mod_firehose will
  92     silently stand down and not record anything, and the server will
  93     keep running as normal.</p>
  94
  95     <p>By default, all attempts to write will block the server. If the
  96     webserver has been built against APR v2.0 or later, and an optional
  97     "nonblock" parameter is specified all file writes will be non
  98     blocking, and buffer overflows will cause debugging data to be lost.
  99     In this case it is possible to prioritise the running of the server
 100     over the recording of firehose data.</p>
 101
 102 </div><div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 103 <div class="section">
 104 <h2><a name="format" id="format">Stream Format</a></h2>
 105
 106
 107     <p>The server typically serves multiple connections simultaneously,
 108     and as a result requests and responses need to be multiplexed before
 109     being written to the firehose.</p>
 110
 111     <p>The fragment format is designed as clear text, so that a firehose
 112     can be opened with and inspected by a normal text editor.
 113     Alternatively, the <code class="program"><a href="../programs/firehose.html">firehose</a></code> tool can be used to
 114     demultiplex the firehose back into individual requests or
 115     connections.</p>
 116
 117     <p>The size of the multiplexed fragments is governed by PIPE_BUF,
 118     the maximum size of write the system is prepared to perform
 119     atomically. By keeping the multiplexed fragments below PIPE_BUF in
 120     size, the module guarantees that data from different fragments does
 121     not interleave. The size of PIPE_BUF varies on different operating
 122     systems.</p>
 123
 124     <p>The BNF for the fragment format is as follows:</p>
 125
 126     <pre> stream = 0*(fragment)
 127
 128  fragment = header CRLF body CRLF
 129
 130  header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count
 131
 132  length = &lt;up to 16 byte hex fragment length&gt;
 133  timestamp = &lt;up to 16 byte hex timestamp microseconds since 1970&gt;
 134  request = "&lt;"
 135  response = "&gt;"
 136  uuid = &lt;formatted uuid of the connection&gt;
 137  count = &lt;hex fragment number in the connection&gt;
 138
 139  body = &lt;the binary content of the fragment&gt;
 140
 141  SPC = &lt;a single space&gt;
 142  CRLF = &lt;a carriage return, followed by a line feed&gt;</pre>
 143
 144     <p>All fragments for a connection or a request will share the same
 145     UUID, depending on whether connections or requests are being recorded.
 146     If connections are being recorded, multiple requests may appear within
 147     a connection. A fragment with a zero length indicates the end of the
 148     connection.</p>
 149
 150     <p>Fragments may go missing or be dropped if the process reading the
 151     fragments is too slow. If this happens, gaps will exist in the
 152     connection counter numbering. A warning will be logged in the error
 153     log to indicate the UUID and counter of the dropped fragment, so it
 154     can be confirmed the fragment was dropped.</p>
 155
 156     <p>It is possible that the terminating empty fragment may not appear,
 157     caused by the httpd process crashing, or being terminated ungracefully.
 158     The terminating fragment may be dropped if the process reading the
 159     fragments is not fast enough.</p>
 160
 161 </div>
 162 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 163 <div class="directive-section"><h2><a name="FirehoseConnectionInput" id="FirehoseConnectionInput">FirehoseConnectionInput</a> <a name="firehoseconnectioninput" id="firehoseconnectioninput">Directive</a></h2>
 164 <table class="directive">
 165 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the server on each connection</td></tr>
 166 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 167 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 168 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 169 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 170 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 171 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseConnectionInput is only available in Apache 2.5.0 and
 172 later.</td></tr>
 173 </table>
 174     <p>Capture traffic coming into the server on each connection. Multiple
 175     requests will be captured within the same connection if keepalive is
 176     present.</p>
 177
 178     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseConnectionInput connection-input.firehose</pre>
 179 </div>
 180
 181 </div>
 182 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 183 <div class="directive-section"><h2><a name="FirehoseConnectionOutput" id="FirehoseConnectionOutput">FirehoseConnectionOutput</a> <a name="firehoseconnectionoutput" id="firehoseconnectionoutput">Directive</a></h2>
 184 <table class="directive">
 185 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic going out of the server on each connection</td></tr>
 186 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 187 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 188 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 189 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 190 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 191 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseConnectionOutput is only available in Apache 2.5.0 and
 192 later.</td></tr>
 193 </table>
 194     <p>Capture traffic going out of the server on each connection.
 195     Multiple requests will be captured within the same connection if
 196     keepalive is present.</p>
 197
 198     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseConnectionOutput connection-output.firehose</pre>
 199 </div>
 200
 201 </div>
 202 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 203 <div class="directive-section"><h2><a name="FirehoseProxyConnectionInput" id="FirehoseProxyConnectionInput">FirehoseProxyConnectionInput</a> <a name="firehoseproxyconnectioninput" id="firehoseproxyconnectioninput">Directive</a></h2>
 204 <table class="directive">
 205 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the back of mod_proxy</td></tr>
 206 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseProxyConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 207 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 208 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 209 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 210 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 211 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseProxyConnectionInput is only available in Apache 2.5.0 and
 212 later.</td></tr>
 213 </table>
 214     <p>Capture traffic being received by mod_proxy.</p>
 215
 216     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseProxyConnectionInput proxy-input.firehose</pre>
 217 </div>
 218
 219 </div>
 220 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 221 <div class="directive-section"><h2><a name="FirehoseProxyConnectionOutput" id="FirehoseProxyConnectionOutput">FirehoseProxyConnectionOutput</a> <a name="firehoseproxyconnectionoutput" id="firehoseproxyconnectionoutput">Directive</a></h2>
 222 <table class="directive">
 223 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic sent out from the back of mod_proxy</td></tr>
 224 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseProxyConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 225 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 226 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 227 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 228 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 229 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and
 230 later.</td></tr>
 231 </table>
 232     <p>Capture traffic being sent out by mod_proxy.</p>
 233
 234     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseProxyConnectionOutput proxy-output.firehose</pre>
 235 </div>
 236
 237 </div>
 238 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 239 <div class="directive-section"><h2><a name="FirehoseRequestInput" id="FirehoseRequestInput">FirehoseRequestInput</a> <a name="firehoserequestinput" id="firehoserequestinput">Directive</a></h2>
 240 <table class="directive">
 241 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic coming into the server on each request</td></tr>
 242 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseRequestInput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 243 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 244 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 245 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 246 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 247 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseRequestInput is only available in Apache 2.5.0 and
 248 later.</td></tr>
 249 </table>
 250     <p>Capture traffic coming into the server on each request. Requests
 251     will be captured separately, regardless of the presence of keepalive.</p>
 252
 253     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseRequestInput request-input.firehose</pre>
 254 </div>
 255
 256 </div>
 257 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
 258 <div class="directive-section"><h2><a name="FirehoseRequestOutput" id="FirehoseRequestOutput">FirehoseRequestOutput</a> <a name="firehoserequestoutput" id="firehoserequestoutput">Directive</a></h2>
 259 <table class="directive">
 260 <tr><th><a href="directive-dict.html#Description">Description:</a></th><td>Capture traffic going out of the server on each request</td></tr>
 261 <tr><th><a href="directive-dict.html#Syntax">Syntax:</a></th><td><code>FirehoseRequestOutput <var>[ block | nonblock ]</var> <var>filename</var></code></td></tr>
 262 <tr><th><a href="directive-dict.html#Default">Default:</a></th><td><code>none</code></td></tr>
 263 <tr><th><a href="directive-dict.html#Context">Context:</a></th><td>server config</td></tr>
 264 <tr><th><a href="directive-dict.html#Status">Status:</a></th><td>Extension</td></tr>
 265 <tr><th><a href="directive-dict.html#Module">Module:</a></th><td>mod_firehose</td></tr>
 266 <tr><th><a href="directive-dict.html#Compatibility">Compatibility:</a></th><td>FirehoseRequestOutput is only available in Apache 2.5.0 and
 267 later.</td></tr>
 268 </table>
 269     <p>Capture traffic going out of the server on each request. Requests
 270     will be captured separately, regardless of the presence of keepalive.</p>
 271
 272     <div class="example"><h3>Example</h3><pre class="prettyprint lang-config">FirehoseRequestOutput request-output.firehose</pre>
 273 </div>
 274
 275 </div>
 276 </div>
 277 <div class="bottomlang">
 278 <p><span>Available Languages: </span><a href="../en/mod/mod_firehose.html" title="English">&nbsp;en&nbsp;</a></p>
 279 </div><div class="top"><a href="#page-header"><img src="../images/up.gif" alt="top" /></a></div><div class="section"><h2><a id="comments_section" name="comments_section">Comments</a></h2><div class="warning"><strong>Notice:</strong><br />This is not a Q&amp;A section. Comments placed here should be pointed towards suggestions on improving the documentation or server, and may be removed again by our moderators if they are either implemented or considered invalid/off-topic. Questions on how to manage the Apache HTTP Server should be directed at either our IRC channel, #httpd, on Freenode, or sent to our <a href="http://httpd.apache.org/lists.html">mailing lists</a>.</div>
 280 <script type="text/javascript"><!--//--><![CDATA[//><!--
 281 var comments_shortname = 'httpd';
 282 var comments_identifier = 'http://httpd.apache.org/docs/trunk/mod/mod_firehose.html';
 283 (function(w, d) {
 284     if (w.location.hostname.toLowerCase() == "httpd.apache.org") {
 285         d.write('<div id="comments_thread"><\/div>');
 286         var s = d.createElement('script');
 287         s.type = 'text/javascript';
 288         s.async = true;
 289         s.src = 'https://comments.apache.org/show_comments.lua?site=' + comments_shortname + '&page=' + comments_identifier;
 290         (d.getElementsByTagName('head')[0] || d.getElementsByTagName('body')[0]).appendChild(s);
 291     }
 292     else {
 293         d.write('<div id="comments_thread">Comments are disabled for this page at the moment.<\/div>');
 294     }
 295 })(window, document);
 296 //--><!]]></script></div><div id="footer">
 297 <p class="apache">Copyright 2015 The Apache Software Foundation.<br />Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.</p>
 298 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p></div><script type="text/javascript"><!--//--><![CDATA[//><!--
 299 if (typeof(prettyPrint) !== 'undefined') {
 300     prettyPrint();
 301 }
 302 //--><!]]></script>
 303 </body></html>