1 <?xml version="1.0" encoding="ISO-8859-1"?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
4 <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type" />
6 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
7 This file is generated from xml source: DO NOT EDIT
8 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
10 <title>mod_unique_id - Apache HTTP Server Version 2.5</title>
11 <link href="../style/css/manual.css" rel="stylesheet" media="all" type="text/css" title="Main stylesheet" />
12 <link href="../style/css/manual-loose-100pc.css" rel="alternate stylesheet" media="all" type="text/css" title="No Sidebar - Default font size" />
13 <link href="../style/css/manual-print.css" rel="stylesheet" media="print" type="text/css" /><link rel="stylesheet" type="text/css" href="../style/css/prettify.css" />
14 <script src="../style/scripts/prettify.min.js" type="text/javascript">
17 <link href="../images/favicon.ico" rel="shortcut icon" /></head>
19 <div id="page-header">
20 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p>
21 <p class="apache">Apache HTTP Server Version 2.5</p>
22 <img alt="" src="../images/feather.png" /></div>
23 <div class="up"><a href="./"><img title="<-" alt="<-" src="../images/left.gif" /></a></div>
25 <a href="http://www.apache.org/">Apache</a> > <a href="http://httpd.apache.org/">HTTP Server</a> > <a href="http://httpd.apache.org/docs/">Documentation</a> > <a href="../">Version 2.5</a> > <a href="./">Modules</a></div>
26 <div id="page-content">
27 <div id="preamble"><h1>Apache Module mod_unique_id</h1>
29 <p><span>Available Languages: </span><a href="../en/mod/mod_unique_id.html" title="English"> en </a> |
30 <a href="../fr/mod/mod_unique_id.html" hreflang="fr" rel="alternate" title="Français"> fr </a> |
31 <a href="../ja/mod/mod_unique_id.html" hreflang="ja" rel="alternate" title="Japanese"> ja </a> |
32 <a href="../ko/mod/mod_unique_id.html" hreflang="ko" rel="alternate" title="Korean"> ko </a></p>
34 <table class="module"><tr><th><a href="module-dict.html#Description">Description:</a></th><td>Provides an environment variable with a unique
35 identifier for each request</td></tr>
36 <tr><th><a href="module-dict.html#Status">Status:</a></th><td>Extension</td></tr>
37 <tr><th><a href="module-dict.html#ModuleIdentifier">Module Identifier:</a></th><td>unique_id_module</td></tr>
38 <tr><th><a href="module-dict.html#SourceFile">Source File:</a></th><td>mod_unique_id.c</td></tr></table>
42 <p>This module provides a magic token for each request which is
43 guaranteed to be unique across "all" requests under very
44 specific conditions. The unique identifier is even unique
45 across multiple machines in a properly configured cluster of
46 machines. The environment variable <code>UNIQUE_ID</code> is
47 set to the identifier for each request. Unique identifiers are
48 useful for various reasons which are beyond the scope of this
51 <div id="quickview"><h3>Topics</h3>
53 <li><img alt="" src="../images/down.gif" /> <a href="#theory">Theory</a></li>
54 </ul><h3 class="directives">Directives</h3>
55 <p>This module provides no
57 <h3>Bugfix checklist</h3><ul class="seealso"><li><a href="https://www.apache.org/dist/httpd/CHANGES_2.4">httpd changelog</a></li><li><a href="https://bz.apache.org/bugzilla/buglist.cgi?bug_status=__open__&list_id=144532&product=Apache%20httpd-2&query_format=specific&order=changeddate%20DESC%2Cpriority%2Cbug_severity&component=mod_unique_id">Known issues</a></li><li><a href="https://bz.apache.org/bugzilla/enter_bug.cgi?product=Apache%20httpd-2&component=mod_unique_id">Report a bug</a></li></ul><h3>See also</h3>
59 <li><a href="#comments_section">Comments</a></li></ul></div>
60 <div class="top"><a href="#page-header"><img alt="top" src="../images/up.gif" /></a></div>
62 <h2><a name="theory" id="theory">Theory</a></h2>
65 <p>First a brief recap of how the Apache server works on Unix
66 machines. This feature currently isn't supported on Windows NT.
67 On Unix machines, Apache creates several children, the children
68 process requests one at a time. Each child can serve multiple
69 requests in its lifetime. For the purpose of this discussion,
70 the children don't share any data with each other. We'll refer
71 to the children as <dfn>httpd processes</dfn>.</p>
73 <p>Your website has one or more machines under your
74 administrative control, together we'll call them a cluster of
75 machines. Each machine can possibly run multiple instances of
76 Apache. All of these collectively are considered "the
77 universe", and with certain assumptions we'll show that in this
78 universe we can generate unique identifiers for each request,
79 without extensive communication between machines in the
82 <p>The machines in your cluster should satisfy these
83 requirements. (Even if you have only one machine you should
84 synchronize its clock with NTP.)</p>
87 <li>The machines' times are synchronized via NTP or other
88 network time protocol.</li>
90 <li>The machines' hostnames all differ, such that the module
91 can do a hostname lookup on the hostname and receive a
92 different IP address for each machine in the cluster.</li>
95 <p>As far as operating system assumptions go, we assume that
96 pids (process ids) fit in 32-bits. If the operating system uses
97 more than 32-bits for a pid, the fix is trivial but must be
98 performed in the code.</p>
100 <p>Given those assumptions, at a single point in time we can
101 identify any httpd process on any machine in the cluster from
102 all other httpd processes. The machine's IP address and the pid
103 of the httpd process are sufficient to do this. A httpd process
104 can handle multiple requests simultaneously if you use a
105 multi-threaded MPM. In order to identify threads, we use a thread
106 index Apache httpd uses internally. So in order to
107 generate unique identifiers for requests we need only
108 distinguish between different points in time.</p>
110 <p>To distinguish time we will use a Unix timestamp (seconds
111 since January 1, 1970 UTC), and a 16-bit counter. The timestamp
112 has only one second granularity, so the counter is used to
113 represent up to 65536 values during a single second. The
114 quadruple <em>( ip_addr, pid, time_stamp, counter )</em> is
115 sufficient to enumerate 65536 requests per second per httpd
116 process. There are issues however with pid reuse over time, and
117 the counter is used to alleviate this issue.</p>
119 <p>When an httpd child is created, the counter is initialized
120 with ( current microseconds divided by 10 ) modulo 65536 (this
121 formula was chosen to eliminate some variance problems with the
122 low order bits of the microsecond timers on some systems). When
123 a unique identifier is generated, the time stamp used is the
124 time the request arrived at the web server. The counter is
125 incremented every time an identifier is generated (and allowed
128 <p>The kernel generates a pid for each process as it forks the
129 process, and pids are allowed to roll over (they're 16-bits on
130 many Unixes, but newer systems have expanded to 32-bits). So
131 over time the same pid will be reused. However unless it is
132 reused within the same second, it does not destroy the
133 uniqueness of our quadruple. That is, we assume the system does
134 not spawn 65536 processes in a one second interval (it may even
135 be 32768 processes on some Unixes, but even this isn't likely
138 <p>Suppose that time repeats itself for some reason. That is,
139 suppose that the system's clock is screwed up and it revisits a
140 past time (or it is too far forward, is reset correctly, and
141 then revisits the future time). In this case we can easily show
142 that we can get pid and time stamp reuse. The choice of
143 initializer for the counter is intended to help defeat this.
144 Note that we really want a random number to initialize the
145 counter, but there aren't any readily available numbers on most
146 systems (<em>i.e.</em>, you can't use rand() because you need
147 to seed the generator, and can't seed it with the time because
148 time, at least at one second resolution, has repeated itself).
149 This is not a perfect defense.</p>
151 <p>How good a defense is it? Suppose that one of your machines
152 serves at most 500 requests per second (which is a very
153 reasonable upper bound at this writing, because systems
154 generally do more than just shovel out static files). To do
155 that it will require a number of children which depends on how
156 many concurrent clients you have. But we'll be pessimistic and
157 suppose that a single child is able to serve 500 requests per
158 second. There are 1000 possible starting counter values such
159 that two sequences of 500 requests overlap. So there is a 1.5%
160 chance that if time (at one second resolution) repeats itself
161 this child will repeat a counter value, and uniqueness will be
162 broken. This was a very pessimistic example, and with real
163 world values it's even less likely to occur. If your system is
164 such that it's still likely to occur, then perhaps you should
165 make the counter 32 bits (by editing the code).</p>
167 <p>You may be concerned about the clock being "set back" during
168 summer daylight savings. However this isn't an issue because
169 the times used here are UTC, which "always" go forward. Note
170 that x86 based Unixes may need proper configuration for this to
171 be true -- they should be configured to assume that the
172 motherboard clock is on UTC and compensate appropriately. But
173 even still, if you're running NTP then your UTC time will be
174 correct very shortly after reboot.</p>
177 <p>The <code>UNIQUE_ID</code> environment variable is
178 constructed by encoding the 144-bit (32-bit IP address, 32 bit
179 pid, 32 bit time stamp, 16 bit counter, 32 bit thread index)
181 alphabet <code>[A-Za-z0-9@-]</code> in a manner similar to MIME
182 base64 encoding, producing 24 characters. The MIME base64
183 alphabet is actually <code>[A-Za-z0-9+/]</code> however
184 <code>+</code> and <code>/</code> need to be specially encoded
185 in URLs, which makes them less desirable. All values are
186 encoded in network byte ordering so that the encoding is
187 comparable across architectures of different byte ordering. The
188 actual ordering of the encoding is: time stamp, IP address,
189 pid, counter. This ordering has a purpose, but it should be
190 emphasized that applications should not dissect the encoding.
191 Applications should treat the entire encoded
192 <code>UNIQUE_ID</code> as an opaque token, which can be
193 compared against other <code>UNIQUE_ID</code>s for equality
196 <p>The ordering was chosen such that it's possible to change
197 the encoding in the future without worrying about collision
198 with an existing database of <code>UNIQUE_ID</code>s. The new
199 encodings should also keep the time stamp as the first element,
200 and can otherwise use the same alphabet and bit length. Since
201 the time stamps are essentially an increasing sequence, it's
202 sufficient to have a <em>flag second</em> in which all machines
203 in the cluster stop serving any request, and stop using the old
204 encoding format. Afterwards they can resume requests and begin
205 issuing the new encodings.</p>
207 <p>This we believe is a relatively portable solution to this
208 problem. The identifiers
209 generated have essentially an infinite life-time because future
210 identifiers can be made longer as required. Essentially no
211 communication is required between machines in the cluster (only
212 NTP synchronization is required, which is low overhead), and no
213 communication between httpd processes is required (the
214 communication is implicit in the pid value assigned by the
215 kernel). In very specific situations the identifier can be
216 shortened, but more information needs to be assumed (for
217 example the 32-bit IP address is overkill for any site, but
218 there is no portable shorter replacement for it). </p>
221 <div class="bottomlang">
222 <p><span>Available Languages: </span><a href="../en/mod/mod_unique_id.html" title="English"> en </a> |
223 <a href="../fr/mod/mod_unique_id.html" hreflang="fr" rel="alternate" title="Français"> fr </a> |
224 <a href="../ja/mod/mod_unique_id.html" hreflang="ja" rel="alternate" title="Japanese"> ja </a> |
225 <a href="../ko/mod/mod_unique_id.html" hreflang="ko" rel="alternate" title="Korean"> ko </a></p>
226 </div><div class="top"><a href="#page-header"><img src="../images/up.gif" alt="top" /></a></div><div class="section"><h2><a id="comments_section" name="comments_section">Comments</a></h2><div class="warning"><strong>Notice:</strong><br />This is not a Q&A section. Comments placed here should be pointed towards suggestions on improving the documentation or server, and may be removed again by our moderators if they are either implemented or considered invalid/off-topic. Questions on how to manage the Apache HTTP Server should be directed at either our IRC channel, #httpd, on Freenode, or sent to our <a href="http://httpd.apache.org/lists.html">mailing lists</a>.</div>
227 <script type="text/javascript"><!--//--><![CDATA[//><!--
228 var comments_shortname = 'httpd';
229 var comments_identifier = 'http://httpd.apache.org/docs/trunk/mod/mod_unique_id.html';
231 if (w.location.hostname.toLowerCase() == "httpd.apache.org") {
232 d.write('<div id="comments_thread"><\/div>');
233 var s = d.createElement('script');
234 s.type = 'text/javascript';
236 s.src = 'https://comments.apache.org/show_comments.lua?site=' + comments_shortname + '&page=' + comments_identifier;
237 (d.getElementsByTagName('head')[0] || d.getElementsByTagName('body')[0]).appendChild(s);
240 d.write('<div id="comments_thread">Comments are disabled for this page at the moment.<\/div>');
242 })(window, document);
243 //--><!]]></script></div><div id="footer">
244 <p class="apache">Copyright 2017 The Apache Software Foundation.<br />Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.</p>
245 <p class="menu"><a href="../mod/">Modules</a> | <a href="../mod/quickreference.html">Directives</a> | <a href="http://wiki.apache.org/httpd/FAQ">FAQ</a> | <a href="../glossary.html">Glossary</a> | <a href="../sitemap.html">Sitemap</a></p></div><script type="text/javascript"><!--//--><
projects / apache / blob