Stefan Eissing [Thu, 17 Aug 2017 14:34:44 +0000 (14:34 +0000)]
On the trunk:
mod_md v0.7.0:
- LIVE: the real Let's Encrypt CA is now live by default! If you need to experiment, configure
MDCertificateAuthority https://acme-staging.api.letsencrypt.org/directory
- When existing, complete certificates are renewed, the activation of the new ones is
delayed by 24 hours (or until the existing ones expire, whatever is earler) to accomodate
for clients with weird clocks, refs #1.
- Fixed store sync when MDCAChallenges was removed again from an MD.
- Fixed crash when MD matched the base server, fixes #23
- Fixed watchgod resetting staging when server processes disappeared (e.g. reached
max requests or other limits).
Luca Toscano [Wed, 16 Aug 2017 13:40:12 +0000 (13:40 +0000)]
common|synopsis.xsl: rename directive type=sections id generation
This commits is a follow up of r1805189 and it is meant
to allow directives with the same name but different type
to coexist in the same document without triggering errors
while executing validate-xhtml.
For example: mod_md.xml recently introduced the following:
* ManagedDomain
* <ManagedDomain> (this one is type=section)
In my opinion this is a perfectly valid use case and it should
be allowed/handled correctly by the doc generation process/validation.
In order to avoid clashing the directive ids will get a suffix
called "section" if type=section will be present as param.
Quicklinks, <directive> links have been updated to the new
scheme to avoid dandling pointers in the doc.
Comments/reviews are welcome, if I left something behind
please let me know.
Luca Toscano [Wed, 16 Aug 2017 13:04:55 +0000 (13:04 +0000)]
synopsis.xsl: do not render two times the same
directive HTML if more than one
directive share the same name.
This has happened when mod_md.xml was introduced,
and the following directives shared the same name:
* ManagedDomain
* <ManagedDomain>
With the current code each time that a node needs
to be rendered it will emit a duplicate, ending up
in the above example with 4 sections rather than two.
Uniqueness of sections will be ensured by the HTML
elements ids, to avoid errors before committing for
example (accidental duplicates, etc..).
Stefan Eissing [Wed, 16 Aug 2017 12:43:41 +0000 (12:43 +0000)]
On the trunk:
mod_ssl: not using SSLV3 constant that would define what we mean by SSL version 3 if openssl does not know about SSL version 3. Then we pretend to not know about it either.
I have recently got some basic mod-proxy concepts
and these simple info would have helped me a lot:
- connection pooling == reuse. My brain did not pick
up this simple nomenclature at first read for some
reason, probably because we have parameters called
"*reuse*" and not "*pooling*". I find the term
"reuse" clearer than "pooling", but this is
debatable :)
- the concept of "Worker" needs to be read before
the ProxyPass section (that people probably read
directly after checking a search engine) to fully
understand its power.
- the fact that a worker is related to a single process
(and not shared as a Balancer) is a really important
concept that I tried to highlight when talking
about maxconns (question popped up a lot of times
from users).
Hope that my understanding is correct, feedback is welcome.
event: Avoid possible blocking in the listener thread when shutting down
connections. PR 60956.
start_lingering_close_nonblocking() now puts connections in defer_linger_chain
which is emptied by any worker thread (all atomically) after its usual work,
hence any possibly blocking flush and lingering close run outside the listener.
The listener may create a dedicated worker if it fills defer_linger_chain or
while it's not empty, calling push2worker with a NULL cs.
The state machine in process_socket() is slighly modified to be able to enter
with CONN_STATE_LINGER directly w/o clogging_input_filters to interfer.
New abort_socket_nonblocking() allows to reset connections when nonblocking is
required and we can't do much about the connection anymore, nor we want the
system to linger on its own after close().
Many thanks to Stefan Priebe for his heavy testing on many event's changes!
mpm_event: ap_queue_info_try_get_idler() may atomically decrement and then
re-increment the number idlers if it went under or to zero. We can avoid
this by switching to a compare-and-swap scheme.
Joe Orton [Tue, 18 Jul 2017 08:14:42 +0000 (08:14 +0000)]
Fix overriding ExtendedStatus to "off" with mod_systemd loaded, and
give more feedback to systemd during a reload.
* modules/arch/unix/mod_systemd.c (systemd_pre_config): New
function; tell systemd the service is reloading here.
(systemd_pre_mpm): Don't set ap_extended_status here, do nothing
if ExtendedStatus is off.
(register_hooks): Register pre_config hook.
mod_proxy_fcgi: Add the support for mod_proxy's
flushpackets and flushwait params
This change was requested on the development mailing
list in order to fill another gap between mod_fcgi
and mod_proxy_fcgi, namely the -flush funtionality.
The more evolved core trunk code would not need this
feature becuse of the non-blocking writes, but it
is be needed in 2.4.x.
mpm_winnt: Do not redefine the standard CONTAINING_RECORD() macro
in child.c.
This definition has been added in https://svn.apache.org/r88498 — perhaps,
because not every versions of SDK contained it at that time.
But since then, the macro has been available starting from Windows 2000
(https://msdn.microsoft.com/en-us/library/windows/hardware/ff542043),
and any available version of Windows SDK now should also contain it.
mpm_winnt: Remove an obsolete comment in child.c explaining why the
declarations of the structures and functions to access the completion
contexts reside in a header file.
This no longer holds, as all the necessary functions and structures are
located in the single .c file (child.c).
mpm_winnt: Tweak the names of the variables in child.c which are used to
represent a queue of the completion contexts.
Starting from r1801655, the "queue" isn't really a queue, as all the
access happens with a LIFO order. So, instead of that, call it a "pool
of completion contexts", adjust names of all relevant variables and
tweak the comments.
This patch changes
- qlock to ctxpool_lock,
- qhead to ctxpool_head, and
- qwait_event to ctxpool_wait_event.
mpm_winnt: Tweak the listener shutdown code to use a separate event
instead of the global variable (shutdown_in_progress).
This change has two purposes. First of all, it makes the listener threads
which are blocked waiting for a completion context exit immediately during
shutdown. Previously, such threads would only check for exit every second.
The second reason for this change is to put the child_main() function in
charge of controlling the listeners life cycle. Previously, such relation
was circumvented by the fact that the listeners were also waiting for the
global child exit_event. With the new separate listener_shutdown_event,
only the child_main() function is responsible for shutting down the
listeners, and I think that this makes the code a bit clearer.
All the original behavior, including the special APLOG_DEBUG diagnostic
message when we fail to acquire a free completion context in 1 second,
is kept unchanged.
mpm_winnt: Advertise support for preshutdown notifications in the service,
and perform shutdown in respond to SERVICE_CONTROL_PRESHUTDOWN.
The pure shutdown notification leaves a small amount of time for the service
to finish (and the allowed amount of time has been shrinking with every new
version of Windows), and handling only it increases the chance of the process
being killed by SCM, instead of gracefully shutting down. Handling the
preshutdown control code extends this period, and increases the chances of
finishing everything properly when the machine is rebooted or shut down.
(See https://msdn.microsoft.com/en-us/library/windows/desktop/ms683241)
Please note that although the preshutdown notifications are available only
starting from Windows Vista, the code is compatible with the previous versions
of Windows, since the SCM ignores unknown SERVICE_ACCEPT codes, and will
still send an ordinary SERVICE_CONTROL_SHUTDOWN under old Windows
versions.
mpm_winnt: Use a LIFO stack instead of a FIFO queue to hold unused
completion contexts, as that may significantly reduce the memory usage.
This simple change can have a noticeable impact on the amount of memory
consumed by the child process in various cases. Every completion context
in the queue has an associated allocator, and every allocator has it's
ap_max_mem_free memory limit which is not given back to the operating
system. Once the queue grows, it cannot shrink back, and every allocator
in each of the queued completion contexts keeps up to its max_free amount
of memory. The queue can only grow when a server has to serve multiple
concurrent connections at once.
With that in mind, consider a case with a server that doesn't encounter many
concurrent connections most of the time, but has occasional spikes when
it has to serve multiple concurrent connections. During such spikes, the
size of the completion context queue grows.
The actual difference between using LIFO and FIFO orders shows up after
such spikes, when the server is back to light load and doesn't see a lot
of concurrency. With FIFO order, every completion context in the queue
will be used in a round-robin manner, thus using *every* available allocator
one by one and ultimately claiming up to (N * ap_max_mem_free memory) from
the OS. With LIFO order, only the completion contexts that are close to
the top of the stack will be used and reused for subsequent connections.
Hence, only a small part of the allocators will be used, and this can
prevent all other allocators from unnecessarily acquiring memory from
the OS (and keeping it), and this reduces the overall memory footprint.
Please note that this change doesn't affect the worst case behavior, as
it's still (N * ap_max_mem_free memory), but tends to behave better in
practice, for the reasons described above.
Another thing worth considering is the new behavior when the OS decides
to swap out pages of the child process, for example, in a close-to-OOM
condition. Handling every new connection after the swap requires the OS
to load the memory pages for the allocator from the completion context that
is used for this connection. With FIFO order, the completion contexts are
used one by one, and this would cause page loads for every new connection.
With LIFO order, there will be almost no swapping, since the same completion
context is going to be reused for subsequent new connections.
mpm_winnt: Drop the APLOG_DEBUG diagnostic saying how many thread
are blocked on the I/O completion port during the shutdown.
Prior to r1801635, the shutdown code required to know the amount of blocked
threads, as it has been dispatching the same amount of completion packets.
But this no longer holds, and the only reason why we maintain the
corresponding g_blocked_threads variable is because of this debug
diagnostic message.
Drop it in order to reduce complexity of the quite critical code in the
winnt_get_connection() function and to reduce the amount of global
variables.
projects / apache / log