util_filter: follow up to r1835640: pending_filter_cleanup() precedence.
Register pending_filter_cleanup() as a normal cleanup (not pre_cleanup) so
that the pending filters are still there on pool cleanup, and f->bb is set
to NULL where needed.
Then is_pending_filter() check is moved where relevant.
Always favor APR_POLLSET_WAKEABLE over method/implementation.
Probably more about correctness than a real issue since systems are
unlikely to implement more than one/their method...
This also makes use of pruntime for event_pollset (an oversight from r1835845).
MPMs event and worker both need a dedicated pool to handle the creation of
the threads (listener, workers) and synchronization objects (queues, pollset,
mutexes...) in the start_threads() thread, with at least the lifetime of
the connections they handle, and thus survive pchild destruction (notably
in ONE_PROCCESS mode, but SIG_UNGRACEFUL is concerned too).
For instance, without this fix, the below backtrace can happen in ONE_PROCCESS
mode and a signal/^C is received (with active connections):
Thread 1 "httpd" received signal SIGSEGV, Segmentation fault.
(gdb) bt
#0 <BOOM>
#1 0x00007ffff7c7e016 in apr_file_write (thefile=0x0, ...)
^ NULL (cleared)
at file_io/unix/readwrite.c:230
#2 0x00007ffff7c7e4a7 in apr_file_putc (ch=1 '\001', thefile=0x0)
^ NULL (cleared)
at file_io/unix/readwrite.c:377
#3 0x00007ffff7c8da4a in apr_pollset_wakeup (pollset=0x55555568b870)
^ already destroyed by pchild
at poll/unix/pollset.c:224
#4 0x00007ffff7fc16c7 in decrement_connection_count (cs_=0x7fff08000ea0)
at event.c:811
#5 0x00007ffff7c83e15 in run_cleanups (cref=0x7fffe4002b78)
at memory/unix/apr_pools.c:2672
#6 0x00007ffff7c82c2f in apr_pool_destroy (pool=0x7fffe4002b58)
^ master_conn
at memory/unix/apr_pools.c:1007
#7 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x7fff08000c28)
^ ptrans
at memory/unix/apr_pools.c:1004
#8 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x555555638698)
^ pconf
at memory/unix/apr_pools.c:1004
#9 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x555555636688)
^ pglobal
at memory/unix/apr_pools.c:1004
#10 0x00005555555f4709 in ap_terminate ()
at unixd.c:522
#11 0x00007ffff6dbc8f1 in __run_exit_handlers (...)
at exit.c:108
#12 0x00007ffff6dbc9ea in __GI_exit (status=<optimized out>)
at exit.c:139
#13 0x00007ffff7fc1616 in clean_child_exit (code=0)
at event.c:774
^ pchild already destroyed here
#14 0x00007ffff7fc5ae4 in child_main (child_num_arg=0, child_bucket=0)
at event.c:2869
...
While at it, add comments about the lifetimes of MPMs pools and their objects,
and give each pool a tag (e.g. "pchild" accordingly to other MPMs).
Lucien Gentis [Thu, 12 Jul 2018 13:05:22 +0000 (13:05 +0000)]
Rebuild (with 8 bits characters replaced by their HTML entities because -Xbootclasspath/p option disabled in build.sh script because it is no more supported in openjdk 10)
util_filter: keep filters with aside buckets in order.
Read or write of filter's pending data must happen in the same order as the
filter chain, thus we can't use an apr_hash_t to maintain the pending filters
since it provides no garantee on this matter.
Instead use an APR_RING maintained in c->pending_filters, and since both the
name (was c->filters) and the type changed, MAJOR is bumped (trunk only code
anyway so far).
Joe Orton [Wed, 11 Jul 2018 07:46:08 +0000 (07:46 +0000)]
* modules/ssl/ssl_engine_pphrase.c (modssl_load_engine_keypair): Load
the engine associated with the private key (&cert) explicitly
rather than requiring the engine to be set as the default method
for all operations (with "SSLCryptoDevice <engine>").
(Thanks to Anderson Sasaki <ansasaki redhat.com> for suggested
improvement and guidance)
Joe Orton [Fri, 6 Jul 2018 12:01:29 +0000 (12:01 +0000)]
Hook up PKCS#11 PIN entry through configured passphrase entry method.
* modules/ssl/ssl_engine_pphrase.c: Add wrappers for OpenSSL UI * API
around passphrase entry.
(modssl_load_engine_keypair): Take vhost ID and use above rather than
default OpenSSL UI.
Stefan Eissing [Thu, 5 Jul 2018 10:21:00 +0000 (10:21 +0000)]
On the trunk:
* silencing gcc uninitialized warning
* refrainning from apr_table_addn() use since pool debug assumptions are in conflict
* adding more assertions
* copy-porting changes to base64 encoding code from mod_md
Luca Toscano [Wed, 27 Jun 2018 06:32:53 +0000 (06:32 +0000)]
output-filters.xml: add a use case study
While working on http://svn.apache.org/r1833875
I learned a ton of non trivial things about
the output filter chain and how to design a filter,
and I thought to report them as a separate section
of the output filters doc page. Feedback is welcome
for inaccuracies or wrong understanding :)
Remove a compatibility note that is not relevant any more.
The corresponding code was added in r1807709 but reverted in r1827760.
The compatibility note was added as part of r1807869 but was not removed when the code has been axed.
If several parameters are used in a AuthzProviderAlias directive, if these parameters are not enclosed in quotation mark, only the first one is handled. The other ones are silently ignored.
Add a message to warn about such a spurious configuration.
PR 62469
Rainer Jung [Tue, 19 Jun 2018 22:40:19 +0000 (22:40 +0000)]
mod_cache: Per RFC 7234 section 5.3 an invalid
Expires header value must be interpreted as a
time in the past. So apply the logic concerning
"CacheStoreExpired" and "max-age" and "s-maxage"
handling, which we are already using for a valid
Expires header containing a time in the past,
also to the case of an invalid Expires header.
Luca Toscano [Tue, 19 Jun 2018 22:28:24 +0000 (22:28 +0000)]
mod_ratelimit: fix behavior with proxied content
mod_ratelimit works by splitting data in "chunks"
to send to the client, sleeping a predefined amount
of time between them (200ms). So for example,
a rate-limit 40 value would correspond to a chunk size
of 8192 bytes, flushed to the client every 200ms.
The idea works fine when httpd directly serves the
content, since the filter will be called once with
a single bucket brigade. In the context of a proxied
content though the filter is likely to be called multiple
times, with a bucket brigade size that corresponds to
the maximum allowed buffer size. If this value is lower
or higher than the chunk size, the filter will not
properly rate limit the data going to the client.
This patch solves the problem with two fix:
1) do_sleep is now stored in the ctx context struct,
so if the filter is invoked multiple times it
will still sleep when needed. For example, say
that the chunk_size is 8192 and the bucket brigate
len is 10240: the filter will flush 8192 bytes
on the first invocation, sleep 200ms, flush the
remaining bytes and then finish. The next invocation
will do the same, clearly not leading to the
correct "sleeping pattern".
2) The example above highlights also another issue:
mod_ratelimit should flush only chunk_size bytes
at the time (I am now excluding the burst calculation
from the picture), and buffer between invocations
unless the brigade contains EOS.
The change has been tested with various scenarios and
it looks working as expected, but of course more
feedback/testing is welcome.
The original patch was written by me and then Yann
refactored the code to be more precise and efficient,
basically transforming an axe in a wonderful Japanese
katana sword, so credits to him for this work.
Eric Covener [Tue, 19 Jun 2018 12:07:19 +0000 (12:07 +0000)]
add server_rec to log.c fatal startup errors
Not strictly necessary for trunk, but in 2.4.x if the main ErrorLog is
using syslog, these messages are lost. In trunk, the low-level logging
routines reach up and find the syslog provider when no server_rec is provided
but that backport is stalled.
Yann Ylavic [Fri, 15 Jun 2018 14:35:31 +0000 (14:35 +0000)]
mod_ssl: build with LibreSSL.
LibreSSL seems to be openssl-1.1 API compatible only in version 2.8 (master).
So use that for MODSSL_USE_OPENSSL_PRE_1_1_API instead of 2.7, the two 2.7
compatibility-exceptions are handled explicitely but overall it's simpler.
Regarding CRYPTO_malloc_init vs OPENSSL_malloc_init, libreSSL uses none, the
former used to be a no-op but depends is LIBRESSL_INTERNAL in latest versions,
while the latter has never been (and will never be) defined. So don't call any
with LibreSSL.
Yann Ylavic [Fri, 15 Jun 2018 11:12:19 +0000 (11:12 +0000)]
mod_ssl: disable check for client initiated renegotiations with TLS 1.3.
This is already forbidden by the protocol, enforced by OpenSSL, and the
current logic can't work (ssl_callback_Info() may be called multiple times
with TLS 1.3).
Yann Ylavic [Wed, 13 Jun 2018 09:54:16 +0000 (09:54 +0000)]
Follow up to r1833368: share openssl between modules.
Both libapr[-util], the core PRNG, mod_ssl, mod_crypto and mod_session_crypto
can use the same crypto library (e.g. openssl), use the new APR crypto loading
API so that they can work together and initialize/terminate the lib either once
for all or on demand and reusable by the others.
apr_pcalloc can be turned into apr_palloc (the allocated memory is fully initialized by the subsequent memcpy/strcpy) and '(int)strlen(p)' can be replaced by 'plen - 1' to save some cycles.
Joe Orton [Thu, 7 Jun 2018 13:17:27 +0000 (13:17 +0000)]
* configure.in, Makefile.in: Handle no-test-suite case through
check-no rule. Only regenerate the test suite on repeated
"make check" run if a header file has changed.
Yann Ylavic [Wed, 6 Jun 2018 21:04:21 +0000 (21:04 +0000)]
Avoid cyclic dependency by moving ap_set_etag() from module http to core.
This function, along with ap_make_etag(), is used by the default_handler in
core.c, and in several modules other than builtin mod_http, breaking static
linking and httpdunit tests build.
The move is done by "svn move modules/http/http_etag.c server/util_etag.c".
MMN major bumped, not backportable (as is) to 2.4.x.
Joe Orton [Wed, 6 Jun 2018 11:39:33 +0000 (11:39 +0000)]
* modules/http/http_request.c (ap_process_request_after_handler,
ap_process_request): Cache and retrieve the brigade structure used
to send EOR and FLUSH between requests in c->pool userdata, to avoid
allocating a brigade structure per-request out of c->pool.
Stefan Eissing [Wed, 6 Jun 2018 09:56:00 +0000 (09:56 +0000)]
mod_ssl: after code review, changed:
* eliminated SSLPolicyRec as name no longer used
* eliminated some left over parameters in internal functions due to policy def removal
* reverted a NULL test, necessary before
projects / apache / log