Azat Khuzhin [Mon, 5 Nov 2018 15:23:31 +0000 (18:23 +0300)]
cmake: set CMP0074 to NEW (for OPENSSL_ROOT in appveyor)
We have $env:OPENSSL_ROOT (env) equals to -DOPENSSL_ROOT (cmake
variable) anyway.
cmake complains:
Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
Run "cmake --help-policy CMP0074" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
Environment variable OpenSSL_ROOT is set to:
C:/OpenSSL-Win64/bin
For compatibility, CMake is ignoring the variable.
This warning is for project developers. Use -Wno-dev to suppress it.
Azat Khuzhin [Mon, 5 Nov 2018 19:25:15 +0000 (22:25 +0300)]
regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance
Due to inplace callbacks (i.e. no BEV_OPT_DEFER_CALLBACKS) we cannot be
sure that wm_transfer() will not be called recursively and indeed it
still happens sometimes, and the referenced patch increase amount of
this times, especially for linux/poll.
Fixes: 66304a23cf748714159c988e78f35401c5352827 ("Fix
ssl/bufferevent_wm_filter when bev does not reach watermark on break")
Azat Khuzhin [Mon, 5 Nov 2018 18:33:54 +0000 (21:33 +0300)]
regress_http: disable http/read_on_write_error under win32
EVHTTP_CON_READ_ON_WRITE_ERROR works only if an error already read from
the socket, but if we already got EPIPE on write we cannot read from the
socket anymore, and win32 does not guarantee that read will happens
before (although it happens from time to time).
In the referenced patch I just replaced callback with not expecting 417,
but like I already wrote, this is not always true (i.e. it is flacky).
Fixes: 3b581693ac1967f7f8d98491cb772a1b415eb4cd ("test/http:
read_on_write_error: fix it for win32")
Azat Khuzhin [Sun, 4 Nov 2018 18:41:20 +0000 (21:41 +0300)]
Merge branch 'ssl_bufferevent_wm_filter-fix'
* ssl_bufferevent_wm_filter-fix:
Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break
regress_ssl: cover watermarks with deferred callbacks
regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging
Azat Khuzhin [Tue, 30 Oct 2018 22:22:30 +0000 (01:22 +0300)]
Merge branch 'event-ET-#636-v2'
* event-ET-#636-v2:
Preserve ET bit for backends with changelist
Epoll ET setting lost with multiple events for same fd
Cover ET with multiple events for same fd
Add ET flag into event_base_dump_events()
Azat Khuzhin [Sun, 28 Oct 2018 15:11:22 +0000 (18:11 +0300)]
Check existence of IPV6_V6ONLY in evutil_make_listen_socket_ipv6only() (mingw32)
MinGW 32-bit 5.3.0 does not defines it and our appveyour [1] build
reports this instantly:
evutil.c: In function 'evutil_make_listen_socket_ipv6only':
evutil.c:392:40: error: 'IPV6_V6ONLY' undeclared (first use in this function)
return setsockopt(sock, IPPROTO_IPV6, IPV6_V6ONLY, (void*) &one,
Azat Khuzhin [Sun, 28 Oct 2018 13:46:24 +0000 (16:46 +0300)]
Convert evbuffer_strspn() (internal helper) to use size_t
As pointed by @yankeehacker in #590:
Signed to Unsigned Conversion Error - buffer.c:1623
Description: This assignment creates a type mismatch by populating an
unsigned variable with a signed value. The signed integer will be
implicitly cast to an unsigned integer, converting negative values into
positive ones. If an attacker can control the signed value, it may be
possible to trigger a buffer overflow if the value specifies the length
of a memory write.
Remediation: Do not rely on implicit casts between signed and unsigned
values because the result can take on an unexpected value and violate
weak assumptions made elsewhere in the program.
Azat Khuzhin [Sun, 28 Oct 2018 12:16:24 +0000 (15:16 +0300)]
buffer: add an assert for last_with_datap to suppress static analyzer
../buffer.c:2231:6: warning: Access to field 'flags' results in a dereference of a null pointer
if (CHAIN_SPACE_LEN(*firstchainp) == 0) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../buffer.c:130:30: note: expanded from macro 'CHAIN_SPACE_LEN'
#define CHAIN_SPACE_LEN(ch) ((ch)->flags & EVBUFFER_IMMUTABLE ? \
Azat Khuzhin [Sat, 27 Oct 2018 16:34:52 +0000 (19:34 +0300)]
regress_ssl: reset static variables on test setup/cleanup and eliminate leaks
One tricky bit is reply to the BIO_C_GET_FD command, since otherwise it
will try to close(0) and accepted bev in ssl/bufferevent_connect_sleep
will leak. Other seems more or less trivial.
This was done to make sure that for at least generic cases does not
leak (tricky cases was listed here nmathewson/Libevent#83).
Azat Khuzhin [Sat, 27 Oct 2018 15:35:08 +0000 (18:35 +0300)]
be_openssl: avoid leaking of SSL structure
From nmathewson/Libevent#83 by @fancycode:
There are a few code paths where the passed SSL object is not released in error cases, even if BEV_OPT_CLOSE_ON_FREE is passed as option while for others it is released. That way it's impossible for the caller to know it he has to free it on errors himself or not.
Line numbers are from "bufferevent_openssl.c" in 911abf3:
L1414 ("underlying == NULL" passed)
L1416 (bio could not be created)
L1446 (different fd passed)
L1325 (both underlying and fd passed)
L1328 (out-of-memory)
L1333 ("bufferevent_init_common_" failed)
In all error cases after the "bufferevent_ops_openssl" has been assigned, the option is evaluated on "bufferevent_free" (L1399) and the SSL object released (L1226).
Azat Khuzhin [Sat, 27 Oct 2018 11:58:30 +0000 (14:58 +0300)]
cmake: add various warning flags like autotools has
This is mostly to match autotools and reduce amount mixiing declarations
and code.
Added:
- -Wextra (the same as -W), -Wno-unused-parameter -Wstrict-aliasing
- -fno-strict-aliasing (gcc 2.9.5+)
- -Winit-self -Wmissing-field-initializers -Wdeclaration-after-statement (4.0+)
- -Waddress -Wno-unused-function -Wnormalized=id -Woverride-init (4.2+)
- -Wlogical-op (4.5+)
Removed:
- -Wformat (include in -Wall)
Plus use CMAKE_C_COMPILER_ID over CMAKE_COMPILER_IS_GNUCC, as
cmake-variables(7) suggesting, and add common GNUC/CLANG variables.
v2: drop checks for flags, since add_compiler_flags() will check if such
flags exists anyway (but just to note, gcc ignores non existing warning
flags by default).
Murat Demirten [Mon, 4 Jun 2018 13:43:34 +0000 (16:43 +0300)]
listener: ipv6only socket bind support
According to RFC3493 and most Linux distributions, default value is to
work in IPv4-mapped mode. If there is a requirement to bind same port
on same ip addresses but different handlers for both IPv4 and IPv6,
it is required to set IPV6_V6ONLY socket option to be sure that the
code works as expected without affected by bindv6only sysctl setting
in system.
Azat Khuzhin [Wed, 24 Oct 2018 21:50:50 +0000 (00:50 +0300)]
Merge branch 'evutil_found_ifaddr-dev'
* evutil_found_ifaddr-dev:
Cover evutil_v4addr_is_local_()/evutil_v6addr_is_local_()
Split evutil_found_ifaddr() into helpers (evutil_v{4,6}addr_is_local())
Use INADDR_ANY over 0 in evutil_found_ifaddr()
Replace EVUTIL_V4ADDR_IS_*() macroses with static inline functions
Filter link-local IPv4 addresses in evutil_found_ifaddr()
Azat Khuzhin [Mon, 22 Oct 2018 21:12:23 +0000 (00:12 +0300)]
Merge branch 'http-request-line-parsing'
* http-request-line-parsing:
Fix http https_basic/https_filter_basic under valgrind (increase timeout)
http: cover various non RFC3986 conformant URIs
http: allow non RFC3986 conformant during parsing request-line (http server)
http: do not try to parse request-line if we do not have enough bytes
http: allow trailing spaces (and only them) in request-line (like nginx)
http: cleanup of the request-line parsing
Azat Khuzhin [Sun, 21 Oct 2018 22:06:48 +0000 (01:06 +0300)]
Disable parallel jobs for the osx (due to CPU time deficit) in travis-ci
As you can see right now linux workers has zero failed tests, while osx
workers has 18 failed tests:
[bufferevent_connect_hostname_emfile FAILED]
[bufferevent_pair_release_lock FAILED]
[bufferevent_timeout FAILED]
[bufferevent_timeout_filter FAILED]
[bufferevent_timeout_pair FAILED]
[common_timeout FAILED]
[del_wait FAILED]
[immediatesignal FAILED]
[loopexit FAILED]
[loopexit_multiple FAILED]
[monotonic_res FAILED]
[no_events FAILED]
[persistent_active_timeout FAILED]
[persistent_timeout_jump FAILED]
[signal_switchbase FAILED]
[signal_while_processing FAILED]
[simpletimeout FAILED]
[usleep FAILED]
And this patch should remove from this list time related failures
(though maybe not all of them).
Azat Khuzhin [Sun, 21 Oct 2018 15:31:01 +0000 (18:31 +0300)]
Simplify bufferevent timeout tests to reduce CPU usage in between start/compare
Between start (setting "started_at") and comparing the time when
timeouts triggered with the start (test_timeval_diff_eq), there is too
much various things that can introduce extra delays and eventually could
fail the test on machine with shortage of CPU.
And this is exactly what happend on:
- travis-ci
- #262
Here is a simple reproducer that I came up with for this issue:
docker run --cpus=0.01 -e LD_LIBRARY_PATH=$PWD/lib -e PATH=/usr/bin:/bin:$PWD/bin -v $PWD:$PWD --rm -it debian:testing regress --no-fork --verbose bufferevent/bufferevent_timeout
Under limited CPU (see reproducer) the test almost always has problems
with that "write_timeout_at" exceed default timeval diff tolerance
(test_timeval_diff_eq() has 50 tolerance), i.e.:
FAIL ../test/regress_bufferevent.c:1040: assert(labs(timeval_msec_diff(((&started_at)), ((&res1.write_timeout_at))) - (100)) <= 50): 101 vs 50
But under some setup write timeout can even not triggered, and the
reason for this is that we write to the bufferevent 1024*1024 bytes, and
hence if evbuffer_write_iovec() will has some delay after writev() and
not send more then one vector at a time [1], it is pretty simple to
trigger, i.e.:
FAIL ../test/regress_bufferevent.c:1040: assert(labs(timeval_msec_diff(((&started_at)), ((&res1.write_timeout_at))) - (100)) <= 50): 1540155888478 vs 50
So this patch just send static small payload for all cases (plus a few
more asserts added).
The outcome of this patch is that all regression tests passed on
travis-ci for linux box [2]. While before it fails almost always [3].
Also reproducer with CPU limiting via docker also survive some
iterations (and strictly speaking it should has less CPU then travis-ci
workers I guess).
Azat Khuzhin [Sun, 21 Oct 2018 00:15:34 +0000 (03:15 +0300)]
Merge branch 'regress-dns-fixes'
* regress-dns-fixes:
Do not rely on getservbyname() for most of the dns regression tests
Turn off dns/getaddrinfo_race_gotresolve by default
Fix an error for debug locking in dns/getaddrinfo_race_gotresolve
Jiri Luznicky [Wed, 23 May 2018 13:39:13 +0000 (15:39 +0200)]
Fix missing LIST_HEAD
Despite the presence of 'sys/queue.h' in some stdlib implementations
(i.e. uclibc) 'LIST_HEAD' macro can be missing. This fix defines this
macro in the same manner as was done previously for 'TAILQ_'.
Azat Khuzhin [Wed, 17 Oct 2018 20:21:32 +0000 (23:21 +0300)]
Merge branch 'be-wm-overrun-v2'
* be-wm-overrun-v2:
Fix hangs due to watermarks overruns in bufferevents implementations
test: cover watermarks (with some corner cases) in ssl bufferevent
avoid warnings with any modern C99 compiler due to implicit function
declaration for pthread_create, as shown by the following :
test/regress_dns.c:2226:2: warning: implicit declaration of function
'pthread_create' is invalid in C99 [-Wimplicit-function-declaration]
THREAD_START(thread[0], race_base_run, &rp);
^
test/regress_thread.h:35:2: note: expanded from macro 'THREAD_START'
pthread_create(&(threadvar), NULL, fn, arg)
^
test/regress_dns.c:2226:2: warning: this function declaration is not a prototype
[-Wstrict-prototypes]
test/regress_thread.h:35:2: note: expanded from macro 'THREAD_START'
pthread_create(&(threadvar), NULL, fn, arg)
^
$ clang --version
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Eliminate compiler warnings (at least for gcc/linux)
Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com> Closes: #646 # cherry-picked from the PR
(cherry picked from commit be371163027d3988a30620537bf3fcb81fb7f077)
Azat Khuzhin [Tue, 19 Jun 2018 07:15:08 +0000 (10:15 +0300)]
Cleanup __func__ detection
First of all __func__ is not a macro, it is char[] array, so the code
that we had before in cmake, was incorrect, i.e.:
#if defined (__func__)
#define EVENT____func__ __func__
#elif defined(__FUNCTION__)
#define EVENT____func__ __FUNCTION__
#else
#define EVENT____func__ __FILE__
#endif
So just detect do we have __func__/__FUNCTION__ in configure/cmake
before build and define EVENT__HAVE___func__/EVENT__HAVE___FUNCTION__
to use the later to choose which should be used as a __func__ (if it is
not presented).
Azat Khuzhin [Wed, 1 Aug 2018 06:48:42 +0000 (09:48 +0300)]
Merge branch 'official/pr/671' -- README cleanup
* official/pr/671:
Capitalise project names consistently in README.md
Indent configure flag section to make markdown format them as code
Use https for resources that support it
Rewords awkward sentences in README.md
Fix typos in README.md
Azat Khuzhin [Tue, 31 Jul 2018 21:58:02 +0000 (00:58 +0300)]
autotools: include win32 specific headers for socklen_t detection on win32/mingw
The [1] removes EVENT__ prefix, and now if we will incorrectly detect
that "foobar" (or socklen_t in our case) type is not available, but
somewhere later it will be available then we will get next error [2]:
error: two or more data types in declaration specifiers
According to [3]:
- Compile something in Cygwin and you are compiling it for Cygwin.
- Compile something in MinGW and you are compiling it for Windows.
And I can confirm this, since there is _WIN32 defined (according to [4])
And since according to [5] our image in appveyour (Visual Studion 2015)
has mingw (and we use it, not cygwin) we need ws2tcpip.h (over
sys/socket.h -- which does not exist in win32) header to detect
socklen_t existence.
The script make-event-config.sed was mangling all the symbols by
prefixing them with "EVENT__". The problem here is that some
symbols aren't for local consumption within libevent, but rather
influence other system header files (ex: __USE_FILE_OFFSET64 is
used by dozens of header files including <sys/sendfile.h>).
As a workaround, all symbols starting with a capital letter only
(with the exception of STDC_HEADERS which must also be left
untouched) will be mangled.
Future contributors will need to be aware of this distinction.
Azat Khuzhin [Sun, 22 Apr 2018 22:50:55 +0000 (01:50 +0300)]
Fix CheckFunctionExistsEx() cmake macro on win32
For example under mingw64 it could not detect that strtok_r() exists,
because it checks with:
void *p = func_name;
And for this you need the function to be defined, so just sync our
CheckFunctionExistsEx.c with CheckFunctionExists.c from cmake (and later
we should drop them out) since it does correct things to detech
functions existence.
Also for WIN32 there is -FIwinsock2.h -FIws2tcpip.h, and I guess that is
not works for mingw gcc (since -F in gcc is framework, and in windows
-FI is like -include in gcc). But looks like we do not need them
already (due to fixed CheckFunctionExistsEx()).
Greg Hazel [Mon, 12 Feb 2018 00:28:58 +0000 (16:28 -0800)]
Fix evhttp_connection_get_addr() fox incomming http connections
Install conn_address of the bufferevent on incomping http connections
(even though this is kind of subsytem violation, so let's fix it in a
simplest way and thinkg about long-term solution).
Jesse Fang [Fri, 23 Feb 2018 11:15:12 +0000 (19:15 +0800)]
bufferevent_socket_connect{,_hostname}() missing event callback and use ret code
- When socket() failed in bufferevent_socket_connect() , the event
callback should be called also in
bufferevent_socket_connect_hostname(). eg. when use
bufferevent_socket_connect_hostname() to resolve and connect an IP
address but process have a smaller ulimit open files, socket() fails
always but caller is not notified.
- Make bufferevent_socket_connect()'s behavior more consistent: function
return error then no callback, function return ok then error passed by
event callback.
Azat Khuzhin [Sun, 22 Apr 2018 21:26:08 +0000 (00:26 +0300)]
Adopt ignore rules for cmake + ninja
In case we have build directory differs from source directory there will be
bunch of files we should ignore, so just remove leading "/" for some or rules.
And fix others.
Bernard Spil [Mon, 2 Apr 2018 11:18:27 +0000 (13:18 +0200)]
Fix build with LibreSSL 2.7
LibreSSL 2.7 implements OpenSSL 1.1 API except for BIO_get_init()
See also: https://bugs.freebsd.org/226900 Signed-off-by: Bernard Spil <brnrd@FreeBSD.org> Closes: #617 (cherry-pick)
(cherry picked from commit 28b8075400c70b2d2da2ce07e590c2ec6d11783d)
"Upon successful completion, the select() function may modify the object
pointed to by the timout argument."
If "struct timeval" pointer is a "static const", it could potentially
be allocated in a RO text segment. The kernel would then try to copy
back the modified value (with the time remaining) into a read-only
address and SEGV.
Azat Khuzhin [Tue, 27 Feb 2018 18:12:14 +0000 (21:12 +0300)]
Fix base unlocking in event_del() if event_base_set() runned in another thread
Image next situation:
T1: T2:
event_del_()
lock the event.ev_base.th_base_lock
event_del_nolock_() event_set_base()
unlock the event.ev_base.th_base_lock
In this case we will unlock the wrong base after event_del_nolock_()
returns, and deadlock is likely to happens, since event_base_set() do
not check any mutexes (due to it is possible to do this only if event is
not inserted anywhere).
So event_del_() has to cache the base before removing the event, and
cached base.th_base_lock after.
Fixes: #577
* evconnlistener-do-not-close-client-fd:
listener: cover closing of fd in case evconnlistener_free() called from acceptcb
Revert "Fix potential fd leak in listener_read_cb()"
Andrey Okoshkin [Wed, 29 Nov 2017 08:13:51 +0000 (11:13 +0300)]
Fix generation of LibeventConfig.cmake for the installation tree
'LIBEVENT_INCLUDE_DIRS' is properly initialized in 'LibeventConfig.cmake' as
'LibeventConfig.cmake.in' contains usage of 'LIBEVENT_CMAKE_DIR' and
'EVENT_INSTALL_INCLUDE_DIR' variables but not 'EVENT_CMAKE_DIR' and
'EVENT__INCLUDE_DIRS'.
Related typos are fixed.
ejurgensen [Sun, 5 Nov 2017 11:18:49 +0000 (12:18 +0100)]
Fix incorrect ref to evhttp_get_decoded_uri in http.h
Replaces reference in the http.h include header file to evhttp_get_decoded_uri
with evhttp_uridecode. There is no function called evhttp_get_decoded_uri.
Azat Khuzhin [Sun, 29 Oct 2017 19:53:41 +0000 (22:53 +0300)]
Allow bodies for GET/DELETE/OPTIONS/CONNECT
I checked with nginx, and via it's lua bindings it allows body for all
this methods. Also everybody knows that some of web-servers allows body
for GET even though this is not RFC conformant.