Mark Ellzey [Fri, 15 May 2015 09:58:14 +0000 (02:58 -0700)]
Debug mode option to error on evthread init AFTER other event calls.
- A handy event_enable_debug_mode() feature which will error and abort the
application if any thread-aware libevent functions are called BEFORE the
evthread API has been initialized (manually, or through
evthread_use_windows_threads() / evthread_use_pthreads()
- This is done by setting the global debug variable
'event_debug_created_threadable_ctx_' whenever the following functions
are called:
evthreadimpl_lock_alloc_()
evthreadimpl_cond_alloc_()
event_base_new_with_config() <- this checks to see if the thread
callbacks are enabled first, so we
have to manually set the variable.
Mark Ellzey [Mon, 11 May 2015 16:47:40 +0000 (12:47 -0400)]
Fix the link for appveyor OpenSSL installer (WIN32)
* change and move the openssl self installer off to a site we own
- the old link was 404, probably due to being replaced with a newer
non-vuln version. But since we are only using this installer to
auto-build with on appveyor (not as a release), then having a file
we don't change and own seems to be a better solution.
* reduce verbosity
Mark Ellzey [Mon, 11 May 2015 16:06:01 +0000 (12:06 -0400)]
Fix garbage value in socketpair util function, stdint?
* Fixed an issue with evutil_ersatz_socketpair_, listen_addr could all
be compared against with agarbage values. So just memset it before
using it anywhere.
* Nick might punch me in the face, but if we have stdint.h; (as in
EVENT__HAVE_STDINT_H is defined), might as well use those instead of
the manual [U]INT[X}_MAX/MIN muck in there now.
Mark Ellzey [Tue, 5 May 2015 17:37:00 +0000 (10:37 -0700)]
Call underlying bev ctrl SET_FD on filtered bufferevents
If a bufferevent_filter is set on an underlying bufferevent which has
ctrl functions, bufferevent_filter needs to handle this.
For now I have added just BEV_CTRL_SET_FD, since this is needed for
bufferevent_sock to assign file descriptors to the proper
bufferevent_read/write callbacks.
A good example of the problem can be found in issue #237
https://github.com/libevent/libevent/issues/237
This does the same thing as Travis-CI but for windows.
@nmathewson
Go to: https://ci.appveyor.com/login -> Login using Github
Click **+New Project** -> Choose **Github** to the left -> Find **Libevent** in the list and click **Add**
CMake configuration files are intended to be used by other projects to find the library. Specifically the CMake find_package command can use it to find all files related to the project.
The idea is to support 2 different CMake configuration files for Libevent. One if you simply build libevent that is generated for the build tree.
And a second one that is generated for an install target that will be installed on the system and point to where on the system the lib files and such can be find.
So for instance, in the build tree the config would set the cmake variable `LIBEVENT_INCLUDE_DIRS` to `/path/to/libevent/build/include`.
And for the system config it would be set to `/usr/local/include` (or whatever target the user chose when running cmake).
Azat Khuzhin [Thu, 8 Jan 2015 01:45:27 +0000 (04:45 +0300)]
event: call event_disable_debug_mode() in libevent_global_shutdown()
This will avoid leaking of event_debug_map_HT_GROW
I buildin it into libevent_glboal_shutdown() because
event_disable_debug_mode() -> event_free_debug_globals() ->
event_free_debug_globals_locks() will clean event_debug_map_lock_ that
used in event_disable_debug_mode().
This fixes following problems in shared library build:
* visibility=hidden was not enabled for gcc because of incorrect variable name
* test programs that need internal APIs caused link errors
There is a race between manual event_active and natural event activation. If both happen at the same time on the same FD, they would both be protected by the same event base lock except for 1 LoC where the fields of struct event are read without any kind of lock. This commit does those reads into local variables inside the lock and then invokes the callback with those local arguments outside the lock. In 2.0-stable, none of this is inside the lock; in HEAD, only the callback is read inside the lock. This gets the callback and all 3 arguments inside the lock before calling it outside the lock.
Acer Yang [Sat, 20 Sep 2014 07:25:53 +0000 (15:25 +0800)]
Fix cmake error when the Module path has more than one entry.
CMAKE_MODULE_PATH is usually a list instead of single entry. Especially
for projects contain sub cmake projects. My patch replace the
CMAKE_MODULE_PATH with fixed path, to locate the `.in` file.
Azat Khuzhin [Wed, 12 Nov 2014 17:16:18 +0000 (20:16 +0300)]
evdns: avoid read-after-free in evdns_request_timeout_callback()
In evdns_request_timeout_callback() in case we a giving up, we call
request_finished() which will free() req structure, however we ns from
it to fail it, so save pointer to ns to call nameserver_failed() on
them.
Founded with valgrind:
$ valgrind regress dns/retry
==10497== Memcheck, a memory error detector
==10497== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==10497== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==10497== Command: regress dns/retry
==10497==
dns/retry: [forking] ==10498== Invalid read of size 8
==10498== at 0x4C309D: evdns_request_timeout_callback (evdns.c:2179)
==10498== by 0x49EA95: event_process_active_single_queue (event.c:1576)
==10498== by 0x49EFDD: event_process_active (event.c:1668)
==10498== by 0x49F6DD: event_base_loop (event.c:1891)
==10498== by 0x49F063: event_base_dispatch (event.c:1702)
==10498== by 0x44C7F1: dns_retry_test_impl (regress_dns.c:724)
==10498== by 0x44CF60: dns_retry_test (regress_dns.c:749)
==10498== by 0x48A8A1: testcase_run_bare_ (tinytest.c:105)
==10498== by 0x48A94E: testcase_run_forked_ (tinytest.c:189)
==10498== by 0x48AB73: testcase_run_one (tinytest.c:247)
==10498== by 0x48B4C2: tinytest_main (tinytest.c:434)
==10498== by 0x477FC7: main (regress_main.c:459)
==10498== Address 0x6176ef8 is 40 bytes inside a block of size 342 free'd
==10498== at 0x4C29E90: free (vg_replace_malloc.c:473)
==10498== by 0x4A4411: event_mm_free_ (event.c:3443)
==10498== by 0x4BE8C5: request_finished (evdns.c:702)
==10498== by 0x4C3098: evdns_request_timeout_callback (evdns.c:2178)
==10498== by 0x49EA95: event_process_active_single_queue (event.c:1576)
==10498== by 0x49EFDD: event_process_active (event.c:1668)
==10498== by 0x49F6DD: event_base_loop (event.c:1891)
==10498== by 0x49F063: event_base_dispatch (event.c:1702)
==10498== by 0x44C7F1: dns_retry_test_impl (regress_dns.c:724)
==10498== by 0x44CF60: dns_retry_test (regress_dns.c:749)
==10498== by 0x48A8A1: testcase_run_bare_ (tinytest.c:105)
==10498== by 0x48A94E: testcase_run_forked_ (tinytest.c:189)
==10498==
==10498==
==10498== HEAP SUMMARY:
==10498== in use at exit: 0 bytes in 0 blocks
==10498== total heap usage: 83 allocs, 83 frees, 10,020 bytes allocated
==10498==
==10498== All heap blocks were freed -- no leaks are possible
==10498==
==10498== For counts of detected and suppressed errors, rerun with: -v
==10498== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
OK
1 tests ok. (0 skipped)
==10497==
==10497== HEAP SUMMARY:
==10497== in use at exit: 0 bytes in 0 blocks
==10497== total heap usage: 3 allocs, 3 frees, 96 bytes allocated
==10497==
==10497== All heap blocks were freed -- no leaks are possible
==10497==
==10497== For counts of detected and suppressed errors, rerun with: -v
==10497== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
This will fix some invalid read/write:
==556== Invalid read of size 8
==556== at 0x4E4EEC6: event_queue_remove_timeout (minheap-internal.h:178)
==556== by 0x4E508AA: event_del_nolock_ (event.c:2764)
==556== by 0x4E53535: event_base_loop (event.c:3088)
==556== by 0x406FCFA: dispatch (libcrawl.c:271)
==556== by 0x402863: main (crawler.c:49)
==556== Address 0x68a3f18 is 152 bytes inside a block of size 400 free'd
==556== at 0x4C29C97: free (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==556== by 0x406F140: renew (libcrawl.c:625)
==556== by 0x4E6CDE9: evhttp_connection_cb_cleanup (http.c:1331)
==556== by 0x4E6E2B2: evhttp_connection_cb (http.c:1424)
==556== by 0x4E4DF2D: bufferevent_writecb (bufferevent_sock.c:310)
==556== by 0x4E52D1D: event_process_active_single_queue (event.c:1584)
==556== by 0x4E53676: event_base_loop (event.c:1676)
==556== by 0x406FCFA: dispatch (libcrawl.c:271)
==556== by 0x402863: main (crawler.c:49)
But this one because of some invalid write before (I guess).
It is 100% reproduced during massive crawling (because this process
has many different servers), but after spending some time for trying to
reproduce this using some simple tests/utils I gave up for a few days (I
have a lot of work to do), but I'm sending this patch as a reminder.
Just in case, I've tried next tests:
- mixing timeouts/retries
- shutdown http server and return it back
- slow dns server for first request
- sleep before accept
- hacking libevent sources to change the behaviour of http layer (so it
will go into that function which I'm insterested in).
Azat Khuzhin [Mon, 29 Sep 2014 21:34:04 +0000 (01:34 +0400)]
evdns: fail ns after we are failing/retrasmitting request
In case we are failing request (evdns_request_timeout_callback()), we
delete timeout_event in request_finished(), while just before calling
request_finished() (for failing request) there was a call to
nameserver_failed(), that add event for timeout_event, IOW we must fail
ns after request because otherwise we will not have timeout_event
actived, and we will waiting forever.
Before this patch the dns/retry_disable_when_inactive will wait forever,
after - OK.
Azat Khuzhin [Sun, 21 Sep 2014 14:29:17 +0000 (18:29 +0400)]
bufferevent_openssl: reset fd_is_set when setfd with -1 is called
Otherwise we will use old fd after close(2) called on it:
(Traces trimmed, to minimize it)
$ strace -keclose,open,socket,epoll_ctl https-client -url https://libevent.org
socket(PF_INET, SOCK_STREAM|SOCK_NONBLOCK, IPPROTO_IP) = 5
> /lib/x86_64-linux-gnu/libc-2.19.so(socket+0x7) [0xe6da7]
> /usr/lib/libevent-2.1.so.4.0.0(evutil_socket_+0x25) [0x27835]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0xe9) [0x1b1a9]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0x21f) [0x1b2df]
> /usr/lib/libevent-2.1.so.4.0.0(evutil_getaddrinfo_async_+0x65) [0x271a5]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect_hostname+0x106) [0x1b416]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0xe7) [0x39f07]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_make_request+0xb8) [0x3a218]
epoll_ctl(3, EPOLL_CTL_ADD, 5, {EPOLLIN, {u32=5, u64=5}}) = 0
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xfdd) [0x2ced]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_setfd+0x3b) [0x16d9b]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0x75) [0x1b135]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0x21f) [0x1b2df]
> /usr/lib/libevent-2.1.so.4.0.0(evutil_getaddrinfo_async_+0x65) [0x271a5]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect_hostname+0x106) [0x1b416]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0xe7) [0x39f07]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_make_request+0xb8) [0x3a218]
epoll_ctl(3, EPOLL_CTL_MOD, 5, {EPOLLIN|EPOLLOUT, {u32=5, u64=5}}) = 0
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xfee) [0x2cfe]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_setfd+0x3b) [0x16d9b]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0x75) [0x1b135]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect+0x21f) [0x1b2df]
> /usr/lib/libevent-2.1.so.4.0.0(evutil_getaddrinfo_async_+0x65) [0x271a5]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_socket_connect_hostname+0x106) [0x1b416]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0xe7) [0x39f07]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_make_request+0xb8) [0x3a218]
epoll_ctl(3, EPOLL_CTL_MOD, 5, {EPOLLOUT, {u32=5, u64=5}}) = 0
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xbbc) [0x28cc]
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xd96) [0x2aa6]
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xeb8) [0x2bc8]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x3fe) [0x1fd1e]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
epoll_ctl(3, EPOLL_CTL_DEL, 5, 7fffa1f841e0) = 0
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xbc4) [0x28d4]
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xd96) [0x2aa6]
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xeb8) [0x2bc8]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x3fe) [0x1fd1e]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
close(5) = 0
> /lib/x86_64-linux-gnu/libpthread-2.19.so(__close_nocancel+0x7) [0xeb20]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_reset_+0x55) [0x373f5]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0x1d) [0x39e3d]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x3fe) [0x1fd1e]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
epoll_ctl(3, EPOLL_CTL_ADD, 5, {EPOLLIN, {u32=5, u64=5}}) = -1 EBADF (Bad file descriptor)
) = -1 EBADF (Bad file descriptor)
/src/oss/strace-code/strace: Can't initiate libunwind: No such process
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xfdd) [0x2ced]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_setfd+0x3b) [0x16d9b]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0x14c) [0x39f6c]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x3fe) [0x1fd1e]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
/src/oss/strace-code/strace: Exit of unknown pid 28185 seen
[warn] Epoll ADD(1) on fd 5 failed. Old events were 0; read change was 1 (add); write change was 0 (none); close change was 0 (none): Bad file descriptor
epoll_ctl(3, EPOLL_CTL_ADD, 5, {EPOLLOUT, {u32=5, u64=5}}) = -1 EBADF (Bad file descriptor)
) = -1 EBADF (Bad file descriptor)
/src/oss/strace-code/strace: Can't initiate libunwind: No such process
> ...
> /usr/lib/libevent_openssl-2.1.so.4.0.0(_init+0xfee) [0x2cfe]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_setfd+0x3b) [0x16d9b]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_connect_+0x14c) [0x39f6c]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x3fe) [0x1fd1e]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
/src/oss/strace-code/strace: Exit of unknown pid 28186 seen
[warn] Epoll ADD(4) on fd 5 failed. Old events were 0; read change was 0 (none); write change was 1 (add); close change was 0 (none): Bad file descriptor
<getaddrinfo traces trimmed>
close(5) = -1 EBADF (Bad file descriptor)
) = -1 EBADF (Bad file descriptor)
/src/oss/strace-code/strace: Can't initiate libunwind: No such process
> /lib/x86_64-linux-gnu/libpthread-2.19.so(__close_nocancel+0x7) [0xeb20]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_reset_+0x55) [0x373f5]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_fail_+0xc1) [0x3aed1]
> /usr/lib/libevent-2.1.so.4.0.0(evhttp_connection_fail_+0x4a3) [0x3b2b3]
> /usr/lib/libevent-2.1.so.4.0.0(bufferevent_enable+0x192) [0x16722]
> /usr/lib/libevent-2.1.so.4.0.0(event_free+0x5df) [0x1feff]
> /usr/lib/libevent-2.1.so.4.0.0(event_base_loop+0x407) [0x20677]
/src/oss/strace-code/strace: Exit of unknown pid 28187 seen
some request failed - no idea which one though!
socket error = Bad file descriptor (9)
Before this patch:
$ sample/https-client -retries 1 -url https://libevent.org
[warn] Epoll ADD(1) on fd 6 failed. Old events were 0; read change was 1 (add); write change was 0 (none); close change was 0 (none): Bad file descriptor
[warn] Epoll ADD(4) on fd 6 failed. Old events were 0; read change was 0 (none); write change was 1 (add); close change was 0 (none): Bad file descriptor
some request failed - no idea which one though!
socket error = Bad file descriptor (9)
After this patch:
$ sample/https-client -retries 1 -url https://libevent.org
Response line: 0 (null)