granicus.if.org Git - libevent/log

http: lingering close (like nginx have) for entity-too-large

By lingering close I mean something what nginx have for this name, by this term
I mean that we need to read all the body even if it's size greater then
`max_body_size`, otherwise browsers on win32 (including chrome) failed read the
http status - entity-too-large (while on linux chrome for instance are good),
and also this includes badly written http clients.

Refs: #321

v2: do this only under EVHTTP_SERVER_LINGERING_CLOSE

http: read server response even after server closed the connection

Otherwise if we will try to write more data than server can accept
(see `evhttp_set_max_body_size()` for libevent server) we will get `EPIPE` and
will not try to read server's response which must contain 400 error for now
(which is not strictly correct though, it must 413).
```
  $ strace regress --no-fork http/data_length_constraints
  ...
  connect(10, {sa_family=AF_INET, sin_port=htons(43988), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
  ...
  writev(10, [{"POST / HTTP/1.1\r\nHost: somehost\r"..., 60}, {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16324}], 2) = 16384
  epoll_wait(5, [{EPOLLOUT, {u32=10, u64=10}}, {EPOLLIN, {u32=11, u64=11}}], 32, 50000) = 2
  writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = 16384
  ioctl(11, FIONREAD, [32768])            = 0
  readv(11, [{"POST / HTTP/1.1\r\nHost: somehost\r"..., 4096}], 1) = 4096
  epoll_ctl(5, EPOLL_CTL_DEL, 11, 0x7fff09d41e50) = 0
  epoll_ctl(5, EPOLL_CTL_ADD, 11, {EPOLLOUT, {u32=11, u64=11}}) = 0
  epoll_wait(5, [{EPOLLOUT, {u32=10, u64=10}}, {EPOLLOUT, {u32=11, u64=11}}], 32, 50000) = 2
  writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = 16384
  writev(11, [{"HTTP/1.1 400 Bad Request\r\nConten"..., 129}, {"<HTML><HEAD>\n<TITLE>400 Bad Requ"..., 94}], 2) = 223
  epoll_ctl(5, EPOLL_CTL_DEL, 11, 0x7fff09d42080) = 0
  shutdown(11, SHUT_WR)                   = 0
  close(11)                               = 0
  epoll_wait(5, [{EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=10, u64=10}}], 32, 50000) = 1
  writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = -1 EPIPE (Broken pipe)
  --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=13954, si_uid=1000} ---
  epoll_ctl(5, EPOLL_CTL_DEL, 10, 0x7fff09d42010) = 0
  shutdown(10, SHUT_WR)                   = -1 ENOTCONN (Transport endpoint is not connected)
  close(10)                               = 0
  write(1, "\n  FAIL ../test/regress_http.c:3"..., 37
```
Careful reader can ask why it send error even when it didn't read
`evcon->max_body_size`, and the answer will be checks for `evcon->max_body_size
against `Content-Length` header, which contains ~8MB (-2 bytes).

And also if we will not drain the output buffer than we will send buffer that
we didn't send in previous request and instead of sending method via
`evhttp_make_header()`.

Fixes: http/data_length_constraints
Refs: #321

v2: do this only under EVHTTP_CON_READ_ON_WRITE_ERROR flag

test: increase buffer size for http/data_length_constraints to trigger EPIPE

With greater buffer it can't be written with one writev(2), and hence we can
trigger more tricky cases, like calling writecb/readcb more then once.

Refs: #321

sample/hello-world: exAmple, not eXMple

Fixes: #334

http: fix conflicts EVHTTP_CON_AUTOFREE and EVHTTP_CON_REUSE_CONNECTED_ADDR

And we can't make them continuous, since the latest is a public API, and
otherwise we will break binary compatibility.
Also extra check for EVHTTP_CON_PUBLIC_FLAGS_END, in case somebody forgot about
this (implementer I mean).

Refs: #182

http: coding style issue

util: fix "%zu" format on TDM-gcc/MinGW-w64

From: @gvanem
Fixes: #326

test/tinytest_demo: include <windows.h> for win32 to fix tdm-gcc

From: @gvanem
Fixes: #325

test/regress_ssl: Fix compile problems for win32

Windows doesn't have unistd.h, but have the required
functionality in io.h.

azat: use ev_ssize_t instead of ssize_t

evdns: name_parse(): fix remote stack overread

@asn-the-goblin-slayer:
  "the name_parse() function in libevent's DNS code is vulnerable to a buffer overread.

   971         if (cp != name_out) {
   972             if (cp + 1 >= end) return -1;
   973             *cp++ = '.';
   974         }
   975         if (cp + label_len >= end) return -1;
   976         memcpy(cp, packet + j, label_len);
   977         cp += label_len;
   978         j += label_len;
   No check is made against length before the memcpy occurs.

   This was found through the Tor bug bounty program and the discovery should be credited to 'Guido Vranken'."

Reproducer for gdb (https://gist.github.com/azat/e4fcf540e9b89ab86d02):
  set $PROT_NONE=0x0
  set $PROT_READ=0x1
  set $PROT_WRITE=0x2
  set $MAP_ANONYMOUS=0x20
  set $MAP_SHARED=0x01
  set $MAP_FIXED=0x10
  set $MAP_32BIT=0x40

  start

  set $length=202
  # overread
  set $length=2
  # allocate with mmap to have a seg fault on page boundary
  set $l=(1<<20)*2
  p mmap(0, $l, $PROT_READ|$PROT_WRITE, $MAP_ANONYMOUS|$MAP_SHARED|$MAP_32BIT, -1, 0)
  set $packet=(char *)$1+$l-$length
  # hack the packet
  set $packet[0]=63
  set $packet[1]='/'

  p malloc(sizeof(int))
  set $idx=(int *)$2
  set $idx[0]=0
  set $name_out_len=202

  p malloc($name_out_len)
  set $name_out=$3

  # have WRITE only mapping to fail on read
  set $end=$1+$l
  p (void *)mmap($end, 1<<12, $PROT_NONE, $MAP_ANONYMOUS|$MAP_SHARED|$MAP_FIXED|$MAP_32BIT, -1, 0)
  set $m=$4

  p name_parse($packet, $length, $idx, $name_out, $name_out_len)
  x/2s (char *)$name_out

Before this patch:
$ gdb -ex 'source gdb' dns-example
$1 = 1073741824
$2 = (void *) 0x633010
$3 = (void *) 0x633030
$4 = (void *) 0x40200000

Program received signal SIGSEGV, Segmentation fault.
__memcpy_sse2_unaligned () at memcpy-sse2-unaligned.S:33

After this patch:
$ gdb -ex 'source gdb' dns-example
$1 = 1073741824
$2 = (void *) 0x633010
$3 = (void *) 0x633030
$4 = (void *) 0x40200000
$5 = -1
0x633030:       "/"
0x633032:       ""
(gdb) p $m
$6 = (void *) 0x40200000
(gdb) p $1
$7 = 1073741824
(gdb) p/x $1
$8 = 0x40000000
(gdb) quit

P.S. plus drop one condition duplicate.

Fixes: #317

evutil_parse_sockaddr_port(): fix buffer overflow

@asn-the-goblin-slayer:
  "Length between '[' and ']' is cast to signed 32 bit integer on line 1815. Is
   the length is more than 2<<31 (INT_MAX), len will hold a negative value.
   Consequently, it will pass the check at line 1816. Segfault happens at line
   1819.

   Generate a resolv.conf with generate-resolv.conf, then compile and run
   poc.c. See entry-functions.txt for functions in tor that might be
   vulnerable.

   Please credit 'Guido Vranken' for this discovery through the Tor bug bounty
   program."

Reproducer for gdb (https://gist.github.com/azat/be2b0d5e9417ba0dfe2c):
  start
  p (1ULL<<31)+1ULL
  # $1 = 2147483649
  p malloc(sizeof(struct sockaddr))
  # $2 = (void *) 0x646010
  p malloc(sizeof(int))
  # $3 = (void *) 0x646030
  p malloc($1)
  # $4 = (void *) 0x7fff76a2a010
  p memset($4, 1, $1)
  # $5 = 1990369296
  p (char *)$4
  # $6 = 0x7fff76a2a010 '\001' <repeats 200 times>...
  set $6[0]='['
  set $6[$1]=']'
  p evutil_parse_sockaddr_port($4, $2, $3)
  # $7 = -1

Before:
  $ gdb bin/http-connect < gdb
  (gdb) $1 = 2147483649
  (gdb) (gdb) $2 = (void *) 0x646010
  (gdb) (gdb) $3 = (void *) 0x646030
  (gdb) (gdb) $4 = (void *) 0x7fff76a2a010
  (gdb) (gdb) $5 = 1990369296
  (gdb) (gdb) $6 = 0x7fff76a2a010 '\001' <repeats 200 times>...
  (gdb) (gdb) (gdb) (gdb)
  Program received signal SIGSEGV, Segmentation fault.
  __memcpy_sse2_unaligned () at memcpy-sse2-unaligned.S:36

After:
  $ gdb bin/http-connect < gdb
  (gdb) $1 = 2147483649
  (gdb) (gdb) $2 = (void *) 0x646010
  (gdb) (gdb) $3 = (void *) 0x646030
  (gdb) (gdb) $4 = (void *) 0x7fff76a2a010
  (gdb) (gdb) $5 = 1990369296
  (gdb) (gdb) $6 = 0x7fff76a2a010 '\001' <repeats 200 times>...
  (gdb) (gdb) (gdb) (gdb) $7 = -1
  (gdb) (gdb) quit

Fixes: #318

cmake: don't define EVENT__NEED_DLLIMPORT always (fixes VS2013 static build)

Otherwise that #ifdef in visibility.h is useless, and __declspec(dllimport)
will be always on.

Fixes: #314
Fixes: 4545fa9b6866df47ce2f908631a84477a94d5f49 ("Add option to build shared
library")

Add missing return statement to del_wait_thread so libevent can build.

http-connect: make it win32 compilable

Fixes: commit 1d34498ea31134bd7a8fcadcb95910b30a479e19 ("sample: add HTTP
CONNECT tunnelling example using libevent http layer")

cmake: fix building dns-example under win32 (missing getopt)

visibility: align it to make it more readable

cmake: Fix detection of ssize_t/SSIZE_T

Since ssize_it is POSIX, windows/VS also have this but with BaseTsd.h, plus the
logic prefers "ssize_t" (lower) instead of "SSIZE_T" (upper) when the latest
only available -- fix this too.

Refs: #311

test/regress: cover event_del() waiting mechanism

Since we have some issues (see refs) for changing waiting order in event_del()
I wrote this simple test, so maybe this test can explain something or at least
cover what we have before and show it will be broken.

P.S. we really need avoid such stuff like lets-test-with-sleep/usleep.

Refs: #225
Refs: #226
Refs: #236

Ignore more configure stuff (configure.lineno)

Link: https://lists.gnu.org/archive/html/autoconf/2004-01/msg00043.html

Merge branch 'event_reinit-for-signals-v3'

* event_reinit-for-signals-v3:
  test/regress: cover existing signal callbacks and fork() + event_reinit()
  test/regress: cover signals after fork() + event_reinit()
  test/regress: main/fork: rewrite assertions by just removing event in callback
  event_reinit: make signals works after fork() without evsig_add()
  event_reinit: always re-init signal's socketpair

Fixes #307

test/regress: cover existing signal callbacks and fork() + event_reinit()

Regression-for: 88640aa ("event_reinit: make signals works after fork() without
evsig_add()")

test/regress: cover signals after fork() + event_reinit()

Regression-for: ad0c237 ("event_reinit: always re-init signal's socketpair")

test/regress: main/fork: rewrite assertions by just removing event in callback

Instead of assigning some variable value (got_child), and schedule exit from
loop from that callback, just remove event for that signal, and event loop will
exit automatically when there will be no events.

event_reinit: make signals works after fork() without evsig_add()

event_reinit() removes the event, but only evsig_add puts it back. So any
signals set up before event_reinit will be ignored until another signal is
added.

Fixes: #307

event_reinit: always re-init signal's socketpair

Before this patch event_reinit() only closes the signal socketpair fds and
recreates them if signals have been added, but this is wrong, since socketpair
fds created on backend init, and if we will not re-create them bad things in
child/parent signal handling will happens (and indeed this is what happens for
non-reinit backends like select).

Fixes: #307

Update README.md

Fixed issue with cmake version generation

Merge pull request #125 from libevent/master

update upstream

Merge pull request #305 from libevent/cmake_updates

CMake updates that fix things I thought I fixed.

Cmake is now officially working.

Though I am currently breaking out things that seem to be repetitive, and a
cooler version macro and changelog updater.

More to come.

More cmake updates, lot's of missing definitions

Merge pull request #300 from ellzey/cmake_updates

Cmake updates

CMake syntax fixes fo .in files

Merge pull request #299 from libevent/revert-297-master

Revert "The Windows socket type is defined as SOCKET."

Revert "The Windows socket type is defined as SOCKET."

CMAKE CMAKE CMAKE CLEANUPS

It's almost useful.

Lot's of cmake updates

This is still not done, cmake here was a horrid mess, but we're
getting our act together now.

Merge pull request #297 from billsegall/master

The Windows socket type is defined as SOCKET.

Provide a mechanism for building the library on Windows with different compiler flags. Add a batch file that builds it for the M[DT][d] options and performs a hunt and gather of the different output libraries.

The Windows socket type is defined as SOCKET.

Under the hood it's an unsigned rather than a signed type and whilst C
compilers are largely happy with this C++ compilers tend to be fussy
about class function signatures which makes C++ usage of libevent
problematic.

Merge pull request #296 from lzmths/master

Refactoring conditional directives that break parts of statements.

Merge pull request #289 from mc-server/UnicodeErrMsgFix

Fixed Unicode issue in error messages.

Refactoring conditional directives that break parts of statements.

Fixed Unicode issue in error messages.

be_sock: bufferevent_socket_connect_hostname(): make it thread-safe

If you use bufferevent_socket_connect_hostname() to resolve, then ipv4 answer
can be returned before ipv6 scheduled and if you will destroy bufferevent after
ipv4 answer will come (in a separate thread of course) then ipv6 will trigger
UAF:

$ a.out
=================================================================
==29733==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200000ef50 at pc 0x0000004b7aef bp 0x7fffffffd940 sp 0x7fffffffd0f8
READ of size 2 at 0x60200000ef50 thread T0
    #0 0x4b7aee in __interceptor_index (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4b7aee)
    #1 0x5060eb in string_num_dots /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2739
    #2 0x5078df in search_request_new /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:3214
    #3 0x506afd in evdns_base_resolve_ipv6 /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2935
    #4 0x50aa94 in evdns_getaddrinfo /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:4719
    #5 0x51de4f in evutil_getaddrinfo_async_ /src/oss/libevent/libevent-github/.cmake-debug/../evutil.c:1567
    #6 0x4fe023 in bufferevent_socket_connect_hostname /src/oss/libevent/libevent-github/.cmake-debug/../bufferevent_sock.c:519
    #7 0x524f54 in evhttp_connection_connect_ /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2493
    #8 0x525156 in evhttp_make_request /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2548
    #9 0x52d373 in main (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d373)
    #10 0x7ffff6849b44 in __libc_start_main /tmp/buildd/glibc-2.19/csu/libc-start.c:287
    #11 0x445806 in _start (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x445806)

0x60200000ef50 is located 0 bytes inside of 15-byte region [0x60200000ef50,0x60200000ef5f)
freed by thread T1 here:
    #0 0x4cc4f2 in __interceptor_free (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4cc4f2)
    #1 0x5141c1 in event_mm_free_ /src/oss/libevent/libevent-github/.cmake-debug/../event.c:3512
    #2 0x522402 in evhttp_connection_free /src/oss/libevent/libevent-github/.cmake-debug/../http.c:1206
    #3 0x52cc5f in connection_closer (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52cc5f)
    #4 0x50e80e in event_process_active_single_queue /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1642
    #5 0x50ed57 in event_process_active /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1734
    #6 0x50f458 in event_base_loop /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1957
    #7 0x50eddf in event_base_dispatch /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1768
    #8 0x52d075 in event_dispatch_thread (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d075)
    #9 0x7ffff74fc0a3 in start_thread /tmp/buildd/glibc-2.19/nptl/pthread_create.c:309

Fixes: #217
Closes: #222
Closes: #219
Gist: https://gist.github.com/azat/92cbb34232ac02d7972b (from #217 but thread-safe)

autotools: fix getservbyname() detection

The mentioned commit adds this check under-the-else of the previous condition
between getaddrinfo()/gethostbyname_r(), so this check is triggered only when
we don't have getaddrinfo() which is wrong, fix this by move it upper.

Fixes [CI] since it uses getservbyname() and it failed with autotools only
(cmake detection is ok).

CI: https://travis-ci.org/libevent/libevent/builds/93125954
Fixes: af08a94085e49e6942835b4c6b50a774536d5b5b ("Check for getservbyname even
if not on win32.")

test/dns: check exit code of evdns_getaddrinfo()

By some reason all autotools builds failed:
CI: https://travis-ci.org/libevent/libevent/builds/93125954

Merge branch 'evdns-fail-requests-v6'

This patchset fixes some issues wit evdns_base_free(..., fail_requests=1), and
there are two cases: with callback wrapper for evdns_getaddrinfo() and not,
both fixed by this patches, and adds regression tests for this.

* evdns-fail-requests-v6:
  evdns: evdns_base_free(): fix UAF of evdns_base with @fail_requests
  test/dns: cover evdns_getaddrinfo() and evdns_base_free() with @fail_requests
  evdns: evdns_base_free(): free requests before namservers
  test/dns: cover @fail_requests for evdns_base_free()
  test/dns: more graceful coverage of @fail_requests

evdns: evdns_base_free(): fix UAF of evdns_base with @fail_requests

If you call evdns_base_free() with @fail_requests == 1, then it will defer
callback with DNS_ERR_SHUTDOWN, but that callback (internal) uses
data->evdns_base, but we already freed that evdns base, so we can't do
this, fix this by checking @result to DNS_ERR_SHUTDOWN.

Fixes: regress dns/client_fail_requests_getaddrinfo
Fixes: #269

test/dns: cover evdns_getaddrinfo() and evdns_base_free() with @fail_requests

evdns: evdns_base_free(): free requests before namservers

Otherwise we will trigger next UAF:
$ valgrind --vgdb-error=1 regress --no-fork +dns/client_fail_requests
==24733== Memcheck, a memory error detector
==24733== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24733== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==24733== Command: regress --no-fork +dns/client_fail_requests
==24733==
==24733==
==24733== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==24733==   /path/to/gdb regress
==24733== and then give GDB the following command
==24733==   target remote | /usr/lib/valgrind/../../bin/vgdb --pid=24733
==24733== --pid is optional if only one valgrind process is running
==24733==
dns/client_fail_requests: ==24733== Invalid read of size 4
==24733==    at 0x4C3352: request_finished (evdns.c:662)
==24733==    by 0x4CC8B7: evdns_base_free_and_unlock (evdns.c:4048)
==24733==    by 0x4CCAFD: evdns_base_free (evdns.c:4088)
==24733==    by 0x458E95: dns_client_fail_requests_test (regress_dns.c:2039)
==24733==    by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733==    by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733==    by 0x48F67E: tinytest_main (tinytest.c:434)
==24733==    by 0x47C0DA: main (regress_main.c:461)
==24733==  Address 0x61e6f70 is 448 bytes inside a block of size 456 free'd
==24733==    at 0x4C29EAB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24733==    by 0x4A8F4D: event_mm_free_ (event.c:3512)
==24733==    by 0x4CC7A1: evdns_nameserver_free (evdns.c:4021)
==24733==    by 0x4CC7DC: evdns_base_free_and_unlock (evdns.c:4037)
==24733==    by 0x4CCAFD: evdns_base_free (evdns.c:4088)
==24733==    by 0x458E95: dns_client_fail_requests_test (regress_dns.c:2039)
==24733==    by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733==    by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733==    by 0x48F67E: tinytest_main (tinytest.c:434)
==24733==    by 0x47C0DA: main (regress_main.c:461)
==24733==  Block was alloc'd at
==24733==    at 0x4C28C4F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24733==    by 0x4A8D5A: event_mm_malloc_ (event.c:3437)
==24733==    by 0x4C8B96: evdns_nameserver_add_impl_ (evdns.c:2505)
==24733==    by 0x4C916D: evdns_base_nameserver_ip_add (evdns.c:2629)
==24733==    by 0x458DA3: dns_client_fail_requests_test (regress_dns.c:2031)
==24733==    by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733==    by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733==    by 0x48F67E: tinytest_main (tinytest.c:434)
==24733==    by 0x47C0DA: main (regress_main.c:461)
==24733==
==24733== (action on error) vgdb me ...

Fixes: regress dns/client_fail_requests
Fixes: #269

test/dns: cover @fail_requests for evdns_base_free()

test/dns: more graceful coverage of @fail_requests

In case when evdns_base_free() called with @fail_requests, we can potentially
have leaks, but we can avoid them if we will run event loop once again to
trigger defer cbs, so let's do this, instead of magical decrements (and also
this will give an example how to avoid leaks for evdns).

sample/https-client: allow to change path to ca-certificates

And default to one that linux have, for freebsd you will need:
$ https-client -url https://google.com -crt /usr/local/share/certs/ca-root-nss.crt

sample/https-client: check for ERR_remove_thread_state() existence

Fixes: freebsd 9.2 build

http: avoid epoll_ctl() on already closed fd (triggers by http/chunk_out)

Before:
$ strace -eepoll_ctl,close regress --verbose --no-fork +http/chunk_out
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
close(3)                                = 0
http/chunk_out: epoll_ctl(5, EPOLL_CTL_ADD, 8, {EPOLLIN, {u32=8, u64=8}}) = 0
close(10)                               = 0
close(9)                                = 0
epoll_ctl(5, EPOLL_CTL_ADD, 9, {EPOLLIN, {u32=9, u64=9}}) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 10, {EPOLLOUT, {u32=10, u64=10}}) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 11, {EPOLLIN, {u32=11, u64=11}}) = 0
epoll_ctl(5, EPOLL_CTL_DEL, 10, 7ffffb10b870) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 10, {EPOLLIN, {u32=10, u64=10}}) = 0
close(12)                               = 0
epoll_ctl(5, EPOLL_CTL_MOD, 11, {EPOLLIN|EPOLLOUT, {u32=11, u64=11}}) = 0
epoll_ctl(5, EPOLL_CTL_MOD, 11, {EPOLLIN, {u32=11, u64=11}}) = 0
close(11)                               = 0
epoll_ctl(5, EPOLL_CTL_DEL, 11, 7ffffb10b770) = -1 EBADF (Bad file descriptor)
...

And trace for latest:
close(11)                               = 0
> regress(evutil_closesocket+0x15) [0xadac4]
> regress(evhttp_connection_free+0x19d) [0xbada2]
> regress(evhttp_send_done+0x14d) [0xbde43]
> regress(evhttp_write_cb+0x4a) [0xba1a1]
> regress(bufferevent_run_writecb_+0xa2) [0x97bba]
> regress(bufferevent_trigger_nolock_+0xb1) [0x9e330]
> regress(bufferevent_writecb+0x2a3) [0x9e91a]
> regress(event_persist_closure+0x2bb) [0xa2e09]
> regress(event_process_active_single_queue+0x2a8) [0xa30b4]
> regress(event_process_active+0x13f) [0xa3696]
> regress(event_base_loop+0x2ab) [0xa3d97]
> regress(event_base_dispatch+0x1d) [0xa371e]
> regress(http_chunk_out_test_impl+0x132) [0x74940]
epoll_ctl(5, EPOLL_CTL_DEL, 11, 7fff09439fe0) = -1 EBADF (Bad file descriptor)
> regress(epoll_ctl+0x36) [0xb49a3]
> regress(epoll_apply_one_change+0x14e) [0xb4cd4]
> regress(epoll_nochangelist_del+0x87) [0xb51e4]
> regress(evmap_io_del_+0x247) [0xaab04]
> regress(event_del_nolock_+0x2f7) [0xa6aa8]
> regress(event_callback_cancel_nolock_+0x6e) [0xa747e]
> regress(event_callback_finalize_many_+0xeb) [0xa4dd6]
> regress(bufferevent_decref_and_unlock_+0x21d) [0x98da7]
> regress(bufferevent_writecb+0x312) [0x9e989]
> regress(event_persist_closure+0x2bb) [0xa2e09]
> regress(event_process_active_single_queue+0x2a8) [0xa30b4]
> regress(event_process_active+0x13f) [0xa3696]
> regress(event_base_loop+0x2ab) [0xa3d97]
> regress(event_base_dispatch+0x1d) [0xa371e]
> regress(http_chunk_out_test_impl+0x132) [0x74940]
...

http: install timeout for read too during connect for ssl

Since during ssl handshake we can read too, and if something nasty will happens
during this handshake (because of too many events in the loop of something like
this) we can wait forever since there is no read timeout:
  (gdb) p *$2.task.connection.bufev
  $11 = {
    ...
    be_ops = 0x7f78c2864b00 <bufferevent_ops_openssl>,
    ev_read = {
      ...
      ev_ = {
        ev_io = {
          ...
          ev_timeout = { tv_sec = 0, tv_usec = 0 }
        },
        ev_signal = { ... }
      },
      ev_events = 82, ev_res = 2,
      ev_timeout = { tv_sec = 10889976, tv_usec = 418753 }
    },
    ev_write = {
      ...
      ev_ = {
        ev_io = {
          ...
          ev_timeout = { tv_sec = 20, tv_usec = 0 }
        },
        ev_signal = { ... }
      },
      ev_events = 84, ev_res = 4,
      ev_timeout = { tv_sec = 10889977, tv_usec = 598753 }
    },
    ...
    errorcb = 0x7f78c287de70 <evhttp_connection_cb>,
    ...
    timeout_read = { tv_sec = 0, tv_usec = 0 },
    timeout_write = { tv_sec = 20, tv_usec = 0 },
    enabled = 4
  }
  (gdb) bt
  #0  0x00007f78c17c3633 in __epoll_wait_nocancel () at syscall-template.S:81
  #1  0x00007f78c2aaf508 in epoll_dispatch (base=0x18f76d0, tv=<optimized out>) at epoll.c:463
  ...

Found-with: massive crawling
Tested-with: massive crawling

test/ssl: cover busy-loop (i.e. {read,write}-blocked-on-{write,read} stuff)

This covers SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE error codes from ssl,
under which we must block read/write to avoid busy looping, and hence extra CPU
usage.
This test introduces custom BIO that will count read/write and validates
counters, with patches for be_openssl that drops handling
SSL/SSL_ERROR_WANT_READ there are more then 43K reads, so 100 is pretty ok.

test/http: write_during_read for https

test/http: connection_fail for https

test/http: stream_out for https

be_openssl: don't call do_write() directly from outbuf_cb

Otherwise we can trigger incorrect callback, the simplest way to trigger this
is using http regression tests -- https_chunk_out, since all it do is:
  evhttp_send_reply_end()
    evbuffer_add()
      do_write()
    evhttp_write_buffer()
      evcon->cb = cb

And indeed this is what happens:
  (gdb) bt
  #0  do_write (bev_ssl=0x738a90, atmost=16384) at bufferevent_openssl.c:717
  #1  0x00000000004b69f7 in consider_writing (bev_ssl=0x738a90) at bufferevent_openssl.c:875
  #2  0x00000000004b7386 in be_openssl_outbuf_cb (buf=0x7387b0, cbinfo=0x7fffffffd590, arg=0x738a90) at bufferevent_openssl.c:1147
  #3  0x0000000000490100 in evbuffer_run_callbacks (buffer=0x7387b0, running_deferred=0) at buffer.c:508
  #4  0x00000000004901e5 in evbuffer_invoke_callbacks_ (buffer=0x7387b0) at buffer.c:529
  #5  0x0000000000493a30 in evbuffer_add (buf=0x7387b0, data_in=0x4ecfb2, datlen=5) at buffer.c:1803
  #6  0x00000000004be2e3 in evhttp_send_reply_end (req=0x7371a0) at http.c:2794
  #7  0x000000000045c407 in http_chunked_trickle_cb (fd=-1, events=1, arg=0x75aaf0) at regress_http.c:402
  ...
  (gdb) p bev.writecb
  $4 = (bufferevent_data_cb) 0x4ba17e <evhttp_write_cb>
  $5 = (void *) 0x7379b0
  (gdb) p (struct evhttp_connection *)bev.cbarg
  $6 = (struct evhttp_connection *) 0x7379b0
  (gdb) p $6->cb
  $7 = (void (*)(struct evhttp_connection *, void *)) 0x0

And be_sock don't do like this anyway.

Fixes: https_chunk_out

test/http: chunk_out for https

epoll: introduce PRINT_CHANGES() macro to avoid copy-pasting

And also this will use change_to_string() for successfully returned
epoll_ctl()

sample/https-client: replace ERR_remove_state() by ERR_remove_thread_state()

Since ERR_remove_state() is deprecated:
  $ git log --grep ERR_remove_thread_state
  commit 2ecd2ededece66bf090fefc93ef3ddb672d9e71a
  Author: Bodo Möller <bodo@openssl.org>
  Date:   Wed Aug 13 19:30:01 2008 +0000

      Mention ERR_remove_state() deprecation, and ERR_remove_thread_state(NULL).

Link: https://www.openssl.org/docs/manmaster/crypto/ERR_remove_state.html

Add missing <string.h> for openssl_hostname_validation module

Now it included by openssl, but nfter
openssl/openssl@master-post-reformat-1494-g6329b60 it will print warning
(apparently they dropped <string.h> from the generic headers).

Merge branch 'be-openssl-more-common-with-be-sock'

* be-openssl-more-common-with-be-sock:
  be_openssl: use bufferevent_enable() instead of bufferevent_add_event_()
  be_sock: drop be_sock_add() macro (useless and debug unfriendly)
  be: introduce bufferevent_generic_adj_existing_timeouts_()
  be_openssl: don't add events during bev creation (like be_sock)
  be: add_event: use evutil_timerisset()

test/regress: fix ssl-less builds (need to make this prettier)

CI: https://travis-ci.org/libevent/libevent/jobs/89590839

be_openssl: use bufferevent_enable() instead of bufferevent_add_event_()

By using bufferevent_enable() there will be no event for READ *or* WRITE if
they are not enabled before, and this patch reduces difference for
be_sock_enable/be_openssl_enable (handshake)

be_sock: drop be_sock_add() macro (useless and debug unfriendly)

be: introduce bufferevent_generic_adj_existing_timeouts_()

And use it in openssl/sock layers to avoid copy-pasting it's variants.

be_openssl: don't add events during bev creation (like be_sock)

Using the following examples you can get changes between be_openssl and
be_sock:
$ function diff_addr()
{
eval diff -u $(printf "<(strip_addr %s) " "$@")
}
$ function strip_addr()
{
sed 's/0x[a-zA-Z0-9]*/0xFFFF/g' "$@"
}
$ EVENT_DEBUG_LOGGING_ALL= regress --verbose --no-fork +http/https_connection_retry 2> /tmp/https-retry.log >&2
$ EVENT_DEBUG_LOGGING_ALL= regress --verbose --no-fork +http/connection_retry 2> /tmp/http-retry.log >&2
$ diff_addr /tmp/http-retry.log /tmp/https-retry.log

be: add_event: use evutil_timerisset()

Merge branch 'https-coverage-v6'

This patchset adds some basic tests to cover some https cases:
$ regress +http/https_..
http/https_basic: [forking] OK
http/https_simple: [forking] OK
http/https_simple_dirty: [forking] OK
http/https_incomplete: [forking] OK
http/https_incomplete_timeout: [forking] OK
http/https_connection_retry: [forking] OK
http/https_connection_retry_conn_address: [forking] OK
7 tests ok.  (0 skipped)

But there are some leaks in http regression tests (like init_ssl() and others),
must be fixed by using custom setup routine.

* https-coverage-v6:
  test/http: allow dirty shutdown for ssl to fix https_incomplete
  test/http: https basic
  test/http: incomplete{,_timeout} for https
  test/http: add simplest test for http/https/https_dirty_shutdown
  test/http: https: retry coverage
  test/http: https server support (plus some helpers)
  test/http: more sanity checks
  test/ssl: export getkey()/getcert()/get_ssl_ctx()/init_ssl() for https

test/http: allow dirty shutdown for ssl to fix https_incomplete

test/http: https basic

test/http: incomplete{,_timeout} for https

test/http: add simplest test for http/https/https_dirty_shutdown

There is "basic" before, but it uses bufferevent's for creating requests for
http layer, while this test uses http layer for client requests too.

test/http: https: retry coverage

test/http: https server support (plus some helpers)

TODO:
- check BEV_OPT_DEFER_CALLBACKS separately

test/http: more sanity checks

test/ssl: export getkey()/getcert()/get_ssl_ctx()/init_ssl() for https

Fix lock leak in be_pair_flush() if flush type is BEV_NORMAL

Fixes: bufferevent/bufferevent_pair_flush_normal
  $ regress --no-fork bufferevent/bufferevent_pair_flush_normal
  bufferevent/bufferevent_pair_flush_normal:
    FAIL ../test/regress_bufferevent.c:181: assert(bufferevent_pair_get_partner(bev1) == NULL): 0x177fa20 vs (nil)
    [bufferevent_pair_flush_normal FAILED]

Fixes #266

test/regress_be: basic coverage bufferevent_flush() for pair/sock layers

Refs #266

test/regress_be: socket_filter_inactive: check bufferevent after creation

Fixes:
../test/regress_bufferevent.c: In function ‘test_bufferevent_socket_filter_inactive’:
../test/regress_bufferevent.c:1180:1: warning: label ‘end’ defined but not used [-Wunused-label]
end:

Free event queues even for recursive finalizers

For finalizers we can register yet another finalizer out from finalizer, and
iff finalizer will be in active_later_queue we can add finalizer to
activequeues, and we will have events in activequeues after event_base_free()
returns, which is not what we want (we even have an assertion for this).

A simple case is bufferevent with underlying (i.e. filters) in inactive queue.

Fixes: regress bufferevent/bufferevent_socket_filter_inactive

test/regress_be: cover finalizers from inactive to active queue

Right now this will fail with the next assertion:
$ regress --no-fork --verbose bufferevent/bufferevent_socket_filter_inactive
bufferevent/bufferevent_socket_filter_inactive: [err] ../event.c:862: Assertion TAILQ_EMPTY(&base->activequeues[i]) failed in event_base_free_
Aborted

buffer_compat: fix comment -- we have EVBUFFER_EOL_ANY not EOL_STYLE_ANY

test/regress_buffer: fix clang compilation warnings

../test/regress_buffer.c:201:12: warning: logical not is only applied to the left hand side of this comparison [-Wlogical-not-parentheses]
tt_assert(!memcmp((char*)EVBUFFER_DATA(evb), "1/hello", 7) != 0);

test/regress_http: fix compilation warnings (-Wmissing-field-initializers)

test/regress_dns: fix compilation warnings (-Wmissing-field-initializers/for)

I don't have an error for loop because gcc5 have --std=gnu11 by default.
We need some options-consistency for all versions/compilers and build systems
to avoid such patches.

Fixes: https://travis-ci.org/libevent/libevent/jobs/84403473
Fixes: https://travis-ci.org/libevent/libevent/builds/84403463

Merge branch 'evdns-randomization'

* evdns-randomization:
  evdns: fix randomize-case by make case-insensitive as required
  tests/regress_dns: cover that randomize-case works case-insensitive
  evnds: inline TEST_NAME macro to make debuggin easier

Fixes #288

evdns: fix randomize-case by make case-insensitive as required

Interesting that this wasn't found by regression tests since they respond with
that SoME-rAndDom-CaSe domains, and no case-insensitive mode is required during
comparing response from the server and request.

Fixes #288
Covered-by: regress dns/search_lower

tests/regress_dns: cover that randomize-case works case-insensitive

Regression-for: #288

evnds: inline TEST_NAME macro to make debuggin easier

make test/regress_ssl.c compile without warnings

evthread: fix evthread_setup_global_lock_() for debug-lock with a real-lock case

Sample bt:
  (gdb) bt
  #0  0x00000000004a95d2 in evthread_setup_global_lock_ (lock_=0x0, locktype=0, enable_locks=1)
  #1  0x00000000004afbc7 in evsig_global_setup_locks_ (enable_locks=1) at signal.c:475
  #2  0x00000000004a5da8 in event_global_setup_locks_ (enable_locks=1) at event.c:3838
  #3  0x00000000004a8983 in evthread_set_lock_callbacks (cbs=0x7fffffffdde0) at evthread.c:129
  #4  0x0000000000445a8f in use_lock_unlock_profiler () at regress_bufferevent.c:306
  #5  0x0000000000445af2 in test_bufferevent_pair_release_lock (arg=0x70e710) at regress_bufferevent.c:327
  #6  0x000000000048c9a8 in testcase_run_bare_ (testcase=0x6fdc10 <bufferevent_testcases+80>) at tinytest.c:105
  #7  0x000000000048cc8a in testcase_run_one (group=0x6ff270 <testgroups+112>, testcase=0x6fdc10 <bufferevent_testcases+80>) at tinytest.c:252
  #8  0x000000000048d5c9 in tinytest_main (c=3, v=0x7fffffffe008, groups=0x6ff200 <testgroups>) at tinytest.c:434
  #9  0x000000000047a08f in main (argc=3, argv=0x7fffffffe008) at regress_main.c:459

Found-with: regress bufferevent/bufferevent_pair_release_lock (-DEVENT__DISABLE_DEBUG_MODE=ON)

Fix checking for make_base_notifiable()

Fixes: a068f2e5 ("event_debug_created_threadable_ctx_: fix compilation without
debug mode")
Found-after: 3e56da23 ("travis: add builds without debug mode into matrix")

test/regress_be: drop debug __asm__(int3) to fix arm build

Closes #284

travis: add builds without debug mode into matrix

To cover compilation with this flag, since brew for example uses it by default.