]> granicus.if.org Git - libevent/log
libevent
5 years agobuffer: fix evbuffer_remove_buffer() with empty chain in front
Azat Khuzhin [Sat, 2 Mar 2019 19:50:00 +0000 (22:50 +0300)]
buffer: fix evbuffer_remove_buffer() with empty chain in front

In case we have empty chain (chain that do not have any data, i.e. ->off
== 0) at the beginning of the buffer, and no more full chains to move to
the dst, we will skip moving of this empty chain, and hence
last_with_datap will not be adjusted, and things will be broken after.

Fix this by not relying on ->off, just count if we have something to
move that's it.

Test case from:
  https://github.com/envoyproxy/envoy/pull/6062

Fixes: #774
5 years agotest: verify content of the buffer in evbuffer/remove_buffer_with_empty*
Azat Khuzhin [Sun, 3 Mar 2019 14:43:37 +0000 (17:43 +0300)]
test: verify content of the buffer in evbuffer/remove_buffer_with_empty*

And replace spaces with tab in remove_buffer_with_empty

5 years agoRemove needless check for arc4_seeded_ok
Seong-Joong Kim [Thu, 21 Feb 2019 16:19:10 +0000 (16:19 +0000)]
Remove needless check for arc4_seeded_ok

5 years agoFix typos: warnigns -> warnings
Enji Cooper [Fri, 1 Mar 2019 01:47:51 +0000 (17:47 -0800)]
Fix typos: warnigns -> warnings

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
5 years agoFix typos in a comment describing the purpose of `VersionViaGit.cmake`
Enji Cooper [Fri, 1 Mar 2019 01:22:30 +0000 (17:22 -0800)]
Fix typos in a comment describing the purpose of `VersionViaGit.cmake`

Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
5 years agoDefine `_GNU_SOURCE` properly/consistently per autoconf
Enji Cooper [Mon, 25 Feb 2019 19:59:15 +0000 (11:59 -0800)]
Define `_GNU_SOURCE` properly/consistently per autoconf

Although `_GNU_SOURCE` can be defined as an arbitrary #define per the
glibc docs [1], it's best to define it in a manner consistent with the way
that autoconf defines it, i.e., `1`.

While this shouldn't matter in most cases, it does when the headers from
other projects follow the poorly defined GNU convention implemented by
autoconf and are included after the libevent's util.h header. An example
failure with clang, similar to the failure I encountered, is as follows:
```
$ printf "#define _GNU_SOURCE\n#define _GNU_SOURCE 1" | clang -c -x c -
<stdin>:2:9: warning: '_GNU_SOURCE' macro redefined [-Wmacro-redefined]
        ^
<stdin>:1:9: note: previous definition is here
        ^
1 warning generated.
```

This happened when compiling python [2] with a stale homebrew util.h file from
libevent (which admittedly would not happen in a correct libevent install, as the
header should be installed under /usr/local/include/event2/util.h). However, if
both headers had been combined (which is more likely), it would have failed as
shown above.

Removing the ad hoc definition unbreaks compiling python's pyconfig.h.in header
when included after util.h from libevent.

1. http://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html
2. https://github.com/python/cpython/blob/master/configure.ac#L126

Closes: #773 (cherry-picked)
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
5 years agoDon't loose top error in SSL
Yury Korzhetsky [Tue, 26 Feb 2019 09:33:49 +0000 (12:33 +0300)]
Don't loose top error in SSL

Closes: #775 (cherry-picked)
5 years agoIgnore bitrise branches for travis-ci/appvyeor [ci skip]
Azat Khuzhin [Sun, 24 Feb 2019 21:26:39 +0000 (00:26 +0300)]
Ignore bitrise branches for travis-ci/appvyeor [ci skip]

5 years agoUpdate link to the appveyor project
Azat Khuzhin [Sun, 24 Feb 2019 19:59:33 +0000 (22:59 +0300)]
Update link to the appveyor project

Since the migration request has been hanged for a while, let's switch it
for now without beauty API URL.

Fixes: #555
5 years agoMatch definition with prototype for bufferevent_socket_set_conn_address_fd_()
Azat Khuzhin [Sun, 24 Feb 2019 14:34:20 +0000 (17:34 +0300)]
Match definition with prototype for bufferevent_socket_set_conn_address_fd_()

Fixes: #772
Fixes: b29207dc ("Eliminate fd conversion warnings and introduce EVUTIL_INVALID_SOCKET (windows)")
5 years agoRevert "test: avoid regress hanging in macOS"
Azat Khuzhin [Sun, 24 Feb 2019 14:25:31 +0000 (17:25 +0300)]
Revert "test: avoid regress hanging in macOS"

After we started to use kill() over raise() everything should work just
fine.

This reverts commit a86f89d333d870e6714bd28c695ba1774df3d7f5.

Fixed-in: 728c5dc1 ("Use kill() over raise() for raising the signal (fixes osx 10.14 with kqueue)")
Fixes: #747
5 years agoUse kill() over raise() for raising the signal (fixes osx 10.14 with kqueue)
Azat Khuzhin [Sun, 24 Feb 2019 14:07:18 +0000 (17:07 +0300)]
Use kill() over raise() for raising the signal (fixes osx 10.14 with kqueue)

On OSX 10.14+ the raise() uses pthread_kill() (verified with dtruss) and
by some reason signals that has been raised with pthread_kill() do not
received by kqueue EVFILT_SIGNAL.

While on OSX 10.11 the raise()/pthread_kill() uses plain kill() and
everything work just fine (linux also does the same, but instead of
kill() it uses tgkill())

Here is a simple reproducer that installs alarm to show that the signal
does not received by the kqueue backend:
  https://gist.github.com/azat/73638b8e3b0fa563a20dadcca9e652a1

Refs: #747
Fixes: #765
5 years agoMerge pull request #769 from sungjungk/fix-return-handling
Nathan French [Thu, 21 Feb 2019 15:21:02 +0000 (10:21 -0500)]
Merge pull request #769 from sungjungk/fix-return-handling

Add error-handling routine for arc4_seed()

5 years agoAdd error-handling routine for arc4_seed()
Seong-Joong Kim [Thu, 21 Feb 2019 09:13:05 +0000 (18:13 +0900)]
Add error-handling routine for arc4_seed()

5 years agoAdded DNS header mask definitions.
Nathan French [Thu, 7 Feb 2019 16:43:14 +0000 (11:43 -0500)]
Added DNS header mask definitions.

Currently, we do a lot of data munging with manual hex. This is ugly
and can lead to bugs. I defined the following:

_QR_MASK    0x8000U
_OP_MASK    0x7800U
_AA_MASK    0x0400U
_TC_MASK    0x0200U
_RD_MASK    0x0100U
_RA_MASK    0x0080U
_Z_MASK     0x0040U
_AD_MASK    0x0020U
_CD_MASK    0x0010U
_RCODE_MASK 0x000fU

So that we can more easily twiddle flags.

v2: make evdns flag masks unsigned literal
Closes: #756 (cherry-picked)
5 years agoUse the logo from the libevent.org
Azat Khuzhin [Tue, 5 Feb 2019 21:12:35 +0000 (00:12 +0300)]
Use the logo from the libevent.org

5 years agotest/dns: in solaris under EMFILE devpoll does not dispatch (due DP_POLL failure)
Azat Khuzhin [Mon, 4 Feb 2019 19:34:10 +0000 (22:34 +0300)]
test/dns: in solaris under EMFILE devpoll does not dispatch (due DP_POLL failure)

5 years agocmake: fix checking of devpoll backend (like in autotools, by devpoll.h existence)
Azat Khuzhin [Mon, 4 Feb 2019 19:18:45 +0000 (22:18 +0300)]
cmake: fix checking of devpoll backend (like in autotools, by devpoll.h existence)

5 years agotest/dns: in solaris under EMFILE the error is EAI_FAIL
Azat Khuzhin [Sun, 3 Feb 2019 15:54:00 +0000 (18:54 +0300)]
test/dns: in solaris under EMFILE the error is EAI_FAIL

5 years agotest/ssl/bufferevent_wm: explicitly break the loop once client/server received enough
Azat Khuzhin [Sun, 3 Feb 2019 15:47:14 +0000 (18:47 +0300)]
test/ssl/bufferevent_wm: explicitly break the loop once client/server received enough

There can be tricky cases (that can be reproduced by reducing
SO_RCVBUF/SO_SNDBUF to 6144, on linux, and be aware, since linux doubles
this const), when there is still write event pending, although we read
enough.

This should be fixed in a more sophisticated way, but to backport the
patch, let's simply break the loop manually.

The ssl/bufferevent_wm originally failed on solaris.

5 years agoEliminate fd conversion warnings and introduce EVUTIL_INVALID_SOCKET (windows)
Azat Khuzhin [Tue, 29 Jan 2019 18:12:33 +0000 (21:12 +0300)]
Eliminate fd conversion warnings and introduce EVUTIL_INVALID_SOCKET (windows)

windows has intptr_t instead of regular int.

Also tt_fd_op() had been introduced, since we cannot use tt_int_op() for
comparing fd, since it is not always int.

5 years agoevmap: use approparite type for changes to avoid warnings under windows
Azat Khuzhin [Tue, 29 Jan 2019 18:31:19 +0000 (21:31 +0300)]
evmap: use approparite type for changes to avoid warnings under windows

5 years agotest/et/et: use evutil_socket_t* over int* for pointer to the pair
Azat Khuzhin [Tue, 29 Jan 2019 18:06:37 +0000 (21:06 +0300)]
test/et/et: use evutil_socket_t* over int* for pointer to the pair

Next code will not work correctly under win x64:
  evutil_socket_t very_long_pair_name[2];
  int *pair = very_long_pair_name; // <-- accessing the second word of the first element

Because sizeof(evutil_socket_t) == sizeof(intptr_t) == 8

P.S. in the 5334762f another test had been fixed instead of the one that
really fails.

Fixes: 5334762f ("test/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)")
Refs: #750

5 years agoappveyor: disable verbosity of MSBuild
Azat Khuzhin [Tue, 29 Jan 2019 17:58:01 +0000 (20:58 +0300)]
appveyor: disable verbosity of MSBuild

5 years agoMerge branch 'win64-fixes'
Azat Khuzhin [Tue, 29 Jan 2019 07:54:39 +0000 (10:54 +0300)]
Merge branch 'win64-fixes'

* win64-fixes:
  test/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)
  test/et/et: verify return codes
  appveyor: switch to new VS/MinGW and x64

5 years agotest/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)
Azat Khuzhin [Tue, 29 Jan 2019 07:42:50 +0000 (10:42 +0300)]
test/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)

Fixes: #750
5 years agotest/et/et: verify return codes
Azat Khuzhin [Mon, 28 Jan 2019 22:44:33 +0000 (01:44 +0300)]
test/et/et: verify return codes

5 years agoappveyor: switch to new VS/MinGW and x64
Azat Khuzhin [Sat, 24 Nov 2018 17:22:40 +0000 (20:22 +0300)]
appveyor: switch to new VS/MinGW and x64

The cache had been reseted with the following REST API requests:
  DELETE https://ci.appveyor.com/api/projects/nmathewson/libevent/buildCache
  DELETE https://ci.appveyor.com/api/projects/libevent/libevent/buildCache
  DELETE https://ci.appveyor.com/api/projects/azat/libevent/buildCache

5 years agowin32select: print an error in case of failure
Azat Khuzhin [Tue, 29 Jan 2019 04:56:21 +0000 (07:56 +0300)]
win32select: print an error in case of failure

5 years agoAdd OS-specific exclude patterns of the branch names for the CI
Azat Khuzhin [Tue, 29 Jan 2019 07:10:23 +0000 (10:10 +0300)]
Add OS-specific exclude patterns of the branch names for the CI

5 years agos/http-server: fix cleanup routines
Azat Khuzhin [Mon, 28 Jan 2019 22:23:02 +0000 (01:23 +0300)]
s/http-server: fix cleanup routines

Fixes: bdd71f18 ("s/http-server: graceful cleanup")
5 years agoMerge branch 'http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2'
Azat Khuzhin [Mon, 28 Jan 2019 22:18:39 +0000 (01:18 +0300)]
Merge branch 'http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2'

* http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2:
  http: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR
  test: add logging for http/read_on_write_error and rearrange code
  http: do not call deferred readcb if readcb is not set

Refs: #749

5 years agohttp: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR
Azat Khuzhin [Mon, 28 Jan 2019 22:09:44 +0000 (01:09 +0300)]
http: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR

There are two possible ways of getting response from the server:
- processing existing bufferevent buffer
- reading from the socket (even after write() errored with -1, it is
still possible)

But we did not tried the first option, only the second one.

Fixes: http/read_on_write_error (on freebsd/osx)
5 years agotest: add logging for http/read_on_write_error and rearrange code
Azat Khuzhin [Sun, 27 Jan 2019 12:28:28 +0000 (15:28 +0300)]
test: add logging for http/read_on_write_error and rearrange code

5 years agohttp: do not call deferred readcb if readcb is not set
Azat Khuzhin [Sun, 27 Jan 2019 12:16:39 +0000 (15:16 +0300)]
http: do not call deferred readcb if readcb is not set

Otherwise evhttp_read_cb can be called with invalid connection state:
    http/read_on_write_error: [forking] [msg] libevent using: kqueue
      FAIL ../test/regress_http.c:4079: assert(req)
      FAIL ../test/regress_http.c:4087: assert(req)[err] evhttp_read_cb: illegal connection state 0

5 years agoMerge branch 'travis-ci-osx-fixes'
Azat Khuzhin [Mon, 28 Jan 2019 19:19:20 +0000 (22:19 +0300)]
Merge branch 'travis-ci-osx-fixes'

* travis-ci-osx-fixes:
  travis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8
  travis-ci/osx: install gcc and fix CC

5 years agotravis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8
Azat Khuzhin [Mon, 28 Jan 2019 18:22:27 +0000 (21:22 +0300)]
travis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8

Refs: travis-ci/travis-ci#9640

5 years agotravis-ci/osx: install gcc and fix CC
Azat Khuzhin [Mon, 28 Jan 2019 17:30:57 +0000 (20:30 +0300)]
travis-ci/osx: install gcc and fix CC

Since there is no gcc in osx_image [1]:
    ls: /usr/local/bin/gcc-?.?: No such file or directory

  [1]: https://travis-ci.org/libevent/libevent/jobs/484794192#L95

And use ls(1) over echo(1) to show an error if there is no such file.

5 years agotest: adjust expecting error for getaddrinfo() under EMFILE
Azat Khuzhin [Sat, 26 Jan 2019 15:52:33 +0000 (18:52 +0300)]
test: adjust expecting error for getaddrinfo() under EMFILE

When getaddrinfo() cannot allocate file descriptor glibc/musl-libc on
linux report EAI_SYSTEM error. But this is not true for freebsd libc [1]
(and hence apple libc [2]), they report EAI_NONAME error instead, so
adjust expectation.

  [1]: https://github.com/freebsd/freebsd/blob/master/lib/libc/net/getaddrinfo.c
  [2]: https://opensource.apple.com/source/Libc/

Refs: #749
Refs: https://github.com/libevent/libevent/issues/749#issuecomment-457838159

5 years agotest/nonpersist_readd: use assert helpers
Azat Khuzhin [Sat, 12 Jan 2019 10:58:50 +0000 (13:58 +0300)]
test/nonpersist_readd: use assert helpers

To debug failure under win32 in appveyor:
  https://ci.appveyor.com/project/nmathewson/libevent/builds/21559140/job/dn16qdo1j6sr497t#L1620

5 years agorpc: use *_new_with_arg() to match function prototype
Azat Khuzhin [Fri, 11 Jan 2019 18:52:11 +0000 (21:52 +0300)]
rpc: use *_new_with_arg() to match function prototype

In 755fbf16c ("Add void* arguments to request_new and reply_new
evrpc hooks") this new functions had been introduced, but newer used,
what for? So let's use them.

5 years agohttp: Preserve socket error from listen across closesocket cleanup
Luke Dashjr [Fri, 28 Dec 2018 04:42:20 +0000 (04:42 +0000)]
http: Preserve socket error from listen across closesocket cleanup

Closes: #738 (cherry-picked)
5 years agocmake: support static runtime (MSVC)
Azat Khuzhin [Mon, 17 Dec 2018 18:31:54 +0000 (21:31 +0300)]
cmake: support static runtime (MSVC)

Fixes: #737
5 years agoappveyor: cover static libs for MSVC
Azat Khuzhin [Mon, 17 Dec 2018 18:41:32 +0000 (21:41 +0300)]
appveyor: cover static libs for MSVC

5 years agofix a comment error
baixiangcpp [Sat, 15 Dec 2018 13:45:36 +0000 (21:45 +0800)]
fix a comment error

Closes: #736 (cherry-picked)
5 years agocmake: Fix some typos in option descriptions
Fredrik Strupe [Sat, 15 Dec 2018 11:54:31 +0000 (12:54 +0100)]
cmake: Fix some typos in option descriptions

5 years agohttp: fix connection retries when there more then one request for connection
Azat Khuzhin [Fri, 7 Dec 2018 18:46:27 +0000 (21:46 +0300)]
http: fix connection retries when there more then one request for connection

We should not attemp to establishe the connection if there is retry
timer active, since otherwise there will be a bug.

Imagine next situation:
  con = evhttp_connection_base_new()
  evhttp_connection_set_retries(con, 2)
  req = evhttp_request_new()
  evhttp_make_request(con, req, ...)
  # failed during connecting, and timer for 2 second scheduler (retry_ev)

Then another request scheduled for this evcon:
  evhttp_make_request(con, req, ...)
  # got request from server,
  # and now it tries to read the response from the server
  # (req.kind == EVHTTP_RESPONSE)
  #
  # but at this point retry_ev scheduled,
  # and it schedules the connect again,
  # and after the connect will succeeed, it will pick request with
  # EVHTTP_RESPONSE for sending and this is completelly wrong and will
  # fail in evhttp_make_header_response() since there is no
  # "http_server" for this evcon

This was a long standing issue, that I came across few years ago
firstly, bad only now I had time to dig into it (but right now it was
pretty simple, by limiting amount of CPU for the process and using rr
for debug to go back and forth).

5 years agoDo not check O_NONBLOCK for invalid fds
Azat Khuzhin [Sun, 9 Dec 2018 11:48:44 +0000 (14:48 +0300)]
Do not check O_NONBLOCK for invalid fds

Fixes: 6f988ee1 ("Merge branch 'check-O_NONBLOCK-in-debug'")
5 years agotest-fdleak: fix memory leaks
Azat Khuzhin [Sat, 8 Dec 2018 14:35:53 +0000 (17:35 +0300)]
test-fdleak: fix memory leaks

Fixes: #726
5 years agotravis-ci: do not run coveralls with clang (SIGSEGV)
Azat Khuzhin [Wed, 5 Dec 2018 08:40:40 +0000 (11:40 +0300)]
travis-ci: do not run coveralls with clang (SIGSEGV)

This will fix coveralls badge

See: https://travis-ci.org/libevent/libevent/jobs/458590276#L1425

5 years agoAdd mailmap
Azat Khuzhin [Wed, 5 Dec 2018 08:46:30 +0000 (11:46 +0300)]
Add mailmap

5 years agotest: add TT_RETRIABLE for http/cancel_by_host_no_ns
Azat Khuzhin [Thu, 22 Nov 2018 21:42:46 +0000 (00:42 +0300)]
test: add TT_RETRIABLE for http/cancel_by_host_no_ns

Could fail from time to time in travis-ci:
  https://travis-ci.org/libevent/libevent/jobs/458554097#L1702

Follow-up-for: fe5b0719 ("Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)")

5 years agoappveyor: enable parallel build
Azat Khuzhin [Wed, 21 Nov 2018 04:04:33 +0000 (07:04 +0300)]
appveyor: enable parallel build

5 years agocmake: do not build both (SHARED and STATIC) for MSVC/win32
Azat Khuzhin [Thu, 22 Nov 2018 21:29:55 +0000 (00:29 +0300)]
cmake: do not build both (SHARED and STATIC) for MSVC/win32

MSVC does not support SHARED and STATIC libraries with the same name,
so let's just build SHARED libraries by default instead (yes we can add
prefix but let's stick with this).

The reason for this is that in windows shared libraries requires .lib
file too, but this is not static library it is imported library for
shared (doh...), for more info [1] and [2].

  [1]: https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-creation
  [2]: https://blogs.msdn.microsoft.com/oldnewthing/20091013-00/?p=16403

And when we build both static library can and will override shared
library imported part, let's take a look at event_extra.lib:

- before patch [3]:
  $ less libevent-fail/lib/Debug/event_extra.lib | head
  ==> use library:contained_file to view a file in the archive
  rw-rw-rw- 100666/100666  59568 Nov 21 23:55 2018 event_extra_static.dir/Debug/evrpc.obj
  rw-rw-rw- 100666/100666 252219 Nov 21 23:55 2018 event_extra_static.dir/Debug/evdns.obj
  rw-rw-rw- 100666/100666 203850 Nov 21 23:55 2018 event_extra_static.dir/Debug/http.obj
  rw-rw-rw- 100666/100666  25907 Nov 21 23:55 2018 event_extra_static.dir/Debug/event_tagging.obj

  [3]: https://ci.appveyor.com/project/azat/libevent/builds/20472024/job/t0o93v042jai0dj7

- "after patch" [4] (not after but the same effect):
  $ less libevent-ok/lib/Debug/event_extra.lib | head
  ==> use library:contained_file to view a file in the archive
  --------- 0/0    509 Nov 21 23:38 2018 event_extra.dll
  ...

  [4]: https://ci.appveyor.com/project/azat/libevent/builds/20478998/job/ca9k3c76amc4qr76

Refs: #691

5 years agocmake: introduce EVENT__LIBRARY_TYPE option
Azat Khuzhin [Thu, 22 Nov 2018 20:00:11 +0000 (23:00 +0300)]
cmake: introduce EVENT__LIBRARY_TYPE option

Long time ago in [1] cmake build was forced to compile both libraries
(SHARED and STATIC), since this is how our autotools build works.

  [1]: 7182c2f561570cd9ceb704623ebe9ae3608c7b43 ("cmake: build SHARED and STATIC libraries (like autoconf does)")

And there is no way to configure this (and indeed you need to do this
for MSVC for example), so let's introduce option for this --
EVENT__LIBRARY_TYPE.

Plus now we have INTERFACE libraries, that we can use internally in
libevent's cmake rules to avoid strict to _shared/_static variant of the
libraries to link with samples/tests (we prefer SHARED over STATIC for
linking).

Also bump minimal cmake required version to 3.1 by the following
reasons:
- 3.1 is required for RPATH configuration under APPLE
- 3.0 is required for add_library(INTERFACE) (did not found it in 2.8.x
documentation)
- remove extra conditions
(anyway 3.1 was release 4 years ago, so I guess that most of the systems
will have it)

5 years agocmake: drop redundant add_dependencies()
Azat Khuzhin [Wed, 21 Nov 2018 04:31:57 +0000 (07:31 +0300)]
cmake: drop redundant add_dependencies()

5 years agoMerge branch 'TT_RETRIABLE'
Azat Khuzhin [Tue, 20 Nov 2018 21:10:43 +0000 (00:10 +0300)]
Merge branch 'TT_RETRIABLE'

* TT_RETRIABLE:
  Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)
  regress: introduce TT_RETRIABLE

Fixes: #704
5 years agoMark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)
Azat Khuzhin [Tue, 20 Nov 2018 08:46:44 +0000 (11:46 +0300)]
Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)

This patch mark testcases that only fail under travis-ci/appveyor with
TT_RETRIABLE, since otherwise there is too much noise, other issues
(like failures under vagrant boxes) would be investigated separatelly.

linux (from travis-ci only):
- http/cancel_by_host_no_ns
- http/cancel_by_host_inactive_server
- http/cancel_by_host_ns_timeout
- http/cancel_by_host_ns_timeout_inactive_server
- thread/conditions_simple
- util/monotonic_prc_precise
- util/usleep
- main/del_wait

vagrant/ubuntu box (this is the only exception):
- thread/no_events

win32 (from appveyor only):
- main/active_later
- main/persistent_active_timeout

And we should use TT_RETRIABLE over TT_OFF_BY_DEFAULT/TT_SKIP when it
make sense.

But there is still "test-ratelim__group_lim" left.

5 years agoregress: introduce TT_RETRIABLE
Azat Khuzhin [Mon, 19 Nov 2018 22:06:04 +0000 (01:06 +0300)]
regress: introduce TT_RETRIABLE

We have some tests that has false-positive due to real/CPU time bound,
but they are pretty generic and we do not want to skip them by default.

TT_RETRIABLE is the flag that will indicate tinytest to retry the test
in case of failure, use it to avoid next possible false-positives:
- real time-related
- CPU time-related
Since I guess it is better to see/grepping RETRYING messages over
ignoring completely failed builds.

No configuration switch for number of retries was done on purpose (only
3 retries and no more).

And this is how it looks BTW:
  $ gcc ../test/tinytest_demo.c ../test/tinytest.c
  $ ./a.out --verbose --no-fork
  demo/timeout_retry
  demo/timeout_retry:
    FAIL ../test/tinytest_demo.c:201: assert(i != 1): 1 vs 1
    [timeout_retry FAILED]

    [RETRYING timeout_retry (3)]
  demo/timeout_retry:
           OK ../test/tinytest_demo.c:201: assert(i != 1): 2 vs 1
           OK ../test/tinytest_demo.c:213: assert(t2-t1 >= 4): 5 vs 4
           OK ../test/tinytest_demo.c:215: assert(t2-t1 <= 6): 5 vs 6
  1 tests ok.  (0 skipped)

5 years agobufferevent: add debug messages when .setfd/.getfd/.enable/.disable failed
Azat Khuzhin [Mon, 19 Nov 2018 21:58:47 +0000 (00:58 +0300)]
bufferevent: add debug messages when .setfd/.getfd/.enable/.disable failed

5 years agohttp: improve error path for bufferevent_{setfd,enable,disable}()
Azat Khuzhin [Tue, 13 Nov 2018 21:20:20 +0000 (00:20 +0300)]
http: improve error path for bufferevent_{setfd,enable,disable}()

We have calls to the next functions but do not check return values,
though they can be invalid and it is better to show this somehow.

Also do bufferevent_setfd() first and only after it
bufferevent_enable()/bufferevent_disable() since:
a) it is more natural
b) it will avoid extra operations
c) it will not fail first bufferevent_enable() (this is the case for
   buffbufferevent_async at least)

In this case we could add more information for issues like #709

5 years agoregress_http: fix compilation with !EVENT__HAVE_OPENSSL
Azat Khuzhin [Tue, 20 Nov 2018 03:20:51 +0000 (06:20 +0300)]
regress_http: fix compilation with !EVENT__HAVE_OPENSSL

Fixes: 811c63f7 ("regress: test for HTTP/HTTPS with IOCP enabled")
5 years agoregress_bufferevent: add TT_IOCP_LEGACY/TT_IOCP
Azat Khuzhin [Tue, 13 Nov 2018 20:20:10 +0000 (23:20 +0300)]
regress_bufferevent: add TT_IOCP_LEGACY/TT_IOCP

5 years agot/bench_http: disable buffering (win32 do not show anything otherwise)
Azat Khuzhin [Tue, 13 Nov 2018 20:30:51 +0000 (23:30 +0300)]
t/bench_http: disable buffering (win32 do not show anything otherwise)

Refs: #255

5 years agos/http-server: check for EVTHREAD_USE_WINDOWS_THREADS_IMPLEMENTED
Azat Khuzhin [Tue, 13 Nov 2018 20:23:09 +0000 (23:23 +0300)]
s/http-server: check for EVTHREAD_USE_WINDOWS_THREADS_IMPLEMENTED

5 years agoMerge branch 'iocp-fixes'
Azat Khuzhin [Tue, 13 Nov 2018 19:47:43 +0000 (22:47 +0300)]
Merge branch 'iocp-fixes'

* iocp-fixes:
  regress: test for HTTP/HTTPS with IOCP enabled
  bev_async: trigger/run only deferred callbacks
  bev_async: do not initialize timeouts multiple times
  bev_async: set "ok" on setfd if fd>=0 (like we do during creation)
  bev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp

Closes: #709
Refs: nmathewson/Libevent#160

5 years agoregress: test for HTTP/HTTPS with IOCP enabled
Azat Khuzhin [Tue, 13 Nov 2018 08:25:35 +0000 (11:25 +0300)]
regress: test for HTTP/HTTPS with IOCP enabled

Next tests added:
- iocp/http/simple
- iocp/http/https_simple

5 years agobev_async: trigger/run only deferred callbacks
Azat Khuzhin [Tue, 13 Nov 2018 18:31:44 +0000 (21:31 +0300)]
bev_async: trigger/run only deferred callbacks

Otherwise callbacks will be runned even without event_loop, due to
nature of IOCP.

A simple example is:
  evhttp_connection_free(client)
  # freeing the client will trigger evhttp_connection_free() for the
  # client on the server side, and hence there will double free
  evhttp_free(server)

Fixes: iocp/http/simple
5 years agobev_async: do not initialize timeouts multiple times
Azat Khuzhin [Tue, 13 Nov 2018 08:47:14 +0000 (11:47 +0300)]
bev_async: do not initialize timeouts multiple times

You cannot event_assign() event multiple times, this is UB, and most
likely will fail.

Fixes: af9b2a7ae0be11c79a909d212b1833a9379e4ba0 ("Initialize async
bufferevent timeout CBs unconditionally")

5 years agobev_async: set "ok" on setfd if fd>=0 (like we do during creation)
Azat Khuzhin [Sun, 11 Nov 2018 18:51:46 +0000 (21:51 +0300)]
bev_async: set "ok" on setfd if fd>=0 (like we do during creation)

Otherwise after .setfd, .enable will not work.

5 years agobev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp
Azat Khuzhin [Sun, 11 Nov 2018 18:35:20 +0000 (21:35 +0300)]
bev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp

listener already calls event_iocp_port_associate_() the second call will
return ERROR_INVALID_PARAMETER.
Plus we already ignore it on creation, so why we should care about it
here?

5 years agoFix conceivable UAF of the bufferevent in evhttp_connection_free()
Azat Khuzhin [Tue, 13 Nov 2018 18:26:12 +0000 (21:26 +0300)]
Fix conceivable UAF of the bufferevent in evhttp_connection_free()

Although this is not a problem, since bufferevent uses finalizers and
will free itself only from the loop (well this is not a problem if you
do not play games with various event_base in different threads) it
generates questions, so rewrite it in more reliable way.

Fixes: #712
5 years agoUse BEV_UPCASE() everywhere
Azat Khuzhin [Fri, 30 Nov 2018 19:25:41 +0000 (22:25 +0300)]
Use BEV_UPCASE() everywhere

Done with coccinelle and manual line rewrap:
  $ cat > BEV_UPCAST.cocci
  @@
  expression field_;
  expression var;
  @@

  - EVUTIL_UPCAST(var, struct bufferevent_private, field_)
  + BEV_UPCAST(var)

  $ spatch --sp-file BEV_UPCASE.cocci --in-place bufferevent*.c > /dev/null

5 years agoregress: add EVENT_NO_FILE_BUFFERING, to disable buffering for stdout/stderr
Azat Khuzhin [Tue, 13 Nov 2018 06:25:13 +0000 (09:25 +0300)]
regress: add EVENT_NO_FILE_BUFFERING, to disable buffering for stdout/stderr

Useful for win32

5 years agoevent: add some debug information into loop for event_base_free_queues_()
Azat Khuzhin [Tue, 13 Nov 2018 05:24:55 +0000 (08:24 +0300)]
event: add some debug information into loop for event_base_free_queues_()

Refs: 7c8d0152dda18ecc52d3099fea235b04ddb850d9 ("Free event queues even
for recursive finalizers")

5 years agoMerge branch 'sample-http-server'
Azat Khuzhin [Tue, 13 Nov 2018 08:10:25 +0000 (11:10 +0300)]
Merge branch 'sample-http-server'

Some improvements for http-server sample:
- getopt
- persistent port via -p option
- IOCP for win32 via -I
- disable buffering
- enable debug logging via -v/EVENT_DEBUG_LOGGING_ALL
- cleanup (by signal and separate error path on errors)

* sample-http-server:
  s/http-server: graceful cleanup
  s/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset
  s/http-server: turn off buffering (otherwise do output on win32)
  s/http-server: add an option to use IOCP
  s/http-server: add options (for persistent port)

Refs: #709

5 years agos/http-server: graceful cleanup
Azat Khuzhin [Tue, 13 Nov 2018 07:48:41 +0000 (10:48 +0300)]
s/http-server: graceful cleanup

5 years agos/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset
Azat Khuzhin [Tue, 13 Nov 2018 07:26:17 +0000 (10:26 +0300)]
s/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset

5 years agos/http-server: turn off buffering (otherwise do output on win32)
Azat Khuzhin [Thu, 8 Nov 2018 08:19:16 +0000 (11:19 +0300)]
s/http-server: turn off buffering (otherwise do output on win32)

5 years agos/http-server: add an option to use IOCP
Azat Khuzhin [Thu, 8 Nov 2018 08:13:21 +0000 (11:13 +0300)]
s/http-server: add an option to use IOCP

5 years agos/http-server: add options (for persistent port)
Azat Khuzhin [Tue, 13 Nov 2018 07:33:24 +0000 (10:33 +0300)]
s/http-server: add options (for persistent port)

5 years agoRemove Vagrantfile (will be moved into libevent-extras)
Azat Khuzhin [Wed, 7 Nov 2018 21:21:08 +0000 (00:21 +0300)]
Remove Vagrantfile (will be moved into libevent-extras)

Since:
- it is not a library
- this file should have (if I had enough time) enough fixes in itself
  and should not polute libevent history
- it "requires" (it more cleaner to use it in this way) script --
  tools/vagrant-tests.py (indeed, from libevent-extras)
- will has it's own issues/README/...

https://github.com/libevent/libevent-extras

5 years agoregress_ssl: fix ssl/bufferevent_wm_filter for non defered callbacks
Azat Khuzhin [Wed, 7 Nov 2018 21:36:07 +0000 (00:36 +0300)]
regress_ssl: fix ssl/bufferevent_wm_filter for non defered callbacks

Even after referenced patch there is still possible recursive callbacks
from evbuffer_drain(bev_input), i.e.:
  wm_transfer() -> evbuffer_drain() -> wm_transfer()
                   inc(ctx->get)

But if we will increment ctx->get before drain that we will not add more
data to buffer.

Refs: 54c6fe3c ("regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance")
CI: https://ci.appveyor.com/project/nmathewson/libevent/build/job/f0rv299i71wnuxdq#L2546

5 years agoappveyor: skip travis patches (by commit subject/files)
Azat Khuzhin [Tue, 6 Nov 2018 08:30:18 +0000 (11:30 +0300)]
appveyor: skip travis patches (by commit subject/files)

5 years agoappveyor: cache build directory to reduce overall time (6x time faster)
Azat Khuzhin [Mon, 5 Nov 2018 14:04:47 +0000 (17:04 +0300)]
appveyor: cache build directory to reduce overall time (6x time faster)

various build checks (i.e. detecting headers/macroses/functions) takes
7 minutes (from 13 minutes in total) for cmake, which is too high.

By using cache we can reduce this to ~0.

And set APPVEYOR_SAVE_CACHE_ON_ERROR so that cmake checks will be
cached (anyway all sources will be built from scratch due to timestamp
updates while extracting from sources).

5 years agoappveyor: disable verbosity
Azat Khuzhin [Mon, 5 Nov 2018 22:22:13 +0000 (01:22 +0300)]
appveyor: disable verbosity

5 years agoappveyor: do not run on branches that has "travis" in it's name
Azat Khuzhin [Mon, 5 Nov 2018 15:00:41 +0000 (18:00 +0300)]
appveyor: do not run on branches that has "travis" in it's name

5 years agocmake: set CMP0075 to NEW (for ws2_32.lib in win32)
Azat Khuzhin [Mon, 5 Nov 2018 15:30:38 +0000 (18:30 +0300)]
cmake: set CMP0075 to NEW (for ws2_32.lib in win32)

Otherwise cmake complains:
  Policy CMP0075 is not set: Include file check macros honor
  CMAKE_REQUIRED_LIBRARIES.  Run "cmake --help-policy CMP0075" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  CMAKE_REQUIRED_LIBRARIES is set to:
    ws2_32.lib

  For compatibility with CMake 3.11 and below this check is ignoring it.

5 years agocmake: set CMP0074 to NEW (for OPENSSL_ROOT in appveyor)
Azat Khuzhin [Mon, 5 Nov 2018 15:23:31 +0000 (18:23 +0300)]
cmake: set CMP0074 to NEW (for OPENSSL_ROOT in appveyor)

We have $env:OPENSSL_ROOT (env) equals to -DOPENSSL_ROOT (cmake
variable) anyway.

cmake complains:
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  Environment variable OpenSSL_ROOT is set to:
    C:/OpenSSL-Win64/bin

  For compatibility, CMake is ignoring the variable.
This warning is for project developers.  Use -Wno-dev to suppress it.

5 years agotravis: use homebrew via addon (-~4min/16% from the build time)
Azat Khuzhin [Mon, 5 Nov 2018 19:58:25 +0000 (22:58 +0300)]
travis: use homebrew via addon (-~4min/16% from the build time)

5 years agoregress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance
Azat Khuzhin [Mon, 5 Nov 2018 19:25:15 +0000 (22:25 +0300)]
regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance

Due to inplace callbacks (i.e. no BEV_OPT_DEFER_CALLBACKS) we cannot be
sure that wm_transfer() will not be called recursively and indeed it
still happens sometimes, and the referenced patch increase amount of
this times, especially for linux/poll.

Fixes: 66304a23cf748714159c988e78f35401c5352827 ("Fix
ssl/bufferevent_wm_filter when bev does not reach watermark on break")

5 years agoregress_http: disable http/read_on_write_error under win32
Azat Khuzhin [Mon, 5 Nov 2018 18:33:54 +0000 (21:33 +0300)]
regress_http: disable http/read_on_write_error under win32

EVHTTP_CON_READ_ON_WRITE_ERROR works only if an error already read from
the socket, but if we already got EPIPE on write we cannot read from the
socket anymore, and win32 does not guarantee that read will happens
before (although it happens from time to time).

In the referenced patch I just replaced callback with not expecting 417,
but like I already wrote, this is not always true (i.e. it is flacky).

Fixes: 3b581693ac1967f7f8d98491cb772a1b415eb4cd ("test/http:
read_on_write_error: fix it for win32")

5 years agotravis-ci: exclude appveyor branches
Azat Khuzhin [Mon, 5 Nov 2018 15:02:05 +0000 (18:02 +0300)]
travis-ci: exclude appveyor branches

5 years agocmake: do not detect _GNU_SOURCE/__GNU_LIBRARY__ if it is cached
Azat Khuzhin [Sun, 4 Nov 2018 16:48:11 +0000 (19:48 +0300)]
cmake: do not detect _GNU_SOURCE/__GNU_LIBRARY__ if it is cached

5 years agoMerge branch 'ssl_bufferevent_wm_filter-fix'
Azat Khuzhin [Sun, 4 Nov 2018 18:41:20 +0000 (21:41 +0300)]
Merge branch 'ssl_bufferevent_wm_filter-fix'

* ssl_bufferevent_wm_filter-fix:
  Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break
  regress_ssl: cover watermarks with deferred callbacks
  regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging

5 years agoFix ssl/bufferevent_wm_filter when bev does not reach watermark on break
Azat Khuzhin [Sun, 4 Nov 2018 17:40:04 +0000 (20:40 +0300)]
Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break

For the ssl/bufferevent_wm* we have next configuration:
- payload_len = 1024
- wm_high     = 5120
- limit       = 40960
- to_read     = 512

In this test we expect that with high watermark installed to "wm_high"
we will read "limit" bytes by reading "to_read" at a time, but adding
"payload_len" at a time (this "to_read"/"payload_len" limits is
installed to finally overflow watermark).

Once we read "limit" bytes we break, by disable EV_READ and reset
callbacks. Although this will not work if when we want to break we do
not reach watermark, this is because watermarks installs evbuffer
callback for the input buffer and if the watermark does not reached it
will enable EV_READ while be_openssl_enable() will read from the
underlying buffer (in case the openssl bufferevent created via
bufferevent_openssl_filter_new()) and call callback again (until it will
reach watermark or read al from the underlying buffer -- this is why it
stops in our caes).

And this is exactly what happened in win32, you can see this in the
following logs:

- win32 before:
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 40960
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 41472
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 41984
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 42496
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break

- win32 after:
    OK C:\vagrant\test\regress_ssl.c:821: wm_transfer-client(00FC26F0): break
    OK C:\vagrant\test\regress_ssl.c:836: wm_transfer-client(00FC26F0): in: 4800, out: 0, got: 40960

- linux before:
    OK ../test/regress_ssl.c:829: wm_transfer-client(0x55555566f5e0): in: 5120, out: 0, got: 40960
    OK ../test/regress_ssl.c:834: wm_transfer-client(0x55555566f5e0): break

- linux after:
    OK ../test/regress_ssl.c:821: wm_transfer-client(0x55555566f5e0): break
    OK ../test/regress_ssl.c:836: wm_transfer-client(0x55555566f5e0): in: 5120, out: 0, got: 40960

(As you can see in linux case we already reach watermark hence it passed
before).

So fix the issue by breaking before draining.

But during fixing this I was thinking is this right? I.e. reading from
the be_openssl_enable(), maybe we should force deferred callbacks at
least?

5 years agoregress_ssl: cover watermarks with deferred callbacks
Azat Khuzhin [Sun, 4 Nov 2018 17:15:14 +0000 (20:15 +0300)]
regress_ssl: cover watermarks with deferred callbacks

5 years agoregress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging
Azat Khuzhin [Sun, 4 Nov 2018 17:03:50 +0000 (20:03 +0300)]
regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging

- add bev pointer
- use EV_SIZE_FMT over %zu (win32)

5 years agoregress_http: make https_basic non time dependent
Azat Khuzhin [Fri, 2 Nov 2018 20:43:57 +0000 (23:43 +0300)]
regress_http: make https_basic non time dependent

Fixes: #454
5 years agoMerge branch 'check-O_NONBLOCK-in-debug'
Azat Khuzhin [Sat, 3 Nov 2018 22:21:48 +0000 (01:21 +0300)]
Merge branch 'check-O_NONBLOCK-in-debug'

* check-O_NONBLOCK-in-debug:
  regress: use non blocking descriptors whenever it is possible
  assert that fds are nonblocking in debug mode

Closes: nmathewson/Libevent#90