]> granicus.if.org Git - libevent/log
libevent
5 years agoMerge branch 'win64-fixes'
Azat Khuzhin [Tue, 29 Jan 2019 07:54:39 +0000 (10:54 +0300)]
Merge branch 'win64-fixes'

* win64-fixes:
  test/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)
  test/et/et: verify return codes
  appveyor: switch to new VS/MinGW and x64

5 years agotest/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)
Azat Khuzhin [Tue, 29 Jan 2019 07:42:50 +0000 (10:42 +0300)]
test/et/et: fix it by using appropriate type for the SOCKET (evutil_socket_t)

Fixes: #750
5 years agotest/et/et: verify return codes
Azat Khuzhin [Mon, 28 Jan 2019 22:44:33 +0000 (01:44 +0300)]
test/et/et: verify return codes

5 years agoappveyor: switch to new VS/MinGW and x64
Azat Khuzhin [Sat, 24 Nov 2018 17:22:40 +0000 (20:22 +0300)]
appveyor: switch to new VS/MinGW and x64

The cache had been reseted with the following REST API requests:
  DELETE https://ci.appveyor.com/api/projects/nmathewson/libevent/buildCache
  DELETE https://ci.appveyor.com/api/projects/libevent/libevent/buildCache
  DELETE https://ci.appveyor.com/api/projects/azat/libevent/buildCache

5 years agowin32select: print an error in case of failure
Azat Khuzhin [Tue, 29 Jan 2019 04:56:21 +0000 (07:56 +0300)]
win32select: print an error in case of failure

5 years agoAdd OS-specific exclude patterns of the branch names for the CI
Azat Khuzhin [Tue, 29 Jan 2019 07:10:23 +0000 (10:10 +0300)]
Add OS-specific exclude patterns of the branch names for the CI

5 years agos/http-server: fix cleanup routines
Azat Khuzhin [Mon, 28 Jan 2019 22:23:02 +0000 (01:23 +0300)]
s/http-server: fix cleanup routines

Fixes: bdd71f18 ("s/http-server: graceful cleanup")
5 years agoMerge branch 'http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2'
Azat Khuzhin [Mon, 28 Jan 2019 22:18:39 +0000 (01:18 +0300)]
Merge branch 'http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2'

* http-EVHTTP_CON_READ_ON_WRITE_ERROR-fixes-v2:
  http: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR
  test: add logging for http/read_on_write_error and rearrange code
  http: do not call deferred readcb if readcb is not set

Refs: #749

5 years agohttp: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR
Azat Khuzhin [Mon, 28 Jan 2019 22:09:44 +0000 (01:09 +0300)]
http: try to read existing data in buffer under EVHTTP_CON_READ_ON_WRITE_ERROR

There are two possible ways of getting response from the server:
- processing existing bufferevent buffer
- reading from the socket (even after write() errored with -1, it is
still possible)

But we did not tried the first option, only the second one.

Fixes: http/read_on_write_error (on freebsd/osx)
5 years agotest: add logging for http/read_on_write_error and rearrange code
Azat Khuzhin [Sun, 27 Jan 2019 12:28:28 +0000 (15:28 +0300)]
test: add logging for http/read_on_write_error and rearrange code

5 years agohttp: do not call deferred readcb if readcb is not set
Azat Khuzhin [Sun, 27 Jan 2019 12:16:39 +0000 (15:16 +0300)]
http: do not call deferred readcb if readcb is not set

Otherwise evhttp_read_cb can be called with invalid connection state:
    http/read_on_write_error: [forking] [msg] libevent using: kqueue
      FAIL ../test/regress_http.c:4079: assert(req)
      FAIL ../test/regress_http.c:4087: assert(req)[err] evhttp_read_cb: illegal connection state 0

5 years agoMerge branch 'travis-ci-osx-fixes'
Azat Khuzhin [Mon, 28 Jan 2019 19:19:20 +0000 (22:19 +0300)]
Merge branch 'travis-ci-osx-fixes'

* travis-ci-osx-fixes:
  travis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8
  travis-ci/osx: install gcc and fix CC

5 years agotravis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8
Azat Khuzhin [Mon, 28 Jan 2019 18:22:27 +0000 (21:22 +0300)]
travis-ci/osx: switch to xcode 10.1, since 9.4 is not compatible with gcc-8

Refs: travis-ci/travis-ci#9640

5 years agotravis-ci/osx: install gcc and fix CC
Azat Khuzhin [Mon, 28 Jan 2019 17:30:57 +0000 (20:30 +0300)]
travis-ci/osx: install gcc and fix CC

Since there is no gcc in osx_image [1]:
    ls: /usr/local/bin/gcc-?.?: No such file or directory

  [1]: https://travis-ci.org/libevent/libevent/jobs/484794192#L95

And use ls(1) over echo(1) to show an error if there is no such file.

5 years agotest: adjust expecting error for getaddrinfo() under EMFILE
Azat Khuzhin [Sat, 26 Jan 2019 15:52:33 +0000 (18:52 +0300)]
test: adjust expecting error for getaddrinfo() under EMFILE

When getaddrinfo() cannot allocate file descriptor glibc/musl-libc on
linux report EAI_SYSTEM error. But this is not true for freebsd libc [1]
(and hence apple libc [2]), they report EAI_NONAME error instead, so
adjust expectation.

  [1]: https://github.com/freebsd/freebsd/blob/master/lib/libc/net/getaddrinfo.c
  [2]: https://opensource.apple.com/source/Libc/

Refs: #749
Refs: https://github.com/libevent/libevent/issues/749#issuecomment-457838159

5 years agotest/nonpersist_readd: use assert helpers
Azat Khuzhin [Sat, 12 Jan 2019 10:58:50 +0000 (13:58 +0300)]
test/nonpersist_readd: use assert helpers

To debug failure under win32 in appveyor:
  https://ci.appveyor.com/project/nmathewson/libevent/builds/21559140/job/dn16qdo1j6sr497t#L1620

5 years agorpc: use *_new_with_arg() to match function prototype
Azat Khuzhin [Fri, 11 Jan 2019 18:52:11 +0000 (21:52 +0300)]
rpc: use *_new_with_arg() to match function prototype

In 755fbf16c ("Add void* arguments to request_new and reply_new
evrpc hooks") this new functions had been introduced, but newer used,
what for? So let's use them.

5 years agohttp: Preserve socket error from listen across closesocket cleanup
Luke Dashjr [Fri, 28 Dec 2018 04:42:20 +0000 (04:42 +0000)]
http: Preserve socket error from listen across closesocket cleanup

Closes: #738 (cherry-picked)
6 years agocmake: support static runtime (MSVC)
Azat Khuzhin [Mon, 17 Dec 2018 18:31:54 +0000 (21:31 +0300)]
cmake: support static runtime (MSVC)

Fixes: #737
6 years agoappveyor: cover static libs for MSVC
Azat Khuzhin [Mon, 17 Dec 2018 18:41:32 +0000 (21:41 +0300)]
appveyor: cover static libs for MSVC

6 years agofix a comment error
baixiangcpp [Sat, 15 Dec 2018 13:45:36 +0000 (21:45 +0800)]
fix a comment error

Closes: #736 (cherry-picked)
6 years agocmake: Fix some typos in option descriptions
Fredrik Strupe [Sat, 15 Dec 2018 11:54:31 +0000 (12:54 +0100)]
cmake: Fix some typos in option descriptions

6 years agohttp: fix connection retries when there more then one request for connection
Azat Khuzhin [Fri, 7 Dec 2018 18:46:27 +0000 (21:46 +0300)]
http: fix connection retries when there more then one request for connection

We should not attemp to establishe the connection if there is retry
timer active, since otherwise there will be a bug.

Imagine next situation:
  con = evhttp_connection_base_new()
  evhttp_connection_set_retries(con, 2)
  req = evhttp_request_new()
  evhttp_make_request(con, req, ...)
  # failed during connecting, and timer for 2 second scheduler (retry_ev)

Then another request scheduled for this evcon:
  evhttp_make_request(con, req, ...)
  # got request from server,
  # and now it tries to read the response from the server
  # (req.kind == EVHTTP_RESPONSE)
  #
  # but at this point retry_ev scheduled,
  # and it schedules the connect again,
  # and after the connect will succeeed, it will pick request with
  # EVHTTP_RESPONSE for sending and this is completelly wrong and will
  # fail in evhttp_make_header_response() since there is no
  # "http_server" for this evcon

This was a long standing issue, that I came across few years ago
firstly, bad only now I had time to dig into it (but right now it was
pretty simple, by limiting amount of CPU for the process and using rr
for debug to go back and forth).

6 years agoDo not check O_NONBLOCK for invalid fds
Azat Khuzhin [Sun, 9 Dec 2018 11:48:44 +0000 (14:48 +0300)]
Do not check O_NONBLOCK for invalid fds

Fixes: 6f988ee1 ("Merge branch 'check-O_NONBLOCK-in-debug'")
6 years agotest-fdleak: fix memory leaks
Azat Khuzhin [Sat, 8 Dec 2018 14:35:53 +0000 (17:35 +0300)]
test-fdleak: fix memory leaks

Fixes: #726
6 years agotravis-ci: do not run coveralls with clang (SIGSEGV)
Azat Khuzhin [Wed, 5 Dec 2018 08:40:40 +0000 (11:40 +0300)]
travis-ci: do not run coveralls with clang (SIGSEGV)

This will fix coveralls badge

See: https://travis-ci.org/libevent/libevent/jobs/458590276#L1425

6 years agoAdd mailmap
Azat Khuzhin [Wed, 5 Dec 2018 08:46:30 +0000 (11:46 +0300)]
Add mailmap

6 years agotest: add TT_RETRIABLE for http/cancel_by_host_no_ns
Azat Khuzhin [Thu, 22 Nov 2018 21:42:46 +0000 (00:42 +0300)]
test: add TT_RETRIABLE for http/cancel_by_host_no_ns

Could fail from time to time in travis-ci:
  https://travis-ci.org/libevent/libevent/jobs/458554097#L1702

Follow-up-for: fe5b0719 ("Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)")

6 years agoappveyor: enable parallel build
Azat Khuzhin [Wed, 21 Nov 2018 04:04:33 +0000 (07:04 +0300)]
appveyor: enable parallel build

6 years agocmake: do not build both (SHARED and STATIC) for MSVC/win32
Azat Khuzhin [Thu, 22 Nov 2018 21:29:55 +0000 (00:29 +0300)]
cmake: do not build both (SHARED and STATIC) for MSVC/win32

MSVC does not support SHARED and STATIC libraries with the same name,
so let's just build SHARED libraries by default instead (yes we can add
prefix but let's stick with this).

The reason for this is that in windows shared libraries requires .lib
file too, but this is not static library it is imported library for
shared (doh...), for more info [1] and [2].

  [1]: https://docs.microsoft.com/en-us/windows/desktop/dlls/dynamic-link-library-creation
  [2]: https://blogs.msdn.microsoft.com/oldnewthing/20091013-00/?p=16403

And when we build both static library can and will override shared
library imported part, let's take a look at event_extra.lib:

- before patch [3]:
  $ less libevent-fail/lib/Debug/event_extra.lib | head
  ==> use library:contained_file to view a file in the archive
  rw-rw-rw- 100666/100666  59568 Nov 21 23:55 2018 event_extra_static.dir/Debug/evrpc.obj
  rw-rw-rw- 100666/100666 252219 Nov 21 23:55 2018 event_extra_static.dir/Debug/evdns.obj
  rw-rw-rw- 100666/100666 203850 Nov 21 23:55 2018 event_extra_static.dir/Debug/http.obj
  rw-rw-rw- 100666/100666  25907 Nov 21 23:55 2018 event_extra_static.dir/Debug/event_tagging.obj

  [3]: https://ci.appveyor.com/project/azat/libevent/builds/20472024/job/t0o93v042jai0dj7

- "after patch" [4] (not after but the same effect):
  $ less libevent-ok/lib/Debug/event_extra.lib | head
  ==> use library:contained_file to view a file in the archive
  --------- 0/0    509 Nov 21 23:38 2018 event_extra.dll
  ...

  [4]: https://ci.appveyor.com/project/azat/libevent/builds/20478998/job/ca9k3c76amc4qr76

Refs: #691

6 years agocmake: introduce EVENT__LIBRARY_TYPE option
Azat Khuzhin [Thu, 22 Nov 2018 20:00:11 +0000 (23:00 +0300)]
cmake: introduce EVENT__LIBRARY_TYPE option

Long time ago in [1] cmake build was forced to compile both libraries
(SHARED and STATIC), since this is how our autotools build works.

  [1]: 7182c2f561570cd9ceb704623ebe9ae3608c7b43 ("cmake: build SHARED and STATIC libraries (like autoconf does)")

And there is no way to configure this (and indeed you need to do this
for MSVC for example), so let's introduce option for this --
EVENT__LIBRARY_TYPE.

Plus now we have INTERFACE libraries, that we can use internally in
libevent's cmake rules to avoid strict to _shared/_static variant of the
libraries to link with samples/tests (we prefer SHARED over STATIC for
linking).

Also bump minimal cmake required version to 3.1 by the following
reasons:
- 3.1 is required for RPATH configuration under APPLE
- 3.0 is required for add_library(INTERFACE) (did not found it in 2.8.x
documentation)
- remove extra conditions
(anyway 3.1 was release 4 years ago, so I guess that most of the systems
will have it)

6 years agocmake: drop redundant add_dependencies()
Azat Khuzhin [Wed, 21 Nov 2018 04:31:57 +0000 (07:31 +0300)]
cmake: drop redundant add_dependencies()

6 years agoMerge branch 'TT_RETRIABLE'
Azat Khuzhin [Tue, 20 Nov 2018 21:10:43 +0000 (00:10 +0300)]
Merge branch 'TT_RETRIABLE'

* TT_RETRIABLE:
  Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)
  regress: introduce TT_RETRIABLE

Fixes: #704
6 years agoMark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)
Azat Khuzhin [Tue, 20 Nov 2018 08:46:44 +0000 (11:46 +0300)]
Mark a lot of flacky tests with TT_RETRIABLE (for linux/win32 only)

This patch mark testcases that only fail under travis-ci/appveyor with
TT_RETRIABLE, since otherwise there is too much noise, other issues
(like failures under vagrant boxes) would be investigated separatelly.

linux (from travis-ci only):
- http/cancel_by_host_no_ns
- http/cancel_by_host_inactive_server
- http/cancel_by_host_ns_timeout
- http/cancel_by_host_ns_timeout_inactive_server
- thread/conditions_simple
- util/monotonic_prc_precise
- util/usleep
- main/del_wait

vagrant/ubuntu box (this is the only exception):
- thread/no_events

win32 (from appveyor only):
- main/active_later
- main/persistent_active_timeout

And we should use TT_RETRIABLE over TT_OFF_BY_DEFAULT/TT_SKIP when it
make sense.

But there is still "test-ratelim__group_lim" left.

6 years agoregress: introduce TT_RETRIABLE
Azat Khuzhin [Mon, 19 Nov 2018 22:06:04 +0000 (01:06 +0300)]
regress: introduce TT_RETRIABLE

We have some tests that has false-positive due to real/CPU time bound,
but they are pretty generic and we do not want to skip them by default.

TT_RETRIABLE is the flag that will indicate tinytest to retry the test
in case of failure, use it to avoid next possible false-positives:
- real time-related
- CPU time-related
Since I guess it is better to see/grepping RETRYING messages over
ignoring completely failed builds.

No configuration switch for number of retries was done on purpose (only
3 retries and no more).

And this is how it looks BTW:
  $ gcc ../test/tinytest_demo.c ../test/tinytest.c
  $ ./a.out --verbose --no-fork
  demo/timeout_retry
  demo/timeout_retry:
    FAIL ../test/tinytest_demo.c:201: assert(i != 1): 1 vs 1
    [timeout_retry FAILED]

    [RETRYING timeout_retry (3)]
  demo/timeout_retry:
           OK ../test/tinytest_demo.c:201: assert(i != 1): 2 vs 1
           OK ../test/tinytest_demo.c:213: assert(t2-t1 >= 4): 5 vs 4
           OK ../test/tinytest_demo.c:215: assert(t2-t1 <= 6): 5 vs 6
  1 tests ok.  (0 skipped)

6 years agobufferevent: add debug messages when .setfd/.getfd/.enable/.disable failed
Azat Khuzhin [Mon, 19 Nov 2018 21:58:47 +0000 (00:58 +0300)]
bufferevent: add debug messages when .setfd/.getfd/.enable/.disable failed

6 years agohttp: improve error path for bufferevent_{setfd,enable,disable}()
Azat Khuzhin [Tue, 13 Nov 2018 21:20:20 +0000 (00:20 +0300)]
http: improve error path for bufferevent_{setfd,enable,disable}()

We have calls to the next functions but do not check return values,
though they can be invalid and it is better to show this somehow.

Also do bufferevent_setfd() first and only after it
bufferevent_enable()/bufferevent_disable() since:
a) it is more natural
b) it will avoid extra operations
c) it will not fail first bufferevent_enable() (this is the case for
   buffbufferevent_async at least)

In this case we could add more information for issues like #709

6 years agoregress_http: fix compilation with !EVENT__HAVE_OPENSSL
Azat Khuzhin [Tue, 20 Nov 2018 03:20:51 +0000 (06:20 +0300)]
regress_http: fix compilation with !EVENT__HAVE_OPENSSL

Fixes: 811c63f7 ("regress: test for HTTP/HTTPS with IOCP enabled")
6 years agoregress_bufferevent: add TT_IOCP_LEGACY/TT_IOCP
Azat Khuzhin [Tue, 13 Nov 2018 20:20:10 +0000 (23:20 +0300)]
regress_bufferevent: add TT_IOCP_LEGACY/TT_IOCP

6 years agot/bench_http: disable buffering (win32 do not show anything otherwise)
Azat Khuzhin [Tue, 13 Nov 2018 20:30:51 +0000 (23:30 +0300)]
t/bench_http: disable buffering (win32 do not show anything otherwise)

Refs: #255

6 years agos/http-server: check for EVTHREAD_USE_WINDOWS_THREADS_IMPLEMENTED
Azat Khuzhin [Tue, 13 Nov 2018 20:23:09 +0000 (23:23 +0300)]
s/http-server: check for EVTHREAD_USE_WINDOWS_THREADS_IMPLEMENTED

6 years agoMerge branch 'iocp-fixes'
Azat Khuzhin [Tue, 13 Nov 2018 19:47:43 +0000 (22:47 +0300)]
Merge branch 'iocp-fixes'

* iocp-fixes:
  regress: test for HTTP/HTTPS with IOCP enabled
  bev_async: trigger/run only deferred callbacks
  bev_async: do not initialize timeouts multiple times
  bev_async: set "ok" on setfd if fd>=0 (like we do during creation)
  bev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp

Closes: #709
Refs: nmathewson/Libevent#160

6 years agoregress: test for HTTP/HTTPS with IOCP enabled
Azat Khuzhin [Tue, 13 Nov 2018 08:25:35 +0000 (11:25 +0300)]
regress: test for HTTP/HTTPS with IOCP enabled

Next tests added:
- iocp/http/simple
- iocp/http/https_simple

6 years agobev_async: trigger/run only deferred callbacks
Azat Khuzhin [Tue, 13 Nov 2018 18:31:44 +0000 (21:31 +0300)]
bev_async: trigger/run only deferred callbacks

Otherwise callbacks will be runned even without event_loop, due to
nature of IOCP.

A simple example is:
  evhttp_connection_free(client)
  # freeing the client will trigger evhttp_connection_free() for the
  # client on the server side, and hence there will double free
  evhttp_free(server)

Fixes: iocp/http/simple
6 years agobev_async: do not initialize timeouts multiple times
Azat Khuzhin [Tue, 13 Nov 2018 08:47:14 +0000 (11:47 +0300)]
bev_async: do not initialize timeouts multiple times

You cannot event_assign() event multiple times, this is UB, and most
likely will fail.

Fixes: af9b2a7ae0be11c79a909d212b1833a9379e4ba0 ("Initialize async
bufferevent timeout CBs unconditionally")

6 years agobev_async: set "ok" on setfd if fd>=0 (like we do during creation)
Azat Khuzhin [Sun, 11 Nov 2018 18:51:46 +0000 (21:51 +0300)]
bev_async: set "ok" on setfd if fd>=0 (like we do during creation)

Otherwise after .setfd, .enable will not work.

6 years agobev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp
Azat Khuzhin [Sun, 11 Nov 2018 18:35:20 +0000 (21:35 +0300)]
bev_async: ignore ERROR_INVALID_PARAMETER on .setfd for iocp

listener already calls event_iocp_port_associate_() the second call will
return ERROR_INVALID_PARAMETER.
Plus we already ignore it on creation, so why we should care about it
here?

6 years agoFix conceivable UAF of the bufferevent in evhttp_connection_free()
Azat Khuzhin [Tue, 13 Nov 2018 18:26:12 +0000 (21:26 +0300)]
Fix conceivable UAF of the bufferevent in evhttp_connection_free()

Although this is not a problem, since bufferevent uses finalizers and
will free itself only from the loop (well this is not a problem if you
do not play games with various event_base in different threads) it
generates questions, so rewrite it in more reliable way.

Fixes: #712
6 years agoUse BEV_UPCASE() everywhere
Azat Khuzhin [Fri, 30 Nov 2018 19:25:41 +0000 (22:25 +0300)]
Use BEV_UPCASE() everywhere

Done with coccinelle and manual line rewrap:
  $ cat > BEV_UPCAST.cocci
  @@
  expression field_;
  expression var;
  @@

  - EVUTIL_UPCAST(var, struct bufferevent_private, field_)
  + BEV_UPCAST(var)

  $ spatch --sp-file BEV_UPCASE.cocci --in-place bufferevent*.c > /dev/null

6 years agoregress: add EVENT_NO_FILE_BUFFERING, to disable buffering for stdout/stderr
Azat Khuzhin [Tue, 13 Nov 2018 06:25:13 +0000 (09:25 +0300)]
regress: add EVENT_NO_FILE_BUFFERING, to disable buffering for stdout/stderr

Useful for win32

6 years agoevent: add some debug information into loop for event_base_free_queues_()
Azat Khuzhin [Tue, 13 Nov 2018 05:24:55 +0000 (08:24 +0300)]
event: add some debug information into loop for event_base_free_queues_()

Refs: 7c8d0152dda18ecc52d3099fea235b04ddb850d9 ("Free event queues even
for recursive finalizers")

6 years agoMerge branch 'sample-http-server'
Azat Khuzhin [Tue, 13 Nov 2018 08:10:25 +0000 (11:10 +0300)]
Merge branch 'sample-http-server'

Some improvements for http-server sample:
- getopt
- persistent port via -p option
- IOCP for win32 via -I
- disable buffering
- enable debug logging via -v/EVENT_DEBUG_LOGGING_ALL
- cleanup (by signal and separate error path on errors)

* sample-http-server:
  s/http-server: graceful cleanup
  s/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset
  s/http-server: turn off buffering (otherwise do output on win32)
  s/http-server: add an option to use IOCP
  s/http-server: add options (for persistent port)

Refs: #709

6 years agos/http-server: graceful cleanup
Azat Khuzhin [Tue, 13 Nov 2018 07:48:41 +0000 (10:48 +0300)]
s/http-server: graceful cleanup

6 years agos/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset
Azat Khuzhin [Tue, 13 Nov 2018 07:26:17 +0000 (10:26 +0300)]
s/http-server: enable debug logging if EVENT_DEBUG_LOGGING_ALL env isset

6 years agos/http-server: turn off buffering (otherwise do output on win32)
Azat Khuzhin [Thu, 8 Nov 2018 08:19:16 +0000 (11:19 +0300)]
s/http-server: turn off buffering (otherwise do output on win32)

6 years agos/http-server: add an option to use IOCP
Azat Khuzhin [Thu, 8 Nov 2018 08:13:21 +0000 (11:13 +0300)]
s/http-server: add an option to use IOCP

6 years agos/http-server: add options (for persistent port)
Azat Khuzhin [Tue, 13 Nov 2018 07:33:24 +0000 (10:33 +0300)]
s/http-server: add options (for persistent port)

6 years agoRemove Vagrantfile (will be moved into libevent-extras)
Azat Khuzhin [Wed, 7 Nov 2018 21:21:08 +0000 (00:21 +0300)]
Remove Vagrantfile (will be moved into libevent-extras)

Since:
- it is not a library
- this file should have (if I had enough time) enough fixes in itself
  and should not polute libevent history
- it "requires" (it more cleaner to use it in this way) script --
  tools/vagrant-tests.py (indeed, from libevent-extras)
- will has it's own issues/README/...

https://github.com/libevent/libevent-extras

6 years agoregress_ssl: fix ssl/bufferevent_wm_filter for non defered callbacks
Azat Khuzhin [Wed, 7 Nov 2018 21:36:07 +0000 (00:36 +0300)]
regress_ssl: fix ssl/bufferevent_wm_filter for non defered callbacks

Even after referenced patch there is still possible recursive callbacks
from evbuffer_drain(bev_input), i.e.:
  wm_transfer() -> evbuffer_drain() -> wm_transfer()
                   inc(ctx->get)

But if we will increment ctx->get before drain that we will not add more
data to buffer.

Refs: 54c6fe3c ("regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance")
CI: https://ci.appveyor.com/project/nmathewson/libevent/build/job/f0rv299i71wnuxdq#L2546

6 years agoappveyor: skip travis patches (by commit subject/files)
Azat Khuzhin [Tue, 6 Nov 2018 08:30:18 +0000 (11:30 +0300)]
appveyor: skip travis patches (by commit subject/files)

6 years agoappveyor: cache build directory to reduce overall time (6x time faster)
Azat Khuzhin [Mon, 5 Nov 2018 14:04:47 +0000 (17:04 +0300)]
appveyor: cache build directory to reduce overall time (6x time faster)

various build checks (i.e. detecting headers/macroses/functions) takes
7 minutes (from 13 minutes in total) for cmake, which is too high.

By using cache we can reduce this to ~0.

And set APPVEYOR_SAVE_CACHE_ON_ERROR so that cmake checks will be
cached (anyway all sources will be built from scratch due to timestamp
updates while extracting from sources).

6 years agoappveyor: disable verbosity
Azat Khuzhin [Mon, 5 Nov 2018 22:22:13 +0000 (01:22 +0300)]
appveyor: disable verbosity

6 years agoappveyor: do not run on branches that has "travis" in it's name
Azat Khuzhin [Mon, 5 Nov 2018 15:00:41 +0000 (18:00 +0300)]
appveyor: do not run on branches that has "travis" in it's name

6 years agocmake: set CMP0075 to NEW (for ws2_32.lib in win32)
Azat Khuzhin [Mon, 5 Nov 2018 15:30:38 +0000 (18:30 +0300)]
cmake: set CMP0075 to NEW (for ws2_32.lib in win32)

Otherwise cmake complains:
  Policy CMP0075 is not set: Include file check macros honor
  CMAKE_REQUIRED_LIBRARIES.  Run "cmake --help-policy CMP0075" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  CMAKE_REQUIRED_LIBRARIES is set to:
    ws2_32.lib

  For compatibility with CMake 3.11 and below this check is ignoring it.

6 years agocmake: set CMP0074 to NEW (for OPENSSL_ROOT in appveyor)
Azat Khuzhin [Mon, 5 Nov 2018 15:23:31 +0000 (18:23 +0300)]
cmake: set CMP0074 to NEW (for OPENSSL_ROOT in appveyor)

We have $env:OPENSSL_ROOT (env) equals to -DOPENSSL_ROOT (cmake
variable) anyway.

cmake complains:
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  Environment variable OpenSSL_ROOT is set to:
    C:/OpenSSL-Win64/bin

  For compatibility, CMake is ignoring the variable.
This warning is for project developers.  Use -Wno-dev to suppress it.

6 years agotravis: use homebrew via addon (-~4min/16% from the build time)
Azat Khuzhin [Mon, 5 Nov 2018 19:58:25 +0000 (22:58 +0300)]
travis: use homebrew via addon (-~4min/16% from the build time)

6 years agoregress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance
Azat Khuzhin [Mon, 5 Nov 2018 19:25:15 +0000 (22:25 +0300)]
regress_ssl: make ssl/bufferevent_wm_filter more fault-tolerance

Due to inplace callbacks (i.e. no BEV_OPT_DEFER_CALLBACKS) we cannot be
sure that wm_transfer() will not be called recursively and indeed it
still happens sometimes, and the referenced patch increase amount of
this times, especially for linux/poll.

Fixes: 66304a23cf748714159c988e78f35401c5352827 ("Fix
ssl/bufferevent_wm_filter when bev does not reach watermark on break")

6 years agoregress_http: disable http/read_on_write_error under win32
Azat Khuzhin [Mon, 5 Nov 2018 18:33:54 +0000 (21:33 +0300)]
regress_http: disable http/read_on_write_error under win32

EVHTTP_CON_READ_ON_WRITE_ERROR works only if an error already read from
the socket, but if we already got EPIPE on write we cannot read from the
socket anymore, and win32 does not guarantee that read will happens
before (although it happens from time to time).

In the referenced patch I just replaced callback with not expecting 417,
but like I already wrote, this is not always true (i.e. it is flacky).

Fixes: 3b581693ac1967f7f8d98491cb772a1b415eb4cd ("test/http:
read_on_write_error: fix it for win32")

6 years agotravis-ci: exclude appveyor branches
Azat Khuzhin [Mon, 5 Nov 2018 15:02:05 +0000 (18:02 +0300)]
travis-ci: exclude appveyor branches

6 years agocmake: do not detect _GNU_SOURCE/__GNU_LIBRARY__ if it is cached
Azat Khuzhin [Sun, 4 Nov 2018 16:48:11 +0000 (19:48 +0300)]
cmake: do not detect _GNU_SOURCE/__GNU_LIBRARY__ if it is cached

6 years agoMerge branch 'ssl_bufferevent_wm_filter-fix'
Azat Khuzhin [Sun, 4 Nov 2018 18:41:20 +0000 (21:41 +0300)]
Merge branch 'ssl_bufferevent_wm_filter-fix'

* ssl_bufferevent_wm_filter-fix:
  Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break
  regress_ssl: cover watermarks with deferred callbacks
  regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging

6 years agoFix ssl/bufferevent_wm_filter when bev does not reach watermark on break
Azat Khuzhin [Sun, 4 Nov 2018 17:40:04 +0000 (20:40 +0300)]
Fix ssl/bufferevent_wm_filter when bev does not reach watermark on break

For the ssl/bufferevent_wm* we have next configuration:
- payload_len = 1024
- wm_high     = 5120
- limit       = 40960
- to_read     = 512

In this test we expect that with high watermark installed to "wm_high"
we will read "limit" bytes by reading "to_read" at a time, but adding
"payload_len" at a time (this "to_read"/"payload_len" limits is
installed to finally overflow watermark).

Once we read "limit" bytes we break, by disable EV_READ and reset
callbacks. Although this will not work if when we want to break we do
not reach watermark, this is because watermarks installs evbuffer
callback for the input buffer and if the watermark does not reached it
will enable EV_READ while be_openssl_enable() will read from the
underlying buffer (in case the openssl bufferevent created via
bufferevent_openssl_filter_new()) and call callback again (until it will
reach watermark or read al from the underlying buffer -- this is why it
stops in our caes).

And this is exactly what happened in win32, you can see this in the
following logs:

- win32 before:
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 40960
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 41472
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 41984
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break
    OK C:\vagrant\test\regress_ssl.c:829: wm_transfer-client(00DC2750): in: 4608, out: 0, got: 42496
    OK C:\vagrant\test\regress_ssl.c:834: wm_transfer-client(00DC2750): break

- win32 after:
    OK C:\vagrant\test\regress_ssl.c:821: wm_transfer-client(00FC26F0): break
    OK C:\vagrant\test\regress_ssl.c:836: wm_transfer-client(00FC26F0): in: 4800, out: 0, got: 40960

- linux before:
    OK ../test/regress_ssl.c:829: wm_transfer-client(0x55555566f5e0): in: 5120, out: 0, got: 40960
    OK ../test/regress_ssl.c:834: wm_transfer-client(0x55555566f5e0): break

- linux after:
    OK ../test/regress_ssl.c:821: wm_transfer-client(0x55555566f5e0): break
    OK ../test/regress_ssl.c:836: wm_transfer-client(0x55555566f5e0): in: 5120, out: 0, got: 40960

(As you can see in linux case we already reach watermark hence it passed
before).

So fix the issue by breaking before draining.

But during fixing this I was thinking is this right? I.e. reading from
the be_openssl_enable(), maybe we should force deferred callbacks at
least?

6 years agoregress_ssl: cover watermarks with deferred callbacks
Azat Khuzhin [Sun, 4 Nov 2018 17:15:14 +0000 (20:15 +0300)]
regress_ssl: cover watermarks with deferred callbacks

6 years agoregress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging
Azat Khuzhin [Sun, 4 Nov 2018 17:03:50 +0000 (20:03 +0300)]
regress_ssl: improve bufferevent_wm/bufferevent_wm_filter logging

- add bev pointer
- use EV_SIZE_FMT over %zu (win32)

6 years agoregress_http: make https_basic non time dependent
Azat Khuzhin [Fri, 2 Nov 2018 20:43:57 +0000 (23:43 +0300)]
regress_http: make https_basic non time dependent

Fixes: #454
6 years agoMerge branch 'check-O_NONBLOCK-in-debug'
Azat Khuzhin [Sat, 3 Nov 2018 22:21:48 +0000 (01:21 +0300)]
Merge branch 'check-O_NONBLOCK-in-debug'

* check-O_NONBLOCK-in-debug:
  regress: use non blocking descriptors whenever it is possible
  assert that fds are nonblocking in debug mode

Closes: nmathewson/Libevent#90
6 years agoregress: use non blocking descriptors whenever it is possible
Azat Khuzhin [Sat, 3 Nov 2018 21:59:33 +0000 (00:59 +0300)]
regress: use non blocking descriptors whenever it is possible

Next tests uses fds without O_NONBLOCK flag
- main/free_active_base
- main/many_events
- et/et (has some other bits cleaned up by using TT_* flags and test
  setup/cleanup callbacks)

And hence they will fail in debug mode (EVENT_DEBUG_MODE=):
  Assertion flags & O_NONBLOCK failed in event_debug_assert_socket_nonblocking_

6 years agoassert that fds are nonblocking in debug mode
Greg Hazel [Wed, 8 Jan 2014 22:57:14 +0000 (14:57 -0800)]
assert that fds are nonblocking in debug mode

Check that each fd that had been added with some event do has O_NOBLOCK
after event_enable_debug_mode()

Rebased and do not check signals (EV_SIGNAL) by azat.

Refs: nmathewson/Libevent#90
Refs: #96

6 years agoAdd cmake rules into dist archive
Azat Khuzhin [Wed, 31 Oct 2018 03:34:40 +0000 (06:34 +0300)]
Add cmake rules into dist archive

Fixes: #502
Refs: #551

6 years agoMerge branch 'event-ET-#636-v2'
Azat Khuzhin [Tue, 30 Oct 2018 22:22:30 +0000 (01:22 +0300)]
Merge branch 'event-ET-#636-v2'

* event-ET-#636-v2:
  Preserve ET bit for backends with changelist
  Epoll ET setting lost with multiple events for same fd
  Cover ET with multiple events for same fd
  Add ET flag into event_base_dump_events()

Fixes: #636
6 years agoPreserve ET bit for backends with changelist
Azat Khuzhin [Tue, 30 Oct 2018 20:59:24 +0000 (23:59 +0300)]
Preserve ET bit for backends with changelist

6 years agoEpoll ET setting lost with multiple events for same fd
Isidor Kouvelas [Tue, 30 Oct 2018 15:50:08 +0000 (08:50 -0700)]
Epoll ET setting lost with multiple events for same fd

After two or more events have been registered for the same file
descriptor using EV_ET, if one of the events is deleted, then the
epoll_ctl() call issued by libevent drops the EPOLLET flag resulting in
level triggered notifications.

[ azat: use existing "et" in the evmap_io_del_() ]

6 years agoCover ET with multiple events for same fd
Isidor Kouvelas [Tue, 30 Oct 2018 21:09:38 +0000 (00:09 +0300)]
Cover ET with multiple events for same fd

[ azat: test cleanup ]

6 years agoAdd ET flag into event_base_dump_events()
Azat Khuzhin [Mon, 29 Oct 2018 19:22:00 +0000 (22:22 +0300)]
Add ET flag into event_base_dump_events()

6 years agoappveyor: reconfigure tests with allow_failure and fast_finish
Azat Khuzhin [Sun, 28 Oct 2018 17:52:32 +0000 (20:52 +0300)]
appveyor: reconfigure tests with allow_failure and fast_finish

6 years agoMerge branch 'win32-visibility-event_debug_logging_mask_'
Azat Khuzhin [Sun, 28 Oct 2018 16:48:37 +0000 (19:48 +0300)]
Merge branch 'win32-visibility-event_debug_logging_mask_'

* win32-visibility-event_debug_logging_mask_:
  Introduce EVENT_VISIBILITY_WANT_DLLIMPORT
  regress_http: use TT_BLAZER() over event_debug()

Fixes: #702
6 years agoIntroduce EVENT_VISIBILITY_WANT_DLLIMPORT
Azat Khuzhin [Sun, 28 Oct 2018 16:30:34 +0000 (19:30 +0300)]
Introduce EVENT_VISIBILITY_WANT_DLLIMPORT

And use it in places where event_debug() should be called (since it
requires access to "event_debug_logging_mask_" and in win32 it is
tricky).

One of this places that is covered by this patch is the test for
event_debug().

6 years agoregress_http: use TT_BLAZER() over event_debug()
Azat Khuzhin [Sun, 28 Oct 2018 16:27:05 +0000 (19:27 +0300)]
regress_http: use TT_BLAZER() over event_debug()

Later is pretty tricky due to exporting event_debug_logging_mask_ symbol
for win32.

6 years agoCheck existence of IPV6_V6ONLY in evutil_make_listen_socket_ipv6only() (mingw32)
Azat Khuzhin [Sun, 28 Oct 2018 15:11:22 +0000 (18:11 +0300)]
Check existence of IPV6_V6ONLY in evutil_make_listen_socket_ipv6only() (mingw32)

MinGW 32-bit 5.3.0 does not defines it and our appveyour [1] build
reports this instantly:
    evutil.c: In function 'evutil_make_listen_socket_ipv6only':
    evutil.c:392:40: error: 'IPV6_V6ONLY' undeclared (first use in this function)
      return setsockopt(sock, IPPROTO_IPV6, IPV6_V6ONLY, (void*) &one,

  [1]: https://www.appveyor.com/docs/windows-images-software/#mingw-msys-cygwin

Another solution will be to use mingw64 which has it, but I guess we do
want that #ifdef anyway.

6 years agoConvert evbuffer_strspn() (internal helper) to use size_t
Azat Khuzhin [Sun, 28 Oct 2018 13:46:24 +0000 (16:46 +0300)]
Convert evbuffer_strspn() (internal helper) to use size_t

As pointed by @yankeehacker in #590:
  Signed to Unsigned Conversion Error - buffer.c:1623

  Description: This assignment creates a type mismatch by populating an
  unsigned variable with a signed value. The signed integer will be
  implicitly cast to an unsigned integer, converting negative values into
  positive ones. If an attacker can control the signed value, it may be
  possible to trigger a buffer overflow if the value specifies the length
  of a memory write.

  Remediation: Do not rely on implicit casts between signed and unsigned
  values because the result can take on an unexpected value and violate
  weak assumptions made elsewhere in the program.

Fixes: #590
6 years agobuffer: add an assert for last_with_datap to suppress static analyzer
Azat Khuzhin [Sun, 28 Oct 2018 12:16:24 +0000 (15:16 +0300)]
buffer: add an assert for last_with_datap to suppress static analyzer

  ../buffer.c:2231:6: warning: Access to field 'flags' results in a dereference of a null pointer
          if (CHAIN_SPACE_LEN(*firstchainp) == 0) {
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ../buffer.c:130:30: note: expanded from macro 'CHAIN_SPACE_LEN'
  #define CHAIN_SPACE_LEN(ch) ((ch)->flags & EVBUFFER_IMMUTABLE ? \

6 years agolibevent.pc: link against core/extra
Mike Frysinger [Fri, 13 Jun 2014 20:08:04 +0000 (16:08 -0400)]
libevent.pc: link against core/extra

Since we want people to stop using -levent, have the pkg-config file
also stop linking against that.  This makes it easier to delete the
libevent.so library entirely.

Closes: #141
6 years agobench_cascase: include getopt.h only for _WIN32 (like in other places)
Azat Khuzhin [Sat, 27 Oct 2018 23:00:16 +0000 (02:00 +0300)]
bench_cascase: include getopt.h only for _WIN32 (like in other places)

Fixes: #561
6 years agoDefine __EXT_POSIX2 for QNX
Maya Rashish [Sat, 23 Sep 2017 03:57:16 +0000 (06:57 +0300)]
Define __EXT_POSIX2 for QNX

POSIX 1003.2 extension is necessary for getopt interface.
Reported here [1] (not mine, I can't test)

  [1]: https://mail-index.netbsd.org/pkgsrc-users/2017/09/20/msg025601.html

Closes: #563 (cherry-pick)
6 years agoregress_ssl: reset static variables on test setup/cleanup and eliminate leaks
Azat Khuzhin [Sat, 27 Oct 2018 16:34:52 +0000 (19:34 +0300)]
regress_ssl: reset static variables on test setup/cleanup and eliminate leaks

One tricky bit is reply to the BIO_C_GET_FD command, since otherwise it
will try to close(0) and accepted bev in ssl/bufferevent_connect_sleep
will leak. Other seems more or less trivial.

This was done to make sure that for at least generic cases does not
leak (tricky cases was listed here nmathewson/Libevent#83).

And this will allow run ssl/.. with --no-fork

6 years agotest: export basic_test_setup/basic_test_cleanup to extend them
Azat Khuzhin [Sat, 27 Oct 2018 16:41:52 +0000 (19:41 +0300)]
test: export basic_test_setup/basic_test_cleanup to extend them

6 years agotravis-ci: do not notify irc.oftc.net#libevent
Azat Khuzhin [Sat, 27 Oct 2018 20:58:18 +0000 (23:58 +0300)]
travis-ci: do not notify irc.oftc.net#libevent

I think that this just introduce noise that people just turns off. So
very questionable let's remove it.

6 years agobe_openssl: avoid leaking of SSL structure
Azat Khuzhin [Sat, 27 Oct 2018 15:35:08 +0000 (18:35 +0300)]
be_openssl: avoid leaking of SSL structure

From nmathewson/Libevent#83 by @fancycode:
  There are a few code paths where the passed SSL object is not released in error cases, even if BEV_OPT_CLOSE_ON_FREE is passed as option while for others it is released. That way it's impossible for the caller to know it he has to free it on errors himself or not.

  Line numbers are from "bufferevent_openssl.c" in 911abf3:

  L1414 ("underlying == NULL" passed)
  L1416 (bio could not be created)
  L1446 (different fd passed)
  L1325 (both underlying and fd passed)
  L1328 (out-of-memory)
  L1333 ("bufferevent_init_common_" failed)
  In all error cases after the "bufferevent_ops_openssl" has been assigned, the option is evaluated on "bufferevent_free" (L1399) and the SSL object released (L1226).

Fixes: nmathewson/Libevent#83
6 years agobe_openssl: drop close_flag parameter of the BIO_new_bufferevent()
Azat Khuzhin [Sat, 27 Oct 2018 15:29:57 +0000 (18:29 +0300)]
be_openssl: drop close_flag parameter of the BIO_new_bufferevent()

6 years agoAdd evhttp_parse_query_str_flags()
Azat Khuzhin [Sat, 27 Oct 2018 14:21:35 +0000 (17:21 +0300)]
Add evhttp_parse_query_str_flags()

And a set of flags:
- EVHTTP_URI_QUERY_LAST
- EVHTTP_URI_QUERY_NONCONFORMANT

Fixes: #15