Mike Smellie [Mon, 19 Jul 2010 03:31:19 +0000 (15:31 +1200)]
Change bufferevent_openssl::do_write so it doesn't call SSL_write with a 0 length buffer
I was running into a problem when using bufferevent_openssl with a
very simple echo server. My server simply bufferevent_read_buffer 'd
data into an evbuffer and then passed that evbuffer straight to
bufferevent_write_buffer.
The problem was every now and again the write would fail for no
apparent reason. I tracked it down to SSL_write being called with the
amount of data to send being 0.
This patch alters do_write in bufferevent_openssl so that it skips
io_vecs with 0 length.
Nick Mathewson [Mon, 19 Jul 2010 13:14:10 +0000 (15:14 +0200)]
Make test/test.sh call test-changelist
Eventually test-changelist should expand to try more cases, maybe
query the status of the actual changelist somehow, and integrate it
with the rest of the unit tests.
Mike Smellie [Mon, 19 Jul 2010 01:44:56 +0000 (13:44 +1200)]
Possible fix to 100% cpu usage with epoll and openssl
I'm running a fairly simple bit of test code using libevent2 with epoll and
openssl bufferevents and I've run into a 100% cpu usage problem.
Looking into it 100% usage was caused by epoll_wait constantly
returning write events on the openssl socket when it shouldn't really have
been looking for write events at all (N_ACTIVE_CALLBACKS() was returning 0
also).
Looking a bit deeper eventbuffer_openssl socket seems to be requesting
that the EV_WRITE event be removed when it should, but the event isn't
actually being removed from epoll.
Continuing to follow this I think I've found a bug in
event_changelist_del.
For evpoll event_del calls event_changelist_del which caches the change
which is then actioned later when evpoll_dispatch is called.
In event_changlist_del there is a check so that if the currently changed
action is an add then the cached action is changed to a no-op rather than a
delete (which makes sense). The problem arises if there are more than
two add or delete operations between calls to dispatch, in this case it's
possible that the delete is turned into a no-op when it shouldn't have
been.
For example starting with the event on, a delete followed by an add and
then another delete results in a no-op when it should have been a delete (I
added a fair bit of debug output that seems to confirm this behaviour).
I've applied a small change that checks the original old_event stored with
the change and only converts the delete to a no-op if the event isn't on in
old_event. This seems to have fixed my problem.
Fix wrong sie calculation of iovec buffers when exact=1
The old code had a bug where the 'exact' flag to 1 in
_evbuffer_read_setup_vecs would never actually make the iov_len field
of the last iovec get truncated. This patch fixes that.
Nick Mathewson [Tue, 13 Jul 2010 15:06:08 +0000 (11:06 -0400)]
Pass flags to fcntl(F_SETFL) and fcntl(F_SETFD) as int, not long
Everybody but Linux documents this as taking an int, and Linux is
very tolerant of getting an int instead. If it weren't, everybody
doing fcntl(fd,F_SETFL,O_NONBLOCK) would break, since the glibc
headers define O_NONBLOCK as an int literal.
Nick Mathewson [Thu, 8 Jul 2010 18:41:02 +0000 (14:41 -0400)]
Suppress a spurious EPERM warning in epoll.c
It's okay for us to get an EPERM when doing an EPOLL_DEL on an fd; it
just means that before we got a chance to the EPOLL_DEL, we closed the
fd and reopened a new non-socket that wound up having the same fd.
Nick Mathewson [Mon, 5 Jul 2010 18:39:39 +0000 (14:39 -0400)]
Don't race when calling event_active/event_add on a running signal event
There was previously no lock protecting the signal event's
ev_ncalls/ev_pncalls fields, which were accessed by all of
event_signal_closure, event_add_internal, event_del_internal, and
event_active_nolock. This patch fixes this race by using the
current_event_lock in the same way it's used to prevent
event_del_internal from touching an event that's currently running.
Nick Mathewson [Mon, 5 Jul 2010 17:17:47 +0000 (13:17 -0400)]
Fix a deadlock related to event-base notification. Diagnosed by Zhou Li, Avi Bab, and Scott Lamb.
The problem was that the thread doing the notification could block on
write in evthread_notify_base_default while holding the th_base_lock.
The main thread would never drain th_notify_fd[0], since it would need
th_base_lock to actually trigger events.
Nick Mathewson [Mon, 21 Jun 2010 16:26:21 +0000 (12:26 -0400)]
Add bufferevent_lock()/bufferevent_unlock()
Although bufferevent operations are threadsafe, sometimes you need
to make sure that a few operations on a single bufferevent will all
be executed with nothing intervening. That's what these functions
are for.
Nick Mathewson [Thu, 17 Jun 2010 14:33:06 +0000 (10:33 -0400)]
Have autogen.sh pass --force-missing to automake
Previously, our autogen.sh script wouldn't tell automake to update
older versions of its copied-in scripts, which would cause problems if
they got sufficiently out-of-date.
Felix Nawothnig [Sun, 30 May 2010 01:17:48 +0000 (03:17 +0200)]
Fix possible nullptr dereference in evhttp_send_reply_end()
(The existing implementation had sanity-checking code for the case where
its argument was NULL, but it erroneously dereferenced it before actually
doing the sanity-check. --nickm)
Nick Mathewson [Thu, 3 Jun 2010 15:25:54 +0000 (11:25 -0400)]
Add test for behavior on remote socket close
On all the backends on this little mac laptop, that behavior is to
report a remote socket close as both EV_READ and EV_WRITE.
Historically, we had problem for some of these behaviors on some
backends, so let's make sure that such behaviors don't come back.
Felix Nawothnig [Wed, 26 May 2010 16:50:59 +0000 (12:50 -0400)]
Fix the default HTTP error template
The current template...
<HTML><HEAD><TITLE>%s</TITLE>
</HEAD><BODY>
<H1>Method Not Implemented</H1>
Invalid method in request<P>
</BODY></HTML>
is highly confusing. The given title is easily overlooked and the
hard-coded content is just plain wrong in most cases (I really read
this as "the server did not understand the requested HTTP method)
This patch changes the template to include the error reason in the
body as well as in the header, and to infer the proper reason from
the status code whenever the reason argument is NULL.
This patch also removes a redundant evhttp_add_header from
evhttp_send_error; evhttp_send_page already adds a "Connection:
close" header.
Nick Mathewson [Wed, 26 May 2010 17:32:32 +0000 (13:32 -0400)]
Have the unit tests report errors from test.sh
The default behavior of test.sh was to suppress all output from
test/regress, and say nothing but OKAY or FAILED. This wasn't so good
for getting bugs reported, since lots of people didn't know to set
TEST_OUTPUT_FILE, or re-run ./test/regress on its own.
Now, when you don't specify an output file for test.sh, it runs
regress with the --quiet option. This option makes the unit tests
only print output on failure, which is what we probably wanted.
Nick Mathewson [Fri, 14 May 2010 18:36:49 +0000 (14:36 -0400)]
Replace (safe) use of strcpy with memcpy to appease OpenBSD
If Libevent uses strcpy, even safely, it seems OpenBSD's linker will
complain every time a library links Libevent. It's easier just not to
use the old thing, even when it's safe to do so.
Nick Mathewson [Thu, 13 May 2010 16:01:30 +0000 (12:01 -0400)]
Add options to test-ratelim.c to check its results
The new options let you specify a maximum deviation of bandwidth used
from expected bandwidth used, and make test-ratelim.c exit with a
nonzero status when those deviations are violated.
This patch also adds a test-ratelim.sh script to run test-ratelim with
a few sensible options for testing.
Nick Mathewson [Thu, 13 May 2010 14:57:30 +0000 (10:57 -0400)]
Mark the event_err() functions as __attribute__((noreturn))
This attribute tells gcc (and anything else that understands gcc
attributes) that the functions will never return control, and helps
the optimizer a little. With luck, it will also tell
less-than-full-program dataflow analysis tools that they don't need to
worry about any code path that involves calling one of these functions
and then returning.
This patch also forces event_exit() to always exit, no matter what the
user-supplied fatal_callback does. This means that the old unit tests
for the event_err* functions don't work any more, since they assume it
is safe to call event_err* if you've given it a bogus fatal_callback
that doesn't exit. Instead, we have to make the unit tests fork
before calling event_err(), and have the main unit test process wait
for the event_err() test to exit with a sane exit code. On unix,
that's trivial. On windows, let's not bother and just assume that
event_err* works.
Nick Mathewson [Tue, 11 May 2010 15:44:07 +0000 (11:44 -0400)]
Remove the obsolete evthread interfaces
These were added in 2.0.1, and deprecated in 2.0.4 and 2.0.5; we've
promised that they would be removed, and warned whenever they were
invoked. Users should call evthread_set_lock_callbacks instead... or
ideally just call evthread_use_windows_threads or
evthread_use_pthreads.
Nick Mathewson [Sun, 9 May 2010 03:29:29 +0000 (23:29 -0400)]
Fix some autoconf issues on OpenBSD
Issue 1: autoconf gets accept when a header works properly with cpp
but not with cc. This was true of the sys/sysctl.h header on
openbsd. The fix: include sys/param.h (if present) when testing for
sys/sysctl.h
Issue 2: Somehow, autoconf's macro generation code is messed up on
some versions of openbsd (including mine, and other people's too) so
that instead of SIZEOF_VOID_P, it makes SIZEOF_VOID__.
evutil/util.h now works around that.
Nick Mathewson [Sun, 9 May 2010 02:21:52 +0000 (22:21 -0400)]
Define _REENTRANT as needed on Solaris, elsewhere
It turns out that _REENTRANT isn't only needed to make certain
functions visible; we also need it to make pthreads work properly
some places (like Solaris, where forgetting _REENTRANT basically
means that all threads are sharing the same errno). Fortunately,
our ACX_PTHREAD() configure macro already gives us a PTHREAD_CFLAG
variable, so all we have to do is use it.
Nick Mathewson [Sat, 8 May 2010 23:56:25 +0000 (19:56 -0400)]
Fix test.sh on freebsd
It turns out that in all conformant shells, "unset FOO" removes FOO
both from the shell's variables and from the exported environment.
(I've tested this on msys, opensolaris, linux, osx, and freebsd.)
And in nearly every shell I can find, "unset FOO; export FOO" does
the same as unset FOO... except in my FreeBSD VM, where the "export
FOO" sets the exported value of FOO equal to "". This broke test.sh
for us.
Nick Mathewson [Sat, 8 May 2010 22:00:26 +0000 (18:00 -0400)]
Make test.sh support mingw/msys on win32
This required:
- Adding another WIN32 section in test.sh
- not running "touch /dev/null"
- calling WSAStartup in all the test binaries
- Fixing a dumb windows-only bug in test-time.c
Nick Mathewson [Sat, 8 May 2010 23:16:47 +0000 (19:16 -0400)]
Fix another nasty solaris getaddrinfo() behavior
Everybody else thinks that when you getaddrinfo() on an ip address
and don't specify the protocol and the socktype, it should give you
multiple answers , one for each protocol/socktype implementation.
OpenSolaris takes a funny view of RFC3493, and leaves the results set
to 0.
This patch post-processes the getaddrinfo() results for consistency.
Nick Mathewson [Wed, 21 Apr 2010 15:57:55 +0000 (11:57 -0400)]
Fix getaddrinfo with protocol unset on Solaris 9. Found by Dagobert Michelsen
Apparently when you call Solaris 9's getaddrinfo(), it likes to leave
ai_protocol unset in the result. This is no way to behave, if I'm
reading RFC3493 right.
This patch makes us check for a getaddrinfo() that's broken in this way,
and work around it by trying to infer socktype and protocol from one
another.
Nick Mathewson [Sat, 8 May 2010 22:09:27 +0000 (18:09 -0400)]
Numerous opensolaris compilation fixes
For future note, opensolaris doesn't have sys/sysctl.h, doesn't like
comparing iov_buf to a chain_space_ptr without a cast, and is (predictably)
unforgiving of dumb syntax errors.
Also, we had accidentally broken the devpoll backend test in configure.in
Nick Mathewson [Sat, 8 May 2010 23:11:50 +0000 (19:11 -0400)]
Make test for bufferevent_connect_hostname system-neutral
Previously, the be5_outcome field for the dns error would be set to
something dependent on our system resolver. It turns out that you
can't rely on nameservers to really give you an NEXIST answer for
xyz.example.com nowadays: too many of them are annoyingly broken and
like to redirect you to their locked-in portals. This patch changes
the bufferevent_connect_hostname test so that it makes sure that the
dns_error of be5_outcome is "whatever you would get from resolving
the target hostname"
Nick Mathewson [Sat, 8 May 2010 23:09:09 +0000 (19:09 -0400)]
Make unit test for add_file able to tell "error" from "done"
Importantly, we don't actually want to call evbuffer_write() when
the buffer is empty. This makes it an error to ever get a -1 return
value from evbuffer_add_file(), which makes it safe for us to test
the return value.
Nick Mathewson [Thu, 6 May 2010 18:37:23 +0000 (14:37 -0400)]
Only specify -no-undefined on mingw
It turns out that commit 3cbca8661f broke building with shared
libraries on OSX. Since -no-undefined is only necessary on platforms
like win32, only use it there.
There may be a better fix for this. Should fix bug 2997775.
Nick Mathewson [Thu, 6 May 2010 17:26:05 +0000 (13:26 -0400)]
Stop distributing and installing manpages: they were too inaccurate
It would be great to have the manpages come back some time, perhaps
from a refactoring of my asciidoc book, but for now the existing
manpages were the single worst, most incomplete, and most misleading
libevent documentation we had. (Less misleading: the doxygen output,
the header files, and my reference book.)
Nick Mathewson [Tue, 4 May 2010 17:27:36 +0000 (13:27 -0400)]
Rename current_base symbol to event_global_current_base_
The "current_base" symbol was never actually declared in an exported
header; it's hideously deprecated, and it was the one remaining
exported symbol (fwict) that was prefixed with neither ev nor
bufferevent nor _ev nor _bufferevent.
codesearch.google.com turns up no actual attempts to use our
current_base from outside libevent.
Nick Mathewson [Tue, 4 May 2010 16:57:40 +0000 (12:57 -0400)]
Fix symbol conflict between mm_*() macros and libmm
Our mm_malloc, mm_calloc, etc functions were all exported, since C
hasn't got a nice portable way to say "we want to use this function
inside our library but not export it to others". But they apparently
conflict with anything else that calls its symbols mm_*, as libmm does.
This patch renames the mm_*() functions to event_mm_*_(, and defines
maros in mm_internal so that all the code we have that uses mm_*()
will still work. New code should also prefer the mm_*() macro names.
Nick Mathewson [Mon, 3 May 2010 17:00:00 +0000 (13:00 -0400)]
Try /proc on Linux as entropy fallback; use sysctl as last resort
It turns out that the happy fun Linux kernel is deprecating sysctl,
and using sysctl to fetch entropy will spew messages in the kernel
logs. Let's not do that. Instead, let's call sysctl for our
entropy only when all other means fail.
Additionally, let's add another means, and try
/proc/sys/kernel/random/uuid if /dev/urandom fails.
Every current BSD system providing TAILQ_* macros define
TAILQ_FOREACH_REVERSE in this order:
TAILQ_FOREACH_REVERSE(var, head, field, headname)
However, libevent defines it in another order:
TAILQ_FOREACH_REVERSE(var, head, headname, field)
Here's a trivial patch to have libevent compatible with stock queue.h headers.
-Frank.
[From sourceforge patch 2995179. codesearch.google.com confirms that
the only people defining TAILQ_FOREACH_REVERSE our way are people
using it in a compatibility header like us. Did we copy this from
OpenSSH or something?]
Nick Mathewson [Wed, 28 Apr 2010 19:16:32 +0000 (15:16 -0400)]
Remove redundant checks for lock!=NULL before calling EVLOCK_LOCK
The EVLOCK_LOCK and EVLOCK_UNLOCK macros already check to make sure
that the lock is present before messing with it, so there's no point
in checking the lock before calling them.
A good compiler should be able to simplify code like
if (lock) {
if (lock)
acquire(lock);
}
, but why count on it?
Nick Mathewson [Wed, 28 Apr 2010 16:03:08 +0000 (12:03 -0400)]
Catch attempts to enable debug_mode too late
Debug mode needs to be enabled before any event is setup or any
event_base is created. Otherwise, we will not have recorded when events
were first setup or added, and so it will look like a bug later when we
delete or free them.
I have already confused myself because of this requirement, so let's
make Libevent catch it for the next poor forgetful developer like me.
Nick Mathewson [Wed, 28 Apr 2010 15:51:56 +0000 (11:51 -0400)]
Make debug mode catch mixed ET and non-ET events on an fd
Of the backends that support edge-triggered IO, most (all?) do not
support attempts to mix edge-triggered and level-triggered IO on the
same FD. With debugging mode enabled, we now detect and refuse attempts
to add a level-triggered IO event to an fd that already has an
edge-triggered IO event, and vice versa.