Nick Mathewson [Fri, 6 Aug 2010 20:36:23 +0000 (16:36 -0400)]
Fix unit tests with -DUSE_DEBUG enabled
If you were to enable USE_DEBUG and slog through all 700+ MB of
debugging output, you'd find that one of the unit tests failed,
since it tested the debug logging code, but the string it expected
and the string it logged differed by a tab vs 2 spaces.
Nick Mathewson [Fri, 6 Aug 2010 17:01:32 +0000 (13:01 -0400)]
Turn our socketpair() replacement into its own function
This patch splits the formerly windows-only case of evutil_socketpair()
into an (internal-use-only) function named evutil_ersatz_socketpair(), and
makes it build and work right on non-Windows hosts.
We need this for convenience to test sendfile on solaris, where socketpair
can't give you an AF_INET pair, and sendfile() won't work on AF_UNIX.
Nick Mathewson [Wed, 4 Aug 2010 19:52:32 +0000 (15:52 -0400)]
Fix an assertion bug in test-ratelim
If the rate limit was low enough, then the echo_conns wouldn't finish
inside the 300 msec we allowed for them to close. Instead, count the
number of connections we have, and keep waiting until they are all
closed.
Nick Mathewson [Wed, 4 Aug 2010 18:54:38 +0000 (14:54 -0400)]
Fix rate-limit calculation on openssl bufferevents.
When you're doing rate limiting on an openssl connection, you nearly
always want to limit the number of bytes sent and received over the
wire, not the number of bytes read or written over the secure
transport.
Nick Mathewson [Sat, 31 Jul 2010 21:10:04 +0000 (17:10 -0400)]
Build more cleanly with NetBSDs that dislike toupper(char)
To be fair, when char can be signed, if toupper doesn't take negative
characters, toupper(char) is a very bad idea. So let's just use the
nice safe EVUTIL_TOUPPER instead. (It explicitly only upcases ASCII,
but we only use it for identifiers that we know to be ASCII anyway).
Joachim Bauch [Fri, 30 Jul 2010 00:32:40 +0000 (20:32 -0400)]
Fix badly-behaved subtest of dns/bufferevent_connect_hostname
The bufferevent_connect_hostname test was specifying AF_INET, but the
gethostbyname test we were using to see what error to expect was using
PF_UNSPEC, leading to possible divergence of results.
Mike Smellie [Mon, 19 Jul 2010 03:31:19 +0000 (15:31 +1200)]
Change bufferevent_openssl::do_write so it doesn't call SSL_write with a 0 length buffer
I was running into a problem when using bufferevent_openssl with a
very simple echo server. My server simply bufferevent_read_buffer 'd
data into an evbuffer and then passed that evbuffer straight to
bufferevent_write_buffer.
The problem was every now and again the write would fail for no
apparent reason. I tracked it down to SSL_write being called with the
amount of data to send being 0.
This patch alters do_write in bufferevent_openssl so that it skips
io_vecs with 0 length.
Nick Mathewson [Mon, 19 Jul 2010 13:14:10 +0000 (15:14 +0200)]
Make test/test.sh call test-changelist
Eventually test-changelist should expand to try more cases, maybe
query the status of the actual changelist somehow, and integrate it
with the rest of the unit tests.
Mike Smellie [Mon, 19 Jul 2010 01:44:56 +0000 (13:44 +1200)]
Possible fix to 100% cpu usage with epoll and openssl
I'm running a fairly simple bit of test code using libevent2 with epoll and
openssl bufferevents and I've run into a 100% cpu usage problem.
Looking into it 100% usage was caused by epoll_wait constantly
returning write events on the openssl socket when it shouldn't really have
been looking for write events at all (N_ACTIVE_CALLBACKS() was returning 0
also).
Looking a bit deeper eventbuffer_openssl socket seems to be requesting
that the EV_WRITE event be removed when it should, but the event isn't
actually being removed from epoll.
Continuing to follow this I think I've found a bug in
event_changelist_del.
For evpoll event_del calls event_changelist_del which caches the change
which is then actioned later when evpoll_dispatch is called.
In event_changlist_del there is a check so that if the currently changed
action is an add then the cached action is changed to a no-op rather than a
delete (which makes sense). The problem arises if there are more than
two add or delete operations between calls to dispatch, in this case it's
possible that the delete is turned into a no-op when it shouldn't have
been.
For example starting with the event on, a delete followed by an add and
then another delete results in a no-op when it should have been a delete (I
added a fair bit of debug output that seems to confirm this behaviour).
I've applied a small change that checks the original old_event stored with
the change and only converts the delete to a no-op if the event isn't on in
old_event. This seems to have fixed my problem.
Fix wrong sie calculation of iovec buffers when exact=1
The old code had a bug where the 'exact' flag to 1 in
_evbuffer_read_setup_vecs would never actually make the iov_len field
of the last iovec get truncated. This patch fixes that.
Nick Mathewson [Tue, 13 Jul 2010 15:06:08 +0000 (11:06 -0400)]
Pass flags to fcntl(F_SETFL) and fcntl(F_SETFD) as int, not long
Everybody but Linux documents this as taking an int, and Linux is
very tolerant of getting an int instead. If it weren't, everybody
doing fcntl(fd,F_SETFL,O_NONBLOCK) would break, since the glibc
headers define O_NONBLOCK as an int literal.
Nick Mathewson [Thu, 8 Jul 2010 18:41:02 +0000 (14:41 -0400)]
Suppress a spurious EPERM warning in epoll.c
It's okay for us to get an EPERM when doing an EPOLL_DEL on an fd; it
just means that before we got a chance to the EPOLL_DEL, we closed the
fd and reopened a new non-socket that wound up having the same fd.
Nick Mathewson [Mon, 5 Jul 2010 18:39:39 +0000 (14:39 -0400)]
Don't race when calling event_active/event_add on a running signal event
There was previously no lock protecting the signal event's
ev_ncalls/ev_pncalls fields, which were accessed by all of
event_signal_closure, event_add_internal, event_del_internal, and
event_active_nolock. This patch fixes this race by using the
current_event_lock in the same way it's used to prevent
event_del_internal from touching an event that's currently running.
Nick Mathewson [Mon, 5 Jul 2010 17:17:47 +0000 (13:17 -0400)]
Fix a deadlock related to event-base notification. Diagnosed by Zhou Li, Avi Bab, and Scott Lamb.
The problem was that the thread doing the notification could block on
write in evthread_notify_base_default while holding the th_base_lock.
The main thread would never drain th_notify_fd[0], since it would need
th_base_lock to actually trigger events.
Nick Mathewson [Mon, 21 Jun 2010 16:26:21 +0000 (12:26 -0400)]
Add bufferevent_lock()/bufferevent_unlock()
Although bufferevent operations are threadsafe, sometimes you need
to make sure that a few operations on a single bufferevent will all
be executed with nothing intervening. That's what these functions
are for.
Nick Mathewson [Thu, 17 Jun 2010 14:33:06 +0000 (10:33 -0400)]
Have autogen.sh pass --force-missing to automake
Previously, our autogen.sh script wouldn't tell automake to update
older versions of its copied-in scripts, which would cause problems if
they got sufficiently out-of-date.
Felix Nawothnig [Sun, 30 May 2010 01:17:48 +0000 (03:17 +0200)]
Fix possible nullptr dereference in evhttp_send_reply_end()
(The existing implementation had sanity-checking code for the case where
its argument was NULL, but it erroneously dereferenced it before actually
doing the sanity-check. --nickm)
Nick Mathewson [Thu, 3 Jun 2010 15:25:54 +0000 (11:25 -0400)]
Add test for behavior on remote socket close
On all the backends on this little mac laptop, that behavior is to
report a remote socket close as both EV_READ and EV_WRITE.
Historically, we had problem for some of these behaviors on some
backends, so let's make sure that such behaviors don't come back.
Felix Nawothnig [Wed, 26 May 2010 16:50:59 +0000 (12:50 -0400)]
Fix the default HTTP error template
The current template...
<HTML><HEAD><TITLE>%s</TITLE>
</HEAD><BODY>
<H1>Method Not Implemented</H1>
Invalid method in request<P>
</BODY></HTML>
is highly confusing. The given title is easily overlooked and the
hard-coded content is just plain wrong in most cases (I really read
this as "the server did not understand the requested HTTP method)
This patch changes the template to include the error reason in the
body as well as in the header, and to infer the proper reason from
the status code whenever the reason argument is NULL.
This patch also removes a redundant evhttp_add_header from
evhttp_send_error; evhttp_send_page already adds a "Connection:
close" header.
Nick Mathewson [Wed, 26 May 2010 17:32:32 +0000 (13:32 -0400)]
Have the unit tests report errors from test.sh
The default behavior of test.sh was to suppress all output from
test/regress, and say nothing but OKAY or FAILED. This wasn't so good
for getting bugs reported, since lots of people didn't know to set
TEST_OUTPUT_FILE, or re-run ./test/regress on its own.
Now, when you don't specify an output file for test.sh, it runs
regress with the --quiet option. This option makes the unit tests
only print output on failure, which is what we probably wanted.
Nick Mathewson [Fri, 14 May 2010 18:36:49 +0000 (14:36 -0400)]
Replace (safe) use of strcpy with memcpy to appease OpenBSD
If Libevent uses strcpy, even safely, it seems OpenBSD's linker will
complain every time a library links Libevent. It's easier just not to
use the old thing, even when it's safe to do so.
Nick Mathewson [Thu, 13 May 2010 16:01:30 +0000 (12:01 -0400)]
Add options to test-ratelim.c to check its results
The new options let you specify a maximum deviation of bandwidth used
from expected bandwidth used, and make test-ratelim.c exit with a
nonzero status when those deviations are violated.
This patch also adds a test-ratelim.sh script to run test-ratelim with
a few sensible options for testing.
Nick Mathewson [Thu, 13 May 2010 14:57:30 +0000 (10:57 -0400)]
Mark the event_err() functions as __attribute__((noreturn))
This attribute tells gcc (and anything else that understands gcc
attributes) that the functions will never return control, and helps
the optimizer a little. With luck, it will also tell
less-than-full-program dataflow analysis tools that they don't need to
worry about any code path that involves calling one of these functions
and then returning.
This patch also forces event_exit() to always exit, no matter what the
user-supplied fatal_callback does. This means that the old unit tests
for the event_err* functions don't work any more, since they assume it
is safe to call event_err* if you've given it a bogus fatal_callback
that doesn't exit. Instead, we have to make the unit tests fork
before calling event_err(), and have the main unit test process wait
for the event_err() test to exit with a sane exit code. On unix,
that's trivial. On windows, let's not bother and just assume that
event_err* works.
Nick Mathewson [Tue, 11 May 2010 15:44:07 +0000 (11:44 -0400)]
Remove the obsolete evthread interfaces
These were added in 2.0.1, and deprecated in 2.0.4 and 2.0.5; we've
promised that they would be removed, and warned whenever they were
invoked. Users should call evthread_set_lock_callbacks instead... or
ideally just call evthread_use_windows_threads or
evthread_use_pthreads.
Nick Mathewson [Sun, 9 May 2010 03:29:29 +0000 (23:29 -0400)]
Fix some autoconf issues on OpenBSD
Issue 1: autoconf gets accept when a header works properly with cpp
but not with cc. This was true of the sys/sysctl.h header on
openbsd. The fix: include sys/param.h (if present) when testing for
sys/sysctl.h
Issue 2: Somehow, autoconf's macro generation code is messed up on
some versions of openbsd (including mine, and other people's too) so
that instead of SIZEOF_VOID_P, it makes SIZEOF_VOID__.
evutil/util.h now works around that.
Nick Mathewson [Sun, 9 May 2010 02:21:52 +0000 (22:21 -0400)]
Define _REENTRANT as needed on Solaris, elsewhere
It turns out that _REENTRANT isn't only needed to make certain
functions visible; we also need it to make pthreads work properly
some places (like Solaris, where forgetting _REENTRANT basically
means that all threads are sharing the same errno). Fortunately,
our ACX_PTHREAD() configure macro already gives us a PTHREAD_CFLAG
variable, so all we have to do is use it.
Nick Mathewson [Sat, 8 May 2010 23:56:25 +0000 (19:56 -0400)]
Fix test.sh on freebsd
It turns out that in all conformant shells, "unset FOO" removes FOO
both from the shell's variables and from the exported environment.
(I've tested this on msys, opensolaris, linux, osx, and freebsd.)
And in nearly every shell I can find, "unset FOO; export FOO" does
the same as unset FOO... except in my FreeBSD VM, where the "export
FOO" sets the exported value of FOO equal to "". This broke test.sh
for us.
Nick Mathewson [Sat, 8 May 2010 22:00:26 +0000 (18:00 -0400)]
Make test.sh support mingw/msys on win32
This required:
- Adding another WIN32 section in test.sh
- not running "touch /dev/null"
- calling WSAStartup in all the test binaries
- Fixing a dumb windows-only bug in test-time.c
Nick Mathewson [Sat, 8 May 2010 23:16:47 +0000 (19:16 -0400)]
Fix another nasty solaris getaddrinfo() behavior
Everybody else thinks that when you getaddrinfo() on an ip address
and don't specify the protocol and the socktype, it should give you
multiple answers , one for each protocol/socktype implementation.
OpenSolaris takes a funny view of RFC3493, and leaves the results set
to 0.
This patch post-processes the getaddrinfo() results for consistency.
Nick Mathewson [Wed, 21 Apr 2010 15:57:55 +0000 (11:57 -0400)]
Fix getaddrinfo with protocol unset on Solaris 9. Found by Dagobert Michelsen
Apparently when you call Solaris 9's getaddrinfo(), it likes to leave
ai_protocol unset in the result. This is no way to behave, if I'm
reading RFC3493 right.
This patch makes us check for a getaddrinfo() that's broken in this way,
and work around it by trying to infer socktype and protocol from one
another.
Nick Mathewson [Sat, 8 May 2010 22:09:27 +0000 (18:09 -0400)]
Numerous opensolaris compilation fixes
For future note, opensolaris doesn't have sys/sysctl.h, doesn't like
comparing iov_buf to a chain_space_ptr without a cast, and is (predictably)
unforgiving of dumb syntax errors.
Also, we had accidentally broken the devpoll backend test in configure.in
Nick Mathewson [Sat, 8 May 2010 23:11:50 +0000 (19:11 -0400)]
Make test for bufferevent_connect_hostname system-neutral
Previously, the be5_outcome field for the dns error would be set to
something dependent on our system resolver. It turns out that you
can't rely on nameservers to really give you an NEXIST answer for
xyz.example.com nowadays: too many of them are annoyingly broken and
like to redirect you to their locked-in portals. This patch changes
the bufferevent_connect_hostname test so that it makes sure that the
dns_error of be5_outcome is "whatever you would get from resolving
the target hostname"
Nick Mathewson [Sat, 8 May 2010 23:09:09 +0000 (19:09 -0400)]
Make unit test for add_file able to tell "error" from "done"
Importantly, we don't actually want to call evbuffer_write() when
the buffer is empty. This makes it an error to ever get a -1 return
value from evbuffer_add_file(), which makes it safe for us to test
the return value.
Nick Mathewson [Thu, 6 May 2010 18:37:23 +0000 (14:37 -0400)]
Only specify -no-undefined on mingw
It turns out that commit 3cbca8661f broke building with shared
libraries on OSX. Since -no-undefined is only necessary on platforms
like win32, only use it there.
There may be a better fix for this. Should fix bug 2997775.
Nick Mathewson [Thu, 6 May 2010 17:26:05 +0000 (13:26 -0400)]
Stop distributing and installing manpages: they were too inaccurate
It would be great to have the manpages come back some time, perhaps
from a refactoring of my asciidoc book, but for now the existing
manpages were the single worst, most incomplete, and most misleading
libevent documentation we had. (Less misleading: the doxygen output,
the header files, and my reference book.)
Nick Mathewson [Tue, 4 May 2010 17:27:36 +0000 (13:27 -0400)]
Rename current_base symbol to event_global_current_base_
The "current_base" symbol was never actually declared in an exported
header; it's hideously deprecated, and it was the one remaining
exported symbol (fwict) that was prefixed with neither ev nor
bufferevent nor _ev nor _bufferevent.
codesearch.google.com turns up no actual attempts to use our
current_base from outside libevent.
Nick Mathewson [Tue, 4 May 2010 16:57:40 +0000 (12:57 -0400)]
Fix symbol conflict between mm_*() macros and libmm
Our mm_malloc, mm_calloc, etc functions were all exported, since C
hasn't got a nice portable way to say "we want to use this function
inside our library but not export it to others". But they apparently
conflict with anything else that calls its symbols mm_*, as libmm does.
This patch renames the mm_*() functions to event_mm_*_(, and defines
maros in mm_internal so that all the code we have that uses mm_*()
will still work. New code should also prefer the mm_*() macro names.