Nick Mathewson [Tue, 29 Dec 2009 22:59:55 +0000 (17:59 -0500)]
Allow the user to redirect the verbose output of test/test.sh to a file
By default, the test.sh script still suppresses the output of all the
tests it invokes. Now, however, you can have that output written to
a file specified in the TEST_OUTPUT_FILE shell variable.
Nick Mathewson [Tue, 29 Dec 2009 21:04:16 +0000 (16:04 -0500)]
Make the initial nameserver probe timeout configurable.
When we decide that a nameserver is down, we stop sending queries to
it, except to periodically probe it to see if it has come back up.
Our previous probe sechedule was an ad-hoc and hard-wired "10 seconds,
one minute, 5 minues, 15 minutes, 1 hour, 1 hour, 1 hour...". There
was nothing wrong with having it be ad-hoc, but making it hard-wired
served no good purpose.
Now the user can set the initial timeout via a new
"initial-probe-timeout:" option; future timeouts back off by a factor
of 3 on every failure to a maximum of 1 hour.
As a side-benefit, this lets us cut the runtime of the dns/retry test
from about 40 seconds to about 3 seconds. Faster unit tests are
always a good thing.
Nick Mathewson [Sat, 19 Dec 2009 04:37:50 +0000 (23:37 -0500)]
Set all instances of the version number correctly.
Note that we've made two subtle mistakes: we are supposed to suffix
any non-released version with "-dev", and we're supposed to use the
last byte of the numeric version to indicate whether we have done this.
For example, when 2.0.4-alpha is released, its numeric versin will be
0x 02 00 04 00. As soon as we tag it, we will change the version in
the git repository to 2.0.4-alpha-dev, whose numeric version will be
0x 02 00 04 01 or something.
Nick Mathewson [Mon, 23 Nov 2009 23:34:32 +0000 (18:34 -0500)]
Refactor our 'suspend operation' logic on bufferevents.
There are lots of things we do internally in bufferevents to indicate
"the user would like this operation to happen, but we aren't going to
try until some other condition goes away." Our logic here has gotten
entirely too complicated.
This patch tries to fix that by adding the idea of 'suspend flags' for
read and write. To say "don't bother reading or writing until
condition X no longer holds," bufferevent_suspend_read/write(bev,
BEV_SUSPEND_X). When X no longer holds, call
bufferevent_unsuspend_read/write(bev, BEV_SUSPEND_X).
Right now, only the read-watermark logic uses this.
William Ahern [Sun, 29 Nov 2009 15:20:46 +0000 (10:20 -0500)]
Valgrind fix: Clear struct kevent before checking for OSX bug.
William's original commit message:
Valgrind complains on startup because kq_init passes to kevent only
a partially initialized structure. The code doesn't expect kevent
to look at .fflags, .udata, or .data, I suppose, because it merely
tickles the kernel looking for an error response. But perhaps
that's unwarranted chuminess (notwithstanding that it's checking
for an OS X bug), and needless noise nonetheless.
Nick Mathewson [Fri, 27 Nov 2009 22:22:19 +0000 (17:22 -0500)]
Improved optional lock debugging.
There were a couple of places in the code where we manually kept lock
counts to make sure we never accessed resources without holding a
lock, and that we never released a lock we didn't have. The
lock-debugging code already puts counts on _every_ lock when lock
debugging is enabled, so there is no need to keep these counts around
otherwise. This patch rewrites the ASSERT_FOO_LOCKED macros to all
use a common EVLOCK_ASSERT_LOCKED().
We also teach the lock debugging code to keep track of who exactly
holds each lock, so that EVLOCK_ASSERT_LOCKED() means "locked by this
thread."
Zhuang Yuyao [Fri, 27 Nov 2009 21:02:49 +0000 (16:02 -0500)]
Fix an evdns lock violation.
Original message:
evdns contains a bug related to thread lock.
enable thread lock by evthread_use_pthreads() will cause successive
evdns_base_resolve_ipv4() (and other resolve functions i think) to
hang on EVDNS_LOCK(base) after one or several successful call to
evdns_base_resolve_ipv4().
Nick Mathewson [Fri, 27 Nov 2009 21:44:47 +0000 (16:44 -0500)]
Stop passing EVTHREAD_READ and EVTHREAD_WRITE to non-rw locks.
Previously, our default lock model kind of assumed that every lock was
potentially a read-write lock. This was a poor choice, since
read-write locks are far more expensive than regular locks, and so the
lock API should only use them when we can actually take advantage of
them. Neither our pthreads or win32 lock implementation provided rw
locks.
Now that we have a way (not currently used!) to indicate that we
really want a read-write lock, we shouldn't actually say "lock this
for reading" or "lock this for writing" unless we mean it.
Nick Mathewson [Fri, 27 Nov 2009 20:20:43 +0000 (15:20 -0500)]
Revise the locking API: deprecate the old locking callbacks and add trylock.
Previously, there was no good way to request different kinds of lock
(say, read/write vs writeonly or recursive vs nonrecursive), or for a
lock function to signal failure (which would be important for a
trylock mode).
This patch revises the lock API to be a bit more useful. The older
lock calls are still supported for now.
We also add a debugging mode to catch common errors in using the
locking APIs.
Nick Mathewson [Sat, 21 Nov 2009 06:11:49 +0000 (01:11 -0500)]
Fix memory-leak of signal handler array with kqueue.
It turns out that kqueue_dealloc wasn't calling evsig_dealloc()
(because it doesn't use the main signal handler logic) so the sh_old
array was leaking.
This patch also introduces a fix in evsig_dealloc() where we set
the sh_old array to NULL when we free it, so that main/fork can pass.
Nick Mathewson [Wed, 18 Nov 2009 23:18:55 +0000 (23:18 +0000)]
Do the proper hack for the (Open)BSD getaddrinfo quirk.
From evutil.c:
Some older BSDs (like OpenBSD up to 4.6) used to believe that
giving a numeric port without giving an ai_socktype was verboten.
We test for this so we can apply an appropriate workaround. If it
turns out that the bug is present, then:
- If nodename==NULL and servname is numeric, we build an answer
ourselves using evutil_getaddrinfo_common().
- If nodename!=NULL and servname is numeric, then we set
servname=NULL when calling getaddrinfo, and post-process the
result to set the ports on it.
We test for this bug at runtime, since otherwise we can't have the
same binary run on multiple BSD versions.
Nick Mathewson [Wed, 18 Nov 2009 21:17:00 +0000 (21:17 +0000)]
Make our failing-connection bufferevent test more tolerant.
I thought we had a way to do connect() that would never fail
immediately, but always wait for a moment before failing. It
turns out that on FreeBSD it can fail immediately. This is not
FreeBSD's fault, or even a real bug anywhere but in the unit test.
Nick Mathewson [Wed, 18 Nov 2009 21:16:53 +0000 (21:16 +0000)]
Add a temporary workaround for an ssl bug found on FreeBSD.
Basically, we only want to report the 'connected' event because of
the socket connect() finishing when we have an actual socket
bufferevent; on an SSL bufferevent, 'connected' means 'SSL
connection finished.'
This isn't FreeBSD's fault: it just has a connect() that tends to
succeed pretty early.
Nick Mathewson [Tue, 17 Nov 2009 20:31:09 +0000 (20:31 +0000)]
Move responsibility for IOCP callback into bufferevent_async.
This patch from Chris Davis saves some callback depth, and adds proper
ref-counting to bufferevents when there's a deferred evbuffer callback
inflight. It could use a couple more comments to really nail down what
its invariants are.
Nick Mathewson [Mon, 9 Nov 2009 19:37:27 +0000 (19:37 +0000)]
Change event_base.activequeues to "array of eventlist".
Previously, event_base.activequeues was of type "array of pointers to
eventlist." This was pointless: none of the eventlists were allowed
to be NULL. Worse, it was inefficient:
- It made looking up an active event queue take two pointer
deferences instead of one, thus risking extra cache misses.
- It used more RAM than it needed to, because of the extra pointer
and the malloc overhead.
Also, this patch fixes a bug where we were saying
calloc(N,N*sizeof(X)) instead of calloc(N,sizeof(X)) when allocating
activequeues. That part, I'll backport.
Also, we warn and return -1 on failure to allocate activequeues,
rather than calling event_err.
Nick Mathewson [Mon, 9 Nov 2009 18:30:57 +0000 (18:30 +0000)]
Make persistent timeouts more accurate.
Previously, if the user scheduled a persistent timeout for {1,0}, we
would schedule the first one at "now+one second", and then when we
were about to run its callback, we would schedule it again for one
second after that. This would introduce creeping delays to the event
that was supposed to run every second.
Now, we schedule the event for one second after it was _last
scheduled_. To do this, we introduce internal code to add an event at
an _absolute_ tv rather than at now+tv.
Nick Mathewson [Mon, 9 Nov 2009 17:16:30 +0000 (17:16 +0000)]
Implement queued timeouts for case where many timeouts are the same.
Libevent's current timeout code is relatively optimized for the
randomly scattered timeout case, where events are added with their
timeouts in no particular order. We add and remove timeouts with
O(lg n) behavior.
Frequently, however, an application will want to have many timeouts
of the same value. For example, we might have 1000 bufferevents,
each with a 2 second timeout on reading or writing. If we knew this
were always the case, we could just put timeouts in a queue and get
O(1) add and remove behavior. Of course, a queue would give O(n)
performance for a scattered timeout pattern, so we don't want to
just switch the implementation.
This patch gives the user the ability to explicitly tag certain
timeout values as being "very common". These timeout values have a
cookie encoded in the high bits of their tv_usec field to indicate
which queue they belong on. The queues themselves are each
triggered by an entry in the minheap.
Nick Mathewson [Fri, 6 Nov 2009 17:12:39 +0000 (17:12 +0000)]
Fix a miscalculated realloc() size in win32select.c.
This bug was introduced by the code to make the backend able to safely release the base lock while calling select().
Also, we change win32select.c to the same 32-fds-to-start default as the rest of the backends, so that the main/many_events test can test it. It was at 64-to-start, so the test wasn't hitting it.
Nick Mathewson [Wed, 4 Nov 2009 05:19:26 +0000 (05:19 +0000)]
Commit ConnectEx code to get connect working with async bufferevents.
This is code by Chris Davis, with changes to get the unit tests failing less aggressively.
The unit tests for this code do not completely pass yet; Chris is looking into that. If they aren't passing by the next release, I'll turn off this code.
Nick Mathewson [Tue, 3 Nov 2009 20:40:48 +0000 (20:40 +0000)]
Add a bufferevent function to resolve a name then connect to it.
This function, bufferevent_socket_connect_hostname() can either use
evdns to do the resolve, or use a new function (evutil_resolve) that
uses getaddrinfo or gethostbyname, like http.c does now.
This function is meant to eventually replace the hostname resolution mess in
http.c.