Nick Mathewson [Mon, 9 Nov 2009 17:16:30 +0000 (17:16 +0000)]
Implement queued timeouts for case where many timeouts are the same.
Libevent's current timeout code is relatively optimized for the
randomly scattered timeout case, where events are added with their
timeouts in no particular order. We add and remove timeouts with
O(lg n) behavior.
Frequently, however, an application will want to have many timeouts
of the same value. For example, we might have 1000 bufferevents,
each with a 2 second timeout on reading or writing. If we knew this
were always the case, we could just put timeouts in a queue and get
O(1) add and remove behavior. Of course, a queue would give O(n)
performance for a scattered timeout pattern, so we don't want to
just switch the implementation.
This patch gives the user the ability to explicitly tag certain
timeout values as being "very common". These timeout values have a
cookie encoded in the high bits of their tv_usec field to indicate
which queue they belong on. The queues themselves are each
triggered by an entry in the minheap.
Nick Mathewson [Fri, 6 Nov 2009 17:12:39 +0000 (17:12 +0000)]
Fix a miscalculated realloc() size in win32select.c.
This bug was introduced by the code to make the backend able to safely release the base lock while calling select().
Also, we change win32select.c to the same 32-fds-to-start default as the rest of the backends, so that the main/many_events test can test it. It was at 64-to-start, so the test wasn't hitting it.
Nick Mathewson [Wed, 4 Nov 2009 05:19:26 +0000 (05:19 +0000)]
Commit ConnectEx code to get connect working with async bufferevents.
This is code by Chris Davis, with changes to get the unit tests failing less aggressively.
The unit tests for this code do not completely pass yet; Chris is looking into that. If they aren't passing by the next release, I'll turn off this code.
Nick Mathewson [Tue, 3 Nov 2009 20:40:48 +0000 (20:40 +0000)]
Add a bufferevent function to resolve a name then connect to it.
This function, bufferevent_socket_connect_hostname() can either use
evdns to do the resolve, or use a new function (evutil_resolve) that
uses getaddrinfo or gethostbyname, like http.c does now.
This function is meant to eventually replace the hostname resolution mess in
http.c.
Nick Mathewson [Tue, 3 Nov 2009 19:54:56 +0000 (19:54 +0000)]
Remove compat/sys/_time.h
I've gone through everything that it declared to see where it was used,
and it seems that we probably don't need it anywhere.
Here's what it declared, and why I think we're okay dropping it.
o struct timeval {}
(Used all over, and we can't really get away with declaring it ourselves;
we need the same definition the system uses. If we can't find struct
timeval, we're pretty much sunk.)
o struct timespec {}
(Used in event.c, evdns.c, kqueue.c, evport.c. Of these,
kqueue.c and event.c include sys/_time.h. event.c conditions its use on
_EVENT_HAVE_CLOCK_GETTIME, and kqueue() only works if timespec is defined.)
o TIMEVAL_TO_TIMESPEC
(Used in kqueue.c, but every place with kqueue has sys/time.h)
o struct timezone {}
(event2/util.h has a forward declaration; only evutil.c references it and
doesn't look at its contents.)
o timerclear, timerisset, timercmp, timeradd, timersub
(Everything now uses the evutil_timer* variants.)
o ITIMER_REAL, ITIMER_VIRTUAL, ITIMER_PROF, struct itemerval
(These are only used in test/regress.c, which does not include _time.h)
o CLOCK_REALTIME
(Only used in evdns.c, which does not include _time.h)
o TIMESPEC_TO_TIMEVAL
o DST_*
o timespecclear, timespecisset, timespeccmp, timespecadd, timespecsub
o struct clockinfo {}
o CLOCK_VIRTUAL, CLOCK_PROF
o TIMER_RELTIME, TIMER_ABSTIME
(unused)
Nick Mathewson [Mon, 2 Nov 2009 19:51:26 +0000 (19:51 +0000)]
Refactor IOCP callback interface
Chris Davis points out that GetQueuedCompletionStatus
sometimes returns false not to report "No events for
you!" but instead to report "An overlapped operation
failed." Add a way to tell an event_overlapped that
its operation failed.
Nick Mathewson [Fri, 30 Oct 2009 22:43:30 +0000 (22:43 +0000)]
Add a "many events" regression test.
This is a glass-box test to get more coverage on the event loop
backends. We've run into bugs here before with fencepost errors, and
it turns out that none of our unit tests had enough events to
exercise the resize code.
Most of the backends have some kind of logic that resizes an array
when:
- The highest fd is too high
- The number of events added since the last iteration of the loop
is too high
- The number of active events is too high.
This test hits all 3 cases, and increases coverage in select.c by 7%,
in poll by 1%, and in kqueue by 9%.
Nick Mathewson [Fri, 30 Oct 2009 21:08:29 +0000 (21:08 +0000)]
Keep openssl errors associated with the right bufferevent object.
OpenSSL has a per-thread error stack, and really doesn't like you
leaving errors on the stack. Rather than discard the errors or force
the user to handle them, this patch pulls them off the openssl stack
and puts them on a stack associated with the bufferevent_openssl. If
the user leaves them on the stack then, it won't affect any other
connections.
Nick Mathewson [Tue, 27 Oct 2009 06:47:25 +0000 (06:47 +0000)]
Avoid calling exit() during event_base_new*()
Previously, each of the three make-an-event-base functions would exit
under different, weird circumstances, but return NULL on others.
- All three would exit on OOM sometimes.
- event_base_new() and event_init() would die if all backends were
disabled.
- None of them would die if the socketpair() call failed.
Now, only event_init() exits on failure, and it exits on every kind of
failure. event_base_new() and event_base_new_with_config() never do.
Nick Mathewson [Mon, 26 Oct 2009 20:00:08 +0000 (20:00 +0000)]
Add an EVUTIL_ASSERT() to replace our calls to assert().
The big difference here is that EVUTIL_ASSERT() passes its message on
via event_errx() before aborting, so that the application has a prayer
of noticing and recording it.
Nick Mathewson [Wed, 21 Oct 2009 19:21:05 +0000 (19:21 +0000)]
Make the bufferevent_connect_fail test faster on OSX.
It seems that connecting to a listener that is bound but not accepting
or listening doesn't give a 'connection refused' error on OSX, but
rather makes the connect() time out after 75 seconds. I couldn't find
any way to make the timout shorter. Fortunately, closing the listener
after a second or so makes the desired error occur after another
second or so.
Nick Mathewson [Wed, 21 Oct 2009 18:48:22 +0000 (18:48 +0000)]
Treat the bitwise OR of two enum values as an int.
This makes our interfaces usable from C++, which doesn't believe
you can say "bufferevent_socket_nase(base, -1,
BEV_OPT_CLOSE_ON_FREE|BEV_OPT_DEFER_CALLBACKS)" but which instead
would demand "static_cast<bufferevent_options>(BEV_OPT_CLOSE_ON_FREE|
BEV_OPT_DEFER_CALLBACKS))" for the last argument.
Nick Mathewson [Wed, 21 Oct 2009 07:00:14 +0000 (07:00 +0000)]
Fix win32 connect() event handling.
Christopher Davis reported:
Connection failures aren't reported on Windows when
using bufferevent_socket_connect, because Windows uses
select's exceptfds to notify of failure, and libevent
treats them like read events. Only the write event
handler is currently used to handle connection events.
We should think hard about this one, since it changes
behavior from 1.4.x. Anything that worked on Mac/Unix before
will work more consistently on Windows now... but this might
break stuff that worked only on Windows, but nowhere else.
Nick Mathewson [Wed, 21 Oct 2009 03:54:00 +0000 (03:54 +0000)]
Add locking to event_base_loop.
This is harder than it sounds, since we need to make sure to
release the lock around the key call to the kernel (e.g.,
select, epoll_wait, kevent), AND we need to make sure that
none of the fields that are used in that call are touched by
anything that might be running concurrently in another
thread. I managed to do this pretty well for everything but
poll(). With poll, I needed to introduce a copy of the
event_set structure.
This patch also fixes a bug in win32.c where we called
realloc() instead of mm_realloc().
Nick Mathewson [Fri, 2 Oct 2009 03:03:58 +0000 (03:03 +0000)]
Do not notify the main thread more than needed.
Basically, we suppress the notification when an event is added or deleted
and:
- The event has no fd, or there is no change in whether we are
reading/writing on the event's fd.
- The event has no timeout, or adding the event did not make the earliest
timeout become earlier.
This should be a big efficiency win in applications with multiple threads and
lots of timeouts.
Nick Mathewson [Fri, 11 Sep 2009 18:47:35 +0000 (18:47 +0000)]
Make epoll use less RAM.
We do this by not allocating the maximum epoll_event array for the epoll
backend at startup. Instead, we start out accepting 32 events at a time, and
double the array's size when it seems that the OS is generating events faster
than we're requesting them. This saves up to 374K per epoll-based
event_base. Resolves bug 2839240.
Nick Mathewson [Wed, 19 Aug 2009 20:55:25 +0000 (20:55 +0000)]
On connect, call only one of BEV_EVENT_CONNECTED or writecb.
Previously, if we had a socket bufferevent in connect state, we'd send
both of these to indicate that the connection was done. That was broken
since the point of adding BEV_EVENT_CONNECTED was so that we could
distinguish "we're connected" and "we wrote something".
Now, writecb is called only when
A) the connection finished but the user never put the socket into a
"connecting" state, or
B) data was actually written.
Nick Mathewson [Fri, 7 Aug 2009 17:16:52 +0000 (17:16 +0000)]
Add an evbuffer_search_range() to search a bounded range of a buffer
This can be handy when you have one search to find the end of a header
section, and then you want to find a substring within the header
section without looking at the body.
projects / libevent / log