Nick Mathewson [Mon, 27 Feb 2012 22:57:01 +0000 (17:57 -0500)]
Remove an unsafe programming practice from test-fdleak; add XX comments
It's a bad idea to have a read callback read a fixed amount; Instead,
we should drain it, or have some way to know that more data is
arrived. Not unsafe in this case, but let's not encourage people to
learn bad habits.
Ross Lagerwall [Thu, 23 Feb 2012 19:40:02 +0000 (21:40 +0200)]
Add a new test: test-fdleak which tests for fd leaks by creating many sockets.
This test opens a server socket, and forks a child which connects to that
server socket many times. It sets a low number for the max open file limit
to catch any file descriptor leaks.
It would not work on Windows since it uses fork() to be able to create both the
server and the clients.
Nick Mathewson [Sun, 12 Feb 2012 02:01:53 +0000 (21:01 -0500)]
Refactor the functions that run over every event.
Now there are appropriate "for each event", "for each fd", and "for
each signal" helpers that they can use; this makes the code a bit
simpler, and far less duplicated.
This lets me turn back on the functions I disabled when removing
eventlist.
Additionally, check more lists for circularity in
event_base_assert_ok().
Add typedefs for the callback types.
Name fewer things "ctx".
Adds an implementation of Floyd's tortoise-and-hare algorithm to check
for circularity in TAILQs and LISTs, to avoid the abuse of flags that
event_base_assert_ok() was doing before.
Mark Ellzey [Tue, 14 Feb 2012 23:04:52 +0000 (18:04 -0500)]
Support TCP_DEFER_ACCEPT sockopts for listeners
A listening socket can be enabled with the sockopt
TCP_DEFER_ACCEPT. This informs the kernel to not call the user-land
accept() until real data has been written to the socket.
A new flag LEV_OPT_DEFERRED_ACCEPT has been introduced to
automatically set this option. Optionally
evutil_make_tcp_listen_socket_deferred() can be called manually.
Nick Mathewson [Thu, 16 Feb 2012 01:12:32 +0000 (20:12 -0500)]
Stop crashing in evdns when nameserver probes give a weird error
When a nameserver is down, we periodically try sending a "probe"
message to that nameserver to see if it has come back up. If a
nameserver comes up, we cancel any pending probe messages.
Cancelling a probe message while handling the probe's response would
result in a access-after-free or a double-free, so when we notice that
we're about to call a nameserver up because of having received a probe
from it, we need to check whether current response is the response
from the probe.
There was a case where we didn't to that, though: when the resolver
gave us an unusual error response to our request that it resolve
google.com. This is pretty rare, but apparently it can happen with
some weird cacheing nameservers -- the one on the mikrotik router, for
example. Without this patch, we would crash with a NULL pointer
derefernce.
Thanks to Hannes Sowa for finding this issue and helping me track it
down.
Nick Mathewson [Fri, 10 Feb 2012 20:55:15 +0000 (15:55 -0500)]
Infrastructure for using faster/fewer syscalls when creating sockets
Linux provides some features that allow avoiding extra calls to
fcntl when creating new nonblocking/close-on-exec sockets, so
we can add wrapper functions to emulate those when they are not
available.
Additionally, even when we are emulating those functions, we can
take a fast path that cuts our fcntl calls in half: we don't need to
look up the previous value of a file's flags when we have just
created it.
Nick Mathewson [Fri, 10 Feb 2012 16:24:51 +0000 (11:24 -0500)]
In the kqueue backend, do not report EBADF as an EV_READ
We were doing this because of (correct) reports that NetBSD gives an
EBADF when you try to add the write side of a pipe for which the
read side has been closed. But on most kqueue platforms, that
doesn't happen, and on *all* kqueue platforms, reporting a
nonexistent fd (which we usually have if we have seen EBADF) as
readable tends to give programs a case of the vapors.
Nicholas Marriott wrote the original patch here; I did the comment
fixes.
Nick Mathewson [Mon, 6 Feb 2012 17:24:49 +0000 (12:24 -0500)]
Loop on filtering SSL reads until we are blocked or exhausted.
This is not a perfect fix, but it's much much better than the
current buggy behavior, which could lead to filtering SSL
connections that just stopped reading.
Based on ideas by Maseeb Abdul Qadir and Mark Ellzey.
Nick Mathewson [Fri, 27 Jan 2012 21:35:04 +0000 (16:35 -0500)]
Remove the eventqueue list and the ev_next pointers.
Those pointers were once used to maintain a complete list of
inserted IO and signal events. But such a list is now available by
walking over ev_io_map and ev_signal_map! So all they did was
require extra pointer operations to maintain, and extra 8-16 bytes
of storage in each struct event.
To be cowardly and keep the option of going back to having this
redundancy, I'm wrapping the removed code in a set of ifdefs.
This is a first cut; it needs cleanups and stress-testing!! In
particular, it just plain disables a couple of functions that could
probably be saved.
There seems to be a need for an evmap_{io,signal}_foreach() or something.
Nick Mathewson [Fri, 27 Jan 2012 19:30:41 +0000 (14:30 -0500)]
Restore fast-path event_reinit() for slower backends
We used to use the needs_reinit flag in struct eventop to indicate
whether an event backend had shared state across a fork(), and
therefore would require us to construct a new event backend. But
when we realized that the signal notification fds and the thread
notification fds would always be shared across forks, we stopped
looking at it.
This patch restores the old behavior so that poll, select, and
win32select don't need to do a linear scan over all pending
fds/signals when they do a reinit. Their life is hard enough
already.
Nick Mathewson [Fri, 27 Jan 2012 18:54:05 +0000 (13:54 -0500)]
Make event_reinit() more robust and maintainable
Previously, event_reinit required a bunch of really dubious hacks,
and violated a lot of abstraction barriers to mess around with lists
of internal events and "pretend" to re-add them.
The new (and fairly well commented!) implementation tries to be much
smarter, by isolating the changes as much as possible to the backend
state, and minimizing the amount of abstraction violations.
Specifically, we now use event_del() to remove events we want to
remove, rather than futzing around with queues in event_reinit().
To avoid bogus calls to evsel->del(), we temporarily replace evsel
with a null-object stub.
Also, we now push the responsibility for calling evsel->add() down
into the evmap code, so that we don't actually need to unlink and
re-link all of our events.
Nick Mathewson [Tue, 24 Jan 2012 21:08:00 +0000 (16:08 -0500)]
When including an -internal.h header outside the main tree, do so early
Some of our unit tests and sample code need functions and structures
defined in an -internal.h header. But that can freak out OpenSolaris,
where stdio.h wants to define _FILE_OFFSET_BITS unless it's already
defined, and then evconfig-internal.h defines it. Regular users
should never ever use our -internal.h headers, so the solution is
to make sure that if we're going to use them ourselves, we do so
before system headers.
Nick Mathewson [Tue, 24 Jan 2012 19:33:10 +0000 (14:33 -0500)]
Make regression tests run over 3x faster.
This was mainly a matter of reducing timeouts and delays, paying
special attention to every test that took longer than a second to
finish.
We could do better here; IMO anything over .7 sec is probably too
long, but it's a big win as it is.
Remember, interactive computing is a big win over batch processing:
anything that makes you get up and walk away from the terminal might
as well be making you carry your punch cards over to the mainframe.