note that unlike the originals, these do not print the program
name/argv[0] because we have not saved it anywhere. this could be
changed in __libc_start_main if desired.
fix misaligned read on early string termination in strchr
this could actually cause rare crashes in the case where a short
string is located at the end of a page and the following page is not
readable, and in fact this was seen in gcc compiling certain files.
fix rare but nasty under-allocation bug in malloc with large requests
the bug appeared only with requests roughly 2*sizeof(size_t) to
4*sizeof(size_t) bytes smaller than a multiple of the page size, and
only for requests large enough to be serviced by mmap instead of the
normal heap. it was only ever observed on 64-bit machines but
presumably could also affect 32-bit (albeit with a smaller window of
opportunity).
simplify vdprintf implementation greatly based on recent vfprintf changes
since vfprintf will provide a temporary buffer in the case where the
target FILE has a zero buffer size, don't bother setting up a real
buffer for vdprintf. this also allows us to skip the call to fflush
since we know everything will be written out before vfprintf returns.
use a local temp buffer for unbuffered streams in vfprintf
this change makes it so most calls to fprintf(stderr, ...) will result
in a single writev syscall, as opposed to roughly 2*N syscalls (and
possibly more) where N is the number of format specifiers. in
principle we could use a much larger buffer, but it's best not to
increase the stack requirements too much. most messages are under 80
chars.
the problem: there is a (single-instruction) race condition window
between a thread flagging itself dead and decrementing itself from the
thread count. if it receives the rsyscall signal at this exact moment,
the rsyscall caller will never succeed in signalling enough flags to
succeed, and will deadlock forever. in previous versions of musl, the
about-to-terminate thread masked all signals prior to decrementing
the thread count, but this cost a whole syscall just to account for
extremely rare races.
the solution is a huge hack: rather than blocking in the signal
handler if the thread is dead, modify the signal mask of the saved
context and return in order to prevent further signal handling by the
dead thread. this allows the dead thread to continue decrementing the
thread count (if it had not yet done so) and exiting, even while the
live part of the program blocks for rsyscall.
for some inexplicable reason, linux allows the sender of realtime
signals to spoof its identity. permission checks for sending signals
should limit the impact to same-user processes, but just to be safe,
we avoid trusting the siginfo structure and instead simply examine the
program state to see if we're in the middle of a legitimate rsyscall.
if init_malloc returns positive (successful first init), malloc will
retry getting a chunk from the free bins rather than expanding the
heap again. also pass init_malloc a hint for the size of the initial
allocation.
reorganize the __libc structure for threaded performance issues
we want to keep atomically updated fields (locks and thread count) and
really anything writable far away from frequently-needed function
pointers. stuff some rarely-needed function pointers in between to
pad, hopefully up to a cache line boundary.
Rich Felker [Wed, 30 Mar 2011 17:04:55 +0000 (13:04 -0400)]
avoid all malloc/free in timer creation/destruction
instead of allocating a userspace structure for signal-based timers,
simply use the kernel timer id. we use the fact that thread pointers
will always be zero in the low bit (actually more) to encode integer
timerid values as pointers.
also, this change ensures that the timer_destroy syscall has completed
before the library timer_destroy function returns, in case it matters.
Rich Felker [Wed, 30 Mar 2011 16:06:39 +0000 (12:06 -0400)]
optimize timer creation and possibly protect against some minor races
the major idea of this patch is not to depend on having the timer
pointer delivered to the signal handler, and instead use the thread
pointer to get the callback function address and argument. this way,
the parent thread can make the timer_create syscall while the child
thread is starting, and it should never have to block waiting for the
barrier.
Rich Felker [Wed, 30 Mar 2011 14:32:45 +0000 (10:32 -0400)]
avoid crash on stupid but allowable usage of pthread_mutex_unlock
unlocking an unlocked mutex is not UB for robust or error-checking
mutexes, so we must avoid calling __pthread_self (which might crash
due to lack of thread-register initialization) until after checking
that the mutex is locked.
Rich Felker [Wed, 30 Mar 2011 13:29:49 +0000 (09:29 -0400)]
rename __simple_malloc.c to lite_malloc.c - yes this affects behavior!
why does this affect behavior? well, the linker seems to traverse
archive files starting from its current position when resolving
symbols. since calloc.c comes alphabetically (and thus in sequence in
the archive file) between __simple_malloc.c and malloc.c, attempts to
resolve the "malloc" symbol for use by calloc.c were pulling in the
full malloc.c implementation rather than the __simple_malloc.c
implementation.
as of now, lite_malloc.c and malloc.c are adjacent in the archive and
in the correct order, so malloc.c should never be used to resolve
"malloc" unless it's already needed to resolve another symbol ("free"
or "realloc").
Rich Felker [Wed, 30 Mar 2011 13:06:00 +0000 (09:06 -0400)]
streamline mutex unlock to remove a useless branch, use a_store to unlock
this roughly halves the cost of pthread_mutex_unlock, at least for
non-robust, normal-type mutexes.
the a_store change is in preparation for future support of archs which
require a memory barrier or special atomic store operation, and also
should prevent the possibility of the compiler misordering writes.
Rich Felker [Wed, 30 Mar 2011 12:58:25 +0000 (08:58 -0400)]
cheap special-case optimization for normal mutexes
cycle-level benchmark on atom cpu showed typical pthread_mutex_lock
call dropping from ~120 cycles to ~90 cycles with this change. benefit
may vary with compiler options and version, but this optimization is
very cheap to make and should always help some.
Rich Felker [Tue, 29 Mar 2011 17:01:25 +0000 (13:01 -0400)]
implement POSIX timers
this implementation is superior to the glibc/nptl implementation, in
that it gives true realtime behavior. there is no risk of timer
expiration events being lost due to failed thread creation or failed
malloc, because the thread is created as time creation time, and
reused until the timer is deleted.
Rich Felker [Tue, 29 Mar 2011 16:58:22 +0000 (12:58 -0400)]
major improvements to cancellation handling
- there is no longer any risk of spoofing cancellation requests, since
the cancel flag is set in pthread_cancel rather than in the signal
handler.
- cancellation signal is no longer unblocked when running the
cancellation handlers. instead, pthread_create will cause any new
threads created from a cancellation handler to unblock their own
cancellation signal.
- various tweaks in preparation for POSIX timer support.
Rich Felker [Mon, 28 Mar 2011 05:14:44 +0000 (01:14 -0400)]
major stdio overhaul, using readv/writev, plus other changes
the biggest change in this commit is that stdio now uses readv to fill
the caller's buffer and the FILE buffer with a single syscall, and
likewise writev to flush the FILE buffer and write out the caller's
buffer in a single syscall.
making this change required fundamental architectural changes to
stdio, so i also made a number of other improvements in the process:
- the implementation no longer assumes that further io will fail
following errors, and no longer blocks io when the error flag is set
(though the latter could easily be changed back if desired)
- unbuffered mode is no longer implemented as a one-byte buffer. as a
consequence, scanf unreading has to use ungetc, to the unget buffer
has been enlarged to hold at least 2 wide characters.
- the FILE structure has been rearranged to maintain the locations of
the fields that might be used in glibc getc/putc type macros, while
shrinking the structure to save some space.
- error cases for fflush, fseek, etc. should be more correct.
- library-internal macros are used for getc_unlocked and putc_unlocked
now, eliminating some ugly code duplication. __uflow and __overflow
are no longer used anywhere but these macros. switch to read or
write mode is also separated so the code can be better shared, e.g.
with ungetc.
Rich Felker [Sat, 26 Mar 2011 02:13:57 +0000 (22:13 -0400)]
match glibc/lsb cancellation abi on i386
glibc made the ridiculous choice to use pass-by-register calling
convention for these functions, which is impossible to duplicate
directly on non-gcc compilers. instead, we use ugly asm to wrap and
convert the calling convention. presumably this works with every
compiler anyone could potentially want to use.
Rich Felker [Fri, 25 Mar 2011 20:34:03 +0000 (16:34 -0400)]
fix all implicit conversion between signed/unsigned pointers
sadly the C language does not specify any such implicit conversion, so
this is not a matter of just fixing warnings (as gcc treats it) but
actual errors. i would like to revisit a number of these changes and
possibly revise the types used to reduce the number of casts required.
Rich Felker [Thu, 24 Mar 2011 18:18:00 +0000 (14:18 -0400)]
overhaul cancellation to fix resource leaks and dangerous behavior with signals
this commit addresses two issues:
1. a race condition, whereby a cancellation request occurring after a
syscall returned from kernelspace but before the subsequent
CANCELPT_END would cause cancellable resource-allocating syscalls
(like open) to leak resources.
2. signal handlers invoked while the thread was blocked at a
cancellation point behaved as if asynchronous cancellation mode wer in
effect, resulting in potentially dangerous state corruption if a
cancellation request occurs.
the glibc/nptl implementation of threads shares both of these issues.
with this commit, both are fixed. however, cancellation points
encountered in a signal handler will not be acted upon if the signal
was received while the thread was already at a cancellation point.
they will of course be acted upon after the signal handler returns, so
in real-world usage where signal handlers quickly return, it should
not be a problem. it's possible to solve this problem too by having
sigaction() wrap all signal handlers with a function that uses a
pthread_cleanup handler to catch cancellation, patch up the saved
context, and return into the cancellable function that will catch and
act upon the cancellation. however that would be a lot of complexity
for minimal if any benefit...
Rich Felker [Sun, 20 Mar 2011 01:36:10 +0000 (21:36 -0400)]
syscall overhaul part two - unify public and internal syscall interface
with this patch, the syscallN() functions are no longer needed; a
variadic syscall() macro allows syscalls with anywhere from 0 to 6
arguments to be made with a single macro name. also, manually casting
each non-integer argument with (long) is no longer necessary; the
casts are hidden in the macros.
some source files which depended on being able to define the old macro
SYSCALL_RETURNS_ERRNO have been modified to directly use __syscall()
instead of syscall(). references to SYSCALL_SIGSET_SIZE and SYSCALL_LL
have also been changed.
x86_64 has not been tested, and may need a follow-up commit to fix any
minor bugs/oversights.
Rich Felker [Sat, 19 Mar 2011 22:51:42 +0000 (18:51 -0400)]
overhaul syscall interface
this commit shuffles around the location of syscall definitions so
that we can make a syscall() library function with both SYS_* and
__NR_* style syscall names available to user applications, provides
the syscall() library function, and optimizes the code that performs
the actual inline syscalls in the library itself.
previously on i386 when built as PIC (shared library), syscalls were
incurring bus lock (lock prefix) overhead at entry and exit, due to
the way the ebx register was being loaded (xchg instruction with a
memory operand). now the xchg takes place between two registers.
further cleanup to arch/$(ARCH)/syscall.h is planned.
Rich Felker [Sat, 19 Mar 2011 01:53:30 +0000 (21:53 -0400)]
some linux headers useful from user apps.
i'm still not sure whether it's a good idea to include or use any of
these, but i'll add them for now. it may make more sense to just add
official kernel headers to the include path for compiling programs
that need them.
Rich Felker [Fri, 18 Mar 2011 02:55:43 +0000 (22:55 -0400)]
implement wprintf family of functions
this implementation is extremely ugly and inefficient, but it avoids a
good deal of code duplication and bloat. it may be cleaned up later to
eliminate the remaining code duplication and some of the warts, but i
don't really care about its performance.
Rich Felker [Fri, 18 Mar 2011 00:41:37 +0000 (20:41 -0400)]
implement robust mutexes
some of this code should be cleaned up, e.g. using macros for some of
the bit flags, masks, etc. nonetheless, the code is believed to be
working and correct at this point.