time64 syscall is used only if it's the only one defined for the arch,
or if the requested timeout length does not fit in 32 bits. on current
32-bit archs where time_t is a 32-bit type, this makes it statically
unreachable.
on 64-bit archs, there are only superficial changes to the code after
preprocessing. on current 32-bit archs, the timeout is passed via an
intermediate copy to remove the assumption that time_t is a 32-bit
type.
time64 syscall is used only if it's the only one defined for the arch,
or if the requested absolute timeout does not fit in 32 bits. on
current 32-bit archs where time_t is a 32-bit type, this makes it
statically unreachable.
on 64-bit archs, there is no change to the code after preprocessing.
on current 32-bit archs, the timeout is passed via an intermediate
copy to remove the assumption that time_t is a 32-bit type.
time64 syscall is used only if it's the only one defined for the arch,
or if the requested time does not fit in 32 bits. on current 32-bit
archs where time_t is a 32-bit type, this makes it statically
unreachable.
on 64-bit archs, there is no change to the code after preprocessing.
on current 32-bit archs, the time is moved through an intermediate
copy to remove the assumption that time_t is a 32-bit type.
implement settimeofday in terms of clock_settime, not old syscall
this is yet another place where special handling of time syscalls can
and should be avoided by implementing legacy functions in terms of
their modern replacements. in theory a fallback to SYS_settimeofday
could be added to clock_settime, but SYS_clock_settime has been
available since Linux 2.6.0 or earlier, i.e. all the way back to the
minimum supported version.
internally, define plain syscalls, if missing, as their time64 variants
this commit has no effect whatsoever right now, but is in preparation
for a future riscv32 port and other future 32-bit archs that will be
"time64-only" from the start on the kernel side.
together with the previous x32 changes, this commit ensures that
syscall call points that don't care about time (passing null timeouts,
etc.) can continue to do so without having to special-case time64-only
archs, and allows code using the time64 syscalls to uniformly test for
the need to fallback with SYS_foo != SYS_foo_time64, rather than
needing to check defined(SYS_foo) && SYS_foo != SYS_foo_time64.
internally, define time64 syscalls on x32 as the existing syscalls
x32 is odd in that it's the only ILP32 arch/ABI we have where time_t
is 64-bit rather than (32-bit) long, and this has always been
problematic in that it results in struct timespec having unused
padding space, since tv_nsec has type long, which the kernel insists
be zero- or sign-extended (due to negative tv_nsec being invalid, it
doesn't matter which) to match the x86_64 type.
up til now, we've had really ugly hacks in x32/syscall_arch.h to patch
up the timespecs passed to the kernel. but the same requirement to
zero- or sign-extend tv_nsec also applies to all the new time64
syscalls on true 32-bit archs. so let's take advantage of this to
clean things up.
this patch defines all of the time64 syscalls for x32 as aliases for
the existing syscalls by the same name. this establishes the following
invariants:
- if the time64 form is defined, it takes time arguments as 64-bit
objects, and tv_nsec inputs must be zero-/sign-extended to 64-bit.
- if the time64 form is not defined, or if the time64 form is defined
and is not equal to the "plain" form, the plain form takes time
arguments as longs.
this will avoid the need for protocols for archs to define appropriate
types for each family of syscalls, and for the reader of the code to
have to be aware of such type definitions.
in some sense it might be simpler if the plain syscall form were
undefined for x32, so that it would always take longs if defined.
however, a number of these syscalls are used in contexts with a null
time argument, or (e.g. futex) for commands that don't involve time at
all, and having to introduce time64-specific logic to all those call
points does not make sense. thus, while the "plain" forms are kept now
just because they're needed until the affected code is converted over,
they'll also almost surely be kept in the future as well.
don't use futimesat syscall as utimensat fallback on x32
kernel support for x32 was added long after the utimensat syscall was
already available, so having a fallback is just wasted code size.
also, for changes related to time64 support on 32-bit archs, I want to
be able to assume the old futimesat syscall always works with longs,
which is true except for x32. by ensuring that it's not used on x32,
the needed invariant is established.
previously the fallback wrongly failed with EINVAL rather than ENOSYS
when UTIME_NOW was used with one component but not both. commit dd5f50da6f6c3df5647e922e47f8568a8896a752 introduced this behavior when
initially adding the fallback support.
instead, detect the case where both are UTIME_NOW early and replace
with a null times pointer; this may improve performance slightly (less
copy from user), and removes the complex logic from the fallback case.
it also makes things slightly simpler for adding time64 code paths.
refactor thrd_sleep and nanosleep in terms of clock_nanosleep
for namespace-safety with thrd_sleep, this requires an alias, which is
also added. this eliminates all but one direct call point for
nanosleep syscalls, and arranges that 64-bit time_t conversion logic
will only need to exist in one file rather than three.
as a bonus, clock_nanosleep with CLOCK_REALTIME and empty flags is now
implemented as SYS_nanosleep, thereby working on older kernels that
may lack POSIX clocks functionality.
Samuel Holland [Sun, 21 Jul 2019 04:52:26 +0000 (23:52 -0500)]
use the correct stat structure in the fstat path
commit 01ae3fc6d48f4a45535189b7a6db286535af08ca modified fstatat to
translate the kernel's struct stat ("kstat") into the libc struct stat.
To do this, it created a local kstat object, and copied its contents
into the user-provided object.
However, the commit neglected to update the fstat compatibility path and
its fallbacks. They continued to pass the user-supplied object to the
kernel, later overwiting it with the uninitialized memory in the local
temporary.
this sets the stage for having the conversion logic for 64-bit time_t
all in one file, and as a bonus makes clock_adjtime for CLOCK_REALTIME
work even on kernels too old to have the clock_adjtime syscall.
implement fstatat with SYS_statx, conditional on undersized kstat time
this commit adds a new backend for fstatat (and thereby the whole stat
family) using the SYS_statx syscall, but conditions the new code on
the kernel stat structure's time fields being smaller than time_t. in
principle that should make it all dead code at present, but mips64 has
a broken stat structure with 32-bit time fields despite having 64-bit
time_t elsewhere, so on mips64 it is a functional change that makes
post-Y2038 filesystem timestamps accessible.
whenever the 32-bit archs end up getting 64-bit time_t, regardless of
how that happens, the changes in this commit will automatically take
effect for them too.
restore property that fstat(AT_FDCWD) fails with EBADF
AT_FDCWD is not a valid file descriptor, so POSIX requires fstat to
fail with EBADF. if passed to fstatat, the call would spuriously
succeed and return results for the working directory.
remove mips/n32/64 stat struct hacks from syscall machinery
now that we have a kstat structure decoupled from the public struct
stat, we can just use the broken kernel structures directly and let
the code in fstatat do the translation.
presently, all archs/ABIs have struct stat matching the kernel
stat[64] type, except mips/mipsn32/mips64 which do conversion hacks in
syscall_arch.h to work around bugs in the kernel type. this patch
completely decouples them and adds a translation step to the success
path of fstatat. at present, this is just a gratuitous copying, but it
opens up multiple possibilities for future support for 64-bit time_t
on 32-bit archs and for cleaned-up/unified ABIs.
for clarity, the mips hacks are not yet removed in this commit, so the
mips kstat structs still correspond to the output of the hacks in
their syscall_arch.h files, not the raw kernel type. a subsequent
commit will fix this.
equivalent logic for fstat+O_PATH fallback and direct use of
stat/lstat syscalls where appropriate is kept, now in the fstatat
function. this change both improves functionality (now, fstatat forms
equivalent to fstat/lstat/stat will work even on kernels too old to
have the at functions) and localizes direct interfacing with the
kernel stat structure to one file.
remove utterly wrong includes from mips64/n32 bits/stat.h
these were overlooked during review. bits headers are not allowed to
pull in additional headers (note: that rule is currently broken in
other places but just for endian.h). string.h has no place here
anyway, and including bits/alltypes.h without defining macros to
request types from it is a nop.
use register constraint instead of memory operand for riscv64 atomics
the "A" constraint is simply for an address expression that's a single
register, but it's not yet supported by clang, and has no advantage
here over just using a register operand for the address. the latter is
actually preferable in the a_cas_p case because it avoids aliasing an
lvalue onto the memory.
most egregious problem was the lack of memory clobber and lack of
volatile asm; this made the atomics memory barriers but not compiler
barriers. use of "+r" rather than "=r" for a clobbered temp was also
wrong, since the initial value is indeterminate.
having "+r"(a0) is redundant with "0"(a0) in syscalls with at least 1
arg, which is arguably a constraint violation (clang treats it as
such), and an invalid input with indeterminate value in the 0-arg
case. use the "=r"(a0) form instead.
fix broken lseek on mipsn32 with offsets larger than LONG_MAX
mips n32 has 32-bit long, and generally uses long syscall arguments
and return values, but provides only SYS_lseek, not SYS_llseek. we
have some framework (syscall_arg_t, added for x32) to make syscall
arguments 64-bit in such a setting, but it's not clear whether this
could match the sign-extension semantics needed for 32-bit args to all
the other syscalls, and we don't have any existing mechanism to allow
the return value of syscalls to be something other than long.
instead, just provide a custom mipsn32 version of the lseek function
doing its own syscall asm with 64-bit arguments. as a result of commit 03919b26ed41c31876db41f7cee076ced4513fad, stdio will also get the new
code, fixing fseeko/ftello too.
ever since inline syscalls were added for (o32) mips in commit 328810d32524e4928fec50b57e37e1bf330b2e40, the asm has nonsensically
loaded the syscall number, rather than taking $2 as an input
constraint to let the compiler load it. commit cfc09b1ecf0c6981494fd73dffe234416f66af10 improved on this somewhat by
allowing a constant syscall number to propagate into an immediate, but
missed that the whole operation made no sense.
now, only $4, $5, $6, $8, and $9 are potential input-only registers.
$2 is always input and output, and $7 is both when it's an argument,
otherwise output-only. previously, $7 was treated as an input (with a
"1" constraint matching its output position) even when it was not an
input, which was arguably undefined behavior (asm input from
indeterminate value). this is corrected.
this patch is not purely non-functional changes, since before, $8 and
$9 were wrongly in the clobberlist for syscalls with fewer than 5 or 6
arguments. of course it's impossible for syscalls to have different
clobbers depending on their number of arguments. the clobberlist for
the recently-added 5- and 6-argument forms was correct, and for the 0-
to 4-argument forms was erroneously copied from the mips o32 ABI where
the additional arguments had to be passed on the stack.
in making this change, I reviewed the kernel sources, and $8 and $9
are always saved for 64-bit kernels since they're part of the syscall
argument list for n32 and n64 ABIs.
use namespace-safe __lseek for __stdio_seek instead of direct syscall
this probably saves a few bytes, avoids duplicating the clunky
lseek/_llseek syscall convention in two places, and sets the stage for
fixing broken seeks on x32 and mipsn32.
as before my aim was adding everyone with either substantial code
contributions or a pattern of ongoing simple patch submission; any
omissions are unintentional.
fix build failure on arm building C code in thumb1 mode
a fully thumb1 build is not supported because some asm files are
incompatible with thumb1, but apparently it works to compile the C
code as thumb1
commit 06fbefd10046a0fae7e588b7c6d25fb51811b931 caused this regression
but introducing use of the clz instruction, which is not supported in
arm mode prior to v5, and not supported in thumb prior to thumb2
(v6t2). commit 1b9406b03c0a94ebe2076a8fc1746a8c45e78a83 fixed the
issue only for arm mode pre-v5 but left thumb1 broken.
Samuel Holland [Sat, 29 Jun 2019 23:19:05 +0000 (18:19 -0500)]
use the correct attributes for ___errno_location
In the public header, __errno_location is declared with the "const"
attribute, conditional on __GNUC__. Ensure that its internal alias has
the same attributes.
Maintainer's note: This change also fixes a regression in quality of
code generation -- multiple references to errno in a single function
started generating multiple calls again -- introduced by commit e13063aad7aee341d278d2a879a76ec7b59b2ad8.
Samuel Holland [Sat, 29 Jun 2019 23:19:06 +0000 (18:19 -0500)]
fix conflicting mips and powerpc definitions for TIOCSER_TEMT macro
Commit 3517d74a5e04a377192d1f4882ad6c8dc22ce69a changed the token in
sys/ioctl.h from 0x01 to 1, so bits/termios.h no longer matches. Revert
the bits/termios.h change to keep the headers in sync.
Samuel Holland [Sat, 29 Jun 2019 23:19:04 +0000 (18:19 -0500)]
fix restrict violations in internal use of several functions
The old/new parameters to pthread_sigmask, sigprocmask, and setitimer
are marked restrict, so passing the same address to both is
prohibited. Modify callers of these functions to use a separate object
for each argument.
prevent dup2 action for posix_spawn internal pipe fd
as reported by Tavian Barnes, a dup2 file action for the internal pipe
fd used by posix_spawn could cause it to remain open after execve and
allow the child to write an artificial error into it, confusing the
parent. POSIX allows internal use of file descriptors by the
implementation, with undefined behavior for poking at them, so this is
not a conformance problem, but it seems preferable to diagnose and
prevent the error when we can do so easily.
catch attempts to apply a dup2 action to the internal pipe fd and
emulate EBADF for it instead.
fix inadvertent use of uninitialized variable in dladdr
commit c8b49b2fbc7faa8bf065220f11963d76c8a2eb93 introduced code that
checked bestsym to determine whether a matching symbol was found, but
bestsym is uninitialized if not. instead use best, consistent with use
in the rest of the function.
simplified from bug report and patch by Cheng Liu.
remove spurious MAP_32BIT definition from riscv64 arch
this was apparently copied from x86_64; it's not part of the kernel
API for riscv64. this change eliminates the need for a
riscv64-specific bits header and lets it use the generic one.
Szabolcs Nagy [Thu, 9 May 2019 20:44:27 +0000 (20:44 +0000)]
add new syscall numbers from linux v5.1
syscall numbers are now synced up across targets (starting from 403 the
numbers are the same on all targets other than an arch specific offset)
IPC syscalls sem*, shm*, msg* got added where they were missing (except
for semop: only semtimedop got added), the new semctl, shmctl, msgctl
imply IPC_64, see
a new syscall got added that uses the fd of /proc/<pid> as a stable
handle for processes: allows sending signals without pid reuse issues,
intended to eventually replace rt_sigqueueinfo, kill, tgkill and
rt_tgsigqueueinfo, see
Szabolcs Nagy [Thu, 9 May 2019 22:49:28 +0000 (22:49 +0000)]
ipc: prefer SYS_ipc when it is defined
Linux v5.1 introduced ipc syscalls on targets where previously only
SYS_ipc was available, change the logic such that the ipc code keeps
using SYS_ipc which works backward compatibly on older kernels.
This changes behaviour on microblaze which had both mechanisms, now
SYS_ipc will be used instead of separate syscalls.
Szabolcs Nagy [Thu, 9 May 2019 18:51:53 +0000 (18:51 +0000)]
fcntl.h: add F_SEAL_FUTURE_WRITE from linux v5.1
needed for android so it can migrate from its ashmem to memfd.
allows making the memfd readonly for future users while keeping
a writable mmap of it. see
Samuel Holland [Mon, 1 Jul 2019 03:44:28 +0000 (22:44 -0500)]
fix deadlock in synccall after threaded fork
synccall may be called by AS-safe functions such as setuid/setgid after
fork. although fork() resets libc.threads_minus_one, causing synccall to
take the single-threaded path, synccall still takes the thread list
lock. This lock may be held by another thread if for example fork()
races with pthread_create(). After fork(), the value of the lock is
meaningless, so clear it.
Rich Felker [Fri, 28 Jun 2019 21:58:03 +0000 (17:58 -0400)]
cap getdents length argument to INT_MAX
the linux syscall treats this argument as having type int, so passing
extremely long buffer sizes would be misinterpreted by the kernel.
since "short reads" are always acceptable, just cap it down.
patch based on report and suggested change by Florian Weimer.
Rich Felker [Tue, 25 Jun 2019 22:50:05 +0000 (18:50 -0400)]
remove unnecessary and problematic _Noreturn from crt/ldso startup
after commit a48ccc159a5fa061a18419296100ee48a1cd6cc9 removed the use
of _Noreturn on the stage3_func type (which only worked due to it
being defined to the "GNU C" attribute in C99 mode), GCC could no
longer assume that the ends of __dls2 and __dls2b are unreachable, and
produced a warning that a function marked _Noreturn returns.
also, since commit 4390383b32250a941ec616e8bff6f568a801b1c0, the
_Noreturn declaration for __libc_start_main in crt1/rcrt1 has been not
only inconsistent with the definition, but wrong. formally,
__libc_start_main does return, via a (hopefully) tail call to a helper
function after the barrier. incorrect usage of _Noreturn in the
declaration was probably formal UB.
the _Noreturn specifiers were not useful in any of these places, so
remove them all. now, the only remaining usage of _Noreturn is in
public interfaces where _Noreturn is part of their contract.
Rich Felker [Tue, 25 Jun 2019 21:47:12 +0000 (17:47 -0400)]
allow fmemopen with zero size
previously, POSIX erroneously required this to fail with EINVAL
despite the traditional glibc implementation, on which the POSIX
interface was based, allowing it. the resolution of Austin Group issue
818 removes the requirement to fail.
Rich Felker [Fri, 21 Jun 2019 19:49:38 +0000 (15:49 -0400)]
remove implicit include of sys/sysmacros.h from sys/types.h
this reverts commit f552c792c7ce5a560f214e1104d93ee5b0833967, which
exposed the sysmacros.h macros (device major/minor calculations) for
BSD and GNU profiles to mimic an unintentional glibc behavior some
code depended on. glibc has deprecated and since removed them as the
resolution to bug #19239, so it makes no sense for us to keep this
behavior. affected code should all have been fixed by now, and if it's
not yet fixed it needs to be for use with modern glibc anyway.
Rich Felker [Fri, 24 May 2019 14:46:08 +0000 (10:46 -0400)]
add riscv64 architecture support
Author: Alex Suykov <alex.suykov@gmail.com>
Author: Aric Belsito <lluixhi@gmail.com>
Author: Drew DeVault <sir@cmpwn.com>
Author: Michael Clark <mjc@sifive.com>
Author: Michael Forney <mforney@mforney.org>
Author: Stefan O'Rear <sorear2@gmail.com>
This port has involved the work of many people over several years. I
have tried to ensure that everyone with substantial contributions has
been credited above; if any omissions are found they will be noted
later in an update to the authors/contributors list in the COPYRIGHT
file.
The version committed here comes from the riscv/riscv-musl repo's
commit 3fe7e2c75df78eef42dcdc352a55757729f451e2, with minor changes by
me for issues found during final review:
- a_ll/a_sc atomics are removed (according to the ISA spec, lr/sc
are not safe to use in separate inline asm fragments)
- a_cas[_p] is fixed to be a memory barrier
- the call from the _start assembly into the C part of crt1/ldso is
changed to allow for the possibility that the linker does not place
them nearby each other.
- DTP_OFFSET is defined correctly so that local-dynamic TLS works
- reloc.h LDSO_ARCH logic is simplified and made explicit.
- unused, non-functional crti/n asm files are removed.
- an empty .sdata section is added to crt1 so that the
__global_pointer reference is resolvable.
- indentation style errors in some asm files are fixed.
Rich Felker [Sun, 26 May 2019 23:27:20 +0000 (19:27 -0400)]
optimize aarch64 dynamic tlsdesc function to spill fewer registers
with the glibc generation counter model for reusing dynamic tls slots
after dlclose, it's really not possible to get away with fewer than 4
working registers. for us however it's always been possible, but
tricky, and only became apparent after the switch to installing new
dynamic tls at dlopen time. by merging the negated thread pointer into
the addend early, the register holding the thread pointer can
immediately be reused, bringing the working register count down to
three. this allows saving/restoring via a single stp/ldp pair, since
the return register x0 does not need to be saved.
net reduction of 3 instructions, 2 of which were push/pop.
Rich Felker [Wed, 22 May 2019 22:28:32 +0000 (18:28 -0400)]
make powerpc64 vrregset_t logical layout match expected API
between v2 and v3 of the powerpc64 port patch, the change was made
from a 32x4 array of 32-bit unsigned ints for vrregs[] to a 32-element
array of __int128. this mismatches the API applications working with
mcontext_t expect from glibc, and seems to have been motivated by a
misinterpretation of a comment on how aarch64 did things as a
suggestion to do the same on powerpc64.
Rich Felker [Wed, 22 May 2019 19:17:12 +0000 (15:17 -0400)]
fix vrregset_t layout and member naming on powerpc64
the mistaken layout seems to have been adapted from 32-bit powerpc,
where vscr and vrsave are packed into the same 128-bit slot in a way
that looks like it relies on non-overlapping-ness of the value bits in
big endian.
the powerpc64 port accounted for the fact that the 64-bit ABI puts
each in its own 128-bit slot, but ordered them incorrectly (matching
the bit order used on the 32-bit ABI), and failed to account for vscr
being padded according to endianness so that it can be accessed via
vector moves.
in addition to ABI layout, our definition used different logical
member layout/naming from glibc, where vscr is a structure to
facilitate access as a 32-bit word or a 128-bit vector. the
inconsistency here was unintentional, so fix it.
Szabolcs Nagy [Mon, 13 May 2019 18:47:11 +0000 (18:47 +0000)]
fix tls offsets when p_vaddr%p_align != 0 on TLS_ABOVE_TP targets
currently the bfd linker does not seem to create tls segments where
p_vaddr%p_align != 0, but this is valid in ELF and then the runtime
computed tls offset must satisfy
offset%p_align == (base+p_vaddr)%p_align
and in case of local exec tls (main executable) the smallest such
offset must be used (otherwise it is incompatible with the offset
computed by the static linker). the !TLS_ABOVE_TP case is handled
correctly (the offset is negative then in the formula).
the ldso code for TLS_ABOVE_TP is changed so the static tls offset
of each module satisfies the formula.
Szabolcs Nagy [Thu, 16 May 2019 17:15:33 +0000 (17:15 +0000)]
fix static tls offsets of shared libs on TLS_ABOVE_TP targets
tls_offset should always point to the end of the allocated static tls
area, but this was not handled correctly on "tls variant 1" targets
in the dynamic linker:
after application tls was allocated, tls_offset was aligned up,
potentially wasting tls space. (alignment may be needed at the
begining of the tls area, not at the end, but that will be fixed
separately as it is unlikely to affect real binaries.)
when static tls was allocated for a shared library, tls_offset was
only updated with the size of the tls segment which does not include
alignment gaps, which can easily happen if the tls size update for
one library leaves tls_offset misaligned for the next one. this can
cause oob access in __copy_tls or arbitrary breakage at tls access.
(the issue was observed on aarch64 with rust binaries)
Rich Felker [Thu, 16 May 2019 21:12:56 +0000 (17:12 -0400)]
fix format strings for uid/gid values in putpwent/putgrent
commit 648c3b4e18b2ce2b6af7d44783e42ca267ea49f5 omitted this change,
which is needed to be able to use uid/gid values greater than INT_MAX
with these interfaces. it fixes alpine linux bug #10460.
Rich Felker [Sat, 11 May 2019 23:44:21 +0000 (19:44 -0400)]
improve i386 inline syscall asm on non-broken compilers
we have to avoid using ebx unconditionally in asm constraints for
i386, because gcc 3 and 4 and possibly other simplistic compilers
(pcc?) implement PIC via making ebx a fixed-use register, and disallow
its use for anything else. rather than hard-coding knowledge of which
compilers work (at least gcc 5+ and clang), perform a configure test;
this should give us the good codegen on any new compilers we don't yet
know about.
swapping ebx and edx is kept for 1- and 2-arg syscalls because it
avoids having any spills/stack-frame at all in small functions. for
6-arg, if ebx is directly usable, the complex shuffling introduced in
commit c8798ef974d21c338a7d8d874a402978ffc6168e can be avoided, and
ebp can be loaded the same way ebx is in 5-arg syscalls for compilers
that don't support direct use of ebx.
Rich Felker [Sat, 11 May 2019 00:56:19 +0000 (20:56 -0400)]
fix regression in i386 inline syscall asm producing invalid code
commit 22e5bbd0deadcbd767864bd714e890b70e1fe1df inlined the i386
syscall mechanism, but wrongly assumed memory operands to the 5- and
6-argument syscall asm would be esp-based. however, nothing in the
constraints prevented them from being ebx- or ebp-based, and in those
cases, ebx and ebp could be clobbered before use of the memory operand
was complete. in the 6-argument case, this prevented restoration of
the original register values before the end of the asm block, breaking
the asm contract since ebx and ebp are not marked as clobbered. (they
can't be, because lots of compilers don't accept these registers in
constraints or clobbers if PIC or frame pointer is enabled).
doing this right is complicated by the fact that, after a single push,
no operands which might be memory operands are usable. if they are
esp-based, the value of esp has changed, rendering them invalid.
introduce some new dances to load the registers. for the 5-arg case,
push the operand that may be a memory operand first, and after that,
it doesn't matter if the operand is invalid, since we'll just use the
newly pushed value. for the 6-arg case, we need to put both operands
in memory to begin with, like the old non-inline code prior to commit 22e5bbd0deadcbd767864bd714e890b70e1fe1df accepted, so that there's
only one potentially memory-based operand to the asm. this can then be
saved with a single push, and after that the values can be read off
into the registers they're needed in.
there's some size overhead, but still a lot less execution overhead
than the old out-of-line code. doing it better depends on a modern
compiler that lets you use ebx and ebp in asm constraints without
restriction. the failure modes on compilers where this doesn't work
are inconsistent and dangerous (on at least some gcc versions 4.x and
earlier, wrong codegen!), so this is a delicate matter. it can be
addressed later if needed.
Rich Felker [Mon, 6 May 2019 02:50:57 +0000 (22:50 -0400)]
make fgetwc set error indicator for stream on encoding errors
this is a requirement in POSIX that's omitted, and seemed potentially
non-conforming, in the C standard. as such it was omitted here.
however, as part of Austin Group issue #1170, the discrepancy was
raised with WG14 and determined to be unintended; future versions of
the C standard will require the error indicator to be set, as POSIX
does.
Rich Felker [Sun, 5 May 2019 15:24:57 +0000 (11:24 -0400)]
fix broken posix_fadvise on mips due to missing 7-arg syscall support
commit 788d5e24ca19c6291cebd8d1ad5b5ed6abf42665 exposed the breakage
at build time by removing support for 7-argument syscalls; however,
the external __syscall function provided for mips before did not pass
a 7th argument from the stack, so the behavior was just silently
broken.
Rich Felker [Sun, 5 May 2019 15:15:23 +0000 (11:15 -0400)]
allow archs to provide a 7-argument syscall if needed
commit 788d5e24ca19c6291cebd8d1ad5b5ed6abf42665 noted that we could
add this if needed, and in fact it is needed, but not for one of the
archs documented as having a 7th syscall arg register. rather, it's
needed for mips (o32), where all but the first 4 arguments are passed
on the stack, and the stack can accommodate a 7th.
Rich Felker [Sun, 5 May 2019 14:52:41 +0000 (10:52 -0400)]
fix passing of 64-bit syscall arguments on microblaze
this has been wrong since the beginning of the microblaze port: the
syscall ABI for microblaze does not align 64-bit arguments on even
register boundaries. commit 788d5e24ca19c6291cebd8d1ad5b5ed6abf42665
exposed the problem by introducing references to a nonexistent
__syscall7. the ABI is not documented well anywhere, but I was able to
confirm against both strace source and glibc source that microblaze is
not using the alignment.
per the syscall(2) man page, posix_fadvise, ftruncate, pread, pwrite,
readahead, sync_file_range, and truncate were all affected and either
did not work at all, or only worked by chance, e.g. when the affected
argument slots were all zero.
make new math code compatible with unused variable warning/error
commit b50d315fd23f0fbc4c11e2583801dd123d933745 introduced
fp_force_eval implemented by default with a dead store to a volatile
variable. unfortunately introduces warnings with -Wunused-variable and
breaks the ability to use -Werror with the default warning options set
by configure when warnings are enabled.
we could just call fp_barrier instead, but that results in a spurious
load after the store due to volatile semantics.
the fix committed here avoids the load. it will still produce warnings
without -Wno-unused-but-set-variable, but that's part of our default
warning profile, and there are already other locations in the source
where an unused variable warning will occur without it.
Szabolcs Nagy [Sun, 2 Dec 2018 18:53:37 +0000 (18:53 +0000)]
math: add configuration macros
Musl currently aims to support non-nearest rounding mode and does not
support SNaNs. These macros allow marking relevant code paths in case
these decisions are changed later (they also help documenting the
corner cases involved).
Szabolcs Nagy [Sun, 22 Oct 2017 13:51:35 +0000 (13:51 +0000)]
math: add single precision error handling functions
These are supposed to be used in tail call positions when handling
special cases in new code. (fp exceptions may be raised "naturally"
by the common code path if special casing is more effort.)
This implements the error handling apis used in
https://github.com/ARM-software/optimized-routines
without errno setting.
Szabolcs Nagy [Sat, 1 Dec 2018 23:52:34 +0000 (23:52 +0000)]
math: add eval_as_float and eval_as_double
Previously type casts or assignments were used for handling excess
precision, which assumed standard C99 semantics, but since it's a
rarely needed obscure detail, it's better to use explicit helper
functions to document where we rely on this. It also helps if the
code is used outside of the libc in non-C99 compilation mode: with the
default excess precision handling of gcc, explicit inline asm barriers
are needed for narrowing on FLT_EVAL_METHOD!=0 targets.
I plan to use this in new code with the existing style that uses
double_t and float_t as much as possible.
One ugliness is that it is required for almost every return statement
since that does not drop excess precision (the standard changed this
in C11 annex F, but that does not help in non-standard compilation
modes or with old compilers).
Szabolcs Nagy [Mon, 26 Nov 2018 23:30:00 +0000 (23:30 +0000)]
math: add fp_arch.h with fp_barrier and fp_force_eval
C99 has ways to support fenv access, but compilers don't implement it
and assume nearest rounding mode and no fp status flag access. (gcc has
-frounding-math and then it does not assume nearest rounding mode, but
it still assumes the compiled code itself does not change the mode.
Even if the C99 mechanism was implemented it is not ideal: it requires
all code in the library to be compiled with FENV_ACCESS "on" to make it
usable in non-nearest rounding mode, but that limits optimizations more
than necessary.)
The math functions should give reasonable results in all rounding modes
(but the quality may be degraded in non-nearest rounding modes) and the
fp status flag settings should follow the spec, so fenv side-effects are
important and code transformations that break them should be prevented.
Unfortunately compilers don't give any help with this, the best we can
do is to add fp barriers to the code using volatile local variables
(they create a stack frame and undesirable memory accesses to it) or
inline asm (gcc specific, requires target specific fp reg constraints,
often creates unnecessary reg moves and multiple barriers are needed to
express that an operation has side-effects) or extern call (only useful
in tail-call position to avoid stack-frame creation and does not work
with lto).
We assume that in a math function if an operation depends on the input
and the output depends on it, then the operation will be evaluated at
runtime when the function is called, producing all the expected fenv
side-effects (this is not true in case of lto and in case the operation
is evaluated with excess precision that is not rounded away). So fp
barriers are needed (1) to prevent the move of an operation within a
function (in case it may be moved from an unevaluated code path into an
evaluated one or if it may be moved across a fenv access), (2) force the
evaluation of an operation for its side-effect when it has no input
dependency (may be constant folded) or (3) when its output is unused. I
belive that fp_barrier and fp_force_eval can take care of these and they
should not be needed in hot code paths.
Szabolcs Nagy [Thu, 29 Nov 2018 22:09:53 +0000 (22:09 +0000)]
math: move complex math out of libm.h
This makes it easier to build musl math code with a compiler that
does not support complex types (tcc) and in general more sensible
factorization of the internal headers.
Szabolcs Nagy [Tue, 16 Oct 2018 22:20:39 +0000 (22:20 +0000)]
define FP_FAST_FMA* when fma* can be inlined
FP_FAST_FMA can be defined if "the fma function generally executes about
as fast as, or faster than, a multiply and an add of double operands",
which can only be true if the fma call is inlined as an instruction.
gcc sets __FP_FAST_FMA if __builtin_fma is inlined as an instruction,
but that does not mean an fma call will be inlined (e.g. it is defined
with -fno-builtin-fma), other compilers (clang) don't even have such
macro, but this is the closest we can get.
(even if the libc fma implementation is a single instruction, the extern
call overhead is already too big when the macro is used to decide between
x*y+z and fma(x,y,z) so it cannot be based on libc only, defining the
macro unconditionally on targets which have fma in the base isa is also
incorrect: the compiler might not inline fma anyway.)
this solution works with gcc unless fma inlining is explicitly turned off.
A. Wilcox [Wed, 13 Mar 2019 16:16:11 +0000 (11:16 -0500)]
fcntl.h: define O_TTY_INIT to 0
POSIX: "[If] either O_TTY_INIT is set in oflag or O_TTY_INIT has the
value zero, open() shall set any non-standard termios structure
terminal parameters to a state that provides conforming behavior."
The Linux kernel tty drivers always perform initialisation on their
devices to set known good termios values during the open(2) call. This
means that setting O_TTY_INIT to zero is conforming.
remove external __syscall function and last remaining users
the weak version of __syscall_cp_c was using a tail call to __syscall
to avoid duplicating the 6-argument syscall code inline in small
static-linked programs, but now that __syscall no longer exists, the
inline expansion is no longer duplication.
the syscall.h machinery suppported up to 7 syscall arguments, only via
an external __syscall function, but we presently have no syscall call
points that actually make use of that many, and the kernel only
defines 7-argument calling conventions for arm, powerpc (32-bit), and
sh. if it turns out we need them in the future, they can easily be
added.
implement inline 5- and 6-argument syscalls for mipsn32 and mips64
n32 and n64 ABIs add new argument registers vs o32, so that passing on
the stack is not necessary, so it's not clear why the 5- and
6-argument versions were special-cased to begin with; it seems to have
been pattern-copying from arch/mips (o32).
i've treated the new argument registers like the first 4 in terms of
clobber status (non-clobbered). hopefully this is correct.
implement inline 5- and 6-argument syscalls for mips
the OABI passes these on the stack, using the convention that their
position on the stack is as if the first four arguments (in registers)
also had stack slots. originally this was deemed too awkward to do
inline, falling back to external __syscall, but it's not that bad and
now that external __syscall is being removed, it's necessary.
projects / musl / log