For a given set of live values, the spill cost will always be the
same for each call. Compute the cost once and multiply it by the
number of calls.
(I'm not sure this spill cost modeling makes sense if there are
multiple calls, as the spill cost will likely be shared across
calls in that case. But that's how it currently works.)
hwasan: Improve precision of checks using short granule tags.
A short granule is a granule of size between 1 and `TG-1` bytes. The size
of a short granule is stored at the location in shadow memory where the
granule's tag is normally stored, while the granule's actual tag is stored
in the last byte of the granule. This means that in order to verify that a
pointer tag matches a memory tag, HWASAN must check for two possibilities:
* the pointer tag is equal to the memory tag in shadow memory, or
* the shadow memory tag is actually a short granule size, the value being loaded
is in bounds of the granule and the pointer tag is equal to the last byte of
the granule.
Pointer tags between 1 to `TG-1` are possible and are as likely as any other
tag. This means that these tags in memory have two interpretations: the full
tag interpretation (where the pointer tag is between 1 and `TG-1` and the
last byte of the granule is ordinary data) and the short tag interpretation
(where the pointer tag is stored in the granule).
When HWASAN detects an error near a memory tag between 1 and `TG-1`, it
will show both the memory tag and the last byte of the granule. Currently,
it is up to the user to disambiguate the two possibilities.
Because this functionality obsoletes the right aligned heap feature of
the HWASAN memory allocator (and because we can no longer easily test
it), the feature is removed.
Also update the documentation to cover both short granule tags and
outlined checks.
Philip Reames [Tue, 9 Jul 2019 19:59:39 +0000 (19:59 +0000)]
[PoisonChecking] Flesh out complete todo list for full coverage
Note: I don't actually plan to implement all of the cases at the moment, I'm just documenting them for completeness. There's a couple of cases left which are practically useful for me in debugging loop transforms, and I'll probably stop there for the moment.
[X86][AMDGPU][DAGCombiner] Move call to allowsMemoryAccess into isLoadBitCastBeneficial/isStoreBitCastBeneficial to allow X86 to bypass it
Basically the problem is that X86 doesn't set the Fast flag from
allowsMemoryAccess on certain CPUs due to slow unaligned memory
subtarget features. This prevents bitcasts from being folded into
loads and stores. But all vector loads and stores of the same width
are the same cost on X86.
This patch merges the allowsMemoryAccess call into isLoadBitCastBeneficial to allow X86 to skip it.
Philip Reames [Tue, 9 Jul 2019 19:26:12 +0000 (19:26 +0000)]
[PoisonCheker] Support for out of bounds operands on shifts + insert/extractelement
These are sources of poison which don't come from flags, but are clearly documented in the LangRef. Left off support for scalable vectors for the moment, but should be easy to add if anyone is interested.
Sean Fertile [Tue, 9 Jul 2019 19:21:01 +0000 (19:21 +0000)]
Boilerplate for producing XCOFF object files from the PowerPC backend.
Stubs out a number of the classes needed to produce a new object file format
(XCOFF) for the powerpc-aix target. For testing input is an empty module which
produces an object file with just a file header.
Bob Haarman [Tue, 9 Jul 2019 18:50:55 +0000 (18:50 +0000)]
[ThinLTO] only emit used or referenced CFI records to index
Summary: We emit CFI_FUNCTION_DEFS and CFI_FUNCTION_DECLS to
distributed ThinLTO indices to implement indirect function call
checking. This change causes us to only emit entries for functions
that are either defined or used by the module we're writing the index
for (instead of all functions in the combined index), which can make
the indices substantially smaller.
Philip Reames [Tue, 9 Jul 2019 18:49:29 +0000 (18:49 +0000)]
Add a transform pass to make the executable semantics of poison explicit in the IR
Implements a transform pass which instruments IR such that poison semantics are made explicit. That is, it provides a (possibly partial) executable semantics for every instruction w.r.t. poison as specified in the LLVM LangRef. There are obvious parallels to the sanitizer tools, but this pass is focused purely on the semantics of LLVM IR, not any particular source language.
The target audience for this tool is developers working on or targetting LLVM from a frontend. The idea is to be able to take arbitrary IR (with the assumption of known inputs), and evaluate it concretely after having made poison semantics explicit to detect cases where either a) the original code executes UB, or b) a transform pass introduces UB which didn't exist in the original program.
At the moment, this is mostly the framework and still needs to be fleshed out. By reusing existing code we have decent coverage, but there's a lot of cases not yet handled. What's here is good enough to handle interesting cases though; for instance, one of the recent LFTR bugs involved UB being triggered by integer induction variables with nsw/nuw flags would be reported by the current code.
(See comment in PoisonChecking.cpp for full explanation and context)
Sean Fertile [Tue, 9 Jul 2019 18:44:28 +0000 (18:44 +0000)]
Try to appease the Windows build bots.
Several of the conditonal operators commited in llvm-svn: 365524 fail to compile
on the windows buildbots. Converting to an if and early return to try to fix.
Sean Fertile [Tue, 9 Jul 2019 18:09:11 +0000 (18:09 +0000)]
[Object][XCOFF] Add support for 64-bit file header and section header dumping.
Adds a readobj dumper for 32-bit and 64-bit section header tables, and extend
support for the file-header dumping to include 64-bit object files. Also
refactors the binary file parsing to be done in a helper function in an attempt
to cleanup error handeling.
I'm not sure if transforming any of these is valid as
a target-independent fold, but we might as well have
a few tests here to confirm or deny our position.
- The test had extension .yaml, which lit doesn't execute in this
directory. Rename to .test to make it run, and move the yaml bits
into a dedicated file, like with all other tests in this dir.
- llvm-pdbdump got renamed to llvm-pdbutil long ago, update test.
- -dbi-module-info got renamed in r305032, update test for this too.
[AMDGPU] Created a sub-register class for the return address operand in the return instruction.
Function return instruction lowering, currently uses the fixed register pair s[30:31] for holding
the return address. It can be any SGPR pair other than the CSRs. Created an SGPR pair sub-register class
exclusive of the CSRs, and used this regclass while lowering the return instruction.
Sam Elliott [Tue, 9 Jul 2019 16:24:16 +0000 (16:24 +0000)]
[RISCV] Fix ICE in isDesirableToCommuteWithShift
Summary:
There was an error being thrown from isDesirableToCommuteWithShift in
some tests. This was tracked down to the method being called before
legalisation, with an extended value type, not a machine value type.
In the case I diagnosed, the error was only hit with an instruction sequence
involving `i24`s in the add and shift. `i24` is not a Machine ValueType, it is
instead an Extended ValueType which was causing the issue.
I have added a test to cover this case, and fixed the error in the callback.
[AArch64][GlobalISel] Optimize conditional branches followed by unconditional branches
If we have an icmp->brcond->br sequence where the brcond just branches to the
next block jumping over the br, while the br takes the false edge, then we can
modify the conditional branch to jump to the br's target while inverting the
condition of the incoming icmp. This means we can eliminate the br as an
unconditional branch to the fallthrough block.
Yonghong Song [Tue, 9 Jul 2019 15:28:41 +0000 (15:28 +0000)]
[BPF] Support for compile once and run everywhere
Introduction
============
This patch added intial support for bpf program compile once
and run everywhere (CO-RE).
The main motivation is for bpf program which depends on
kernel headers which may vary between different kernel versions.
The initial discussion can be found at https://lwn.net/Articles/773198/.
Currently, bpf program accesses kernel internal data structure
through bpf_probe_read() helper. The idea is to capture the
kernel data structure to be accessed through bpf_probe_read()
and relocate them on different kernel versions.
On each host, right before bpf program load, the bpfloader
will look at the types of the native linux through vmlinux BTF,
calculates proper access offset and patch the instruction.
To accommodate this, three intrinsic functions
preserve_{array,union,struct}_access_index
are introduced which in clang will preserve the base pointer,
struct/union/array access_index and struct/union debuginfo type
information. Later, bpf IR pass can reconstruct the whole gep
access chains without looking at gep itself.
This patch did the following:
. An IR pass is added to convert preserve_*_access_index to
global variable who name encodes the getelementptr
access pattern. The global variable has metadata
attached to describe the corresponding struct/union
debuginfo type.
. An SimplifyPatchable MachineInstruction pass is added
to remove unnecessary loads.
. The BTF output pass is enhanced to generate relocation
records located in .BTF.ext section.
Typical CO-RE also needs support of global variables which can
be assigned to different values to different hosts. For example,
kernel version can be used to guard different versions of codes.
This patch added the support for patchable externals as well.
Example
=======
The following is an example.
struct pt_regs {
long arg1;
long arg2;
};
struct sk_buff {
int i;
struct net_device *dev;
};
#define _(x) (__builtin_preserve_access_index(x))
static int (*bpf_probe_read)(void *dst, int size, const void *unsafe_ptr) =
(void *) 4;
extern __attribute__((section(".BPF.patchable_externs"))) unsigned __kernel_version;
int bpf_prog(struct pt_regs *ctx) {
struct net_device *dev = 0;
// ctx->arg* does not need bpf_probe_read
if (__kernel_version >= 41608)
bpf_probe_read(&dev, sizeof(dev), _(&((struct sk_buff *)ctx->arg1)->dev));
else
bpf_probe_read(&dev, sizeof(dev), _(&((struct sk_buff *)ctx->arg2)->dev));
return dev != 0;
}
In the above, we want to translate the third argument of
bpf_probe_read() as relocations.
-bash-4.4$ clang -target bpf -O2 -g -S trace.c
The compiler will generate two new subsections in .BTF.ext,
OffsetReloc and ExternReloc.
OffsetReloc is to record the structure member offset operations,
and ExternalReloc is to record the external globals where
only u8, u16, u32 and u64 are supported.
BPFOffsetReloc Size
struct SecLOffsetReloc for ELF section #1
A number of struct BPFOffsetReloc for ELF section #1
struct SecOffsetReloc for ELF section #2
A number of struct BPFOffsetReloc for ELF section #2
...
BPFExternReloc Size
struct SecExternReloc for ELF section #1
A number of struct BPFExternReloc for ELF section #1
struct SecExternReloc for ELF section #2
A number of struct BPFExternReloc for ELF section #2
struct BPFOffsetReloc {
uint32_t InsnOffset; ///< Byte offset in this section
uint32_t TypeID; ///< TypeID for the relocation
uint32_t OffsetNameOff; ///< The string to traverse types
};
struct BPFExternReloc {
uint32_t InsnOffset; ///< Byte offset in this section
uint32_t ExternNameOff; ///< The string for external variable
};
Note that only externs with attribute section ".BPF.patchable_externs"
are considered for Extern Reloc which will be patched by bpf loader
right before the load.
For the above test case, two offset records and one extern record
will be generated:
OffsetReloc records:
.long .Ltmp12 # Insn Offset
.long 7 # TypeId
.long 242 # Type Decode String
.long .Ltmp18 # Insn Offset
.long 7 # TypeId
.long 242 # Type Decode String
For instruction .Ltmp12 and .Ltmp18, "r2 = 8", the number
8 is the structure offset based on the current BTF.
Loader needs to adjust it if it changes on the host.
For instruction .Ltmp5, "r2 = 0", the external variable
got a default value 0, loader needs to supply an appropriate
value for the particular host.
Petar Avramovic [Tue, 9 Jul 2019 14:36:17 +0000 (14:36 +0000)]
[MIPS GlobalISel] Register bank select for G_PHI. Select i64 phi
Select gprb or fprb when def/use register operand of G_PHI is
used/defined by either:
copy to/from physical register or
instruction with only one mapping available for that use/def operand.
Integer s64 phi is handled with narrowScalar when mapping is applied,
produced artifacts are combined away. Manually set gprb to all register
operands of instructions created during narrowScalar.
Matt Arsenault [Tue, 9 Jul 2019 14:30:57 +0000 (14:30 +0000)]
AMDGPU/GlobalISel: Prepare some tests for store selection
Mostsly these would fail due to trying to use SI with a flat
operation. Implementing global loads with MUBUF is more work than
flat, so these won't be handled in the initial load selection.
Others fail because store of s64 won't initially work, as the current
set of patterns expect everything to be turned into v2i32.
Petar Avramovic [Tue, 9 Jul 2019 14:30:29 +0000 (14:30 +0000)]
[MIPS GlobalISel] Regbanks for G_SELECT. Select i64, f32 and f64 select
Select gprb or fprb when def/use register operand of G_SELECT is
used/defined by either:
copy to/from physical register or
instruction with only one mapping available for that use/def operand.
Integer s64 select is handled with narrowScalar when mapping is applied,
produced artifacts are combined away. Manually set gprb to all register
operands of instructions created during narrowScalar.
For selection of floating point s32 or s64 select it is enough to set
fprb of appropriate size and selectImpl will do the rest.
Simon Atanasyan [Tue, 9 Jul 2019 12:55:55 +0000 (12:55 +0000)]
[mips] Implement sge/sgeu pseudo instructions
The `sge/sgeu Dst, Src1, Src2/Imm` pseudo instructions set register
`Dst` to 1 if register `Src1` is greater than or equal `Src2/Imm` and
to 0 otherwise.
Tim Northover [Tue, 9 Jul 2019 11:35:35 +0000 (11:35 +0000)]
OpaquePtr: add Type parameter to Loads analysis API.
This makes the functions in Loads.h require a type to be specified
independently of the pointer Value so that when pointers have no structure
other than address-space, it can still do its job.
Most callers had an obvious memory operation handy to provide this type, but a
SROA and ArgumentPromotion were doing more complicated analysis. They get
updated to merge the properties of the various instructions they were
considering.
Dump the DWARF information about call sites and call site parameters into
debug info sections.
The patch also provides an interface for the interpretation of instructions
that could load values of a call site parameters in order to generate DWARF
about the call site parameters.
([13/13] Introduce the debug entry values.)
Co-authored-by: Ananth Sowda <asowda@cisco.com> Co-authored-by: Nikola Prica <nikola.prica@rt-rk.com> Co-authored-by: Ivan Baev <ibaev@cisco.com>
Differential Revision: https://reviews.llvm.org/D60716
Alex Bradbury [Tue, 9 Jul 2019 10:56:18 +0000 (10:56 +0000)]
[RISCV] Fix RISCVTTIImpl::getIntImmCost for immediates where getMinSignedBits() > 64
APInt::getSExtValue will assert if getMinSignedBits() > 64. This can happen,
for instance, if examining an i128. Avoid this assertion by checking
Imm.getMinSignedBits() <= 64 before doing
getTLI()->isLegalAddImmediate(Imm.getSExtValue()). We could directly check
getMinSignedBits() <= 12 but it seems better to reuse the isLegalAddImmediate
helper for this.
[LegalizeTypes] Fix saturation bug for smul.fix.sat
Summary:
Make sure we use SETGE instead of SETGT when checking
if the sign bit is zero at SMULFIXSAT expansion.
The faulty expansion occured when doing "expand" of
SMULFIXSAT and the scale was exactly matching the
size of the smaller type. For example doing
i64 Z = SMULFIXSAT X, Y, 32
and expanding X/Y/Z into using two i32 values.
The problem was that we sometimes did not saturate
to min when overflowing.
Here is an example using Q3.4 numbers:
Consider that we are multiplying X and Y.
X = 0x80 (-8.0 as Q3.4)
Y = 0x20 (2.0 as Q3.4)
To avoid loss of precision we do a widening
multiplication, getting a 16 bit result
Z = 0xF000 (-16.0 as Q7.8)
To detect negative overflow we should check if
the five most significant bits in Z are less than -1.
Assume that we name the 4 most significant bits
as HH and the next 4 bits as HL. Then we can do the
check by examining if
(HH < -1) or (HH == -1 && "sign bit in HL is zero").
The fault was that we have been doing the check as
(HH < -1) or (HH == -1 && HL > 0)
instead of
(HH < -1) or (HH == -1 && HL >= 0).
In our example HH is -1 and HL is 0, so the old
code did not trigger saturation and simply truncated
the result to 0x00 (0.0). With the bugfix we instead
detect that we should saturate to min, and the result
will be set to 0x80 (-8.0).
Reland "[LiveDebugValues] Emit the debug entry values"
Emit replacements for clobbered parameters location if the parameter
has unmodified value throughout the funciton. This is basic scenario
where we can use the debug entry values.
([12/13] Introduce the debug entry values.)
Co-authored-by: Ananth Sowda <asowda@cisco.com> Co-authored-by: Nikola Prica <nikola.prica@rt-rk.com> Co-authored-by: Ivan Baev <ibaev@cisco.com>
Differential Revision: https://reviews.llvm.org/D58042
[Loop Peeling] Add support for peeling of loops with multiple exits
This patch modifies the loop peeling transformation so that
it does not expect that there is only one loop exit from latch.
It modifies only transformation. Update of branch weights remains
only for exit from latch.
The motivation is that in follow-up patch I plan to enable loop peeling for
loops with multiple exits but only if other exits then from latch one goes to
block with call to deopt.
Prepare for making SwitchInstProfUpdateWrapper strict
This patch removes the test part that relates to the non-strict
behavior of SwitchInstProfUpdateWrapper and changes
the assertion to llvm_unreachable() to allow the check in
release builds.
This patch prepares SwitchInstProfUpdateWrapper to become
strict with one line change. That is need to revert it easily
if any failure will arise.
[LoopInfo] Update getExitEdges to accept vector of pairs for non const BasicBlock
D63921 requires getExitEdges fills a vector of Edge pairs where
BasicBlocks are not constant.
The rest Loop API mostly returns non-const BasicBlocks, so to be more consistent with
other Loop API getExitEdges is modified to return non-const BasicBlocks as well.
Kai Luo [Tue, 9 Jul 2019 02:55:08 +0000 (02:55 +0000)]
[PowerPC][Peephole] Combine extsw and sldi after instruction selection
Summary:
`extsw` and `sldi` are supposed to be combined if they are in the same
BB in instruction selection phase. This patch handles the case where
extsw and sldi are not in the same BB.
Jinsong Ji [Tue, 9 Jul 2019 02:27:35 +0000 (02:27 +0000)]
[MachinePipeliner] Fix Phi refers to Phi in same stage in 1st epilogue
Summary:
This is exposed by functional testing on PowerPC.
In some pipelined loops, Phi refer to phi did not get value defined by
the Phi, hence getting wrong value later.
As the comment mentioned, we should "use the value defined by the Phi,
unless we're generating the firstepilog and the Phi refers to a Phi
in a different stage.", so Phi refering to same stage Phi should use
the value defined by the Phi here.
[WebAssembly] Make sret parameter work with AddMissingPrototypes
Summary:
Even with functions with `no-prototype` attribute, there can be an
argument `sret` (structure return) attribute, which is an optimization
when a function return type is a struct. Fixes PR42420.
Yonghong Song [Tue, 9 Jul 2019 01:51:36 +0000 (01:51 +0000)]
[BPF] add new intrinsics preserve_{array,union,struct}_access_index
For background of BPF CO-RE project, please refer to
http://vger.kernel.org/bpfconf2019.html
In summary, BPF CO-RE intends to compile bpf programs
adjustable on struct/union layout change so the same
program can run on multiple kernels with adjustment
before loading based on native kernel structures.
In order to do this, we need keep track of GEP(getelementptr)
instruction base and result debuginfo types, so we
can adjust on the host based on kernel BTF info.
Capturing such information as an IR optimization is hard
as various optimization may have tweaked GEP and also
union is replaced by structure it is impossible to track
fieldindex for union member accesses.
Three intrinsic functions, preserve_{array,union,struct}_access_index,
are introducted.
addr = preserve_array_access_index(base, index, dimension)
addr = preserve_union_access_index(base, di_index)
addr = preserve_struct_access_index(base, gep_index, di_index)
here,
base: the base pointer for the array/union/struct access.
index: the last access index for array, the same for IR/DebugInfo layout.
dimension: the array dimension.
gep_index: the access index based on IR layout.
di_index: the access index based on user/debuginfo types.
For example, for the following example,
$ cat test.c
struct sk_buff {
int i;
int b1:1;
int b2:2;
union {
struct {
int o1;
int o2;
} o;
struct {
char flags;
char dev_id;
} dev;
int netid;
} u[10];
};
static int (*bpf_probe_read)(void *dst, int size, const void *unsafe_ptr)
= (void *) 4;
Note that @llvm.preserve.{struct,union}.access.index calls have metadata llvm.preserve.access.index
attached to instructions to provide struct/union debuginfo type information.
For &ctx->u[5].dev.dev_id,
. The "%6 = ..." represents struct member "u" with index 2 for IR layout and index 3 for DI layout.
. The "%7 = ..." represents array subscript "5".
. The "%8 = ..." represents union member "dev" with index 1 for DI layout.
. The "%10 = ..." represents struct member "dev_id" with index 1 for both IR and DI layout.
Basically, traversing the use-def chain recursively for the 3rd argument of bpf_probe_read() and
examining all preserve_*_access_index calls, the debuginfo struct/union/array access index
can be achieved.
The intrinsics also contain enough information to regenerate codes for IR layout.
For array and structure intrinsics, the proper GEP can be constructed.
For union intrinsics, replacing all uses of "addr" with "base" should be enough.
The test case ThinLTO/X86/lazyload_metadata.ll is adjusted to reflect the
new addition of the metadata.
Signed-off-by: Yonghong Song <yhs@fb.com>
Differential Revision: https://reviews.llvm.org/D61810
Philip Reames [Tue, 9 Jul 2019 01:27:45 +0000 (01:27 +0000)]
[LoopPred] Extend LFTR normalization to the inverse EQ case
A while back, I added support for NE latches formed by LFTR. I didn't think that quite through, as LFTR will also produce the inverse EQ form for some loops and I hadn't handled that. This change just adds handling for that case as well.
Let unaliased Args track which Alias they were created from, and use that in Arg::getAsString() for diagnostics
With this, `clang-cl /source-charset:utf-16 test.cc` now prints `invalid
value 'utf-16' in '/source-charset:utf-16'` instead of `invalid value
'utf-16' in '-finput-charset=utf-16'` before, and several other clang-cl
flags produce much less confusing output as well.
Fixes PR29106.
Since an arg and its alias can have different arg types (joined vs not)
and different values (because of AliasArgs<>), I chose to give the Alias
its own Arg object. For convenience, I just store the alias directly in
the unaliased arg – there aren't many arg objects at runtime, so that
seems ok.
Finally, I changed Arg::getAsString() to use the alias's representation
if it's present – that function was already documented as being the
suitable function for diagnostics, and most callers already used it for
diagnostics.
Implementation-wise, Arg::accept() previously used to parse things as
the unaliased option. The core of that switch is now extracted into a
new function acceptInternal() which parses as the _aliased_ option, and
the previously-intermingled unaliasing is now done as an explicit step
afterwards.
(This also changes one place in lld that didn't use getAsString() for
diagnostics, so that that one place now also prints the flag as the user
wrote it, not as it looks after it went through unaliasing.)
[Attributor] Deduce the "returned" argument attribute
Deduce the "returned" argument attribute by collecting all potentially
returned values.
Not only the unique return value, if any, can be used by subsequent
attributes but also the set of all potentially returned values as well
as the mapping from returned values to return instructions that they
originate from (see AAReturnedValues::checkForallReturnedValues).
Change in statistics (-stats) for LLVM-TS + Spec2006, totaling ~19% more "returned" arguments.
Rong Xu [Mon, 8 Jul 2019 21:03:12 +0000 (21:03 +0000)]
llvm-profdata] Handle the cases of overlapping input file and output file
Currently llvm-profdata does not expect the same file name for the input profile
and the output profile.
>llvm-profdata merge A.profraw B.profraw -o B.profraw
The above command runs successfully but the resulted B.profraw is not correct.
This patch fixes the issue by moving the initialization of writer after loading
the profile.
For the show command, the following will report a confusing error of
"Empty raw profile file":
>llvm-profdata show B.profraw -o B.profraw
It's harder to fix as we need to output something before loading the input profile.
I don't think that a fix for this is worth the effort. I just make the error explicit for
the show command.
[InstCombine] fold insertelement into splat of same scalar
Forming the canonical splat shuffle improves analysis and
may allow follow-on transforms (although some possibilities
are missing as shown in the test diffs).
The backend generically turns these patterns into build_vector,
so there should be no codegen regressions. All targets are
expected to be able to lower splats efficiently.
projects / llvm / log