John Brawn [Mon, 27 Nov 2017 11:29:15 +0000 (11:29 +0000)]
[CGP] Fix handling of null pointer values in optimizeMemoryInst
The current way that trivial addressing modes are detected incorrectly thinks
that null pointers are non-trivial, leading to an infinite loop where we keep
duplicating the same select. Fix this by aware of null when deciding if an
addressing mode is trivial.
Simon Pilgrim [Mon, 27 Nov 2017 10:41:32 +0000 (10:41 +0000)]
[X86][FMA] Tag all FMA/FMA4 instructions with WriteFMA schedule class
As mentioned on PR17367, many instructions are missing scheduling tags preventing us from setting 'CompleteModel = 1' for better instruction analysis. This patch deals with FMA/FMA4 which is one of the bigger offenders (along with AVX512 in general).
Annoyingly all scheduler models need to define WriteFMA (now that its actually used), even for older targets without FMA/FMA4 support, but that is an existing problem shared by other schedule classes.
Momchil Velikov [Mon, 27 Nov 2017 10:13:14 +0000 (10:13 +0000)]
[ARM] Fix an off-by-one error when restoring LR for 16-bit Thumb
The commit https://reviews.llvm.org/rL318143 computes incorrectly to offset to
restore LR from.
The number of tPOP operands is 2 (condition) + 2 (implicit def and use of SP) +
count of the popped registers. We need to load LR from just past the last
register, hence the correct offset should be either getNumOperands() - 4 and
getNumExplicitOperands() - 2 (multiplied by 4).
Jatin Bhateja [Sun, 26 Nov 2017 15:08:41 +0000 (15:08 +0000)]
[SCEV] Adding a check on outgoing branches of a terminator instr for SCEVBackedgeConditionFolder, NFC.
Summary:
For a given loop, getLoopLatch returns a non-null value
when a loop has only one latch block. In the modified
context adding an assertion to check that both the outgoing branches of
a terminator instruction (of latch) does not target same header.
+
few minor code reorganization.
Oren Ben Simhon [Sun, 26 Nov 2017 13:02:45 +0000 (13:02 +0000)]
Control-Flow Enforcement Technology - Shadow Stack support (LLVM side)
Shadow stack solution introduces a new stack for return addresses only.
The HW has a Shadow Stack Pointer (SSP) that points to the next return address.
If we return to a different address, an exception is triggered.
The shadow stack is managed using a series of intrinsics that are introduced in this patch as well as the new register (SSP).
The intrinsics are mapped to new instruction set that implements CET mechanism.
The patch also includes initial infrastructure support for IBT.
For more information, please see the following:
https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf
Jatin Bhateja [Sun, 26 Nov 2017 02:01:01 +0000 (02:01 +0000)]
[SCEV] NFC : Removing unnecessary check on outgoing branches of a branch instr.
Summary:
For a given loop, getLoopLatch returns a non-null value
when a loop has only one latch block. In the modified
context a check on both the outgoing branches of a terminator instruction (of latch) to same header is redundant.
David Blaikie [Sat, 25 Nov 2017 20:06:04 +0000 (20:06 +0000)]
Remove dead code
(this header is not fully implemented (the out of line function
writeTypeRecordKind is called in an inline function but never
implemented - this fails to link under modular code generation) and not
included anywhere)
Craig Topper [Sat, 25 Nov 2017 18:32:43 +0000 (18:32 +0000)]
[X86] Add separate intrinsics for scalar FMA4 instructions.
Summary:
These instructions zero the non-scalar part of the lower 128-bits which makes them different than the FMA3 instructions which pass through the non-scalar part of the lower 128-bits.
I've only added fmadd because we should be able to derive all other variants using operand negation in the intrinsic header like we do for AVX512.
I think there are still some missed negate folding opportunities with the FMA4 instructions in light of this behavior difference that I hadn't noticed before.
I've split the tests so that we can use different intrinsics for scalar testing between the two. I just copied the tests split the RUN lines and changed out the scalar intrinsics.
fma4-fneg-combine.ll is a new test to make sure we negate the fma4 intrinsics correctly though there are a couple TODOs in it.
Craig Topper [Sat, 25 Nov 2017 18:09:37 +0000 (18:09 +0000)]
[X86] Don't report gather is legal on Skylake CPUs when AVX2/AVX512 is disabled. Allow gather on SKX/CNL/ICL when AVX512 is disabled by using AVX2 instructions.
Summary:
This adds a new fast gather feature bit to cover all CPUs that support fast gather that we can use independent of whether the AVX512 feature is enabled. I'm only using this new bit to qualify AVX2 codegen. AVX512 is still implicitly assuming fast gather to keep tests working and to match the scatter behavior.
Test command lines have been added for these two cases.
Craig Topper [Sat, 25 Nov 2017 17:59:00 +0000 (17:59 +0000)]
[SelectionDAG] Remove some dead code from vector scalaring
Summary:
Currently ScalarizeVecRes_SETCC checks for the result type being a vector and jumps to ScalarizeVecRes_VSETCC. But if we're scalarizing a vector result, aren't we guaranteed to be looking at a vector type?
This patch deletes the current ScalarizeVecRes_SETCC and renames ScalarizeVecRes_VSETCC to ScalarizeVecRes_SETCC.
Craig Topper [Sat, 25 Nov 2017 07:20:21 +0000 (07:20 +0000)]
[X86] Add some early DAG combines to turn v4i32 AND/OR/XOR into FAND/FOR/FXOR whe only SSE1 is available.
v4i32 isn't a legal type with sse1 only and would end up getting scalarized otherwise.
This isn't completely ideal as it doesn't handle cases like v8i32 that would get split to v4i32. But it at least helps with code written using the clang intrinsic header.
Craig Topper [Fri, 24 Nov 2017 20:29:04 +0000 (20:29 +0000)]
Recommit r318963 "[APInt] Don't print debug messages from the APInt knuth division algorithm by default"
The previous commit had the condition in the do/while backwards.
Debug builds currently print out low level details of the Knuth division algorithm when -debug is used. This information isn't useful in most cases and just adds noise to the log.
This adds a new preprocessor flag to enable the prints in the knuth division code in APInt.
Craig Topper [Fri, 24 Nov 2017 19:57:48 +0000 (19:57 +0000)]
[X86] Prevent using X * rsqrt(X) to approximate sqrt when only sse1 is enabled.
This optimization can occur after type legalization and emit a vselect with v4i32 type. But that type is not legal with sse1. This ultimately gets scalarized by the second type legalization that runs after vector op legalization, but that's really intended to handle the scalar types that might be introduced by legalizing vector ops.
For now just stop this from happening by disabling the optimization with sse1.
Craig Topper [Fri, 24 Nov 2017 19:13:24 +0000 (19:13 +0000)]
[APInt] Don't print debug messages from the APInt knuth division algorithm by default
Debug builds currently print out low level details of the Knuth division algorithm when -debug is used. This information isn't useful in most cases and just adds noise to the log.
This adds a new preprocessor flag to enable the prints in the knuth division code in APInt.
Simon Dardis [Fri, 24 Nov 2017 16:45:28 +0000 (16:45 +0000)]
[CodeGenPrepare] Check that erased sunken address are not reused
CodeGenPrepare sinks address computations from one basic block to another
and attempts to reuse address computations that have already been sunk. If
the same address computation appears twice with the first instance as an
operand of a load whose result is an operand to a simplifable select,
CodeGenPrepare simplifies the select and recursively erases the now dead
instructions. CodeGenPrepare then attempts to use the erased address
computation for the second load.
Fix this by erasing the cached address value if it has zero uses before
looking for the address value in the sunken address map.
This partially resolves PR35209.
Thanks to Alexander Richardson for reporting the issue!
This fixed version relands r318032 which was reverted in r318049 due to
sanitizer buildbot failures.
John Brawn [Fri, 24 Nov 2017 14:10:45 +0000 (14:10 +0000)]
[CGP] Make optimizeMemoryInst able to combine more kinds of ExtAddrMode fields
This patch extends the recent work in optimizeMemoryInst to make it able to
combine more ExtAddrMode fields than just the BaseReg.
This fixes some benchmark regressions introduced by r309397, where GVN PRE is
hoisting a getelementptr such that it can no longer be combined into the
addressing mode of the load or store that uses it.
Craig Topper [Thu, 23 Nov 2017 19:25:45 +0000 (19:25 +0000)]
[X86] Don't invert NewCC variable while processing the jcc/setcc/cmovcc instructions in optimizeCompareInstr.
The NewCC variable is calculated outside of the loop that processes jcc/setcc/cmovcc instructions. If we invert it during the loop it can cause an incorrect value to be used by a later iteration. Instead only read it during the loop and use a new variable to store the possibly inverted value.
Diana Picus [Thu, 23 Nov 2017 13:26:07 +0000 (13:26 +0000)]
[ARM GlobalISel] Support G_FDIV for s32 and s64
TableGen already generates code for selecting a G_FDIV, so we only need
to add a test.
For the legalizer and reg bank select, we do the same thing as for the
other floating point binary operations: either mark as legal if we have
a FP unit or lower to a libcall, and map to the floating point
registers.
Ying Yi [Thu, 23 Nov 2017 12:48:41 +0000 (12:48 +0000)]
[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands internally
Summary:
The internal shell already supports 'cd', ‘export’ and ‘echo’ commands.
This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’
commands as the internal shell builtins.
Diana Picus [Thu, 23 Nov 2017 12:44:20 +0000 (12:44 +0000)]
[ARM GlobalISel] Support G_FMUL for s32 and s64
TableGen already generates code for selecting a G_FMUL, so we only need
to add a test for that part.
For the legalizer and reg bank select, we do the same thing as the other
floating point binary operators: either mark as legal if we have a FP
unit or lower to a libcall, and map to the floating point registers.
Simon Dardis [Thu, 23 Nov 2017 12:38:04 +0000 (12:38 +0000)]
[mips] Use the delay slot filler to convert branches for microMIPSR6.
The MIPS delay slot filler converts delay slot branches into compact
forms for the MIPS ISAs which support them. For branches that compare
(in)equality with with zero, it converts them into branches with implict
zero register operands. These branches have a slightly greater range
than normal two register operands branches.
Changing the branches at this point in the pipeline offers the long
branch pass the ability to mark better judgements if a long branch
sequence is required.
[MSan] Move the access address check before the shadow access for that address
MSan used to insert the shadow check of the store pointer operand
_after_ the shadow of the value operand has been written.
This happens to work in the userspace, as the whole shadow range is
always mapped. However in the kernel the shadow page may not exist, so
the bug may cause a crash.
This patch moves the address check in front of the shadow access.
Max Kazantsev [Thu, 23 Nov 2017 06:14:39 +0000 (06:14 +0000)]
[IRCE][NFC] Add no wrap flags to no-wrapping SCEV calculation
In a lambda where we expect to have result within bounds, add respective `nsw/nuw` flags to
help SCEV just in case if it fails to figure them out on its own.
Yaxun Liu [Thu, 23 Nov 2017 03:08:51 +0000 (03:08 +0000)]
[NFC] CodeGen: Handle shift amount type in DAGTypeLegalizer::SplitInteger
This patch reverts change to X86TargetLowering::getScalarShiftAmountTy in
rL318727 and move the logic to DAGTypeLegalizer::SplitInteger.
The reason is that getScalarShiftAmountTy returns a shift amount type that
is suitable for common use cases in CodeGen. DAGTypeLegalizer::SplitInteger
is a rare situation which requires a shift amount type larger than what
getScalarShiftAmountTy. In this case, it is more reasonable to do special
handling of shift amount type in DAGTypeLegalizer::SplitInteger only. If
similar situations arises the logic may be moved to a separate function.
Fedor Sergeev [Wed, 22 Nov 2017 20:59:53 +0000 (20:59 +0000)]
IR printing improvement for loop passes
Summary:
Loop-pass printing is somewhat deficient since it does not provide the
context around the loop (e.g. preheader). This context information becomes
pretty essential when analyzing transformations that move stuff out of the loop.
Extending printLoop to cover preheader and exit blocks (if any).
[Hexagon] Implement buildVector32 and buildVector64 as utility functions
Change LowerBUILD_VECTOR to use those functions. This commit will tempora-
rily affect constant vector generation (it will generate constant-extended
values instead of non-extended combines), but the code for the general case
should be better. The constant selection part will be fixed later.
Rafael Espindola [Wed, 22 Nov 2017 19:59:05 +0000 (19:59 +0000)]
Allow TempFile::discard to be called twice.
We already allowed keep+discard. It is important to be able to discard
a temporary if a rename fail. It is also convenient as it allows the
use of RAII for discarding.
CachePruning: Allow limiting the number of files in the cache directory.
The default limit is 1000000 but it can be configured with a cache
policy. The motivation is that some filesystems (notably ext4) have
a limit on the number of files that can be contained in a directory
(separate from the inode limit).
Paul Robinson [Wed, 22 Nov 2017 15:33:17 +0000 (15:33 +0000)]
[DWARFv5] Support DW_FORM_strp in the .debug_line.dwo header.
As a side effect, the .debug_line section will be dumped in physical
order, rather than in the order that compile units refer to their
associated portions of the .debug_line section. These are probably
always the same order anyway, and no tests noticed the difference.
Nicolai Haehnle [Wed, 22 Nov 2017 12:25:21 +0000 (12:25 +0000)]
AMDGPU: Consider memory dependencies with moved instructions in SILoadStoreOptimizer
Summary:
This bug seems to have gone unnoticed because critical cases with LDS
instructions are eliminated by the peephole optimizer.
However, equivalent situations arise with buffer loads and stores
as well, so this fixes regressions since r317751 ("AMDGPU: Merge
S_BUFFER_LOAD_DWORD_IMM into x2, x4").
Fixes at least:
KHR-GL45.shader_storage_buffer_object.basic-operations-case1-cs
KHR-GL45.cull_distance.functional
piglit tes-input-gl_ClipDistance.shader_test
... and probably more
George Rimar [Wed, 22 Nov 2017 07:53:48 +0000 (07:53 +0000)]
[llvm-tblgen] - Stop using std::string in RecordKeeper.
RecordKeeper::getDef() is a hot place, it shows up in profiling
and it creates std::string instance for each search in RecordMap
though RecordKeeper::RecordMap can use StringRef as a key
instead to avoid that. Patch do that change.
Craig Topper [Wed, 22 Nov 2017 07:11:03 +0000 (07:11 +0000)]
[X86] Lower all ISD::MGATHER nodes to X86ISD:MGATHER.
Now we consistently represent the mask result without relying on isel ignoring it.
We now have a more general SDNode and type constraints to represent these nodes in isel patterns. This allows us to present both both vXi1 and XMM/YMM mask types with a single set of constraints.
Max Kazantsev [Wed, 22 Nov 2017 06:21:39 +0000 (06:21 +0000)]
[SCEV] Strengthen variance condition in calculateLoopDisposition
Given loops `L1` and `L2` with AddRecs `AR1` and `AR2` varying in them respectively.
When identifying loop disposition of `AR2` w.r.t. `L1`, we only say that it is varying if
`L1` contains `L2`. But there is also a possible situation where `L1` and `L2` are
consecutive sibling loops within the parent loop. In this case, `AR2` is also varying
w.r.t. `L1`, but we don't correctly identify it.
It can lead, for exaple, to attempt of incorrect folding. Consider:
AR1 = {a,+,b}<L1>
AR2 = {c,+,d}<L2>
EXAR2 = sext(AR1)
MUL = mul AR1, EXAR2
If we incorrectly assume that `EXAR2` is invariant w.r.t. `L1`, we can end up trying to
construct something like: `{a * {c,+,d}<L2>,+,b * {c,+,d}<L2>}<L1>`, which is incorrect
because `AR2` is not available on entrance of `L1`.
Both situations "`L1` contains `L2`" and "`L1` preceeds sibling loop `L2`" can be handled
with one check: "header of `L1` dominates header of `L2`". This patch replaces the old
insufficient check with this one.
Davide Italiano [Wed, 22 Nov 2017 03:04:55 +0000 (03:04 +0000)]
[SCCP] Pick the right lattice value for constants.
After the dataflow algorithm proves that an argument is constant,
it replaces it value with the integer constant and drops the lattice
value associated to the DEF.
e.g. in the example we have @f() that's called twice:
call @f(undef, ...)
call @f(2, ...)
`undef` MEET 2 = 2 so we replace the argument and all its uses with
the constant 2.
Shortly after, tryToReplaceWithConstantRange() tries to get the lattice
value for the argument we just replaced, causing an assertion.
This function is a little peculiar as it runs when we're doing replacement
and not as part of the solver but still queries the solver.
The fix is that of checking whether we replaced the value already and
get a temporary lattice value for the constant.
Craig Topper [Tue, 21 Nov 2017 23:36:42 +0000 (23:36 +0000)]
[X86] Move the information about the feature bits used by compiler-rt and shared by Host.cpp to a .def file and TargetParser.h so clang can make use of it.
Since we keep Host.cpp and compiler-rt relatively in sync, clang can use this information as a proxy.
Change the representation of COFF comdats so that a COFF linker
is able to accurately resolve comdats between IR and native object
files. Specifically, apply name mangling to comdat names consistently
with native object files, and do not export comdats with an internal
leader because they do not affect symbol resolution.
projects / llvm / log