Kristof Beyls [Thu, 8 Jan 2015 15:09:14 +0000 (15:09 +0000)]
Fix large stack alignment codegen for ARM and Thumb2 targets
This partially fixes PR13007 (ARM CodeGen fails with large stack
alignment): for ARM and Thumb2 targets, but not for Thumb1, as it
seems stack alignment for Thumb1 targets hasn't been supported at
all.
Producing an aligned stack pointer is done by zero-ing out the lower
bits of the stack pointer. The BIC instruction was used for this.
However, the immediate field of the BIC instruction only allows to
encode an immediate that can zero out up to a maximum of the 8 lower
bits. When a larger alignment is requested, a BIC instruction cannot
be used; llvm was silently producing incorrect code in this case.
This commit fixes code generation for large stack aligments by
using the BFC instruction instead, when the BFC instruction is
available. When not, it uses 2 instructions: a right shift,
followed by a left shift to zero out the lower bits.
The lowering of ARM::Int_eh_sjlj_dispatchsetup still has code
that unconditionally uses BIC to realign the stack pointer, so it
very likely has the same problem. However, I wasn't able to
produce a test case for that. This commit adds an assert so that
the compiler will fail the assert instead of silently generating
wrong code if this is ever reached.
Its functionality has been replaced by calling
SIInstrInfo::legalizeOperands() from
SIISelLowering::AdjstInstrPostInstrSelection() and running the
SIFoldOperands and SIShrinkInstructions passes.
PEI tries to keep track of how much starting or ending a call sequence adjusts the stack pointer by, so that it can resolve frame-index references. Currently, it takes a very simplistic view of how SP adjustments are done - both FrameStartOpcode and FrameDestroyOpcode adjust it exactly by the amount written in its first argument.
This view is in fact incorrect for some targets (e.g. due to stack re-alignment, or because it may want to adjust the stack pointer in multiple steps). However, that doesn't cause breakage, because most targets (the only in-tree exception appears to be 32-bit ARM) rely on being able to simplify the call frame pseudo-instructions earlier, so this code is never hit.
Moving the computation into TargetInstrInfo allows targets to override the way the adjustment is computed if they need to have a non-zero SPAdj.
[RegAllocGreedy] Introduce a late pass to repair broken hints.
A broken hint is a copy where both ends are assigned different colors. When a
variable gets evicted in the neighborhood of such copies, it is likely we can
reconcile some of them.
** Context **
Copies are inserted during the register allocation via splitting. These split
points are required to relax the constraints on the allocation problem. When
such a point is inserted, both ends of the copy would not share the same color
with respect to the current allocation problem. When variables get evicted,
the allocation problem becomes different and some split point may not be
required anymore. However, the related variables may already have been colored.
This usually shows up in the assembly with pattern like this:
def A
...
save A to B
def A
use A
restore A from B
...
use B
Whereas we could simply have done:
def B
...
def A
use A
...
use B
** Proposed Solution **
A variable having a broken hint is marked for late recoloring if and only if
selecting a register for it evict another variable. Indeed, if no eviction
happens this is pointless to look for recoloring opportunities as it means the
situation was the same as the initial allocation problem where we had to break
the hint.
Finally, when everything has been allocated, we look for recoloring
opportunities for all the identified candidates.
The recoloring is performed very late to rely on accurate copy cost (all
involved variables are allocated).
The recoloring is simple unlike the last change recoloring. It propagates the
color of the broken hint to all its copy-related variables. If the color is
available for them, the recoloring uses it, otherwise it gives up on that hint
even if a more complex coloring would have worked.
The recoloring happens only if it is profitable. The profitability is evaluated
using the expected frequency of the copies of the currently recolored variable
with a) its current color and b) with the target color. If a) is greater or
equal than b), then it is profitable and the recoloring happen.
** Example **
Consider the following example:
BB1:
a =
b =
BB2:
...
= b
= a
Let us assume b gets split:
BB1:
a =
b =
BB2:
c = b
...
d = c
= d
= a
Because of how the allocation work, b, c, and d may be assigned different
colors. Now, if a gets evicted to make room for c, assuming b and d were
assigned to something different than a.
We end up with:
BB1:
a =
st a, SpillSlot
b =
BB2:
c = b
...
d = c
= d
e = ld SpillSlot
= e
This is likely that we can assign the same register for b, c, and d,
getting rid of 2 copies.
** Performances **
Both ARM64 and x86_64 show performance improvements of up to 3% for the
llvm-testsuite + externals with Os and O3. There are a few regressions too that
comes from the (in)accuracy of the block frequency estimate.
Ahmed Bougacha [Thu, 8 Jan 2015 00:51:32 +0000 (00:51 +0000)]
[SelectionDAG] Allow targets to specify legality of extloads' result
type (in addition to the memory type).
The *LoadExt* legalization handling used to only have one type, the
memory type. This forced users to assume that as long as the extload
for the memory type was declared legal, and the result type was legal,
the whole extload was legal.
However, this isn't always the case. For instance, on X86, with AVX,
this is legal:
v4i32 load, zext from v4i8
but this isn't:
v4i64 load, zext from v4i8
Whereas v4i64 is (arguably) legal, even without AVX2.
Note that the same thing was done a while ago for truncstores (r46140),
but I assume no one needed it yet for extloads, so here we go.
Calls to getLoadExtAction were changed to add the value type, found
manually in the surrounding code.
Calls to setLoadExtAction were mechanically changed, by wrapping the
call in a loop, to match previous behavior. The loop iterates over
the MVT subrange corresponding to the memory type (FP vectors, etc...).
I also pulled neighboring setTruncStoreActions into some of the loops;
those shouldn't make a difference, as the additional types are illegal.
(e.g., i128->i1 truncstores on PPC.)
Matthias Braun [Thu, 8 Jan 2015 00:21:23 +0000 (00:21 +0000)]
RegisterCoalescer: Do not remove IMPLICIT_DEFS if they are required for subranges.
The register coalescer used to remove implicit_defs when they are
covered by the main range anyway. With subreg liveness tracking we can't
do that anymore in places where the IMPLICIT_DEF is required as begin of
a subregister liverange.
Matthias Braun [Wed, 7 Jan 2015 23:58:38 +0000 (23:58 +0000)]
RegisterCoalescer: Fix valuesIdentical() in some subrange merge cases.
I got confused and assumed SrcIdx/DstIdx of the CoalescerPair is a
subregister index in SrcReg/DstReg, but they are actually subregister
indices of the coalesced register that get you back to SrcReg/DstReg
when applied.
Fixed the bug, improved comments and simplified code accordingly.
"This patch started out as an exploration of gc.relocate, and an attempt
to write a simple test in call-lowering. I then noticed that the
arguments of gc.relocate were not checked fully, so I went in and fixed
a few things. Finally, the most important outcome of this patch is that
my new error handling code caught a bug in a callsite in
stackmap-format."
Ahmed Bougacha [Wed, 7 Jan 2015 22:47:46 +0000 (22:47 +0000)]
[CodeGen] Add MVT::isValid to replace manual validity checks. NFC.
Now that we have MVT::FIRST_VALUETYPE (r225362), we can provide a method
checking that the MVT is valid, that is, it's in
[FIRST_VALUETYPE, LAST_VALUETYPE[.
This commit also uses it in a few asserts, that would previously accept
invalid MVTs, such as the default constructed -1. In that case,
the code following those asserts would do an out-of-bounds array access.
Using MVT::isValid, those assertions fail as expected when passed
invalid MVTs.
It feels clunky to have such a validity checking function, but it's
at least better than the alternative of broken manual checks.
Allow distinct `MDNode`s to be explicitly created. There's no way (yet)
of representing their distinctness in assembly/bitcode, however, so this
still isn't first-class.
Add API to indicate whether an `MDNode` is distinct. A distinct node is
not stored in the MDNode uniquing tables, and will never be returned by
`MDNode::get()`.
Although distinct nodes are only currently created by uniquing
collisions (when operands change), PR22111 will allow these nodes to be
explicitly created.
`MDNode::replaceOperandWith()` changes all instances of metadata. Stop
using it when linking module flags, since (due to uniquing) the flag
values could be used by other metadata.
Instead, use new API `NamedMDNode::setOperand()` to update the reference
directly.
Kevin Enderby [Wed, 7 Jan 2015 21:02:18 +0000 (21:02 +0000)]
Slightly refactor things for llvm-objdump and the -macho option so it can be used with
options other than just -disassemble so that universal files can be used with other
options combined with -arch options.
No functional change to existing options and use. One test case added for the
additional functionality with a universal file an a -arch option.
Tom Stellard [Wed, 7 Jan 2015 20:59:25 +0000 (20:59 +0000)]
R600/SI: Remove VReg_32 register class
Use VGPR_32 register class instead. These two register classes were
identical and having separate classes was causing
SIInstrInfo::isLegalOperands() to be overly conservative in some cases.
This change is necessary to prevent future paches from missing a folding
opportunity in fneg-fabs.ll.
Philip Reames [Wed, 7 Jan 2015 19:07:50 +0000 (19:07 +0000)]
Introduce an example statepoint GC strategy
This change includes the most basic possible GCStrategy for a GC which is using the statepoint lowering code. At the moment, this GCStrategy doesn't really do much - aside from actually generate correct stackmaps that is - but I went ahead and added a few extra correctness checks as proof of concept. It's mostly here to provide documentation on how to do one, and to provide a point for various optimization legality hooks I'd like to add going forward. (For context, see the TODOs in InstCombine around gc.relocate.)
Most of the validation logic added here as proof of concept will soon move in to the Verifier. That move is dependent on http://reviews.llvm.org/D6811
There was discussion in the review thread about addrspace(1) being reserved for something. I'm going to follow up on a seperate llvmdev thread. If needed, I'll update all the code at once.
Note that I am deliberately not making a GCStrategy required to use gc.statepoints with this change. I want to give folks out of tree - including myself - a chance to migrate. In a week or two, I'll make having a GCStrategy be required for gc.statepoints. To this end, I added the gc tag to one of the test cases but not others.
Ahmed Bougacha [Wed, 7 Jan 2015 18:39:00 +0000 (18:39 +0000)]
[CodeGen] Add MVT::FIRST_VALUETYPE to avoid explicit 0. NFC.
Many places reference MVT::LAST_VALUETYPE when iterating over all
valid MVTs, but they usually start with 0.
With FIRST_VALUETYPE, we can avoid explicit constants when we really
should be using MVT::SimpleValueType.
David Majnemer [Wed, 7 Jan 2015 18:14:07 +0000 (18:14 +0000)]
X86: Allow the stack probe size to be configurable per function
LLVM emits stack probes on Windows targets to ensure that the stack is
correctly accessed. However, the amount of stack allocated before
emitting such a probe is hardcoded to 4096.
It is desirable to have this be configurable so that a function might
opt-out of stack probes. Our level of granularity is at the function
level instead of, say, the module level to permit proper generation of
code after LTO.
Patch by Andrew H!
N.B. The inliner needs to be updated to properly consider what happens
after inlining a function with a specific stack-probe-size into another
function with a different stack-probe-size.
Ahmed Bougacha [Wed, 7 Jan 2015 17:33:03 +0000 (17:33 +0000)]
[X86] Teach FCOPYSIGN lowering to recognize constant magnitudes.
For code like:
float foo(float x) { return copysign(1.0, x); }
We used to generate:
andps <-0.000000e+00,0,0,0>, %xmm0
movss <1.000000e+00>, %xmm1
andps <nan>, %xmm1
orps %xmm0, %xmm1
Basically doing an abs(1.0f) in the two middle instructions.
We now generate:
andps <-0.000000e+00,0,0,0>, %xmm0
orps <1.000000e+00,0,0,0>, %xmm0
Builds on cleanups r223415, r223542.
rdar://19049548
Differential Revision: http://reviews.llvm.org/D6555
Improvements to emacs packages for llvm and tablegen mode.
* Both files have valid package headers and footers (you can verify
with M-x checkdoc).
* Fixed style warnings generated by checkdoc.
* Fixed a byte-compiler warning in llvm-mode.el.
* Ensure that the modes are autoloaded, so users do not need to
(require 'llvm-mode) to use them.
Aaron Ballman [Wed, 7 Jan 2015 14:26:07 +0000 (14:26 +0000)]
Manually specify the folder that Kaleidescope should reside in for CMake-produced solutions that care about such things (like MSVC). This takes the Kaleidescope target out of the root solution folder and places it into the Examples folder where it belongs.
Aaron Ballman [Wed, 7 Jan 2015 14:19:15 +0000 (14:19 +0000)]
Manually specify the folder that llvm-ranlib should reside in for CMake-produced solutions that care about such things (like MSVC). This takes llvm-ranlib out of the root solution folder and places it into the Tools folder where it belongs.
Jonas Paulsson [Wed, 7 Jan 2015 13:38:29 +0000 (13:38 +0000)]
New method SDep::isNormalMemoryOrBarrier() in ScheduleDAGInstrs.cpp.
Used to iterate over previously added memory dependencies in
adjustChainDeps() and iterateChainSucc().
SDep::isCtrl() was previously used in these places, that also gave
anti and output edges. The code may be worse if these are followed,
because MisNeedChainEdge() will conservatively return true since a
non-memory instruction has no memory operands, and a false chain dep
will be added. It is also unnecessary since all memory accesses of
interest will be reached by memory dependencies, and there is a budget
limit for the number of edges traversed.
This problem was found on an out-of-tree target with enabled alias
analysis. No test case for an in-tree target has been found.
[PM] Give slightly less horrible names to the utility pass templates for
requiring and invalidating specific analyses. Also make their printed
names match their class names. Writing these out as prose really doesn't
make sense to me any more.
Craig Topper [Wed, 7 Jan 2015 08:10:38 +0000 (08:10 +0000)]
[X86] Merge a switch statement inside a default case of another switch statement on the same variable. There was no additional code in the default so this should be no functional change.
Karthik Bhat [Wed, 7 Jan 2015 06:34:34 +0000 (06:34 +0000)]
Revert r225165 and r225169
Even thouh gcc produces simialr instructions as Owen pointed out the two patterns aren’t equivalent in the case
where the original subtraction could have caused an overflow.
Reverting the same.
Ahmed Bougacha [Wed, 7 Jan 2015 02:42:01 +0000 (02:42 +0000)]
[ADT][SmallVector] Flip an assert comparison to avoid overflows yielding false-negatives. NFC.
r221973 changed SmallVector::operator[] to use size_t instead of unsigned.
Before that, on 64bit platforms, when a large index (say -1) was passed,
truncating it to unsigned avoided an overflow when computing 'begin() + idx',
and failed the range checking assertion, as expected.
With r221973, idx isn't truncated, so the addition wraps to
'(char*)begin() - 1', and doesn't fire anymore when it should have done so.
This commit changes the comparison to instead compute 'end() - begin()'
(i.e., 'size()'), which avoids potentially overflowing additions, and
correctly triggers the assertion when values such as -1 are passed.
Note that the problem already existed before that revision, on platforms
where sizeof(size_t) == sizeof(unsigned).
We can't drop support for RAUW entirely in `MDNode`s, since it's
required for graph construction. This comment was from before I'd done
the math on that (out-of-tree), and never should have been committed.
[PM] Fix a pretty nasty bug where the new pass manager would invalidate
passes too many time.
I think this is actually the issue that someone raised with me at the
developer's meeting and in an email, but that we never really got to the
bottom of. Having all the testing utilities made it much easier to dig
down and uncover the core issue.
When a pass manager is running many passes over a single function, we
need it to invalidate the analyses between each run so that they can be
re-computed as needed. We also need to track the intersection of
preserved higher-level analyses across all the passes that we run (for
example, if there is one module analysis which all the function analyses
preserve, we want to track that and propagate it). Unfortunately, this
interacted poorly with any enclosing pass adaptor between two IR units.
It would see the intersection of preserved analyses, and need to
invalidate any other analyses, but some of the un-preserved analyses
might have already been invalidated *and recomputed*! We would fail to
propagate the fact that the analysis had already been invalidated.
The solution to this struck me as really strange at first, but the more
I thought about it, the more natural it seemed. After a nice discussion
with Duncan about it on IRC, it seemed even nicer. The idea is that
invalidating an analysis *causes* it to be preserved! Preserving the
lack of result is trivial. If it is recomputed, great. Until something
*else* invalidates it again, we're good.
The consequence of this is that the invalidate methods on the analysis
manager which operate over many passes now consume their
PreservedAnalyses object, update it to "preserve" every analysis pass to
which it delivers an invalidation (regardless of whether the pass
chooses to be removed, or handles the invalidation itself by updating
itself). Then we return this augmented set from the invalidate routine,
letting the pass manager take the result and use the intersection of
*that* across each pass run to compute the final preserved set. This
accounts for all the places where the early invalidation of an analysis
has already "preserved" it for a future run.
I've beefed up the testing and adjusted the assertions to show that we
no longer repeatedly invalidate or compute the analyses across nested
pass managers.
David Majnemer [Wed, 7 Jan 2015 00:39:50 +0000 (00:39 +0000)]
Analysis: Reformulate WillNotOverflowUnsignedAdd for reusability
WillNotOverflowUnsignedAdd's smarts will live in ValueTracking as
computeOverflowForUnsignedAdd. It now returns a tri-state result:
never overflows, always overflows and sometimes overflows.
Hal Finkel [Wed, 7 Jan 2015 00:15:29 +0000 (00:15 +0000)]
[PowerPC] Transform a README.txt entry into a FIXME
Remove the README.txt entry regarding register allocation of CR logical ops,
and replace it with a FIXME in PPCInstrInfo.td. The text in the README.txt was
not really accurate, and thanks goes to Pat Haugen (and Bill Schmidt) from IBM
for clarifying what was intended and highlighting the relevant text in the ISA
specification.
Add the missing `DEPENDS` keyword. r225319 did almost the right thing
(I didn't notice the problem with it because `Kaleidoscope-Ch8` wasn't
building at all).
Lang Hames [Tue, 6 Jan 2015 23:04:36 +0000 (23:04 +0000)]
Revert r224935 "Refactor duplicated code. No intended functionality change."
This is affecting the behavior of some ObjC++ / AArch64 test cases on Darwin.
Reverting to get the bots green while I track down the source of the changed
behavior.
Change the .ll syntax for comdats and add a syntactic sugar.
In order to make comdats always explicit in the IR, we decided to make
the syntax a bit more compact for the case of a GlobalObject in a
comdat with the same name.
Hal Finkel [Tue, 6 Jan 2015 22:31:02 +0000 (22:31 +0000)]
[PowerPC] Reuse a load operand in int->fp conversions
int->fp conversions on PPC must be done through memory loads and stores. On a
modern core, this process begins by storing the int value to memory, then
loading it using a (sometimes special) FP load instruction. Unfortunately, we
would do this even when the value to be converted was itself a load, and we can
just use that same memory location instead of copying it to another first.
There is a slight complication when handling int_to_fp(fp_to_int(x)) pairs,
because the fp_to_int operand has not been lowered when the int_to_fp is being
lowered. We handle this specially by invoking fp_to_int's lowering logic
(partially) and getting the necessary memory location (some trivial refactoring
was done to make this possible).
This is all somewhat ugly, and it would be nice if some later CodeGen stage
could just clean this stuff up, but because doing so would involve modifying
target-specific nodes (or instructions), it is not immediately clear how that
would work.
Also, remove a related entry from the README.txt for which we now generate
reasonable code.
Mehdi Amini [Tue, 6 Jan 2015 20:05:02 +0000 (20:05 +0000)]
Use a Factory Method for MachineFunctionInfo Creation
The goal is to allows MachineFunctionInfo to override this create
function to customize the creation.
No change intended in existing backend in this patch.
Tom Stellard [Tue, 6 Jan 2015 19:52:04 +0000 (19:52 +0000)]
R600/SI: Fix dependency calculation for DS writes instructions in SIInsertWaits
In DS write instructions, the address operand comes before the value
operand(s) which is reversed from every other instruction type.
The SIInsertWait assumed that the first use for each instruction
was the value, so for DS write it was protecting the address
operand with s_waitcnt instructions when it should have been
protecting the value operand.
Sanjoy Das [Tue, 6 Jan 2015 19:02:56 +0000 (19:02 +0000)]
This patch teaches IndVarSimplify to add nuw and nsw to certain kinds
of operations that provably don't overflow. For example, we can prove
%civ.inc below does not sign-overflow. With this change,
IndVarSimplify changes %civ.inc to an add nsw.
projects / llvm / log