Adam Nemet [Tue, 10 Mar 2015 18:54:26 +0000 (18:54 +0000)]
[LAA-memchecks 3/3] Introduce pointer partitions for memchecks
This is the final patch that actually introduces the new parameter of
partition mapping to RuntimePointerCheck::needsChecking.
Another API (LAI::getInstructionsForAccess) is also exposed that helps
to map pointers to instructions because ultimately we partition
instructions.
The WIP version of the Loop Distribution pass in D6930 has been adapted
to use all this. See for example, how
InstrPartitionContainer::computePartitionSetForPointers sets up the
partitions using the above API and then calls to LAI::addRuntimeCheck
with the pointer partitions.
Adam Nemet [Tue, 10 Mar 2015 18:54:23 +0000 (18:54 +0000)]
[LAA-memchecks 2/3] Move number of memcheck threshold checking to LV
Now the analysis won't "fail" if the memchecks exceed the threshold. It
is the transform pass' responsibility to perform the check.
This allows the transform pass to further analyze/eliminate the
memchecks. E.g. in Loop distribution we only need to check pointers
that end up in different partitions.
Note that there is a slight change of functionality here. The logic in
analyzeLoop is that if dependence checking fails due to non-constant
distance between the pointers, another attempt is made to prove safety
of the dependences purely using run-time checks.
Before this patch we could fail the loop due to exceeding the memcheck
threshold after the first step, now we only check the threshold in the
client after the full analysis. There is no measurable compile-time
effect but I wanted to record this here.
Adam Nemet [Tue, 10 Mar 2015 18:54:19 +0000 (18:54 +0000)]
[LAA-memchecks 1/3] Split out NumComparisons checks. NFC
The check for the number of memchecks will be moved to the client of
this analysis. Besides allowing for transform-specific thresholds, this
also lets Loop Distribution post-process the memchecks; Loop
Distribution only needs memchecks between pointers of different
partitions.
The motivation for this first patch is to untangle the CanDoRT check
from the NumComparison check before moving the NumComparison part.
CanDoRT means that we couldn't determine the bounds for the pointer.
Note that NumComparison is set independent of this flag.
Adam Nemet [Tue, 10 Mar 2015 17:40:37 +0000 (17:40 +0000)]
[LoopAccesses 2/3] Allow querying of interesting dependences
Gather an array of interesting dependences rather than just failing
after the first unsafe one and regarding the loop unsafe. Loop
Distribution needs to be able to collect all dependences in order to
isolate the dependence cycles into their own partition.
Since the dependence checking algorithm is quadratic in terms of
accesses sharing the same underlying pointer, I am applying a cut-off
threshold (MaxInterestingDependence). Exceeding that, the logic reverts
back to the original approach deeming the loop unsafe upon encountering
the first unsafe dependence.
The main idea of the patch is to split isDepedent from directly
answering the question whether the dep is safe for vectorization to
return a dependence type which then gets mapped to old boolean result
using Dependence::isSafeForVectorization.
Tested that this was compile-time neutral on SpecINT2006 LTO bitcode
inputs. No assembly change on the testsuite including external.
Adam Nemet [Tue, 10 Mar 2015 17:40:34 +0000 (17:40 +0000)]
[LoopAccesses 1/3] Expose MemoryDepChecker to LAA users
LoopDistribution needs to query various results of the dependence
analysis. This series will expose some more APIs and state of the
dependence checker.
This patch is a simple one to just expose the DepChecker instance. The
set is compile-time neutral measured with LTO bitcode files of
SpecINT2006. Also there is no assembly change on the testsuite.
Igor Laevsky [Tue, 10 Mar 2015 16:26:48 +0000 (16:26 +0000)]
Teach lowering to correctly handle invoke statepoint and gc results tied to them. Note that we still can not lower gc.relocates for invoke statepoints.
Also it extracts getCopyFromRegs helper function in SelectionDAGBuilder as we need to be able to customize type of the register exported from basic block during lowering of the gc.result.
(Resubmitting this change after not being able to reproduce buildbot failure)
Chad Rosier [Tue, 10 Mar 2015 16:22:52 +0000 (16:22 +0000)]
[BranchFolding] Remove MMOs during tail merge to preserve dependencies.
When tail merging it may be necessary to remove MMOs from memory operations to
ensures later passes (e.g., MI sched) conservatively compute dependencies.
Currently, we only remove the MMO from the common tail if the MMO doesn't match
with the relative instruction in the non-common tail(s).
A more robust solution would be to add multiple MMOs from the duplicate MIs to
the new MI. Currently ScheduleDAGInstrs.cpp ignores all MMOs on instructions
with multiple MMOs, so this solution is equivalent for the time being.
No test case included as this is incredibly difficult to reproduce.
Patch was a collaborative effort between Ana Pazos and myself.
Phabricator: http://reviews.llvm.org/D7769
Sanjay Patel [Tue, 10 Mar 2015 16:08:36 +0000 (16:08 +0000)]
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles
We want to replace as much custom x86 shuffling via intrinsics
as possible because pushing the code down the generic shuffle
optimization path allows for better codegen and less complexity
in LLVM.
This is the sibling patch for the Clang half of this change:
http://reviews.llvm.org/D8088
Karthik Bhat [Tue, 10 Mar 2015 14:32:02 +0000 (14:32 +0000)]
Fix a memory corruption in Dependency Analysis.
This crash occurs due to memory corruption when trying to update dependency
direction based on Constraints.
This crash was observed during lnt regression of Polybench benchmark test case dynprog.
Karthik Bhat [Tue, 10 Mar 2015 13:31:03 +0000 (13:31 +0000)]
Fix a crash in Dependency Analysis.
This crash in Dependency analysis is because we assume here that in case of UsefulGEP
both source and destination have the same number of operands which may not be true.
This incorrect assumption results in crash while populating Pairs. Fix the same.
This crash was observed during lnt regression for code such as-
struct s{
int A[10][10];
int C[10][10][10];
} S;
void dep_constraint_crash_test(int k,int N) {
for( int i=0;i<N;i++)
for( int j=0;j<N;j++)
S.A[0][0] = S.C[0][0][k];
}
Review: http://reviews.llvm.org/D8162
Daniel Sanders [Tue, 10 Mar 2015 10:42:59 +0000 (10:42 +0000)]
The operand flag word used in ISD::INLINEASM is an i32 not a pointer. NFC.
Summary:
This is part of the work to support memory constraints that behave
differently to 'm'. The subsequent patches will expand on the existing
encoding (which is a 32-bit int) and as a result in some flag words will no
longer fit into an i16. This problem only affected the MSP430 target which
appears to have 16-bit pointers.
Yaron Keren [Tue, 10 Mar 2015 07:33:23 +0000 (07:33 +0000)]
Teach raw_ostream to accept SmallString.
Saves adding .str() call to any raw_ostream << SmallString usage
and a small step towards making .str() consistent in the ADTs by
removing one of the SmallString::str() use cases, discussion at
Owen Anderson [Tue, 10 Mar 2015 05:58:21 +0000 (05:58 +0000)]
Fix an issue in the verifier where we could try to read information out of a malformed statepoint intrinsic.
In this situation we would always have already flagged an error on the statepoint intrinsic,
but then we carry on to parse other, related GC intrinsics, and could end up crashing during that
verification when they try to access data from the malformed statepoint.
Owen Anderson [Tue, 10 Mar 2015 05:13:47 +0000 (05:13 +0000)]
Fix an infinite loop in InstCombine when an instruction with no users and side effects can be constant folded.
ReplaceInstUsesWith needs to return nullptr when the input has no users,
because in that case it does not mutate the program. Otherwise, we can
get stuck in an infinite loop of repeatedly attempting to constant fold
and instruction with no users.
Last commit fixed the handling of hash collisions, but it introdcuced
unneeded bucket terminators in some places. The generated table was
correct, it can just be a tiny bit smaller. As the previous table was
correct, the test doesn't need updating. If we really wanted to test
this, I could add the section size to the dwarf dump and test for a
precise value there. IMO the correctness test is sufficient.
CFLAA didn't know how to properly handle ConstantExprs; it would silently
ignore them. This was a problem if the ConstantExpr is, say, a GEP of a global,
because CFLAA wouldn't realize that there's a global there. :)
Mehdi Amini [Tue, 10 Mar 2015 02:37:25 +0000 (02:37 +0000)]
DataLayout is mandatory, update the API to reflect it with references.
Summary:
Now that the DataLayout is a mandatory part of the module, let's start
cleaning the codebase. This patch is a first attempt at doing that.
This patch is not exactly NFC as for instance some places were passing
a nullptr instead of the DataLayout, possibly just because there was a
default value on the DataLayout argument to many functions in the API.
Even though it is not purely NFC, there is no change in the
validation.
I turned as many pointer to DataLayout to references, this helped
figuring out all the places where a nullptr could come up.
I had initially a local version of this patch broken into over 30
independant, commits but some later commit were cleaning the API and
touching part of the code modified in the previous commits, so it
seemed cleaner without the intermediate state.
Frederic Riss [Tue, 10 Mar 2015 00:46:31 +0000 (00:46 +0000)]
DwarfAccelTable: Fix handling of hash collisions.
It turns out accelerator tables where totally broken if they contained
entries with colliding hashes. The failure mode is pretty bad, as it not
only impacted the colliding entries, but would basically make all the
entries after the first hash collision pointing in the wrong place.
The testcase uses the symbol names that where found to collide during a
clang build.
From a performance point of view, the patch adds a sort and a linear
walk over each bucket contents. While it has a measurable impact on the
accelerator table emission, it's not showing up significantly in clang
profiles (and I'd argue that correctness is priceless :-)).
Eric Christopher [Tue, 10 Mar 2015 00:33:27 +0000 (00:33 +0000)]
Temporarily revert r231726 and r231724 as they're breaking the build.:
Author: Lang Hames <lhames@gmail.com>
Date: Mon Mar 9 23:51:09 2015 +0000
[Orc][MCJIT][RuntimeDyld] Add header that was accidentally left out of r231724.
Author: Lang Hames <lhames@gmail.com>
Date: Mon Mar 9 23:44:13 2015 +0000
[Orc][MCJIT][RuntimeDyld] Add symbol flags to symbols in RuntimeDyld. Thread the
new types through MCJIT and Orc.
In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
will allow us to distinguish between weak and strong definitions and find the
right ones during symbol resolution.
Lang Hames [Mon, 9 Mar 2015 23:44:13 +0000 (23:44 +0000)]
[Orc][MCJIT][RuntimeDyld] Add symbol flags to symbols in RuntimeDyld. Thread the
new types through MCJIT and Orc.
In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
will allow us to distinguish between weak and strong definitions and find the
right ones during symbol resolution.
Ahmed Bougacha [Mon, 9 Mar 2015 22:51:05 +0000 (22:51 +0000)]
[CodeGen] Replace the reused stores' chain for extractelt expansion.
This fixes a subtle issue that was introduced in r205153.
When reusing a store for the extractelement expansion (to load directly
from it, inserting of going through the stack), later stores to the
same location might have overwritten the data we were expecting to
extract from.
To fix that, we need to explicitly replace the chain going out of the
reused store, so that later stores also have an explicit dependency on
the generated element-extracting loads, and can't clobber them.
Fix the double-deletion of AnalysisResolver when delegating through to
Dwarf EH preparation by creating one from scratch. Hopefully the new
pass manager simplifies this.
Sanjoy Das [Mon, 9 Mar 2015 21:43:43 +0000 (21:43 +0000)]
[SCEV] Unify getUnsignedRange and getSignedRange
Summary:
This removes some duplicated code, and also helps optimization: e.g. in
the test case added, `%idx ULT 128` in `@x` is not currently optimized
to `true` by `-indvars` but will be, after this change.
The only functional change in ths commit is that for add recurrences,
ScalarEvolution::getRange will be more aggressive -- computing the
unsigned (resp. signed) range for a SCEVAddRecExpr will now look at the
NSW (resp. NUW) bits and check for signed (resp. unsigned) overflow.
This can be a strict improvement in some cases (such as the attached
test case), and should be no worse in other cases.
Frederic Riss [Mon, 9 Mar 2015 21:09:50 +0000 (21:09 +0000)]
DwarfAccelTable: fix obvious typo.
I have a test for that issue, but I didn't include it in the commit as it's
a 200KB file for a pretty minor issue. (The reason the file is so big is
that it needs > 1024 variables/functions to trigger and that with debug
information.
The issue/fix on the other side is totally trivial. If poeple want the test
commited, I can do that. It just didn't seem worth it to me.
Reid Kleckner [Mon, 9 Mar 2015 20:23:14 +0000 (20:23 +0000)]
TableGen: Use 'enum : uint64_t' for feature flags to fix -Wmicrosoft
clang-cl would warn that this value is not representable in 'int':
enum { FeatureX = 1ULL << 31 };
All MS enums are 'ints' unless otherwise specified, so we have to use an
explicit type. The AMDGPU target just hit 32 features, triggering this
warning.
Now that we have C++11 strong enum types, we can also eliminate the
'const uint64_t' codepath from tablegen and just use 'enum : uint64_t'.
In the case where just tables are part of the function section, this produces
more readable assembly by avoiding switching to the eh section and back
to .text.
This would also break with non unique section names, as trying to switch to
a unique section actually creates a new one.
Owen Anderson [Mon, 9 Mar 2015 07:13:42 +0000 (07:13 +0000)]
Fix a bug in the LLParser where we failed to diagnose landingpads with non-constant clause operands.
Fixing this also exposed a related issue where the landingpad under construction was not
cleaned up when an error was raised, which would cause bad reference errors before the
error could actually be printed.
Kevin Qin [Mon, 9 Mar 2015 06:14:28 +0000 (06:14 +0000)]
[AArch64] Enable partial & runtime unrolling on cortex-a57
For inner one of nested loops, it is more likely to be a hot loop,
and the runtime check can be promoted out from patch 0001, so the
overhead is less, we can try a doubled threshold to unroll more loops.
projects / llvm / log