Summary:
This patch moves the getIndexExpressionsFromGEP function from polly
into ScalarEvolution so that both polly and DependenceAnalysis can
use it for the purpose of subscript delinearization when the array
sizes are not parametric.
Authored By: bmahjour
Reviewer: Meinersbur, sebpop, fhahn, dmgreen, grosser, etiotto, bollu
Reviewed By: Meinersbur
Subscribers: hiraditya, arphaman, Whitney, ppc-slack, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73995
Previously, the enums didn't account for all the possible cases, which
could cause misleading results (particularly for a "switch" on
FunctionModRefBehavior).
Fixes regression in polly from recent patch to add writeonly to memset.
While I'm here, also fix a few dubious uses of the FMRB_* enum values.
Differential Revision: https://reviews.llvm.org/D73154
Scope of changes:
1) Moved RecordedAssumptions vector to ScopBuilder. RecordedAssumptions are used only for Scop constructions.
2) Moved definition of RecordedAssumptionsTy to ScopHelper. It is required both by ScopBuilder and SCEVAffinator.
3) Add new function recordAssumption to ScopHelper. One of its argument is a reference to RecordedAssumption vector. This function is used by ScopBuilder and SCEVAffinator.
4) All RecordedAssumptions are created by ScopBuilder. isl::pw_aff
objects for corresponding SCEVs are created inside ScopBuilder. Scop
functions do not record any assumptions. Scop can use isl::pw_aff
objects which were created by ScopBuilder.
5) Removed functions for handling RecordedAssumptions from Scop class.
6) Removed constness from getScopArrayInfo functions.
7) Replaced SCEVVisitor struct from SCEVAffinator with taylored version, which allow to pass pointer to RecordedAssumptions as function argument.
Differential Revision: https://reviews.llvm.org/D68056
ScopBuilder::buildEqivClassBlockStmts creates ScopStmts for instruction
groups in basic block and inserts these ScopStmts into Scop::StmtMap,
however, as described in llvm.org/PR38358, comment #5, StmtScops are
inserted into vector ScopStmt[BB] in wrong order. As a result,
ScopBuilder::buildSchedule creates wrong order sequence node.
Looking closer to code, it's clear there is no equivalent classes with
interleaving isOrderedInstruction(memory access) instructions after
joinOrderedInstructions. Afterwards, ScopStmts need to be created and
inserted in the original order of memory access instructions, however,
at the moment ScopStmts are inserted in the order of leader instructions
which are probably not memory access instructions.
The fix is simple with a standalone loop scanning
isOrderedInstruction(memory access) instructions in basic block and
inserting elements into LeaderToInstList one by one. The patch also
removes double reversing operations which are now unnecessary.
New test preserve-equiv-class-order-in-basic_block.ll is also added.
Differential Revision: https://reviews.llvm.org/D68941
llvm-svn: 375192
Function joinOrderedInstructions merges instructions when a leader is encountered twice.
It also notices that leaders in SeenLeaders may lose their leadership in previous merging,
and tries to handle the case using following code:
Instruction *PrevLeader = UnionFind.getLeaderValue(SeenLeaders.back());
However, this is wrong because it always gets leader for the last element of SeenLeaders,
and I believe it's wrong even we get leader for Prev here. As a result, Statements in cases
like the one in patch aren't merged as expected. After investigation, I believe it's
unnecessary to get leader instruction at all. This is based on fact: Although leaders in
SeenLeaders could lose leadership, they only lose to others in SeenLeaders, in other words,
one existing leader will be chosen as new leader of merged equivalent statements. We can
take advantage of this and simply check if current leader equals to Prev and break merging
if it does.
The patch also adds a new test.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D67007
llvm-svn: 371801
The while loop iterating parent loop in ScopBuilder::buildDomains is
unnecessary because either L or LD are later unused, this is a simple
patch removing it.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D66698
llvm-svn: 370368
When reading code in ScopBuilder::buildEqivClassBlockStmts, I think the
main statement flag computation can be simplified, here is the patch.
It's based on two simple facts that:
1. Instruction won't be removed once it's inserted into UnionFind.
2. Main statement must be set if there is non-trivial statement besides the last one.
The patch also saves std::find call.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D66477
llvm-svn: 369972
Scope of changes:
1) Moved buildDomains function to ScopBuilder class.
2) Moved buildDomainsWithBranchConstraints function to ScopBuilder class.
3) Moved propagateDomainConstraints to ScopBuilder class.
4) Moved propagateDomainConstraintsToRegionExit to ScopBuilder class.
5) Moved propagateInvalidStmtDomains to ScopBuilder class.
6) Moved getPredecessorDomainConstraints function to ScopBuilder class.
7) Moved addLoopBoundsToHeaderDomain function to ScopBuilder class.
8) Moved getPwAff function to ScopBuilder class.
9) Moved buildConditionSets functions to ScopBuilder class.
10) Added updateMaxLoopDepth, notifyErrorBlock, getOrInitEmptyDomain, isDomainDefined, setDomain functions to Scop class. They are used by ScopBuilder.
11) Moved helper functions: getRegionNodeBasicBlock, getRegionNodeSuccessor, containsErrorBlock, createNextIterationMap, collectBoundedParts, partitionSetParts, buildConditionSet to ScopBuilder.cpp file.
Differential Revision: https://reviews.llvm.org/D65729
llvm-svn: 368100
Scope of changes:
1) Moved addUserAssumptions function to ScopBuilder class.
2) Moved buildConditionSets functions to polly namespace.
3) Moved getRepresentingInvariantLoadSCEV to public section of the Scop class
Differential Revision: https://reviews.llvm.org/D65241
llvm-svn: 368089
Scope of changes:
1. Moved buildSchedule functions to ScopBuilder.
2. Moved combineInSequence function to ScopBuilder.
3. Moved mapToDimension function to ScopBuilder.
4. Moved LoopStackTy to ScopBuilder.
5. Moved getLoopSurroundingScop to ScopHelper.
6. Moved getNumBlocksInLoop to ScopHelper.
7. Moved getNumBlocksInRegionNode to ScopHelper.
8. Moved getRegionNodeLoop to ScopHelper.
Differential Revision: https://reviews.llvm.org/D64223
llvm-svn: 366377
Scope of changes:
1) Moved finalizeAccesses to ScopBuilder
2) Moved updateAccessDimensionality to ScopBuilder
3) Moved foldSizeConstantsToRight to ScopBuilder
4) Moved foldSizeConstantsToRight to ScopBuilder
5) Moved assumeNoOutOfBounds to ScopBuilder
6) Moved markFortranArrays to ScopBuilder
7) Added iterator range for AccessFunctions vector.
Differential Revision: https://reviews.llvm.org/D63794
llvm-svn: 366374
Scope of changes:
1) Moved addUserContext to ScopBuilder.
2) Moved command line option UserContextStr to ScopBuilder.
Differential Revision: https://reviews.llvm.org/D63740
llvm-svn: 366266
Scope of changes:
1) Moved buildAliasChecks to ScopBuilder.
2) Moved buildAliasGroup to ScopBuilder.
3) Moved buildAliasGroups to ScopBuilder.
4) Moved buildAliasGroupsForAccesses to ScopBuilder.
5) Moved splitAliasGroupsByDomain to ScopBuilder.
6) Moved addNonEmptyDomainConstraints to ScopBuilder.
7) Moved buildMinMaxAccess to ScopBuilder.
8) Moved calculateMinMaxAccess to ScopBuilder.
9) Moved getAccessDomain to ScopBuilder.
10) Moved command line options used only by buildAliasChecks functions to ScopBuilder.
11) Refactored buildAliasGroup function. Added addAliasGroup function to Scop class for pushing back calculated min/max accesses.
12) Added function incrementNumberOfAliasingAssumptions which increments number of statistic variable AssumptionsAliasing. AssumptionsAliasing variable is defined by STATISTIC macro inside ScopInfo.cpp and it is also used by function trackAssumption from Scop class.
13) Added reference to OptimizationRemarkEmitter to ScopBuilder class.
14) Moved calculateMinMaxAccess function to ScopBuilder class.
Differential Revision: https://reviews.llvm.org/D63693
llvm-svn: 366262
Scope of changes:
1) Moved addRecordedAssumptions to ScopBuilder.
2) Moved Assumption struct outside Scop class.
3) Refactored addRecordedAssumptions function. Replaced while loop by
for range loop.
4) Added function to clear processed Assumptions.
Differential Revision: https://reviews.llvm.org/D63572
llvm-svn: 366260
The Loads.h API changed so that a Type parameter is now mandatory in
preparation for pointer types being opaque. Unfortunately I don't build
polly routinely and it still had some uses. This just provides the
(obvious) load type in each case.
llvm-svn: 365470
Moved addInvariantLoads and functions listed below to ScopBuilder:
isAParameter
canAlwaysBeHoisted
These functions were referenced only by getNonHoistableCtx.
Moved CLI parameter PollyAllowDereferenceOfAllFunctionParams to
ScopBuilder.
Added iterator range through InvariantEquivClasses.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D63172
llvm-svn: 363216
This review is based on review: https://reviews.llvm.org/D62925 . It is
part of moving hoistInvariantLoads function and all functions referenced
only by hoistInvariantLoads to ScopBuilder.
Moved getNonHoistableCtx and functions listed below to ScopBuilder:
isRequiredInvariantLoad
hasNonHoistableBasePtrInScop
isAccessRangeTooComplex
These functions were referenced only by getNonHoistableCtx.
MaxDimensionsInAccessRange and MaxDisjunctsInDomain constant is marked
as extern and it is added to polly namespace. It is used by Scop and
ScopBuilder classes.
MaxDimensionsInAccessRange constant moved to ScopBuilder. It is not used
outside ScopBuilder.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D63066
llvm-svn: 363214
Refactor Scop and ScopBuilder class:
1. Move hoistInvariantLoads function from Scop to ScopBuilder class.
2. Private functions (addInvariantLoads, getNonHoistableCtx) are moved
to public section of Scop class. hoistInvariantLoads function
references these functions. These functions will be moved to
ScopBuilder as well in the next steps.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D62925
llvm-svn: 363121
Refactor Scop and ScopBuilder class. Move canonicalizeDynamicsBasePtrs
and corresponding static functions from ScopInfo.cpp to ScopBuilder.cpp
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D62781
llvm-svn: 362554
Refactor Scop and ScopBuilder class. Move
buildInvariantEquivalenceClasses function from Scop class to ScopBuilder
class.
Patch by: Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D62351
llvm-svn: 361902
This removes unused includes (and forward declarations) as
suggested by include-what-you-use. If a transitive include of a removed
include is required to compile a file, I added the required header (or
forward declaration if suggested by include-what-you-use).
This should reduce compilation time and reduce the number of iterative
recompilations when a header was changed.
llvm-svn: 357209
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This patch should not introduce any behavior changes. It consists of
mostly one of two changes:
1. Replacing fall through comments with the LLVM_FALLTHROUGH macro
2. Inserting 'break' before falling through into a case block consisting
of only 'break'.
We were already using this warning with GCC, but its warning behaves
slightly differently. In this patch, the following differences are
relevant:
1. GCC recognizes comments that say "fall through" as annotations, clang
doesn't
2. GCC doesn't warn on "case N: foo(); default: break;", clang does
3. GCC doesn't warn when the case contains a switch, but falls through
the outer case.
I will enable the warning separately in a follow-up patch so that it can
be cleanly reverted if necessary.
Reviewers: alexfh, rsmith, lattner, rtrieu, EricWF, bollu
Differential Revision: https://reviews.llvm.org/D53950
llvm-svn: 345882
This is a bit awkward in a handful of places where we didn't even have
an instruction and now we have to see if we can build one. But on the
whole, this seems like a win and at worst a reasonable cost for removing
`TerminatorInst`.
All of this is part of the removal of `TerminatorInst` from the
`Instruction` type hierarchy.
llvm-svn: 340701
First build the surrounding loops and then build up the polyhedral
structures. Before r326664 we had to mix these updates, clean this
up to improve readability (slightly).
llvm-svn: 334412
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
Differential Revision: https://reviews.llvm.org/D44978
llvm-svn: 332352
Add the switch -polly-debug-func to define the name of a debug
function. This function is ignored for any validity check.
Its purpose is to allow to observe a value after transformation by a
SCoP, and to follow which statements are executed in which order. For
instance, consider the following code:
static void dbg_printf(int sum, int i) {
fprintf(stderr, "The value of sum is %d, i=%d\n", sum, i);
fflush(stderr);
}
void func(int n) {
int sum = 0;
for (int i = 0; i < 16; i+=1) {
sum += i;
dbg_printf(sum, i);
}
}
Executing this after Polly's codegen with -polly-debug-func=dbg_printf
reveals the new execution order and the assumed values at that point of
execution.
Differential Revision: https://reviews.llvm.org/D45728
llvm-svn: 330466
isl does not guarantee that set dimension ids will be preserved, so using them
to carry information is not a good idea. Furthermore, the loop information can
be derived without problem from the statement itself. As this even requires
less code than propagating loop information on set dimension ids, starting from
this commit we just derive the loop information in collectSurroundingLoops
directly from the IR.
Interestingly this also results in a couple of isl sets to take a simpler
representation.
llvm-svn: 326664
Two or more PHIs mutually using each other directly or indirectly as
incoming value could cause that a PHI WRITE be added before the PHI READ
(i.e. it overwrites the current incoming value with the next incoming
value before it being read).
Fix by ensuring that the PHI WRITE and PHI READ are in the same statement.
This should fix the miscompile of SingleSource/Benchmark/Misc/whetstone
from the test-suite.
llvm-svn: 324934
Splitting basic blocks into multiple statements if there are now
additional scalar dependencies gives more freedom to the scheduler, but
more statements also means higher compile-time complexity. Switch to
finer statement granularity, the additional compile time should be
limited by the number of operations quota.
The regression tests are written for the -polly-stmt-granularity=bb
setting, therefore we add that flag to those tests that break with the
new default. Some of the tests only fail because the statements are
named differently due to a basic block resulting in multiple statements,
but which are removed during simplification of statements without
side-effects. Previous commits tried to reduce this effect, but it is
not completely avoidable.
Differential Revision: https://reviews.llvm.org/D42151
llvm-svn: 324169
Do not add a "_last" suffix to the statement name if there is no (other)
main statement for a basic block. In other words, it becomes the main
statement itself. This further reduces the statement naming difference
between -polly-stmt-granularity=bb and
-polly-stmt-granularity=scalar-indep.
llvm-svn: 324168
Theoretically, a PHI write can be added to any statement that represents
the incoming basic block. We previously always chose the last because
the incoming value's definition is guaranteed to be defined.
With this patch the PHI write is added to the statement that defines the
incoming value. It avoids the requirement for a scalar dependency between
the defining statement and the statement containing the write. As such the
logic for -polly-stmt-granularity=scalar-indep that ensures that there is
such scalar dependencies can be removed.
Differential Revision: https://reviews.llvm.org/D42147
llvm-svn: 323284
The goal is to have -polly-stmt-granularity=bb and
-polly-stmt-granularity=scalar-indep to have the same names if there is
just one statement per basic block.
This fixes a fluke when Polybench's jacobi-2d is optimized differently
depending on the -polly-stmt-granularity option, although both options
create the same SCoP, just with different statement names.
The new naming scheme is:
With -polly-use-llvm-names=0:
Stmt<BBIdx as decimal><Idx within BB as letter>
With -polly-use-llvm-names=1:
Stmt_BBName_<Idx within BB as letter>
The <Idx within BB> suffix is omitted for the main statement of a BB. The
main statement is either the one containing the first store or call
(those cannot be removed by the simplifyer), or if there is no such
instruction, the first. If after simplification there is just a single
statement left, it should be the main statement and have the same names as
with -polly-stmt-granularity=bb.
Differential Revision: https://reviews.llvm.org/D42136
llvm-svn: 322852
This will give control of the statement's name to the caller.
Required to give -polly-stmt-granularity=scalar-indep more control
over the name of the generated statement in a follow-up commit.
llvm-svn: 322851
The option splits BasicBlocks into minimal statements such that no
additional scalar dependencies are introduced.
The algorithm is based on a union-find structure, and unites sets if
putting them into separate statements would introduce a scalar
dependencies. As a consequence, instructions may be split into separate
statements such their relative order is different than the statements
they are in. This is accounted for instructions whose relative order
matters (e.g. memory accesses).
The algorithm is generic in that heuristic changes can be made
relatively easily. We might relax the order requirement for read-reads
or accesses to different base pointers. Forwardable instructions can be
made to not cause a join.
This implementation gives us a speed-up of 82% in SPEC 2006 456.hmmer
benchmark by allowing loop-distribution in a hot loop such that one of
the loops can be vectorized.
Differential Revision: https://reviews.llvm.org/D38403
llvm-svn: 314983
The option is introduced with only one possible value
-polly-stmt-granularity=bb which represents the current behaviour, which
is outlined into the new function buildSequentialBlockStmts().
More options will be added in future commits.
llvm-svn: 314900
Iterate over statement instructions instead over basic block
instructions when creating MemoryAccesses. It allows making the creation
of MemoryAccesses independent of how the basic blocks are split into
multiple ScopStmts.
llvm-svn: 314665
Create the MemoryAccesses of invariant loads separately and before
all other MemoryAccesses.
Invariant loads are classified as synthesizable and therefore are not
contained in any statement. When iterating over all instructions of all
statements, the invariant loads are consequently not processed and
iterating over them separately becomes necessary.
This patch can change the order in which MemoryAccesses are created, but
otherwise has no functional change.
Some temporary code is introduced to ensure correctness, but will be
removed in the next commit.
llvm-svn: 314664
Instructions that compute escaping values might be synthesizable and
therefore not contained in any ScopStmt. When buildAccessFunctions is
changed to only iterate over the instruction list of statement,
"free" instructions still need to be written. We do this after the
main MemoryAccesses have been created.
This can change the order in which MemoryAccesses are created, but has
otherwise no functional change.
llvm-svn: 314663
Decouple handling of exit block PHIs and other MemoryAccesses. Exit PHIs
only need the PHI handling part of buildAccessFunctions but requires
code for skipping them in while creating other MemoryAcesses.
This change will make it easier to modify how statement MemoryAccesses
are created without considering the exit block special case.
llvm-svn: 314662
Eli Friedman