168dfffdb8
typo + pasto
...
llvm-svn: 140769
2011-09-29 01:53:08 +00:00
bc6de90a5f
LSR: rewrite inner loops only.
...
Rewriting the entire loop nest now requires -enable-lsr-nested.
See PR11035 for some performance data.
A few unit tests specifically test nested LSR, and are now under a flag.
llvm-svn: 140762
2011-09-29 01:33:38 +00:00
581243919d
Disable LSR retry by default.
...
Disabling aggressive LSR saves compilation time, and with the new
indvars behavior usually improves performance.
llvm-svn: 140590
2011-09-27 00:44:14 +00:00
8868faec63
LSR, one of the new Cost::isLoser() checks did not get merged in the previous checkin.
...
llvm-svn: 140583
2011-09-26 23:35:25 +00:00
784729d408
LSR cost metric minor fix and verification.
...
The minor bug heuristic was noticed by inspection. I added the
isLoser/isValid helpers because they will become more
important with subsequent checkins.
llvm-svn: 140580
2011-09-26 23:11:04 +00:00
8b2fe2f744
LSR minor bug fix in RateRegister.
...
No test case. Noticed by inspection and I doubt it ever affects the
outcome of the overall heuristic, let alone final codegen.
llvm-svn: 140431
2011-09-23 23:05:19 +00:00
3fb137f7ef
LSR wants to split the landing pad's critical edge. Let it do it, but use the
...
proper function to do it.
llvm-svn: 138550
2011-08-25 05:55:40 +00:00
07efd6f1e0
When inserting new instructions, use getFirstInsertionPt instead of
...
getFirstNonPHI so that it will skip over the landingpad instructions as well.
llvm-svn: 138537
2011-08-25 01:08:34 +00:00
86c5cbe613
Skip the landingpad instruction when determining the insertion point.
...
llvm-svn: 138481
2011-08-24 21:06:46 +00:00
bd243d0dfe
LSR, correct fix for rdar://9786536. Silly casting bug.
...
llvm-svn: 135654
2011-07-21 01:45:54 +00:00
858e9f083d
LSR must sometimes sign-extend before generating double constants.
...
rdar://9786536
llvm-svn: 135650
2011-07-21 01:05:01 +00:00
8acb434402
LSR crashes on an empty IVUsers list.
...
rdar://9786536
llvm-svn: 135644
2011-07-21 00:40:04 +00:00
229907cd11
land David Blaikie's patch to de-constify Type, with a few tweaks.
...
llvm-svn: 135375
2011-07-18 04:54:35 +00:00
b1a1512119
start using the new helper methods a bit.
...
llvm-svn: 135251
2011-07-15 06:08:15 +00:00
411daa5e81
SCEVExpander: give new insts a name that identifies the reponsible pass.
...
llvm-svn: 133992
2011-06-28 05:07:32 +00:00
fc4ccb20c6
IVUsers no longer needs to record the phis.
...
llvm-svn: 133518
2011-06-21 15:43:52 +00:00
6c4a319088
When checking for signed multiplication overflow, watch out for INT_MIN and -1.
...
This fixes PR9845.
llvm-svn: 131919
2011-05-23 21:07:39 +00:00
f44aadf0fd
indvars: Prototyping Sign/ZeroExtend elimination without canonical IVs.
...
No functionality enabled by default. Use -disable-iv-rewrite.
Extended IVUsers to keep track of the phi that represents the users' IV.
Added the WidenIV transform to replace a narrow IV with a wide IV
by doing a one-for-one replacement of IV users instead of expanding the
SCEV expressions. [sz]exts are removed and truncs are inserted.
llvm-svn: 131744
2011-05-20 18:25:42 +00:00
3268e4d692
When forming an ICmpZero LSRUse, normalize the non-IV operand
...
of the comparison, so that the resulting expression is fully
normalized. This fixes PR9939.
llvm-svn: 131576
2011-05-18 21:02:18 +00:00
6136e94897
Add an unfolded offset field to LSR's Formula record. This is used to
...
model constants which can be added to base registers via add-immediate
instructions which don't require an additional register to materialize
the immediate.
llvm-svn: 130743
2011-05-03 00:46:49 +00:00
0ab5e2cded
Fix a ton of comment typos found by codespell. Patch by
...
Luis Felipe Strano Moraes!
llvm-svn: 129558
2011-04-15 05:18:47 +00:00
b902f1dd88
Reapply r129401 with patch for clang.
...
llvm-svn: 129419
2011-04-13 00:36:11 +00:00
dbfde42468
Revert r129401 for now. Clang is using the old way of doing things.
...
llvm-svn: 129403
2011-04-12 22:59:27 +00:00
47c24875a1
Remove the unaligned load intrinsics in favor of using native unaligned loads.
...
Now that we have a first-class way to represent unaligned loads, the unaligned
load intrinsics are superfluous.
First part of <rdar://problem/8460511>.
llvm-svn: 129401
2011-04-12 22:46:31 +00:00
52131344a2
Remove PHINode::reserveOperandSpace(). Instead, add a parameter to
...
PHINode::Create() giving the (known or expected) number of operands.
llvm-svn: 128537
2011-03-30 11:28:46 +00:00
e0938d8a87
(Almost) always call reserveOperandSpace() on newly created PHINodes.
...
llvm-svn: 128535
2011-03-30 11:19:20 +00:00
8b55b736b1
Added SCEV::NoWrapFlags to manage unsigned, signed, and self wrap
...
properties.
Added the self-wrap flag for SCEV::AddRecExpr.
A slew of temporary FIXMEs indicate the intention of the no-self-wrap flag
without changing behavior in this revision.
llvm-svn: 127590
2011-03-14 16:50:06 +00:00
97dae4d361
If we can't avoid running loop-simplify twice for now, at least avoid running
...
iv-users twice.
llvm-svn: 125318
2011-02-10 23:53:14 +00:00
da6bd45088
Revert this in an attempt to bring the builders back.
...
llvm-svn: 125257
2011-02-10 01:48:24 +00:00
58c8670ab2
Turn this pass ordering:
...
Natural Loop Information
Loop Pass Manager
Canonicalize natural loops
Scalar Evolution Analysis
Loop Pass Manager
Induction Variable Users
Canonicalize natural loops
Induction Variable Users
Loop Strength Reduction
into this:
Scalar Evolution Analysis
Loop Pass Manager
Canonicalize natural loops
Induction Variable Users
Loop Strength Reduction
This fixes <rdar://problem/8869639>. I also filed PR9184 on doing this sort of
thing automatically, but it seems easier to just change the ordering of the
passes if this is the only case.
llvm-svn: 125254
2011-02-10 01:07:54 +00:00
de7f699754
Don't split any loop backedges, including backedges of loops other than
...
the active loop. This is generally desirable, and it avoids trouble
in situations such as the testcase in PR9123, though the failure
mode depends on use-list order, so it is infeasible to test.
llvm-svn: 125065
2011-02-08 00:55:13 +00:00
b703654edc
Remove code for updating dominance frontiers and some outdated references to
...
dominance and post-dominance frontiers.
llvm-svn: 123725
2011-01-18 04:11:31 +00:00
bf0aa927cc
split dom frontier handling stuff out to its own DominanceFrontier header,
...
so that Dominators.h is *just* domtree. Also prune #includes a bit.
llvm-svn: 122714
2011-01-02 22:09:33 +00:00
20d9ce21ef
Move SCEV::dominates and properlyDominates to ScalarEvolution.
...
llvm-svn: 119570
2010-11-17 21:41:58 +00:00
afd6db9932
Move SCEV::isLoopInvariant and hasComputableLoopEvolution to be member
...
functions of ScalarEvolution, in preparation for memoization and
other optimizations.
llvm-svn: 119562
2010-11-17 21:23:15 +00:00
a4fefc1949
Passes do not need to recursively initialize passes that they preserve, if
...
they do not also require them. This allows us to reduce inter-pass linkage
dependencies.
llvm-svn: 116854
2010-10-19 20:08:44 +00:00
6c18d1aac0
Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which
...
must be called in the pass's constructor. This function uses static dependency declarations to recursively initialize
the pass's dependencies.
Clients that only create passes through the createFooPass() APIs will require no changes. Clients that want to use the
CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h
before parsing commandline arguments.
I have tested this with all standard configurations of clang and llvm-gcc on Darwin. It is possible that there are problems
with the static dependencies that will only be visible with non-standard options. If you encounter any crash in pass
registration/creation, please send the testcase to me directly.
llvm-svn: 116820
2010-10-19 17:21:58 +00:00
8ac477ffb5
Begin adding static dependence information to passes, which will allow us to
...
perform initialization without static constructors AND without explicit initialization
by the client. For the moment, passes are required to initialize both their
(potential) dependencies and any passes they preserve. I hope to be able to relax
the latter requirement in the future.
llvm-svn: 116334
2010-10-12 19:48:12 +00:00
2fd85d7cd2
Filter out illegal formulae after updating offsets, not before, so that
...
formulae which become illegal as a result of the offset updating don't
escape.
This is for rdar://8529692. No testcase yet, because the given cases
hit use-list ordering differences.
llvm-svn: 116093
2010-10-08 19:33:26 +00:00
5947e1626a
Delete the FormulaSorter class and inline its one method into its
...
one user. This code will be restructured soon and FormulaSorter
is getting in the way.
llvm-svn: 116012
2010-10-07 23:52:18 +00:00
1b61fd9bff
Fix a spello.
...
llvm-svn: 116011
2010-10-07 23:43:09 +00:00
34f37e0d04
Charge a formula for explicit multiplies on scaled registers too,
...
not just base registers.
llvm-svn: 116010
2010-10-07 23:41:58 +00:00
49d638b45a
Use size_t for consistency.
...
llvm-svn: 116009
2010-10-07 23:37:58 +00:00
8e72611058
When merging one use into another, transfer the offsets from
...
the old use to the new one.
llvm-svn: 116008
2010-10-07 23:36:45 +00:00
a7b68d6d95
Fix LSR to keep the RegUseTracker up to date when combining users.
...
This doesn't usually matter, because the other heuristics usually
succeed regardless, but it's good to keep the register use
bookkeeping consistent.
llvm-svn: 116005
2010-10-07 23:33:43 +00:00
df7a4f2515
Now with fewer extraneous semicolons!
...
llvm-svn: 115996
2010-10-07 22:25:06 +00:00
16ef49686c
Delete an unused function.
...
llvm-svn: 114841
2010-09-27 16:58:21 +00:00
a06741b356
do not rely on the implicit-dereference semantics of dyn_cast_or_null
...
llvm-svn: 114278
2010-09-18 11:55:34 +00:00
110ed64fbb
Revert 112442 and 112440 until the compile time problems introduced
...
by 112440 are resolved.
llvm-svn: 112692
2010-09-01 01:45:53 +00:00
002ff89cbd
Optionally rerun dedicated-register filtering after applying
...
other filtering techniques, as those may allow it to filter
out more obviously unprofitable candidates.
llvm-svn: 112441
2010-08-29 16:39:22 +00:00
f031792cc6
Fix several areas in LSR to do a better job keeping the main
...
LSRInstance data structures up to date. This fixes some
pessimizations caused by stale data which will be exposed
in an upcoming change.
llvm-svn: 112440
2010-08-29 16:32:54 +00:00
e9e0873b08
Refactor the three main groups of code out of
...
NarrowSearchSpaceUsingHeuristics into separate functions.
llvm-svn: 112439
2010-08-29 16:09:42 +00:00
37a0f68036
Delete a bogus check.
...
llvm-svn: 112438
2010-08-29 15:30:29 +00:00
b6a520d63c
Add some comments.
...
llvm-svn: 112437
2010-08-29 15:27:08 +00:00
bf673e0652
Move this debug output into GenerateAllReuseFormula, to declutter
...
the high-level logic.
llvm-svn: 112436
2010-08-29 15:21:38 +00:00
d366b6d5c8
Delete an unused declaration.
...
llvm-svn: 112435
2010-08-29 15:19:11 +00:00
4f13bbfefc
Do one lookup instead of two.
...
llvm-svn: 112434
2010-08-29 15:18:49 +00:00
129a816ee6
Process the step before the start, because it's usually the simpler
...
of the two.
llvm-svn: 111495
2010-08-19 01:02:31 +00:00
89fdbaf99a
Instead of having CollectSubexpr's categorize operands as interesting or
...
uninteresting, just put all the operands on one list and make
GenerateReassociations make the decision about what's interesting.
This is simpler, and it avoids an extra ScalarEvolution::getAddExpr call.
llvm-svn: 111133
2010-08-16 15:50:00 +00:00
9b7632df26
Put add operands in ScalarEvolution-canonical order, when convenient.
...
This isn't necessary, because ScalarEvolution sorts them anyway,
but it's tidier this way.
llvm-svn: 111132
2010-08-16 15:39:27 +00:00
081ffcd00b
Fix LSR's ExtractImmediate and ExtractSymbol to avoid calling
...
ScalarEvolution::getAddExpr, which can be pretty expensive, when nothing
has changed, which is pretty common.
llvm-svn: 111042
2010-08-13 21:17:19 +00:00
a7aed18624
Reapply r110396, with fixes to appease the Linux buildbot gods.
...
llvm-svn: 110460
2010-08-06 18:33:48 +00:00
bda59bd247
Revert r110396 to fix buildbots.
...
llvm-svn: 110410
2010-08-06 00:23:35 +00:00
755aceb5d0
Don't use PassInfo* as a type identifier for passes. Instead, use the address of the static
...
ID member as the sole unique type identifier. Clean up APIs related to this change.
llvm-svn: 110396
2010-08-05 23:42:04 +00:00
ba81fc16a5
Fix whitespace.
...
llvm-svn: 110223
2010-08-04 17:43:57 +00:00
40b31ad3ee
Prefix `next' iterator operation with `llvm::'.
...
Fixes potential ambiguity problems on VS 2010.
Patch by nobled!
llvm-svn: 110029
2010-08-02 06:00:15 +00:00
a57b97e7e7
Fix batch of converting RegisterPass<> to INTIALIZE_PASS().
...
llvm-svn: 109045
2010-07-21 22:09:45 +00:00
1415208292
Don't merge uses when they are targetting fixup sites with
...
different widths. In a use with a narrower fixup, formulae
may be wider than the fixup, in which case the high bits
aren't necessarily meaningful, so it isn't safe to reuse
them for uses with wider fixups.
This fixes PR7618, though the testcase is too large for a
reasonable regression test, since it heavily dependes on
hitting LSR's heuristics in a certain way.
llvm-svn: 108455
2010-07-15 20:24:58 +00:00
a1501b9c50
Use dbgs() instead of errs() in a DEBUG.
...
llvm-svn: 108453
2010-07-15 20:12:42 +00:00
4afd412d6b
Watch out for a constant offset cancelling out a base register, forming
...
a zero. This situation arrises in Fortran code with induction variables
that start at 1 instead of 0. This fixes PR7651.
llvm-svn: 108424
2010-07-15 15:14:45 +00:00
8ae3095286
use getArgOperand instead of getOperand
...
llvm-svn: 107270
2010-06-30 09:15:28 +00:00
fb9712bdae
In GenerateReassociations, don't bother thinking about individual
...
SCEVUnknown values which are loop-variant, as LSR can't do anything
interesting with these values in any case. This fixes very slow compile
times on loops which have large numbers of such values.
llvm-svn: 106897
2010-06-25 22:32:18 +00:00
963b1c142e
A few minor micro-optimizations.
...
llvm-svn: 106764
2010-06-24 16:57:52 +00:00
47ddf76d89
Teach getExactSDiv to evaluate x/1 to x up front, as it's a common
...
enough special case, and it theoretically allows more folding because
it works even when x is unanalyzable.
llvm-svn: 106763
2010-06-24 16:51:25 +00:00
ab5422200b
Fix copy+pasto issues in isMulSExtable.
...
llvm-svn: 106759
2010-06-24 16:45:11 +00:00
1081f1a0f5
Fix OptimizeMax to handle an odd case where one of the max operands
...
is another max which folds. This fixes PR7454.
llvm-svn: 106594
2010-06-22 23:07:13 +00:00
dd41bba517
Use A.append(...) instead of A.insert(A.end(), ...) when A is a
...
SmallVector, and other SmallVector simplifications.
llvm-svn: 106452
2010-06-21 19:47:52 +00:00
32655906e4
Add a TODO comment.
...
llvm-svn: 106397
2010-06-19 21:30:18 +00:00
51d00092b6
Include the use kind along with the expression in the key of the
...
use sharing map. The reconcileNewOffset logic already forces a
separate use if the kinds differ, so incorporating the kind in the
key means we can track more sharing opportunities.
More sharing means fewer total uses to track, which means smaller
problem sizes, which means the conservative throttles don't kick
in as often.
llvm-svn: 106396
2010-06-19 21:29:59 +00:00
297fb8b9fc
Don't include things in anonymous namespaces that don't need it.
...
llvm-svn: 106395
2010-06-19 21:21:39 +00:00
67b4403101
Don't track users of undef values; they aren't interesting for
...
register pressure.
llvm-svn: 105501
2010-06-04 23:16:05 +00:00
9b48b856ea
DominatorTree.getNode can return null for unreachable blocks.
...
llvm-svn: 104290
2010-05-20 22:46:54 +00:00
86110fa2bb
Minor code cleanups.
...
llvm-svn: 104287
2010-05-20 22:25:20 +00:00
6295f2ebb8
Make Solve check its own post-condition, to reduce clutter in the
...
top-level LSRInstance logic.
llvm-svn: 104278
2010-05-20 20:59:23 +00:00
a4ca28a3ae
Add comments.
...
llvm-svn: 104276
2010-05-20 20:52:00 +00:00
927bcaadda
More code cleanups. Use iterators instead of indices when indices
...
aren't needed.
llvm-svn: 104273
2010-05-20 20:33:18 +00:00
4c4043cf34
Fix OptimizeShadowIV to set Changed. Change OptimizeLoopTermCond to set
...
Changed directly instead of using a return value.
Rename FilterOutUndesirableDedicatedRegisters's Changed variable to
distinguish it from LSRInstance's Changed member.
llvm-svn: 104269
2010-05-20 20:05:31 +00:00
8ec018cedf
Add some comments.
...
llvm-svn: 104268
2010-05-20 20:00:41 +00:00
8ce95cc3c5
Simplify this code. Don't do a DomTreeNode lookup for each visited block.
...
llvm-svn: 104267
2010-05-20 20:00:25 +00:00
ab5fb7f559
Minor code cleanups.
...
llvm-svn: 104263
2010-05-20 19:44:23 +00:00
ee2fea3cd7
When canonicalizing icmp operand order to put the loop invariant
...
operand on the left, the interesting operand is on the right. This
fixes a bug where LSR was failing to recognize ICmpZero uses,
which led it to be unable to reverse the induction variable in the
attached testcase.
Delete test/CodeGen/X86/stack-color-with-reg-2.ll, because its test
is extremely fragile and hard to meaningfully update.
llvm-svn: 104262
2010-05-20 19:26:52 +00:00
fdf9874ba7
Set Changed to true when canonicalizing ICmp operand order; even though
...
it isn't a very interesting change, it's a change nonetheless.
llvm-svn: 104260
2010-05-20 19:16:03 +00:00
981563d0ba
Rename a variable to avoid shadowing.
...
llvm-svn: 104234
2010-05-20 16:41:11 +00:00
6b733fc189
Minor code simplification.
...
llvm-svn: 104232
2010-05-20 16:23:28 +00:00
80a9608442
Move the code for deleting BaseRegs and LSRUses into helper functions,
...
and fix a bug that valgrind noticed where the code would std::swap an
element with itself.
llvm-svn: 104225
2010-05-20 15:17:54 +00:00
20fab456da
Teach LSR how to cope better with unrolled loops on targets where
...
the addressing modes don't make this trivially easy. This allows
it to avoid falling into the less precise heuristics in more
cases.
llvm-svn: 104186
2010-05-19 23:43:12 +00:00
beebef4137
Add a comment.
...
llvm-svn: 104089
2010-05-18 23:55:57 +00:00
50f8f2c23d
Fix the predicate which checks for non-sensical formulae which have
...
constants in registers which partially cancel out their immediate fields.
llvm-svn: 104088
2010-05-18 23:48:08 +00:00
4cf99b5303
Factor out the code for recomputing an LSRUse's Regs set after some
...
of its formulae have been removed into a helper function, and also
teach it how to update the RegUseTracker.
llvm-svn: 104087
2010-05-18 23:42:37 +00:00
a4eca05174
Factor out code for estimating search space complexity into a helper
...
function.
llvm-svn: 104082
2010-05-18 22:51:59 +00:00
63e9015248
Add some more debug output.
...
llvm-svn: 104080
2010-05-18 22:41:32 +00:00
f1c7b1b42f
Factor out the code for deleting a formula from an LSRUse into
...
a helper function.
llvm-svn: 104079
2010-05-18 22:39:15 +00:00
8aca7ef903
Make some debug output more informative.
...
llvm-svn: 104078
2010-05-18 22:37:37 +00:00
06ab08f795
Print an error message in Formula::print if the HasBaseReg flag
...
is inconsistent with the BaseRegs field. It's not print's job to
assert on an invalid condition, but it can make one more obvious.
llvm-svn: 104077
2010-05-18 22:35:55 +00:00
248c41d108
Rename RegUseTracker's RegUses member to RegUsesMap to avoid
...
confusion with LSRInstance's RegUses member.
llvm-svn: 104076
2010-05-18 22:33:00 +00:00
6739a89117
Fixes for Microsoft Visual Studio 2010, from Steven Watanabe!
...
llvm-svn: 103457
2010-05-11 06:17:44 +00:00
d0800241d2
When pruning candidate formulae out of an LSRUse, update the
...
LSRUse's Regs set after all pruning is done, rather than trying
to do it on the fly, which can produce an incomplete result.
This fixes a case where heuristic pruning was stripping all
formulae from a use, which led the solver to enter an infinite
loop.
Also, add a few asserts to diagnose this kind of situation.
llvm-svn: 103328
2010-05-07 23:36:59 +00:00
1d2ded75e2
Use getConstant instead of getIntegerSCEV. The two are basically the
...
same, now that getConstant has overloads consistent with ConstantInt::get.
llvm-svn: 102965
2010-05-03 22:09:21 +00:00
534ba376f6
Generalize LSR's OptimizeMax to handle the new kinds of max expressions
...
that indvars may use, now that indvars is recognizing le and ge loops.
llvm-svn: 102235
2010-04-24 03:13:44 +00:00
997bbc54d6
Fix LSR to tolerate cases where ScalarEvolution initially
...
misses an opportunity to fold add operands, but folds them
after LSR has separated them out. This fixes rdar://7886751.
llvm-svn: 102157
2010-04-23 01:55:05 +00:00
e637ff5e9a
Remove the Expr member from IVUsers. Instead of remembering the expression,
...
just ask ScalarEvolution for it on demand. This helps IVUsers be more robust
in the case of expressions changing underneath it. This fixes PR6862.
llvm-svn: 101819
2010-04-19 21:48:58 +00:00
27c8e79839
Delete this code, which is no longer needed.
...
llvm-svn: 101033
2010-04-12 08:00:22 +00:00
607e02b33a
When determining a canonical insert position, don't climb deeper
...
into adjacent loops. Also, ensure that the insert position is
dominated by the loop latch of any loop in the post-inc set which
has a latch.
llvm-svn: 100906
2010-04-09 22:07:05 +00:00
42ec4eb351
When looking for loop-invariant users, look through no-op instructions,
...
so that an unfortunately placed bitcast doesn't pin a value in a
register.
llvm-svn: 100883
2010-04-09 19:12:34 +00:00
d2df643ddb
Refactor the code for computing the insertion point for an expression into
...
a separate function.
llvm-svn: 100845
2010-04-09 02:00:38 +00:00
9b5d0bb774
Avoid allocating a value of zero in a register if the initial formula
...
inputs happen to negate each other.
llvm-svn: 100828
2010-04-08 23:36:27 +00:00
4ce1fb1448
Add variants of ult, ule, etc. which take a uint64_t RHS, for convenience.
...
llvm-svn: 100824
2010-04-08 23:03:40 +00:00
4506539d84
When expanding expressions which are using post-inc mode for multiple loops,
...
ensure that the expansion is dominated by the increments of those loops.
llvm-svn: 100748
2010-04-08 05:57:57 +00:00
d006ab90dd
Generalize IVUsers to track arbitrary expressions rather than expressions
...
explicitly split into stride-and-offset pairs. Also, add the
ability to track multiple post-increment loops on the same expression.
This refines the concept of "normalizing" SCEV expressions used for
to post-increment uses, and introduces a dedicated utility routine for
normalizing and denormalizing expressions.
This fixes the expansion of expressions which are post-increment users
of more than one loop at a time. More broadly, this takes LSR another
step closer to being able to reason about more than one loop at a time.
llvm-svn: 100699
2010-04-07 22:27:08 +00:00
d42e09d91e
Ignore debug intrinsics in yet more places.
...
llvm-svn: 99580
2010-03-26 00:33:27 +00:00
c78d720f02
rename use_const_iterator to const_use_iterator for consistency's sake
...
llvm-svn: 99564
2010-03-25 23:06:16 +00:00
29707de4fe
Make SCEVExpander and LSR more aggressive about hoisting expressions out
...
of loops.
llvm-svn: 97642
2010-03-03 05:29:13 +00:00
52f5563973
Non-affine post-inc SCEV expansions have more code which must be
...
emitted after the increment. Make sure the insert position
reflects this. This fixes PR6453.
llvm-svn: 97537
2010-03-02 01:59:21 +00:00
8b0a419eb1
Spelling fixes.
...
llvm-svn: 97453
2010-03-01 17:49:51 +00:00
8c16b38262
Remove unused variables and parameters.
...
llvm-svn: 96780
2010-02-22 04:11:59 +00:00
4506fcb3c2
When emitting an instruction which depends on both a post-incremented
...
induction variable value and a loop-variant value, don't force the
insert position to be at the post-increment position, because it may
not be dominated by the loop-variant value. This fixes a
use-before-def problem noticed on PPC.
llvm-svn: 96774
2010-02-22 03:59:54 +00:00
4eebb94094
Rename getSDiv to getExactSDiv to reflect its behavior in cases where
...
the division would have a remainder.
llvm-svn: 96693
2010-02-19 19:35:48 +00:00
85af256779
Check for overflow when scaling up an add or an addrec for
...
scaled reuse.
llvm-svn: 96692
2010-02-19 19:32:49 +00:00
2446f57503
When determining the set of interesting reuse factors, consider
...
strides in foreign loops. This helps locate reuse opportunities
with existing induction variables in foreign loops and reduces
the need for inserting new ones. This fixes rdar://7657764.
llvm-svn: 96629
2010-02-19 00:05:23 +00:00
13ac3b2139
Delete some unneeded casts.
...
llvm-svn: 96429
2010-02-17 00:42:19 +00:00
5f10d6c52c
Don't attempt to divide INT_MIN by -1; consider such cases to
...
have overflowed.
llvm-svn: 96428
2010-02-17 00:41:53 +00:00
6deab96c81
Refactor rewriting for PHI nodes into a separate function.
...
llvm-svn: 96382
2010-02-16 20:25:07 +00:00
0849ed5e26
Fix whitespace.
...
llvm-svn: 96372
2010-02-16 19:42:34 +00:00
19d0b47b1f
There are two ways of checking for a given type, for example isa<PointerType>(T)
...
and T->isPointerTy(). Convert most instances of the first form to the second form.
Requested by Chris.
llvm-svn: 96344
2010-02-16 11:11:14 +00:00
521efe68ab
Split the main for-each-use loop again, this time for GenerateTruncates,
...
as it also peeks at which registers are being used by other uses. This
makes LSR less sensitive to use-list order.
llvm-svn: 96308
2010-02-16 01:42:53 +00:00
e4e51a63da
Fix whitespace.
...
llvm-svn: 96179
2010-02-14 18:51:39 +00:00
e7f74bb16c
Fix a comment.
...
llvm-svn: 96178
2010-02-14 18:51:20 +00:00
bb7d52213c
When complicated expressions are broken down into subexpressions
...
with multiplication by constants distributed through, occasionally
those subexpressions can include both x and -x. For now, if this
condition is discovered within LSR, just prune such cases away,
as they won't be profitable. This fixes a "zero allocated in a
base register" assertion failure.
llvm-svn: 96177
2010-02-14 18:50:49 +00:00
2d0f96d49a
Actually, this code doesn't have to be quite so conservative in
...
the no-TLI case. But it should still default to declining the
transformation.
llvm-svn: 96152
2010-02-14 03:21:49 +00:00
cb76a806f0
Don't attempt aggressive post-inc uses if TargetLowering is not available,
...
because profitability can't be sufficiently approximated.
llvm-svn: 96148
2010-02-14 02:45:21 +00:00
0daaf13b97
Make LSR not crash if invoked without target lowering info, e.g. if invoked
...
from opt.
llvm-svn: 96135
2010-02-13 23:40:16 +00:00
5b18f039eb
Fix a pruning heuristic which implicitly assumed that SmallPtrSet is
...
deterministically sorted.
llvm-svn: 96071
2010-02-13 02:06:02 +00:00
2b75de97c0
Reapply 95979, a compile-time speedup, now that the bug it exposed is fixed.
...
llvm-svn: 96005
2010-02-12 19:35:25 +00:00
363f847ec6
Fix this code to avoid dereferencing an end() iterator in
...
offset distributions it doesn't expect.
llvm-svn: 96002
2010-02-12 19:20:37 +00:00
e0b2c69d3c
Revert "Reverse the order for collecting the parts of an addrec. The order", it
...
is breaking llvm-gcc bootstrap.
llvm-svn: 95988
2010-02-12 17:27:08 +00:00
0194f58047
Reverse the order for collecting the parts of an addrec. The order
...
doesn't matter, except that ScalarEvolution tends to need less time
to fold the results this way.
llvm-svn: 95979
2010-02-12 11:08:26 +00:00
45774ce0ad
Reapply the new LoopStrengthReduction code, with compile time and
...
bug fixes, and with improved heuristics for analyzing foreign-loop
addrecs.
This change also flattens IVUsers, eliminating the stride-oriented
groupings, which makes it easier to work with.
llvm-svn: 95975
2010-02-12 10:34:29 +00:00
48816a0b3f
Generic reformatting and comment fixing. No functionality change.
...
llvm-svn: 94771
2010-01-29 00:52:43 +00:00
8277838cf8
Add newline to debugging output, and fix some grammar-os in comment.
...
llvm-svn: 94765
2010-01-29 00:27:39 +00:00
045f81981a
Revert LoopStrengthReduce.cpp to pre-r94061 for now.
...
llvm-svn: 94123
2010-01-22 00:46:49 +00:00
b1ee154b6b
When inserting expressions for post-increment users which contain
...
loop-variant components, adds must be inserted after the increment.
Keep track of the increment position for this case, and insert
these adds in the correct location.
llvm-svn: 94110
2010-01-21 23:01:22 +00:00
cb8d577eb2
Include IVUsers information in LSR's debug output.
...
llvm-svn: 94108
2010-01-21 22:46:32 +00:00
29916e023d
Prune the search for candidate formulae if the number of register
...
operands exceeds the number of registers used in the initial
solution, as that wouldn't lead to a profitable solution anyway.
llvm-svn: 94107
2010-01-21 22:42:49 +00:00
c903499ff8
Add a comment.
...
llvm-svn: 94104
2010-01-21 21:31:09 +00:00
51ad99d2c5
Re-implement the main strength-reduction portion of LoopStrengthReduction.
...
This new version is much more aggressive about doing "full" reduction in
cases where it reduces register pressure, and also more aggressive about
rewriting induction variables to count down (or up) to zero when doing so
reduces register pressure.
It currently uses fairly simplistic algorithms for finding reuse
opportunities, but it introduces a new framework allows it to combine
multiple strategies at once to form hybrid solutions, instead of doing
all full-reduction or all base+index.
llvm-svn: 94061
2010-01-21 02:09:26 +00:00
b5358003fb
Set Changed properly after calling DeleteDeadPHIs.
...
llvm-svn: 92735
2010-01-05 16:31:45 +00:00
2330f78075
Remove dump routine and the associated Debug.h from a header. Patch up
...
other files to compensate.
llvm-svn: 92075
2009-12-23 22:58:38 +00:00
18fa5686f6
Add Loop contains utility methods for testing whether a loop
...
contains another loop, or an instruction. The loop form is
substantially more efficient on large loops than the typical
code it replaces.
llvm-svn: 91654
2009-12-18 01:24:09 +00:00
fd7231f1fe
Minor code simplification.
...
llvm-svn: 91653
2009-12-18 01:20:44 +00:00
b1924e8a0f
Don't pass const pointers by reference.
...
llvm-svn: 91647
2009-12-18 00:38:08 +00:00
92c3696524
Reapply LoopStrengthReduce and IVUsers cleanups, excluding the part
...
of 91296 that caused trouble -- the Processed list needs to be
preserved for the livetime of the pass, as AddUsersIfInteresting
is called from other passes.
llvm-svn: 91641
2009-12-18 00:06:20 +00:00
090ac0865a
Revert 91280-91283, 91286-91289, 91291, 91293, 91295-91296. It apparently introduced a non-deterministic behavior in the optimizer somewhere.
...
llvm-svn: 91598
2009-12-17 09:39:49 +00:00
265ce318b8
Delete an unused function.
...
llvm-svn: 91432
2009-12-15 16:30:09 +00:00
fbeec7270c
Fix a thinko; isNotAlreadyContainedIn had a built-in negative, so the
...
condition was inverted when the code was converted to contains().
llvm-svn: 91295
2009-12-14 17:31:01 +00:00
416d5b7361
Remove unnecessary #includes.
...
llvm-svn: 91293
2009-12-14 17:19:06 +00:00
163fb26927
Instead of having a ScalarEvolution pointer member in BasedUser, just pass
...
the ScalarEvolution pointer into the functions which need it.
llvm-svn: 91289
2009-12-14 17:12:51 +00:00
8dbd4e3d16
Don't bother cleaning up if there's nothing to clean up.
...
llvm-svn: 91288
2009-12-14 17:10:44 +00:00
88c7e61c5b
Delete an unused variable.
...
llvm-svn: 91287
2009-12-14 17:08:09 +00:00
838f604543
LSR itself doesn't need LoopInfo.
...
llvm-svn: 91283
2009-12-14 17:02:34 +00:00
273e692952
LSR itself doesn't need DominatorTree.
...
llvm-svn: 91282
2009-12-14 16:57:08 +00:00
c3513095cf
Remove the code in LSR that manually hoists expansions out of loops;
...
SCEVExpander does this automatically.
llvm-svn: 91281
2009-12-14 16:52:55 +00:00
ec2a7c58e8
Minor code cleanups.
...
llvm-svn: 91280
2009-12-14 16:37:29 +00:00
6bf5305f5d
grammar
...
llvm-svn: 89145
2009-11-17 21:37:04 +00:00
e4e018ae67
80-column violations
...
llvm-svn: 89123
2009-11-17 19:05:35 +00:00
ba4e5da727
Generalize OptimizeLoopTermCond to optimize more loop terminating icmp to use postinc iv.
...
llvm-svn: 89116
2009-11-17 18:10:11 +00:00
60f4854c76
Remove trailing whitespace
...
llvm-svn: 89110
2009-11-17 17:53:56 +00:00
85a9f430e9
- Teach LSR to avoid changing cmp iv stride if it will create an immediate that
...
cannot be folded into target cmp instruction.
- Avoid a phase ordering issue where early cmp optimization would prevent the
later count-to-zero optimization.
- Add missing checks which could cause LSR to reuse stride that does not have
users.
- Fix a bug in count-to-zero optimization code which failed to find the pre-inc
iv's phi node.
- Remove, tighten, loosen some incorrect checks disable valid transformations.
- Quite a bit of code clean up.
llvm-svn: 86969
2009-11-12 07:35:05 +00:00
12f146d8f7
Block terminator may be a switch.
...
llvm-svn: 86761
2009-11-11 00:00:21 +00:00
87fe40b32d
Generalize lsr code that optimize loop to count down towards zero.
...
llvm-svn: 86715
2009-11-10 21:14:05 +00:00
a1bf0c0acc
Teach LSR to avoid calling SplitCriticalEdge on edges with indirectbr.
...
llvm-svn: 86193
2009-11-05 23:34:59 +00:00
a83ac2d9e7
Update various Loop optimization passes to cope with the possibility that
...
LoopSimplify form may not be available.
llvm-svn: 86175
2009-11-05 21:11:53 +00:00
d632f89596
Make LSR's OptimizeShadowIV ignore induction variables with negative
...
strides for now, because it doesn't handle them correctly. This fixes a
miscompile of SingleSource/Benchmarks/Misc-C++/ray.
This problem was usually hidden because indvars transforms such induction
variables into negations of canonical induction variables.
llvm-svn: 85118
2009-10-26 15:32:57 +00:00
54d7179a25
Remove ICmpInst::isSignedPredicate which was a reimplementation
...
CmpInst::isSigned.
llvm-svn: 85037
2009-10-25 05:20:17 +00:00
42fb7452df
Instruction::clone does not need to take an LLVMContext&. Remove that and
...
update all the callers.
llvm-svn: 82889
2009-09-27 07:38:41 +00:00
3ddbc242fb
Re-apply r80926, with fixes: keep the domtree informed of new blocks
...
that get created during loop unswitching, and fix SplitBlockPredecessors'
LCSSA updating code to create new PHIs instead of trying to just move
existing ones.
Also, optimize Loop::verifyLoop, since it gets called a lot. Use
searches on a sorted list of blocks instead of calling the "contains"
function, as is done in other places in the Loop class, since "contains"
does a linear search. Also, don't call verifyLoop from LoopSimplify or
LCSSA, as the PassManager is already calling verifyLoop as part of
LoopInfo's verifyAnalysis.
llvm-svn: 81221
2009-09-08 15:45:00 +00:00
904199547b
Revert r80926. It causes loop unswitch assertion and slow down some JIT tests significantly.
...
llvm-svn: 81101
2009-09-06 02:26:10 +00:00
4c1bdcf5d7
Add a verifyAnalysis to LoopInfo, LoopSimplify, and LCSSA form that verify
...
that these passes are properly preserved.
Fix several transformation passes that claimed to preserve LoopSimplify
form but weren't.
llvm-svn: 80926
2009-09-03 16:31:42 +00:00
2dd09dbdf7
eliminate VISIBILITY_HIDDEN from Transforms/Scalar. PR4861
...
llvm-svn: 80766
2009-09-02 06:11:42 +00:00
b25de3ff60
eliminate the "Value" printing methods that print to a std::ostream.
...
This required converting a bunch of stuff off DOUT and other cleanups.
llvm-svn: 79819
2009-08-23 04:37:46 +00:00
55f1c09e31
Push LLVMContexts through the IntegerType APIs.
...
llvm-svn: 78948
2009-08-13 21:58:54 +00:00
53aa7a960c
Rename MVT to EVT, in preparation for splitting SimpleValueType out into its own struct type.
...
llvm-svn: 78610
2009-08-10 22:56:29 +00:00
298bce2aa9
Check for !isa<Constant> instead of isa<Instruction>. This
...
matches what the comment says, and it avoids spurious BitCast
instructions for Argument values.
llvm-svn: 78121
2009-08-04 23:23:56 +00:00
5a1acd9912
Move a few more APIs back to 2.5 forms. The only remaining ones left to change back are
...
metadata related, which I'm waiting on to avoid conflicting with Devang.
llvm-svn: 77721
2009-07-31 20:28:14 +00:00
47d02732e0
Eliminate a few unused-variable warnings
...
llvm-svn: 77519
2009-07-29 22:41:10 +00:00
4056ca9568
Move types back to the 2.5 API.
...
llvm-svn: 77516
2009-07-29 22:17:13 +00:00
487375e9a2
Move ConstantExpr to 2.5 API.
...
llvm-svn: 77494
2009-07-29 18:55:55 +00:00
69c464dec4
Move ConstantFP construction back to the 2.5-ish API.
...
llvm-svn: 77247
2009-07-27 20:59:43 +00:00
6115b39ffd
Remove Value::getName{Start,End}, the last of the old Name APIs.
...
llvm-svn: 77152
2009-07-26 09:48:23 +00:00
edb4a70325
Revert the ConstantInt constructors back to their 2.5 forms where possible, thanks to contexts-on-types. More to come.
...
llvm-svn: 77011
2009-07-24 23:12:02 +00:00
47db941fd3
Get rid of the Pass+Context magic.
...
llvm-svn: 76702
2009-07-22 00:24:57 +00:00
Andrew Trick