Commit Graph

511 Commits

Author SHA1 Message Date
Vedant Kumar 282b26ec4d [GVN+LICM] Use line 0 locations for better crash attribution
This is a follow-up to r291037+r291258, which used null debug locations
to prevent jumpy line tables.

Using line 0 locations achieves the same effect, but works better for
crash attribution because it preserves the right inline scope.

Differential Revision: https://reviews.llvm.org/D60913

llvm-svn: 358791
2019-04-19 22:36:40 +00:00
Alina Sbirlea 43709f7233 [LICM & MemorySSA] Make limit flags pass tuning options.
Summary:
Make the flags in LICM + MemorySSA tuning options in the old and new
pass managers.

Subscribers: mehdi_amini, jlebar, Prazek, george.burgess.iv, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60490

llvm-svn: 358772
2019-04-19 17:46:50 +00:00
Nick Lewycky 1e1e212d27 [NFC] Remove dead parameter "FreeInLoop", fix some typos and trailing whitespace.
Differential Revision: https://reviews.llvm.org/D60084

llvm-svn: 357427
2019-04-01 20:37:56 +00:00
Simon Pilgrim 92cbcfc325 Fix -Wmisleading-indentation gcc7 warning. NFCI.
llvm-svn: 356658
2019-03-21 11:58:22 +00:00
Mikael Holmen 5b1754f93d Silence warning about unused variable in builds without asserts [NFC]
llvm-svn: 356648
2019-03-21 07:54:44 +00:00
Alina Sbirlea f69f807321 [NFC] Fix brace indentation.
llvm-svn: 356596
2019-03-20 19:18:55 +00:00
Alina Sbirlea 5baa72ea74 [LICM & MemorySSA] Don't sink/hoist stores in the presence of ordered loads.
Summary:
Before this patch, if any Use existed in the loop, with a defining
access in the loop, we conservatively decide to not move the store.
What this approach was missing, is that ordered loads are not Uses, they're Defs
in MemorySSA. So, even when the clobbering walker does not find that
volatile load to interfere, we still cannot hoist a store past a
volatile load.
Resolves PR41140.

Reviewers: george.burgess.iv

Subscribers: sanjoy, jlebar, Prazek, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59564

llvm-svn: 356588
2019-03-20 18:33:37 +00:00
Philip Reames 2226e9a745 [LICM] Infer proper alignment from loads during scalar promotion
This patch fixes an issue where we would compute an unnecessarily small alignment during scalar promotion when no store is not to be guaranteed to execute, but we've proven load speculation safety. Since speculating a load requires proving the existing alignment is valid at the new location (see Loads.cpp), we can use the alignment fact from the load.

For non-atomics, this is a performance problem. For atomics, this is a correctness issue, though an *incredibly* rare one to see in practice. For atomics, we might not be able to lower an improperly aligned load or store (i.e. i32 align 1). If such an instruction makes it all the way to codegen, we *may* fail to codegen the operation, or we may simply generate a slow call to a library function. The part that makes this super hard to see in practice is that the memory location actually *is* well aligned, and instcombine knows that. So, to see a failure, you have to have a) hit the bug in LICM, b) somehow hit a depth limit in InstCombine/ValueTracking to avoid fixing the alignment, and c) then have generated an instruction which fails codegen rather than simply emitting a slow libcall. All around, pretty hard to hit.

Differential Revision: https://reviews.llvm.org/D58809

llvm-svn: 355217
2019-03-01 18:45:05 +00:00
Alina Sbirlea fcfa7c5f92 [MemorySSA] Make insertDef insert corresponding phi nodes.
Summary:
The original assumption for the insertDef method was that it would not
materialize Defs out of no-where, hence it will not insert phis needed
after inserting a Def.

However, when cloning an instruction (use case used in LICM), we do
materialize Defs "out of no-where". If the block receiving a Def has at
least one other Def, then no processing is needed. If the block just
received its first Def, we must check where Phi placement is needed.
The only new usage of insertDef is in LICM, hence the trigger for the bug.

But the original goal of the method also fails to apply for the move()
method. If we move a Def from the entry point of a diamond to either the
left or right blocks, then the merge block must add a phi.
While this usecase does not currently occur, or may be viewed as an
incorrect transformation, MSSA must behave corectly given the scenario.

Resolves PR40749 and PR40754.

Reviewers: george.burgess.iv

Subscribers: sanjoy, jlebar, Prazek, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58652

llvm-svn: 355040
2019-02-27 22:20:22 +00:00
Max Kazantsev deaf2ba280 [NFC] Refactor LICM code for better readability
llvm-svn: 354013
2019-02-14 09:04:12 +00:00
Alina Sbirlea 8567ff0c34 [LICM] Cap the clobbering calls in LICM.
Summary:
Unlimitted number of calls to getClobberingAccess can lead to high
compile times in pathological cases.
Switching EnableLicmCap flag from bool to int, and enabling to default 100.
(tested to be appropriate for current bechmarks)
We can revisit this value when enabling MemorySSA.

Reviewers: sanjoy, chandlerc, george.burgess.iv

Subscribers: jlebar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57968

llvm-svn: 353897
2019-02-12 23:05:40 +00:00
Alina Sbirlea 605b21739d [LICM&MSSA] Limit store hoisting.
Summary:
If there is no clobbering access for a store inside the loop, that store
can only be hoisted if there are no interfearing loads.
A more general verification introduced here: there are no loads that are
not optimized to an access outside the loop.
Addresses PR40586.

Reviewers: george.burgess.iv

Subscribers: sanjoy, jlebar, Prazek, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57967

llvm-svn: 353734
2019-02-11 19:07:15 +00:00
Alina Sbirlea 6cba96ed52 [LICM/MSSA] Add promotion to scalars by building an AliasSetTracker with MemorySSA.
Summary:
Experimentally we found that promotion to scalars carries less benefits
than sinking and hoisting in LICM. When using MemorySSA, we build an
AliasSetTracker on demand in order to reuse the current infrastructure.
We only build it if less than AccessCapForMSSAPromotion exist in the
loop, a cap that is by default set to 250. This value ensures there are
no runtime regressions, and there are small compile time gains for
pathological cases. A much lower value (20) was found to yield a single
regression in the llvm-test-suite and much higher benefits for compile
times. Conservatively we set the current cap to a high value, but we will
explore lowering it when MemorySSA is enabled by default.

Reviewers: sanjoy, chandlerc

Subscribers: nemanjai, jlebar, Prazek, george.burgess.iv, jfb, jsji, llvm-commits

Differential Revision: https://reviews.llvm.org/D56625

llvm-svn: 353339
2019-02-06 20:25:17 +00:00
James Y Knight 14359ef1b6 [opaque pointer types] Pass value type to LoadInst creation.
This cleans up all LoadInst creation in LLVM to explicitly pass the
value type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57172

llvm-svn: 352911
2019-02-01 20:44:24 +00:00
Haojian Wu b9613a39b8 Fix a compiler error introduced in r352093.
llvm-svn: 352098
2019-01-24 20:30:48 +00:00
Alina Sbirlea 0a4367209c [LICM] Cleanup duplicated code. [NFCI]
llvm-svn: 352093
2019-01-24 19:57:30 +00:00
Alina Sbirlea 52f6e2a173 [MemorySSA +LICM CFHoist] Solve PR40317.
Summary:
MemorySSA needs updating each time an instruction is moved.
LICM and control flow hoisting re-hoists instructions, thus needing another update when re-moving those instructions.
Pending cleanup: the MSSA update is duplicated, should be moved inside moveInstructionBefore.

Reviewers: jnspaulsson

Subscribers: sanjoy, jlebar, Prazek, george.burgess.iv, llvm-commits

Differential Revision: https://reviews.llvm.org/D57176

llvm-svn: 352092
2019-01-24 19:48:35 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Benjamin Kramer b17d2136ea Give helper classes/functions local linkage. NFC.
llvm-svn: 351016
2019-01-12 18:36:22 +00:00
Alina Sbirlea cae12edaaa Use MemorySSA in LICM to do sinking and hoisting.
Summary:
Step 2 in using MemorySSA in LICM:
Use MemorySSA in LICM to do sinking and hoisting, all under "EnableMSSALoopDependency" flag.
Promotion is disabled.

Enable flag in LICM sink/hoist tests to test correctness of this change. Moved one test which
relied on promotion, in order to test all sinking tests.

Reviewers: sanjoy, davide, gberry, george.burgess.iv

Subscribers: llvm-commits, Prazek

Differential Revision: https://reviews.llvm.org/D40375

llvm-svn: 350879
2019-01-10 19:29:04 +00:00
Max Kazantsev 4615a505f8 [IPT] Drop cache less eagerly in GVN and LoopSafetyInfo
Current strategy of dropping `InstructionPrecedenceTracking` cache is to
invalidate the entire basic block whenever we change its contents. In fact,
`InstructionPrecedenceTracking` has 2 internal strictures: `OrderedInstructions`
that is needed to be invalidated whenever the contents changes, and the map
with first special instructions in block. This second map does not need an
update if we add/remove a non-special instuction because it cannot
affect the contents of this map.

This patch changes API of `InstructionPrecedenceTracking` so that it now
accounts for reasons under which we invalidate blocks. This should lead
to much less recalculations of the map and should save us some compile time
because in practice we don't typically add/remove special instructions.

Differential Revision: https://reviews.llvm.org/D54462
Reviewed By: efriedma

llvm-svn: 350694
2019-01-09 07:28:13 +00:00
John Brawn 39ac159c24 [LICM] Adjust how moving the re-hoist point works
In some cases the order that we hoist instructions in means that when rehoisting
(which uses the same order as hoisting) we can rehoist to a block A, then a
block B, then block A again. This currently causes an assertion failure as it
expects that when changing the hoist point it only ever moves to a block that
dominates the hoist point being moved from.

Fix this by moving the re-hoist point when it doesn't dominate the dominator of
hoisted instruction, or in other words when it wouldn't dominate the uses of
the instruction being rehoisted.

Differential Revision: https://reviews.llvm.org/D55266

llvm-svn: 350408
2019-01-04 17:12:09 +00:00
Alina Sbirlea 0e216854f9 [LICM] *Actually* disable ControlFlowHoisting.
Summary:
The remaining code paths that ControlFlowHoisting introduced that were
not disabled, increased compile time by 3x for some benchmarks.
The time is spent in DominatorTree updates.

Reviewers: john.brawn, mkazantsev

Subscribers: sanjoy, jlebar, llvm-commits

Differential Revision: https://reviews.llvm.org/D55313

llvm-svn: 348345
2018-12-05 10:16:21 +00:00
John Brawn a7eb2c863f [LICM] Reapply r347776 "Make LICM able to hoist phis" with fix
This commit caused a large compile-time slowdown in some cases when NDEBUG is
off due to the dominator tree verification it added. Fix this by only doing
dominator tree and loop info verification when something has been hoisted.

Differential Revision: https://reviews.llvm.org/D52827

llvm-svn: 347889
2018-11-29 17:10:00 +00:00
Martin Storsjo bfd1d27585 Revert "[LICM] Enable control flow hoisting by default" and "[LICM] Reapply r347190 "Make LICM able to hoist phis" with fix"
This reverts commits r347776 and r347778.

The first one, r347776, caused significant compile time regressions
for certain input files, see PR39836 for details.

llvm-svn: 347867
2018-11-29 14:39:39 +00:00
John Brawn 4557ffeb63 [LICM] Enable control flow hoisting by default
Differential Revision: https://reviews.llvm.org/D54949

llvm-svn: 347778
2018-11-28 17:23:03 +00:00
John Brawn 31c9769580 [LICM] Reapply r347190 "Make LICM able to hoist phis" with fix
This commit caused failures because it failed to correctly handle cases where
we hoist a phi, then hoist a use of that phi, then have to rehoist that use. We
need to make sure that we rehoist the use to _after_ the hoisted phi, which we
do by always rehoisting to the immediate dominator instead of just rehoisting
everything to the original preheader.

An option is also added to control whether control flow is hoisted, which is
off in this commit but will be turned on in a subsequent commit.

Differential Revision: https://reviews.llvm.org/D52827

llvm-svn: 347776
2018-11-28 17:21:49 +00:00
Benjamin Kramer 2cad359c91 Revert "[LICM] Make LICM able to hoist phis"
This reverts commit r347190.

llvm-svn: 347225
2018-11-19 16:51:57 +00:00
John Brawn 12c046fba0 [LICM] Make LICM able to hoist phis
The general approach taken is to make note of loop invariant branches, then when
we see something conditional on that branch, such as a phi, we create a copy of
the branch and (empty versions of) its successors and hoist using that.

This has no impact by itself that I've been able to see, as LICM typically
doesn't see such phis as they will have been converted into selects by the time
LICM is run, but once we start doing phi-to-select conversion later it will be
important.

Differential Revision: https://reviews.llvm.org/D52827

llvm-svn: 347190
2018-11-19 11:31:24 +00:00
Max Kazantsev 7d49a3a816 [LICM] Hoist guards from non-header blocks
This patch relaxes overconservative checks on whether or not we could write
memory before we execute an instruction. This allows us to hoist guards out of
loops even if they are not in the header block.

Differential Revision: https://reviews.llvm.org/D50891
Reviewed By: fedor.sergeev

llvm-svn: 346643
2018-11-12 09:29:58 +00:00
Max Kazantsev 9883d1e1a7 [NFC] Add utility function for SafetyInfo updates for moveBefore
llvm-svn: 346472
2018-11-09 05:39:04 +00:00
Max Kazantsev 0042c06e8b [LICM] Remove too conservative IsMustExecute variable
LICM relies on variable `MustExecute` which is conservatively set to `false`
in all non-headers. It is used when we decide whether or not we want to hoist
an instruction or a guard.

For the guards, it might be too conservative to use this variable, we can
instead use a more precise logic from LoopSafetyInfo. Currently it is only NFC
because `IsMemoryNotModified` is also conservatively set to `false` for all
non-headers, and we cannot hoist guards from non-header blocks. However once we
give up using `IsMemoryNotModified` and use a smarter check instead, this will
allow us to hoist guards from all mustexecute non-header blocks.

Differential Revision: https://reviews.llvm.org/D50888
Reveiwed By: fedor.sergeev

llvm-svn: 346204
2018-11-06 04:17:40 +00:00
Max Kazantsev 69f6dfa0f8 [LICM] Use ICFLoopSafetyInfo in LICM
This patch makes LICM use `ICFLoopSafetyInfo` that is a smarter version
of LoopSafetyInfo that leverages power of Implicit Control Flow Tracking
to keep track of throwing instructions and give less pessimistic answers
to queries related to throws.

The ICFLoopSafetyInfo itself has been introduced in rL344601. This patch
enables it in LICM only.

Differential Revision: https://reviews.llvm.org/D50377
Reviewed By: apilipenko

llvm-svn: 346201
2018-11-06 02:44:49 +00:00
Max Kazantsev 872bb74b0a [NFC][LICM] Factor out instruction erasing logic
This patch factors out a function that makes all required updates
whenever an instruction gets erased.

Differential Revision: https://reviews.llvm.org/D54011
Reviewed By: apilipenko

llvm-svn: 345914
2018-11-02 00:21:45 +00:00
Max Kazantsev 9c90ec2fae [NFC] Make LoopSafetyInfo abstract to allow alternative implementations
llvm-svn: 344592
2018-10-16 08:31:05 +00:00
Max Kazantsev 8d56be7070 [NFC] Encapsulate work with BlockColors in LoopSafetyInfo
llvm-svn: 344590
2018-10-16 08:07:14 +00:00
Max Kazantsev c8466f937c [NFC] Turn isGuaranteedToExecute into a method
llvm-svn: 344587
2018-10-16 06:34:53 +00:00
George Burgess IV 6ef8002c2c Replace most users of UnknownSize with LocationSize::unknown(); NFC
Moving away from UnknownSize is part of the effort to migrate us to
LocationSizes (e.g. the cleanup promised in D44748).

This doesn't entirely remove all of the uses of UnknownSize; some uses
require tweaks to assume that UnknownSize isn't just some kind of int.
This patch is intended to just be a trivial replacement for all places
where LocationSize::unknown() will Just Work.

llvm-svn: 344186
2018-10-10 21:28:44 +00:00
Serguei Katkov 5f4a9e9ea0 [LICM] Avoid duplicate work during building AliasSetTracker
Currently we re-use cached info from sub loops or traverse them
to populate AliasSetTracker. But after that we traverse all basic blocks
from the current loop. This is redundant work.

All what we need is traversing the all basic blocks from the loop except
those which are used to get the data from the cache.

This should improve compile time only.

Reviewers: mkazantsev, reames, kariddi, anna
Reviewed By: reames
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D51715

llvm-svn: 341896
2018-09-11 04:07:36 +00:00
Philip Reames 1f52e38e8e [LICM] (re-)simplify code using MemoryLocation API [NFC]
I'd made exactly this same change before, but it appears to have been accidentally reverted in another change.  (I'm assuming accidental since it was without comment or test case, and in an unrelated change.)

llvm-svn: 341892
2018-09-11 03:28:28 +00:00
Max Kazantsev 3c284bde3f Re-enable "[NFC] Unify guards detection"
rL340921 has been reverted by rL340923 due to linkage dependency
from Transform/Utils to Analysis which is not allowed. In this patch
this has been fixed, a new utility function moved to Analysis.

Differential Revision: https://reviews.llvm.org/D51152

llvm-svn: 341014
2018-08-30 03:39:16 +00:00
Philip Reames 1887c40b22 Add a todo and tests to Address a review commnt from D50925 [NFC]
llvm-svn: 340978
2018-08-29 22:09:21 +00:00
Philip Reames f562fc8dbf [LICM] Hoist stores of invariant values to invariant addresses out of loops
Teach LICM to hoist stores out of loops when the store writes to a location otherwise unused in the loop, writes a value which is invariant, and is guaranteed to execute if the loop is entered.

Worth noting is that this transformation is partially overlapping with the existing promotion transformation. Reasons this is worthwhile anyway include:
 * For multi-exit loops, this doesn't require duplication of the store.
 * It kicks in for case where we can't prove we exit through a normal exit (i.e. we may throw), but can prove the store executes before that possible side exit.

Differential Revision: https://reviews.llvm.org/D50925

llvm-svn: 340974
2018-08-29 21:49:30 +00:00
Hans Wennborg 2c390c54f6 Revert r340921 "[NFC] Unify guards detection"
This broke the build, see e.g.

http://lab.llvm.org:8011/builders/clang-cmake-armv8-lnt/builds/4626/
http://lab.llvm.org:8011/builders/clang-ppc64be-linux-lnt/builds/18647/
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-avx2-linux/builds/5856/
http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/22800/

> We have multiple places in code where we try to identify whether or not
> some instruction is a guard. This patch factors out this logic into a separate
> utility function which works uniformly in all places.
>
> Differential Revision: https://reviews.llvm.org/D51152
> Reviewed By: fedor.sergeev

llvm-svn: 340923
2018-08-29 12:21:32 +00:00
Max Kazantsev 1dafaa87d9 [NFC] Unify guards detection
We have multiple places in code where we try to identify whether or not
some instruction is a guard. This patch factors out this logic into a separate
utility function which works uniformly in all places.

Differential Revision: https://reviews.llvm.org/D51152
Reviewed By: fedor.sergeev

llvm-svn: 340921
2018-08-29 11:37:34 +00:00
Philip Reames 1c0fde61a6 [AST] Simplify code minorly using pattern match [NFC]
llvm-svn: 340638
2018-08-24 19:13:39 +00:00
Philip Reames 9ec15faf20 [LICM] Hoist an invariant_start out of loops if there are no stores executed before it
Once the invariant_start is reached, we know that no instruction *after* it can modify the memory. So, if we can prove the location isn't read *between entry into the loop and the execution of the invariant_start*, we can execute the invariant_start before entering the loop.

Differential Revision: https://reviews.llvm.org/D51181

llvm-svn: 340617
2018-08-24 16:24:48 +00:00
Alina Sbirlea ab6f84f763 Update MemorySSA in BasicBlockUtils.
Summary:
Extend BasicBlocksUtils to update MemorySSA.

Subscribers: sanjoy, arsenm, nhaehnle, jlebar, Prazek, llvm-commits

Differential Revision: https://reviews.llvm.org/D45300

llvm-svn: 340365
2018-08-21 23:32:03 +00:00
Marcello Maggioni 883fe455f1 [LICM] Refactor some AliasSetTracker code to get rid of new/deletes. NFC
Differential Revision: https://reviews.llvm.org/D51024

llvm-svn: 340333
2018-08-21 20:30:14 +00:00
Philip Reames c3c23e8cf2 [AST] Remove notion of volatile from alias sets [NFCI]
Volatility is not an aliasing property. We used to model volatile as if it had extremely conservative aliasing implications, but that hasn't been true for several years now. So, it doesn't make sense to be in AliasSet.

It also turns out the code is entirely a noop. Outside of the AST code to update it, there was only one user: load store promotion in LICM. L/S promotion doesn't need the check since it walks all the users of the address anyway. It already checks each load or store via !isUnordered which causes us to bail for volatile accesses. (Look at the lines immediately following the two remove asserts.)

There is the possibility of some small compile time impact here, but the only case which will get noticeably slower is a loop with a large number of loads and stores to the same address where only the last one we inspect is volatile. This is sufficiently rare it's not worth optimizing for..

llvm-svn: 340312
2018-08-21 17:59:11 +00:00
Max Kazantsev 097ef69182 [LICM] Hoist guards with invariant conditions
This patch teaches LICM to hoist guards from the loop if they are guaranteed to execute and
if there are no side effects that could prevent that.

Differential Revision: https://reviews.llvm.org/D50501
Reviewed By: reames

llvm-svn: 340256
2018-08-21 08:11:31 +00:00
Anna Thomas 1962621a7e [LICM] Add a diagnostic analysis for identifying alias information
Summary:
Currently, in LICM, we use the alias set tracker to identify if the
instruction (we're interested in hoisting) aliases with instruction that
modifies that memory location.

This patch adds an LICM alias analysis diagnostic tool that checks the
mod ref info of the instruction we are interested in hoisting/sinking,
with every instruction in the loop.  Because of O(N^2) complexity this
is now only a diagnostic tool to show the limitation we have with the
alias set tracker and is OFF by default.

Test cases show the difference with the diagnostic analysis tool, where
we're able to hoist out loads and readonly + argmemonly calls from the
loop, where the alias set tracker analysis is not able to hoist these
instructions out.

Reviewers: reames, mkazantsev, fedor.sergeev, hfinkel

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D50854

llvm-svn: 340026
2018-08-17 13:44:00 +00:00
Philip Reames 0e2f9b9e30 [LICM][NFC] Restructure pointer invalidation API in terms of MemoryLocation
Main value is just simplifying code.  I'll further simply the argument handling case in a bit, but that involved a slightly orthogonal change so I went with the mildy ugly intermediate for this patch.

Note that the isSized check in the old LICM code was not carried across.  It turns out that check was dead.  a) no test exercised it, and b) langref and verifier had been updated to disallow unsized types used in loads.

llvm-svn: 339930
2018-08-16 20:11:15 +00:00
Max Kazantsev 72d7d649e3 [NFC] Remove const modifier to allow further development in LICM
llvm-svn: 339846
2018-08-16 08:30:15 +00:00
Max Kazantsev 530b8d1c3d [NFC] Refactoring of LoopSafetyInfo, step 1
Turn structure into class, encapsulate methods, add clarifying comments.

Differential Revision: https://reviews.llvm.org/D50693
Reviewed By: reames

llvm-svn: 339752
2018-08-15 05:55:43 +00:00
Max Kazantsev 68290f838a [NFC][LICM] Make hoist method void
Method hoist always returns true. This patch makes it void.

Differential Revision: https://reviews.llvm.org/D50696
Reviewed By: hiraditya

llvm-svn: 339750
2018-08-15 02:49:12 +00:00
Philip Reames 85afd1a9a0 [LICM] Hoist assumes out of loops
If we have an assume which is known to execute and whose operand is invariant, we can lift that into the pre-header. So long as we don't change which paths the assume executes on, this is a legal transformation. It's likely to be a useful canonicalization as other transforms only look for dominating assumes.

Differential Revision: https://reviews.llvm.org/D50364

llvm-svn: 339481
2018-08-10 22:21:56 +00:00
Philip Reames 7d79433136 [LICM] Suppress a compiler warning noticed by one of the bots
llvm-svn: 339388
2018-08-09 21:15:33 +00:00
Philip Reames ca256d93fb [LICM] hoist fences out of loops w/o memory operations
The motivating case is an otherwise dead loop with a fence in it. At the moment, this goes all the way through the optimizer and we end up emitting an entirely pointless loop on x86. This case may seem a bit contrived, but we've seen it in real code as the result of otherwise reasonable lowering strategies combined w/thread local memory optimizations (such as escape analysis).

To handle this simple case, we can teach LICM to hoist must execute fences when there is no other memory operation within the loop.

Differential Revision: https://reviews.llvm.org/D50489

llvm-svn: 339378
2018-08-09 20:18:42 +00:00
Philip Reames 22b20a09a0 [LICM] Add an assert to ensure all instruction types needing aliasing are handled [NFC]
llvm-svn: 339308
2018-08-09 03:44:28 +00:00
Philip Reames 3b35aaacb6 [LICM] Extract a helper function for readability [NFC]
llvm-svn: 339069
2018-08-06 22:07:37 +00:00
Philip Reames 5937368d4f [LICM] Remove unneccessary safety check to increase sinking effectiveness
This one requires a bit of explaination.  It's not every day you simply delete code to implement an optimization.  :)

The transform in question is sinking an instruction from a loop to the uses in loop exiting blocks.  We know (from LCSSA) that all of the uses outside the loop must be phi nodes, and after predecessor splitting, we know all phi users must have a single operand.  Since the use must be strictly dominated by the def, we know from the definition of dominance/ssa that the exit block must execute along a (non-strict) subset of paths which reach the def.  As a result, duplicating a potentially faulting instruction can not *introduce* a fault that didn't previously exist in the program.  

The full story is that this patch builds on "rL338671: [LICM] Factor out fault legality from canHoistOrSinkInst [NFC]" which pulled this logic out of a common helper routine.  As best I can tell, this check was originally added to the helper function for hoisting legality, later an incorrect fastpath for loads/calls was added, and then the bug was fixed by duplicating the fault safety check in the hoist path.  This left the redundant check in the common code to pessimize sinking for no reason.  I split it out in an NFC, and am not removing the unneccessary check.  I wanted there to be something easy to revert in case I missed something.

Reviewed by: Anna Thomas (in person)

llvm-svn: 338794
2018-08-03 00:21:56 +00:00
Philip Reames 32cb80b9d3 [LICM] Factor out fault legality from canHoistOrSinkInst [NFC]
This method has three callers, each of which wanted distinct handling:
1) Sinking into a loop is moving an instruction known to execute before a loop into the loop.  We don't need to worry about introducing a fault at all in this case.
2) Hoisting from a loop into a preheader already duplicated the check in the caller.
3) Sinking from the loop into an exit block was the only true user of the code within the routine.  For the moment, this has just been lifted into the caller, but up next is examining the logic more carefully.  Whitelisting of loads and calls - while consistent with the previous code - is rather suspicious.  Either way, a behavior change is worthy of it's own patch.  

llvm-svn: 338671
2018-08-02 04:08:04 +00:00
Philip Reames 09de470e9e [LICM] hoisting/sinking legality - bail early for unsupported instructions
Originally, this was part of a larger refactoring I'd planned, but had to abandoned.  I figured the minor improvement in readability was worthwhile.

llvm-svn: 338663
2018-08-02 00:54:14 +00:00
Fangrui Song f78650a8de Remove trailing space
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338293
2018-07-30 19:41:25 +00:00
Alina Sbirlea 0e15501fa7 Replace "Replacable" with "Replaceable". [NFC]
llvm-svn: 336133
2018-07-02 18:53:40 +00:00
Stanislav Mekhanoshin d8c9374797 Fix invariant fdiv hoisting in LICM
FDiv is replaced with multiplication by reciprocal and invariant
reciprocal is hoisted out of the loop, while multiplication remains
even if invariant.

Switch checks for all invariant operands and only invariant
denominator to fix the issue.

Differential Revision: https://reviews.llvm.org/D48447

llvm-svn: 335411
2018-06-23 04:01:28 +00:00
David Blaikie 31b98d2e99 Move Analysis/Utils/Local.h back to Transforms
Review feedback from r328165. Split out just the one function from the
file that's used by Analysis. (As chandlerc pointed out, the original
change only moved the header and not the implementation anyway - which
was fine for the one function that was used (since it's a
template/inlined in the header) but not in general)

llvm-svn: 333954
2018-06-04 21:23:21 +00:00
Jun Bum Lim dfbe6fa832 [LICM] Preserve DT and LoopInfo specifically
Summary:
In LICM, CFG could be changed in splitPredecessorsOfLoopExit(), which update
only DT and LoopInfo. Therefore, we should preserve only DT and LoopInfo specifically,
instead of all analyses that depend on the CFG (setPreservesCFG()).

This change should fix PR37323.

Reviewers: uabelho, davide, dberlin, Ka-Ka

Reviewed By: dberlin

Subscribers: mzolotukhin, bjope, mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D46775

llvm-svn: 333198
2018-05-24 15:58:34 +00:00
Nicola Zaghen d34e60ca85 Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624

llvm-svn: 332240
2018-05-14 12:53:11 +00:00
Philip Reames 5b39acd111 [LICM] Compute a must execute property for the prefix of the header as we go
Computing this property within the existing walk ensures that the cost is linear with the size of the block. If we did this from within isGuaranteedToExecute, it would be quadratic without some very fancy caching.

This allows us to reliably catch a hoistable instruction within a header which may throw at some point *after* our hoistable instruction. It doesn't do anything for non-header cases, but given how common single block loops are, this seems very worthwhile.

llvm-svn: 331557
2018-05-04 21:35:00 +00:00
Philip Reames 5a6482450a [LICM] Reduce nesting with an early return [NFC]
llvm-svn: 331080
2018-04-27 20:58:30 +00:00
Fangrui Song 956ee79795 Fix a bunch of typoes. NFC
llvm-svn: 328907
2018-03-30 22:22:31 +00:00
Andrew Kaylor a237866faf Fix a block copying problem in LICM
Differential Revision: https://reviews.llvm.org/D44817

llvm-svn: 328336
2018-03-23 17:36:18 +00:00
David Blaikie 2be3922807 Fix a couple of layering violations in Transforms
Remove #include of Transforms/Scalar.h from Transform/Utils to fix layering.

Transforms depends on Transforms/Utils, not the other way around. So
remove the header and the "createStripGCRelocatesPass" function
declaration (& definition) that is unused and motivated this dependency.

Move Transforms/Utils/Local.h into Analysis because it's used by
Analysis/MemoryBuiltins.cpp.

llvm-svn: 328165
2018-03-21 22:34:23 +00:00
Anastasis Grammenos 3a589103a4 [LICM] Salvage DI from dying Instructions
LICM deletes trivially dead instructions which it won't attempt to sink.
Attempt to salvage debug values which reference these instructions.

llvm-svn: 327800
2018-03-18 15:59:19 +00:00
Anna Thomas 5ac72f94f3 Test Commit NFC. Updated comment
llvm-svn: 327436
2018-03-13 19:38:45 +00:00
David Green 0d5f9651f2 Move llvm::computeLoopSafetyInfo from LICM.cpp to LoopUtils.cpp. NFC
Move computeLoopSafetyInfo, defined in Transforms/Utils/LoopUtils.h,
into the corresponding LoopUtils.cpp, as opposed to LICM where it resides
at the moment. This will allow other functions from Transforms/Utils
to reference it.

llvm-svn: 325151
2018-02-14 18:34:53 +00:00
Jun Bum Lim 144eb593dd [LICM] update BlockColors after splitting predecessors
Update BlockColors after splitting predecessors. Do not allow splitting
EHPad for sinking when the BlockColors is not empty, so we can
simply assign predecessor's color to the new block.

Fixes PR36184

llvm-svn: 324916
2018-02-12 17:56:55 +00:00
Jun Bum Lim 44c58d35c1 Re-commit : [LICM] Allow sinking when foldable in loop
This recommits r320823 reverted due to the test failure in sink-foldable.ll and
an unused variable. Added "REQUIRES: aarch64-registered-target" in the test
and removed unused variable.

Original commit message:

  Continue trying to sink an instruction if its users in the loop is foldable.
  This will allow the instruction to be folded in the loop by decoupling it from
  the user outside of the loop.

  Reviewers: hfinkel, majnemer, davidxl, efriedma, danielcdh, bmakam, mcrosier

  Reviewed By: hfinkel

  Subscribers: javed.absar, bmakam, mcrosier, llvm-commits

  Differential Revision: https://reviews.llvm.org/D37076

llvm-svn: 320858
2017-12-15 20:33:24 +00:00
Jun Bum Lim 5efd4d8b5e Revert "Re-commit : [LICM] Allow sinking when foldable in loop"
This reverts commit r320833.

llvm-svn: 320836
2017-12-15 18:12:49 +00:00
Jun Bum Lim 83ccad6684 Re-commit : [LICM] Allow sinking when foldable in loop
This recommit r320823 after fixing a test failure.

 Original commit message:

    Continue trying to sink an instruction if its users in the loop is foldable.
    This will allow the instruction to be folded in the loop by decoupling it from
    the user outside of the loop.

    Reviewers: hfinkel, majnemer, davidxl, efriedma, danielcdh, bmakam, mcrosier

    Reviewed By: hfinkel

    Subscribers: javed.absar, bmakam, mcrosier, llvm-commits

    Differential Revision: https://reviews.llvm.org/D37076

llvm-svn: 320833
2017-12-15 17:58:59 +00:00
Jun Bum Lim 6136d87f5d Revert "[LICM] Allow sinking when foldable in loop"
This reverts commit r320823.

llvm-svn: 320828
2017-12-15 16:35:09 +00:00
Jun Bum Lim 22855c26a5 [LICM] Allow sinking when foldable in loop
Summary:
Continue trying to sink an instruction if its users in the loop is foldable.
This will allow the instruction to be folded in the loop by decoupling it from
the user outside of the loop.

Reviewers: hfinkel, majnemer, davidxl, efriedma, danielcdh, bmakam, mcrosier

Reviewed By: hfinkel

Subscribers: javed.absar, bmakam, mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D37076

llvm-svn: 320823
2017-12-15 16:09:54 +00:00
Alina Sbirlea ff8b8aea2e Add MemorySSA as loop dependency, disabled by default [NFC].
Summary:
First step in adding MemorySSA as dependency for loop pass manager.
Adding the dependency under a flag.

New pass manager: MSSA pointer in LoopStandardAnalysisResults can be null.
Legacy and new pass manager: Use cl::opt EnableMSSALoopDependency. Disabled by default.

Reviewers: sanjoy, davide, gberry

Subscribers: mehdi_amini, Prazek, llvm-commits

Differential Revision: https://reviews.llvm.org/D40274

llvm-svn: 318772
2017-11-21 15:45:46 +00:00
Jun Bum Lim 0f90672ae9 [LICM] Fix PR35342
Summary: This change fix PR35342 by replacing only the current use with undef in unreachable blocks.

Reviewers: efriedma, mcrosier, igor-laevsky

Reviewed By: efriedma

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D40184

llvm-svn: 318551
2017-11-17 20:38:25 +00:00
Jun Bum Lim f5fb3d745d [LICM] sink through non-trivially replicable PHI
Summary:
The current LICM allows sinking an instruction only when it is exposed to exit
blocks through a trivially replacable PHI of which all incoming values are the
same instruction. This change enhance LICM to sink a sinkable instruction
through non-trivially replacable PHIs by spliting predecessors of loop
exits.

Reviewers: hfinkel, majnemer, davidxl, bmakam, mcrosier, danielcdh, efriedma, jtony

Reviewed By: efriedma

Subscribers: nemanjai, dberlin, llvm-commits

Differential Revision: https://reviews.llvm.org/D37163

llvm-svn: 317335
2017-11-03 16:24:53 +00:00
Philip Reames 21cc2fa3f6 [LICM] Restructure implicit exit handling to be more clear [NFCI]
When going to explain this to someone else, I got tripped up by the complicated meaning of IsKnownNonEscapingObject in load-store promotion.  Extract a helper routine and clarify naming/scopes to make this a bit more obvious.

llvm-svn: 316699
2017-10-26 21:00:15 +00:00
Vivek Pandya 9590658fb8 [NFC] Convert OptimizationRemarkEmitter old emit() calls to new closure
parameterized emit() calls

Summary: This is not functional change to adopt new emit() API added in r313691.

Reviewed By: anemet

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38285

llvm-svn: 315476
2017-10-11 17:12:59 +00:00
Max Kazantsev 0c8dd052b8 [LICM] Disallow sinking of unordered atomic loads into loops
Sinking of unordered atomic load into loop must be disallowed because it turns
a single load into multiple loads. The relevant section of the documentation
is: http://llvm.org/docs/Atomics.html#unordered, specifically the Notes for
Optimizers section. Here is the full text of this section:

> Notes for optimizers
> In terms of the optimizer, this **prohibits any transformation that
> transforms a single load into multiple loads**, transforms a store into
> multiple stores, narrows a store, or stores a value which would not be
> stored otherwise. Some examples of unsafe optimizations are narrowing
> an assignment into a bitfield, rematerializing a load, and turning loads
> and stores into a memcpy call. Reordering unordered operations is safe,
> though, and optimizers should take advantage of that because unordered
> operations are common in languages that need them.

Patch by Daniil Suchkov!

Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D38392

llvm-svn: 315438
2017-10-11 07:26:45 +00:00
Adam Nemet 0965da2055 Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*
Sync it up with the name of the class actually defined here.  This has been
bothering me for a while...

llvm-svn: 315249
2017-10-09 23:19:02 +00:00
Alina Sbirlea 7ed5856a32 Refactor collectChildrenInLoop to LoopUtils [NFC]
Summary: Move to LoopUtils method that collects all children of a node inside a loop.

Reviewers: majnemer, sanjoy

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37870

llvm-svn: 313322
2017-09-15 00:04:16 +00:00
Alina Sbirlea 80b806bf30 Make promoteLoopAccessesToScalars independent of AliasSet [NFC]
Summary:
The current promoteLoopAccessesToScalars method receives an AliasSet, but
the information used is in fact a list of Value*, known to must alias.
Create the list ahead of time to make this method independent of the AliasSet class.

While there is no functionality change, this adds overhead for creating
a set of Value*, when promotion would normally exit earlier.
This is meant to be as a first refactoring step in order to start replacing
AliasSetTracker with MemorySSA.
And while the end goal is to redesign LICM, the first few steps will focus on
adding MemorySSA as an alternative to the AliasSetTracker using most of the
existing functionality.

Reviewers: mkuper, danielcdh, dberlin

Subscribers: sanjoy, chandlerc, gberry, davide, llvm-commits

Differential Revision: https://reviews.llvm.org/D35439

llvm-svn: 313075
2017-09-12 21:18:44 +00:00
David Majnemer e6bb895ab5 [LICM] Make sinkRegion and hoistRegion non-recursive
Large CFGs can cause us to blow up the stack because we would have a
recursive step for each basic block in a region.

Instead, create a worklist and iterate it. This limits the stack usage
to something more manageable.

Differential Revision: https://reviews.llvm.org/D35609

llvm-svn: 308582
2017-07-20 03:27:02 +00:00
Davide Italiano 79eb3b0366 [IR] Prefer use_empty() to !hasNUsesOrMore(1) for clarity.
llvm-svn: 303218
2017-05-16 22:38:40 +00:00
Evgeniy Stepanov 58ccc0949a Revert "Compute safety information in a much finer granularity."
Use-after-free in llvm::isGuaranteedToExecute.

llvm-svn: 301214
2017-04-24 18:25:07 +00:00
Xin Tong a266923d57 Compute safety information in a much finer granularity.
Summary:
Instead of keeping a variable indicating whether there are early exits
in the loop.  We keep all the early exits. This improves LICM's ability to
move instructions out of the loop based on is-guaranteed-to-execute.

I am going to update compilation time as well soon.

Reviewers: hfinkel, sanjoy, efriedma, mkuper

Reviewed By: hfinkel

Subscribers: llvm-commits, mzolotukhin

Differential Revision: https://reviews.llvm.org/D32433

llvm-svn: 301196
2017-04-24 17:12:22 +00:00
Hal Finkel b63ed91549 [LICM] Hoist fp division from the loops and replace by a reciprocal
When allowed, we can hoist a division out of a loop in favor of a
multiplication by the reciprocal. Fixes PR32157.

Patch by vit9696!

Differential Revision: https://reviews.llvm.org/D30819

llvm-svn: 299911
2017-04-11 02:22:54 +00:00
Xin Tong ccee0e0c05 Make default value for disable-licm-promotion in licm explicit.
llvm-svn: 295767
2017-02-21 20:53:48 +00:00
Brian Cain 6dedf65cc9 Correct a typo, s/hosting/hoisting/
llvm-svn: 295066
2017-02-14 16:41:10 +00:00