Commit Graph

6555 Commits

Author SHA1 Message Date
OCHyams e3cd498ff7 [Assignment Tracking][21/*] Account for assignment tracking in inliner
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

The inliner requires two additions:

fixupAssignments - Update inlined instructions' DIAssignID metadata so that
inlined DIAssignID attachments are unique to the inlined instance.

trackInlinedStores - Treat inlined stores to caller-local variables
(i.e. callee stores to argument pointers that point to the caller's allocas) as
assignments. Track them using trackAssignments, which is the same method as is
used by the AssignmentTrackingPass. This means that we're able to detect stale
memory locations due to DSE after inlining. Because the stores are only tracked
_after_ inlining, any DSE or movement of stores _before_ inlining will not be
accounted for. This is an accepted limitation mentioned in the RFC.

One change is also required:

Update CloneBlock to preserve debug use-before-defs. Otherwise the assignments
will be dropped due to having the intrinsic operands replaced with empty
metadata (see use-before-def.ll in this patch and this related discourse post.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133318
2022-11-18 11:55:05 +00:00
OCHyams 86464ed3df [Assignment Tracking][15/*] Account for assignment tracking in simplifycfg
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Update simplifycfg:
sinkLastInstruction - preserve debug use-before-defs.

SpeculativelyExecuteBB - replace the value component of dbg.assign intrinsics
when stores are hoisted and merged using a select, and don't delete them.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133310
2022-11-18 10:15:55 +00:00
OCHyams 4898568caa [Assignment Tracking][11/*] Update RemoveRedundantDbgInstrs
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Update the RemoveRedundantDbgInstrs utility to avoid sometimes losing
information when deleting dbg.assign intrinsics.

removeRedundantDbgInstrsUsingBackwardScan - treat dbg.assign intrinsics that
are not linked to any instruction just like dbg.values. That is, in a block of
contiguous debug intrinsics, delete all other than the last definition for a
fragment. Leave linked dbg.assign intrinsics in place.

removeRedundantDbgInstrsUsingForwardScan - Don't delete linked dbg.assign
intrinsics and don't delete the next intrinsic found even if it would otherwise
be eligible for deletion.

remomveUndefDbgAssignsFromEntryBlock - Delete undef and unlinked dbg.assign
intrinsics encountered in the entry block that come before non-undef
non-unlinked intrinsics for the same variable.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133294
2022-11-16 12:27:18 +00:00
OCHyams 0946e463e8 [Assignment Tracking][12/*] Account for assignment tracking in mem2reg
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

The changes for assignment tracking in mem2reg don't require much of a
deviation from existing behaviour. dbg.assign intrinsics linked to an alloca
are treated much in the same way as dbg.declare users of an alloca, except that
we don't insert dbg.value intrinsics to describe assignments when there is
already a dbg.assign intrinsic present, e.g. one linked to a store that is
going to be removed.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133295
2022-11-15 11:11:57 +00:00
OCHyams 23bb4735ca [Assignment Tracking][10/*] salvageDebugInfo for dbg.assign intrinsics
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Plumb in salvaging for the address part of dbg.assign intrinsics.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133293
2022-11-09 11:49:46 +00:00
Nikita Popov ce2f9ba2c9 [SCCP] Add helper for getting constant range (NFC)
Add a helper for the recurring pattern of getting a constant range
if the value lattice element is one, or a full range otherwise.
2022-11-09 12:42:36 +01:00
OCHyams a9025f57ba [Assignment Tracking][8/*] Add DIAssignID merging utilities
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Add method:

  Instruction::mergeDIAssignID(
      ArrayRef<const Instruction* > SourceInstructions)

which merges the DIAssignID metadata attachments on `SourceInstructions` and
`this` and replaces uses of the original IDs with the new shared one.

This is used when stores are merged, for example sinking stores out of a
if-diamond CFG or vectorizing contiguous stores.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133291
2022-11-09 10:46:04 +00:00
OCHyams fd16ff3a7e Reapply: [NFC] Move getDebugValueLoc from static in Local.cpp to DebugInfo.h
Reverted in b22d80dc6a.

Move getDebugValueLoc so that it can be accessed from DebugInfo.h for the
Assignment Tracking patch stack and remove redundant parameter Src.

Reviewed By: jryans

Differential Revision: https://reviews.llvm.org/D132357
2022-11-08 16:25:39 +00:00
Shubham Sandeep Rastogi b22d80dc6a Revert "[NFC] Move getDebugValueLoc from static in Local.cpp to DebugInfo.h"
This reverts commit 80378a4ca7.

    I am reverting this patch because I need to revert 171f7024cc and without reverting this patch, reverting 171f7024cc causes conflicts.

     Patch 171f7024cc introduced a cyclic dependancy in the module build.

        https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48197/consoleFull#-69937453049ba4694-19c4-4d7e-bec5-911270d8a58c

        In file included from <module-includes>:1:
        /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/IR/Argument.h:18:10: fatal error: cyclic dependency in module 'LLVM_IR': LLVM_IR -> LLVM_intrinsic_gen -> LLVM_IR
                 ^
        While building module 'LLVM_MC' imported from /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/lib/MC/MCAsmInfoCOFF.cpp:14:
        While building module 'LLVM_IR' imported from /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/MC/MCPseudoProbe.h:57:
        In file included from <module-includes>:12:
        /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/IR/DebugInfo.h:24:10: fatal error: could not build module 'LLVM_intrinsic_gen'
         ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
        While building module 'LLVM_MC' imported from /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/lib/MC/MCAsmInfoCOFF.cpp:14:
        In file included from <module-includes>:15:
        In file included from /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/MC/MCContext.h:23:
        /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/include/llvm/MC/MCPseudoProbe.h:57:10: fatal error: could not build module 'LLVM_IR'
         ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
        /Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/llvm/lib/MC/MCAsmInfoCOFF.cpp:14:10: fatal error: could not build module 'LLVM_MC'
         ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
        4 errors generated.
2022-11-07 15:19:04 -08:00
Miguel Saldivar de36d39e24 [InstCombine] Avoid passing pow attributes to sqrt
As described in issue #58475, we could pass the attributes of pow to sqrt and crash.

Differential Revision: https://reviews.llvm.org/D137454
2022-11-07 12:07:37 -05:00
OCHyams 80378a4ca7 [NFC] Move getDebugValueLoc from static in Local.cpp to DebugInfo.h
Move getDebugValueLoc so that it can be accessed from DebugInfo.h for the
Assignment Tracking patch stack and remove redundant parameter Src.

Reviewed By: jryans

Differential Revision: https://reviews.llvm.org/D132357
2022-11-07 15:14:43 +00:00
Sanjay Patel bff6880a5f [SimplifyLibCalls] improve code readability for AttributeList propagation; NFC
It is possible that we can do better on some of these transforms
by passing some subset of attributes, but we were not doing that
in any of the changed code. So it's better to give that a name
to indicate we're clearing attributes or make that more obvious
by using the default-constructed empty list.
2022-11-06 09:07:17 -05:00
Nikita Popov 304f1d59ca [IR] Switch everything to use memory attribute
This switches everything to use the memory attribute proposed in
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
The old argmemonly, inaccessiblememonly and inaccessiblemem_or_argmemonly
attributes are dropped. The readnone, readonly and writeonly attributes
are restricted to parameters only.

The old attributes are auto-upgraded both in bitcode and IR.
The bitcode upgrade is a policy requirement that has to be retained
indefinitely. The IR upgrade is mainly there so it's not necessary
to update all tests using memory attributes in this patch, which
is already large enough. We could drop that part after migrating
tests, or retain it longer term, to make it easier to import IR
from older LLVM versions.

High-level Function/CallBase APIs like doesNotAccessMemory() or
setDoesNotAccessMemory() are mapped transparently to the memory
attribute. Code that directly manipulates attributes (e.g. via
AttributeList) on the other hand needs to switch to working with
the memory attribute instead.

Differential Revision: https://reviews.llvm.org/D135780
2022-11-04 10:21:38 +01:00
Nikita Popov 01ec0ff2dc [SimplifyCFG] Allow speculating block containing assume()
SpeculativelyExecuteBB(), which converts a branch + phi structure
into a select, currently bails out if the block contains an assume
(because it is not speculatable).

Adjust the fold to ignore ephemeral values (i.e. assumes and values
only used in assumes) for cost modelling purposes, and drop them
when performing the fold.

Theoretically, we could try to preserve the assume information by
generating a assume(br_cond || assume_cond) style assume, but this
is very unlikely to to be useful (because we don't do anything
useful with assumes of this form) and it would make things
substantially more complicated once we take operand bundle assumes
into account (which don't really support a || operation).
I'd prefer not to do that without good motivation.

Differential Revision: https://reviews.llvm.org/D137339
2022-11-04 09:26:35 +01:00
Nikita Popov b03f7c3365 [SimplifyCFG] Use range based for loop (NFC) 2022-11-03 15:21:45 +01:00
Nikita Popov 592a96c03b [SimplifyCFG] Extract code for tracking ephemeral values (NFC)
To allow reusing this in more places in SimplifyCFG.
2022-11-03 14:28:12 +01:00
OCHyams 88d34d4626 [DebugInfo] Fix minor debug info bug in deleteDeadLoop
Using a DebugVariable as the set key rather than std::pair<DIVariable *,
DIExpression *> ensures we don't accidently confuse multiple instances of
inlined variables.

Reviewed By: jryans

Differential Revision: https://reviews.llvm.org/D133303
2022-11-02 13:28:05 +00:00
Nikita Popov 134bda4b61 [ValueLattice] Use DL-aware folding in getCompare()
Use DL-aware ConstantFoldCompareInstOperands() API instead of
ConstantExpr API. The practical effect of this is that SCCP can
now fold comparisons that require DL.
2022-11-02 10:41:11 +01:00
Nikita Popov 7ad3bb527e Reapply [ValueLattice] Fix getCompare() for undef values
Relative to the previous attempt, this also updates the
ValueLattice unit tests.

-----

Resolve the TODO about incorrect getCompare() behavior. This can
be made more precise (e.g. by materializing the undef value and
performing constant folding on it), but for now just return an
unknown result to fix the correctness issue.

This should be NFC in terms of user-visible behavior, because the
only user of this method (SCCP) was already guarding against
UndefValue results.
2022-11-02 10:23:06 +01:00
Florian Hahn e83e289fd1
[SCEVExpander] Forget SCEV when replacing congruent phi.
Otherwise there may be stale values left over in the cache, causing a
SCEV verification failure.

Fixes #58702.
2022-11-01 17:49:38 +00:00
Nikita Popov 094d1298d1 Revert "[ValueLattice] Fix getCompare() for undef values"
This reverts commit 6de41e6d76.

Missed a unit test affected by this change, revert for now.
2022-11-01 17:03:40 +01:00
Nikita Popov 6de41e6d76 [ValueLattice] Fix getCompare() for undef values
Resolve the TODO about incorrect getCompare() behavior. This can
be made more precise (e.g. by materializing the undef value and
performing constant folding on it), but for now just return an
unknown result.
2022-11-01 16:41:24 +01:00
Brendon Cahoon f59205aef9 [BasicBlockUtils] Add a new way for CreateControlFlowHub()
The existing way of creating the predicate in the guard blocks uses
a boolean value per outgoing block. This increases the number of live
booleans as the number of outgoing blocks increases. The new way added
in this change is to store one integer to represent the outgoing block
we want to branch to, then at each guard block, an integer equality
check is performed to decide which a specific outgoing block is taken.

Using an integer reduces the number of live values and decreases
register pressure especially in cases where there are a large number
of outgoing blocks. The integer based approach is used when the
number of outgoing blocks crosses a threshold, which is currently set
to 32.

Patch by Ruiling Song.

Differential review: https://reviews.llvm.org/D127831
2022-10-31 08:58:54 -05:00
Brendon Cahoon 9265b7fa83 NFC: restructure code for CreateControlFlowHub()
Differential review: https://reviews.llvm.org/D127830
2022-10-31 08:39:09 -05:00
Momchil Velikov c9b4dc3a81 [FuncSpec][NFC] Avoid redundant computations of DominatorTree/LoopInfo
The `FunctionSpecialization` pass needs loop analysis results for its
cost function. For this purpose, it computes the `DominatorTree` and
`LoopInfo` for a function in `getSpecializationBonus`.  This function,
however, is called O(number of call sites x number of arguments), but
the DominatorTree/LoopInfo can be computed just once.

This patch plugs into the PassManager infrastructure to obtain
LoopInfo for a function and removes ad-hoc computation from
`getSpecializatioBonus`.

Reviewed By: ChuanqiXu, labrinea

Differential Revision: https://reviews.llvm.org/D136332
2022-10-28 16:08:41 +01:00
Guillaume Chatelet 1a726cfa83 Take memset_inline into account in analyzeLoadFromClobberingMemInst
This appeared in https://reviews.llvm.org/D126903#3884061

Differential Revision: https://reviews.llvm.org/D136752
2022-10-26 09:50:13 +00:00
Alina Sbirlea d1b19da854 [LoopPeeling] Add flag to disable support for peeling loops with non-latch exits
Add a flag to allow disabling the changes in
https://reviews.llvm.org/D134803.

Differential Revision: https://reviews.llvm.org/D136643
2022-10-25 12:19:14 -07:00
Simon Pilgrim 50fe87a5c8 [Transforms] classifyArgUse - don't deference pointer before null test
Reported here: https://pvs-studio.com/en/blog/posts/cpp/1003/ (N11)
2022-10-25 17:24:00 +01:00
Yaxun (Sam) Liu 9d5adc7e49 Revert "reland e5581df60a [SimplifyCFG] accumulate bonus insts cost"
This reverts commit bd7949bcd8.

Revert this patch since reviwers have different opinions regarding
the approach in post-commit review.

Will open RFC for further discussion.

Differential Revision: https://reviews.llvm.org/D132408
2022-10-25 12:15:39 -04:00
Yaxun (Sam) Liu bd7949bcd8 reland e5581df60a [SimplifyCFG] accumulate bonus insts cost
Fixed compile time increase due to always constructing LocalCostTracker.
Now only construct LocalCostTracker when needed.
2022-10-24 15:43:53 -04:00
Kevin P. Neal cfb88ee3ba [StrictFP][IPSCCP] Constant fold intrinsics with metadata arguments
This teaches the SCCP Solver how to constant fold more intrinsics. Constant
folding appears to be just as good as D115737 but much, much lower in code
change impact as suggested by nikic.

The constrained floating-point intrinsics all take at least one metadata
argument and were the motivation for the change.

Differential Revision: https://reviews.llvm.org/D136466
2022-10-24 11:43:20 -04:00
Matt Arsenault 597b9b7e8e CodeExtractor: Fix assertion with non-0 alloca address spaces
emitCallAndSwitchStatement creates placeholder allocas to pass
to these, so the types need to match.
2022-10-23 15:16:55 -07:00
Florian Hahn fd236772f5
[IndVars] Forget SCEV for value after simplifying condition.
Additional SCEV verification highlighted a case where the cached loop
dispositions where incorrect after simplifying a condition in IndVars
and moving the user in LoopDeletion. Fix it by invalidating ICmp and all
its users.

Fixes #58515.
2022-10-21 11:18:01 +01:00
Nikita Popov e9754f0211 [IR] Add support for memory attribute
This implements IR and bitcode support for the memory attribute,
as specified in https://reviews.llvm.org/D135597.

The new attribute is not used for anything yet (and as such, the
old memory attributes are unaffected).

Differential Revision: https://reviews.llvm.org/D135592
2022-10-21 12:11:25 +02:00
Michael Francis 922f42d531 [clang][AIX] Fix mcount name and call arguments
Currently, compiling a program with the `-pg` flag will result in an
undefined symbol error for `.mcount`. This revision fixes the call to
use `__mcount`, which requires a pointer argument to a pointer-sized
object (unique per inserted call) on AIX.

This is only a partial fix. This patch should fix the `-pg` flag's
behaviour on AIX to work with code you are compiling, but it will not
link against standard libraries with `mcount` instrumentation calls. The
next step is to add profiled libraries to the linker search paths in the
Clang driver for the AIX toolchain when linking with `-pg`.

Differential Review: https://reviews.llvm.org/D135384
2022-10-20 16:20:00 -04:00
OCHyams f825214411 [DebugInfo][NFC] Refactor debug intrinsic copy and delete to instead just move
Reviewed By: jryans

Differential Revision: https://reviews.llvm.org/D133304
2022-10-20 15:12:49 +01:00
Nikita Popov 747f27d97d [AA] Rename getModRefBehavior() to getMemoryEffects() (NFC)
Follow up on D135962, renaming the method name to match the new
type name.
2022-10-19 11:03:54 +02:00
Nikita Popov 1a9d9823c5 [AA] Rename uses of FunctionModRefBehavior (NFC)
Followup to D135962 to rename remaining uses of
FunctionModRefBehavior to MemoryEffects. Does not touch API names
yet, but also updates variables names FMRB/MRB to ME, to match the
new type name.
2022-10-19 10:54:47 +02:00
Florian Hahn c65513444b
[IndVars] Forget SCEV for instruction and users before replacing it.
Extra invalidation is needed here to clear stale values to fix a
verification failure.

Fixes #58440.
2022-10-18 17:38:14 +01:00
uabkaka da137d041b [SimplifyLibCalls] Add NoUndef/NonNull/Dereferenceable attributes to iprintf/siprintf
When SimplifyLibCalls fail to optimize printf and sprintf it add
NoUndef/NonNull/Dereferenceable attributes. This patch add the same attributes
if SimplifyLibCalls optimize printf/sprintf into the integer only
iprintf/siprintf.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D136140
2022-10-18 16:36:35 +02:00
Florian Hahn e302fa89aa
[LoopUnroll] Forget exit values when making changes.
When unrolling, the exit values in LCSSA phis will get updated.
Invalidate cached SCEV values for those phis in case SCEV looked through
a exit phi.

Fixes #58340.
2022-10-18 15:12:24 +01:00
Florian Hahn 16cf666bb7
[Loop] Move block and loop dispo invalidation to makeLoopInvariant.
makeLoopInvariant may recursively move its operands to make them
invariant, before moving the passed in instruction. Those recursively
moved instructions are currently missed when invalidating block and loop
dispositions.

To address this, move the invalidation code to Loop::makeLoopInvariant.

Fixes #58314.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D135909
2022-10-14 21:58:14 +01:00
Florian Hahn be611ef7fa
[LoopRotation] Also drop block dispositions.
LoopRotation may also fold basic blocks, so cached block dispositions
also need to be dropped.

Fixes #58291.
2022-10-11 15:25:27 +01:00
Max Kazantsev a8a07890aa [NFC] Refine API: add missing const notion in hasPartialIVCondition 2022-10-11 19:35:16 +07:00
Nikita Popov df8264c46a [SimplifyLibCalls] Use helper methods to query attributes (NFC) 2022-10-11 11:41:28 +02:00
Florian Hahn 19ad1cd5ce
Recommit "[SCEV] Support clearing Block/LoopDispositions for a single value."
This reverts commit 92f698f01f.

The updated version of the patch includes handling for non-SCEVable
types. A test case has been added in ec86e9a99b.
2022-10-07 20:15:44 +01:00
Florian Hahn 92f698f01f
Revert "[SCEV] Support clearing Block/LoopDispositions for a single value."
This reverts commit 9e931439dd.

This commit causes a crash when TSan, e.g. with
https://lab.llvm.org/buildbot/#/builders/70/builds/28309/steps/10/logs/stdio

Reverting while I extract a reproducer and submit a fix.
2022-10-07 17:58:54 +01:00
Florian Hahn 9e931439dd
[SCEV] Support clearing Block/LoopDispositions for a single value.
Extend forgetBlockAndLoopDisposition to allow clearing information for a
single value. This can be useful when only a single value is changed,
e.g. because the instruction is moved.

We also need to clear the cached values for all SCEV users, because they
may depend on the starting value's disposition.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D134614
2022-10-07 16:07:17 +01:00
Nikita Popov b43a4d0850 [LoopPeeling] Support peeling loops with non-latch exits
Loop peeling currently requires that a) the latch is exiting
b) a branch and c) other exits are unreachable/deopt. This patch
removes all of these limitations, and adds the necessary branch
weight updating support. It essentially works the same way as
before with latch -> exiting terminator and
loop trip count -> per exit trip count.

It's worth noting that there are still other limitations in
profitability heuristics: This patch enables peeling of loops to
make conditions invariant (which is pretty much always highly
profitable if possible), while peeling to make loads dereferenceable
still checks that non-latch exits are unreachable and PGO-based
peeling has even more conditions. Those checks could be relaxed
later if we consider those cases profitable.

The motivation for this change is that loops using iterator adaptors
in Rust often optimize very badly, and end up with a loop phi of the
form phi(true, false) in the final result. Peeling eliminates that
phi and conditions based on it, which enables a lot of follow-on
simplification.

Differential Revision: https://reviews.llvm.org/D134803
2022-10-07 12:35:52 +02:00
Nikita Popov ccf53cae32 [ValueTracking] Remove unused Offset argument in getConstantStringInfo() (NFC) 2022-10-07 11:35:55 +02:00