Commit Graph

4442 Commits

Author SHA1 Message Date
Mehdi Amini a407ec9b6d Revert "Revert "[NFC][llvm] Make the contructors of `ElementCount` private.""
Was reverted because MLIR/Flang builds were broken, these APIs have been
fixed in the meantime.
2020-08-19 17:26:36 +00:00
Mehdi Amini 4fc56d70aa Revert "[NFC][llvm] Make the contructors of `ElementCount` private."
This reverts commit 264afb9e6a.
(and dependent 6b742cc48 and fc53bd610f)

MLIR/Flang are broken.
2020-08-19 17:21:37 +00:00
Francesco Petrogalli 264afb9e6a [NFC][llvm] Make the contructors of `ElementCount` private.
Differential Revision: https://reviews.llvm.org/D86120
2020-08-19 16:26:44 +00:00
Alex Zinenko 874aef875d [llvm] support graceful failure of DataLayout parsing
Existing implementation always aborts on syntax errors in a DataLayout
description. While this is meaningful for consuming textual IR modules, it is
inconvenient for users that may need fine-grained control over the layout from,
e.g., command-line options. Propagate errors through the parsing functions and
only abort in the top-level parsing function instead.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D85650
2020-08-17 15:10:37 +02:00
Philip Reames 6b2105456a [Statepoint] Remove code related to inline operand bundles
This code becomes dead for valid IR after 48f4312 and a96fc46.  The reason for the test change is that the verifier reports the first verification error encountered, in some non-specified visit order.  By removing the verification code in gc.relocates for a statepoint with inline gc operands, I change the error the verifier reports.  And in one case, the checked for error is no longer possible with the bundle representation, so I simply delete the file.
2020-08-14 20:29:41 -07:00
Philip Reames 48f4312d4e Remove inline gc arguments from statepoints
The "gc-live" operand bundles were recently added, and all tests have been updated to use that format.  A migration period was provided, though it's worth noting these intrinsics are experimental, so formally there is no compatibile requirement.

This is an extension to a96fc46.  "gc-live" hadn't been implemented at the point that patch was initially posted.
2020-08-14 19:44:24 -07:00
Philip Reames a96fc4638b Remove deopt and gc transition arguments from gc.statepoint intrinsic
(Forgot to land this a couple of weeks back.)

In a recent series of changes, I've introduced support for using the respective operand bundle kinds on the statepoint. At the moment, code supports either/or, but there's no need to keep the old support around. For the moment, I am simply changing the specification and verifier to require zero length argument sets in the intrinsic.

The intrinsic itself is experimental. Given that, there's no forward serialization needed. The in tree uses and generation have already been updated to use the new operand bundle based forms, the only folks broken by the change will be those with frontends generating statepoints directly and the updates should be easy.

Why not go ahead and just remove the arguments entirely? Well, I plan to. But while working on this I've found that almost all of the arguments to the statepoint can be expressed via operand bundles or attributes. Given that, I'm planning a radical simplification of the arguments and figured I'd do one update not several small ones.

Differential Revision: https://reviews.llvm.org/D80892
2020-08-14 16:07:40 -07:00
Vitaly Buka fc4fd89852 [StackSafety] Use ValueInfo in ParamAccess::Call
This avoid GUID lookup in Index.findSummaryInModule.
Follow up for D81242.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D85269
2020-08-14 12:42:44 -07:00
Ben Dunbobbin 4cb016cd2d [X86][ELF] Prefer lowering MC_GlobalAddress operands to .Lfoo$local for STV_DEFAULT only
This patch restricts the behaviour of referencing via .Lfoo$local
local aliases, introduced in https://reviews.llvm.org/D73230, to
STV_DEFAULT globals only.

Hidden symbols via --fvisiblity=hidden (https://gcc.gnu.org/wiki/Visibility)
is an important scenario.

Benefits:

- Improves the size of object files by using fewer STT_SECTION symbols.

- The code reads a bit better (it was not obvious to me without going
  back to the code reviews why the canBenefitFromLocalAlias function
  currently doesn't consider visibility).

- There is also a side benefit in restoring the effectiveness of the
  --wrap linker option and making the behavior of --wrap consistent
  between LTO and normal builds for references within a translation-unit.
  Note: this --wrap behavior (which is specific to LLD) should not be
  considered reliable. See comments on https://reviews.llvm.org/D73230
  for more.

Differential Revision: https://reviews.llvm.org/D85782
2020-08-14 00:09:15 +01:00
Arthur Eubanks 41f49736a9 [ConstProp] Handle insertelement constants
Previously ConstantFoldExtractElementInstruction() would only work with
insertelement instructions, not contants. This properly handles
insertelement constants as well.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D85865
2020-08-13 15:59:17 -07:00
Craig Topper a7a06ded8b Recommit "[InstSimplify] Remove select ?, undef, X -> X and select ?, X, undef -> X transforms" and its follow up patches
This recommits the following patches now that D85684 has landed

1cf6f210a2 [IR] Disable select ? C : undef -> C fold in ConstantFoldSelectInstruction unless we know C isn't poison.
469da663f2 [InstSimplify] Re-enable select ?, undef, X -> X transform when X is provably not poison
122b0640fc [InstSimplify] Don't fold vectors of partial undef in SimplifySelectInst if the non-undef element value might produce poison
ac0af12ed2 [InstSimplify] Add test cases for opportunities to fold select ?, X, undef -> X when we can prove X isn't poison
9b1e95329a [InstSimplify] Remove select ?, undef, X -> X and select ?, X, undef -> X transforms
2020-08-12 10:45:27 -07:00
Vedant Kumar 30c1633386 Revert "[Instruction] Add updateLocationAfterHoist helper"
This reverts commit 4a646ca9e2.

This is causing some bots to fail with "!dbg attachment points at wrong
subprogram for function", like:

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/67958/steps/stage%201%20check/logs/stdio
2020-08-11 14:54:09 -07:00
Vedant Kumar 4a646ca9e2 [Instruction] Add updateLocationAfterHoist helper
Introduce a helper on Instruction which can be used to update the debug
location after hoisting.

Use this in GVN and LICM, where we were mistakenly introducing new line
0 locations after hoisting (the docs recommend dropping the location in
this case).

For more context, see the discussion in https://reviews.llvm.org/D60913.

Differential Revision: https://reviews.llvm.org/D85670
2020-08-11 14:05:20 -07:00
Benjamin Kramer 8134c2c7ff [AutoUpgrade] Simplify code
No need to set the name on an instruction that's going away, just move
it from the old instruction to the new one.
2020-08-11 13:22:58 +02:00
Benjamin Kramer 1de173c049 [X86][FPEnv] Fix a use after free
Found by asan!
2020-08-11 13:00:47 +02:00
Wang, Pengfei 9512525947 [X86][FPEnv] Teach X86 mask compare intrinsics to respect strict FP semantics.
When we use mask compare intrinsics under strict FP option, the masked
elements shouldn't raise any exception. So, we cann't replace the
intrinsic with a full compare + "and" operation.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D85385
2020-08-11 10:28:41 +08:00
Johannes Doerfert fa5d22a045 [OpenMP][NFC] Reuse OMPIRBuilder `struct ident_t` handling in Clang
Replace the `ident_t` handling in Clang with the methods offered by the
OMPIRBuilder. This cuts down on the clang code as well as the
differences between the two, making further transitions easier. Tests
have changed but there should not be a real functional change. The most
interesting difference is probably that we stop generating local ident_t
allocations for now and just use globals. Given that this happens only
with debug info, the location part of the `ident_t` is probably bigger
than the test anyway. As the location part is already a global, we can
avoid the allocation, memcpy, and store in favor of a constant global
that is slightly bigger. This can be revisited if there are
complications.

Reviewed By: ABataev

Differential Revision: https://reviews.llvm.org/D80735
2020-08-10 17:13:26 -05:00
Vitaly Buka 654266bea9 [StackSafety] Use getSignedMin() to serialize ranges
Almost NFC as it's important only for full sets which should not
be serialized at all.
2020-08-09 14:53:13 -07:00
Jianzhou Zhao aedaa077f5 Reduce dropTriviallyDeadConstantArrays cumulative time percentage from 17% to 4%
The history of dropTriviallyDeadConstantArrays is like this. Because the appending linkage uses too much memory (http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20150105/251381.html), dropTriviallyDeadConstantArrays was introduced (https://reviews.llvm.org/rG81f385b0c6ea37dd7195a65be162c75bbdef29d2) to release unused constant arrays. Recently, dropTriviallyDeadConstantArrays was improved (https://reviews.llvm.org/rG81f385b0c6ea37dd7195a65be162c75bbdef29d2) to reduce its quadratic cost.

Our recent LTO profiling shows that when a target is large, 15-20% of time cost is from the SetVector::insert called by dropTriviallyDeadConstantArrays.

A large application has hundreds or thousands of modules; each module calls dropTriviallyDeadConstantArrays once for cleaning up tens of thousands of ConstantArrays a module has. In those ConstantArrays, usually around 5 can be deleted; a very very few deleted ConstantArrays reference other ConstantArrays: less than 10 out of millions.

Given this, the cost of SetVector::insert is mainly from the construction of WorkList from ArrayConstants. This motivated the fix that iterates ArrayConstants directly, and uses WorkList only when necessary.

Our evaluation shows that
1) The cumulative time percentage of dropTriviallyDeadConstantArrays is reduced from 15-17% to 4-6%.
2) For targets with LTO time > 20min, the time reduction is about 20%.
3) No observable performance impact for build without using LTO.

{F12506218}
{F12506221}

Reviewed By: mehdi_amini, tejohnson, jdoerfert

Differential Revision: https://reviews.llvm.org/D85379
2020-08-07 11:36:30 -07:00
Bevin Hansson 5de6c56f7e [Intrinsic] Add sshl.sat/ushl.sat, saturated shift intrinsics.
Summary:
This patch adds two intrinsics, llvm.sshl.sat and llvm.ushl.sat,
which perform signed and unsigned saturating left shift,
respectively.

These are useful for implementing the Embedded-C fixed point
support in Clang, originally discussed in
http://lists.llvm.org/pipermail/llvm-dev/2018-August/125433.html
and
http://lists.llvm.org/pipermail/cfe-dev/2018-May/058019.html

Reviewers: leonardchan, craig.topper, bjope, jdoerfert

Subscribers: hiraditya, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83216
2020-08-07 15:09:24 +02:00
Simon Pilgrim 807467009d [X86] getX86MaskVec - replace mask limit from NumElts < 8 with NumElts <= 4
As noted on PR46885, the number of mask elements should always be a power of 2, so to fix the static analyzer warning we are better off replacing the condition to <= 4, and I've added a pow2 assertion as well.
2020-08-06 11:46:19 +01:00
Shinji Okumura 1c2777f585 [NFC][APInt][DenseMapInfo] Move DenseMapAPIntKeyInfo into DenseMap.h as DenseMapInfo<APInt>
`DenseMapAPIntKeyInfo` is now located in `lib/IR/LLVMContextImpl.h`.
Moved it into `include/ADT/DenseMapInfo.h` to use it.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D85131
2020-08-03 23:31:13 +09:00
Simon Pilgrim e202236721 [IR] Add IRBuilderBase::CreateVectorSplat(ElementCount EC) variant
As discussed on D81500, this adds a more general ElementCount variant of the build helper and converts the (non-scalable) unsigned NumElts variant to use it internally.
2020-08-02 16:55:38 +01:00
Shinji Okumura 376b64926b Revert "[Attributor] AAPotentialValues Interface"
The commit cause build failure.
2020-08-02 22:49:52 +09:00
Shinji Okumura d3f01b6681 [Attributor] AAPotentialValues Interface
This is a split patch of D80991.
This patch introduces AAPotentialValues and its interface only.
For more detail of AAPotentialValues abstract attribute, see the original patch.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D83283
2020-08-02 19:12:17 +09:00
Teresa Johnson 1479cdfe4f [ThinLTO] Compile time improvement to propagateAttributes
I found that propagateAttributes was ~23% of a thin link's run time
(almost 4x higher than the second hottest function). The main reason is
that it re-examines a global var each time it is referenced. This
becomes unnecessary once it is marked both non read only and non write
only. I added a set to avoid doing redundant work, which dropped the
runtime of that thin link by almost 15%.

I made a smaller efficiency improvement (no measurable impact) to skip
all summaries for a VI if the first copy is dead. I added an assert to
ensure that all copies are dead if any is. The code in
computeDeadSymbols marks all summaries for a VI as live. There is one
corner case where it was skipping marking an alias as live, that I
fixed. However, since the code earlier marked all copies of a preserved
GUID's VI as live, and each 'visit' marks all copies live, the only case
where this could make a difference is summaries that were marked live
when they were built initially, and that is only a few special compiler
generated symbols and inline assembly symbols, so it likely is never
provoked in practice.

Differential Revision: https://reviews.llvm.org/D84985
2020-07-31 10:54:02 -07:00
Nikita Popov 9ebeac6788 [ConstantRange][CVP] Make use of abs poison flag
Pass the abs poison flag to the underlying ConstantRange
implementation, allowing CVP to simplify based on it.

Importantly, this recognizes that abs with poison flag is actually
non-negative...
2020-07-30 23:06:10 +02:00
Nikita Popov 94f8120cb9 [ConstantRange] Support abs with poison flag
This just adds the ConstantRange support, including exhaustive
testing. It's not wired up to the IR intrinsic flag yet.
2020-07-30 22:49:28 +02:00
Nikita Popov d8a98a9c35 [ConstantRange][CVP] Compute min/max/abs intrinsic ranges
Wire up ConstantRange::intrinsic() to the existing primitives for
min, max and abs.

The poison flag on abs is not yet taken into account.
2020-07-30 22:21:34 +02:00
Yuanfang Chen 555cf42f38 [NewPM][PassInstrument] Add PrintPass callback to StandardInstrumentations
Problem:
Right now, our "Running pass" is not accurate when passes are wrapped in adaptor because adaptor is never skipped and a pass could be skipped. The other problem is that "Running pass" for a adaptor is before any "Running pass" of passes/analyses it depends on. (for example, FunctionToLoopPassAdaptor). So the order of printing is not the actual order.

Solution:
Doing things like PassManager::Debuglogging is very intrusive because we need to specify Debuglogging whenever adaptor is created. (Actually, right now we're not specifying Debuglogging for some sub-PassManagers. Check PassBuilder)

This patch move debug logging for pass as a PassInstrument callback. We could be sure that all running passes are logged and in the correct order.

This could also be used to implement hierarchy pass logging in legacy PM. We could also move logging of pass manager to this if we want.

The test fixes looks messy. It includes changes:
- Remove PassInstrumentationAnalysis
- Remove PassAdaptor
- If a PassAdaptor is for a real pass, the pass is added
- Pass reorder (to the correct order), related to PassAdaptor
- Add missing passes (due to Debuglogging not passed down)

Reviewed By: asbirlea, aeubanks

Differential Revision: https://reviews.llvm.org/D84774
2020-07-30 10:07:57 -07:00
Nikita Popov 897bdca4b8 [ConstantRange] Add API for intrinsics (NFC)
This adds a common API for compute constant ranges of intrinsics.
The intention here is that
a) we can reuse the same code across different passes that handle
   constant ranges, i.e. this can be reused in SCCP
b) we only have to add knowledge about supported intrinsics to
   ConstantRange, not any consumers.

Differential Revision: https://reviews.llvm.org/D84587
2020-07-29 22:16:27 +02:00
Yuanfang Chen 7a2e1122ae [NewPM][PassInstrument] Make PrintIR and TimePasses to use before-pass-run callback
Reviewed By: asbirlea, aeubanks

Differential Revision: https://reviews.llvm.org/D84773
2020-07-29 08:26:36 -07:00
Chuanqi Xu dd4106d22e [NFC] Edit the comment in User::replaceUsesOfWith 2020-07-29 10:02:04 +08:00
Johannes Doerfert 450dc09d69 [SROA][Mem2Reg] Use efficient droppable use API (after D83976)
Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D84804
2020-07-28 17:41:01 -05:00
Jon Roelofs 736423af53 [OldPM] Print out a bit more when passes lie about changing IR
https://reviews.llvm.org/D84686
2020-07-28 10:01:24 -06:00
serge-sans-paille 3218c064d6 [legacyPM] Do not compute preserved analysis if there's no local change
All analysis are preserved if there's no local change, and thanks to
3667d87a33 this property is enforced for all
passes.

Skipping the dependency computation improves the performance when there's a lot
of small functions, where only a few change happen.

Thanks to Nikita Popov who provided this numbers (extract below)

https://llvm-compile-time-tracker.com/compare.php?from=183342c0a9850e60dd7a004b651c83dfb3a7d25e&to=f2f91e6a2743070471cc9471e4e8c646e50c653c&stat=instructions

O3: (number of instructions)
Benchmark               Old             New
kimwitu++               60783M          59968M          (-1.34%)
sqlite3                 73200M          73083M          (-0.16%)
consumer-typeset        52776M          52712M          (-0.12%)
Bullet                  133709M         132940M         (-0.58%)
tramp3d-v4              123864M         123186M         (-0.55%)
mafft                   55534M          55477M          (-0.10%)
ClamAV                  76292M          76164M          (-0.17%)
lencod                  103190M         103061M         (-0.13%)
SPASS                   64068M          63713M          (-0.55%)
7zip                    197332M         196308M         (-0.52%)
geomean                 85750M          85389M          (-0.42%)

Differential Revision: https://reviews.llvm.org/D80707
2020-07-28 11:01:04 +02:00
Simon Pilgrim 5b5b3ce0ad IRPrintingPasses.h - simplify unnecessary header with forward declarations. NFC.
Remove duplicate PassManager.h include in IRPrintingPasses.cpp
2020-07-27 14:51:28 +01:00
Alina Sbirlea 8bf4c1f4fb Reapply "[DomTree] Replace ChildrenGetter with GraphTraits over GraphDiff."
This is the part of the patch that's moving the Updates to a CFGDiff
object. Splitting off from the clean-up work merging the two branches when BUI is null.

Differential Revision: https://reviews.llvm.org/D77341
2020-07-24 14:10:50 -07:00
Johannes Doerfert ce8928f2e4 [Mem2Reg] Teach promote to register about droppable instructions
This is the first of two patches to address PR46753. We basically allow
mem2reg to promote allocas that are used in doppable instructions, for
now that means `llvm.assume`. The uses of the alloca (or a bitcast or
zero offset GEP from there) are replaced by `undef` in the droppable
instructions.

Reviewed By: Tyker

Differential Revision: https://reviews.llvm.org/D83976
2020-07-24 15:15:38 -05:00
Simon Pilgrim 0128b9505c Revert rG5dd566b7c7b78bd- "PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI."
This reverts commit 5dd566b7c7.

Causing some buildbot failures that I'm not seeing on MSVC builds.
2020-07-24 13:02:33 +01:00
Simon Pilgrim 5dd566b7c7 PassManager.h - remove unnecessary Function.h/Module.h includes. NFCI.
PassManager.h is one of the top headers in the ClangBuildAnalyzer frontend worst offenders list.

This exposes a large number of implicit dependencies on various forward declarations/includes in other headers that need addressing.
2020-07-24 12:40:50 +01:00
Simon Pilgrim 9c81c2372d PassTimingInfo.h - remove unused includes. NFC.
Remove duplicate includes from PassTimingInfo.cpp that already exist in PassTimingInfo.h
2020-07-23 16:20:13 +01:00
Sebastian Neubauer 2a6c871596 [InstCombine] Move target-specific inst combining
For a long time, the InstCombine pass handled target specific
intrinsics. Having target specific code in general passes was noted as
an area for improvement for a long time.

D81728 moves most target specific code out of the InstCombine pass.
Applying the target specific combinations in an extra pass would
probably result in inferior optimizations compared to the current
fixed-point iteration, therefore the InstCombine pass resorts to newly
introduced functions in the TargetTransformInfo when it encounters
unknown intrinsics.
The patch should not have any effect on generated code (under the
assumption that code never uses intrinsics from a foreign target).

This introduces three new functions:
TargetTransformInfo::instCombineIntrinsic
TargetTransformInfo::simplifyDemandedUseBitsIntrinsic
TargetTransformInfo::simplifyDemandedVectorEltsIntrinsic

A few target specific parts are left in the InstCombine folder, where
it makes sense to share code. The largest left-over part in
InstCombineCalls.cpp is the code shared between arm and aarch64.

This allows to move about 3000 lines out from InstCombine to the targets.

Differential Revision: https://reviews.llvm.org/D81728
2020-07-22 15:59:49 +02:00
Jay Foad 5e5bda74b6 [IR] Simplify Use::swap. NFCI.
The new implementation makes it clear that there are exactly two
conditional stores (after the initial no-op optimization). By contrast
the old implementation had seven conditionals, some hidden inside other
functions.

This commit can change the order of operands in operand lists, hence the
tweak to one test case.

Differential Revision: https://reviews.llvm.org/D80116
2020-07-21 12:15:12 +01:00
Matt Arsenault 780cef1f34 Verifier: Check byref address space for AMDGPU calling conventions 2020-07-20 11:13:11 -04:00
Matt Arsenault ad8e900cb3 Verifier: Disallow byval and similar for AMDGPU calling conventions
These imply stack-like semantics, which doesn't make any sense for
entry points.
2020-07-20 10:58:57 -04:00
Alok Kumar Sharma 2d10258a31 [DebugInfo] Support for DW_AT_associated and DW_AT_allocated.
Summary:
This support is needed for the Fortran array variables with pointer/allocatable
attribute. This support enables debugger to identify the status of variable
whether that is currently allocated/associated.

  for pointer array (before allocation/association)
  without DW_AT_associated

(gdb) pt ptr
type = integer (140737345375288:140737354129776)
(gdb) p ptr
value requires 35017956 bytes, which is more than max-value-size

  with DW_AT_associated

(gdb) pt ptr
type = integer (:)
(gdb) p ptr
$1 = <not associated>

  for allocatable array (before allocation)

  without DW_AT_allocated

(gdb) pt arr
type = integer (140737345375288:140737354129776)
(gdb) p arr
value requires 35017956 bytes, which is more than max-value-size

  with DW_AT_allocated

(gdb) pt arr
type = integer, allocatable (:)
(gdb) p arr
$1 = <not allocated>

    Testing
- unit test cases added
- check-llvm
- check-debuginfo

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D83544
2020-07-20 19:54:35 +05:30
Matt Arsenault 5e999cbe8d IR: Define byref parameter attribute
This allows tracking the in-memory type of a pointer argument to a
function for ABI purposes. This is essentially a stripped down version
of byval to remove some of the stack-copy implications in its
definition.

This includes the base IR changes, and some tests for places where it
should be treated similarly to byval. Codegen support will be in a
future patch.

My original attempt at solving some of these problems was to repurpose
byval with a different address space from the stack. However, it is
technically permitted for the callee to introduce a write to the
argument, although nothing does this in reality. There is also talk of
removing and replacing the byval attribute, so a new attribute would
need to take its place anyway.

This is intended avoid some optimization issues with the current
handling of aggregate arguments, as well as fixes inflexibilty in how
frontends can specify the kernel ABI. The most honest representation
of the amdgpu_kernel convention is to expose all kernel arguments as
loads from constant memory. Today, these are raw, SSA Argument values
and codegen is responsible for turning these into loads.

Background:

There currently isn't a satisfactory way to represent how arguments
for the amdgpu_kernel calling convention are passed. In reality,
arguments are passed in a single, flat, constant memory buffer
implicitly passed to the function. It is also illegal to call this
function in the IR, and this is only ever invoked by a driver of some
kind.

It does not make sense to have a stack passed parameter in this
context as is implied by byval. It is never valid to write to the
kernel arguments, as this would corrupt the inputs seen by other
dispatches of the kernel. These argumets are also not in the same
address space as the stack, so a copy is needed to an alloca. From a
source C-like language, the kernel parameters are invisible.
Semantically, a copy is always required from the constant argument
memory to a mutable variable.

The current clang calling convention lowering emits raw values,
including aggregates into the function argument list, since using
byval would not make sense. This has some unfortunate consequences for
the optimizer. In the aggregate case, we end up with an aggregate
store to alloca, which both SROA and instcombine turn into a store of
each aggregate field. The optimizer never pieces this back together to
see that this is really just a copy from constant memory, so we end up
stuck with expensive stack usage.

This also means the backend dictates the alignment of arguments, and
arbitrarily picks the LLVM IR ABI type alignment. By allowing an
explicit alignment, frontends can make better decisions. For example,
there's real no advantage to an aligment higher than 4, so a frontend
could choose to compact the argument layout. Similarly, there is a
high penalty to using an alignment lower than 4, so a frontend could
opt into more padding for small arguments.

Another design consideration is when it is appropriate to expose the
fact that these arguments are all really passed in adjacent
memory. Currently we have a late IR optimization pass in codegen to
rewrite the kernel argument values into explicit loads to enable
vectorization. In most programs, unrelated argument loads can be
merged together. However, exposing this property directly from the
frontend has some disadvantages. We still need a way to track the
original argument sizes and alignments to report to the driver. I find
using some side-channel, metadata mechanism to track this
unappealing. If the kernel arguments were exposed as a single buffer
to begin with, alias analysis would be unaware that the padding bits
betewen arguments are meaningless. Another family of problems is there
are still some gaps in replacing all of the available parameter
attributes with metadata equivalents once lowered to loads.

The immediate plan is to start using this new attribute to handle all
aggregate argumets for kernels. Long term, it makes sense to migrate
all kernel arguments, including scalars, to be passed indirectly in
the same manner.

Additional context is in D79744.
2020-07-20 10:23:09 -04:00
Nikita Popov f7dce88915 [IR] Fix MSVC warning (NFC)
As requested by Andrew Kaylor, rewrite this code in a way that does
not warn on old MSVC versions.

Avoid the buggy constexpr warning by just not using constexpr and
removing the static_assert that depends on it.
2020-07-17 18:27:39 +02:00
Eric Christopher 7bfaa40086 Temporarily Revert "[AssumeBundles] Use operand bundles to encode alignment assumptions"
due to the performance bugs filed in https://bugs.llvm.org/show_bug.cgi?id=46753.

An SROA change soon may obviate some of these problems.

This reverts commit 8d09f20798.
2020-07-16 11:54:04 -07:00
Matt Arsenault d909764cc7 Use findEnumAttribute helper for preallocated 2020-07-16 13:50:49 -04:00
Matt Arsenault 023883a834 IR: Rename Argument::hasPassPointeeByValueAttr to prepare for byref
When the byref attribute is added, there will need to be two similar
functions for the existing cases which have an associate value copy,
and byref which does not. Most, but not all of the existing uses will
use the existing version.

The associated size function added by D82679 also needs to
contextually differ, and will help eliminate a few places still
relying on pointee element types.
2020-07-16 13:50:49 -04:00
Craig Topper 00f3579aea Revert "[InstSimplify] Remove select ?, undef, X -> X and select ?, X, undef -> X transforms" and subsequent patches
This reverts most of the following patches due to reports of miscompiles.
I've left the added test cases with comments updated to be FIXMEs.

1cf6f210a2 [IR] Disable select ? C : undef -> C fold in ConstantFoldSelectInstruction unless we know C isn't poison.
469da663f2 [InstSimplify] Re-enable select ?, undef, X -> X transform when X is provably not poison
122b0640fc [InstSimplify] Don't fold vectors of partial undef in SimplifySelectInst if the non-undef element value might produce poison
ac0af12ed2 [InstSimplify] Add test cases for opportunities to fold select ?, X, undef -> X when we can prove X isn't poison
9b1e95329a [InstSimplify] Remove select ?, undef, X -> X and select ?, X, undef -> X transforms
2020-07-15 22:02:33 -07:00
Craig Topper 81cfb90f85 [IR] Add a few asserts to provide a better failure signature if you try to create a load/store/alloca with no alignment or insertion position
If no alignment is specified we try to find the datalayout by using the insert position to get the module so we can get the datalayout. But if those are null, then we deference a null pointer.

This patch adds asserts to make the failure a little more obvious than just seg faulting.

Differential Revision: https://reviews.llvm.org/D83829
2020-07-14 18:47:44 -07:00
Giorgis Georgakoudis aef60af34e [CallGraph] Ignore callback uses
Summary:
Ignore callback uses when adding a callback function
in the CallGraph. Callback functions are typically
created when outlining, e.g. for OpenMP, so they have
internal scope and linkage. They should not be added
to the ExternalCallingNode since they are only callable
by the specified caller function at creation time.

A CGSCC pass, such as OpenMPOpt, may need to update
the CallGraph by adding a new outlined callback function.
Without ignoring callback uses, adding breaks CGSCC
pass restrictions and results to a broken CallGraph.

Reviewers: jdoerfert

Subscribers: hiraditya, sstefan1, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83370
2020-07-14 13:08:49 -07:00
Teresa Johnson 6014c46c80 Restore "[WPD/LowerTypeTests] Delay lowering/removal of type tests until after ICP"
This restores commit 80d0a137a5, and the
follow on fix in 873c0d0786, with a new
fix for test failures after a 2-stage clang bootstrap, and a more robust
fix for the Chromium build failure that an earlier version partially
fixed. See also discussion on D75201.

Reviewers: evgeny777

Subscribers: mehdi_amini, Prazek, hiraditya, steven_wu, dexonsmith, arphaman, davidxl, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D73242
2020-07-14 12:16:57 -07:00
Sjoerd Meijer 2b3c505d0f [Matrix] Intrinsic descriptions
This changes the matrix load/store intrinsic definitions to load/store from/to
a pointer, and not from/to a pointer to a vector, as discussed in D83477.

This also includes the recommit of "[Matrix] Tighten LangRef definitions and
Verifier checks" which adds improved language reference descriptions of the
matrix intrinsics and verifier checks.

Differential Revision: https://reviews.llvm.org/D83785
2020-07-14 19:58:16 +01:00
serge-sans-paille 3667d87a33 Double check that passes correctly set their Modified status
The approach is simple: if a pass reports that it's not modifying a
Function/Module, compute a loose hash of that Function/Module and compare it
with the original one. If we report no change but there's a hash change, then we
have an error.

This approach misses a lot of change but it's not super intrusive and can
detect most of the simple mistakes.

Differential Revision: https://reviews.llvm.org/D80916
2020-07-14 09:56:49 +02:00
Tyker 8d09f20798 [AssumeBundles] Use operand bundles to encode alignment assumptions
Summary:
NOTE: There is a mailing list discussion on this: http://lists.llvm.org/pipermail/llvm-dev/2019-December/137632.html

Complemantary to the assumption outliner prototype in D71692, this patch
shows how we could simplify the code emitted for an alignemnt
assumption. The generated code is smaller, less fragile, and it makes it
easier to recognize the additional use as a "assumption use".

As mentioned in D71692 and on the mailing list, we could adopt this
scheme, and similar schemes for other patterns, without adopting the
assumption outlining.

Reviewers: hfinkel, xbolva00, lebedev.ri, nikic, rjmccall, spatel, jdoerfert, sstefan1

Reviewed By: jdoerfert

Subscribers: thopre, yamauchi, kuter, fhahn, merge_guards_bot, hiraditya, bollu, rkruppe, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D71739
2020-07-14 01:05:58 +02:00
Sjoerd Meijer 4ff7ed3310 Revert "[Matrix] Tighten LangRef definitions and Verifier checks."
This reverts commit f4d29d6e8c.

Hm, some build bot failures, reverting it while I investigate that.
2020-07-12 19:19:25 +01:00
Sjoerd Meijer f4d29d6e8c [Matrix] Tighten LangRef definitions and Verifier checks.
This tightens the matrix intrinsic definitions in LLVM LangRef and adds
correspondings checks to the IR Verifier.

Differential Revision: https://reviews.llvm.org/D83477
2020-07-12 19:07:22 +01:00
Arthur Eubanks 21b4cc1db9 Reland [NFC] Derive from PassInfoMixin for no-op/printing passes
PassInfoMixin should be used for all NPM passes, rater than a custom
`name()`.

This caused ambiguous references in LegacyPassManager.cpp, so had to
remove "using namespace llvm::legacy" and move some things around.

Reviewed By: ychen, asbirlea

Differential Revision: https://reviews.llvm.org/D83498
2020-07-10 12:51:28 -07:00
Davide Italiano fdb7856d54 Revert "[NFC] Derive from PassInfoMixin for no-op/printing passes"
This reverts commit 8039d2c3bf as
it breaks the modules build on macOS.
2020-07-10 11:19:13 -07:00
Craig Topper 1cf6f210a2 [IR] Disable select ? C : undef -> C fold in ConstantFoldSelectInstruction unless we know C isn't poison.
This matches the recent change to InstSimplify from D83440.

Differential Revision: https://reviews.llvm.org/D83535
2020-07-10 10:42:25 -07:00
Arthur Eubanks 8039d2c3bf [NFC] Derive from PassInfoMixin for no-op/printing passes
PassInfoMixin should be used for all NPM passes, rater than a custom
`name()`.

This caused ambiguous references in LegacyPassManager.cpp, so had to
remove "using namespace llvm::legacy" and move some things around.

The passes had to be moved to the llvm namespace, or else they would get
printed as "(anonymous namespace)::FooPass".

Reviewed By: ychen, asbirlea

Differential Revision: https://reviews.llvm.org/D83498
2020-07-09 16:58:30 -07:00
Roman Lebedev c2a61ef388
Revert "[CallGraph] Ignore callback uses"
This likely has broken test/Transforms/Attributor/IPConstantProp/ tests.
http://45.33.8.238/linux/22502/step_12.txt

This reverts commit 205dc0922d.
2020-07-10 00:02:07 +03:00
Giorgis Georgakoudis 205dc0922d [CallGraph] Ignore callback uses
Summary:
Ignore callback uses when adding a callback function
in the CallGraph. Callback functions are typically
created when outlining, e.g. for OpenMP, so they have
internal scope and linkage. They should not be added
to the ExternalCallingNode since they are only callable
by the specified caller function at creation time.

A CGSCC pass, such as OpenMPOpt, may need to update
the CallGraph by adding a new outlined callback function.
Without ignoring callback uses, adding breaks CGSCC
pass restrictions and results to a broken CallGraph.

Reviewers: jdoerfert

Subscribers: hiraditya, sstefan1, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83370
2020-07-09 13:13:46 -07:00
Matt Arsenault 6f5d9136b2 OpaquePtr: Don't check pointee type for byval/preallocated
Since none of these users really care about the actual type, hide the
type under a new size-getting attribute to go along with
hasPassPointeeByValueAttr. This will work better for the future byref
attribute, which may end up only tracking the byte size and not the IR
type.

We currently have 3 parameter attributes that should carry the type
(technically inalloca does not yet). The APIs are somewhat awkward
since preallocated/inalloca piggyback on byval in some places, but in
others are treated as distinct attributes. Since these are all
mutually exclusive, we should probably just merge all the attribute
infrastructure treating these as totally distinct attributes.
2020-07-09 09:37:41 -04:00
Benjamin Kramer b44470547e Make helpers static. NFC. 2020-07-09 13:48:56 +02:00
Gui Andrade ff7900d5de [LLVM] Accept `noundef` attribute in function definitions/calls
The `noundef` attribute indicates an argument or return value which
may never have an undef value representation.

This patch allows LLVM to parse the attribute.

Differential Revision: https://reviews.llvm.org/D83412
2020-07-08 19:02:04 +00:00
serge-sans-paille bf9a940c3f Revert "Double check that passes correctly set their Modified status"
This reverts commit 37afd99c76.
2020-07-08 18:14:40 +02:00
Arthur Eubanks 470bf7b5a2 [Preallocated] Add @llvm.call.preallocated.teardown
This cleans up the stack allocated by a @llvm.call.preallocated.setup.
Should either call the teardown or the preallocated call to clean up the
stack. Calling both is UB.

Add LangRef.

Add verifier check that the token argument is a @llvm.call.preallocated.setup.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D83354
2020-07-08 08:48:44 -07:00
serge-sans-paille 37afd99c76 Double check that passes correctly set their Modified status
The approach is simple: if a pass reports that it's not modifying a
Function/Module, compute a loose hash of that Function/Module and compare it
with the original one. If we report no change but there's a hash change, then we
have an error.

This approach misses a lot of change but it's not super intrusive and can
detect most of the simple mistakes.

Differential Revision: https://reviews.llvm.org/D80916
2020-07-08 17:36:13 +02:00
Christopher Tetreault 021d56abb9 [SVE] Make Constant::getSplatValue work for scalable vector splats
Summary:
Make Constant::getSplatValue recognize scalable vector splats of the
form created by ConstantVector::getSplat. Add unit test to verify that
C == ConstantVector::getSplat(C)->getSplatValue() for fixed width and
scalable vector splats

Reviewers: efriedma, spatel, fpetrogalli, c-rhodes

Reviewed By: efriedma

Subscribers: sdesmalen, tschuett, hiraditya, rkruppe, psnobl, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82416
2020-07-07 13:45:51 -07:00
Guillaume Chatelet 74c723757e [NFC] Adding the align attribute on Atomic{CmpXchg|RMW}Inst
This is the first step to add support for the align attribute to AtomicRMWInst and AtomicCmpXchgInst.
Next step is to add support in IRBuilder and BitcodeReader.
Bug: https://bugs.llvm.org/show_bug.cgi?id=27168

Differential Revision: https://reviews.llvm.org/D83136
2020-07-07 09:54:13 +00:00
Roman Lebedev 69dca6efc6
[NFCI][IR] Introduce CallBase::Create() wrapper
Summary:
It is reasonably common to want to clone some call with different bundles.
Let's actually provide an interface to do that.

Reviewers: chandlerc, jdoerfert, dblaikie, nickdesaulniers

Reviewed By: nickdesaulniers

Subscribers: llvm-commits, hiraditya

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83248
2020-07-07 01:16:36 +03:00
Roman Lebedev 7ea46aee36
Revert "[AssumeBundles] Use operand bundles to encode alignment assumptions"
Assume bundle can have more than one entry with the same name,
but at least AlignmentFromAssumptionsPass::extractAlignmentInfo() uses
getOperandBundle("align"), which internally assumes that it isn't the
case, and happily crashes otherwise.

Minimal reduced reproducer: run `opt -alignment-from-assumptions` on

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

%0 = type { i64, %1*, i8*, i64, %2, i32, %3*, i8* }
%1 = type opaque
%2 = type { i8, i8, i16 }
%3 = type { i32, i32, i32, i32 }

; Function Attrs: nounwind
define i32 @f(%0* noalias nocapture readonly %arg, %0* noalias %arg1) local_unnamed_addr #0 {
bb:
  call void @llvm.assume(i1 true) [ "align"(%0* %arg, i64 8), "align"(%0* %arg1, i64 8) ]
  ret i32 0
}

; Function Attrs: nounwind willreturn
declare void @llvm.assume(i1) #1

attributes #0 = { nounwind "reciprocal-estimates"="none" }
attributes #1 = { nounwind willreturn }


This is what we'd have with -mllvm -enable-knowledge-retention

This reverts commit c95ffadb24.
2020-07-04 23:49:23 +03:00
jasonliu 572dde55ee [XCOFF][AIX] Use 'L..' instead of '.L' for getPrivateGlobalPrefix in DataLayout
Summary:
D80831 changed part of the prefix usage for AIX.
But there are other places getting prefix from DataLayout.
This patch intends to make prefix usage consistent on AIX.

Reviewed by: hubert.reinterpretcast, daltenty

Differential Revision: https://reviews.llvm.org/D81270
2020-07-03 18:25:14 +00:00
Danila Malyutin 38909f31bd [IR] Short-circuit comparison with itself for Attributes
Differential Revision: https://reviews.llvm.org/D82295
2020-07-03 16:07:14 +03:00
Guillaume Chatelet 063258eb6e [Alignment][NFC] Use 5 bits to store Instructions Alignment
As per [MaxAlignmentExponent]{b7338fb1a6/llvm/include/llvm/IR/Value.h (L688)} alignment is not allowed to be more than 2^29.
Encoded as Log2, this means that storing alignment uses 5 bits.
This patch makes sure all instructions store their alignment in a consistent way, encoded as Log2 and using 5 bits.

Differential Revision: https://reviews.llvm.org/D83119
2020-07-03 08:54:27 +00:00
Guillaume Chatelet 87e2751cf0 [Alignment][NFC] Use proper getter to retrieve alignment from ConstantInt and ConstantSDNode
This patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D83082
2020-07-03 08:06:43 +00:00
Guillaume Chatelet 3587c9c427 [NFC] Use ADT/Bitfields in Instructions
This is an example patch for D81580.

Differential Revision: https://reviews.llvm.org/D81662
2020-07-03 07:20:22 +00:00
Eli Friedman 779e4d82de [IR] Add classof methods to ConstantExpr subclasses.
I didn't notice these were missing when I wrote 1544019.
2020-07-01 11:56:12 -07:00
Simon Pilgrim c83ec0a633 Make dyn_cast results explicitly auto* instead of just auto.
Noticed by clang-tidy llvm-qualified-auto warning.
2020-07-01 16:38:52 +01:00
Simon Pilgrim 0ae989a1fe Pass DebugLoc::appendInlinedAt DebugLoc arg by const reference not value.
Noticed by clang-tidy performance-unnecessary-value-param warning.
2020-07-01 16:38:51 +01:00
Guillaume Chatelet d3085c2501 [Alignment][NFC] Transition and simplify calls to DL::getABITypeAlignment
This patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D82956
2020-07-01 14:31:56 +00:00
Alexey Bataev e35a5876e4 Revert "[StackSafety,NFC] Remove unneded constexpr"
This reverts commit 38470baa54 because it
breaks builds with lld and gold linkers.
2020-07-01 08:41:45 -04:00
Simon Pilgrim 65647ed1e5 Pass DIEnumerator APInt args by const reference not value.
Noticed by clang-tidy performance-unnecessary-value-param warning.
2020-07-01 13:16:07 +01:00
Simon Pilgrim 2c7af6dffc Pass stripNonLineTableDebugInfo remapDebugLoc lambda DebugLoc arg by const reference not value.
Noticed by clang-tidy performance-unnecessary-value-param warning.
2020-07-01 12:37:47 +01:00
Simon Pilgrim 0144f501a6 AttrBuilder::merge/remove - use const& for iterator values in for-range loops.
Noticed by clang-tidy performance-for-range-copy warning.
2020-07-01 12:01:18 +01:00
Vitaly Buka 38470baa54 [StackSafety,NFC] Remove unneded constexpr
Differential Revision: https://reviews.llvm.org/D80908
2020-07-01 02:54:27 -07:00
Eli Friedman df1f371e28 [SVE] Reject vector struct indexes for scalable vectors.
It's messy to pattern-match, and completely unnecessary: scalar indexes
work equally well.

See also discussion on D81620 and D82061.

Differential Revision: https://reviews.llvm.org/D82430
2020-06-30 13:52:38 -07:00
Eli Friedman 15440191b5 [IR] Delete llvm::Constants using the correct type.
In most cases, this doesn't have much impact: the destructors just call
the base class destructor anyway.  A few subclasses of ConstantExpr
actually store non-trivial data, though. Make sure we clean up
appropriately.

This is sort of ugly, but I don't see a good alternative given the
constraints.

Issue found by asan buildbots running the testcase for D80330.

Differential Revision: https://reviews.llvm.org/D82509
2020-06-30 12:37:53 -07:00
Simon Pilgrim 32f8cd9a6a Pass MDFieldPrinter::printAPInt APInt arg by reference not value.
Noticed by clang-tidy performance-unnecessary-value-param warning.
2020-06-30 17:18:20 +01:00
Guillaume Chatelet 2c5ff48e61 [Alignment][NFC] Migrate AtomicExpandPass to Align
This is a followup on D78403.
I'm unsure about `getAtomicOpAlign` overloads that take `AtomicRMWInst` and `AtomicCmpXchgInst`, shouldn't `getAlign` provide the correct answer already?

Differential Revision: https://reviews.llvm.org/D81369
2020-06-30 09:54:45 +00:00
Reid Kleckner 6d01a94193 Silence unused var warning in NDEBUG build 2020-06-29 11:40:49 -07:00
Sebastian Neubauer 874fcd4e8f Add intrinsic helper function
It simplifies getting generic argument types from intrinsics.

Differential Revision: https://reviews.llvm.org/D81084
2020-06-29 14:47:46 +02:00
Guillaume Chatelet 368a5e3a66 [Alignment][NFC] migrate DataLayout::getPreferredAlignment
This patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D82752
2020-06-29 11:24:36 +00:00
Nikita Popov 9a334a4d20 [IR] Store attributes that are available "somewhere" (NFC)
I noticed that for some benchmarks we spend quite a bit of time
inside AttributeList::hasAttrSomewhere(), mainly when checking
for the "returned" attribute. Most of the time the attribute will
not be present, in which case this function has to walk through
the whole attribute list and check for the attribute at each index.

This patch adds a cache of all "available somewhere" attributes
inside AttributeListImpl. This makes the structure 12 bytes larger,
but I don't think that's problematic, as attribute lists are uniqued.
Compile-time in terms of instructions retired improves by 0.4% on
average, but >1% for sqlite.

Differential Revision: https://reviews.llvm.org/D81867
2020-06-27 10:44:59 +02:00
Simon Pilgrim ba2ac689e6 AsmWriter - printConstVCalls/printNonConstVCalls - avoid std::vector pass by value. NFCI. 2020-06-27 09:38:37 +01:00