Commit Graph

31928 Commits

Author SHA1 Message Date
Alexandre Ganea aba5b91b69 Re-land [CodeView] Add full repro to LF_BUILDINFO record
This patch writes the full -cc1 command into the resulting .OBJ, like MSVC does. This allows for external tools (Recode, Live++) to rebuild a source file without any external dependency but the .OBJ itself (other than the compiler) and without knowledge of the build system.

The LF_BUILDINFO record stores a full path to the compiler, the PWD (CWD at program startup), a relative or absolute path to the source, and the full CC1 command line. The stored command line is self-standing (does not depend on the environment). In the same way, MSVC doesn't exactly store the provided command-line, but an expanded version (a somehow equivalent of CC1) which is also self-standing.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2022-01-19 19:44:37 -05:00
Johannes Doerfert dd75a6b2ae [DWARF][FIX] Try not to crash for nvptx with missing debug information
This prevents crashes in the OpenMP offload pipeline as not everything
is properly annotated with debug information, e.g., the runtimes we link
in. While we might want to have them annotated, it seems to be generally
useful to gracefully handle missing debug info rather than crashing.

TODO: A test is missing and can hopefully be distilled prior to landing.

This fixes #51079.

Differential Revision: https://reviews.llvm.org/D116959
2022-01-19 18:40:13 -06:00
Mircea Trofin 073e09683d Fix build break introduced by D117147 2022-01-19 11:43:51 -08:00
Mircea Trofin e67430cca4 [MLGO] ML Regalloc Eviction Advisor
The bulk of the implementation is common between 'release' mode (==AOT-ed
model) and 'development' mode (for training), the main difference is
that in development mode, we may also log features (for training logs),
inject scoring information (currently after the Virtual Register
Rewriter) and then produce the log file.

This patch also introduces the score injection pass, 'Register
Allocation Pass Scoring', which is trivially just logging the score in
development mode.

Differential Revision: https://reviews.llvm.org/D117147
2022-01-19 11:00:32 -08:00
Simon Pilgrim d6fee6c3b0 [DAG] SelectionDAG::computeKnownBits - add mul(x,x) self-multiply handling (PR48683)
Pass the SelfMultiply flag to KnownBits::mul() - added at D108992

https://alive2.llvm.org/ce/z/NN_eaR
2022-01-19 17:39:32 +00:00
Daniel Thornburgh 2e2999cd44 [NFC] Test commit to verify commit access. 2022-01-18 18:03:26 -08:00
Matt Arsenault 5599c43124 GlobalISel: Swap order of operand checks in ConstantFoldVectorBinop
Since constants are canonicalized to the RHS, this is more likely to
exit early.
2022-01-18 17:21:02 -05:00
Matt Arsenault da72822763 GlobalISel: Fix CSEMIRBuilder mishandling constant folds of vectors
This was ignoring the requested result register, resulting in a
missing def when this happened in the IRTranslator. Fixes some crashes
and verifier errors at -O0.

Alternatively we could pass DstOps to the constant fold functions.
2022-01-18 17:21:02 -05:00
David Green 100763a88f [DAG] Extend SearchForAndLoads with any_extend handling
This extends the code in SearchForAndLoads to be able to look through
ANY_EXTEND nodes, which can be created from mismatching IR types where
the AND node we begin from only demands the low parts of the register.
That turns zext and sext into any_extends as only the low bits are
demanded. To be able to look through ANY_EXTEND nodes we need to handle
mismatching types in a few places, potentially truncating the mask to
the size of the final load.

Recommitted with a more conservative check for the type of the extend.

Differential Revision: https://reviews.llvm.org/D117457
2022-01-18 21:03:08 +00:00
Matt Arsenault 984451eafc PostRAPseudos: Don't preserve kills on some implicit copy operands
This fixes a verifier error I ran into at -O0. A subregister copy had
an implicit kill of an overlapping superregister, which was partially
redefined by the copy. The preserved implicit operand killed
subregisters made live earlier in the sequence. AMDGPU already uses
similar logic for whether to preserve the kill of the superregister on
the final instruction if there's overlap.
2022-01-18 13:52:04 -05:00
Fraser Cormack c8e33978fb [VP] Propagate align parameter attr on VP gather/scatter to ISel
This patch fixes a case where the 'align' parameter attribute on the
pointer operands to llvm.vp.gather and llvm.vp.scatter was being dropped
during the conversion to the SelectionDAG. The default alignment equal
to the ABI type alignment of the vector type was kept. It also updates
the documentation to reflect the fact that the parameter attribute is
now properly supported.

The default alignment of these intrinsics was previously documented as
being equal to the ABI alignment of the *scalar* type, when in fact that
wasn't the case: the ABI alignment of the vector type was used instead.
This has also been fixed in this patch.

Reviewed By: simoll, craig.topper

Differential Revision: https://reviews.llvm.org/D114423
2022-01-18 17:33:24 +00:00
Sanjay Patel 870591200d [SDAG] remove duplicate functionality when getting shift type for demanded bits; NFCI
This was noted as a potential cleanup in D117508.
getShiftAmountTy() has checks for vector, phase, etc. so it should
handle anything that the caller was trying to account for.
2022-01-18 12:13:45 -05:00
Nikita Popov 0d51b6ab15 [DebugInstrRef] Add some missing const qualifiers (NFC) 2022-01-18 17:19:23 +01:00
Nikita Popov cbaae61422 [DebugInstrRef] Use DenseMap for ValueToLoc (NFC)
Just replacing std::map with DenseMap here is a major regression
-- because this code used an identity hash for ValueIDNum.
Because ValueIDNum is composed of multiple components, it is
important that we use a reasonably good hash function here, so
switch it to hash_value. DenseMapInfo::getHashValue<uint64_t>
would not be sufficient.

This gives a -0.8% geomean improvement on CTMark ReleaseLTO-g.
2022-01-18 17:02:14 +01:00
Vang Thao 10ed1eca24 [MachineSink] Allow sinking of constant or ignorable physreg uses
For AMDGPU, any use of the physical register EXEC prevents sinking even if it is not a real physical register read. Add check to see if a physical
register use can be ignored for sinking.

Also perform same constant and ignorable physical register check when considering sinking in loops.

https://reviews.llvm.org/D116053
2022-01-18 14:17:40 +00:00
Victor Perez b7bf96a258 [LegalizeTypes][VP] Add widening support for vp.reduce.*
When widening these intrinsics, we do not have to insert neutral
elements at the end of the vector as when widening vector.reduce.*
intrinsics, thanks to vector predication semantics.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117467
2022-01-18 10:21:01 +00:00
Hans Wennborg f4615feaa1 Revert "[DAG] Extend SearchForAndLoads with any_extend handling"
This caused builds to fail with

  llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp:5638:
  bool (anonymous namespace)::DAGCombiner::BackwardsPropagateMask(llvm::SDNode *):
  Assertion `NewLoad && "Shouldn't be masking the load if it can't be narrowed"' failed.

See the code review for a link to a reproducer.

> This extends the code in SearchForAndLoads to be able to look through
> ANY_EXTEND nodes, which can be created from mismatching IR types where
> the AND node we begin from only demands the low parts of the register.
> That turns zext and sext into any_extends as only the low bits are
> demanded. To be able to look through ANY_EXTEND nodes we need to handle
> mismatching types in a few places, potentially truncating the mask to
> the size of the final load.
>
> Differential Revision: https://reviews.llvm.org/D117457

This reverts commit 578008789f.
2022-01-18 10:50:55 +01:00
Victor Perez fd1dce35bd [LegalizeTypes][VP] Add splitting support for vp.reduction.*
Split vp.reduction.* intrinsics by splitting the vector to reduce in
two halves, perform the reduction operation in each one of them and
accumulate the results of both operations.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117469
2022-01-18 09:29:24 +00:00
Bjorn Pettersson 65fbe38f0a [DwarfDebug] Restore code that make comments stay aligned in DwarfDebug::emitDebugLocEntry
Commit 2bddab25db removed a piece of code from
DwarfDebug::emitDebugLocEntry that according to code comments
"Make sure comments stay aligned".

This patch restores that piece of code, together with the addition
of some extra checks in an existing lit test to work as a regression
test. Without this patch we incorrectly get
  .byte   159                             # 0
instead of
  .byte   159                             # DW_OP_stack_value

Differential Revision: https://reviews.llvm.org/D117441
2022-01-18 09:46:03 +01:00
David Sherwood f4515ab858 Revert "[CodeGen][AArch64] Ensure isSExtCheaperThanZExt returns true for negative constants"
This reverts commit 197f3c0deb.

Reverting after miscompilation errors discovered with ffmpeg.
2022-01-18 08:40:20 +00:00
Sanjay Patel ba6485e25f [SDAG] add demanded bits transform for bswap
A possible codegen regression for PowerPC is noted in D117406
because we don't recognize a pattern that demands only 1 byte
from a bswap.

This fold  has existed in IR since close to the beginning of LLVM:
https://github.com/llvm/llvm-project/blame/main/llvm/lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp#L794
...so this patch copies that code as much as possible and adapts
it for SDAG.

The test for PowerPC that would change in D117406 is over-reduced
with undefs, so I recreated it for AArch64 and x86 by passing in
pointer args and renamed the values to make the logic clearer.

Differential Revision: https://reviews.llvm.org/D117508
2022-01-17 18:25:42 -05:00
David Green 578008789f [DAG] Extend SearchForAndLoads with any_extend handling
This extends the code in SearchForAndLoads to be able to look through
ANY_EXTEND nodes, which can be created from mismatching IR types where
the AND node we begin from only demands the low parts of the register.
That turns zext and sext into any_extends as only the low bits are
demanded. To be able to look through ANY_EXTEND nodes we need to handle
mismatching types in a few places, potentially truncating the mask to
the size of the final load.

Differential Revision: https://reviews.llvm.org/D117457
2022-01-17 15:25:11 +00:00
David Sherwood 197f3c0deb [CodeGen][AArch64] Ensure isSExtCheaperThanZExt returns true for negative constants
When we know the value we're extending is a negative constant then it
makes sense to use SIGN_EXTEND because this may improve code quality in
some cases, particularly when doing a constant splat of an unpacked vector
type. For example, for SVE when splatting the value -1 into all elements
of a vector of type <vscale x 2 x i32> the element type will get promoted
from i32 -> i64. In this case we want the splat value to sign-extend from
(i32 -1) -> (i64 -1), whereas currently it zero-extends from
(i32 -1) -> (i64 0xFFFFFFFF). Sign-extending the constant means we can use
a single mov immediate instruction.

New tests added here:

  CodeGen/AArch64/sve-vector-splat.ll

I believe we see some code quality improvements in these existing
tests too:

  CodeGen/AArch64/reduce-and.ll
  CodeGen/AArch64/unfold-masked-merge-vector-variablemask.ll

The apparent regressions in CodeGen/AArch64/fast-isel-cmp-vec.ll only
occur because the test disables codegen prepare and branch folding.

Differential Revision: https://reviews.llvm.org/D114357
2022-01-17 11:08:57 +00:00
Nikita Popov 873a7ee7e4 [MachineInstr] Don't include debug uses in bundle header (PR52817)
Following the recommendation in
https://github.com/llvm/llvm-project/issues/52817#issuecomment-1007635426,
this excludes debug instructions when finalizing the bundle. As uses
in debug instructions don't have effects, they will no longer be
included in the BUNDLE header.

Fixes https://github.com/llvm/llvm-project/issues/52817.

Differential Revision: https://reviews.llvm.org/D116945
2022-01-17 10:43:21 +01:00
Bjorn Pettersson 9f237c9e7d [DAGCombine] Refactor DAGCombiner::ReduceLoadWidth. NFCI
Update code comments in DAGCombiner::ReduceLoadWidth and refactor
the handling of SRL a bit. The refactoring is done with the intent
of adding support for folding away SRA by using SEXTLOAD in a
follow-up patch.

The function is also renamed as DAGCombiner::reduceLoadWidth.

Differential Revision: https://reviews.llvm.org/D117104
2022-01-16 20:24:52 +01:00
Fangrui Song 5456249736 [SelectionDAG] Fix -Wunused-variable in -DLLVM_ENABLE_ASSERTIONS=off builds after D117235 2022-01-15 17:13:09 -08:00
Nikita Popov c63a3175c2 [AttrBuilder] Remove ctor accepting AttributeList and Index
Use the AttributeSet constructor instead. There's no good reason
why AttrBuilder itself should exact the AttributeSet from the
AttributeList. Moving this out of the AttrBuilder generally results
in cleaner code.
2022-01-15 22:39:31 +01:00
Fraser Cormack 877d1b3d07 [SelectionDAG][VP] Add splitting/widening for VP_LOAD and VP_STORE
Original patch by @hussainjk.

This patch was split off from D109377 to keep vector legalization
(widening/splitting) separate from vector element legalization
(promoting).

While the original patch added a third overload of
SelectionDAG::getVPStore, this patch takes the liberty of collapsing
those all down to 1, as three overloads seems excessive for a
little-used node.

The original patch also used ModifyToType in places, but that method
still crashes on scalable vector types. Seeing as the other VP
legalization methods only work when all operands need identical
widening, this patch follows in that vein.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117235
2022-01-15 11:41:29 +00:00
Craig Topper e0841f6920 [SelectionDAGBuilder] Remove unneeded vector bitcast from visitTargetIntrinsic.
This seems to be a leftover from a long time ago when there was
an ISD::VBIT_CONVERT and a MVT::Vector. It looks like in those days
the vector type was carried in a VTSDNode.

As far as I know, these days ComputeValueTypes would have already
assigned "Result" the same type we're getting from TLI.getValueType
here. Thus the BITCAST is always a NOP. Verified by adding an assert
and running check-llvm.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D117335
2022-01-14 12:52:49 -08:00
James Y Knight a97e20a3a8 Revert "GlobalISel: Add G_ASSERT_ALIGN hint instruction"
This commit sometimes causes a crash when compiling a vtable thunk. E.g.:

clang '--target=aarch64-grtev4-linux-gnu' -xc++ - -c -o /dev/null <<EOF
struct a {
  virtual int f();
};
struct c {
  virtual int &g() const;
};
struct d : a, c {
  int &g() const;
};
int &d::g() const {}
EOF

Some follow-up commits have been reverted as well:
Revert "IR: Make getRetAlign check callee function attributes"
Revert "Fix MSVC "32-bit shift implicitly converted to 64 bits" warning. NFC."
Revert "Fix MSVC "32-bit shift implicitly converted to 64 bits" warning. NFC."

This reverts commit 4f414af6a7.
This reverts commit a5507d2e25.
This reverts commit 3d2d208f6a.
This reverts commit 07ddfa95e3.
2022-01-14 04:50:07 +00:00
David Sherwood ba471ba8d2 Revert "[CodeGen][AArch64] Ensure isSExtCheaperThanZExt returns true for negative constants"
This reverts commit 31009f0b5a.

It seems to be causing SVE VLA buildbot failures and has introduced a
genuine regression. Reverting for now.
2022-01-13 15:59:43 +00:00
Eugene Zhulenev 764e52f0d4 [DebugInfo][InstrRef] Short-circuit unnecessary preferred location map construction
Reviewed By: cota

Differential Revision: https://reviews.llvm.org/D117162
2022-01-13 06:24:52 -08:00
Simon Pilgrim 4f414af6a7 Fix MSVC "32-bit shift implicitly converted to 64 bits" warning. NFC. 2022-01-13 11:10:50 +00:00
Sebastian Neubauer f4139440f1 [Docs] Fix IR and TableGen grammar inconsistencies
IR:
- globals (and functions, ifuncs, aliases) can have a partition
- catchret has a `to` before the label
- the sint/int types do not exist
- signext comes after the type
- a variable was missing its type

TableGen:
- The second value after a `#` concatenation is optional
  See e.g. llvm/lib/Target/X86/X86InstrAVX512.td:L3351
- IncludeDirective and PreprocessorDirective were never referenced in
  the grammar
- Add some missing ;
- Parent classes of multiclasses can have generic arguments.
  Reuse the `ParentClassList` that is already used in other places.

MIR:
- liveins only allows physical registers, which start with a $

Differential Revision: https://reviews.llvm.org/D116674
2022-01-13 11:55:13 +01:00
David Sherwood 31009f0b5a [CodeGen][AArch64] Ensure isSExtCheaperThanZExt returns true for negative constants
When we know the value we're extending is a negative constant then it
makes sense to use SIGN_EXTEND because this may improve code quality in
some cases, particularly when doing a constant splat of an unpacked vector
type. For example, for SVE when splatting the value -1 into all elements
of a vector of type <vscale x 2 x i32> the element type will get promoted
from i32 -> i64. In this case we want the splat value to sign-extend from
(i32 -1) -> (i64 -1), whereas currently it zero-extends from
(i32 -1) -> (i64 0xFFFFFFFF). Sign-extending the constant means we can use
a single mov immediate instruction.

New tests added here:

  CodeGen/AArch64/sve-vector-splat.ll

I believe we see some code quality improvements in these existing
tests too:

  CodeGen/AArch64/dag-numsignbits.ll
  CodeGen/AArch64/reduce-and.ll
  CodeGen/AArch64/unfold-masked-merge-vector-variablemask.ll

The apparent regressions in CodeGen/AArch64/fast-isel-cmp-vec.ll only
occur because the test disables codegen prepare and branch folding.

Differential Revision: https://reviews.llvm.org/D114357
2022-01-13 09:43:07 +00:00
Matt Arsenault 5a16306c09 GlobalISel: Always enable GISelKnownBits for InstructionSelect
This wasn't running at -O0, and causing crashes for AMDGPU. AMDGPU
needs this to match the addressing modes of stack access instructions,
which is even more important at -O0 than with optimizations.

It currently costs nothing to run ahead of time, so just always enable
it.
2022-01-12 18:57:24 -05:00
Matt Arsenault 5f39a02ea9 RegScavenger: Remove used regs from scavenge candidates
In a future change, AMDGPU will have 2 emergency scavenging indexes in
some situations. The secondary scavenging index ends up being used
recursively when the scavenger calls eliminateFrameIndex for the
emergency spill slot. Without this, it would end up seeing the same
register which was just scavenged in the parent call as free, inserts
a second emergency spill to the same location and returns the same
register when 2 unique free registers are required.

We need to only do this if the register is used. SystemZ uses 2
scavenging slots, but calls the scavenger twice in sequence and not
recursively. In this case the previously scavenged register can be
re-clobbered, but is still tracked in the scavenger until it sees the
deferred restore instruction.
2022-01-12 18:56:52 -05:00
Matt Arsenault 07ddfa95e3 GlobalISel: Add G_ASSERT_ALIGN hint instruction
Insert it for call return values only for now, which is the only case
the DAG handles also.
2022-01-12 18:20:58 -05:00
Matt Arsenault 8a16201a0b GlobalISel: Fix insert point in localizer
This was inserting the new G_CONSTANT after the use, and the later
block scan would run off the end. Fix calling SkipPHIsAndLabels for no
apparent reason.
2022-01-12 13:44:05 -05:00
Mircea Trofin b2d2e93138 [NFC][MLGO] The regalloc reward is float, not int64_t 2022-01-12 09:32:41 -08:00
Mircea Trofin 3150bce078 [NFC][MLGO] Prep a few files before the main ML Regalloc adviser
To avoid trivial changes.
2022-01-12 08:54:00 -08:00
Petar Avramovic c8c5dc766b GlobalIsel: Fix fma combine when one of the operands comes from unmerge
Fma combine assumes that MRI.getVRegDef(Reg)->getOperand(0).getReg() = Reg
which is not true when Reg is defined by instruction with multiple defs
e.g. G_UNMERGE_VALUES.
Fix is to keep register and the instruction that defines register in
DefinitionAndSourceRegister and use when needed.

Differential Revision: https://reviews.llvm.org/D117032
2022-01-12 17:47:25 +01:00
Leonard Grey 0f85393004 [MachO] Port call graph profile section and directive
This ports the `.cg_profile` assembly directive and call graph profile section
generation to MachO from COFF/ELF. Due to MachO section naming rules, the
section is called `__LLVM,__cg_profile` rather than `.llvm.call-graph-profile`
as in COFF/ELF. Support for llvm-readobj is included to facilitate testing.

Corresponding LLD change is D112164

Differential Revision: https://reviews.llvm.org/D112160
2022-01-12 09:22:26 -05:00
Jeremy Morse 6a605b97a2 [DebugInfo] Move flag for instr-ref to LLVM option, from TargetOptions
This feature was previously controlled by a TargetOptions flag, and I
figured that codegen::InitTargetOptionsFromCodeGenFlags would default it
to "on" for all frontends. Enabling by default was discussed here:

  https://lists.llvm.org/pipermail/llvm-dev/2021-November/153653.html

and originally supposed to happen in 3c04507088, but it didn't actually
take effect, as it turns out frontends initialize TargetOptions themselves.
This patch moves the flag from a TargetOptions flag to a global flag to
CodeGen, where it isn't immediately affected by the frontend being used.
Hopefully this will actually cause instr-ref to be on by default on x86_64
now!

This patch is easily reverted, and chances of turbulence are moderately
high. If you need to revert, please consider instead commenting out the
'return true' part of llvm::debuginfoShouldUseDebugInstrRef to turn the
feature off, and dropping me an email.

Differential Revision: https://reviews.llvm.org/D116821
2022-01-12 13:28:01 +00:00
Alexey Lapshin 39385d4cd1 [CodeGen][Debuginfo][NFC] Refactor DIE values SizeOf method to not depend on AsmPrinter.
SizeOf() method of DIE values(unsigned SizeOf(const AsmPrinter *AP, dwarf::Form Form) const)
depends on AsmPrinter. AsmPrinter is too specific class here. This patch removes dependency
on AsmPrinter and use dwarf::FormParams structure instead. It allows calculate DIE values
size without using AsmPrinter. That refactoring is useful for D96035([dsymutil][DWARFlinker]
implement separate multi-thread processing for compile units.)

Differential Revision: https://reviews.llvm.org/D116997
2022-01-12 13:15:26 +03:00
Craig Topper 63b17eb9ec [RISCV] Add strictfp support for compares.
This adds support for STRICT_FSETCC(quiet) and STRICT_FSETCCS(signaling).

FEQ matches well to STRICT_FSETCC oeq.
FLT/FLE matches well to STRICT_FSETCCS olt/ole.

Others require commuting operands or multiple instructions.

STRICT_FSETCC olt/ole/ogt/oge/ult/ule/ugt/uge uses FLT/FLE,
but we need to save/restore FFLAGS around them to avoid spurious
exceptions. I've implemented pseudo instructions with a
CustomInserter to insert the save/restore CSR instructions.
Unfortunately, this doesn't honor exceptions for signaling NANs
but I'm not sure if signaling nans are really supported by the
constrained intrinsics.

STRICT_FSETCC one and ueq expand to a pair of FLT instructions
with a save/restore of fflags around each. This could be improved
in the future.

There may be some opportunities to generate better code for strict
comparisons mixed with nonans fast math flags. I've left FIXMEs in
the .td files for that.

Co-Authored-by: ShihPo Hung <shihpo.hung@sifive.com>

Reviewed By: arcbbb

Differential Revision: https://reviews.llvm.org/D116694
2022-01-11 20:01:41 -08:00
Matt Arsenault 5a434ceafb GlobalISel: Use cloneVirtualRegister in localizer 2022-01-11 16:10:12 -05:00
Nick Desaulniers 4edb9983cb [SelectionDAG] treat X constrained labels as i for asm
Completely rework how we handle X constrained labels for inline asm.

X should really be treated as i. Then existing tests can be moved to use
i D115410 and clang can just emit i D115311. (D115410 and D115311 are
callbr, but this can be done for label inputs, too).

Coincidentally, this simplification solves an ICE uncovered by D87279
based on assumptions made during D69868.

This is the third approach considered. See also discussions v1 (D114895)
and v2 (D115409).

Reported-by: kernel test robot <lkp@intel.com>
Fixes: https://github.com/ClangBuiltLinux/linux/issues/1512

Reviewed By: void, jyknight

Differential Revision: https://reviews.llvm.org/D115688
2022-01-11 10:29:40 -08:00
Nick Desaulniers 9c4b49db19 [ShrinkWrap] check for PPC's non-callee-saved LR
As pointed out in https://reviews.llvm.org/D115688#inline-1108193, we
don't want to sink the save point past an INLINEASM_BR, otherwise
prologepilog may incorrectly sink a prolog past the MBB containing an
INLINEASM_BR and into the wrong MBB.

ShrinkWrap is getting this wrong because LR is not in the list of callee
saved registers. Specifically, ShrinkWrap::useOrDefCSROrFI calls
RegisterClassInfo::getLastCalleeSavedAlias which reads
CalleeSavedAliases which was populated by
RegisterClassInfo::runOnMachineFunction by iterating the list of
MCPhysReg returned from MachineRegisterInfo::getCalleeSavedRegs.

Because PPC's LR is non-allocatable, it's NOT considered callee saved.
Add an interface to TargetRegisterInfo for such a case and use it in
Shrinkwrap to ensure we don't sink a prolog past an INLINEASM or
INLINEASM_BR that clobbers LR.

Reviewed By: jyknight, efriedma, nemanjai, #powerpc

Differential Revision: https://reviews.llvm.org/D116424
2022-01-11 10:01:34 -08:00
David Sherwood 51497dc0b2 [IR] Change vector.splice intrinsic to reject out-of-bounds indices
I've changed the definition of the experimental.vector.splice
instrinsic to reject indices that are known to be or possibly
out-of-bounds. In practice, this means changing the definition so that
the index is now only valid in the range [-VL, VL-1] where VL is the
known minimum vector length. We use the vscale_range attribute to
take the minimum vscale value into account so that we can permit
more indices when the attribute is present.

The splice intrinsic is currently only ever generated by the vectoriser,
which will never attempt to splice vectors with out-of-bounds values.
Changing the definition also makes things simpler for codegen since we
can always assume that the index is valid.

This patch was created in response to review comments on D115863

Differential Revision: https://reviews.llvm.org/D115933
2022-01-11 09:37:39 +00:00
Nick Desaulniers 649b11ef8b git-clang-format HEAD~ 2022-01-10 18:34:30 -08:00
Nick Desaulniers 301e911740 [TargetLowering] precommit refactor from D115688 NFC
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2022-01-10 18:32:13 -08:00
Mircea Trofin b191c1f0f9 [NFC][regalloc] Pull out some AllocationOrder/CostPerUseLimit eviction logic
We are reusing that logic in the ML implementation.

Differential Revision: https://reviews.llvm.org/D116075
2022-01-10 15:47:31 -08:00
Nadav Rotem e2cc091a7d Fix a missed opportunity to merge stores.
This commit fixes a missed opportunity in merging consecutive stores.
The code that searches for stores skipped the case of stores that
directly connect to the root. The comment above the implementation lists
this case but the code did not handle it. I found this pattern when
looking into the shared_ptr destructor. GCC generates the right
sequence. Here is a small repo:

    int foo(int* buff) {
        buff[0] = 0;
        int x = buff[1];
        buff[1] = 0;
        return x;
    }

Differential Revision: https://reviews.llvm.org/D116895
2022-01-10 13:49:02 -08:00
Mircea Trofin e121269131 [NFC][regalloc] Pass RAGreedy to eviction adviser
This patch simplifies the interface between RAGreedy and the eviction
adviser by passing the allocator to the adviser, which allows the latter
to extract needed information as needed, rather than requiring it be passed
piecemeal at construction time (which would also complicate later
evolution).

Part of this, the patch also moves ExtraRegInfo back to RAGreedy. We
keep the encapsulation of ExtraRegInfo because it has benefits (e.g.
improved readability by abstracting access to the cascade info) and also
simpler re-initialization at regalloc pass re-entry time (we just flush
the Optional).

Differential Revision: https://reviews.llvm.org/D116669
2022-01-10 11:55:16 -08:00
Matt Arsenault 0ba4e4b500 GlobalISel: Pass DebugLoc to getFunctionLiveInPhysReg
Fixes crash in assertion about dropping debug info.
2022-01-10 13:50:52 -05:00
Serge Guelton d2cc6c2d0c Use a sorted array instead of a map to store AttrBuilder string attributes
Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor slightly heavier, and involves extra
allocation for each new string attribute. Storing the attribute key/value as
strings implies extra allocation/copy step.

Use a sorted vector instead. Given the low number of attributes generally
involved, this is cheaper, as showcased by

https://llvm-compile-time-tracker.com/compare.php?from=5de322295f4ade692dc4f1823ae4450ad3c48af2&to=05bc480bf641a9e3b466619af43a2d123ee3f71d&stat=instructions

Differential Revision: https://reviews.llvm.org/D116599
2022-01-10 14:49:53 +01:00
Chen Zheng 2c46ca96e2 [PowerPC] fast isel can lower intrinsics call on AIX.
Reviewed By: qiucf

Differential Revision: https://reviews.llvm.org/D114778
2022-01-10 02:30:05 +00:00
Craig Topper a500f7f48f [SelectionDAG] Add FP_TO_UINT_SAT/FP_TO_SINT_SAT to computeKnownBits/computeNumSignBits.
These nodes should saturate to their saturating VT. We can use this
information to know the bits past the VT are all zeros or all sign bits.

I think we might only have test coverage for the unsigned case. I'll
verify and add tests.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D116870
2022-01-09 17:48:05 -08:00
Alexander Shaposhnikov 22430ede7e [CodeGen] Rename emitCalleeSavedFrameMoves
This diff renames emitCalleeSavedFrameMoves to avoid conflicts with
non-virtual methods of derived classes having the same name but different semantics.
E.g. the class AArch64FrameLowering used to have (non-virtual) "emitCalleeSavedFrameMoves"
but it started to override TargetFrameLowering::emitCalleeSavedFrameMoves after
https://github.com/llvm/llvm-project/commit/c3e6555616 though its usage and semantics didn't change.
P.S. for x86 there was no conflict because the signature of
non-virtual X86FrameLowering::emitCalleeSavedFrameMoves is different

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D114140
2022-01-10 01:33:04 +00:00
Jay Foad 50fb44eebb [GlobalISel] Use getPreferredShiftAmountTy in one more G_UBFX combine
Change CombinerHelper::matchBitfieldExtractFromShrAnd to use
getPreferredShiftAmountTy for the shift-amount-like operands of G_UBFX
just like all the other G_[SU]BFX combines do. This better matches the
AMDGPU legality rules for these instructions.

Differential Revision: https://reviews.llvm.org/D116803
2022-01-08 09:20:44 +00:00
Jay Foad ff971873b3 [GlobalISel] Fix legality checks for G_UBFX combines
1. Fix CombinerHelper::matchBitfieldExtractFromAnd to check legality
   with the correct types for the G_UBFX that it builds.
2. Fix AMDGPUTargetLowering::isConstantUnsignedBitfieldExtractLegal to
   match the legality rules: result and first operand can be s32 or s64
   but the "shift amount" operands are always s32.
3. Add AMDGPU tests where the post-legalizer combiner would create
   illegal MIR without the above fixes.

Differential Revision: https://reviews.llvm.org/D116802
2022-01-08 09:20:44 +00:00
Kazu Hirata 4e2ec7e38d [llvm] Remove unused forward declarations (NFC) 2022-01-07 20:00:34 -08:00
Kazu Hirata b932bdf59f [llvm] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-01-07 17:45:09 -08:00
Jay Foad 3f3fe4a5cf [GlobalISel] Fix typo Extact to Extract in function name. NFC. 2022-01-07 11:13:35 +00:00
Nikita Popov 0312fe2901 [CodeGen] Support opaque pointers for inline asm
This is the last part of D116531. Fetch the type of the indirect
inline asm operand from the elementtype attribute, rather than
the pointer element type.

Fixes https://github.com/llvm/llvm-project/issues/52928.
2022-01-07 10:57:38 +01:00
Nikita Popov e4d1779990 [IR] Add ConstraintInfo::hasArg() helper (NFC)
Checking whether a constraint corresponds to an argument is a
recurring pattern.
2022-01-07 10:44:38 +01:00
Victor Perez 38efa68b08 [LegalizeTypes][VP] Add splitting support for vp.select
Split vp.select in a similar way as vselect, splitting also the length
parameter.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D116651
2022-01-07 08:46:01 +00:00
Kazu Hirata 2aed08131d [llvm] Use true/false instead of 1/0 (NFC)
Identified with modernize-use-bool-literals.
2022-01-07 00:39:14 -08:00
Kazu Hirata 410480e32b Ensure newlines at the end of files (NFC) 2022-01-06 23:44:02 -08:00
Arlo Siemsen 3d10997e42 Add Rust to CodeView SourceLanguage (CV_CFL_LANG) enum
Microsoft has added several new entries to the CV_CFL_LANG enum, including Rust:
    https://docs.microsoft.com/en-us/visualstudio/debugger/debug-interface-access/cv-cfl-lang

This change adds Rust to the corresponding LLVM enum and translates `dwarf::DW_LANG_Rust` to `SourceLanguage::Rust` in the CodeView AsmPrinter.

This means that Rust will no longer emit as Masm.

Differential Revision: https://reviews.llvm.org/D115300
2022-01-06 14:27:08 -08:00
Mircea Trofin 68ac7b1701 [NFC][mlgo] Add feature declarations for the ML regalloc advisor
This just adds feature declarations and some boilerplate.

Differential Revision: https://reviews.llvm.org/D116076
2022-01-05 11:54:01 -08:00
David Green fffd663c87 [CodeGen] Initialize MaxBytesForAlignment in TargetLoweringBase::TargetLoweringBase.
This appears to be missing from D114590, causing sanitizer errors.
2022-01-05 19:34:27 +00:00
Luís Ferreira 34435fd105 [llvm] Add support for DW_TAG_immutable_type
Added documentation about DW_TAG_immutable_type too.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D113633
2022-01-05 19:17:08 +00:00
Craig Topper 88ecdd30f6 [LegalizeTypes] Remove IsVP argument from type legalization methods. NFC
We can either check the opcode or number of operands or use
ISD::isVPOpcode inside the methods.

In some places I've used number of operands figuring that it is
cheaper than isVPOpcode. I've included isVPOpcode in an assert to
verify.

Reviewed By: frasercrmck

Differential Revision: https://reviews.llvm.org/D116578
2022-01-05 09:00:48 -08:00
Nicholas Guy 73d92faa2f [CodeGen] Emit alignment "Max Skip" operand
The current AsmPrinter has support to emit the "Max Skip" operand
(the 3rd of .p2align), however has no support for it to actually be specified.
Adding MaxBytesForAlignment to MachineBasicBlock provides this capability on a
per-block basis. Leaving the value as default (0) causes no observable differences
in behaviour.

Differential Revision: https://reviews.llvm.org/D114590
2022-01-05 12:54:30 +00:00
Victor Perez 96e220e688 [LegalizeTypes][VP] Add integer promotion support for vp.select
Promote select, vselect and vp.select in a similar way.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D116400
2022-01-05 11:01:52 +00:00
Victor Perez df5226dfb3 [LegalizeTypes][VP] Add widening support for vp.select
Widen vp.select the same way as select and vselect.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D116407
2022-01-05 09:21:11 +00:00
Craig Topper a04b532505 [LegalizeIntegerTypes][RISCV] Teach PromoteSetCCOperands to check sign bits of unsigned compares.
Unsigned compares work with either zero extended or sign extended
inputs just like equality comparisons. I didn't allow this when
I refactored the code in D116421 due to lack of tests. But I've
since found a simple C test case that demonstrates when this can be
useful.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D116617
2022-01-04 12:38:47 -08:00
Jack Andersen 5b1337184b [DebugInfo] Avoid triggering global location assert for 2-byte pointer sizes.
D111404 moved a 4/8 byte check assert into a block taken by 2-byte platforms.
Since these platforms do not take the branches where the pointer size is used,
sink the assert accordingly.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D116480
2022-01-04 15:16:36 -05:00
Michael Liao 56ec762a76 [regalloc] Fix GCC warning `-Wattributes`. NFC.
- Mark it with LLVM_LIBRARY_VISIBILITY to preserve the legacy
  visibility.
2022-01-04 12:05:57 -05:00
Mircea Trofin 64e56f8356 [NFC] Expose isRematerializable and copyHint from CalcSpillWeights
We need to reuse them for the ML regalloc eviction advisor, as we
'explode' the weight calculation into sub-features.

Differential Revision: https://reviews.llvm.org/D116074
2022-01-04 08:11:49 -08:00
Mircea Trofin c41610778b [NFC][regalloc] Introduce RegAllocGreedy.h
This was suggested in D114831. It should simplify the relation between
eviction advisor and the allocator, and simplify ingesting more features
tied to the internals of the allocator, in the future.

This change simply pulls out RAGreedy, places it in the llvm namespace,
and cleans up a bit the includes in the new header file.

Differential Revision: https://reviews.llvm.org/D116114
2022-01-04 08:04:55 -08:00
Simon Moll 4c2aba999e [VP][ISel] use LEGALPOS for legalization action
Use the VPIntrinsics.def's LEGALPOS that is specified with every VP
SDNode to determine which return or operand value type shall be used to
infer the legalization action.

Reviewed By: frasercrmck

Differential Revision: https://reviews.llvm.org/D116594
2022-01-04 14:50:49 +01:00
Simon Pilgrim 882c083889 [DAG] TargetLowering::SimplifySetCC - use APInt::getMinSignedBits() helper. NFC. 2022-01-04 13:48:36 +00:00
Nikita Popov 4ef560ec60 [ELF] Handle .init_array prefix consistently
Currently, the code in TargetLoweringObjectFile only assigns
@init_array section type to plain .init_array sections, but not
prioritized sections like .init_array.00001.

This is inconsistent with the interpretation in the AsmParser
(see 791523bae6/llvm/lib/MC/MCParser/ELFAsmParser.cpp (L621-L632))
and upcoming expectations in LLD
(see https://github.com/rust-lang/rust/issues/92181 for context).

This patch assigns @init_array section type to all sections with an
.init_array prefix. The same is done for .fini_array and
.preinit_array as well. With that, the logic matches the AsmParser.

Differential Revision: https://reviews.llvm.org/D116528
2022-01-04 09:42:58 +01:00
Craig Topper cbcbbd6ac8 [ValueTracking][SelectionDAG] Rename ComputeMinSignedBits->ComputeMaxSignificantBits. NFC
This function returns an upper bound on the number of bits needed
to represent the signed value. Use "Max" to match similar functions
in KnownBits like countMaxActiveBits.

Rename APInt::getMinSignedBits->getSignificantBits. Keeping the old
name around to keep this patch size down. Will do a bulk rename as
follow up.

Rename KnownBits::countMaxSignedBits->countMaxSignificantBits.

Reviewed By: lebedev.ri, RKSimon, spatel

Differential Revision: https://reviews.llvm.org/D116522
2022-01-03 11:33:30 -08:00
Kazu Hirata e5947760c2 Revert "[llvm] Remove redundant member initialization (NFC)"
This reverts commit fd4808887e.

This patch causes gcc to issue a lot of warnings like:

  warning: base class ‘class llvm::MCParsedAsmOperand’ should be
  explicitly initialized in the copy constructor [-Wextra]
2022-01-03 11:28:47 -08:00
Victor Perez 5527139302 [RISCV][VP] Add RVV codegen for [nX]vXi1 vp.select
Expand [nX]vXi1 vp.select the same way as [nX]vXi1 vselect.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D115546
2022-01-02 23:12:32 -08:00
Kazu Hirata 7e163afd9e Remove redundant void arguments (NFC)
Identified by modernize-redundant-void-arg.
2022-01-02 10:20:19 -08:00
Kazu Hirata fd4808887e [llvm] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-01-01 16:18:18 -08:00
Kazu Hirata 69ccc96162 [llvm] Use the default constructor for SDValue (NFC) 2022-01-01 10:36:59 -08:00
Craig Topper 243b7aaf51 [SelectionDAG] Use KnownBits::countMinSignBits() to simplify the end of ComputeNumSignBits.
This matches what is done in ValueTracking.cpp

Reviewed By: RKSimon, foad

Differential Revision: https://reviews.llvm.org/D116423
2021-12-31 17:29:57 -08:00
Craig Topper d00e438cfe [RISCV][LegalizeIntegerTypes] Teach PromoteSetCCOperands not to sext i32 comparisons for RV64 if the promoted values are already zero extended.
This is similar to what is done for targets that prefer zero extend
where we avoid using a zero extend if the promoted values are sign
extended.

We'll also check for zero extended operands for ugt, ult, uge, and ule when the
target prefers sign extend. This is different than preferring zero extend, where
we only check for sign bits on equality comparisons.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D116421
2021-12-31 17:15:20 -08:00
Craig Topper 7d659c6ac7 [LegalizeIntegerTypes] Rename NewLHS/NewRHS arguments to DAGTypeLegalizer::PromoteSetCCOperands. NFC
The 'New' only makes sense in the context of these being
output arguments, but they are also used as inputs first.
Drop the 'New' and just call them LHS/RHS.

Factored out of D116421.
2021-12-30 15:31:43 -08:00
Craig Topper 15787ccd45 [RISCV] Add support for STRICT_LRINT/LLRINT/LROUND/LLROUND. Tests for other strict intrinsics.
This patch adds isel support for STRICT_LRINT/LLRINT/LROUND/LLROUND.

It also adds test cases for f32 and f64 constrained intrinsics that
correspond to the intrinsics in float-intrinsics.ll and
double-intrinsics.ll. Support for promoting the integer argument of
STRICT_FPOWI was added.

I've skipped adding tests for f16 intrinsics, since we don't have libcalls
for them and we have inconsistent support for promoting them in LegalizeDAG.
This will need to be examined more closely.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D116323
2021-12-30 11:54:32 -08:00
modimo ba51d26ec4 [CodeView] Clamp Frontend version
D43002 introduced a test debug-info-objname.cpp that outputted the current compiler version into CodeView. Internally we appended a date to the patch version and overflowed the 16-bits allocated to that space. This change clamps the Frontend version outputted values to 16-bits like rGd1185fc081ead71a8bf239ff1814f5ff73084c15 did for the Backend version.

Testing:
ninja check-all
newly added tests correctly clamps and no longer asserts when trying to output the field

Reviewed By: aganea

Differential Revision: https://reviews.llvm.org/D116243
2021-12-28 15:22:18 -08:00
Craig Topper 1c6b740d4b [TargetLowering] Remove workaround for old behavior of getShiftAmountTy. NFC
getShiftAmountTy used to directly return the shift amount type from
the target which could be too small for large illegal types. For
example, X86 always returns i8.

The code here detected this and used i32 instead if it won't fit. This
behavior was added to getShiftAmountTy in D112469 so we no longer need
this workaround.
2021-12-28 14:08:25 -08:00
Kazu Hirata 5a667c0e74 [llvm] Use nullptr instead of 0 (NFC)
Identified with modernize-use-nullptr.
2021-12-28 08:52:25 -08:00
Kazu Hirata d09a284dfb [CodeGen] Drop unnecessary const from return types (NFC)
Identified with readability-const-return-type.
2021-12-28 00:38:11 -08:00
Petar Avramovic 508e39afe0 GlobalISel: remove redundant line added in D114198. NFC 2021-12-27 12:14:13 +01:00
David Blaikie 2bddab25db DebugInfo: Don't hash DIE offsets before they're computed
Instead of hashing DIE offsets, hash DIE references the same as they
would be when used outside of a loclist - that is, deep hash the type on
first use, and hash the numbering on subsequent uses.

This does produce different hashes for different type references, where
it did not before (because we were hashing zero all the time - so it
didn't matter what type was referenced, the hash would be identical).

This also allows us to enforce that the DIE offset (& size) is not
queried before it is used (which came up while investigating another bug
recently).
2021-12-25 16:09:12 -08:00
Kazu Hirata 2d303e6781 Remove redundant return and continue statements (NFC)
Identified with readability-redundant-control-flow.
2021-12-24 23:17:54 -08:00
Fangrui Song ea2d4c5881 [GlobalISel] Fix -Wunused-function in -DLLVM_ENABLE_ASSERTIONS=off builds after D114198 2021-12-24 00:55:54 -08:00
David Blaikie b05df0287b Revert "[DWARF] Fix PR51087 Extraneous enum record in DWARF with type units"
Causes invalid debug_gnu_pubnames (& I think non-gnu pubnames too) -
visible as 0 values for the offset in gnu pubnames. More details on the
original review in D115325.

This reverts commit 78d15a112c.
This reverts commit 54586582d3.
2021-12-23 20:50:30 -08:00
Kristina Bessonova 81378f7e56 Revert "[DwarfDebug] Support emitting function-local declaration for a lexical block" & dependent patches
Try to revert D113741 once again.

This also reverts 0ac75e82ff (D114705)
as it causes LLDB's lldb-api.lang/cpp/nsimport.TestCppNsImport.py test
failure w/o D113741.

This reverts commit f9607d45f3.

Differential Revision: https://reviews.llvm.org/D116225
2021-12-24 00:47:04 +02:00
Simon Pilgrim 71fc4bbdd2 [X86][SSE] Add ISD::ROTR support
Fix issue in TargetLowering::expandROT where we only attempt to flip a rotation if the other direction has better support - this matches TargetLowering::expandFunnelShift

This allows us to enable ISD::ROTR lowering on SSE targets, which particularly simplifies/improves codegen for splat amount and AVX2 per-element shifts.
2021-12-23 15:07:30 +00:00
Petar Avramovic 29f88b93fd [GlobalISel] Rework more/fewer elements for vectors
Artifact combiner is not able to access individual elements after using
LCMTy style merge/unmerge, extract and insert to change vector number of
elements (pad with undef or split to sub-vector instructions).
Use unmerge to individual elements instead and then merge elements into
requested types.
Change argument lowering for vectors and moreElementsVector to use
buildPadVectorWithUndefElements and buildDeleteTrailingVectorElements.
FewerElementsVector had a few helpers that had different behavior,
introduce new helper for most of the opcodes.
FewerElementsVector helper is more flexible since it can create leftover
instruction smaller then requested type (useful in case target wants to
avoid pad with undef and use fewer registers). If target does not want
leftover of different type it should call more elements first.
Some helpers were performing more elements first to have split without
leftover. Opcodes that used this helper use clampMaxNumElementsStrict
(does more elements first) in LegalizerInfo to avoid test changes.
Fixes failures caused by failing to combine artifacts created during
more/fewer elements vector.

Differential Revision: https://reviews.llvm.org/D114198
2021-12-23 14:30:02 +01:00
Muhammad Omair Javaid f9607d45f3 Revert "Revert "[DwarfDebug] Support emitting function-local declaration for a lexical block" & dependent patches"
This has broke following LLDB buildbots:

https://lab.llvm.org/buildbot/#/builders/17/builds/14984
https://lab.llvm.org/buildbot/#/builders/96/builds/15928
https://lab.llvm.org/buildbot/#/builders/68/builds/23600

This reverts commit 62a6b9e9ab.
2021-12-23 14:09:48 +05:00
Shivam Gupta 0489e89119 [DAGCombiner] Avoid combining adjacent stores at -O0 to improve debug experience
When the source has a series of assignments, users reasonably want to
have the debugger step through each one individually. Turn off the combine
for adjacent stores so we get this behavior at -O0.

Similar to D7181.

Reviewed By: spatel, xgupta

Differential Revision: https://reviews.llvm.org/D115808
2021-12-23 10:48:28 +05:30
David Blaikie 62a6b9e9ab Revert "[DwarfDebug] Support emitting function-local declaration for a lexical block" & dependent patches
This patch causes invalid DWARF to be generated in some cases of LTO +
Split DWARF - follow-up on the original review thread (D113741) contains
further detail and test cases.

This reverts commit 75b622a795.
This reverts commit b6ccca217c.
This reverts commit 514d374419.
2021-12-22 15:27:09 -08:00
Simon Pilgrim 4639461531 [DAG][X86] Add TargetLowering::isSplatValueForTargetNode override
Add callback to enable us to test target nodes if they are splat vectors

Added some basic X86ISD::VBROADCAST + X86ISD::VBROADCAST_LOAD handling
2021-12-22 16:57:44 +00:00
Alexandre Ganea a282ea4898 Reland - [CodeView] Emit S_OBJNAME record
Reland integrates build fixes & further review suggestions.

Thanks to @zturner for the initial S_OBJNAME patch!

Differential Revision: https://reviews.llvm.org/D43002
2021-12-21 19:02:14 -05:00
Alexandre Ganea 5bb5142e80 Revert [CodeView] Emit S_OBJNAME record
Also revert all subsequent fixes:
- abd1cbf5e5 [Clang] Disable debug-info-objname.cpp test on Unix until I sort out the issue.
- 00ec441253 [Clang] debug-info-objname.cpp test: explictly encode a x86 target when using %clang_cl to avoid falling back to a native CPU triple.
- cd407f6e52 [Clang] Fix build by restricting debug-info-objname.cpp test to x86.
2021-12-21 19:02:14 -05:00
Alexandre Ganea f44e3fbadd [CodeView] Emit S_OBJNAME record
Thanks to @zturner for the initial patch!

Differential Revision: https://reviews.llvm.org/D43002
2021-12-21 09:26:36 -05:00
Jay Foad 17006033f9 [GlobalISel] Verify operand types for G_SHL, G_LSHR, G_ASHR
Differential Revision: https://reviews.llvm.org/D115868
2021-12-21 11:59:33 +00:00
Simon Pilgrim 592e89e636 [DAG] Constify SelectionDAG::isSplatValue()
This doesn't generate any nodes so should be usable by methods with const SelectionDAG &.
2021-12-21 11:19:23 +00:00
Kazu Hirata 500c4b68dc [llvm] Construct SmallVector with iterator ranges (NFC) 2021-12-20 23:43:24 -08:00
Mircea Trofin 07622368a8 [NFC] Fix clang-tidy issues in CalcSpillWeights.cpp 2021-12-20 19:24:44 -08:00
Sami Tolvanen 5dc8aaac39 [llvm][IR] Add no_cfi constant
With Control-Flow Integrity (CFI), the LowerTypeTests pass replaces
function references with CFI jump table references, which is a problem
for low-level code that needs the address of the actual function body.

For example, in the Linux kernel, the code that sets up interrupt
handlers needs to take the address of the interrupt handler function
instead of the CFI jump table, as the jump table may not even be mapped
into memory when an interrupt is triggered.

This change adds the no_cfi constant type, which wraps function
references in a value that LowerTypeTestsModule::replaceCfiUses does not
replace.

Link: https://github.com/ClangBuiltLinux/linux/issues/1353

Reviewed By: nickdesaulniers, pcc

Differential Revision: https://reviews.llvm.org/D108478
2021-12-20 12:55:32 -08:00
Shivam Gupta eb66f0662a Revert "[DAGCombiner] Avoid combining adjacent stores at -O0 to improve debug experience"
This reverts commit 731bde1ed3.
2021-12-20 21:43:40 +05:30
Shivam Gupta 731bde1ed3 [DAGCombiner] Avoid combining adjacent stores at -O0 to improve debug experience
When the source has a series of assignments, users reasonably want to
have the debugger step through each one individually. Turn off the combine
for adjacent stores so we get this behavior at -O0.

Similar to D7181.

Differential Revision: https://reviews.llvm.org/D115808
2021-12-19 20:58:49 +05:30
Simon Pilgrim efec3a26b4 [DAG] visitADDSAT/visitSUBSAT - merge scalar/vector canonicalization and constant folding.
Match order of most of the other integer opcode combines
2021-12-19 13:19:40 +00:00
Simon Pilgrim c1340b9e78 [DAG] Improve FMINNUM/FMAXNUM/FMINIMUM/FMAXIMUM constant folding
Merge the node combines into a common DAGCombiner::visitFMinMax (like we do for IMINMAX).

Move the constant folding into SelectionDAG::foldConstantFPMath.

This allows us to fold the vecreduce-propagate-sd-flags.ll test as it reduces constants - so I've refactored it to take variables instead.

Differential Revision: https://reviews.llvm.org/D115952
2021-12-19 11:45:51 +00:00
Kazu Hirata fee57711fe Use DenseMap::lookup (NFC) 2021-12-17 18:19:25 -08:00
Kazu Hirata 26bd534a79 [llvm] Use none_of instead of \!any_of (NFC) 2021-12-17 13:48:57 -08:00
Sanjay Patel 79932211f9 [SDAG] remove FP-to-int cast attribute check in fold to FTRUNC
We were using a function attribute to indicate a non-standard FP mode,
but now we can use intrinsics for that job as shown in the new tests.
Presumably the x86 asm could be improved for that IR with intrinsics,
but I have not worked out exactly how to do that. Note that the
transform to FTRUNC still requires a hacky check for "nsz" (because
FMF are not applied to FP casts).

This is a cleanup based on the clang change in D115804 / 8c7f2a4f87 .
This is effectively a revert of 5a90285bd9 + D46237 .

Differential Revision: https://reviews.llvm.org/D115885
2021-12-17 16:01:37 -05:00
Kazu Hirata 90bd4873d6 [CodeGen] Fix an unused variable warning
This patch fixes:

  llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp:22617:11: error:
  unused variable 'Ops' [-Werror,-Wunused-variable]
2021-12-17 09:43:42 -08:00
Simon Pilgrim 35c7b1aeae [DAG] SimplifyVBinOp - remove FoldConstantArithmetic call.
Constant folding (scalar/vector) is now consistently handled before the SimplifyVBinOp calls.
2021-12-17 17:22:23 +00:00
Simon Pilgrim f602723bfa [DAG] Constant fold + canonicalize fp binops before SimplifyVBinOp call
Replace custom constant scalar/splat folding with FoldConstantArithmetic call and canonicalize commutative constant ops to the RHS before the SimplifyVBinOp call
2021-12-17 17:02:54 +00:00
Simon Pilgrim 9d2994311a [DAG] Move foldConstantFPMath() inside FoldConstantArithmetic
Further merging of integer and fp constant folding paths.

This allows us to handle undef vector arguments the same as scalar cases.
2021-12-17 16:06:41 +00:00
Simon Pilgrim 512ab9968d [DAG] foldConstantFPMath - fold vector splats as well as scalar constants 2021-12-17 15:19:26 +00:00
Simon Pilgrim 52611702ea Revert rG22dbc7a48bf7a3942a7e5ff57977ef828d240bd3 "[DAG] foldConstantFPMath - fold vector splats as well as scalar constants"
A followup patch uncovered an issue with allowing undef elements in the splat - I will reapply this with a fixed implementation.
2021-12-17 15:19:25 +00:00
David Truby 5c9684704d [DAG][sve] Lowering for VLS masked truncating stores
This extends the custom lowering for truncating stores on
fixed length vectors in SVE to support masked truncating stores.
It also adds a DAG combine for truncates followed by masked
stores.

Reviewed By: peterwaller-arm, paulwalker-arm

Differential Revision: https://reviews.llvm.org/D108115
2021-12-17 15:04:45 +00:00
Simon Pilgrim 22dbc7a48b [DAG] foldConstantFPMath - fold vector splats as well as scalar constants 2021-12-17 14:24:36 +00:00
Simon Pilgrim d91b5b0f57 [DAG] foldConstantFPMath - use APFloat& for read-only constant fold arg. NFC.
We just need to copy the 1st arg (which we use for the constant fold result) - use a cheaper const reference for the 2nd arg.
2021-12-17 12:34:03 +00:00
Simon Pilgrim 42f00106b7 [DAG] Constant fold + canonicalize integer binops before SimplifyVBinOp call
SimplifyVBinOp still has a FoldConstantArithmetic call, which now it isn't vector specific we should be able to remove (once fp binops are tidied up); but we can at least clean up the integer opcodes to perform the basic constant/undef handling in common code first.
2021-12-17 12:02:27 +00:00
OCHyams 78d15a112c [DWARF] Fix PR51087 Extraneous enum record in DWARF with type units
Fixes https://llvm.org/PR51087: Extraneous enum record in DWARF with type units.

As explained in PR51087 we sometimes get skeleton DIEs for enums in a Dwarf
Compile Unit (CU) that are not referenced from any CU and are already described
by a type unit.

Types for enums are emitted whether used or not, all together before most types
in the CU. Mechanically, the extraneous CU records are generated because the
enum types are generated with a call to CU->getOrCreateTypeDIE. This function
will recursively get-or-create the parent DIE (in the CU) and the type unit for
each. We don't need the CU-side DIEs if the type units are sucesfully
emitted. Fix by only emitting the type units for enums if possible, falling back
to a call to getOrCreateTypeDIE if not. Do the same for retained types.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D115325
2021-12-17 10:10:55 +00:00
Mircea Trofin 09103807e7 [NFC][regalloc] Introduce the RegAllocEvictionAdvisorAnalysis
This patch introduces the eviction analysis and the eviction advisor,
the default implementation, and the scaffolding for introducing the
other implementations of the advisor.

Differential Revision: https://reviews.llvm.org/D115707
2021-12-16 17:56:46 -08:00
Ellis Hoag 58d9c1aec8 [Try2][InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115693
2021-12-16 14:20:30 -08:00
Nathan Sidwell dd073e08ae Avoid by-value copies of referenced objects
These were detected by the new -Wauto-by-value-copy (D114989) warning, these by-value
constant copies need only be references.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D114990
2021-12-16 07:22:46 -08:00
Jay Foad cce93b3397 [MachineVerifier] Undef subreg operands do not require subranges
D112556 added verification that the live interval for a subreg operand
must have subranges. This patch fixes a corner case, where if all subreg
operands for a particular register are undef uses then no subranges
are required. This matches how LiveIntervalCalc would build the live
intervals in the first place, since an undef use is not considered
to read the register.

Before this patch, CodeGen/AMDGPU/no-remat-indirect-mov.mir would fail
with -early-live-intervals:

 # After Live Interval Analysis
...
*** Bad machine code: Live interval for subreg operand has no subranges ***
- function:    index_vgpr_waterfall_loop
- basic block: %bb.1  (0x6a9a968) [352B;496B)
- instruction: 432B	%24:vgpr_32 = V_MOV_B32_e32 undef %18.sub0:vreg_512, implicit $exec, implicit %18:vreg_512, implicit $m0
- operand 1:   undef %18.sub0:vreg_512

Differential Revision: https://reviews.llvm.org/D115360
2021-12-16 09:49:27 +00:00
Arthur Eubanks eba7b26815 [SafeStack] Use Align instead of uint64_t
It is better typed, and the calls to getAlignment() are deprecated.

Differential Revision: https://reviews.llvm.org/D115466
2021-12-15 14:40:56 -08:00
Arnold Schwaighofer d87e617048 Teach the backend to make references to swift_async_extendedFramePointerFlags weak if it emits it
When references to the symbol `swift_async_extendedFramePointerFlags`
are emitted they have to be weak.

References to the symbol `swift_async_extendedFramePointerFlags` get
emitted only by frame lowering code. Therefore, the backend needs to track
references to the symbol and mark them weak.

Differential Revision: https://reviews.llvm.org/D115672
2021-12-15 10:02:06 -08:00
Simon Pilgrim b88f4f271b [DAG] SelectionDAG::isSplatValue - add *_EXTEND_VECTOR_INREG handling
Fixes #52719
2021-12-15 12:26:39 +00:00
Esme-Yi c0529efc95 [DebugInfo][DWARF] emit DW_AT_accessibility attribute for class/struct/union types.
Summary:
	This patch emits the DW_AT_accessibility attribute for
class/struct/union types in the LLVM part.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D115606
2021-12-15 07:38:12 +00:00
John Brawn dc9f65be45 [AArch64][SVE] Fix handling of stack protection with SVE
Fix a couple of things that were causing stack protection to not work
correctly in functions that have scalable vectors on the stack:
 * Use TypeSize when determining if accesses to a variable are
   considered out-of-bounds so that the behaviour is correct for
   scalable vectors.
 * When stack protection is enabled move the stack protector location
   to the top of the SVE locals, so that any overflow in them (or the
   other locals which are below that) will be detected.

Fixes: https://github.com/llvm/llvm-project/issues/51137

Differential Revision: https://reviews.llvm.org/D111631
2021-12-14 11:30:48 +00:00
Chen Zheng 062d9b7d43 [LegalizeVectorOps] code refactor for LegalizeOp; NFC
Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D115636
2021-12-14 03:45:53 +00:00
Ellis Hoag c809da7d9c Revert "[InstrProf] Attach debug info to counters"
This reverts commit 800bf8ed29.

The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.

Differential Revision: https://reviews.llvm.org/D115689
2021-12-13 18:15:17 -08:00
Ellis Hoag 800bf8ed29 [InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D114565
2021-12-13 17:51:22 -08:00
Chen Zheng 8c107bee70 [LegalizeVectorOps] fix a typo
Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D115637
2021-12-14 00:22:58 +00:00
Fangrui Song a6a07a514b [MachineOutliner] Don't outline functions starting with PATCHABLE_FUNCTION_ENTER/FENTRL_CALL
MachineOutliner may outline a "patchable-function-entry" function whose body has
a TargetOpcode::PATCHABLE_FUNCTION_ENTER MachineInstr. This is incorrect because
the special code sequence must stay unchanged to be used at run-time.
Avoid outlining PATCHABLE_FUNCTION_ENTER. While here, avoid outlining FENTRY_CALL too
(which doesn't reproduce currently) to allow phase ordering flexibility.

Fixes #52635

Reviewed By: paquette

Differential Revision: https://reviews.llvm.org/D115614
2021-12-13 13:24:29 -08:00
Stanislav Mekhanoshin c4aef9c281 Check subrange liveness at rematerialization
LiveRangeEdit::allUsesAvailableAt checks that VNI at use is the same
as at the original use slot. However, the VNI can be the same while
a specific subrange needed for use can be dead at the new index.

This patch adds subrange liveness check if there is a subreg use.

Fixes: SWDEV-312810

Differential Revision: https://reviews.llvm.org/D115278
2021-12-13 11:11:55 -08:00
Mircea Trofin 657adcb077 [NFC][regalloc] Move ExtraRegInfo and related to LiveRangeStageManager
This would allow sharing the LiveRangeStageManager between different
RegAllocEvictionAdvisors. One scenario is for ML training, where we want
to capture what the default advisor would do, for bootstrapping (speeds
up training).

Differential Revision: https://reviews.llvm.org/D114831
2021-12-13 10:10:57 -08:00
Roman Lebedev c1a36ba002
[DAGCombine][X86][ARM] EXTRACT_SUBVECTOR(VECTOR_SHUFFLE(?,?,Mask)) -> VECTOR_SHUFFLE(EXTRACT_SUBVECTOR(?, ?), EXTRACT_SUBVECTOR(?, ?), Mask')
In most test changes this allows us to drop some broadcasts/shuffles.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D104156
2021-12-13 20:03:44 +03:00
Peter Waller 921e89c59a [SVE] Only combine (fneg (fma)) => FNMLA with nsz
-(Za + Zm * Zn) != (-Za + Zm * (-Zn))
when the FMA produces a zero output (e.g. all zero inputs can produce -0
output)

Add a PatFrag to check presence of nsz on the fneg, add tests which
ensure the combine does not fire in the absense of nsz.

See https://reviews.llvm.org/D90901 for a similar discussion on X86.

Differential Revision: https://reviews.llvm.org/D109525
2021-12-13 11:33:07 +00:00
Fraser Cormack b0319ab79b [PR52475] Ensure a correct chain in copies to/from hidden sret parameter
This patch fixes an issue during SelectionDAG construction. When the
target is unable to lower the function's return value, a hidden sret
parameter is created. It is initialized and copied to a stored variable
(DemoteRegister) with CopyToReg and is later fetched with
CopyFromReg. The bug is that the chains used for each copy are
inconsistent, and thus in rare cases the scheduler may issue them out of
order.

The fix is to ensure that the CopyFromReg uses the DAG root which is set
as the chain corresponding to the initial CopyToReg.

Fixes https://llvm.org/PR52475

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D114795
2021-12-13 10:46:32 +00:00
Kazu Hirata bb6447a78c [llvm] Use llvm::reverse (NFC) 2021-12-12 16:13:49 -08:00
Kazu Hirata 67aeae0138 [llvm] Use range-based for loops (NFC) 2021-12-11 22:34:07 -08:00
Adrian Prantl c7c84b9087 [DwarfDebug] Refuse to emit DW_OP_LLVM_arg values wider than 64 bits
DwarfExpression::addUnsignedConstant(const APInt &Value) only supports
wider-than-64-bit values when it is used to emit a top-level DWARF
expression representing the location of a variable. Before this change,
it was possible to call addUnsignedConstant on >64 bit values within a
subexpression when substituting DW_OP_LLVM_arg values.

This can trigger an assertion failure (e.g. PR52584, PR52333) when it
happens in a fragment (DW_OP_LLVM_fragment) expression, as
addUnsignedConstant on >64 bit values splits the constant into separate
DW_OP_pieces, which modifies DwarfExpression::OffsetInBits.

This change papers over the assertion errors by bailing on overly wide
DW_OP_LLVM_arg values. A more comprehensive fix might be to be to split
wide values into pointer-sized fragments.

[0] https://github.com/llvm/llvm-project/blob/e71fa03/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp#L799-L805

Patch by Ricky Zhou!

Differential Revision: https://reviews.llvm.org/D115343
2021-12-10 09:33:27 -08:00
David Sherwood 652faed353 [CodeGen] Improve SelectionDAGBuilder lowering code for get.active.lane.mask intrinsic
Previously we were using UADDO to generate a two-result value with
the unsigned addition and the overflow mask. We then combined the
overflow mask with the trip count comparison to get a result.
However, we don't need to do this - we can simply use a UADDSAT
saturating add node to add the vector index splat and the stepvector
together. Then we can just compare this to a splat of the trip count.
This results in overall better code quality for both Thumb2 and AArch64.

Differential Revision: https://reviews.llvm.org/D115354
2021-12-10 13:39:38 +00:00
Brian Cain 1e68c79987 Reapply [xray] add support for hexagon
Adds x-ray support for hexagon to llvm codegen, clang driver,
compiler-rt libs.

Differential Revision: https://reviews.llvm.org/D113638

Reapplying this after 543a9ad7c4,
which fixes the leak introduced there.
2021-12-10 05:32:28 -08:00
Sameer Sahasrabuddhe 1d0244aed7 Reapply CycleInfo: Introduce cycles as a generalization of loops
Reverts 02940d6d22. Fixes breakage in the modules build.

LLVM loops cannot represent irreducible structures in the CFG. This
change introduce the concept of cycles as a generalization of loops,
along with a CycleInfo analysis that discovers a nested
hierarchy of such cycles. This is based on Havlak (1997), Nesting of
Reducible and Irreducible Loops.

The cycle analysis is implemented as a generic template and then
instatiated for LLVM IR and Machine IR. The template relies on a new
GenericSSAContext template which must be specialized when used for
each IR.

This review is a restart of an older review request:
https://reviews.llvm.org/D83094

Original implementation by Nicolai Hähnle <nicolai.haehnle@amd.com>,
with recent refactoring by Sameer Sahasrabuddhe <sameer.sahasrabuddhe@amd.com>

Differential Revision: https://reviews.llvm.org/D112696
2021-12-10 14:36:43 +05:30
Konstantin Schwarz a344653725 [GlobalISel] Fix IRTranslator for constexpr fcmp
The existing code assumed fcmp to always be an Instruction, but it can also be a ConstExpr.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D115450
2021-12-10 08:49:12 +01:00
Kazu Hirata f829630d2e [llvm] Use llvm::count (NFC) 2021-12-09 20:50:38 -08:00
Arthur Eubanks 1172712f46 [NFC] Replace some deprecated getAlignment() calls with getAlign()
Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D115370
2021-12-09 08:43:19 -08:00
Brian Cain ab28cb1c5c Revert "[xray] add support for hexagon"
This reverts commit 543a9ad7c4.
2021-12-09 07:30:40 -08:00
Brian Cain 543a9ad7c4 [xray] add support for hexagon
Adds x-ray support for hexagon to llvm codegen, clang driver,
compiler-rt libs.

Differential Revision: https://reviews.llvm.org/D113638
2021-12-09 05:47:53 -08:00
Mircea Trofin 4afae6f7c7 [NFC] Rename MachineFunction::cloneMachineInstrBundle (coding style) 2021-12-08 21:12:54 -08:00
Mircea Trofin b012742405 [NFC] Rename MachineFunction::deleteMachineInstr (coding style) 2021-12-08 20:36:13 -08:00
Kazu Hirata c23ebf1714 [llvm] Use range-based for loops (NFC) 2021-12-08 20:35:39 -08:00
Mircea Trofin 91a0da0142 [NFC] Rename MachineFunction::DeleteMachineBasicBlock
Renamed to conform to coding style
2021-12-08 18:12:51 -08:00
David Sherwood 3257f63bbd [NFC][CodeGen] Remove rarely used DL variable from SelectionDAGBuilder
There is a pointer to the DataLayout in SelectionDAGBuilder called
'DL' that is hardly ever used. In most cases the code seems to just
use `DAG.getDataLayout()` instead. Given that DL is also often used
as a shadowed variable for the debug location it seems sensible to
just kill off the few remaining uses and be consistent with the rest
of the code.

Differential Revision: https://reviews.llvm.org/D114451
2021-12-08 17:05:46 +00:00
David Green 5d7efd4758 [SDAG] Refine MMO size when converting masked load/store to normal load/store
After D113888 / 32b6c17b29 the MMO size of a masked loads/store is
unknown. When we are converting back to a standard load/store because
the mask is known all ones, we can refine that to the correct size from
the size of the vector being loaded/stored.

Differential Revision: https://reviews.llvm.org/D114582
2021-12-08 10:13:25 +00:00
Alex Lorenz 0756aa3978 [macho] add support for emitting macho files with two build version load commands
This patch extends LLVM IR to add metadata that can be used to emit macho files with two build version load commands.
It utilizes "darwin.target_variant.triple" and "darwin.target_variant.SDK Version" metadata names for that,
which will be set by a future patch in clang.

MachO uses two build version load commands to represent an object file / binary that is targeting both the macOS target,
and the Mac Catalyst target. At runtime, a dynamic library that supports both targets can be loaded from either a native
macOS or a Mac Catalyst app on a macOS system. We want to add support to this to upstream to LLVM to be able to build
compiler-rt for both targets, to finish the complete support for the Mac Catalyst platform, which is right now targetable
by upstream clang, but the compiler-rt bits aren't supported because of the lack of this multiple build version support.

Differential Revision: https://reviews.llvm.org/D112189
2021-12-07 18:17:47 -08:00
Jonas Devlieghere 02940d6d22 Revert "CycleInfo: Introduce cycles as a generalization of loops"
This reverts commit 0fe61ecc2c because it
breaks the modules build.

https://green.lab.llvm.org/green/job/clang-stage2-rthinlto/4858/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/39112/
2021-12-07 13:06:34 -08:00
Chih-Ping Chen b5c42ef3da [NFC][CodeView] Use one unified access to the module in beginModule.
Differential Revision: https://reviews.llvm.org/D115257
2021-12-07 13:45:48 -05:00
Simon Pilgrim d298c32407 Remove unused variable. NFC. 2021-12-07 18:37:07 +00:00
Simon Pilgrim 52d2f35323 [DAG] Update expandFunnelShift/expandROT to return the expansion directly. NFCI.
Don't return a bool to indicate if the expansion was successful, just return the SDValue result directly, like we do for most other basic expansions.
2021-12-07 18:09:43 +00:00
Kazu Hirata 630c847b1b [llvm] Use range-based for loops (NFC) 2021-12-07 09:17:03 -08:00
Mircea Trofin fa99cb64ff [mlgo][regalloc] Add score calculation for training
Add the calculation of a score, which will be used during ML training. The
score qualifies the quality of a regalloc policy, and is independent of
what we train (currently, just eviction), or the regalloc algo itself.
We can then use scores to guide training (which happens offline), by
formulating a reward based on score variation - the goal being lowering
scores (currently, that reward is percentage reduction relative to
Greedy's heuristic)

Currently, we compute the score by factoring different instruction
counts (loads, stores, etc) with the machine basic block frequency,
regardless of the instructions' provenance - i.e. they could be due to
the regalloc policy or be introduced previously. This is different from
RAGreedy::reportStats, which accummulates the effects of the allocator
alone. We explored this alternative but found (at least currently) that
the more naive alternative introduced here produces better policies. We
do intend to consolidate the two, however, as we are actively
investigating improvements to our reward function, and will likely want
to re-explore scoring just the effects of the allocator.

In either case, we want to decouple score calculation from allocation
algorighm, as we currently evaluate it after a few more passes after
allocation (also, because score calculation should be reusable
regardless of allocation algorithm).

We intentionally accummulate counts independently because it facilitates
per-block reporting, which we found useful for debugging - for instance,
we can easily report the counts indepdently, and then cross-reference
with perf counter measurements.

Differential Revision: https://reviews.llvm.org/D115195
2021-12-07 09:00:27 -08:00
spupyrev f573f6866e ext-tsp basic block layout
A new basic block ordering improving existing MachineBlockPlacement.

The algorithm tries to find a layout of nodes (basic blocks) of a given CFG
optimizing jump locality and thus processor I-cache utilization. This is
achieved via increasing the number of fall-through jumps and co-locating
frequently executed nodes together. The name follows the underlying
optimization problem, Extended-TSP, which is a generalization of classical
(maximum) Traveling Salesmen Problem.

The algorithm is a greedy heuristic that works with chains (ordered lists)
of basic blocks. Initially all chains are isolated basic blocks. On every
iteration, we pick a pair of chains whose merging yields the biggest increase
in the ExtTSP value, which models how i-cache "friendly" a specific chain is.
A pair of chains giving the maximum gain is merged into a new chain. The
procedure stops when there is only one chain left, or when merging does not
increase ExtTSP. In the latter case, the remaining chains are sorted by
density in decreasing order.

An important aspect is the way two chains are merged. Unlike earlier
algorithms (e.g., based on the approach of Pettis-Hansen), two
chains, X and Y, are first split into three, X1, X2, and Y. Then we
consider all possible ways of gluing the three chains (e.g., X1YX2, X1X2Y,
X2X1Y, X2YX1, YX1X2, YX2X1) and choose the one producing the largest score.
This improves the quality of the final result (the search space is larger)
while keeping the implementation sufficiently fast.

Differential Revision: https://reviews.llvm.org/D113424
2021-12-07 07:31:10 -08:00
Paulo Matos 2fd634a5e3 [WebAssembly] Implement table instruction intrinsics
This change implements intrinsics for table.grow, table.fill,
table.size, and table.copy.

Differential Revision: https://reviews.llvm.org/D113420
2021-12-07 13:25:59 +01:00
Fraser Cormack 40d51de5cb [SelectionDAG] Use UnknownSize for VP memory ops
In the style of D113888, this patch updates the various VP memory
operations (load, store, gather, scatter) to use UnknownSize. This is
for the same reason as for masked loads and stores: the number of
elements accessed is not generally known at compile time.

This is somewhat pessimistic in the sense that we may still find
un-canonicalized intrinsics featuring both an all-true mask and an EVL
equal to the vector size. Arguably those should be canonicalized before
the SelectionDAG, so those have been left for future work.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D115036
2021-12-07 10:51:02 +00:00
Fraser Cormack 3460cc2585 [VP] Propagate align parameter attr on VP load/store to ISel
This patch fixes a case where the 'align' parameter attribute on the
pointer operands to llvm.vp.load and llvm.vp.store was being dropped
during the conversion to the SelectionDAG. The default alignment
equal to the ABI type alignment of the vector type was kept. It also
updates the documentation to reflect the fact that the parameter
attribute is now properly supported.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D114422
2021-12-07 10:16:16 +00:00
Sameer Sahasrabuddhe 0fe61ecc2c CycleInfo: Introduce cycles as a generalization of loops
LLVM loops cannot represent irreducible structures in the CFG. This
change introduce the concept of cycles as a generalization of loops,
along with a CycleInfo analysis that discovers a nested
hierarchy of such cycles. This is based on Havlak (1997), Nesting of
Reducible and Irreducible Loops.

The cycle analysis is implemented as a generic template and then
instatiated for LLVM IR and Machine IR. The template relies on a new
GenericSSAContext template which must be specialized when used for
each IR.

This review is a restart of an older review request:
https://reviews.llvm.org/D83094

Original implementation by Nicolai Hähnle <nicolai.haehnle@amd.com>,
with recent refactoring by Sameer Sahasrabuddhe <sameer.sahasrabuddhe@amd.com>

Differential Revision: https://reviews.llvm.org/D112696
2021-12-07 12:02:34 +05:30
Mircea Trofin 2bd7384d3a [NFC][MachineInstr] Pass-by-value DebugLoc in CreateMachineInstr
DebugLoc is cheap to move, passing it by-val rather than const ref to
take advantage of the fact that it is consumed that way by the
MachineInstr ctor, which creates some optimization oportunities.

Differential Revision: https://reviews.llvm.org/D115208
2021-12-06 19:42:19 -08:00
Kai Luo b206ee6906 [MachineVerifier] Make TiedOpsRewritten computable in MIRParser
This patch is to address post-commit comment https://reviews.llvm.org/D80538#anchor-inline-1091625, which make the constraint stronger based on what https://reviews.llvm.org/D80538 does, i.e., "TiedOpsRewritten is set iff leave-ssa and all tied operands share the same register".

Reviewed By: MatzeB

Differential Revision: https://reviews.llvm.org/D114573
2021-12-07 02:25:15 +00:00
Mircea Trofin 615e374252 [NFC][MachineInstr] Rename some vars to conform to coding style 2021-12-06 17:19:11 -08:00
Nico Weber 3678326d28 Revert "ext-tsp basic block layout"
This reverts commit c68f71eb37.

Breaks tests on arm hosts, see comments on https://reviews.llvm.org/D113424
2021-12-06 19:08:20 -05:00
James Nagurne cc3bb85580 [llvm][Hexagon] Generalize VLIWResourceModel, VLIWMachineScheduler, and ConvergingVLIWScheduler
The Pre-RA VLIWMachineScheduler used by Hexagon is a relatively generic
implementation that would make sense to use on other VLIW targets.

This commit lifts those classes into their own header/source file with the
root VLIWMachineScheduler. I chose this path rather than adding the
strategy et al. into MachineScheduler to avoid bloating the file with other
implementations.

Target-specific behaviors have been captured and replicated through
function overloads.

- Added an overloadable DFAPacketizer creation member function. This is
  mainly done for our downstream, which has the capability to override
  the DFAPacketizer with custom implementations. This is an upstreamable
  TODO on our end. Currently, it always returns the result of
  TargetInstrInfo::CreateTargetScheduleState
- Added an extra helper which returns the number of instructions in the
  current packet. This is used in our downstream, and may be useful
  elsewhere.
- Placed the priority heuristic values into the ConvergingVLIWscheduler
  class instead of defining them as local statics in the implementation
- Added a overridable helper in ConvergingVLIWScheduler so that targets
  can create their own VLIWResourceModel

Differential Revision: https://reviews.llvm.org/D113150
2021-12-06 16:23:48 -06:00
spupyrev c68f71eb37 ext-tsp basic block layout
A new basic block ordering improving existing MachineBlockPlacement.

The algorithm tries to find a layout of nodes (basic blocks) of a given CFG
optimizing jump locality and thus processor I-cache utilization. This is
achieved via increasing the number of fall-through jumps and co-locating
frequently executed nodes together. The name follows the underlying
optimization problem, Extended-TSP, which is a generalization of classical
(maximum) Traveling Salesmen Problem.

The algorithm is a greedy heuristic that works with chains (ordered lists)
of basic blocks. Initially all chains are isolated basic blocks. On every
iteration, we pick a pair of chains whose merging yields the biggest increase
in the ExtTSP value, which models how i-cache "friendly" a specific chain is.
A pair of chains giving the maximum gain is merged into a new chain. The
procedure stops when there is only one chain left, or when merging does not
increase ExtTSP. In the latter case, the remaining chains are sorted by
density in decreasing order.

An important aspect is the way two chains are merged. Unlike earlier
algorithms (e.g., based on the approach of Pettis-Hansen), two
chains, X and Y, are first split into three, X1, X2, and Y. Then we
consider all possible ways of gluing the three chains (e.g., X1YX2, X1X2Y,
X2X1Y, X2YX1, YX1X2, YX2X1) and choose the one producing the largest score.
This improves the quality of the final result (the search space is larger)
while keeping the implementation sufficiently fast.

Differential Revision: https://reviews.llvm.org/D113424
2021-12-06 08:56:39 -08:00
Kazu Hirata c4a8928b51 [CodeGen] Use range-based for loops (NFC) 2021-12-06 08:49:10 -08:00
Jack Andersen f108c7f59d [GlobalISel] Allow DBG_VALUE to use undefined vregs before LiveDebugValues.
Expanding on D109750.

Since `DBG_VALUE` instructions have final register validity determined in
`LDVImpl::handleDebugValue`, there is no apparent reason to immediately prune
unused register operands as their defs are erased. Consequently, this renders
`MachineInstr::eraseFromParentAndMarkDBGValuesForRemoval` moot; gaining a
substantial performance improvement.

The only necessary changes involve making relevant passes consider invalid
DBG_VALUE vregs uses as valid.

Reviewed By: MatzeB

Differential Revision: https://reviews.llvm.org/D112852
2021-12-05 15:55:59 -05:00
Michael Liao b6ccca217c Fix `-Wunused-variable` warning. NFC. 2021-12-05 13:40:35 -05:00
Kazu Hirata 1457e78352 [llvm] Use range-based for loops (NFC) 2021-12-05 08:33:02 -08:00
Kristina Bessonova 75b622a795 Reland [DwarfDebug] Support emitting function-local declaration for a lexical block
This is another attempt to make function-local declarations
(like static variables, structs/classes and other) be correctly
emitted within a lexical (bracketed) block.

Fixes https://bugs.llvm.org/show_bug.cgi?id=19238.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D113741
2021-12-05 13:56:45 +02:00
Kristina Bessonova 0ac75e82ff Reland [DwarfDebug] Move emission of global vars, types and imports to endModule()
This patch proposes to move emission of global variables, types,
imported entities, etc from DwarfDebug::beginModule() to DwarfDebug::endModule().
Effectively, this changes nothing but the order of debug entities which
will be as follows:
* subprograms (including related context, local variables/labels,
  local imported entities; related types can be created as a part of
  the emission of local entities of an abstract subprogram);
* global variables (including related context and types);
* retained types and enums;
* non-local-scoped imported entities;
* basic types;
* other types left (as a part of local variables attributes emission).

Note that the order of emitted compile units may also be changed as now we emit
units that contain subprograms first and then all other non-empty units.

The motivation behind this change is the following:
(1) DwarfDebug::beginModule() is run at the very beginning of backend's pipeline,
    from this time IR can be significantly changed by target-specific passes.
    If it happens for debug metadata of global entities, those changes will not
    be reflected in the emitted DWARF.
(2) imported subprogram names should refer to an abstract subprogram if it exists,
    but it isn't known in DwarfDebug::beginModule() (it's possible to make some
    guesses based on location info, but it's not quite reliable);
(3) aforementioned entities if they are scoped within a bracketed block
    (subject of D113741) couldn't be emitted in DwarfDebug::beginModule()
    (they need parent emitted first). Another problem is if to try to gather
    some information about local entities and defer their emission
    (till subprogram's processing or DwarfDebug::endModule()) all the gathered
    details might be irrelevant / invalid by the time the entities are being
    emitted (because of (1)).

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D114705
2021-12-05 13:56:45 +02:00
David Green 57ff805a6d [DAG] Create fptoui.sat from clamped fptosi
As an extension to D111976, this converts clamp fptosi, clamped between
0 and (2^n)-1 to a fptoui.sat. This can greatly help on targets with
conversions that naturally saturate, such as Arm.

X86 disables the transform as some of the test cases increases in size.
A fptoui.sat necessitates a fp clamp without native support, so there is
little use in converting if the instruction is just going to be
expanded.

Differential Revision: https://reviews.llvm.org/D112428
2021-12-05 09:25:52 +00:00
Kazu Hirata ca2f53897a [CodeGen] Use range-based for loops (NFC) 2021-12-04 08:48:05 -08:00