Clang has support of virtual file system for the purpose of testing, but
treatment of config files did not use it. This change enables VFS in it
as well.
Differential Revision: https://reviews.llvm.org/D132867
"Z" was so named when we had both gABI ELFCOMPRESS_ZLIB and the legacy .zdebug support.
Now we have just one zlib format, we should use the more descriptive name.
LLVM contains a helpful function for getting the size of a C-style
array: `llvm::array_lengthof`. This is useful prior to C++17, but not as
helpful for C++17 or later: `std::size` already has support for C-style
arrays.
Change call sites to use `std::size` instead.
Differential Revision: https://reviews.llvm.org/D133429
This patch introduces the priority analysis and the priority advisor,
the default implementation, and the scaffolding for introducing the
other implementations of the advisor.
Reviewed By: mtrofin
Differential Revision: https://reviews.llvm.org/D132835
as high-level API on top of `llvm::compression::{zlib,zstd}::*`:
* getReasonIfUnsupported: return nullptr if the specified format is
supported, or (if unsupported) a string like `LLVM was not built with LLVM_ENABLE_ZLIB ...`
* compress: dispatch to zlib::uncompress or zstd::uncompress
* decompress: dispatch to zlib::uncompress or zstd::uncompress
Move `llvm::DebugCompressionType` from MC to Support to avoid Support->MC cyclic
dependency. There are 40+ uses in llvm-project.
Add another enum class `llvm::compression::Format` to represent supported
compression formats, which may be a superset of ELF compression formats.
See D130458 (llvm-objcopy --{,de}compress-debug-sections for zstd) for a use
case.
Link: https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399
("[RFC] Zstandard as a second compression method to LLVM")
---
Note: this patch alone will cause -Wswitch to llvm/lib/ObjCopy/ELF/ELFObject.cpp
Reviewed By: ckissane, dblaikie
Differential Revision: https://reviews.llvm.org/D130506
as high-level API on top of `llvm::compression::{zlib,zstd}::*`:
* getReasonIfUnsupported: return nullptr if the specified format is
supported, or (if unsupported) a string like `LLVM was not built with LLVM_ENABLE_ZLIB ...`
* compress: dispatch to zlib::uncompress or zstd::uncompress
* decompress: dispatch to zlib::uncompress or zstd::uncompress
Move `llvm::DebugCompressionType` from MC to Support to avoid Support->MC cyclic
dependency. There are 40+ uses in llvm-project.
Add another enum class `llvm::compression::Format` to represent supported
compression formats, which may be a superset of ELF compression formats.
See D130458 (llvm-objcopy --{,de}compress-debug-sections for zstd) for a use
case.
Link: https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399
("[RFC] Zstandard as a second compression method to LLVM")
Differential Revision: https://reviews.llvm.org/D130506
In order to avoid the patch being too large, the atomicrmw xchg operation
on LA32 will be added later
Differential Revision: https://reviews.llvm.org/D131228
NewPM -filter-passes (D86360) uses ClassName instead of pass-name as used in
`-passes`, `-print-after`, etc. D87216 has added a mechanism to map
ClassName to pass-name. Adopt it for -filter-passes.
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D133263
Introduces the SanitizerBinaryMetadata instrumentation pass which uses
the new MD_pcsections metadata kinds to instrument certain types of
instructions and functions required for breakpoint-based sanitizers.
The first intended user of the binary metadata emitted will be a variant
of GWP-TSan [1]. GWP-TSan will require information about atomic
accesses; to unambiguously determine if an access is atomic or not, we
also require "covered" information which code has been compiled with
SanitizerBinaryMetadata instrumentation enabled.
[1] https://llvm.org/devmtg/2020-09/slides/Morehouse-GWP-Tsan.pdf
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D130887
This patch mostly reverts commit 70b37f4c03 which fixed PR50725.
In case of explicit consumption of multiple partially overlapping group
resources, the ResourceManager was not correctly checking pipeline
esources availability.
The fix for PR50725 only partially addressed a few instances of that issue.
This is a more general (although, technically slower) fix for that same issue.
It also fixes Issue #57548
Thanks to Haohai Wen for the small reproducible.
Interpret MD_pcsections in AsmPrinter emitting the requested metadata to
the associated sections. Functions and normal instructions are handled.
Differential Revision: https://reviews.llvm.org/D130879
Propagate (most) PC sections metadata to MachineInstr when GlobalISel is
doing instruction selection.
This change results in support for architectures using GlobalISel (such
as -O0 with AArch64). Not all instructions may be supported yet, and
requires further target-specific handling (such as done for AArch64
pseudo-atomics). Expanding supported instructions is planned on a
case-by-case basis and new use cases for PC sections metadata.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130886
Propagate PC sections metadata to MachineInstr when FastISel is doing
instruction selection.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130884
In many places DebugLoc and PCSections metadata are just copied along to
propagate them through MachineInstrs. Simplify doing so by bundling them
up in a MIMetadata class that replaces the DebugLoc argument to most
BuildMI() variants.
The DebugLoc-only constructors allow implicit construction, so that
existing usage of `BuildMI(.., DL, ..)` works as before, and the rest of
the codebase using BuildMI() does not require changes.
NFC.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130883
Add a new entry to SDNodeExtraInfo to propagate PCSections through
SelectionDAG.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130882
Document (in comments) the optional fourth "stride" argument to the
sequence operator, which was added in svn r157416.
Differential Revision: https://reviews.llvm.org/D133297
This change adds an assert to dyn_cast that the value passed-in is present. In the past, this relied on the isa_impl assertion (which still works in many cases) but which we can tighten up for a better QoI.
The PointerUnion change is because it seems like (based on the call sites) the semantics of the member dyn_cast are actually dyn_cast_if_present.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D133221
Add mapped_file_region::sync(), equivalent to POSIX msync,
synchronizing written content to disk without unmapping the region.
Asserts if the mode is not mapped_file_region::readwrite.
Note that I don't have access to a Windows machine, so I can't
easily run those unit tests.
Change by dexonsmith
Differential Revision: https://reviews.llvm.org/D95494
This patch is essentially an alternative to https://reviews.llvm.org/D75836 and was mentioned by @lhames in a comment.
The gist of the issue is that Mach-O has restrictions on which kind of sections are allowed after debug info has been emitted, which is also properly asserted within LLVM. Problem is that stack maps are currently emitted as one of the last sections in each target-specific AsmPrinter so far, which would cause the assertion to trigger. The current approach of special casing for the `__LLVM_STACKMAPS` section is not viable either, as downstream users can overwrite the stackmap format using plugins, which may want to use different sections.
This patch fixes the issue by emitting the stack map earlier, right before debug info is emitted. The way this is implemented is by taking the choice when to emit the StackMap away from the target AsmPrinter and doing so in the base class. The only disadvantage of this approach is that the `StackMaps` member is now part of the base class, even for targets that do not support them. This is functionaly not a problem however, as emitting an empty `StackMaps` is a no-op.
Differential Revision: https://reviews.llvm.org/D132708
OpenMP has a list of of optimistic attributes that can be attached to
known runtime functions to aid some analysis. The `omp_get_wtime`
function incorrectly used the `readonly` attribute. This is not correct
at the `omp_get_wtime` function changes values depending on some
external state. This is more correctly modeled with
`inaccessiblememonly` meaning that the value does not depend on anything
within the module, but can not be removes as it depends on external
state.
Fixes#57578
Reviewed By: tianshilei1992
Differential Revision: https://reviews.llvm.org/D133360
During SelectionDAG legalization SDNodes with associated extra info may
be replaced with a new SDNode. Preserve associated extra info on
ReplaceAllUsesWith and remove entries in DeallocateNode.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130881
For information infrequently attached to SDNodes, it is useful to
provide a way to add this information out-of-line. This is already done
for call-site specific information.
Rename CallSiteDbgInfo to NodeExtraInfo in preparation of adding
additional information not necessarily related to call sites only.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130880
This adds the ExpandLargeDivRem to the default pass pipeline.
The limit at which it expands div/rem instructions is configured
via a new TargetTransformInfo hook (default: no expansion)
X86, Arm and AArch64 backends implement this hook to expand div/rem
instructions with more than 128 bits.
Differential Revision: https://reviews.llvm.org/D130076
We currently extract offload binaries inside of the linker wrapper.
Other tools may wish to do the same extraction operation. This patch
simply factors out this handling into the `OffloadBinary.h` interface.
Reviewed By: yaxunl
Differential Revision: https://reviews.llvm.org/D132689
Provide MachineInstr::setPCSection(), to propagate relevant metadata
through the backend. Use ExtraInfo to store the metadata.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D130876
Part of patchset to add initial support for ARM64EC.
Per discussion on review, using the triple arm64ec-pc-windows-msvc. The
parsing works the same way as Apple's alternate Arm ABI "arm64e".
Differential Revision: https://reviews.llvm.org/D125412
GNU uses a different hashing function compared to the sys-V standard
function already provided in libObject. This is already used internally
in LLD for generating synthetic sections. This patch simply extracts
this definition and makes it availible to other users of `libObject`.
This is done in preparation for supporting symbol name lookups via the
GNU hash table.
Reviewed By: MaskRay, jhenderson
Differential Revision: https://reviews.llvm.org/D132696
CGP uses a raw `getInstructionCost(I, TargetTransformInfo::TCK_SizeAndLatency) >= TCC_Expensive` check to see if its better to move an expensive instruction used in a select behind a branch instead.
This is causing issues with upcoming improvements to TCK_SizeAndLatency costs on X86 as we need to use TCK_SizeAndLatency as an uop count (so its compatible with various target-specific buffer sizes - see D132288), but we can have instructions that have a low TCK_SizeAndLatency value but should still be treated as 'expensive' (FDIV for example) - by adding a isExpensiveToSpeculativelyExecute wrapper we can keep the current behaviour but still add an x86 override in a future patch when the cost tables are updated to compensate.
Current implementation of registerModuleReference() function not only
"registers" module reference, but also clones referenced module
(inside loadClangModule()). That may lead to cloning the module with
incorrect options (registerModuleReference() examines module references
and additionally accumulates MaxDwarfVersion and accel tables info).
Since accumulated options may differ from the current values,
it is incorrect to clone modules before options are fully accumulated.
This patch separates "cloning" code from "registering" code. So,
that accumulating option is done in the "registering stage" and
"cloning" is done after all modules are registered and options accumulated.
It also adds a callback for loaded compile units which can be used for
D132755 and D132371(to allow doing options accumulation outside
of DWARFLinker).
Differential Revision: https://reviews.llvm.org/D133047
LEGALPOS appears to only be used by LegalizeVectorOps. It needs
to point at a vector operand. Stores need to point at the second
operand since the result and the first operand are MVT::Other.
Reductions need to point at the second operand since the result
and the first operand are scalsrs.
Reviewed By: rogfer01
Differential Revision: https://reviews.llvm.org/D133048
I'm not sure how much to add to the description as we've tried to allow targets to interpret the TargetCostKind enums in their own way. But we need to make it clear that certain cost kinds need to match threshold numbers used by various passes (and vice-versa when passes are determining a cost-benefit threshold).
I'm not keen on the "The weighted sum of size and latency" description, but its very difficult to come up with anything else that's suitably generic (e.g. X86 will use uop counts here to easily work with LoopMicroOpBufferSize thresholds, even though high latency fdiv/fsqrt instructions still often have low uop counts).
Differential Revision: https://reviews.llvm.org/D132288
[MachineFunctionPass] Support -filter-passes for -print-changed
-filter-passes specifies a `PassID` (a lower-case dashed-separated pass name,
also used by -print-after, -stop-after, etc) instead of a CamelCasePass.
`-filter-passes=CamelCaseNewPMPass` seems like a workaround for new PM passes before
we can use lower-case dashed-separated pass names (as used by `-passes=`).
Example:
```
# getPassName() is "IRTranslator". PassID is "irtranslator"
llc -mtriple=aarch64 -print-changed -filter-passes=irtranslator < print-changed-machine.ll
```
Close https://github.com/llvm/llvm-project/issues/57453
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D133055
The patch adds the support of OpenCL and SPIR-V built-in types. It also
implements ExtInst selection and adds spv_unreachable and spv_alloca
intrinsics which improve the generation of the corresponding SPIR-V code.
Five LIT tests are included to demonstrate the improvement.
Differential Revision: https://reviews.llvm.org/D132648
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
Although we only currently have one error produced in this function I am
working on changes right now that add some more. This change makes the
error location more accurate.
Differential Revision: https://reviews.llvm.org/D133016
VP_STORE has a Chain for operand 0, so the LEGALPOS should be 1.
VP_STORE is always considered Legal for MVT::Other. So I suspect this
was causing vp_store to be ignored by LegalizeVectorOps and instead
handled in LegalizeDAG.
VP_LOAD is Custom expanded in LegalizeVectorOps for RISC-V.
Differential Revision: https://reviews.llvm.org/D132972
Printing support enables the production of more useful error messages in unit testing e.g. when using matchers such as `UnorderedElementsAre()` to inspect the contents of a `StringMap`.
Reviewed By: gribozavr2, sgatev, ymandel
Differential Revision: https://reviews.llvm.org/D132747
This adds support for backtrace generation to the llvm-symbolizer markup
filter, which is likely the largest use case.
Reviewed By: peter.smith
Differential Revision: https://reviews.llvm.org/D132706
This flag instructs dyld to make the segment read-only after fixups have
been performed.
I'm not sure why this flag is needed, as on macOS 13 beta at least,
__DATA_CONST is read-only even without this flag; but ld64 sets it as
well.
Differential Revision: https://reviews.llvm.org/D133010
This also modifies llvm-readobj to be more future-proof when printing
the guard FIDs table by calculating the entry size correctly according
to MS docs.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D132924
Transforms occasionally want to insert an instruction directly
after the definition point of a value. This involves quite a few
different edge cases, e.g. for phi nodes the next insertion point
is not the next instruction, and for invokes and callbrs its not
even in the same block. Additionally, the insertion point may not
exist at all if catchswitch is involved.
This adds a general Instruction::getInsertionPointAfterDef() API to
implement the necessary logic. For now it is used in two places
where this should be mostly NFC. I will follow up with additional
uses where this fixes specific bugs in the existing implementations.
Differential Revision: https://reviews.llvm.org/D129660
This section stores 32-bit `__TEXT` segment offsets of initializer
functions, and is used instead of `__mod_init_func` when chained fixups
are enabled.
Storing the offsets lets us avoid emitting fixups for the initializers.
Differential Revision: https://reviews.llvm.org/D132947
Also, some local variables were renamed in accordance with the code
style as well as `std::tie` occurrences and `.first`/`.second` member
uses were replaced with structure bindings.
Differential Revision: https://reviews.llvm.org/D132806
Current implementation promotes a non-cold function in the SampleFDO profile
into a hot function in the FDO profile. This is too aggressive. This patch
promotes a hot functions in the SampleFDO profile into a hot function, and a
warm function in SampleFDO into a warm function in FDO.
Differential Revision: https://reviews.llvm.org/D132601
This patch has the following changes:
(1) Handling of internal linkage functions (static functions)
Static functions in FDO have a prefix of source file name, while they do not
have one in SampleFDO. Current implementation does not handle this and we are
not updating the profile for static functions. This patch fixes this.
(2) Handling of -funique-internal-linakge-symbols
Again this is for the internal linkage functions. Option
-funique-internal-linakge-symbols can now be applied to both FDO and SampleFDO
compilation. When it is used, it demangles internal linkage function names and
adds a hash value as the postfix.
When both SampleFDO and FDO profiles use this option, or both
not use this option, changes in (1) should handle this.
Here we also handle when the SampleFDO profile using this option while FDO
profile not using this option, or vice versa.
There is one case where this patch won't work: If one of the profiles used
mangled name and the other does not. For example, if the SampleFDO profile
uses clang c-compiler and without -funique-internal-linakge-symbols, while
the FDO profile uses -funique-internal-linakge-symbols. The SampleFDO profile
contains unmangled names while the FDO profile contains mangled names. If
both profiles use c++ compiler, this won't happen. We think this use case
is rare and does not justify the effort to fix.
Differential Revision: https://reviews.llvm.org/D132600
Mostly just modeled after vp.fneg except there is a
"functional instruction" for fneg while fabs is always an
intrinsic.
Reviewed By: fakepaper56
Differential Revision: https://reviews.llvm.org/D132793
This option outputs the location, encoded value and target of chained
fixups, using the same format as `otool -dyld_info`.
This initial implementation only supports the DYLD_CHAINED_PTR_64 and
DYLD_CHAINED_PTR_64_OFFSET pointer encodings, which are used in x86_64
and arm64 userspace binaries.
When Apple's effort to upstream their chained fixups code continues,
we'll replace this code with the then-upstreamed code. But we need
something in the meantime for testing ld64.lld's chained fixups code.
Differential Revision: https://reviews.llvm.org/D132036
MapperJITLinkMemoryManager has slab allocation. Combined with
InProcessMapper, it can replace InProcessMemoryManager.
It can also replace JITLinkSlabAllocator through the InProcessDeltaMapper
that adds an offset to the executor addresses for use in tests.
Differential Revision: https://reviews.llvm.org/D132315
Reapplies f14cb494a3 (which was reverted in 2f08f8426c) with a fix for UB in
the ExecutorAddr::Unwrap::Unwrap constructor (which caused failures on some
bots).
The simpler diff-checks require pointers with add-recs from the same
innermost loop, but this property wasn't check completely. Add the
missing check to ensure both addrecs are in the innermost loop.
Fixes#57315.
The wrap/unwrap operations are applied to pointers after/before conversion to/from
raw addresses. They can be used to tag, untag, sign, or strip signing from
pointers. They currently default to 'rawPtr' (identity) on all platforms, but it
is expected that the default will be set based on the host architecture, e.g.
they would default to signing/stripping for arm64e.
Adds a pass ExpandLargeDivRem to expand div/rem instructions
with more than 128 bits into a loop computing that value.
As discussed on https://reviews.llvm.org/D120327, this approach has the advantage
that it is independent of the runtime library. This also helps the clang driver,
which otherwise would need to understand enough about the runtime library
to know whether to allow _BitInts with more than 128 bits.
Targets are still free to disable this pass and instead provide a faster
implementation in a runtime library.
Fixes https://github.com/llvm/llvm-project/issues/44994
Differential Revision: https://reviews.llvm.org/D126644
While this does not matter for most targets, when building for Arm Morello,
we have to mark the symbol as a function and add size information, so that
LLD can correctly evaluate relocations against the local symbol.
Since Morello is an out-of-tree target, I tried to reproduce this with
in-tree backends and with the previous reviews applied this results in
a noticeable difference when targeting Thumb.
Background: Morello uses a method similar Thumb where the encoding mode is
specified in the LSB of the symbol. If we don't mark the target as a
function, the relocation will not have the LSB set and calls will end up
using the wrong encoding mode (which will almost certainly crash).
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D131429
Follow the pattern used in MLIR for the cl::opt instances.
v2:
- make DebugCounter::isCountingEnabled public so that the
DebugCounterOwner doesn't have to be a nested class. This simplifies
later changes
v3:
- remove the indirection via DebugCounterOwner::instance()
Differential Revision: https://reviews.llvm.org/D129116
`llvm::identity` is similar to `std::identity` from C++20, but one surprising
thing is that `llvm::identity` is not a transparent function object. Add the
`is_transparent` type alias to denote it can be used as a transparent function
object.
Differential Revision: https://reviews.llvm.org/D132628
This reverts commit 51d82502d9.
There is a regression in the flang-aarch64-dylib buildbot which is most
likely caused by this change. Reverting until I can investigate.
It was observed in D129117 that the subtle dependency between statistic
and timer code is not entirely robust: the global destructor
~StatisticInfo indirectly calls CreateInfoOutputFile, which requires
the LibSupportInfoOutputFilename to not have been destructed.
By constructing LibSupportInfoOutputFilename before the StatisticInfo
object, the order of destruction is guaranteed.
Differential Revision: https://reviews.llvm.org/D131059
Follow the pattern used in MLIR for the cl::opt instances.
v2:
- make DebugCounter::isCountingEnabled public so that the
DebugCounterOwner doesn't have to be a nested class. This simplifies
later changes
Differential Revision: https://reviews.llvm.org/D129116
Match Clang's sorting, so that longer (more specific) prefix paths will match
before less specific paths.
Reviewed By: MaskRay, raj.khem, #debug-info
Differential Revision: https://reviews.llvm.org/D132390
The goal is to separate collecting items for post-processing
and processing them. Post processing also outlined as
dedicated method.
Differential Revision: https://reviews.llvm.org/D132603
TLite is a lightweight, statically linkable[1], model evaluator, supporting a
subset of what the full tensorflow library does, sufficient for the
types of scenarios we envision having. It is also faster.
We still use saved models as "source of truth" - 'release' mode's AOT
starts from a saved model; and the ML training side operates in terms of
saved models.
Using TFLite solves the following problems compared to using the full TF
C API:
- a compiler-friendly implementation for runtime-loadable (as opposed
to AOT-embedded) models: it's statically linked; it can be built via
cmake;
- solves an issue we had when building the compiler with both AOT and
full TF C API support, whereby, due to a packaging issue on the TF
side, we needed to have the pip package and the TF C API library at
the same version. We have no such constraints now.
The main liability is it supporting a subset of what the full TF
framework does. We do not expect that to cause an issue, but should that
be the case, we can always revert back to using the full framework
(after also figuring out a way to address the problems that motivated
the move to TFLite).
Details:
This change switches the development mode to TFLite. Models are still
expected to be placed in a directory - i.e. the parameters to clang
don't change; what changes is the directory content: we still need
an `output_spec.json` file; but instead of the saved_model protobuf and
the `variables` directory, we now just have one file, `model.tflite`.
The change includes a utility showing how to take a saved model and
convert it to TFLite, which it uses for testing.
The full TF implementation can still be built (not side-by-side). We
intend to remove it shortly, after patching downstream dependencies. The
build behavior, however, prioritizes TFLite - i.e. trying to enable both
full TF C API and TFLite will just pick TFLite.
[1] thanks to @petrhosek's changes to TFLite's cmake support and its deps!
The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type identifier for each
function and injects verification code before indirect calls.
Unlike the current CFI schemes implemented in LLVM, KCFI does not
require LTO, does not alter function references to point to a jump
table, and never breaks function address equality. KCFI is intended
to be used in low-level code, such as operating system kernels,
where the existing schemes can cause undue complications because
of the aforementioned properties. However, unlike the existing
schemes, KCFI is limited to validating only function pointers and is
not compatible with executable-only memory.
KCFI does not provide runtime support, but always traps when a
type mismatch is encountered. Users of the scheme are expected
to handle the trap. With `-fsanitize=kcfi`, Clang emits a `kcfi`
operand bundle to indirect calls, and LLVM lowers this to a
known architecture-specific sequence of instructions for each
callsite to make runtime patching easier for users who require this
functionality.
A KCFI type identifier is a 32-bit constant produced by taking the
lower half of xxHash64 from a C++ mangled typename. If a program
contains indirect calls to assembly functions, they must be
manually annotated with the expected type identifiers to prevent
errors. To make this easier, Clang generates a weak SHN_ABS
`__kcfi_typeid_<function>` symbol for each address-taken function
declaration, which can be used to annotate functions in assembly
as long as at least one C translation unit linked into the program
takes the function address. For example on AArch64, we might have
the following code:
```
.c:
int f(void);
int (*p)(void) = f;
p();
.s:
.4byte __kcfi_typeid_f
.global f
f:
...
```
Note that X86 uses a different preamble format for compatibility
with Linux kernel tooling. See the comments in
`X86AsmPrinter::emitKCFITypeId` for details.
As users of KCFI may need to locate trap locations for binary
validation and error handling, LLVM can additionally emit the
locations of traps to a `.kcfi_traps` section.
Similarly to other sanitizers, KCFI checking can be disabled for a
function with a `no_sanitize("kcfi")` function attribute.
Relands 67504c9549 with a fix for
32-bit builds.
Reviewed By: nickdesaulniers, kees, joaomoreira, MaskRay
Differential Revision: https://reviews.llvm.org/D119296
The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type identifier for each
function and injects verification code before indirect calls.
Unlike the current CFI schemes implemented in LLVM, KCFI does not
require LTO, does not alter function references to point to a jump
table, and never breaks function address equality. KCFI is intended
to be used in low-level code, such as operating system kernels,
where the existing schemes can cause undue complications because
of the aforementioned properties. However, unlike the existing
schemes, KCFI is limited to validating only function pointers and is
not compatible with executable-only memory.
KCFI does not provide runtime support, but always traps when a
type mismatch is encountered. Users of the scheme are expected
to handle the trap. With `-fsanitize=kcfi`, Clang emits a `kcfi`
operand bundle to indirect calls, and LLVM lowers this to a
known architecture-specific sequence of instructions for each
callsite to make runtime patching easier for users who require this
functionality.
A KCFI type identifier is a 32-bit constant produced by taking the
lower half of xxHash64 from a C++ mangled typename. If a program
contains indirect calls to assembly functions, they must be
manually annotated with the expected type identifiers to prevent
errors. To make this easier, Clang generates a weak SHN_ABS
`__kcfi_typeid_<function>` symbol for each address-taken function
declaration, which can be used to annotate functions in assembly
as long as at least one C translation unit linked into the program
takes the function address. For example on AArch64, we might have
the following code:
```
.c:
int f(void);
int (*p)(void) = f;
p();
.s:
.4byte __kcfi_typeid_f
.global f
f:
...
```
Note that X86 uses a different preamble format for compatibility
with Linux kernel tooling. See the comments in
`X86AsmPrinter::emitKCFITypeId` for details.
As users of KCFI may need to locate trap locations for binary
validation and error handling, LLVM can additionally emit the
locations of traps to a `.kcfi_traps` section.
Similarly to other sanitizers, KCFI checking can be disabled for a
function with a `no_sanitize("kcfi")` function attribute.
Reviewed By: nickdesaulniers, kees, joaomoreira, MaskRay
Differential Revision: https://reviews.llvm.org/D119296
This commit adds definitions for the `dyld_chained_import*` structs.
The imports array is now printed with `llvm-otool -chained_fixups`. This
completes this option's implementation.
A slight difference from cctools otool is that we don't yet dump the
raw bytes of the imports entries.
When Apple's effort to upstream their chained fixups code continues,
we'll replace this code with the then-upstreamed code. But we need
something in the meantime for testing ld64.lld's chained fixups code.
Differential Revision: https://reviews.llvm.org/D131982
The diff modifies ext-tsp code layout algorithm in the following ways:
(i) fixes merging of cold block chains (this is a port of D129397);
(ii) adjusts the cost model utilized for optimization;
(iii) adjusts some APIs so that the implementation can be used in BOLT; this is
a prerequisite for D129895.
The only non-trivial change is (ii). Here we introduce different weights for
conditional and unconditional branches in the cost model. Based on the new model
it is slightly more important to increase the number of "fall-through
unconditional" jumps, which makes sense, as placing two blocks with an
unconditional jump next to each other reduces the number of jump instructions in
the generated code. Experimentally, this makes a mild impact on the performance;
I've seen up to 0.2%-0.3% perf win on some benchmarks.
Reviewed By: hoy
Differential Revision: https://reviews.llvm.org/D129893
We currently process one OutputSection at a time and for each OutputSection
write contained input sections in parallel. This strategy does not leverage
multi-threading well. Instead, parallelize writes of different OutputSections.
The default TaskSize for parallelFor often leads to inferior sharding. We
prepare the task in the caller instead.
* Move llvm::parallel::detail::TaskGroup to llvm::parallel::TaskGroup
* Add llvm::parallel::TaskGroup::execute.
* Change writeSections to declare TaskGroup and pass it to writeTo.
Speed-up with --threads=8:
* clang -DCMAKE_BUILD_TYPE=Release: 1.11x as fast
* clang -DCMAKE_BUILD_TYPE=Debug: 1.10x as fast
* chrome -DCMAKE_BUILD_TYPE=Release: 1.04x as fast
* scylladb build/release: 1.09x as fast
On M1, many benchmarks are a small fraction of a percentage faster. Mozilla showed the largest difference with the patch being about 1.03x as fast.
Differential Revision: https://reviews.llvm.org/D131247
Teach LLVM about filesets. Filesets were added in macOS 11 (Big Sur) to
combine multiple Mach-O files. They introduce a new load command
(LC_FILESET_ENTRY) consisting of a fileset_entry_command.
struct fileset_entry_command {
uint32_t cmd; /* LC_FILESET_ENTRY */
uint32_t cmdsize; /* includes entry_id string */
uint64_t vmaddr; /* memory address of the entry */
uint64_t fileoff; /* file offset of the entry */
union lc_str entry_id; /* contained entry id */
uint32_t reserved; /* reserved */
};
This patch teaches LLVM about the new load command and the corresponding
data.
Differential revision: https://reviews.llvm.org/D132432
This patch adds a Type operand to the TLI isCheapToSpeculateCttz/isCheapToSpeculateCtlz callbacks, allowing targets to decide whether branches should occur on a type-by-type/legality basis.
For X86, this patch proposes to allow CTTZ speculation for i8/i16 types that will lower to promoted i32 BSF instructions by masking the operand above the msb (we already do something similar for i8/i16 TZCNT). This required a minor tweak to CTTZ lowering - if the src operand is known never zero (i.e. due to the promotion masking) we can remove the CMOV zero src handling.
Although BSF isn't very fast, most CPUs from the last 20 years don't do that bad a job with it, although there are some annoying passthrough EFLAGS dependencies. Additionally, now that we emit 'REP BSF' in most cases, we are tending towards assuming this will most likely be executed as a TZCNT instruction on any semi-modern CPU.
Differential Revision: https://reviews.llvm.org/D132520
Allows things like `(G_PTR_ADD (G_PTR_ADD a, b), c)` to be
simplified into a single ADD3 instruction instead of two adds.
Reviewed By: foad
Differential Revision: https://reviews.llvm.org/D131254
This commit moves the information on whether a register is constant into
the Tablegen files to allow generating the implementaiton of
isConstantPhysReg(). I've marked isConstantPhysReg() as final in this
generated file to ensure that changes are made to tablegen instead of
overriding this function, but if that turns out to be too restrictive,
we can remove the qualifier.
This should be pretty much NFC, but I did notice that e.g. the AMDGPU
generated file also includes the LO16/HI16 registers now.
The new isConstant flag will also be used by D131958 to ensure that
constant registers are marked as call-preserved.
Differential Revision: https://reviews.llvm.org/D131962
The existing code resulted in the max size and access counts being equal
to the min. Compute the max instead (max lifetime was already correct).
Differential Revision: https://reviews.llvm.org/D132515
Enables fixed sized vectors to detect SK_Splice shuffle patterns and provides basic X86 cost support
Differential Revision: https://reviews.llvm.org/D132374
This has the effect of exposing the power-of-two property for use in memory op costing, but no target actually uses it yet. The main point of this change is simple consistency with the recently changes getArithmeticInstrCost, and to remove the last (interface) use of OperandValueKind.
This patch adds the last of the changes required to enable
DBG_VALUE_LIST handling in InstrRefLDV, handling variadic debug values
during the transfer tracking step. Most of the changes are fairly
straightforward, and based around tracking multiple locations per
variable in TransferTracker::VLocTracker.
Differential Revision: https://reviews.llvm.org/D128211
Callsites could be marked as `builtin` while calling `nobuiltin`
functions. This can lead to problems, if local optimizations apply
transformations based on the semantics of the builtin, but then IPO
treats the function as `nobuiltin` and applies a transform that breaks
builtin semantics (assumed earlier).
To avoid this, mark such functions as maybey-derefined, to avoid IPO
transforms on them that may break assumptions of earlier calls.
Fixes#57075Fixes#48366
Reviewed By: ychen
Differential Revision: https://reviews.llvm.org/D97735
This patch proposes to move the llvm::detail::PopulationCounter internal helpers into ADT/bit.h and provide a llvm::popcount implementation.
I've left the countPopulation implementation in place in MathExtras.h for now, but updated it to use llvm::popcount.
Hopefully I've got the type_traits correct - I don't use them very often.
Someday we'll move to C++20 with an actual <bit> std header, and we already have this header in place to simplify matters. We'd probably benefit from moving the other <bit> helpers here at some point, but this is a first step.
Differential Revision: https://reviews.llvm.org/D132407
`llvm::all_equal` checks if all values in the given range are equal, i.e., there are no two elements that are not equal.
Similar to `llvm::all_of`, it returns `true` when the range is empty.
`llvm::all_equal` is intended to supersede `llvm::is_splat`, which will be deprecated and removed in future patches.
See the discussion thread for more details:
https://discourse.llvm.org/t/adt-is-splat-and-empty-ranges/64692.
Reviewed By: dblaikie, shchenz
Differential Revision: https://reviews.llvm.org/D132334
This change completes the process of replacing OperandValueKind and OperandValueProperties which were previously passed independently in this API with a single container class which contains both.
This is the change which motivated the whole sequence which preceeded it. In an original spike version of this change, I'd noticed a nasty bug: I'd changed the signature without changing names, and as result, we silently passed additional information through a callsite which previously dropped the power-of-two fact. This might be harmless in most cases, but at least a couple clearly dependend for correctness on not passing that property through.
I did my best to split off prior changes which reduced the scope of this one, and which made it possible to use compiler assistance. For instance, every parameter which changes type in this change also changes name. This was intentional to make sure that every call site possible effected must show up in the diff. This let me audit each one closely.
This is part of an ongoing transition to use OperandValueInfo which combines OperandValueKind and OperandValueProperties. This change adds some accessor methods and uses them to simplify backend code. The primary motivation of doing so is removing uses of the parameters so that an upcoming api change is less error prone.
This interface allows a target to reject a proposed
SMS schedule. For Hexagon/PowerPC, all schedules
are accepted, leaving behavior unchanged. For ARM,
schedules which exceed register pressure limits are
rejected.
Also, two RegisterPressureTracker methods now need to be public so
that register pressure can be computed by more callers.
Reapplication of D128941/(reversion:D132037) with small fix.
Differential Revision: https://reviews.llvm.org/D132170
This removes the last use of OperandValueKind from the client side API, and (once this is fully plumbed through TTI implementation) allow use of the same properties in store costing as arithmetic costing.
This completes the client side transition to the OperandValueInfo version of this routine. Backend TTI implementations still use the prior versions for now.
OperandValueKind and OperandValueProperties both provide facts about the operands of an instruction for purposes of cost modeling. We've discussed merging them several times; before I plumb through more flags, let's go ahead and do so.
This change only adds the client side interface for getArithmeticInstrCost and makes a couple of minor changes in client code to prove that it works. Target TTI implementations still use the split flags. I'm deliberately splitting what could be one big change into a series of smaller ones so that I can lean on the compiler to catch errors along the way.
`RegisterClassInfo` caches information like allocation orders and reuses
it for multiple machine functions where possible. However the `MCPhysReg
*CalleeSavedRegs` field used to test whether the set of callee saved
registers changed did not work: After D28566
`MachineRegisterInfo::getCalleeSavedRegs()` can return dynamically
computed CSR sets that are only valid while the `MachineRegisterInfo`
object of the current function exists.
This changes the code to make a copy of the CSR list instead of keeping
a possibly invalid pointer around.
Differential Revision: https://reviews.llvm.org/D132080
Check ScopedPrinter pointer before attempting to print the attribute's
parsed information.
Patch by Michael Platings and Victor Campos
Reviewed By: pratlucas
Differential Revision: https://reviews.llvm.org/D132214
Initial implementation had too weak requirements to positive/negative
range crossings. Not crossing zero with nuw is not enough for two reasons:
- If ArLHS has negative step, it may turn from positive to negative
without crossing 0 boundary from left to right (and crossing right to
left doesn't count for unsigned);
- If ArLHS crosses SINT_MAX boundary, it still turns from positive to
negative;
In fact we require that ArLHS always stays non-negative or negative,
which an be enforced by the following set of preconditions:
- both nuw and nsw;
- positive step (looks liftable);
Because of positive step, boundary crossing is only possible from left
part to the right part. And because of no-wrap flags, it is guaranteed
to never happen.
When reviewing https://reviews.llvm.org/D132330, I noticed a few pre-existing
comments regarding the implementation of `llvm::bit_cast`. One comment is a bit
misleading since `std::bit_cast` is a C++20 standard library thing, not a
C++17 one (otherwise we could use it directly). Clarify that in the comment.
Differential Revision: https://reviews.llvm.org/D132332
This patch removes macro tricks to check GCC versions.
The commit message from 19262fc596
states that "is_trivially_copyable is only in GCC 5.1 and later".
Note that we now require GCC 7.1 or higher.
Since both std::is_trivially_constructible and
std::is_trivially_copyable are C++11 features, and we now require
C++17, we probably don't need to worry about the availability of the
C++11 features.
Differential Revision: https://reviews.llvm.org/D132330
This patch removes llvm::is_trivially_copyable as it seems to be dead.
Once I remove it, HAVE_STD_IS_TRIVIALLY_COPYABLE has no users, so this
patch removes the macro also.
The comment on llvm::is_trivially_copyable mentions GCC 4.9, but note
that we now require GCC 7.1 or higher.
Differential Revision: https://reviews.llvm.org/D132328
This patch adds support for the device clause on `Target` directive.
Device clause was added in OpenMP specification version 4.5 to
create a device data environment for the extent of a region. On
target construct, the device expression be either be `ancestor`
(taking after the parent) or assign a new `device_num`.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D126441
Use `std::void_t` instead of defining our own equivalent in `STLExtras.h` now
that C++17 is available for use.
Differential Revision: https://reviews.llvm.org/D132319
Defaults to TCK_RecipThroughput - as most explicit calls were assuming TCK_RecipThroughput (vectorizers) or was just doing a before-vs-after comparison (vectorcombiner). Calls via getInstructionCost were just dropping the CostKind, so again there should be no change at this time (as getShuffleCost and its expansions don't use CostKind yet) - but it will make it easier for us to better account for size/latency shuffle costs in inline/unroll passes in the future.
Differential Revision: https://reviews.llvm.org/D132287
This patch removes llvm::is_trivially_{copy/move}_constructible in
favor of std::is_trivially_{copy/move}_constructible.
The previous attempt to remove them in Dec 2020,
c8d406c93c, broke builds with "some
versions of GCC" according to
6cd9608fb3.
It's been 20 months since then, and the minimum requirement for GCC
has been updated to 7.1 from 5.1.
FWIW, I was able to build llvm with gcc 8.4.0.
Differential Revision: https://reviews.llvm.org/D132311
makeAbsolute was not updating the symbol address when applied to external
symbols.
This commit adds a unit test for makeAbsolute, and updates the makeExternal unit
test to check that makeExternal works correctly for absolute symbols.
This patch deprecates Any::hasValue as I've migrated all known uses of
it to Any::has_value. I'm planning to remove the deprecated method in
3 months or so.
Differential Revision: https://reviews.llvm.org/D132304
This patch implements Any::has_value for consistency with std::any in
C++17.
My plan is to deprecate Any::hasValue after migrating all of its uses
to Any::has_value. Since I am about to do so, this patch simply
replaces hasValue with has_value in the unit test instead of adding
tests for has_value.
Differential Revision: https://reviews.llvm.org/D132278
Adds a builtin that serves as an optimization hint to apply specific optimized
DAG mutations during scheduling. This also disables any other mutations or
clustering that may interfere with the desired pipeline. The first optimization
strategy that is added here is designed to improve the performance of small gemm
kernels on gfx90a.
Reviewed By: jrbyrnes
Differential Revision: https://reviews.llvm.org/D132079
SLP vectorizer tries to find the reductions starting the operands of the
instructions with no-users/void returns/etc. But such operands can be
postponable instructions, like Cmp, InsertElement or InsertValue. Such
operands still must be postponed, vectorizer should not try to vectorize
them immediately.
Differential Revision: https://reviews.llvm.org/D131965
The current llvm.prefetch intrinsic docs state "The rw, locality and
cache type arguments must be constant integers."
This change:
- Makes arg 3 (cache type) an ImmArg
- Improves the verifier error messages to reference the incorrect
argument.
- Fixes two tests which contradict the docs.
This is needed as the lowering to GlobalISel is different for ImmArgs
compared to other constants. The non-ImmArgs create a G_CONSTANT MIR
instruction, the for ImmArgs the constant is put directly on the
intrinsic's MIR instruction as an immediate.
Differential Revision: https://reviews.llvm.org/D132042
The previous implementation translated from names like sifive-7-series
to sifive-7-rv32 or sifive-7-rv64. This also required sifive-7-rv32
and sifive-7-rv64 to be valid CPU names. As those are not real
CPUs it doesn't make sense to accept them in -mcpu.
This patch does away with the translation and adds sifive-7-series
directly to RISCV.td. Removing sifive-7-rv32 and sifive-7-rv64.
sifive-7-series is only allowed in -mtune.
I've also added "rocket" to RISCV.td but have not removed rocket-rv32
or rocket-rv64.
To prevent -mcpu=sifive-7-series or -mcpu=rocket being used with llc,
I've added a Feature32Bit to all rv32 CPUs. And made it an error to
have an rv32 triple without Feature32Bit. sifive-7-series and rocket
do not have Feature32Bit or Feature64Bit set so the user would need
to provide -mattr=+32bit or -mattr=+64bit along with the -mcpu to
avoid the error.
SiFive no longer names their newer products with 3, 5, or 7 series.
Instead we have p200 series, x200 series, p500 series, and p600 series.
Following the previous behavior would require a sifive-p500-rv32 and
sifive-p500-rv64 in order to support -mtune=sifive-p500-series. There
is currently no p500 product, but it could start getting confusing if
there was in the future.
I'm open to hearing alternatives for how to achieve my main goal
of removing sifive-7-rv32/rv64 as a CPU name.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D131708
This patch adds OMPIRBuilder support for the safelen clause for the
simd directive.
Reviewed By: shraiysh, Meinersbur
Differential Revision: https://reviews.llvm.org/D131526
If the incoming previous value of a fixed-order recurrence is a phi in
the header, go through incoming values from the latch until we find a
non-phi value. Use this as the new Previous, all uses in the header
will be dominated by the original phi, but need to be moved after
the non-phi previous value.
At the moment, fixed-order recurrences are modeled as a chain of
first-order recurrences.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D119661
Extends findMoreOptimalIndexType to allow ISD::BUILD_VECTOR based
indices to be truncated when such truncation is lossless. This can
enable the use of 32bit gather/scatter indices thus making it less
likely to have to split a gather/scatter in two.
Depends on D125194
Differential Revision: https://reviews.llvm.org/D130533
* Replace getUserCost with getInstructionCost, covering all cost kinds.
* Remove getInstructionLatency, it's not implemented by any backends, and we should fold the functionality into getUserCost (now getInstructionCost) to make it easier for targets to handle the cost kinds with their existing cost callbacks.
Original Patch by @samparker (Sam Parker)
Differential Revision: https://reviews.llvm.org/D79483
This commit adds the definitions for `dyld_chained_starts_in_image`,
`dyld_chained_starts_in_segment`, and related enums. Dumping their
contents is possible with the -chained_fixups flag of llvm-otool.
The chained-fixups.yaml test was changed to cover bindings/rebases, as
well as weak imports, weak symbols and flat namespace symbols. Now that
we have actual fixup entries, the __DATA segment contains data that
would need to be hexdumped in YAML. We also test empty pages (to look
for the "DYLD_CHAINED_PTR_START_NONE" annotation), so the YAML would end
up quite large. So instead, this commit includes a binary file.
When Apple's effort to upstream their chained fixups code continues,
we'll replace this code with the then-upstreamed code. But we need
something in the meantime for testing ld64.lld's chained fixups code.
Differential Revision: https://reviews.llvm.org/D131961
Also switch from orc::ExecutorAddrDiff to uint64_t for the Symbol::Size field.
These changes help to prepare for the introduction of symbol alias support:
Aliases will require an auxiliary data structure which will also need to be
allocated (hence the need to pass the allocator down). The Size field will be
re-tasked to track the auxiliary data (which will hold a replacement Size field)
if the symbol is either an alias, or aliased by some other symbol.
TragetLowering had two last InstructionCost related `getTypeLegalizationCost()`
and `getScalingFactorCost()` members, but all other costs are processed in TTI.
E.g. it is not comfortable to use other TTI members in these two functions
overrided in a target.
Minor refactoring: `getTypeLegalizationCost()` now doesn't need DataLayout
parameter - it was always passed from TTI.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D117723
This reverts commit 8c4aea438c.
Needed because buildbot failures (warnings) gave a clue that there was
a functional bug in the ARM rejection logic.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D132037
This interface allows a target to reject a proposed
SMS schedule. For Hexagon/PowerPC, all schedules
are accepted, leaving behavior unchanged. For ARM,
schedules which exceed register pressure limits are
rejected.
Also, two RegisterPressureTracker methods now need to be public so
that register pressure can be computed by more callers.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D128941
In D131869 we noticed that we jump through some hoops because we parse the
tolerance option used in MisExpect.cpp into a 64-bit integer. This is
unnecessary, since the value can only be in the range [0, 100).
This patch changes the underlying type to be 32-bit from where it is
parsed in Clang through to it's use in LLVM.
Reviewed By: jloser
Differential Revision: https://reviews.llvm.org/D131935
There are two different senses in which a block can be "address-taken".
There can be a BlockAddress involved, which means we need to map the
IR-level value to some specific block of machine code. Or there can be
constructs inside a function which involve using the address of a basic
block to implement certain kinds of control flow.
Mixing these together causes a problem: if target-specific passes are
marking random blocks "address-taken", if we have a BlockAddress, we
can't actually tell which MachineBasicBlock corresponds to the
BlockAddress.
So split this into two separate bits: one for BlockAddress, and one for
the machine-specific bits.
Discovered while trying to sort out related stuff on D102817.
Differential Revision: https://reviews.llvm.org/D124697
Add initial support for NonNull attribute.
(https://github.com/llvm/llvm-project/issues/57113)
Test plan:
verify that for
__thread int x;
int main() {
int* y = &x;
return *y;
}
(with this patch) clang -O -fsanitize=null -S -emit-llvm -o -
doesn't emit a null-pointer check
Differential revision: https://reviews.llvm.org/D131872
There is an existing mechanism to escape strings, therefore the
functions created to escape Tag_also_compatible_with values are not
really needed. We can simply use the pre-existing utilities.
Reviewed By: pratlucas
Differential Revision: https://reviews.llvm.org/D131680
The ARM Attribute Parser used to parse the value of also_compatible_with
as it is, disregarding the way it is encoded.
This patch does a context aware parsing of the also_compatible_with
attribute. Additionally, some error handling is also done for incorrect
cases.
Reviewed By: pratlucas
Differential Revision: https://reviews.llvm.org/D130913
This reverts commit 354fa0b480.
Returning as is. The patch was reverted due to a miscompile, but
this patch is not causing it. This patch made it possible to infer
some nuw flags in code guarded by `false` condition, and then someone
else to managed to propagate the flag from dead code outside.
Returning the patch to be able to reproduce the issue.
I (lhames) accidentally pushed 5f300397c6 on
Kshitij Jain's behalf without updating the patch author first (my apologies
Kshitij!).
Re-applying with correct authorship.
https://reviews.llvm.org/D131347
Warns on potentially unintended use of C++17 Class Template Argument
Deduction. Use of this feature with types that aren't intended to
support it may may future refactorings of those types more difficult -
so this warning fires whenever the feature is used with a type that may
not have intended to be used with CTAD (the warning uses the existence
of at least one explicit deduction guide to indicate that a type
intentionally supports CTAD - absent that, it's assumed to not be
intended to support CTAD & produces a warning).
This is disabled in libcxx because lots of the standard library is
assumed to provide ctad-usable APIs and the false positive suppression
in the diagnostic is based on system header classification which doesn't
apply in the libcxx build itself.
Differential Revision: https://reviews.llvm.org/D131727
Using the legacy PM for the optimization pipeline is deprecated and in
the process of being removed. This is a small step in that direction.
For an example of migrating to the new PM:
853b57fe80
This class will be used to properly solve the `__imp_` symbol and jump-thunk generation issues. It is assumed to be the last definition generator to be called, and as it's the last generator the only symbols remaining in the lookup set are the symbols that are supposed to be queried outside this jitdylib. Instead of just letting them through, we issue another lookup invocation and fetch the allocated addresses, and then create jitlink graph containing `__imp_` GOT symbols and jump-thunks targetting the fetched addresses.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D131833
And --chained-fixups for llvm-objdump.
For now, this only prints the dyld_chained_fixups_header and adds
plumbing for the flag. This will be expanded in future commits.
When Apple's effort to upstream their chained fixups code continues,
we'll replace this code with the then-upstreamed code. But we need
something in the meantime for testing ld64.lld's chained fixups
code.
Update chained-fixups.yaml with a file that actually contains
the chained fixup data (`LinkEditData` doesn't encode it yet,
so use `__LINKEDIT` via `--raw-segment=data`).
Differential Revision: https://reviews.llvm.org/D131890
These members were made private here 6177386b05 without an explanation.
Our customers have an own implementation inherited from PredIterator with updated advancePastNonTerminators().
The access specifier protected looks resonable and safe here.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D131608
This reverts commit 34ae308c73.
Our internal testing found a miscompile. Not sure if it's caused by
this patch or it revealed something else. Reverting while investigating.
This patch makes `SmallSet::insert(const T &)` return
`std::pair<const_iterator, bool>` instead of
`std::pair<NoneType, bool>`. This will exactly match std::set's behavior
and make deduplicating items with SmallSet easier.
Reviewed By: dblaikie, lattner
Differential Revision: https://reviews.llvm.org/D131549
`getContext().setMCLineTableRootFile` (from D62074) sets `RootFile.Name` to
`FirstCppHashFilename`. `RootFile.Name` is not processed by -fdebug-prefix-map
and will go to DW_TAG_compile_unit's DT_AT_name and DW_TAG_label's
DW_AT_decl_file. Remap `RootFile.Name`.
Fix another issue reported by https://github.com/llvm/llvm-project/issues/56609
Reviewed By: #debug-info, dblaikie, raj.khem
Differential Revision: https://reviews.llvm.org/D131848
This patch deprecates Optional::map in favor of Optional::transform
for consistency with std::optional::transform in C++23.
Note that I've migrated all known users of Optional::map.
Differential Revision: https://reviews.llvm.org/D131842
Add a fix to check that FDE pc-begin targets are defined before calling
getBlock (which will crash if the target is not defined). FDE pc-begins
pointing at undefined symbols are expected to arise only in obscure
circumstances (malformed objects, or removal of targets by JITLink
passes), but we want to handle them gracefully. With this patch the
FDE will be retained, but without any keepalive edge to it. Unless
some pass takes action to mark it as live it will be dead-stripped.
To make it easier for passes to connect FDEs to their targets a new
EHFrameCFIBlockInspector utility is added. This allows clients to
quickly determine whether a CFI record is a CIE or an FDE (assuming
that it's valid), and retrieve any personality, pc-begin, cie, or
LSDA edges associated with it.
MapperJITLinkMemoryManager uses a free list to keep track of available
memory regions. Using an IntervalMap instead of vector allow automatic
coalescing of memory regions as they are freed.
Differential Revision: https://reviews.llvm.org/D131831
Replace C-style array with `std::array` since `std::array<T, N>::operator[]` is
`constexpr` in C++17. This also allows us to replace `array_lengthof` calls with
member `size()` function.
Differential Revision: https://reviews.llvm.org/D131826
This patch implements Optional::transform for consistency with
std::optional::transform in C++23.
Note that the new function is identical to Optional::map. My plan is
to deprecate Optional::map after migrating all of its uses to
Optional::transform.
Differential Revision: https://reviews.llvm.org/D131829
When memory is deallocated from MapperJITLinkMemoryManager deinitialize
actions are run through mapper and in case of InProcessMapper, memory
protections of the region are reset to read/write as they were previously
changed and can be reused in future.
Differential Revision: https://reviews.llvm.org/D131768
Initial platform support for COFF/x86_64.
Completed features:
* Statically linked orc runtime.
* Full linking/initialization of static/dynamic vc runtimes and microsoft stl libraries.
* SEH exception handling.
* Full static initializers support
* dlfns
* JIT side symbol lookup/dispatch
Things to note:
* It uses vc runtime libraries found in vc toolchain installations.
* Bootstrapping state is separated because when statically linking orc runtime it needs microsoft stl functions to initialize the orc runtime, but static initializers need to be ran in order to fully initialize stl libraries.
* Process symbols can't be used blidnly on msvc platform; otherwise duplicate definition error gets generated. If process symbols are used, it's destined to get out-of-reach error at some point.
* Atexit currently not handled -- will be handled in the follow-up patches.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130479
When memory is deallocated from MapperJITLinkMemoryManager deinitialize
actions are run through mapper and in case of InProcessMapper, memory
protections of the region are reset to read/write as they were previously
changed and can be reused in future.
Differential Revision: https://reviews.llvm.org/D131768
For generated assembly debug info, MCDwarfLineTableHeader::CompilationDir is an
unmapped path set in MCContext::setGenDwarfRootFile. Remap it.
A relative destination path of -fdebug-prefix-map= exposes a llvm-dwarfdump bug
which joins relative DW_AT_comp_dir and directories[0].
Fix https://github.com/llvm/llvm-project/issues/56609
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D131749
As a follow-up of e8578968f6 which replaced the
callers to use the C++17 equivalents, remove the equivalents from
`STLForwardCompat.h` entirely and their corresponding tests.
Differential Revision: https://reviews.llvm.org/D131769
The value of the attribute is a size in bytes. It has the effect of
suppressing inlining of functions whose stacksizes exceed the given value.
Reviewed By: mtrofin
Differential Revision: https://reviews.llvm.org/D129904
The code was relicensed by its owner (Unicode.org) a long time back,
but we still had the old (problematic) license in our fork.
Note that the source files have not been distributed from unicode.org
since 2009 (due to being buggy and unmaintained upstream), but they
were given this license before that.
Fixes https://github.com/llvm/llvm-project/issues/32309
Differential Revision: https://reviews.llvm.org/D66390
STLForwardCompat.h defines several utilities and type traits to mimic that of
the ones in the C++17 standard library. Now that LLVM is built with the C++17
standards mode, remove use of these equivalents in favor of the ones from the
standard library.
Differential Revision: https://reviews.llvm.org/D131717
Contextual knowledge may be used to prove invariance of some conditions.
For example, in this case:
```
; %len >= 0
guard(%iv = {start,+,1}<nuw> <s %len)
guard(%iv = {start,+,1}<nuw> <u %len)
```
the 2nd check always fails if `start` is negative and always passes otherwise.
It looks like there are more opportunities of this kind that are still to be
implemented in the future.
Differential Revision: https://reviews.llvm.org/D129753
Reviewed By: apilipenko
Since c5b3de6745 (git main,
August 11th), Clang does generate working hidden visibility
on MinGW targets. Using that reduces the number of exports from
a dylib build of LLVM significantly, which is vital for fitting
within the limit of 64k exported symbols from a DLL.
It's essential that if we set CMAKE_CXX_VISIBILITY_PRESET=hidden
(which passes -fvisibility=hidden on the command line), we also
must define LLVM_EXTERNAL_VISIBILITY consistently to override
it. (If there are mismatches, e.g. setting hidden visibility generally
but never overriding it back to default for the symbols that do need
to be exported, we'd get broken builds in such configurations.)
We don't want to be using __attribute__((visibility("hidden"))) on
MinGW with GCC, because GCC produces a warning about it. (GCC hasn't
warned about the command line options that set hidden visibility
though.) Clang has historically not warned about either of them, so
it is harmless to use the hidden visibility when building with older
Clang (so we don't need to detect the exact version of Clang/LLVM where
it has an effect).
This reduces the number of exported symbols for a dylib build of LLVM;
previously libLLVM exported around 64650 symbols (when the maximum is
65536) when the ARM, AArch64 and X86 targets were enabled. If enabling
more targets (or if building with e.g. assertions enabled), it would
exceed the limit. Now with visibility flags in use, the same build
with ARM, AArch64 and X86 ends up at around 35k exported symbols.
Differential Revision: https://reviews.llvm.org/D131661
Coroutine splitting is not possible if the one-to-one mapping between the two is
lost. Every suspend point must have a matching continuation function
pointer.
rdar://98404664
Differential Revision: https://reviews.llvm.org/D131684
Other sanitizers (ASan, TSan, see added tests) already handle
memcpy.inline and memset.inline by not relying on InstVisitor to turn
the intrinsics into calls. Only MSan instrumentation currently does not
support them due to missing InstVisitor callbacks.
Fix it by actually making InstVisitor handle Mem*InlineInst.
While the mem*.inline intrinsics promise no calls to external functions
as an optimization, for the sanitizers we need to break this guarantee
since access into the runtime is required either way, and performance
can no longer be guaranteed. All other cases, where generating a call is
incorrect, should instead use no_sanitize.
Fixes: https://github.com/llvm/llvm-project/issues/57048
Reviewed By: vitalybuka, dvyukov
Differential Revision: https://reviews.llvm.org/D131577
Introduces COFFVCRuntimeBootstrapper that loads/initialize vc runtime libraries. In COFF, we *must* jit-link vc runtime libraries as COFF relocation types have no proper way to deal with out-of-reach data symbols ragardless of linking mode. (even dynamic version msvcrt.lib have tons of static data symbols that must be jit-linked) This class tries to load vc runtime library files from msvc installations with an option to override the path.
There are some complications when dealing with static version of vc runtimes. First, they need static initializers to be ran that requires COFFPlatform support but orc runtime will not be usable before vc runtimes are fully initialized. (as orc runtime will use msvc stl libraries) COFFPlatform that will be introduced in a following up patch will collect static initializers and run them manually in host before boostrapping itself. So, the user will have to do the following.
1. Create COFFPlatform that addes static initializer collecting passes.
2. LoadVCRuntime
3. InitializeVCRuntime
4. COFFPlatform.bootstrap()
Second, the internal crt initialization function had to be reimplemented in orc side. There are other ways of doing this, but this is the simplest implementation that makes platform fully responsible for static initializer. The complication comes from the fact that crt initialization functions (such as acrt_initialize or dllmain_crt_process_attach) actually run all static initializers by traversing from `__xi_a` symbol to `__xi_z`. This requires symbols to be contiguously allocated in sections alphabetically sorted in memory, which is not possible right now and not practical in jit setting. We might ignore emission of `__xi_a` and `__xi_z` symbol and allocate them ourselves, but we have to take extra care after orc runtime boostrap has been done -- as that point orc runtime should be the one running the static initializers.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130456
Implements SECTION/SECREL relocation. These are used by debug info (pdb) data.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130275
This is not used as general CPU alias. Only to support -mtune. Name it as such.
Reviewed By: kito-cheng
Differential Revision: https://reviews.llvm.org/D131602
This patch makes the variants of `mm*_cast*` intel intrinsics that use `shufflevector(freeze(poison), ..)` emit efficient assembly.
(These intrinsics are planned to use `shufflevector(freeze(poison), ..)` after shufflevector's semantics update; relevant thread: D103874)
To do so, this patch
1. Updates `LowerAVXCONCAT_VECTORS` in X86ISelLowering.cpp to recognize `FREEZE(UNDEF)` operand of `CONCAT_VECTOR` in addition to `UNDEF`
2. Updates X86InstrVecCompiler.td to recognize `insert_subvector` of `FREEZE(UNDEF)` vector as its first operand.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D130339
This patch teaches llvm-profdata to output the sample profile in the
JSON format. The new option is intended to be used for research and
development purposes. For example, one can write a Python script to
take a JSON file and analyze how similar different inline instances of
a given function are to each other.
I've chosen JSON because Python can parse it reasonably fast, and it
just takes a couple of lines to read the whole data:
import json
with open ('profile.json') as f:
profile = json.load(f)
Differential Revision: https://reviews.llvm.org/D130944
After D121595 was commited, I noticed regressions assosicated with small trip
count numbersvectorisation by tail folding with scalable vectors. As a solution
for those issues I propose to introduce the minimal trip count threshold value.
Differential Revision: https://reviews.llvm.org/D130755
This patch adds the names of the Arm Architecture Reference Manual (ARM)
features to the corresponding Subtarget Features in the AArch64 backend
and target parser.
The aim of this is to make it clearer what architectural features a
subtarget feature might enable (so, which features a CPU must provide to
support that subtarget feature), and so make it easier to add new CPUs
in the future.
Differential Revision: https://reviews.llvm.org/D131257
This patch adds the ability to deconstruct the `value_type` returned by `llvm::enumarate` into index and value of the wrapping range. Main use case is the common occurence of using it during loop iteration. After this patch it'd then be possible to write code such as:
```
for (auto [index, value] : enumerate(container)) {
...
}
```
where `index` is the current index and `value` a reference to elements in the given container.
Differential Revision: https://reviews.llvm.org/D131486
Serge Pavlov