Commit Graph

3141 Commits

Author SHA1 Message Date
Kazu Hirata fd4808887e [llvm] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-01-01 16:18:18 -08:00
Kazu Hirata f4ffcab178 Remove redundant string initialization (NFC)
Identified by readability-redundant-string-init.
2022-01-01 12:34:11 -08:00
Kazu Hirata 5a667c0e74 [llvm] Use nullptr instead of 0 (NFC)
Identified with modernize-use-nullptr.
2021-12-28 08:52:25 -08:00
Lang Hames 02fc8d5c9e [ORC] Add custom object interface support to StaticLibaryDefinitionGenerator.
This adds a GetObjectFileInterface callback member to
StaticLibraryDefinitionGenerator, and adds an optional argument for initializing
that member to StaticLibraryDefinitionGenerator's named constructors. If not
supplied, it will default to getObjectFileInterface from ObjectFileInterface.h.

To enable testing a `-hidden-l<x>` option is added to the llvm-jitlink tool.
This allows archives to be loaded with all contained symbol visibilities demoted
to hidden.

The ObjectLinkingLayer::setOverrideObjectFlagsWithResponsibilityFlags method is
(belatedly) hooked up, and enabled in llvm-jitlink when `-hidden-l<x>` is used
so that the demotion is also applied at symbol resolution time (avoiding any
"mismatched symbol flags" crashes).
2021-12-16 19:46:51 +11:00
Lang Hames 4299d8d0ce [ORC] Add MaterializationUnit::Interface parameter to ObjectLayer::add.
Also moves object interface building functions out of Mangling.h and in to the
new ObjectFileInterfaces.h header, and updates the llvm-jitlink tool to use
custom object interfaces rather than a custom link layer.

ObjectLayer::add overloads are added to match the old signatures (which
do not take a MaterializationUnit::Interface). These overloads use the
standard getObjectFileInterface function to build an interface.

Passing a MaterializationUnit::Interface explicitly makes it easier to alter
the effective interface of the object file being added, e.g. by changing symbol
visibility/linkage, or renaming symbols (in both cases the changes will need to
be mirrored by a JITLink pass at link time to update the LinkGraph to match the
explicit interface). Altering interfaces in this way can be useful when lazily
compiling (e.g. for renaming function bodies) or emulating linker options (e.g.
demoting all symbols to hidden visibility to emulate -load_hidden).
2021-12-15 12:03:55 +11:00
Lang Hames 3ce1e9428b [ORC] Add early-out to OL_applyQueryPhase1.
If all symbols in a lookup match before we reach the end of the search order
then bail out of the search-order loop early.

This should reduce unnecessary contention on the session lock and improve
readability of the debug logs.
2021-12-15 10:01:02 +11:00
Jan Svoboda d0262c2394 [llvm] Add null-termination capability to SmallVectorMemoryBuffer
Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (`mmap` vs `open`),
* assert newly initialized buffer indeed has an implicit null terminator.

This patch adds the paramater to the `SmallVectorMemoryBuffer` constructors, meaning:
* null terminator can now be added to `SmallVector`s that didn't have one before,
* `SmallVectors` that had a null terminator before keep it even after the move.

In line with existing code, the new parameter is defaulted to `true`. This patch makes sure all calls to the `SmallVectorMemoryBuffer` constructor set it to `false` to preserve the current semantics.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115331
2021-12-09 11:32:13 +01:00
Kazu Hirata c23ebf1714 [llvm] Use range-based for loops (NFC) 2021-12-08 20:35:39 -08:00
Lang Hames ae73f3fdd6 [ORC] Add a MaterializationUnit::Interface struct.
MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.

Having a type for this will make functions that build materializer interfaces
more readable and maintainable.
2021-12-08 13:41:15 +11:00
Lang Hames 02095f2dd7 [ORC] Pad section start to account for alignment offset in MachO GDB JIT plugin.
In order to present a well-formed MachO debug object for debugger registration
the first block in each section must have a zero alignment offset (since there
is no way to represent a non-zero offset in a MachO section load command). This
patch updates the MachODebugObjectSynthesizer class to introduce a padding
padding block at the start of the section if necessary to guarantee a zero
alignment offset.
2021-12-07 14:28:31 +11:00
Jack Andersen b66339575a [CMake] Installable find modules for terminfo and libffi
Improves cross-distro portability of LLVM cmake package by resolving paths for
terminfo and libffi via import targets.

When LLVMExports.cmake is generated for installation, it contains absolute
library paths which are likely to be a common cause of portability issues. To
mitigate this, the discovery logic for these dependencies is refactored into
find modules which get installed alongside LLVMConfig.cmake. The result is
cleaner, cmake-friendly management of these dependencies that respect the
environment of the LLVM package importer.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D114327
2021-12-05 14:46:23 -05:00
Lang Hames 21562c03ed [JITLink][ELF][x86-64] Adjust addends for R_X86_64_PLT32 relocations.
R_X86_64_PLT32 explicitly represents the '-4' PC-adjustment in the relocation's
addend, but JITLink's x86_64::Branch32PCRel includes the PC-adjustment
implicitly. We have been zeroing the addend to account for the difference, but
this breaks for branches to non-zero offsets past labels. This patch updates the
relocation parsing code to unconditionally adjust the offset by '+4' instead.
For branches directly to labels the result is still 0, for branches to offsets
past labels the result is the correct addend for x86_64::Branch32PCRel.
2021-12-05 20:37:55 +11:00
Lang Hames c22b110612 [JITLink][ELF][x86-64] Use the right edge-naming function for debugging output.
Graph edges use the generic x86-64 edge set (the ELF specific edges are only
used during parsing).
2021-12-05 17:12:39 +11:00
Kazu Hirata 22d82949b0 [llvm] Fix "unused variable" warnings 2021-12-02 09:20:17 -08:00
Lang Hames 758d54b462 [ORC] Add support for removing JITDylibs.
This allows JITDylibs to be removed from the ExecutionSession. Calling
ExecutionSession::removeJITDylib will disconnect the JITDylib from the
ExecutionSession and clear it (removing all trackers associated with it). The
JITDylib object will then be destroyed as soon as the last JITDylibSP pointing
at it is destroyed.
2021-12-02 17:36:32 +11:00
Lang Hames 9eb591f0ac [ORC] Only use JITDylib::GeneratorsMutex while running generators.
GeneratorsMutex should prevent lookups from proceeding through the
generators of a single JITDylib concurrently (since this could
result in redundant attempts to generate definitions). Mutation of
the generators list itself should be done under the session lock.
2021-12-02 17:36:32 +11:00
Lang Hames 9355d11597 [ORC] Hold ResourceTracker in MaterializationResponsibility.
This keeps the tracker alive for the lifetime of the MR. This is needed so that
we can check whether the tracker has become defunct before posting results (or
failure) for the MR.
2021-12-02 17:36:31 +11:00
Steven Wu c737d4d203 [JITLink][ELF] Don't skip sections of size 0
Size 0 sections can have symbols that have size 0. Build those sections
and symbols into the LinkGraph so they can be used properly if needed.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D114749
2021-11-30 09:19:14 -08:00
Steven Wu 5e3200f3ce [JITLink][ELF] Add support for reading extended table
Add support for reading extended table in ELF object file. This allows
JITLink to support ELF object files with many sections.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D114747
2021-11-30 09:18:42 -08:00
Zhuo Zhang d96f92ff16 fix typos in comments 2021-11-29 14:06:33 +01:00
Kazu Hirata fd7d40640d [llvm] Use range-based for loops (NFC) 2021-11-28 18:14:49 -08:00
Lang Hames 43f5f6916f [ORC][JITLink] Move JITDylib name into JITLinkDylib base class.
This will enable better error messages and debug logs in JITLink.
2021-11-19 20:54:28 -08:00
Ben Langmuir 4c94760f36 [ORC] Fix materialization of weak local symbols
We were adding all defined weak symbols to the materialization
responsibility, but local symbols will not be in the symbol table, so it
failed to materialize due to the "missing" symbol.

Local weak symbols come up in practice when using `ld -r` with a hidden
weak symbol.

rdar://85574696
2021-11-19 07:25:56 -08:00
Zarko Todorovski 59c84774d2 [NFC][llvm] Inclusive language: remove uses of sanity in llvm/lib/ExecutionEngine/
Reworded and removed code comments to avoid using `sanity check` and `sanity
test`.
2021-11-17 22:17:54 -05:00
Steven Wu fcd07f8107 [JITLink] Fix splitBlock if there are symbols span across the boundary
Fix `splitBlock` so that it can handle the case when the block being
split has symbols span across the split boundary. This is an error
case in general but for EHFrame splitting on macho platforms, there is an
anonymous symbol that marks the entire block. Current implementation
will leave a symbol that is out of bound of the underlying block. Fix
the problem by dropping such symbols when the block is split.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D113912
2021-11-15 13:55:21 -08:00
Lang Hames 69be352a19 Reapply "[ORC] Initial MachO debugging support (via GDB JIT debug.." with fixes.
This reapplies e1933a0488 (which was reverted in
f55ba3525e due to bot failures, e.g.
https://lab.llvm.org/buildbot/#/builders/117/builds/2768).

The bot failures were due to a missing symbol error: We use the input object's
mangling to decide how to mangle the debug-info registration function name. This
caused lookup of the registration function to fail when the input object
mangling didn't match the host mangling.

Disbaling the test on non-Darwin platforms is the easiest short-term solution.
I have filed https://llvm.org/PR52503 with a proposed longer term solution.
2021-11-14 14:44:07 -08:00
Lang Hames f55ba3525e Revert "[ORC] Initial MachO debugging support (via GDB JIT debug..."
This reverts commit e1933a0488 until I can look
into bot failures.
2021-11-14 00:14:39 -08:00
Lang Hames e1933a0488 [ORC] Initial MachO debugging support (via GDB JIT debug registration interface)
This commit adds a new plugin, GDBJITDebugInfoRegistrationPlugin, that checks
for objects containing debug info and registers any debug info found via the
GDB JIT registration API.

To enable this registration without redundantly representing non-debug sections
this plugin synthesizes a new embedded object within a section of the LinkGraph.
An allocation action is used to make the registration call.

Currently MachO only. ELF users can still use the DebugObjectManagerPlugin. The
two are likely to be merged in the near future.
2021-11-13 13:21:01 -08:00
Lang Hames 2272ec1c63 [JITLink][MachO] Fix "find-symbol-by-address" logic.
Only search within the requested section, and allow one-past-then-end addresses.

This is needed to support section-end-address references to sections with no
symbols in them.
2021-11-12 21:28:32 -08:00
Ben Langmuir 2a739f2789 [ORC][ORC-RT] Register type metadata from __swift5_types MachO section
Similar to how the other swift sections are registered by the ORC
runtime's macho platform, add the __swift5_types section, which contains
type metadata. Add a simple test that demonstrates that the swift
runtime recognized the registered types.

rdar://85358530

Differential Revision: https://reviews.llvm.org/D113811
2021-11-12 16:39:59 -08:00
Lang Hames 9d5e647428 [JITLink] Fix think-o in handwritten CWrapperFunctionResult -> Error converter.
We need to skip the length field when generating error strings.

No test case: This hand-hacked deserializer should be removed in the near future
once JITLink can use generic ORC APIs (including SPS and WrapperFunction).
2021-11-12 10:36:17 -08:00
Ben Langmuir a2639dcbe6 [ORC] Add a utility for adding missing "self" relocations to a Symbol
If a tool wants to introduce new indirections via stubs at link-time in
ORC, it can cause fidelity issues around the address of the function if
some references to the function do not have relocations. This is known
to happen inside the body of the function itself on x86_64 for example,
where a PC-relative address is formed, but without a relocation.

```
_foo:
  leaq -7(%rip), %rax ## form pointer to '_foo' without relocation

_bar:
  leaq (%rip), %rax ##  uses X86_64_RELOC_SIGNED to '_foo'
```

The consequence of introducing a stub for such a function at link time
is that if it forms a pointer to itself without relocation, it will not
have the same value as a pointer from outside the function. If the
function pointer is used as a key, this can cause problems.

This utility provides best-effort support for adding such missing
relocations using MCDisassembler and MCInstrAnalysis to identify the
problematic instructions. Currently it is only implemented for x86_64.

Note: the related issue with call/jump instructions is not handled
here, only forming function pointers.

rdar://83514317

Differential revision: https://reviews.llvm.org/D113038
2021-11-04 15:01:05 -07:00
Lang Hames e9014d9743 [ORC] Run incoming jit-dispatch calls via the TaskDispatcher in SimpleRemoteEPC.
Handlers for jit-dispatch calls are allowed to make their own EPC calls, so we
don't want to run these on the handler thread.
2021-11-01 15:49:14 -07:00
Lang Hames ff846fcb64 [ORC][ORC-RT] Switch MachO EH/TLV registration from EPC-calls to alloc actions.
MachOPlatform used to make an EPC-call (registerObjectSections) to register the
eh-frame and thread-data sections for each linked object with the ORC runtime.

Now that JITLinkMemoryManager supports allocation actions we can use these
instead of an EPC call. This saves us one EPC-call per object linked, and
manages registration/deregistration in the executor, rather than the controller
process. In the future we may use this to allow JIT'd code in the executor to
outlive the controller object while still being able to be cleanly destroyed.

Since the code for allocation actions must be available when the actions are
run, and since the eh-frame registration code lives in the ORC runtime itself,
this change required that MachO eh-frame support be split out of
macho_platform.cpp and into its own macho_ehframe_registration.cpp file that has
no other dependencies. During bootstrap we start by forcing emission of
macho_ehframe_registration.cpp so that eh-frame registration is guaranteed to be
available for the rest of the bootstrap process. Then we load the rest of the
MachO-platform runtime support, erroring out if there is any attempt to use
TLVs. Once the bootstrap process is complete all subsequent code can use all
features.
2021-10-31 10:27:40 -07:00
Lang Hames b77c6db959 [JITLink] Fix alloc action call signature in InProcessMemoryManager.
Alloc actions should return a CWrapperFunctionResult. JITLink does not have
access to this type yet, due to library layering issues, so add a cut-down
version with a fixme.
2021-10-31 10:27:40 -07:00
Lang Hames 213666f804 [ORC] Move CWrapperFunctionResult out of the detail:: namespace.
This type has been moved up into the llvm::orc::shared namespace.

This type was originally put in the detail:: namespace on the assumption that
few (if any) LLVM source files would need to use it. In practice it has been
needed in many places, and will continue to be needed until/unless
OrcTargetProcess is fully merged into the ORC runtime.
2021-10-30 16:12:45 -07:00
Lang Hames afeb1e4ac7 [ORC] Move all pass config into MachOPlatformPlugin::modifyPassConfig.
NFC, this just makes it easier to see and reason about pass ordering.
2021-10-29 20:07:45 -07:00
Lang Hames 12b2cc2294 [ORC] Rename SupportFunctionCall to WrapperFunctionCall.
The new name better suits the type.

This patch also changes the signature of the run method (it now returns a
WrapperFunctionResult), and adds runWithSPSRet methods that deserialize the
function result using SPS.

Together these chages bring this type into close alignment with its ORC runtime
counterpart.
2021-10-28 17:48:54 -07:00
Lang Hames 999c6a235e Reapply e32b1eee6a "[ORC] Change SPSExecutorAddr serialization,..." with fixes.
This re-applies e32b1eee6a, which was reverted in
20675d8f7d due to broken unit tests. This patch
includes fixes for the tests.
2021-10-28 16:40:25 -07:00
Lang Hames 20675d8f7d Revert "[ORC] Change SPSExecutorAddr serialization, SupportFunctionCall struct."
This reverts commit e32b1eee6a.

Reverting while I fix some broken unit tests.
2021-10-27 16:39:56 -07:00
Lang Hames e32b1eee6a [ORC] Change SPSExecutorAddr serialization, SupportFunctionCall struct.
SPSExecutorAddr will now be serializable to/from ExecutorAddr, rather than
uint64_t. This improves type safety when working with serialized addresses.

Also updates the SupportFunctionCall to use an ExecutorAddrRange (rather than
a separate ExecutorAddr addr and uint64_t size field), and updates the
tpctypes::*Write data structures to use ExecutorAddr rather than
JITTargetAddress.
2021-10-27 16:20:46 -07:00
Ben Langmuir 3d13ee2891 [ORC][ORC-RT] Enable the MachO platform for arm64
Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are just handled as GOT accesses.

rdar://84671534

Differential Revision: https://reviews.llvm.org/D112656
2021-10-27 13:36:03 -07:00
Lang Hames 91434d4469 [JITLink] Fix element-present check in MachOLinkGraphParser.
Not all symbols are added to the index-to-symbol map, so we shouldn't use the
size of the map as a proxy for the highest valid index.
2021-10-26 20:48:40 -07:00
Lang Hames bfb40e83ee [ORC] Don't try to perform empty deallocations. 2021-10-26 20:48:40 -07:00
Stefan Gränitz cdb335ffaf [JITLink] Fix warning 'shift count exceeds width' in AArch64 backend 2021-10-25 20:44:07 +02:00
Lang Hames cc3115cd1d [JITLink][x86-64] Lift GOT, PLT table managers into x86_64.h; reuse for MachO.
This lifts the global offset table and procedure linkage table builders out of
ELF_x86_64.h and into x86_64.h, renaming them with generic names
x86_64::GOTTableBuilder and x86_64::PLTTableBuilder. MachO_x86_64.cpp is updated
to use these classes instead of the older PerGraphGOTAndStubsBuilder tool.
2021-10-18 21:47:24 -07:00
Lang Hames bc03a9c066 Simplify the TableManager class and move it into a public header.
Moves visitEdge into the TableManager derivatives, replacing the fixEdgeKind
methods in those classes. The visitEdge method takes on responsibility for
updating the edge target, as well as its kind.
2021-10-18 18:20:33 -07:00
luxufan 849b36bf6f [JITLink][NFC] Add TableManager to replace PerGraph...Builder pass
This patch add a TableManager which reponsible for fixing edges that need entries to reference the target symbol and constructing such entries.

In the past, the PerGraphGOTAndPLTStubsBuilder pass was used to build GOT and PLT entry, and the PerGraphTLSInfoEntryBuilder pass was used to build TLSInfo entry. By generalizing the behavior of building entry, I added a TableManager which could be reused when built GOT, PLT and TLSInfo entries.

If this patch makes sense and can be accepted, I will apply the TableManager to other targets(MachO_x86_64, MachO_arm64, ELF_riscv), and delete the file PerGraphGOTAndPLTStubsBuilder.h

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D110383
2021-10-14 23:09:53 +08:00
Simon Pilgrim 16729d0f62 [Orc] ELFNixPlatform::setupJITDylib - remove dead return. NFCI.
2 returns, one after the other - reported by coverity
2021-10-14 12:17:40 +01:00
Lang Hames 4fcc0ac15e [ORC] Use a Setup object for SimpleRemoteEPC construction.
SimpleRemoteEPC notionally allowed subclasses to override the
createMemoryManager and createMemoryAccess methods to use custom objects, but
could not actually be subclassed in practice (The construction process in
SimpleRemoteEPC::Create could not be re-used).

Instead of subclassing, this commit adds a SimpleRemoteEPC::Setup class that
can be used by clients to set up the memory manager and memory access members.
A default-constructed Setup object results in no change from previous behavior
(EPCGeneric* memory manager and memory access objects used by default).
2021-10-13 16:47:00 -07:00
Lang Hames 447d3017e4 [JITLink][MachO][arm64] Mask high bits out of immediate for LDRLiteral19.
Negative deltas for LDRLiteral19 have their high bits set. If these bits aren't
masked out then they will overwrite other instruction bits, leading to a bogus
encoding.

This long-standing relocation bug was exposed by e50aea58d5, "[JITLink][ORC]
Major JITLinkMemoryManager refactor.", which caused memory layouts to be
reordered, which in turn lead to a previously unseen negative delta. (Unseen
because LDRLiteral19s were only created in JITLink passes where they always
pointed at segments that were layed-out-after in the old layout).

No testcase yet: Our existing regression test infrastructure is good at checking
that operand bits are correct, but provides no easy way to test for bad opcode
bits. I'll have a think about the right way to approach this.

https://llvm.org/PR52153
2021-10-12 22:19:47 -07:00
Lang Hames 2815ed57e3 [ORC] Shut down dispatcher in ExecutorProcessControl implementations.
f341161689 added a task dispatcher for async handlers, but didn't add a
TaskDispatcher::shutdown call to SelfExecutorProcessControl or SimpleRemoteEPC.
This patch adds the missing call, which ensures that we don't destroy the
dispatcher while tasks are still running.

This should fix the use-after-free crash seen in
https://lab.llvm.org/buildbot/#/builders/5/builds/13063
2021-10-12 13:40:15 -07:00
Lang Hames 5829ba7afc [ORC] More attempts to work around compiler failures.
Commit 731f991cdc seems to have helped, but did not catch all instances (see
https://lab.llvm.org/buildbot/#/builders/193/builds/104). Switch more inner
structs to C++98 initializers to work around the issue. Add FIXMEs to revisit
in the future.
2021-10-11 22:36:56 -07:00
Lang Hames 3a52a639b1 [ORC] Add more explicit narrowing casts.
This should fix the buildbot failure at
https://lab.llvm.org/buildbot/#/builders/187/builds/2140
2021-10-11 22:00:06 -07:00
Lang Hames 9ca5064153 [ORC] Fix a typo in a variable name. 2021-10-11 21:50:46 -07:00
Lang Hames 962a2479b5 Re-apply e50aea58d5, "Major JITLinkMemoryManager refactor". with fixes.
Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70.
2021-10-11 21:39:00 -07:00
Lang Hames 6641d29b70 Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."
This reverts commit e50aea58d5 while I
investigate bot failures.
2021-10-11 19:23:41 -07:00
Lang Hames e50aea58d5 [JITLink][ORC] Major JITLinkMemoryManager refactor.
This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
                                     (size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.
2021-10-11 19:12:42 -07:00
Lang Hames 17a0858f9d [ORC] Propagate errors to handlers when sendMessage fails.
In SimpleRemoteEPC, calls to from callWrapperAsync to sendMessage may fail.
The handlers may or may not be sent failure messages by handleDisconnect,
depending on when that method is run. This patch adds a check for an un-failed
handler, and if it finds one sends it a failure message.
2021-10-11 14:23:50 -07:00
Lang Hames 4fc2a4cc01 [ORC] Destroy FinalizeErr if there is a serialization error.
If there is a serialization error then FinalizeErr should never be set, so we
can use cantFail rather than consumeError here.
2021-10-11 14:23:50 -07:00
Stefan Gränitz a6c9506365 [Orc] Handle hangup messages in SimpleRemoteEPC
On the controller-side, handle `Hangup` messages from the executor. The executor passed `Error::success()` or a failure message as payload.

Hangups cause an immediate disconnect of the transport layer. The disconnect function may be called later again and so implementations should be prepared. `FDSimpleRemoteEPCTransport::disconnect()` already has a flag to check that:
cd1bd95d87/llvm/lib/ExecutionEngine/Orc/Shared/SimpleRemoteEPCUtils.cpp (L112)

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D111527
2021-10-11 21:04:56 +02:00
Tommi Pisto 42b588a200 [ORC] Add static and dynamic library generator support to C API.
Adds LLVMOrcCreateStaticLibrarySearchGeneratorForPath and
LLVMOrcCreateDynamicLibrarySearchGeneratorForPath functions to create generators
for static and dynamic libraries.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D108535
2021-10-11 09:25:59 -07:00
Lang Hames 4d7cea3d2e [ORC] Add optional RunPolicy to ExecutorProcessControl::callWrapperAsync.
The callWrapperAsync and callSPSWrapperAsync methods take a handler object
that is run on the return value of the call when it is ready. The new RunPolicy
parameters allow clients to control how these handlers are run. If no policy is
specified then the handler will be packaged as a GenericNamedTask and dispatched
using the ExecutorProcessControl's TaskDispatch member. Callers can use the
ExecutorProcessControl::RunInPlace policy to cause the handler to be run
directly instead, which may be preferrable for simple handlers, or they can
write their own policy object (e.g. to dispatch as some other kind of Task,
rather than GenericNamedTask).
2021-10-10 20:41:59 -07:00
Lang Hames 771e69484a [ORC] Add dependence on pthreads library to ORC.
f341161689 introduced a dependence (for builds with LLVM_ENABLE_THREADS) on
pthreads. This commit updates the CMakeLists.txt file to include a LINK_LIBS
entry for pthreads.
2021-10-10 19:34:34 -07:00
Lang Hames f341161689 [ORC] Add TaskDispatch API and thread it through ExecutorProcessControl.
ExecutorProcessControl objects will now have a TaskDispatcher member which
should be used to dispatch work (in particular, handling incoming packets in
the implementation of remote EPC implementations like SimpleRemoteEPC).

The GenericNamedTask template can be used to wrap function objects that are
callable as 'void()' (along with an optional name to describe the task).
The makeGenericNamedTask functions can be used to create GenericNamedTask
instances without having to name the function object type.

In a future patch ExecutionSession will be updated to use the
ExecutorProcessControl's dispatcher, instead of its DispatchTaskFunction.
2021-10-10 18:39:55 -07:00
Lang Hames da7f993a8d [ORC] Reorder callWrapperAsync and callSPSWrapperAsync parameters.
The callee address is now the first parameter and the 'SendResult' function
the second. This change improves consistentency with the non-async functions
where the callee is the first address and the return value the second.
2021-10-10 13:10:43 -07:00
Dávid Bolvanský 943b304848 Fixed some errors detected by PVS Studio 2021-10-09 17:27:41 +02:00
Dávid Bolvanský 3649fb14d1 Fixed some errors detected by PVS Studio 2021-10-09 17:20:04 +02:00
luxufan 02ac5e5cf1 [Orc] Fix global variable destructor function support when --jit-kind=orc-lazy
The bug was reported here https://bugs.llvm.org/show_bug.cgi?id=52030

This patch follows the idea that @lhames commented in the above webpage.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D110990
2021-10-09 15:58:21 +08:00
luxufan 590326382d [Orc] Support atexit in Orc(JITLink)
There is a bug reported at https://bugs.llvm.org/show_bug.cgi?id=48938

After looking through the glibc, I found the `atexit(f)` is the same as `__cxa_atexit(f, NULL, NULL)`. In orc runtime, we identify different JITDylib by their dso_handle value, so that a NULL dso_handle is invalid. So in this patch, I added a `PlatformJDDSOHandle` to ELFNixRuntimeState, and functions which are registered by atexit will be registered at PlatformJD.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D111413
2021-10-09 12:25:47 +08:00
Reid Kleckner 89b57061f7 Move TargetRegistry.(h|cpp) from Support to MC
This moves the registry higher in the LLVM library dependency stack.
Every client of the target registry needs to link against MC anyway to
actually use the target, so we might as well move this out of Support.

This allows us to ensure that Support doesn't have includes from MC/*.

Differential Revision: https://reviews.llvm.org/D111454
2021-10-08 14:51:48 -07:00
Lang Hames 8fe3d9df0e Revert "[ORC] Move SimpleRemoteEPCServer::Dispatcher into OrcShared."
This reverts commit dfd74db981.

SimpleRemoteEPC should share dispatch with the ExecutionSession, rather than
having two different dispatch systems on the controller side.
SimpleRemoteEPCServer::Dispatch doesn't need to be shared.
2021-10-08 13:43:42 -07:00
Lang Hames a129305b28 [ORC] Remove a stale comment.
SimpleRemoteEPCServer Service shutdown (c965fde7c2) takes care of this.
2021-10-08 13:43:42 -07:00
Lang Hames dfd74db981 [ORC] Move SimpleRemoteEPCServer::Dispatcher into OrcShared.
Renames SimpleRemoteEPCServer::Dispatcher to SimpleRemoteEPCDispatcher and
moves it into OrcShared. SimpleRemoteEPCServer::ThreadDispatcher is similarly
moved and renamed to DynamicThreadPoolSimpleRemoteEPCDispatcher.

This will allow these classes to be reused by SimpleRemoteEPC on the controller
side of the connection.
2021-10-08 11:29:57 -07:00
Simon Pilgrim e5fa68457a [ExecutionEngine] remove unused <string> includes 2021-10-07 12:07:15 +01:00
luxufan b384736b20 Revert "[JITLink][NFC] Add TableManager to replace PerGraph...Builder pass"
This reverts commit 50a278c2ae.
2021-10-06 21:34:18 +08:00
luxufan 50a278c2ae [JITLink][NFC] Add TableManager to replace PerGraph...Builder pass
This patch add a TableManager which reponsible for fixing edges that need entries to reference the target symbol and constructing such entries.

In the past, the PerGraphGOTAndPLTStubsBuilder pass was used to build GOT and PLT entry, and the PerGraphTLSInfoEntryBuilder pass was used to build TLSInfo entry. By generalizing the behavior of building entry, I added a TableManager which could be reused when built GOT, PLT and TLSInfo entries.

If this patch makes sense and can be accepted, I will apply the TableManager to other targets(MachO_x86_64, MachO_arm64, ELF_riscv), and delete the file PerGraphGOTAndPLTStubsBuilder.h

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D110383
2021-10-06 21:24:34 +08:00
Simon Pilgrim 21661607ca [llvm] Replace report_fatal_error(std::string) uses with report_fatal_error(Twine)
As described on D111049, we're trying to remove the <string> dependency from error handling and replace uses of report_fatal_error(const std::string&) with the Twine() variant which can be forward declared.
2021-10-06 12:04:30 +01:00
Simon Pilgrim 2e5daac217 [llvm] Update report_fatal_error calls from raw_string_ostream to use Twine(OS.str())
As described on D111049, we're trying to remove the <string> dependency from error handling and replace uses of report_fatal_error(const std::string&) with the Twine() variant which can be forward declared.

We can use the raw_string_ostream::str() method to perform the implicit flush() and return a reference to the std::string container that we can then wrap inside Twine().
2021-10-05 18:42:12 +01:00
Lang Hames 33dd98e9e4 [ORC] Remove ORC RPC.
With the removal of OrcRPCExecutorProcessControl and OrcRPCTPCServer in
6aeed7b19c the ORC RPC library no longer has any in-tree users.

Clients needing serialization for ORC should move to Simple Packed
Serialization (usually by adopting SimpleRemoteEPC for remote JITing).
2021-10-01 11:17:33 -07:00
Lang Hames 1f2f1a4414 [JITLink][MachO][x86-64] Add support for splitting compact-unwind sections.
Follow-up to fc734da795 to enable compact-unwind splitting on x86-64.
2021-09-28 19:12:56 -07:00
Lang Hames fc734da795 [JITLink][MachO][arm64] Add support for splitting compact-unwind sections.
CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions back to their respective records.

In MachO_arm64.cpp, a CompactUnwindSplitter pass is added to the pre-prune pass
list when setting up the standard pipeline.

This patch does not provide runtime support for compact-unwind, but is a first
step towards enabling it.
2021-09-28 17:29:05 -07:00
Lang Hames 21a06254a3 [ORC] Switch from JITTargetAddress to ExecutorAddr for EPC-call APIs.
Part of the ongoing move to ExecutorAddr.
2021-09-27 16:53:09 -07:00
Lang Hames 6fe2e9a9cc [ORC] Hold shared_ptr<SymbolStringPool> in errors containing SymbolStringPtrs.
This allows these error values to remain valid, even if they tear down the JIT
itself.
2021-09-27 15:46:56 -07:00
Lang Hames 1ea8d12510 [ORC] Add missing lock to CompileOnDemandLayer::getPerDylibResources.
The getPerDylibResources method may be called concurrently from multiple
threads, so we need to protect access to the underlying map.

Possible for fix https://llvm.org/PR51064
2021-09-26 18:35:58 -07:00
Lang Hames 4b37462aab [ORC] Fix SimpleRemoteEPC data races.
Adds a 'start' method to SimpleRemoteEPCTransport to defer transport startup
until the client has been configured. This avoids races on client members if the
first messages arrives while the client is being configured.

Also fixes races on the file descriptors in FDSimpleRemoteEPCTransport.
2021-09-26 18:11:48 -07:00
Lang Hames 6498b0e991 Reintroduce "[ORC] Introduce EPCGenericRTDyldMemoryManager."
This reintroduces "[ORC] Introduce EPCGenericRTDyldMemoryManager."
(bef55a2b47) and "[lli] Add ChildTarget dependence
on OrcTargetProcess library." (7a219d801b) which were
reverted in 99951a5684 due to bot failures.

The root cause of the bot failures should be fixed by "[ORC] Fix uninitialized
variable." (0371049277) and "[ORC] Wait for
handleDisconnect to complete in SimpleRemoteEPC::disconnect."
(320832cc9b).
2021-09-27 03:24:33 +10:00
Lang Hames 175c1a39e8 [ORC][llvm-jitlink] Add debugging output to SimpleRemoteEPC (and Server).
Also adds an optional 'debug' argument to the llvm-jitlink-executor tool to
enable debug-logging.
2021-09-26 10:00:29 -07:00
Lang Hames 320832cc9b [ORC] Wait for handleDisconnect to complete in SimpleRemoteEPC::disconnect.
Disconnect should block until handleDisconnect completes, otherwise we might
destroy the SimpleRemoteEPC instance while it's still in use.

Thanks to Dave Blaikie for helping me track this down.
2021-09-26 10:19:26 +10:00
Lang Hames 0371049277 [ORC] Fix uninitialized variable.
Spotted by Dave Blaikie. Thanks Dave!
2021-09-26 10:19:25 +10:00
Lang Hames 99951a5684 Revert "[ORC] Introduce EPCGenericRTDyldMemoryManager."
This reverts commit bef55a2b47 while I investigate
failures on some bots. Also reverts "[lli] Add ChildTarget dependence on
OrcTargetProcess library." (7a219d801b) which was
a fallow-up to bef55a2b47.
2021-09-25 11:19:14 -07:00
Lang Hames bef55a2b47 [ORC] Introduce EPCGenericRTDyldMemoryManager.
EPCGenericRTDyldMemoryMnaager is an EPC-based implementation of the
RuntimeDyld::MemoryManager interface. It enables remote-JITing via EPC (backed
by a SimpleExecutorMemoryManager instance on the executor side) for RuntimeDyld
clients.

The lli and lli-child-target tools are updated to use SimpleRemoteEPC and
SimpleRemoteEPCServer (rather than OrcRemoteTargetClient/Server), and
EPCGenericRTDyldMemoryManager for MCJIT tests.

By enabling remote-JITing for MCJIT and RuntimeDyld-based ORC clients,
EPCGenericRTDyldMemoryManager allows us to deprecate older remote-JITing
support, including OrcTargetClient/Server, OrcRPCExecutorProcessControl, and the
Orc RPC system itself. These will be removed in future patches.
2021-09-25 10:42:10 -07:00
Lang Hames ef391df2b6 [ORC] Rename ExecutorAddress to ExecutorAddr.
Removing the 'ess' suffix improves the ergonomics without sacrificing clarity.
Since this class is likely to be used more frequently in the future it's worth
some short term pain to fix this now.
2021-09-23 20:35:17 -07:00
Lang Hames a2c1cf09df [ORC] Introduce EPCGenericDylibManager / SimpleExecutorDylibManager.
EPCGenericDylibManager provides an interface for loading dylibs and looking up
symbols in the executor, implemented using EPC-calls to functions in the
executor.

SimpleExecutorDylibManager is an executor-side service that provides the
functions used by EPCGenericDylibManager.

SimpleRemoteEPC is updated to use an EPCGenericDylibManager instance to
implement the ExecutorProcessControl loadDylib and lookup methods. In a future
commit these methods will be removed, and clients updated to use
EPCGenericDylibManagers directly.
2021-09-23 19:59:35 -07:00
Lang Hames c965fde7c2 [ORC] Shut down services in SimpleRemoteEPCServer.
This should have been included with ExecutorBootstrapService in 78b083dbb7,
but was accidentally left out. It give services a chance to release any
resources that they have acquired.
2021-09-23 16:27:28 -07:00
Lang Hames 2ce73f13c9 [ORC] Fix file header. 2021-09-23 15:48:08 -07:00
Stefan Gränitz 767b328e50 [ORC] Minor renaming and typo fixes (NFC)
Two typos, one unsused include and some leftovers from the TargetProcessControl -> ExecutorProcessControl renaming

Reviewed By: xgupta

Differential Revision: https://reviews.llvm.org/D110260
2021-09-23 21:33:34 +02:00
Geoffrey Martin-Noble 01b097afd0
Fix bad merge the removed switch case
When https://reviews.llvm.org/D109520 was landed, it reverted the addition of this switch
case added in https://reviews.llvm.org/D109293. This caused `-Wswitch` failures (and
presumably broke the functionality added in the latter patch).
2021-09-20 10:58:58 -07:00
Stefan Gränitz e8d81d80f6 [JITLink] Adopt forEachRelocation() helper in ELF RISCV backend (NFC)
Following D109516, this patch re-uses the new helper function for ELF relocation traversal in the RISCV backend.

Reviewed By: StephenFan

Differential Revision: https://reviews.llvm.org/D109522
2021-09-20 15:46:41 +02:00
Stefan Gränitz 68914dc990 [JITLink] Adopt forEachRelocation() helper in ELF x86-64 backend (NFC)
Following D109516, this patch re-uses the new helper function for ELF relocation traversal in the x86-64 backend.

Reviewed By: StephenFan

Differential Revision: https://reviews.llvm.org/D109520
2021-09-20 15:46:41 +02:00