PointerToGOT lowering was accidentally changed from Delta32 to Delta64 in
db37225803. This patch moves it back to Delta32 and renames the generic
aarch64 edge to Delta32ToGOT to avoid the ambiguity.
No test case yet -- I haven't figured out how to write a succinct test case
(this typically appears in CIEs in eh-frames).
Unifies GOT/PLT table managers of ELF and MachO on aarch64 architecture. Additionally, it migrates table managers from PerGraphGOTAndPLTStubsBuilder to generic crtp TableManager.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D127558
This patch moves the aarch64 fixup logic from the MachO/arm64 backend to
aarch64.h header so that it can be re-used in the ELF/aarch64 backend. This
significantly expands relocation support in the ELF/aarch64 backend.
Reviewed By: lhames, sgraenitz
Differential Revision: https://reviews.llvm.org/D126286
It's idiomatic to require that plugins (especially platform plugins) be
installed to handle special edge kinds. If the plugins are not installed and an
object is loaded that uses one of the special edge kinds then we want to error
out rather than asserting.
Adds support for pointer encodings commonly used in large/static models,
including non-pcrel, sdata/udata8, indirect, and omit.
Also refactors pointer-encoding handling to consolidate error generation inside
common functions, rather than callees of those functions.
With 229d576b31 the class EHFrameSplitter was renamed to DWARFRecordSectionSplitter. This change merely moves it to it's own .cpp/.h file
Differential Revision: https://reviews.llvm.org/D121721
EHFrameSplitter does the exact same work to split up the eh_frame as it would need for any section that follows the DWARF record, therefore this patch just changes the name of it to DWARFRecordSectionSplitter to be more general.
Differential Revision: https://reviews.llvm.org/D121486
This reapplies 253ce92844, which was reverted in
66b2ed477f due to bot failures.
I have added the `-phony-externals` option added, which should fix the
unresolved symbol errors.
This is needed for DWARF eh-frame exception handling on AArch64.
https://github.com/llvm/llvm-project/issues/52921.
Original patch by David Nadlinger <code@klickverbot.at> (thanks David!),
testcase and comments added by me.
This consistent with ld64's treatment of the GOT, but the main aim here is a
short-term workaround for a bad interaction between stub code sequences and
memory layout: Stubs use LDRLiteral19 relocations to reference the GOT, but
BasicLayout currently puts RW- segments between R-- and R-X segments -- a large
RW- segment (or a large R-- for that matter) can cause the relocation to fail
with an out-of-range error.
Putting the GOT in R-X fixes this efficiently in practice. A more robust fix
will be to use a longer code sequence to materialize the GOT pointer and then
rewrite the stub to use a shorter sequence where possible.
This re-applies 133f86e954, which was reverted in
c5965a411c while I investigated bot failures.
The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.
Only search within the requested section, and allow one-past-then-end addresses.
This is needed to support section-end-address references to sections with no
symbols in them.
Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are just handled as GOT accesses.
rdar://84671534
Differential Revision: https://reviews.llvm.org/D112656
Negative deltas for LDRLiteral19 have their high bits set. If these bits aren't
masked out then they will overwrite other instruction bits, leading to a bogus
encoding.
This long-standing relocation bug was exposed by e50aea58d5, "[JITLink][ORC]
Major JITLinkMemoryManager refactor.", which caused memory layouts to be
reordered, which in turn lead to a previously unseen negative delta. (Unseen
because LDRLiteral19s were only created in JITLink passes where they always
pointed at segments that were layed-out-after in the old layout).
No testcase yet: Our existing regression test infrastructure is good at checking
that operand bits are correct, but provides no easy way to test for bad opcode
bits. I'll have a think about the right way to approach this.
https://llvm.org/PR52153
Adds explicit narrowing casts to JITLinkMemoryManager.cpp.
Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.
This effectively reverts commit 6641d29b70.
This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.
Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.
To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).
In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).
Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.
Major features:
(1) More asynchrony:
The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.
(2) Improved control over graph address layout:
Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.
(3) Efficient tracking of allocated memory.
The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.
Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.
(4) Support for "allocation actions" and finalize-lifetime memory.
Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as
((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);
At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).
The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.
These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.
Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.
CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions back to their respective records.
In MachO_arm64.cpp, a CompactUnwindSplitter pass is added to the pre-prune pass
list when setting up the standard pipeline.
This patch does not provide runtime support for compact-unwind, but is a first
step towards enabling it.
During the generic x86-64 support refactor in ecf6466f01 the implementation
of MachO_arm64_GOTAndStubsBuilder::isGOTEdgeToFix was altered to only return
true for external symbols. This behavior is incorrect: GOT entries may be
required for defined symbols (e.g. in the large code model).
This patch fixes the bug and adds a test case for it (renaming an old test
case to avoid any ambiguity).
This patch introduces new operations on jitlink::Blocks: setMutableContent,
getMutableContent and getAlreadyMutableContent. The setMutableContent method
will set the block content data and size members and flag the content as
mutable. The getMutableContent method will return a mutable copy of the existing
content value, auto-allocating and populating a new mutable copy if the existing
content is marked immutable. The getAlreadyMutableMethod asserts that the
existing content is already mutable and returns it.
setMutableContent should be used when updating the block with totally new
content backed by mutable memory. It can be used to change the size of the
block. The argument value should *not* be shared with any other block.
getMutableContent should be used when clients want to modify the existing
content and are unsure whether it is mutable yet.
getAlreadyMutableContent should be used when clients want to modify the existing
content and know from context that it must already be immutable.
These operations reduce copy-modify-update boilerplate and unnecessary copies
introduced when clients couldn't me sure whether the existing content was
mutable or not.
Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builder, but may also be used by pass writers who
want to create pointer stubs for indirection.
This patch also switches the underlying type for LinkGraph content from
StringRef to ArrayRef<char>. This avoids any confusion when working with buffers
that contain null bytes in the middle like, for example, a newly added null
pointer content array. ;)
Introduces DefineExternalSectionStartAndEndSymbols.h, which defines a template
for a JITLink pass that transforms external symbols meeting a user-supplied
predicate into defined symbols pointing at the start and end of a Section
identified by the predicate. JITLink.h is updated with a new makeAbsolute
function to support this pass.
Also renames BasicGOTAndStubsBuilder to PerGraphGOTAndPLTStubsBuilder -- the new
name better describes the intent of this GOT and PLT stubs builder, and will
help to distinguish it from future GOT and PLT stub builders that build entries
that may be shared between multiple graphs.
This patch introduces generic x86-64 edge kinds, and refactors the MachO/x86-64
backend to use these edge kinds. This simplifies the implementation of the
MachO/x86-64 backend and makes it possible to write generic x86-64 passes and
utilities.
The new edge kinds are different from the original set used in the MachO/x86-64
backend. Several edge kinds that were not meaningfully distinguished in that
backend (e.g. the PCRelMinusN edges) have been merged into single edge kinds in
the new scheme (these edge kinds can be reintroduced later if we find a use for
them). At the same time, new edge kinds have been introduced to convey extra
information about the state of the graph. E.g. The Request*AndTransformTo**
edges represent GOT/TLVP relocations prior to synthesis of the GOT/TLVP
entries, and the 'Relaxable' suffix distinguishes edges that are candidates for
optimization from edges which should be left as-is (e.g. to enable runtime
redirection).
ELF/x86-64 will be refactored to use these generic edges at some point in the
future, and I anticipate a similar refactor to create a generic arm64 support
header too.
Differential Revision: https://reviews.llvm.org/D98305
This makes the target triple, graph name, and full graph content available
when making decisions about how to populate the linker pass pipeline.
Also updates the LLJITWithObjectLinkingLayerPlugin example to show more
API use, including use of the API changes in this patch.
Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is updated to support emission
of raw LinkGraphs in addition to object buffers.
Raw LinkGraphs can be created by in-memory compilers to bypass object encoding /
decoding (though this prevents caching, as LinkGraphs have do not have an
on-disk representation), and by utility code to add programatically generated
data structures to the JIT target process.
Correctly sign extend the addend, and fix implicit shift operand decoding
(it incorrectly returned 0 for some cases), and check that the initial
encoded immediate is 0.
Identify relocations by (section name, offset) pairs, rather than plain
vmaddrs. This makes it easier to cross-reference debugging output for
relocations with output from standard object inspection tools (otool,
readelf, objdump, etc.).
When processing a MachO SUBTRACTOR/UNSIGNED pair, if the UNSIGNED target
is non-extern then check the r_symbolnum field of the relocation to find
the targeted section and use the section's address to find 'ToSymbol'.
Previously 'ToSymbol' was found by loading the initial value stored at
the fixup location and treating this as an address to search for. This
is incorrect, however: the initial value includes the addend and will
point to the wrong block if the addend is less than zero or greater than
the block size.
rdar://65756694
Debug sections will not be linked into the final executable and may contain
ambiguous relocations*. Skipping them avoids both some unnecessary processing
cost and the hassle of dealing with the problematic relocations.
* E.g. __debug_ranges contains non-extern relocations to the end of functions
hat begin with named symbols. Under the usual rules for interpreting non-extern
relocations these will be incorrectly associated with the following block, or
no block at all (if there is a gap between one block and the next).
This removes the conditional layout of relocation_info bitfields that was
introduced in 3ccd677bf (svn r358839). The platform relocation_info
struct (defined in usr/include/mach-o/reloc.h) does not define the layout of
this struct differently on big-endian platforms and we want to keep the LLVM
and platform definitions in sync.
To fix the bug that 3ccd677bf addressed this patch modifies JITLink to construct
its relocation_info structs from the raw relocation words using shift and mask
operations.
Global symbols with linker-private prefixes should be resolvable across object
boundaries, but internal symbols with linker-private prefixes should not.
Some targets (E.g. MachO/arm64) use relocations to fix some CFI record fields
in the eh-frame section. When relocations are used the initial (pre-relocation)
content of the eh-frame section can no longer be interpreted by following the
eh-frame specification. This causes errors in the existing eh-frame parser.
This patch moves eh-frame handling into two LinkGraph passes that are run after
relocations have been parsed (but before they are applied). The first] pass
breaks up blocks in the eh-frame section into per-CFI-record blocks, and the
second parses blocks of (potentially multiple) CFI records and adds the
appropriate edges to any CFI fields that do not have existing relocations.
These passes can be run independently of one another. By handling eh-frame
splitting/fixing with LinkGraph passes we can both re-use existing relocations
for CFI record fields and avoid applying eh-frame fixups before parsing the
section (which would complicate the linker and require extra temporary
allocations of working memory).
This implementation has support for all relocation types except TLV.
Compact unwind sections are not yet supported, so exceptions/unwinding will not
work.
llvm-svn: 374476
Lang Hames