Commit Graph

222 Commits

Author SHA1 Message Date
Lang Hames 0d8ae1e343 Reapply r360194 "[JITLink] Add support for MachO .alt_entry atoms." with fixes.
This patch modifies MachOAtomGraphBuilder to use setLayoutNext rather than
addEdge, and fixes a bug in the section layout algorithm that could result in
atoms appearing more than once in the section ordering (which resulted in those
atoms being assigned invalid addresses during layout).

llvm-svn: 360205
2019-05-07 22:56:40 +00:00
Lang Hames 1a10101e21 Revert r360194 "[JITLink] Add support for MachO .alt_entry atoms."
The testcase is asserting on some bots - reverting while I investigate.

llvm-svn: 360200
2019-05-07 22:19:29 +00:00
Lang Hames 2b09b25e48 [JITLink] Add support for MachO .alt_entry atoms.
The MachO .alt_entry directive is applied to a symbol to indicate that it is
locked (in terms of address layout and liveness) to its predecessor atom. I.e.
it is an alternate entry point, at a fixed offset, for the previous atom.

This patch updates MachOAtomGraphBuilder to check for the .alt_entry flag on
symbols and add a corresponding LayoutNext edge to the atom-graph. It also
updates MachOAtomGraphBuilder_x86_64 to generalize handling of the
X86_64_RELOC_SUBTRACTOR relocation: previously either the minuend or
subtrahend of the subtraction had to be the same as the atom being fixed up,
now it is only necessary for the minuend or subtrahend to be locked (via any
chain of alt_entry directives) to the atom being fixed up.

llvm-svn: 360194
2019-05-07 21:35:14 +00:00
Lang Hames ce8255f3e2 [JITLink] Add two useful Section operations: find by name, get address range.
These operations were already used in eh-frame registration, and are likely to
be used in other runtime registrations, so this commit moves them into a header
where they can be re-used.

llvm-svn: 359950
2019-05-04 00:23:09 +00:00
Lang Hames 3181b87cb6 [JITLink] Make sure we explicitly deallocate memory on failure.
JITLinkGeneric phases 2 and 3 (focused on applying fixups and finalizing memory,
respectively) may fail for various reasons. If this happens, we need to
explicitly de-allocate the memory allocated in phase 1 (explicitly, because
deallocation may also fail and so is implemented as a method returning error).

No testcase yet: I am still trying to decide on the right way to test totally
platform agnostic code like this.

llvm-svn: 359643
2019-05-01 02:43:52 +00:00
Lang Hames d407b4b980 [JITLink] Add debugging output to print resolved external atoms.
llvm-svn: 359614
2019-04-30 21:28:07 +00:00
Lang Hames 88816bdd2f [ORC][JITLink] Name in-memory compiled objects after their source modules.
In-memory compiled object buffer identifiers will now be derived from the
identifiers of their source IR modules. This makes it easier to connect
in-memory objects with their source modules in debugging output.

llvm-svn: 359613
2019-04-30 21:27:56 +00:00
Lang Hames a9fdf375b3 [ORC] Add a 'plugin' interface to ObjectLinkingLayer for events/configuration.
ObjectLinkingLayer::Plugin provides event notifications when objects are loaded,
emitted, and removed. It also provides a modifyPassConfig callback that allows
plugins to modify the JITLink pass configuration.

This patch moves eh-frame registration into its own plugin, and teaches
llvm-jitlink to only add that plugin when performing execution runs on
non-Windows platforms. This should allow us to re-enable the test case that was
removed in r359198.

llvm-svn: 359357
2019-04-26 22:58:39 +00:00
Lang Hames b1ba4d8a8a [JITLink] Refer to FDE's CIE (not the most recent CIE) when parsing eh-frame.
Frame Descriptor Entries (FDEs) have a pointer back to a Common Information
Entry (CIE) that describes how the rest FDE should be parsed. JITLink had been
assuming that FDEs always referred to the most recent CIE encountered, but the
spec allows them to point back to any previously encountered CIE. This patch
fixes JITLink to look up the correct CIE for the FDE.

The testcase is a MachO binary with an FDE that refers to a CIE that is not the
one immediately proceeding it (the layout can be viewed wit
'dwarfdump --eh-frame <testcase>'. This test case had to be a binary as llvm-mc
now sorts FDEs (as of r356216) to ensure FDEs *do* point to the most recent CIE.

llvm-svn: 359105
2019-04-24 15:15:55 +00:00
Simon Pilgrim ddd225d1a9 Fix MSVC "32-bit shift implicitly converted to 64 bits" warning. NFCI.
llvm-svn: 358970
2019-04-23 11:16:16 +00:00
Lang Hames 1233c15be5 [JITLink] Remove a lot of reduntant 'JITLink_' prefixes. NFC.
llvm-svn: 358869
2019-04-22 03:03:09 +00:00
Lang Hames d3dac47aa2 [JITLink] Fix section start address calculation in eh-frame recorder.
Section atoms are not sorted, so we need to scan the whole section to find the
start address.

No test case: Found by inspection, and any reproduction would depend on pointer
ordering.

llvm-svn: 358865
2019-04-22 01:35:16 +00:00
Lang Hames a97032e947 [JITLink] Remove an overly strict error check in JITLink's eh-frame parser.
The error check required FDEs to refer to the most recent CIE, but the eh-frame
spec allows them to refer to any previously seen CIE. This patch removes the
offending check.

llvm-svn: 358840
2019-04-21 04:48:32 +00:00
Lang Hames 0191531a76 [JITLink] Factor basic common GOT and stub creation code into its own class.
llvm-svn: 358838
2019-04-21 03:14:42 +00:00
Lang Hames 65e1ddd713 [JITLink] Add yet more detail to MachO/x86-64 unsupported relocation errors.
Knowing the address/symbolnum field values makes it easier to identify the
unsupported relocation, and provides enough information for the full bit
pattern of the relocation to be reconstructed.

llvm-svn: 358833
2019-04-20 22:59:43 +00:00
Lang Hames 7f77a231fa [JITLink] Fix a bad formatv format string.
llvm-svn: 358831
2019-04-20 22:06:12 +00:00
Lang Hames daed9b10f1 [JITLink] Add BinaryFormat to JITLink's dependencies.
Hopefully this will fix the missing dependence on llvm::identify_magic that is
showing up on some PPC bots. E.g.

http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/9617

llvm-svn: 358827
2019-04-20 19:48:45 +00:00
Lang Hames c283fc5ebb [JITLink] Add more detail to MachO/x86-64 "unsupported relocation" errors.
The extra information here will be helpful in diagnosing errors, like the
ones currently occuring on the PPC big-endian bots. :)

llvm-svn: 358826
2019-04-20 18:50:13 +00:00
Lang Hames dfc3a4f6ff [JITLink] Silence some MSVC implicit cast warnings.
llvm-svn: 358824
2019-04-20 18:30:16 +00:00
Lang Hames b39109585a [JITLink] Use memset instead of bzero.
llvm-svn: 358822
2019-04-20 17:49:58 +00:00
Lang Hames 68b0b8c192 [JITLink] Fix a missing header and bad prototype.
llvm-svn: 358819
2019-04-20 17:29:57 +00:00
Lang Hames 11c8dfa583 Initial implementation of JITLink - A replacement for RuntimeDyld.
Summary:

JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.

JITLink aims to improve on RuntimeDyld in several ways:

(1) A clear design intended to maximize code-sharing while minimizing coupling.

RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.

(2) Support for native code models.

RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.

(3) Native support for asynchronous linking.

JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.

To maximise sharing, the design has a hierarchy of common code:

(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
 |  memory allocation) that are intended to be shared by all architectures.
 |
 + -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
       |  atom-graph parsing.
       |
       + -- (3) Architecture specific code that uses (1) and (2). E.g.
                JITLinkerMachO_x86_64, which adds x86-64 specific relocation
                support to (2) to build and patch up the atom graph.

To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:

  using JITLinkAsyncLookupContinuation =
      std::function<void(Expected<AsyncLookupResult> LR)>;

  using JITLinkAsyncLookupFunction =
      std::function<void(const DenseSet<StringRef> &Symbols,
                         JITLinkAsyncLookupContinuation LookupContinuation)>;

  using FinalizeContinuation = std::function<void(Error)>;

  virtual void finalizeAsync(FinalizeContinuation OnFinalize);

In addition to its headline features, JITLink also makes other improvements:

  - Dead stripping support: symbols that are not used (e.g. redundant ODR
    definitions) are discarded, and take up no memory in the target process
    (In contrast, RuntimeDyld supported pointer equality for weak definitions,
    but the redundant definitions stayed resident in memory).

  - Improved exception handling support. JITLink provides a much more extensive
    eh-frame parser than RuntimeDyld, and is able to correctly fix up many
    eh-frame sections that RuntimeDyld currently (silently) fails on.

  - More extensive validation and error handling throughout.

This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.

Differential Revision: https://reviews.llvm.org/D58704

llvm-svn: 358818
2019-04-20 17:10:34 +00:00