Commit Graph

686 Commits

Author SHA1 Message Date
Xing GUO ff6a0b6a8e [Object] Change ObjectFile::getSymbolValue() return type to Expected<uint64_t>
Summary:
In D77860, we have changed `getSymbolFlags()` return type to `Expected<uint32_t>`.
This change helps bubble the error further up the stack.

Reviewers: jhenderson, grimar, JDevlieghere, MaskRay

Reviewed By: jhenderson

Subscribers: hiraditya, MaskRay, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79075
2020-05-02 14:04:44 +08:00
Simon Pilgrim 39abe45951 RuntimeDyldELF.h - make the object namespace explicit for ELFObjectFileBase. NFC. 2020-04-23 13:52:49 +01:00
Simon Pilgrim 9202902ae6 RuntimeDyldImpl.h - remove unused Twine forward declaration. NFC. 2020-04-22 15:07:17 +01:00
vgxbj ac00376a13 [Object] Change uint32_t getSymbolFlags() to Expected<uint32_t> getSymbolFlags().
This change enables getSymbolFlags() to return errors which benefit error reporting in clients.

Differential Revision: https://reviews.llvm.org/D77860
2020-04-18 21:27:57 +08:00
Benjamin Kramer e64e516790 [RuntimeDyld] Replace hand-rolled endian conversion with support::endian. NFCI. 2020-04-06 13:22:53 +02:00
Lang Hames 39253a50f0 [ORC] Re-apply 98f2bb4461, enable JITEventListeners in OrcV2, with fixes.
Updates the object buffer ownership scheme in jitLinkForOrc and related
functions: Ownership of both the object::ObjectFile and underlying
MemoryBuffer is passed into jitLinkForOrc and passed back to the onEmit
callback once linking is complete. This avoids the use-after-free errors
that were seen in 98f2bb4461.
2020-03-19 16:30:08 -07:00
Fangrui Song 536ba6373f [Object] Change ELFObjectFile<ELFT>::getFileFormatName() to use BFD names
Follow-up for D74433

What the function returns are almost standard BFD names, except that "ELF" is
in uppercase instead of lowercase.

This patch changes "ELF" to "elf" and changes ARM/AArch64 to use their BFD names.
MIPS and PPC64 have endianness differences as well, but this patch does not intend to address them.

Advantages:

* llvm-objdump: the "file format " line matches GNU objdump on ARM/AArch64 objects
* "file format " line can be extracted and fed into llvm-objcopy -O literally.
  (https://github.com/ClangBuiltLinux/linux/issues/779 has such a use case)

Affected tools: llvm-readobj, llvm-objdump, llvm-dwarfdump, MCJIT (internal implementation detail, not exposed)

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D76046
2020-03-16 07:42:04 -07:00
Lang Hames 337e131ca7 [RuntimeDyld][COFF] Build stubs for COFF dllimport symbols.
Summary:
Enables JIT-linking by RuntimeDyld of COFF objects that contain references to
dllimport symbols. This is done by recognizing symbols that start with the
reserved "__imp_" prefix and building a pointer entry to the target symbol in
the stubs area of the section. References to the "__imp_" symbol are updated to
point to this pointer.

Work in progress: The generic code is in place, but only RuntimeDyldCOFFX86_64
and RuntimeDyldCOFFI386 have been updated to look for and update references to
dllimport symbols.

Reviewers: compnerd

Subscribers: hiraditya, ributzka, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75884
2020-03-10 16:08:40 -07:00
Lang Hames aed5712522 [RuntimeDyld] Allow multi-line rtdyld-check and jitlink-check expressions.
This patch allows rtdyld-check / jitlink-check expressions to be extended over
multiple lines by terminating each line with a '\'. E.g.

  # llvm-rtdyld: *{8}X = \
  # llvm-rtdyld:   Y
  X:
    .quad Y

This will be used to break up some long lines in upcoming test cases.
2020-03-10 16:08:40 -07:00
serge-sans-paille cb06571a44 Avoid dangling reference on SectionList
Bug spotted by https://cookieplmonster.github.io/2020/02/01/emulator-bug-llvm-bug/

Basically, holding references to object inside a resized vector is a bad idea.

Differential Revision: https://reviews.llvm.org/D75110
2020-03-05 09:42:24 +01:00
Stefan Gränitz ec1e7faf65 [ExecutionEngine] Add JITSymbolFlags::fromSummary(GlobalValueSummary*)
Summary: A function that creates JITSymbolFlags from a GlobalValueSummary. Similar functions exist: fromGlobalValue(), fromObjectSymbol()

Reviewers: lhames

Reviewed By: lhames

Subscribers: hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75082
2020-02-29 11:54:03 +00:00
Bill Wendling c55cf4afa9 Revert "Remove redundant "std::move"s in return statements"
The build failed with

  error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a793.
2020-02-10 07:07:40 -08:00
Bill Wendling 1c2241a793 Remove redundant "std::move"s in return statements 2020-02-10 06:39:44 -08:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Fangrui Song 6fdd6a7b3f [Disassembler] Delete the VStream parameter of MCDisassembler::getInstruction()
The argument is llvm::null() everywhere except llvm::errs() in
llvm-objdump in -DLLVM_ENABLE_ASSERTIONS=On builds. It is used by no
target but X86 in -DLLVM_ENABLE_ASSERTIONS=On builds.

If we ever have the needs to add verbose log to disassemblers, we can
record log with a member function, instead of passing it around as an
argument.
2020-01-11 13:34:52 -08:00
Tom Stellard ab411801b8 [cmake] Explicitly mark libraries defined in lib/ as "Component Libraries"
Summary:
Most libraries are defined in the lib/ directory but there are also a
few libraries defined in tools/ e.g. libLLVM, libLTO.  I'm defining
"Component Libraries" as libraries defined in lib/ that may be included in
libLLVM.so.  Explicitly marking the libraries in lib/ as component
libraries allows us to remove some fragile checks that attempt to
differentiate between lib/ libraries and tools/ libraires:

1. In tools/llvm-shlib, because
llvm_map_components_to_libnames(LIB_NAMES "all") returned a list of
all libraries defined in the whole project, there was custom code
needed to filter out libraries defined in tools/, none of which should
be included in libLLVM.so.  This code assumed that any library
defined as static was from lib/ and everything else should be
excluded.

With this change, llvm_map_components_to_libnames(LIB_NAMES, "all")
only returns libraries that have been added to the LLVM_COMPONENT_LIBS
global cmake property, so this custom filtering logic can be removed.
Doing this also fixes the build with BUILD_SHARED_LIBS=ON
and LLVM_BUILD_LLVM_DYLIB=ON.

2. There was some code in llvm_add_library that assumed that
libraries defined in lib/ would not have LLVM_LINK_COMPONENTS or
ARG_LINK_COMPONENTS set.  This is only true because libraries
defined lib lib/ use LLVMBuild.txt and don't set these values.
This code has been fixed now to check if the library has been
explicitly marked as a component library, which should now make it
easier to remove LLVMBuild at some point in the future.

I have tested this patch on Windows, MacOS and Linux with release builds
and the following combinations of CMake options:

- "" (No options)
- -DLLVM_BUILD_LLVM_DYLIB=ON
- -DLLVM_LINK_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_BUILD_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_LINK_LLVM_DYLIB=ON

Reviewers: beanz, smeenai, compnerd, phosek

Reviewed By: beanz

Subscribers: wuzish, jholewinski, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, mgorny, mehdi_amini, sbc100, jgravelle-google, hiraditya, aheejin, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, steven_wu, rogfer01, MartinMosbeck, brucehoult, the_o, dexonsmith, PkmX, jocewei, jsji, dang, Jim, lenary, s.egerton, pzheng, sameer.abuasal, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70179
2019-11-21 10:48:08 -08:00
Martin Storsjö f67534afd6 [ExecutionEngine] Add a missing break to avoid warnings
This fixes buildbot errors since dc3ee33089.
2019-11-20 12:20:05 +02:00
Adam Kallai dc3ee33089 ExecutionEngine: add preliminary support for COFF ARM64
Differential Revision: https://reviews.llvm.org/D69434
2019-11-20 10:59:42 +02:00
George Rimar 2bf01dcbaa [llvm/Object] - Make ELFObjectFile::getRelocatedSection return Expected<section_iterator>
It returns just a section_iterator currently and have a report_fatal_error call inside.
This change adds a way to return errors and handle them on caller sides.

The patch also changes/improves current users and adds test cases.

Differential revision: https://reviews.llvm.org/D69167

llvm-svn: 375408
2019-10-21 11:06:38 +00:00
Guillaume Chatelet 18f805a7ea [Alignment][NFC] Remove unneeded llvm:: scoping on Align types
llvm-svn: 373081
2019-09-27 12:54:21 +00:00
Benjamin Kramer ce74c3b19f [Orc] Address the remaining move-capture FIXMEs
This required spreading unique_function a bit more, which I think is a
good thing.

llvm-svn: 371843
2019-09-13 11:35:33 +00:00
Guillaume Chatelet af11cc7eb5 [Alignment] Move OffsetToAlignment to Alignment.h
Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson

Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D67499

llvm-svn: 371742
2019-09-12 15:20:36 +00:00
Tim Northover f1c2892912 AArch64: support arm64_32, an ILP32 slice for watchOS.
This is the main CodeGen patch to support the arm64_32 watchOS ABI in LLVM.
FastISel is mostly disabled for now since it would generate incorrect code for
ILP32.

llvm-svn: 371722
2019-09-12 10:22:23 +00:00
Jonas Devlieghere 0eaee545ee [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013
2019-08-15 15:54:37 +00:00
George Rimar bcc00e1afb Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
Changes: no changes. A fix for the clang code will be landed right on top.

Original commit message:

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368826
2019-08-14 11:10:11 +00:00
George Rimar 468919e182 Revert r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
It broke clang BB: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/16455

llvm-svn: 368813
2019-08-14 08:56:55 +00:00
George Rimar a0c6a35714 [llvm/Object] - Convert SectionRef::getName() to return Expected<>
SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368812
2019-08-14 08:46:54 +00:00
Benjamin Kramer 3d5360a439 Replace llvm::MutexGuard/UniqueLock with their standard equivalents
All supported platforms have <mutex> now, so we don't need our own
copies any longer. No functionality change intended.

llvm-svn: 368149
2019-08-07 10:57:25 +00:00
Nick Desaulniers b380846a12 [RuntimeDyld] fix too-small-bitmask error
Summary:
This was flagged in https://www.viva64.com/en/b/0629/ under "Snippet No.
33".

It seems that this statement is doing the standard bitwise trick for
adjusting a value to have a specific alignment.

The issue is that getStubAlignment() returns an unsigned, while DataSize
is declared a uint64_t. The right hand side of the expression is not
extended to 64b before bitwise negation, resulting in the top half of
the mask being 0s, which is not correct for realignment.

Reviewers: lhames, MaskRay

Reviewed By: MaskRay

Subscribers: RKSimon, MaskRay, hiraditya, llvm-commits, srhines

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62227

llvm-svn: 362286
2019-06-01 04:51:26 +00:00
Lang Hames a100042b27 [RuntimeDyld] Update reserveAllocationSpace to account for stub padding.
This should fix the buildbot failures caused by r362139.

llvm-svn: 362151
2019-05-30 20:58:28 +00:00
Lang Hames 0e124b37bd [RuntimeDyld] Apply padding and alignment bumps to all sections with stubs, and
increase the MachO/x86-64 stub alignment to 8.

Stub alignment should be guaranteed for any section containing RuntimeDyld
stubs/GOT-entries. To do this we should pad and align all sections containing
stubs, not just code sections.

This commit also bumps the MachO/x86-64 stub alignment to 8, so that GOT entries
will be aligned.

llvm-svn: 362139
2019-05-30 19:59:20 +00:00
Lang Hames 23343c5d90 [RuntimeDyld][ARM] Fix an incorrect assertion condition.
Fixes https://llvm.org/PR42036

llvm-svn: 361782
2019-05-27 21:34:31 +00:00
Xing Xue 4246b75295 Disable EHFrameSupport in JITLink/RuntimeDyld on AIX
Summary:
EH Frames aren't supported on AIX with the system compiler, but the definition of HAVE_EHTABLE_SUPPORT misses this which causes linking problems on AIX. This patch updates the definition of HAVE_EHTABLE_SUPPORT in both JITLink and RuntimeDyld.

Author: daltenty

Reviewers: sfertile, xingxue, hubert.reinterpretcase

Reviewed By: xingxue

Subscribers: hiraditya, jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62203

llvm-svn: 361410
2019-05-22 17:41:27 +00:00
Fangrui Song e183340c29 Recommit [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
r360876 didn't fix 2 call sites in clang.

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360892
2019-05-16 13:24:04 +00:00
Hans Wennborg 4da9ff9fcf Revert r360876 "[Object] Change object::SectionRef::getContents() to return Expected<StringRef>"
It broke the Clang build, see llvm-commits thread.

> Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
>
> Follow-up of D61781.

llvm-svn: 360878
2019-05-16 12:08:34 +00:00
Fangrui Song a076ec54be [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360876
2019-05-16 11:33:48 +00:00
Lang Hames 23085ec36d [JITLink] Add a test for zero-filled content.
Also updates RuntimeDyldChecker and llvm-rtdyld to support zero-fill tests by
returning a content address of zero (but no error) for zero-fill atoms, and
treating loads from zero as returning zero.

llvm-svn: 360547
2019-05-12 22:26:33 +00:00
Lang Hames 11c8dfa583 Initial implementation of JITLink - A replacement for RuntimeDyld.
Summary:

JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.

JITLink aims to improve on RuntimeDyld in several ways:

(1) A clear design intended to maximize code-sharing while minimizing coupling.

RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.

(2) Support for native code models.

RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.

(3) Native support for asynchronous linking.

JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.

To maximise sharing, the design has a hierarchy of common code:

(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
 |  memory allocation) that are intended to be shared by all architectures.
 |
 + -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
       |  atom-graph parsing.
       |
       + -- (3) Architecture specific code that uses (1) and (2). E.g.
                JITLinkerMachO_x86_64, which adds x86-64 specific relocation
                support to (2) to build and patch up the atom graph.

To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:

  using JITLinkAsyncLookupContinuation =
      std::function<void(Expected<AsyncLookupResult> LR)>;

  using JITLinkAsyncLookupFunction =
      std::function<void(const DenseSet<StringRef> &Symbols,
                         JITLinkAsyncLookupContinuation LookupContinuation)>;

  using FinalizeContinuation = std::function<void(Error)>;

  virtual void finalizeAsync(FinalizeContinuation OnFinalize);

In addition to its headline features, JITLink also makes other improvements:

  - Dead stripping support: symbols that are not used (e.g. redundant ODR
    definitions) are discarded, and take up no memory in the target process
    (In contrast, RuntimeDyld supported pointer equality for weak definitions,
    but the redundant definitions stayed resident in memory).

  - Improved exception handling support. JITLink provides a much more extensive
    eh-frame parser than RuntimeDyld, and is able to correctly fix up many
    eh-frame sections that RuntimeDyld currently (silently) fails on.

  - More extensive validation and error handling throughout.

This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.

Differential Revision: https://reviews.llvm.org/D58704

llvm-svn: 358818
2019-04-20 17:10:34 +00:00
Lang Hames c7c1f21525 Simplify decoupling between RuntimeDyld/RuntimeDyldChecker, add 'got_addr' util.
This patch reduces the number of functions in the interface between RuntimeDyld
and RuntimeDyldChecker by combining "GetXAddress" and "GetXContent" functions
into "GetXInfo" functions that return a struct describing both the address and
content. The GetStubOffset function is also replaced with a pair of utilities,
GetStubInfo and GetGOTInfo, that fit the new scheme. For RuntimeDyld both of
these functions will return the same result, but for the new JITLink linker
(https://reviews.llvm.org/D58704) these will provide the addresses of PLT stubs
and GOT entries respectively.

For JITLink's use, a 'got_addr' utility has been added to the rtdyld-check
language, and the syntax of 'got_addr' and 'stub_addr' has been changed: both
functions now take two arguments, a 'stub container name' and a target symbol
name. For llvm-rtdyld/RuntimeDyld the stub container name is the object file
name and section name, separated by a slash. E.g.:

rtdyld-check: *{8}(stub_addr(foo.o/__text, y)) = y

For the upcoming llvm-jitlink utility, which creates stubs on a per-file basis
rather than a per-section basis, the container name is just the file name. E.g.:

jitlink-check: *{8}(got_addr(foo.o, y)) = y
llvm-svn: 358295
2019-04-12 18:07:28 +00:00
Nico Weber af5834596b gn build: Fix Windows builds after r357797
llvm-svn: 358004
2019-04-09 14:02:02 +00:00
Lang Hames d250238abd [RuntimeDyld] Fix an ambiguous make_unique call.
llvm-svn: 357950
2019-04-08 22:19:05 +00:00
Lang Hames 941f247d30 [RuntimeDyld] Decouple RuntimeDyldChecker from RuntimeDyld.
This will allow RuntimeDyldChecker (and rtdyld-check tests) to test a new JIT
linker: JITLink (https://reviews.llvm.org/D58704).

llvm-svn: 357947
2019-04-08 21:50:48 +00:00
Fangrui Song 6a0746a92f Change some StringRef::data() reinterpret_cast to bytes_begin() or arrayRefFromStringRef()
llvm-svn: 357852
2019-04-07 03:58:42 +00:00
Richard Trieu 108b892939 Add namespace to some types.
llvm-svn: 352725
2019-01-31 04:33:11 +00:00
Zachary Turner 3c35f774de [RuntimeDyld] Don't try to allocate sections with align 0.
ELF sections allow 0 for the alignment, which is specified to
be the same as 1.  However many clients do not expect this and
will behave poorly in the presence of a 0-aligned section (for
example by trying to modulo something by the section alignment).
We can be more polite by making sure that we always pass a
non-zero value to clients.

Differential Revision: https://reviews.llvm.org/D57482

llvm-svn: 352694
2019-01-30 23:52:32 +00:00
Yonghong Song 61bc1d7ed5 [RuntimeDyld] load all sections with ProcessAllSections
This patch tried to address the following use case.
  . bcc (https://github.com/iovisor/bcc) utilizes llvm JIT to
    compile for BTF target.
  . with -g, .BTF and .BTF.ext sections (BPF debug info)
    will be generated by LLVM.
  . .BTF does not have relocations and .BTF.ext has some
    relocations.
  . With ProcessAllSections, .BTF.ext is loaded by JIT dynamic linker
    and is available to application. But .BTF is not loaded.

The bcc application needs both .BTF.ext and .BTF for debugging
purpose, and .BTF is not loaded. This patch addressed this issue
by iterating over all sections and loading any missing
sections, after symbol/relocation processing in loadObjectImpl().

Signed-off-by: Yonghong Song <yhs@fb.com>

Differential Revision: https://reviews.llvm.org/D55943

llvm-svn: 352432
2019-01-28 21:35:23 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Nathan Lanza 893083ae5e Implement IMAGE_REL_AMD64_SECREL for RuntimeDyldCOFFX86_64
lldb on Windows uses the ExecutionEngine for expression evaluation
and hits the llvm_unreachable due to this relocation. Thus, implement
the relocation and add a test to verify it's function.

llvm-svn: 348904
2018-12-12 00:04:06 +00:00
Jonas Devlieghere 45eb84f340 [Support] Make error banner optional in logAllUnhandledErrors
In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
simplify the call sites.

llvm-svn: 346604
2018-11-11 01:46:03 +00:00
Lang Hames 776f1d50c8 [RuntimeDyld][COFF] Skip non-loaded sections when calculating ImageBase.
Non-loaded sections (whose unused load-address defaults to zero) should not
be taken into account when calculating ImageBase, or ImageBase will be
incorrectly set to 0.

Patch by Andrew Scheidecker. Thanks Andrew!

https://reviews.llvm.org/D51343

+        // The Sections list may contain sections that weren't loaded for
+        // whatever reason: they may be debug sections, and ProcessAllSections
+        // is false, or they may be sections that contain 0 bytes. If the
+        // section isn't loaded, the load address will be 0, and it should not
+        // be included in the ImageBase calculation.

llvm-svn: 344995
2018-10-23 01:36:33 +00:00
Lang Hames 98440293fb [ORC] Add partitioning support to CompileOnDemandLayer2.
CompileOnDemandLayer2 now supports user-supplied partition functions (the
original CompileOnDemandLayer already supported these).

Partition functions are called with the list of requested global values
(i.e. global values that currently have queries waiting on them) and have an
opportunity to select extra global values to materialize at the same time.

Also adds testing infrastructure for the new feature to lli.

llvm-svn: 343396
2018-09-29 23:49:57 +00:00
Lang Hames abeedf1812 [ORC] Add an asynchronous jit-link function, jitLinkForORC, to RuntimeDyld and
switch RTDyldObjectLinkingLayer2 to use it.

RuntimeDyld::loadObject is currently a blocking operation. This means that any
JIT'd code whose call-graph contains an embedded complete K graph will require
at least K threads to link, which precludes the use of a fixed sized thread
pool for concurrent JITing of arbitrary code (whatever K the thread-pool is set
at, any code with a K+1 complete subgraph will deadlock at JIT-link time).

To address this issue, this commmit introduces a function called jitLinkForORC
that uses continuation-passing style to pass the fix-up and finalization steps
to the asynchronous symbol resolver interface so that linking can be performed
without blocking.

llvm-svn: 343043
2018-09-25 22:57:44 +00:00
Lang Hames 079c0df47d Remove 'orc' namespace from MSVCErrorWorkarounds.h, fix some typos that were
breaking windows builds.

The 'orc' namespace was accidentally left in when the workarounds were moved
out of orc in r343011.

llvm-svn: 343025
2018-09-25 20:48:57 +00:00
Lang Hames ffa72ef903 Fix a missing includes and a use of the MSVC promise/future workaround that
were left out of r343011/r343012.

llvm-svn: 343022
2018-09-25 20:16:06 +00:00
Lang Hames adde5ba4b2 [ORC] Reapply r342939 with a fix for MSVC's promise/future restrictions.
llvm-svn: 343012
2018-09-25 19:48:46 +00:00
Lang Hames 99bfa95ae3 Revert "[ORC] Switch to asynchronous resolution in JITSymbolResolver."
This reverts commit r342939.

MSVC's promise/future implementation does not like types that are not default
constructible. Reverting while I figure out a solution.

llvm-svn: 342941
2018-09-25 04:54:03 +00:00
Lang Hames 0e5b60326e [ORC] Switch to asynchronous resolution in JITSymbolResolver.
Asynchronous resolution (where the caller receives a callback once the requested
set of symbols are resolved) is a core part of the new concurrent ORC APIs. This
change extends the asynchronous resolution model down to RuntimeDyld, which is
necessary to prevent deadlocks when compiling/linking on a fixed number of
threads: If RuntimeDyld's linking process were a blocking operation, then any
complete K-graph in a program will require at least K threads to link in the
worst case, as each thread would block waiting for all the others to complete.
Using callbacks instead allows the work to be passed between dependent threads
until it is complete.

For backwards compatibility, all existing RuntimeDyld functions will continue
to operate in blocking mode as before. This change will enable the introduction
of a new async finalization process in a subsequent patch to enable asynchronous
JIT linking.

llvm-svn: 342939
2018-09-25 04:43:38 +00:00
Lang Hames 6cadc7c06b [ORC] Replace lookupFlags in JITSymbolResolver with getResponsibilitySet.
The new method name/behavior more closely models the way it was being used.
It also fixes an assertion that can occur when using the new ORC Core APIs,
where flags alone don't necessarily provide enough context to decide whether
the caller is responsible for materializing a given symbol (which was always
the reason this API existed).

The default implementation of getResponsibilitySet uses lookupFlags to determine
responsibility as before, so existing JITSymbolResolvers should continue to
work.

llvm-svn: 340874
2018-08-28 21:18:05 +00:00
Lang Hames 8e296229b5 [RuntimeDyld] Fix a bug in RuntimeDyld::loadObjectImpl that was over-allocating
space for common symbols.

Patch by Dmitry Sidorov. Thanks Dmitry!

Differential revision: https://reviews.llvm.org/D50240

llvm-svn: 340125
2018-08-18 18:38:37 +00:00
Lang Hames bfea8cdc69 [ORC] Add a 'Callable' flag to JITSymbolFlags.
The callable flag can be used to indicate that a symbol is callable. If present,
the symbol is callable. If absent, the symbol may or may not be callable (the
client must determine this by context, for example by examining the program
representation that will provide the symbol definition).

This flag will be used in the near future to enable creation of lazy compilation
stubs based on SymbolFlagsMap instances only (without having to provide
additional information to determine which symbols need stubs).

llvm-svn: 338649
2018-08-01 22:42:23 +00:00
Fangrui Song f78650a8de Remove trailing space
sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338293
2018-07-30 19:41:25 +00:00
Reid Kleckner 330f65b3e8 [RuntimeDyld] Implement the ELF PIC large code model relocations
Prerequisite for https://reviews.llvm.org/D47211 which improves our ELF
large PIC codegen.

llvm-svn: 335402
2018-06-22 23:53:22 +00:00
Sean Fertile 80b8f82f17 [PPC64] Support "symbol@high" and "symbol@higha" symbol modifers.
Add support for the "@high" and "@higha" symbol modifiers in powerpc64 assembly.
The modifiers represent accessing the segment consiting of bits 16-31 of a
64-bit address/offset.

Differential Revision: https://reviews.llvm.org/D47729

llvm-svn: 334855
2018-06-15 19:47:11 +00:00
Lang Hames 22a7440200 [RuntimeDyld] Add an assert to catch misbehaving symbol resolvers.
Resolvers are required to find results for all requested symbols or return an
error, but if a resolver fails to adhere to this contract (by returning results
for only a subset of the requested symbols) then this code will infinite loop.
This assertion catches resolvers that fail to adhere to the contract.

llvm-svn: 334536
2018-06-12 20:43:17 +00:00
Hiroshi Inoue 3872c6c633 [PowerPC] fix broken JIT-compiled code with tail call optimization
The relocation for branch instructions in the dynamic loader of ExecutionEngine assumes branch instructions with R_PPC64_REL24 relocation type are only bl. However, with the tail call optimization, b instructions can be also used to jump into another function.
This patch makes the relocation to keep bits in the branch instruction other than the jump offset to avoid relocation rewrites a b instruction into bl.

Differential Revision: https://reviews.llvm.org/D47456

llvm-svn: 333502
2018-05-30 04:48:29 +00:00
Lang Hames 4c4a2ba353 [RuntimeDyld][MachO] Add support for MachO::ARM64_RELOC_POINTER_TO_GOT reloc.
llvm-svn: 333130
2018-05-23 21:27:07 +00:00
Nicola Zaghen d34e60ca85 Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624

llvm-svn: 332240
2018-05-14 12:53:11 +00:00
Lang Hames 3ae85708c9 [RuntimeDyld][MachO] Properly handle thumb to thumb calls within a section.
Previously thumb bits were only checked for external relocations (thumb to arm
code and vice-versa). This patch adds detection for thumb callees in the same
section asthe (also thumb) caller.

The MachO/Thumb test case is updated to cover this, and redundant checks
(handled by the MachO/ARM test) are removed.

llvm-svn: 331838
2018-05-09 01:38:13 +00:00
Adrian Prantl 4dfcc4a788 Remove @brief commands from doxygen comments, too.
This is a follow-up to r331272.

We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by
  for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done

https://reviews.llvm.org/D46290

llvm-svn: 331275
2018-05-01 16:10:38 +00:00
Adrian Prantl 5f8f34e459 Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

  for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46290

llvm-svn: 331272
2018-05-01 15:54:18 +00:00
Weiming Zhao 1bd40005ba Add missing vtable anchors
Summary: This patch adds anchor() for MemoryBuffer, raw_fd_ostream, RTDyldMemoryManager, SectionMemoryManager, etc.

Reviewers: jlebar, eli.friedman, dblaikie

Reviewed By: dblaikie

Subscribers: mehdi_amini, mgorny, dblaikie, weimingz, llvm-commits

Differential Revision: https://reviews.llvm.org/D45244

llvm-svn: 329861
2018-04-11 23:09:20 +00:00
Lang Hames 7a598477aa [RuntimeDyld][PowerPC] Use global entry points for calls between sections.
Functions in different objects may use different TOCs, so calls between such
functions should use the global entry point of the callee which updates the
TOC pointer.

This should fix a bug that the Numba developers encountered (see
https://github.com/numba/numba/issues/2451).

Patch by Olexa Bilaniuk. Thanks Olexa!

No RuntimeDyld checker test case yet as I am not familiar enough with how
RuntimeDyldELF fixes up call-sites, but I do not want to hold up landing
this. I will continue to work on it and see if I can rope some powerpc
experts in.

llvm-svn: 329335
2018-04-05 19:37:05 +00:00
Lang Hames b1e5043cff Reapply r329133 with fix.
llvm-svn: 329136
2018-04-04 00:34:54 +00:00
Lang Hames 4e319acd84 Revert r329133 "[RuntimeDyld][AArch64] Add some error pluming / generation..."
This broke a number of buildbots. Looking in to it now...

llvm-svn: 329135
2018-04-04 00:12:12 +00:00
Lang Hames b92b10f3ec [RuntimeDyld][AArch64] Add some error pluming / generation to catch unhandled
relocation types on AArch64.

llvm-svn: 329133
2018-04-03 23:19:20 +00:00
Lang Hames 2d603a1860 [RuntimeDyld] Silence a compiler error.
This should fix the error at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/19008

llvm-svn: 327478
2018-03-14 06:39:49 +00:00
Lang Hames 313f590aee [ExecutionEngine] Add a getSymbolTable method to RuntimeDyld.
This can be used to extract the symbol table from a RuntimeDyld instance prior
to disposing of it.

This patch also updates RTDyldObjectLinkingLayer to use the new method, rather
than requesting symbols one at a time via getSymbol.

llvm-svn: 327476
2018-03-14 06:25:07 +00:00
Lang Hames 811343cfd8 [RuntimeDyld][MachO] Fix assertion in encodeAddend, add missing directive to
test case.

r326290 fixed the assertion for decodeAddend, but not encodeAddend. The
regression test failed to catch this because it was missing the
subsections_via_symbols flag, so the desired relocation was not applied.

This patch also fixes the formatting of the assertion from r326290.

llvm-svn: 326406
2018-03-01 01:44:33 +00:00
Lang Hames 6588f14a6c [RuntimeDyld][MachO] Support ARM64_RELOC_BRANCH26 for BL instructions by
relaxing an assertion.

llvm-svn: 326290
2018-02-28 00:58:21 +00:00
Frederich Munch 33ef594c58 Handle IMAGE_REL_AMD64_ADDR32NB in RuntimeDyldCOFF
Summary:
IMAGE_REL_AMD64_ADDR32NB relocations are currently set to zero in all cases.
This patch sets the relocation to the correct value when possible and shows an error when not.

Reviewers: enderby, lhames, compnerd

Reviewed By: compnerd

Subscribers: LepelTsmok, compnerd, martell, llvm-commits

Differential Revision: https://reviews.llvm.org/D30709

llvm-svn: 325700
2018-02-21 17:18:20 +00:00
Rafael Espindola d5f76ad37f Move getPlatformFlags to ELFObjectFileBase and simplify.
This removes a few std::error_code results that were ignored on every
call.

llvm-svn: 323674
2018-01-29 18:27:30 +00:00
Rafael Espindola fc16f76edb Handle R_386_PLT32 in RuntimeDyldELF.
This should fix the 32 bit buildbots.

llvm-svn: 323344
2018-01-24 17:36:08 +00:00
Lang Hames 635fd9092b [ORC] Add orc::SymbolResolver, a Orc/Legacy API interop header, and an
orc::SymbolResolver to JITSymbolResolver adapter.

The new orc::SymbolResolver interface uses asynchronous queries for better
performance. (Asynchronous queries with bulk lookup minimize RPC/IPC overhead,
support parallel incoming queries, and expose more available work for
distribution). Existing ORC layers will soon be updated to use the
orc::SymbolResolver API rather than the legacy llvm::JITSymbolResolver API.

Because RuntimeDyld still uses JITSymbolResolver, this patch also includes an
adapter that wraps an orc::SymbolResolver with a JITSymbolResolver API.

llvm-svn: 323073
2018-01-22 03:00:31 +00:00
Rui Ueyama 20b49240b8 Fix -Wunused-variable.
llvm-svn: 323004
2018-01-19 22:56:04 +00:00
Lang Hames b72f48452c [ORC] Re-apply r322913 with a fix for a read-after-free error.
ExternalSymbolMap now stores the string key (rather than using a StringRef),
as the object file backing the key may be removed at any time.

llvm-svn: 323001
2018-01-19 22:24:13 +00:00
Lang Hames 44efd042a2 [ORC] Revert r322913 while I investigate an ASan failure.
llvm-svn: 322914
2018-01-19 01:40:26 +00:00
Lang Hames 817df9fa0c [ORC] Redesign the JITSymbolResolver interface to support bulk queries.
Bulk queries reduce IPC/RPC overhead for cross-process JITing and expose
opportunities for parallel compilation.

The two new query methods are lookupFlags, which finds the flags for each of a
set of symbols; and lookup, which finds the address and flags for each of a
set of symbols. (See doxygen comments for more details.)

The existing JITSymbolResolver class is renamed LegacyJITSymbolResolver, and
modified to extend the new JITSymbolResolver class using the following scheme:

- lookupFlags is implemented by calling findSymbolInLogicalDylib for each of the
symbols, then returning the result of calling getFlags() on each of these
symbols. (Importantly: lookupFlags does NOT call getAddress on the returned
symbols, so lookupFlags will never trigger materialization, and lookupFlags will
never call findSymbol, so only symbols that are part of the logical dylib will
return results.)

- lookup is implemented by calling findSymbolInLogicalDylib for each symbol and
falling back to findSymbol if findSymbolInLogicalDylib returns a null result.
Assuming a symbol is found its getAddress method is called to materialize it and
the result (if getAddress succeeds) is stored in the result map, or the error
(if getAddress fails) is returned immediately from lookup. If any symbol is not
found then lookup returns immediately with an error.

This change will break any out-of-tree derivatives of JITSymbolResolver. This
can be fixed by updating those classes to derive from LegacyJITSymbolResolver
instead.

llvm-svn: 322913
2018-01-19 01:12:40 +00:00
Lang Hames 4a793c0667 [ExecutionEngine] Rename JITSymbol::isStrongDefinition to isStrong.
For symmetry with isWeak, isCommon.

llvm-svn: 322594
2018-01-16 20:39:51 +00:00
Rui Ueyama fc63551082 Remove ELFDataTypeTypedefHelper class.
Differential Revision: https://reviews.llvm.org/D41973

llvm-svn: 322395
2018-01-12 19:59:43 +00:00
Rui Ueyama 478d635156 Instead of ELFFile<ELFT>::Type, use ELFT::Type. NFC.
llvm-svn: 322346
2018-01-12 02:28:31 +00:00
Michael Zolotukhin a44d5fe333 Remove redundant includes from lib/ExecutionEngine.
llvm-svn: 320621
2017-12-13 21:30:50 +00:00
Saleem Abdulrasool 9e802eaf60 ExecutionEngine: make COFF Thumb2 assertions non-tautological
The overflow detection assertions were tautological due to truncation.
Adjust them to no longer be tautological.

Patch by Alex Langford!

llvm-svn: 316303
2017-10-22 20:51:25 +00:00
Nitesh Jain 757f74c2d3 [mips] Adds support for R_MIPS_26, HIGHER, HIGHEST relocations in RuntimeDyld.
Reviewers: sdardis

Subscribers: jaydeep, bhushan, llvm-commits

Differential Revision: https://reviews.llvm.org/D38314

llvm-svn: 316287
2017-10-22 09:47:41 +00:00
Saleem Abdulrasool 1261151912 ExecutionEngine: adjust COFF i386 tautological asserts
Modify static_casts to not be tautological in some COFF i386
relocations.

Patch by Alex Langford!

llvm-svn: 316169
2017-10-19 16:57:40 +00:00
Shoaib Meenai a42f60e7f7 [ExecutionEngine] Correct the size of a write in a COFF i386 relocation
We want to be writing a 32bit value, so we should be writing 4 bytes
instead of 2.

Patch by Alex Langford <apl@fb.com>.

Differential Revision: https://reviews.llvm.org/D38872

llvm-svn: 315964
2017-10-17 01:41:14 +00:00
Rafael Espindola 1a0e5a1933 Convert an ErrorOr to Expected.
getRelocationAddend should never be called on non SHT_RELA sections,
but changing that requires changing RelocVisitor.h.

llvm-svn: 315473
2017-10-11 16:56:33 +00:00
Rafael Espindola ef421f9c18 Make the ELFObjectFile constructor private.
This forces every user to use the new create method that returns an
Expected. This in turn propagates better error messages.

llvm-svn: 315371
2017-10-10 21:21:16 +00:00
Rafael Espindola 8ed1198774 Try to make gcc happy.
llvm-svn: 315349
2017-10-10 19:27:51 +00:00
Rafael Espindola ff9f5f3372 Return Expected from createRTDyldELFObject.
No functionality change, it just makes it easier to use Expected in
Object.

llvm-svn: 315348
2017-10-10 19:14:30 +00:00
Rafael Espindola 563ede149a Simplify. NFC.
llvm-svn: 315347
2017-10-10 19:07:10 +00:00