Commit Graph

852 Commits

Author SHA1 Message Date
Sunho Kim 9189a26664 [ORC_RT][COFF] Initial platform support for COFF/x86_64.
Initial platform support for COFF/x86_64.

Completed features:
* Statically linked orc runtime.
* Full linking/initialization of static/dynamic vc runtimes and microsoft stl libraries.
* SEH exception handling.
* Full static initializers support
* dlfns
* JIT side symbol lookup/dispatch

Things to note:
* It uses vc runtime libraries found in vc toolchain installations.
* Bootstrapping state is separated because when statically linking orc runtime it needs microsoft stl functions to initialize the orc runtime, but static initializers need to be ran in order to fully initialize stl libraries.
* Process symbols can't be used blidnly on msvc platform; otherwise duplicate definition error gets generated. If process symbols are used, it's destined to get out-of-reach error at some point.
* Atexit currently not handled -- will be handled in the follow-up patches.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D130479
2022-08-13 13:48:40 +09:00
Anubhab Ghosh 143555b2ed [Orc] Properly deallocate mapped memory in MapperJITLinkMemoryManager
When memory is deallocated from MapperJITLinkMemoryManager deinitialize
actions are run through mapper and in case of InProcessMapper, memory
protections of the region are reset to read/write as they were previously
changed and can be reused in future.

Differential Revision: https://reviews.llvm.org/D131768
2022-08-13 10:08:25 +05:30
Anubhab Ghosh 0aaa74f7e6 [Orc] Reorder operations in ExecutorSharedMemoryMapperService shutdown
Differential Revision: https://reviews.llvm.org/D131510
2022-08-11 19:34:10 +05:30
Sunho Kim 2813151730 [ORC] Add missing std::move. 2022-08-11 15:41:06 +09:00
Sunho Kim 7260cdd2e1 [ORC][COFF] Introduce COFFVCRuntimeBootstrapper.
Introduces COFFVCRuntimeBootstrapper that loads/initialize vc runtime libraries. In COFF, we *must* jit-link vc runtime libraries as COFF relocation types have no proper way to deal with out-of-reach data symbols ragardless of linking mode. (even dynamic version msvcrt.lib have tons of static data symbols that must be jit-linked) This class tries to load vc runtime library files from msvc installations with an option to override the path.

There are some complications when dealing with static version of vc runtimes. First, they need static initializers to be ran that requires COFFPlatform support but orc runtime will not be usable before vc runtimes are fully initialized. (as orc runtime will use msvc stl libraries) COFFPlatform that will be introduced in a following up patch will collect static initializers and run them manually in host before boostrapping itself. So, the user will have to do the following.
1. Create COFFPlatform that addes static initializer collecting passes.
2. LoadVCRuntime
3. InitializeVCRuntime
4. COFFPlatform.bootstrap()
Second, the internal crt initialization function had to be reimplemented in orc side. There are other ways of doing this, but this is the simplest implementation that makes platform fully responsible for static initializer. The complication comes from the fact that crt initialization functions (such as acrt_initialize or dllmain_crt_process_attach) actually run all static initializers by traversing from `__xi_a` symbol to `__xi_z`. This requires symbols to be contiguously allocated in sections alphabetically sorted in memory, which is not possible right now and not practical in jit setting. We might ignore emission of `__xi_a` and `__xi_z` symbol and allocate them ourselves, but we have to take extra care after orc runtime boostrap has been done -- as that point orc runtime should be the one running the static initializers.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D130456
2022-08-11 15:27:47 +09:00
Sunho Kim 5cf0082ae3 [JITLink][COFF][x86_64] Implement SECTION/SECREL relocation.
Implements SECTION/SECREL relocation. These are used by debug info (pdb) data.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D130275
2022-08-11 15:12:24 +09:00
Anubhab Ghosh 1eee6de873 [Orc][JITLink] Slab based memory allocator to reduce RPC calls
Calling reserve() used to require an RPC call. This commit allows large
ranges of executor address space to be reserved. Subsequent calls to
reserve() will return subranges of already reserved address space while
there is still space available.

Differential Revision: https://reviews.llvm.org/D130392
2022-08-08 15:13:41 +05:30
Kazu Hirata a2d4501718 [llvm] Fix comment typos (NFC) 2022-08-07 00:16:14 -07:00
Lang Hames bc062e034f [ORC] Fix a memory leak in LLVMOrcIRTransformLayerSetTransform.
This function heap-allocates a ThreadSafeModule (the current C bindings assume
that TSMs are always heap-allocated), but was failing to free it.

Should fix http://llvm.org/PR56953.
2022-08-05 13:52:03 -07:00
Martin Storsjö 46bc1b5689 [ORC] Actually propagate memory unmapping errors on Windows
This fixes warnings like these:

    ../lib/ExecutionEngine/Orc/MemoryMapper.cpp:364:9: warning: ignoring return value of function declared with 'warn_unused_result' attribute [-Wunused-result]
            joinErrors(std::move(Err),
            ^~~~~~~~~~ ~~~~~~~~~~~~~~~

Differential Revision: https://reviews.llvm.org/D131056
2022-08-04 11:14:52 +03:00
Martin Storsjö 46196db4d3 [ORC] Fix a warning about an unused variable on Windows. NFC.
Differential Revision: https://reviews.llvm.org/D131055
2022-08-04 11:14:52 +03:00
Anubhab Ghosh ac3cb4ecd0 [Orc] Disable use of shared memory on Android
shm_open and shm_unlink are not available on Android. This commit
disables SharedMemoryMapper on Android until a better solution is
available.

https://android.googlesource.com/platform/bionic/+/refs/heads/master/docs/status.md
https://github.com/llvm/llvm-project/issues/56812

Differential Revision: https://reviews.llvm.org/D130814
2022-08-01 18:48:39 +05:30
Sunho Kim 6b27890b2c [ORC][COFF] Handle COFF import files of static archive.
Handles COFF import files of static archive. Changes static library genrator to build up object file map keyed by symbol name that excludes the symbols from dllimported symbols so that static generator will not be responsible for them. It exposes the list of dynamic libraries that need to be imported. Client should properly load the libraries in this list beforehand. Object file map is also an improvment from the past in terms of performance. Archive.findSym does a slow O(n) linear serach of symbol list to find the symbol. (we call findSym O(n) times, thus full time complexity is O(n^2); we were the only user of findSym function in fact)

There is a room for improvements in how to load the libraries in the list. We currently just hand the responsibility over to the clinet. A better way would be let ORC read this list and hand them over to JITLink side that would also help validation (e.g. not trying to generate stub for non dllimported targets) Nevertheless, we will have to exclude the symbols from COFF import object file list and need a way to access this list, which this patch offers.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D129952
2022-07-29 16:25:08 +09:00
Sunho Kim 72ea1a721e [ORC] Fix weak hidden symbols failure on PPC with runtimedyld
Fix "JIT session error: Symbols not found: [ DW.ref.__gxx_personality_v0 ] error" which happens when trying to use exceptions on ppc linux. To do this, it expands AutoClaimSymbols option in RTDyldObjectLinkingLayer to also claim weak symbols before they are tried to be resovled. In ppc linux, DW.ref symbols is emitted as weak hidden symbols in the later stage of MC pipeline. This means when using IRLayer (i.e. LLJIT), IRLayer will not claim responsibility for such symbols and RuntimeDyld will skip defining this symbol even though it couldn't resolve corresponding external symbol.

Reviewed By: sgraenitz

Differential Revision: https://reviews.llvm.org/D129175
2022-07-28 21:12:25 +09:00
Sunho Kim c7ea209068 [ORC][COFF] Properly set weak flag to COMDAT symbols.
Properly set weak flag to COMDAT symbols so that no duplicate definition error will be generated. There is an inaccuracy in setting plain weak for largest selection type, which will be dealt with soon when largest type is properly implemented.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D129764
2022-07-25 23:24:25 +09:00
Kazu Hirata 5bbe452e75 Revert "[Orc] Use default member initialization (NFC)"
This reverts commit d534967b66.

The patch causes build failures, such as:

https://lab.llvm.org/buildbot/#/builders/121/builds/21760
2022-07-23 21:10:10 -07:00
Kazu Hirata d534967b66 [Orc] Use default member initialization (NFC)
Identified with modernize-use-default-member-init
2022-07-23 18:36:23 -07:00
Anubhab Ghosh 4fcf8434dd [ORC] Add a new MemoryMapper-based JITLinkMemoryManager implementation.
MapperJITLinkMemoryManager supports executor memory management using any
implementation of MemoryMapper to do the transfer such as InProcessMapper or
SharedMemoryMapper.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D129495
2022-07-20 17:52:37 -07:00
Lang Hames aabc4b13e8 [ORC] Don't try to copy from an empty segment in SimpleExecutorMemoryManager.
Since 67220c2ad7 empty SPSSequence<char>s deserialize to default-constructed
ArrayRef<char>s, which have a null data field. We need to check for this to
avoid memcpy'ing from a nullptr.

This should fix the bot failure in
https://lab.llvm.org/buildbot/#/builders/85/builds/9323
2022-07-20 16:47:00 -07:00
Anubhab Ghosh 1b1f1c7786 Re-re-apply 5acd471698, Add a shared-memory based orc::MemoryMapper...
...with more fixes.

The original patch was reverted in 3e9cc543f2 due to bot failures caused by
a missing dependence on librt. That issue was fixed in 32d8d23cd0, but that
commit also broke sanitizer bots due to a bug in SimplePackedSerialization:
empty ArrayRef<char>s triggered a zero-byte memcpy from a null source. The
ArrayRef<char> serialization issue was fixed in 67220c2ad7, and this patch has
also been updated with a new custom SharedMemorySegFinalizeRequest message that
should avoid serializing empty ArrayRefs in the first place.

https://reviews.llvm.org/D128544
2022-07-19 15:35:33 -07:00
Mitch Phillips 4162aefad1 Revert "Re-apply 5acd471698, Add a shared-memory based orc::MemoryMapper, with fixes."
This reverts commit 32d8d23cd0.

Reason: Broke the UBSan buildbots. See more details on Phabricator:
https://reviews.llvm.org/D128544
2022-07-15 17:11:55 -07:00
Anubhab Ghosh 32d8d23cd0 Re-apply 5acd471698, Add a shared-memory based orc::MemoryMapper, with fixes.
The original commit was reverted in 3e9cc543f2 due to buildbot failures, which
should be fixed by the addition of dependencies on librt.

Differential Revision: https://reviews.llvm.org/D128544
2022-07-15 09:45:30 -07:00
Rainer Orth 8922adf646 [Orc] Honor hasJIT in JITTargetMachineBuilder::createTargetMachine
Even with D129349 <https://reviews.llvm.org/D129349>, various SPARC JIT
tests still `FAIL`.  They are guarded by `host-supports-jit`.  This is set
from `clang-repl --host-supports-jit` which still returns `true`.  It turns
out the `JITTargetMachineBuilder` doesn't consider `hasJIT` at all.

This patch corrects this, turning the affected tests `UNSUPPORTED`.

Tested on `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D129350
2022-07-15 08:20:57 +02:00
Cullen Rhodes 3e9cc543f2 Revert "[ORC] Add a shared-memory based orc::MemoryMapper."
This reverts commit 5acd471698.

Breaks shared library build with:

  ld.lld-12: error: undefined symbol: shm_open
  >>> referenced by ExecutorSharedMemoryMapperService.cpp:68
  (/home/culrho01/llvm-project/llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp:68)
  >>>
  lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/ExecutorSharedMemoryMapperService.cpp.o:(llvm::orc::rt_bootstrap::ExecutorSharedMemoryMapperService::reserve[abi:cxx11](unsigned
  long))
  >>> did you mean: sem_open
  >>> defined in:
  /usr/bin/../lib/gcc/aarch64-linux-gnu/9/../../../aarch64-linux-gnu/libpthread.so
2022-07-14 09:52:57 +00:00
Cullen Rhodes f3eacb4f35 Revert "[ORC] Fix compilation on mingw"
This reverts commit 46b1a7c5f9.

Parent commit breaks shared library build, reverting both commits.
2022-07-14 09:52:57 +00:00
Martin Storsjö 46b1a7c5f9 [ORC] Fix compilation on mingw
Explicitly call the -W suffixed API functions when passing wchar based
strings.
2022-07-14 12:05:08 +03:00
Anubhab Ghosh 5acd471698 [ORC] Add a shared-memory based orc::MemoryMapper.
This is an implementation of orc::MemoryMapper that maps shared memory
pages in both executor and controller process and writes directly to
them avoiding transferring content over EPC. All allocations are properly
deinitialized automatically on the executor side at shutdown by the
ExecutorSharedMemoryMapperService.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D128544
2022-07-13 15:24:28 -07:00
Sunho Kim db995d72db [JITLink][COFF] Initial COFF support.
Adds initial COFF support in JITLink. This is able to run a hello world c program in x86 windows successfully.

Implemented
- COFF object loader
- Static local symbols
- Absolute symbols
- External symbols
- Weak external symbols
- Common symbols
- COFF jitlink-check support
- All COMDAT selection type execpt largest
- Implicit symobl size calculation
- Rel32 relocation with PLT stub.
- IMAGE_REL_AMD64_ADDR32NB relocation

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D128968
2022-07-13 03:52:43 +09:00
Nicolai Hähnle 60cbf3fade ManagedStatic: Remove from JITLoaderGDB
This change originally landed as part of
e6f1f06245 (D129120), which caused a
Fuchsia buildbot regression in ExecutionEngine tests.

I am resubmitting the backed out parts in smaller pieces after a careful
review.
2022-07-10 10:32:29 +02:00
Nicolai Hähnle ede600377c ManagedStatic: remove many straightforward uses in llvm
(Reapply after revert in e9ce1a5880 due to
Fuchsia test failures. Removed changes in lib/ExecutionEngine/ other
than error categories, to be checked in more detail and reapplied
separately.)

Bulk remove many of the more trivial uses of ManagedStatic in the llvm
directory, either by defining a new getter function or, in many cases,
moving the static variable directly into the only function that uses it.

Differential Revision: https://reviews.llvm.org/D129120
2022-07-10 10:29:15 +02:00
Nicolai Hähnle e9ce1a5880 Revert "ManagedStatic: remove many straightforward uses in llvm"
This reverts commit e6f1f06245.

Reverting due to a failure on the fuchsia-x86_64-linux buildbot.
2022-07-10 09:54:30 +02:00
Nicolai Hähnle e6f1f06245 ManagedStatic: remove many straightforward uses in llvm
Bulk remove many of the more trivial uses of ManagedStatic in the llvm
directory, either by defining a new getter function or, in many cases,
moving the static variable directly into the only function that uses it.

Differential Revision: https://reviews.llvm.org/D129120
2022-07-10 09:15:08 +02:00
Xiaodong Liu 17c790ca84 [ORC][MIPS} Correct template parameter for stubAndPointerRangesOk function on Mips
It might be an oversight that pass OrcAArch64 as template parameter to stubAndPointerRangesOk on MIps.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D129076
2022-07-07 21:00:25 +08:00
Sunho Kim 30b6c51f51 [ORC][ORC_RT][AArch64] Implement TLS descriptor in ELFNixPlatform.
Implements TLS descriptor relocations in JITLink ELF/AARCH64 backend and support the relevant runtime functions in ELFNixPlatform.

Unlike traditional TLS model, TLS descriptor model requires linker to return the "offset" from thread pointer via relocaiton not the actual pointer to thread local variable. There is no public libc api for adding new allocations to TLS block dynamically which thread pointer points to. So, we support this by taking delta from thread base pointer to the actual thread local variable in our allocated section.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D128601
2022-07-06 20:12:22 +09:00
Jonas Hahnfeld a4e2c1f762 [Orc][LLJIT] Use JITLink on RISC-V
RuntimeDyld does not support RISC-V, so it makes sense to enable
JITLink by default. This also makes relocations work without support
for a large code model.

Differential Revision: https://reviews.llvm.org/D129092
2022-07-04 23:09:38 +02:00
Sunho Kim 7d101e43cd [ORC][LLJIT] Define atexit symbol in GenericLLVMIRPlatformSupport.
Define atexit symbol in GenericLLVMIRPlatformSupport so that it doesn't need to be defined by user.

On windows, llvm codegen emits atexit runtime calls to support global deinitializers as there is no lower function like cxa_atexit as in Itanium C++ ABI. ORC JIT user had to define custom atexit symbol manually. This was a hassle as it has to deal with dso_handle and cxa_atexit internals of LLJIT. If client didn't provide atexit definition, the default behaviour is just linking with host atexit function which is destined to fail as it calls dtors when the host program exits. This is after jit instances and buffers are freed, so users would see weird access violation exception from the uknown location. (in console application, the debugger thinks exception happened in scrt_common_main_seh)

This is a hack that has some caveats. (e.g. memory address is not identical) But, it's better than the situation described in the above. Ultimately, we will move on to ORC runtime that is able to solve the memory address issue properly.

Reviewed By: sgraenitz

Differential Revision: https://reviews.llvm.org/D128037
2022-06-26 03:50:08 +09:00
Anubhab Ghosh 79fbee3cc5 Re-apply "[JITLink][Orc] Add MemoryMapper interface with InProcess implementation"
[JITLink][Orc] Add MemoryMapper interface with InProcess implementation

MemoryMapper class takes care of cross-process and in-process address space
reservation, mapping, transferring content and applying protections.

Implementations of this class can support different ways to do this such
as using shared memory, transferring memory contents over EPC or just
mapping memory in the same process (InProcessMemoryMapper).

The original patch landed with commit 6ede652050
It was reverted temporarily in commit 6a4056ab2a

Reviewed By: sgraenitz, lhames

Differential Revision: https://reviews.llvm.org/D127491
2022-06-21 23:53:16 +02:00
Nico Weber 6a4056ab2a Revert "[JITLink][Orc] Add MemoryMapper interface with InProcess implementation"
This reverts commit 6ede652050.
Doesn't build on Windows, see https://reviews.llvm.org/D127491#3598773
2022-06-21 09:56:49 -04:00
Anubhab Ghosh 6ede652050 [JITLink][Orc] Add MemoryMapper interface with InProcess implementation
MemoryMapper class takes care of cross-process and in-process address space
reservation, mapping, transferring content and applying protections.

Implementations of this class can support different ways to do this such
as using shared memory, transferring memory contents over EPC or just
mapping memory in the same process (InProcessMemoryMapper).

Reviewed By: sgraenitz, lhames

Differential Revision: https://reviews.llvm.org/D127491
2022-06-21 13:44:17 +02:00
Kazu Hirata 7a47ee51a1 [llvm] Don't use Optional::getValue (NFC) 2022-06-20 22:45:45 -07:00
Kazu Hirata ad7ce1e769 Don't use Optional::hasValue (NFC) 2022-06-20 11:49:10 -07:00
Kazu Hirata e0e687a615 [llvm] Don't use Optional::hasValue (NFC) 2022-06-20 10:38:12 -07:00
Kazu Hirata b254d67160 [llvm] Call *set::insert without checking membership first (NFC) 2022-06-18 08:32:54 -07:00
Lang Hames 14b7c108a2 [C-API][ORC] Add C API to suspend lookups during definition generation.
Slow definition generators may suspend lookups to temporarily release the
session lock, allowing unrelated lookups to proceed.

Using this functionality is discouraged: it is best to make definition
generation fast, rather than suspending the lookup. As a last resort where
this is not possible, suspension may be used.
2022-06-13 17:20:07 -07:00
Lang Hames 803c770ee0 [C-API][ORC] Add LLVMOrcExecutionSessionLookup -- generic async symbol lookup.
An API to wrap ExecutionSession::lookup, this allows C API clients to use async
lookup.

The immediate motivation for adding this is to simplify upcoming
definition-generator unit tests.

As we're adding more tests that need to convert between C and C++ flag values
this commit adds helper functions to support this. This patch also updates the
CAPIDefinitionGenerator to use these new utilities.
2022-06-13 16:37:35 -07:00
Jez Ng d4bcb45db7 [MC][re-land] Omit DWARF unwind info if compact unwind is present where eligible
This reverts commit d941d59783.

Differential Revision: https://reviews.llvm.org/D122258
2022-06-12 17:24:19 -04:00
Jez Ng d941d59783 Revert "[MC] Omit DWARF unwind info if compact unwind is present where eligible"
This reverts commit ef501bf85d.
2022-06-12 10:47:08 -04:00
Jez Ng ef501bf85d [MC] Omit DWARF unwind info if compact unwind is present where eligible
Previously, omitting unnecessary DWARF unwinds was only done in two
cases:
* For Darwin + aarch64, if no DWARF unwind info is needed for all the
  functions in a TU, then the `__eh_frame` section would be omitted
  entirely. If any one function needed DWARF unwind, then MC would emit
  DWARF unwind entries for all the functions in the TU.
* For watchOS, MC would omit DWARF unwind on a per-function basis, as
  long as compact unwind was available for that function.

This diff makes it so that we omit DWARF unwind on a per-function basis
for Darwin + aarch64 as well. In addition, we introduce the flag
`--emit-dwarf-unwind=` which can toggle between `always`,
`no-compact-unwind` (only emit DWARF when CU cannot be emitted for a
given function), and the target platform `default`.  `no-compact-unwind`
is particularly useful for newer x86_64 platforms: we don't want to omit
DWARF unwind for x86_64 in general due to possible backwards compat
issues, but we should make it possible for people to opt into this
behavior if they are only targeting newer platforms.

**Motivation:** I'm working on adding support for `__eh_frame` to LLD,
but I'm concerned that we would suffer a perf hit. Processing compact
unwind is already expensive, and that's a simpler format than EH frames.
Given that MC currently produces one EH frame entry for every compact
unwind entry, I don't think processing them will be cheap. I tried to do
something clever on LLD's end to drop the unnecessary EH frames at parse
time, but this made the code significantly more complex. So I'm looking
at fixing this at the MC level instead.

**Addendum:** It turns out that there was a latent bug in the X86
backend when `OmitDwarfIfHaveCompactUnwind` is naively enabled, which is
not too surprising given that this combination has not been heretofore
used.

For functions that have unwind info that cannot be encoded with CU, MC
would end up dropping both the compact unwind entry (OK; existing
behavior) as well as the DWARF entries (not OK).  This diff fixes things
so that we emit the DWARF entry, as well as a CU entry with encoding
`UNWIND_X86_MODE_DWARF` -- this basically tells the unwinder to look for
the DWARF entry. I'm not 100% sure the `UNWIND_X86_MODE_DWARF` CU entry
is necessary, this was the simplest fix. ld64 seems to be able to handle
both the absence and presence of this CU entry. Ultimately ld64 (and
LLD) will synthesize `UNWIND_X86_MODE_DWARF` if it is absent, so there
is no impact to the final binary size.

Reviewed By: davide, lhames

Differential Revision: https://reviews.llvm.org/D122258
2022-06-12 10:03:56 -04:00
Peter S. Housel 2be5abb7e9 [ORC][ORC_RT] Handle ELF .init_array with non-default priority
ELF-based platforms currently support defining multiple static
initializer table sections with differing priorities, for example
.init_array.0 or .init_array.100; the default .init_array corresponds
to a priority of 65535. When building a shared library or executable,
the system linker normally sorts these sections and combines them into
a single .init_array section. This change adds the capability to
recognize ELF static initializers with priorities other than the
default, and to properly sort them by priority, to Orc and the Orc
runtime.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D127056
2022-06-09 22:47:58 -07:00
Peter S. Housel 1aa71f8679 [ORC][ORC_RT] Integrate ORC platforms with LLJIT and lli
This change enables integrating orc::LLJIT with the ORCv2
platforms (MachOPlatform and ELFNixPlatform) and the compiler-rt orc
runtime. Changes include:

- Adding SPS wrapper functions for the orc runtime's dlfcn emulation
  functions, allowing initialization and deinitialization to be invoked
  by LLJIT.

- Changing the LLJIT code generation default to add UseInitArray so
  that .init_array constructors are generated for ELF platforms.

- Integrating the ORCv2 Platforms into lli, and adding a
  PlatformSupport implementation to the LLJIT instance used by lli which
  implements initialization and deinitialization by calling the new
  wrapper functions in the runtime.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D126492
2022-06-09 22:47:58 -07:00
Sunho Kim e093e42107 [ORC][AArch64] Add initial support for aarch64 in ELFNixPlatform.
Adds the aarch64 support in ELFNixPlatform. These are few simple changes, but it allows us to use the orc runtime in ELF/AARCH64 backend. It succesfully run the static initializers of stdlibc++ iostream so that "cout << Hello world" testcase starts to work.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D127060
2022-06-10 13:37:36 +09:00
Lang Hames 3fcd3669e3 [ORC] Add an output stream operator for SymbolStringPool.
Handy for checking string pool state, e.g. when debugging dangling-pool-entry
errors.
2022-06-08 16:49:51 -07:00
Kazu Hirata e0039b8d6a Use llvm::less_second (NFC) 2022-06-04 22:48:32 -07:00
Lang Hames 7be783ab9d [ORC] Check for errors when materializing absolute symbols.
This code previously used cantFail, but both steps (resolution and emission)
can fail if the resource tracker associated with the
AbsoluteSymbolsMaterializationUnit is removed. Checking these errors is
necessary for correct error propagation.
2022-05-21 14:11:53 -07:00
Lang Hames 55e8f721d4 [ORC] Allow FailedToMaterialize errors to outlive ExecutionSessions.
Idiomatic llvm::Error usage can result in a FailedToMaterialize error tearing
down an ExecutionSession instance. Since the FailedToMaterialize error holds
SymbolStringPtrs and JITDylib references this leads to crashes when accessing
or logging the error.

This patch modifies FailedToMaterialize to retain the SymbolStringPool and
JITDylibs involved in the failure so that we can safely report an error message
to the client, even if the error tears down the session.

The contract for JITDylibs allows the getName method to be used even after the
session has been torn down, but no other JITDylib fields should be accessed via
the FailedToMaterialize error if the ssesion has been torn down. Logging the
error is guaranteed to be safe in all cases.
2022-05-21 13:51:02 -07:00
Lang Hames f3428dafdc [ORC] Add a ~ExectionSession destructor to verify that endSession was called.
Clients are required to call ExecutionSession::endSession before destroying the
ExecutionSession. Failure to do so can lead to memory leaks and other difficult
to debug issues. Enforcing this requirement by assertion makes it easy to spot
or debug situations where the contract was not followed.
2022-05-21 09:02:01 -07:00
Lang Hames 441f691b37 [ORC] Remove usused Error variable.
This was left in by accident, and caused unchecked-error assertions
during program termination in BuildingAJIT-Ch4.
2022-05-20 19:33:07 -07:00
Lang Hames 2ca81cd914 [ORC] Avoid more SymbolStringPtr copies. 2022-05-19 10:51:20 -07:00
Lang Hames d27e4bcebd [ORC] Add a FIXME. 2022-05-19 10:51:20 -07:00
Lang Hames 4bb18a89c4 [ORC] Add missing std::moves, pass SymbolLookupSet by value.
Avoids some unnecessary SymbolStringPtr copies.
2022-05-19 10:51:20 -07:00
Jonas Hahnfeld e451d55234 [ORC] Fix sorting of contructors by priority
The code was incorrectly sorting by the function address.

Differential Revision: https://reviews.llvm.org/D123311
2022-05-11 09:06:41 +02:00
Peter S. Housel 981523b2e4 [ORC-RT][ORC] Handle dynamic unwind registration for libunwind
This changes the ELFNix platform Orc runtime to use, when available,
the __unw_add_dynamic_eh_frame_section interface provided by libunwind
for registering .eh_frame sections loaded by JITLink. When libunwind
is not being used for unwinding, the ELFNix platform detects this and
defaults to the __register_frame interface provided by libgcc_s.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D114961
2022-05-06 14:00:29 -07:00
Lang Hames 16dcbb53dc [ORC] Return ExecutorAddrs rather than JITEvaluatedSymbols from LLJIT::lookup.
Clients don't care about linkage, and ExecutorAddr is much more ergonomic.
2022-05-05 13:56:00 -07:00
Lang Hames 45e390cfee [ORC] Report paths in errors when creating StaticLibrarySearchGenerators. 2022-04-17 12:01:06 -07:00
Alex Fan acb408fbbc [ORC] add lazy jit support for riscv64
This adds resolver, indirection and trampoline stubs for riscv64,
allowing lazy compilation to work.

It assumes hard float extension exists. I don't know the proper way to detect it as Triple doesn't provide the interface to check riscv +f +d abi.

I am also not sure if orclazy tests should be enabled because lli needs an additional -codemodel=melany for tests to pass.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D122543
2022-04-10 18:44:50 +08:00
Lang Hames a76209c265 [ORC] Fix handling of casts in llvm.global_ctors.
Removes a bogus dyn_cast_or_null that was breaking cast-expression handling when
parsing llvm.global_ctors.

The intent of this code was to identify Functions nested within cast
expressions, but the offending dyn_cast_or_null was actually blocking that:
Since a function is not a cast expression, we would set FuncC to null and break
the loop without finding the Function. The cast was not necessary either:
Functions are already Constants, and we didn't need to do anything
ConstantExpr-specific with FuncC, so we could just drop the cast.

Thanks to Jonas Hahnfeld for tracking this down.

http://llvm.org/PR54797
2022-04-07 17:06:38 -07:00
Pavel Labath ec6d621050 Remove a top-level using-directive from EPCDebugObjectRegistrar.h
The directive pollutes the namespace of all files which include the
header.

Use alternate ways to reference the namespace constituents instead.
2022-03-28 15:14:20 +02:00
Lang Hames 36f0dfd07a [ORC][JITLink] Fix MachO absolute symbol handling, add test case.
This patch removes the unintended resolution of locally scoped absolute symbols
(which was causing unexpected definition errors).

It stops using the JITSymbolFlags::Absolute flag (it isn't set or used elsewhere,
and causes mismatch-flags asserts), and adds JITSymbolFlags::Exported to default
scoped absolute symbols.

Finally, we now set the scope of absolute symbols correctly in
MachOLinkGraphBuilder.
2022-03-07 20:54:55 -08:00
Lang Hames 34e539dcd7 [ORC] Set ResolverBlockAddr in EPCIndirectionUtils::writeResolverBlock.
Without this, EPCIndirectionUtils::getResolverBlockAddr (and lazy compilation
via EPC) won't work.

No test case: lli is still using LocalLazyCallThroughManager. I'll revisit this
soon when I look at adding lazy compilation support to the ORC runtime.
2022-03-01 16:44:55 -08:00
Lang Hames f9aef477eb [ORC][ORC-RT] Rewrite the MachO platform to use allocation actions.
This patch updates the MachO platform (both the ORC MachOPlatform class and the
ORC-Runtime macho_platform.* files) to use allocation actions, rather than EPC
calls, to transfer the initializer information scraped from each linked object.
Interactions between the ORC and ORC-Runtime sides of the platform are
substantially redesigned to accomodate the change.

The high-level changes in this patch are:

1. The MachOPlatform::setupJITDylib method now calls into the runtime to set up
   a dylib name <-> header mapping, and a dylib state object (JITDylibState).

2. The MachOPlatformPlugin builds an allocation action that calls the
   __orc_rt_macho_register_object_platform_sections and
   __orc_rt_macho_deregister_object_platform_sections functions in the runtime
   to register the address ranges for all "interesting" sections in the object
   being allocated (TLS data sections, initializers, language runtime metadata
   sections, etc.).

3. The MachOPlatform::rt_getInitializers method (the entry point in the
   controller for requests from the runtime for initializer information) is
   replaced by MachOPlatform::rt_pushInitializers. The former returned a data
   structure containing the "interesting" section address ranges, but these are
   now handled by __orc_rt_macho_register_object_platform_sections. The new
   rt_pushInitializers method first issues a lookup to trigger materialization
   of the "interesting" sections, then returns the dylib dependence tree rooted
   at the requested dylib for dlopen to consume. (The dylib dependence tree is
   returned by rt_pushInitializers, rather than being handled by some dedicated
   call, because rt_pushInitializers can alter the dependence tree).

The advantage of these changes (beyond the performance advantages of using
allocation actions) is that it moves more information about the materialized
portions of the JITDylib into the executor. This tends to make the runtime
easier to reason about, e.g. the implementation of dlopen in the runtime is now
recursive, rather than relying on recursive calls in the controller to build a
linear data structure for consumption by the runtime. This change can also make
some operations more efficient, e.g. JITDylibs can be dlclosed and then
re-dlopened without having to pull all initializers over from the controller
again.

In addition to the high-level changes, there are some low-level changes to ORC
and the runtime:

* In ORC, at ExecutionSession teardown time JITDylibs are now destroyed in
reverse creation order. This is on the assumption that the ORC runtime will be
loaded into an earlier dylib that will be used by later JITDylibs. This is a
short-term solution to crashes that arose during testing when the runtime was
torn down before its users. Longer term we will likely destroy dylibs in
dependence order.

* toSPSSerializable(Expected<T> E) is updated to explicitly initialize the T
value, allowing it to be used by Ts that have explicit constructors.

* The ORC runtime now (1) attempts to track ref-counts, and (2) distinguishes
not-yet-processed "interesting" sections from previously processed ones. (1)
is necessary for standard dlopen/dlclose emulation. (2) is intended as a step
towards better REPL support -- it should enable future runtime calls that
run only newly registered initializers ("dlopen_more", "dlopen_additions",
...?).
2022-02-08 17:28:13 +11:00
Kazu Hirata 3a3cb929ab [llvm] Use = default (NFC) 2022-02-06 22:18:35 -08:00
Lang Hames 4111fafa5b [ORC] Fix JITDylib debug output: don't output symbol table entry flags twice. 2022-02-04 18:34:32 +11:00
Lang Hames d5ae7a99d1 [ORC] Fix another missing std::move from 9eb4939b86. 2022-01-20 17:58:37 +11:00
Lang Hames f8752e28fd [ORC] Fix missing std::move from 9eb4939b86. 2022-01-20 17:56:56 +11:00
Lang Hames 9eb4939b86 [ORC] Allow JITDylib::getDFSLinkOrder and friends to fail for defunct JITDylibs.
Calls to JITDylib's getDFSLinkOrder and getReverseDFSLinkOrder methods (both
static an non-static versions) are now valid to make on defunct JITDylibs, but
will return an error if any JITDylib in the link order is defunct.

This means that platforms can safely lookup link orders by name in response to
jit-dlopen calls from the ORC runtime, even if the call names a defunct
JITDylib -- the call will just fail with an error.
2022-01-20 17:45:32 +11:00
Lang Hames ade71641dc [ORC] Add Platform::teardownJITDylib method.
This is a counterpart to Platform::setupJITDylib, and is called when JITDylib
instances are removed (via ExecutionSession::removeJITDylib).

Upcoming MachOPlatform patches will use this to clear per-JITDylib data when
JITDylibs are removed.
2022-01-18 16:27:02 +11:00
Lang Hames c0fdc74887 [ORC] Add helper functions for running finalize / dealloc actions.
runFinalizeActions takes an AllocActions vector and attempts to run its finalize
actions. If any finalize action fails then all paired dealloc actions up to the
failing pair are run, and the error(s) returned. If all finalize actions succeed
then a vector containing the dealloc actions is returned.

runDeallocActions takes a vector<WrapperFunctionCall> containing dealloc action
calls and runs them all, returning any error(s).

These helpers are intended to simplify the implementation of
JITLinkMemoryManager::InFlightAlloc::finalize and
JITLinkMemoryManager::deallocate overrides by taking care of execution (and
potential roll-back) of allocation actions.
2022-01-10 19:13:58 +11:00
Lang Hames 089acf2522 [ORC][JITLink] Merge JITLink AllocActionCall and ORC WrapperFunctionCall.
These types performed identical roles. Merging them simplifies interoperability
between JITLink and ORC APIs (allowing us to address a few FIXMEs).
2022-01-08 16:46:15 +11:00
Kazu Hirata 2aed08131d [llvm] Use true/false instead of 1/0 (NFC)
Identified with modernize-use-bool-literals.
2022-01-07 00:39:14 -08:00
Qiu Chaofan c2cc70e4f5 [NFC] Fix endif comments to match with include guard 2022-01-07 15:52:59 +08:00
Lang Hames 118e953b18 Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.
This re-applies 133f86e954, which was reverted in
c5965a411c while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.
2022-01-06 17:22:21 +11:00
Lang Hames c5965a411c Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."
This reverts commit 133f86e954 while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.
2022-01-06 15:20:21 +11:00
Lang Hames 133f86e954 [JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.
ExecutorAddr is the preferred representation for executor process addresses now.
2022-01-06 13:48:12 +11:00
Lang Hames 9e2cfb061a [ORC] Make ExecutorAddrDiff an alias for uint64_t.
We don't need to restrict operations on ExecutorAddrDiff as carefully as we do
for ExecutorAddr.
2022-01-06 13:48:11 +11:00
Kazu Hirata 5a667c0e74 [llvm] Use nullptr instead of 0 (NFC)
Identified with modernize-use-nullptr.
2021-12-28 08:52:25 -08:00
Lang Hames 02fc8d5c9e [ORC] Add custom object interface support to StaticLibaryDefinitionGenerator.
This adds a GetObjectFileInterface callback member to
StaticLibraryDefinitionGenerator, and adds an optional argument for initializing
that member to StaticLibraryDefinitionGenerator's named constructors. If not
supplied, it will default to getObjectFileInterface from ObjectFileInterface.h.

To enable testing a `-hidden-l<x>` option is added to the llvm-jitlink tool.
This allows archives to be loaded with all contained symbol visibilities demoted
to hidden.

The ObjectLinkingLayer::setOverrideObjectFlagsWithResponsibilityFlags method is
(belatedly) hooked up, and enabled in llvm-jitlink when `-hidden-l<x>` is used
so that the demotion is also applied at symbol resolution time (avoiding any
"mismatched symbol flags" crashes).
2021-12-16 19:46:51 +11:00
Lang Hames 4299d8d0ce [ORC] Add MaterializationUnit::Interface parameter to ObjectLayer::add.
Also moves object interface building functions out of Mangling.h and in to the
new ObjectFileInterfaces.h header, and updates the llvm-jitlink tool to use
custom object interfaces rather than a custom link layer.

ObjectLayer::add overloads are added to match the old signatures (which
do not take a MaterializationUnit::Interface). These overloads use the
standard getObjectFileInterface function to build an interface.

Passing a MaterializationUnit::Interface explicitly makes it easier to alter
the effective interface of the object file being added, e.g. by changing symbol
visibility/linkage, or renaming symbols (in both cases the changes will need to
be mirrored by a JITLink pass at link time to update the LinkGraph to match the
explicit interface). Altering interfaces in this way can be useful when lazily
compiling (e.g. for renaming function bodies) or emulating linker options (e.g.
demoting all symbols to hidden visibility to emulate -load_hidden).
2021-12-15 12:03:55 +11:00
Lang Hames 3ce1e9428b [ORC] Add early-out to OL_applyQueryPhase1.
If all symbols in a lookup match before we reach the end of the search order
then bail out of the search-order loop early.

This should reduce unnecessary contention on the session lock and improve
readability of the debug logs.
2021-12-15 10:01:02 +11:00
Jan Svoboda d0262c2394 [llvm] Add null-termination capability to SmallVectorMemoryBuffer
Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (`mmap` vs `open`),
* assert newly initialized buffer indeed has an implicit null terminator.

This patch adds the paramater to the `SmallVectorMemoryBuffer` constructors, meaning:
* null terminator can now be added to `SmallVector`s that didn't have one before,
* `SmallVectors` that had a null terminator before keep it even after the move.

In line with existing code, the new parameter is defaulted to `true`. This patch makes sure all calls to the `SmallVectorMemoryBuffer` constructor set it to `false` to preserve the current semantics.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115331
2021-12-09 11:32:13 +01:00
Lang Hames ae73f3fdd6 [ORC] Add a MaterializationUnit::Interface struct.
MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.

Having a type for this will make functions that build materializer interfaces
more readable and maintainable.
2021-12-08 13:41:15 +11:00
Lang Hames 02095f2dd7 [ORC] Pad section start to account for alignment offset in MachO GDB JIT plugin.
In order to present a well-formed MachO debug object for debugger registration
the first block in each section must have a zero alignment offset (since there
is no way to represent a non-zero offset in a MachO section load command). This
patch updates the MachODebugObjectSynthesizer class to introduce a padding
padding block at the start of the section if necessary to guarantee a zero
alignment offset.
2021-12-07 14:28:31 +11:00
Kazu Hirata 22d82949b0 [llvm] Fix "unused variable" warnings 2021-12-02 09:20:17 -08:00
Lang Hames 758d54b462 [ORC] Add support for removing JITDylibs.
This allows JITDylibs to be removed from the ExecutionSession. Calling
ExecutionSession::removeJITDylib will disconnect the JITDylib from the
ExecutionSession and clear it (removing all trackers associated with it). The
JITDylib object will then be destroyed as soon as the last JITDylibSP pointing
at it is destroyed.
2021-12-02 17:36:32 +11:00
Lang Hames 9eb591f0ac [ORC] Only use JITDylib::GeneratorsMutex while running generators.
GeneratorsMutex should prevent lookups from proceeding through the
generators of a single JITDylib concurrently (since this could
result in redundant attempts to generate definitions). Mutation of
the generators list itself should be done under the session lock.
2021-12-02 17:36:32 +11:00
Lang Hames 9355d11597 [ORC] Hold ResourceTracker in MaterializationResponsibility.
This keeps the tracker alive for the lifetime of the MR. This is needed so that
we can check whether the tracker has become defunct before posting results (or
failure) for the MR.
2021-12-02 17:36:31 +11:00
Lang Hames 43f5f6916f [ORC][JITLink] Move JITDylib name into JITLinkDylib base class.
This will enable better error messages and debug logs in JITLink.
2021-11-19 20:54:28 -08:00
Ben Langmuir 4c94760f36 [ORC] Fix materialization of weak local symbols
We were adding all defined weak symbols to the materialization
responsibility, but local symbols will not be in the symbol table, so it
failed to materialize due to the "missing" symbol.

Local weak symbols come up in practice when using `ld -r` with a hidden
weak symbol.

rdar://85574696
2021-11-19 07:25:56 -08:00
Lang Hames 69be352a19 Reapply "[ORC] Initial MachO debugging support (via GDB JIT debug.." with fixes.
This reapplies e1933a0488 (which was reverted in
f55ba3525e due to bot failures, e.g.
https://lab.llvm.org/buildbot/#/builders/117/builds/2768).

The bot failures were due to a missing symbol error: We use the input object's
mangling to decide how to mangle the debug-info registration function name. This
caused lookup of the registration function to fail when the input object
mangling didn't match the host mangling.

Disbaling the test on non-Darwin platforms is the easiest short-term solution.
I have filed https://llvm.org/PR52503 with a proposed longer term solution.
2021-11-14 14:44:07 -08:00
Lang Hames f55ba3525e Revert "[ORC] Initial MachO debugging support (via GDB JIT debug..."
This reverts commit e1933a0488 until I can look
into bot failures.
2021-11-14 00:14:39 -08:00
Lang Hames e1933a0488 [ORC] Initial MachO debugging support (via GDB JIT debug registration interface)
This commit adds a new plugin, GDBJITDebugInfoRegistrationPlugin, that checks
for objects containing debug info and registers any debug info found via the
GDB JIT registration API.

To enable this registration without redundantly representing non-debug sections
this plugin synthesizes a new embedded object within a section of the LinkGraph.
An allocation action is used to make the registration call.

Currently MachO only. ELF users can still use the DebugObjectManagerPlugin. The
two are likely to be merged in the near future.
2021-11-13 13:21:01 -08:00