Commit Graph

278 Commits

Author SHA1 Message Date
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Pavel Labath 859d302349 [RuntimeDyld] Fix debug section relocation (pr20457)
Summary:
Debug info sections, (or non-SHF_ALLOC sections in general) should be
linked as if their load address was zero to emulate the behavior of the
static linker.

This bug was discovered because it was breaking lldb expression evaluation on
linux.

Reviewers: lhames

Subscribers: aprantl, eugene, clayborg, lldb-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D32899

llvm-svn: 303239
2017-05-17 08:47:28 +00:00
Lang Hames c936ac7f37 [ExecutionEngine] Make RuntimeDyld::MemoryManager responsible for tracking EH
frames.

RuntimeDyld was previously responsible for tracking allocated EH frames, but it
makes more sense to have the RuntimeDyld::MemoryManager track them (since the
frames are allocated through the memory manager, and written to memory owned by
the memory manager). This patch moves the frame tracking into
RTDyldMemoryManager, and changes the deregisterFrames method on
RuntimeDyld::MemoryManager from:

void deregisterEHFrames(uint8_t *Addr, uint64_t LoadAddr, size_t Size);

to:

void deregisterEHFrames();

Separating this responsibility will allow ORC to continue to throw the
RuntimeDyld instances away post-link (saving a few dozen bytes per lazy
function) while properly deregistering frames when modules are unloaded.

This patch also updates ORC to call deregisterEHFrames when modules are
unloaded. This fixes a bug where an exception that tears down the JIT can then
unwind through dangling EH frames that have been deallocated but not
deregistered, resulting in UB.

For people using SectionMemoryManager this should be pretty much a no-op. For
people with custom allocators that override registerEHFrames/deregisterEHFrames,
you will now be responsible for tracking allocated EH frames.

Reviewed in https://reviews.llvm.org/D32829

llvm-svn: 302589
2017-05-09 21:32:18 +00:00
Lang Hames d22badef45 [RuntimeDyld] Make RuntimeDyld honor the ProcessAllSections flag.
When the ProcessAllSections flag (introduced in r204398) is set RuntimeDyld is
supposed to make a call to the client's memory manager for every section in each
object that is loaded. Due to some missing checks, this was not happening in all
cases. This patch adds the missing cases, and fixes the Orc unit test that
verifies correct behavior for ProcessAllSections (The unit test had been
silently bailing out due to an ordering issue: a change in the test order meant
that this unit-test was running before the native target was registered. This
issue has also been fixed in this patch).

This fixes <rdar://problem/22789965>

llvm-svn: 299449
2017-04-04 17:03:49 +00:00
Eugene Leviant 3e582c8855 RuntimeDyldELF/AArch64: Implement basic GOT support
This patch implements two GOT relocations:
R_AARCH64_ADR_GOT_PAGE and R_AARCH64_LD64_GOT_LO12_NC

Differential revision: https://reviews.llvm.org/D28571

llvm-svn: 294191
2017-02-06 15:31:28 +00:00
Simon Dardis c97cfb69ba [mips][rtdyld] Move MIPS relocation resolution to a subclass and implement N32 relocations
N32 relocations are only correct for individual relocations at the moment.
Support for relocation composition will follow in a later patch.

Patch By: Daniel Sanders

Reviwers: vkalintiris, atanasyan

Differential Revision: https://reviews.llvm.org/D27467

llvm-svn: 289532
2016-12-13 11:39:18 +00:00
Lang Hames 90370702c5 [RuntimeDyld] Skip undefined symbols when building the symbol table.
Storing these in the symbol table (with zero values) is just wasted space.

llvm-svn: 288225
2016-11-30 01:12:07 +00:00
Reid Kleckner 990504e625 Remove LLVM_NOEXCEPT and replace it with noexcept
Now that we have dropped MSVC 2013, all supported compilers support
noexcept and we can drop this portability macro.

llvm-svn: 284672
2016-10-19 23:52:38 +00:00
Lang Hames bb9431acda Re-apply r278065 (Weak symbol support in RuntimeDyld) with a fix for ELF.
llvm-svn: 278149
2016-08-09 19:27:17 +00:00
Lang Hames 072728d419 Revert r278065 while I investigate some build-bot breakage.
llvm-svn: 278069
2016-08-08 22:57:30 +00:00
Lang Hames 33c0b6bfca [RuntimeDyld][Orc][MCJIT] Add partial weak-symbol support to RuntimeDyld.
This patch causes RuntimeDyld to check for existing definitions when it
encounters weak symbols. If a definition already exists then the new weak
definition is discarded. All symbol lookups within a "logical dylib" should now
agree on the address of any given weak symbol. This allows the JIT to better
match the behavior of the static linker for C++ code.

This support is only partial, as it does not allow strong definitions that
occur after the first weak definition (in JIT symbol lookup order) to override
the previous weak definitions. Support for this will be added in a future
patch.

llvm-svn: 278065
2016-08-08 22:53:37 +00:00
Lang Hames 71f089c82b [RuntimeDyld] Remove symbol that is unused as of r277943.
llvm-svn: 277944
2016-08-07 01:12:44 +00:00
Lang Hames 00769a0904 [RuntimeDyld] Replace manual flag checks with JITSymbolFlags::fromObjectSymbol.
llvm-svn: 277943
2016-08-07 00:18:14 +00:00
Lang Hames aac59a26a5 [ExecutionEngine] Refactor - Roll JITSymbolFlags functionality into JITSymbol.h
and remove the JITSymbolFlags header.

llvm-svn: 277766
2016-08-04 20:32:37 +00:00
Lang Hames 7643d98d86 [Orc] Fix common symbol support in ORC.
Common symbol support in ORC was broken in r270716 when the symbol resolution
rules in RuntimeDyld were changed. With the switch to lazily materialized
symbols in r277386, common symbols can be supported by having
RuntimeDyld::emitCommonSymbols search for (but not materialize!) definitions
elsewhere in the logical dylib.

This patch adds the 'Common' flag to JITSymbolFlags, and the necessary check
to RuntimeDyld::emitCommonSymbols.

llvm-svn: 277397
2016-08-01 22:23:24 +00:00
Lang Hames ad4a911fea [ExecutionEngine][MCJIT][Orc] Replace RuntimeDyld::SymbolInfo with JITSymbol.
This patch replaces RuntimeDyld::SymbolInfo with JITSymbol: A symbol class
that is capable of lazy materialization (i.e. the symbol definition needn't be
emitted until the address is requested). This can be used to support common
and weak symbols in the JIT (though this is not implemented in this patch).

For consistency, RuntimeDyld::SymbolResolver is renamed to JITSymbolResolver.

For space efficiency a new class, JITEvaluatedSymbol, is introduced that
behaves like the old RuntimeDyld::SymbolInfo - i.e. it is just a pair of an
address and symbol flags. Instances of JITEvaluatedSymbol can be used in
symbol-tables to avoid paying the space cost of the materializer.

llvm-svn: 277386
2016-08-01 20:49:11 +00:00
Nitesh Jain 7481e40ef8 [LLVM][MIPS] Fix createStubFunction to emit JR encoding based on Arch.
Reviewers: vkalintiris, dsanders

Subscribers: jaydeep, bhushan, mohit.bhakkad, slthakur, llvm-commits

Differential Revision: https://reviews.llvm.org/D21172

llvm-svn: 275559
2016-07-15 12:56:37 +00:00
NAKAMURA Takumi 4cb46e6747 Reformat blank lines.
llvm-svn: 274481
2016-07-04 01:26:33 +00:00
NAKAMURA Takumi f252951e90 Reformat comment lines.
llvm-svn: 274480
2016-07-04 01:26:27 +00:00
NAKAMURA Takumi 940cd9368d Untabify.
llvm-svn: 274479
2016-07-04 01:26:21 +00:00
NAKAMURA Takumi f4c6441b01 Reformat.
llvm-svn: 274478
2016-07-04 01:26:14 +00:00
Kevin Enderby 931cb65df2 Thread Expected<...> up from libObject’s getSymbolAddress() for symbols to allow
a good error message to be produced.

This is nearly the last libObject interface that used ErrorOr and the last one
that appears in llvm/include/llvm/Object/MachO.h .  For Mach-O objects this is
just a clean up because it’s version of getSymbolAddress() can’t return an
error.

I will leave it to the experts on COFF and ELF to actually add meaning full
error messages in their tests if they wish.  And also leave it to these experts
to change the last two ErrorOr interfaces in llvm/include/llvm/Object/ObjectFile.h
for createCOFFObjectFile() and createELFObjectFile() if they wish.

Since there are no test cases for COFF and ELF error cases with respect to
getSymbolAddress() in the test suite this is no functional change (NFC).

llvm-svn: 273701
2016-06-24 18:24:42 +00:00
Lang Hames bf9d1aa931 [RuntimeDyld] Call the SymbolResolver::findSymbolInLogicalDylib method when
searching for external symbols, and fall back to the SymbolResolver::findSymbol
method if the former returns null.

This makes RuntimeDyld behave more like a static linker: Symbol definitions
from within the current module's "logical dylib" will be preferred to
external definitions. We can build on this behavior in the future to properly
support weak symbol handling.

Custom symbol resolvers that override the findSymbolInLogicalDylib method may
notice changes due to this patch. Clients who have not overridden this method
should generally be unaffected, however users of the OrcMCJITReplacement class
may notice changes.

llvm-svn: 270716
2016-05-25 16:23:59 +00:00
Peter Collingbourne 4718f8b5f1 Add FIXMEs to all derived classes of std::error_category.
This helps make clear that we're moving away from std::error_code.

Differential Revision: http://reviews.llvm.org/D20592

llvm-svn: 270604
2016-05-24 20:13:46 +00:00
Kevin Enderby 7bd8d99497 Thread Expected<...> up from libObject’s getType() for symbols to allow llvm-objdump to produce a good error message.
Produce another specific error message for a malformed Mach-O file when a symbol’s
section index is more than the number of sections.  The existing test case in test/Object/macho-invalid.test
for macho-invalid-section-index-getSectionRawName now reports the error with the message indicating
that a symbol at a specific index has a bad section index and that bad section index value.

Again converting interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. Where the existing code reported the error with a
string message or an error code it was converted to do the same.

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values.  So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
"// TODO: Actually report errors helpfully" and a call something like
consumeError(NameOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

llvm-svn: 268298
2016-05-02 20:28:12 +00:00
Lang Hames 8959531c51 [RuntimeDyld] Plumb Error/Expected through the internals of RuntimeDyld.
Also replaces a number of calls to report_fatal_error with Error returns.

The plumbing will make it easier to return errors originating in libObject.

Replacing report_fatal_errors with Error returns will give JIT clients the
opportunity to recover gracefully when the JIT is unable to produce/relocate
code, as well as providing meaningful error messages that can be used to file
bug reports.

llvm-svn: 267776
2016-04-27 20:24:48 +00:00
Lang Hames 543e0dcc0a [RuntimeDyld] Fix conservative over-allocation of memory for common symbols.
The previous allocation code was over-estimating the amount of memory required.

No test case: we don't currently have a good way to detect conervative
over-allocation.

llvm-svn: 267041
2016-04-21 20:08:06 +00:00
Kevin Enderby 81e8b7d949 Thread Expected<...> up from libObject’s getName() for symbols to allow llvm-objdump to produce a good error message.
Produce another specific error message for a malformed Mach-O file when a symbol’s
string index is past the end of the string table.  The existing test case in test/Object/macho-invalid.test
for macho-invalid-symbol-name-past-eof now reports the error with the message indicating
that a symbol at a specific index has a bad sting index and that bad string index value.
 
Again converting interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. Where the existing code reported the error with a
string message or an error code it was converted to do the same.  There is some
code for this that could be factored into a routine but I would like to leave that for
the code owners post-commit to do as they want for handling an llvm::Error.  An
example of how this could be done is shown in the diff in
lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h which had a Check() routine
already for std::error_code so I added one like it for llvm::Error .

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values.  So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
“// TODO: Actually report errors helpfully” and a call something like
consumeError(NameOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

Note there fixes needed to lld that goes along with this that I will commit right after this.
So expect lld not to built after this commit and before the next one.

llvm-svn: 266919
2016-04-20 21:24:34 +00:00
Kevin Enderby 5afbc1cda7 Fix a crash in running llvm-objdump -t with an invalid Mach-O file already
in the test suite. While this is not really an interesting tool and option to run
on a Mach-O file to show the symbol table in a generic libObject format
it shouldn’t crash.

The reason for the crash was in MachOObjectFile::getSymbolType() when it was
calling MachOObjectFile::getSymbolSection() without checking its return value
for the error case.

What makes this fix require a fair bit of diffs is that the method getSymbolType() is
in the class ObjectFile defined without an ErrorOr<> so I needed to add that all
the sub classes.  And all of the uses needed to be updated and the return value
needed to be checked for the error case.

The MachOObjectFile version of getSymbolType() “can” get an error in trying to
come up with the libObject’s internal SymbolRef::Type when the Mach-O symbol
symbol type is an N_SECT type because the code is trying to select from the
SymbolRef::ST_Data or SymbolRef::ST_Function values for the SymbolRef::Type.
And it needs the Mach-O section to use isData() and isBSS to determine if
it will return SymbolRef::ST_Data.

One other possible fix I considered is to simply return SymbolRef::ST_Other
when MachOObjectFile::getSymbolSection() returned an error.  But since in
the past when I did such changes that “ate an error in the libObject code” I
was asked instead to push the error out of the libObject code I chose not
to implement the fix this way.

As currently written both the COFF and ELF versions of getSymbolType()
can’t get an error.  But if isReservedSectionNumber() wanted to check for
the two known negative values rather than allowing all negative values or
the code wanted to add the same check as in getSymbolAddress() to use
getSection() and check for the error then these versions of getSymbolType()
could return errors.

At the end of the day the error printed now is the generic “Invalid data was
encountered while parsing the file” for object_error::parse_failed.  In the
future when we thread Lang’s new TypedError for recoverable error handling
though libObject this will improve.  And where the added // Diagnostic(…
comment is, it would be changed to produce and error message
like “bad section index (42) for symbol at index 8” for this case.

llvm-svn: 264187
2016-03-23 20:27:00 +00:00
Lang Hames b0934294b2 [RuntimeDyld] Add a notifyObjectLoaded method to RuntimeDyld::MemoryManager.
This is a more generic version of the MCJITMemoryManager::notifyObjectLoaded
method: It provides only a RuntimeDyld reference (rather than an
ExecutionEngine), and so can be used with ORC JIT stacks.

llvm-svn: 257296
2016-01-10 23:59:41 +00:00
Lang Hames b2b7a3c179 [RuntimeDyld] Add alignment arguments to the reserveAllocationSpace method of
RuntimeDyld::MemoryManager.

The RuntimeDyld::MemoryManager::reserveAllocationSpace method is called when
object files are loaded, and gives clients a chance to pre-allocate memory for
all segments. Previously only the size of each segment (code, ro-data, rw-data)
was supplied but not the alignment. This hasn't caused any problems so far, as
most clients allocate via the MemoryBlock interface which returns page-aligned
blocks. Adding alignment arguments enables finer grained allocation while still
satisfying alignment restrictions.

llvm-svn: 257294
2016-01-10 18:51:50 +00:00
Lang Hames 859d73ce95 [Orc][RuntimeDyld] Prevent duplicate calls to finalizeMemory on shared memory
managers.

Prior to this patch, recursive finalization (where finalization of one
RuntimeDyld instance triggers finalization of another instance on which the
first depends) could trigger memory access failures: When the inner (dependent)
RuntimeDyld instance and its memory manager are finalized, memory allocated
(but not yet relocated) by the outer instance is locked, and relocation in the
outer instance fails with a memory access error.

This patch adds a latch to the RuntimeDyld::MemoryManager base class that is
checked by a new method: RuntimeDyld::finalizeWithMemoryManagerLocking, ensuring
that shared memory managers are only finalized by the outermost RuntimeDyld
instance.

This allows ORC clients to supply the same memory manager to multiple calls to
addModuleSet. In particular it enables the use of user-supplied memory managers
with the CompileOnDemandLayer which must reuse the supplied memory manager for
each function that is lazily compiled.

llvm-svn: 257263
2016-01-09 19:50:40 +00:00
Keno Fischer eb59d468d9 [RuntimeDyld] DenseMap -> std::unordered_map
DenseMap is most applicable when both keys and values are small.
In this case, the value violates that assumption, causing quite
significant memory overhead. A std::unordered_map is more appropriate
in this case (or at least fixed the memory problems I was seeing).

Differential Revision: http://reviews.llvm.org/D14910

llvm-svn: 254651
2015-12-03 21:27:59 +00:00
Sanjoy Das d5658b0896 [RuntimeDyld] Don't allocate unnecessary stub buffer space
Summary:
For relocation types that are known to not require stub functions, there
is no need to allocate extra space for the stub functions.

Reviewers: lhames, reames, maksfb

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14676

llvm-svn: 253920
2015-11-23 21:47:51 +00:00
Sanjoy Das 8082592ac9 [RuntimeDyld] Add bounds checking to SectionEntry::advanceStubOffset
Summary:
Change SectionEntry to keep track of the size of its underlying
allocation, and use that to bounds check advanceStubOffset.

Reviewers: lhames, andrew.w.kaylor, reames

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14675

llvm-svn: 253919
2015-11-23 21:47:46 +00:00
Sanjoy Das 277776a520 [RuntimeDyld] Add accessors to `SectionEntry`; NFC
Summary:
Remove naked access to the data members in `SectionEntry` and route
accesses through accessor functions.  This makes it obvious how the
instances of the class are used, and will also facilitate adding bounds
checking to `advanceStubOffset` in a later change.

Reviewers: lhames, loladiro, andrew.w.kaylor

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14674

llvm-svn: 253918
2015-11-23 21:47:41 +00:00
Keno Fischer ddad187ce7 [RuntimeDyld] Ignore ST_FILE symbols when constructing GlobalSymbolTable
Summary: ELF's STT_File symbols may overlap with regular globals in
other files, so we should ignore them here in order to avoid having
bogus entries in the symbol table that confuse us when resolving relocations.

Reviewers: lhames

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13888

llvm-svn: 250942
2015-10-21 20:22:04 +00:00
Lang Hames a32d71be4c [RuntimeDyld] Add support for absolute symbols.
llvm-svn: 250639
2015-10-18 01:41:37 +00:00
Lang Hames 86a4593dd2 [RuntimeDyld] Don't try to get the contents of sections that don't have any
(e.g. bss sections).

MachO and ELF have been silently letting this pass, but COFFObjectFile contains
an assertion to catch this kind of (ab)use of the getSectionContents, and this
was causing the JIT to crash on COFF objects with BSS sections. This patch
should fix that.

llvm-svn: 250371
2015-10-15 06:41:45 +00:00
Keno Fischer 2cd66e9270 [RuntimeDyld] Fix performance problem in resolveRelocations with many sections
Summary:
Rather than just iterating over all sections and checking whether we have relocations for them, iterate over the relocation map instead. This showed up heavily in an artificial julia benchmark that does lots of compilation. On that particular benchmark, this patch gives
~15% performance improvements. As far as I can tell the primary reason why the original
loop was so expensive is that Relocations[i] actually constructs a relocationList (allocating memory & doing lots of other unnecessary computing) if none is found.

Reviewers: lhames

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D13545

llvm-svn: 249942
2015-10-10 05:37:02 +00:00
Lang Hames 79fce4711b [RuntimeDyld] Fix a bug in debugging output: all sections should be dumped
before any relocations have been applied, and again after all relocations have
been applied.

Previously each section was dumped before and after relocations targetting it
were applied, but this only shows the impact of relocations that point to other
symbols in the same section.

llvm-svn: 247335
2015-09-10 20:44:36 +00:00
Lang Hames 7e66c6c5e3 [RuntimeDyld] Make sure code-sections aren't under-aligned.
Code-section alignment should be at least as high as the minimum
stub alignment. If the section alignment is lower it can cause
padding to be emitted resulting in alignment errors if the section
is mapped to a higher alignment on the target.

E.g. If a text section with a 4-byte alignment gets 4-bytes of
padding to guarantee 8-byte alignment for stubs but is re-mapped to
an 8-byte alignment on the target, the 4-bytes of padding will push
the stubs to 4-byte alignment causing a crash.

No test case: There is currently no way to control host section
alignment in llvm-rtdyld. This could be made testable by adding
a custom memory manager. I'll look at that in a follow-up patch.

llvm-svn: 245031
2015-08-14 06:26:42 +00:00
Petar Jovanovic d22164dc3b [mips][mcjit] Calculate correct addend for HI16 and PCHI16 reloc
Previously, for O32 ABI we did not calculate correct addend for R_MIPS_HI16
and R_MIPS_PCHI16 relocations. This patch fixes that.

Patch by Vladimir Radosavljevic.

Differential Revision: http://reviews.llvm.org/D11186

llvm-svn: 244897
2015-08-13 15:12:49 +00:00
Rafael Espindola 8bab889b0f Convert getSymbolSection to return an ErrorOr.
This function can actually fail since the symbol contains an index to the
section and that can be invalid.

llvm-svn: 244375
2015-08-07 23:27:14 +00:00
Lang Hames 2e88f4fc5f [RuntimeDyld] Make LoadedObjectInfo::getLoadedSectionAddress take a SectionRef
rather than a string section name.

llvm-svn: 243456
2015-07-28 17:52:11 +00:00
Rafael Espindola ef888a4db6 Simplify by passing in the section of the symbol. NFC.
llvm-svn: 241603
2015-07-07 16:45:55 +00:00
Lang Hames 78937c2ae5 [RuntimeDyld] Skip relocations for external symbols with 64-bit address ~0ULL.
Requested by Eugene Rozenfeld of the LLILC team, this feature allows JIT
clients to skip relocations for selected external symbols by returning ~0ULL
from their symbol resolver. If this value is returned for a given symbol,
RuntimeDyld will skip all relocations for that symbol. The client will be
responsible for applying the skipped relocations manually before the code
is executed.

llvm-svn: 241383
2015-07-04 01:35:26 +00:00
Rafael Espindola ed067c45d4 Return ErrorOr from getSymbolAddress.
It can fail trying to get the section on ELF and COFF. This makes sure the
error is handled.

llvm-svn: 241366
2015-07-03 18:19:00 +00:00
Rafael Espindola 5d0c2ffadf Return ErrorOr from SymbolRef::getName.
This function can really fail since the string table offset can be out of
bounds.

Using ErrorOr makes sure the error is checked.

Hopefully a lot of the boilerplate code in tools/* can go away once we have
a diagnostic manager in Object.

llvm-svn: 241297
2015-07-02 20:55:21 +00:00
Rafael Espindola edd5f84419 Expose getFlags via ELFSectionRef.
llvm-svn: 240779
2015-06-26 12:44:10 +00:00
Rafael Espindola 41401e9c80 Add a ELFSectionRef class and use it to expose getSectionType.
llvm-svn: 240778
2015-06-26 12:33:37 +00:00
Rafael Espindola 2fa80cc5fd Simplify getSymbolType.
This is still a really odd function. Most calls are in object format specific
contexts and should probably be replaced with a more direct query, but at least
now this is not too obnoxious to use.

llvm-svn: 240777
2015-06-26 12:18:49 +00:00
Rafael Espindola d7a32ea4b8 Change how symbol sizes are handled in lib/Object.
COFF and MachO only define symbol sizes for common symbols. Reflect that
in the class hierarchy by having a method for common symbols only in the base
and a general one in ELF.

This avoids the need of using a magic value for the size, which had a few
problems
* Most callers didn't check for it.
* The ones that did could not tell the magic value from a file actually having
  that value.

llvm-svn: 240529
2015-06-24 10:20:30 +00:00
Rui Ueyama 7d09919534 Remove object_error::success and use std::error_code() instead
make_error_code(object_error) is slow because object::object_category()
uses a ManagedStatic variable. But the real problem is that the function is
called too frequently. This patch uses std::error_code() instead of
object_error::success. In most cases, we return "success", so this patch
reduces number of function calls to that function.

http://reviews.llvm.org/D10333

llvm-svn: 239409
2015-06-09 15:20:42 +00:00
Rafael Espindola 5eb02e45e3 Simplify another function that doesn't fail.
llvm-svn: 238703
2015-06-01 00:27:26 +00:00
Rafael Espindola a4d22472f3 Simplify interface of function that doesn't fail.
llvm-svn: 238700
2015-05-31 23:52:50 +00:00
Petar Jovanovic 9720283e99 [Mips64] Add support for MCJIT for MIPS64r2 and MIPS64r6
Add support for resolving MIPS64r2 and MIPS64r6 relocations in MCJIT.

Patch by Vladimir Radosavljevic.

Differential Revision: http://reviews.llvm.org/D9667

llvm-svn: 238424
2015-05-28 13:48:41 +00:00
Keno Fischer e6892c8149 Reapply [RuntimeDyldELF] Fold Placeholder into Addend
This reapplies r235060 and 235070, which were reverted because of test failures
in LLDB. The failure was caused because at  moment RuntimeDyld is processing
relocations for all sections, irrespective of whether we actually load them
into memory or not, but RuntimeDyld was not actually remembering where in memory
the unrelocated section is. This commit includes a fix for that issue by
remembering that pointer, though the longer term fix should be to stop processing
unneeded sections.

Original Summary:

This allows us to get rid of the original unrelocated object file after
we're done processing relocations (but before applying them).
MachO and COFF already do not require this (currently we have temporary hacks
to prevent ownership from being released, but those are brittle and should be
removed soon).

The placeholder mechanism allowed the relocation resolver to look at original
object file to obtain more information that are required to apply the
relocations. This is usually necessary in two cases:

- For relocations targetting sub-word memory locations, there may be pieces
  of the instruction at the target address which we should not override.
- Some relocations on some platforms allow an extra addend to be encoded in
  their immediate fields.

The problem is that in the second case the information cannot be recovered
after the relocations have been applied once because they will have been
overridden. In the first case we also need to be careful to not use any bits
that aren't fixed and may have been overriden by applying a first relocation.

In the past both have been fixed by just looking at original object file. This
patch attempts to recover the information from the first by looking at the
relocated object file, while the extra addend in the second case is read
upon relocation processing and addend to the regular addend.

I have tested this on X86. Other platforms represent my best understanding
of how those relocations should work, but I may have missed something because
I do not have access to those platforms.
We will keep the ugly workarounds in place for a couple of days, so this commit
can be reverted if it breaks the bots.

Differential Revision: http://reviews.llvm.org/D9028

llvm-svn: 236341
2015-05-01 20:21:45 +00:00
Keno Fischer 02628def32 [RuntimeDyldELF] Improve GOT support
Summary:
This is the first in a series of patches to eventually add support for TLS relocations to RuntimeDyld. This patch resolves an issue in the current GOT handling, where GOT entries would be reused between object files, which leads to the same situation that necessitates the GOT in the first place, i.e. that the 32-bit offset can not cover all of the address space. Thus this patch makes the GOT object-file-local.
Unfortunately, this still isn't quite enough, because the MemoryManager does not yet guarantee that sections are allocated sufficiently close to each other, even if they belong to the same object file. To address this concern, this patch also adds a small API abstraction on top of the GOT allocation mechanism that will allow (temporarily, until the MemoryManager is improved) using the stub mechanism instead of allocating a different section. The actual switch from separate section to stub mechanism will be part of a follow-on commit, so that it can be easily reverted independently at the appropriate time.

Test Plan: Includes a test case where the GOT of two object files is artificially forced to be apart by several GB.

Reviewers: lhames

Reviewed By: lhames

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D8813

llvm-svn: 234839
2015-04-14 02:10:35 +00:00
Lang Hames 2be1bbea7e [RuntimeDyld] Always allocate at least 1 byte for object sections in the JIT to
ensure that section addresses are distinct.

mapSectionAddress will fail if two sections are allocated the same address,
which can happen if any section has zero size (since malloc(0) is implementation
defined). Unfortunately I've been unable to repro this with a simple test case.

Fixes <rdar://problem/20314015>.

llvm-svn: 234299
2015-04-07 06:27:56 +00:00
Alexei Starovoitov 36df1ca5f1 [MCJIT] In debug memory dump output, don't truncate 64 bit addresses
Summary: In dumpMemorySections a cast was too short, and in resolveRelocations a format string was too short.

Test Plan:
Enable debug build and run a program which invokes MCJIT::finalizeObject(). Saw valid input as below (highlighted addresses were previously truncated):

```
Parse relocations:
Resolving relocations Section #0	**0x7f4c1337b000**
----- Contents of section socket1 before relocations -----
**0x00007f4c1337b000**: 18 01 00 00 01 01 01 0a 00 00 00 00 04 03 02 01
0x00007f4c1337b010: 7b 1a f8 ff 00 00 00 00 18 11 00 00 05 00 00 00

```

Reviewers: lhames

Reviewed By: lhames

Subscribers: llvm-commits, ast

Differential Revision: http://reviews.llvm.org/D8681

llvm-svn: 233512
2015-03-30 05:15:57 +00:00
Lang Hames 633fe146e9 [MCJIT][Orc] Refactor RTDyldMemoryManager, weave RuntimeDyld::SymbolInfo through
MCJIT.

This patch decouples the two responsibilities of the RTDyldMemoryManager class,
memory management and symbol resolution, into two new classes:
RuntimeDyld::MemoryManager and RuntimeDyld::SymbolResolver.

The symbol resolution interface is modified slightly, from:

  uint64_t getSymbolAddress(const std::string &Name);

to:

  RuntimeDyld::SymbolInfo findSymbol(const std::string &Name);

The latter passes symbol flags along with symbol addresses, allowing RuntimeDyld
and others to reason about non-strong/non-exported symbols.


The memory management interface removes the following method:

  void notifyObjectLoaded(ExecutionEngine *EE,
                          const object::ObjectFile &) {}

as it is not related to memory management. (Note: Backwards compatibility *is*
maintained for this method in MCJIT and OrcMCJITReplacement, see below).


The RTDyldMemoryManager class remains in-tree for backwards compatibility.
It inherits directly from RuntimeDyld::SymbolResolver, and indirectly from
RuntimeDyld::MemoryManager via the new MCJITMemoryManager class, which
just subclasses RuntimeDyld::MemoryManager and reintroduces the
notifyObjectLoaded method for backwards compatibility).

The EngineBuilder class retains the existing method:

  EngineBuilder&
  setMCJITMemoryManager(std::unique_ptr<RTDyldMemoryManager> mcjmm);

and includes two new methods:

  EngineBuilder&
  setMemoryManager(std::unique_ptr<MCJITMemoryManager> MM);

  EngineBuilder&
  setSymbolResolver(std::unique_ptr<RuntimeDyld::SymbolResolver> SR);

Clients should use EITHER:

A single call to setMCJITMemoryManager with an RTDyldMemoryManager.

OR (exclusive)

One call each to each of setMemoryManager and setSymbolResolver.

This patch should be fully compatible with existing uses of RTDyldMemoryManager.
If it is not it should be considered a bug, and the patch either fixed or
reverted.

If clients find the new API to be an improvement the goal will be to deprecate
and eventually remove the RTDyldMemoryManager class in favor of the new classes.

llvm-svn: 233509
2015-03-30 03:37:06 +00:00
Lang Hames b11860362d [Orc][MCJIT][RuntimeDyld] Re-apply r231726 and r231724 with fix suggested by
Dave Blaikie. Thanks Dave!

llvm-svn: 231896
2015-03-11 00:43:26 +00:00
Eric Christopher 0c3c1893c4 Temporarily revert r231726 and r231724 as they're breaking the build.:
Author: Lang Hames <lhames@gmail.com>
Date:   Mon Mar 9 23:51:09 2015 +0000

    [Orc][MCJIT][RuntimeDyld] Add header that was accidentally left out of r231724.

Author: Lang Hames <lhames@gmail.com>
Date:   Mon Mar 9 23:44:13 2015 +0000

    [Orc][MCJIT][RuntimeDyld] Add symbol flags to symbols in RuntimeDyld. Thread the
    new types through MCJIT and Orc.

    In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
    will allow us to distinguish between weak and strong definitions and find the
    right ones during symbol resolution.

llvm-svn: 231731
2015-03-10 00:33:27 +00:00
Lang Hames 3197fb4a89 [Orc][MCJIT][RuntimeDyld] Add symbol flags to symbols in RuntimeDyld. Thread the
new types through MCJIT and Orc.

In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
will allow us to distinguish between weak and strong definitions and find the
right ones during symbol resolution.

llvm-svn: 231724
2015-03-09 23:44:13 +00:00
David Majnemer 1a666e0f69 ExecutionEngine: Preliminary support for dynamically loadable coff objects
Provide basic support for dynamically loadable coff objects. Only handles a subset of x64 currently.

Patch by Andy Ayers!

Differential Revision: http://reviews.llvm.org/D7793

llvm-svn: 231574
2015-03-07 20:21:27 +00:00
Rafael Espindola 127b6c3ba7 Don't deference the section_end() iterator.
Hard to test given the undefined behavior nature.

llvm-svn: 229530
2015-02-17 20:07:28 +00:00
Lang Hames 33c9433ed4 Revert r227247 and r227228: "Add weak symbol support to RuntimeDyld".
This has wider implications than I expected when I reviewed the patch: It can
cause JIT crashes where clients have used the default value for AbortOnFailure
during symbol lookup. I'm currently investigating alternative approaches and I
hope to have this back in tree soon.

llvm-svn: 227287
2015-01-28 01:30:37 +00:00
Keno Fischer 88cc26811b [ExecutionEngine] Add weak symbol support to RuntimeDyld
Support weak symbols by first looking up if there is an externally visible symbol we can find,
and only if that fails using the one in the object file we're loading.

Reviewed By: lhames
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6950

llvm-svn: 227228
2015-01-27 20:02:31 +00:00
Lang Hames 93de2a12a3 [Orc] New JIT APIs.
This patch adds a new set of JIT APIs to LLVM. The aim of these new APIs is to
cleanly support a wider range of JIT use cases in LLVM, and encourage the
development and contribution of re-usable infrastructure for LLVM JIT use-cases.

These APIs are intended to live alongside the MCJIT APIs, and should not affect
existing clients.

Included in this patch:

1) New headers in include/llvm/ExecutionEngine/Orc that provide a set of
   components for building JIT infrastructure.
   Implementation code for these headers lives in lib/ExecutionEngine/Orc.

2) A prototype re-implementation of MCJIT (OrcMCJITReplacement) built out of the
   new components.

3) Minor changes to RTDyldMemoryManager needed to support the new components.
   These changes should not impact existing clients.

4) A new flag for lli, -use-orcmcjit, which will cause lli to use the
   OrcMCJITReplacement class as its underlying execution engine, rather than
   MCJIT itself.

Tests to follow shortly.

Special thanks to Michael Ilseman, Pete Cooper, David Blaikie, Eric Christopher,
Justin Bogner, and Jim Grosbach for extensive feedback and discussion.

llvm-svn: 226940
2015-01-23 21:25:00 +00:00
Lang Hames 2996895f28 [RuntimeDyld] Tidy up emitCommonSymbols a little. NFC.
llvm-svn: 226358
2015-01-17 00:55:05 +00:00
Lang Hames 1f7eab338f [RuntimeDyld] Remove the brace initialization that was introduced in r226341.
Evidently MSVC doesn't like it.

llvm-svn: 226349
2015-01-17 00:32:56 +00:00
Lang Hames 6bfd398022 [RuntimeDyld] Track symbol visibility in RuntimeDyld.
RuntimeDyld symbol info previously consisted of just a Section/Offset pair. This
patch replaces that pair type with a SymbolInfo class that also tracks symbol
visibility. A new method, RuntimeDyld::getExportedSymbolLoadAddress, is
introduced which only returns a non-zero result for exported symbols. For
non-exported or non-existant symbols this method will return zero. The
RuntimeDyld::getSymbolAddress method retains its current behavior, returning
non-zero results for all symbols regardless of visibility.

No in-tree clients of RuntimeDyld are changed. The newly introduced
functionality will be used by the Orc APIs.

No test case: Since this patch doesn't modify the behavior for any in-tree
clients we don't have a good tool to test this with yet. Once Orc is in we can
use it to write regression tests that test these changes.

llvm-svn: 226341
2015-01-16 23:13:56 +00:00
Rafael Espindola 0e77a94fd6 Move three methods only used by MCJIT to MCJIT.
These methods are only used by MCJIT and are very specific to it. In fact, they
are also fairly specific to the fact that we have a dynamic linker of
relocatable objects.

llvm-svn: 223964
2014-12-10 20:46:55 +00:00
Lang Hames a5cd950c73 [MCJIT] Remove the local symbol table from RuntimeDlyd - it's not needed.
All symbols have to be stored in the global symbol to enable
cross-rtdyld-instance linking, so the local symbol table content is
redundant.

llvm-svn: 222867
2014-11-27 05:40:13 +00:00
Lang Hames b5c7b1ff83 [MCJIT] Reapply r222828 and r222810-r222812 with fix for MSVC move-op issues.
llvm-svn: 222840
2014-11-26 16:54:40 +00:00
Aaron Ballman 9fb411431d Reverting r222828 and r222810-r222812 as they broke the build on Windows.
http://bb.pgr.jp/builders/ninja-clang-i686-msc17-R/builds/11753

llvm-svn: 222833
2014-11-26 15:27:39 +00:00
Lang Hames 829a19ae74 [MCJIT] Clean up RuntimeDyld's quirky object-ownership/modification scheme.
Previously, when loading an object file, RuntimeDyld (1) took ownership of the
ObjectFile instance (and associated MemoryBuffer), (2) potentially modified the
object in-place, and (3) returned an ObjectImage that managed ownership of the
now-modified object and provided some convenience methods. This scheme accreted
over several years as features were tacked on to RuntimeDyld, and was both
unintuitive and unsafe (See e.g. http://llvm.org/PR20722).

This patch fixes the issue by removing all ownership and in-place modification
of object files from RuntimeDyld. Existing behavior, including debugger
registration, is preserved.

Noteworthy changes include:

(1) ObjectFile instances are now passed to RuntimeDyld by const-ref.
(2) The ObjectImage and ObjectBuffer classes have been removed entirely, they
    existed to model ownership within RuntimeDyld, and so are no longer needed.
(3) RuntimeDyld::loadObject now returns an instance of a new class,
    RuntimeDyld::LoadedObjectInfo, which can be used to construct a modified
    object suitable for registration with the debugger, following the existing
    debugger registration scheme.
(4) The JITRegistrar class has been removed, and the GDBRegistrar class has been
    re-written as a JITEventListener.

This should fix http://llvm.org/PR20722 .

llvm-svn: 222810
2014-11-26 06:53:26 +00:00
Michael J. Spencer 21245af8e7 Fix covered switch warning
llvm-svn: 222209
2014-11-18 01:26:46 +00:00
Daniel Sanders 66e799ff1b [JIT] Fix more missing endian conversions (opcodes for AArch64, ARM, and Mips stub functions, and ARM target in general)
Summary:
Fixed all of the missing endian conversions that Lang Hames and I identified in
RuntimeDyldMachOARM.h.

Fixed the opcode emission in RuntimeDyldImpl::createStubFunction() for AArch64,
ARM, Mips when the host endian doesn't match the target endian.
PowerPC will need changing if it's opcodes are affected by endianness but I've
left this for now since I'm unsure if this is the case and it's the only path
that specifies the target endian.

This patch fixes MachO_ARM_PIC_relocations.s on a big-endian Mips host. This
is the last of the known issues on this host.

Reviewers: lhames

Reviewed By: lhames

Subscribers: aemerson, llvm-commits

Differential Revision: http://reviews.llvm.org/D6130

llvm-svn: 221446
2014-11-06 09:53:05 +00:00
Lang Hames 2d0d096bd1 [MCJIT] Temporarily revert r220245 - it broke several bots.
(See e.g. http://bb.pgr.jp/builders/cmake-llvm-x86_64-linux/builds/17653)

llvm-svn: 220249
2014-10-21 00:24:02 +00:00
Lang Hames 84801c217c [MCJIT] Make MCJIT honor symbol visibility settings when populating the global
symbol table.

Patch by Anthony Pesch. Thanks Anthony!

llvm-svn: 220245
2014-10-20 23:39:54 +00:00
Rafael Espindola 802912743e Remove bogus std::error_code returns form SectionRef.
There are two methods in SectionRef that can fail:

* getName: The index into the string table can be invalid.
* getContents: The section might point to invalid contents.

Every other method will always succeed and returning and std::error_code just
complicates the code. For example, a section can have an invalid alignment,
but if we are able to get to the section structure at all and create a
SectionRef, we will always be able to read that invalid alignment.

llvm-svn: 219314
2014-10-08 15:28:58 +00:00
Rafael Espindola fdf5f7a9dd Fix indentation.
llvm-svn: 219312
2014-10-08 15:12:20 +00:00
Lang Hames 24f0c24de9 [MCJIT] Don't crash in debugging output for sections that aren't emitted.
llvm-svn: 218836
2014-10-01 21:57:47 +00:00
Lang Hames da01602647 [MCJIT] Fix some more RuntimeDyld debugging output format specifiers.
llvm-svn: 218328
2014-09-23 19:20:57 +00:00
Lang Hames 8d4d0260a1 [MCJIT] Fix a debugging-output formatting bug in RuntimeDyld.
The mismatched mask (7 vs (ColsPerRow-1)) could lead to partial lines being
printed out of place.

llvm-svn: 218061
2014-09-18 16:43:24 +00:00
Lang Hames 69abd72e49 [MCJIT] Fix a bug RuntimeDyldImpl's read/writeBytesUnaligned methods.
The previous implementation was writing to the high-bytes of integers on BE
targets (when run on LE hosts).

http://llvm.org/PR20640

llvm-svn: 217325
2014-09-07 02:05:26 +00:00
Lang Hames 3e930a3e83 [MCJIT] Const-ify the symbol lookup operations on RuntimeDyld.
llvm-svn: 217263
2014-09-05 18:00:16 +00:00
David Blaikie 847d2a061b unique_ptrify RuntimeDyld::Dyld
llvm-svn: 217180
2014-09-04 18:37:29 +00:00
Lang Hames 778ef5b240 [MCJIT] Add command-line argument to llvm-rtdyld to specify target addresses for
sections.

This allows fine-grained control of the memory layout of hypothetical target
processes for testing purposes.

llvm-svn: 217122
2014-09-04 04:19:54 +00:00
David Blaikie fba8b20d98 unique_ptrify RuntimeDyldImpl::loadObject
I'm not sure this is a particularly helpful API (to pass ownership and
then return it unconditionally) rather than just pass the underlying
object by non-const reference, but this was the original API so I'll
just make it more safe/stable and anyone else is free to adjust that at
their whim, of course.

llvm-svn: 217081
2014-09-03 21:34:34 +00:00
David Blaikie ed9709d928 unique_ptrify a bunch of stuff through RuntimeDyld::loadObject
llvm-svn: 217065
2014-09-03 19:48:09 +00:00
Lang Hames 587ee6ab4f [MCJIT] Add a 'section_addr' builtin function to RuntimeDyldChecker.
The syntax of the new builtin is 'section_addr(<filename>, <section-name>)'
(similar to the stub_addr builtin, but without a symbol name). It returns the
base address of the given section in the given object file. This builtin makes
it possible to refer to the contents of sections that cannot contain symbols,
e.g. sections added by the linker itself, like __eh_frame.

llvm-svn: 217010
2014-09-03 05:01:46 +00:00
Lang Hames e1287c01be [MCJIT] Move endian-aware read/writes from RuntimeDyldMachO into
RuntimeDyldImpl.

These are platform independent, and moving them to the base class allows
RuntimeDyldChecker to use them too.

llvm-svn: 216801
2014-08-29 23:17:47 +00:00
Lang Hames c5cafbb074 [MCJIT] Fix format specifiers for debug output in RuntimeDyld.
More work on http://llvm.org/PR20640

llvm-svn: 216648
2014-08-28 04:25:17 +00:00
Benjamin Kramer 01e1789d38 Silence unused function warning in Release builds.
llvm-svn: 216458
2014-08-26 14:22:05 +00:00
Lang Hames f4b3b67f57 [MCJIT] Dump section memory both before and after relocations are applied.
Also switch section memory dump format from 8 to 16 columns.

llvm-svn: 216413
2014-08-25 22:19:14 +00:00
Lang Hames 86b08f02c0 [MCJIT] Make RuntimeDyld dump section contents in -debug mode.
llvm-svn: 216400
2014-08-25 18:37:38 +00:00
Tim Northover e19bed7d33 AArch64: remove arm64 triple enumerator.
Having both Triple::arm64 and Triple::aarch64 is extremely confusing, and
invites bugs where only one is checked. In reality, the only legitimate
difference between the two (arm64 usually means iOS) is also present in the OS
part of the triple and that's what should be checked.

We still parse the "arm64" triple, just canonicalise it to Triple::aarch64, so
there aren't any LLVM-side test changes.

llvm-svn: 213743
2014-07-23 12:32:47 +00:00