This flag is implemented similarly to --reproduce in the ELF linker.
This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.
One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.
Differential Revision: https://reviews.llvm.org/D22418
llvm-svn: 276719
r275711 for "speedng up symbol version handling" was committed
by misunderstanding; the benchmark number was measured with
a debug build. The number with a release build didn't actually change.
This patch removes false optimizations added in that patch.
llvm-svn: 276267
--trace-symbol is a command line option to watch a symbol.
Previosly, we looked up a hash table for a new symbol if the
option is given. Any code that looks up a hash table for each
symbol is expensive because the linker handles a lot of symbols.
In our design, we look up a hash table strictly only once
for a symbol, so --trace-symbol was an exception.
This patch improves efficiency of the option by merging the
hash table into the symbol table.
Instead of looking up a separate hash table with a string,
this patch sets `Traced` flag to symbols specified by --trace-symbol.
So, if you insert a symbol and get a symbol with `Traced` flag on,
you know that you need to print out a log message for the symbol.
This is nearly zero cost.
llvm-svn: 275716
Versions can be assigned to symbols in two different ways.
One is the usual version scripts, and the other is special
symbol suffix '@'. If a symbol contains '@', the string after
that is considered to specify a version name.
Previously, we look for '@' for all symbols.
Anything that works on every symbol can be expensive because
the linker has to handle a lot of symbols. The search for '@'
was not an exception.
In this patch, I made two optimizations.
The first optimization is to handle '@' only when at least one
version is defined. If no versions are defined, no versions can
be assigned to any symbols, so it's waste of time to search for '@'.
The second optimization is to scan only suffixes of symbol names
instead of entire symbol names. Symbol names can be very long, but
symbol versions are usually short, so scanning entire symbol names
is waste of time, too.
There are some error cases which we no longer be able to detect
with this patch. I don't think it's a major drawback because they
are minor errors. Speed is more important.
This change improves LLD with debug info self-link time from
6.6993 seconds to 6.3426 seconds (or -5.3%).
Differential Revision: https://reviews.llvm.org/D22433
llvm-svn: 275711
Previously, it checked for the EC parameter and set HasError
only when there was an error. But in most places we called
error only when error had occurred, so this behavior was confusing.
llvm-svn: 275517
Config members are named after corresponding command line options.
This patch renames VAStart ImageBase so that they are in line with
--image-base.
Differential Revision: http://reviews.llvm.org/D22277
llvm-svn: 275298
Patch by H.J. Lu.
This patch adds -m elf32_x86_64 to lld. But it doesn't generate working
x32 binaries.
Differential Revision: http://reviews.llvm.org/D22268
llvm-svn: 275236
When building executable usually version script is absent.
Before this patch error was shown in the case when
symbol name contained version and there was no script to match it.
Instead of error out patch allows
to create new version declaration in this case and use it.
gnu linkers do the same.
That is PR28359.
Differential revision: http://reviews.llvm.org/D21890
llvm-svn: 274828
Option has next description (http://linux.die.net/man/1/ld):
"--unresolved-symbols=method
Determine how to handle unresolved symbols. There are four possible values for method
according to documentation:
ignore-all: Do not report any unresolved symbols.
report-all: Report all unresolved symbols. This is the default.
ignore-in-object-files: Report unresolved symbols that are contained in shared libraries, but ignore them if they come from regular object files.
ignore-in-shared-libs: Report unresolved symbols that come from regular object files, but ignore them if they come from shared libraries."
Since report-all is default and we traditionally do not report about undefined symbols in lld,
report-all does not report about undefines from DSO.
ignore-in-object-files also does not do that. Handling of that option differs from what gnu linkers do.
Option works in next way in lld:
ignore-all: Do not report any unresolved symbols.
report-all: Report all unresolved symbols except symbols from DSOs. This is the default.
ignore-in-object-files: The same as ignore-all.
gnore-in-shared-libs: The same as report-all.
This is PR24524.
Differential revision: http://reviews.llvm.org/D21794
llvm-svn: 274123
Previously, we initialized Config->EKind and Config->EMachine when
we instantiate ELF objects. That was not an ideal location to do that
because the logic was buried too deep inside a concrete logic.
This patch moves the code to the driver so that the initialization
becomes explicit.
Differential Revision: http://reviews.llvm.org/D21784
llvm-svn: 274089
Option checks for cases where a version script explicitly lists
a symbol, but the symbol is not defined and errors out such
cases if any.
Differential revision: http://reviews.llvm.org/D21745
llvm-svn: 273998
Patch implements support of zlib style compressed sections.
SHF_COMPRESSED flag is used to recognize that decompression is required.
After that decompression is performed and flag is removed from output.
Differential revision: http://reviews.llvm.org/D20272
llvm-svn: 273661
Patch by Shridhar Joshi.
This option provides names of all the link time modules which define and
reference symbols requested by user. This helps to speed up application
development by detecting references causing undefined symbols.
It also helps in detecting symbols being resolved to wrong (unintended)
definitions in case of applications containing multiple definitions for
same symbols with different types, bindings.
Implements PR28226.
llvm-svn: 273536
Doing that in an anonymous version is a bit silly, but this opens the
way for supporting it in general.
Since we don't support actual versions, for now we just disable the
version script if we detect that it is missing a local.
llvm-svn: 273000
Add support for an ARM Target and the initial set of relocations
and PLT entries that are necessary for an ARM only hello world to
link. This has been tested against an ARM only sysroot from the
4.2.0 CodeSourcery Lite release.
Tests have been added to test/ELF for the support that has been
implemented.
Main limitations:
- No Thumb support
- Relocations incomplete
- No C++ exceptions support
- No TLS support
- No range extension or interworking veneer (thunk) support
- No Build Attribute support
- No Big-endian support
The deprecated relocations R_ARM_PLT32 and R_ARM_PC24 have been
implemented as these are used by the 4.2.0 CodeSourcery Lite release.
llvm-svn: 271993
Previously, mergeable section's constructors did more than just
setting member variables; it split section contents into small
pieces. It is not always computationally cheap task because if
the section is a mergeable string section, it needs to scan the
entire section to split them by NUL characters.
If a section would be thrown away by GC, that cost ended up
being a waste of time. It is going to be larger problem if the
section is compressed -- the whole time to uncompress it and
split it up is going to be a waste.
Luckily, we can defer section splitting after GC. We just have
to remember which offsets are in use during GC and apply that later.
This patch implements it.
Differential Revision: http://reviews.llvm.org/D20516
llvm-svn: 270455
If you specify the option in the form of --build-id=0x<hexstring>,
that hexstring is set as a build ID. We observed that the feature
is actually in use in some builds, so we want this feature.
llvm-svn: 269495
Just do not allow to link shared library if there are
undefined symbols.
This fixes PR27447
Differential revision: http://reviews.llvm.org/D20169
llvm-svn: 269183
This is the option which sorts relocs to optimize dynamic linker performance.
-z combelocs is the default in gold, also it ignores -z nocombreloc,
this patch do the same.
Patch sorts relocations by symbols only and do not create any
DT_REL[A]COUNT entries. That is different with what gold/bfd do.
More information about option is here:
http://www.airs.com/blog/archives/186http://people.redhat.com/jakub/prelink.pdf, p.2
Differential revision: http://reviews.llvm.org/D19528
llvm-svn: 269066
Also improves the error message. Previously it would just print out
the cause (e.g. "permission denied"). Now it prints out something like
"--reproduce: failed to open foo.cpio: permission denied".
llvm-svn: 268551
This is both simpler and safer. If we crash at any point, there is a
valid cpio file to reproduce the crash.
Thanks to Rui for the suggestion.
llvm-svn: 268495
Both bfd and gold have this. It allows disabling build-id when it is the
default with by adding -Wl,--build-id=none no the clang command line.
llvm-svn: 268435
We want --reproduce to
* not rewrite scripts and thin archives
* work with absolute paths
Given that, it pretty much has to create a full directory tree. On windows that
is problematic because of the very short maximum path limit. On most cases
users can still work around it with "--repro c:\r", but that is annoying and
not viable for automated testing.
We then need to produce some form of archive with the files. The first option
that comes to mind is .a files since we already have code for writing them.
There are a few problems with them
The format has a dedicated string table, so we cannot start writing it until
all members are known.
Regular implementations don't support creating directories. We could make
llvm-ar support that, but that is probably not a good idea.
The next natural option would be tar. The problem is that to support long path
names (which is how this started) it needs a "pax extended header" making this
an annoying format to write.
The next option I looked at seems a natural fit: cpio files.
They are available on pretty much every unix, support directories and long path
names and are really easy to write. The only slightly annoying part is a
terminator, but at least gnu cpio only prints a warning if it is missing, which
is handy for crashes. This patch still makes an effort to always create it.
llvm-svn: 268404
This patch implements a new design for the symbol table that stores
SymbolBodies within a memory region of the Symbol object. Symbols are mutated
by constructing SymbolBodies in place over existing SymbolBodies, rather
than by mutating pointers. As mentioned in the initial proposal [1], this
memory layout helps reduce the cache miss rate by improving memory locality.
Performance numbers:
old(s) new(s)
Without debug info:
chrome 7.178 6.432 (-11.5%)
LLVMgold.so 0.505 0.502 (-0.5%)
clang 0.954 0.827 (-15.4%)
llvm-as 0.052 0.045 (-15.5%)
With debug info:
scylla 5.695 5.613 (-1.5%)
clang 14.396 14.143 (-1.8%)
Performance counter results show that the fewer required indirections is
indeed the cause of the improved performance. For example, when linking
chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and
instructions per cycle increases from 0.78 to 0.83. We are also executing
many fewer instructions (15,516,401,933 down to 15,002,434,310), probably
because we spend less time allocating SymbolBodies.
The new mechanism by which symbols are added to the symbol table is by calling
add* functions on the SymbolTable.
In this patch, I handle local symbols by storing them inside "unparented"
SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating
these SymbolBodies, we can probably do that separately.
I also removed a few members from the SymbolBody class that were only being
used to pass information from the input file to the symbol table.
This patch implements the new design for the ELF linker only. I intend to
prepare a similar patch for the COFF linker.
[1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html
Differential Revision: http://reviews.llvm.org/D19752
llvm-svn: 268178
The aim of this patch is to make it easy to re-run the command without
updating paths in the command line. Here is a use case.
Assume that Alice is having an issue with lld and is reporting the issue
to developer Bob. Alice's current directly is /home/alice/work and her
command line is "ld.lld -o foo foo.o ../bar.o". She adds "--reproduce repro"
to the command line and re-run. Then the following text will be produced as
response.txt (notice that the paths are rewritten so that they are
relative to /home/alice/work/repro.)
-o home/alice/work/foo home/alice/work/foo.o home/alice/bar.o
The command also produces the following files by copying inputs.
/home/alice/repro/home/alice/work/foo.o
/home/alice/repro/home/alice/bar.o
Alice zips the directory and send it to Bob. Bob get an archive from Alice
and extract it to his home directory as /home/bob/repro. Now his directory
have the following files.
/home/bob/repro/response.txt
/home/bob/repro/home/alice/work/foo.o
/home/bob/repro/home/alice/bar.o
Bob then re-run the command with these files by the following commands.
cd /home/bob/repro
ld.lld @response.txt
This command will run the linker with the same command line options and
the same input files as Alice's, so it is very likely that Bob will see
the same issue as Alice saw.
Differential Revision: http://reviews.llvm.org/D19737
llvm-svn: 268169
This patch redefines the default optimization level as 1 and adds
new level 0. In the command line, it is -O0. The flag disables
costly but optional features so that the linker produces semantically
correct but larger output quickly. Currently it only disables
section merging.
This flag is not intended to be used for final production linking.
It is intended to be used in compile-link-test cycle.
Time to link clang with debug info is about 2x faster with the flag.
Head:
13.24 seconds
Output size: 1227189664 bytes
With this patch:
7.41 seconds
Output size: 2490281784 bytes
Differential Revision: http://reviews.llvm.org/D19705
llvm-svn: 268056
Using multiple context used to be a really big memory saving because we
could free memory from each file while the linker proceeded with the
symbol resolution. We are getting lazier about reading data from the
bitcode, so I was curious if this was still a good tradeoff.
One thing that is a bit annoying is that we still have to copy the
symbol names. The problem is that the names are stored in the Module and
get freed when we move the module bits during linking.
Long term I think the solution is to add a symbol table to the bitcode.
That way IRObject file will not need to use a Module or a Context and we
can drop it while still keeping a StringRef to the names.
This patch is still be an interesting medium term improvement.
When linking llvm-as without debug info this patch is a small speedup:
master: 29.861877513 seconds
patch: 29.814533787 seconds
With debug info the numbers are
master: 34.765181469 seconds
patch: 34.563351584 seconds
The peak memory usage when linking llvm-as with debug info was
master: 599.10MB
patch: 600.13MB
llvm-svn: 267921
The semantics of the -u flag are to load the lazy symbol named by the flag. We
were previously relying on this behavior falling out of symbol resolution
against a synthetic undefined symbol, but that didn't quite give us the
correct behavior, so we needed a flag to mark symbols created with -u so
we could treat them specially in the writer. However, it's simpler and less
error prone to implement the required behavior directly and remove the flag.
This fixes an issue where symbols loaded with -u would receive hidden
visibility even when the definition in an object file had wider visibility.
Differential Revision: http://reviews.llvm.org/D19560
llvm-svn: 267639
When --reproduce <path> is given, then we need to concatenate input
file paths to the given path to save input files to the directory.
Previously, path concatenation didn't handle Windows drive letters
so it could generate invalid paths such as "C:\D:\foo". It also didn't
handle ".." path components, so it could produce some bad paths
such as "foo/../../etc/passwd".
In this patch, Windows drive letters and ".." are removed before
concatenating paths.
Differential Revision: http://reviews.llvm.org/D19551
llvm-svn: 267600
--reproduce dumps the object files in a directory chosen
(preserving the file system layout) and the linker invocation
so that people can create an archive and upload for debugging.
Differential Revision: http://reviews.llvm.org/D19494
llvm-svn: 267497
This patch is to remove -lto-no-discard-value-names flag and
instead to use -save-temps as we discussed in the post-commit
review thread for r267020.
Differential Revision: http://reviews.llvm.org/D19437
llvm-svn: 267230
This patch only implements support for version scripts of the form:
{ [ global: symbol1; symbol2; [...]; symbolN; ] local: *; };
No wildcards are supported, other than for the local entry. Symbol versioning
is also not supported.
It works by introducing a new Symbol flag which tracks whether a symbol
appears in the global section of a version script.
This patch also simplifies the logic in SymbolBody::isPreemptible(), and
teaches it to handle the case where symbols with default visibility in DSOs
do not appear in the dynamic symbol table because of a version script.
Fixes PR27482.
Differential Revision: http://reviews.llvm.org/D19430
llvm-svn: 267208
These are properties of a symbol name, rather than a particular instance
of a symbol in an object file. We can simplify the code by collecting these
properties in Symbol.
The MustBeInDynSym flag has been renamed ExportDynamic, as its semantics
have been changed to be the same as those of --dynamic-list and
--export-dynamic-symbol, which do not cause hidden symbols to be exported.
Differential Revision: http://reviews.llvm.org/D19400
llvm-svn: 267183
Rafael reported on the mailing list that this reduces peak memory
usage while linking llvm-as by 15%. It makes sense to make it
the default, and introduce an inverse knob -lto-no-discard-value-names
for those who want to restore the old behavior.
llvm-svn: 267020
Originally, linker scripts were basically an alternative way to specify
options to the command line options. But as we add more features to hanlde
symbols and sections, many member functions needed to be templated.
Now most the members are templated. It is probably time to template the
entire class.
Previously, LinkerScript is an executor of the linker script as well as
a storage of linker script configurations. This is not suitable to template
the class because when we are reading linker script files, we don't know
the ELF type yet, so we can't instantiate ELF-templated classes.
In this patch, I defined a new class, ScriptConfiguration, to store
linker script configurations. ScriptParser writes parse results to it,
and LinkerScript uses them.
Differential Revision: http://reviews.llvm.org/D19302
llvm-svn: 266908
Parallelism level can be chosen using the new --lto-jobs=K option
where K is the number of threads used for CodeGen. It currently
defaults to 1.
llvm-svn: 266484
This patch implements the --dynamic-list option, which adds a list of
global symbol that either should not be bounded by default definition
when creating shared libraries, or add in dynamic symbol table in the
case of creating executables.
The patch modifies the ScriptParserBase class to use a list of Token
instead of StringRef, which contains information if the token is a
quoted or unquoted strings. It is used to use a faster search for
exact match symbol name.
The input file follow a similar format of linker script with some
simplifications (it does not have scope or node names). It leads
to a simplified parser define in DynamicList.{cpp,h}.
Different from ld/gold neither glob pattern nor mangled names
(extern 'C++') are currently supported.
llvm-svn: 266227
Previously each archive file was reported no matter were it's member used or not,
like:
lib/libLLVMSupport.a
Now lld prints line for each used internal file, like:
lib/libLLVMSupport.a(lib/Support/CMakeFiles/LLVMSupport.dir/StringSaver.cpp.o)
lib/libLLVMSupport.a(lib/Support/CMakeFiles/LLVMSupport.dir/Host.cpp.o)
lib/libLLVMSupport.a(lib/Support/CMakeFiles/LLVMSupport.dir/ConvertUTF.c.o)
That should be consistent with what gold do.
This fixes PR27243.
Differential revision: http://reviews.llvm.org/D19011
llvm-svn: 266220
Previously, we supported only one hash function, FNV-1, so
BuildIdSection directly handled hash computation. In this patch,
I made BuildIdSection an abstract class and defined two subclasses,
BuildIdFnv1 and BuildIdMd5.
llvm-svn: 265737
start-lib and end-lib are options to link object files in the same
semantics as archive files. If an object is in start-lib and end-lib,
the object is linked only when the file is needed to resolve
undefined symbols. That means, if an object is in start-lib and end-lib,
it behaves as if it were in an archive file.
In this patch, I introduced a new notion, LazyObjectFile. That is
analogous to Archive file type, but that works for a single object
file instead of for an archive file.
http://reviews.llvm.org/D18814
llvm-svn: 265710
Our symbol representation was redundant, and some times would get out of
sync. It had an Elf_Sym, but some fields were copied to SymbolBody.
Different parts of the code were checking the bits in SymbolBody and
others were checking Elf_Sym.
There are two general approaches to fix this:
* Copy the required information and don't store and Elf_Sym.
* Don't copy the information and always use the Elf_Smy.
The second way sounds tempting, but has a big problem: we would have to
template SymbolBody. I started doing it, but it requires templeting
*everything* and creates a bit chicken and egg problem at the driver
where we have to find ELFT before we can create an ArchiveFile for
example.
As much as possible I compared the test differences with what gold and
bfd produce to make sure they are still valid. In most cases we are just
adding hidden visibility to a local symbol, which is harmless.
In most tests this is a small speedup. The only slowdown was scylla
(1.006X). The largest speedup was clang with no --build-id, -O3 or
--gc-sections (i.e.: focus on the relocations): 1.019X.
llvm-svn: 265293
So, there are some cases when the IR Linker produces a broken
module (which doesn't pass the verifier) and we end up asserting
inside the verifier. I think it's always a bug producing a module
which does not pass the verifier but there are some cases in which
people can live with the broken module (e.g. if only DebugInfo
metadata are broken). The gold plugin has something similar.
This commit is motivated by a situation I found in the
wild. It seems that somebody else discovered it independently
and reported in PR24923.
llvm-svn: 265258
Currently we create a file called .lto.bc. In UNIX,
ls(1) by default doesn't show up files starting with
a dot, as they're (only by convention) hidden.
This makes the output of -save-temps a little bit
hard to find. Use "a.out.lto.bc" instead if the
output file is not specified.
While here, change an existing -save-temps test to
exercise this more interesting behaviour.
llvm-svn: 265254
c:\b\slave\sanitizer-windows\llvm\tools\lld\elf\Config.h(94) : error C2797: 'lld:🧝:Configuration::MLlvm': list initialization inside member initializer list or non-static data member initializer is not implemented
llvm-svn: 265207
-pie
--pic-executable
Create a position independent executable. This is currently only
supported on ELF platforms. Position independent executables are
similar to shared libraries in that they are relocated by the
dynamic linker to the virtual address the OS chooses for them
(which can vary between invocations). Like normal dynamically
linked executables they can be executed and symbols defined in the
executable cannot be overridden by shared libraries.
Differential revision: http://reviews.llvm.org/D18183
llvm-svn: 263693
Rui Ueyama