During D88827 it was requested to remove the local implementation
of Memory/File Buffers:
// TODO: refactor the buffer classes in LLVM to enable us to use them here
// directly.
This patch uses raw_ostream instead of Buffers. Generally, using streams
could allow us to reduce memory usages. No need to load all data into the
memory - the data could be streamed through a smaller buffer.
Thus, this patch uses raw_ostream as an interface for output data:
Error executeObjcopyOnBinary(CopyConfig &Config,
object::Binary &In,
raw_ostream &Out);
Note 1. This patch does not change the implementation of Writers
so that data would be directly stored into raw_ostream.
This is assumed to be done later.
Note 2. It would be better if Writers would be implemented in a such way
that data could be streamed without seeking/updating. If that would be
inconvenient then raw_ostream could be replaced with raw_pwrite_stream
to have a possibility to seek back and update file headers.
This is assumed to be done later if necessary.
Note 3. Current FileOutputBuffer allows using a memory-mapped file.
The raw_fd_ostream (which could be used if data should be stored in the file)
does not allow us to use a memory-mapped file. Memory map functionality
could be implemented for raw_fd_ostream:
It is possible to add resize() method into raw_ostream.
class raw_ostream {
void resize(uint64_t size);
}
That method, implemented for raw_fd_ostream, could create a memory-mapped file.
The streamed data would be written into that memory file then.
Thus we would be able to use memory-mapped files with raw_fd_ostream.
This is assumed to be done later if necessary.
Differential Revision: https://reviews.llvm.org/D91028
The code was using the standard isalnum function which doesn't handle
values outside the non-ascii range. Switching to using llvm::isAlnum
instead ensures we don't provoke undefined behaviour, which can in some
cases result in crashes.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D97663
The check for whether an extended symbol index table was required
dropped the first SHN_LORESERVE sections from the sections array before
checking whether the remaining sections had symbols. Unfortunately, the
null section header is not present in this list, so the check was
skipping the first section that might be important. If that section
contained a symbol, and no subsequent ones did, the .symtab_shndx
section would not be emitted, leading to a corrupt object.
Also consolidate and expand test coverage in the area to cover this bug
and other aspects of the SYMTAB_SHNDX section.
Reviewed by: alexshap, MaskRay
Differential Revision: https://reviews.llvm.org/D97661
This is consistent with BFD objcopy.
Previously llvm objcopy would allocate space for SHT_NOBITS sections
often resulting in enormous binary files.
New test case (binary-paddr.test %t6).
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D95569
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)
Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).
This patch places such an empty segment according to its parent segment. This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D90897
Remove usages of special error reporting functions(error(),
reportError()). Errors are reported as Expected<>/Error returning
values. This part is for ELF subfolder of llvm-objcopy.
Testing: check-all.
Differential Revision: https://reviews.llvm.org/D87987
`ELFFile<ELFT>` has many methods that take pointers,
though they assume that arguments are never null and
hence could take references instead.
This patch performs such clean-up.
Differential revision: https://reviews.llvm.org/D87385
This patch resolves crash that occurs when user wanted to remove all
symbols and add a brand new one using:
```
llvm-objcopy -R .symtab --add-symbol foo=1234 in.o out.o
```
Before these changes the symbol table internally being null when adding
new symbols. For now we will regenerate symtab in this case.
This fixes: https://bugs.llvm.org/show_bug.cgi?id=43930
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D82935
SHT_GROUP sections contain a reference to a symbol indicating their
"signature" symbol. The symbol table containing this symbol is referred
to by the group section's sh_link field. If llvm-objcopy is instructed
to remove the symbol table, it will emit an error.
This fixes https://bugs.llvm.org/show_bug.cgi?id=46153.
Reviewed By: jhenderson, Higuoxing
Differential Revision: https://reviews.llvm.org/D82274
When a group member is removed, the corresponding record in the
SHT_GROUP section has to be deleted.
This fixes PR46064.
Differential Revision: https://reviews.llvm.org/D80568
When a SHT_GROUP section is removed, but other sections of the group are
kept, the SHF_GROUP flag of these sections should be dropped, otherwise
the resulting ELF file will be malformed.
Differential Revision: https://reviews.llvm.org/D80511
To avoid undefined behavior caught by -fsanitize=undefined on binary-paddr.test
void SectionWriter::visit(const Section &Sec) {
if (Sec.Type != SHT_NOBITS)
// Sec.Contents is empty while Sec.Offset may be out of bound
llvm::copy(Sec.Contents, Out.getBufferStart() + Sec.Offset);
}
After SHF_ALLOC sections are ordered by LMA:
* If initial sections are empty, GNU objcopy skips their contents while we
emit leading zeros. (binary-paddr.test %t4)
* If trailing sections are empty, GNU objcopy skips their contents while we
emit trailing zeros. (binary-paddr.test %t5)
This patch matches GNU objcopy's behavior. Linkers don't keep p_memsz
PT_LOAD segments. Such empty sections would not have a containing
PT_LOAD and `Section::ParentSegment` might be null if linkers fail to
optimize the file offsets (lld D79254).
In particular, without D79254, the arm Linux kernel's multi_v5_defconfig
depends on this behavior: in `vmlinux`, an empty .text_itcm is mapped at
a very high address (0xfffe0000) but the kernel does not expect
`objcopy -O binary` to create a very large `arch/arm/boot/Image`
(0xfffe0000-0xc0000000 ~= 1GiB). See https://bugs.llvm.org/show_bug.cgi?id=45632
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D79229
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.
Differential revision: https://reviews.llvm.org/D78016
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
.text sh_address=0x1000 sh_offset=0x1000
.data sh_address=0x3000 sh_offset=0x2000
In an objcopy -O binary output, the distance between two sections equal
their LMA differences (0x3000-0x1000), instead of their sh_offset
differences (0x2000-0x1000). This patch changes our behavior to match
GNU.
This rule gets more complex when the containing PT_LOAD has
p_vaddr!=p_paddr. GNU objcopy essentially computes
sh_offset-p_offset+p_paddr for each candidate section, and removes the
gap before the first address.
Added tests to binary-paddr.test to catch the compatibility problem.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D71035
--only-keep-debug produces a debug file as the output that only
preserves contents of sections useful for debugging purposes (the
binutils implementation preserves SHT_NOTE and non-SHF_ALLOC sections),
by changing their section types to SHT_NOBITS and rewritting file
offsets.
See https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html
The intended use case is:
```
llvm-objcopy --only-keep-debug a a.dbg
llvm-objcopy --strip-debug a b
llvm-objcopy --add-gnu-debuglink=a.dbg b
```
The current layout algorithm is incapable of deleting contents and
shrinking segments, so it is not suitable for implementing the
functionality.
This patch adds a new algorithm which assigns sh_offset to sections
first, then modifies p_offset/p_filesz of program headers. It bears a
resemblance to lld/ELF/Writer.cpp.
Reviewed By: jhenderson, jakehehrlich
Differential Revision: https://reviews.llvm.org/D67137
`llvm::objcopy:🧝:*Section::classof` matches Type and Flags, yet Type
and Flags are mutable (by setSectionFlagsAndTypes and upcoming
--only-keep-debug feature). Add OriginalType & OriginalFlags to be used
in classof, to prevent classof results from changing.
Reviewed By: jakehehrlich, jhenderson, alexshap
Differential Revision: https://reviews.llvm.org/D69739
* Improve comments.
* Reorder the assignment to Obj.SectionNames before the symbol table
creation code. Add a test.
Reviewed By: grimar
Differential Revision: https://reviews.llvm.org/D69526
It was revealed by D69260.
Tool crashed when scanned relocations in a object without a symbol table.
This patch teaches it either to handle such objects (when relocations
does not use symbols we do not need a symbol table to proceed)
or to show an appropriate error otherwise.
Differential revision: https://reviews.llvm.org/D69304
Exposed by D69041. If SHT_SYMTAB does not exist, ELFObjcopy.cpp:handleArgs will crash due
to a null pointer dereference.
for (const NewSymbolInfo &SI : Config.ELF->SymbolsToAdd) {
...
Obj.SymbolTable->addSymbol(
Fix this by creating .symtab and .strtab on demand in ELFBuilder<ELFT>::readSections,
if --add-symbol is specified.
Reviewed By: grimar
Differential Revision: https://reviews.llvm.org/D69093
llvm-svn: 375105
GNU objcopy documents that -B is only useful with architecture-less
input (i.e. "binary" or "ihex"). After D67144, -O defaults to -I, and
-B is essentially a NOP.
* If -O is binary/ihex, GNU objcopy ignores -B.
* If -O is elf*, -B provides the e_machine field in GNU objcopy.
So to convert a blob to an ELF, `-I binary -B i386:x86-64 -O elf64-x86-64` has to be specified.
`-I binary -B i386:x86-64 -O elf64-x86-64` creates an ELF with its
e_machine field set to EM_NONE in GNU objcopy, but a regular x86_64 ELF
in elftoolchain elfcopy. Follow the elftoolchain approach (ignoring -B)
to simplify code. Users that expect their command line portable should
specify -B.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D67215
llvm-svn: 371914
"Section" can refer to the type llvm::objcopy:🧝:Section or the
variable name. Rename it to "Sec" for clarity. "Sec" is already used a
lot, so this change improves consistency as well.
Also change `auto` to `const SectionBase` for readability.
Reviewed By: grimar
Differential Revision: https://reviews.llvm.org/D67143
llvm-svn: 370852
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
llvm-svn: 369013
This reverts r365193 (git commit 194f16b354)
This patch doesn't work with binaries built w/ `--emit-relocs`, e.g.
```
$ echo 'int main() { return 0; }' | clang -Wl,--emit-relocs -x c - -o foo && llvm-objcopy --strip-unneeded foo
llvm-objcopy: error: 'foo': not stripping symbol '__gmon_start__' because it is named in a relocation
```
llvm-svn: 365712
Summary:
The directive defines a symbol as an group/local memory (LDS) symbol.
LDS symbols behave similar to common symbols for the purposes of ELF,
using the processor-specific SHN_AMDGPU_LDS as section index.
It is the linker and/or runtime loader's job to "instantiate" LDS symbols
and resolve relocations that reference them.
It is not possible to initialize LDS memory (not even zero-initialize
as for .bss).
We want to be able to link together objects -- starting with relocatable
objects, but possible expanding to shared objects in the future -- that
access LDS memory in a flexible way.
LDS memory is in an address space that is entirely separate from the
address space that contains the program image (code and normal data),
so having program segments for it doesn't really make sense.
Furthermore, we want to be able to compile multiple kernels in a
compilation unit which have disjoint use of LDS memory. In that case,
we may want to place LDS symbols differently for different kernels
to save memory (LDS memory is very limited and physically private to
each kernel invocation), so we can't simply place LDS symbols in a
.lds section.
Hence this solution where LDS symbols always stay undefined.
Change-Id: I08cbc37a7c0c32f53f7b6123aa0afc91dbc1748f
Reviewers: arsenm, rampitec, t-tye, b-sumner, jsjodin
Subscribers: kzhuravl, jvesely, wdng, yaxunl, dstuttard, tpr, rupprecht, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61493
llvm-svn: 364296
Summary:
When llvm-objcopy sorts sections during finalization, it only sorts based on the offset, which can cause the group section to come after the sections it contains. This causes link failures when using gold to link objects created by llvm-objcopy.
Fix this for now by copying GNU objcopy's behavior of placing SHT_GROUP sections first. In the future, we may want to remove this sorting entirely to more closely preserve the input file layout.
This fixes https://bugs.llvm.org/show_bug.cgi?id=42052.
Reviewers: jakehehrlich, jhenderson, MaskRay, espindola, alexshap
Reviewed By: MaskRay
Subscribers: phuongtrang148993, emaste, arichardson, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62620
llvm-svn: 362973
This implements the functionality described in
https://lld.llvm.org/Partitions.html. It works as follows:
- Reads the section headers using the ELF header at file offset 0;
- If extracting a loadable partition:
- Finds the section containing the required partition ELF header by looking it up in the section table;
- Reads the ELF and program headers from the section.
- If extracting the main partition:
- Reads the ELF and program headers from file offset 0.
- Filters the section table according to which sections are in the program headers that it read:
- If ParentSegment != nullptr or section is not SHF_ALLOC, then it goes in.
- Sections containing partition ELF headers or program headers are excluded as there are no headers for these in ordinary ELF files.
Differential Revision: https://reviews.llvm.org/D62364
llvm-svn: 362818
This is https://bugs.llvm.org/show_bug.cgi?id=42122.
If an object file has a size less than program header's file [offset + size]
(i.e. if we have overflow), llvm-objcopy crashes instead of reporting a
error.
The patch fixes this issue.
Differential revision: https://reviews.llvm.org/D62898
llvm-svn: 362778
Alexey Lapshin