As for now, llvm-objcopy renames only sections that are specified
explicitly in --rename-section, while GNU objcopy keeps names of
relocation sections in sync with their targets. For example:
> readelf -S test.o
...
[ 1] .foo PROGBITS
[ 2] .rela.foo RELA
> objcopy --rename-section .foo=.bar test.o gnu.o
> readelf -S gnu.o
...
[ 1] .bar PROGBITS
[ 2] .rela.bar RELA
> llvm-objcopy --rename-section .foo=.bar test.o llvm.o
> readelf -S llvm.o
...
[ 1] .bar PROGBITS
[ 2] .rela.foo RELA
This patch makes llvm-objcopy to match the behavior of GNU objcopy better.
Differential Revision: https://reviews.llvm.org/D110352
This patch continues refactoring done by D99055. It puts format specific
options into the correponding CopyConfig structures.
Differential Revision: https://reviews.llvm.org/D102277
As for now, llvm-objcopy sorts section headers according to the offsets
of the sections in the input file. That can corrupt section references
in the dynamic symbol table because it is a loadable section and as such
is not updated by the tool. Even though the section references are not
required for loading the binary correctly, they are still handy for a
user who analyzes the file.
While the patch removes global reordering of section headers, it layouts
the sections in the same way as before, i.e. according to their original
offsets. All that helps the output file to resemble the input better.
Note that the patch removes sorting SHT_GROUP sections to the start of
the list, which was introduced in D62620 in order to ensure that they
come before the group members, along with the corresponding test. The
original issue was caused by the sorting of section headers, so dropping
the sorting also resolves the issue.
Differential Revision: https://reviews.llvm.org/D107653
The patch removes mutable accessor methods for sections and segments.
As for now, const variants of them are not used because all callers have
mutable access to an instance of Object. On the other hand, they do not
actually modify the sets, so it looks better to keep only const ones.
Differential Revision: https://reviews.llvm.org/D107652
As detailed on https://pvs-studio.com/en/blog/posts/cpp/0771/ and raised on D62583, the SecNo++ increment is not guaranteed to occur before the second use of SecNo in the same addSection() call.
This patch pulls out the increment (just for clarity) and replaces the second use of SecNo with a constant zero value (we're using stable_sort so the value isn't critical).
Differential Revision: https://reviews.llvm.org/D107273
See [GRP_COMDAT group with STB_LOCAL signature](https://groups.google.com/g/generic-abi/c/2X6mR-s2zoc)
objcopy PR: https://sourceware.org/bugzilla/show_bug.cgi?id=27931
GRP_COMDAT deduplication is purely based on the signature symbol name in
ld.lld/GNU ld/gold. The local/global status is not part of the equation.
If the signature symbol is localized by --localize-hidden or
--keep-global-symbol, the intention is likely to make the group fully
localized. Drop GRP_COMDAT to suppress deduplication.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D106782
There is no need to differentiate whether `UseSegments` is true or
false. Unifying the cases makes the behavior closer to BinaryWriter.
This improves compatibility with objcopy because SHF_ALLOC sections not in
a PT_LOAD will not be skipped. Such cases are usually erroneous input, though.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D104186
Since this only comes up with inputs containing sections at least 4GB
large (I guess I could use a bzero section or something, so the input
file doesn't have to be 4GB, but even then the output file would have to
be 4GB, right?) I've skipped testing this. If there's a nice way to test
this without needing 4GB inputs or output files.
The subtlety here is demonstrated by this code:
struct t { operator uint64_t(); };
static_assert(std::is_same_v<int, decltype(std::declval<bool>() ? 0 : std::declval<t>())>);
static_assert(std::is_same_v<uint64_t, decltype(std::declval<bool>() ? 0 : std::declval<uint64_t>())>);
Because of this difference, the original source code was getting an int
type (truncating the actual size) and then extending it again, resulting
in bogus values (I haven't thought through this hard enough to explain
why the resulting value was 0xffff... - sign extension, possible UB, but
in any case it's the wrong answer - in this particular case I was
looking at that resulted in a size so large that we couldn't open a file
large enough to write to and ended up with a rather vague:
error: 'file_name.o': Invalid argument
IHexWriter was evaluating a section's physical address when deciding if
that section should be written to an output. This approach does not
account for a zero-sized section that has the same physical address as a
sized section. The behavior varies from GNU objcopy, and may result in a
HEX file that does not include all program sections.
The IHexWriter now excludes zero-sized sections when deciding what
should be written to the output. This affects the contents of the
writer's `Sections` collection; we will not try to insert multiple
sections that could have the same physical address. The behavior seems
consistent with GNU objcopy, which always excludes empty sections,
no matter the address.
The new test case evaluates the IHexWriter behavior when provided a
variety of empty sections that overlap or append a filled section. See
the input file's comments for more information. Given that test input,
and the change to the IHexWriter, GNU objcopy and llvm-objcopy produce
the same output.
Reviewed By: jhenderson, MaskRay, evgeny777
Differential Revision: https://reviews.llvm.org/D101332
During reviewing D102277 it was decided to remove lazy options processing
from llvm-objcopy CopyConfig structure. This patch transforms processing of ELF
lazy options into the in-place processing.
Differential Revision: https://reviews.llvm.org/D103260
This patch prepares llvm-objcopy to move its implementation
into a separate library. To make it possible it is necessary
to minimize internal dependencies.
Differential Revision: https://reviews.llvm.org/D99055
PR50160: we currently ignore non-PT_PHDR segments with no sections, not
accounting for its p_offset and p_filesz: this can cause an out-of-bounds write
in `writeSegmentData` if the p_offset+p_filesz is larger than the total file
size.
This can be fixed by setting p_offset=p_filesz=0. The logic nicely unifies with
the logic added in D90897.
Reviewed By: jhenderson, rupprecht
Differential Revision: https://reviews.llvm.org/D101560
This patch removes creation of the resulting file from the
executeObjcopyOnBinary() function. For the most use cases, the
executeObjcopyOnBinary receives output file as a parameter
- raw_ostream &Out. The splitting .dwo file is implemented differently:
file containg .dwo tables is created inside executeObjcopyOnBinary().
When objcopy functionality would be moved into separate library,
current implementation will become inconvenient. The goal of that
refactoring is to separate concerns: It might be convenient to
to do dwo tables splitting but to create resulting file differently.
Differential Revision: https://reviews.llvm.org/D98582
During D88827 it was requested to remove the local implementation
of Memory/File Buffers:
// TODO: refactor the buffer classes in LLVM to enable us to use them here
// directly.
This patch uses raw_ostream instead of Buffers. Generally, using streams
could allow us to reduce memory usages. No need to load all data into the
memory - the data could be streamed through a smaller buffer.
Thus, this patch uses raw_ostream as an interface for output data:
Error executeObjcopyOnBinary(CopyConfig &Config,
object::Binary &In,
raw_ostream &Out);
Note 1. This patch does not change the implementation of Writers
so that data would be directly stored into raw_ostream.
This is assumed to be done later.
Note 2. It would be better if Writers would be implemented in a such way
that data could be streamed without seeking/updating. If that would be
inconvenient then raw_ostream could be replaced with raw_pwrite_stream
to have a possibility to seek back and update file headers.
This is assumed to be done later if necessary.
Note 3. Current FileOutputBuffer allows using a memory-mapped file.
The raw_fd_ostream (which could be used if data should be stored in the file)
does not allow us to use a memory-mapped file. Memory map functionality
could be implemented for raw_fd_ostream:
It is possible to add resize() method into raw_ostream.
class raw_ostream {
void resize(uint64_t size);
}
That method, implemented for raw_fd_ostream, could create a memory-mapped file.
The streamed data would be written into that memory file then.
Thus we would be able to use memory-mapped files with raw_fd_ostream.
This is assumed to be done later if necessary.
Differential Revision: https://reviews.llvm.org/D91028
This diff introduces --keep-undefined in llvm-objcopy/llvm-strip for Mach-O
which makes the tools preserve undefined symbols.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D97040
The code was using the standard isalnum function which doesn't handle
values outside the non-ascii range. Switching to using llvm::isAlnum
instead ensures we don't provoke undefined behaviour, which can in some
cases result in crashes.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D97663
The check for whether an extended symbol index table was required
dropped the first SHN_LORESERVE sections from the sections array before
checking whether the remaining sections had symbols. Unfortunately, the
null section header is not present in this list, so the check was
skipping the first section that might be important. If that section
contained a symbol, and no subsequent ones did, the .symtab_shndx
section would not be emitted, leading to a corrupt object.
Also consolidate and expand test coverage in the area to cover this bug
and other aspects of the SYMTAB_SHNDX section.
Reviewed by: alexshap, MaskRay
Differential Revision: https://reviews.llvm.org/D97661
The few options are niche. They solved a problem which was traditionally solved
with more shell commands (`llvm-readelf -n` fetches the Build ID. Then
`ln` is used to hard link the file to a directory derived from the Build ID.)
Due to limitation, they are no longer used by Fuchsia and they don't appear to
be used elsewhere (checked with Google Search and Debian Code Search). So delete
them without a transition period.
Announcement: https://lists.llvm.org/pipermail/llvm-dev/2021-February/148446.html
Differential Revision: https://reviews.llvm.org/D96310
This is consistent with BFD objcopy.
Previously llvm objcopy would allocate space for SHT_NOBITS sections
often resulting in enormous binary files.
New test case (binary-paddr.test %t6).
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D95569
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)
Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).
This patch places such an empty segment according to its parent segment. This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D90897
This matches behavior GNU objcopy and can simplify clang-offload-bundler
(which currently works around the issue by invoking llvm-objcopy twice).
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D90438
Remove usages of special error reporting functions(error(),
reportError()). Errors are reported as Expected<>/Error returning
values. This part is for ELF subfolder of llvm-objcopy.
Testing: check-all.
Differential Revision: https://reviews.llvm.org/D87987
`ELFFile<ELFT>` has many methods that take pointers,
though they assume that arguments are never null and
hence could take references instead.
This patch performs such clean-up.
Differential revision: https://reviews.llvm.org/D87385
This patch resolves crash that occurs when user wanted to remove all
symbols and add a brand new one using:
```
llvm-objcopy -R .symtab --add-symbol foo=1234 in.o out.o
```
Before these changes the symbol table internally being null when adding
new symbols. For now we will regenerate symtab in this case.
This fixes: https://bugs.llvm.org/show_bug.cgi?id=43930
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D82935
SHT_GROUP sections contain a reference to a symbol indicating their
"signature" symbol. The symbol table containing this symbol is referred
to by the group section's sh_link field. If llvm-objcopy is instructed
to remove the symbol table, it will emit an error.
This fixes https://bugs.llvm.org/show_bug.cgi?id=46153.
Reviewed By: jhenderson, Higuoxing
Differential Revision: https://reviews.llvm.org/D82274
Reorder `DumpSection` under `handleArgs` in file `ELFObjcopy.cpp`.
`DumpSection` is placed before `replaceAndRemoveSections` and is
therefore now the first operation under `handleArgs`. Thus, it is now
performed before both `add` and `remove` section operations.
Change for the MachO format at D81123. Together fixes https://bugs.llvm.org/show_bug.cgi?id=44283.
Reviewed By: alexshap, jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D81097
When a group member is removed, the corresponding record in the
SHT_GROUP section has to be deleted.
This fixes PR46064.
Differential Revision: https://reviews.llvm.org/D80568
When a SHT_GROUP section is removed, but other sections of the group are
kept, the SHF_GROUP flag of these sections should be dropped, otherwise
the resulting ELF file will be malformed.
Differential Revision: https://reviews.llvm.org/D80511
cctools strip has the option "-T" which removes Swift symbols.
This diff implements this option in llvm-strip for MachO.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D80099
To avoid undefined behavior caught by -fsanitize=undefined on binary-paddr.test
void SectionWriter::visit(const Section &Sec) {
if (Sec.Type != SHT_NOBITS)
// Sec.Contents is empty while Sec.Offset may be out of bound
llvm::copy(Sec.Contents, Out.getBufferStart() + Sec.Offset);
}
After SHF_ALLOC sections are ordered by LMA:
* If initial sections are empty, GNU objcopy skips their contents while we
emit leading zeros. (binary-paddr.test %t4)
* If trailing sections are empty, GNU objcopy skips their contents while we
emit trailing zeros. (binary-paddr.test %t5)
This patch matches GNU objcopy's behavior. Linkers don't keep p_memsz
PT_LOAD segments. Such empty sections would not have a containing
PT_LOAD and `Section::ParentSegment` might be null if linkers fail to
optimize the file offsets (lld D79254).
In particular, without D79254, the arm Linux kernel's multi_v5_defconfig
depends on this behavior: in `vmlinux`, an empty .text_itcm is mapped at
a very high address (0xfffe0000) but the kernel does not expect
`objcopy -O binary` to create a very large `arch/arm/boot/Image`
(0xfffe0000-0xc0000000 ~= 1GiB). See https://bugs.llvm.org/show_bug.cgi?id=45632
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D79229
Don't error on Config.KeepFileSymbols for COFF and Mach-O.
Original description:
GNU objcopy removes STT_FILE symbols for strip-debug operations, and
keeps them for --discard-all operation. Match their behaviour for
llvm-objcopy.
Bug: https://github.com/android/ndk/issues/1212
Differential Revision: https://reviews.llvm.org/D76675
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.
Differential revision: https://reviews.llvm.org/D78016
Igor Kudrin