Commit Graph

164 Commits

Author SHA1 Message Date
George Rimar 99f73ebe5c [llvm-objcopy] - Reimplement strip-dwo-groups.test to stop using the precompiled object.
When llvm-copy removes .dwo sections the index of symbol table,
the indices of the symbols and the indices of the sections which go
after the removed ones changes. That affects on SHT_GROUP sections,
which needs to be updated.

Initially this test used a precompiled object, I rewrote it to use YAML
and improved a bit.

Differential revision: https://reviews.llvm.org/D65273

llvm-svn: 367202
2019-07-29 07:55:39 +00:00
Eugene Leviant c7e6d14c6c [llvm-objcopy] Allow strip symtab in executables and DSOs
Re-commit of the patch after addressing -Wl,--emit-relocs case.
Differential revision: https://reviews.llvm.org/D61672

llvm-svn: 366787
2019-07-23 08:03:30 +00:00
David Blaikie 9ec6f9e07e llvm-objcopy/test: add REQUIRES: shell for use of umask
(follow-up to r365162)

llvm-svn: 366755
2019-07-22 22:23:52 +00:00
Alex Brachet 553c29faa2 [test] [llvm-objcopy] Fix broken test case
Summary: The test case added in D62718 did not work unless the user was root because write bits were not set for the output file. This change uses only permissions with user write (0200) to ensure tests pass regardless of the users permissions.

Reviewers: jhenderson, rupprecht, MaskRay, espindola, alexshap

Reviewed By: MaskRay

Subscribers: emaste, arichardson, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64302

llvm-svn: 366527
2019-07-19 02:31:21 +00:00
Fangrui Song 6dc5962957 [llvm-objcopy] Don't change permissions of non-regular output files
There is currently an EPERM error when a regular user executes `llvm-objcopy a.o /dev/null`.
Worse, root can even change the mode bits of /dev/null.

Fix it by checking if the output file is special.

A new overload of llvm::sys::fs::setPermissions with FD as the parameter
is added. Users should provide `perm & ~umask` as the parameter if they
intend to respect umask.

The existing overload of llvm::sys::fs::setPermissions may be deleted if
we can find an implementation of fchmod() on Windows. fchmod() is
usually better than chmod() because it saves syscalls and can avoid race
condition.

Reviewed By: jakehehrlich, jhenderson

Differential Revision: https://reviews.llvm.org/D64236

llvm-svn: 365753
2019-07-11 10:17:59 +00:00
Jordan Rupprecht 77d3590a87 Revert [llvm-objcopy] Allow strip symtab from executables and DSOs
This reverts r365193 (git commit 194f16b354)

This patch doesn't work with binaries built w/ `--emit-relocs`, e.g.

```
$ echo 'int main() { return 0; }' | clang -Wl,--emit-relocs -x c - -o foo && llvm-objcopy --strip-unneeded foo
llvm-objcopy: error: 'foo': not stripping symbol '__gmon_start__' because it is named in a relocation
```

llvm-svn: 365712
2019-07-10 23:32:44 +00:00
Eugene Leviant 194f16b354 [llvm-objcopy] Allow strip symtab from executables and DSOs
Differential revision: https://reviews.llvm.org/D61672

llvm-svn: 365193
2019-07-05 12:10:44 +00:00
George Rimar d0921a4696 [Object/ELF.h] - Improve error reporting.
The errors coming from ELF.h are usually not very
useful because they are uninformative. This patch is a
first step to improve the situation.

I tested this patch with a run of check-llvm and found
that few messages are untested. In this patch, I did not
add more tests but marked all such cases with a "TODO" comment.

For all tested messages I extended the error text to
provide more details (see test cases changed).

Differential revision: https://reviews.llvm.org/D64014

llvm-svn: 365183
2019-07-05 11:28:49 +00:00
Fangrui Song 3c47d38f61 [llvm-objcopy][test] Fix respect-umask.test after D62718/r365162
llvm-svn: 365172
2019-07-05 05:10:28 +00:00
Alex Brachet de7da3f694 Fix patch not passing test cases
llvm-svn: 365170
2019-07-05 01:28:41 +00:00
Alex Brachet f0e5aa035e Temporarily stop failing test case
llvm-svn: 365168
2019-07-05 01:13:09 +00:00
Alex Brachet 7439a9a921 [llvm-objcopy] Change handling of output file permissions
Summary: Address bug [[ https://bugs.llvm.org/show_bug.cgi?id=42082 | 42082 ]] where files were always outputted with 0775 permissions. Now, the output file is given either 0666 or 0777 if the object is executable.

Reviewers: espindola, alexshap, rupprecht, jhenderson, jakehehrlich, MaskRay

Reviewed By: rupprecht, jhenderson, jakehehrlich, MaskRay

Subscribers: emaste, arichardson, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62718

llvm-svn: 365162
2019-07-04 22:45:27 +00:00
George Rimar fa1c7d9bdf [llvm-objcopy] - Get rid of dynrel.elf precompiled binary from inputs.
We do not have to spread using the precompiled binaries in the tests,
when we can use YAML. This patch removes the dynrel.elf binary and adds
a few comments to the test cases.

Differential revision: https://reviews.llvm.org/D63641

llvm-svn: 364052
2019-06-21 14:15:15 +00:00
Hubert Tong 1f6ddfb6a3 [NFC][llvm-objcopy] Fix overly restrictive od output check
The check against the output of `od` in the affected tests expect a
specific input offset format. They also expect a specific offset value,
not consistent with the EXAMPLE section for `od` in POSIX.1-2017
Chapter 4, while using the `-j` option. In particular, the example shows
that the input offset begins at 0 following the bytes skipped.

This patch adjusts the matching of the input offset to be more generic.
In order to avoid false matches, it restricts the number of bytes to be
formatted.

llvm-svn: 363829
2019-06-19 16:04:24 +00:00
Fangrui Song 677423997d [llvm-readobj] Allow --hex-dump/--string-dump to dump multiple sections
1) `-x foo` currently dumps one `foo`. This change makes it dump all `foo`.
2) `-x foo -x foo` currently dumps `foo` twice. This change makes it dump `foo` once.
   In addition, if foo has section index 9, `-x foo -x 9` dumps `foo` once.
3) Give a warning instead of an error if `foo` does not exist.

The new behaviors match GNU readelf.

Also, print a new line as a separator between two section dumps.
GNU readelf uses two lines, but one seems good enough.

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D63475

llvm-svn: 363683
2019-06-18 14:01:03 +00:00
Alex Brachet 7747700937 [llvm-strip] Error when using stdin twice
Summary: Implements bug [[ https://bugs.llvm.org/show_bug.cgi?id=42204 | 42204 ]]. llvm-strip now warns when the same input file is used more than once, and errors when stdin is used more than once.

Reviewers: jhenderson, rupprecht, espindola, alexshap

Reviewed By: jhenderson, rupprecht

Subscribers: emaste, arichardson, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63122

llvm-svn: 363638
2019-06-18 00:39:10 +00:00
Seiya Nuta 13de174b4c [llvm-objcopy] Add elf32-sparc and elf32-sparcel target
Summary:
The "sparc"/"sparcel" architectures appears in ArchMap (used by -B option) but not in OutputFormatMap (used by -I/-O option). Add their targets into OutputFormatMap for consistency.

Note that AFAIK there're no targets for 32-bit little-endian SPARC ("elf32-sparcel") in GNU binutils.

Reviewers: espindola, alexshap, rupprecht, jhenderson, compnerd, jakehehrlich

Reviewed By: jhenderson, compnerd, jakehehrlich

Subscribers: jyknight, emaste, arichardson, fedor.sergeev, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63238

llvm-svn: 363524
2019-06-17 02:03:45 +00:00
Alex Brachet 899a3072f0 [objcopy] Error when --preserve-dates is specified with standard streams
Summary: llvm-objcopy/strip now error when -p is specified when reading from stdin or writing to stdout

Reviewers: jhenderson, rupprecht, espindola, alexshap

Reviewed By: jhenderson, rupprecht

Subscribers: emaste, arichardson, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63090

llvm-svn: 363485
2019-06-15 05:32:23 +00:00
Eugene Leviant d46ebd207b [llvm-objcopy][IHEX] Improve test case formatting. NFC
Differential revision: https://reviews.llvm.org/D63258

llvm-svn: 363359
2019-06-14 08:09:10 +00:00
Alex Brachet d54d4f9905 [llvm-objcopy] Changed command line parsing errors
Summary: Tidied up errors during command line parsing to be more consistent with the rest of llvm-objcopy errors.

Reviewers: jhenderson, rupprecht, espindola, alexshap

Reviewed By: jhenderson, rupprecht

Subscribers: emaste, arichardson, MaskRay, llvm-commits, jakehehrlich

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62973

llvm-svn: 363350
2019-06-14 02:04:02 +00:00
Seiya Nuta b1027a480a [llvm-objcopy] Fix sparc target endianness
Summary: AFAIK, the "sparc" target is big endian and the target for 32-bit little-endian SPARC is denoted as "sparcel". This patch fixes the endianness of "sparc" target and adds "sparcel" target for 32-bit little-endian SPARC.

Reviewers: espindola, alexshap, rupprecht, jhenderson

Reviewed By: jhenderson

Subscribers: jyknight, emaste, arichardson, fedor.sergeev, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63251

llvm-svn: 363336
2019-06-13 23:24:12 +00:00
Eugene Leviant 86b7f865ac [llvm-objcopy] Implement IHEX reader
This is the final part of IHEX format support in llvm-objcopy
Differential revision: https://reviews.llvm.org/D62583

llvm-svn: 363243
2019-06-13 09:56:14 +00:00
Jordan Rupprecht f8f9d65f85 [llvm-objcopy] Fix SHT_GROUP ordering.
Summary:
When llvm-objcopy sorts sections during finalization, it only sorts based on the offset, which can cause the group section to come after the sections it contains. This causes link failures when using gold to link objects created by llvm-objcopy.

Fix this for now by copying GNU objcopy's behavior of placing SHT_GROUP sections first. In the future, we may want to remove this sorting entirely to more closely preserve the input file layout.

This fixes https://bugs.llvm.org/show_bug.cgi?id=42052.

Reviewers: jakehehrlich, jhenderson, MaskRay, espindola, alexshap

Reviewed By: MaskRay

Subscribers: phuongtrang148993, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62620

llvm-svn: 362973
2019-06-10 18:35:01 +00:00
Peter Collingbourne 8d58a98c59 llvm-objcopy: Implement --extract-partition and --extract-main-partition.
This implements the functionality described in
https://lld.llvm.org/Partitions.html. It works as follows:

- Reads the section headers using the ELF header at file offset 0;
- If extracting a loadable partition:
  - Finds the section containing the required partition ELF header by looking it up in the section table;
  - Reads the ELF and program headers from the section.
- If extracting the main partition:
  - Reads the ELF and program headers from file offset 0.
- Filters the section table according to which sections are in the program headers that it read:
  - If ParentSegment != nullptr or section is not SHF_ALLOC, then it goes in.
  - Sections containing partition ELF headers or program headers are excluded as there are no headers for these in ordinary ELF files.

Differential Revision: https://reviews.llvm.org/D62364

llvm-svn: 362818
2019-06-07 17:57:48 +00:00
George Rimar 33044a7ae2 [llvm-objcopy] - Emit error and don't crash if program header reaches past end of file.
This is https://bugs.llvm.org/show_bug.cgi?id=42122.

If an object file has a size less than program header's file [offset + size]
(i.e. if we have overflow), llvm-objcopy crashes instead of reporting a
error.

The patch fixes this issue.

Differential revision: https://reviews.llvm.org/D62898

llvm-svn: 362778
2019-06-07 08:34:18 +00:00
Alex Brachet 375d5fb9ca [test][llvm-objcopy] Test llvm-objcopy with standard streams
Differential Revision: https://reviews.llvm.org/D62817

llvm-svn: 362556
2019-06-04 22:17:27 +00:00
Owen Reynolds 5d5078e341 [llvm-ar] Reapply Fix relative thin archive path handling
Includes a fix for an introduced build failure due to a post c++11 use of std::mismatch. 

This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.

Differential Revision: https://reviews.llvm.org/D59491

llvm-svn: 362484
2019-06-04 10:13:03 +00:00
Dmitri Gribenko 857de979a7 Revert "[llvm-ar] Fix relative thin archive path handling"
This reverts commit r362407.  It broke compilation of
llvm/lib/Object/ArchiveWriter.cpp:

error: type 'llvm::sys::path::const_iterator' does not provide a call
operator

llvm-svn: 362413
2019-06-03 16:21:37 +00:00
Owen Reynolds fade9cbed7 [llvm-ar] Fix relative thin archive path handling
This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.

Differential Revision: https://reviews.llvm.org/D59491

llvm-svn: 362407
2019-06-03 15:26:07 +00:00
Eugene Leviant fa147c97d6 [llvm-objcopy] Remove %p format specifiers
On 32-bit machines %p expects 32 bit values, however
addresses in llvm-objcopy are always 64 bits.

llvm-svn: 362074
2019-05-30 09:09:01 +00:00
Eugene Leviant c98b288b03 Yet another attempt to fix buildbot after r361949
Looks like %p format specifier of createStringError behaves
differently on different platforms

llvm-svn: 361993
2019-05-29 17:14:48 +00:00
Eugene Leviant a6fb183c98 [llvm-objcopy] Implement IHEX writer
Differential revision: https://reviews.llvm.org/D60270

llvm-svn: 361949
2019-05-29 11:37:16 +00:00
George Rimar c1cc8d0eca [llvm-objcopy] - Strip undefined symbols if they are no longer referenced following --only-section
This is https://bugs.llvm.org/show_bug.cgi?id=40004.

In this patch I teach llvm-objcopy to remove undefined symbols if
them are not used anymore after applying -j/--only-section option.

Differential revision: https://reviews.llvm.org/D62317

llvm-svn: 361642
2019-05-24 15:04:50 +00:00
Seiya Nuta ada9d2d884 [llvm-objcopy] Add file names to error messages
Summary:
This patch adds the file names to llvm-objcopy error messages. It makes easy to identify which file causes an error.

Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=41798

Reviewers: espindola, alexshap, rupprecht, jhenderson, jakehehrlich

Reviewed By: rupprecht, jhenderson, jakehehrlich

Subscribers: emaste, arichardson, jakehehrlich, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61993

llvm-svn: 361450
2019-05-23 00:42:46 +00:00
James Henderson 5316a0d200 [llvm-objcopy] Tidy up error messages
This patch brings various error messages into line with each other, by
removing trailing full stops, and making the first letter lower-case.
This addresses https://bugs.llvm.org/show_bug.cgi?id=40859.

Reviewed by: jhenderson, rupprecht, jakehehrlich

Differential Revision: https://reviews.llvm.org/D62072

Patch by Alex Brachet

llvm-svn: 361384
2019-05-22 13:23:26 +00:00
Seiya Nuta 3fcf1f5d29 [NFC] Remove a blank line (test commit)
llvm-svn: 361349
2019-05-22 04:48:26 +00:00
Eugene Leviant ec767b0b4a [llvm-objcopy] Strip file symbols with --strip-unneeded
Differential revision: https://reviews.llvm.org/D61641

llvm-svn: 361231
2019-05-21 09:09:33 +00:00
James Henderson 5772e02bd0 [llvm-objcopy] Add additional testing for various cases
This patch adds a number of tests to test various cases not covered by
existing tests. All of them work correctly, with no need to change
llvm-objcopy itself, although some do indicate possible areas for
improvement.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D61727

llvm-svn: 360422
2019-05-10 12:58:52 +00:00
James Henderson 9ca48f26a3 [NFC]Fix British English -> American English issues
llvm-svn: 360252
2019-05-08 13:30:48 +00:00
James Henderson 5a2b5ca7d2 [llvm-objcopy] Improve error message for unrecognised archive member
Prior to this patch, llvm-objcopy's error messages for archives with
unsupported members only mentioned the archive name, not the member
name, making them unhelpful. This change improves it by approximately
following GNU objcopy's error message syntax of
"<archive name>(<member name>): <problem>".

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D61674

llvm-svn: 360251
2019-05-08 13:28:58 +00:00
James Henderson fa11fb33ad [llvm-objcopy] Add --prefix-alloc-sections
This patch adds support for --prefix-alloc-sections, which adds a prefix
to every allocated section names.

It adds a prefix after renaming section names by --rename-section as GNU
objcopy does.

Fixes PR41266: https://bugs.llvm.org/show_bug.cgi?id=41266

Differential Revision: https://reviews.llvm.org/D60042

Patch by Seiya Nuta.

llvm-svn: 360233
2019-05-08 09:49:35 +00:00
Sid Manning 5ad18a7d59 Let --discard-all imply --strip-debug.
This will match gnu strip's behavior.

Differential Revision: https://reviews.llvm.org/D61092

llvm-svn: 359887
2019-05-03 14:14:01 +00:00
James Henderson e4a89a1bee [llvm-strip]Add --no-strip-all to disable --strip-all behaviour (including default stripping)
If certain switches are not specified, llvm-strip behaves as if
--strip-all were specified. This means that for testing, when we don't
want the stripping behaviour, we have to specify one of these switches,
which can be confusing. This change adds --no-strip-all to allow an
alternative way of suppressing the default stripping, in a less
confusing manner.

Reviewed by: jakehehrlich, MaskRay

Differential Revision: https://reviews.llvm.org/D61377

llvm-svn: 359781
2019-05-02 11:53:02 +00:00
George Rimar f5345a3f4c [yaml2obj] - Report when unknown section is referenced from program header declaration block.
Previously we did not report this.
Also this removes multiple lookups in the map
what cleanups the code.

Differential revision: https://reviews.llvm.org/D61322

llvm-svn: 359663
2019-05-01 09:45:55 +00:00
Fangrui Song 6afcdcf9ab [llvm-readobj] Change -t to --symbols in tests. NFC
-t is --symbols in llvm-readobj but --section-details (unimplemented) in readelf.
The confusing option should not be used since we aim for improving
compatibility.

Keep just one llvm-readobj -t use case in test/tools/llvm-readobj/symbols.test

llvm-svn: 359661
2019-05-01 09:28:24 +00:00
Fangrui Song 97c17e83f8 [llvm-readobj] llvm-readobj --elf-output-style=GNU => llvm-readelf. NFC
The latter is much more common.

A dedicated --elf-output-style=GNU test demonstrating it is the same as
llvm-readelf is sufficient.

llvm-svn: 359652
2019-05-01 05:55:22 +00:00
Fangrui Song e29e30b139 [llvm-readobj] Change -long-option to --long-option in tests. NFC
We use both -long-option and --long-option in tests. Switch to --long-option for consistency.

In the "llvm-readelf" mode, -long-option is discouraged as it conflicts with grouped short options and it is not accepted by GNU readelf.

While updating the tests, change llvm-readobj -s to llvm-readobj -S to reduce confusion ("s" is --section-headers in llvm-readobj but --symbols in llvm-readelf).

llvm-svn: 359649
2019-05-01 05:27:20 +00:00
Fangrui Song aa1f2c50a8 [llvm-objcopy] Simplify SHT_NOBITS -> SHT_PROGBITS promotion
GNU objcopy uses bfd_elf_get_default_section_type to decide the candidate section type,
which roughly translates to our [a] (I assume SEC_COMMON implies SHF_ALLOC):

  (!(Sec.Flags & ELF::SHF_ALLOC) || Flags & (SectionFlag::SecContents | SectionFlag::SecLoad)))

Then, it updates the section type in bfd/elf.c:elf_fake_sections if:

  if (this_hdr->sh_type == SHT_NULL)
    this_hdr->sh_type = sh_type; // common case
  else if (this_hdr->sh_type == SHT_NOBITS
           && sh_type == SHT_PROGBITS
           && (asect->flags & SEC_ALLOC) != 0)  // uncommon case
    ...
    this_hdr->sh_type = sh_type;

If the following condition is met the uncommon branch is executed:

  if (elf_section_type (osec) == SHT_NULL
      && (osec->flags == isec->flags
	  || (final_link
	      && ((osec->flags ^ isec->flags)
		  & ~(SEC_LINK_ONCE | SEC_LINK_DUPLICATES | SEC_RELOC)) == 0)))

I suggest we just ignore this clause and follow the common case
behavior, which is done in this patch. Rationales to do so:

If --set-section-flags is a no-op (osec->flags == isec->flags)
(corresponds to the "readonly" test in set-section-flags.test), GNU
objcopy will require (Sec.Flags & ELF::SHF_ALLOC). [a] is essentially:

  Flags & (SectionFlag::SecContents | SectionFlag::SecLoad)

This special case is not really useful. Non-SHF_ALLOC SHT_NOBITS
sections do not make much sense and it doesn't matter if they are
SHT_NOBITS or SHT_PROGBITS.

For all other RUN lines in set-section-flags.test, the new behavior
matches GNU objcopy, i.e. this patch improves compatibility.

Differential Revision: https://reviews.llvm.org/D60189

llvm-svn: 359639
2019-05-01 00:39:31 +00:00
Jordan Rupprecht 96bbb1dc2b [llvm-objcopy] Add RISC-V support for -B/-O
Reviewers: jorgbrown, espindola, alexshap, jhenderson

Subscribers: emaste, arichardson, fedor.sergeev, jakehehrlich, kito-cheng, shiva0217, MaskRay, rogfer01, rkruppe, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61272

llvm-svn: 359568
2019-04-30 15:21:36 +00:00
George Rimar 67f590e286 [llvm-objcopy] - Check dynamic relocation sections for broken references.
This is a fix for https://bugs.llvm.org/show_bug.cgi?id=41371.

Currently, it is possible to break the sh_link field of the dynamic relocation section
by removing the section it refers to. The patch fixes an issue and adds 2 test cases.

One of them shows that it does not seem possible to break the sh_info field.
I added an assert to verify this.

Differential revision: https://reviews.llvm.org/D60825

llvm-svn: 359552
2019-04-30 11:02:09 +00:00
Alexandre Ganea 8245140d3f Fix llvm-objcopy/ELF/preserve-segment-contents test on UTF-8 locale
Differential Revision: https://reviews.llvm.org/D61137

llvm-svn: 359302
2019-04-26 13:09:26 +00:00
Fangrui Song 2db79e9d2c [llvm-objcopy] Accept --long-option but not -long-option
Summary:

llvm-{objcopy,strip} (and many other LLVM binary utilities) accept
cl::opt style -long-option as well as many short options (e.g. -p -S
-x). People who use them as replacement of GNU binutils often use the
grouped option syntax (POSIX Utility Conventions), e.g. -Sx => -S -x,
-Wd => -W -d, -sj.text => -s -j.text

There is ambiguity if a long option starts with the character used by a
short option. Drop the support for -long-option to resolve the ambiguity.

This divergence from other utilities is accepted (other utilities
continue supporting -long-option).
https://lists.llvm.org/pipermail/llvm-dev/2019-April/131786.html

Reviewers: alexshap, jakehehrlich, jhenderson, rupprecht, espindola

Reviewed By: jakehehrlich, jhenderson, rupprecht

Subscribers: grimar, emaste, arichardson, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60439

llvm-svn: 359265
2019-04-26 02:10:10 +00:00
Jordan Rupprecht 2b32902a88 [llvm-objcopy] Add -B mips
llvm-svn: 358667
2019-04-18 14:22:37 +00:00
James Henderson 66a9d0f8c6 [llvm-objcopy][llvm-strip] Add switch to allow removing referenced sections
llvm-objcopy currently emits an error if a section to be removed is
referenced by another section. This is a reasonable thing to do, but is
different to GNU objcopy. We should allow users who know what they are
doing to have a way to produce the invalid ELF. This change adds a new
switch --allow-broken-links to both llvm-strip and llvm-objcopy to do
precisely that. The corresponding sh_link field is then set to 0 instead
of an error being emitted.

I cannot use llvm-readelf/readobj to test the link fields because they
emit an error if any sections, like the .dynsym, cannot be properly
loaded.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D60324

llvm-svn: 358649
2019-04-18 09:13:30 +00:00
Jordan Rupprecht b0b65cae59 [llvm-objcopy] Support full list of bfd targets that lld uses.
Summary:
This change takes the full list of bfd targets that lld supports (see `ScriptParser.cpp`), including generic handling for `*-freebsd` targets (which uses the same settings but with a FreeBSD OSABI). In particular this adds mips support for `--output-target` (but not yet via `--binary-architecture`).

lld and llvm-objcopy use their own different custom data structures, so I'd prefer to check this in as-is (add support directly in llvm-objcopy, including all the test coverage) and do a separate NFC patch(s) that consolidate the two by putting this mapping into libobject.

See [[ https://bugs.llvm.org/show_bug.cgi?id=41462 | PR41462 ]].

Reviewers: jhenderson, jakehehrlich, espindola, alexshap, arichardson

Reviewed By: arichardson

Subscribers: fedor.sergeev, emaste, sdardis, krytarowski, atanasyan, llvm-commits, MaskRay, arichardson

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60773

llvm-svn: 358562
2019-04-17 07:42:31 +00:00
Eugene Leviant 88089fed9c [llvm-objcopy] Fill .symtab_shndx section correctly
Differential revision: https://reviews.llvm.org/D60555

llvm-svn: 358278
2019-04-12 11:59:30 +00:00
Don Hinton 98e3954fe9 [llvm-objcopy] [llvm-symbolizer] Fix failing tests
Summary: Fix failing tests that matched substrings in path.

Reviewers: evgeny777, mattd, espindola, alexshap, rupprecht, jhenderson

Reviewed By: jhenderson

Subscribers: Bulletmagnet, emaste, arichardson, jakehehrlich, MaskRay, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60170

llvm-svn: 357709
2019-04-04 17:35:41 +00:00
George Rimar 6da44ad75d [yaml2obj][obj2yaml] - Change how symbol's binding is descibed when parsing/dumping.
Currently, YAML has the following syntax for describing the symbols:

Symbols:
  Local:
    LocalSymbol1:
    ...
    LocalSymbol2:
    ...
  ...
  Global:
    GlobalSymbol1:
  ...
  Weak:
  ...
  GNUUnique:

I.e. symbols are grouped by their bindings. That is not very convenient,
because:

It does not allow to set a custom binding, what can be useful for producing
broken/special outputs for test cases. Adding a new binding would require to
change a syntax (what we observed when added GNUUnique recently).

It does not allow to change the order of the symbols in .symtab/.dynsym,
i.e. currently all Local symbols are placed first, then Global, Weak and GNUUnique
are following, but we are not able to change the order.

It is not consistent. Binding is just one of the properties of the symbol,
we do not group them by other properties.

It makes the code more complex that it can be. This patch shows it can be simplified
with the change performed.

The patch changes the syntax to just:

Symbols:
  Symbol1:
  ...
  Symbol2:
  ...
...

With that, we are able to work with the binding field just like with any other symbol property.

Differential revision: https://reviews.llvm.org/D60122

llvm-svn: 357595
2019-04-03 14:53:42 +00:00
James Henderson f5b181e16d [NFC] Address missed review comment for test
Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D60200

llvm-svn: 357594
2019-04-03 14:50:50 +00:00
James Henderson d931cf3e46 [llvm-objcopy] Make section rename/set flags case-insensitive
This fixes https://bugs.llvm.org/show_bug.cgi?id=41305. GNU objcopy
--set-section-flags/--rename-section flags are case-insensitive, so this
patch updates llvm-objcopy to match.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D60200

llvm-svn: 357590
2019-04-03 14:40:27 +00:00
Jordan Rupprecht 017deaf1ae [llvm-objcopy] Change SHT_NOBITS to SHT_PROBITS for some --set-section-flags
Summary:
Some flags accepted by --set-section-flags and --rename-section can change a SHT_NOBITS section to a SHT_PROGBITS section. Note that none of them can change a SHT_PROGBITS to SHT_NOBITS.

The full list (found via experimentation of individually setting each flag) that does this is: contents, load, noload, code, data, rom, and debug.

This was found by testing llvm-objcopy with the gnu binutils test suite, specifically this test case: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/testsuite/binutils-all/copy-1.d;h=f2b0d9e90df738c2891b4d5c7b62f62894b556ca;hb=HEAD

Reviewers: jhenderson, grimar, jakehehrlich, alexshap, espindola

Reviewed By: jhenderson

Subscribers: emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59958

llvm-svn: 357492
2019-04-02 16:49:56 +00:00
James Henderson 38cb238f75 [llvm-objcopy]Allow llvm-objcopy to be used on an ELF file with no section headers
This patch fixes https://bugs.llvm.org/show_bug.cgi?id=41293 and
https://bugs.llvm.org/show_bug.cgi?id=41045. llvm-objcopy assumed that
it could always read a section header string table. This isn't the case
when the sections were previously all stripped, and the e_shstrndx field
was set to 0. This patch fixes this. It also fixes a double space in an
error message relating to this issue, and prevents llvm-objcopy from
adding extra space for non-existent section headers, meaning that
--strip-sections on the output of a previous --strip-sections run
produces identical output, simplifying the test.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D59989

llvm-svn: 357475
2019-04-02 14:11:13 +00:00
Yi Kong f2baddb0fc [llvm-objcopy] Add --keep-symbols option
Differential Revision: https://reviews.llvm.org/D60054

llvm-svn: 357418
2019-04-01 18:12:43 +00:00
George Rimar 279898b315 [llvm-objcopy] - Strip sections before symbols.
This is a fix for https://bugs.llvm.org/show_bug.cgi?id=40007.

Idea is to swap the order of stripping. So that we strip sections before
symbols what allows us to strip relocation sections without emitting
the error about relative symbols.

Differential revision: https://reviews.llvm.org/D59763

llvm-svn: 357017
2019-03-26 18:42:15 +00:00
James Henderson 1f44814952 [llvm-objcopy]Preserve data in segments not covered by sections
llvm-objcopy previously knew nothing about data in segments that wasn't
covered by section headers, meaning that it wrote zeroes instead of what
was there. As it is possible for this data to be useful to the loader,
this patch causes llvm-objcopy to start preserving this data. Data in
sections that are explicitly removed continues to be written as zeroes.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41005.

Reviewed by: jakehehrlich, rupprecht

Differential Revision: https://reviews.llvm.org/D59483

llvm-svn: 356919
2019-03-25 16:36:26 +00:00
George Rimar 272571718c Recommit r356738 "[llvm-objcopy] - Implement replaceSectionReferences for GroupSection class."
Fix: r356853 + set AddressAlign to 4 in 
Inputs/compress-debug-sections.yaml for the new group section introduced.

Original commit message:

Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.

The patch does that.

Differential revision: https://reviews.llvm.org/D59638

llvm-svn: 356856
2019-03-24 14:41:45 +00:00
George Rimar 0a5d4b8472 [llvm-objcopy] - Report SHT_GROUP sections with invalid alignment.
This patch fixes the reason of ubsan failure (UB detected) 
happened after landing the D59638 (I had to revert it).
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/11760/steps/check-llvm%20ubsan/logs/stdio)

Problem is the following. Our implementation of GroupSection assumes that
its address is 4 bytes aligned when writes it:

template <class ELFT>
void ELFSectionWriter<ELFT>::visit(const GroupSection &Sec) {
  ELF::Elf32_Word *Buf =
      reinterpret_cast<ELF::Elf32_Word *>(Out.getBufferStart() + Sec.Offset);
...

But the test case for D59638 did not set AddressAlign in YAML. So address was
not 4 bytes aligned since Sec.Offset was odd. That triggered the issue.

This patch teaches llvm-objcopy to report an error for such sections (which should
not met in reality), what is better than having UB.

Differential revision: https://reviews.llvm.org/D59695

llvm-svn: 356853
2019-03-24 13:31:08 +00:00
James Henderson c069d9fd36 [llvm-objcopy]Add coverage for --split-dwo and --output-format
Also fix up a couple of minor issues in the test being updated, where
FileCheck could match on incorrect output and fix the test case order to
match the struct order.

Reviewed by: grimar

Differential Revision: https://reviews.llvm.org/D59691

llvm-svn: 356746
2019-03-22 12:45:27 +00:00
George Rimar d822018dbe Revert r356738 "[llvm-objcopy] - Implement replaceSectionReferences for GroupSection class."
Seems this broke ubsan bot:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/11760

llvm-svn: 356745
2019-03-22 12:14:04 +00:00
George Rimar 1ed6a745db [llvm-objcopy] - Fix a st_name of the first symbol table entry.
Spec says about the first symbol table entry that index 0 both designates the first entry in the table
and serves as the undefined symbol index. It should have zero value.
Hence the first symbol table entry has no name. And so has to have a st_name == 0.
(http://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html)

Currently, we do not emit zero value for the first symbol table entry.
That happens because we add empty strings to the string builder, which
for each such case adds a zero byte:
(https://github.com/llvm-mirror/llvm/blob/master/lib/MC/StringTableBuilder.cpp#L185)
After the string optimization performed it might return non zero indexes for the
empty string requested.

The patch fixes this issue for the case above and other sections with no names.

Differential revision: https://reviews.llvm.org/D59496

llvm-svn: 356739
2019-03-22 10:28:56 +00:00
George Rimar 73e1c4a030 [llvm-objcopy] - Implement replaceSectionReferences for GroupSection class.
Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.

The patch does that.

Differential revision: https://reviews.llvm.org/D59638

llvm-svn: 356738
2019-03-22 10:24:37 +00:00
James Henderson c040d5de25 [llvm-objcopy]Add support for *-freebsd output formats
GNU objcopy can support output formats like elf32-i386-freebsd and
elf64-x86-64-freebsd. The only difference from their regular non-freebsd
counterparts that I have observed is that the freebsd versions set the
OS/ABI field to ELFOSABI_FREEBSD. This patch sets the OS/ABI field
according based on the format whenever --output-format is specified.

Reviewed by: rupprecht, grimar

Differential Revision: https://reviews.llvm.org/D59645

llvm-svn: 356737
2019-03-22 10:21:09 +00:00
George Rimar 0373bedb41 [llvm-objcopy] - Use replaceSectionReferences to update the sections for symbols in symbol table.
If the compression was used and we had a symbol not involved in relocation,
we never updated its section and it was silently removed from the output.

Differential revision: https://reviews.llvm.org/D59542

llvm-svn: 356554
2019-03-20 13:57:47 +00:00
George Rimar faf308b11a [llvm-objcopy] - Calculate the string table section sizes correctly.
This fixes the https://bugs.llvm.org/show_bug.cgi?id=40980.

Previously if string optimization occurred as a result of
StringTableBuilder's finalize() method, the size wasn't updated.

This hopefully also makes the interaction between sections during finalization
processes a bit more clear.

Differential revision: https://reviews.llvm.org/D59488

llvm-svn: 356371
2019-03-18 14:27:41 +00:00
Jordan Rupprecht 12ed01dcf9 [llvm-strip] Hook up (unimplemented) --only-keep-debug
For ELF, we accept but ignore --only-keep-debug. Do the same for llvm-strip.

COFF does implement this, so update the test that it is supported.

llvm-svn: 356207
2019-03-14 21:51:42 +00:00
James Henderson b5de5e25de [llvm-objcopy]Don't implicitly strip sections in segments
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.

Reviewed by: grimar, rupprecht, jakehehrlich

Differential Revision: https://reviews.llvm.org/D59293

This is a reland of r356129, attempting to fix greendragon failures
due to a suspected compatibility issue with od on the greendragon bots
versus other versions.

llvm-svn: 356136
2019-03-14 11:47:41 +00:00
James Henderson e81f5f91b4 Revert r356129 due to greendragon bot failures
llvm-svn: 356133
2019-03-14 11:23:04 +00:00
James Henderson c03a95d465 [llvm-objcopy]Don't implicitly strip sections in segments
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.

This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.

Reviewed by: grimar, rupprecht, jakehehrlich

Differential Revision: https://reviews.llvm.org/D59293

llvm-svn: 356129
2019-03-14 10:20:27 +00:00
Jordan Rupprecht 42bc1e241c [llvm-objcopy] Cleanup errors from CopyConfig and remove llvm-objcopy.h dependency
error() was previously cleaned up from CopyConfig, but new uses were introduced.

This also tweaks the error message for --add-symbol to report all invalid flags.

llvm-svn: 356105
2019-03-13 22:26:01 +00:00
George Rimar d8a5c6cf19 [llvm-objcopy] - Fix --compress-debug-sections when there are relocations.
When --compress-debug-sections is given,
llvm-objcopy removes the uncompressed sections and adds compressed to the section list.
This makes all the pointers to old sections to be outdated.

Currently, code already has logic for replacing the target sections of the relocation
sections. But we also have to update the relocations by themselves.

This fixes https://bugs.llvm.org/show_bug.cgi?id=40885.

Differential revision: https://reviews.llvm.org/D58960

llvm-svn: 355821
2019-03-11 11:01:24 +00:00
George Rimar ade3c70537 [llvm-objcopy] - Simplify `isCompressable` and fix the issue relative.
When --compress-debug-sections is given, llvm-objcopy do not compress
sections that have "ZLIB" header in data. Normally this signature is used
in zlib-gnu compression format. But if zlib-gnu used then the name of the compressed
section should start from .z* (e.g .zdebug_info). If it does not, then it is not
a zlib-gnu format and section should be treated as a normal uncompressed section.

Differential revision: https://reviews.llvm.org/D58908

llvm-svn: 355399
2019-03-05 13:07:43 +00:00
George Rimar bf447a5bb5 [llvm-objdump] - Improve the error message for "removing a section that is used by relocation" case.
This refines/improves the error message introduced in D58625

Differential revision: https://reviews.llvm.org/D58709

llvm-svn: 355074
2019-02-28 08:21:50 +00:00
George Rimar 79fb858053 [llvm-objcopy] - Check for invalidated relocations when removing a section.
This is https://bugs.llvm.org/show_bug.cgi?id=40818

Removing a section that is used by relocation is an error
we did not report. The patch fixes that.

Differential revision: https://reviews.llvm.org/D58625

llvm-svn: 354962
2019-02-27 11:18:27 +00:00
Eugene Leviant 53350d0411 [llvm-objcopy] Add --set-start, --change-start and --adjust-start
Differential revision: https://reviews.llvm.org/D58173

llvm-svn: 354854
2019-02-26 09:24:22 +00:00
Eugene Leviant 51c1f640aa [llvm-objcopy] Add --add-symbol
Differential revision: https://reviews.llvm.org/D58234

llvm-svn: 354787
2019-02-25 14:12:41 +00:00
Jordan Rupprecht ad29d29130 [llvm-objcopy][NFC] More error cleanup
Summary:
This removes calls to `error()`/`reportError()` in the main driver (llvm-objcopy.cpp) as well as the associated argv-parsing (CopyConfig.cpp). `logAllUnhandledErrors()` is now the main way to print errors.

There are still a few uses from within the per-arch drivers, so we can't delete them yet... but almost!

Reviewers: jhenderson, alexshap, espindola

Reviewed By: jhenderson

Subscribers: emaste, arichardson, jakehehrlich, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58316

llvm-svn: 354600
2019-02-21 17:05:19 +00:00
Eugene Leviant 2db1062906 [llvm-objcopy] Add --strip-unneeded-symbol(s)
Differential revision: https://reviews.llvm.org/D58027

llvm-svn: 353919
2019-02-13 07:34:54 +00:00
Eugene Leviant e08fe35d79 [llvm-objcopy] Add few file processing directives
Differential revision: https://reviews.llvm.org/D57877

llvm-svn: 353521
2019-02-08 14:37:54 +00:00
Eugene Leviant fc6d29dff9 Attempt to fix build bot after r353509
llvm-svn: 353510
2019-02-08 10:51:08 +00:00
Eugene Leviant 340cb87e83 [llvm-objcopy] Add --redefine-syms
Differential revision: https://reviews.llvm.org/D57738

llvm-svn: 353509
2019-02-08 10:33:16 +00:00
Eugene Leviant ef6eba2401 Attempt to fix buildbot after r353289
llvm-svn: 353294
2019-02-06 13:45:22 +00:00
Eugene Leviant f324f6dcfb [llvm-objcopy] Allow regular expressions in name comparison
Differential revision: https://reviews.llvm.org/D57517

llvm-svn: 353289
2019-02-06 11:00:07 +00:00
Eugene Leviant 05a3f9978e [llvm-strip] Implement --keep-file-symbols
Differential revision: https://reviews.llvm.org/D57582

llvm-svn: 352878
2019-02-01 15:25:15 +00:00
Jordan Rupprecht bd7735f797 [llvm-objcopy] Skip --localize-symbol for undefined symbols
Summary:
Include the symbol being defined in the list of requirements for using --localize-symbol.

This is used, for example, when someone is depending on two different projects that have the same (or close enough) method defined in each library, and using "-L sym" for a conflicting symbol in one of the libraries so that the definition from the other one is used. However, the library may have internal references to the symbol, which cause program crashes when those are used, i.e.:

```
$ cat foo.c
int foo() { return 5; }
$ cat bar.c
int foo();
int bar() { return 2 * foo(); }
$ cat foo2.c
int foo() { /* Safer implementation */ return 42; }
$ cat main.c
int bar();
int main() {
  __builtin_printf("bar = %d\n", bar());
  return 0;
}
$ ar rcs libfoo.a foo.o bar.o
$ ar rcs libfoo2.a foo2.o
# Picks the wrong foo() impl
$ clang main.o -lfoo -lfoo2 -L. -o main
# Picks the right foo() impl
$ objcopy -L foo libfoo.a && clang main.o -lfoo -lfoo2 -L. -o main
# Links somehow, but crashes at runtime
$ llvm-objcopy -L foo libfoo.a && clang main.o -lfoo -lfoo2 -L. -o main
```

Reviewers: jhenderson, alexshap, jakehehrlich, espindola

Subscribers: emaste, arichardson

Differential Revision: https://reviews.llvm.org/D57417

llvm-svn: 352767
2019-01-31 16:45:16 +00:00
Eugene Leviant 2267c58aea [llvm-strip] Add --strip-symbol
Differential revision: https://reviews.llvm.org/D57440

llvm-svn: 352746
2019-01-31 12:16:20 +00:00
Jordan Rupprecht d0f7bcfbe1 [llvm-objcopy] Support -X|--discard-locals.
Summary:
This adds support for the --discard-locals flag, which acts similarly to --discard-all, except it only applies to compiler-generated symbols (i.e. symbols starting with `.L` in ELF).

I am not sure about COFF local symbols: those appear to also use `.L` in most cases, but also use just `L` in other cases, so for now I am just leaving it unimplemented there.

Fixes PR36160

Reviewers: jhenderson, alexshap, jakehehrlich, mstorsjo, espindola

Reviewed By: jhenderson

Subscribers: llvm-commits, emaste, arichardson

Differential Revision: https://reviews.llvm.org/D57248

llvm-svn: 352626
2019-01-30 14:58:13 +00:00
Jordan Rupprecht 307deab40a [llvm-objcopy][NFC] More error propagation
Summary: Do some more error cleanup, removing some dependencies from llvm-objcopy's error/reportError in [ELF/COFF]Objcopy methods.

Reviewers: jhenderson, alexshap, jakehehrlich, mstorsjo, espindola

Subscribers: emaste, arichardson

Differential Revision: https://reviews.llvm.org/D57423

llvm-svn: 352625
2019-01-30 14:36:53 +00:00
Jordan Rupprecht c892741e74 [llvm-objcopy] Implement --set-section-flags.
Summary:
--set-section-flags is used to change the section flags (e.g. SHF_ALLOC) for given sections. The flags allowed are the same from the existing --rename-section=.old=.new[,flags] feature.

Additionally, make sure that --set-section-flag cannot be used with --rename-section (either the source or destination), since --rename-section accepts flags. This avoids ambiguity for something like "--rename-section=.foo=.bar,alloc --set-section-flag=.bar,code".

Reviewers: jhenderson, jakehehrlich, alexshap, espindola

Reviewed By: jhenderson, jakehehrlich

Subscribers: llvm-commits, emaste, arichardson

Differential Revision: https://reviews.llvm.org/D57198

llvm-svn: 352505
2019-01-29 15:05:38 +00:00
Jordan Rupprecht b2702d6a45 [llvm-objcopy] Fix crash when writing empty binary output
Summary: When using llvm-objcopy -O binary and the resulting file will be empty (e.g. removing the only section that would be written, or using --only-keep with a section that doesn't exist/isn't SHF_ALLOC), we crash because FileOutputBuffer expects Size > 0. Add a regression test, and change Buffer to open/truncate the output file in this case.

Reviewers: alexshap, jhenderson, jakehehrlich, espindola

Reviewed By: alexshap, jhenderson

Subscribers: jfb, llvm-commits, emaste, arichardson

Differential Revision: https://reviews.llvm.org/D56806

llvm-svn: 352371
2019-01-28 15:02:40 +00:00
Douglas Yung 914e838e63 [llvm-objcopy] Add support for -g as an alias for --strip-debug
This change adds an option -g to llvm-objcopy which is an alias for the existing option --strip-debug.

This fixes PR40003.

Reviewed by: alexshap

Differential Revision: https://reviews.llvm.org/D57217

llvm-svn: 352182
2019-01-25 09:57:20 +00:00