This is a resurrection of D106421 with the change that it keeps backward-compatibility. This means decoding the previous version of `LLVM_BB_ADDR_MAP` will work. This is required as the profile mapping tool is not released with LLVM (AutoFDO). As suggested by @jhenderson we rename the original section type value to `SHT_LLVM_BB_ADDR_MAP_V0` and assign a new value to the `SHT_LLVM_BB_ADDR_MAP` section type. The new encoding adds a version byte to each function entry to specify the encoding version for that function. This patch also adds a feature byte to be used with more flexibility in the future. An use-case example for the feature field is encoding multi-section functions more concisely using a different format.
Conceptually, the new encoding emits basic block offsets and sizes as label differences between each two consecutive basic block begin and end label. When decoding, offsets must be aggregated along with basic block sizes to calculate the final offsets of basic blocks relative to the function address.
This encoding uses smaller values compared to the existing one (offsets relative to function symbol).
Smaller values tend to occupy fewer bytes in ULEB128 encoding. As a result, we get about 17% total reduction in the size of the bb-address-map section (from about 11MB to 9MB for the clang PGO binary).
The extra two bytes (version and feature fields) incur a small 3% size overhead to the `LLVM_BB_ADDR_MAP` section size.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D121346
Instead of the GNU extension `SHF_GNU_RETAIN`, Solaris provides equivalent
functionality with `SHF_SUNW_NODISCARD`. This patch implements the necessary
support.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D107955
Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post processing tools. For example, if we run strip -s on the object files the symbol table changes, but indices in that section do not. In non-visible behavior indices point to wrong symbols. The visible behavior indices point outside of Symbol table: "invalid symbol index".
This patch changes the format by using R_*_NONE relocations to indicate the from/to symbols. The Frequency (Weight) will still be in the .llvm.call-graph-profile, but symbol information will be in relocation section. In LLD information from both sections is used to reconstruct call graph profile. Relocations themselves will never be applied.
With this approach post processing tools that handle relocations correctly work for this section also. Tools can add/remove symbols and as long as they handle relocation sections with this approach information stays correct.
Doing a quick experiment with clang-13.
The size went up from 107KB to 322KB, aggregate of all the input sections. Size of clang-13 binary is ~118MB. For users of -fprofile-use/-fprofile-sample-use the size of object files will go up slightly, it will not impact final binary size.
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D104080
This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table. This also allows
combining the string table with another string table in the object, e.g.
the symbol name string table. The field is optional. By default,
.shstrtab will continue to be used.
This partially fixes https://bugs.llvm.org/show_bug.cgi?id=50506.
Reviewed by: Higuoxing
Differential Revision: https://reviews.llvm.org/D104035
As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D96831
As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D96831
This patch let the yaml encoding use Hex64 values for NumBlocks, BB AddressOffset, BB Size, and BB Metadata.
Additionally, it changes the decoded values in elf2yaml to uint64_t to match DataExtractor::getULEB128 return type.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D95767
We already set the `sh_entsize` field in a single place
for all non-implicit sections.
This patch reorders the logic slightly and with it
we finally have the only one place where the `sh_entsize` is set.
obj2yaml will not dump the `EntSize` key for `SHT_DYNSYM/SHT_SYMTAB` sections anymore,
when the value of `sh_entsize` is equal to `sizeof(Elf_Sym)`
Note that this also seems revealed an issue in llvm-objcopy:
Previously yaml2obj set the `sh_entsize` for the `.symtab` section to 0x18,
now we it sets it for `SHT_SYMTAB` sections, i.e. by type.
But the `llvm-objcopy/ELF/only-keep-debug.test` has a `.symtab` section of type `SHT_STRTAB`,
and now yaml2obj sets the `sh_entsize` to 0 for it.
I had to update the corresponding check lines for `ES`, but the behavior of
`llvm-objcopy` should be fixed instead I think.
I've added a TODO and a comment.
Differential revision: https://reviews.llvm.org/D95364
This was discussed in D93678 thread.
Currently we have one special chunk - Fill.
This patch re implements the "SectionHeaderTable" key to become a special chunk too.
With that we are able to place the section header table at any location,
just like we place sections.
Differential revision: https://reviews.llvm.org/D95140
This makes the following improvements.
For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.
Also, simplifies few test cases.
Differential revision: https://reviews.llvm.org/D94956
This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.
Note that it also fixes a bug of `yaml2obj`: we do not
set the `sh_entsize` field for the `SHT_ARM_EXIDX` section properly.
Differential revision: https://reviews.llvm.org/D93858
Currently we don't support multiple SHT_SYMTAB_SHNDX sections
and the DT_SYMTAB_SHNDX tag currently.
This patch implements it and fixes the
https://bugs.llvm.org/show_bug.cgi?id=43991.
I had to introduce the `struct DataRegion` to ELF.h,
it is used to represent a region that might have no known size.
It is needed, because we don't know the size of the extended
section indices table when it is located via DT_SYMTAB_SHNDX.
In this case we still want to validate that we don't read
past the end of the file.
Differential revision: https://reviews.llvm.org/D92923
Currently we crash when we have an object with SHT_SYMTAB/SHT_DYNSYM sections
of size 0.
With this patch instead of the crash we start to dump them properly.
Differential revision: https://reviews.llvm.org/D93697
We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the `EntSize`
key is currently ignored.
2) obj2yaml is unable to dump the group section which `sh_entsize != 4`.
3) obj2yaml always dumps the "EntSize" for group sections, though
usually we are trying to omit dumping default values when dumping keys.
I.e. we should not print the "EntSize" key when `sh_entsize` == 4.
This patch fixes (1),(3) and adds the test case to document the behavior of (2).
Differential revision: https://reviews.llvm.org/D93854
It has multiple issues fixed by this patch:
1) It shouldn't test how llvm-readelf/yaml2obj works.
2) It should use "-NEXT" prefix for check lines.
3) It can use YAML macros, that allows to use a single YAML.
4) It should probably test the case when a group member is a null section.
Differential revision: https://reviews.llvm.org/D93753
`getUniquedSectionName(const Elf_Shdr *Sec)` assumes that
`Sec` is not `nullptr`.
I've found one place in `getUniquedSymbolName` where it is
not true (because of that we crash when trying to dump
unnamed null section symbols).
Patch fixes the crash and changes the signature of the
`getUniquedSectionName` section to accept a reference.
Differential revision: https://reviews.llvm.org/D93754
This is similar to D93760.
When something is wrong with the hash table header we dump
its context as a raw data.
Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).
The patch fixes it.
Differential revision: https://reviews.llvm.org/D93799
When something is wrong with the GNU hash table header we dump
its context as a raw data.
Currently we have the calculation overflow issue and it is possible to
bypass the validation we have (and crash).
The patch fixes it.
Differential revision: https://reviews.llvm.org/D93760
Currently, `ELFFile<ELFT>::getEntry` does not check an index of
an entry. Because of that the code might read past the end of the symbol
table silently. I've added a test to `llvm-readobj\ELF\relocations.test`
to demonstrate the possible issue. Also, I've added a unit test for
this method.
After this change, `getEntry` stops reporting the section index and
reuses the `getSectionContentsAsArray` method, which already has
all the validation needed. Our related warnings now provide
more and better context sometimes.
Differential revision: https://reviews.llvm.org/D93209
When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and adds test cases for them.
Differential revision: https://reviews.llvm.org/D93010
It is allowed to have multiple `SHT_SYMTAB_SHNDX` sections, though
we currently don't implement it.
The current implementation assumes that there is a maximum of one SHT_SYMTAB_SHNDX
section and that it is always linked with .symtab section.
This patch drops this limitations.
Differential revision: https://reviews.llvm.org/D92644
Currently when we dump sections, we dump them in the order,
which is specified in the sections header table.
With that the order in the output might not match the order in the file.
This patch starts sorting them by by file offsets when dumping.
When the order in the section header table doesn't match the order
in the file, we should emit the "SectionHeaderTable" key. This patch does it.
Differential revision: https://reviews.llvm.org/D91249
This patch starts emitting the `EShNum` key, when the `e_shnum = 0`
and the section header table exists.
`e_shnum` might be 0, when the the number of entries in the section header
table is larger than or equal to SHN_LORESERVE (0xff00).
In this case the real number of entries
in the section header table is held in the `sh_size`
member of the initial entry in section header table.
Currently, obj2yaml crashes when an object has `e_shoff != 0` and the `sh_size`
member of the initial entry in section header table is `0`.
This patch fixes it.
Differential revision: https://reviews.llvm.org/D92098
The following line asserts when `sh_addralign > MAX_UINT32 && (uint32_t)sh_addralign == 0`:
```
ExpectedOffset = alignTo(ExpectedOffset,
SecHdr.sh_addralign ? SecHdr.sh_addralign : 1);
```
it happens because `sh_addralign` is truncated to 32-bit value, but `alignTo`
doesn't accept `Align == 0`. We should change `1` to `1uLL`.
Differential revision: https://reviews.llvm.org/D92163
Currently we never dump the `sh_offset` key.
Though it sometimes an important information.
To reduce the noise this patch implements the following logic:
1) The "Offset" key for the first section is always emitted.
2) If we can derive the offset for a next section naturally,
then the "Offset" key is omitted.
By "naturally" I mean that section[X] offset is expected to be:
```
offsetOf(section[X]) == alignTo(section[X - 1].sh_offset + section[X - 1].sh_size, section[X].sh_addralign)
```
So, when it has the expected value, we omit it from the output.
Differential revision: https://reviews.llvm.org/D91152
When we produce an YAML output, we also print leading zeroes currently.
An output might look like this:
```
- Name: .dynsym
Type: SHT_DYNSYM
Address: 0x0000000000001000
EntSize: 0x0000000000000018
```
There are probably no reason to print leading zeroes.
It just makes harder to read values. This patch stops printing them.
The output becomes like:
```
- Name: .dynsym
Type: SHT_DYNSYM
Address: 0x1000
EntSize: 0x18
```
This affects obj2yaml mostly, but also dsymutil and llvm-xray tools output.
Differential revision: https://reviews.llvm.org/D90930
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.
To put them into segment we can do (1*):
```
Sections:
- Section: foo1
- Section: foo4
```
or we can use (2*):
```
Sections:
- Section: foo1
- Section: foo3
- Section: foo4
```
or (3*) :
```
Sections:
- Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
- Section: (index 2)
- Section: foo3
- Section: foo4
```
It looks really confusing that we don't have to list all of sections.
At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).
This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.
Differential revision: https://reviews.llvm.org/D90458
YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stack_sizes section.
Reviewed By: jhenderson, grimar
Differential Revision: https://reviews.llvm.org/D88717
Currently it is impossible to create an instance of ELFObjectFile when the
SHT_SYMTAB_SHNDX can't be read. We error out when fail to parse the
SHT_SYMTAB_SHNDX section in the factory method.
This change delays reading of the SHT_SYMTAB_SHNDX section entries,
with it llvm-readobj is now able to work with such inputs.
Differential revision: https://reviews.llvm.org/D89379
`Link` is not an optional field currently.
Because of this it is not convenient to write macros.
This makes it optional and fixes corresponding test cases.
Differential revision: https://reviews.llvm.org/D90390
Specification for SHT_HASH table says (https://refspecs.linuxbase.org/elf/gabi4+/ch5.dynamic.html#hash)
that it contains Elf32_Word entries for both 32/64 bit objects.
Currently both GNU linkers and LLD sets the `sh_entsize` field to `4`.
At the same time, `yaml2obj` ignores the `EntSize` field for SHT_HASH sections.
This patch fixes this and also adds a support for obj2yaml: it will not
dump this field when the `sh_entsize` contains the default value (`4`).
Differential revision: https://reviews.llvm.org/D88652
Currently we are always recognizing the `SHT_MIPS_ABIFLAGS` section,
even on non-MIPS targets.
The problem of doing this is briefly discussed in D88228 which does the same for `SHT_ARM_EXIDX`:
"The problem is that `SHT_ARM_EXIDX` shares the value with `SHT_X86_64_UNWIND (0x70000001U)`.
We might have other machine specific conflicts, e.g.
`SHT_ARM_ATTRIBUTES` vs `SHT_MSP430_ATTRIBUTES` vs `SHT_RISCV_ATTRIBUTES (0x70000003U)`."
I think we should only recognize target specific sections when the machine type
matches. I.e. `SHT_MIPS_*` should be recognized only on `MIPS`, `SHT_ARM_*`
only on `ARM` etc.
This patch stops recognizing `SHT_MIPS_ABIFLAGS` on `non-MIPS` targets.
Note: I had to update `ScalarEnumerationTraits<ELFYAML::MIPS_ISA>::enumeration`, because
otherwise test crashes, calling `llvm_unreachable`.
Differential revision: https://reviews.llvm.org/D88294
We have the `Object/Mips/abi-flags.yaml` which tests how yaml2obj/obj2yaml
handle `SHT_MIPS_ABIFLAGS` sections.
This patch splits it into two tests: one for obj2yaml and one for yaml2obj
and moves the result to right places.
Differential revision: https://reviews.llvm.org/D88231
When ELF header's `e_machine == 0`, we emit:
```
Machine: EM_NONE
```
We can avoid doing this, because yaml2obj sets the
`e_machine` field to `EM_NONE` by default.
Differential revision: https://reviews.llvm.org/D87829
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.
Reviewed By: grimar, MaskRay
Differential Revision: https://reviews.llvm.org/D87727
Without this patch, obj2yaml decodes the content of only one ".stack_size" section. Other sections are dumped with their full contents.
Reviewed By: grimar, MaskRay
Differential Revision: https://reviews.llvm.org/D87727
This patch adds support for dumping the .debug_addr(v5) section to
obj2yaml.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D87601
This patch adds support for dumping the .debug_ranges section to
elf2yaml.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D87429
Weining Lu