Commit Graph

592 Commits

Author SHA1 Message Date
Xing GUO 8632931787 [DWARFYAML] Make the length field of compilation units optional. NFC.
This patch makes the length field of compilation units optional (0 by
default).
2020-07-22 12:16:19 +08:00
Xing GUO c657602f3f [DWARFYAML] Add dependency 'BinaryFormat'. NFC.
This patch is trying to fix build failure.

Failed tests:
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/12574
2020-07-20 13:58:22 +08:00
Xing GUO 65c63eb69c [DWARFYAML] Remove 'default' tag. NFC.
This patch is trying to make build bots happy.

Failed bots:
http://lab.llvm.org:8011/builders/ppc64le-lld-multistage-test/builds/10705
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/33595
2020-07-20 11:14:50 +08:00
Xing GUO 1ab3d6c819 [DWARFYAML] Implement the .debug_rnglists section.
This patch implements the .debug_rnglists section. We are able to
produce the .debug_rnglists section by the following syntax.

```
debug_rnglists:
  - Format:              DWARF32 ## Optional
    Length:              0x1234  ## Optional
    Version:             5       ## Optional
    AddressSize:         0x08    ## Optional
    SegmentSelectorSize: 0x00    ## Optional
    OffsetEntryCount:    2       ## Optional
    Offsets:             [1, 2]  ## Optional
    Lists:
      - Entries:
          - Operator: DW_RLE_base_address
            Values:   [ 0x1234 ]
```

The generated .debug_rnglists is verified by llvm-dwarfdump, except for
the operator DW_RLE_startx_endx, since llvm-dwarfdump doesn't support
it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83624
2020-07-20 10:42:27 +08:00
Georgii Rymar 6227f04a09 [llvm-readobj] - Add proper testing for the SHT_MIPS_ABIFLAGS section.
This rewrites the mips-abiflags.test to stop using recompiled objects,
adds testing for all missed bits and also adds two missing enum values
to lib/ObjectYAML, which are used in the new test.

Differential revision: https://reviews.llvm.org/D83954
2020-07-17 15:24:39 +03:00
Xing GUO dc65f57124 [DWARFYAML] Merge forms that use same encodings. NFC. 2020-07-17 11:31:49 +08:00
Xing GUO 10478c9c64 [DWARFYAML] Implement the .debug_str_offsets section.
This patch helps add support for emitting the .debug_str_offsets section
to yaml2elf.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D83853
2020-07-16 22:33:13 +08:00
Georgii Rymar ed605b7b96 [yaml2obj] - Fix an issue with NoHeaders key.
When setting the NoHeaders to false,
the e_shnum field wasn't set correctly.

This patch fixes this bug.

Differential revision: https://reviews.llvm.org/D83941
2020-07-16 14:22:36 +03:00
Xing GUO a9e10a0965 [DWARFYAML] Remove useless conditional statement. NFC.
The conditional statement is useless after
3a48a632d0.
2020-07-15 19:04:58 +08:00
Georgii Rymar 7a587ca932 [yaml2obj] - Rename FileHeader::SH* fields.
In D83482 we agreed to name e_* fields that are used for overriding
values (like e_phoff) as EPh* (e.g. EPhOff).

Currently we have a set of e_sh* fields that are named inconsistently
with this rule. This patch renames all of them.

Differential revision: https://reviews.llvm.org/D83766
2020-07-15 12:47:31 +03:00
Georgii Rymar 90e34b563a [yaml2obj] - Refine handling of the NoHeaders key.
Imagine we have an YAML description for some object and we want to
produce 2 outputs: with and without the section header.
A natural way to do it would look like:

```
--- !ELF
FileHeader:
  Class:   ELFCLASS64
  Data:    ELFDATA2LSB
  Type:    ET_REL
  Machine: EM_X86_64
Sections:
...
SectionHeaderTable:
  NoHeaders: [[NOHEADERS]]

```
But currently, we do not distinguish between no `NoHeaders` key case
and `NoHeaders == false`. Because of this we can't simply specify
`NOHEADERS = false`, as tool starts to complain.

With this patch the behavior changed. When we have:

```
SectionHeaderTable:
  NoHeaders: false

```
it is the same as we have no `SectionHeaderTable` at all.
(`NoHeaders` key still can't be used with `Sections/Excluded` keys)

Differential revision: https://reviews.llvm.org/D83672
2020-07-14 15:09:06 +03:00
Xing GUO b539f01221 [DWARFYAML] Add support for emitting value forms of strx, addrx, etc.
This patch adds support for emitting value forms of DW_FORM_strx,
DW_FORM_addrx, DW_FORM_loclistx and DW_FORM_rnglistx.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83749
2020-07-14 19:10:15 +08:00
Georgii Rymar 3001569323 [yaml2obj] - Add a syntax to override e_phoff, e_phentsize and e_phnum fields.
This adds `EPhOff`, `EPhEntSize` and `EPhNum` keys.
Will be useful for creating broken objects for testing llvm-readelf.

Differential revision: https://reviews.llvm.org/D83482
2020-07-14 13:16:19 +03:00
Xing GUO 830a7c2ad4 [DWARFYAML] Replace Is64bit with Is64BitAddrSize. NFC.
Is64bit is ambiguous. In this patch, we replace it with Is64BitAddrSize
to make it clearer.
2020-07-14 10:58:35 +08:00
Xing GUO 47c4ce41a1 [DWARFYAML] Use override instead of virtual for better safety.
Functions in DWARFYML::FixupVisitor are declared as
virtual functions in its base class DWARFYAML::Visitor.
We should use the mordern "override" keyword instead
of "virtual" for virtual functions in subclasses for
better safety.

Besides, the visibility is changed from private to
protected to make it consistent with
DWARFYAML::FixupVisitor class and DWARFYAML::Visitor
class.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83452
2020-07-09 18:55:42 +08:00
Wouter van Oortmerssen 4d135b0446 [WebAssembly] 64-bit memory limits 2020-07-06 12:40:45 -07:00
Xing GUO b954cb408f [ObjectYAML][ELF] Add support for emitting the .debug_gnu_pubnames/pubtypes sections.
This patch helps add support for emitting the .debug_gnu_pubnames and .debug_gnu_pubtypes sections.

The .debug_gnu_pub* sections is verified by llvm-dwarfdump.

Known issues:
- Doesn't support emitting multiple pub-tables.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82367
2020-07-03 22:40:18 +08:00
Xing GUO b6f08b7cfd [DWARFYAML][debug_gnu_*] Add the missing context `IsGNUStyle`. NFC.
This patch helps add the missing context `IsGNUStyle`. Before this patch, yaml2obj cannot parse the YAML description of 'debug_gnu_pubnames' and 'debug_gnu_pubtypes' correctly due to the missing context.

In other words, if we have

```
DWARF:
  debug_gnu_pubtypes:
    Length:
      TotalLength: 0x1234
    Version:    2
    UnitOffset: 0x1234
    UnitSize:   0x4321
    Entries:
      - DieOffset:  0x12345678
        Name:       abc
        Descriptor: 0x00      ## Descriptor can never be mapped into Entry.Descriptor
```

yaml2obj will complain that "error: unknown key 'Descriptor'".

This patch helps resolve this problem.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82435
2020-07-03 18:12:58 +08:00
Xing GUO 38907b696c [DWARFYAML][debug_abbrev] Emit 0 byte for terminating abbreviations.
The abbreviations for a given compilation unit end with an entry
consisting of a 0 byte for the abbreviation code.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82933
2020-07-02 00:09:29 +08:00
Georgii Rymar 64bae035ef [yaml2obj] - Support reading a content as an array of bytes using the new 'ContentArray' key.
It implements the way to describe a section content using a multi line description. E.g:

```
- Name:         .foo
  Type:         SHT_PROGBITS
  ContentArray: [ 0x11, 0x22, 0x33, 0x44,                                ## .long 11223344
                  0x55, 0x66,                                            ## .short 5566.
                  0x77,                                                  ## .byte 0x77
                  0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0x00 ] ## .quad 0x8899aabbccddeeff
```

It was briefly discussed in D75123 thread previously.

Differential revision: https://reviews.llvm.org/D82366
2020-06-30 12:13:23 +03:00
Xing GUO fe08ab542b [DWARFYAML][debug_info] Replace 'InitialLength' with 'Format' and 'Length'.
'InitialLength' is replaced with 'Format' (DWARF32 by default) and 'Length' in this patch.
Besides, test cases for DWARFv4 and DWARFv5, DWARF32 and DWARF64 is
added.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82622
2020-06-30 16:28:39 +08:00
Xing GUO 8f9ca561a2 [ObjectYAML][DWARF] Collect diagnostic message when YAMLParser fails.
Before this patch, the diagnostic message is printed to `errs()` directly, which makes it difficult to use `FailedWithMessage()` in unit testing.
In this patch, we add a custom error handler for YAMLParser, which helps collect diagnostic messages and make it easy to use `FailedWithMessage()` to check error messages.

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D82630
2020-06-29 16:13:53 +08:00
Xing GUO 7686b29584 [DWARFYAML][debug_info] Teach yaml2obj emit correct DWARF64 unit header.
This patch helps teach yaml2obj emit correct DWARF64 unit header of the .debug_info section.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82621
2020-06-26 19:25:26 +08:00
Xing GUO 17326ebbd6 [ObjectYAML][DWARF] Format codes. NFC. 2020-06-25 21:53:06 +08:00
Xing GUO 93bc571d47 [DWARFYAML][debug_gnu_*] 'Descriptor' field should be 1-byte. NFC.
The 'Descriptor' field of .debug_gnu_pubnames and .debug_gnu_pubtypes
section should be 1-byte rather than 4-byte. This patch helps resolve
this issue.
2020-06-25 08:21:13 +08:00
Xing GUO 45fa936855 [ObjectYAML][DWARF] Remove unused context. NFC.
The context is unused. This patch helps remove it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82351
2020-06-24 00:02:51 +08:00
Xing GUO fad54c50e4 [ObjectYAML][ELF] Add support for emitting the .debug_pubtypes section.
This patch helps add support for emitting the .debug_pubtypes section.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82347
2020-06-24 00:01:07 +08:00
Xing GUO 8c7775e9a7 [ObjectYAML][ELF] Add support for emitting the .debug_pubnames section.
This patch helps add support for emitting the .debug_pubnames section to yaml2elf.

Known issues:
- Current implementation doesn't support emitting multiple sets of entries.
- Doesn't support DWARF64.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82296
2020-06-23 20:40:33 +08:00
Sam Clegg 79aad89d8d [WebAssembly] Add support for externalref to MC and wasm-ld
This allows code for handling externref values to be processed by the
assembler and linker.

Differential Revision: https://reviews.llvm.org/D81977
2020-06-22 15:57:24 -07:00
Xing GUO 03480c80d3 [DWARFYAML][debug_info] Add support for error handling.
This patch helps add support for error handling in `DWARFYAML::emitDebugInfo()`.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82275
2020-06-22 21:36:13 +08:00
Xing GUO 3a48a632d0 [DWARFYAML][debug_info] Use 'AbbrCode' to index the abbreviation.
Before this patch, we use `(uint32_t)AbbrCode - (uint32_t)FirstAbbrCode` to index the abbreviation. It's impossible for we to use the preceeding abbreviation of the previous one (e.g., if the previous DIE's `AbbrCode` is 2, we are unable to use the abbreviation with index 1). In this patch, we use `AbbrCode` to index the abbreviation directly.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82173
2020-06-22 21:34:02 +08:00
Xing GUO 6770349592 [DWARFYAML][debug_info] Fix array index out of bounds error
This patch is trying to fix the array index out of bounds error. I observed it in (https://reviews.llvm.org/harbormaster/unit/view/99638/).

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D82139
2020-06-20 15:08:24 +08:00
Xing GUO 1cfdda57fa [ObjectYAML][ELF] Add support for emitting the .debug_info section.
This patch helps add support for emitting the .debug_info section to yaml2elf.

Reviewed By: jhenderson, grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D82073
2020-06-20 12:13:01 +08:00
Xing GUO d261a1c0e0 [DWARFYAML][debug_abbrev] Make the abbreviation code optional.
This patch helps make the `Code` optional in abbreviations table.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81826
2020-06-18 13:02:54 +08:00
Xing GUO 1f391afbf4 [ObjectYAML][ELF] Add support for emitting the .debug_abbrev section.
This patch enables yaml2elf emit the .debug_abbrev section.

The generated .debug_abbrev is verified using `llvm-dwarfdump`.

Known issues that will be addressed later:
- Current implementation doesn't support generating multiple abbreviation tables in one .debug_abbrev section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81820
2020-06-18 10:50:38 +08:00
Xing GUO 9aaa32cfcb [ObjectYAML][DWARF] Let writeVariableSizedInteger() return Error.
This patch helps change the return type of `writeVariableSizedInteger()` from `void` to `Error`.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81915
2020-06-17 09:30:14 +08:00
Xing GUO 8aaeaddec8 [ObjectYAML][DWARF] Implement the .debug_addr section.
This patch implements the .debug_addr section.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81541
2020-06-16 10:53:10 +08:00
Stanislav Mekhanoshin 9ee272f13d [AMDGPU] Add gfx1030 target
Differential Revision: https://reviews.llvm.org/D81886
2020-06-15 16:18:05 -07:00
Wouter van Oortmerssen 3b29376e3f [WebAssembly] Adding 64-bit version of R_WASM_MEMORY_ADDR_* relocs
This adds 4 new reloc types.

A lot of code that previously assumed any memory or offset values could be contained in a uint32_t (and often truncated results from functions returning 64-bit values) have been upgraded to uint64_t. This is not comprehensive: it is only the values that come in contact with the new relocation values and their dependents.

A new tablegen mapping was added to automatically upgrade loads/stores in the assembler, which otherwise has no way to select for these instructions (since they are indentical other than for the offset immediate). It follows a similar technique to https://reviews.llvm.org/D53307

Differential Revision: https://reviews.llvm.org/D81704
2020-06-15 10:07:42 -07:00
Georgii Rymar ec4e68e667 [yaml2obj] - Introduce the "NoHeaders" key for "SectionHeaderTable"
We have an issue currently. The following YAML piece just ignores the `Excluded` key.

```
SectionHeaderTable:
  Sections: []
  Excluded:
    - Name: .foo
```

Currently the meaning is: exclude the whole table.

The code checks that the `Sections` key is empty and doesn't catch/check
invalid/duplicated/missed `Excluded` entries.

Also there is no way to exclude all sections except the first null section,
because `Sections: []` currently just excludes the whole the sections header table.

To fix it, I suggest a change of the behavior.

1) A new `NoHeaders` key is added. It provides an explicit syntax to drop the whole table.
2) The meaning of the following is changed:

```
SectionHeaderTable:
  Sections: []
  Excluded:
    - Name: .foo

```
Assuming there are 2 sections in the object (a null section and `.foo`), with this patch it
means: exclude the `.foo` section, keep the null section. The null section is an implicit
section and I think it is reasonable to make "Sections: []" to mean it is implicitly added.
It will be consistent with the global "Sections" tag that is used to describe sections.

3) `SectionHeaderTable->Sections` is now optional. No `Sections` is the same as
   `Sections: []` (I think it avoids a confusion).
4) Using of `NoHeaders` together with `Sections`/`Excluded` is not allowed.
5) It is possible to use the `Excluded` key without the `Sections` key now (in this case
   `Excluded` must contain all sections).
6) `SectionHeaderTable:` or `SectionHeaderTable: []` is not allowed.
7) When the `SectionHeaderTable` key is present, we still require all sections to be
   present in `Sections` and `Excluded` lists. No changes here, we are still strict.

Differential revision: https://reviews.llvm.org/D81655
2020-06-15 12:43:16 +03:00
Xing GUO ff9c1ae213 [ObjectYAML][DWARF] Let the target address size be inferred from FileHeader.
This patch adds a new field `bool Is64bit` in `DWARFYAML::Data` to indicate the address size of target. It's helpful for inferring the `AddrSize` in some DWARF sections.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D81709
2020-06-14 12:42:20 +08:00
Xing GUO 0431e4bcb2 Recommit "[DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`."
This recommits fcc0c186e9
2020-06-13 23:39:11 +08:00
Xing GUO 325f7607b0 Revert "[DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`."
This reverts commit fcc0c186e9.
2020-06-13 17:57:02 +08:00
Xing GUO fcc0c186e9 [DWARFYAML][debug_line] Replace `InitialLength` with `Format` and `Length`. 2020-06-13 17:47:06 +08:00
Xing GUO 613c4a87ba [ObjectYAML][DWARF] Add one helper function `writeInitialLength()`. NFC. 2020-06-12 21:10:58 +08:00
Georgii Rymar 818ab3d654 [yaml2obj] - Allocate the file space for SHT_NOBITS sections in some cases.
This teaches yaml2obj to allocate file space for a no-bits section
when there is a non-nobits section in the same segment that follows it.

It was discussed in D78005 thread and matches GNU linkers and LLD behavior.

Differential revision: https://reviews.llvm.org/D80629
2020-06-11 12:54:53 +03:00
Xing GUO 99c2335434 [DWARFYAML][debug_ranges] Make the "Offset" field optional.
Before this patch, we have to calculate the offset for the current range list entry. This patch helps make the "Offset" field optional.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81220
2020-06-11 08:36:44 +08:00
Xing GUO 502a2a80c2 [DWARFYAML] Add support for emitting DWARF64 .debug_aranges section.
The `debug_info_offset`(`CuOffset`) should be 64-bit width rather than 32-bit width in DWARF64 .debug_aranges section. This patch helps resolve it.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81528
2020-06-11 08:35:17 +08:00
Georgii Rymar 3c123acf57 [yaml2obj] - Introduce a 10 Mb limit of the output by default and a --max-size option.
Multiple times we faced an issue of huge outputs due to unexpected behavior
or incorrect test cases. The last one was https://reviews.llvm.org/D80629#2073066.

This patch limits the output to 10 Mb for ELF and introduces the --max-size to change this
limit.

I've tried to keep the implementation non-intrusive.

The current logic we have is that we prepare section content in a buffer first and write
it to the output later. This patch checks the available limit on each writing attempt to this buffer
and stops writing when the limit is reached and raises the internal error flag.
Later, this flag is is checked before the actual writing to a file happens and
an error is reported.

Differential revision: https://reviews.llvm.org/D81258
2020-06-10 15:23:59 +03:00
Xing GUO 9728000395 [ObjectYAML][ELF] Add support for emitting the .debug_line section.
This patch enables yaml2elf emit the .debug_line section.

Test cases for emitting the dwarf64 .debug_line section and opcodes will be added later.

Known issues:
- We should replace `InitialLength` with `Format` and `Length`
- Currently implementation of the .debug_line section only fully supports DWARFv2, some header fields in DWARFv4 and DWARFv5 is missing, e.g., `header_length` in DWARFv4, `address_size` and `segment_selector_size` in DWARFv5.
- Some opcodes relies on the .debug_info section, we should warn user about it.

These issues will be addressed in a follow-up patch.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81450
2020-06-09 23:42:40 +08:00
Xing GUO e4344e1466 [DWARFYAML][debug_ranges] Emit an error message for invalid offset.
This patch helps make yaml2obj emit an error message when we try to assign an invalid offset to the entry of the 'debug_ranges' section.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81357
2020-06-09 18:53:38 +08:00
Xing GUO 9939f231e6 [ObjectYAML] Add support for error handling in DWARFYAML. NFC.
This patch intends to be an NFC-patch. Test cases will be added in a follow-up patch.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81356
2020-06-08 22:49:29 +08:00
Xing GUO 67b4afc41a [DWARFYAML] Rename function names to match the coding style. NFC. 2020-06-08 17:31:02 +08:00
Xing GUO d527690103 [DWARFYAML][debug_ranges] Fix inappropriate assertion. NFC. 2020-06-07 22:45:52 +08:00
Xing GUO 288025494e [ObjectYAML][DWARF] Support emitting .debug_ranges section in ELFYAML.
This patch enables yaml2elf to emit the .debug_ranges section.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81217
2020-06-07 15:47:47 +08:00
Xing GUO 929edd8bd2 [DWARFYAML][debug_aranges] Replace InitialLength with Format and Length.
This patch addresses the comment in [D80972](https://reviews.llvm.org/D80972#inline-744217).

Before this patch, the initial length field of .debug_aranges section should be declared as:

```
## 32-bit DWARF
debug_aranges:
  - Length:
      TotalLength: 0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Length:
      TotalLength:   0xffffffff
      TotalLength64: 0x20
    Version: 2
    ...
```

After this patch:

```
## 32-bit DWARF
debug_aranges:
  - [[Format:  DWARF32]] ## Optional
    Length:  0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Format:  DWARF64
    Length:  0x20
    Version: 2
```

Current implementation of generating DWARF64 .debug_aranges section is buggy. A follow-up patch will improve it and add test cases for DWARF64.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D81063
2020-06-05 12:16:44 +08:00
Georgii Rymar 9d739a9157 [ObjectYAML] - Remove unused function. NFC.
Was introduced in D81005 by mistake.

Catched by BB:
http://lab.llvm.org:8011/builders/clang-ppc64le-rhel/builds/4070/steps/build%20stage%201/logs/stdio
2020-06-04 14:22:51 +03:00
Georgii Rymar c781e7370e [yaml2obj] - Add a way to exclude specified sections from the section header.
This implements a new "Excluded" key that can be used
to exclude entries from section header:

```
SectionHeaderTable:
  Sections:
    ...
  Excluded:
    - Name: .foo
```

Differential revision: https://reviews.llvm.org/D81005
2020-06-04 13:50:35 +03:00
Georgii Rymar 5750f12b82 Revert "[yaml2obj] - Allocate the file space for SHT_NOBITS sections in some cases."
This reverts commit aa3a85cdaa.

There are problems with it. See here: https://reviews.llvm.org/D80629
2020-06-04 13:17:48 +03:00
Xing GUO b6020c330d [ObjectYAML][ELF] Let the endianness of DWARF sections be inferred from FileHeader.
This patch helps infer the endianness of DWARF sections from `FileHeader`.

Reviewed By: jhenderson, grimar

Differential Revision: https://reviews.llvm.org/D81051
2020-06-04 09:26:05 +08:00
Xing GUO a50f4eb2e9 [ObjectYAML][DWARF] Support emitting the .debug_aranges section in ELFYAML.
This patch enables yaml2obj to emit the .debug_aranges section in ELFYAML.

Known issues:
- The current implementation of `debug_aranges` doesn't support emitting `segment` in the `(segment, address, length)` tuple. I will fix it in a follow-up patch.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D80972
2020-06-04 08:56:00 +08:00
Georgii Rymar aa3a85cdaa [yaml2obj] - Allocate the file space for SHT_NOBITS sections in some cases.
This teaches yaml2obj to allocate file space for a no-bits section
when there is a non-nobits section in the same segment that follows it.

It was discussed in D78005 thread and matches GNU linkers and LLD behavior.

Differential revision: https://reviews.llvm.org/D80629
2020-06-02 17:19:24 +03:00
Xing GUO ea7db621d2 [ObjectYAML][DWARF] Make the `PubSection` optional.
This patch helps make the `PubSection` optional in the DWARF structure.

Reviewed By: jhenderson, aprantl

Differential Revision: https://reviews.llvm.org/D80722
2020-05-29 20:11:53 +08:00
Xing GUO 373e98a3a5 [ObjectYAML][DWARF] Add DWARF entry in ELFYAML.
This patch adds a new DWARF entry in ELF YAML file.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D80203
2020-05-29 10:57:18 +08:00
Georgii Rymar ad07d5f394 [yaml2obj] - Implement the "SectionHeaderTable" tag.
With the "SectionHeaderTable" it is now possible to reorder
entries in the section header table.

It also allows to stop emitting the table.

Differential revision: https://reviews.llvm.org/D80002
2020-05-28 13:42:43 +03:00
Kazushi (Jam) Marukawa 5921782f74 [VE] Implements minimum MC layer for VE (3/4)
Summary:
Define ELF binary code for VE and modify code where should use this new code.

Depends on D79544.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D79545
2020-05-28 10:07:48 +02:00
Xing GUO 3c3a6e26e7 [ObjectYAML][MachO] Add error handling in MachOEmitter.
Currently, `yaml2macho` doesn't support error handling. This patch helps improve it.

Differential Revision: https://reviews.llvm.org/D80535
2020-05-28 09:54:46 +08:00
Xing GUO 2c04b8aacd [ObjectYAML][DWARF] Make variable names consistent. 2020-05-26 17:45:10 +08:00
Xing GUO 590f3a72c2 [ObjectYAML][DWARF] Use .empty() to indicate if the DWARF sections are empty. 2020-05-26 17:45:10 +08:00
Georgii Rymar 3d4c873a14 [yaml2obj] - Map section names to chunks for each ELFYAML::ProgramHeader early. NFCI.
Each `ELFYAML::ProgramHeader` currently contains a list of section names
included. We are trying to map them to Fill/Sections very late,
though we can create such mapping early, in `initProgramHeaders`.

The benefit is that with such change it is possible to access mapped
chunks earlier (for example during writing section content) and have
simpler code.

Differential revision: https://reviews.llvm.org/D80520
2020-05-26 12:32:10 +03:00
Georgii Rymar 304b0ed403 [yaml2obj] - Move "repeated section/fill name" check earlier.
This allows to simplify the code.
Doing checks early is generally useful.

Differential revision: https://reviews.llvm.org/D79985
2020-05-23 17:40:48 +03:00
Georgii Rymar 38c5d6f700 [yaml2obj] - Add a technical prefix for each unnamed chunk.
This change does not affect the produced binary.

In this patch I assign a technical suffix to each section/fill
(i.e. chunk) name when it is empty. It allows to simplify the code
slightly and improve error messages reported.

In the code we have the section to index mapping, SN2I, which is
globally used. With this change we can use it to map "empty"
names to indexes now, what is helpful.

Differential revision: https://reviews.llvm.org/D79984
2020-05-23 17:22:23 +03:00
Georgii Rymar baf3225987 [yaml2obj] - Implement the "Offset" property for the Fill Chunk.
Similar to a regular section chunk, a Fill should have this property.
This patch implements it.

Differential revision: https://reviews.llvm.org/D80190
2020-05-20 13:38:48 +03:00
Georgii Rymar e2b134b01a [yaml2obj] - Stop using square brackets for unique suffixes.
For describing section/symbol names we can use unique suffixes,
e.g:

```
- Name: '.foo [1]`
- Name: '.foo [2]`
```

It can be a problem (see https://reviews.llvm.org/D79984#inline-734829),
because `[]` are sometimes used to describe a macros:

```
- Name: "[[a0]]"
```

Seems the better approach is to use something else, like "()".
This patch does it and refactors the code related.

Differential revision: https://reviews.llvm.org/D80123
2020-05-19 12:59:13 +03:00
Xing GUO 42011fb1c8 [ObjectYAML][DWARF] Take into account other debug sections in DWARFYAML::Data::isEmpty(). 2020-05-17 22:53:27 +08:00
Georgii Rymar 7ccae2cece [yaml2obj] - Introduce the "Offset" property for sections.
Currently there is no good way to set a physical offset for a section:

* We have the `ShOffset` that allows to override the `sh_offset`, but
  it does not affect the real data written.
* We can use a `Filler` to create an artificial gap, but it is more like a hack
  rather than a proper solution for this problem.

This patch adds the `Offset` property which allows setting physical
offsets for sections.

It also generalizes the code, so that we set sh_offset field in one place

Differential revision: https://reviews.llvm.org/D78927
2020-05-15 11:23:44 +03:00
Greg Clayton 6e73f12a64 Fix buildbots errors after comitting D78782.
Rename "Ranges" variables to "DebugRanges" to avoid warnings/errors on machines that have extra settings enabled.

https://reviews.llvm.org/D78782
2020-05-13 22:01:57 -07:00
Greg Clayton 6025fc2243 Add .debug_ranges support to the DWARF YAML.
Summary: This allows DIEs with DW_AT_ranges to be encoded and decoded _and_ actually have their address ranges be included instead of having DW_AT_ranges with a section offset value for a section that doesn't exist.

Reviewers: labath, aprantl, JDevlieghere, dblaikie, probinson

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78782
2020-05-13 16:21:45 -07:00
Reid Kleckner 7af4bb1641 [PDB] Remove unique_ptr wrapper around C13 line table subsections
This accounts for a large portion of the memory allocations in LLD.
This DebugSubsectionRecordBuilder object can be stored directly in
C13Builders, it mostly wraps other subsections.

Remove the container kind field from the object. It is always the same
for all elements in the vector, and we can pass it in during writing.
2020-05-02 16:35:07 -07:00
Georgii Rymar 9f9a08e19c [obj2yaml] - Program headers: simplify the computation of p_filesz.
Currently we have computations of `p_filesz` and `p_memsz` mixed together
with the use of a loop over fragments. After recent changes it is possible to
avoid using a loop for the computation of `p_filesz`, since we know that fragments
are sorted by their file offsets.

The main benefit of this change is that splits the computation of `p_filesz`
and `p_memsz` what is simpler and allows us to fix the computation of the
`p_memsz` independently (D78005 shows the issue that we have currently).

Differential revision: https://reviews.llvm.org/D78628
2020-04-24 15:23:16 +03:00
Georgii Rymar 34b3d5b66c [obj2yaml] - Zero initialize program headers. NFCI.
It allows to simplify the current code and also
might help for the code around.

It is also consistent with what we do for another headers,
e.g. section headers, elf file header etc.

Differential revision: https://reviews.llvm.org/D78627
2020-04-23 12:39:14 +03:00
Alexander Shaposhnikov c19c3293d3 [ObjectYAML][MachO] Add support for relocations
Add support for relocations for MachO to ObjectYAML / yaml2obj / obj2yaml.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D77844
2020-04-22 11:50:55 -07:00
Georgii Rymar 2bf5674317 [yaml2obj] - Program headers: add an additional check for `Offset`
The `Offset` field is used to set the file offset of a program header.
In a normal object it should not be greater than the minimal offset
of sections included into segment.

This patch adds a check for that and adds tests.

Differential revision: https://reviews.llvm.org/D78304
2020-04-22 12:49:05 +03:00
Georgii Rymar 87d33d9e09 [yaml2obj] - Change how p_offset is calculated when creating segments. NFCI.
This depends on D78361 and simplifies the computation of the `p_offset`.

Differential revision: https://reviews.llvm.org/D78363
2020-04-22 12:37:44 +03:00
Georgii Rymar 3471ae9dad [yaml2obj] - Verify that sections are sorted by their file offsets when creating segments.
This validates that sections listed for a segment in the YAML
declaration are ordered by their file offsets.

It might help to simplify the file size computation, but also
is useful by itself as helps to avoid issues in test cases and
to maintain their readability.

Differential revision: https://reviews.llvm.org/D78361
2020-04-21 15:50:42 +03:00
Wouter van Oortmerssen 3eaeebec95 [WebAssembly] Fix WasmEmitter link error. 2020-04-16 17:23:18 -07:00
Wouter van Oortmerssen 48139ebc3a [WebAssembly] Add int32 DW_OP_WASM_location variant
This to allow us to add reloctable global indices as a symbol.
Also adds R_WASM_GLOBAL_INDEX_I32 relocation type to support it.

See discussion in https://github.com/WebAssembly/debugging/issues/12
2020-04-16 16:32:17 -07:00
Georgii Rymar 93fc0ba145 [yaml2obj] - Add NBucket and NChain fields for the SHT_HASH section.
These fields allows to override nchain and nbucket fields of a SHT_HASH section.

Differential revision: https://reviews.llvm.org/D76834
2020-04-01 12:28:16 +03:00
Kai Wang 581ba35291 [RISCV] ELF attribute section for RISC-V.
Leverage ARM ELF build attribute section to create ELF attribute section
for RISC-V. Extract the common part of parsing logic for this section
into ELFAttributeParser.[cpp|h] and ELFAttributes.[cpp|h].

Differential Revision: https://reviews.llvm.org/D74023
2020-03-31 16:16:19 +08:00
Heejin Ahn f93426c5b9 [WebAssembly] Move event section before global section
Summary:
https://github.com/WebAssembly/exception-handling/issues/98

Also this moves many parts of code to make code align with the section
order, even if they don't affect the output.

Reviewers: tlively

Subscribers: dschuff, sbc100, hiraditya, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76752
2020-03-25 11:49:03 -07:00
Georgii Rymar 4dd5f1ca9b [yaml2obj] - Add `ELFYAML::YAMLIntUInt` to fix how we parse a relocation `Addend` key.
This patch makes `Relocation::Addend` to be `ELFYAML::YAMLIntUInt` and not `int64_t`.

`ELFYAML::YAMLIntUInt` it is a new type and it has the following benefits/features:

1) For an 64-bit object any hex/decimal addends
   in the range [INT64_MIN, UINT64_MAX] is accepted.
2) For an 32-bit object any hex/decimal addends
   in range [INT32_MIN, UINT32_MAX] is accepted.
3) Negative hex numbers like -0xffffffff are not accepted.
4) It is printed as decimal. I.e. obj2yaml will print
   something like "Addend: 125", this matches the current behavior.

This fixes all FIXMEs in `relocation-addend.yaml`.

Differential revision: https://reviews.llvm.org/D75527
2020-03-17 14:22:19 +03:00
Georgii Rymar b236b4cb43 [yaml2obj] - Set a default value for `PAddr` property of a program header to a value of `VAddr`
`PAddr` corresponds to `p_paddr` of a program header, which is the segment's physical
address for systems in which physical addressing is relevant. `p_paddr` is often equal
to `p_vaddr`, which is the virtual address of a segment.

This patch changes the default for `PAddr` from 0 to a value of `VAddr`.

Differential revision: https://reviews.llvm.org/D76131
2020-03-14 17:44:57 +03:00
Georgii Rymar e4ceb8f421 [lib/ObjectYAML] - Make `ELFYAML::Relocation::Offset` optional.
Currently `yaml2obj` require `Offset` field in a relocation description.
There are many cases when `Offset` is insignificant in a context of a test case.

Making `Offset` optional allows to simplify our test cases.
This is what this patch does.

Also, with this patch `obj2yaml` does not dump a zero offset of a relocation.

Differential revision: https://reviews.llvm.org/D75608
2020-03-06 13:59:58 +03:00
Georgii Rymar 157b3d505f [yaml2obj] - Address post commit comments for D74764
It removes a stale comment and fixes the comment in the test
and section names related accordingly.
2020-02-25 13:26:46 +03:00
Georgii Rymar 31f2ad9c36 [yaml2obj] - Automatically assign sh_addr for allocatable sections.
I've noticed that it is not convenient to create YAMLs from
binaries (using obj2yaml) that have to be test cases for obj2yaml
later (after applying yaml2obj).

The problem, for example is that obj2yaml emits "DynamicSymbols:"
key instead of .dynsym. It also does not create .dynstr.
And when a YAML document without explicitly defined .dynsym/.dynstr
is given to yaml2obj, we have issues:

1) These sections are placed after non-allocatable sections (I've fixed it in D74756).
2) They have VA == 0. User needs create descriptions for such sections explicitly manually
    to set a VA.

This patch addresses (2). I suggest to let yaml2obj assign virtual addresses by itself.
It makes an output binary to be much closer to "normal" ELF.
(It is still possible to use "Address: 0x0" for a section to get the original behavior
if it is needed)

Differential revision: https://reviews.llvm.org/D74764
2020-02-22 14:43:54 +03:00
Georgii Rymar 3439d4ee0e [yaml2obj] - Change the order of implicitly created sections.
.dynsym and .dynstr are allocatable and therefore normally are placed
before non-allocatable .strtab, .shstrtab, .symtab sections.
But we are placing them after currently what creates a mix of
alloc/non-alloc sections and does not look normal.

Differential revision: https://reviews.llvm.org/D74756
2020-02-19 15:09:19 +03:00
Bill Wendling c55cf4afa9 Revert "Remove redundant "std::move"s in return statements"
The build failed with

  error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a793.
2020-02-10 07:07:40 -08:00
Bill Wendling 1c2241a793 Remove redundant "std::move"s in return statements 2020-02-10 06:39:44 -08:00
Georgii Rymar fd0abcbfc1 [yaml2obj] - Change NameIndex to StName for Symbol.
It is consistent with the approach we use for Section struct.

Differential revision: https://reviews.llvm.org/D74034
2020-02-06 12:04:19 +03:00
Georgii Rymar 60f161eb62 [yaml2obj][obj2yaml] - Simplify format of the SHT_LLVM_ADDRSIG section.
Previously the description allowed to describe symbols with use of
`Name` and `Index` keys. This patch removes them and now it is still
possible to use either names or symbol indexes, but the code is simpler
and the format is slightly different.

Such a change will be useful for another patches, e.g:
https://reviews.llvm.org/D73788#inline-671077

Differential revision: https://reviews.llvm.org/D73888
2020-02-05 12:33:14 +03:00
Georgii Rymar bec54e464e [yaml2obj/obj2yaml] - Add support for the SHT_LLVM_CALL_GRAPH_PROFILE section.
This is a LLVM specific section that is well described here:
https://llvm.org/docs/Extensions.html#sht-llvm-call-graph-profile-section-call-graph-profile

This patch teaches yaml2obj and obj2yaml about how to work with it.

Differential revision: https://reviews.llvm.org/D73788
2020-02-04 15:13:20 +03:00
Georgii Rymar 1649c0098a [yaml2obj] - Add a way to set sh_entsize for relocation sections.
We are missing ability to override the sh_entsize field for
SHT_REL[A] sections. It would be useful for writing test cases.

Differential revision: https://reviews.llvm.org/D73621
2020-01-30 14:51:33 +03:00
Krzysztof Parzyszek 305bf5b21d [Hexagon] Add support for Hexagon v67t microarchitecture (tiny core) 2020-01-21 11:35:10 -06:00
Krzysztof Parzyszek c12a5917d2 [Hexagon] Add support for Hexagon/HVX v67 ISA 2020-01-20 16:16:49 -06:00
Krzysztof Parzyszek ecf0766cf1 [Hexagon] Add ELF flags for Hexagon v66 to ELFYAML.cpp 2020-01-16 15:01:00 -06:00
Georgii Rymar 7570d387c2 [yaml2obj/obj2yaml] - Add support for SHT_RELR sections.
Note: this is a reland with a trivial 2 lines fix in ELFState<ELFT>::writeSectionContent.
      It adds a check similar to ones we already have for other sections to fix the case revealed
      by bots, like http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/60744.

The encoded sequence of Elf*_Relr entries in a SHT_RELR section looks
like [ AAAAAAAA BBBBBBB1 BBBBBBB1 ... AAAAAAAA BBBBBB1 ... ]
i.e. start with an address, followed by any number of bitmaps. The address
entry encodes 1 relocation. The subsequent bitmap entries encode up to 63(31)
relocations each, at subsequent offsets following the last address entry.

More information is here:
https://github.com/llvm-mirror/llvm/blob/master/lib/Object/ELF.cpp#L272

This patch adds a support for these sections.

Differential revision: https://reviews.llvm.org/D71872
2020-01-15 15:15:24 +03:00
Georgii Rymar ca6f616532 Revert "[yaml2obj/obj2yaml] - Add support for SHT_RELR sections."
This reverts commit 46d11e30ee.

It broke bots. E.g. http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/60744
2020-01-15 14:19:00 +03:00
Georgii Rymar 46d11e30ee [yaml2obj/obj2yaml] - Add support for SHT_RELR sections.
The encoded sequence of Elf*_Relr entries in a SHT_RELR section looks
like [ AAAAAAAA BBBBBBB1 BBBBBBB1 ... AAAAAAAA BBBBBB1 ... ]
i.e. start with an address, followed by any number of bitmaps. The address
entry encodes 1 relocation. The subsequent bitmap entries encode up to 63(31)
relocations each, at subsequent offsets following the last address entry.

More information is here:
https://github.com/llvm-mirror/llvm/blob/master/lib/Object/ELF.cpp#L272

This patch adds a support for these sections.

Differential revision: https://reviews.llvm.org/D71872
2020-01-15 13:54:08 +03:00
Mark de Wever 8dc7b982b4 [NFC] Fixes -Wrange-loop-analysis warnings
This avoids new warnings due to D68912 adds -Wrange-loop-analysis to -Wall.

Differential Revision: https://reviews.llvm.org/D71857
2020-01-01 20:01:37 +01:00
Georgii Rymar f027e1a68d [yaml2obj] - Allow using an arbitrary value for OSABI.
There was no way to set an unsupported or unknown OS ABI.
With this patch it is possible to use any numeric value.

Differential revision: https://reviews.llvm.org/D71765
2019-12-23 13:29:52 +03:00
Georgii Rymar 1f98577556 [yaml2obj] - Add support for ELFOSABI_LINUX.
ELFOSABI_LINUX is an alias for ELFOSABI_GNU.
It is not that obvious probably.

Differential revision: https://reviews.llvm.org/D71764
2019-12-23 13:25:58 +03:00
Georgii Rymar 86e652f828 [yaml2obj] - Add a way to override sh_flags section field.
Currently we have the `Flags` property that allows to
set flags for a section. The problem is that it does not
allow us to set an arbitrary value, because of bit fields
validation under the hood. An arbitrary values can be used
to test specific broken cases.

We probably do not want to relax the validation, so this
patch adds a `ShSize` property that allows to
override the `sh_size`. It is inline with others `Sh*` properties
we have already.

Differential revision: https://reviews.llvm.org/D71411
2019-12-13 11:54:37 +03:00
Peter Smith 2120612e46 [ELF] Support for PT_GNU_PROPERTY in header and tools
The PT_GNU_PROPERTY is generated by a linker to describe the
.note.gnu.property section. The Linux kernel uses this program header to
locate the .note.gnu.property section.

It is described in "The Linux gABI extension"

Include support for llvm-readelf, llvm-readobj and the yaml reader and
writers.

Differential Revision: https://reviews.llvm.org/D70959
2019-12-04 15:38:12 +00:00
Georgii Rymar daff7b8589 [yaml2obj] - Make DynamicSymbols to be Optional<> too.
We already have Symbols property to list regular symbols and
it is currently Optional<>. This patch makes DynamicSymbols to be optional
too. With this there is no need to define a dummy symbol anymore to trigger
creation of the .dynsym and it is now possible to define an empty .dynsym using
just the following line:

DynamicSymbols: []
(it is important to have when you do not want to have dynamic symbols,
but want to have a .dynsym)

Now the code is consistent and it helped to fix a bug: previously we
did not report an error when both Content/Size and an empty
Symbols/DynamicSymbols list were specified.

Differential revision: https://reviews.llvm.org/D70956
2019-12-04 13:12:02 +03:00
Georgii Rymar 13cbcf1c1a [yaml2obj] - Add a way to describe content of the SHT_GNU_verneed section with "Content".
There is no way to set raw content for SHT_GNU_verneed section.
This patch implements it.

Differential revision: https://reviews.llvm.org/D70816
2019-11-29 10:50:00 +03:00
Georgii Rymar f69ac55d60 [yaml2obj] - Teach tool to describe SHT_GNU_verdef section with a "Content" property.
There is no way to set raw content for SHT_GNU_verdef section.
This patch implements it.

Differential revision: https://reviews.llvm.org/D70710
2019-11-26 15:35:05 +03:00
Georgii Rymar 9659464d7e [yaml2obj/obj2yaml] - Add support for SHT_LLVM_DEPENDENT_LIBRARIES sections.
This section contains strings specifying libraries to be added to the link by the linker.
The strings are encoded as standard null-terminated UTF-8 strings.

This patch adds a way to describe and dump SHT_LLVM_DEPENDENT_LIBRARIES sections.

I introduced a new YAMLFlowString type here. That used to teach obj2yaml to dump
them like:

```
Libraries: [ foo, bar ]
```

instead of the following (if StringRef would be used):

```
Libraries:
  - foo
  - bar
```

Differential revision: https://reviews.llvm.org/D70598
2019-11-25 12:57:53 +03:00
Tom Stellard ab411801b8 [cmake] Explicitly mark libraries defined in lib/ as "Component Libraries"
Summary:
Most libraries are defined in the lib/ directory but there are also a
few libraries defined in tools/ e.g. libLLVM, libLTO.  I'm defining
"Component Libraries" as libraries defined in lib/ that may be included in
libLLVM.so.  Explicitly marking the libraries in lib/ as component
libraries allows us to remove some fragile checks that attempt to
differentiate between lib/ libraries and tools/ libraires:

1. In tools/llvm-shlib, because
llvm_map_components_to_libnames(LIB_NAMES "all") returned a list of
all libraries defined in the whole project, there was custom code
needed to filter out libraries defined in tools/, none of which should
be included in libLLVM.so.  This code assumed that any library
defined as static was from lib/ and everything else should be
excluded.

With this change, llvm_map_components_to_libnames(LIB_NAMES, "all")
only returns libraries that have been added to the LLVM_COMPONENT_LIBS
global cmake property, so this custom filtering logic can be removed.
Doing this also fixes the build with BUILD_SHARED_LIBS=ON
and LLVM_BUILD_LLVM_DYLIB=ON.

2. There was some code in llvm_add_library that assumed that
libraries defined in lib/ would not have LLVM_LINK_COMPONENTS or
ARG_LINK_COMPONENTS set.  This is only true because libraries
defined lib lib/ use LLVMBuild.txt and don't set these values.
This code has been fixed now to check if the library has been
explicitly marked as a component library, which should now make it
easier to remove LLVMBuild at some point in the future.

I have tested this patch on Windows, MacOS and Linux with release builds
and the following combinations of CMake options:

- "" (No options)
- -DLLVM_BUILD_LLVM_DYLIB=ON
- -DLLVM_LINK_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_BUILD_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_LINK_LLVM_DYLIB=ON

Reviewers: beanz, smeenai, compnerd, phosek

Reviewed By: beanz

Subscribers: wuzish, jholewinski, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, mgorny, mehdi_amini, sbc100, jgravelle-google, hiraditya, aheejin, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, steven_wu, rogfer01, MartinMosbeck, brucehoult, the_o, dexonsmith, PkmX, jocewei, jsji, dang, Jim, lenary, s.egerton, pzheng, sameer.abuasal, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70179
2019-11-21 10:48:08 -08:00
Pavel Labath a03435ec8e Recommit "[DWARF] Add an api to get "interpreted" location lists"
This recommits 089c0f5814, which was
reverted due to failing tests on big endian machines. It includes a fix
which I believe (I don't have BE machine) should fix this issue. The fix
consists of correcting the invocation DWARFYAML::EmitDebugSections,
which was missing one (default) function arguments, and so didn't
actually force the little-endian mode.

The original commit message follows.

Summary:
This patch adds DWARFDie::getLocations, which returns the location
expressions for a given attribute (typically DW_AT_location). It handles
both "inline" locations and references to the external location list
sections (currently only of the DW_FORM_sec_offset type). It is
implemented on top of DWARFUnit::findLoclistFromOffset, which is also
added in this patch. I tried to make their signatures similar to the
equivalent range list functionality.

The actual location list interpretation logic is in
DWARFLocationTable::visitAbsoluteLocationList. This part is not
equivalent to the range list code, but this deviation is motivated by a
desire to reuse the same location list parsing code within lldb.

The functionality is tested via a c++ unit test of the DWARFDie API.

Reviewers: dblaikie, JDevlieghere, SouraVX

Subscribers: mgorny, hiraditya, cmtice, probinson, llvm-commits, aprantl

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70394
2019-11-20 16:24:11 +01:00
Pavel Labath 72d2929c52 Revert "[DWARF] Add an api to get "interpreted" location lists"
The test fails on big endian machines.

This reverts commit 089c0f5814 and the
subsequent attempt to fix in 82dc32e2d4.
2019-11-20 15:15:22 +01:00
Pavel Labath 089c0f5814 [DWARF] Add an api to get "interpreted" location lists
Summary:
This patch adds DWARFDie::getLocations, which returns the location
expressions for a given attribute (typically DW_AT_location). It handles
both "inline" locations and references to the external location list
sections (currently only of the DW_FORM_sec_offset type). It is
implemented on top of DWARFUnit::findLoclistFromOffset, which is also
added in this patch. I tried to make their signatures similar to the
equivalent range list functionality.

The actual location list interpretation logic is in
DWARFLocationTable::visitAbsoluteLocationList. This part is not
equivalent to the range list code, but this deviation is motivated by a
desire to reuse the same location list parsing code within lldb.

The functionality is tested via a c++ unit test of the DWARFDie API.

Reviewers: dblaikie, JDevlieghere, SouraVX

Subscribers: mgorny, hiraditya, cmtice, probinson, llvm-commits, aprantl

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70394
2019-11-20 13:25:18 +01:00
Sergey Dmitriev 4d02263af0 [yaml2obj][COFF] Add support for extended relocation tables
Summary:
The tool does not correctly handle COFF sections with extended relocation tables (with IMAGE_SCN_LNK_NRELOC_OVFL bit set), this patch fixes this problem.

But I have cheated a bit in the test (to make it smaller) because extended relocation table is supposed to be used when the number of relocations exceeds 65534. Otherwise the test size would be pretty big.

Reviewers: jhenderson, MaskRay, mstorsjo

Reviewed By: mstorsjo

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70251
2019-11-14 12:39:28 -08:00
Georgii Rymar dd101539da [yaml2obj/obj2yaml] - Add support for SHT_LLVM_LINKER_OPTIONS sections.
SHT_LLVM_LINKER_OPTIONS section contains pairs of null-terminated strings.
This patch adds support for them.

Differential revision: https://reviews.llvm.org/D69895
2019-11-12 09:55:20 +03:00
Simon Pilgrim 8383be0f75 Remove superfluous ';' to fix Wpedantic. NFC. 2019-11-11 10:54:00 +00:00
Georgii Rymar 6b15c5dfac [FixBB] - Fix one more std::min -> std::min<uint64_t> to make BB happy.
BB: http://lab.llvm.org:8011/builders/clang-armv7-linux-build-cache/builds/22133/steps/build%20stage%201/logs/stdio
2019-11-11 12:11:54 +03:00
Georgii Rymar a26d7b6298 [FixBB] - An attemp to fix clang-armv7-linux-build-cache builder.
http://lab.llvm.org:8011/builders/clang-armv7-linux-build-cache/builds/22130/steps/build%20stage%201/logs/stdio

/usr/bin/c++   -DGTEST_HAS_RTTI=0 -D_DEBUG -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/ObjectYAML -I/home/buildslave/buildslave/clang-armv7-linux-build-cache/llvm/llvm/lib/ObjectYAML -I/usr/include/libxml2 -Iinclude -I/home/buildslave/buildslave/clang-armv7-linux-build-cache/llvm/llvm/include -mthumb -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3    -UNDEBUG  -fno-exceptions -fno-rtti -std=c++14 -MMD -MT lib/ObjectYAML/CMakeFiles/LLVMObjectYAML.dir/YAML.cpp.o -MF lib/ObjectYAML/CMakeFiles/LLVMObjectYAML.dir/YAML.cpp.o.d -o lib/ObjectYAML/CMakeFiles/LLVMObjectYAML.dir/YAML.cpp.o -c /home/buildslave/buildslave/clang-armv7-linux-build-cache/llvm/llvm/lib/ObjectYAML/YAML.cpp
/home/buildslave/buildslave/clang-armv7-linux-build-cache/llvm/llvm/lib/ObjectYAML/YAML.cpp:42:41: error: no matching function for call to 'min'
    OS.write((const char *)Data.data(), std::min(N, Data.size()));
                                        ^~~~~~~~
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/algorithmfwd.h:370:5: note: candidate template ignored: deduced conflicting types for parameter '_Tp' ('unsigned long long' vs. 'unsigned int')
    min(const _Tp&, const _Tp&);
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/stl_algo.h:3451:5: note: candidate template ignored: could not match 'initializer_list<type-parameter-0-0>' against 'unsigned long long'
    min(initializer_list<_Tp> __l, _Compare __comp)
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/algorithmfwd.h:375:5: note: candidate function template not viable: requires 3 arguments, but 2 were provided
    min(const _Tp&, const _Tp&, _Compare);
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/stl_algo.h:3445:5: note: candidate function template not viable: requires single argument '__l', but 2 arguments were provided
    min(initializer_list<_Tp> __l)
    ^
/home/buildslave/buildslave/clang-armv7-linux-build-cache/llvm/llvm/lib/ObjectYAML/YAML.cpp:46:28: error: no matching function for call to 'min'
  for (uint64_t I = 0, E = std::min(N, Data.size() / 2); I != E; ++I) {
                           ^~~~~~~~
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/algorithmfwd.h:370:5: note: candidate template ignored: deduced conflicting types for parameter '_Tp' ('unsigned long long' vs. 'unsigned int')
    min(const _Tp&, const _Tp&);
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/stl_algo.h:3451:5: note: candidate template ignored: could not match 'initializer_list<type-parameter-0-0>' against 'unsigned long long'
    min(initializer_list<_Tp> __l, _Compare __comp)
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/algorithmfwd.h:375:5: note: candidate function template not viable: requires 3 arguments, but 2 were provided
    min(const _Tp&, const _Tp&, _Compare);
    ^
/usr/bin/../lib/gcc/arm-linux-gnueabihf/5.4.0/../../../../include/c++/5.4.0/bits/stl_algo.h:3445:5: note: candidate function template not viable: requires single argument '__l', but 2 arguments were provided
    min(initializer_list<_Tp> __l)

Fix: specify the type for std::min call.
2019-11-11 12:02:29 +03:00
Georgii Rymar 06456daa9e [yaml2obj] - Add a way to describe the custom data that is not part of an output section.
Currently there is no way to describe the data that is not a part of an output section.
It can be a data used to align sections or to fill the gaps with something,
or another kind of custom data. In this patch I suggest a way to describe it. It looks like that:

```
Sections:
  - Type:    CustomFiller
    Pattern: "CCDD"
    Size:    4
  - Name:    .bar
    Type:    SHT_PROGBITS
    Content: "FF"
```

I.e. I've added a kind of synthetic section with a synthetic type "CustomFiller".
In the code it is called a "SyntheticFiller", which is "a synthetic section which
might be used to write the custom data around regular output sections. It does
not present in the sections header table, but it might affect the output file size and
program headers produced. Think about it as about piece of data."

`SyntheticFiller` currently has a `Pattern` field and a `Size` field + an optional `Name`.
When written, `Size` of bytes in the output will be filled with a `Pattern`.
It is possible to reference a named filler it by name from the program headers description,
just like any other normal section.

Differential revision: https://reviews.llvm.org/D69709
2019-11-11 11:48:23 +03:00
Steven Wu 2293b3f169 Fix a typo in my previous commit 2019-11-06 11:42:30 -08:00
Steven Wu 6da58e7e0f [Object][MachO] Rewrite macho-invalid-fat-arch-size into YAML
Summary:
Rewrite one of the invalid macho test input file with YAML file. The
original invalid macho is breaking our internal test infrastusture
because it is too broken to be copy around.

Need to relax an assertion in the YAML/MachoEmitter to allow yaml2obj to
write an invalid object like this.

rdar://problem/56879982

Reviewers: beanz, mtrent

Reviewed By: beanz

Subscribers: hiraditya, jkorous, dexonsmith, ributzka, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69856
2019-11-06 11:26:25 -08:00
Pavel Labath e9430231ac minidump: Add an "arm64" constant
This is the "official" constant for arm64. We also have another constant
for arm64 (called BP_ARM64), which was used by breakpad while there was
no official constant for arm64 available.
2019-10-31 11:20:46 +01:00
georgerim a7aee6c47a [yaml2obj/obj2yaml] - Add support for SHT_GNU_HASH section.
This adds parsing and dumping support for GNU hash sections.
They are described nicely here: https://blogs.oracle.com/solaris/gnu-hash-elf-sections-v2

Differential revision: https://reviews.llvm.org/D69399
2019-10-31 12:25:37 +03:00
Pavel Labath 83a55c6a57 minidump: Rename some architecture constants
The architecture enum contains two kinds of contstants: the "official" ones
defined by Microsoft, and unofficial constants added by breakpad to cover the
architectures not described by the first ones.

Up until now, there was no big need to differentiate between the two. However,
now that Microsoft has defined
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-system_info
a constant for ARM64, we have a name clash.

This patch renames all breakpad-defined constants with to include the prefix
"BP_". This frees up the name "ARM64", which I'll re-introduce with the new
"official" value in a follow-up patch.

Reviewers: amccarth, clayborg

Subscribers: lldb-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D69285
2019-10-30 14:46:00 +01:00
Georgii Rymar 3fe7f1dcf4 [yaml2obj] - Make .symtab to be not mandatory section for SHT_REL[A] section.
Before this change .symtab section was required for SHT_REL[A] section
declarations. yaml2obj automatically defined it in case when YAML document
did not have it.

With this change it is now possible to produce an object that
has a relocation section, but has no symbol table.

It simplifies the code and also it is inline with how we handle Link fields
for another special sections.

Differential revision: https://reviews.llvm.org/D69260
2019-10-29 11:43:12 +03:00
Georgii Rymar 5b118a0471 [yaml2obj] - Improve handling of the SHT_GROUP section.
Currently, when we do not specify "Info" field in a YAML description
for SHT_GROUP section, yaml2obj reports an error:
"error: unknown symbol referenced: '' by YAML section '.group1'"

Also, we do not link it with a symbol table by default,
though it is what we do for AddrsigSection, HashSection, RelocationSection.
(http://www.sco.com/developers/gabi/latest/ch4.sheader.html#sh_link)

The patch fixes missings mentioned.

Differential revision: https://reviews.llvm.org/D69299
2019-10-29 11:09:12 +03:00
Georgii Rymar 073ab70b72 [ObjectYAML] - Do not use auto. NFC.
Using 'auto' when the type is not obvious is undesired.

(it is just a test commit actually)
2019-10-26 15:08:49 +03:00
georgerim de3cef1d5d [yaml2obj, obj2yaml] - Add support for SHT_NOTE sections.
SHT_NOTE is the section that consists of
namesz, descsz, type, name + padding, desc + padding data.
This patch teaches yaml2obj, obj2yaml to dump and parse them.

This patch implements the section how it is described here:
https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter6-18048.html
Which says: "For 64–bit objects and 32–bit objects, each entry is an array of 4-byte words in
the format of the target processor"

The official specification is different
http://www.sco.com/developers/gabi/latest/ch5.pheader.html#note_section
And says: "n 64-bit objects (files with e_ident[EI_CLASS] equal to ELFCLASS64), each entry is an array
of 8-byte words in the format of the target processor. In 32-bit objects (files with e_ident[EI_CLASS]
equal to ELFCLASS32), each entry is an array of 4-byte words in the format of the target processor"

Since LLVM uses the first, 32-bit way, this patch follows it.

Differential revision: https://reviews.llvm.org/D68983
2019-10-25 13:25:56 +03:00
georgerim 64df708400 [lib/ObjectYAML] - Add a full stop to the comment. NFC.
A test commit.
2019-10-23 12:35:43 +03:00
George Rimar 2779987d0e [yaml2obj][obj2yaml] - Do not create a symbol table by default.
This patch tries to resolve problems faced in D68943
and uses some of the code written by Konrad Wilhelm Kleine
in that patch.

Previously, yaml2obj tool always created a .symtab section.
This patch changes that. With it we only create it when
have a "Symbols:" tag in the YAML document or when
we need to create it because it is used by another section(s).

obj2yaml follows the new behavior and does not print "Symbols:"
anymore when there is no symbol table.

Differential revision: https://reviews.llvm.org/D69041

llvm-svn: 375361
2019-10-20 14:47:17 +00:00
Thomas Lively 393d0f799f [WebAssembly] Allow multivalue signatures in object files
Summary:
Also changes the wasm YAML format to reflect the possibility of having
multiple return types and to put the returns after the params for
consistency with the binary encoding.

Reviewers: aheejin, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, arphaman, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69156

llvm-svn: 375283
2019-10-18 20:27:30 +00:00
Joseph Tremoulet a50272f826 Update MinidumpYAML to use minidump::Exception for exception stream
Reviewers: labath, jhenderson, clayborg, MaskRay, grimar

Reviewed By: grimar

Subscribers: lldb-commits, grimar, MaskRay, hiraditya, llvm-commits

Tags: #llvm, #lldb

Differential Revision: https://reviews.llvm.org/D68657

llvm-svn: 375242
2019-10-18 14:56:19 +00:00
Pavel Labath 3aa7e76677 MinidumpYAML: Add support for the memory info list stream
Summary:
The implementation is fairly straight-forward and uses the same patterns
as the existing streams. The yaml form does not attempt to preserve the
data in the "gaps" that can be created by setting a larger-than-required
header or entry size in the stream header, because the existing consumer
(lldb) does not make use of the information in the gap in any way, and
attempting to preserve that would make the implementation more
complicated.

Reviewers: amccarth, jhenderson, clayborg

Subscribers: llvm-commits, lldb-commits, markmentovai, zturner, JosephTremoulet

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68645

llvm-svn: 374337
2019-10-10 13:05:46 +00:00
George Rimar c18585e32e [yaml2obj] - Add a Size tag support for SHT_LLVM_ADDRSIG sections.
It allows using "Size" with or without "Content" in YAML descriptions of
SHT_LLVM_ADDRSIG sections.

Differential revision: https://reviews.llvm.org/D68334

llvm-svn: 373610
2019-10-03 15:02:18 +00:00
George Rimar fc9104d42a Recommit r373598 "[yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections."
Fix: call `consumeError()` for a case missed.

Original commit message:

SHT_LLVM_ADDRSIG is described here:
https://llvm.org/docs/Extensions.html#sht-llvm-addrsig-section-address-significance-table

This patch teaches tools to dump them and to parse the YAML declarations of such sections.

Differential revision: https://reviews.llvm.org/D68333

llvm-svn: 373606
2019-10-03 14:52:33 +00:00
George Rimar 9f6cf2a081 Revert r373598 "[yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections."
It broke BB:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/18655/steps/test/logs/stdio

llvm-svn: 373599
2019-10-03 14:04:47 +00:00
George Rimar 32cbabfecb [yaml2obj/obj2yaml] - Add support for SHT_LLVM_ADDRSIG sections.
SHT_LLVM_ADDRSIG is described here:
https://llvm.org/docs/Extensions.html#sht-llvm-addrsig-section-address-significance-table

This patch teaches tools to dump them and to parse the YAML declarations of such sections.

Differential revision: https://reviews.llvm.org/D68333

llvm-svn: 373598
2019-10-03 13:57:08 +00:00
George Rimar 6fa696fb08 [yaml2obj] - Alow Size tag for describing SHT_HASH sections.
This is a follow-up for D68085 which allows using "Size"
tag together with "Content" tag or alone.

Differential revision: https://reviews.llvm.org/D68216

llvm-svn: 373473
2019-10-02 13:52:37 +00:00
George Rimar 0210a1a530 [yaml2obj] - Allow specifying custom Link values for SHT_HASH section.
This allows setting any sh_link values for SHT_HASH sections.

Differential revision: https://reviews.llvm.org/D68214

llvm-svn: 373316
2019-10-01 09:54:40 +00:00
George Rimar e5163ebf8d [yaml2obj/obj2yaml] - Add support for SHT_HASH sections.
SHT_HASH specification is:
http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#hash

In short the format is the following: it has 2 uint32 fields
in its header: nbucket and nchain followed by (nbucket + nchain)
uint32 values.

This patch allows dumping and parsing such sections.

Differential revision: https://reviews.llvm.org/D68085

llvm-svn: 373315
2019-10-01 09:45:59 +00:00
George Rimar cfc2bccfd8 [yaml2elf] - Support describing .stack_sizes sections using unique suffixes.
Currently we can't use unique suffixes in section names to describe
stack sizes sections. E.g. '.stack_sizes [1]' will be treated as a regular section.
This happens because we recognize stack sizes section by name and
do not yet drop the suffix before the check.

The patch fixes it.

Differential revision: https://reviews.llvm.org/D68018

llvm-svn: 372853
2019-09-25 12:09:30 +00:00
George Rimar f302436a0a [yaml2obj] - Add a Size field for StackSizesSection.
It is a follow-up requested in the review comment
for D67757. Allows to use Content + Size or just Size
when describing .stack_sizes sections in YAML document

Differential revision: https://reviews.llvm.org/D67958

llvm-svn: 372845
2019-09-25 11:40:11 +00:00
George Rimar 1a219aa8df [yaml2obj/obj2yaml] - Add support for .stack_sizes sections.
.stack_sizes is a SHT_PROGBITS section that contains pairs of
<address (4/8 bytes), stack size (uleb128)>.

This patch teach tools to parse and dump it.

Differential revision: https://reviews.llvm.org/D67757

llvm-svn: 372762
2019-09-24 14:22:37 +00:00
George Rimar 4d69967f44 [yaml2obj/obj2yaml] - Do not trigger llvm_unreachable when dumping/parsing relocations and e_machine is unsupported.
Currently when e_machine is set to something that is not supported by YAML lib,
then tools fail with llvm_unreachable.

In this patch I allow them to handle relocations in this case.
It can be used to dump and create objects for broken or unsupported targets.

Differential revision: https://reviews.llvm.org/D67657

llvm-svn: 372377
2019-09-20 09:15:36 +00:00
Simon Pilgrim 6e1a9b0fdf WasmEmitter - Don't dereference a dyn_cast result. NFCI.
llvm-svn: 372165
2019-09-17 19:14:11 +00:00
George Rimar 82d83733dd [obj2yaml] - Support PPC64 relocation types.
We do not support them and fail with llvm_unreachable currently.
This is not the only target we do not support and also seems we are missing
the tests for those we have already. But I needed this one for another patch,
so posted it separatelly.

Relocation names are taken from llvm\include\llvm\BinaryFormat\ELFRelocs\PowerPC64.def

Differential revision: https://reviews.llvm.org/D67615

llvm-svn: 372109
2019-09-17 12:00:55 +00:00
George Rimar cfc0ba3852 [yaml2obj/obj2yaml] - Allow setting an arbitrary values for e_machine.
Currently we only allow using a known named constants
for `Machine` field in YAML documents.

This patch allows using any numbers (valid or "unknown")
and adds test cases for current and new functionality.

With this it is possible to write a test cases for really unknown
EM_* targets.

Differential revision: https://reviews.llvm.org/D67652

llvm-svn: 372108
2019-09-17 11:51:26 +00:00
George Rimar 8501102727 [yaml2obj/ObjectYAML] - Cleanup the error reporting API, add custom errors handlers.
This is a continuation of the YAML library error reporting
refactoring/improvement and the idea by itself was mentioned
in the following thread:
https://reviews.llvm.org/D67182?id=218714#inline-603404

This performs a cleanup of all object emitters in the library.
It allows using the custom one provided by the caller.

One of the nice things is that each tool can now print its tool name,
e.g: "yaml2obj: error: <text>"

Also, the code became a bit simpler.

Differential revision: https://reviews.llvm.org/D67445

llvm-svn: 371865
2019-09-13 16:00:16 +00:00
George Rimar 7da559f2f6 [lib/ObjectYAML] - Change interface to return `bool` instead of `int`. NFCI
It was suggested in comments for D67445 to split this part.

Differential revision: https://reviews.llvm.org/D67488

llvm-svn: 371828
2019-09-13 09:12:38 +00:00
Fangrui Song 1da4f47195 [yaml2obj] Set p_align to the maximum sh_addralign of contained sections
The address difference between two sections in a PT_LOAD is a constant.
Consider a hypothetical case (pagesize can be very small, say, 4).

```
.text     sh_addralign=4
.text.hot sh_addralign=16
```

If we set p_align to 4, the PT_LOAD will be loaded at an address which
is a multiple of 4. The address of .text.hot is guaranteed to be a
multiple of 4, but not necessarily a multiple of 16.

This patch deletes the constraint

  if (SHeader->sh_offset == PHeader.p_offset)

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D67260

llvm-svn: 371501
2019-09-10 09:16:34 +00:00
Fangrui Song c28f3e6e2c [yaml2obj] Simplify p_filesz/p_memsz computing
This fixes a bug as well. When "FileSize:" (p_filesz) is specified and
different from the actual value, the following code probably should not
use PHeader.p_filesz:

  if (SHeader->sh_offset == PHeader.p_offset + PHeader.p_filesz)
    PHeader.p_memsz += SHeader->sh_size;

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D67256

llvm-svn: 371420
2019-09-09 16:45:17 +00:00
George Rimar 3212ecfea8 [lib/ObjectYAML] - Improve and cleanup error reporting in ELFState<ELFT> class.
The aim of this patch is to refactor how we handle and report error.

I suggest to use the same approach we use in LLD: delayed error reporting.
For that I introduced 'HasError' flag which triggers when we report an error.
Now we do not exit instantly on any error. The benefits are:

1) There are no more 'exit(1)' calls in the library code.
2) Code was simplified significantly in a few places.
3) It is now possible to print multiple errors instead of only one.

Also, I changed the messages to be lower case and removed a full stop.

Differential revision: https://reviews.llvm.org/D67182

llvm-svn: 371380
2019-09-09 09:43:03 +00:00
Fangrui Song d20c41dd31 [yaml2obj] Rename SHOffset (e_shoff) field to SHOff. NFC
`struct Elf*_Shdr` has a field `sh_offset`, named `ShOffset` in
llvm::ELFYAML::Section. Rename SHOffset (e_shoff) to SHOff to prevent confusion.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D67254

llvm-svn: 371185
2019-09-06 09:23:17 +00:00
Alex Brachet 0b69c59656 [yaml2obj] Make e_phoff and e_phentsize 0 if there are no program headers
Summary: It says [[ http://www.sco.com/developers/gabi/latest/ch4.eheader.html | here ]] that if there are no program headers than e_phoff should be 0, but currently it is always set after the header. GNU's `readelf` (but not `llvm-readelf`) complains about this: `readelf: Warning: possibly corrupt ELF header - it has a non-zero program header offset, but no program headers`.

Reviewers: jhenderson, grimar, MaskRay, rupprecht

Reviewed By: jhenderson, grimar, MaskRay

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D67054

llvm-svn: 371162
2019-09-06 02:27:55 +00:00
Fangrui Song 9d2504b6d8 [llvm-readobj][yaml2obj] Support SHT_LLVM_SYMPART, SHT_LLVM_PART_EHDR and SHT_LLVM_PART_PHDR
See http://lists.llvm.org/pipermail/llvm-dev/2019-February/130583.html
and D60242 for the lld partition feature.

This patch:

* Teaches yaml2obj to parse the 3 section types.
* Teaches llvm-readobj/llvm-readelf to dump the 3 section types.

There is no test for SHT_LLVM_DEPENDENT_LIBRARIES in llvm-readobj. Add
it as well.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D67228

llvm-svn: 371157
2019-09-06 00:53:28 +00:00
Fangrui Song c3bc697974 [yaml2obj] Write the section header table after section contents
Linkers (ld.bfd/gold/lld) place the section header table at the very
end. This allows tools to strip it, which is optional in executable/shared objects.
In addition, if we add or section, the size of the section header table
will change. Placing the section header table in the end keeps section
offsets unchanged.

yaml2obj currently places the section header table immediately after the
program header. Follow what linkers do to make offset updating easier.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D67221

llvm-svn: 371074
2019-09-05 14:25:57 +00:00
George Rimar 33b1a0eb5e [lib/ObjectYAML] - Cleanup the private interface of ELFState<ELFT>. NFCI.
In D62809 I accidentally added "ELFState<ELFT> &State" as the
first parameter to two methods. There is no reason for having that.
I removed this argument and also moved finalizeStrings declaration to
remove an excessive 'private:' tag.

Differential revision: https://reviews.llvm.org/D67157

llvm-svn: 371033
2019-09-05 08:59:28 +00:00
George Rimar e7b4d20998 Recommit r371023 "[lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol."
Fix: added missing return "return 0;"

Original commit message:
This eliminates one of the error(1) call in this lib.
It is different from the others because happens on a fields mapping stage
and can be easily fixed.

Differential revision: https://reviews.llvm.org/D67150

llvm-svn: 371030
2019-09-05 08:52:26 +00:00
George Rimar faa2a08987 Revert r371023 "[lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol." (2)
Forgot to revert the cpp file.

llvm-svn: 371025
2019-09-05 08:39:44 +00:00
George Rimar 2c9c432256 [lib/ObjectYAML] - Stop calling error(1) when mapping the st_other field of a symbol.
This eliminates one of the error(1) call in this lib.
It is different from the others because happens on a fields mapping stage
and can be easily fixed.

Differential revision: https://reviews.llvm.org/D67150

llvm-svn: 371023
2019-09-05 08:28:43 +00:00
Fangrui Song 441d450115 [yaml2obj] Support PT_GNU_STACK and PT_GNU_RELRO
PT_GNU_STACK is used in an llvm-objcopy test.

I plan to use PT_GNU_RELRO in a patch to improve nested segment
processing in llvm-objcopy (PR42963).

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D67146

llvm-svn: 370857
2019-09-04 09:19:31 +00:00
George Rimar 86cc736df1 [yaml2obj] - Allow overriding sh_name fields of the sections.
This is in line with the previous changes which allowed to
override the sh_offset/sh_size and useful for writing test cases.

Differential revision: https://reviews.llvm.org/D66998

llvm-svn: 370633
2019-09-02 09:47:17 +00:00
George Rimar 4e71702cd4 [yaml2obj][obj2yaml] - Use a single "Other" field instead of "Other", "Visibility" and "StOther".
Currenly we can encode the 'st_other' field of symbol using 3 fields.
'Visibility' is used to encode STV_* values.
'Other' is used to encode everything except the visibility, but it can't handle arbitrary values.
'StOther' is used to encode arbitrary values when 'Visibility'/'Other' are not helpfull enough.

'st_other' field is used to encode symbol visibility and platform-dependent
flags and values. Problem to encode it is that it consists of Visibility part (STV_* values)
which are enumeration values and the Other part, which is different and inconsistent.

For MIPS the Other part contains flags for all STO_MIPS_* values except STO_MIPS_MIPS16.
(Like comment in ELFDumper says: "Someones in their infinite wisdom decided to make
STO_MIPS_MIPS16 flag overlapped with other ST_MIPS_xxx flags."...)

And for PPC64 the Other part might actually encode any value.

This patch implements custom logic for handling the st_other and removes
'Visibility' and 'StOther' fields.

Here is an example of a new YAML style this patch allows:

- Name:  foo
  Other: [ 0x4 ]
- Name:  bar
  Other: [ STV_PROTECTED, 4 ]
- Name:  zed
  Other: [ STV_PROTECTED, STO_MIPS_OPTIONAL, 0xf8 ]

Differential revision: https://reviews.llvm.org/D66886

llvm-svn: 370472
2019-08-30 13:39:22 +00:00
Dan Gohman da84b688f9 [WebAssembly] Make __attribute__((used)) not imply export.
Add an WASM_SYMBOL_NO_STRIP flag, so that __attribute__((used)) doesn't
need to imply exporting. When targeting Emscripten, have
WASM_SYMBOL_NO_STRIP imply exporting.

Differential Revision: https://reviews.llvm.org/D62542

llvm-svn: 370415
2019-08-29 22:40:00 +00:00
George Rimar de0bc44883 [yaml2obj] - Allow placing local symbols after globals.
This allows us to produce broken binaries with local
symbols placed after global in '.dynsym'/'.symtab'

Also, simplifies the code.

Differential revision: https://reviews.llvm.org/D66799

llvm-svn: 370331
2019-08-29 10:58:47 +00:00
Vlad Tsyrklevich b8a96f4bf5 Reland "[yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time."
This relands this commit, I mistakenly reverted the original change
thinking it was the cause of the observed MSan failures but it was not.

llvm-svn: 370206
2019-08-28 14:04:09 +00:00
Vlad Tsyrklevich aba62e9c00 Revert "[yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time."
This reverts commit r370032, it was causing check-llvm failures on
sanitizer-x86_64-linux-bootstrap-msan

llvm-svn: 370198
2019-08-28 13:15:08 +00:00
George Rimar 7a2e21d9f4 [yaml2obj] - Don't allow setting StOther and Other/Visibility at the same time.
This is a follow up discussed in the comments of D66583.

Currently, if for example, we have both StOther and Other set in YAML document for a symbol,
then yaml2obj reports an "unknown key 'Other'" error.
It happens because 'mapOptional()' is never called for 'Other/Visibility' in this case,
leaving those unhandled.

This message does not describe the reason of the error well. This patch fixes it.

Differential revision: https://reviews.llvm.org/D66642

llvm-svn: 370032
2019-08-27 09:58:39 +00:00
George Rimar 668b11b2c8 [yaml2obj] - Allow setting the symbol st_other field to any integer.
st_other field of a symbol usually contains its visibility.
Other bits are usually 0, though some targets, like
MIPS can set them using the named bit field values.

Problem is that there is no way to set an arbitrary value now,
though that might be useful for our test cases.

In this patch I introduced a way to set st_other to any numeric
value using the new StOther field.
I added a test and simplified the existent one to show the effect/benefit

Differential revision: https://reviews.llvm.org/D66583

llvm-svn: 369742
2019-08-23 09:31:07 +00:00
George Rimar 91208447d0 [yaml2obj] - Lookup relocation symbols in dynamic symbol when .dynsym referenced.
This fixes https://bugs.llvm.org/show_bug.cgi?id=40337.

Previously, it was always assumed that relocations referenced symbols in the static symbol table. 
Now, if the Link field references a section called ".dynsym" it will look up these symbols
in the dynamic symbol table.

This patch is heavily based on D59097 by James Henderson

Differential revision: https://reviews.llvm.org/D66532

llvm-svn: 369645
2019-08-22 12:39:56 +00:00
Pavel Labath 82275ec51d MinidumpYAML: move serialization code to MinidumpEmitter.cpp
Summary:
The code for serializing minidumps was living in MinidumpYAML.cpp
so that it would be accessible from unit tests. While this had its
advantages, it was also unfortunate because it broke symmetry with all
other yaml2obj serializers.

Fortunately, nowadays all of yaml2obj is a library, so we don't need to
do anything special. This patch improves the code consistency by moving
the serialization code to MinidumpEmitter.cpp to match the style used in
other backends. It also removes the writeAsBinary entry point in favor
of the more general convertYAML interface.

This patch is just massaging the code a bit. There shouldn't be any
functional change here.

Reviewers: jhenderson, abrachet

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66474

llvm-svn: 369517
2019-08-21 11:30:48 +00:00
Seiya Nuta 522377494b [yaml2obj/obj2yaml][MachO] Allow setting custom section data
Reviewers: alexshap, jhenderson, rupprecht

Reviewed By: alexshap, jhenderson

Subscribers: abrachet, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65799

llvm-svn: 369348
2019-08-20 08:49:07 +00:00
Jonas Devlieghere 0eaee545ee [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013
2019-08-15 15:54:37 +00:00
George Rimar d3963051c4 [yaml2obj/obj2yaml] - Add a basic support for extended section indexes.
In some cases a symbol might have section index == SHN_XINDEX.
This is an escape value indicating that the actual section header index
is too large to fit in the containing field.
Then the SHT_SYMTAB_SHNDX section is used. It contains the 32bit values
that stores section indexes.

ELF gABI says that there can be multiple SHT_SYMTAB_SHNDX sections,
i.e. for example one for .symtab and one for .dynsym
(1) https://groups.google.com/forum/#!topic/generic-abi/-XJAV5d8PRg
(2) DT_SYMTAB_SHNDX: http://www.sco.com/developers/gabi/latest/ch5.dynamic.html

In this patch I am only supporting a single SHT_SYMTAB_SHNDX associated
with a .symtab. This is a more or less common case which is used a few tests I saw in LLVM.

I decided not to create the SHT_SYMTAB_SHNDX section as "implicit",
but implement is like a kind of regular section for now.
i.e. tools do not recreate this section or its content, like they do for
symbol table sections, for example. That should allow to write all kind of
possible broken test cases for our needs and keep the output closer to requested.

Differential revision: https://reviews.llvm.org/D65446

llvm-svn: 368272
2019-08-08 09:49:05 +00:00
Alex Brachet c22d9666fc [yaml2obj] Move core yaml2obj code into lib and include for use in unit tests
Reviewers: jhenderson, rupprecht, MaskRay, grimar, labath

Reviewed By: rupprecht

Subscribers: gribozavr, mgrang, seiya, mgorny, sbc100, hiraditya, aheejin, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65255

llvm-svn: 368119
2019-08-07 02:44:49 +00:00
Dmitri Gribenko fc21bb661f Revert "[yaml2obj] Move core yaml2obj code into lib and include for use in unit tests"
This reverts commit r368021, it broke tests.

llvm-svn: 368035
2019-08-06 13:39:50 +00:00
Alex Brachet 3cfeaa4d2c [yaml2obj] Move core yaml2obj code into lib and include for use in unit tests
Reviewers: jhenderson, rupprecht, MaskRay, grimar, labath

Reviewed By: rupprecht

Subscribers: seiya, mgorny, sbc100, hiraditya, aheejin, jakehehrlich, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65255

llvm-svn: 368021
2019-08-06 12:15:18 +00:00
Nilanjana Basu da60fc813c Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367867
2019-08-05 14:16:58 +00:00
Nilanjana Basu b5e4d7de17 Revert "Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability"
This reverts commit a885afa9fa.

llvm-svn: 367861
2019-08-05 13:55:21 +00:00
Nilanjana Basu a885afa9fa Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367850
2019-08-05 13:11:51 +00:00
Eric Christopher 5fb56b1966 Temporarily Revert "Changing representation of cv_def_range directives in Codeview debug info assembly format for better readability"
This is breaking bots and the author asked me to revert.

This reverts commit 367704.

llvm-svn: 367707
2019-08-02 19:10:37 +00:00
Nilanjana Basu 1c67521591 Changing representation of cv_def_range directives in Codeview debug info assembly format for better readability
llvm-svn: 367704
2019-08-02 18:44:39 +00:00
George Rimar eb41f7f081 [yaml2obj] - Allow overriding the sh_size field.
There is no way to set broken sh_size field currently
for sections. It can be usefull for writing the
test cases. 

Differential revision: https://reviews.llvm.org/D64401

llvm-svn: 365766
2019-07-11 12:59:29 +00:00
Stanislav Mekhanoshin 22b2c3d651 [AMDGPU] gfx908 target
Differential Revision: https://reviews.llvm.org/D64429

llvm-svn: 365525
2019-07-09 18:10:06 +00:00
George Rimar 9df825f429 [yaml2obj] - Allow overriding sh_offset field from the YAML.
Some of our test cases are using objects which
has sections with a broken sh_offset field.

There was no way to set it from YAML until this patch.

Differential revision: https://reviews.llvm.org/D63879

llvm-svn: 364898
2019-07-02 10:20:12 +00:00
George Rimar 687d47c2b0 [yaml2obj] - Allow overriding e_shentsize, e_shoff, e_shnum and e_shstrndx fields in the YAML.
This allows setting different values for e_shentsize, e_shoff, e_shnum
and e_shstrndx fields and is useful for producing broken inputs for various
test cases.

Differential revision: https://reviews.llvm.org/D63771

llvm-svn: 364517
2019-06-27 11:08:42 +00:00
Nicolai Haehnle 08e8cb5760 AMDGPU/MC: Add .amdgpu_lds directive
Summary:
The directive defines a symbol as an group/local memory (LDS) symbol.
LDS symbols behave similar to common symbols for the purposes of ELF,
using the processor-specific SHN_AMDGPU_LDS as section index.

It is the linker and/or runtime loader's job to "instantiate" LDS symbols
and resolve relocations that reference them.

It is not possible to initialize LDS memory (not even zero-initialize
as for .bss).

We want to be able to link together objects -- starting with relocatable
objects, but possible expanding to shared objects in the future -- that
access LDS memory in a flexible way.

LDS memory is in an address space that is entirely separate from the
address space that contains the program image (code and normal data),
so having program segments for it doesn't really make sense.

Furthermore, we want to be able to compile multiple kernels in a
compilation unit which have disjoint use of LDS memory. In that case,
we may want to place LDS symbols differently for different kernels
to save memory (LDS memory is very limited and physically private to
each kernel invocation), so we can't simply place LDS symbols in a
.lds section.

Hence this solution where LDS symbols always stay undefined.

Change-Id: I08cbc37a7c0c32f53f7b6123aa0afc91dbc1748f

Reviewers: arsenm, rampitec, t-tye, b-sumner, jsjodin

Subscribers: kzhuravl, jvesely, wdng, yaxunl, dstuttard, tpr, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61493

llvm-svn: 364296
2019-06-25 11:51:35 +00:00
George Rimar b6e20937b3 [yaml2obj/obj2yaml] - Make RawContentSection::Info Optional<>
This allows to customize this field for "implicit" sections properly.

Differential revision: https://reviews.llvm.org/D63487

llvm-svn: 363777
2019-06-19 08:57:38 +00:00
George Rimar cfa1a62a4c [yaml2obj] - Allow setting cutom Flags for implicit sections.
With this patch we get ability to set any flags we want
for implicit sections defined in YAML.

Differential revision: https://reviews.llvm.org/D63136

llvm-svn: 363367
2019-06-14 11:01:14 +00:00
Stanislav Mekhanoshin c43e67bfff [AMDGPU] gfx1011/gfx1012 targets
Differential Revision: https://reviews.llvm.org/D63307

llvm-svn: 363344
2019-06-14 00:33:31 +00:00
George Rimar 1e41007aeb [yaml2obj/obj2yaml] - Make RawContentSection::Content and RawContentSection::Size optional
This is a follow-up for D62809.

Content and Size fields should be optional as was discussed in comments
of the D62809's thread. With that, we can describe a specific string table and
symbol table sections in a more correct way and also show appropriate errors.

The patch adds lots of test cases where the behavior is described in details.

Differential revision: https://reviews.llvm.org/D62957

llvm-svn: 362931
2019-06-10 12:43:18 +00:00
Fangrui Song f090e6f7b6 [llvm-objdump/llvm-readobj/obj2yaml/yaml2obj] Support DT_PPC_GOT and DT_PPC_OPT
In glibc, DT_PPC_GOT indicates that PowerPC32 Secure PLT ABI is used.
I plan to use it in D62464.

DT_PPC_OPT currently indicates if a TLSDESC inspired TLS optimization is
enabled.

Reviewed By: grimar, jhenderson, rupprecht

Differential Revision: https://reviews.llvm.org/D62851

llvm-svn: 362569
2019-06-05 01:36:48 +00:00