Currently, DWARFLinker receives kind of accel tables as predefined sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Dwarf, ///< DWARF v5 .debug_names.
Default, ///< Dwarf for DWARF5 or later, Apple otherwise.
Pub, ///< .debug_pubnames, .debug_pubtypes
```
This patch removes implicit sets of tables(Default, Dwarf) and allows to ask for several sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Pub, ///< .debug_pubnames, .debug_pubtypes
DebugNames ///< .debug_names.
```
It allows seamlessness adding more accel tables in the future: .gdb_index, .debug_cu_index...
Doing things that way, DWARFLinker will be independent of consumers' requirements.
f.e. dsymutil and llvm-dwarfutil may have different variants for Default set
(so, instead of implementing these differencies inside DWARFLinker it could be
implemented in the corresponding module).
Differential Revision: https://reviews.llvm.org/D132371
Currently, DWARFLinker determines the target DWARF version internally.
It examines incoming object files, detects maximal
DWARF version and uses that version for the output file.
This patch allows explicitly setting output DWARF version by the consumer
of DWARFLinker. So that DWARFLinker uses a specified version instead
of autodetected one. It allows consumers to use different logic for
setting the target DWARF version. f.e. instead of the maximally used version
someone could set a higher version to convert from DWARFv4 to DWARFv5
(This possibility is not supported yet, but it would be good if
the interface will support it). Or another variant is to set the target
version through the command line. In this patch, the autodetection is moved
into the consumers(DwarfLinkerForBinary.cpp, DebugInfoLinker.cpp).
Differential Revision: https://reviews.llvm.org/D132755
Current implementation of registerModuleReference() function not only
"registers" module reference, but also clones referenced module
(inside loadClangModule()). That may lead to cloning the module with
incorrect options (registerModuleReference() examines module references
and additionally accumulates MaxDwarfVersion and accel tables info).
Since accumulated options may differ from the current values,
it is incorrect to clone modules before options are fully accumulated.
This patch separates "cloning" code from "registering" code. So,
that accumulating option is done in the "registering stage" and
"cloning" is done after all modules are registered and options accumulated.
It also adds a callback for loaded compile units which can be used for
D132755 and D132371(to allow doing options accumulation outside
of DWARFLinker).
Differential Revision: https://reviews.llvm.org/D133047
DWARFLinker::DIECloner::cloneAddressAttribute() contains call to
relocateIndexedAddr(StartOffset, EndOffset). StartOffset is
incorrectly calculated. Val.getRawUValue() is an index into the
.debug_addr table, so it should be multiplied
by Unit.getOrigUnit().getAddressByteSize().
Differential Revision: https://reviews.llvm.org/D132644
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
This patch is extracted from D86539.
Current implementation of lookForDIEsToKeep() function skips types
duplications basing on the getCanonicalDIEOffset() data:
```
if (AttrSpec.Form != dwarf::DW_FORM_ref_addr && (UseOdr || IsModuleRef) &&
Info.Ctxt &&
Info.Ctxt != ReferencedCU->getInfo(Info.ParentIdx).Ctxt &&
Info.Ctxt->getCanonicalDIEOffset() && isODRAttribute(AttrSpec.Attr)) <<<<<
continue;
```
But that field is set after all compile units inside object file are processed:
```
for (auto &CurrentUnit : OptContext.CompileUnits)
lookForDIEsToKeep(.., &CurrentUnit, ..); // check CanonicalDIEOffset
DIECloner.cloneAllCompileUnits(); // set CanonicalDIEOffset
```
Thus, if the object file contains several compilation units - types would
not be deduplicated. The above solution works well for the case when the object file
contains only one compilation unit. But if the object file contains several compilation
units then types would not be deduplicated between these compilation units.
This patch changes the algorithm so that types were deduplicated between
compilation units from the same object file.
It produces binary incompatible output for the cases when several compilation units
are located inside the same object file.
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D125469
this review is extracted from D86539.
1. Rename AccelTableKind to DwarfLinkerAccelTableKind
(to differentiate from AccelTableKind from CodeGen/AsmPrinter/DwarfDebug.h)
2. Add None value to the DwarfLinkerAccelTableKind.
3. added 'None' value for 'accelerator' option of dsymutil.
Differential Revision: https://reviews.llvm.org/D125474
Currently you can run the DWARF verifier on the linked dsymutil output.
This patch extends this functionality and makes it possible to
run the DWARF verifier on the input as well.
A new option --verify-dwarf allows you to specify input, output, all and
none. The existing --verify flag remains unchanged and acts and alias
for --verify-dwarf=output.
Input verification issues do not result in a non-zero exit code because
dsymutil is capable of taking invalid DWARF as input and producing valid
DWARF as output.
Differential revision: https://reviews.llvm.org/D89216
This makes a bunch of these call sites independent of a follow-up change
I'm making to have getAsCString return Expected<const char*> for more
descriptive error messages so that the failures there can be
communicated up to DWARFVerifier (or other callers who want to provide
more verbose diagnostics) so DWARFVerifier doesn't have to re-implement
the string lookup logic and error checking.
In debug info, we expect variables to have location info if they are used and we don't want location info for functions that are not used. However, if an unused function is inlined, we could have the scenario where a function is not in the final binary but its static variable is in the final binary. Ensure that variables in the final binary have location debug info even if their scope was inlined.
Also add `--implicit-check-not` to a test for clarity.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D115565
When determining the incompleteness of a DIE based on its children, make
sure we propagate it across union types. See test case for an example.
Without this patch we never emit the definition of Container_ivars.
Differential revision: https://reviews.llvm.org/D110443
list for attributes that don't have the loclist class.
Summary: The overflow error occurs when we try to dump
location list for those attributes that do not have the
loclist class, like DW_AT_count and DW_AT_byte_size.
After re-reviewed the entire list, I sorted those
attributes into two parts, one for dumping location list
and one for dumping the location expression.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D105613
Add a flag to change dsymutil's behavior and force a static variable to
keep its enclosing function. The test shows a situation where that could
be useful. I'm not convinced this behavior makes sense as a default,
which is why it's behind a flag.
rdar://74918374
Differential revision: https://reviews.llvm.org/D101337
Consider the .debug_pubnames and .debug_pubtypes their own kind of
accelerator and stop emitting them together with the Apple-style
accelerator tables. The only reason we were still emitting both was for
(byte-for-byte) compatibility with dsymutil-classic.
- This patch adds a new accelerator table kind "Pub" which can be
specified with --accelerator=Pub.
- This patch removes the ability to emit both pubnames/types and apple
style accelerator tables. I don't think anyone is relying on that but
it's worth pointing out.
- This patch removes the --minimize option and makes this behavior the
default. Specifying the flag will result in a warning but won't abort
the program.
Differential revision: https://reviews.llvm.org/D99907
dsymutil is not relocating the DW_AT_low_pc for a DW_TAG_label. This
patch fixes that and adds a test.
Differential revision: https://reviews.llvm.org/D99534
Split from D91844.
The local variable `Unit` in function `DWARFLinker::loadClangModule`
in file `llvm/lib/DWARFLinker/DWARFLinker.cpp`. If the variable is not set
in the loop below its definition, it will trigger a null pointer dereference
after the loop.
Patch By: OikawaKirie
Reviewed By: avl
Differential Revision: https://reviews.llvm.org/D97185
Current dsymutil implementation of hasLiveMemoryLocation()/hasLiveAddressRange()
and applyValidRelocs() assume that calls should be done in certain order
(from first Dies to last). Multi-thread implementation might call these methods
in other order(it might process compilation units in order other than they are physically
located), so we remove restriction that searching for relocations should be done
in ascending order. This change does not introduce noticable performance degradation.
The testing results for clang binary:
golden-dsymutil/dsymutil 23787992
clang MD5: 5efa8fd9355ebf81b65f24db5375caa2
elapsed time=91sec
build-Release/bin/dsymutil 23855616
clang MD5: 5efa8fd9355ebf81b65f24db5375caa2
elapsed time=91sec
Differential Revision: https://reviews.llvm.org/D93106
Currently dsymutil will silently fail when processing binaries with
Dwarf 5 debug info. This patch adds rudimentary support for Dwarf 5 in
dsymutil.
- Recognize relocations in the debug_addr section.
- Recognize (a subset of) Dwarf 5 form values.
- Emits valid Dwarf 5 compile unit header chains.
To simplify things (and avoid having to emit indexed sections) I decided
to emit the relocated addresses directly in the debug info section.
- DW_FORM_strx gets relocated and rewritten to DW_FORM_strp
- DW_FORM_addrx gets relocated and rewritten to DW_FORM_addr
Obviously there's a lot of work left, but this should be a step in the
right direction.
rdar://62345491
Differential revision: https://reviews.llvm.org/D94323
That refactoring is helpful since it reduces data inter-dependencies.
Which is good for current implementation and even more good for
fully multi-thread implementation. The idea of the refactoring
is to delete UniquingStringPool from the global DWARFLinker level.
It is used to unique type names while ODR deduplication is done.
Thus we move UniquingStringPool into the DeclContextTree which
matched to UniquingStringPool usage scope.
golden-dsymutil/dsymutil 23787992
clang MD5: 7d9873ff94f0246b6ab1ec3e8d0f3f06
build-Release/bin/dsymutil 23921272
clang MD5: 7d9873ff94f0246b6ab1ec3e8d0f3f06
Differential Revision: https://reviews.llvm.org/D93460
Current interface of AddressMap assumes that relocations exist.
That is correct for not-linked object file but is not correct
for linked executable. This patch changes interface in such way
that AddressMap could be used not only with not-linked object files:
hasValidRelocationAt()
replaced with:
hasLiveMemoryLocation()
hasLiveAddressRange()
Differential Revision: https://reviews.llvm.org/D87723
Convert analyzeContextInfo to a work list using the same approach I used
to remove the recursion from lookForDIEsToKeep. This fixes the crash
reported in https://llvm.org/PR48029.
Tested using the reproducer attached to PR48029 as well as by comparing
the clang MD5 hashes before and after the change (with and without
gmodules).
Differential revision: https://reviews.llvm.org/D90873
Eliminate the need to go through the DIE index by passing the DIE to
CompileUnit::getInfo directly.
Before:
unsigned Idx = Unit->getOrigUnit().getDIEIndex(Die);
CompileUnit::DIEInfo &Info = Unit->getInfo(Idx);
After:
CompileUnit::DIEInfo &Info = Unit->getInfo(Die);
Fangrui Song