Commit Graph

1079 Commits

Author SHA1 Message Date
Spyridoula Gravani 890eedc4e4 [DWARF] Introduce verification for the unit header chain in .debug_info section to llvm-dwarfdump.
This patch adds verification checks for the unit header chain in the .debug_info section.
Specifically, for each unit in the .debug_info section, the verifier checks that:

The unit length is valid (i.e. the unit can actually fit in the .debug_info section)
The dwarf version of the unit is valid
The address size is valid (4 or 8)
The unit type (if the unit is in dwarf5) is valid
The debug_abbrev_offset is valid

llvm-svn: 307975
2017-07-13 23:25:24 +00:00
Reid Kleckner 6597c28d76 [PDB] Fix type server handling for archives
Summary:
This fixes type indices for SDK or CRT static archives. Previously we'd
try to look next to the archive object file path, which would not exist
on the local machine.

Also error out if we can't resolve a type server record. Hypothetically
we can recover from this error by discarding debug info for this object,
but that is not yet implemented.

Reviewers: ruiu, amccarth

Subscribers: aprantl, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35369

llvm-svn: 307946
2017-07-13 20:12:23 +00:00
Wolfgang Pieb 515d0e5001 [DWARF] Fixing a bug with processing of DWARF v5 indexed strings in Mach-O objects.
Code to convert MachO - specific section debug section names to standard DWARF v5
section names was in the wrong place.

Differential Revision: https://reviews.llvm.org/D35321

llvm-svn: 307872
2017-07-13 01:03:28 +00:00
Rafael Espindola 5e5dfa1fc5 Don't expose a map in the DWARFContext interface.
Doing so is leaking an implementation detail.

I have an implementation that uses the lld infrastructure and doesn't
use a map or object::SectionRef.

llvm-svn: 307846
2017-07-12 21:08:24 +00:00
Reid Kleckner 0962cb2e3a Fix non-Windows build after PDB native builtin type change
Some C++14 features slipped in along with an extra member qualification.

llvm-svn: 307835
2017-07-12 19:46:35 +00:00
Adrian McCarthy 8d090fc531 [PDB] Enable NativeSession to create symbols for built-in types on demand
Summary:
There is a reserved range of type indexes for built-in types (like integers).
This will create a symbol for a built-in type if the caller askes for one by
type index.  This is also plumbing for being able to recall symbols by type
index in general, but user-defined types will come in subsequent patches.

Reviewers: rnk, zturner

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D35163

llvm-svn: 307834
2017-07-12 19:38:11 +00:00
Reid Kleckner 8d8888ff42 [codeview] Change readobj symbol dumping format
Avoid duplicating DictScope with hand-written names everywhere.  Print
the S_-prefixed symbol kind for every record. This should make it easier
to search for certain kinds of records when debugging PDB linking.

llvm-svn: 307732
2017-07-11 23:41:41 +00:00
Reid Kleckner 447f677133 [codeview] Fix type index discovery for four symbol records
I encountered these when linking LLD, which uses atls.lib. Those objects
appear to use these uncommon symbol records:

  0x115E S_HEAPALLOCSITE
  0x113D S_ENVBLOCK
  0x1113 S_GTHREAD32
  0x1153 S_FILESTATIC

llvm-svn: 307725
2017-07-11 22:37:25 +00:00
Zachary Turner 7eaf1d96ad [lld/pdb] Create an empty public symbol record stream.
This is part of the continuing effort to increase parity between
LLD and MSVC PDBs.  link still doesn't like our PDBs, so the most
obvious thing to check was whether adding an empty publics stream
would get it to do something else.  It still fails in the same way
but at least this removes one more variable from the equation.
The next logical step would be to try creating an empty globals
stream.

Differential Revision: https://reviews.llvm.org/D35224

llvm-svn: 307598
2017-07-10 22:40:20 +00:00
George Rimar c7147b368a [DWARF] - Rename variable. NFC.
Variable was called 'Name' and contained text
name of relocation type. Problem was that
outside of this error handling scope we already
have different 'Name' variable that contains
section name.

Change helps to avoid confusion. 

llvm-svn: 307530
2017-07-10 10:04:51 +00:00
George Rimar 4be1388ebb [DWARF] - Remove unused variables. NFC.
llvm-svn: 307528
2017-07-10 09:36:44 +00:00
Zachary Turner 3a11fdf8ce [PDB] More changes to bring lld PDBs to parity with MSVC.
1) Don't write a /src/headerblock stream.  This appears to be
   written conditionally by MSVC, but it's not clear what the
   condition is.  For now, just remove it since we dont' know
   what it is anyway and the particular pdb we've checked in
   for the test doesn't have one.
2) Write a valid timestamp for the PDB file signature.  This
   leads to non-reproducible builds, but it matches the default
   behavior of link, so it should be out default as well.  If
   we need reproducibility, we should add a separate command
   line option for it that is off by default.
3) Write an empty FPO stream.  MSVC seems to always write an
   FPO stream.  This change makes the stream directory match
   up, although we still need to make the contents of the FPO
   stream match.

llvm-svn: 307436
2017-07-07 20:25:39 +00:00
Zachary Turner c1e93e5fa4 Fix some differences between lld and MSVC generated PDBs.
A couple of things were different about our generated PDBs.

1) We were outputting the wrong Version on the PDB Stream.
   The version we were setting was newer than what MSVC is setting.
   It's not clear what the implications are, but we change LLD
   to use PdbImplVC70, as MSVC does.
2) For the optional debug stream indices in the DBI Stream, we
   were outputting 0 to mean "the stream is not present".  MSVC
   outputs uint16_t(-1), which is the "correct" way to specify
   that a stream is not present.  So we fix that as well.
3) We were setting the PDB Stream signature to 0.  This is supposed
   to be the result of calling time(nullptr).  Although this leads
   to non-deterministic builds, a better way to solve that is by
   having a command line option explicitly for generating a
   reproducible build, and have the default behavior of lld-link
   match the default behavior of link.

To test this, I'm making use of the new and improved `pdb diff`
sub command.  To make it suitable for writing tests against, I had
to modify the diff subcommand slightly to print less verbose output.
Previously it would always print | <column> | <value1> | <value2> |
which is quite verbose, and the values are fragile.  All we really
want to know is "did we produce the same value as link?"  So I added
command line options to print a single character representing the
result status (different, identical, equivalent), and another to
hide the value display.  Note that just inspecting the diff output
used to write the test, you can see some things that are obviously
wrong.  That is just reflective of the fact that this is the state
of affairs today, not that we're asserting that this is "correct".
We can use this as a starting point to discover differences, fix
them, and update the test.

Differential Revision: https://reviews.llvm.org/D35086

llvm-svn: 307422
2017-07-07 18:45:56 +00:00
Zachary Turner f3b4b2d89d [llvm-pdbutil] Improve diff mode.
We're getting to the point that some MS tools (e.g. DIA) can recognize
our PDBs but others (e.g. link.exe) cannot. I think the way forward is
to improve our tooling to help us find differences more easily. For
example, if we can compile the same program with clang-cl and cl and
have a tool tell us all the places where the PDBs differ, this could
tell us what we're doing wrong. It's tricky though, because there are a
lot of "benign" differences in a PDB. For example, if the string table
in one PDB consists of "foo" followed by "bar" and in the other PDB it
consists of "bar" followed by "foo", this is not necessarily a critical
difference, as long as the uses of these strings also refer to the
correct location. On the other hand, if the second PDB doesn't even
contain the string "foo" at all, this is a critical difference.

diff mode has been in llvm-pdbutil for quite a while, but because of the
above challenge along with some others, it's been hard to make it
useful. I think this patch addresses that. It looks for all the same
things, but it now prints the output in tabular format (carefully
formatted and aligned into tables and fields), and it highlights
critical differences in red, non-critical differences in yellow, and
identical fields in green.  This makes it easy to spot the places we
differ, and the general concept of outputting arbitrary fields in
tabular format can be extended to provide analysis into many of the
different types of information that show up in a PDB.

Differential Revision: https://reviews.llvm.org/D35039

llvm-svn: 307421
2017-07-07 18:45:37 +00:00
Rafael Espindola 7413496f8d Fix variable names. NFC.
llvm-svn: 307406
2017-07-07 15:20:55 +00:00
Rafael Espindola 77e87b3444 Reduce code duplication.
By addding a mapNameToDWARFSection we only need to check section names
in one place.

llvm-svn: 307359
2017-07-07 05:36:53 +00:00
Zachary Turner 6c4bfba8f3 [PDB] Teach libpdb to write DBI Stream ECNames.
Based strictly on the name, this seems to have something to do
width edit & continue.  The goal of this patch has nothing to do
with supporting edit and continue though.  msvc link.exe writes
very basic information into this area even when *not* compiling
with support for E&C, and so the goal here is to bring lld-link
to parity.  Since we cannot know what assumptions standard tools
make about the content of PDB files, we need to be as close as
possible.

This ECNames data structure is a standard PDB string hash table.
link.exe puts a single string into this hash table, which is the
full path to the PDB file on disk.  It then references this string
from the module descriptor for the compiler generated `* Linker *`
module.

With this patch, lld-link will generate the exact same sequence of
bytes as MSVC link for this subsection for a given object file
input (as reported by `llvm-pdbutil bytes -ec`).

llvm-svn: 307356
2017-07-07 05:04:36 +00:00
Hiroshi Inoue ddb34d84c9 fix trivial typos in comments; NFC
llvm-svn: 307004
2017-07-03 06:32:59 +00:00
Eugene Zelenko 4fcfc19976 [CodeView, PDB] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306911
2017-06-30 23:06:03 +00:00
Zachary Turner af8c75a8c0 [llvm-pdbutil] Output the symbol offset when dumping.
Type records have a unique type index, but symbol records do
not.  Instead, symbol records refer to other symbol records
by referencing their offset in the symbol stream.  In a sense
this is the analogue of the TypeIndex, but we are not printing
it in the dumper.  Printing it not only gives us more useful
information when manually investigating the contents of a PDB,
but also allows us to write better tests by enabling us to
verify that fields that reference other symbol records do
so correctly.

Differential Revision: https://reviews.llvm.org/D34906

llvm-svn: 306890
2017-06-30 21:35:00 +00:00
Zachary Turner 02a267758e [llvm-pdbutil] Add the ability to dump the dependency tree for a type
Previously we had the -type-index option which would dump the record of
a single, but we had no way to follow the dependency graph backwards and
also dump all dependent types.

Having this option makes test-writing better, because we can limit the
test to only those records that are of importance for the thing we're
trying to test, which allows us to use things like CHECK-NEXT to reduce
fragility.

Differential Revision: https://reviews.llvm.org/D34899

llvm-svn: 306852
2017-06-30 18:15:47 +00:00
Spyridoula Gravani 837c110cb1 [DWARF] Added verification checks for the .apple_names section.
This patch verifies the number of atoms, the validity of the form for each atom, as well as the validity of the
hashdata. For hashdata, we're verifying that the hashdata offset is correct and that the offset in the .debug_info for
each DIE in the hashdata is also valid.

llvm-svn: 306735
2017-06-29 20:13:05 +00:00
Paul Robinson 17536b935a [DWARF] NFC: DWARFDataExtractor combines relocs with DataExtractor.
Requires callers to directly associate relocations with a DataExtractor
used to read data from a DWARF section, which helps a callee not make
assumptions about which section it is reading.
This is the next step in reducing DWARFFormValue's dependence on DWARFUnit.

Differential Revision: https://reviews.llvm.org/D34704

llvm-svn: 306699
2017-06-29 16:52:08 +00:00
George Rimar b25a09d0f5 [DWARF] - Fix message reporting about broken relocation.
Because of mistake introduced in r306517,
wrong variable ("name" instead of "Name") was used
in error message.
As a result it reported section name instead of
relocation name.

This file still needs cleanup to match LLVM coding style
and more tests I think.

llvm-svn: 306677
2017-06-29 14:05:18 +00:00
Eugene Zelenko 8456b16ea9 [CodeView] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306616
2017-06-29 00:05:44 +00:00
Adrian McCarthy bf0afc3246 Introduce symbol cache to PDB NativeSession
Instead of creating symbols directly in the findChildren methods of the native
symbol implementations, they will rely on the NativeSession to act as a factory
for these types.  This lets NativeSession cache the NativeRawSymbols in its
new symbol cache and makes that cache the source of unique IDs for the symbols.

Right now, this affects only NativeCompilandSymbols.  There's no external
change yet, so I think the existing tests are still sufficient.  Coming soon
are patches to extend this to built-in types and enums.

llvm-svn: 306610
2017-06-28 22:47:40 +00:00
George Rimar 1af3cb2912 Recommit "[ELF] - Add ability for DWARFContextInMemory to exit early when any error happen."
With fix in include folder character case:
#include "llvm/Codegen/AsmPrinter.h" -> #include "llvm/CodeGen/AsmPrinter.h"

Original commit message:

Change introduces error reporting policy for DWARFContextInMemory.
New callback provided by client is able to handle error on it's
side and return Halt or Continue.

That allows to either keep current behavior when parser prints all errors
but continues parsing object or implement something very different, like
stop parsing on a first error and report an error in a client style.

Differential revision: https://reviews.llvm.org/D34328

llvm-svn: 306517
2017-06-28 08:21:19 +00:00
George Rimar 7a82cffd68 Revert r306512 "[ELF] - Add ability for DWARFContextInMemory to exit early when any error happen."
It broke BB:

[13/106] 13 0.022 Generating VCSRevision.h
[25/106] 24 1.209 Building CXX object unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o
FAILED: unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o 
/home/bb/bin/g++  -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_GLOBAL_ISEL -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iunittests/DebugInfo/DWARF -I../llvm-project/llvm/unittests/DebugInfo/DWARF -Iinclude -I../llvm-project/llvm/include -I../llvm-project/llvm/utils/unittest/googletest/include -I../llvm-project/llvm/utils/unittest/googlemock/include -fPIC -fvisibility-inlines-hidden -m32 -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3    -UNDEBUG  -Wno-variadic-macros -fno-exceptions -fno-rtti -MD -MT unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o -MF unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o.d -o unittests/DebugInfo/DWARF/CMakeFiles/DebugInfoDWARFTests.dir/DWARFDebugInfoTest.cpp.o -c ../llvm-project/llvm/unittests/DebugInfo/DWARF/DWARFDebugInfoTest.cpp
../llvm-project/llvm/unittests/DebugInfo/DWARF/DWARFDebugInfoTest.cpp:18:37: fatal error: llvm/Codegen/AsmPrinter.h: No such file or directory
 #include "llvm/Codegen/AsmPrinter.h"
                                     ^
compilation terminated.

llvm-svn: 306513
2017-06-28 07:06:17 +00:00
George Rimar 397a70425b [ELF] - Add ability for DWARFContextInMemory to exit early when any error happen.
Change introduces error reporting policy for DWARFContextInMemory.
New callback provided by client is able to handle error on it's
side and return Halt or Continue.

That allows to either keep current behavior when parser prints all errors
but continues parsing object or implement something very different, like
stop parsing on a first error and report an error in a client style.

Differential revision: https://reviews.llvm.org/D34328

llvm-svn: 306512
2017-06-28 06:57:20 +00:00
Paul Robinson d66ee0f9a7 [DWARF] NFC: Make string-offset handling more like address-table handling;
do the indirection and relocation all in the same method.

llvm-svn: 306418
2017-06-27 15:40:18 +00:00
Paul Robinson 36e85a867b [DWARF] NFC: Give DwarfFormat a 1-byte base type.
In particular this reduces DWARFFormParams from 64 to 32 bits; pass it
around by value.

llvm-svn: 306324
2017-06-26 19:52:32 +00:00
Paul Robinson 75c068c50b [DWARF] NFC: Collect info used by DWARFFormValue into a helper.
Some forms have sizes that depend on the DWARF version, DWARF format
(32/64-bit), or the size of an address.  Collect these into a struct
to simplify passing them around.  Require callers to provide one when
they query a form's size.

Differential Revision: http://reviews.llvm.org/D34570

llvm-svn: 306315
2017-06-26 18:43:01 +00:00
Zachary Turner 1affd805fc [pdb] Fix reading of llvm-generated PDBs by cvdump.
If you dump a pdb to yaml, and then round-trip it back to a pdb,
and run cvdump -l <file> on the new pdb, cvdump will generate
output such as this.

*** LINES

** Module: "d:\src\llvm\test\DebugInfo\PDB\Inputs\empty.obj"

Error: Line number corrupted: invalid file id 0
  <Unknown> (MD5), 0001:00000010-0000001A, line/addr pairs = 3

        5 00000010      6 00000013      7 00000018

Note the error message about the corrupted line number.

It turns out that the problem is that cvdump cannot find the
/names stream (e.g. the global string table), and the reason it
can't find the /names stream is because it doesn't understand
the NameMap that we serialize which tells pdb consumers which
stream has the string table.

Some experimentation shows that if we add items to the hash
table in a specific order before serializing it, cvdump can read
it. This suggests that either we're using the wrong hash function,
or we're serializing something incorrectly, but it will take some
deeper investigation to figure out how / why.  For now, this at
least allows cvdump to read our line information (and incidentally,
produces an identical byte sequence to what Microsoft tools
produce when writing the named stream map).

Differential Revision: https://reviews.llvm.org/D34491

llvm-svn: 306233
2017-06-25 03:51:42 +00:00
Zachary Turner fa33282774 [llvm-pdbutil] Dump raw bytes of module symbols and debug chunks.
llvm-svn: 306179
2017-06-23 23:08:57 +00:00
Eugene Zelenko 2db0cfa617 [DebugInfo] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 306169
2017-06-23 21:57:40 +00:00
Zachary Turner c2f5b4bfd9 [llvm-pdbutil] Dump raw bytes of type and id records.
llvm-svn: 306167
2017-06-23 21:50:54 +00:00
Zachary Turner dd73968256 [llvm-pdbutil] Dump raw bytes of various DBI stream subsections.
llvm-svn: 306160
2017-06-23 21:11:54 +00:00
Zachary Turner 6c3e41bbd3 [llvm-pdbutil] Dump raw bytes of pdb name map.
This patch dumps the raw bytes of the pdb name map which contains
the mapping of stream name to stream index for the string table
and other reserved streams.

llvm-svn: 306148
2017-06-23 20:18:38 +00:00
Zachary Turner 0b36c3ebd0 [llvm-pdbutil] Add a function for formatting MSF data.
The goal here is to make it possible to display absolute
file offsets when dumping byets from an MSF.  The problem is
that when dumping bytes from an MSF, often the bytes will
cross a block boundary and encounter a discontinuity.  We
can't use the normal formatBinary() function for this because
this would just treat the sequence as entirely ascending, and
not account out-of-order blocks.

This patch adds a formatMsfData() function to our printer, and
then uses this function to improve the output of the -stream-data
command line option for dumping bytes from a particular stream.

Test coverage is also expanded to make sure to include all possible
scenarios of offsets, sizes, and crossing block boundaries.

llvm-svn: 306141
2017-06-23 18:52:13 +00:00
Adrian McCarthy 4aedc81b8c Fix build break by using llvm::make_unique instead of std::make_unique.
llvm-svn: 306043
2017-06-22 18:57:51 +00:00
Adrian McCarthy 31bcb6f680 Add IDs and clone methods to NativeRawSymbol
All NativeRawSymbols will have a unique symbol ID (retrievable via
getSymIndexId).  For now, these are initialized to 0, but soon the
NativeSession will be responsible for creating the raw symbols, and it will
assign unique IDs.

The symbol cache in the NativeSession will also require the ability to clone
raw symbols, so I've provided implementations for that as well.

llvm-svn: 306042
2017-06-22 18:43:18 +00:00
Adrian McCarthy 6a4b080a5f Make IPDBSession::getGlobalScope a non-const method
There doesn't seem to be a compelling reason why this method should be const
other than it was possible with the DIA implementation.  The native session
is going to act as a symbol factory and cache.  This could be acheived with
mutable (and the existing const_cast), but it seems cleaner to accept that
this method affects the state of the session.

This change eliminates an existing const_cast.

llvm-svn: 306041
2017-06-22 18:42:23 +00:00
Wolfgang Pieb 258927e3da [DWARF] Support for DW_FORM_strx3 and complete support for DW_FORM_strx{1,2,4}
(consumer).

Reviewer: aprantl

Differential Revision:  https://reviews.llvm.org/D34418

llvm-svn: 305944
2017-06-21 19:37:44 +00:00
Reid Kleckner d0e6e24a53 [PDB] Add symbols to the PDB
Summary:
The main complexity in adding symbol records is that we need to
"relocate" all the type indices. Type indices do not have anything like
relocations, an opaque data structure describing where to find existing
type indices for fixups. The linker just has to "know" where the type
references are in the symbol records. I added an overload of
`discoverTypeIndices` that works on symbol records, and it seems to be
able to link the standard library.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34432

llvm-svn: 305933
2017-06-21 17:25:56 +00:00
Zachary Turner 297b6eb20d [PDB] Don't write uninitialized bytes to a PDB file.
There were certain fields that we didn't know how to write, as
well as various padding bytes that we would ignore.  This leads
to garbage data in the PDB.  While not strictly necessary, we
should initialize these bytes to something meaningful, as it
makes for easier binary comparison between PDBs.

llvm-svn: 305819
2017-06-20 18:50:55 +00:00
David Blaikie 6ab0eb4764 Remove convenient but probably not worthwhile macro for lambda workaround
Cleanup from r305405

llvm-svn: 305731
2017-06-19 19:01:08 +00:00
Reid Kleckner 44cdb10964 [PDB] Start emitting source file and line information
Summary:
This is a first step towards getting line info to show up in VS and
windbg. So far, only llvm-pdbutil can parse the PDBs that we produce.
cvdump doesn't like something about our file checksum tables. I'll have
to dig into that next.

This patch adds a new DebugSubsectionRecordBuilder which takes bytes
directly from some other producer, such as a linker, and sticks it into
the PDB. Line tables only need to be relocated. No data needs to be
rewritten.

File checksums and string tables, on the other hand, need to be re-done.

Reviewers: zturner, ruiu

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D34257

llvm-svn: 305713
2017-06-19 17:21:45 +00:00
Reid Kleckner 18d90e17ad [CodeView] Fix dumping of public symbol record flags
I noticed nonsensical type information while dumping PDBs produced by
MSVC.

llvm-svn: 305708
2017-06-19 16:54:51 +00:00
Zachary Turner 26dbc5420d Delete TypeDatabase.
Merge the functionality into the random access type collection.
This class was only being used in 2 places, so getting rid of it
simplifies the code.

llvm-svn: 305653
2017-06-18 20:52:45 +00:00
Zachary Turner b0fdd214b7 Don't crash if a type record can't be found.
This was a regression introduced in a previous patch.  Adding
back the code that handles this case.

llvm-svn: 305617
2017-06-17 00:02:24 +00:00
Zachary Turner ad859bd472 [CodeView] Fix random access of type names.
Suppose we had a type index offsets array with a boundary at type index
N. Then you request the name of the type with index N+1, and that name
requires the name of index N-1 (think a parameter list, for example). We
didn't handle this, and we would print something like (<unknown UDT>,
<unknown UDT>).

The fix for this is not entirely trivial, and speaks to a larger
problem. I think we need to kill TypeDatabase, or at the very least kill
TypeDatabaseVisitor. We need a thing that doesn't do any caching
whatsoever, just given a type index it can compute the type name "the
slow way". The reason for the bug is that we don't have anything like
that. Everything goes through the type database, and if we've visited a
record, then we're "done". It doesn't know how to do the expensive thing
of re-visiting dependent records if they've not yet been visited.

What I've done here is more or less copied the code (albeit greatly
simplified) from TypeDatabaseVisitor, but wrapped it in an interface
that just returns a std::string. The logic of caching the name is now in
LazyRandomTypeCollection. Eventually I'd like to move the record
database here as well and the visited record bitfield here as well, at
which point we can actually just delete TypeDatabase. I don't see any
reason for it if a "sequential" collection is just a special case of a
random access collection with an empty partial offsets array.

Differential Revision: https://reviews.llvm.org/D34297

llvm-svn: 305612
2017-06-16 23:42:44 +00:00
Zachary Turner 59224cba2e Remove some dead code / includes.
I'm trying to get rid of the TypeDatabase class, so the first
step is to minimize its footprint.

llvm-svn: 305611
2017-06-16 23:42:15 +00:00
Spyridoula Gravani 32614fcf42 [DWARF] Corrected behavior for when no .apple_names section is present in the object.
The verifier should not output any message in such a case.
Added test case with no .apple_name section in the file to verify new functionality.
Made existing test case more specific.

llvm-svn: 305597
2017-06-16 22:03:21 +00:00
Zachary Turner 4e950647fb [llvm-pdbutil] Add support for dumping lines and inlinee lines.
llvm-svn: 305529
2017-06-15 23:56:19 +00:00
Zachary Turner 0e327d0360 [llvm-pdbutil] Add back support for dumping file checksums.
When dumping module source files, also dump checksums.

llvm-svn: 305526
2017-06-15 23:12:41 +00:00
Zachary Turner f8a2e04812 [llvm-pdbutil] Add back the ability to dump hashes and index offsets.
This was regressed in a previous patch that re-wrote the dumper,
and I'm incrementally adding back the pieces that are missing.

llvm-svn: 305524
2017-06-15 23:04:42 +00:00
Zachary Turner 6305545527 Resubmit "[llvm-pdbutil] rewrite the "raw" output style."
This resubmits commit c0c249e9f2ef83e1d1e5f166b50673d92f3579d7.

It was broken due to some weird template issues, which have
since been fixed.

llvm-svn: 305517
2017-06-15 22:24:24 +00:00
Zachary Turner da504b794c Revert "[llvm-pdbutil] rewrite the "raw" output style."
This reverts commit 83ea17ebf2106859a51fbc2a86031b44d33696ad.

This is failing due to some strange template problems, so reverting
until it can be straightened out.

llvm-svn: 305505
2017-06-15 20:55:51 +00:00
Spyridoula Gravani 7a27a26db0 [DWARF] Removed dead code. The verifier functionality is provided by
the DWARFVerifier class (as it should).

llvm-svn: 305503
2017-06-15 20:40:08 +00:00
Zachary Turner b560fdf3b8 [llvm-pdbutil] rewrite the "raw" output style.
After some internal discussions, we agreed that the raw output style had
outlived its usefulness. It was originally created before we had even
thought of dumping to YAML, and it was intended to give us some insight
into the internals of a PDB file. Now we have YAML mode which does
almost exactly this but is more powerful in that it can round-trip back
to a PDB, which the raw mode could not do. So the raw mode had become
purely a maintenance burden.

One option was to just delete it. However, its original goal was to be
as readable as possible while staying close to the "metal" - i.e.
presenting the output in a way that maps directly to the underlying file
format. We don't actually need that last requirement anymore since it's
covered by the yaml mode, so we could repurpose "raw" mode to actually
just be as readable as possible.

This patch implements about 80% of the functionality previously in raw
mode, but in a completely different style that is more akin to what
cvdump outputs. Records are very compressed, often times appearing on
just one line. One nice thing about this is that it makes full record
matching easier, because you can grep for indices, names, and leaf types
on a single line often.

See the tests for some examples of what the new output looks like.

Note that this patch actually regresses the functionality of raw mode in
a few areas, but only because the patch was already unreasonably large
and going 100% would have been even worse. Specifically, this patch is
missing:

The ability to dump module debug subsections (checksums, lines, etc)
The ability to dump section headers
Aside from that everything is here. While goign through the tests fixing
them all up, I found many duplicate tests. They've been deleted. In
subsequent patches I will go through and re-add the missing
functionality.

Differential Revision: https://reviews.llvm.org/D34191

llvm-svn: 305495
2017-06-15 19:34:41 +00:00
Galina Kistanova 3c0505d30c Specified ReportError as noreturn friendly to old compilers.
llvm-svn: 305405
2017-06-14 17:32:53 +00:00
Zachary Turner a8cfc29c9a Resubmit "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This was originally reverted because of some non-deterministic
failures on certain buildbots.  Luckily ASAN eventually caught
this as a stack-use-after-scope, so the fix is included in
this patch.

llvm-svn: 305393
2017-06-14 15:59:27 +00:00
Zachary Turner 0085dce221 Revert "[codeview] Make obj2yaml/yaml2obj support .debug$S..."
This is causing failures on linux bots with an invalid stream
read.  It doesn't repro in any configuration on Windows, so
reverting until I have a chance to investigate on Linux.

llvm-svn: 305371
2017-06-14 06:24:24 +00:00
Zachary Turner a3da4467fa [codeview] Make obj2yaml/yaml2obj support .debug$S/T sections.
This allows us to use yaml2obj and obj2yaml to round-trip CodeView
symbol and type information without having to manually specify the bytes
of the section. This makes for much easier to maintain tests. See the
tests under lld/COFF in this patch for example. Before they just said
SectionData: <blob> whereas now we can use meaningful record
descriptions. Note that it still supports the SectionData yaml field,
which could be useful for initializing a section to invalid bytes for
testing, for example.

Differential Revision: https://reviews.llvm.org/D34127

llvm-svn: 305366
2017-06-14 05:31:00 +00:00
Spyridoula Gravani e41823bb89 Added partial verification for .apple_names accelerator table in llvm-dwarfdump output.
This patch adds code which verifies that each bucket in the .apple_names
accelerator table is either empty or has a valid hash index.

Differential Revision: https://reviews.llvm.org/D34177

llvm-svn: 305344
2017-06-14 00:17:55 +00:00
Galina Kistanova 41def9b72c Reverted r305339 as MSVC is not happy with noreturn in lambda.
llvm-svn: 305343
2017-06-13 23:57:51 +00:00
Galina Kistanova 680c7605a7 Specified LLVM_ATTRIBUTE_NORETURN for ReportError.
llvm-svn: 305339
2017-06-13 23:39:42 +00:00
Reid Kleckner 8cbdd0c0f2 [PDB] Add a module descriptor for every object file
Summary:
Expose the module descriptor index and fill it in for section
contributions.

Reviewers: zturner

Subscribers: llvm-commits, ruiu, hiraditya

Differential Revision: https://reviews.llvm.org/D34126

llvm-svn: 305296
2017-06-13 15:49:13 +00:00
Zachary Turner 68ea80d0a7 Slightly better fix for dealing with no-id-stream PDBs.
The last fix required the user to manually add the required
feature.  This caused an LLD test to fail because I failed to
update LLD.  In practice we can hide this logic so it can just
be transparently added when we write the PDB.

llvm-svn: 305236
2017-06-12 21:46:51 +00:00
Zachary Turner 990d0c8158 [llvm-pdbdump] Don't fail on PDBs with no ID stream.
Older PDBs don't have this.  Its presence is detected by using
the various "feature" flags that come at the end of the PDB
Stream.  Detect this, and don't try to dump the ID stream if the
features tells us it's not present.

llvm-svn: 305235
2017-06-12 21:34:53 +00:00
Zachary Turner d334cebac4 Fix a null pointer dereference in llvm-pdbutil pretty.
Static data members were causing a problem because I mistakenly
assumed all members would affect a class's layout and so the
Layout member would be non-null.

llvm-svn: 305229
2017-06-12 20:46:35 +00:00
Sylvestre Ledru 337804d86a Same expressions on both sides of the return
Summary:
I guess we want PointerToMemberFunction & PointerToDataMember


Fix coverity cid 1376038 


Reviewers: zturner

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34110

llvm-svn: 305219
2017-06-12 18:53:46 +00:00
David Blaikie a91885a08c dwarfdump: Handle relocs to zlib (.zdebug*) compressed sections
llvm-svn: 305152
2017-06-10 19:32:50 +00:00
Galina Kistanova 038f9854ec Added llvm_unreachable as ReportError cannot be specified as noreturn.
llvm-svn: 305143
2017-06-10 07:50:14 +00:00
Zachary Turner 3226fe95bb [pdb] Support CoffSymbolRVA debug subsection.
llvm-svn: 305108
2017-06-09 20:46:52 +00:00
Zachary Turner 7e62cd17d6 Allow VarStreamArray to use stateful extractors.
Previously extractors tried to be stateless with any additional
context information needed in order to parse items being passed
in via the extraction method.  This led to quite cumbersome
implementation challenges and awkwardness of use.  This patch
brings back support for stateful extractors, making the
implementation and usage simpler.

llvm-svn: 305093
2017-06-09 17:54:36 +00:00
Bob Haarman fdf499bf2d [codeview] use 32-bit integer for RelocOffset in DebugLinesSubsection
Summary:
RelocOffset is a 32-bit value, but we previously truncated it to 16 bits.

Fixes PR33335.

Reviewers: zturner, hiraditya!

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33968

llvm-svn: 305043
2017-06-09 01:18:10 +00:00
Zachary Turner 28c22c83e3 [pdb] Don't crash on unknown debug subsections.
More and more unknown debug subsection kinds are being discovered
so we should make it possible to dump these and display the
bytes.

llvm-svn: 305041
2017-06-09 00:53:59 +00:00
Zachary Turner deb391309c [CodeView] Support remaining debug subsection types
This adds support for Symbols, StringTable, and FrameData subsection
types.  Even though these subsections rarely if ever appear in a PDB
file (they are usually in object files), there's no theoretical reason
why they *couldn't* appear in a PDB.  The real issue though is that in
order to add support for dumping and writing them (which will be useful
for object files), we need a way to test them.  And since there is no
support for reading and writing them to / from object files yet, making
PDB support them is the best way to both add support for the underlying
format and add support for tests at the same time.  Later, when we go
to add support for reading / writing them from object files, we'll need
only minimal changes in the underlying read/write code.

llvm-svn: 305037
2017-06-09 00:28:08 +00:00
Zachary Turner 1bf7762049 [llvm-pdbdump] Support native ordering of subsections in raw mode.
This is the same change for the YAML Output style applied to the
raw output style.  Previously we would queue up all subsections
until every one had been read, and then output them in a pre-
determined order.  This was because some subsections need to be
read first in order to properly dump later subsections.  This
patch allows them to be dumped in the order they appear.

Differential Revision: https://reviews.llvm.org/D34015

llvm-svn: 305034
2017-06-08 23:49:01 +00:00
Zachary Turner 15eb237fd3 [PDB] Don't crash on /debug:fastlink PDBs.
Apparently support for /debug:fastlink PDBs isn't part of the
DIA SDK (!), and it was causing llvm-pdbdump to crash because
we weren't checking for a null pointer return value.  This
manifests when calling findChildren on the IDiaSymbol, and
it returns E_NOTIMPL.

llvm-svn: 304982
2017-06-08 16:00:40 +00:00
NAKAMURA Takumi 92c99cd6dc Update libdeps to add BinaryFormat, introduced in r304864.
llvm-svn: 304869
2017-06-07 04:48:49 +00:00
Zachary Turner 264b5d9e88 Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

llvm-svn: 304864
2017-06-07 03:48:56 +00:00
Zachary Turner 1bfb9f47af Fix uninitialized read.
llvm-svn: 304846
2017-06-06 23:54:23 +00:00
Adrian Prantl 318d1195f2 Introduce -brief command line option to llvm-dwarfdump
This patch introduces a new command line option, called brief, to
llvm-dwarfdump.  When -brief is used, the attribute forms for the
.debug_info section will not be emitted to output.

Patch by Spyridoula Gravani!

rdar://problem/21474365
Differential Revision: https://reviews.llvm.org/D33867

llvm-svn: 304844
2017-06-06 23:28:45 +00:00
Chandler Carruth aaeada6c75 Fix another ordering constraint with windows.h and comment about
a revers constraint that we got right (by chance).

llvm-svn: 304792
2017-06-06 12:43:20 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Wolfgang Pieb 77d3e938f8 [DWARF] Adding support for the DWARF v5 string offsets table (consumer/reader part only).
Reviewers: dblaikie, aprantl

Differential Revision: https://reviews.llvm.org/D32779

llvm-svn: 304759
2017-06-06 01:22:34 +00:00
Zachary Turner 88101dadcc [CodeView] Fix endianness bug.
We should be outputting in little endian, but we were writing
in host endianness.

llvm-svn: 304741
2017-06-05 22:12:23 +00:00
Zachary Turner 349c18f837 [CodeView] Handle Cross Module Imports and Exports.
While it's not entirely clear why a compiler or linker might
put this information into an object or PDB file, one has been
spotted in the wild which was causing llvm-pdbdump to crash.

This patch adds support for reading-writing these sections.
Since I don't know how to get one of the native tools to
generate this kind of debug info, the only test here is one
in which we feed YAML into the tool to produce a PDB and
then spit out YAML from the resulting PDB and make sure that
it matches.

llvm-svn: 304738
2017-06-05 21:40:33 +00:00
Zachary Turner 5b74ff33e7 [PDB] Fix use after free.
Previously MappedBlockStream owned its own BumpPtrAllocator that
it would allocate from when a read crossed a block boundary.  This
way it could still return the user a contiguous buffer of the
requested size.  However, It's not uncommon to open a stream, read
some stuff, close it, and then save the information for later.
After all, since the entire file is mapped into memory, the data
should always be available as long as the file is open.

Of course, the exception to this is when the data isn't *in* the
file, but rather in some buffer that we temporarily allocated to
present this contiguous view.  And this buffer would get destroyed
as soon as the strema was closed.

The fix here is to force the user to specify the allocator, this
way it can provide an allocator that has whatever lifetime it
chooses.

Differential Revision: https://reviews.llvm.org/D33858

llvm-svn: 304623
2017-06-03 00:33:35 +00:00
Zachary Turner 92dcdda623 [CodeView] Support CodeView subsections in any order.
Previously we would expect certain subsections to appear
in a certain order because some subsections would reference
other subsections, but in practice we need to support
arbitrary orderings since some object file and PDB file
producers generate them this way.  This also paves the
way for supporting Yaml <-> Object File conversion of
CodeView, since Object Files typically have quite a
large number of subsections in their debug info.

Differential Revision: https://reviews.llvm.org/D33807

llvm-svn: 304588
2017-06-02 19:49:14 +00:00
Zachary Turner afb81a83a9 Fix 2 more -Wreorder warnings.
llvm-svn: 304494
2017-06-01 23:24:50 +00:00
Zachary Turner ebd3ae8371 [CodeView] Properly align symbol records on read/write.
Object files have symbol records not aligned to any particular
boundary (e.g. 1-byte aligned), while PDB files have symbol
records padded to 4-byte aligned boundaries.  Since they share
the same reading / writing code, we have to provide an option to
specify the alignment and propagate it up to the producer or
consumer who knows what the alignment is supposed to be for the
given container type.

Added a test for this by modifying the existing PDB -> YAML -> PDB
round-tripping code to round trip symbol records as well as types.

Differential Revision: https://reviews.llvm.org/D33785

llvm-svn: 304484
2017-06-01 21:52:41 +00:00
Adrian Prantl f4bc1f77b7 [DWARF] Introduce Dump Options
This commit introduces a structure that holds all the flags that
control the pretty printing of dwarf output.

Patch by Spyridoula Gravani!

Differential Revision: https://reviews.llvm.org/D33749

llvm-svn: 304446
2017-06-01 18:18:23 +00:00
Zachary Turner d427383cb8 [CodeView] Move CodeView YAML code to ObjectYAML.
This is the beginning of an effort to move the codeview yaml
reader / writer into ObjectYAML so that it can be shared.
Currently the only consumer / producer of CodeView YAML is
llvm-pdbdump, but CodeView can exist outside of PDB files, and
indeed is put into object files and passed to the linker to
produce PDB files.  Furthermore, there are subtle differences
in the types of records that show up in object file CodeView
vs PDB file CodeView, but they are otherwise 99% the same.

By having this code in ObjectYAML, we can have llvm-pdbdump
reuse this code, while teaching obj2yaml and yaml2obj to use
this syntax for dealing with object files that can contain
CodeView.

This patch only adds support for CodeView type information
to ObjectYAML.  Subsequent patches will add support for
CodeView symbol information.

llvm-svn: 304248
2017-05-30 21:53:05 +00:00
Galina Kistanova 8c1e2f9108 Added missing break.
llvm-svn: 304230
2017-05-30 19:02:49 +00:00
Zachary Turner 591312c5c1 [CodeView] Add more DebugSubsection implementations.
This adds implementations for Symbols and FrameData, and renames
the existing codeview::StringTable class to conform to the
DebugSectionStringTable convention.

llvm-svn: 304222
2017-05-30 17:13:33 +00:00
Zachary Turner 8c099fe06e [CodeView] Rename ModuleDebugFragment -> DebugSubsection.
This is more concise, and matches the terminology used in other
parts of the codebase more closely.

llvm-svn: 304218
2017-05-30 16:36:15 +00:00
George Rimar a25d329b33 Recommit "[DWARF] - Make collectAddressRanges() return section index in addition to Low/High PC"
With fix of uninitialized variable.

Original commit message:

This change is intended to use for LLD in D33183. 
Problem we have in LLD when building .gdb_index is that we need to know section which address range belongs to.

Previously it was solved on LLD side by providing fake section addresses with use of llvm::LoadedObjectInfo
interface. We assigned file offsets as addressed. Then after obtaining ranges lists, for each range we had to find section ID's.
That not only was slow, but also complicated implementation and was the reason of incorrect behavior when
sections share the same offsets, like D33176 shows.

This patch makes DWARF parsers to return section index as well. That solves problem mentioned above.

Differential revision: https://reviews.llvm.org/D33184

llvm-svn: 304078
2017-05-27 18:10:23 +00:00