Commit Graph

97 Commits

Author SHA1 Message Date
Zachary Turner 36efbfa6d8 [pdb] Print out some more info when dumping a raw stream.
We have various command line options that print the type of a
stream, the size of a stream, etc but nowhere that it can all be
viewed together.

Since a previous patch introduced the ability to dump the bytes
of a stream, this seems like a good place to present a full view
of the stream's properties including its size, what kind of data
it represents, and the blocks it occupies.  So I added the
ability to print that information to the -stream-data command
line option.

llvm-svn: 281077
2016-09-09 19:00:49 +00:00
Zachary Turner 72c5b6451f [pdb] Add command line options for dumping individual streams and blocks
I ran into a situation where I wanted to print out the contents of
page 6 of a PDB as a binary blob, and there was no straightforward
way to do that.

In addition to adding that, this patch also adds the ability to dump
a stream by index as a binary blob, and it will stitch together all
the blocks and dump the whole thing as one seemingly contiguous
sequence of bytes.

llvm-svn: 281070
2016-09-09 18:17:52 +00:00
Zachary Turner c6d54da891 [pdb] Write PDB TPI Stream from Yaml.
This writes the full sequence of type records described in
Yaml to the TPI stream of the PDB file.

Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D24316

llvm-svn: 281063
2016-09-09 17:46:17 +00:00
Zachary Turner c998571493 Re-add "Make FieldList records print as a YAML sequence"
This was originally submitted in r280549, and reverted in r280577
due to breaking one MSVC buildbot.  The issue is that MSVC 2013
doesn't synthesize move constructors.  So even though i was
writing std::move(A) it was copying it, leading to a bogus ArrayRef.
The solution here is to simply remove the std::vector<> from the
type, since it is unused and unnecessary.  This way the ArrayRef
continues to point into the original memory backing the CVType.

llvm-svn: 280769
2016-09-06 23:45:47 +00:00
Nico Weber 05e78450be Revert r280549.
The test it added doesn't pass:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/15318/steps/ninja%20check%201/logs/FAIL%3A%20LLVM%3A%3Apdbdump-yaml-types.test

Command Output (stdout):
--
$ "D:/buildslave/clang-x64-ninja-win7/stage1/./bin\llvm-pdbdump.EXE" "pdb2yaml" "-tpi-stream" "D:\buildslave\clang-x64-ninja-win7\llvm\test\DebugInfo\PDB/Inputs/empty.pdb"
$ "D:/buildslave/clang-x64-ninja-win7/stage1/./bin\FileCheck.EXE" "-check-prefix=YAML" "D:\buildslave\clang-x64-ninja-win7\llvm\test\DebugInfo\PDB\pdbdump-yaml-types.test"
# command stderr:
D:\buildslave\clang-x64-ninja-win7\llvm\test\DebugInfo\PDB\pdbdump-yaml-types.test:36:7: error: expected string not found in input
YAML: Name: apartment
      ^
<stdin>:153:10: note: scanning from here
 Value: 161
         ^

llvm-svn: 280577
2016-09-03 03:18:49 +00:00
Zachary Turner 83849415de [codeview] Make FieldList records print as a yaml sequence.
Before we were kind of imitating the behavior of a Yaml sequence
by outputting each record one after the other.  This makes it a
little cumbersome when we want to go the other direction -- from
Yaml to Pdb.  So this treats FieldList records as no different than
any other list of records, by printing them as a Yaml sequence with
the exact same format.

llvm-svn: 280549
2016-09-02 22:19:01 +00:00
Zachary Turner ac5763eca4 Resubmit "Write the TPI stream from a PDB to Yaml."
The original patch was breaking some buildbots due to an
incorrect ordering of function definitions which caused some
compilers to recognize a definition but others to not.

llvm-svn: 279089
2016-08-18 16:49:29 +00:00
Justin Bogner 39eec466a2 Revert "Write the TPI stream from a PDB to Yaml."
This is hitting a "use of undeclared identifier 'skipPadding' error
locally and on some bots.

This reverts r278869.

llvm-svn: 278871
2016-08-16 23:37:10 +00:00
Zachary Turner 8321ba5437 Write the TPI stream from a PDB to Yaml.
Reviewed By: ruiu, rnk
Differential Revision: https://reviews.llvm.org/D23226

llvm-svn: 278869
2016-08-16 23:28:54 +00:00
Rui Ueyama 7e49549d4f pdbdump: Add a test to verify the result of PDB -> YAML -> PDB conversions.
Currently not all information can be restored from YAML.
This test verifies only the PDB header.

llvm-svn: 277682
2016-08-03 23:54:39 +00:00
Rui Ueyama d1d8c8312a pdbdump: Fix crash bug.
pdbdump calls DbiStreamBuilder::commit through PDBFileBuilder::commit
without calling DbiStreamBuilder::finalize. Because `finalize` initializes
`Header` member, `Header` remained nullptr which caused a crash bug.

Differential Revision: https://reviews.llvm.org/D23143

llvm-svn: 277681
2016-08-03 23:43:23 +00:00
Rui Ueyama 057625f616 Fix a test for r277545.
This change should have been submitted with that commit.

llvm-svn: 277548
2016-08-02 23:25:59 +00:00
Zachary Turner d3c7b8e303 [msf] Teach LLVM to parse a split Fpm.
The FPM is split at regular intervals across the MSF file, as the MS code
suggests. It turns out that the value of the interval is precisely the
block size. If the block size is 4096, then there are two Fpm pages every
4096 blocks.

So here we teach the PDBFile class to parse a split FPM, and also add more
options when dumping the FPM to display some additional information such
as orphaned pages (pages which the FPM says are allocated, but which
nothing appears to use), use after free pages (pages which the FPM says
are not allocated, but which are referenced by a stream), and multiple use
pages (pages which the FPM says are allocated but are used more than
once).

Reviewed By: ruiu
Differential Revision: https://reviews.llvm.org/D23022

llvm-svn: 277388
2016-08-01 21:19:45 +00:00
Rui Ueyama 7a5cdc6225 pdbdump: Dump Free Page Map contents.
Differential Revision: https://reviews.llvm.org/D22974

llvm-svn: 277216
2016-07-29 21:38:00 +00:00
Rui Ueyama d8388aaecb [pdbdump] Use the "flow" style to print out a sequence of uint32_t.
Summary: Lists can be written either with "-" or "[]" in YAML.

Differential Revision: https://reviews.llvm.org/D22579

llvm-svn: 276168
2016-07-20 19:41:47 +00:00
Zachary Turner b927e02e1b [pdb] Teach MsfBuilder and other classes about the Free Page Map.
Block 1 and 2 of an MSF file are bit vectors that represent the
list of blocks allocated and free in the file.  We had been using
these blocks to write stream data and other data, so we mark them
as the free page map now.  We don't yet serialize these pages to
the disk, but at least we make a note of what it is, and avoid
writing random data to them.

Doing this also necessitated cleaning up some of the tests to be
more general and hardcode fewer values, which is nice.

llvm-svn: 275629
2016-07-15 22:17:19 +00:00
Zachary Turner 5e534c7fb3 [pdb] Round trip the NameMap data structure to YAML.
llvm-svn: 275628
2016-07-15 22:17:08 +00:00
Zachary Turner faa554b2fd [pdb] Use MsfBuilder to handle the writing PDBs.
Previously we would read a PDB, then write some of it back out,
but write the directory, super block, and other pertinent metadata
back out unchanged.  This generates incorrect PDBs since the amount
of data written was not always the same as the amount of data read.

This patch changes things to use the newly introduced `MsfBuilder`
class to write out a correct and accurate set of Msf metadata for
the data *actually* written, which opens up the door for adding and
removing type records, symbol records, and other types of data to
an existing PDB.

llvm-svn: 275627
2016-07-15 22:16:56 +00:00
Zachary Turner dbeaea7b35 Refactor the PDB writing to use a builder approach
llvm-svn: 275110
2016-07-11 21:45:26 +00:00
Zachary Turner f6b9382467 [pdb] Add a pdb2yaml option to not dump file headers.
This will be useful once we start adding the ability to dump type
records and symbol records, since it will allow us to generate
mergeable information instead of information that specifies an
entire file.

llvm-svn: 275109
2016-07-11 21:45:09 +00:00
David Majnemer 7abd269aa9 [CodeView] Emit an appropriate symbol kind for globals
We emitted debug info for globals/functions as if they all had external
linkage.  Instead, emit local symbol records when appropriate.

llvm-svn: 274676
2016-07-06 21:07:47 +00:00
Zachary Turner 8848a7a6b2 [pdb] Round trip the PDB stream between YAML and binary PDB.
This gets writing of the PDB stream working.

llvm-svn: 274647
2016-07-06 18:05:57 +00:00
Zachary Turner ab58ae8730 [pdb] Re-add code to write PDB files.
Somehow all the functionality to write PDB files got removed,
probably accidentally when uploading the patch perhaps the wrong
one got uploaded.  This re-adds all the code, as well as the
corresponding test.

llvm-svn: 274248
2016-06-30 17:43:00 +00:00
Zachary Turner a30bd1a1bc Update llvm-pdbdump to use subcommands.
llvm-svn: 274247
2016-06-30 17:42:48 +00:00
Reid Kleckner 5b335b864b [codeview] Add support for splitting field list records over 64KB
The basic structure is that once a list record goes over 64K, the last
subrecord of the list is an LF_INDEX record that refers to the next
record. Because the type record graph must be toplogically sorted, this
means we have to emit them in reverse order. We build the type record in
order of declaration, so this means that if we don't want extra copies,
we need to detect when we were about to split a record, and leave space
for a continuation subrecord that will point to the eventual split
top-level record.

Also adds dumping support for these records.

Next we should make sure that large method overload lists work properly.

llvm-svn: 273294
2016-06-21 18:33:01 +00:00
Rui Ueyama 5dbea9db10 [Codeview] Add a class for LF_UDT_MOD_SRC_LINE.
Differential Revision: http://reviews.llvm.org/D21406

llvm-svn: 272843
2016-06-15 21:25:29 +00:00
Zachary Turner 1dc9fd3c4a Resubmit "[pdb] Actually write a PDB to disk from YAML.""
Reviewed By: ruiu
Differential Revision: http://reviews.llvm.org/D21220

llvm-svn: 272708
2016-06-14 20:48:36 +00:00
Zachary Turner 07c229c9e7 Revert "[pdb] Actually write a PDB to disk from YAML."
This reverts commit 879139e1c6577b09df52de56a6bab856a19ed185.

This was committed accidentally when I blindly typed git svn
dcommit instead of the command to generate a patch.

llvm-svn: 272693
2016-06-14 18:51:35 +00:00
Zachary Turner fe5bc02492 [pdb] Actually write a PDB to disk from YAML.
llvm-svn: 272692
2016-06-14 18:49:36 +00:00
Rui Ueyama f14a74c102 [pdbdump] Print out # of hash buckets.
In the reference code, the field name is `cHashBuckets`.

llvm-svn: 272075
2016-06-07 23:53:43 +00:00
Rui Ueyama d833917f98 [pdbdump] Print out TPI hash key size.
llvm-svn: 272073
2016-06-07 23:44:27 +00:00
Rui Ueyama 4bc848047b Fix CRLF -> LF.
llvm-svn: 271960
2016-06-06 23:35:52 +00:00
Rui Ueyama 2c5384ae4c [pdbdump] Print section header flags.
llvm-svn: 271943
2016-06-06 21:34:55 +00:00
Zachary Turner 7120a478fa [llvm-pdbdump] Dump MSF headers to YAML.
This is the simplest possible patch to get some kind of YAML
output.  All it dumps is the MSF header fields so that in
theory an empty MSF file could be reconstructed.

Reviewed By: ruiu, majnemer
Differential Revision: http://reviews.llvm.org/D20971

llvm-svn: 271939
2016-06-06 20:37:05 +00:00
Rui Ueyama ef2b488482 [pdbdump] Print out New FPO stream contents.
The data strucutre in the new FPO stream is described in the
PE/COFF spec. There is one record per function if frame pointer
is omitted.

Differential Revision: http://reviews.llvm.org/D20999

llvm-svn: 271926
2016-06-06 18:39:21 +00:00
Rui Ueyama fd97bf1f76 pdbdump: print out TPI hashes.
Differential Revision: http://reviews.llvm.org/D20945

llvm-svn: 271736
2016-06-03 20:48:51 +00:00
Zachary Turner 71f725735c [pdb] Add string table offsets to check output.
llvm-svn: 271674
2016-06-03 14:22:46 +00:00
Zachary Turner 3df1bfaaec [pdb] Print out file names instead of file offsets.
When printing line information and file checksums, we were printing
the file offset field from the struct header.  This teaches
llvm-pdbdump how to turn those numbers into the filename.  In the
case of file checksums, this is done by looking in the global
string table.  In the case of line contributions, this is done
by indexing into the file names buffer of the DBI stream.  Why
they use a different technique I don't know.

llvm-svn: 271630
2016-06-03 05:52:57 +00:00
Zachary Turner d0563f29f9 [pdb] Dump file checksums from pdb codeview line info.
llvm-svn: 271622
2016-06-03 04:01:48 +00:00
Zachary Turner a96cce64a5 [codeview] Dump line number and column information.
To facilitate this, a couple of changes had to be made:

1. `ModuleSubstream` got moved from `DebugInfo/PDB` to
`DebugInfo/CodeView`, and various codeview related types are defined
there.  It turns out `DebugInfo/CodeView/Line.h` already defines many of
these structures, but this is really old code that is not endian aware,
doesn't interact well with `StreamInterface` and not very helpful for
getting stuff out of a PDB.  Eventually we should migrate the old readobj
`COFFDumper` code to these new structures, or at least merge their
functionality somehow.

2. A `ModuleSubstream` visitor is introduced.  Depending on where your
module substream array comes from, different subsets of record types can
be expected.  We are already hand parsing these substream arrays in many
places especially in `COFFDumper.cpp`.  In the future we can migrate these
paths to the visitor as well, which should reduce a lot of code in
`COFFDumper.cpp`.

Differential Revision: http://reviews.llvm.org/D20936
Reviewed By: ruiu, majnemer

llvm-svn: 271621
2016-06-03 03:25:59 +00:00
Zachary Turner 7eb6d358af [llvm-pdbdump] Dump CodeView line information.
This first pass only splits apart the records and dumps the line
info kinds and binary data.  Subsequent patches will parse out
the binary data into more useful information and dump it in
detail.

llvm-svn: 271576
2016-06-02 20:11:22 +00:00
Rui Ueyama 90db78816b pdbdump: print out COFF section headers.
Unlike other sections that can grow to any size, the COFF section header
stream has maximum length because each record is fixed size and the COFF
file format limits the maximum number of sections. So I decided to not
create a specific stream class for it. Instead, I added a member function
to DbiStream class which returns a vector of COFF headers.

Differential Revision: http://reviews.llvm.org/D20717

llvm-svn: 271557
2016-06-02 18:20:20 +00:00
Zachary Turner 93839cb4ac [pdb] Parse and dump section map and section contribs
Differential Revision: http://reviews.llvm.org/D20876
Reviewed By: rnk, ruiu

llvm-svn: 271488
2016-06-02 05:07:49 +00:00
David Majnemer 8d106d5ea4 Update test to deal with non-zero exit codes
llvm-svn: 271135
2016-05-28 19:02:12 +00:00
Rui Ueyama 6816367a27 pdbdump: print out the name of the stream 0.
Differential Revision: http://reviews.llvm.org/D20712

llvm-svn: 270943
2016-05-27 00:32:07 +00:00
Rui Ueyama 9dc034dba1 pdbdump: Add -raw-all to enable all -raw-* flags.
Differential Revision: http://reviews.llvm.org/D20707

llvm-svn: 270937
2016-05-26 23:26:55 +00:00
Zachary Turner d3076ab36f [llvm-pdbdump] Decipher the remaining PDB streams.
We know at least know the meaning of every stream of the
PDB file.  Yay!

llvm-svn: 270669
2016-05-25 05:49:48 +00:00
Zachary Turner c9972c64f5 [llvm-pdbdump] Dump the IPI stream and all records.
llvm-svn: 270661
2016-05-25 04:35:22 +00:00
Rui Ueyama b12b158f20 pdbdump: fix bug in name hash table.
name_ids() did not return all IDs but only the first NameCount items.
The number of non-zero entries in IDs vector is NameCount, but it
does not mean that all non-zero entries are at the beginning of IDs
vector.

Differential Revision: http://reviews.llvm.org/D20611

llvm-svn: 270656
2016-05-25 04:07:17 +00:00
Zachary Turner c59261ca37 [llvm-pdbdump] Stream 0 isn't actually the MSF superblock.
Oddly enough, I realized we don't actually know what stream
0 is (if anything).

llvm-svn: 270655
2016-05-25 03:53:16 +00:00