The new key/value pairs that are added to each module's stats are:
"debugInfoByteSize": The size in bytes of debug info for each module.
"debugInfoIndexTime": The time in seconds that it took to index the debug info.
"debugInfoParseTime": The time in seconds that debug info had to be parsed.
At the top level we add up all of the debug info size, parse time and index time with the following keys:
"totalDebugInfoByteSize": The size in bytes of all debug info in all modules.
"totalDebugInfoIndexTime": The time in seconds that it took to index all debug info if it was indexed for all modules.
"totalDebugInfoParseTime": The time in seconds that debug info was parsed for all modules.
Differential Revision: https://reviews.llvm.org/D112501
Skeleton vs. DWO units mismatch has been fixed in D106270. As they both
have type DWARFUnit it is a bit difficult to debug. So it is better to
make it safe against future changes.
Reviewed By: kimanh, clayborg
Differential Revision: https://reviews.llvm.org/D107659
When going through the CU entries in the name index,
make sure to compare the name entry's CU
offset against the skeleton CU's offset.
Previously there would be a mismatch, since the
wrong offset was compared, and thus no suitable
entry was found.
Reviewed By: jankratochvil
Differential Revision: https://reviews.llvm.org/D106270
LLDB can often appear deadlocked to users that use IDEs when it is indexing DWARF, or parsing symbol tables. These long running operations can make a debug session appear to be doing nothing even though a lot of work is going on inside LLDB. This patch adds a public API to allow clients to listen to debugger events that report progress and will allow UI to create an activity window or display that can show users what is going on and keep them informed of expensive operations that are going on inside LLDB.
Differential Revision: https://reviews.llvm.org/D97739
This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to the
LLDB_LOG(F) macro.
Differential revision: https://reviews.llvm.org/D93663
Add an optimal thread strategy to execute specified amount of tasks.
This strategy should prevent us from creating too many threads if we
occasionaly have an unexpectedly small amount of tasks.
Differential Revision: https://reviews.llvm.org/D87765
Pavel Labath wrote in D73206:
The internal representation of DebugNames and Apple indexes is fixed by
the relevant (pseudo-)standards, so we can't really change it. The
question is how to efficiently (and cleanly) convert from the internal
representation to some common thing. The conversion from AppleIndex to
DIERef is trivial (which is not surprising as it was the first and the
overall design was optimized for that). With debug_names, the situation
gets more tricky. The internal representation of debug_names uses
CU-relative DIE offsets, but DIERef wants an absolute offset. That means
the index has to do more work to produce the common representation. And
it needs to do that for all results, even though a lot of the index
users are really interested only in a single entry. With the switch to
user_id_t, _all_ indexes would have to do some extra work to encode it,
only for their users to have to immediately decode it back. Having
a iterator/callback based api would allow us to minimize the impact of
that, as it would only need to happen for the entries that are really
used. And /I think/ we could make it interface returns DWARFDies
directly, and each index converts to that using the most direct approach
available.
Jan Kratochvil:
It also makes all the callers shorter as they no longer need to fetch
DWARFDIE from DIERef (and handling if not found by ReportInvalidDIERef)
but the callers are already served DWARFDIE which they need.
In some cases the DWARFDIE had to be fetched both by callee (DWARFIndex
implementation) and caller.
Differential Revision: https://reviews.llvm.org/D77970
It is an obvious part of D77326.
It removes some needless deep indentation and some redundant statements.
It prepares the code for a more clean next patch - DWARF index callbacks
in D77327.
Summary:
When we added support for type units in dwo files, we changed the
"manual" dwarf index to index _all_ dwarf units in the dwo file instead
of just the split unit belonging to our skeleton unit. This was fine for
dwo files, as they contain only a single compile units and type units do
not have a split type unit which would point to them.
However, this does not work for dwp files because, these files do
contain multiple split compile units, and the current approach means
that each unit gets indexed multiple times (once for each split unit =>
n^2 complexity).
This patch teaches the manual dwarf index to treat dwp files specially.
Any type units in the dwp file added to the main list of compile units
and indexed with them in a single batch. Split compile units in dwp
files are still indexed as a part of their skeleton unit -- this is done
because we need the DW_AT_language attribute from the skeleton unit to
index them properly.
Handling of dwo files remains unchanged -- all units (type and skeleton)
are indexed when we reach the dwo file through the split unit.
Reviewers: clayborg, JDevlieghere, aprantl
Subscribers: arphaman, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74964
Change the return value of SymbolFileDWARF::DebugInfo from a pointer to
a reference, and remove all null checks.
Previously, we were not constructing the DebugInfo object when the
debug_info section was empty. Now we always construct the object but
it will return an empty list of dwarf units (a thing which it already
supported).
Summary:
All of our lookup APIs either use `CompilerDeclContext &` or `CompilerDeclContext *` semi-randomly it seems.
This leads to us constantly converting between those two types (and doing nullptr checks when going from
pointer to reference). It also leads to the confusing situation where we have two possible ways to express
that we don't have a CompilerDeclContex: either a nullptr or an invalid CompilerDeclContext (aka a default
constructed CompilerDeclContext).
This moves all APIs to use references and gets rid of all the nullptr checks and conversions.
Reviewers: labath, mib, shafik
Reviewed By: labath, shafik
Subscribers: shafik, arphaman, abidh, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74607
Summary:
This is a preparatory patch to re-enable DWP support in lldb (we already
have code claiming to do that, but it has been completely broken for a
while now).
The idea of the new approach is to make the SymbolFileDWARFDwo class
handle both dwo and dwo files, similar to how llvm uses one DWARFContext
to handle the two.
The first step is to remove the assumption that a SymbolFileDWARFDwo
holds just a single compile unit, i.e. the GetBaseCompileUnit method.
This requires changing the way how we reach the skeleton compile unit
(and the lldb_private::CompileUnit) from a dwo unit, which was
previously done via GetSymbolFile()->GetBaseCompileUnit() (and some
virtual dispatch).
The new approach reuses the "user data" mechanism of DWARFUnits, which
was used to link dwarf units (both skeleton and split) to their
lldb_private counterparts. Now, this is done only for non-dwo units, and
instead of that, the dwo units holds a pointer to the relevant skeleton
unit.
Reviewers: JDevlieghere, aprantl, clayborg
Reviewed By: JDevlieghere, clayborg
Subscribers: arphaman, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73781
This patchset is removing non-DWARF code from DWARFUnit for better
future merge with LLVM DWARF as discussed with @labath.
Differential revision: https://reviews.llvm.org/D70646
Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).
This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).
Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73258
Summary:
This change is connected with
https://reviews.llvm.org/D69843
In large codebases, we sometimes see Module::FindFunctions (when called from
ClangExpressionDeclMap::FindExternalVisibleDecls) returning huge amounts of
functions.
In current fix I trying to return only function_fullnames from ManualDWARFIndex::GetFunctions when eFunctionNameTypeFull is passed as argument.
Reviewers: labath, jarin, aprantl
Reviewed By: labath
Subscribers: shafik, clayborg, teemperor, arphaman, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D70846
Summary:
With the last round of refactors, supporting type units in dwo files
becomes almost trivial. This patch contains a couple of small fixes,
which taken as a whole make type units work in the split dwarf scenario
(both DWARF4 and DWARF5):
- DWARFContext: make sure we actually read the debug_types.dwo section
- DWARFUnit: set string offsets base on all units in the dwo file, not
just the main CU
- ManualDWARFIndex: index all units in the file
- SymbolFileDWARFDwo: Search for the single compile unit in the file, as
we can no longer assume it will be the first one
The last part makes it obvious that there is still some work to be done
here, namely that we do not support dwo files with multiple compile
units. That is something that should be easier after the DIERef
refactors, but it still requires more work.
Tests are added for the type units+split dwarf + dwarf4/5 scenarios, as
well as a test that checks we behave reasonably in the presence of dwo
files with multiple CUs.
Reviewers: clayborg, JDevlieghere, aprantl
Subscribers: arphaman, lldb-commits
Differential Revision: https://reviews.llvm.org/D63643
llvm-svn: 364274
Summary:
When dwo support was introduced, it used a trick where debug info
entries were referenced by the offset of the compile unit in the main
file, but the die offset was relative to the dwo file. Although there
was some elegance to it, this representation was starting to reach its
breaking point:
- the fact that the skeleton compile unit owned the DWO file meant that
it was impossible (or at least hard and unintuitive) to support DWO
files containing more than one compile unit. These kinds of files are
produced by LTO for example.
- it made it impossible to reference any DIEs in the skeleton compile
unit (although the skeleton units are generally empty, clang still
puts some info into them with -fsplit-dwarf-inlining).
- (current motivation) it made it very hard to support type units placed
in DWO files, as type units don't have any skeleton units which could
be referenced in the main file
This patch addresses this problem by introducing an new
"dwo_num" field to the DIERef class, whose purpose is to identify the
dwo file. It's kind of similar to the dwo_id field in DWARF5 unit
headers, but while this is a 64bit hash whose main purpose is to catch
file mismatches, this is just a smaller integer used to indentify a
loaded dwo file. Currently, this is based on the index of the skeleton
compile unit which owns the dwo file, but it is intended to be
eventually independent of that (to support the LTO use case).
Simultaneously the cu_offset is dropped to conserve space, as it is no
longer necessary. This means we can remove the "BaseObjectOffset" field
from the DWARFUnit class. It also means we can remove some of the
workarounds put in place to support the skeleton-unit+dwo-die combo.
More work is needed to remove all of them, which is out of scope of this
patch.
Reviewers: JDevlieghere, clayborg, aprantl
Subscribers: mehdi_amini, dexonsmith, arphaman, lldb-commits
Differential Revision: https://reviews.llvm.org/D63428
llvm-svn: 364009
Previously it was storing a *pointer*, which left open the possibility
of this pointer being null. We never made use of that possibility (it
does not make sense), and most of the code was already assuming that.
However, there were a couple of null-checks scattered around the code.
This patch replaces the reference with a pointer, making the
non-null-ness explicit, and removes the remaining null-checks.
llvm-svn: 363381
The fix form sizes use to have two arrays: one for 4 byte addresses and in for 8 byte addresses. The table had an issue where DW_FORM_flag_present wasn't being represented as a fixed size form because its actual size _is_ zero and zero was used to indicate the form isn't fixed in size. Any code that needed to quickly access the DWARF had to get a FixedFormSizes instance using the address byte size.
This fix cleans things up by adding a DWARFFormValue::GetFixedSize() both as a static method and as a member function on DWARFFormValue. It correctly can indicate if a form size is zero. This cleanup is a precursor to a follow up patch where I hope to speed up DWARF parsing.
I verified performance doesn't regress by loading hundreds of DWARF files and setting a breakpoint by file and line and by name in files that do not have DWARF indexes. Performance remained consistent between the two approaches.
Differential Revision: https://reviews.llvm.org/D62416
llvm-svn: 361675
Summary:
NFC = [[ https://llvm.org/docs/Lexicon.html#nfc | Non functional change ]]
This commit is the result of modernizing the LLDB codebase by using
`nullptr` instread of `0` or `NULL`. See
https://clang.llvm.org/extra/clang-tidy/checks/modernize-use-nullptr.html
for more information.
This is the command I ran and I to fix and format the code base:
```
run-clang-tidy.py \
-header-filter='.*' \
-checks='-*,modernize-use-nullptr' \
-fix ~/dev/llvm-project/lldb/.* \
-format \
-style LLVM \
-p ~/llvm-builds/debug-ninja-gcc
```
NOTE: There were also changes to `llvm/utils/unittest` but I did not
include them because I felt that maybe this library shall be updated in
isolation somehow.
NOTE: I know this is a rather large commit but it is a nobrainer in most
parts.
Reviewers: martong, espindola, shafik, #lldb, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: arsenm, jvesely, nhaehnle, hiraditya, JDevlieghere, teemperor, rnkovacs, emaste, kubamracek, nemanjai, ki.stfu, javed.absar, arichardson, kbarton, jrtc27, MaskRay, atanasyan, dexonsmith, arphaman, jfb, jsji, jdoerfert, lldb-commits, llvm-commits
Tags: #lldb, #llvm
Differential Revision: https://reviews.llvm.org/D61847
llvm-svn: 361484
Summary:
This patch adds the ability to precisely address debug info in
situations when a single file can have more than one debug-info-bearing
sections (as is the case with type units in DWARF v4).
The changes here can be classified into roughly three categories:
- the code which addresses a debug info by offset gets an additional
argument, which specifies the section one should look into.
- the DIERef class also gets an additional member variable specifying
the section. This way, code dealing with DIERefs can know which
section is the object referring to.
- the user_id_t encoding steals one bit from the dwarf_id field to store
the section. This means the total number of separate object files
(apple .o, or normal .dwo) is limited to 2 billion, but that is fine
as it's not possible to hit that number without switching to DWARF64
anyway.
This patch is functionally equivalent to (and inspired by) the two
patches (D61503 and D61504) by Jan Kratochvil, but there are differences
in the implementation:
- it uses an enum instead of a bool flag to differentiate the sections
- it increases the size of DIERef struct instead of reducing the amount
of addressable debug info
- it sets up DWARFDebugInfo to store the units in a single vector
instead of two. This sets us up for the future in which type units can
also live in the debug_info section, and I believe it's cleaner
because there's no need for unit index remapping
There are no tests with this patch as this is essentially NFC until
we start parsing type units from the debug_types section.
Reviewers: JDevlieghere, clayborg, aprantl
Subscribers: arphaman, jankratochvil, lldb-commits
Differential Revision: https://reviews.llvm.org/D61908
llvm-svn: 360872
apple and manual indexing code were creating a DIERef in a bunch of
places. Though the code itself is not much, it is also easy to simplify
by factoring out the DIERef creation. In HashedNameToDIE I create a
conversion operator from DIEInfo to DIERef, and in ManualDWARFIndex I
just create the DIERef in a global variable up-front.
This also reduces the diff in follow-up patches which change how DIERefs
are constructed.
llvm-svn: 360669
D42892 changed a lot of code to use superclass DWARFUnit instead of its
subclass DWARFCompileUnit.
Finish this change more thoroughly for any *CompileUnit* -> *Unit* names.
Later patch will introduce DWARFTypeUnit which needs to be sometimes different
from DWARFCompileUnit and it would be confusing without this renaming.
Differential Revision: https://reviews.llvm.org/D61501
llvm-svn: 360443
A lot of comments in LLDB are surrounded by an ASCII line to delimit the
begging and end of the comment.
Its use is not really consistent across the code base, sometimes the
lines are longer, sometimes they are shorter and sometimes they are
omitted. Furthermore, it looks kind of weird with the 80 column limit,
where the comment actually extends past the line, but not by much.
Furthermore, when /// is used for Doxygen comments, it looks
particularly odd. And when // is used, it incorrectly gives the
impression that it's actually a Doxygen comment.
I assume these lines were added to improve distinguishing between
comments and code. However, given that todays editors and IDEs do a
great job at highlighting comments, I think it's worth to drop this for
the sake of consistency. The alternative is fixing all the
inconsistencies, which would create a lot more churn.
Differential revision: https://reviews.llvm.org/D60508
llvm-svn: 358135
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
It possible that a single module has indexed and non-indexed compile
units. In this case, we can use the fast indexed lookup for the first
ones and fall back to the manual index for the others.
This patch implements this functionality by adding a units_to_avoid
argument to the ManualDWARFIndex constructor. Any units present in that
list will be ignored for the purposes of manual index. Individual
DebugNamesDWARFIndex then always consult both the manual fallback index
as well as the index in the .debug_names section.
Reviewers: JDevlieghere, clayborg
Subscribers: aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D47832
llvm-svn: 334185
Summary:
This patch adds the ability to lookup variables to the DWARF v5 index
class.
During review we discovered an inconsistency between how the existing
two indexes handle looking up qualified names of the variables:
- manual index would return a value if the input string exactly matched
the demangled name of some variable.
- apple index ignored the context and returned any variable with the
same base name.
So, this patch also rectifies that situation:
- it removes all context handling from the index classes. The
GetGlobalVariables functions now just take a base name. For manual
index, this meant we can stop putting demangled names into the
variable index (this matches the behavior for functions).
- context extraction is put into SymbolFileDWARF, so that it is common
to all indexes.
- additional filtering based on the context is also done in
SymbolFileDWARF. This is done via a simple substring search, which is
not ideal, but it matches what we are doing for functions (cf.
Module::LookupInfo::Prune).
Reviewers: clayborg, JDevlieghere
Subscribers: aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D47781
llvm-svn: 334181
Summary:
This patch adds the skeleton for implementing the DWARF v5 name index
class. All of the methods are stubbed out and will be implemented in
subsequent patches. The interesting part of the patch is the addition of
a "ignore-file-indexes" setting to the dwarf plugin which enables a
user to force using manual indexing path in lldb (for example as a
debugging aid). I have also added a test that verifies that file indexes
are used by default.
Reviewers: JDevlieghere, clayborg, jingham
Subscribers: mgorny, mehdi_amini, aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D47629
llvm-svn: 334088
Now that Apple index determines method-ness straight from the debug
info, we don't need to resolve the functions into SymbolContexts inside
the Index classes. This removes the need for callback arguments and
allows us to pull the common parts out of the two implementations of
these functions back into the SymbolFileDWARF class.
Reviewers: JDevlieghere, clayborg
Subscribers: aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D47147
llvm-svn: 334004
If BuildAddressRangeTable called ExtractDIEsIfNeeded(false), then another
thread started processing data from m_die_array and then the first thread
called final ClearDIEs() the second thread would crash.
It is also required without multithreaded debugger using DW_TAG_partial_unit
for DWZ.
Differential revision: https://reviews.llvm.org/D40470
llvm-svn: 333987
Summary:
When searching for methods only, we need to do extra work to make sure
the functions we get from the apple tables are indeed methods.
Previously we were resolving the DIE into a SymbolContext and then
checked whether the enclosing CompilerDeclContext is a
class (or struct, or union).
This patch changes that to operate on the debug info directly. This
should be:
- simpler
- faster
- more consistent with the ManualDWARFIndex (which does the same check,
only at indexing time).
What we lose this ways is for the language plugin to have a say in what
it considers to be a "class", but that's probably more flexibility than
we need (and if we really wanted to do that in the future, we could
implement a more direct way to consult the plugin about this).
This also fixes the find-method-local-struct test, which was failing
because we were not able to construct a CompilerDeclContext for a local
struct correctly.
As a drive-by, I rename the DWARFDIE's IsStructClassOrUnion method to
match the name on the CompilerDeclContext class.
Reviewers: clayborg, JDevlieghere
Subscribers: aprantl, lldb-commits
Differential Revision: https://reviews.llvm.org/D47470
llvm-svn: 333878
GetUnitDIEPtrOnly() needs to return pointer to the first DIE.
But the first element of m_die_array after ExtractDIEsIfNeeded(true)
may move in memory after later ExtractDIEsIfNeeded(false).
DWARFDebugInfoEntry::collection m_die_array is std::vector,
its data may move during its expansion.
Differential revision: https://reviews.llvm.org/D46810
llvm-svn: 333437
Summary:
We were treating subprograms and inlined subroutines differently when
building the index. The difference was in which indexes were individual
tags inserted (subprograms went to all indexes, where as inlined
subroutines only into the basename and full name indexes).
This seems like an error, because an inlined subroutine can still
represent an C++ or an ObjC method. I don't see anything in the
subprogram branch which should not apply to an inlined subroutine, so I
propose to just treat them identically. This makes searching for an
inlined method behave the same way as for the apple index.
I write an assembly-based test because I did not want to depend on
particular clang inlining behavior (and because I wanted to see how hard
would it be).
Reviewers: clayborg, JDevlieghere
Subscribers: eraman, lldb-commits
Differential Revision: https://reviews.llvm.org/D47368
llvm-svn: 333398
Several functions were passing a list of 8 NameToDIE arguments around.
This puts those variables in a struct and passes that instead, reducing
code duplication and the possibility of error (swapping two arguments
accidentally).
llvm-svn: 333264
Summary:
I think this makes sense for several reasons:
- better separation of concerns: DWARFUnit's job should be to provide a
nice interface to its users to access the unit contents.
ManualDWARFIndex can then use this interface to build an index and
provide it to its users.
- closer alignment with llvm parsers: there is no indexing equivalent in
llvm, and there probably never will be, as the index is very centered
around how lldb wants to access debug info. If we ever switch to
llvm's parser, this will allow us swap out DWARFUnit implementations
and keep indexing as-is.
- closer proximity of the indexing code to AppleDWARFIndex will make it
easier to keep the two in sync (e.g. right now the two use very
different algorithms to determine whether a DW_TAG_subroutine
represents a "method"). This is my primary motivation for making this
change now, but I am leaving this work to a separate patch.
The only interface change to DWARFUnit I needed to make was to add an
efficient way to iterate over the list of all DIEs. Adding this also
aligns us closer to the llvm parser.
Reviewers: JDevlieghere, clayborg, aprantl
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D47253
llvm-svn: 333178
Greg Clayton