The new module stats adds the ability to measure the time it takes to parse and index the symbol tables for each module, and reports modules statistics in the output of "statistics dump" along with the path, UUID and triple of the module. The time it takes to parse and index the symbol tables are also aggregated into new top level key/value pairs at the target level.
Differential Revision: https://reviews.llvm.org/D112279
This patch deals with ObjectFile, ObjectContainer and OperatingSystem
plugins. I'll convert the other types in separate patches.
In order to enable piecemeal conversion, I am leaving some ConstStrings
in the lowest PluginManager layers. I'll convert those as the last step.
Differential Revision: https://reviews.llvm.org/D112061
There is no reason why this function should be returning a ConstString.
While modifying these files, I also fixed several instances where
GetPluginName and GetPluginNameStatic were returning different strings.
I am not changing the return type of GetPluginNameStatic in this patch, as that
would necessitate additional changes, and this patch is big enough as it is.
Differential Revision: https://reviews.llvm.org/D111877
This reverts commits f9aba9a5af and
035217ff51.
As explained in the original commit message, this didn't have the
intended effect of improving the common LLDB use case, but still
provided a marginal improvement for the places where LLDB creates a
scoped time with a string literal.
The reason for the revert is that this change pulls in the os/signpost.h
header in Signposts.h. The former transitively includes loader.h, which
contains a series of macro defines that conflict with MachO.h. There are
ways to work around that, but Adrian and I concluded that none of them
are worth the trade-off in complicating Signposts.h even further.
A MachO userspace corefile may contain LC_THREAD commands which specify
thread exception state.
For arm64* only (for now), report a human-readable version of this state
as the thread stop reason, instead of 'SIGSTOP'.
As a follow-up, similar functionality can be implemented for x86 cores
by translating the trapno/err exception registers.
rdar://82898146
Differential Revision: https://reviews.llvm.org/D109795
A MachO userspace corefile may contain LC_THREAD commands which specify
thread exception state.
For arm64* only (for now), report a human-readable version of this state
as the thread stop reason, instead of 'SIGSTOP'.
As a follow-up, similar functionality can be implemented for x86 cores
by translating the trapno/err exception registers.
rdar://82898146
Differential Revision: https://reviews.llvm.org/D109795
When the corefile reader is adding binaries from the "all image
infos" LC_NOTE in a Mach-O corefile, it would detect if the binary
being added was an executable binary and set it as the Target's
executable binary. This has the side effect of clearing the Target's
image list, so if the executable was in the middle of the all image
infos, the initial images would be dropped. There's no need to set
the executable binary in the Target for these corefile processes,
so instead of doing multiple passes over the list to find the
executable, I'm dropping that.
In all these years, we haven't found a use for this function (it has
zero callers). Lets just remove the boilerplate.
Differential Revision: https://reviews.llvm.org/D109600
When adding a dSYM to a Module and it has different file addresses
from the already-present ObjectFile binary, change the Sections to
use the dSYM's file addresses so the symbol table and DWARF are
properly contained in the Sections. Previously this was only done
for IsInMemory ObjectFiles, but it's more common than that.
Differential Revision: https://reviews.llvm.org/D108889
rdar://81504400
These two tests, TestSkinnyCorefile.py and TestStackCorefile.py,
require a new debugserver on darwin systems to run correctly; for now,
skip them if the system debugserver is in use. There's no easy way to
test if the debugserver being used supports either of these memory
region info features. For end users, the fallback will be a full
corefile and that's not the worst thing, but for the tests it is a
problem.
Add a field to the qMemoryRegionInfo packet where the remote stub
can describe the type of memory -- heap, stack. Keep track of
memory regions that are stack memory in lldb. Add a new "--style
stack" to process save-core to request that only stack memory be
included in the corefile.
Differential Revision: https://reviews.llvm.org/D107625
Modify OpenOptions enum to open the future path into synchronizing
vFile:open bits with GDB. Currently, LLDB and GDB use different flag
models effectively making it impossible to match bits. Notably, LLDB
uses two bits to indicate read and write status, and uses union of both
for read/write. GDB uses a value of 0 for read-only, 1 for write-only
and 2 for read/write.
In order to future-proof the code for the GDB variant:
1. Add a distinct eOpenOptionReadWrite constant to be used instead
of (eOpenOptionRead | eOpenOptionWrite) when R/W access is required.
2. Rename eOpenOptionRead and eOpenOptionWrite to eOpenOptionReadOnly
and eOpenOptionWriteOnly respectively, to make it clear that they
do not mean to be combined and require update to all call sites.
3. Use the intersection of all three flags when matching against
the three possible values.
This commit does not change the actual bits used by LLDB.
Differential Revision: https://reviews.llvm.org/D106984
The pointer to the dyld trie data structure which lldb needs to parse to get
"trampoline kinds" on Darwin used to be a field in the LC_DYLD_INFO load command. A
new load command was added recently dedicated to this purpose: LC_DYLD_EXPORTS_TRIE.
The format of the trie did not change, however. So all we have to do is use the new
command if present. The commands are supposed to be mutually exclusive, so I added
an lldb_assert to warn if they are not.
Differential Revision: https://reviews.llvm.org/D107673
This is a resubmission of https://reviews.llvm.org/D105160 after fixing testing issues.
This fix was created after profiling the target creation of a large C/C++/ObjC application that contained almost 4,000,000 redacted symbol names. The symbol table parsing code was creating names for each of these synthetic symbols and adding them to the name indexes. The code was also adding the object file basename to the end of the symbol name which doesn't allow symbols from different shared libraries to share the names in the constant string pool.
Prior to this fix this was creating 180MB of "___lldb_unnamed_symbol" symbol names and was taking a long time to generate each name, add them to the string pool and then add each of these names to the name index.
This patch fixes the issue by:
not adding a name to synthetic symbols at creation time, and allows name to be dynamically generated when accessed
doesn't add synthetic symbol names to the name indexes, but catches this special case as name lookup time. Users won't typically set breakpoints or lookup these synthetic names, but support was added to do the lookup in case it does happen
removes the object file baseanme from the generated names to allow the names to be shared in the constant string pool
Prior to this fix the startup times for a large application was:
12.5 seconds (cold file caches)
8.5 seconds (warm file caches)
After this fix:
9.7 seconds (cold file caches)
5.7 seconds (warm file caches)
The names of the symbols are auto generated by appending the symbol's UserID to the end of the "___lldb_unnamed_symbol" string and is only done when the name is requested from a synthetic symbol if it has no name.
Differential Revision: https://reviews.llvm.org/D106837
This patch adds code to process save-core for Mach-O files which
embeds an "addrable bits" LC_NOTE when the process is using a
code address mask (e.g. AArch64 v8.3 with ptrauth aka arm64e).
Add code to ObjectFileMachO to read that LC_NOTE from corefiles,
and ProcessMachCore to set the process masks based on it when reading
a corefile back in.
Also have "process status --verbose" print the current address masks
that lldb is using internally to strip ptrauth bits off of addresses.
Differential Revision: https://reviews.llvm.org/D106348
rdar://68630113
This fix was created after profiling the target creation of a large C/C++/ObjC application that contained almost 4,000,000 redacted symbol names. The symbol table parsing code was creating names for each of these synthetic symbols and adding them to the name indexes. The code was also adding the object file basename to the end of the symbol name which doesn't allow symbols from different shared libraries to share the names in the constant string pool.
Prior to this fix this was creating 180MB of "___lldb_unnamed_symbol" symbol names and was taking a long time to generate each name, add them to the string pool and then add each of these names to the name index.
This patch fixes the issue by:
- not adding a name to synthetic symbols at creation time, and allows name to be dynamically generated when accessed
- doesn't add synthetic symbol names to the name indexes, but catches this special case as name lookup time. Users won't typically set breakpoints or lookup these synthetic names, but support was added to do the lookup in case it does happen
- removes the object file baseanme from the generated names to allow the names to be shared in the constant string pool
Prior to this fix the startup times for a large application was:
12.5 seconds (cold file caches)
8.5 seconds (warm file caches)
After this fix:
9.7 seconds (cold file caches)
5.7 seconds (warm file caches)
The names of the symbols are auto generated by appending the symbol's UserID to the end of the "___lldb_unnamed_symbol" string and is only done when the name is requested from a synthetic symbol if it has no name.
Differential Revision: https://reviews.llvm.org/D105160
Reverts commits:
"Fix failing tests after https://reviews.llvm.org/D104488."
"Fix buildbot failure after https://reviews.llvm.org/D104488."
"Create synthetic symbol names on demand to improve memory consumption and startup times."
This series of commits broke the windows lldb bot and then failed to fix all of the failing tests.
This fix was created after profiling the target creation of a large C/C++/ObjC application that contained almost 4,000,000 redacted symbol names. The symbol table parsing code was creating names for each of these synthetic symbols and adding them to the name indexes. The code was also adding the object file basename to the end of the symbol name which doesn't allow symbols from different shared libraries to share the names in the constant string pool.
Prior to this fix this was creating 180MB of "___lldb_unnamed_symbol" symbol names and was taking a long time to generate each name, add them to the string pool and then add each of these names to the name index.
This patch fixes the issue by:
- not adding a name to synthetic symbols at creation time, and allows name to be dynamically generated when accessed
- doesn't add synthetic symbol names to the name indexes, but catches this special case as name lookup time. Users won't typically set breakpoints or lookup these synthetic names, but support was added to do the lookup in case it does happen
- removes the object file baseanme from the generated names to allow the names to be shared in the constant string pool
Prior to this fix the startup times for a large application was:
12.5 seconds (cold file caches)
8.5 seconds (warm file caches)
After this fix:
9.7 seconds (cold file caches)
5.7 seconds (warm file caches)
The names of the symbols are auto generated by appending the symbol's UserID to the end of the "___lldb_unnamed_symbol" string and is only done when the name is requested from a synthetic symbol if it has no name.
Differential Revision: https://reviews.llvm.org/D104488
Add a new feature to process save-core on Darwin systems -- for
lldb to create a user process corefile with only the dirty (modified
memory) pages included. All of the binaries that were used in the
corefile are assumed to still exist on the system for the duration
of the use of the corefile. A new --style option to process save-core
is added, so a full corefile can be requested if portability across
systems, or across time, is needed for this corefile.
debugserver can now identify the dirty pages in a memory region
when queried with qMemoryRegionInfo, and the size of vm pages is
given in qHostInfo.
Create a new "all image infos" LC_NOTE for Mach-O which allows us
to describe all of the binaries that were loaded in the process --
load address, UUID, file path, segment load addresses, and optionally
whether code from the binary was executing on any thread. The old
"read dyld_all_image_infos and then the in-memory Mach-O load
commands to get segment load addresses" no longer works when we
only have dirty memory.
rdar://69670807
Differential Revision: https://reviews.llvm.org/D88387
One nice feature of the os_signpost API is that format string
substitutions happen in the consumer, not the logging
application. LLVM's current Signpost class doesn't take advantage of
this though and instead always uses a static "Begin/End %s" format
string.
This patch uses variadic macros to allow the API to be used as
intended. Unfortunately, the primary use-case I had in mind (the
LLDB_SCOPED_TIMER() macro) does not get much better from this, because
__PRETTY_FUNCTION__ is *not* a macro, but a static string, so
signposts created by LLDB_SCOPED_TIMER() still use a static "%s"
format string. At least LLDB_SCOPED_TIMERF() works as intended.
This reapplies the previously reverted patch with additional include
order fixes for non-modular builds of LLDB.
Differential Revision: https://reviews.llvm.org/D103575
One nice feature of the os_signpost API is that format string
substitutions happen in the consumer, not the logging
application. LLVM's current Signpost class doesn't take advantage of
this though and instead always uses a static "Begin/End %s" format
string.
This patch uses variadic macros to allow the API to be used as
intended. Unfortunately, the primary use-case I had in mind (the
LLDB_SCOPED_TIMER() macro) does not get much better from this, because
__PRETTY_FUNCTION__ is *not* a macro, but a static string, so
signposts created by LLDB_SCOPED_TIMER() still use a static "%s"
format string. At least LLDB_SCOPED_TIMERF() works as intended.
This reapplies the previsously reverted patch with additional MachO.h
macro #undefs.
Differential Revision: https://reviews.llvm.org/D103575
Unfortunately the Darwin signpost header also pulls in the system
MachO header and so we need to make sure to use the LLVM versions of
those definitions.
Some larger projects were loading quite slowly with the current LLDB on macOS and macOS simulator builds. I did some instrument traces and found 3 main culprits:
- a LLDB timer that was put into a function that was called too often
- a std::set that was keeping track of the address of symbols that were already added
- a unnamed function generator in ObjectFile that was going slow due to allocations
In order to see this in action I ran the latest LLDB on a large application with many frameworks using the following method:
(lldb) script import time; start_time = time.perf_counter()
(lldb) file Large.app
(lldb) script print(time.perf_counter() - start_time)
I first range "sudo purge" to clear the system file caches to simulate a cold startup of the debugger, followed by two iterations with warm file caches.
Prior to this fix I was seeing the following timings:
17.68 (cold)
14.56 (warm 1)
14.52 (warm 2)
After this fix I was seeing:
11.32 (cold)
8.43 (warm 1)
8.49 (warm 2)
Differential Revision: https://reviews.llvm.org/D103504
LLDB can often appear deadlocked to users that use IDEs when it is indexing DWARF, or parsing symbol tables. These long running operations can make a debug session appear to be doing nothing even though a lot of work is going on inside LLDB. This patch adds a public API to allow clients to listen to debugger events that report progress and will allow UI to create an activity window or display that can show users what is going on and keep them informed of expensive operations that are going on inside LLDB.
Differential Revision: https://reviews.llvm.org/D97739
This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to the
LLDB_LOG(F) macro.
Differential revision: https://reviews.llvm.org/D93663
Commit f3aa9e36d9 fixed the embedded OS
build by removing all passed args for `GetName`/`GetDemangledName`. The motivation
for this was that these arguments were apparently removed in
commit 22b044877d. However, only `GetName`'s language
argument was removed but the mangling preference argument was *not* removed
(and unfortunately had a default argument). So when that commit removed all
the args it didn't just fix the build but it also changed all the mangling
preferences to 'demangled' for all `GetName` calls.
Also some `GetName` calls were outside the TARGET_OS_EMBEDDED ifdef, so
this change ended up breaking the following tests on macOS:
lldb-api :: lang/objc/objc-static-method-stripped/TestObjCStaticMethodStripped.py
lldb-api :: lang/objc/objc-super/TestObjCSuper.py
From what I can see f3aa9e36d9 removed 12 ePreferMangled args and this patch
re-adds 12 args with roughly the same line numbers, so this *should* restore the
old behaviour and also keep the embedded build working. On the other hand,
ObjectFileMachO::ParseSymtab is a very successful attempt at writing
the longest possible function within LLVM, so this fix is partly based
on the engineering principle known as "hoping for the best".
When a Mach-O corefile has an LC_NOTE "main bin spec" for a
standalone binary / firmware, with only a UUID and no load
address, try to locate the binary and dSYM by UUID and if
found, load it at offset 0 for the user.
Add a test case that tests a firmware/standalone corefile
with both the "kern ver str" and "main bin spec" LC_NOTEs.
<rdar://problem/68193804>
Differential Revision: https://reviews.llvm.org/D88282
Checking if an object file is in memory should use the ObjectFile::IsInMemory(), not test ObjectFile::BaseAddress(). ObjectFile::BaseAddress() is designed to be overridden by all classes and is for mach-o, ELF and COFF plug-ins. They find the header base adddress and return that as a section offset address. The default implementation of ObjectFile::BaseAddress() does try and make an Address() from the ObjectFile::m_memory_addr, but I switched it to a correct function call.
Differential Revision: https://reviews.llvm.org/D86122
The code in ObjectFileMachO didn't disambiguate between ios and
ios-simulator object files for Mach-O objects using the legacy
ambiguous LC_VERSION_MIN load commands. This used to not matter before
taught ArchSpec that ios and ios-simulator are no longer compatible.
rdar://problem/66545307
Differential Revision: https://reviews.llvm.org/D85358
Summary:
The resolver addresses stored in the dyld trie are relative to the base
of the __TEXT segment. This is usually 0 in a dylib, so this was never
noticed, but it is not 0 for most dylibs integrated in the shared cache.
As we started using the shared cache images recently as symbol source,
this causes LLDB to fail to resolve symbols which go through a runtime
resolver.
Reviewers: jasonmolenda, jingham
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D84083
Summary:
On macOS 11, the libraries that have been integrated in the system
shared cache are not present on the filesystem anymore. LLDB was
using those files to get access to the symbols of those libraries.
LLDB can get the images from the target process memory though.
This has 2 consequences:
- LLDB cannot load the images before the process starts, reporting
an error if someone tries to break on a system symbol.
- Loading the symbols by downloading the data from the inferior
is super slow. It takes tens of seconds at the start of the
debug session to populate the Module list.
To fix this, we can use the library images LLDB has in its own
mapping of the shared cache. Shared cache images are somewhat
special as their LINKEDIT segment is moved to the end of the cache
and thus the images are not contiguous in memory. All of this can
hidden in ObjectFileMachO.
This patch fixes a number of test failures on macOS 11 due to the
first problem described above and adds some specific unittesting
for the new SharedCache Host utilities.
Reviewers: jasonmolenda, labath
Subscribers: llvm-commits, lldb-commits
Tags: #lldb, #llvm
Differential Revision: https://reviews.llvm.org/D83023
Greg Clayton a few years ago.
My patch to augment the symbol table in Mach-O files with the
dyld trie exports data structure only categorized symbols as code
or data, but Greg Clayton had a patch to do something similar to
swift a few years ago that had a more extensive categorization of
symbols, as well as extracting some objc class/ivar names from the
entries. This patch is basically just Greg's, updated a bit and
with a test case added to it.
<rdar://problem/50791451>
Differential Revision: https://reviews.llvm.org/D77369
In ObjectFileMachO we construct the symbol table from multiple
sources -- primarily the binary's nlist records, but when the nlist
symbols have been stripped, we would augment those with function
start address from the LC_FUNCTION_STARTS or eh_frame. This patch
adds another source of symbols - the exported symbols that the
dynamic linker, dyld, uses at runtime from its trie structure. This
provides us names and addresses for these functions/data.
This patch removes the code from ParseSymtab that would reject an
empty symbol table / nlist source. It adds a new symbols_added
set which tracks the address of every symbol we've added to the
symtab. We add symbols in most-information-ful order, and before
adding a symbol from less-informational-ful source (e.g.
LC_FUNCTION_STARTS with no function name), we check if that symbol
has already been added.
On targets with thumb code generation, instead of using the 0th bit
in these addresses in FunctionStarts (or now the trie entries), we
use the data field of FunctionStarts (formerly used to track if the
func_start should be added) and a flag for the trie entries to
encode this, and only store the actual addresses in the symbols_seen
and these vectors.
<rdar://problem/50791451>
Differential revision: https://reviews.llvm.org/D76758
The two classes are equivalent, except:
- the former uses a llvm::SmallVector (with a configurable size), while
the latter uses std::vector.
- the former has a typo in one of the functions name
This patch just leaves one class, using llvm::SmallVector, and defaults
the small size to zero. This is the same thing we did with the
RangeDataVector class in D56170.
Greg Clayton