At the moment the expression parser doesn't support evaluating expressions in
static member functions and just pretends the expression is evaluated within a
non-member function. This causes that all static members are inaccessible when
doing unqualified name lookup.
This patch adds support for evaluating in static member functions. It
essentially just does the same setup as what LLDB is already doing for
non-static member functions (i.e., wrapping the expression in a fake member
function) with the difference that we now mark the wrapping function as static
(to prevent access to non-static members).
Reviewed By: shafik, jarin
Differential Revision: https://reviews.llvm.org/D81550
LLDB uses utility functions to run code in the inferior for its own
internal purposes, such as reading classes from the Objective-C runtime
for example. Because these expressions should be transparent to the
user, we ignore breakpoints and unwind the stack on errors, which
makes them hard to debug.
This patch adds a new setting target.debug-utility-expression that, when
enabled, changes these options to facilitate debugging. It enables
breakpoints, disables unwinding and writes out the utility function
source code to disk so it shows up in the source view.
Differential revision: https://reviews.llvm.org/D97249
LLDB tracks where any imported `clang::Decl` originally came from via a simple
map from 'imported decl' to 'original decl'. That information is used to later
complete parts of the Decl when more information is requested about a certain
Decl (e.g., via the ExternalASTSource interface from Clang).
When finding the 'original decl' for a given decl, the ASTImporterDelegate
essentially just recursively follows the previously mentioned map from
'imported' to 'original decl' until it can find any further 'original decl'. The
final found decl is then the one that will be imported. The recursion is
necessary as in LLDB we don't just import decls from one ASTContext to another,
but also from one ASTContext to another via a (potentially temporary)
ASTContext. For example, the expression parser creates a temporary ASTContext
for parsing the current expression.
The problem with the recursion is however that if we somehow get a cycle into
our mapping, then the ASTImporterDelegate will just infinite recurse. As the
infinite recursion usually happens after the cycle was already created in a code
path such as completing a type, the crash backtraces we get for these bugs are
not very useful. However having the backtrace where the faulty map entry is
created usually makes the code trivial to fix (as there should be some rogue
CopyType call or something similar nearby. See for example D96366).
This patch tries to make these issues easier to track down by putting a bunch of
sanity asserts in the code that fills out the map. All the asserts are just
checking that there is no direct cycle (ASTContext maps to itself) when updating
the origin tracking map.
The assert in the ASTImportDelegate constructor is an `lldbassert` (which also
is getting checked in release builds with disabled asserts) as the code path
there is pretty cold and we can reliably detect a rogue CopyType call from
there.
I also had to update some code in
`ClangASTImporter::ASTImporterDelegate::Imported`. This code already had a
safety check for creating a cycle in the origin tracking map, but it still
constructed an ASTImporter while checking for the cycle (by requesting a
delegate via `GetDelegate` and passing two identical ASTContexts which looks
like a rogue CopyType call to the checks).
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D97300
The comment for ValueType claims that all values <1 are errors, but
not all switch statements take this into account. This patch
introduces an explicit Error case and deletes all default: cases, so
we get warned about incomplete switch coverage.
https://reviews.llvm.org/D96537
It looks like a previous change switched these from LLDB_LOGF but did not update the format strings.
Differential Revision: https://reviews.llvm.org/D96550
Clang emits a warning when accessing an Objective-C getter but not using the result.
This gets triggered when just trying to print a getter value in the expression parser (where
Clang just sees a normal expression like `obj.getter` while parsing).
This patch just disables the warning in the expression parser (similar to what we do with
the C++ equivalent of just accessing a member variable but not doing anything with it).
Reviewed By: kastiglione
Differential Revision: https://reviews.llvm.org/D94307
Identical to previous commits that just add a standard library template to the
supported template list and test it. Adding this rather obscure class to the
template list is mostly caused by the std::deque test unexpectedly referencing
this type when testing against newer libc++ versions on macOS.
Fixes TestQueueFromStdModule and TestQueueFromStdModule on macOS.
Fixes rdar://73213589
Migrate to the `FileEntryRef` overload of `SourceManager::createFileID`
(using `FileManager::getOptionalFileRef`) in
`ClangExpressionParser::ParseInternal`.
No functionality change here.
Differential Revision: https://reviews.llvm.org/D92957
Currently when LLDB has enough data in the debug information to import the `std` module,
it will just try to import it. However when debugging libraries where the sources aren't
available anymore, importing the module will generate a confusing diagnostic that
the module couldn't be built.
For the fallback mode (where we retry failed expressions with the loaded module), this
will cause the second expression to fail with a module built error instead of the
actual parsing issue in the user expression.
This patch adds checks that ensures that we at least have any source files in the found
include paths before we try to import the module. This prevents the module from being
loaded in the situation described above which means we don't emit the bogus 'can't
import module' diagnostic and also don't waste any time retrying the expression in the
fallback mode.
For the unit tests I did some refactoring as they now require a VFS with the files in it
and not just the paths. The Python test just builds a binary with a fake C++ module,
then deletes the module before debugging.
Fixes rdar://73264458
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D95096
Replace uses of GetModuleAtIndexUnlocked and
GetModulePointerAtIndexUnlocked with the ModuleIterable and
ModuleIterableNoLocking where applicable.
Differential revision: https://reviews.llvm.org/D94271
7ad49aec12 added a __memory subdirectory to libc++
but the code we use to find libc++ from the debug info support files wasn't
prepared to encounter unknown subdirectories within libc++. The import-std-module
tests automatically fell back to not importing the std module which caused
them to fail.
This patch removes our hardcoded exception for the 'experimental' subdirectory
and instead just ignores all subdirectories of c++/vX/ when searching the
support files.
Right now we have one large AST for all types in LLDB. All ODR violations in
types we reconstruct are resolved by just letting the ASTImporter handle the
conflicts (either by merging types or somehow trying to introduce a duplicated
declaration in the AST). This works ok for the normal types we build from debug
information as most of them are just simple CXXRecordDecls or empty template
declarations.
However, with a loaded `std` C++ module we have alternative versions of pretty
much all declarations in the `std` namespace that are much more fleshed out than
the debug information declarations. They have all the information that is lost
when converting to DWARF, such as default arguments, template default arguments,
the actual uninstantiated template declarations and so on.
When we merge these C++ module types into the big scratch AST (that might
already contain debug information types) we give the ASTImporter the tricky task
of somehow creating a consistent AST out of all these declarations. Usually this
ends in a messy AST that contains a mostly broken mix of both module and debug
info declarations. The ASTImporter in LLDB is also importing types with the
MinimalImport setting, which usually means the only information we have when
merging two types is often just the name of the declaration and the information
that it contains some child declarations. This makes it pretty much impossible
to even implement a better merging logic (as the names of C++ module
declarations and debug info declarations are identical).
This patch works around this whole merging problem by separating C++ module
types from debug information types. This is done by splitting up the single
scratch AST into two: One default AST for debug information and a dedicated AST
for C++ module types.
The C++ module AST is implemented as a 'specialised AST' that lives within the
default ScratchTypeSystemClang. When we select the scratch AST we can explicitly
request that we want such a isolated sub-AST of the scratch AST. I kept the
infrastructure more general as we probably can use the same mechanism for other
features that introduce conflicting types (such as programs that are compiled
with a custom -wchar-size= option).
There are just two places where we explicitly have request the C++ module AST:
When we export persistent declarations (`$mytype`) and when we create our
persistent result variable (`$0`, `$1`, ...). There are a few formatters that
were previously assuming that there is only one scratch AST which I cleaned up
in a preparation revision here (D92757).
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D92759
By now LLDB can import the 'std' C++ module to improve expression evaluation,
but there are still a few problems to solve before we can do this by default.
One is that importing the C++ module is slightly slower than normal expression
evaluation (mostly because the disk access and loading the initial lookup data
is quite slow in comparison to the barebone Clang setup the rest of the LLDB
expression evaluator is usually doing). Another problem is that some complicated
types in the standard library aren't fully supported yet by the ASTImporter, so
we end up types that fail to import (which usually appears to the user as if the
type is empty or there is just no result variable).
To still allow people to adopt this mode in their daily debugging, this patch
adds a setting that allows LLDB to automatically retry failed expression with a
loaded C++ module. All success expressions will behave exactly as they would do
before this patch. Failed expressions get a another parse attempt if we find a
usable C++ module in the current execution context. This way we shouldn't have
any performance/parsing regressions in normal debugging workflows, while the
debugging workflows involving STL containers benefit from the C++ module type
info.
This setting is off by default for now with the intention to enable it by
default on macOS soon-ish.
The implementation is mostly just extracting the existing parse logic into its
own function and then calling the parse function again if the first evaluation
failed and we have a C++ module to retry the parsing with.
Reviewed By: shafik, JDevlieghere, aprantl
Differential Revision: https://reviews.llvm.org/D92784
LLDB is currently always activating C++ when parsing expressions as LLDB itself
is using C++ features when creating the final AST that will be codegen'd
(specifically, references to variables, namespaces and using declarations are
used).
This is causing problems for users that have variables in non-C++ programs (e.g.
plain C or Objective-C) that have names which are keywords in C++. Expressions
referencing those variables fail to parse as LLDB's Clang parser thinks those
identifiers are C++ keywords and not identifiers that may belong to a
declaration.
We can't just disable C++ in the expression parser for those situations as
replacing the functionality of the injected C++ code isn't trivial. So this
patch is just disabling most keywords that are exclusive to C++ in LLDB's Clang
parser when we are in a non-C++ expression. There are a few keywords we can't
disable for now:
* `using` as that's currently used in some situations to inject variables into the expression function.
* `__null` as that's used by LLDB to define `NULL`/`Nil`/`nil`.
Getting rid of these last two keywords is possible but is a large enough change
that this will be handled in follow up patches.
Note that this only changes the keyword status of those tokens but this patch
does not remove any C++ functionality from the expression parser. The type
system still follows C++ rules and so does the rest of the expression parser.
There is another small change that gives the hardcoded macro definitions in LLDB
a higher precedence than the macros imported from the Objective-C modules. The
reason for this is that the Objective-C modules in LLDB are actually parsed in
Objective-C++ mode and they end up providing the C++ definitions of certain
system macros (like `NULL` being defined as `nullptr`). So we have to move the
LLDB definition forward and surround the definition from the module with an
`#ifdef` to make sure that we use the correct LLDB definition that doesn't
reference C++ keywords. Or to give an example, this is how the expression source
code changes:
Before:
```
#define NULL (nullptr) // injected module definition
#ifndef NULL
#define NULL (__null) // hardcoded LLDB definition
#endif
```
After:
```
#ifndef NULL
#define NULL (__null) // hardcoded LLDB definition
#endif
#ifndef NULL
#define NULL (nullptr) // injected module definition
#endif
```
Fixes rdar://10356912
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D82770
For performance reasons the reproducers don't copy the files captured by
the file collector eagerly, but wait until the reproducer needs to be
generated.
This is a problematic when LLDB crashes and we have to do all this
signal-unsafe work in the signal handler. This patch uses a similar
trick to clang, which has the driver invoke a new cc1 instance to do all
this work out-of-process.
This patch moves the writing of the mapping file as well as copying over
the reproducers into a separate process spawned when lldb crashes.
Differential revision: https://reviews.llvm.org/D89600
This patch redesigns the Target::GetUtilityFunctionForLanguage API:
- Use a unique_ptr instead of a raw pointer for the return type.
- Wrap the result in an llvm::Expected instead of using a Status object as an I/O parameter.
- Combine the action of "getting" and "installing" the UtilityFunction as they always get called together.
- Pass std::strings instead of const char* and std::move them where appropriate.
There's more room for improvement but I think this tackles the most
prevalent issues with the current API.
Differential revision: https://reviews.llvm.org/D90011
The UtilityFunction ctor was dropping the text argument. Probably for
that reason ClangUtilityFunction was setting the parent's member
directly instead of deferring to the parent ctor. Also change the
signatures to take strings which are std::moved in place.
Currently one can redefine a persistent variable and LLDB will just silently
ignore the second definition:
```
(lldb) expr int $i = 1
(lldb) expr int $i = 2
(lldb) expr $i
(int) $i = 1
```
This patch makes this an error and rejects the expression with the second
definition.
A nice follow up would be to refactor LLDB's persistent variables to not just be
a pair of type and name, but also contain some way to obtain the original
declaration and source code that declared the variable. That way we could
actually make a full diagnostic as we would get from redefining a variable twice
in the same expression.
Reviewed By: labath, shafik, JDevlieghere
Differential Revision: https://reviews.llvm.org/D89310
While debugging another bug I found out that we currently don't set any limit
for the number of diagnostics Clang emits. If a user does something that
generates a lot of errors (like including some long header file from within the
expression function), then we currently spam the LLDB output with potentially
thousands of Clang error diagnostics.
Clang sets a default limit of 20 errors, but given that LLDB is often used
interactively for small expressions I would say a limit of 5 is enough. The
limit is implemented as a setting, so if a user cares about seeing having a
million errors printed to their terminal then they can just increase the
settings value.
Reviewed By: shafik, mib
Differential Revision: https://reviews.llvm.org/D88889
Both of the if-clauses modify the raw_path variable and only one of them
was resetting the variable for the fallback. Avoid future bugs like that
by always resetting the variable.
Differential revision: https://reviews.llvm.org/D88704
This adds support for substituting std::pair instantiations with enabled
import-std-module.
With the fixes in parent revisions we can currently substitute a single pair
(however, a result that returns a second pair currently causes LLDB to crash
while importing the second template instantiation).
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D85141
The ASTImporter has an `Imported(From, To)` callback that notifies subclasses
that a declaration has been imported in some way. LLDB uses this in the
`CompleteTagDeclsScope` to see which records have been imported into the scratch
context. If the record was declared inside the expression, then the
`CompleteTagDeclsScope` will forcibly import the full definition of that record
to the scratch context so that the expression AST can safely be disposed later
(otherwise we might end up going back to the deleted AST to complete the
minimally imported record). The way this is implemented is that there is a list
of decls that need to be imported (`m_decls_to_complete`) and we keep completing
the declarations inside that list until the list is empty. Every `To` Decl we
get via the `Imported` callback will be added to the list of Decls to be
completed.
There are some situations where the ASTImporter will actually give us two
`Imported` calls with the same `To` Decl. One way where this happens is if the
ASTImporter decides to merge an imported definition into an already imported
one. Another way is that the ASTImporter just happens to get two calls to
`ASTImporter::Import` for the same Decl. This for example happens when importing
the DeclContext of a Decl requires importing the Decl itself, such as when
importing a RecordDecl that was declared inside a function.
The bug addressed in this patch is that when we end up getting two `Imported`
calls for the same `To` Decl, then we would crash in the
`CompleteTagDeclsScope`. That's because the first time we complete the Decl we
remove the Origin tracking information (that maps the Decl back to from where it
came from). The next time we try to complete the same `To` Decl the Origin
tracking information is gone and we hit the `to_context_md->getOrigin(decl).ctx
== m_src_ctx` assert (`getOrigin(decl).ctx` is a nullptr the second time as the
Origin was deleted).
This is actually a regression coming from D72495. Before D72495
`m_decls_to_complete` was actually a set so every declaration in there could
only be queued once to be completed. The set was changed to a vector to make the
iteration over it deterministic, but that also causes that we now potentially
end up trying to complete a Decl twice.
This patch essentially just reverts D72495 and makes the `CompleteTagDeclsScope`
use a SetVector for the list of declarations to be completed. The SetVector
should filter out the duplicates (as the original `set` did) and also ensure that
the completion order is deterministic. I actually couldn't find any way to cause
LLDB to reproduce this bug by merging declarations (this would require that we
for example declare two namespaces in a non-top-level expression which isn't
possible). But the bug reproduces very easily by just declaring a class in an
expression, so that's what the test is doing.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D85648
SemaSourceWithPriorities is a special SemaSource that wraps our normal LLDB
ExternalASTSource and the ASTReader (which is used for the C++ module loading).
It's only active when the `import-std-module` setting is turned on.
The `CompleteType` function there in `SemaSourceWithPriorities` is looping over
all ExternalASTSources and asks each to complete the type. However, that loop is
in another loop that keeps doing that until the type is complete. If that
function is ever called on a type that is a forward decl then that causes LLDB
to go into an infinite loop.
I remember I added that second loop and the comment because I thought I saw a
similar pattern in some other Clang code, but after some grepping I can't find
that code anywhere and it seems the rest of the code base only calls
CompleteType once (It would also be kinda silly to have calling it multiple
times). So it seems that's just a silly mistake.
The is implicitly tested by importing `std::pair`, but I also added a simpler
dedicated test that creates a dummy libc++ module with some forward declarations
and then imports them into the scratch AST context. At some point the
ASTImporter will check if one of the forward decls could be completed by the
ExternalASTSource, which will cause the `SemaSourceWithPriorities` to go into an
infinite loop once it receives the `CompleteType` call.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D87289
Extract all the provider related logic from Reproducer.h and move it
into its own header ReproducerProvider.h. These classes are seeing most
of the development these days and this reorganization reduces
incremental compilation from ~520 to ~110 files when making changes to
the new header.
The search for the complete class definition can also produce entries
which are not of the expected type. This can happen for instance when
there is a function with the same name as the class we're looking up
(which means that the class needs to be disambiguated with the
struct/class tag in most contexts).
Previously we were just picking the first Decl that the lookup returned,
which later caused crashes or assertion failures if it was not of the
correct type. This patch changes that to search for an entry of the
correct type.
Differential Revision: https://reviews.llvm.org/D85904
This patch has no effect for C and C++. In more dynamic languages,
such as Objective-C and Swift GetByteSize() needs to call into the
language runtime, so it's important to pass one in where possible. My
primary motivation for this is some work I'm doing on the Swift
branch, however, it looks like we are also seeing warnings in
Objective-C that this may resolve. Everything in the SymbolFile
hierarchy still passes in nullptrs, because we don't have an execution
context in SymbolFile, since SymbolFile transcends processes.
Differential Revision: https://reviews.llvm.org/D84267
The `intrinsics_gen` target exists in the CMake exports since r309389
(see LLVMConfig.cmake.in), hence projects can depend on `intrinsics_gen`
even it they are built separately from LLVM.
Reviewed By: MaskRay, JDevlieghere
Differential Revision: https://reviews.llvm.org/D83454
The passthrough DiagnosticConsumer is an implementation detail of
ClangDiagnosticManagerAdapter and we can just hide it behind the normal
DiagnosticConsumer interface that ClangDiagnosticManagerAdapter is supposed
to implement.
Summary:
This patch adds support for evaluation of expressions referring to types
which were compiled in -flimit-debug-info (a.k.a -fno-standalone-debug)
in clang. In this mode it's possible that the debug information needed
to fully describe a c++ type is not present in a single shared library
-- for example debug info for a base class or a member of a type can
only be found in another shared library. This situation is not
currently handled well within lldb as we are limited to searching within
a single shared library (lldb_private::Module) when searching for the
definition of these types.
The way that this patch gets around this limitation is by doing the
search at a later stage -- during the construction of the expression ast
context. This works by having the parser (currently SymbolFileDWARF, but
a similar approach is probably needed for PDBs too) mark a type as
"forcefully completed". What this means is that the parser has marked
the type as "complete" in the module ast context (as this is necessary
to e.g. derive classes from it), but its definition is not really there.
This is done via a new field on the ClangASTMetadata struct.
Later, when we are importing such a type into the expression ast, we
check this flag. If the flag is set, we try to find a better definition
for the type in other shared libraries. We do this by initiating a
new lookup for the "forcefully completed" classes, which then imports the
type from a module with a full definition.
This patch only implements this handling for base classes, but other
cases (members, array element types, etc.). The changes for that should
be fairly simple and mostly revolve around marking these types as
"forcefully completed" at an approriate time -- the importing logic is
generic already.
Another aspect, which is also not handled by this patch is viewing these
types via the "frame variable" command. This does not use the AST
importer and so it will need to handle these types on its own -- that
will be the subject of another patch.
Differential Revision: https://reviews.llvm.org/D81561
Jordan Rupprecht