This is a re-submission of 24d2405588
without the hunks in HostNativeThreadBase.{h,cpp}, which break builds
on Windows.
Identified with modernize-use-nullptr.
This reverts commit 913457acf0.
It again broke builds on Windows:
lldb/source/Host/common/HostNativeThreadBase.cpp(37,14): error:
assigning to 'lldb::thread_result_t' (aka 'unsigned int') from
incompatible type 'std::nullptr_t'
This is a re-submission of 24d2405588
without the hunk in HostNativeThreadBase.h, which breaks builds on
Windows.
Identified with modernize-use-nullptr.
This reverts commit 24d2405588.
Breaks building on Windows:
../../lldb/include\lldb/Host/HostNativeThreadBase.h(49,36): error:
cannot initialize a member subobject of type 'lldb::thread_result_t'
(aka 'unsigned int') with an rvalue of type 'std::nullptr_t'
lldb::thread_result_t m_result = nullptr;
^~~~~~~
1 error generated.
Since quite a while Apple's LLDB fork (that contains the Swift debugging
support) is randomly crashing in `CommandLineParser::addOption` with an error
such as `CommandLine Error: Option 'h' registered more than once!`
The backtrace of the crashing thread is shown below. There are also usually many
other threads also performing similar clang::FrontendActions which are all
trying to generate (usually outdated) Clang modules which are used by Swift for
various reasons.
```
[ 6] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, llvm:🆑:SubCommand*) + 856
[ 7] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, llvm:🆑:SubCommand*) + 733
[ 8] LLDB`CommandLineParser::addOption(llvm:🆑:Option*, bool) + 184
[ 9] LLDB`llvm:🆑:ParseCommandLineOptions(...) [inlined] ::CommandLineParser::ParseCommandLineOptions(... + 1279
[ 9] LLDB`llvm:🆑:ParseCommandLineOptions(...) + 497
[ 10] LLDB`setCommandLineOpts(clang::CodeGenOptions const&) + 416
[ 11] LLDB`EmitAssemblyHelper::EmitAssemblyWithNewPassManager(...) + 98
[ 12] LLDB`clang::EmitBackendOutput(...) + 4580
[ 13] LLDB`PCHContainerGenerator::HandleTranslationUnit(clang::ASTContext&) + 871
[ 14] LLDB`clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) + 43
[ 15] LLDB`clang::ParseAST(clang::Sema&, bool, bool) + 579
[ 16] LLDB`clang::FrontendAction::Execute() + 74
[ 17] LLDB`clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 1808
```
The underlying reason for the crash is that the CommandLine code in LLVM isn't
thread-safe and will never be thread-safe with its current architecture. The way
LLVM's CommandLine logic works is that all parts of the LLVM can provide command
line arguments by defining `cl::opt` global variables and their constructors
(which are invoked during static initialisation) register the variable in LLVM's
CommandLineParser (which is also just a global variable). At some later point
after static initialization we actually try to parse command line arguments and
we ask the CommandLineParser to parse our `argv`. The CommandLineParser then
lazily constructs it's internal parsing state in a non-thread-safe way (this is
where the crash happens), parses the provided command line and then goes back to
the respective `cl::opt` global variables and sets their values according to the
parse result.
As all of this is based on global state, this whole mechanism isn't thread-safe
so the only time to ever use it is when we know we only have one active thread
dealing with LLVM logic. That's why nearly all callers of
`llvm:🆑:ParseCommandLineOptions` are at the top of the `main` function of the
some LLVM-based tool. One of the few exceptions to this rule is in the
`setCommandLineOpts` function in `BackendUtil.cpp` which is in our backtrace:
```
static void setCommandLineOpts(const CodeGenOptions &CodeGenOpts) {
SmallVector<const char *, 16> BackendArgs;
BackendArgs.push_back("clang"); // Fake program name.
if (!CodeGenOpts.DebugPass.empty()) {
BackendArgs.push_back("-debug-pass");
BackendArgs.push_back(CodeGenOpts.DebugPass.c_str());
}
if (!CodeGenOpts.LimitFloatPrecision.empty()) {
BackendArgs.push_back("-limit-float-precision");
BackendArgs.push_back(CodeGenOpts.LimitFloatPrecision.c_str());
}
BackendArgs.push_back(nullptr);
llvm:🆑:ParseCommandLineOptions(BackendArgs.size() - 1,
BackendArgs.data());
}
```
This is trying to set `cl::opt` variables in the LLVM backend to their right
value as the passed via CodeGenOptions by invoking the CommandLine parser. As
this is just in some generic Clang CodeGen code (where we allow having multiple
threads) this is code is clearly wrong. If we're unlucky it either overwrites
the value of the global variables or it causes the CommandLine parser to crash.
So the next question is why is this only crashing in LLDB? The main reason seems
to be that easiest way to crash this code is to concurrently enter the initial
CommandLineParser construction where it tries to collect all the registered
`cl::opt` options and checks for sanity:
```
// If it's a DefaultOption, check to make sure it isn't already there.
if (O->isDefaultOption() &&
SC->OptionsMap.find(O->ArgStr) != SC->OptionsMap.end())
return;
// Add argument to the argument map!
if (!SC->OptionsMap.insert(std::make_pair(O->ArgStr, O)).second) {
errs() << ProgramName << ": CommandLine Error: Option '" << O->ArgStr
<< "' registered more than once!\n";
HadErrors = true;
}
```
The `OptionsMap` here is global variable and if we end up in this code with two
threads at once then two threads at the same time can register an option (such
as 'h') when they pass the first `if` and then we fail with the sanity check in
the second `if`.
After this sanity check and initial setup code the only remaining work is just
parsing the provided CommandLine which isn't thread-safe but at least doesn't
crash in all my attempts at breaking it (as it's usually just reading from the
already generated parser state but not further modifying it). The exception to
this is probably that once people actually specify the options in the code
snippet above we might run into some new interesting ways to crash everything.
To go back to why it's only affecting LLDB: Nearly all LLVM tools I could find
(even if they are using threads) seem to call the CommandLine parser at the
start so they all execute the initial parser setup at a point where there is
only one thread. So once the code above is executed they are mostly safe from
the sanity check crashes. We even have some shady code for the gtest `main` in
`TestMain.cpp` which is why this also doesn't affect unit tests.
The only exception to this rule is ... *drum roll* ... LLDB! it's not using that
CommandLine library for parsing options so it also never ends up calling it in
`main`. So when we end up in the `FrontendAction` code from the backtrace we are
already very deep in some LLDB logic and usually already have several threads.
In a situation where Swift decides to compile a large amount of Clang modules in
parallel we then end up entering this code via several threads. If several
threads reach this code at the same time we end up in the situation where the
sanity-checking code of CommandLine crashes. I have a very reliable way of
demonstrating the whole thing in D99650 (just run the unit test several times,
it usually crashes after 3-4 attempts).
We have several ways to fix this:
1. Make the whole CommandLine mechanism in LLVM thread-safe.
2. Get rid of `setCommandLineOpts` in `BackendUtil.cpp` and other callers of the
command line parsing in generic Clang code.
3. Initialise the CommandLine library in a safe point in LLDB.
Option 1 is just a lot of work and I'm not even sure where to start. The whole
mechanism is based on global variables and global state and this seems like a
humongous task.
Option 2 is probably the best thing we can do in the near future. There are only
two callers of the command line parser in generic Clang code. The one in
`BackendUtils.cpp` looks like it can be replaced with some reasonable
refactoring (as it only deals with two specific options). There is another one
in `ExecuteCompilerInvocation` which deals with forwarding the generic `-mllvm`
options to the backend which seems like it will just end up requiring us to do
Option 1.
Option 3 is what this patch is doing. We just parse some dummy command line
invocation in a point of the LLDB execution where we only have one thread that
is dealing with LLVM/Clang stuff. This way we are at least prevent the frequent
crashes for users as parsing the dummy command line invocation will set up the
initial parser state safely.
Fixes rdar://70989856
Reviewed By: mib, JDevlieghere
Differential Revision: https://reviews.llvm.org/D99652
Our code for locating the shared library directory works via dladdr (or
the windows equivalent) to locate the path of an address known to reside
in liblldb. This works great for C++ programs, but there's a catch.
When (lib)lldb is used from python (like in our test suite), this dladdr
call will return a path to the _lldb.so (or such) file in the python
directory. To compensate for this, we have code which attempts to
resolve this symlink, to ensure we get the canonical location. However,
here's the second catch.
On windows, this file is not a symlink (but a copy), so this logic
fails. Since most of our other paths are derived from the liblldb
location, all of these paths will be wrong, when running the test suite.
One effect of this was the failure to find lldb-server in D96202.
To fix this issue, I add some windows-specific code to locate the
liblldb directory. Since it cannot rely on symlinks, it works by
manually walking the directory tree -- essentially doing the opposite of
what we do when computing the python directory.
To avoid python leaking back into the host code, I implement this with
the help of a callback which can be passed to HostInfo::Initialize in
order to assist with the directory location. The callback lives inside
the python plugin.
I also strenghten the existing path test to ensure the returned path is
the right one.
Differential Revision: https://reviews.llvm.org/D96779
This patch introduces a LLDB_SCOPED_TIMER macro to hide the needlessly
repetitive creation of scoped timers in LLDB. It's similar to the
LLDB_LOG(F) macro.
Differential revision: https://reviews.llvm.org/D93663
Instead of having a custom error message, propagate the llvm::Error from
SystemInitializerCommon. I didn't realize we had this overload until
Pavel mentioned it in D90987 today.
During active replay, the ::Initialize call is replayed like any other
SB API call and the return value is ignored. Since we can't intercept
this, we terminate here before the uninitialized debugger inevitably
crashes.
Differential revision: https://reviews.llvm.org/D90987
Renamed ThreadIntelPT to TreaceThread, making it a top-level class. I noticed that this class can and shuld work for any trace plugin and there's nothing intel-pt specific in it.
With that TraceThread change, I was able to move most of the json file parsing logic to the base class TraceSessionFileParser, which makes adding new plug-ins easier.
This originally was part of https://reviews.llvm.org/D89283
Differential Revision: https://reviews.llvm.org/D89408
This patch changes the way we initialize and terminate the plugins in
the system initializer. It uses an approach similar to LLVM's
TARGETS_TO_BUILD with a def file that enumerates the plugins.
Previous attempts to land this failed on the Windows bot because there's
a dependency between the different process plugins. Apparently
ProcessWindowsCommon needs to be initialized after all other process
plugins but before ProcessGDBRemote.
Differential revision: https://reviews.llvm.org/D73067
The WASM and Hexagon plugin check the ArchType rather than the OSType,
so explicitly reject those in the DynamicLoaderStatic.
Differential revision: https://reviews.llvm.org/D74780
Generate the LLDB_PLUGIN_DECLARE macros with CMake and a def file. I'm
landing D73067 in pieces so I can bisect what exactly is breaking the
Windows bot.
Other plugins depend on DynamicLoaderDarwinKernel and which means we
cannot conditionally enable/build this plugin based on the target
platform. This means that it will be past of the list of plugins
initialized once that's autogenerated.
Use LLDB_PLUGIN_DEFINE_ADV to make the name of the generated initializer
match the name of the plugin. This is a step towards generating the
initializers with a def file. I'm landing this change in pieces so I can
narrow down what exactly breaks the Windows bot.
This patch changes the way we initialize and terminate the plugins in
the system initializer. It uses an approach similar to LLVM's
TARGETS_TO_BUILD with a def file that enumerates the plugins.
The previously landed patch got reverted because it was lacking:
(1) A plugin definition for the Objective-C language runtime,
(2) The dependency between the Static and WASM dynamic loader,
(3) Explicit initialization of ScriptInterpreterNone for lldb-test.
All issues have been addressed in this patch.
Differential revision: https://reviews.llvm.org/D73067
This patch changes the way we initialize and terminate the plugins in
the system initializer. It uses an approach similar to LLVM's
TARGETS_TO_BUILD with a def file that enumerates the plugins.
Differential revision: https://reviews.llvm.org/D73067
After the recent change that grouped some of the ABI plugins together,
those plugins ended up with multiple initializers per plugin. This is
incompatible with my proposed approach of generating the initializers
dynamically, which is why I've grouped them together in a new entry
point.
Differential revision: https://reviews.llvm.org/D74451
Move the logic for initialization and termination for DynamicLoaderMacOS
into DynamicLoaderMacOSXDYLD so that there's one initializer for the
DynamicLoaderMacOSXDYLD plugin.
Move the logic for initialization and termination for
SymbolFileDWARFDebugMap into SymbolFileDWARF so that there's one
initializer for the SymbolFileDWARF plugin.
Apparently Linux and Windows have the exact opposite behavior when it
comes to inline declarations of external functions. On Linux they're
considered to be part of the lldb_private namespace, while on Windows
they're considered to be part of the top level namespace. Somehow on
macOS, it doesn't really matter and both are fine...
At this point I don't know what to do, so I'm just adding the
LLDB_PLUGIN_DECLARE macros again as originally proposed in D74245.
This is a step towards making the initialize and terminate calls be
generated by CMake, which in turn is towards making it possible to
disable plugins at configuration time.
Differential revision: https://reviews.llvm.org/D74245
Summary:
There's a fair amount of code duplication between the different ABI plugins for
the same architecture (e.g. ABIMacOSX_arm & ABISysV_arm). Deduplicating this
code is not very easy at the moment because there is no good place where to put
the common code.
Instead of creating more plugins, this patch reduces their number by grouping
similar plugins into a single folder/plugin. This makes it easy to extract
common code to a (e.g.) base class, which can then live in the same folder.
The grouping is done based on the underlying llvm target for that architecture,
because the plugins already require this for their operation.
Reviewers: JDevlieghere, jasonmolenda, jfb
Subscribers: sdardis, nemanjai, mgorny, kristof.beyls, fedor.sergeev, kbarton, jrtc27, atanasyan, jsji, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74138
This patch has a couple of outstanding issues. The test is not python3
compatible, and it also seems to fail with python2 (at least under some
circumstances) due to an overambitious assertion.
This reverts the patch as well as subsequent fixup attempts:
014ea93376,
f5f70d1c8f.
4697e701b8.
5c15e8e682.
3ec28da6d6.
Summary:
This change represents the move of ClangASTImporter, ClangASTMetadata,
ClangExternalASTSourceCallbacks, ClangUtil, CxxModuleHandler, and
TypeSystemClang from lldbSource to lldbPluginExpressionParserClang.h
This explicitly removes knowledge of clang internals from lldbSymbol,
moving towards a more generic core implementation of lldb.
Reviewers: JDevlieghere, davide, aprantl, teemperor, clayborg, labath, jingham, shafik
Subscribers: emaste, mgorny, arphaman, jfb, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73661
Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).
This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).
Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73258
Summary:
This commit renames ClangASTContext to TypeSystemClang to better reflect what this class is actually supposed to do
(implement the TypeSystem interface for Clang). It also gets rid of the very confusing situation that we have both a
`clang::ASTContext` and a `ClangASTContext` in clang (which sometimes causes Clang people to think I'm fiddling
with Clang's ASTContext when I'm actually just doing LLDB work).
I also have plans to potentially have multiple clang::ASTContext instances associated with one ClangASTContext so
the ASTContext naming will then become even more confusing to people.
Reviewers: #lldb, aprantl, shafik, clayborg, labath, JDevlieghere, davide, espindola, jdoerfert, xiaobai
Reviewed By: clayborg, labath, xiaobai
Subscribers: wuzish, emaste, nemanjai, mgorny, kbarton, MaskRay, arphaman, jfb, usaxena95, jingham, xiaobai, abidh, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D72684
AppleObjCRuntime is the main entry point to the plugin with the same
name. This is part of a greater refactoring to auto generate the
initializers. NFC.
Differential revision: https://reviews.llvm.org/D73121
PlatformMacOSX is the main entry point to the plugin with the same name.
This is part of a greater refactoring to auto generate the initializers.
Differential revision: https://reviews.llvm.org/D73116
Jonas Devlieghere