Remove the LLDB_CAPTURE_REPRODUCER as it is inherently dangerous. The
reproducers require careful initialization which cannot be guaranteed by
overwriting the reproducer mode at this level.
If we want to provide this functionality, we should do it in the driver
instead. It was originally added to enable capture in CI, but we now
have a dedicated CI job that captures and replays the test suite.
The FileCollector asserts that paths passed to addDirectory are indeed
directories. For that to work, the file needs to actually exist. In the
downstream Swift fork we have tests that remove files during testing,
which resulted in this assertion getting triggered.
For performance reasons the reproducers don't copy the files captured by
the file collector eagerly, but wait until the reproducer needs to be
generated.
This is a problematic when LLDB crashes and we have to do all this
signal-unsafe work in the signal handler. This patch uses a similar
trick to clang, which has the driver invoke a new cc1 instance to do all
this work out-of-process.
This patch moves the writing of the mapping file as well as copying over
the reproducers into a separate process spawned when lldb crashes.
Differential revision: https://reviews.llvm.org/D89600
Add a reproducer verifier that catches:
- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Missing files from the VFS
The verifier is enabled by default during replay, but can be skipped by
passing --reproducer-no-verify.
Differential revision: https://reviews.llvm.org/D86497
Add a reproducer verifier that catches:
- Missing or invalid home directory
- Missing or invalid working directory
- Missing or invalid module/symbol paths
- Missing files from the VFS
The verifier is enabled by default during replay, but can be skipped by
passing --reproducer-no-verify.
Differential revision: https://reviews.llvm.org/D86497
Extract all the provider related logic from Reproducer.h and move it
into its own header ReproducerProvider.h. These classes are seeing most
of the development these days and this reorganization reduces
incremental compilation from ~520 to ~110 files when making changes to
the new header.
When replaying the reproducer, lldb should source the .lldbinit file
that was captured by the reproducer and not the one in the current home
directory. This requires that we store the home directory as part of the
reproducer. By returning the virtual home directory during replay, we
ensure the correct virtual path gets constructed which the VFS can then
find and remap to the correct file in the reproducer root.
This patch adds a new HomeDirectoryProvider, similar to the existing
WorkingDirectoryProvider. As the home directory is not part of the VFS,
it is stored in LLDB's FileSystem instance.
RecordInterestingDirectory was added to collect dSYM bundles and their
content. For the current working directory we only want the directory to
be part of the VFS, not necessarily its contents. This patch renames the
current method to RecordInterestingDirectoryRecursively and adds a new
one that's not recursive.
This patch does several things that are all closely related:
- It introduces a new YamlRecorder as a counterpart to the existing
DataRecorder. As the name suggests the former serializes data as yaml
while the latter uses raw texts or bytes.
- It introduces a new MultiProvider base class which can be backed by
either a DataRecorder or a YamlRecorder.
- It reimplements the CommandProvider in terms of the new
MultiProvider.
Finally, it adds unit testing coverage for the MultiProvider, a naive
YamlProvider built on top of the new YamlRecorder and the existing
MutliLoader.
Differential revision: https://reviews.llvm.org/D83441
Support passive replay as proposed in the RFC [1] on lldb-dev and
described in more detail on the lldb website [2].
This patch extends the LLDB_RECORD macros to re-invoke the current
function with arguments deserialized from the reproducer. This relies on
the function being called in the exact same order as during replay. It
uses the same mechanism to toggle the API boundary as during recording,
which guarantees that only boundary crossing calls are replayed.
Another major change is that before this patch we could ignore the
result of an API call, because we only cared about the observable
behavior. Now we need to be able to return the replayed result to the
SWIG bindings.
We reuse a lot of the recording infrastructure, which can be a little
confusing. We kept the existing naming to limit the amount of churn, but
might revisit that in a future patch.
[1] http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html
[2] https://lldb.llvm.org/resources/reproducers.html
Differential revision: https://reviews.llvm.org/D77602
The FileCollector in LLDB collects every files that's used during a
debug session when capture is enabled. This ensures that the reproducer
only contains the files necessary to reproduce. This approach is not a
good fit for the dSYM bundle, which is a directory on disk, but should
be treated as a single unit.
On macOS LLDB have automatically find the matching dSYM for a binary by
its UUID. Having a incomplete dSYM in a reproducer can break debugging
even when reproducers are disabled.
This patch adds a was to specify a directory of interest to the
reproducers. It is called from SymbolVendorMacOSX with the path of the
dSYMs used by LLDB.
Differential revision: https://reviews.llvm.org/D76672
The FileCollector in LLDB collects every files that's used during a
debug session when capture is enabled. This ensures that the reproducer
only contains the files necessary to reproduce. This approach is not a
good fit for the dSYM bundle, which is a directory on disk, but should
be treated as a single unit.
On macOS LLDB have automatically find the matching dSYM for a binary by
its UUID. Having a incomplete dSYM in a reproducer can break debugging
even when reproducers are disabled.
This patch adds a was to specify a directory of interest to the
reproducers. It is called from SymbolVendorMacOSX with the path of the
dSYMs used by LLDB.
Differential revision: https://reviews.llvm.org/D76672
Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).
This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).
Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere
Reviewed By: JDevlieghere
Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D73258
Add a flag which always generates a reproducer when normally it would be
discarded. This is meant for testing purposes to capture a debugger
session without modification the session itself.
When running the test suite with always capture on, a handful of tests
are failing because they have multiple targets and therefore multiple
GDB remote connections. The current reproducer infrastructure is capable
of dealing with that.
This patch reworks the GDB remote provider to support multiple GDB
remote connections, similar to how the reproducers support shadowing
multiple command interpreter inputs. The provider now keeps a list of
packet recorders which deal with a single GDB remote connection. During
replay we rely on the order of creation to match the number of packets
to the GDB remote connection.
Differential revision: https://reviews.llvm.org/D71105
Make it possible to override reproducer capture with the
LLDB_CAPTURE_REPRODUCER environment variable.
The goal of this change is twofold.
(1) I want to be able to enable capturing reproducers during regular
test runs, both locally and on the bots. To do so I need a way to
force capture. I cannot do this through the Python API, because
reproducer capture must be enabled *before* the debugger
initialized, which happens automatically when doing `import lldb`.
(2) I want to provide an escape hatch for when reproducers are enabled
by default. Downstream we have reproducer capture enabled by default
in the driver.
This patch solves both problems by overriding the reproducer mode based
on the environment variable. Acceptable values are 0/1 and ON/OFF.
This patch extends the reproducer to capture the debugger's current
working directory. This information will be used later to set the
current working directory of the VFS.
llvm-svn: 375059
The VFS requires files to be have absolute paths. The file collector
makes paths relative to the reproducer root. If the root is a relative
path, this would trigger an assert in the VFS. This patch ensures that
we always make the given path absolute.
Thank you Ted Woodward for pointing this out!
llvm-svn: 373102
This just moves the CommandLoader utility into the reproducer namespace
and makes it accessible outside the API layer. This is setting things up
for a bigger change.
llvm-svn: 371689
Originally the idea was for providers to be defined close to where they
are used. While this helped designing the providers in such a way that
they don't depend on each other, it also means that it's not possible to
access them from a central place. This proved to be a problem for some
providers and resulted in them living in the reproducer class.
The ProcessGDBRemote provider is the last remaining exception. This
patch makes things consistent and moves it into the reproducer like the
other providers.
llvm-svn: 371685
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368933
Utility doesn't link against lldbBase so we cannot call GetVersion in
keep. I already added a string member m_version to deal with that, but
the call was still there.
llvm-svn: 363228
Generally, reproducers are rev-locked to the version of LLDB, so it's
valuable to have the LLDB version in the reproducer. For now I just want
the information to be present, without enforcing it, but I envision
emitting a warning during replay in the future.
Differential revision: https://reviews.llvm.org/D63229
llvm-svn: 363225
This replaces the `info` typedef with a nested struct named Info. This
means we now have FooProvider and FooProvider::Info, instead of two
related but separate classes FooProvider and FooInfo. This change is
mostly cosmetic.
llvm-svn: 363211
Deallocating the data recorder in during the ::Keep() operation causes
problems down the line when exiting the debugger. The command
interpreter still holds a pointer to the now deallocated object and has
no way to know it no longer exists. This is exactly what the m_record
flag was meant for, although it wasn't hooked up properly either.
llvm-svn: 358916
The command interpreter holds a pointer to a DataRecorder. After
generating the reproducer, we deallocated all the DataRecorders, causing
the command interpreter to hold a non-null reference to an invalid
object.
This patch changes the behavior of the command provider to stop the
DataRecorders when a reproducer is generated, rather than deallocating
them.
llvm-svn: 355940
automatic move should not fire when returning type T in a function with
result type Expected<T>. Some compilers seem to allow that nonetheless.
llvm-svn: 355270
This patch adds the necessary logic to capture and replay commands
entered into the command interpreter. A DataRecorder shadows the input
and writes its data to a know file. During replay this file is used as
the command interpreter's input.
It's possible to the command interpreter more than once, with a
different input source. We support this scenario by using multiple
buffers. The synchronization for this takes place at the SB layer, where
we create a new recorder every time the debugger input is changed.
During replay we use the corresponding buffer as input.
Differential revision: https://reviews.llvm.org/D58564
llvm-svn: 355249
As per the discussion on the mailing list:
http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20190218/048007.html
This commit implements option (3):
> Go back to initializing the reproducer before the rest of the debugger.
> The method wouldn't be instrumented and guarantee no other SB methods are
> called or SB objects are constructed. The initialization then becomes part
> of the replay.
Differential revision: https://reviews.llvm.org/D58410
llvm-svn: 354631
This patch adds the file provider which is responsible for capturing
files used by LLDB.
When capturing a reproducer, we use a file collector that is very
similar to the one used in clang. For every file that we touch, we add
an entry with a mapping from its virtual to its real path. When we
decide to generate a reproducer we copy over the files and their
permission into to reproducer folder.
When replaying a reproducer, we load the VFS mapping and instantiate a
RedirectingFileSystem. The latter will transparently use the files
available in the reproducer.
I've tested this on two macOS machines with an artificial example.
Still, it is very likely that I missed some places where we (still) use
native file system calls. I'm hoping to flesh those out while testing
with more advanced examples. However, I will fix those things in
separate patches.
Differential revision: https://reviews.llvm.org/D54617
llvm-svn: 352538
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
In the original reproducer design, I expected providers to be more
dynamic than they turned out. For example, we don't have any instances
where one provider has multiple files. Additionally, I expected there to
be less locality between capture and replay, with the provider being
defined in one place and the replay code to live in another. Both
contributed to the design of the provider info.
This patch refactors the reproducer info to be something static. This
means less magic strings and better type checking. The new design still
allows for the capture and replay code to live in different places as
long as they both have access to the new statically defined info class.
I didn't completely get rid of the index, because it is useful for (1)
sanity checking and (2) knowing what files are used by the reproducer.
Differential revision: https://reviews.llvm.org/D56814
llvm-svn: 351501
As Pavel noted on the mailing list we should only create the bottom-most
directory if it doesn't exist. This should also fix the test case on
Windows as we can use lit's temp directory.
llvm-svn: 348289
This patch changes the way the reproducer is initialized. Rather than
making changes at run time we now do everything at initialization time.
To make this happen we had to introduce initializer options and their SB
variant. This allows us to tell the initializer that we're running in
reproducer capture/replay mode.
Because of this change we also had to alter our testing strategy. We
cannot reinitialize LLDB when using the dotest infrastructure. Instead
we use lit and invoke two instances of the driver.
Another consequence is that we can no longer enable capture or replay
through commands. This was bound to go away form the beginning, but I
had something in mind where you could enable/disable specific providers.
However this seems like it adds very little value right now so the
corresponding commands were removed.
Finally this change also means you now have to control this through the
driver, for which I replaced --reproducer with --capture and --replay to
differentiate between the two modes.
Differential revision: https://reviews.llvm.org/D55038
llvm-svn: 348152
When I landed the initial reproducer framework I knew there were some
things that needed improvement. Rather than bundling it with a patch
that adds more functionality I split it off into this patch. I also
think the API is stable enough to add unit testing, which is included in
this patch as well.
Other improvements include:
- Refactor how we initialize the loader and generator.
- Improve naming consistency: capture and replay seems the least ambiguous.
- Index providers by name and make sure there's only one of each.
- Add convenience methods for creating and accessing providers.
Differential revision: https://reviews.llvm.org/D54616
llvm-svn: 347716
Jonas Devlieghere