Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds the
necessary code for replay. For now this is a NO-OP until the
corresponding reproducer instrumentation changes land.
For more info please refer to the RFC on lldb-dev:
http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html
Differential revision: https://reviews.llvm.org/D77588
Summary:
I added an `abort()` call to some code and noticed that the test suite was still passing and it just marked my test as "UNSUPPORTED".
It seems the reason for that is that we expect failing tests to print "FAIL:" which doesn't happen when we crash. If we then also
have an unsupported because we skipped some debug information in the output, we just mark the test passing because it is unsupported
on the current platform.
This patch marks any test that has a non-zero exit code as failing even if it doesn't print "FAIL:" (e.g., because it crashed).
Reviewers: labath, JDevlieghere
Reviewed By: labath, JDevlieghere
Subscribers: aprantl, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D75031
The current lldbtest format has a number of shortcomings, all related to
how we omit information based on why the test fails. For example, a
successful test would print nothing, even when `-a` is passed to lit.
It's not up to the test format to decide whether to print something or
not, that's handled by lit itself. For other test results we would
sometimes print stdout & stderr, but not always, such as when a timeout
was reached or we couldn't parse the dotest output.
This patch changes the lldbtest format and makes it behave more like
lit. We now always print the dotest invocation, the exit code, the
output to stdout & stderr. If you're used to dealing with ShTests in
lit, this will feel all very familiar.
Differential revision: https://reviews.llvm.org/D73384
The code that works around SIP was unintentionally being triggered for
/usr/local/bin/python as well. That caused trouble on GreenDragon where
we were swapping out a Python 3 executable with the system's Python 2
executable.
LLDB has three major testing strategies: unit tests, tests that exercise
the SB API though dotest.py and what we currently call lit tests. The
later is rather confusing as we're now using lit as the driver for all
three types of tests. As most of this grew organically, the directory
structure in the LLDB repository doesn't really make this clear.
The 'lit' tests are part of the root and among these tests there's a
Unit and Suite folder for the unit and dotest-tests. This layout makes
it impossible to run just the lit tests.
This patch changes the directory layout to match the 3 testing
strategies, each with their own directory and their own configuration
file. This means there are now 3 directories under lit with 3
corresponding targets:
- API (check-lldb-api): Test exercising the SB API.
- Shell (check-lldb-shell): Test exercising command line utilities.
- Unit (check-lldb-unit): Unit tests.
Finally, there's still the `check-lldb` target that runs all three test
suites.
Finally, this also renames the lit folder to `test` to match the LLVM
repository layout.
Differential revision: https://reviews.llvm.org/D68606
llvm-svn: 374184
Jonas Devlieghere