Many tests use (commented out) print statement for debugging the test
itself. This patch adds a new trace method to lldbtest to reuse the
existing tracing infrastructure and replace these print statements.
Differential revision: https://reviews.llvm.org/D80448
The reproducers' working directory is set to the current working
directory when they are initialized. While this is not optimal, as the
cwd can change during a debug session, it has been sufficient so far.
The current approach doesn't work for the API test suite however because
dotest temporarily changes the directory to where the test's Python file
lives.
This patch adds an API to tell the reproducers what to set the CWD to.
This is a NO-OP in every mode but capture.
Differential revision: https://reviews.llvm.org/D79825
Summary:
If we don't have a current frame then we can still run many expressions
as long as we have an active target. With this patch `expect_expr` directly
calls the target's EvaluateExpression function when there is no current frame.
Reviewers: labath
Reviewed By: labath
Subscribers: JDevlieghere
Differential Revision: https://reviews.llvm.org/D77197
Global properties are shared between debugger instances and
if a test doesn't clear changes in settings it made,
this leads to side effects in other tests.
Differential Revision: https://reviews.llvm.org/D75537
This patch forces architecture "arm" if underlying os reports core
armv7l or armv8l. On linux systems 32 bit sysroot running on 64bit
AArch64 hardware reports armv7l or armv8l which is essently arm
32bit mode. This fixes 5 testcases on 32bit arm.
Some tests set settings and don't clean them up, this leads to side effects in other tests.
The patch removes a global debugger instance with a per-test debugger to avoid such effects.
From what I see, lldb.DBG was needed to determine the platform before a test is run,
lldb.selected_platform is used for this purpose now. Though, this required adding a new function
to the SBPlatform interface.
Differential Revision: https://reviews.llvm.org/D74903
Summary:
Currently the test suite runs with enabled automatically applied Clang fix-its for expressions.
This is causing that sometimes incorrect expressions in tests are still evaluated even though they
are actually incorrect. Let's disable this feature in the test suite so that we know when expressions
are wrong and leave the fix-it testing to the dedicated tests for that feature.
Also updates the `lang/cpp/operators/` test as it seems Clang needs the `struct` keywords
before C and would otherwise fail without fixits.
Reviewers: jingham, JDevlieghere, shafik
Reviewed By: JDevlieghere, shafik
Subscribers: shafik, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74957
Summary:
Currently when printing data types we include implicit scopes such as inline namespaces or anonymous namespaces.
This leads to command output like this (for `std::set<X>` with X being in an anonymous namespace):
```
(lldb) print my_set
(std::__1::set<(anonymous namespace)::X, std::__1::less<(anonymous namespace)::X>, std::__1::allocator<(anonymous namespace)::X> >) $0 = size=0 {}
```
This patch removes all the implicit scopes when printing type names in TypeSystemClang::GetDisplayTypeName
so that our output now looks like this:
```
(lldb) print my_set
(std::set<X, std::less<X>, std::allocator<X> >) $0 = size=0 {}
```
As previously GetDisplayTypeName and GetTypeName had the same output we actually often used the
two as if they are the same method (they were in fact using the same implementation), so this patch also
fixes the places where we actually want the display type name and not the actual type name.
Note that this doesn't touch the `GetTypeName` class that for example the data formatters use, so this patch
is only changes the way we display types to the user. The full type name can also still be found when passing
'-R' to see the raw output of a variable in case someone is somehow interested in that.
Partly fixes rdar://problem/59292534
Reviewers: shafik, jingham
Reviewed By: shafik
Subscribers: christof, JDevlieghere, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74478
Summary:
This creates a separate LLDB_TEST_SRC var to match the existing LLDB_TEST var. LLDB_TEST points to the test framework, LLDB_TEST_SRC points to the tests themselves.
The var points to the same place, but a future patch will move the tree + update var.
Reviewers: labath, JDevlieghere
Reviewed By: labath
Subscribers: merge_guards_bot, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71150
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
Re-landing this now that (hopefully) all the failures this caused on the
bots have been addressed.
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
This patch changes the behavior of the substrs argument to self.expect.
Currently, the elements of substrs are unordered and as long as the
string appears in the output, the assertion passes.
We can be more precise by requiring that the substrings be ordered in
the way they appear. My hope is that this will make it harder to
accidentally pass a check because a string appears out of order.
Differential revision: https://reviews.llvm.org/D73766
Currently if 'expect' fails and a custom msg is supplied, then lldbtest
will not print the actual command output. This makes it impossible to know
why the test actually failed. This just prints the command output even
if the msg parameter was supplied.
Summary:
This patch adds a new function to lldbtest: `expect_expr`. This function is supposed to replace the current approach
of calling `expect`/`runCmd` with `expr`, `p` etc.
`expect_expr` allows evaluating expressions and matching their value/summary/type/error message without
having to do any string matching that might allow unintended passes (e.g., `self.expect("expr 3+4", substrs=["7"])`
can unexpectedly pass for results like `(Class7) $0 = 7`, `(int) $7 = 22`, `(int) $0 = 77` and so on).
This only uses the function in a few places to test and demonstrate it. I'll migrate the tests in follow up commits.
Reviewers: JDevlieghere, shafik, labath
Reviewed By: labath
Subscribers: christof, abidh, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D70314
To do so, we need to register the sanitizer libraries with the target
so that they get uploaded before running. This patch adds a helper to
the test class to this effect.
The invocation shown by dotest.py to re-run a single test is misleading:
it ranges from missing arguments (best case scenario) to being totally
wrong (worst case scenario).
In the past I've tried to get it right, but given the dotest
architecture this is harder than it looks. Furthermore, we have pretty
good documentation on the website [1] for most use cases.
This patch removes the rerun invocation.
[1] https://lldb.llvm.org/resources/test.html
Before the test reorganization, everything was part of a single test
suite with a single module cache. Now that things are properly separated
this is no longer the case. Only the shell tests inherited the logic to
properly configure and wipe the module caches. This patch adds that
logic back for the API tests. While doing so, I noticed that we were
configuring a Clang module cache in CMake, but weren't actually using it
from dotest.py. I included a fix for that in this patch as well.
Differential revision: https://reviews.llvm.org/D68755
llvm-svn: 374386
Summary:
In r367234 we introduced a central place to hold the set up commands for
the various ways we have of launching lldb. However, a number of
commands still remained outside of that.
This patch moves the remaining set up commands into this function, which
allows us to remove manual clang module path setting code in
TestBatchMode.
One unfortunate victim of this approach is TestSTTYBeforeAndAfter which,
due to how it launches lldb (pexpect->expect->lldb), fails get the
quoting right. It would be possible to fix the quoting there, it would be a bit
icky, and none of the commands in this list are really relevant for what this
test is doing, so I just remove the commands outright.
Reviewers: JDevlieghere, jankratochvil
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D67173
llvm-svn: 371019
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
The refactoring patch for the option completion broke the completion
for ambiguous long options. As this feature was also untested (as
testing ambiguous options with the current test methods is impossible),
I just noticed now. This patch restores the old behavior and adds a
test for this feature.
llvm-svn: 370185
Today I discovered the skipLongRunningTest decorator and to my surprise
all the tests were passing without the decorator. They don't seem to be
that expensive either, they take a few seconds but we have tests that
take much longer than that. As such I propose to remove the decorator
and enable them by default.
Differential revision: https://reviews.llvm.org/D66774
llvm-svn: 369995
Summary:
This commit fixes some typo I found while exploring LLDB's codebase.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D66452
llvm-svn: 369313
On the matrix bot on GreenDragon [1] we want to run the test suite
against different DWARF versions. The idea here is not to replace
targeted tests for certain DWARF features, but rather to provide an easy
way to support this configuration.
[1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/
Differential revision: https://reviews.llvm.org/D66370
llvm-svn: 369272
Summary:
It seems this was an unintended side-effect of D26698. AFAICT, these
functions did return an empty string before that patch, and the patch
contained code which attempted to ensure that, but those efforts were
negated by ConstString::AsCString, which by default returns a nullptr
even for empty strings.
This patch:
- fixes the GetOutput/Error methods to really return empty strings
- adds and explicit test for that
- removes a workaround in lldbtest.py, which was masking this problem
from our other tests
Reviewers: jingham, clayborg
Subscribers: zturner, lldb-commits
Differential Revision: https://reviews.llvm.org/D65739
llvm-svn: 368806
rL357954 did increase `packet-timeout` 1sec -> 5secs. Which is IMO about the
maximum timeout reasonable for regular use. But for testsuite I think the
timeout should be higher as the testsuite runs in parallel and it can be run
even on slow hosts and with other load (moreover if it runs on some slow arch).
I have chosen 60 secs, that should be enough hopefully. Larger value could
make debugging with hanging `lldb-server` annoying.
This patch was based on this testsuite timeout:
http://lab.llvm.org:8014/builders/lldb-x86_64-fedora/builds/546/steps/test/logs/stdio
FAIL: test_connect (TestGDBRemoteClient.TestGDBRemoteClient)
Test connecting to a remote gdb server
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jkratoch/slave-lldb-x86_64-fedora/lldb-x86_64-fedora/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestGDBRemoteClient.py", line 13, in test_connect
process = self.connect(target)
File "/home/jkratoch/slave-lldb-x86_64-fedora/lldb-x86_64-fedora/llvm/tools/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py", line 480, in connect
self.assertTrue(error.Success(), error.description)
AssertionError: False is not True : failed to get reply to handshake packet
Differential Revision: https://reviews.llvm.org/D65271
llvm-svn: 367234
Dotest contains code to clear DYLD_LIBRARY_PATH for the inferior to not
propagate sanitized builds. However, it's possible that we want to
inject a different library path with `--inferior-env`. To make that work
correctly, we need to do that *after* clearing DYLD_LIBRARY_PATH.
llvm-svn: 367054
Summary: lldb-mi has been removed, but there are still a bunch of references in the code base. This patch removes all of them.
Reviewers: JDevlieghere, jfb
Reviewed By: JDevlieghere
Subscribers: dexonsmith, ki.stfu, mgorny, abidh, jfb, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D64992
llvm-svn: 366590
Disable colors so we don't risk having unexpected ANSI codes in the test
output. Currently, the behavior of a test can change depending on
whether it's run under a color-supporting terminal, or under a dummy
terminal, for example when using lit or multiprocessing.
llvm-svn: 366356
This patch adds a dotest flag for setting environment variables for the
inferior. This is different from the current --env flag, which sets
variables in the debugger's environment. This allows us to set things
like LD_LIBRARY_PATH for testing.
Differential revision: https://reviews.llvm.org/D63790
llvm-svn: 364443
D55859 <https://reviews.llvm.org/D55859> has no effect for some of the
testcases so this patch extends it even for (all?) other testcases known to me.
LLDB was failing when LLDB prints errors reading system debug infos
(`*-debuginfo.rpm`, DWZ-optimized) which should never happen as LLDB testcases
should not be affected by system debug infos.
`lldb/packages/Python/lldbsuite/test/api/multithreaded/driver.cpp.template` is
using only SB API which does not expose `ModuleList` so I had to call
`HandleCommand()` there.
`lldb-test.cpp` could also use `HandleCommand` and then there would be no need
for `ModuleListProperties::SetEnableExternalLookup()` but I think it is cleaner
with API and not on based on text commands.
Differential Revision: https://reviews.llvm.org/D63339
llvm-svn: 363567
Remove the code forcing -stdlib=libstdc++ on NetBSD in getBuildFlags()
method. NetBSD uses libc++ everywhere else, and using libstdc++ here
causes lang/cpp/dynamic-value to fail to build.
Differential Revision: https://reviews.llvm.org/D58871
llvm-svn: 355273
dotest's version comparision function is just a lexicographical compare
of the version strings. This is obviously wrong. This patch implements
the comparison using distutils.version.LooseVersion as suggested by
Zachary.
Reviewers: zturner, labath, serge-sans-paille
Subscribers: jdoerfert, lldb-commits
Differential Revision: https://reviews.llvm.org/D58219
llvm-svn: 354047
We figured out the issue so the logging is no longer necessary. It turns
out we were using a session format that was not unique for inline tests.
llvm-svn: 351902
We're seeing an odd issue on GreenDragon's lldb-cmake-matrix. Dotest is
unable to move a log file (OSError: [Errno 2] No such file or
directory). The os.rename call is guarded with a check that the source
file and destination directory exist.
This wraps the call in a try-except that prints the source and
destination path to see which component seemingly doesn't exist.
llvm-svn: 351611
This saves about 3 redundant gigabytes from the Objective-C test build
directories. Tests that must do unsavory things with the LLDB clang
module cache, already specify a per-test module cache in their .py
test instructions.
<rdar://problem/36002081>
Differential Revision: https://reviews.llvm.org/D54602
llvm-svn: 347057
In January Davide sent an e-mail to the mailing list to suggest removing
unmaintained language plugins such as Go and Java. The plan was to have
some cool down period to allow users to speak up, however after that the
plugins were never actually removed.
This patch removes the Go debugger plugin.
The plugin can be added again in the future if it is mature enough both
in terms of testing and maintenance commitment.
Discussion on the mailing list:
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013171.html
Differential revision: https://reviews.llvm.org/D54057
llvm-svn: 346157
This allows bots which haven't updated to pass in --filecheck to
dotest.py to run more tests. FileCheck-dependent tests will continue to
fail.
Differential Revision: https://reviews.llvm.org/D53175
llvm-svn: 344401
Summary: This is another string/byte conversion issue between Python 2 and 3. In Python 2, the subprocess communication expects a byte string, but in Python 3, it expects bytes. Since both versions are capable of using strings when universal_newlines is set to True AND filecheck operates on strings, force the use of strings.
Reviewers: zturner, asmith, vsk
Reviewed By: zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D53166
llvm-svn: 344386
The 'test_filename' property in TestBase changes over time, so
attempting to find a check file relative to the directory containing
'test_filename' is flaky.
Use the absolute path of the check file as that's always correct (and
simpler). This relies on the test driver changing into the test
directory, which it seems we can safely assume.
As a drive-by, make self.filecheck respect the trace (-t) option.
llvm-svn: 342699
Add a "filecheck" method to the LLDB test base. This allows test authors
to pattern match command output using FileCheck, making it possible to
write stricter tests than what `self.expect` allows.
For context (motivation, examples of stricter checking, etc), see the
lldb-dev thread: "Using FileCheck in lldb inline tests".
Differential Revision: https://reviews.llvm.org/D50751
llvm-svn: 342508
Summary:
This patch adds a framework for adding descriptions to the command completions we provide.
It also adds descriptions for completed top-level commands so that we can test this code.
Completions are in general supposed to be displayed alongside the completion itself. The descriptions
can be used to provide additional information about the completion to the user. Examples for descriptions
are function signatures when completing function calls in the expression command or the binary name
when providing completion for a symbol.
There is still some boilerplate code from the old completion API left in LLDB (mostly because the respective
APIs are reused for non-completion related purposes, so the CompletionRequest doesn't make sense to be
used), so that's why I still had to change some function signatures. Also, as the old API only passes around a
list of matches, and the descriptions are for these functions just another list, I had to add some code that
essentially just ensures that both lists are always the same side (e.g. all the manual calls to
`descriptions->AddString(X)` below a `matches->AddString(Y)` call).
The initial command descriptions that come with this patch are just reusing the existing
short help that is already added in LLDB.
An example completion with descriptions looks like this:
```
(lldb) pl
Available completions:
platform -- Commands to manage and create platforms.
plugin -- Commands for managing LLDB plugins.
```
Reviewers: #lldb, jingham
Reviewed By: #lldb, jingham
Subscribers: jingham, JDevlieghere, lldb-commits
Differential Revision: https://reviews.llvm.org/D51175
llvm-svn: 342181
Summary:
This patch adds initial code completion support for the `expr` command.
We now have a completion handler in the expression CommandObject that
essentially just attempts to parse the given user expression with Clang with
an attached code completion consumer. We filter and prepare the
code completions provided by Clang and send them back to the completion
API.
The current completion is limited to variables that are in the current scope.
This includes local variables and all types used by local variables. We however
don't do any completion of symbols that are not used in the local scope (or
in some other way already in the ASTContext).
This is partly because there is not yet any code that manually searches for additiona
information in the debug information. Another cause is that for some reason the existing
code for loading these additional symbols when requested by Clang doesn't seem to work.
This will be fixed in a future patch.
Reviewers: jingham, teemperor
Reviewed By: teemperor
Subscribers: labath, aprantl, JDevlieghere, friss, lldb-commits
Differential Revision: https://reviews.llvm.org/D48465
llvm-svn: 341086
This patch adds a new lldb-vscode tool that speaks the Microsoft Visual Studio Code debug adaptor protocol. It has full unit tests that test all packets.
This tool can be easily packaged up into a native extension and used with Visual Studio Code, and it can also be used by Nuclide
Differential Revision: https://reviews.llvm.org/D50365
llvm-svn: 339911
Summary:
Instead of just printing the current "False is not True, ..." message when we
fail to run a certain command, this patch also adds the actual command output or
error output that we received to the assertion message.
Reviewers: davide
Reviewed By: davide
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D50492
llvm-svn: 339351
Summary: Otherwise this assertion message is not very useful to whoever is reading the log.
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D49947
llvm-svn: 338179
Summary:
The test suite has often unnecessary trailing whitespace, and sometimes
unnecessary trailing lines or a missing final new line. This patch just strips
trailing whitespace/lines and adds missing newlines at the end.
Subscribers: ki.stfu, JDevlieghere, christof, lldb-commits
Differential Revision: https://reviews.llvm.org/D49943
llvm-svn: 338171
Summary:
test_set_working_dir was testing two scenario: failure to set the working dir because of a non existent directory and succeeding to set the working directory. Since the negative case fails on both Linux and Windows, the positive case was never tested. I split the test into two which allows us to always run both the negative and positive cases. The positive case now succeeds on Linux and the negative case still fails.
During the investigation, it turned out that lldbtest.py will try to execute a process launch command up to 3 times if the command failed. This means that we could be covering up intermittent failures by running any test that does process launch multiple times without ever realizing it. I've changed the counter to 1 (though it can still be overwritten with the environment variable).
This change also fixes both the positive and negative cases on Windows. There were a few issues:
1) In ProcessLauncherWindows::LaunchProcess, the error was not retrieved until CloseHandle was possibly called. Since CloseHandle is also a system API, its success would overwrite any existing error that could be retrieved using GetLastError. So by the time the error was retrieved, it was now a success.
2) In DebuggerThread::StopDebugging TerminateProcess was called on the process handle regardless of whether it was a valid handle. This was causing the process to crash when the handle was LLDB_INVALID_PROCESS (0xFFFFFFFF).
3) In ProcessWindows::DoLaunch we need to check that the working directory exists before launching the process to have the same behavior as other platforms which first check the directory and then launch process. This way we also control the exact error string.
Reviewers: labath, zturner, asmith, jingham
Reviewed By: labath
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D48050
llvm-svn: 334642
Add OpenBSD python module in order to support unit tests.
Reviewers: labath, zturner
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D47692
llvm-svn: 333888
Summary:
In the magic test duplicator, we were making the decision whether to
create a test variant based on the compiler and the target platform.
This meant that the set of known tests was different for each test
configuration.
This patch makes the set of generated test variants static and handles
the skipping via runtime checks instead. This is more consistent with
how we do other test-skipping decision (e.g. for libc++ tests), and
makes it easier to expose the full set of tests to lit, which now does
not need to know anything about what things can potentially cause tests
to appear or disappear.
Reviewers: JDevlieghere, aprantl
Subscribers: eraman, lldb-commits
Differential Revision: https://reviews.llvm.org/D45949
llvm-svn: 330708
Summary:
- Fix test jump for powerpc64le
Jumping directly to the return line on power architecture dos not means
returning the value that is seen on the code. The last test fails, because
it needs the execution of some assembly in the beginning of the function.
Avoiding this test for this architecture.
- Avoid evaluate environ variable name on Linux
On Linux the Symbol environ conflicts with another variable, then in
order to avoid it, this test was moved into a specific test, which is not
supported if the OS is Linux.
- Added PPC64le as MIPS behavior
Checking the disassembler output, on PPC64le machines behaves as MPIS.
Added method to identify PPC64le architecture and checking it when
disassembling instructions in the test case.
Reviewers: labath
Reviewed By: labath
Subscribers: clayborg, labath, luporl, alexandreyy, sdardis, ki.stfu, arichardson
Differential Revision: https://reviews.llvm.org/D44101
Patch by Leonardo Bianconi <leonardo.bianconi@eldorado.org.br>.
llvm-svn: 327977
Summary:
This patch implements a unified way of cleaning the build folder of each
test. This is done by completely removing the build folder before each
test, in the respective setUp() method. Previously, we were using a
combination of several methods, each with it's own drawbacks:
- nuking the entire build tree before running dotest: the issue here is
that this did not take place if you ran dotest manually
- running "make clean" before the main "make" target: this relied on the
clean command being correctly implemented. This was usually true, but
not always.
- for files which were not produced by make, each python file was
responsible for ensuring their deleting, using a variety of methods.
With this approach, the previous methods become redundant. I remove the
first two, since they are centralized. For the other various bits of
clean-up code in python files, I indend to delete it when I come
across it.
Reviewers: aprantl
Subscribers: emaste, ki.stfu, mgorny, eraman, lldb-commits
Differential Revision: https://reviews.llvm.org/D44526
llvm-svn: 327703
This is a more principled approach to disabling Spotlight .dSYM
lookups while running the testsuite, most importantly it also works
for the LIT-based tests, which I overlooked in my initial fix
(renaming the test build dir to lldb-tests.noindex).
Differential Revision: https://reviews.llvm.org/D44342
llvm-svn: 327330
I want to extend the properties on ModuleList to also contain other
more general settings and renaming the settings category to symbols
seems to be the least bad of choices.
llvm-svn: 327193
It turns out that setting the clang module cache after LLDB has a
Target can be too late. In particular, the Swift language plugin needs
to know the setting without having access to a Target. This patch
moves the setting into the *LLDB* module cache, where it is a global
setting that is available before any Target is created and more
importantly, is shared between all Targets.
rdar://problem/37944432
Differential Revision: https://reviews.llvm.org/D43984
llvm-svn: 326628
Summary:
These were not being flaky, but they're still making the tree dirty.
These tests were using lldbutil.append_to_process_working_directory to
derive the file path so I fix them by modifying the function to return
the build directory for local tests.
Technically, now the path returned by this function does not point to
the process working directory for local tests, but I think it makes
sense to keep the function name, as I think we should move towards
launching the process in the build directory (and I intend to change
this for the handful of inferiors that actually care about their PWD,
for example because they need to create files there).
Reviewers: davide, aprantl
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D43506
llvm-svn: 325690
Since we now can run multiple tests from the same directory at once, we
need to include the test name in the remote test directory instead.
I'm not sure if the test_number in the remote path is necessary anymore
(or even if it was ever necessary), but I choose to leave it in for now.
llvm-svn: 325334
Summary:
Due to in-tree builds, we were parallelizing the tests at the directory
level. Now that the tests are built out-of-tree, we can remove this
limitation and paralelize at file level instead.
This decreases test suite time by about 10% for me, which is not
world-shattering, but it makes the code slightly simpler and will also
allow us to merge tests which were artificially spread over multiple
folders (TestConcurrentEvents...) to work-around this limitation.
To make this work, I've also needed to include the test file name in the
build directory name, as just the test method name is not unique enough
(plenty of tests have a test method called "test" or similar).
While doing this, I've found a couple of tests that are taking waaay longer then
they ought to (TestBreakpointCaseSensitivity -- 90 seconds), which I plan to
look into in the future.
Reviewers: aprantl
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D43335
llvm-svn: 325322
This patch makes LLDB's clang module cache path customizable via
settings set target.clang-modules-cache-path <path> and uses it in the
LLDB testsuite to reuse the same location inside the build directory
for LLDB and clang.
Differential Revision: https://reviews.llvm.org/D43099
llvm-svn: 324775
Summary:
Update makefiles to specify -fPIC in Makefile.rules and only throw -fPIC when building a shared library. This change is necessary to allow building the lldb tests on Windows where -fPIC is not a valid option.
Update a few places to Python 3.x syntax
Reviewers: zturner, lldb-commits
Reviewed By: zturner
Subscribers: stella.stamenova, labath, llvm-commits
Differential Revision: https://reviews.llvm.org/D42994
llvm-svn: 324671
This patch creates a <test>.dwarf, <test>.dwo, etc., build directory for each testcase variant.
Most importantly, this eliminates the need for the per-test lock file in the source directory.
Tests that are marked as NO_DEBUG_INFO_TESTCASE and build with
buildDefault() are built in a <test>.default build directory.
Differential Revision: https://reviews.llvm.org/D42763
llvm-svn: 324368
Summary:
This changes the way we store the debug info variant to make it
available earlier in the test bringup: instead of it being set by the
test wrapper method, it is set as a *property* of the wrapper method.
This way, we can inspect it as soon as self.testMethodName is
initialized. The retrieval is implemented by a new function
TestBase.getDebugInfo(), and all that's necessary to make it work is to
change self.debug_info into self.getDebugInfo().
While searching for debug_info occurences i noticed that TestLogging is
being replicated for no good reason, so I removed the replication there.
Reviewers: aprantl, jingham
Subscribers: eraman, JDevlieghere, lldb-commits
Differential Revision: https://reviews.llvm.org/D42836
llvm-svn: 324226
This patch is the result of a discussion on lldb-dev, see
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for
background.
For each test (should be eventually: each test configuration) a
separate build directory is created and we execute
make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test
In order to make this work all LLDB tests need to be updated to find
the executable in the test build directory, since CWD still points at
the test's source directory, which is a requirement for unittest2.
Although we have done extensive testing, I'm expecting that this first
attempt will break a few bots. Please DO NOT HESITATE TO REVERT this
patch in order to get the bots green again. We will likely have to
iterate on this some more.
Differential Revision: https://reviews.llvm.org/D42281
llvm-svn: 323803
The logic was incorrect because on windows, we need to look for
yaml2obj.EXE. I implement the search in terms of
distutils.spawn.find_executable, which should handle the platform
differences for us.
llvm-svn: 323744
Summary:
Adds new utilities that make it easier to write test cases for lldb acting as a client over a gdb-remote connection.
- A GDBRemoteTestBase class that starts a mock GDB server and provides an easy way to check client packets
- A MockGDBServer that, via MockGDBServerResponder, can be made to issue server responses that test client behavior.
- Utility functions for handling common data encoding/decoding
- Utility functions for creating dummy targets from YAML files
----
Split from the review at https://reviews.llvm.org/D42145, which was a new feature that necessitated the new testing capabilities.
Reviewers: clayborg, labath
Reviewed By: clayborg, labath
Subscribers: hintonda, davide, jingham, krytarowski, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D42195
Patch by Owen Shaw <llvm@owenpshaw.net>
llvm-svn: 323636
Summary:
This mechanism was mostly redundant with the file-based .categories
mechanism, and it was interfering with it, as any test which implemented
a getCategories method would not inherit the filesystem categories.
This patch removes it. The existing categories are preserved either by
adding a .categories file, or using the @add_test_categories decorator.
Reviewers: jingham, clayborg, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D39515
llvm-svn: 317277
dwarf&dwo versions were doing it, but gmodules and dsym weren't. All
this function does right now is pass OS=Android to make when targeting
android. This enables us to run dotest without manually passing --env
OS=Android.
llvm-svn: 317130
Normal customer devices won't be able to run these tests, we're hoping to get
a public facing bot set up at some point. Both devices pass the testsuite without
any errors or failures.
I have seen some instability with the armv7 test runs, I may submit additional patches
to address this. arm64 looks good.
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 314132
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point. Both devices pass the testsuite without
any errors or failures.
I have seen some instability with the armv7 test runs, I may submit additional patches
to address this. arm64 looks good.
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 314038
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point.
There will be some smaller follow-on patches. The changes to tools/lldb-server are
verbose and I'm not thrilled with having to skip all of these tests manually.
There are a few places where I'm making the assumption that "armv7", "armv7k", "arm64"
means it's an ios device, and I need to review & clean these up with an OS check
as well. (Android will show up as "arm" and "aarch64" so by pure luck they shouldn't
cause problems, but it's not an assumption I want to rely on).
I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same. I know it's a rather large patch.
One change I had to make specifically for iOS devices was that debugserver can't
create files. There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file. They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.
llvm-svn: 313932
For remote targets we need to call registerSharedLibrariesWithTarget to
make sure they are installed alongside main executable. This also
required a small fixup in the the mentioned function as in this case
"One" was both a directory name and a library name template. I fixed it
to make sure it checks that the string refers to a file before it
assumed it was a full library path.
llvm-svn: 303248
r298203 make SBPlatform::MakeDirectory less recursive, which breaks the
test suite creation of test directory hierarchy creation on the remote
target. Since the function was never fully recursive, and the name does
not imply recursiveness, I fix the problem by modifying the test runner
to do the recursion manually.
I also make the runner complain more loudly when it fails to create the
directory -- previously it just printed the error to stdout and caused
most of the tests to hang, which is not very helpful in diagnosing the
problem.
llvm-svn: 298261
Summary:
This has been broken at least since the new test result framework was
added, which was over a year ago. It looks like nobody has missed it
since.
Removing this makes the gmodules handling code saner, as it already did
not know how to handle the multiple-compilers case.
My motivation for this is libc++ data formatters support on android -- I
am trying make a central way of determining whether libc++ tests can be
run, and without this, I would have to resort to similar hacks as the
gmodules code.
Reviewers: jingham, zturner
Subscribers: danalbert, tfiala, lldb-commits
Differential Revision: https://reviews.llvm.org/D30779
llvm-svn: 297811
Make sure we don't generate extremely long file names for test trace log
file, as this can cause path-too-long errors. As the compilers in the
android ndk are deeply nested, it's very easy to trigger these.
I chose to output at most 4 path components -- this should keep the full
path for common cases like /usr/bin/gcc with room to spare, and should
be enough to uniquely identify the compiler for more deeply nested
cases.
llvm-svn: 296870
This is a large API change that removes the two functions from
StreamString that return a std::string& and a const std::string&,
and instead provide one function which returns a StringRef.
Direct access to the underlying buffer violates the concept of
a "stream" which is intended to provide forward only access,
and makes porting to llvm::raw_ostream more difficult in the
future.
Differential Revision: https://reviews.llvm.org/D26698
llvm-svn: 287152
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.
This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.
Reviewers: tfiala, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D25887
llvm-svn: 285542
Summary:
dotest.py has a framework option that is not respected. This patch makes the framework path properly configurable via the --framework option.
This patch also adds a function to the lldbtest.Base class named "hasDarwinFramework" which allows us to not rely on the host platform to determine if a framework is present. If running on Darwin, and not building a framework, this will follow the *nix code paths which are appropriate for Darwin.
Reviewers: tfiala
Subscribers: lldb-commits, mgorny
Differential Revision: https://reviews.llvm.org/D25886
llvm-svn: 285541
This reverts commit r285357.
I committed this patch accidentally out of order. Will recommit when the change this depends on is landed.
llvm-svn: 285361
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.
This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.
Reviewers: tfiala, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D25887
llvm-svn: 285357
Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.
Reviewers: zturner, tfiala
Subscribers: labath, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25714
llvm-svn: 284893
This change introduces optional marking of the column within a source
line where a thread is stopped. This marking will show up when the
source code for a thread stop is displayed, when the debug info
knows the column information, and if the optional column marking is
enabled.
There are two separate methods for handling the marking of the stop
column:
* via ANSI terminal codes, which are added inline to the source line
display. The default ANSI mark-up is to underline the column.
* via a pure text-based caret that is added in the appropriate column
in a newly-inserted blank line underneath the source line in
question.
There are some new options that control how this all works.
* settings set stop-show-column
This takes one of 4 values:
* ansi-or-caret: use the ANSI terminal code mechanism if LLDB
is running with color enabled; if not, use the caret-based,
pure text method (see the "caret" mode below).
* ansi: only use the ANSI terminal code mechanism to highlight
the stop line. If LLDB is running with color disabled, no
stop column marking will occur.
* caret: only use the pure text caret method, which introduces
a newly-inserted line underneath the current line, where
the only character in the new line is a caret that highlights
the stop column in question.
* none: no stop column marking will be attempted.
* settings set stop-show-column-ansi-prefix
This is a text format that indicates the ANSI formatting
code to insert into the stream immediately preceding the
column where the stop column character will be marked up.
It defaults to ${ansi.underline}; however, it can contain
any valid LLDB format codes, e.g.
${ansi.fg.red}${ansi.bold}${ansi.underline}
* settings set stop-show-column-ansi-suffix
This is the text format that specifies the ANSI terminal
codes to end the markup that was started with the prefix
described above. It defaults to: ${ansi.normal}. This
should be sufficient for the common cases.
Significant leg-work was done by Adrian Prantl. (Thanks, Adrian!)
differential review: https://reviews.llvm.org/D20835
reviewers: clayborg, jingham
llvm-svn: 282105
*** to conform to clang-format’s LLVM style. This kind of mass change has
*** two obvious implications:
Firstly, merging this particular commit into a downstream fork may be a huge
effort. Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit. The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):
find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;
The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.
Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit. There are alternatives available that will attempt
to look through this change and find the appropriate prior commit. YMMV.
llvm-svn: 280751
Summary:
This is a slightly reworked version of D16322, which I had reverted because it did not do what it
advertised. Differences from the previous version are:
- moved the code for cleaning up the remote working dir to a later point as it was removing the
log file before we could get to it.
- specialised log downloading code for gdb-remote tests is not needed, as this will cover that
use case as well.
Reviewers: tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D21898
llvm-svn: 274491
This change adds the capability of building test inferiors
with the -gmodules flag to enable module debug info support.
Windows is excluded per @zturner.
Reviewers: granata.enrico, aprantl, zturner, labath
Subscribers: zturner, labath, lldb-commits
Differential Revision: http://reviews.llvm.org/D19998
llvm-svn: 270848
Summary:
print build errors nicely in test output
This test infrastructure change adds a new Python exception
for test subject builds that fail. The output of the build
command is captured and propagated to both the textual test
output display code and to the test event system.
The ResultsFormatter objects have been modified to do something
more useful with this information. The xUnit formatter
now replaces the non-informative Python build error stacktrace
with the build error content. The curses ResultsFormatter
prints a 'B' for build errors rather than 'E'.
The xUnit output, in particular, makes it much easier for
developers to track down test subject build errors that cause
test failures when reports come in from CI.
Reviewers: granata.enrico
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D20252
llvm-svn: 269525
This change moves all the test event handling and its related
ResultsFormatter classes out of the packages/Python/lldbsuite/test dir
into a packages/Python/lldbsuite/test_event package. Formatters are
moved into a sub-package under that.
I am limiting the scope of this change to just the motion and a few
minor issues caught by a static Python checker (e.g. removing unused
import statements).
This is a pre-step for adding package-level tests to the test event
system. I also intend to simplify test event results formatter selection
after I make sure this doesn't break anybody.
See:
http://reviews.llvm.org/D19288
Reviewed by:
Pavel Labath
llvm-svn: 266885
Previously we had 3 different method to run shell commands on the
target and 4 copy of code waiting until a given file appears on the
target device (used for syncronization). This CL merges these methods
to 1 run_platform_command and 1 wait_for_file_on_target functions
located in some utility classes.
Differential revision: http://reviews.llvm.org/D18789
llvm-svn: 265398
Summary:
Debug info is used only by the client and lldb-server tests do not even have the client component
running, as they communicate with the server directly. Therefore, running the tests for each
debug info type is unnecessarry.
This adds general ability to mark a test class as not dependent on debug info, and marks all
lldb-server tests as such.
Reviewers: tberghammer, tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D18598
llvm-svn: 265017
self.expect() had two problems:
- If there was a substrs argument, then it overwrote the variable containing
the command to run with the last substr. That meant nonsense command text in
testsuite errors.
- The actual output is not printed, which makes fixing testsuite failures a bit
annoying (you end up having to use the -tv arguments to dotest).
This fixes both of these issues. We could do even better, pretty-printing the
criteria for "correct" output, but this at least makes dealing with errors a bit
better.
llvm-svn: 262950
Summary:
This does not yet give us a clean testsuite run but it does help with:
1. Actually building on linux
2. Run the testsuite with over 70% tests passing on linux.
Reviewers: tfiala, labath, zturner
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D17182
llvm-svn: 260721
This doesn't attempt to move every decorator. The reason for
this is that it requires touching every single test file to import
decorators.py. I would like to do this in a followup patch, but
in the interest of keeping the patches as bite-sized as possible,
I've only attempted to move the underlying common decorators first.
A few tests call these directly, so those tests are updated as part
of this patch.
llvm-svn: 259807
My eventual goal is to move all of the test decorators to their
own module such as `decorators.py`. But some of the decorators
use existing functions in `lldbtest.py` and conceptually the
functions are probably more appropriately placed in lldbplatformutil.
Moreover, lldbtest.py is a huge file with a ton of random utility
functions scattered around, so this patch also workds toward the
goal of reducing the footprint of this one module to a more
reasonable size.
So this patch moves some of them over to lldbplatformutil with the
eventual goal of moving decorators over to their own module.
Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D16830
llvm-svn: 259680
This decorator was used in only one test, and it's behaviour was quite complicated. It skipped
if:
- test was remote
- platform was *not* android
I am not aware of anyone running tests with this configuration (and even then, I am not aware of
a reason why the test should not pass), but if TestLoadUnload starts breaking for you after this
commit, please disable the test with
@expectedFailureAll(remote=True, oslist=[YOUR_PLATFORM])
llvm-svn: 259642
Previously we were returning a tuple of (bool, skip_reason) from
the tuple function. This makes for some awkward code, especially
since a value of True for the first argument implies that the
second argument is None, and a value of False implies that the
second argument is not None. So it was basically redundant, and
with this patch we simply return the skip reason or None directly.
llvm-svn: 259590
This should be no functional change, just a refactoring of the
skip decorators to all centralize on a single function,
`skipTestIfFn` that does all the logic. This allows easier
maintenance of the decorators and also centralizes all the
hard-to-understand logic in one place.
Reviewed by: Pavel Labath
Differential Revision: http://reviews.llvm.org/D16741
llvm-svn: 259543
r259433 introduced a regression, where if a compiler is specified without a path (e.g., CC=clang,
relying on the fact that clang is in $PATH), then the test suite would fail (at the compiler
version detection step) because realpath would interpret this as a path relative to cwd). The fix
is to perform the $PATH expansion (via `which`) before the realpath step.
llvm-svn: 259484
This patch attempts to solve the Python 2 / Python 3 incompatibilities by
introducing a new `encoded_file` abstraction that we use instead of
`io.open()`. The problem with the builtin implementation of `io.open` is
that `read` and `write` accept and return `unicode` objects, which are not
always convenient to work with in Python 2. We solve this by making
`encoded_file.open()` return the same object returned by `io.open()` but
with hooked `read()` and `write()` methods. These hooked methods will
accept binary or text data, and conditionally convert what it gets to a
`unicode` object using the correct encoding. When calling `read()` it
also does any conversion necessary to convert the output back into the
native `string` type of the running python version.
Differential Revision: http://reviews.llvm.org/D16736
llvm-svn: 259379
Instead of opening the file in unicode mode, we need only encode
data which potentially has non-ASCII characters as UTF8 before
writing. This should work across both Python versions, and is
also far simpler than anything else discussed.
llvm-svn: 258969
* basestring is not a thing anymore. Must use `six.string_types`.
* Must use from __future__ import print_function in every new test
file.
llvm-svn: 258967
Previously the logic of skipIf and expectedFailure were 99%
the same, but they took different sets of arguments since they
were maintained separately, and had slightly differences in
their behavior. This makes everything consistent, there is now
only one real implementation, and the previous ones are changed
to use the single master implementation.
llvm-svn: 258966
SUMMARY:
Get the load address for the address given by symbol and function.
Earlier, this was done for function only, this patch does it for symbol too.
This patch also adds TestAvoidBreakpointInDelaySlot.py to test this change.
Reviewers: clayborg
Subscribers: labath, zturner, mohit.bhakkad, sagar, jaydeep, lldb-commits
Differential Revision: http://reviews.llvm.org/D16049
llvm-svn: 258919
Previously we were writing in the default encoding, which depends
on the operating system and is not guaranteed to be unicode aware.
On Python 3, this would lead to a situation where writing unicode
text to the log file generates an exception. The fix here is to
write session logs using the proper encoding, which incidentally
fixes another test, so xfail is removed from that.
llvm-svn: 258759
Since Unicode support is different in Py2 and Py3, Py3 was throwing
exceptions about being unable to decode the file with the default
encoding.
llvm-svn: 258588
Unfortunately, this turns out not to be working on the lldb-server tests, as there the server is
started in a different way. Since this was a bit of a hack to start with, I am removing it until
I can solve the problem more holistically.
llvm-svn: 258501
Summary:
We already have the ability to collect the server logs when doing local debugging. This enables
the collection of remote logs as well. This relies on specifying a relative path "server.log" for
LLDB_DEBUGSERVER_LOG_FILE when starting remote platform. Since we always set the platform working
directory to a fresh folder to avoid conflicts, the actual file path will always be different and
we can pick the logs up from there.
Reviewers: tfiala
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D16322
llvm-svn: 258414
Jonas Devlieghere