Currently most of the test files have a separate dwarf and a separate
dsym test with almost identical content (only the build step is
different). With adding dwo symbol file handling to the test suit it
would increase this to a 3-way duplication. The purpose of this change
is to eliminate this redundancy with generating 2 test case (one dwarf
and one dsym) for each test function specified (dwo handling will be
added at a later commit).
Main design goals:
* There should be no boilerplate code in each test file to support the
multiple debug info in most of the tests (custom scenarios are
acceptable in special cases) so adding a new test case is easier and
we can't miss one of the debug info type.
* In case of a test failure, the debug symbols used during the test run
have to be cleanly visible from the output of dotest.py to make
debugging easier both from build bot logs and from local test runs
* Each test case should have a unique, fully qualified name so we can
run exactly 1 test with "-f <test-case>.<test-function>" syntax
* Test output should be grouped based on test files the same way as it
happens now (displaying dwarf/dsym results separately isn't
preferable)
Proposed solution (main logic in lldbtest.py, rest of them are test
cases fixed up for the new style):
* Have only 1 test fuction in the test files what will run for all
debug info separately and this test function should call just
"self.build(...)" to build an inferior with the right debug info
* When a class is created by python (the class object, not the class
instance), we will generate a new test method for each debug info
format in the test class with the name "<test-function>_<debug-info>"
and remove the original test method. This way unittest2 see multiple
test methods (1 for each debug info, pretty much as of now) and will
handle the test selection and the failure reporting correctly (the
debug info will be visible from the end of the test name)
* Add new annotation @no_debug_info_test to disable the generation of
multiple tests for each debug info format when the test don't have an
inferior
Differential revision: http://reviews.llvm.org/D13028
llvm-svn: 248883
Summary:
If test is decorated with expectedFlakey*, run teardown and setup before retry
Don't run retry if the test is already decorated with xfail or skip
Test Plan:
Mark TestMultithreaded.test_sb_api_listener_event_process_state as expectedFlakey
Run ./dotest.py -p TestMultithreaded.py -A x86_64 -C gcc4.9.2
Reviewers: vharron, tberghammer
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D10830
llvm-svn: 241202
SUMMARY
Flakey tests get two chances to pass
Also, switched a bunch of tests to use new decorator.
TEST PLAN
Add one of these decorators to a test
Edit a test to pass on the first invocation, confirm test appears as pass
Edit a test to pass on the first invocation, pass on the second, confirm test appears as xfail
Edit a test to fail on two consecutive runs, confirm test appears in results as fail/error
Differential Revision: http://reviews.llvm.org/D10721
llvm-svn: 240789
Summary:
Currently, the local-only path fails about 50% of the tests, which means that: a) nobody is using
it; and b) the remote debugging path is much more stable. This commit removes the local-only
linux debugging code (ProcessLinux) and makes remote-loopback the only way to debug local
applications (the same architecture as OSX). The ProcessPOSIX code is moved to the FreeBSD
directory, which is now the only user of this class. Hopefully, FreeBSD will soon move to the new
architecture as well and then this code can be removed completely.
Test Plan: Test suite passes via remote stub.
Reviewers: emaste, vharron, ovyalov, clayborg
Subscribers: tberghammer, emaste, lldb-commits
Differential Revision: http://reviews.llvm.org/D10661
llvm-svn: 240543
Summary:
This change adds all the necessary infrastructure required to
selectively enable and make TestLoadUnload work for Android. One test,
which tests the module search paths, is still kept disabled for remote
as search paths (because of module caching) are local anyway.
Reviewers: tberghammer
Reviewed By: tberghammer
Subscribers: emaste, lldb-commits, tberghammer
Differential Revision: http://reviews.llvm.org/D10469
llvm-svn: 239965
Summary: Opened files on Windows cannot be modified, so this test doesn't make sense.
Reviewers: ovyalov, zturner, flackr, clayborg
Reviewed By: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D10295
llvm-svn: 239359
Summary: Updated `append_to_remote_wd` to work for both remote and local.
Reviewers: clayborg, ovyalov
Reviewed By: ovyalov
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10288
llvm-svn: 239203
Summary:
This change also adds the infrastructure required to specify the API
levels for which tests should be skipped.
Reviewers: chying, labath
Reviewed By: labath
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10282
llvm-svn: 239183
Summary:
urlparse.ParseResult.hostname has only lowercase characters even if the
input URL had uppercase characters. Since Android device IDs can have
uppercase characters as well, use urlparse.ParseResult.netloc instead
and extract the device ID from it.
This change also improves the error message when lookup of the Android
device's API fails.
Reviewers: chaoren
Reviewed By: chaoren
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10278
llvm-svn: 239173
Summary:
This change adds the infrastructure to mark tests as xfail for specific
Android API levels.
Test Plan: dotest.py TestChangeProcessGroup on an Android API 16 device.
Reviewers: chying, labath, chaoren
Reviewed By: labath, chaoren
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10261
llvm-svn: 239126
Summary:
Before:
AssertionError: False is not True : Process is launched successfully
After:
AssertionError: False is not True : Command 'run a.out' failed.
>>> error: invalid target, create a target using the 'target create' command
>>> Process could not be launched successfully
Reviewers: clayborg
Reviewed By: clayborg
Subscribers: lldb-commits, vharron
Differential Revision: http://reviews.llvm.org/D9948
llvm-svn: 238363
On non-Windows platforms, os.rename() will silently replace the
destination file if it already exists. On Windows, it doesn't do
this, and the filesystem has no mechanism to simulate the same type
of atomic rename operation. So on Windows, delete the file first
before calling os.rename().
llvm-svn: 238239
`lock` depends on `fcntl`, which doesn't exist on Windows. Until
someone implements an equivalent locking mechanism on Windows, we
can't have lock imported globally.
llvm-svn: 237946
SUMMARY
dosep.py starts lots and lots of dotest instances.
This option helps you find if two (or more) dotest instances are using
the same directory at the same time.
Enable it to cause test failures and stderr messages if dotest
instances try to run in the same directory simultaneously.
It is disabled by default because it litters the test directories with
".dirlock" files
TEST PLAN
Set lldbtest.debug_confirm_directory_exclusivity = True
run ./dosep.py
Differential Revision: http://reviews.llvm.org/D9868
llvm-svn: 237935
That way, if the test gets killed (by a dosep timeout) we get to see the session
trace
Test Plan:
Run dotest.py
Kill it while it's running, check the session dir for Test* files
Differential Revision: http://reviews.llvm.org/D9845
llvm-svn: 237926
The lldb executable was referenced through the code by 7 different
(effectively) global variables.
global lldbExecutablePath
global lldbExecutable
os.environ['LLDB_EXEC']
os.environ['LLDB_TEST']
dotest.lldbExec
dotest.lldbHere
lldbtest.lldbExec
This change uses one global variable lldbtest_config.lldbExec to
replace them all.
Differential Revision: http://reviews.llvm.org/D9817
llvm-svn: 237600
When compiling programs for the test suite we currently choose which stdlib to
use based on the host platform, but should be basing this on the target
platform.
Test Plan: ./dotest.py $DOTEST_OPTS -t -p TestThreadExit.py
This test previously failed mac->linux most of the time due to using the mac
host's atomic declaration.
Differential Revision: http://reviews.llvm.org/D9797
llvm-svn: 237466
TestPluginCommands.py attempts to build a library which links against the host
built lldb library. This will only work if the remote is compatible with
binaries produced by the host.
Test Plan:
./dotest.py $DOTEST_OPTS -v -t -p TestPluginCommands.py
Test is skipped if remote platform is incompatible with host platform (i.e. mac
-> linux).
Differential Revision: http://reviews.llvm.org/D9770
llvm-svn: 237444
Select expected remote libc++ library name based on remote platform. This is
used to verify libc++ is in the loaded image list by various tests which use it.
Test Plan:
Passes the following tests mac -> linux with this patch:
TestDataFormatterLibccIterator.py
TestDataFormatterLibccMap.py
TestDataFormatterLibccMultiMap.py
TestDataFormatterLibcxxMultiSet.py
TestDataFormatterLibcxxSet.py
TestDataFormatterUnordered.py
Differential Revision: http://reviews.llvm.org/D9759
llvm-svn: 237288
This is a very minor change.
Just reshuffle the elements of the log filename so that the test name
comes before the arch/compiler
Before:
Failure-x86_64-clang-TestCModules.CModulesTestCase.test_expr_with_dwarf.log
After:
Failure-TestCModules.CModulesTestCase.test_expr_with_dwarf-x86_64-clang.log
This way, results are grouped by test name, not by arch. I think this
is much more useful.
Differential Revision: http://reviews.llvm.org/D9648
llvm-svn: 237076
Summary:
Since we don't yet have remote windows debugging, it should be safe to assume
that the remote target uses unix path separators.
Reviewers: ovyalov, zturner, clayborg, vharron
Reviewed By: vharron
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D9633
llvm-svn: 237006
Summary:
New dotest options that allow arbitrary log channels and
categories to be enabled. Also enables logging for locally run
debug servers.
Log messages are separated into separate files per test case.
(this makes it possible to log in dosep runs)
These new log files are stored side-by-side with trace files in the
session directory.
These files are deleted by default if the test run is successful.
If --log-success is specified, even successful logs are retained.
--log-success is useful for creating reference log files.
Test Plan:
add '--channel "lldb all" --channel "gdb-remote packets" --log-success'
to your dotest options
Tested on OSX and Linux
Differential Revision: http://reviews.llvm.org/D9594
llvm-svn: 236956
Refactored TestInitializerList to not be an inline test.
Refactored Makefiles to use USE_LIBCPP instead of adding FLAGS directly
Fixed copy/paste error in TestDataFormatterUnordered class name
Differenttial Revision: http://reviews.llvm.org/D9426
llvm-svn: 236401
These tests link against host lldb API. Compiler's target triple
must match liblldb triple. Instead of naively skipping i386, I added
a check of the liblldb arch against the compiler target arch.
This is useful for 32 bit API builds (planned for Windows)
Since remote is disabled, we can assume the os is the same.
Also, removed skipIfLinuxClang because it's passing
llvm-svn: 236396
Summary:
- add decorator functions to xfail and skip test on specific os, architecture and version of comipler
- xfail failing test with gcc-4.9.2 on linux
- add one usage of skipIf function
Test Plan:
Run tests with different archs, and version of compilers to verify decorator function working as expected
Run tests with gcc-4.9.2 and no failure reported
Reviewers: sivachandra, ovyalov, vharron, chaoren
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D8851
llvm-svn: 235368
Enrico Granata