The error is
```
NotADirectoryError: [Errno 20] Not a directory: '<build-dir>/unittests/Analysis/./AnalysisTests/0/40'
```
Exclude unittests when collecting features because
unittests don't make use of feature keywords.
The diagnostics engine is very smart about being passed a NamedDecl to
print as part of a diagnostic; it gets the "right" form of the name,
quotes it properly, etc. However, the result of using an unnamed tag
declaration was to print '' instead of anything useful.
This patch causes us to print the same information we'd have gotten if
we had printed the type of the declaration rather than the name of it,
as that's the most relevant information we can display.
Differential Revision: https://reviews.llvm.org/D134813
Fix the use_lld() to use llvm_shlib_dir similarly to how use_clang()
does it. This fixes use_lld() wrongly prepending llvm_libs_dir,
i.e. the directory with system-installed LLVM libraries before
the build directory of standalone build. As a result, the shared
libraries from an earlier version of clang end up being used instead of
the newly built version when running the test suite prior to installing.
To reproduce the problem, build and install LLVM with dylibs first,
e.g.:
cmake ../llvm -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel \
-DCMAKE_INSTALL_PREFIX="${HOME}"/llvm-test \
-DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON
ninja install
Then build clang against that installation and run tests:
export LD_LIBRARY_PATH=~/llvm-test/lib
export PATh=~/llvm-test/bin:"${PATH}"
cmake ../clang -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel \
-DCMAKE_INSTALL_PREFIX="${HOME}"/llvm-test \
-DCLANG_LINK_CLANG_DYLIB=ON -DLLVM_BUILD_TESTS=ON \
-DLLVM_EXTERNAL_LIT="${PWD}"/bin/llvm-lit
ninja check-clang
The tests will be run with LD_LIBRARY_PATH of:
/home/${USER}/llvm-test/lib:/home/${USER}/llvm-project/build-clang/lib
As a result, installed libclang-cpp will take precedence over the one
from build dir. With the patch, the correct path is used, i.e.:
/home/${USER}/llvm-project/build-clang/lib:/home/${USER}/llvm-test/lib
Differential Revision: https://reviews.llvm.org/D135368
Move the `%clang_dxc` substitution from local definition in clang/test
to lit's `llvm/config.py` module where all other driver definitions
are found. This improves consistency and makes it easier to control
global clang options.
Differential Revision: https://reviews.llvm.org/D134871
Due to CMake mis-configurations, some gtest binaries may be added to the test
list more than once. This patch makes lit avoid such cases and issues a
warning when it happens.
These directives define per-test lit substitutions. The concept was
discussed at
<https://discourse.llvm.org/t/iterating-lit-run-lines/62596/10>.
For example, the following directives can be inserted into a test file
to define `%{cflags}` and `%{fcflags}` substitutions with empty
initial values, which serve as the parameters of another newly defined
`%{check}` substitution:
```
// DEFINE: %{cflags} =
// DEFINE: %{fcflags} =
// DEFINE: %{check} = %clang_cc1 %{cflags} -emit-llvm -o - %s | \
// DEFINE: FileCheck %{fcflags} %s
```
The following directives then redefine the parameters before each use
of `%{check}`:
```
// REDEFINE: %{cflags} = -foo
// REDEFINE: %{fcflags} = -check-prefix=FOO
// RUN: %{check}
// REDEFINE: %{cflags} = -bar
// REDEFINE: %{fcflags} = -check-prefix=BAR
// RUN: %{check}
```
Of course, `%{check}` would typically be more elaborate, increasing
the benefit of the reuse.
One issue is that the strings `DEFINE:` and `REDEFINE:` already appear
in 5 tests. This patch adjusts those tests not to use those strings.
Our prediction is that, in the vast majority of cases, if a test
author mistakenly uses one of those strings for another purpose, the
text appearing after the string will not happen to have the syntax
required for these directives. Thus, the test author will discover
the mistake immediately when lit reports the syntax error.
This patch also expands the documentation on existing lit substitution
behavior.
Reviewed By: jhenderson, MaskRay, awarzynski
Differential Revision: https://reviews.llvm.org/D132513
Make the default module cache path invalid when running lit tests so
that tests are forced to provide a cache path. This avoids accidentally
escaping to the system default location, and would have caught the
failure recently found in ClangScanDeps/multiple-commands.c.
Differential Revision: https://reviews.llvm.org/D133622
There are a variety of issues with using GTest sharding by default for users of
`lit` using the Google Test formatter as mentioned in
https://github.com/llvm/llvm-project/issues/56492 and
https://github.com/llvm/llvm-project/issues/56491.
Currently, there is no way for users to explicitly control the sharding
behavior, even with the environment variables that GTest provides. This patch
teaches the `googletest` formatter to actually respect `GTEST_TOTAL_SHARDS`
and `GTEST_SHARD_INDEX` environment variables if they are set.
In practice, we could go one step further and not do any of the post-processing
of the JSON files if `GTEST_TOTAL_SHARDS` is `1` for example, but that it left
as a follow-up if desired. There may be preferred alternative approaches to
disabling sharding entirely through another mechanism, such as a lit config
variable.
Differential Revision: https://reviews.llvm.org/D133542
This allows reading arguments from file using the response file syntax.
We would like to use this in the LLVM build to pass test suites from
subbuilds.
Differential Revision: https://reviews.llvm.org/D132437
D98179 added a mechanism to sort tests by test time to run slow tests
early, increasing potential parallelism. It also added a feature where
negative tests would be marked as negative, allowing subsequent test
runs to run them earlier. Unfortunately it never actually stored the
negative time, even if all the other code seemed to be inplace to sort
them early. Luckily the fix seems simple.
Differential Revision: https://reviews.llvm.org/D130570
Many distros ship zlib with the IBM Z deflate hardware acceleration
patch [1]. Sometimes it's desirable to disable the acceleration, for
example, for reproducibility. This can be done by exporting DFLTCC=0.
llvm-lit clears this environment variable, which causes
compress-debug-sections-zlib.test fail on z15 and later machines. Add
DFLTCC to the list of variables to keep.
[1] https://github.com/madler/zlib/pull/410
Reviewed By: abrachet
Differential Revision: https://reviews.llvm.org/D130253
While the inferred host triple for macOS is something like
<arch>-apple-darwin, it's also valid to have <arch>-apple-macos.
Currently that globally changes whether an SDKROOT is provided in tests,
so make this check more portable.
Differential Revision: https://reviews.llvm.org/D129684
This environment variable can be set to control module
caching. It disables caching by setting the variable
empty. As such, it needs to be handled differently
from other environment variables here which are
assumed to not be empty.
Currently several buildbots give unsymbolized traces on crash.
I suspect these are configuring the symbolizer in this way and regressed in
D122251 or thereabouts.
Trying this coupled with a reland of patch that failed on a couple of bots with
no useful stacktrace...
This behaves just like the sh/cmd.exe equivalents.
pushd/popd are useful to verify path handling of the driver,
typically testing prefix maps or relative path handling.
Differential Revision: https://reviews.llvm.org/D125502
This syntax allows to modify RUN lines based on features
available. For example:
RUN: ... | FileCheck %s --check-prefix=%if windows %{CHECK-W%} %else %{CHECK-NON-W%}
CHECK-W: ...
CHECK-NON-W: ...
The whole command can be put under %if ... %else:
RUN: %if tool_available %{ %tool %} %else %{ true %}
or:
RUN: %if tool_available %{ %tool %}
If tool_available feature is missing, we'll have an empty command in
this RUN line. LIT used to emit an error for empty commands, but now
it treats such commands as nop in all cases.
Multi-line expressions are also supported:
RUN: %if tool_available %{ \
RUN: %tool \
RUN: %} %else %{ \
RUN: true \
RUN: %}
Background and motivation:
D121727 [NVPTX] Integrate ptxas to LIT tests
https://reviews.llvm.org/D121727
Differential Revision: https://reviews.llvm.org/D122569
When a unit test crashes or timeout, print the shard's stdout and
stderr. When a unit test fails, attaches the test's output to the LIT
output to help debugging.
While at it, concatenating shard's environment variables using space
instead of newline to make the reproducer script user friendly.
Based on D123797. (Thanks to @lenary)
This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation).
The shards are executed by the test runner and the results are stored in the
json format supported by the GoogleTest. Later in the test reporting stage,
all test results in the json file are retrieved to continue the test results
summary etc.
On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s.
On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s.
Reviewed By: yln, rnk, abrachet
Differential Revision: https://reviews.llvm.org/D122251
This helps lit unit test performance by a lot, especially on windows. The performance gain comes from launching one gtest executable for many subtests instead of one (this is the current situation).
The shards are executed by the test runner and the results are stored in the
json format supported by the GoogleTest. Later in the test reporting stage,
all test results in the json file are retrieved to continue the test results
summary etc.
On my Win10 desktop, before this patch: `check-clang-unit`: 177s, `check-llvm-unit`: 38s; after this patch: `check-clang-unit`: 37s, `check-llvm-unit`: 11s.
On my Linux machine, before this patch: `check-clang-unit`: 46s, `check-llvm-unit`: 8s; after this patch: `check-clang-unit`: 7s, `check-llvm-unit`: 4s.
Reviewed By: yln, rnk
Differential Revision: https://reviews.llvm.org/D122251
A version of this logic appears in ~every lit.site.cfg.in (28 copies total).
This patch just removes two, but I'll update the rest of llvm-project next.
Differential Revision: https://reviews.llvm.org/D121427
This adds 2 new lit helpers `%{fs-src-root}` and `%{fs-sep}`, these
allow writing tests that correctly handle slashes on Windows. In the
case of tests like clang/test/CodeGen/debug-prefix-map.c, these are
unable to correctly test behavior on both platforms, unless they fork
and add OS requirements, because the relevant logic hits host specific
codepaths like checking if paths are absolute.
Differential Revision: https://reviews.llvm.org/D111457
This takes care of normalizing newlines back to single LF instead
of CRLF.
Fix up a couple tests that accidentally pass binary data to stdout.
Differential Revision: https://reviews.llvm.org/D120623
This takes care of normalizing newlines back to single LF instead
of CRLF.
This on itself breaks on a couple tests that accidentally seem to
be writing binary data to stdout; make sure those cases are piped
to /dev/null instead of actually written to a terminal.
Differential Revision: https://reviews.llvm.org/D120623
Yuanfang Chen