This is useful for checking runtime properties of the target system.
This is a partial re-application of 3ea9450bda. This part was tested
to work on a Windows host with a SSH executor.
The integration between CMake and executor selection in the new format
wasn't very flexible -- only the default executor and SSH executors were
supported.
This patch makes it possible to specify arbitrary executors with the new
format. With the new testing format, a custom executor is just a script
that gets called with a command-line to execute, and some arguments like
--env, --codesign_identity and --execdir. As such, the default executor
is just run.py.
Remote execution with the SSH executor can be achived by specifying
LIBCXX_EXECUTOR="<path-to-ssh.py> --host <host>". Similarly, arbitrary
scripts can be provided.
Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency. This
simplifies the test harness significantly, by making %T the directory
that contains everything required to run a test. This also removes the
need for the %{file_dependencies} substitution, which is removed by this
patch.
Furthermore, this patch also changes the harness to execute tests locally
inside %T, so as to avoid creating a separate directory for no purpose.
This will allow simplifying executors by always just copying the whole
%T, and assuming that all file dependencies are contained in it.
Superseeds https://reviews.llvm.org/D78245, which tried to make %T unique
in Lit, but which encountered push back.
Unlike parameters in litConfig.params, the config isn't shared across
all test suites. For example, if we want to enable exceptions in the
tests for libcxxabi, but not in the tests for libcxx, we can't set the
enable_exceptions parameter in the litConfig object, cause it will be
used by both. Instead, setting it inside the config object solves that
problem.
The availability markup for bad_optional_access marked it as being added
in MacOS 10.14 and aligned releases, however it appears to have been added
in Mac OS 10.13 and aligned releases.
Otherwise, if %{flags} contain other files like static libraries, those
files are treated as C++ source files instead of object files, and the
compiler gets all confused.
Before this patch, we tried detecting whether small atomics were available
without linking against libatomic. However, that's not really what we want
to know -- instead, we want to know what's required in order to support
atomics fully, which is to link against libatomic when it's provided.
That is both much simpler, and it doesn't suffer the problem that we would
not link against libatomic when small atomics didn't require it, which
lead to non-lockfree atomics never working.
Furthermore, because we understand that some platforms might not want to
(or be able to) ship non-lockfree atomics, we add that notion to the test
suite, independently of a potential extern library.
After this patch, we therefore:
(1) Link against libatomic when it is provided
(2) Independently detect whether non-lockfree atomics are supported in
the test suite, regardless of whether that means we're linking against
an external library or not (which is an implementation detail).
Differential Revision: https://reviews.llvm.org/D81190
It is legitimate for the test suite to use types that are slow to use
with std::atomic, since we need coverage for those too. If we don't
disable the warning, it is promoted to an error, which prevents us
from testing such types.
This commit adds missing support for constexpr in std::array under all
standard modes up to and including C++20. It also transforms the <array>
tests to check for constexpr-friendliness under the right standard modes.
Fixes https://llvm.org/PR40124
Fixes rdar://57522096
Supersedes https://reviews.llvm.org/D60666
Differential Revision: https://reviews.llvm.org/D80452
Summary:
As described in the bug report:
The commit a8b9f59e8caf378d56e8bfcecdb22184cdabf42d "Implement feature test macros using a script" added test features macros for libc++. Among others, it added `__cpp_lib_hardware_interference_size`. However, there is nothing like std::hardware_constructive_interference_size nor std::hardware_destructive_interference_size, that should be in header <new>.
* https://bugs.llvm.org/show_bug.cgi?id=41423
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D80431
Instead of linking the tests against a library in some version of the
SDK, always link against the latest library, but still run against the
specified back-deployment target dylib.
This makes more sense since what we're really trying to test is that
the current library can be used to produce binaries that run on some
deployment target -- not that linking against the library in some
previous SDK makes that possible.
This solves an additional issue that when linking against a system dylib,
the -rpath argument given to the tests is ignored because the install_name
of the system library we link against is absolute.
rdar://63241847
We already set it using -rpath when linking test executables, and using
DYLD_LIBRARY_PATH causes problems when running other commands that
shouldn't run against the just-built libc++ (e.g. `ls` in a ShTest).
rdar://63241847
Since we're using the new testing format, DYLD_LIBRARY_PATH is not passed
to the compiler -- it's only passed to the programs we run as an argument
to the %{exec} substitution.
This is already handled by setting cxx_runtime_root instead -- I don't
see a reason to have two ways of setting the runtime path of the library
we're running against.
Because of Python's funny scoping rules with lambdas, we were always
using the value of `macro` as set in the last iteration of the loop.
This problem was introduced by e7bdfba4f0.
Otherwise, specifying (for example) the libc++.dylib from macos10.13
but the libc++abi.dylib from macos10.12 would end up adding library
paths for both the 10.12 and 10.13 dylibs, which would each contain
a copy of both libc++abi.dylib and libc++.dylib. By using a separate
directory for libc++.dylib and libc++abi.dylib, those do not conflict
anymore.
The back-deployment roots were updated to match this change.
When grepping for unused features in the test suite, we will now find
those features and where they are defined, as opposed to thinking they
are dead features.
Running `export` when there is no environment variable to export will
cause the environment on the remote host to be printed. We don't want
that, so don't run any `export` command on the host when there's no env.
This commit removes minor features of the test suite that I've never
seen used and that are basically just a maintenance burden:
- color_diagnostics: Diagnostics are colored by default when running
from a terminal, and not colored otherwise. This is the right behavior.
Being able to tweak this has minor value, and could be achieved by
modifying the %{compile_flags} instead if absolutely needed.
- ccache: This can be achieved by using a wrapper for the %{cxx}
substitution.
- _dump_macros_verbose is just a dead function now.
When building with modules, always enable local submodule visibility.
It used to be disabled on Apple platforms, but it seems like we want
to use the same flags on Apple and Linux now (see https://reviews.llvm.org/D74892).
This commit migrates some of the Lit features from config.py to the new
DSL. This simplifies config.py and is a first step towards defining all
the features using the DSL instead of the complex logic in config.py.
Differential Revision: https://reviews.llvm.org/D78382
The libcxx.util utilities don't work properly, and we should remove them
when we get rid of compiler.py. In particular, libcxx.util.to_string
appears to be completely broken.
The internal Lit shell requires the current working directory to exist.
This didn't show up locally because the directories were already created
by previous runs of the tests.
Tests that require support for Clang-verify are already marked as such
explicitly by their extension, which is .verify.cpp. Requiring the use
of an explicit Lit feature is, after thought, not really helpful.
This is a change in design: we have been bitten in the past by tests not
being enabled when we thought they were. However, the issue was mostly
with file extensions being ignored. The fix for that is not to blindly
require explicit features all the time, but instead to report all files
that are in the suite but that don't match any known test format. This
can be implemented in a follow-up patch.
This reverts commit 51a60ed14c, since the test still doesn't pass on
Windows. Marking the test as UNSUPORTED on Windows again until I've
figured out the problem.
Since 88af3ddb1e, libc++ will prefer Python 3 when available. It is
available on Apple platforms, so subprocess.check_output will return
bytes instead of str. This lead to comparisons against str to be false,
and the MacOS platform not being detected properly.
This allows defining Lit features that can be enabled or disabled based
on compiler support, and parameters that are passed on the command line.
The main benefits are:
- Feature detection is entirely based on the substitutions provided in
the TestingConfig object, which is simpler and decouples it from the
complicated compiler emulation infrastructure.
- The syntax is declarative, which makes it easy to see what features
and parameters are accepted by the test suite. This is significantly
less entangled than the current config.py logic.
- Since feature detection is based on substitutions, it works really
well on top of the new format, and custom Lit configurations can be
created easily without being based on `config.py`.
This commit is a reapplication of 6d58030c8c, which was reverted in
8f24c4b72f because it broke Python 3 support. This re-application
supports Python 3.
Differential Revision: https://reviews.llvm.org/D78381
Instead of using the libc++ headers provided alongside the toolchain,
use those in the sibling libcxx directory that we know is checked out.
Before the days of the monorepo, we couldn't assume that the libc++
repository was present when building libcxxabi. Since we can now make
that assumption, it's always better to use the version of libc++ that
is in lockstep with libc++abi, to avoid subtle bugs.
This allows defining Lit features that can be enabled or disabled based
on compiler support, and parameters that are passed on the command line.
The main benefits are:
- Feature detection is entirely based on the substitutions provided in
the TestingConfig object, which is simpler and decouples it from the
complicated compiler emulation infrastructure.
- The syntax is declarative, which makes it easy to see what features
and parameters are accepted by the test suite. This is significantly
less entangled than the current config.py logic.
- Since feature detection is based on substitutions, it works really
well on top of the new format, and custom Lit configurations can be
created easily without being based on `config.py`.
Differential Revision: https://reviews.llvm.org/D78381
Instead of having different names for the same Lit feature accross code
bases, use the same name everywhere. This NFC commit is in preparation
for a refactor where all three projects will be using the same Lit
feature detection logic, and hence it won't be convenient to use
different names for the feature.
Differential Revision: https://reviews.llvm.org/D78370
When the new libc++ test format was enabled, warnings were accidentally
dropped cause they were not part of the %{compile_flags} substitution.
This commit adds them back, however `-Werror` is only used for non-verify
tests (cause it doesn't make sense for verify tests).
This commit is a re-application of 20fd624380, which was reverted in
5ec6fdb058 because it broke the C++03 bot. This failure should have
been fixed in b4fb705e77.
When the new libc++ test format was enabled, warnings were accidentally
dropped cause they were not part of the %{compile_flags} substitution.
This commit adds them back, however `-Werror` is only used for non-verify
tests (cause it doesn't make sense for verify tests).
Summary:
This patch add the dataflow option to LLVM_USE_SANITIZER and documents
it.
Tested via check-cxx (wip to fix the errors).
Reviewers: morehouse, #libc!
Subscribers: mgorny, cfe-commits, libcxx-commits
Tags: #clang, #libc
Differential Revision: https://reviews.llvm.org/D78390
This was originally committed as f8452ddfcc and reverted in 7cb1aa9d93.
The issue was that shell builtins were being escaped too, and apparently
Bash won't execute a builtin when it is quoted e.g. '!'. Instead, it
thinks it's a command and it can't find it.
Re-committing the change with that issue fixed.
Cherrypick the upstream fix commit a77d5f7 onto llvm/utils/benchmark
and libcxx/utils/google-benchmark.
This fixes LLVM's 32-bit RISC-V compilation, and the issues
mentioned in https://github.com/google/benchmark/pull/955
An additional cherrypick of ecc1685 fixes some minor formatting
issues introduced by the preceding commit.
Differential Revision: https://reviews.llvm.org/D78084
Instead of creating Lit features for all __config_site macros automatically,
only do so for macros that generate features actually used in the test
suite. This makes it easier to know which ones are supported by the test
suite at a glance.
Note that the `libcpp-abi-version-vN` is dropped altogether, but it
wasn't used anywhere.
These substitutions are strongly tied to the operation of the test
format, so it makes sense to have them defined by the test format
instead of the Lit configuration. They should be defined regardless
of which configuration is in use.
The new format requires using an external shell, and as we transition
and we can simplify config.py as we transition to the new format. Also,
frankly, I'd be quite surprised if that setting was still working anyway
because we have several .sh.cpp tests that likely don't work in Lit's
internal shell.
By renaming .fail.cpp tests that don't need clang-verify to .compile.fail.cpp,
the new test format will not try to compile these tests with clang-verify,
and the old test format will work just the same. However, this allows
removing a workaround that requires parsing each test looking for
clang-verify markup.
After this change, a .fail.cpp test should always have clang-verify markup.
When clang-verify is not supported by the compiler, we will just check that
these tests fail to compile. When clang-verify is supported, these tests
will be compiled with clang-verify whether they have markup or not (so
they should have markup, or they will fail).
This simplifies the test suite and also ensures that all of our .fail.cpp
tests provide clang-verify markup. If it's impossible for a test to have
clang-verify markup, it can be moved to a .compile.fail.cpp test, which
are unconditionally just checked for compilation failure.
With this patch, .verify.cpp tests explicitly require clang-verify, but
no other test types require clang-verify out of the box. This will allow
making several .fail.cpp tests that don't have any clang-verify markup
to be just .compile.fail.cpp tests, which in turn should allow removing
a long standing workaround that requires parsing tests to detect whether
they have any clang-verify markup in them.
There are no such tests in the libc++ test suite, and I want to move
away from `.fail.cpp` tests (in favour of something else) too, which
require a workaround.
Instead of spamming a bunch of available features that are not actually
used anywhere, only set those that are actually used in the test suite.
In the future, this should probably be based on the target triple only,
with the ability to have wildcards in the triple.
Remove mentions of the ValgrindExecutor, which doesn't exist. That
executor is literally nowhere in the code base, so this is dead code
as far as we're concerned.
Also, inline a one-liner function that was called exactly once.
The libc++ test suite currently defines several features that are not
used anywhere in the tests, or that are redundant with other features.
For the purpose of simplifying config.py and to ease the bring up of a
new configuration, this commit removes some of these features:
- rename dylib-has-no-filesystem to c++filesystem-disabled, which exists
- rename apple-darwin to just darwin, which is already set
- remove useless setting of libstdc++, which is already set correctly
- remove libcpp-abi-unstable, which is not used anywhere
- remove the glibc-XXX features, which are not used anywhere
Louis Dionne