Commit Graph

198 Commits

Author SHA1 Message Date
Louis Dionne 4cd6ca102a [libc++] NFC: Normalize `#endif //` comment indentation 2021-04-20 12:03:32 -04:00
Arthur O'Dwyer d7eb797ea5 [libc++] [test] Detect an improperly noexcept'ed __decay_copy.
`__decay_copy` is used by `std::thread`'s constructor to copy its arguments
into the new thread. If `__decay_copy` claims to be noexcept, but then
copying the argument does actually throw, we'd call std::terminate instead
of passing this test. (And I've verified that adding an unconditional `noexcept`
to `__decay_copy` does indeed fail this test.)

Differential Revision: https://reviews.llvm.org/D100277
2021-04-12 12:28:01 -04:00
Martin Storsjö f8013a35b6 [libcxx] [test] Make the condvar wait_for tests a bit more understandable. NFC.
This was requested in the review of D99175; rename the "runs"
variable to clarify what it means wrt the test, and move updating of
it to the main function to clarify its behaviour wrt the two runs
further.

Differential Revision: https://reviews.llvm.org/D99768
2021-04-02 10:46:15 +03:00
Martin Storsjö 01aa9e1f6e [libcxx] [test] Make the condvar wait_for tests less brittle
These seem to fail occasionally (they are marked as possibly requiring
a retry).

When doing a condvar wait_for(), it can wake up before the timeout
as a spurious wakeup. In these cases, the wait_for() method returns that
the timeout wasn't hit, and the test reruns another wait_for().

On Windows, it seems like the wait_for() operation often can end up
returning slightly before the intended deadline - when intending to
wait for 250 milliseconds, it can return after e.g. 235 milliseconds.
In these cases, the wait_for() doesn't indicate a timeout.

Previously, the test then reran a new wait_for() for a full 250
milliseconds each time. So for N consecutive wakeups slightly too early,
we'd wait for (N+1)*250 milliseconds. Now it only reruns wait_for() for
the remaining intended wait duration.

Differential Revision: https://reviews.llvm.org/D99175
2021-04-01 21:42:11 +03:00
Martin Storsjö 4f7fa06a66 [libcxx] [test] Add XFAIL LIBCXX-WINDOWS-FIXME in 124 tests that fail in the future CI configuration
This makes no attempt yet to look into the why/what for each of them,
but makes the CI configuration useful for tracking further regressions.
After looking into each case, they can either be fixed, or converted
into UNSUPPORTED: windows or XFAIL: windows, once the cause is known
and explained.

A number of the filesystem cases can be fixed by patches that are
currently in review.

Differential Revision: https://reviews.llvm.org/D99095
2021-03-22 23:41:11 +02:00
Louis Dionne 3c62198c61 [libc++] NFC: Normalize links to bug reports 2021-03-03 13:45:29 -05:00
Martin Storsjö 4adb4bca05 [libcxx] [test] Fix a test error with condvars with trivial destruction
If the destructor is trivial (_LIBCPP_HAS_TRIVIAL_CONDVAR_DESTRUCTION,
the constructor always is), the compiler warns about the
std::condition_variable being unused.

Add a cast to void to silence the warning about the object being unused.

Differential Revision: https://reviews.llvm.org/D97540
2021-03-02 20:34:22 +02:00
Louis Dionne 642048eea0 [libc++] Allow retries in a few more flaky tests 2021-02-17 11:20:40 -05:00
Casey Carter edecee3826 [libcxx][test] move libc++-specific tests into the libcxx tree
...and rename from `version.pass.cpp` to `version.compile.pass.cpp` to follow the new convention.
2021-01-28 18:01:56 -08:00
Igor Kudrin 7803636057 [libcxx testing] Fix UB in tests for std::lock_guard
If mutex::try_lock() is called in a thread that already owns the mutex,
the behavior is undefined. The patch fixes the issue by creating another
thread, where the call is allowed.

Differential Revision: https://reviews.llvm.org/D94656
2021-01-15 16:11:45 +07:00
Arthur O'Dwyer d430330788 [libc++] Update and normalize the "all the headers" tests.
Some C++20 headers weren't added properly to all three of these
test files. Add them, and take the time to normalize the formatting
so that

    diff <(grep '#include' foo.cpp) <(grep '#include' bar.cpp)

shows no diffs (except that `no_assert_include` deliberately
excludes `<cassert>`).

- Add macro guards to <{barrier,latch,semaphore}>.
- Add macro guards to <experimental/simd>.
- Remove an include of <cassert> from <semaphore>.
- Instead, include <cassert> in the semaphore tests.

Differential Revision: https://reviews.llvm.org/D92525
2020-12-03 15:01:38 -05:00
Arthur O'Dwyer 40950a44b9 [libc++] ADL-proof <thread>, and eliminate `using namespace chrono`.
Since we know exactly which identifiers we expect to find in `chrono`,
a using-directive seems like massive overkill. Remove the directives
and qualify the names as needed.

One subtle trick here: In two places I replaced `*__p` with `*__p.get()`.
The former is an unqualified call to `operator*` on a class type, which
triggers ADL and breaks the new test. The latter is a call to the
built-in `operator*` on pointers, which specifically
does NOT trigger ADL thanks to [over.match.oper]/1.

Differential Revision: https://reviews.llvm.org/D92243
2020-12-01 22:13:39 -05:00
Louis Dionne 564628014c [libc++] Introduce an indirection to create threads in the test suite
We create threads using std::thread in various places in the test suite.
However, the usual std::thread constructor may not work on all platforms,
e.g. on platforms where passing a stack size is required to create a thread.

This commit introduces a simple indirection that makes it easier to tweak
how threads are created inside the test suite on various platforms. Note
that tests that are purposefully calling std::thread's constructor directly
(e.g. because that is what they're testing) were not modified.
2020-11-27 11:54:19 -05:00
Louis Dionne 0e61d02c05 [libc++] Correct XFAILs for the C++20 Synchronization Library
Technically, these tests don't only fail against macosx10.9 to 10.15,
but really against any released macOS yet.
2020-11-05 08:45:57 -05:00
Louis Dionne 88ffc72717 [libc++] Add a libc++ configuration that does not support localization
When porting libc++ to embedded systems, it can be useful to drop support
for localization, which these systems don't implement or care about.

Differential Revision: https://reviews.llvm.org/D90072
2020-10-27 14:56:30 -04:00
Louis Dionne 504bc07d1a [runtimes] Use int main(int, char**) consistently in tests
This is needed when running the tests in Freestanding mode, where main()
isn't treated specially. In Freestanding, main() doesn't get mangled as
extern "C", so whatever runtime we're using fails to find the entry point.

One way to solve this problem is to define a symbol alias from __Z4mainiPPc
to _main, however this requires all definitions of main() to have the same
mangling. Hence this commit.
2020-10-08 14:28:13 -04:00
Louis Dionne 281de8f361 [libc++] Allow retries in two flaky tests 2020-10-06 11:32:19 -04:00
Louis Dionne f7e91e6cc7 [libc++] Allow retries on some slightly flaky mutex tests 2020-10-02 17:21:21 -04:00
Louis Dionne 8d4860aa9e [libc++] Remove workarounds for missing rvalue references
We don't support GCC in C++03 mode, and Clang provides rvalue references
even in C++03 mode. So there's effectively no supported compiler that
doesn't support rvalue references.

Differential Revision: https://reviews.llvm.org/D84943
2020-08-12 12:02:28 -04:00
Louis Dionne 2659663ee3 [libc++] Remove shortcut Lit features for Apple backdeployment
Some time ago, I introduced shortcut features like dylib-has-no-shared_mutex
to encode whether the deployment target supported shared_mutex (say). This
made the test suite annotations cleaner.

However, the problem with building Lit features on top of other Lit
features is that it's easier for them to become stale, especially when
they are generated programmatically. Furthermore, it makes the bar for
defining configurations from scratch higher, since more features have
to be defined. Instead, I think it's better to put the XFAILs in the
tests directly, which allows cleaning them up with a simple grep.
2020-07-16 15:39:08 -04:00
Louis Dionne 305b500eaf [libc++] Fix test failures in C++14 mode 2020-07-09 01:14:30 -04:00
David Zarzycki e56e96a264 [libcxx testing] Remove ALLOW_RETRIES from another test 2020-07-04 10:15:21 -04:00
David Zarzycki 8aff689164 [libcxx testing] Remove ALLOW_RETRIES from another test 2020-07-03 07:00:34 -04:00
Louis Dionne 1fc5010d6b [libc++] Consider everything inside %T to be a dependency of each test
Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency. This
simplifies the test harness significantly, by making %T the directory
that contains everything required to run a test. This also removes the
need for the %{file_dependencies} substitution, which is removed by this
patch.

Furthermore, this patch also changes the harness to execute tests locally
inside %T, so as to avoid creating a separate directory for no purpose.
2020-06-10 22:38:05 -04:00
Louis Dionne 2bbfa6b02b [libc++] Fix test broken in C++03 due to requiring C++11 features from vector 2020-06-03 12:59:17 -04:00
Louis Dionne 62cfa3a0b5 [libc++] Support move construction and assignment in <thread> in C++03
Libc++ provides support for <thread> in C++03 as an extension. Furthermore,
it does not support any compiler that doesn't have rvalue references. It
is hence possible to provide the move constructor and move assignment
operator in C++03.
2020-06-03 12:16:27 -04:00
David Zarzycki e25f01be0c [libcxx testing] Fix bot failure in my last commit 2020-06-03 11:28:14 -04:00
David Zarzycki 6ce71d2dad [libcxx testing] Fix more bogus timeouts: condvarany/notify_all.pass.cpp
On slow/busy machines, timing cannot be guaranteed.
2020-06-03 10:28:12 -04:00
Louis Dionne 31cbe0f240 [libc++] Remove the c++98 Lit feature from the test suite
C++98 and C++03 are effectively aliases as far as Clang is concerned.
As such, allowing both std=c++98 and std=c++03 as Lit parameters is
just slightly confusing, but provides no value. It's similar to allowing
both std=c++17 and std=c++1z, which we don't do.

This was discovered because we had an internal bot that ran the test
suite under both c++98 AND c++03 -- one of which is redundant.

Differential Revision: https://reviews.llvm.org/D80926
2020-06-03 09:37:22 -04:00
David Zarzycki 579d6ed48c [libcxx testing] Fix lingering bugs in notify_one.pass.cpp
This test is arguably fatally flawed, at least as long as C++ condition
variables are just trivial wrappers around POSIX. I've added some notes
to the test for future authors to consider.
2020-06-03 08:50:27 -04:00
David Zarzycki 1c4238e7a0 [libcxx testing] Stop using arbitrary timeouts in one test
On a busy and/or slow system, 100ms might not be long enough. Instead,
we now use atomic variables to communicate between threads.
2020-05-30 06:09:11 -04:00
David Zarzycki a675c1dee4 [libcxx testing] Remove ALLOW_RETRIES from lock_guard tests
These two tests were clumsily using time measurements to determine
whether std::lock_guard was working correctly. In practice, this
approach merely verified that the underlying lock properly waits.

Now these two tests verify that lock is acquired, not dropped
prematurely, and finally, actually dropped at the end of the scope.
2020-05-18 07:44:16 -04:00
David Zarzycki 3f66bb2017 [libcxx testing] Remove ALLOW_RETRIES from last futures test
Like other uses of ALLOW_RETRIES, this test tried to verify that an API
returned "quickly" but quick is not safe to define given slow and/or
busy machines.

Instead, we now verify that these "wait" APIs actually wait, which the
old test did not.
2020-05-16 07:11:49 -04:00
David Zarzycki 1858953395 [libcxx testing] Remove ALLOW_RETRIES from two futures tests
These two tests do not use the "thread sleeps X milliseconds" pattern
that other libcxx tests use, so all we can do in order to remove
ALLOW_RETRIES workaround is remove the assumption that measuring the
"quick" return of `wait()` is possible (it is not). Let the test harness
verify overall that `wait()` does not hang.

As a bonus, have the spin-waiting threads `yield()`, which is what well
behaved code should do.
2020-05-14 06:18:23 -04:00
David Zarzycki 1febe28982 [libcxx testing] Remove ALLOW_RETRIES from wait_for futures test
This test tried to verify that "wait()" returned quickly but "quick" is
impossible to define given a busy and/or slow system.

Instead, I've refactored the test to verify that `wait()` actually
waits which the old test did not verify.
2020-05-13 06:47:29 -04:00
David Zarzycki 4f4d6c81f8 [libcxx testing] Remove ALLOW_RETRIES from sleep_until.pass.cpp
Operating systems are best effort by default, so we cannot assume that
sleep-like APIs return as soon as we'd like.

Even if a sleep-like API returns when we want it to, the potential for
preemption means that attempts to measure time are subject to delays.
2020-05-10 05:59:09 -04:00
David Zarzycki 4f4ce13944 [libcxx testing] Make three locking tests more reliable
The challenge with measuring time in tests is that slow and/or busy
machines can cause tests to fail in unexpected ways. After this change,
three tests should be much more robust. The only remaining and tiny race
that I can think of is preemption after `--countDown`. That being said,
the race isn't fixable because the standard library doesn't provide a
way to count threads that are waiting to acquire a lock.

Reviewers: ldionne, EricWF, howard.hinnant, mclow.lists, #libc

Reviewed By: ldionne, #libc

Subscribers: dexonsmith, jfb, broadwaylamb, libcxx-commits

Tags: #libc

Differential Revision: https://reviews.llvm.org/D79406
2020-05-09 11:11:26 -04:00
Louis Dionne 2fd7d364cd [libc++] Make the verify-support feature implicit
Tests that require support for Clang-verify are already marked as such
explicitly by their extension, which is .verify.cpp. Requiring the use
of an explicit Lit feature is, after thought, not really helpful.

This is a change in design: we have been bitten in the past by tests not
being enabled when we thought they were. However, the issue was mostly
with file extensions being ignored. The fix for that is not to blindly
require explicit features all the time, but instead to report all files
that are in the suite but that don't match any known test format. This
can be implemented in a follow-up patch.
2020-04-30 11:47:12 -04:00
Louis Dionne d7da36c6e0 [libc++] Mark two timed_mutex tests as flaky 2020-04-29 11:52:17 -04:00
Louis Dionne f17eb4ec20 [libc++] Add UNSUPPORTED markup for shared_mutex and shared_timed_mutex tests
The tests were previously disabled entirely whenever availability markup
was enabled. Instead, enable the tests but add the proper UNSUPPORTED
markup.
2020-04-27 04:23:58 -04:00
Louis Dionne 8c61114c53 [libc++/abi/unwind] Rename Lit features for no exceptions to 'no-exceptions'
Instead of having different names for the same Lit feature accross code
bases, use the same name everywhere. This NFC commit is in preparation
for a refactor where all three projects will be using the same Lit
feature detection logic, and hence it won't be convenient to use
different names for the feature.

Differential Revision: https://reviews.llvm.org/D78370
2020-04-22 08:25:27 -04:00
Louis Dionne 9a39d5a2ec [libc++] Move .fail.cpp tests with verify-support to .verify.cpp 2020-04-17 09:05:28 -04:00
Louis Dionne 7a6aaf9b23 [libc++] Remove workaround for .fail.cpp tests that don't have clang-verify markup
By renaming .fail.cpp tests that don't need clang-verify to .compile.fail.cpp,
the new test format will not try to compile these tests with clang-verify,
and the old test format will work just the same. However, this allows
removing a workaround that requires parsing each test looking for
clang-verify markup.

After this change, a .fail.cpp test should always have clang-verify markup.
When clang-verify is not supported by the compiler, we will just check that
these tests fail to compile. When clang-verify is supported, these tests
will be compiled with clang-verify whether they have markup or not (so
they should have markup, or they will fail).

This simplifies the test suite and also ensures that all of our .fail.cpp
tests provide clang-verify markup. If it's impossible for a test to have
clang-verify markup, it can be moved to a .compile.fail.cpp test, which
are unconditionally just checked for compilation failure.
2020-04-15 10:53:37 -04:00
Louis Dionne 7149bb7068 [libc++] NFC: Clean up a lot of old Lit features
The libc++ test suite has a lot of old Lit features used to XFAIL tests
and mark them as UNSUPPORTED. Many of them are to workaround problems on
old compilers or old platforms. As time goes by, it is good to go and
clean those up to simplify the configuration of the test suite, and also
to reflect the testing reality. It's not useful to have markup that gives
the impression that e.g. clang-3.3 is supported, when we don't really
test on it anymore (and hence several new tests probably don't have the
necessary markup on them).
2020-04-10 17:20:29 -04:00
Louis Dionne 7662ad67c5 [libc++] Mark two std::timed_mutex tests as flaky 2020-04-06 12:41:10 -04:00
Louis Dionne aaaa25e23d [libc++] Remove useless nothing_to_do.pass.cpp tests
The testing script used to test libc++ historically did not like directories
without any testing files, so these tests had been added. Since this is
not necessary anymore, we can now remove these files. This has the benefit
that the total number of tests reflects the real number of tests more
closely, and we also skip some unnecessary work (especially relevant when
running tests over SSH).

However, some nothing_to_do.pass.cpp tests actually serve the purpose of
documenting that an area of the Standard doesn't need to be tested, or is
tested elsewhere. These files are not removed by this commit.

Removal done with:

  import os
  import itertools
  for (dirpath, dirnames, filenames) in itertools.chain(os.walk('./libcxx/test'),
                                                        os.walk('./libcxxabi/test')):
      if len(filenames + dirnames) > 1 and \
         any(p == 'nothing_to_do.pass.cpp' for p in filenames):
          os.remove(os.path.join(dirpath, 'nothing_to_do.pass.cpp'))
2020-04-03 13:48:34 -04:00
Louis Dionne 07e462526d [libc++] Allow running .sh.cpp tests with SSHExecutors
This commit adds a script that can be used as an %{exec} substitution
such that .sh.cpp tests can now run on remote hosts when using the
SSHExecutor.
2020-03-31 15:50:42 -04:00
Louis Dionne 08776defa5 [libc++/libc++abi] Properly delimit lit substitutions
lit is not very clever when it performs substitution on RUN lines. It
simply looks for a match anywhere in the line (without tokenization)
and replaces it by the expansion. This means that a RUN line containing
e.g. `-verify-ignore-unexpected=note` wouod be expanded to
`-verify-ignore-unexpected=<substitution for not>e`, which is
surprising and nonsensical.

It also means that something like `%compile_module` could be expanded
to `<substitution-for-%compile>_module` or to the correct substitution,
depending on the order in which substitutions are evaluated by lit.

To avoid such problems, it is a good habit to delimit custom substitutions
with some token. This commit does that for all substitutions used in the
libc++ and libc++abi test suites.
2020-03-27 10:27:38 -04:00
Louis Dionne a5fa5f7cb8 [libc++] Do not force the use of -Werror in verify tests
Forcing -Werror and other warnings means that the test suite isn't
actually testing what most people are seeing in their code -- it seems
better and less arbitrary to compile these tests as close as possible
to the compiler default instead.

Removing -Werror also means that we get to differentiate between
diagnostics that are errors and those that are warnings, which makes
the test suite more precise.

Differential Revision: https://reviews.llvm.org/D76311
2020-03-26 07:54:45 -04:00
Louis Dionne aec82f9256 [libc++] Require the use of clang-verify in .fail.cpp tests that don't fail without it
Some tests do not fail at all when -verify is not supported, unless some
arbitrary warning flag is added to make them fail. We currently used
-Werror=unused-result to make them fail, but doing so makes the test
suite a lot more inscrutable. It seems better to just disable those
tests when -verify is not supported.

Differential Revision: https://reviews.llvm.org/D76256
2020-03-25 16:48:09 -04:00