Large timeout values (one year, positive infinity) trip up Python on
Windows with "OverflowError: timeout value is too large". One week
seems to work and is still large enough in practice.
Thanks to Simon Pilgrim for helping me test this.
https://reviews.llvm.org/rL375171
llvm-svn: 375264
Avoid sending back the whole run.Test object (which needs to be pickled)
from the worker process when we are only interested in the test result.
llvm-svn: 375262
This will allow us to serialize just the result object instead of the
whole lit.Test object back from the worker to the main lit process.
llvm-svn: 375195
We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (now plus one year) to simplify code
and do less busy waiting.
Thanks to Reid Kleckner for diagnosing that a deadline for of "positive
infinity" does not work with Python 3 anymore. See commit:
4ff1e34b60
I tested this patch with Python 2 and Python 3.
llvm-svn: 375165
Python on Windows raises this OverflowError:
gotit = waiter.acquire(True, timeout)
OverflowError: timestamp too large to convert to C _PyTime_t
So it seems this API behave the same way on every OS.
Also reverts the dependent commit a660dc590a.
llvm-svn: 375143
We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (positive infinity) to simplify code
and do less busy waiting.
llvm-svn: 375129
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-` as a command-line option. This patch adds support for
`-` to mean stdin.
Reviewed By: probinson, rnk
Differential Revision: https://reviews.llvm.org/D67643
llvm-svn: 375116
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff file -
# RUN: not diff file1 file2 | FileCheck %s
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` cannot
currently be used in pipelines and doesn't recognize `-` as a
command-line option.
To enable pipelines, this patch moves lit's `diff` implementation into
an out-of-process script, similar to lit's `cat` implementation. A
follow-up patch will implement `-` to mean stdin.
Reviewed By: probinson, stella.stamenova
Differential Revision: https://reviews.llvm.org/D66574
llvm-svn: 375114
* Remove outdated precautions for Python versions < 2.7
* Remove dead code related to `maxIndividualTestTime` option
* Move printing of test and result summary out of main into its own
function
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68847
llvm-svn: 375046
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files in text
mode, which in Windows causes `\r\n` to be replaced with `\n`.
However, with D68664, lit diff reads the files in binary mode instead
and thus reports that every line is different, just as GNU diff does
(at least under Ubuntu). Adding `--strip-trailing-cr` to those tests
restores the previous behavior while permitting the behavior of lit
diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
llvm-svn: 375020
As suggested by rnk at D67643#1673043, instead of reading files
multiple times until an appropriate encoding is found, read them once
as binary, and then try to decode what was read.
For Python >= 3.5, don't fail when attempting to decode the
`diff_bytes` output in order to print it.
Avoid failures for Python 2.7 used on some Windows bots by
transforming diff output with `lit.util.to_string` before writing it
to stdout.
Finally, add some tests for encoding handling.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68664
llvm-svn: 375018
This essentially reverts a commit [1] that removed the adaptor for
Python unittests. The code has been slightly refactored to make it more
additive: all code is contained in LitTestCase.py.
Usage sites will require a small adaption:
```
[old]
import lit.discovery
...
test_suite = lit.discovery.load_test_suite(...)
[new]
import lit.LitTestCase
...
test_suite = lit.LitTestCase.load_test_suite(...)
```
This was put back on request by Daniel Dunbar, since I wrongly assumed
that the functionality is unused. At least llbuild still uses this [2].
[1] 70ca752ccf
[2] https://github.com/apple/swift-llbuild/blob/master/utils/Xcode/LitXCTestAdaptor/LitTests.py#L16
Reviewed By: ddunbar
Differential Revision: https://reviews.llvm.org/D69002
llvm-svn: 374947
The goal is to have 100% fidelity in clang-scan-deps behavior when
--analyze is present in compilation command.
At the same time I don't want to break clang-tidy which expects
__static_analyzer__ macro defined as built-in.
I introduce new cc1 options (-setup-static-analyzer) that controls
the macro definition and is conditionally set in driver.
Differential Revision: https://reviews.llvm.org/D68093
llvm-svn: 374815
When using lit's internal shell, RUN lines like the following
accidentally execute an external `diff` instead of lit's internal
`diff`:
```
# RUN: program | diff -U1 file -
```
Such cases exist now, in `clang/test/Analysis` for example. We are
preparing patches to ensure lit's internal `diff` is called in such
cases, which will then fail because lit's internal `diff` doesn't
recognize `-U` as a command-line option. This patch adds `-U`
support.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68668
llvm-svn: 374814
On that decode, Windows bots fail with:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
That's the same error as before r374665 except it's now at the decode
before the write to stdout.
llvm-svn: 374666
I seem to have misread the bot logs on my last attempt. When lit's
internal diff runs on Windows under Python 2.7, it's text diffs not
binary diffs that need decoding to avoid this error when writing the
diff to stdout:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
There is no `decode` attribute in this case under Python 3.6.8 under
Ubuntu, so this patch checks for the `decode` attribute before using
it here. Hopefully nothing else is needed when `decode` isn't
available.
It might take a couple more attempts to figure out what error
handling, if any, is needed for this decoding.
llvm-svn: 374665
Based on the bot logs, when lit's internal diff runs on Windows, it
looks like binary diffs must be decoded also for Python 2.7.
Otherwise, writing the diff to stdout fails with:
```
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
```
I did not need to decode using Python 2.7.15 under Ubuntu. When I do
it anyway in that case, `errors="backslashreplace"` fails for me:
```
TypeError: don't know how to handle UnicodeDecodeError in error callback
```
However, `errors="ignore"` works, so this patch uses that, hoping
it'll work on Windows as well.
This patch leaves `errors="backslashreplace"` for Python >= 3.5 as
there's no evidence yet that doesn't work and it produces more
informative binary diffs. This patch also adjusts some lit tests to
succeed for either error handler.
This patch adjusts changes introduced by D68664.
llvm-svn: 374657
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before
a `\n` at the end of a line. Without this patch, lit's internal diff
only removes `\r` if it appears as the last character. That seems
useless. This patch fixes that.
This patch also adds `--strip-trailing-cr` to some tests that fail on
Windows bots when D68664 is applied. Based on what I see in the bot
logs, I think the following is happening. In each test there, lit
diff is comparing a file with `\r\n` line endings to a file with `\n`
line endings. Without D68664, lit diff reads those files with
Python's universal newlines support activated, causing `\r` to be
dropped. However, with D68664, lit diff reads the files in binary
mode instead and thus reports that every line is different, just as
GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr`
to those tests restores the previous behavior while permitting the
behavior of lit diff to be more like GNU diff.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D68839
llvm-svn: 374652
To avoid breaking some tests, D66574, D68664, D67643, and D68668
landed together. However, D68664 introduced an issue now addressed by
D68839, with which these are now all relanding.
Differential Revision: https://reviews.llvm.org/D68668
llvm-svn: 374651
Julian Lettner