Commit Graph

11 Commits

Author SHA1 Message Date
Ben Langmuir 83902c4036 Reapply "[clang][deps] Split translation units into individual -cc1 or other commands"
Attempt to fix the test failures observed in CI:
* Add Option dependency, which caused BUILD_SHARED_LIBS builds to fail
* Adapt tests that accidentally depended on the host platform: platforms
  that don't use an integrated assembler (e.g. AIX) get a different set
  of commands from the driver. Most dependency scanner tests can use
  -fsyntax-only or -E instead of -c to avoid this, and in the rare case
  we want to check -c specifically, set an explicit target so the
  behaviour is independent of the host.

Original commit message follows.

---

Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.

This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.

In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.

Differential Revision: https://reviews.llvm.org/D132405
2022-08-31 09:45:11 -07:00
Ben Langmuir 1877d76aa0 Revert "[clang][deps] Split translation units into individual -cc1 or other commands"
Failing on some bots, reverting until I can fix it.

This reverts commit f80a0ea760.
2022-08-30 15:50:09 -07:00
Ben Langmuir f80a0ea760 [clang][deps] Split translation units into individual -cc1 or other commands
Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.

This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.

In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.

Differential Revision: https://reviews.llvm.org/D132405
2022-08-30 15:23:19 -07:00
Jan Svoboda be795ee16a [clang][deps] Allow switching between lazily/eagerly loaded PCMs
This patch introduces new option `-eager-load-pcm` to `clang-scan-deps`, which controls whether the resulting command-lines will load PCM files eagerly (at the start of compilation) or lazily (when handling import directive). This patch also switches the default from eager to lazy.

To reduce the potential for churn in LIT tests in the future, this patch also removes redundant checks of command-line arguments and introduces new test `modules-dep-args.c` as a substitute.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D132066
2022-08-22 18:22:27 -07:00
Jan Svoboda 71e32d5cf0 [clang][deps] Always generate module paths
Since D129389 (and downstream PR https://github.com/apple/llvm-project/pull/4965), the dependency scanner is responsible for generating full command-lines, including the modules paths. This patch removes the flag that was making this an opt-in behavior in clang-scan-deps.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D131420
2022-08-10 11:58:28 -07:00
Jan Svoboda 26053ce05a [clang][deps] Create lit substitution for deps-to-rsp
This patch gets rid of the ridiculous relative path we use to invoke the `module-deps-to-rsp.py` script and creates proper lit substitution, cleaning up the tests.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D121525
2022-03-23 15:42:10 +01:00
Jan Svoboda c4b13f7a2f [clang][deps] Simplify PREFIX definitions in tests
Instead of outputting the test directory into the JSON result file, parsing it with `FileCheck` and then potentially stripping it, simply use `FileCheck`'s `-D` option.

Note that we use `%/t` instead of `%t` in order to normalize to forward slashes on Windows, which matches what we do with `sed 's:\\\\\?:/:g'`.

Differential Revision: https://reviews.llvm.org/D121516
2022-03-12 16:45:09 +01:00
Jan Svoboda 19017c2435 [clang][deps] Return the whole TU command line
The dependency scanner already generates canonical -cc1 command lines that can be used to compile discovered modular dependencies.

For translation unit command lines, the scanner only generates additional driver arguments the build system is expected to append to the original command line.

While this works most of the time, there are situations where that's not the case. For example with `-Wunused-command-line-argument`, Clang will complain about the `-fmodules-cache-path=` argument that's not being used in explicit modular builds. Combine that with `-Werror` and the build outright fails.

To prevent such failures, this patch changes the dependency scanner to return the full driver command line to compile the original translation unit. This gives us more opportunities to massage the arguments into something reasonable.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D118986
2022-02-23 15:46:20 +01:00
Steven Wan 06d4a76309 [NFC][AIX]Disable precompiled module file tests on AIX
The PCH reader looks for `__clangast` section in the precompiled module file, which is not present in the file on AIX, and we don't support writing this custom section in XCOFF yet.

Reviewed By: daltenty

Differential Revision: https://reviews.llvm.org/D114481
2021-11-24 13:22:09 -05:00
Jan Svoboda 17ec9d1f6b [clang][deps] Don't emit `-fmodule-map-file=`
During explicit modules build, when all modules are provided via `-fmodule-file=<path>` and implicit modules and implicit module maps are disabled (`-fno-implicit-modules`, `-fno-implicit-module-maps`), we don't need to load the original module map files at all. This patch stops emitting the `-fmodule-map-file=` arguments we don't need, saving some compilation time due to avoiding parsing such module maps and making the command line shorter.

Reviewed By: bnbarham

Differential Revision: https://reviews.llvm.org/D113473
2021-11-18 12:31:24 +01:00
Jan Svoboda 83c633ea1a [clang][deps] Collect precompiled deps from submodules too
In this patch, the dependency scanner starts collecting precompiled dependencies from all encountered submodules, not only from top-level modules.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D108540
2021-08-25 10:35:34 +02:00