Commit Graph

12 Commits

Author SHA1 Message Date
Ben Langmuir 83902c4036 Reapply "[clang][deps] Split translation units into individual -cc1 or other commands"
Attempt to fix the test failures observed in CI:
* Add Option dependency, which caused BUILD_SHARED_LIBS builds to fail
* Adapt tests that accidentally depended on the host platform: platforms
  that don't use an integrated assembler (e.g. AIX) get a different set
  of commands from the driver. Most dependency scanner tests can use
  -fsyntax-only or -E instead of -c to avoid this, and in the rare case
  we want to check -c specifically, set an explicit target so the
  behaviour is independent of the host.

Original commit message follows.

---

Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.

This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.

In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.

Differential Revision: https://reviews.llvm.org/D132405
2022-08-31 09:45:11 -07:00
Ben Langmuir 1877d76aa0 Revert "[clang][deps] Split translation units into individual -cc1 or other commands"
Failing on some bots, reverting until I can fix it.

This reverts commit f80a0ea760.
2022-08-30 15:50:09 -07:00
Ben Langmuir f80a0ea760 [clang][deps] Split translation units into individual -cc1 or other commands
Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.

This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.

In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.

Differential Revision: https://reviews.llvm.org/D132405
2022-08-30 15:23:19 -07:00
Jan Svoboda be795ee16a [clang][deps] Allow switching between lazily/eagerly loaded PCMs
This patch introduces new option `-eager-load-pcm` to `clang-scan-deps`, which controls whether the resulting command-lines will load PCM files eagerly (at the start of compilation) or lazily (when handling import directive). This patch also switches the default from eager to lazy.

To reduce the potential for churn in LIT tests in the future, this patch also removes redundant checks of command-line arguments and introduces new test `modules-dep-args.c` as a substitute.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D132066
2022-08-22 18:22:27 -07:00
Jan Svoboda 71e32d5cf0 [clang][deps] Always generate module paths
Since D129389 (and downstream PR https://github.com/apple/llvm-project/pull/4965), the dependency scanner is responsible for generating full command-lines, including the modules paths. This patch removes the flag that was making this an opt-in behavior in clang-scan-deps.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D131420
2022-08-10 11:58:28 -07:00
Argyrios Kyrtzidis b58a420ff4 [Tooling/DependencyScanning] Rename refactorings towards transitioning dependency scanning to use pre-lexed preprocessor directive tokens
This is first of a series of patches for making the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`.
This patch only includes NFC renaming changes to make reviewing of the functionality changing parts easier.

Differential Revision: https://reviews.llvm.org/D125484
2022-05-26 12:49:51 -07:00
Gulfem Savrun Yeniceri 6bb836af2a Revert "[clang][deps] NFC: De-duplicate clang-cl tests"
This reverts commit 30cb49b44e.
It caused test failures on Fuchsia Toolchain Mac builds:
https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-mac-x64/b8818888168677577537/overview
2022-03-24 18:10:08 +00:00
Jan Svoboda 30cb49b44e [clang][deps] NFC: De-duplicate clang-cl tests
In D92191, a bunch of test cases were added to check `clang-scan-deps` works in `clang-cl` mode as well.

We don't need to duplicate all test cases, though. Testing the few special cases we have in `clang-scan-deps` for `clang-cl` should be good enough:

1. Deducing output path (and therefore target name in our make output).
2. Ignoring `-Xclang` arguments in step 1.
3. Deducing resource directory by invoking the compiler executuable.

This test de-duplicates the extra clang-cl test cases.

Reviewed By: dexonsmith, saudi

Differential Revision: https://reviews.llvm.org/D121812
2022-03-23 15:42:11 +01:00
Jan Svoboda c4b13f7a2f [clang][deps] Simplify PREFIX definitions in tests
Instead of outputting the test directory into the JSON result file, parsing it with `FileCheck` and then potentially stripping it, simply use `FileCheck`'s `-D` option.

Note that we use `%/t` instead of `%t` in order to normalize to forward slashes on Windows, which matches what we do with `sed 's:\\\\\?:/:g'`.

Differential Revision: https://reviews.llvm.org/D121516
2022-03-12 16:45:09 +01:00
Jan Svoboda 19017c2435 [clang][deps] Return the whole TU command line
The dependency scanner already generates canonical -cc1 command lines that can be used to compile discovered modular dependencies.

For translation unit command lines, the scanner only generates additional driver arguments the build system is expected to append to the original command line.

While this works most of the time, there are situations where that's not the case. For example with `-Wunused-command-line-argument`, Clang will complain about the `-fmodules-cache-path=` argument that's not being used in explicit modular builds. Combine that with `-Werror` and the build outright fails.

To prevent such failures, this patch changes the dependency scanner to return the full driver command line to compile the original translation unit. This gives us more opportunities to massage the arguments into something reasonable.

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D118986
2022-02-23 15:46:20 +01:00
Jan Svoboda 17ec9d1f6b [clang][deps] Don't emit `-fmodule-map-file=`
During explicit modules build, when all modules are provided via `-fmodule-file=<path>` and implicit modules and implicit module maps are disabled (`-fno-implicit-modules`, `-fno-implicit-module-maps`), we don't need to load the original module map files at all. This patch stops emitting the `-fmodule-map-file=` arguments we don't need, saving some compilation time due to avoiding parsing such module maps and making the command line shorter.

Reviewed By: bnbarham

Differential Revision: https://reviews.llvm.org/D113473
2021-11-18 12:31:24 +01:00
Alex Lorenz c68f247275 [clang-scan-deps] ignore top-level module dependencies that aren't actually imported
Whenever -fmodule-name=top_level_module name is parsed, and clang actually tries to
import top_level_module, the headers are imported textually and the module isn't actually
built. However, the dependency scanner could still record it as a potential dependency
if the module was reimported and thus recorded by the preprocessor callbacks.
This change avoids collecting this kind of module as a dependency by verifying that we don't
collect top level modules without actual PCM files.

Differential Revision: https://reviews.llvm.org/D106100
2021-07-20 11:11:28 -07:00