This reverts commit 340654e0f2, essentially reapplying 1d3ba05e4a.
The test VFS/real-path-found-first.m that was failing on Windows is now passing with a workaround.
tapi & clang-extractapi both attempt to construct then check against
how a header was included to determine api information when working
against multiple search paths, headermap, and vfsoverlay mechanisms.
Validating this against what the preprocessor sees during lookup time
makes this check more reliable.
Reviewed By: zixuw, jansvoboda11
Differential Revision: https://reviews.llvm.org/D124638
This patch changes the member of `FrameworkCacheEntry` from `const DirectoryEntry *` to `Optional<DirectoryEntryRef>` in order to remove uses of the deprecated `DirectoryEntry::getName()`.
Reviewed By: bnbarham
Differential Revision: https://reviews.llvm.org/D123854
This patch removes uses of the deprecated `DirectoryEntry::getName()` from `HeaderSearch::load*()` functions by using `DirectoryEntryRef` instead.
Note that we bail out in one case and use the also deprecated `FileEntry::getLastRef()`. That's to prevent this patch from growing, and is addressed in a follow-up.
Reviewed By: bnbarham
Differential Revision: https://reviews.llvm.org/D123771
The dependency scanner can reuse single FileManager instance across multiple translation units. This may lead to non-deterministic output depending on which TU gets processed first.
One of the problems is that Clang uses DirectoryEntry::getName in the header search algorithm. This function returns the path that was first used to construct the (shared) entry in FileManager. Using DirectoryEntryRef::getName instead preserves the case as it was spelled out for the current "get directory entry" request.
rdar://90647508
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D123229
in filesystems
It is simpler to search for module unit by -fprebuilt-module-path
option. However, the separator ':' of partitions is not friendly.
According to the discussion in https://reviews.llvm.org/D118586, I think
we get consensus to use '-' as the separator instead. The '-' is the
choice of GCC too.
Previously I thought it would be better to add an option. But I feel it
is over-engineering now. Another reason here is that there are too many
options for modules (for clang module mainly) now. Given it is not bad
to use '-' when searching, I think it is acceptable to not add an
option.
Reviewed By: iains
Differential Revision: https://reviews.llvm.org/D120874
`HeaderSearch` currently assumes `LookupFileCache` is eventually populated in `LookupFile`. However, that's not always the case with `-fms-compatibility` and its early returns.
This patch adds a defensive check that the iterator pulled out of the cache is actually valid before using it.
(This bug was introduced in D119721. Before that, the cache was initialized to `0` - essentially the `search_dir_begin()` iterator.)
Reviewed By: dexonsmith, erichkeane
Differential Revision: https://reviews.llvm.org/D122237
The iterator is not needed after the loop body anymore, meaning we can use more terse range-based for loop.
Depends on D121295.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D121685
To reduce the number of modules we build in explicit builds (which use strict context hash), we prune unused header search paths. This essentially merges parts of the dependency graph.
Determining whether a search path was used to discover a module (through implicit module maps) proved to be somewhat complicated. Initial support landed in D102923, while D113676 attempts to fix some bugs.
However, now that we don't use implicit module maps in explicit builds (since D120465), we don't need to consider such search paths as used anymore. Modules are no longer discovered through the header search mechanism, so we can drop such search paths (provided they are not needed for other reasons).
This patch removes whatever support for detecting such usage we had, since it's buggy and not required anymore.
Depends on D120465.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D121295
This patch replaces a lot of index-based loops with iterators and ranges.
Depends on D117566.
Reviewed By: ahoppen
Differential Revision: https://reviews.llvm.org/D119722
This patch starts using the new iterator type in `LookupFileCacheInfo`.
Depends on D117566.
Reviewed By: ahoppen
Differential Revision: https://reviews.llvm.org/D119721
The `const DirectoryLookup *` out-parameter of `{HeaderSearch,Preprocessor}::LookupFile()` is assigned the most recently used search directory, which callers use to implement `#include_next`.
From the function signature it's not obvious the `const DirectoryLookup *` is being used as an iterator. This patch introduces `ConstSearchDirIterator` to make that affordance obvious. This would've prevented a bug that occurred after initially landing D116750.
Reviewed By: ahoppen
Differential Revision: https://reviews.llvm.org/D117566
An impilt used of C++ module without prebuild path may cause crash.
For example:
```
// ./dir1/C.cppm
export module C;
// ./dir2/B.cppm
export module B;
import C;
// ./A.cpp
import B;
import C;
```
When we compile A.cpp without the prebuild path of C.pcm, the compiler
will crash.
```
clang++ -std=c++20 --precompile -c ./dir1/C.cppm -o ./dir1/C.pcm
clang++ -std=c++20 --precompile -fprebuilt-module-path=./dir2 -c
./dir2/B.cppm -o ./dir2/B.pcm
clang++ -std=c++20 -fprebuilt-module-path=./dir2 A.cpp
```
The prebuilt path of module C is cached when import module B, and in the
function HeaderSearch::getCachedModuleFileName, the compiler try to get
the filename by modulemap without check if modulemap exists, and there
is no modulemap in C++ module.
Reviewed By: ChuanqiXu
Differential review: https://reviews.llvm.org/D119426
Previously, the Framework name was only set if the file
came from a header mapped framework; now we'll always
set the framework name if the file is in a framework.
Differential Revision: https://reviews.llvm.org/D117830
This patch replaces the exact include count of each file in `HeaderFileInfo` with a set of included files in `Preprocessor`.
The number of includes isn't a property of a header file but rather a preprocessor state. The exact number of includes is not used anywhere except statistic tracking.
Reviewed By: vsapsai
Differential Revision: https://reviews.llvm.org/D114095
The `{HeaderSearch,Preprocessor}::LookupFile()` functions take an out-parameter `const DirectoryLookup *&`. Most callers end up creating a `const DirectoryLookup *` variable that's otherwise unused.
This patch changes the out-parameter from reference to a pointer, making it possible to simply pass `nullptr` to the function without the ceremony.
Reviewed By: ahoppen
Differential Revision: https://reviews.llvm.org/D117312
The elements of `SearchPath::SearchDirs` are being referenced to by their indices. This proved to be error-prone: `HeaderSearch::SearchDirToHSEntry` was accidentally not being updated in `HeaderSearch::AddSearchPath()`. This patch fixes that by referencing `SearchPath::SearchDirs` elements by their address instead, which is stable thanks to the bump-ptr-allocation strategy.
Reviewed By: ahoppen
Differential Revision: https://reviews.llvm.org/D116750
Clang will now search through the framework includes to identify
the framework include path to a file, and then suggest a framework
style include spelling for the file.
Differential Revision: https://reviews.llvm.org/D115183
This patch refactors the code that checks whether a file has just been included for the first time.
The `HeaderSearch::FirstTimeLexingFile` function is removed and the information is threaded to the original call site from `HeaderSearch::ShouldEnterIncludeFile`. This will make it possible to avoid tracking the number of includes in a follow up patch.
Depends on D114092.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D114093
When building frameworks, headermaps responsible for mapping angle-included headers to their source file location are passed via
`-I` and not `-index-header-map`. Also, `-index-header-map` is only used for indexing purposes and not during most builds.
This patch holds on to the framework's name in HeaderFileInfo as this is retrieveable for cases outside of IndexHeaderMaps and
still represents the framework that is being built.
resolves: rdar://84046893
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D111468
For dependency scanning, it would be useful to collect header search paths (provided on command-line via `-I` and friends) that were actually used during preprocessing. This patch adds that feature to `HeaderSearch` along with a new remark that reports such paths as they get used.
Previous version of this patch tried to use the existing `LookupFileCache` to report used paths via `HitIdx`. That doesn't work for `ComputeUserEntryUsage` (which is intended to be called *after* preprocessing), because it indexes used search paths by the file name. This means the values get overwritten when the code contains `#include_next`.
Note that `HeaderSearch` doesn't use `HeaderSearchOptions::UserEntries` directly. Instead, `InitHeaderSearch` pre-processes them (adds platform-specific paths, removes duplicates, removes paths that don't exist) and creates `DirectoryLookup` instances. This means we need a mechanism for translating between those two. It's not possible to go from `DirectoryLookup` back to the original `HeaderSearch`, so `InitHeaderSearch` now tracks the relationships explicitly.
Depends on D111557.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D102923
This patch propagates the import `SourceLocation` into `HeaderSearch::lookupModule`. This enables remarks on search path usage (implemented in D102923) to point to the source code that initiated header search.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D111557
There is a separate field `isPragmaOnce` and when `isImport` combines
both, it complicates HeaderFileInfo serialization as `#pragma once` is
the inherent property of the header while `isImport` reflects how other
headers use it. The usage of the header can be different in different
contexts, that's why `isImport` requires tracking separate from `#pragma once`.
Differential Revision: https://reviews.llvm.org/D104351
C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)
Summary:
suggestPathToFileForDiagnostics is actively used in clangd for converting
an absolute path to a header file to a header name as it should be spelled
in the sources. Current approach converts absolute path to relative path.
This diff implements missing logic that makes a reverse lookup from the
relative path to the key in the header map that should be used in the sources.
Prerequisite diff: https://reviews.llvm.org/D103229
Test Plan: check-clang
Reviewers: dexonsmith, bruno, rsmith
Subscribers: cfe-commits
Tasks:
Tags: #clang
Differential Revision: https://reviews.llvm.org/D103142
Migrate `HeaderSearch::LoadedModuleMaps` and a number of APIs over to
`FileEntryRef`. This should have no functionality change. Note that two
`FileEntryRef`s hash the same if they point at the same `FileEntry`.
Differential Revision: https://reviews.llvm.org/D92975
The behavior is controlled by the `-fprebuilt-implicit-modules` option, and
allows searching for implicit modules in the prebuilt module cache paths.
The current command-line options for prebuilt modules do not allow to easily
maintain and use multiple versions of modules. Both the producer and users of
prebuilt modules are required to know the relationships between compilation
options and module file paths. Using a particular version of a prebuilt module
requires passing a particular option on the command line (e.g.
`-fmodule-file=[<name>=]<file>` or `-fprebuilt-module-path=<directory>`).
However the compiler already knows how to distinguish and automatically locate
implicit modules. Hence this proposal to introduce the
`-fprebuilt-implicit-modules` option. When set, it enables searching for
implicit modules in the prebuilt module paths (specified via
`-fprebuilt-module-path`). To not modify existing behavior, this search takes
place after the standard search for prebuilt modules. If not
Here is a workflow illustrating how both the producer and consumer of prebuilt
modules would need to know what versions of prebuilt modules are available and
where they are located.
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v1 <config 1 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v2 <config 2 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v3 <config 3 options>
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules_v1 <config 1 options>
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap <non-prebuilt config options>
With prebuilt implicit modules, the producer can generate prebuilt modules as
usual, all in the same output directory. The same mechanisms as for implicit
modules take care of incorporating hashes in the path to distinguish between
module versions.
Note that we do not specify the output module filename, so `-o` implicit modules are generated in the cache path `prebuilt_modules`.
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 1 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 2 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 3 options>
The user can now simply enable prebuilt implicit modules and point to the
prebuilt modules cache. No need to "parse" command-line options to decide
what prebuilt modules (paths) to use.
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules -fprebuilt-implicit-modules <config 1 options>
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules -fprebuilt-implicit-modules <non-prebuilt config options>
This is for example particularly useful in a use-case where compilation is
expensive, and the configurations expected to be used are predictable, but not
controlled by the producer of prebuilt modules. Modules for the set of
predictable configurations can be prebuilt, and using them does not require
"parsing" the configuration (command-line options).
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D68997
Simplify `HeaderSearch::LookupFile`. Instead of deconstructing a
`FileEntryRef` into a name and `FileEntry` and then rebuilding it later,
use it as is. This helps to unblock making the constructor of
`FileEntryRef` private to `FileManager`.
Differential Revision:
HeaderSearch was marking requested HeaderFileInfo as Resolved only based on
the presence of ExternalSource. As the result, using any module was enough
to set ExternalSource and headers unknown to this module would have
HeaderFileInfo with empty fields, including `isImport = 0`, `NumIncludes = 0`.
Such HeaderFileInfo was preserved without changes regardless of how the
header was used in other modules and caused incorrect result in
`HeaderSearch::ShouldEnterIncludeFile`.
Fix by marking HeaderFileInfo as Resolved only if ExternalSource knows
about this header.
rdar://problem/62126911
Reviewed By: bruno
Differential Revision: https://reviews.llvm.org/D80263
Previously, this would fail if the builtin headers had been "claimed" by
a different module that wraps these builtin headers. libc++ does this,
for example.
This change adds a test demonstrating this situation; the test fails
without the fix.
Fix a few bugs where we would fail to properly determine header to
module correspondence when determining whether to suggest a #include or
import, and suggest a #include more often in language modes where there
is no import syntax. Generally, if the target is in a header with
include guards or #pragma once, we should suggest either #including or
importing that header, and not importing a module that happens to
textually include it.
In passing, improve the notes we attach to the corresponding
diagnostics: calling an entity that we couldn't see "previous" is
confusing.
Support only preferred spelling 'Modules/module.private.modulemap' and
not the deprecated 'module_private.map'.
rdar://problem/57715533
Reviewed By: bruno
Differential Revision: https://reviews.llvm.org/D75311
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
In `DirectoryLookup::LookupFile` parameter `HasBeenMapped` doesn't cover
the case when clang finds a file through a header map but doesn't remap
the lookup filename because the target path is an absolute path. As a
result, -Wnonportable-include-path suppression for header maps
introduced in r301592 wasn't triggered.
Change parameter `HasBeenMapped` to `IsInHeaderMap` and use parameter
`MappedName` to track the filename remapping. This way we can handle
both relative and absolute paths in header maps, and account for their
specific properties, like filename remapping being a property preserved
across lookups in multiple directories.
rdar://problem/39516483
Reviewers: dexonsmith, bruno
Reviewed By: dexonsmith
Subscribers: jkorous, cfe-commits, ributzka
Differential Revision: https://reviews.llvm.org/D58094
llvm-svn: 371655
accessed name to the directory entry
This commit introduces a parallel API that returns a DirectoryEntryRef
to the FileManager, similar to the parallel FileEntryRef API. All
uses will have to be update in follow-up patches. The immediate use of the new API in this
patch fixes the issue where a file manager was reused in clang-scan-deps,
but reported an different file path whenever a framework lookup was done through a symlink.
Differential Revision: https://reviews.llvm.org/D67026
llvm-svn: 370562
`FileManager::getFileRef` is a modern API which we expect to convert to
over time. We should modernize the error handling as well, using
`llvm::Expected` instead of `llvm::ErrorOr`, to help clients that care
about errors to ensure nothing is missed.
However, not all clients care. I've also added another path for those
that don't:
- `FileEntryRef` is now copy- and move-assignable (using a pointer
instead of a reference).
- `FileManager::getOptionalFileRef` returns an `llvm::Optional` instead
of `llvm::Expected`.
- Added an `llvm::expectedToOptional` utility in case this is useful
elsewhere.
https://reviews.llvm.org/D66705
llvm-svn: 369943
when the FileManager is reused across invocations
This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.
The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.
Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.
Differential Revision: https://reviews.llvm.org/D65907
llvm-svn: 369680
Kazu Hirata