Fills gaps in the time trace when precompiled headers are created/loaded.
Reviewed By: jansvoboda11
Differential Revision: https://reviews.llvm.org/D135657
Make flags `-fmodules-validate-system-headers` and
`-fmodules-validate-once-per-build-session` orthogonal, so they have
their own independent responsibilities - if system headers should be
validated and how often.
rdar://87988889
Differential Revision: https://reviews.llvm.org/D135232
This is a follow-up to D134224. The original patch added new `ExcludedHeader` enumerator to `ModuleMap::ModuleHeaderRole` and started associating headers with the modules they were excluded from. This was necessary to consider their module maps as "affecting" in certain situations and in turn serialize them into the PCM.
The association of the header and module needs to be handled when deserializing the PCM as well, though. This patch fixes a potential assertion failure and a regression. This essentially reverts parts of feb54b6ded.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D135381
Update SourceManager::ContentCache::OrigEntry to keep the original
FileEntryRef, and use that to enable ModuleMap::getModuleMapFile* to
return the original FileEntryRef. This change should be NFC for
most users of SourceManager::ContentCache, but it could affect behaviour
for users of getNameAsRequested such as in compileModuleImpl. I have not
found a way to detect that difference without additional functional
changes, other than incidental cases like changes from / to \ on
Windows so there is no new test.
Differential Revision: https://reviews.llvm.org/D135220
This implements WG14 N2927 and WG14 N2930, which together define the
feature for typeof and typeof_unqual, which get the type of their
argument as either fully qualified or fully unqualified. The argument
to either operator is either a type name or an expression. If given a
type name, the type information is pulled directly from the given name.
If given an expression, the type information is pulled from the
expression. Recursive use of these operators is allowed and has the
expected behavior (the innermost operator is resolved to a type, and
that's used to resolve the next layer of typeof specifier, until a
fully resolved type is determined.
Note, we already supported typeof in GNU mode as a non-conforming
extension and we are *not* exposing typeof_unqual as a non-conforming
extension in that mode, nor are we exposing typeof or typeof_unqual as
a nonconforming extension in other language modes. The GNU variant of
typeof supports a form where the parentheses are elided from the
operator when given an expression (e.g., typeof 0 i = 12;). When in C2x
mode, we do not support this extension.
Differential Revision: https://reviews.llvm.org/D134286
Module map files describing excluded headers do affect compilation. Track them in the compiler, serialize them into the PCM file and report them in the scanner.
Depends on D134222.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D134224
When a framework can be found at a new location, all references to it in
the module cache become outdated. When we try to load such outdated .pcm
file, we shouldn't change any already loaded and processed modules.
If `Module` has `ASTFile`, it means we've read its AST block already and
it is too late to undo that. If `ASTFile` is `None`, there is no value
in setting it to `None` again. So we don't reset `ASTFile` in
`ModuleManager::removeModules` at all.
rdar://97216258
Differential Revision: https://reviews.llvm.org/D134249
This improves consistency with other places (e.g. llvm::compression::decompress,
llvm::object::Decompressor::decompress, llvm-objcopy).
Note: when zstd::uncompress was added, we noticed that the API `ZSTD_decompress`
is fine while the zlib API `uncompress` is a misnomer.
Intend to use `ODRDiagsEmitter` during parsing to diagnose a parsed
definition differing from a definition with the same name from a hidden
[sub]module.
Differential Revision: https://reviews.llvm.org/D128695
When compiling a module, its semantics and Clang's behavior are affected by other modules. These modules are typically the **imported** ones. However, during implicit build, some modules end up being compiled and read without being actually imported. This patch starts tracking such modules and serializing them into `.pcm` files. This enables the dependency scanner to construct explicit compilations that mimic implicit build.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D132430
When clang includes a PCH, it tolerates some amount of differences
between the defines used when creating and when including the PCH
- this seems to be intentionally allowed in
c379c07240 (and later extended in
b636875196).
When using a PCH (or when picking a PCH out of a directory containing
multiple candidates) Clang used to accept the header if there were
defines on the command line when creating the PCH that are missing
when using the PCH, or vice versa, defines only set when using the
PCH.
The only cases where Clang explicitly rejected the use of a PCH
is if there was an explicit conflict between the options, e.g.
-DFOO=1 vs -DFOO=2, or -DFOO vs -UFOO.
The latter commit added a FIXME that we really should check whether
mismatched defines actually were used somewhere in the PCH, so that
the define would affect the outcome. This FIXME has stood unaddressed
since 2012.
This differs from GCC, which rejects PCH files if the defines differ
at all.
When explicitly including a single PCH file, the relaxed policy
of allowing minor differences is harmless for correct use cases
(but may fail to diagnose mismtaches), and potentially allow using
PCHs in wider cases (where the user intentionally know that the
differences in defines are harmless for the PCH).
However, for GCC style PCH directories, with a directory containing
multiple PCH variants and the compiler should pick the correct match
out of them, Clang's relaxed logic was problematic. The directory
could contain two otherwise identical PCHs, but one built with -DFOO
and one without. When attempting to include a PCH and iterating over
the candidates in the directory, Clang would essentially pick the
first one out of the two, even if there existed a better, exact
match in the directory.
Keep the relaxed checking when specificlly including one named
PCH file, but require strict matches when trying to pick the right
candidate out of a GCC style directory with alternatives.
This fixes https://github.com/lhmouse/mcfgthread/issues/63.
Differential Revision: https://reviews.llvm.org/D126676
Use of `ORIGINAL_PCH_DIR` record has been superseeded by making PCH/PCM files with relocatable paths at write time.
Removing this record is useful for producing an output-path-independent PCH file and enable sharing of the same PCH file even
when it was intended for a different output path.
Differential Revision: https://reviews.llvm.org/D131124
Specifically, making the following changes:
* Turn lambdas calculating ODR hashes into static functions.
* Move `ODRCXXRecordDifference` where it is used.
* Rename some variables and move some lines of code.
* Replace `auto` with explicit type when the deduced type is not mentioned.
* Add `const` for unmodified objects, so we can pass them to more functions.
Differential Revision: https://reviews.llvm.org/D128690
It's more natural to use uint8_t * (std::byte needs C++17 and llvm has
too much uint8_t *) and most callers use uint8_t * instead of char *.
The functions are recently moved into `llvm::compression::zlib::`, so
downstream projects need to make adaption anyway.
The two first parameters of checkPreprocessorOptions are "PPOpts, ExistingPPOpts".
All other callers of the function pass them consistently.
This avoids confusion when working on the code.
Differential Revision: https://reviews.llvm.org/D129277
* Refactor compression namespaces across the project, making way for a possible
introduction of alternatives to zlib compression.
Changes are as follows:
* Relocate the `llvm::zlib` namespace to `llvm::compression::zlib`.
Reviewed By: MaskRay, leonardchan, phosek
Differential Revision: https://reviews.llvm.org/D128953
We'll need to add more cases for Objective-C entities and adding
everything to `err_module_odr_violation_mismatch_decl_diff` makes it
harder to work with over time.
Differential Revision: https://reviews.llvm.org/D128488
It helps to avoid copy-paste mistakes and makes custom code paths more
noticeable.
Not funnelling all diagnostic through `ODRDiagDeclError` because plan to
break down `err_module_odr_violation_mismatch_decl_diff` into smaller
pieces instead of making it bigger and bigger.
Differential Revision: https://reviews.llvm.org/D128487
This is a support for " #pragma omp atomic compare fail ". It has Parser & AST support for now.
Reviewed By: tianshilei1992
Differential Revision: https://reviews.llvm.org/D123235
Adds support for the reserved locator 'omp_all_memory' for use
in depend clauses with 'out' or 'inout' dependence-types.
Differential Revision: https://reviews.llvm.org/D125828
This is a 1.9% reduction in PCH size in my measurements.
In abbreviated records, VBR6 seems to be slightl better than VBR8 for locations
that may be delta-encoded (i.e. not the first)
Differential Revision: https://reviews.llvm.org/D125952
Much of the size of PCH/PCM files comes from stored SourceLocations.
These are encoded using (almost) their raw value, VBR-encoded. Absolute
SourceLocations can be relatively large numbers, so this commonly takes
20-30 bits per location.
We can reduce this by exploiting redundancy: many "nearby" SourceLocations are
stored differing only slightly and can be delta-encoded.
Randam-access loading of AST nodes constrains how long these sequences
can be, but we can do it at least within a node that always gets
deserialized as an atomic unit.
TypeLoc is implemented in this patch as it's a relatively small change
that shows most of the API.
This saves ~3.5% of PCH size, I have local changes applying this technique
further that save another 3%, I think it's possible to get to 10% total.
Differential Revision: https://reviews.llvm.org/D125403
This diff changes the serialization of the `ORIGINAL_PCH_DIR`
entry in module files to be serialized relative to the module's
`BaseDirectory`. This will allow for the module to be relocatable
across machines.
The path is restored relative to the module's BaseDirectory on
deserialization.
Reviewed By: urnathan
Differential Revision: https://reviews.llvm.org/D124946
This diff changes the serialization of the `SUBMODULE_TOPHEADER`
entry in module files to be serialized relative to the module's
`BaseDirectory`. This matches the behavior of the
`SUBMODULE_HEADER` entry and will allow for the module to be
relocatable across machines.
The path is restored relative to the module's `BaseDirectory` on
deserialization.
Reviewed By: urnathan
Differential Revision: https://reviews.llvm.org/D124938
Emitting metadata for the same ivar multiple times can lead to
miscompilations. Objective-C runtime adds offsets to calculate ivar
position in memory and presence of duplicate offsets causes wrong final
position thus overwriting unrelated memory.
Such a situation is impossible with modules disabled as clang diagnoses
ivar redeclarations during sema checks after parsing
(`Sema::ActOnFields`). Fix the case with modules enabled by checking
during deserialization if ivar is already declared. We also support
a use case where the same category ends up in multiple modules. We
don't want to treat this case as ivar redeclaration and instead merge
corresponding ivars.
rdar://83468070
Differential Revision: https://reviews.llvm.org/D121177
This reverts commit eadf352707.
The reland fixes a couple of places in clang that were unneccesarily
requesting a null-terminated buffer of the PCH, and hitting assertions.
This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<FileEntryRef>`.
With the API change in place, this patch then removes some uses of the deprecated `FileEntry::getName()` (e.g. in `DependencyGraph.cpp` and `ModuleDependencyCollector.cpp`).
Reviewed By: dexonsmith, bnbarham
Differential Revision: https://reviews.llvm.org/D123574
Previously, if a `#pragma clang assume_nonnull begin` was at the
end of a premable with a `#pragma clang assume_nonnull end` at the
end of the main file, clang would diagnose an unterminated begin in
the preamble and an unbalanced end in the main file.
With this change, those errors no longer occur and the case above is
now properly handled. I've added a corresponding test to clangd,
which makes use of preambles, in order to verify this works as
expected.
Differential Revision: https://reviews.llvm.org/D122179
Update `WeakUndeclaredIdentifiers` to hold a collection of weak
aliases per identifier instead of only one.
This also allows the "used" state to be removed from `WeakInfo`
because it is really only there as an alternative to removing
processed map entries, and we can represent that using an empty set
now. The serialization code is updated for the removal of the field.
Additionally, a PCH test is added for the new functionality.
The records are grouped by the "target" identifier, which was already
being used as a key for lookup purposes. We also store only one record
per alias name; combined, this means that diagnostics are grouped by
the "target" and limited to one per alias (which should be acceptable).
Fixes PR28611.
Fixesllvm/llvm-project#28985.
Reviewed By: aaron.ballman, cebowleratibm
Differential Revision: https://reviews.llvm.org/D121927
Co-authored-by: Rachel Craik <rcraik@ca.ibm.com>
Co-authored-by: Jamie Schmeiser <schmeise@ca.ibm.com>
Current ASTContext.getAttributedType() takes attribute kind,
ModifiedType and EquivType as the hash to decide whether an AST node
has been generated or note. But this is not enough for btf_type_tag
as the attribute might have the same ModifiedType and EquivType, but
still have different string associated with attribute.
For example, for a data structure like below,
struct map_value {
int __attribute__((btf_type_tag("tag1"))) __attribute__((btf_type_tag("tag3"))) *a;
int __attribute__((btf_type_tag("tag2"))) __attribute__((btf_type_tag("tag4"))) *b;
};
The current ASTContext.getAttributedType() will produce
an AST similar to below:
struct map_value {
int __attribute__((btf_type_tag("tag1"))) __attribute__((btf_type_tag("tag3"))) *a;
int __attribute__((btf_type_tag("tag1"))) __attribute__((btf_type_tag("tag3"))) *b;
};
and this is incorrect.
It is very difficult to use the current AttributedType as it is hard to
get the tag information. To fix the problem, this patch introduced
BTFTagAttributedType which is similar to AttributedType
in many ways but with an additional BTFTypeTagAttr. The tag itself can
be retrieved with BTFTypeTagAttr.
With the new BTFTagAttributed type, the debuginfo code can be greatly
simplified compared to previous TypeLoc based approach.
Differential Revision: https://reviews.llvm.org/D120296
Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.
The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.
For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.
The recommit fixes the LLDB build failure.
Differential Revision: https://reviews.llvm.org/D117348
Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.
The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.
For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.
Differential Revision: https://reviews.llvm.org/D117348
ASTReader
This is a cleanup to reduce the lines of code to handle default template
argument in ASTReader.
Reviewed By: urnathan
Differential Revision: https://reviews.llvm.org/D118437
This patch replaces the exact include count of each file in `HeaderFileInfo` with a set of included files in `Preprocessor`.
The number of includes isn't a property of a header file but rather a preprocessor state. The exact number of includes is not used anywhere except statistic tracking.
Reviewed By: vsapsai
Differential Revision: https://reviews.llvm.org/D114095
The patch was reverted because it caused a crash during PCH build -- we
missed to update the RParenLoc in TreeTransform<Derived>::TransformAutoType.
This relands 55d96ac and 37ec65e with a test and fix.
Andreas Hollandt