This reverts commit 794b7ea960, and
thus restores commit a212d8da94, and
follow on fixes 0cd6763fa9,
e9ff53d42f, and
37c6a25e9a.
Use a hash function (BLAKE3) instead of hash_combine/hash_code which are
not guaranteed to be stable across executions.
Additionally, it adds a "REQUIRES: x86_64-linux" to the tests that have
raw profile inputs to avoid failures on big endian bots.
Reviewers: snehasish, davidxl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D128142
This reverts commit a212d8da94, and follow
on fixes 0cd6763fa9,
e9ff53d42f, and
37c6a25e9a.
After re-reading the documentation for hash_combine, I don't think this
is the appropriate hash function to use for computing the hash to use as
a stack id in the metadata, since it is not guaranteed to produce stable
values across executions. I have not hit this problem, but plan to
switch to using an MD5 hash. I am hitting an issue with one of the bots
(https://lab.llvm.org/buildbot/#/builders/171/builds/20732)
where the values produced are only the lower 32 bits of the expected
hash values, however, which I assume is related to the implementation of
hash_combine and hash_code.
I believe I fixed all of the other bot failures with the follow on fixes,
which I'll merge into the new version before reapplying.
Profile matching and IR annotation for memprof profiles.
See also related RFCs:
RFC: Sanitizer-based Heap Profiler [1]
RFC: A binary serialization format for MemProf [2]
RFC: IR metadata format for MemProf [3]*
* Note that the IR metadata format has changed from the RFC during
implementation, as described in the preceeding patch adding the basic
metadata and verification support.
The matching is performed during the normal PGO annotation phase, to
ensure that the inlines applied in the IR at that point are a subset
of the inlines in the profiled binary and thus reflected in the
profile's call stacks. This is important because the call frames are
associated with functions in the profile based on the inlining in the
symbolized call stacks, and this simplifies locating the subset of
profile data relevant for matching onto each function's IR.
The PGOInstrumentationUse pass is enhanced to perform matching for
whatever combination of memprof and regular PGO profile data exists in
the profile.
Using the utilities introduced in D128854:
The memprof profile data for each context is converted to "cold" or
"notcold" based on parameterized thresholds for size, access count, and
lifetime. The memprof allocation contexts are trimmed to the minimal
amount of context required to uniquely identify whether the context is
cold or not cold. For allocations where all profiled contexts have the
same allocation type, no memprof metadata is attached and instead the
allocation call is directly annotated with an attribute specifying the
alloction type. This is the same attributed that will be applied to
allocation calls once cloned for different contexts, and later used
during LibCall simplification to emit allocation hints [4].
Depends on D128141 and D128854.
[1] https://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html
[2] https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html
[3] https://discourse.llvm.org/t/rfc-ir-metadata-format-for-memprof/59165
[4] ab87cf382d
Differential Revision: https://reviews.llvm.org/D128142
Currently, `__attribute__((no_sanitize('hwaddress')))` is not possible. Add this piece of plumbing, and now that we properly support copying attributes between an old and a new global variable, add a regression test for the GlobalOpt bug that previously lost the attribute.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D127544
RE-LAND (reverts a revert):
This reverts commit 8e1f47b596.
This patch adds generation of sanitizer metadata attributes (which were
added in D126100) to the clang frontend.
We still currently generate the llvm.asan.globals that's consumed by
the IR pass, but the plan is to eventually migrate off of that onto
purely debuginfo and these IR attributes.
Reviewed By: vitalybuka, kstoimenov
Differential Revision: https://reviews.llvm.org/D126929
This patch adds generation of sanitizer metadata attributes (which were
added in D126100) to the clang frontend.
We still currently generate the `llvm.asan.globals` that's consumed by
the IR pass, but the plan is to eventually migrate off of that onto
purely debuginfo and these IR attributes.
Reviewed By: vitalybuka, kstoimenov
Differential Revision: https://reviews.llvm.org/D126929
A significant number of our tests in C accidentally use functions
without prototypes. This patch converts the function signatures to have
a prototype for the situations where the test is not specific to K&R C
declarations. e.g.,
void func();
becomes
void func(void);
This is the final batch of tests being updated to add prototypes,
hopefully.
A significant number of our tests in C accidentally use functions
without prototypes. This patch converts the function signatures to have
a prototype for the situations where the test is not specific to K&R C
declarations. e.g.,
void func();
becomes
void func(void);
This is the tenth batch of tests being updated (there are a
significant number of other tests left to be updated).
This reverts commit 640beb38e7.
That commit caused performance degradtion in Quicksilver test QS:sGPU and a functional test failure in (rocPRIM rocprim.device_segmented_radix_sort).
Reverting until we have a better solution to s_cselect_b64 codegen cleanup
Change-Id: Ibf8e397df94001f248fba609f072088a46abae08
Reviewed By: kzhuravl
Differential Revision: https://reviews.llvm.org/D115960
Change-Id: Id169459ce4dfffa857d5645a0af50b0063ce1105
CodeGenAction::ExecuteAction creates a BackendConsumer for the
purpose of handling diagnostics. The BackendConsumer's
DiagnosticHandlerImpl method expects CurLinkModule to be set,
but this did not happen on the code path that goes through
ExecuteAction. This change makes it so that the BackendConsumer
constructor used by ExecuteAction requires the Module to be
specified and passes the appropriate module in ExecuteAction.
The change also adds a test that fails without this change
and passes with it. To make the test work, the FIXME in the
handling of DK_Linker diagnostics was addressed so that warnings
and notes are no longer silently discarded. Since this introduces
a new warning diagnostic, a flag to control it (-Wlinker-warnings)
has also been added.
Reviewed By: xur
Differential Revision: https://reviews.llvm.org/D108603
This will embed bitcode after (Thin)LTO merge, but before optimizations.
In the case the thinlto backend is called from clang, the .llvmcmd
section is also produced. Doing so in the case where the caller is the
linker doesn't yet have a motivation, and would require plumbing through
command line args.
Differential Revision: https://reviews.llvm.org/D87636
This patch adds clang options:
-fbasic-block-sections={all,<filename>,labels,none} and
-funique-basic-block-section-names.
LLVM Support for basic block sections is already enabled.
+ -fbasic-block-sections={all, <file>, labels, none} : Enables/Disables basic
block sections for all or a subset of basic blocks. "labels" only enables
basic block symbols.
+ -funique-basic-block-section-names: Enables unique section names for
basic block sections, disabled by default.
Differential Revision: https://reviews.llvm.org/D68049
For IR input files, we currently use LLVM diagnostic handler even the
compilation is from clang. As a result, we are not able to use -Rpass
to get the transformation reports. Some warnings are not handled
properly either: We found many mysterious warnings in our ThinLTO backend
compilations in SamplePGO and CSPGO. An example of the warning:
"warning: net/proto2/public/metadata_lite.h:51:21: 0.02% (1 / 4999)"
This turns out to be a warning by Wmisexpect, which is supposed to be
filtered out by default. But since the filter is in clang's
diagnostic hander, we emit these incomplete warnings from LLVM's
diagnostic handler.
This patch uses clang diagnostic handler for IR input files. We create
a fake backendconsumer just to install the diagnostic handler.
With this change, we will have proper handling of all the warnings and we can
use -Rpass* options in IR input files compilation.
Also note that with is patch, LLVM's diagnostic options, like
"-mllvm -pass-remarks=*", are no longer be able to get optimization remarks.
Differential Revision: https://reviews.llvm.org/D72523
Also revert follow-up changes to the test.
Reason: the patch breaks our internal clang-tidy integration.
It's also unclear why we should use getRealPath instead of plumbing the
VFS to SanitizerBlacklist, see original commit thread of cfe-commits for
a discussion.
llvm-svn: 374151
Summary:
The is the clang side of the fix in D55047, to handle the case where
two different modules have local variables with the same GUID because
they had the same source file name at compilation time. Allow multiple
symbols with the same GUID to be imported, and test that this case works
with the distributed backend path.
Depends on D55047.
Reviewers: evgeny777
Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D55048
llvm-svn: 347887
Summary:
the previous patch (https://reviews.llvm.org/rC346642) has been reverted because of test failure under windows.
So this patch fix the test cfe/trunk/test/CodeGen/code-coverage-filter.c.
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: cfe-commits, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D54600
llvm-svn: 347144
Summary:
These options are taking regex separated by colons to filter files.
- if both are empty then all files are instrumented
- if -fprofile-filter-files is empty then all the filenames matching any of the regex from exclude are not instrumented
- if -fprofile-exclude-files is empty then all the filenames matching any of the regex from filter are instrumented
- if both aren't empty then all the filenames which match any of the regex in filter and which don't match all the regex in filter are instrumented
- this patch is a follow-up of https://reviews.llvm.org/D52033
Reviewers: marco-c, vsk
Reviewed By: marco-c, vsk
Subscribers: cfe-commits, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D52034
llvm-svn: 346642
In DWARF v5 the Line Number Program Header is extensible, allowing values with
new content types. This vendor extension to DWARF v5 allows source text to be
embedded directly in the line tables of the debug line section.
Add new flag (-g[no-]embed-source) to Driver and CC1 which indicates
that source should be passed through to LLVM during CodeGen.
Differential Revision: https://reviews.llvm.org/D42766
llvm-svn: 326102
Summary:
Gold plugin does not add pass to ThinLTO modules without useful symbols.
In this case ThinLTO can't create corresponding index file and some features, like CFI,
cannot be processes by backed correctly without index.
Given that we don't need the backed output we can request it to avoid
processing the module. This is implemented by this patch using new
"SkipModuleByDistributedBackend" flag.
Reviewers: pcc, tejohnson
Subscribers: mehdi_amini, inglorion, eraman, cfe-commits
Differential Revision: https://reviews.llvm.org/D42995
llvm-svn: 325411
Summary:
This is the follow-up patch to D37924.
This change refactors clang to use the the newly added section headers
in SpecialCaseList to specify which sanitizers blacklists entries
should apply to, like so:
[cfi-vcall]
fun:*bad_vcall*
[cfi-derived-cast|cfi-unrelated-cast]
fun:*bad_cast*
The SanitizerSpecialCaseList class has been added to allow querying by
SanitizerMask, and SanitizerBlacklist and its downstream users have been
updated to provide that information. Old blacklists not using sections
will continue to function identically since the blacklist entries will
be placed into a '[*]' section by default matching against all
sanitizers.
Reviewers: pcc, kcc, eugenis, vsk
Reviewed By: eugenis
Subscribers: dberris, cfe-commits, mgorny
Differential Revision: https://reviews.llvm.org/D37925
llvm-svn: 314171
Summary: The patch makes the integration test cover major sample PGO components.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: sanjoy, cfe-commits
Differential Revision: https://reviews.llvm.org/D34725
llvm-svn: 307445
If a regular LTO module has a summary index, then instead of linking
it into the combined regular LTO module right away, add it to the
combined summary index and associate it with a special module that
represents the combined regular LTO module.
Any such modules are linked during LTO::run(), at which time we use
the results of summary-based dead stripping to control whether to
link prevailing symbols.
Differential Revision: https://reviews.llvm.org/D33922
llvm-svn: 305482
Summary:
Because SamplePGO passes will be invoked twice in ThinLTO build: once at compile phase, the other at backend. We want to make sure the IR at the 2nd phase matches the hot part in pro
file, thus we do not want to inline hot callsites in the first phase.
Reviewers: tejohnson, eraman
Reviewed By: tejohnson
Subscribers: mehdi_amini, cfe-commits, Prazek
Differential Revision: https://reviews.llvm.org/D31202
llvm-svn: 298429
The backend now has the capability to save information from optimizations, the
same information that can be used to generate optimization diagnostics but in
machine-consumable form, into an output file. This can be enabled when using
opt (see r282539), and this change enables it when using clang. The idea is
that other tools will be able to consume these files, and perhaps in
combination with the original source code, produce various kinds of
optimization reports for users (and for compiler developers).
We now have at-least two tools that can consume these files:
* tools/llvm-opt-report
* utils/opt-viewer
Using the flag -fsave-optimization-record will cause the YAML file to be
generated; the file name will be based on the output file name (if we're using
-c or -S and have an output name), or the input file name. When we're using
CUDA, or some other offloading mechanism, separate files are generated for each
backend target. The output file name can be specified by the user using
-foptimization-record-file=filename.
Differential Revision: https://reviews.llvm.org/D25225
llvm-svn: 283834
This new test tests that functions are capable of being imported, rather than
that the import pass is run. This new test is compatible with the approach
being developed in D20268 which runs the importer on its own rather than in
a pass.
Differential Revision: http://reviews.llvm.org/D21542
llvm-svn: 273347
Summary: We need to call PruneEH pass before AutoFDO pass so that some EH-related calls can get inlined in Sample Profile pass.
Reviewers: davidxl, dnovillo
Subscribers: junbuml, llvm-commits
Differential Revision: http://reviews.llvm.org/D21197
llvm-svn: 273298
This patch changes cc1 option for PGO profile use from
-fprofile-instr-use=<path> to -fprofile-instrument-use-path=<path>.
-fprofile-instr-use=<path> is now a driver only option.
In addition to decouple the cc1 option from the driver level option, this patch
also enables IR level profile use. cc1 option handling now reads the profile
header and sets CodeGenOpt ProfileUse (valid values are {None, Clang, LLVM}
-- this is a common enum for -fprofile-instrument={}, for the profile
instrumentation), and invoke the pipeline to enable the respective PGO use pass.
Reviewers: silvas, davidxl
Differential Revision: http://reviews.llvm.org/D17737
llvm-svn: 262515
Teresa Johnson