llvm-project/llvm/test/Transforms/SampleProfile/Inputs
Hongtao Yu b9db70369b [CSSPGO] Split context string to deduplicate function name used in the context.
Currently context strings contain a lot of duplicated function names and that significantly increase the profile size. This change split the context into a series of {name, offset, discriminator} tuples so function names used in the context can be replaced by the index into the name table and that significantly reduce the size consumed by context.

A follow-up improvement made in the compiler and profiling tools is to avoid reconstructing full context strings which is  time- and memory- consuming. Instead a context vector of `StringRef` is adopted to represent the full context in all scenarios. As a result, the previous prevalent profile map which was implemented as a `StringRef` is now engineered as an unordered map keyed by `SampleContext`. `SampleContext` is reshaped to using an `ArrayRef` to represent a full context for CS profile. For non-CS profile, it falls back to use `StringRef` to represent a contextless function name. Both the `ArrayRef` and `StringRef` objects are underpinned by real array and string objects that are stored in producer buffers. For compiler, they are maintained by the sample reader. For llvm-profgen, they are maintained in `ProfiledBinary` and `ProfileGenerator`. Full context strings can be generated only in those cases of debugging and printing.

When it comes to profile format, nothing has changed to the text format, though internally CS context is implemented as a vector. Extbinary format is only changed for CS profile, with an additional `SecCSNameTable` section which stores all full contexts logically in the form of `vector<int>`, which each element as an offset points to `SecNameTable`. All occurrences of contexts elsewhere are redirected to using the offset of `SecCSNameTable`.

Testing
This is no-diff change in terms of code quality and profile content (for text profile).

For our internal large service (aka ads), the profile generation is cut to half, with a 20x smaller string-based extbinary format generated.

The compile time of ads is dropped by 25%.

Differential Revision: https://reviews.llvm.org/D107299
2021-08-30 20:09:29 -07:00
..
bad.extbinary.afdo [SampleFDO] Report error when reading a bad/incompatible profile instead of 2021-01-25 10:28:23 -08:00
bad_discriminator_value.prof
bad_fn_header.prof
bad_line_values.prof
bad_mangle.prof
bad_sample_line.prof
bad_samples.prof
branch.prof
calls.prof
cold-indirect-call.prof
cov-zero-samples.prof
coverage-warning.prof
csspgo-import-list.prof [CSSPGO] Load context profile for external functions in PreLink and populate ThinLTO import list 2021-03-15 12:22:15 -07:00
csspgo-use-preinliner.prof [CSSPGO] Add switch for sample loader to honor global pre-inliner decision from llvm-profgen 2021-08-25 17:20:15 -07:00
ctxsplit.extbinary.afdo [SampleFDO] Add the support to split the function profiles with context into 2021-01-19 15:16:19 -08:00
discriminator.prof
einline.prof
entry_counts.prof
entry_counts_cold.prof
flattened.prof
fnptr.binprof
fnptr.prof
fsafdo.extbinary.afdo [SampleFDO] New hierarchical discriminator for FS SampleFDO (ProfileData part) 2021-06-02 10:32:52 -07:00
fsafdo.prof [SampleFDO] New hierarchical discriminator for FS SampleFDO (ProfileData part) 2021-06-02 10:32:52 -07:00
function_metadata.compact.afdo
function_metadata.prof
gcc-simple.afdo
indirect-call-csspgo.prof [CSSPGO] Call site prioritized inlining for sample PGO 2021-02-01 23:46:34 -08:00
indirect-call.afdo
indirect-call.compact.afdo
indirect-call.prof
inline-act.prof
inline-callee-update.prof
inline-cold.prof
inline-combine.prof
inline-coverage.prof
inline-hint.prof
inline-mergeprof.md5.prof
inline-mergeprof.prof
inline-replay.txt [InlineAdvisor] Add single quotes around caller/callee names 2021-08-10 11:51:31 -07:00
inline-topdown.prof
inline.compactbinary.afdo
inline.extbinary.afdo
inline.fixlenmd5.extbinary.afdo [SampleFDO] Store fixed length MD5 in NameTable instead of using ULEB128 if 2020-12-08 16:21:01 -08:00
inline.md5extbinary.afdo
inline.prof
merge-function-attributes.afdo [AFDO] Merge function attributes after inlining 2021-07-09 16:47:12 -07:00
nodebug.prof
nolocinfo.prof
norepeated-icp-2.prof [SampleFDO] Another fix to prevent repeated indirect call promotion in 2021-03-04 18:44:12 -08:00
norepeated-icp-3.prof [SampleFDO] Don't mix up the existing indirect call value profile with the new 2021-03-18 09:54:34 -07:00
norepeated-icp.prof [SampleFDO] Stop repeated indirect call promotion for the same target. 2021-02-18 17:01:32 -08:00
offset.prof
profile-context-order.prof [CSSPGO] Process functions in a top-down order on a dynamic call graph. 2021-02-11 12:36:59 -08:00
profile-context-tracker.prof
profile-correlation-irreducible-loops.prof A post-processing for BFI inference 2021-06-11 21:46:04 -07:00
profile-symbol-list.ll Revert "[NFC] remove explicit default value for strboolattr attribute in tests" 2021-05-24 19:43:40 +02:00
profile-symbol-list.text [SampleFDO] Add a cutoff flag to control how many symbols will be included 2021-02-27 23:15:31 -08:00
profile-topdown-order.prof [CSSPGO] Process functions in a top-down order on a dynamic call graph. 2021-02-11 12:36:59 -08:00
profsampleacc.extbinary.afdo
propagate.prof
pseudo-probe-func-metadata.prof [CSSPGO] Consume pseudo-probe-based AutoFDO profile 2020-12-16 15:57:18 -08:00
pseudo-probe-icp-factor.prof [CSSPGO] Fix incorrect probe distribution factor computation in top-down inliner 2021-04-07 08:48:59 -07:00
pseudo-probe-inline.prof [CSSPGO] Report zero-count probe in profile instead of dangling probes. 2021-06-16 11:45:29 -07:00
pseudo-probe-profile-metadata.prof [SamplePGO] Keeping prof metadata for IndirectBrInst 2021-03-30 10:44:48 -07:00
pseudo-probe-profile.prof [CSSPGO] Consume pseudo-probe-based AutoFDO profile 2020-12-16 15:57:18 -08:00
pseudo-probe-update.prof [CSSPGO] Introducing distribution factor for pseudo probe. 2021-02-02 11:55:01 -08:00
remap-2.prof
remap.map
remap.prof
remarks-hotness.prof
remarks.prof
summary.prof
syntax.prof
uniqname.nosuffix.afdo [SampleFDO] Support enabling -funique-internal-linkage-name. 2021-03-09 21:41:40 -08:00
uniqname.suffix.afdo [SampleFDO] Support enabling -funique-internal-linkage-name. 2021-03-09 21:41:40 -08:00
uniqname.suffix.prof [SampleFDO] Support enabling -funique-internal-linkage-name. 2021-03-09 21:41:40 -08:00
use-sample-profile-attr.prof
warm-inline-instance.prof