There's a few relevant forward declarations in there that may require downstream
adding explicit includes:
llvm/MC/MCContext.h no longer includes llvm/BinaryFormat/ELF.h, llvm/MC/MCSubtargetInfo.h, llvm/MC/MCTargetOptions.h
llvm/MC/MCObjectStreamer.h no longer include llvm/MC/MCAssembler.h
llvm/MC/MCAssembler.h no longer includes llvm/MC/MCFixup.h, llvm/MC/MCFragment.h
Counting preprocessed lines required to rebuild llvm-project on my setup:
before: 1052436830
after: 1049293745
Which is significant and backs up the change in addition to the usual benefits of
decreasing coupling between headers and compilation units.
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119244
This patch makes llvm-rc/windres prefer <target>-clang over
clang when doing it's preprocessing. This is so that we can
have a .cfg file for <target> and configure sysroot and other
important flags.
Config files not picked up with clang --target=<target>
automatically.
We only look for <target>-clang in the same dir as llvm-windres
and not for all PATHs to minimize the change.
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D119219
This adds a --build-id=<hex build ID> flag to llvm-symbolizer. If --obj
is unspecified, this will attempt to look up the provided build ID using
whatever mechanisms are available to the Symbolizer (typically,
debuginfod). The semantics are then as if the found binary were given
using the --obj flag.
Reviewed By: jhenderson, phosek
Differential Revision: https://reviews.llvm.org/D118633
Debuginfod can pull in libcurl as a dependency, which isn't appropriate
for libLLVM. (See
https://gitlab.freedesktop.org/mesa/mesa/-/issues/5732).
This change breaks out debuginfod into a separate non-component library
that can be used directly in llvm-symbolizer. The tool can inject
debuginfod into the Symbolizer library via an abstract DebugInfoFetcher
interface, breaking the dependency of Symbolizer on debuinfod.
See https://github.com/llvm/llvm-project/issues/52731
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D118413
Tracking optimized-away inlinees based on all probes in a binary is expansive in terms of memory usage I'm making the tracking on-demand based on profiled functions only. This saves about 10% memory overall for a medium-sized benchmark.
Before:
note: After parsePerfTraces
note: Thu Jan 27 18:42:09 2022
note: VM: 8.68 GB RSS: 8.39 GB
note: After computeSizeForProfiledFunctions
note: Thu Jan 27 18:42:41 2022
note: **VM: 10.63 GB RSS: 10.20 GB**
note: After generateProbeBasedProfile
note: Thu Jan 27 18:45:49 2022
note: VM: 25.00 GB RSS: 24.95 GB
note: After postProcessProfiles
note: Thu Jan 27 18:49:29 2022
note: VM: 26.34 GB RSS: 26.27 GB
After:
note: After parsePerfTraces
note: Fri Jan 28 12:04:49 2022
note: VM: 8.68 GB RSS: 7.65 GB
note: After computeSizeForProfiledFunctions
note: Fri Jan 28 12:05:26 2022
note: **VM: 8.68 GB RSS: 8.42 GB**
note: After generateProbeBasedProfile
note: Fri Jan 28 12:08:03 2022
note: VM: 22.93 GB RSS: 22.89 GB
note: After postProcessProfiles
note: Fri Jan 28 12:11:30 2022
note: VM: 24.27 GB RSS: 24.22 GB
This should be a no-diff change in terms of profile quality.
Reviewed By: wenlei
Differential Revision: https://reviews.llvm.org/D118515
Summary:
1.added a helper function isSymbolDefined().
2.Split out sorting code
3.refactor symbol comparing function
Reviewers: James Henderson,Fangrui Song
Differential Revision: https://reviews.llvm.org/D119028
The addProcessSymbols function added a generator for process symbols to every
JITDylib in the session, but this is unhelpful default behavior (e.g. it will
cause the ORC runtime's definition of __cxa_atexit to be shadowed by the
process's definition for all JITDylibs except main).
Since the loadProcessSymbols function already added a generator to main we only
need to drop this function. Other JITDylibs wishing to resolve process symbols
can link against the main JITDylib by passing `-lmain`.
libtool can currently produce 2 warnings:
1. No symbols were in the object file
2. An object file with the same basename was specified multiple times
The first warning here is often harmless and may just mean you have some
translation units with no symbols for the target you're building for.
The second warning can lead to real issues like those mentioned in
https://reviews.llvm.org/D113130 where ODR violations can slip in.
This introduces a new -warnings_as_errors flag that can be used by build
systems that want to verify they never hit these warnings. For example
with bazel the libtool caller first uniques names to make sure the
duplicate base name case is not possible, but if that doesn't work as
expected, having it fail would be preferred.
It's also worth noting that llvm-libtool-darwin works around an issue
that cctools libtool experiences related to debug info and duplicate
basenames, the workaround is described here:
30baa5d2a4/llvm/lib/Object/ArchiveWriter.cpp (L424-L465)
And it avoids this bug:
f0cbbb1c37/DuplicateBasenameIssue
Differential Revision: https://reviews.llvm.org/D118931
Inspired by the discussion on D118694, this adds a straightforward
fuzzer for bitcode reading. Currently it will very quickly run into
OOM, because we do unconditional vector reservations with
user-provided sizes.
When using llvm-libtool-darwin with LTO building llvm itself, it crashed
on a file with an asm module in the bitcode. This fixes that by
correctly registering the targets for this.
Differential Revision: https://reviews.llvm.org/D118575
This change extends the RawMemProfReader to read all the sections of the
raw profile and symbolize the virtual addresses recorded as part of the
callstack for each allocation. For now the symbolization is used to
display the contents of the profile with llvm-profdata.
Differential Revision: https://reviews.llvm.org/D116784
Print out the profile summary in YAML format to make it easier to for
tools and tests to read in the contents of the raw profile.
Differential Revision: https://reviews.llvm.org/D116783
This more closely mirrors apple's libtool, and also potentially makes it
clearer for multi-arch archives where the issue lies.
Differential Revision: https://reviews.llvm.org/D118867
llvm-ar silently converts a thin archive to a regular archive when you
specify a modification operation (e.g. 'r') without the 'T' modifier.
This change stops this from occuring. If a user is trying to convert
between thin and full archives then they can explicitly use the 'L'
command to createa new archive.
Differential Revision: https://reviews.llvm.org/D118693
The -alias option can be used to define aliases within a JITDylib. The
immediate motivation is to simplify testing of ORC runtime functions using
existing testcases (e.g. by aliasing dlfcn functions to their ORC-runtime
counterparts, like -alias dlopen=__orc_rt_macho_dlopen). The option is likely
to be useful for testing in general.
The -define-abs option is shortened to -abs for consistency with -alias.
Change a type from `int64_t` to `long long` to make a buildbot happy.
https://lab.llvm.org/buildbot/#/builders/196/builds/7371
```
error: no matching function for call to 'getAsSignedInteger'
if (getAsSignedInteger(ValStr, 0, Val))
```
Worked With My Compiler (TM)
This is a tool which can handle bitstream and YAML remarks. The idea here is to
provide more insight into which functions changed in a benchmark when testing
compiler changes.
E.g. "foo got 20% bigger, so maybe we should look more closely at that."
To use the tool, you can use...
```
$ llvm-remark-size-diff remarks_file_a remarks_file_b --parser=yaml|bitstream
```
... on two remarks files containing at least instruction count remarks. This
will output some data on instruction count change and also other relevant
information such as stack size change from `remarks_file_a` to `remarks_file_b`.
This is a bit of a WIP so I'm happy to change the format etc. Ultimately I think
it'd be best to have some JSON output which could be consumed by another tool.
But some base-level, greppable output is very handy to have anyway.
The format I'm proposing here is
```
<files> <inc/dec in inst count> <fn name> <inst count change> <stack B change>
```
Where the files and increase/decrease are indicated like below:
- `<files>` is one of `++` (file B), `--` (file A), `==` (both)
- `<inc/dec in inst count>` is one of `>` (increase) or `<` (decrease)
This makes it easy to grep for things like "which functions appeared in A but
did not appear in B?" Or "what are all the instruction count decreases?"
Differential Revision: https://reviews.llvm.org/D112940
Previously the options category given to cl::HideUnrelatedOptions was
local to llvm-reduce.cpp and as a result only options declared in that
file were visible in the -help options listing. This was a bit
unfortunate since there were several useful options declared in other
files. This patch addresses that.
Differential Revision: https://reviews.llvm.org/D118682
Symbol versioning can prevent unintented install-time conflicts
between different llvm versions. Users may need to override this
for particular products (e.g. Julia), but this requires carrying
a source code patch. This patch moves this ability to a
configuration option. NFC for existing usage.
Differential Revision: https://reviews.llvm.org/D118672
Based on the output of include-what-you-use.
This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/
I've tried to summarize the biggest change below:
- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h
And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948
200k lines less to process is no that bad ;-)
Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D118652
The namespace llvm::swift is causing errors to pop up in the apple/llvm-project build when cherry-picking 4ce1f3d47c into apple/llvm-project
Differential Review: https://reviews.llvm.org/D118716
If you don't include a target that directly references CFGuard it
doesn't get built into opt or the llvm library build, which causes some
test cases to fail.
Including this in opt explicitly resolve those issues.
In binutils, ar -s and randlib don't convert a thin archive to a regular one.
This behavior makes sense and this patch ports the behavior.
Reviewed By: gbreynoo
Differential Revision: https://reviews.llvm.org/D117443
In GNU ar (since 2008), the modifier 'T' means creating a thin archive.
In many other ar implementations (FreeBSD, macOS, elfutils, etc), -T
means "allow filename truncation of extracted files", as specified by
X/Open System Interface.
For portability, 'T' with thin archive semantics should be avoided.
See https://sourceware.org/bugzilla/show_bug.cgi?id=28759 binutils 2.38
will deprecate 'T' (without diagnostic) and add --thin.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D116979
When exporting textual IR during reduction the ShouldPreserveUseListOrder
parameter of the IR printer should be set to get predictable results.
Differential Revision: https://reviews.llvm.org/D118585
Summary:
Add code object v5 support (deafult is still v4)
Generate metadata for implicit kernel args for the new ABI
Set the metadata version to be 1.2
Reviewers:
t-tye, b-sumner, arsenm, and bcahoon
Fixes:
SWDEV-307188, SWDEV-307189
Differential Revision:
https://reviews.llvm.org/D118272
Previously you would get this error:
```
error: unsupported load command (cmd=0x2d)
```
If the binary you were redefining the symbols of contained a
LC_LINKER_OPTION load command. This command does not need to be changed
when redefining symbols so we can ignore it like many others.
Differential Revision: https://reviews.llvm.org/D118526
Based on the output of include-what you-use.
Most notably, llvm/Remarks/Remark.h is no longer automatically included by
llvm/Remarks/RemarkParser.h, so client code may need to include explicitly.
clang++ -E -Iinclude -I../llvm/include ../llvm/lib/Remarks/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 770253
after: 759347
Related discourse thread: https://llvm.discourse.group/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D118506
Add support for Swift reflection metadata to dsymutil.
This patch adds support for copying Swift reflection metadata (__swift5_.* sections) from .o files to into the symbol-rich binary in the output .dSYM. The functionality is automatically enabled only if a .o file has reflection metadata sections and the binary doesn't. When copying dsymutil moves the section from the __TEXT segment to the __DWARF segment.
rdar://76973336
Differential Revision: https://reviews.llvm.org/D115007
Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block
The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.
When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).
[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116180
This change refactors the ProfileKind enum into a bitset enum to
represent the different attributes a profile can have. This change
simplifies the logic in the instrprof writer when multiple profiles are
merged together. In the future we plan on introducing a new memory
profile section which will extend the enum by one additional entry.
Without this change when accounting for memory profiles will have to be
maintained separately and will make the logic more complex.
Differential Revision: https://reviews.llvm.org/D115393
Adding -debugify and -check-debugify in the PassRegistry will make
sure the passes are listed properly by -print-pipeline-passes as
well as -print-passes.
It also allows removal of the custom pipeline parsing callback that
has been used in the NewPMDriver.
Differential Revision: https://reviews.llvm.org/D118369
Use the `llvm-profdata show` command to verify debug info for profile correlation using the `--debug-info` option.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D118181
serge-sans-paille