On Windows we don't create symlinks for the binaries (clang++, clang-cl)
since the support requires special setup (group policy settings and
you need to know exactly our distribution story). But if you know
about these things and have a controlled environment there is a lot
of storage to be saved, so let's add a manual opt-in for using symlinks
on Windows with LLVM_FORCE_CREATE_SYMLINKS=ON.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D135578
This is a breaking change. If you were passing one of those three runtimes
in LLVM_ENABLE_PROJECTS, you need to start passing them in LLVM_ENABLE_RUNTIMES
instead. The runtimes in LLVM_ENABLE_RUNTIMES will start being built using
the "bootstrapping build" instead, which means that they will be built
using the just-built Clang. This is usually what you wanted anyway.
If you were using LLVM_ENABLE_PROJECTS=all with the explicit goal of
building these three runtimes, you can now use LLVM_ENABLE_RUNTIMES=all
and these runtimes will be built using the bootstrapping build.
NOTE: This is a re-application of 887b8bd733 which had been reverted
in 6b03a4fea0 because it broke the Sphinx documentation publishers.
The Sphinx documentation publishers have now been moved to using
the runtimes build, so this should not be an issue anymore.
Differential Revision: https://reviews.llvm.org/D132480
GetErrcMessages.cmake module makes use of cmake's try_run which by
default builds its sources in debug mode unless configured with
CMAKE_TRY_COMPILE_CONFIGURATION. Debug builds on Windows sometimes fail
when appropraite DLLs are not included in path. Also on Windows on Arm
machines debug builds sometimes fail to link the correct debug DLLs.
To fix this I am setting CMAKE_TRY_COMPILE_CONFIGURATION to active build
configuration of currently configured LLVM project. This makes sure we
select same build type for try_run/try_compile cmake modules as
currently configured LLVM project.
Reviewed By: zero9178
Differential Revision: https://reviews.llvm.org/D133482
Enable registration of multiple exegesis targets at once. Use idiomatic approach to defining target select macros, but leave code in the llvm-mca sub-directories for now.
This is a stepping stone towards allowing llvm-exegesis benchmarking via simulator or testing in non-target dependent tests.
Differential Revision: https://reviews.llvm.org/D133605
This is a breaking change. If you were passing one of those three runtimes
in LLVM_ENABLE_PROJECTS, you need to start passing them in LLVM_ENABLE_RUNTIMES
instead. The runtimes in LLVM_ENABLE_RUNTIMES will start being built using
the "bootstrapping build" instead, which means that they will be built
using the just-built Clang. This is usually what you wanted anyway.
If you were using LLVM_ENABLE_PROJECTS=all with the explicit goal of
building these three runtimes, you can now use LLVM_ENABLE_RUNTIMES=all
and these runtimes will be built using the bootstrapping build.
Differential Revision: https://reviews.llvm.org/D132480
A simple sed doing these substitutions:
- `${LLVM_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX}\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`
where `\>` means "word boundary".
The only manual modifications were reverting changes in
- `runtimes/CMakeLists.txt`
because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.
There are some `${LLVM_BINARY_DIR}/lib` without the `${LLVM_LIBDIR_SUFFIX}`, but these refer to the lib subdirectory of the source (`llvm/lib`). That `lib` is automatically appended to make the local `CMAKE_CURRENT_BINARY_DIR` value by `add_subdirectory`; since the directory name in the source tree is fixed without any suffix, the corresponding `CMAKE_CURRENT_BINARY_DIR` will also be. We therefore do not replace it but leave it as-is.
This picks up where D133828 left off, getting the occurrences with*out* `CMAKE_CFG_INTDIR`. But this is difficult to do correctly and so not done in the (retroactively) previous diff.
This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.
Reviewed By: sebastian-ne
Differential Revision: https://reviews.llvm.org/D132316
add LLVM_PREFER_STATIC_ZSTD (default TRUE) cmake config flag
(compression test seems to fail for shared zstd on windows, note that zstd multithread is by default disabled in the static build so it may be a hidden variable)
propagate variable zstd_DIR in LLVMConfig.cmake.in
fix llvm-config CMakeLists.txt behavior for absolute libs windows
get zstd lib name
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D132870
This patch is in preparation for removing libcxx, libcxxabi and libunwind
from LLVM_ENABLE_PROJECTS. When we make that switch, folks who were
previously using LLVM_ENABLE_PROJECTS=all in order to build those
runtimes will be able to add LLVM_ENABLE_RUNTIMES=all to start building
those projects using the bootstrapping build.
This is technically a breaking change for folks who had been using
LLVM_ENABLE_RUNTIMES=all, however I suspect the set of people currently
doing that is extremely small, so this is likely acceptable (more than
breaking folks who are using LLVM_ENABLE_PROJECTS=all).
Differential Revision: https://reviews.llvm.org/D132479
TLite is a lightweight, statically linkable[1], model evaluator, supporting a
subset of what the full tensorflow library does, sufficient for the
types of scenarios we envision having. It is also faster.
We still use saved models as "source of truth" - 'release' mode's AOT
starts from a saved model; and the ML training side operates in terms of
saved models.
Using TFLite solves the following problems compared to using the full TF
C API:
- a compiler-friendly implementation for runtime-loadable (as opposed
to AOT-embedded) models: it's statically linked; it can be built via
cmake;
- solves an issue we had when building the compiler with both AOT and
full TF C API support, whereby, due to a packaging issue on the TF
side, we needed to have the pip package and the TF C API library at
the same version. We have no such constraints now.
The main liability is it supporting a subset of what the full TF
framework does. We do not expect that to cause an issue, but should that
be the case, we can always revert back to using the full framework
(after also figuring out a way to address the problems that motivated
the move to TFLite).
Details:
This change switches the development mode to TFLite. Models are still
expected to be placed in a directory - i.e. the parameters to clang
don't change; what changes is the directory content: we still need
an `output_spec.json` file; but instead of the saved_model protobuf and
the `variables` directory, we now just have one file, `model.tflite`.
The change includes a utility showing how to take a saved model and
convert it to TFLite, which it uses for testing.
The full TF implementation can still be built (not side-by-side). We
intend to remove it shortly, after patching downstream dependencies. The
build behavior, however, prioritizes TFLite - i.e. trying to enable both
full TF C API and TFLite will just pick TFLite.
[1] thanks to @petrhosek's changes to TFLite's cmake support and its deps!
This will make the following patches to migrate projects off of the
LLVM_ENABLE_PROJECTS build onto the LLVM_ENABLE_RUNTIMES build much
easier to comprehend. This patch should be a NFC since it keeps the
same set of runtimes being built by default.
Differential Revision: https://reviews.llvm.org/D132478
A simple sed doing these substitutions:
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`
where `\>` means "word boundary".
The only manual modifications were reverting changes in
- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake
- `runtimes/CMakeLists.txt`
because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.
This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.
Reviewed By: sebastian-ne
Differential Revision: https://reviews.llvm.org/D132316
This commit reverts the following commits:
- 952f90b72b
- e6a0800532 (D132298)
- 176db3b3ab (D132324)
These commits caused CI instability and need to be reverted in order
to figure things out again. See the discussion in https://llvm.org/D132324
for more information.
That commit (D132324) made putting libc++ in LLVM_ENABLE_RUNTIMES a hard
error, but ther are some out-of-tree CI jobs (especially docs that need
fixing).
I am thus making it a "soft error" for now, until that is resolved.
This has been officially deprecated since D112724, meaning the
deprecation warning is present in released 14 and 15.
This makes me think that now, shortly after the 15 release is branched,
is a good time to pull the trigger.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D132324
Working back towards D130586.
Bolt didn't use `LLVM_LIBDIR_SUFFIX` before, and has no in-tree reverse
dependencies, it seems easier to add.
The change in LLVM itself is to prevent some unexpected `lib64` from
cropping up due to the `CMAKE_INSTALL_LIBDIR` defaulting logic.
Differential Revision: https://reviews.llvm.org/D132297
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.
`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.
I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.
Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi
Differential Revision: https://reviews.llvm.org/D130586
Include utils/unittest before projects and runtimes so that downstream
projects can check for the existence of the llvm_gtest target. This is
motivated by 9c6c4d675b which fixes the stand-alone build
configuration where LLVM_MAIN_SRC_DIR does not exist.
Differential revision: https://reviews.llvm.org/D124314
Move the `LLVM_INSTALL_PACKAGE_DIR` declaration from llvm/cmake/modules
directory to the top-level llvm/CMakeLists.txt, in order to fix
the regression in `llvm-config --cmakedir` output for installed LLVM.
Since the tools directory is processed by CMake prior to
llvm/cmake/modules, the llvm-config executable ended up using
the variable prior to it being defined and incorrectly used an empty
path, resulting in e.g.:
$ llvm-config --cmakedir
/usr/lib/llvm/16/
With this patch, the path is defined (and therefore the default value
is being set) prior to adding the tools subdirectory and llvm-config
starts working correctly:
$ llvm-config --cmakedir
/usr/lib/llvm/16/lib64/cmake/llvm
This fixes a regression introduced by D130539. Thanks to Petr Polezhaev
for reporting the problem @ https://bugs.gentoo.org/865165
Differential Revision: https://reviews.llvm.org/D131878
It fittingly already makes use of add_llvm_utility during target creation and is usually only used in conjunction with other (test) related utilities such as FileCheck and not, which are already in utils.
Differential Revision: https://reviews.llvm.org/D131713
Also nuke the old value out of the cache if it's there, otherwise
it could lead to problems when the project increases the standard
and the developer just runs the "make/ninja" program.
Reviewed By: royjacobson
Differential Revision: https://reviews.llvm.org/D131367
Also make the soft toolchain requirements hard. This allows
us to use C++17 features in LLVM now.
If we find patterns with C++17 that improve readability
it should be recommended in the coding standards.
Reviewed By: jhenderson, cor3ntin, MaskRay
Differential Revision: https://reviews.llvm.org/D130689
In the top-level llvm `CMakeLists.txt`, we need to call
`find_package(Python3)` *before* including `config-ix.cmake`, otherwise
the latter will not be able to successfully search for python modules
using `find_python_module()`. Also set `LLVM_MINIMUM_PYTHON_VERSION`
before calling `find_package(Python3)`, moving it to `CMakeLists.txt`
from `HandleLLVMOptions.cmake`.
Reviewed By: compnerd
Differential Revision: https://reviews.llvm.org/D131191
- add zstd to `llvm::compression` namespace
- add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB`
- add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp`
- debian users should install libzstd when using `LLVM_ENABLE_ZSTD=FORCE_ON` from source due to this bug https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1941956
Reviewed By: leonardchan, MaskRay
Differential Revision: https://reviews.llvm.org/D128465
- add `FindZSTD.cmake`
- add zstd to `llvm::compression` namespace
- add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB`
- add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp`
Reviewed By: leonardchan, MaskRay
Differential Revision: https://reviews.llvm.org/D128465
- add `FindZSTD.cmake`
- add zstd to `llvm::compression` namespace
- add a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB`
- add tests for zstd to `llvm/unittests/Support/CompressionTest.cpp`
Reviewed By: leonardchan, MaskRay
Differential Revision: https://reviews.llvm.org/D128465
Summary:
This is an essential piece of infrastructure for us to be
continuously testing debug info with BOLT. We can't only make changes
to a test repo because we need to change debuginfo tests to call BOLT,
hence, this diff needs to sit in our opensource repo. But when upstreaming
to LLVM, this should be kept BOLT-only outside of LLVM. When upstreaming,
we need to git diff and check all folders that are being modified by our
commits and discard this one (and leave as an internal diff).
To test BOLT in debuginfo tests, configure it with -DLLVM_TEST_BOLT=ON.
Then run check-lldb and check-debuginfo.
Manual rebase conflict history:
https://phabricator.intern.facebook.com/D29205224https://phabricator.intern.facebook.com/D29564078https://phabricator.intern.facebook.com/D33289118https://phabricator.intern.facebook.com/D34957174
Test Plan:
tested locally
Configured with:
-DLLVM_ENABLE_PROJECTS="clang;lld;lldb;compiler-rt;bolt;debuginfo-tests"
-DLLVM_TEST_BOLT=ON
Ran test suite with:
ninja check-debuginfo
ninja check-lldb
Reviewers: #llvm-bolt
Subscribers: ayermolo, phabricatorlinter
Differential Revision: https://phabricator.intern.facebook.com/D35317341
Tasks: T92898286
Implements [[ https://wg21.link/p2071r1 | P2071 Named Universal Character Escapes ]] - as an extension in all language mode, the patch not warn in c++23 mode will be done later once this paper is plenary approved (in July).
We add
* A code generator that transforms `UnicodeData.txt` and `NameAliases.txt` to a space efficient data structure that can be queried in `O(NameLength)`
* A set of functions in `Unicode.h` to query that data, including
* A function to find an exact match of a given Unicode character name
* A function to perform a loose (ignoring case, space, underscore, medial hyphen) matching
* A function returning the best matching codepoint for a given string per edit distance
* Support of `\N{}` escape sequences in String and character Literals, with loose and typos diagnostics/fixits
* Support of `\N{}` as UCN with loose matching diagnostics/fixits.
Loose matching is considered an error to match closely the semantics of P2071.
The generated data contributes to 280kB of data to the binaries.
`UnicodeData.txt` and `NameAliases.txt` are not committed to the repository in this patch, and regenerating the data is a manual process.
Reviewed By: tahonermann
Differential Revision: https://reviews.llvm.org/D123064
This reverts commit 90c9d41c8a.
Keeping arm* LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on is sufficient to work around
LLVM_ENABLE_RUNTIMES not working well with arm. It is more important for
LLVM_ENABLE_PROJECTS and LLVM_ENABLE_RUNTIMES to match in the file hierarchy.
I did this before but the other change got reverted and relanded
recently.
arm vs armhf isn't handled correctly so this setting doesn't work
for 32 bit Arm Linux. Causing failures like:
https://lab.llvm.org/buildbot/#/builders/178/builds/2293
This makes the LLVM_ENABLE_PROJECTS mode (supported for compiler-rt, deprecated
(D112724) for libcxx/libcxxabi/libunwind) closer to
https://libcxx.llvm.org/BuildingLibcxx.html#bootstrapping-build .
The layout is arguably superior because different libraries of target triples
are in different directories, similar to GCC/Debian multiarch.
When LLVM_DEFAULT_TARGET_TRIPLE is x86_64-unknown-linux-gnu,
`lib/clang/15.0.0/lib/libclang_rt.asan-x86_64.a`
is moved to
`lib/clang/15.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.asan.a`.
In addition, if the host compiler supports -m32 (multilib),
`lib/clang/15.0.0/lib/libclang_rt.asan-i386.a`
is moved to
`lib/clang/15.0.0/lib/i386-unknown-linux-gnu/libclang_rt.asan.a`.
Reviewed By: mstorsjo, ldionne, #libc
Differential Revision: https://reviews.llvm.org/D107799
This patch adds an llvm-driver multicall tool that can combine multiple
LLVM-based tools. The build infrastructure is enabled for a tool by
adding the GENERATE_DRIVER option to the add_llvm_executable CMake
call, and changing the tool's main function to a canonicalized
tool_name_main format (i.e. llvm_ar_main, clang_main, etc...).
As currently implemented llvm-driver contains dsymutil, llvm-ar,
llvm-cxxfilt, llvm-objcopy, and clang (if clang is included in the
build).
llvm-driver can be enabled from builds by setting
LLVM_TOOL_LLVM_DRIVER_BUILD=On.
There are several limitations in the current implementation, which can
be addressed in subsequent patches:
(1) the multicall binary cannot currently properly handle
multi-dispatch tools. This means symlinking llvm-ranlib to llvm-driver
will not properly result in llvm-ar's main being called.
(2) the multicall binary cannot be comprised of tools containing
conflicting cl::opt options as the global cl::opt option list cannot
contain duplicates.
These limitations can be addressed in subsequent patches.
Differential revision: https://reviews.llvm.org/D109977
Nikita Popov