UNIX03 conformance requires utilities to flush stdout before exiting and raise
an error if writing fails. Flushing already happens on a call to exit
and thus automatically on a return from main. Write failure is then
detected by LLVM's default SIGPIPE handler. The handler already exits with
a non-zero code, but conformance additionally requires an error message.
First reapply attempt I hadn't noticed the test had changed, hopefully this
goes better.
UNIX03 conformance requires utilities to flush stdout before exiting and raise
an error if writing fails. Flushing already happens on a call to exit
and thus automatically on a return from main. Write failure is then
detected by LLVM's default SIGPIPE handler. The handler already exits with
a non-zero code, but conformance additionally requires an error message.
This patch adds an llvm-driver multicall tool that can combine multiple
LLVM-based tools. The build infrastructure is enabled for a tool by
adding the GENERATE_DRIVER option to the add_llvm_executable CMake
call, and changing the tool's main function to a canonicalized
tool_name_main format (i.e. llvm_ar_main, clang_main, etc...).
As currently implemented llvm-driver contains dsymutil, llvm-ar,
llvm-cxxfilt, llvm-objcopy, and clang (if clang is included in the
build).
llvm-driver can be enabled from builds by setting
LLVM_TOOL_LLVM_DRIVER_BUILD=On.
There are several limitations in the current implementation, which can
be addressed in subsequent patches:
(1) the multicall binary cannot currently properly handle
multi-dispatch tools. This means symlinking llvm-ranlib to llvm-driver
will not properly result in llvm-ar's main being called.
(2) the multicall binary cannot be comprised of tools containing
conflicting cl::opt options as the global cl::opt option list cannot
contain duplicates.
These limitations can be addressed in subsequent patches.
Differential revision: https://reviews.llvm.org/D109977
It can happen on macOS that terminal doesn't report the "colors"
capability in the terminfo database, in which case `tigetnum` returns -1.
This doesn't mean however that the terminal doesn't supports color, it
just means that the capability is absent from the terminal description.
In that case, we should still fallback to the checking the $TERM
environment variable to see if it supports ANSI escapes codes.
Differential Revision: https://reviews.llvm.org/D125914
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
On Apple Silicon Macs, using a Darwin thread priority of PRIO_DARWIN_BG seems to
map directly to the QoS class Background. With this priority, the thread is
confined to efficiency cores only, which makes background indexing take forever.
Introduce a new ThreadPriority "Low" that sits in the middle between Background
and Default, and maps to QoS class "Utility" on Mac. Make this new priority the
default for indexing. This makes the thread run on all cores, but still lowers
priority enough to keep the machine responsive, and not interfere with
user-initiated actions.
I didn't change the implementations for Windows and Linux; on these systems,
both ThreadPriority::Background and ThreadPriority::Low map to the same thread
priority. This could be changed as a followup (e.g. by using SCHED_BATCH for Low
on Linux).
See also https://github.com/clangd/clangd/issues/1119.
Reviewed By: sammccall, dgoldman
Differential Revision: https://reviews.llvm.org/D124715
UNIX03 conformance requires utilities to flush stdout before exiting and raise
an error if writing fails. Flushing already happens on a call to exit
and thus automatically on a return from main. Write failure is then
detected by LLVM's default SIGPIPE handler. The handler already exits with
a non-zero code, but conformance additionally requires an error message.
lib/Support/ThreadLocal.cpp has been uncompilable since rL158346 (2012-06) when
`data` became a char array. The error looks like
```
...llvm/lib/Support/Unix/ThreadLocal.inc:66:57: error: array type 'char[8]' is not assignable
void ThreadLocalImpl::setInstance(const void* d) { data = const_cast<void*>(d);}
```
In contrast to Linux it does not provide entries which can be readlinked
-- these are just regular files, not giving the expected outcome. That's
on top of procfs not being mounted by default to begin with.
This is probably the case on other BSDs as well, so I expect there will
be more ifdefs added down the road.
Reviewed By: emaste, dim
Differential Revision: https://reviews.llvm.org/D122545
This is a followup to D119695 using the suggestion by joerg. Rather
than manually declaring madvise() on __sun__, this uses
posix_madvise() if available, which does get declared properly on
Illumos.
Differential Revision: https://reviews.llvm.org/D119856
This reverts commit ef82063207.
- It conflicts with the existing llvm::size in STLExtras, which will now
never be called.
- Calling it without llvm:: breaks C++17 compat
Replace a `reserve()`/`set_size()` pair with `resize_for_overwrite()`
and `truncate()`. The out parameter also needs a `clear()` call on the
error path.
Differential Revision: https://reviews.llvm.org/D115389
mapped_file_region::dontNeedImpl added in D116366 calls madvise, which
causes problems for z/OS and AIX.
For z/OS, we don't have either madvise, so treat this as a no-op, same
as Windows does.
For AIX, it doesn't have any effect, doesn't have a standardized
signature, and it needs certain feature test macros (i.e. _ALL_SOURCE)
we don't set by default for LLVM on AIX, so just make it a no-op too.
Differential Revision: https://reviews.llvm.org/D116603
On *NIX systems, this API calls madvise(MADV_DONTNEED) on read-only file mappings.
It should not be used on a writable buffer.
The API is used to implement ld.lld LTO memory saving trick (D116367).
Note: on read-only file mappings, Linux's MADV_DONTNEED semantics match POSIX
POSIX_MADV_DONTNEED and BSD systems' MADV_DONTNEED.
On Windows, VirtualAllocEx MEM_COMMIT/MEM_RESET have similar semantics
but are unfortunately not drop-in replacements. dontNeedIfMmap is currently a no-op.
Reviewed By: aganea
Differential Revision: https://reviews.llvm.org/D116366
This reverts 3816c53f04 and removes follow-up
fixups.
The original intention was to show error earlier (posix_fallocate time) than
later for ld.lld but it appears to cause some problems which make it not free.
* FreeBSD ZFS: EINVAL, not too bad.
* FreeBSD UFS: according to khng "devastatingly slow on freebsd because UFS on freebsd does not have preallocation support like illumos. It zero-fills."
* NetBSD: maybe EOPNOTSUPP
* Linux tmpfs: unless tmpfs is set up to use huge pages (requires CONFIG_TRANSPARENT_HUGE_PAGECACHE=y), I can consistently demonstrate ~300ms delay for a 1.4GiB output.
* Linux ext4: I don't measure any benefit, either backed by a hard disk or by a file in tmpfs.
* The current code organization of `defined(HAVE_POSIX_FALLOCATE)` costs us a macro dispatch for AIX.
I think we should just remove it. I think if posix_fallocate ever finds demonstrable benefit,
it is likely Linux specific and will not need HAVE_POSIX_FALLOCATE, and possibly opt-in by some specific programs.
In a filesystem with CoW and compression, the ENOSPC benefit may be lost as well.
Reviewed By: khng300
Differential Revision: https://reviews.llvm.org/D115957
fs::copy_file() on Darwin has a nice optimization to clone the file when
possible. Change the implementation to use clonefile() directly, instead
of the higher-level copyfile(). The latter does the wrong thing for
symlinks, which requires calling `stat` first...
With that out of the way, optimistically call clonefile() all the time,
and then for any error that's recoverable try again with copyfile()
(without the COPYFILE_CLONE flag, as before).
Differential Revision: https://reviews.llvm.org/D112250
The mips-specific includes have been unnecessary ever since the
__clear_cache() builtin replaced cacheflush().
Differential Revision: https://reviews.llvm.org/D111486
As described on D111049, we're trying to remove the <string> dependency from error handling and replace uses of report_fatal_error(const std::string&) with the Twine() variant which can be forward declared.
Many `flang` tests currently `FAIL` on Solaris because the module files
aren't found. I could trace this to `sys::fs::getMainExecutable` not being
implemented.
This patch does this and fixes all affected `flang` tests.
Tested on `amd64-pc-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D109374
[[noreturn]] can be used since Oct 2016 when the minimum compiler requirement was bumped to GCC 4.8/MSVC 2015.
Note: the definition of LLVM_ATTRIBUTE_NORETURN is kept for now.
FreeBSD's condvar.h (included by user.h in Threading.inc) uses a "struct
thread" that conflicts with llvm::thread if both are visible when it's
included.
So this moves our #include after the FreeBSD code.
This adds a new llvm::thread class with the same interface as std::thread
except there is an extra constructor that allows us to set the new thread's
stack size. On Darwin even the default size is boosted to 8MB to match the main
thread.
It also switches all users of the older C-style `llvm_execute_on_thread` API
family over to `llvm::thread` followed by either a `detach` or `join` call and
removes the old API.
Moved definition of DefaultStackSize into the .cpp file to hopefully
fix the build on some (GCC-6?) machines.
This adds a new llvm::thread class with the same interface as std::thread
except there is an extra constructor that allows us to set the new thread's
stack size. On Darwin even the default size is boosted to 8MB to match the main
thread.
It also switches all users of the older C-style `llvm_execute_on_thread` API
family over to `llvm::thread` followed by either a `detach` or `join` call and
removes the old API.
On FreeBSD, absolute paths are passed unmodified in AT_EXECPATH, but
relative paths are resolved to absolute paths, and any symlinks will be
followed in the process. This means that the resource dir calculation
will be wrong if Clang is invoked as an absolute path to a symlink, and
this currently causes clang/test/Driver/rocm-detect.hip to fail on
FreeBSD. Thus, make sure to call realpath on the result, just like is
done on macOS.
Whilst here, clean up the old fallback auxargs loop to use the actual
type for auxargs rather than using lots of hacky casts that rely on
addresses and pointers being the same (which is not the case on CHERI,
and thus Arm's prototype Morello, although for little-endian systems it
happens to work still as the word-sized integer will be padded to a full
pointer, and it's someone academic given dereferencing past the end of
environ will give a bounds fault, but CheriBSD is new enough that the
elf_aux_info path will be used). This also makes the code easier to
follow, and removes the confusing double-increment of p.
Reviewed By: dim, arichardson
Differential Revision: https://reviews.llvm.org/D103346
If exiting using _Exit or ExitProcess, DLLs are still unloaded
cleanly before exiting, running destructors and other cleanup in those
DLLs. When the caller expects to exit without cleanup, running
destructors in some loaded DLLs (which can be either libLLVM.dll or
e.g. libc++.dll) can cause deadlocks occasionally.
This is an alternative to D102684.
Differential Revision: https://reviews.llvm.org/D102944
This patch adds the basic functions needed for controlling auto conversion on z/OS.
Auto conversion is enabled on untagged input file to ASCII by making the assumption that all untagged files are EBCDIC encoded. Output files are auto converted to EBCDIC IBM-1047.
This change also enables conversion for stdin/stdout/stderr.
For more information on how fcntl controls codepage https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-fcntl-bpx1fct-bpx4fct-control-open-file-descriptors
Reviewed By: anirudhp
Differential Revision: https://reviews.llvm.org/D100483
Note to BuryPointer.cpp:GraveYard. 'unused' cannot prevent (1) dead store
elimination and (2) removal of the global pointer variable (D69428) but 'used' can.
Discovered when comparing link maps between HEAD+D69428 and HEAD.
Reviewed By: lattner
Differential Revision: https://reviews.llvm.org/D101217
On Windows, we want to open a file in Binary mode if OF_CRLF bit is not set. On z/OS, we want to open a file in Binary mode if the OF_Text bit is not set.
This patch creates two new functions called ChangeStdinMode and ChangeStdoutMode which will take OpenFlags as an arg to determine which mode to set stdin and stdout to. This will enable patches like https://reviews.llvm.org/D100056 to not affect Windows when setting the OF_Text flag for raw_fd_streams.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D100130
Update llvm::sys::fs::mapped_file_region to have a move constructor and
a move assignment operator, allowing it to be used as an Optional. Also,
update FileOutputBuffer's OnDiskBuffer to take advantage of this,
avoiding an extra allocation from the unique_ptr.
A nice follow-up would be to make the mapped_file_region constructor
private and replace its use with a factory function, such as
mapped_file_region::create(), that returns an Expected (or ErrorOr). I
don't plan on doing that immediately, but I might swing back later.
No functionality change, besides the saved allocation in OnDiskBuffer.
Differential Revision: https://reviews.llvm.org/D100159
This allows mapping larger files, delaying OOM failures until too many
pages of them are accessed. This is makes the behavior of the
mapped_file_region in this regard consistent between its "Unix" and
"Windows" implementations.
Guard the code witih #if defined(MAP_NORESERVE), consistent with other
uses of MAP_NORESERVE in llvm-project, because some FreeBSD versions do
not provide this flag.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D96626
Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.
Solution:
This patch adds two new flags
- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.
Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.
So this is the behaviour per platform with my patch:
z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode
Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return
The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```
These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D99426
Valgrind is reporting this bogus warning because it doesn't model
pthread_sigmask fully accurately. This is a valgrind bug, but
silencing it has effectively no cost, so just do it.
==73662== Syscall param __pthread_sigmask(set) points to uninitialised byte(s)
==73662== at 0x101E9D4C2: __pthread_sigmask (in /usr/lib/system/libsystem_kernel.dylib)
==73662== by 0x101EFB5EA: pthread_sigmask (in /usr/lib/system/libsystem_pthread.dylib)
==73662== by 0x1000D9F6D: llvm::sys::Process::SafelyCloseFileDescriptor(int) (in /Users/chrisl/Projects/circt/build/bin/firtool)
==73662== by 0x100072795: llvm::ErrorOr<std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer> > > getFileAux<llvm::MemoryBuffer>(llvm::Twine const&, long long, unsigned long long, unsigned long long, bool, bool) (in /Users/chrisl/Projects/circt/build/bin/firtool)
==73662== by 0x100072573: llvm::MemoryBuffer::getFileOrSTDIN(llvm::Twine const&, long long, bool) (in /Users/chrisl/Projects/circt/build/bin/firtool)
==73662== by 0x100282C25: mlir::openInputFile(llvm::StringRef, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) (in /Users/chrisl/Projects/circt/build/bin
Differential Revision: https://reviews.llvm.org/D98830
The call to "set_curterm" inside the "terminalHasColors" function breaks
the EditLine configuration on some Linux distributions, causing certain
characters that have functions bound to them to not show up and
backspace to stop deleting characters (only visually). This patch
ensures that term struct is restored after the routine for cheking if
terminal supports colors is done, which fixes the aforementioned issue.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D95230
As of binutils 2.36, GNU strip calls chown(2) for "sudo strip foo" and
"sudo strip foo -o foo", but no "sudo strip foo -o bar" or "sudo strip
foo -o ./foo". In other words, while "sudo strip foo -o bar" creates a
new file bar with root access, "sudo strip foo" will keep the owner and
group of foo unchanged. Currently llvm-objcopy and llvm-strip behave
differently, always changing the owner and gropu to root. The
discrepancy prevents Chrome OS from migrating to llvm-objcopy and
llvm-strip as they change file ownership and cause intended users/groups
to lose access when invoked by sudo with the following sequence
(recommended in man page of GNU strip).
1.<Link the executable as normal.>
1.<Copy "foo" to "foo.full">
1.<Run "strip --strip-debug foo">
1.<Run "objcopy --add-gnu-debuglink=foo.full foo">
This patch makes llvm-objcopy and llvm-strip follow GNU's behavior.
Link: crbug.com/1108880
glibc deprecates `mallinfo` in the latest version of 2.33. This patch replaces the usage of `mallinfo` with the new `mallinfo2` when it's available.
Reviewed By: lattner
Differential Revision: https://reviews.llvm.org/D96359
This change fixes two issues with building LLVM on Haiku. The first issue is
that LLVM requires wait4(), which on Haiku is hidden behind the _BSD_SOURCE
feature flag when using the --std=c++14 flag. Additionally, the wait4()
function is only available in libbsd.so, so this is now a dependency.
The other fix is that Haiku does not have the (non-standard) rusage.maxrss
member, so by default the used memory info will be set to 0 on this platform.
Reviewed By: sepavloff
Differential Revision: https://reviews.llvm.org/D87920
Patch by Niels Sascha Reedijk.
The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:
> start /B /AFFINITY 0xF lld-link.exe ...
Would let LLD only use 4 hyper-threads.
Previously, there was an outstanding issue on Windows Server 2019 on dual-CPU machines, which was preventing from using both CPU sockets. In normal conditions, when no affinity mask was set, ProcessorGroup::AllThreads was different from ProcessorGroup::UsableThreads. The previous code in llvm/lib/Support/Windows/Threading.inc L201 was improperly assuming those two values to be equal, and consequently was limiting the execution to only one CPU socket.
Differential Revision: https://reviews.llvm.org/D92419
Tim Northover