Commit Graph

260 Commits

Author SHA1 Message Date
Fangrui Song d9b948b6eb Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC
F_{None,Text,Append} are kept for compatibility since r334221.

llvm-svn: 367800
2019-08-05 05:43:48 +00:00
Yi Kong 1755abe1fb Fix macOS build after r358716
COPYFILE_CLONE is only defined on newer macOS versions, using it without
check breaks build on systems running legacy OS and toolchain.

Differential Revision: https://reviews.llvm.org/D65317

llvm-svn: 367084
2019-07-26 05:17:14 +00:00
Jordan Rose 887d31ccee FileSystem: Check for DTTOIF alone, not _DIRENT_HAVE_D_TYPE
While 'd_type' is a non-standard extension to `struct dirent`, only
glibc signals its presence with a macro '_DIRENT_HAVE_D_TYPE'.
However, any platform with 'd_type' also includes a way to convert to
mode_t values using the macro 'DTTOIF', so we can check for that alone
and still be confident that the 'd_type' member exists.

(If this turns out to be wrong, I'll go back and set up an actual
CMake check.)

I couldn't think of how to write a test for this, because I couldn't
think of how to test that a 'stat' call doesn't happen without
controlling the filesystem or intercepting 'stat', and there's no good
cross-platform way to do that that I know of.

Follow-up (almost a year later) to r342089.

rdar://problem/50592673
https://reviews.llvm.org/D64940

llvm-svn: 366486
2019-07-18 20:05:11 +00:00
Fangrui Song 6dc5962957 [llvm-objcopy] Don't change permissions of non-regular output files
There is currently an EPERM error when a regular user executes `llvm-objcopy a.o /dev/null`.
Worse, root can even change the mode bits of /dev/null.

Fix it by checking if the output file is special.

A new overload of llvm::sys::fs::setPermissions with FD as the parameter
is added. Users should provide `perm & ~umask` as the parameter if they
intend to respect umask.

The existing overload of llvm::sys::fs::setPermissions may be deleted if
we can find an implementation of fchmod() on Windows. fchmod() is
usually better than chmod() because it saves syscalls and can avoid race
condition.

Reviewed By: jakehehrlich, jhenderson

Differential Revision: https://reviews.llvm.org/D64236

llvm-svn: 365753
2019-07-11 10:17:59 +00:00
Reid Kleckner cc418a3af4 [Support] Move llvm::MemoryBuffer to sys::fs::file_t
Summary:
On Windows, Posix integer file descriptors are a compatibility layer
over native file handles provided by the C runtime. There is a hard
limit on the maximum number of file descriptors that a process can open,
and the limit is 8192. LLD typically doesn't run into this limit because
it opens input files, maps them into memory, and then immediately closes
the file descriptor. This prevents it from running out of FDs.

For various reasons, I'd like to open handles to every input file and
keep them open during linking. That requires migrating MemoryBuffer over
to taking open native file handles instead of integer FDs.

Reviewers: aganea, Bigcheese

Reviewed By: aganea

Subscribers: smeenai, silvas, mehdi_amini, hiraditya, steven_wu, dexonsmith, dang, llvm-commits, zturner

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63453

llvm-svn: 365588
2019-07-10 00:34:13 +00:00
Sam McCall edf904efff getMainExecutable: handle realpath() failure, falling back to getprogpath().
Summary:
Previously, we'd pass a nullptr to std::string and crash().

This case happens when the binary is deleted while being used (e.g. rebuilding clangd).

Reviewers: kadircet

Subscribers: ilya-biryukov, kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64068

llvm-svn: 364936
2019-07-02 15:42:37 +00:00
Alex Brachet 3b715d67dd [Support] Add fs::getUmask() function and change fs::setPermissions
Summary: This patch changes fs::setPermissions to optionally set permissions while respecting the umask. It also adds the function fs::getUmask() which returns the current umask.

Reviewers: jhenderson, rupprecht, aprantl, lhames

Reviewed By: jhenderson, rupprecht

Subscribers: sanaanajjar231288, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D63583

llvm-svn: 364621
2019-06-28 03:21:00 +00:00
Keno Fischer 5f4ae7c457 [Support] Fix build under Emscripten
Summary:
Emscripten's libc doesn't define MNT_LOCAL, thus causing a build
failure in the fallback path. However, to the best of my knowledge,
it also doesn't support remote file system mounts, so we may simply
return `true` here (as we do for e.g. Fuchsia). With this fix, the
core LLVM libraries build correctly under emscripten (though some
of the tools and utils do not).

Reviewers: kripken
Differential Revision: https://reviews.llvm.org/D63688

llvm-svn: 364143
2019-06-23 00:29:59 +00:00
Xing Xue 2dee094a08 Fixes for builds that require strict X/Open and POSIX compatiblity
Summary:
- Use alternative to MAP_ANONYMOUS for allocating mapped memory if it isn't available
- Use strtok_r instead of strsep as part of getting program path
- Don't try to find the width of a terminal using "struct winsize" and TIOCGWINSZ on POSIX builds. These aren't defined under POSIX (even though some platforms make them available when they shouldn't), so just check if we are doing a X/Open or POSIX compliant build first.

Author: daltenty

Reviewers: hubert.reinterpretcast, xingxue, andusy

Reviewed By: hubert.reinterpretcast

Subscribers: MaskRay, jsji, hiraditya, kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61326

llvm-svn: 360898
2019-05-16 14:02:13 +00:00
Lang Hames e4b4ab6d26 [Support] Add error handling to sys::Process::getPageSize().
This patch changes the return type of sys::Process::getPageSize to
Expected<unsigned> to account for the fact that the underlying syscalls used to
obtain the page size may fail (see below).

For clients who use the page size as an optimization only this patch adds a new
method, getPageSizeEstimate, which calls through to getPageSize but discards
any error returned and substitues a "reasonable" page size estimate estimate
instead. All existing LLVM clients are updated to call getPageSizeEstimate
rather than getPageSize.

On Unix, sys::Process::getPageSize is implemented in terms of getpagesize or
sysconf, depending on which macros are set. The sysconf call is documented to
return -1 on failure. On Darwin getpagesize is implemented in terms of sysconf
and may also fail (though the manpage documentation does not mention this).
These failures have been observed in practice when highly restrictive sandbox
permissions have been applied. Without this patch, the result is that
getPageSize returns -1, which wreaks havoc on any subsequent code that was
assuming a sane page size value.

<rdar://problem/41654857>

Reviewers: dblaikie, echristo

Subscribers: kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59107

llvm-svn: 360221
2019-05-08 02:11:07 +00:00
Adrian Prantl ccdefb24ad Guard __builtin_available() with __has_builtin to support older host compilers.
llvm-svn: 360174
2019-05-07 17:10:27 +00:00
David Chisnall 714a4425de Try to use /proc on FreeBSD for getExecutablePath
Currently, clang's libTooling passes this function a fake argv0, which
means that no libTooling tools can find the standard headers on FreeBSD.
With this change, these will now work on any FreeBSD systems that have
procfs mounted.  This isn't the right fix for the libTooling issue, but
it does bring the FreeBSD implementation of getExecutablePath closer to
the Linux and macOS implementations.

llvm-svn: 359427
2019-04-29 09:24:51 +00:00
Adrian Prantl c90ff5e123 Revert using fcopyfile(3) to implement sys::fs::copy_file(Twine, int) on macOS
It turns out that I mesread the man page and fcopyfile(3) does not
actually support COPYFILE_CLONE for files.

<rdar://problem/50148757>

llvm-svn: 359127
2019-04-24 19:08:43 +00:00
Adrian Prantl fac7875704 Implement sys::fs::copy_file using the macOS copyfile(3) API
to support APFS clones.

This patch adds a Darwin-specific implementation of
llvm::sys::fs::copy_file() that uses the macOS copyfile(3) API to
support APFS copy-on-write clones, which should be faster and much
more space efficient.

https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/ToolsandAPIs/ToolsandAPIs.html

Differential Revision: https://reviews.llvm.org/D60802

This reapplies 358628 with an additional bugfix handling the case
where the destination file already exists. (Caught by the clang testsuite).

llvm-svn: 358716
2019-04-18 21:22:50 +00:00
Adrian Prantl 00d97ea202 Revert Implement sys::fs::copy_file using the macOS copyfile(3) API to support APFS clones.
This reverts r358628 (git commit 91a06bee78)
while investigating a crash reproducer bot failure.

llvm-svn: 358634
2019-04-18 01:21:10 +00:00
Adrian Prantl 91a06bee78 Implement sys::fs::copy_file using the macOS copyfile(3) API
to support APFS clones.

This patch adds a Darwin-specific implementation of
llvm::sys::fs::copy_file() that uses the macOS copyfile(3) API to
support APFS copy-on-write clones, which should be faster and much
more space efficient.

https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/ToolsandAPIs/ToolsandAPIs.html

Differential Revision: https://reviews.llvm.org/D60802

llvm-svn: 358628
2019-04-18 00:01:05 +00:00
Hubert Tong 7f8b3bf247 [Support] On AIX, Check ENOTSUP on posix_fallocate instead of EOPNOTSUPP
Summary:
`posix_fallocate` can fail if the underlying filesystem does not support
it; and, on AIX, such a failure is reported by a return value of
`ENOTSUP`. The existing code checks only for `EOPNOTSUPP`, which may
share the same value as `ENOTSUP`, but is not required to.

Reviewers: xingxue, sfertile, jasonliu

Reviewed By: xingxue

Subscribers: kristina, jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60175

llvm-svn: 357662
2019-04-04 00:40:34 +00:00
Hubert Tong 24168852e8 [Support] Implement is_local_impl with AIX mntctl
Summary:
On AIX, we can determine whether a filesystem is remote using `mntctl`.

If the information is not found, then claim that the file is remote
(since that is the more restrictive case). Testing for the associated
interface is restored with a modified version of the unit test from
rL295768.

Reviewers: jasonliu, xingxue

Reviewed By: xingxue

Subscribers: jsji, apaprocki, Hahnfeld, zturner, krytarowski, kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58801

llvm-svn: 357333
2019-03-29 23:32:47 +00:00
Jonas Hahnfeld e59746f8f8 [Support] Treat truncation of fullpath as error
If the concatenation of arguments dir and bin has at least PATH_MAX
characters the call to snprintf will truncate. The result will usually
not exist, but if it does it's actually incorrect to return that the
path exists.
(Motivated by GCC compiler warning about format truncation.)

Differential Revision: https://reviews.llvm.org/D58835

llvm-svn: 356036
2019-03-13 10:37:56 +00:00
Michal Gorny 2d8be64401 [llvm] [Support] Revert "Reimplement getMainExecutable() using sysctl on NetBSD"
This apparently does not work reliably after all (non-reentrant?)
and causes test failures such as:

http://lab.llvm.org:8011/builders/netbsd-amd64/builds/19254/steps/run%20unit%20tests/logs/FAIL%3A%20libc%2B%2B%3A%3Asize.pass.cpp

llvm-svn: 355302
2019-03-04 04:53:50 +00:00
Michal Gorny 8f04766d13 [llvm] [Support] Reimplement getMainExecutable() using sysctl on NetBSD
Use sysctl() to implement getMainExecutable() on NetBSD, rather than
trying to guess the correct path from argv[0].  This is one
of the fixes to recent clang-check-mac-libcxx-fixed-compilation-db.cpp
test failure on NetBSD.

This has been historically done on both FreeBSD and NetBSD in r303015,
and reverted in r303285 due to buggy implementation on FreeBSD.
However, FWIK the NetBSD implementation does not suffer from the same
bugs and is more reliable than playing with argv[0].

Differential Revision: https://reviews.llvm.org/D56975

llvm-svn: 355283
2019-03-03 10:06:40 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Eli Friedman f457470286 [Support] Fix GNU/kFreeBSD build
Patch by James Clarke.

Differential Revision: https://reviews.llvm.org/D55296

llvm-svn: 349434
2018-12-18 01:38:20 +00:00
Argyrios Kyrtzidis 5167c1389b [Support/FileSystem] Add sub-second precision for atime/mtime of sys::fs::file_status on unix platforms
Summary:
getLastAccessedTime() and getLastModificationTime() provided times in nanoseconds but with only 1 second resolution, even when the underlying file system could provide more precise times than that.
These changes add sub-second precision for unix platforms that support improved precision.

Also add some comments to make sure people are aware that the resolution of times can vary across different file systems.

Reviewers: labath, zturner, aaron.ballman, kristina

Reviewed By: aaron.ballman, kristina

Subscribers: lebedev.ri, mgorny, kristina, llvm-commits

Differential Revision: https://reviews.llvm.org/D54826

llvm-svn: 347530
2018-11-26 00:03:39 +00:00
Jonas Devlieghere b23f430ec9 [FileSystem] Add expand_tilde function
In D54435 there was some discussion about the expand_tilde flag for
real_path that I wanted to expose through the VFS. The consensus is that
these two things should be separate functions. Since we already have the
code for this I went ahead and added a function expand_tilde that does
just that.

Differential revision: https://reviews.llvm.org/D54448

llvm-svn: 346776
2018-11-13 18:23:32 +00:00
David Carlier b79eee487d Fix DragonFlyBSD build
Reviewers: rnk, thakis

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D54363

llvm-svn: 346577
2018-11-10 01:01:03 +00:00
Reid Kleckner 5fa1e35bcc Commit missing comment edit and use correct cast to fix std::min overload
llvm-svn: 345105
2018-10-23 23:44:44 +00:00
Reid Kleckner 1500effacd [hurd] Make getMainExecutable get the real binary path
On GNU/Hurd, llvm-config is returning bogus value, such as:

$ llvm-config-6.0 --includedir
/usr/include

while it should be:
$ llvm-config-6.0 --includedir
/usr/lib/llvm-6.0/include

This is because getMainExecutable does not get the actual installation
path. On GNU/Hurd, /proc/self/exe is indeed a symlink to the path that
was used to start the program, and not the eventual binary file. Llvm's
getMainExecutable thus needs to run realpath over it to get the actual
place where llvm was installed (/usr/lib/llvm-6.0/bin/llvm-config), and
not /usr/bin/llvm-config-6.0. This will not change the result on Linux,
where /proc/self/exe already points to the eventual file.

Patch by Samuel Thibault!

While making changes here, I reformatted this block a bit to reduce
indentation and match 2 space indent style.

Differential Revision: https://reviews.llvm.org/D53557

llvm-svn: 345104
2018-10-23 23:35:43 +00:00
Sylvestre Ledru f4719c4d2c Add support for GNU Hurd in Path.inc and other places
Summary: Patch by Svante Signell & myself

Reviewers: rnk, JDevlieghere, efriedma

Reviewed By: efriedma

Subscribers: efriedma, JDevlieghere, krytarowski, llvm-commits, kristina

Differential Revision: https://reviews.llvm.org/D53409

llvm-svn: 345007
2018-10-23 07:13:47 +00:00
Nico Weber d4ed32c526 Remove dead function user_cache_directory()
It's been unused since it was added almost 3 years ago in
https://reviews.llvm.org/D13801

Motivated by https://reviews.llvm.org/rL342002 since it removes one of the
functions keeping a ref to SHGetKnownFolderPath.

Differential Revision: https://reviews.llvm.org/D52184

llvm-svn: 342485
2018-09-18 15:06:16 +00:00
Kristina Brooks 3a55d1ef27 [Support] sys::fs::directory_entry includes the file_type.
This is available on most platforms (Linux/Mac/Win/BSD) with no extra syscalls.
On other platforms (e.g. Solaris) we stat() if this information is requested.

This will allow switching clang's VFS to efficiently expose (path, type) when
traversing a directory. Currently it exposes an entire Status, but does so by
calling fs::status() on all platforms.
Almost all callers only need the path, and all callers only need (path, type).

Patch by sammccall (Sam McCall)

Differential Revision: https://reviews.llvm.org/D51918

llvm-svn: 342089
2018-09-12 22:08:10 +00:00
Hans Wennborg 0a0e411204 Use a lambda for calls to ::open in RetryAfterSignal
In Bionic, open can be overloaded for _FORTIFY_SOURCE support, causing
compile errors of RetryAfterSignal due to overload resolution. Wrapping
the call in a lambda avoids this.

Based on a patch by Chih-Wei Huang <cwhuang@linux.org.tw>!

llvm-svn: 340751
2018-08-27 15:55:39 +00:00
Jordan Rupprecht 97ea485041 [Support] NFC: Allow modifying access/modification times independently in sys::fs::setLastModificationAndAccessTime.
Summary:
Add an overload to sys::fs::setLastModificationAndAccessTime that allows setting last access and modification times separately. This will allow tools to use this API when they want to preserve both the access and modification times from an input file, which may be different.

Also note that both the POSIX (futimens/futimes) and Windows (SetFileTime) APIs take the two timestamps in the order of (1) access (2) modification time, so this renames the method to "setLastAccessAndModificationTime" to make it clear which timestamp is which.

For existing callers, the 1-arg overload just sets both timestamps to the same thing.

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D50521

llvm-svn: 339628
2018-08-13 23:03:45 +00:00
Tim Northover 43d64b0b36 [Support] Build fix for Haiku when checking for a local filesystem
Haiku does not expose information about local versus remote mounts, so just
return false, like Cygwin.

Patch by Niels Sascha Reedijk.

llvm-svn: 337389
2018-07-18 13:42:18 +00:00
Peter Collingbourne 881ba10465 LTO: Keep file handles open for memory mapped files.
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).

We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.

One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether.  Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.

A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.

Differential Revision: https://reviews.llvm.org/D48051

llvm-svn: 334630
2018-06-13 18:03:14 +00:00
Pavel Labath b2c73c4cc0 Fix build errors on some configurations
It's been reported
<http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20180611/559616.html>
that template argument deduction for RetryAfterSignal fails if open is
not prefixed with "::".

This should help us build correctly on those platforms and explicitly
specifying the namespace is more correct anyway.

llvm-svn: 334403
2018-06-11 13:30:47 +00:00
Fangrui Song 69d6418d60 Cleanup. NFC
llvm-svn: 334357
2018-06-10 04:53:14 +00:00
Zachary Turner 6edfecb883 Add a file open flag that disables O_CLOEXEC.
O_CLOEXEC is the right default, but occasionally you don't
want this.  This is especially true for tools like debuggers
where you might need to spawn the child process with specific
files already open, but it's occasionally useful in other
scenarios as well, like when you want to do some IPC between
parent and child.

llvm-svn: 334293
2018-06-08 15:15:56 +00:00
Zachary Turner 9d2cfa6ccc Expose a single global file open function.
This one allows much more flexibility than the standard
openFileForRead / openFileForWrite functions.  Since there is now
just one "real" function that does the work, all other implementations
simply delegate to this one.

llvm-svn: 334246
2018-06-07 23:25:13 +00:00
Zachary Turner 1f67a3cba9 [FileSystem] Split up the OpenFlags enumeration.
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition.  The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum.  The second controls more flags-like values.

This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before.  This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.

llvm-svn: 334221
2018-06-07 19:58:58 +00:00
Zachary Turner 63db25ba0d [Support] Add functions that operate on native file handles on Windows.
Windows' CRT has a limit of 512 open file descriptors, and fds which are
generated by converting a HANDLE via _get_osfhandle count towards this
limit as well.

Regardless, often you find yourself marshalling back and forth between
native HANDLE objects and fds anyway. If we know from the getgo that
we're going to need to work directly with the handle, we can cut out the
marshalling layer while also not contributing to filling up the CRT's
very limited handle table.

On Unix these functions just delegate directly to the existing set of
functions since an fd *is* the native file type. It would be nice, very
long term, if we could convert most uses of fds to file_t.

Differential Revision: https://reviews.llvm.org/D47688

llvm-svn: 333945
2018-06-04 19:38:11 +00:00
Petr Hosek 87f1343a73 [Support] Support building LLVM for Fuchsia
These are necessary changes to support building LLVM for Fuchsia.
While these are not sufficient to run on Fuchsia, they are still
useful when cross-compiling LLVM libraries and runtimes for Fuchsia.

Differential Revision: https://reviews.llvm.org/D46345

llvm-svn: 331423
2018-05-03 01:38:49 +00:00
Nico Weber f3db8e3c70 Remove HAVE_DIRENT_H.
The autoconf manual: "This macro is obsolescent, as all current systems with
directory libraries have <dirent.h>. New programs need not use this macro."

llvm-svn: 328989
2018-04-02 17:17:29 +00:00
Zachary Turner adad33011f [Support] Add WriteThroughMemoryBuffer.
This is like MemoryBuffer (read-only) and WritableMemoryBuffer
(writable private), but where the underlying file can be modified
after writing.  This is useful when you want to open a file, make
some targeted edits, and then write it back out.

Differential Revision: https://reviews.llvm.org/D44230

llvm-svn: 327057
2018-03-08 20:34:47 +00:00
Zachary Turner 2061ad2f83 Silence warning about unused private variable.
llvm-svn: 325275
2018-02-15 18:46:59 +00:00
Davide Italiano 4762c069de [Support] Use realpath(3) instead of trying to open a file.
If we don't have read permissions on the directory the call would
fail.

<rdar://problem/35871293>

llvm-svn: 322095
2018-01-09 17:27:45 +00:00
Davide Italiano 1f465aa64a [Support/UNIX] posix_fallocate() can fail with EINVAL.
According to the docs on opegroup.org, the function can return
EINVAL if:

The len argument is less than zero, or the offset argument is less
than zero, or the underlying file system does not support this
operation.

I'd say it's a peculiar choice (when EONOTSUPP is right there), but
let's keep POSIX happy for now. This was independently discovered
by Mark Millard (on FreeBSD/ZFS).

Quickly ack'ed by Rui on IRC.

llvm-svn: 317535
2017-11-07 00:47:04 +00:00
Peter Collingbourne 0dfdb44797 Support: Have directory_iterator::status() return FindFirstFileEx/FindNextFile results on Windows.
This allows clients to avoid an unnecessary fs::status() call on each
directory entry. Because the information returned by FindFirstFileEx
is a subset of the information returned by a regular status() call,
I needed to extract a base class from file_status that contains only
that information.

On my machine, this reduces the time required to enumerate a ThinLTO
cache directory containing 520k files from almost 4 minutes to less
than 2 seconds.

Differential Revision: https://reviews.llvm.org/D38716

llvm-svn: 315378
2017-10-10 22:19:46 +00:00
Roman Lebedev 21b013ebc1 [Support] mapped_file_region::size() returns size_t
Fixup last commit, found by clang-stage1-cmake-RA-incremental bot.

llvm-svn: 314313
2017-09-27 16:08:33 +00:00
Roman Lebedev 7c983671f2 [Support] mapped_file_region: store size as size_t
Summary:
Found when testing stage-2 build with D38101.

```
In file included from /build/llvm/lib/Support/Path.cpp:1045:
/build/llvm/lib/Support/Unix/Path.inc:648:14: error: comparison 'uint64_t' (aka 'unsigned long') > 18446744073709551615 is always false [-Werror,-Wtautological-constant-compare]
  if (length > std::numeric_limits<size_t>::max()) {
      ~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

`size_t` is `uint64_t` here, apparently, thus any `uint64_t` value
always fits into `size_t`.

Initial patch was to use some preprocessor logic to
not check if the size is known to fit at compile time.
But Zachary Turner suggested using this approach.

Reviewers: Bigcheese, rafael, zturner, mehdi_amini

Reviewed by (via email): zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38132

llvm-svn: 314312
2017-09-27 15:59:16 +00:00
Reid Kleckner af3e93ac93 [Support] Remove getPathFromOpenFD, it was unused
Summary:
It was added to support clang warnings about includes with case
mismatches, but it ended up not being necessary.

Reviewers: twoh, rafael

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D36328

llvm-svn: 310078
2017-08-04 17:43:49 +00:00
Erich Keane d8f61f8f7e Remove Bitrig: LLVM Changes
Bitrig code has been merged back to OpenBSD, thus the OS has been abandoned.

Differential Revision: https://reviews.llvm.org/D35707

llvm-svn: 308799
2017-07-21 22:48:47 +00:00
Pavel Labath fe09f506b6 Recommit "[Support] Add RetryAfterSignal helper function"
The difference from the previous version is the use of decltype, as the
implementation of std::result_of in libc++ did not work correctly for
variadic function like open(2).

Original summary:
This function retries an operation if it was interrupted by a signal
(failed with EINTR). It's inspired by the TEMP_FAILURE_RETRY macro in
glibc, but I've turned that into a template function. I've also added a
fail-value argument, to enable the function to be used with e.g.
fopen(3), which is documented to fail for any reason that open(2) can
fail (which includes EINTR).

The main user of this function will be lldb, but there were also a
couple of uses within llvm that I could simplify using this function.

Reviewers: zturner, silvas, joerg

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D33895

llvm-svn: 306671
2017-06-29 13:15:31 +00:00
Pavel Labath efd57a8aec Revert "[Support] Add RetryAfterSignal helper function" and subsequent fix
The fix in r306003 uncovered a pretty fundamental problem that libc++
implementation of std::result_of does not handle the prototype of
open(2) correctly (presumably because it contains ...). This makes the
whole function unusable in its current form, so I am also reverting the
original commit (r305892), which introduced the function, at least until
I figure out a way to solve the libc++ issue.

llvm-svn: 306005
2017-06-22 14:18:55 +00:00
Pavel Labath 1f6aea2eb3 [Support] Add RetryAfterSignal helper function
Summary:
This function retries an operation if it was interrupted by a signal
(failed with EINTR). It's inspired by the TEMP_FAILURE_RETRY macro in
glibc, but I've turned that into a template function. I've also added a
fail-value argument, to enable the function to be used with e.g.
fopen(3), which is documented to fail for any reason that open(2) can
fail (which includes EINTR).

The main user of this function will be lldb, but there were also a
couple of uses within llvm that I could simplify using this function.

Reviewers: zturner, silvas, joerg

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D33895

llvm-svn: 305892
2017-06-21 10:55:34 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Kamil Rytarowski 07c81b1856 [Solaris] Fix PR33228 - llvm::sys::fs::is_local_impl done right
Summary:
Solaris-specific implementation for llvm::sys::fs::is_local_impl.
FStype pattern matching might be a bit unreliable, but at least it fixes the build failure.



Reviewers: mgorny, nlopes, llvm-commits, krytarowski

Reviewed By: krytarowski

Subscribers: voskresensky.vladimir, krytarowski

Differential Revision: https://reviews.llvm.org/D33695

llvm-svn: 304412
2017-06-01 12:57:00 +00:00
Dimitry Andric ebc8779301 Revert r303015, because it has the unintended side effect of breaking
driver-mode recognition in clang (this is because the sysctl method
always returns one and only one executable path, even for an executable
with multiple links):

Fix DynamicLibraryTest.cpp on FreeBSD and NetBSD

Summary:

After rL301562, on FreeBSD the DynamicLibrary unittests fail, because
the test uses getMainExecutable("DynamicLibraryTests", Ptr), and since
the path does not contain any slashes, retrieving the main executable
will not work.

Reimplement getMainExecutable() for FreeBSD and NetBSD using sysctl(3),
which is more reliable than fiddling with relative or absolute paths.

Also add retrieval of the original argv[] from the GoogleTest framework,
to use as a fallback for other OSes.

Reviewers: emaste, marsupial, hans, krytarowski

Reviewed By: krytarowski

Subscribers: krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D33171

llvm-svn: 303285
2017-05-17 19:33:10 +00:00
Dimitry Andric 4043373e84 Fix DynamicLibraryTest.cpp on FreeBSD and NetBSD
Summary:

After rL301562, on FreeBSD the DynamicLibrary unittests fail, because
the test uses getMainExecutable("DynamicLibraryTests", Ptr), and since
the path does not contain any slashes, retrieving the main executable
will not work.

Reimplement getMainExecutable() for FreeBSD and NetBSD using sysctl(3),
which is more reliable than fiddling with relative or absolute paths.

Also add retrieval of the original argv[] from the GoogleTest framework,
to use as a fallback for other OSes.

Reviewers: emaste, marsupial, hans, krytarowski

Reviewed By: krytarowski

Subscribers: krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D33171

llvm-svn: 303015
2017-05-14 18:35:38 +00:00
Joerg Sonnenberger ecfb876eac If posix_fallocate returns EOPNOTSUPP, fallback to ftruncate.
This can happen at least on NetBSD.

llvm-svn: 302263
2017-05-05 17:55:58 +00:00
Zachary Turner a3cf70bfa0 Make home_directory look in the password database in addition to $HOME.
This is something of an edge case, but when the $HOME environment
variable is not set, we can still look in the password database
to get the current user's home directory.

Added a test for this by getting the value of $HOME, then unsetting
it, then calling home_directory() and verifying that it succeeds
and that the value is the same as what we originally read from
the environment.

llvm-svn: 298513
2017-03-22 15:24:59 +00:00
Zachary Turner 5821a3bf36 [Support] Fill the file_status struct with link count.
Differential Revision: https://reviews.llvm.org/D31110

llvm-svn: 298326
2017-03-20 23:55:20 +00:00
Zachary Turner 1875334f32 Fix linux build.
llvm-svn: 298007
2017-03-16 22:34:18 +00:00
James Henderson 566fdf4a2a [Support] Add support for getting file system permissions on Windows and implement sys::fs::set/getPermissions to work with them
This change adds support for functions to set and get file permissions, in a similar manner to the C++17 permissions() function in <filesystem>. The setter uses chmod on Unix systems and SetFileAttributes on Windows, setting the permissions as passed in. The getter simply uses the existing status() function.

Prior to this change, status() would always return an unknown value for the permissions on a Windows file, making it impossible to test the new function on Windows. I have therefore added support for this as well. On Linux, prior to this change, the permissions included the file type, which should actually be accessed via a different member of the file_status class.

Note that on Windows, only the *_write permission bits have any affect - if any are set, the file is writable, and if not, the file is read-only. This is in common with what MSDN describes for their behaviour of std::filesystem::permissions(), and also what boost::filesystem does.

The motivation behind this change is so that we can easily test behaviour on read-only files in LLVM unit tests, but I am sure that others may find it useful in some situations.

Reviewers: zturner, amccarth, aaron.ballman

Differential Revision: https://reviews.llvm.org/D30736

llvm-svn: 297945
2017-03-16 11:22:09 +00:00
Aaron Ballman 345012dfa0 Reverting r297617 because it broke some bots:
http://bb.pgr.jp/builders/cmake-llvm-x86_64-linux/builds/49970

llvm-svn: 297618
2017-03-13 12:24:51 +00:00
Aaron Ballman f5cba91591 Add support for getting file system permissions and implement sys::fs::permissions to set them.
Patch by James Henderson.

llvm-svn: 297617
2017-03-13 12:17:14 +00:00
Zachary Turner e48ace6a65 Add llvm::sys::fs::real_path.
LLVM already has real_path like functionality, but it is
cumbersome to use and involves clean up after (e.g. you have
to call openFileForRead, then close the resulting FD).

Furthermore, on Windows it doesn't work for directories since
opening a directory and opening a file require slightly
different flags.

So I add a simple function `real_path` which works for all
paths on all platforms and has a simple to use interface.

In doing so, I add the ability to opt in to resolving tilde
expressions (e.g. ~/foo), which are normally handled by
the shell.

Differential Revision: https://reviews.llvm.org/D30668

llvm-svn: 297483
2017-03-10 17:39:21 +00:00
Nuno Lopes 7a1bbd4d73 fix build on Cygwin
llvm-svn: 297378
2017-03-09 13:43:31 +00:00
Zachary Turner 260bda3fbc [Support] Add llvm::sys::fs::remove_directories.
We already have a function create_directories() which can create
an entire tree, and remove() which can remove an empty directory,
but we do not have remove_directories() which can remove an entire
tree.  This patch adds such a function.

Because removing a directory tree can have dangerous consequences
when the tree contains a directory symlink, the patch here updates
the existing directory_iterator construct to optionally not follow
symlinks (previously it would always follow symlinks).  The delete
algorithm uses this flag so that for symlinks, only the links are
removed, and not the targets.

On Windows this is implemented with SHFileOperation, which also
does not recurse into symbolic links or junctions.

Differential Revision: https://reviews.llvm.org/D30676

llvm-svn: 297314
2017-03-08 22:49:32 +00:00
Zachary Turner 3b9fb88476 [fs] Make sure to check S_ISLNK() in fillStatus.
llvm-svn: 297167
2017-03-07 17:48:47 +00:00
Zachary Turner 82dd5421fb [Support] Add the option to not follow symlinks on stat.
llvm-svn: 297154
2017-03-07 16:10:10 +00:00
Zachary Turner 842972b740 [Support] Re-add the special OSX flags on mmap.
The problem appears to be that these flags can only be used
when mapping a file for read-only, not for readwrite.  So
we do that here.

llvm-svn: 295880
2017-02-22 21:24:06 +00:00
Michal Gorny 5ddd2a5bda [Support] Provide linux/magic.h fallback for older kernels
The function for distinguishing local and remote files added in r295768
unconditionally uses linux/magic.h header to provide necessary
filesystem magic numbers. However, in kernel headers predating 2.6.18
the magic numbers are spread throughout multiple include files.
Furthermore, LLVM did not require kernel headers being installed so far.

To increase the portability across different versions of Linux kernel
and different Linux systems, add CMake header checks for linux/magic.h
and -- if it is missing -- the linux/nfs_fs.h and linux/smb.h headers
which contained the numbers previously.

Furthermore, since the numbers are static and the feature does not seem
critical enough to make LLVM require kernel headers at all, add fallback
constants for the case when none of the necessary headers is available.

Differential Revision: https://reviews.llvm.org/D30261

llvm-svn: 295854
2017-02-22 18:09:15 +00:00
Zachary Turner e1ca5a294c Try to fix the buildbot on OSX.
Since I'm only seeing failures on OSX, and it's saying
permission denied, I'm suspecting this is due to the addition
of the MAP_RESILIENT_CODESIGN and/or MAP_RESILIENT_MEDIA flags.
Speculatively trying to remove those to get the bots working.

llvm-svn: 295770
2017-02-21 21:31:28 +00:00
Zachary Turner 6bc2dac132 Try to fix Android build.
llvm-svn: 295769
2017-02-21 21:13:10 +00:00
Zachary Turner 392ed9d342 [Support] Add a function to check if a file resides locally.
Differential Revision: https://reviews.llvm.org/D30010

llvm-svn: 295768
2017-02-21 20:55:47 +00:00
Omair Javaid f5d560bc84 Fix LLDB Android AArch64 GCC debug info build
Committing after fixing suggested changes and tested release/debug builds on 
x86_64-linux and arm/aarch64 builds.

Differential revision: https://reviews.llvm.org/D29042

llvm-svn: 293850
2017-02-02 01:17:49 +00:00
Pavel Labath f726dfa65e [Support] Use O_CLOEXEC only when declared
Summary:
Use the O_CLOEXEC flag only when it is available. Some old systems (e.g.
SLES10) do not support this flag. POSIX explicitly guarantees that this
flag can be checked for using #if, so there is no need for a CMake
check.

In case O_CLOEXEC is not supported, fall back to fcntl(FD_CLOEXEC)
instead.

Reviewers: rnk, rafael, mgorny

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D28894

llvm-svn: 292912
2017-01-24 10:57:01 +00:00
Pavel Labath 2f0960970f [Support] Add sys::fs::set_current_path() (aka chdir)
Summary:
This adds a cross-platform way of setting the current working directory
analogous to the existing current_path() function used for retrieving
it. The function will be used in lldb.

Reviewers: rafael, silvas, zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D29035

llvm-svn: 292907
2017-01-24 10:32:03 +00:00
Pavel Labath 97c7cf1d5c raw_fd_ostream: Make file handles non-inheritable by default
Summary:
This makes the file descriptors on unix platform non-inheritable (O_CLOEXEC).

There is no change in behavior on windows, as the handles were already
non-inheritable there.

Reviewers: rnk, rafael

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D28854

llvm-svn: 292401
2017-01-18 15:46:50 +00:00
Mehdi Amini 8e13bc4562 [ThinLTO] Add an API to trigger file-based API for returning objects to the linker
Summary:
The motivation is to support better the -object_path_lto option on
Darwin. The linker needs to write down the generate object files on
disk for later use by lldb or dsymutil (debug info are not present
in the final binary). We're moving this into libLTO so that we can
be smarter when a cache is enabled and hard-link when possible
instead of duplicating the files.

Reviewers: tejohnson, deadalnix, pcc

Subscribers: dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D27507

llvm-svn: 289631
2016-12-14 04:56:42 +00:00
Simon Pilgrim f2fbf43704 Fix comment typos. NFC.
Identified by Pedro Giffuni in PR27636.

llvm-svn: 287490
2016-11-20 13:47:59 +00:00
Pavel Labath 676a875b06 [Chrono] Fix !HAVE_FUTIMENS build
If we don't have futimens(), we fall back to futimes(), which only supports
microsecond timestamps. In that case, we need to explicitly cast away the extra
precision in setLastModificationAndAccessTime().

llvm-svn: 284977
2016-10-24 14:19:28 +00:00
Pavel Labath 757ca886cd Remove TimeValue usage from llvm/Support
Summary:
This is a follow-up to D25416. It removes all usages of TimeValue from
llvm/Support library (except for the actual TimeValue declaration), and replaces
them with appropriate usages of std::chrono. To facilitate this, I have added
small utility functions for converting time points and durations into appropriate
OS-specific types (FILETIME, struct timespec, ...).

Reviewers: zturner, mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25730

llvm-svn: 284966
2016-10-24 10:59:17 +00:00
Chandler Carruth 2aff750cb8 Add AIX support to Path.inc, Host.h, and CMake.
Patch by Andrew Paprocki!

Differential Revision: https://reviews.llvm.org/D18359

llvm-svn: 276045
2016-07-19 22:46:39 +00:00
Rafael Espindola 3816c53f04 Use posix_fallocate instead of ftruncate.
This makes sure that space is actually available. With this change
running lld on a full file system causes it to exit with

failed to open foo: No space left on device

instead of crashing with a sigbus.

llvm-svn: 276017
2016-07-19 20:19:56 +00:00
Taewook Oh d91532725e In openFileForRead, attempt to fetch the actual name of the file on disk -- including case -- so that clang can later warn about non-portable #include and #import directives.
Differential Revision: http://reviews.llvm.org/D19842
Corresponding clang patch: http://reviews.llvm.org/D19843

Re-commit after addressing issues with of generating too many warnings for Windows and asan test failures

Patch by Eric Niebler

llvm-svn: 272555
2016-06-13 15:54:56 +00:00
Taewook Oh 99497fdebd Revert commit r271704, a patch that enables warnings for non-portable #include and #import paths (Corresponding clang patch has been reverted by r271761). Patches are reverted because they generate lots of unadressable warnings for windows and fail tests under ASAN.
llvm-svn: 271764
2016-06-04 03:36:12 +00:00
Taewook Oh dfec58e80c In openFileForRead, attempt to fetch the actual name of the file on disk -- including case -- so that clang can later warn about non-portable #include and #import directives.
Differential Revision: http://reviews.llvm.org/D19842

Patch by Eric Niebler

llvm-svn: 271704
2016-06-03 18:38:39 +00:00
Mehdi Amini 3ba84ca62d Fix missing include on OpenBSD
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265803
2016-04-08 16:45:05 +00:00
Duncan P. N. Exon Smith 91d3cfed78 Revert "Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes."
This reverts commit r265454 since it broke the build.  E.g.:

  http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-incremental_build/22413/

llvm-svn: 265459
2016-04-05 20:45:04 +00:00
Eugene Zelenko 1760dc2a23 Fix Clang-tidy modernize-deprecated-headers warnings in remaining files; other minor fixes.
Some Include What You Use suggestions were used too.

Use anonymous namespaces in source files.

Differential revision: http://reviews.llvm.org/D18778

llvm-svn: 265454
2016-04-05 20:19:49 +00:00
Mehdi Amini e2d8f1b8fc Add disk_space() to llvm::fs
Summary: Adapted from Boost::filesystem.
(This is a reapply by reverting commit r265080 and fixing the WinAPI part)

Differential Revision: http://reviews.llvm.org/D18467

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265082
2016-04-01 00:18:08 +00:00
Mehdi Amini 640de72a1e Revert "Add disk_space() to llvm::fs"
This reverts commit r265074 and r265068.
Breaks windows build

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265080
2016-04-01 00:13:31 +00:00
Mehdi Amini e503a71df1 Use const ref instead of value for Twine in the disk_space() API
Thanks Rui for noticing!

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265074
2016-03-31 23:14:45 +00:00
Mehdi Amini 4c82356ad3 Add disk_space() to llvm::fs
Summary: Adapted from Boost::filesystem.
(This is a reapply by reverting commit r265062 and fixing the WinAPI part)

Differential Revision: http://reviews.llvm.org/D18467

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265068
2016-03-31 23:05:26 +00:00
Mehdi Amini b880144703 Revert "Add disk_space() to llvm::fs"
Breaks windows bot.
This reverts commit r265050.
This reverts commit r265055.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265062
2016-03-31 21:55:35 +00:00
Mehdi Amini 9defda528e Add disk_space() to llvm::fs
Summary: Adapted from Boost::filesystem.

Reviewers: bruno, silvas

Subscribers: tberghammer, danalbert, llvm-commits, srhines

Differential Revision: http://reviews.llvm.org/D18467

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265050
2016-03-31 20:48:27 +00:00
Mehdi Amini 1e39ef331b Add lastAccessedTime to file_status
Differential Revision: http://reviews.llvm.org/D18456

This is a re-commit of r264387 and r264388 after fixing a typo.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 264392
2016-03-25 07:30:21 +00:00
Mehdi Amini ec68482e53 Revert "Add lastAccessedTime to file_status"
This reverts commit r264387.
Bots are broken in various ways, I need to take one commit at a time...

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 264390
2016-03-25 06:51:43 +00:00