Commit Graph

22 Commits

Author SHA1 Message Date
Alex Shlyapnikov 01676883cd [Sanitizers] 64 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator64 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow
allocator_may_return_null=1 policy
(LargeMmapAllocator: D34243, SizeClassAllocator64: D34433).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34540

llvm-svn: 306342
2017-06-26 22:54:10 +00:00
Alex Shlyapnikov f3cc7cc3d8 [Sanitizers] 32 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).

Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34433

llvm-svn: 305972
2017-06-22 00:02:37 +00:00
Kostya Kortchinsky 38199b2a30 [sanitizer] Cache SizeClassForTransferBatch in the 32-bit local cache
Summary:
`SizeClassForTransferBatch` is expensive and is called for every `CreateBatch`
and `DestroyBatch`. Caching it means `kNumClasses` calls in `InitCache`
instead. This should be a performance gain if more than `kNumClasses / 2`
batches are created and destroyed during the lifetime of the local cache.

I have chosen to fully remove the function and putting the code in `InitCache`,
which is a debatable choice.

In single threaded benchmarks leveraging primary backed allocations, this turns
out to be a sizeable gain in performances (greater than 5%). In multithreaded
benchmarks leveraging everything, it is less significant but still an
improvement (about 1%).

Reviewers: kcc, dvyukov, alekseyshl

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32365

llvm-svn: 301184
2017-04-24 14:53:38 +00:00
Alex Shlyapnikov 8b2caae996 Cache size per class size in SizeClassAllocatorXLocalCache.
Summary:
Allocator::ClassIdToSize() is not free and calling it in every
Allocate/Deallocate has noticeable impact on perf.
Reapplying D31991 with the appropriate fixes.

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32024

llvm-svn: 300216
2017-04-13 16:49:16 +00:00
Diana Picus 7e5db62ad5 Revert "Cache size per class size in SizeClassAllocatorXLocalCache."
This reverts commit r300107 because it broke the ARM and AArch64
buildbots.

llvm-svn: 300180
2017-04-13 07:39:04 +00:00
Alex Shlyapnikov daa342d9c3 Cache size per class size in SizeClassAllocatorXLocalCache.
Summary:
Allocator::ClassIdToSize() is not free and calling it in every
Allocate/Deallocate has noticeable impact on perf.

Reviewers: eugenis, kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D31991

llvm-svn: 300107
2017-04-12 21:44:56 +00:00
Alex Shlyapnikov 05bf27ac3f Avoid calling SizeClassMap::MaxCachedHint on hot path, it's not free.
Summary: Remove unecessary SizeClassMap::MaxCachedHint call.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D31989

llvm-svn: 300103
2017-04-12 20:51:42 +00:00
Kostya Serebryany 4fd30769c1 [sanitizer] remove kBatchClassID that is not used any more; NFC
llvm-svn: 280185
2016-08-31 00:37:33 +00:00
Kostya Serebryany 8e7ea9ddb5 [sanitizer] re-apply r279572 and r279595 reverted in r279643: change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279664
2016-08-24 21:20:10 +00:00
Vitaly Buka 769ec705bc Revert r279572 "[sanitizer] change the 64-bit..." because of failures on ubsan
This reverts commit r279572 and r279595.

llvm-svn: 279643
2016-08-24 17:40:29 +00:00
Kostya Serebryany f46d50e360 [sanitizer] change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279572
2016-08-23 21:19:47 +00:00
Kostya Serebryany ce2163459f [sanitizer] allocator: split the local cache class into two, one for 32-bit allocator and one for 64-bit one. NFC. The two imlementations will diverge in the following changes.
llvm-svn: 279495
2016-08-23 00:30:43 +00:00
Kostya Serebryany 1ab649649d [sanitizer] use 32-bit offset instead of 64-bit pointers in the 64-bit allocator's transfer batches. This saves 2x memory for the transfer batches (up to ~1.5% overall in some cases)
llvm-svn: 278179
2016-08-09 23:30:22 +00:00
Kostya Serebryany d4a5749677 [sanitizer] minor refactoring in the allocator, NFC
llvm-svn: 278163
2016-08-09 20:54:50 +00:00
Kostya Serebryany 4cd2845e6a [sanitizer] allocator: move TransferBatch into SizeClassAllocator64/SizeClassAllocator32 because we actually need different iplementations for the 64- and 32-bit case. NFC; the following patches will make the TransferBatch implementations differ
llvm-svn: 277899
2016-08-06 01:24:11 +00:00
Kostya Serebryany dcaec77936 [sanitizer] remove one redundant loop from the allocator
llvm-svn: 277563
2016-08-03 03:42:55 +00:00
Kostya Serebryany c49e296805 [sanitizer] refactor TransferBatch to hide the implementation. NFC expected. Second attempt after failed r276383 which was reverted.
llvm-svn: 277554
2016-08-03 00:14:10 +00:00
Kostya Serebryany d251e94a01 [sanitizer] revert r276383 while investigating failures on bot
llvm-svn: 276456
2016-07-22 19:02:59 +00:00
Kostya Serebryany 5bc01c108d [sanitizer] refactor TransferBatch to hide the implementation. NFC
llvm-svn: 276383
2016-07-22 02:21:12 +00:00
Kostya Serebryany bfd69e0580 [sanitizer] allocator: introduce kUseSeparateSizeClassForBatch (false by default). When true, it will cause all TransferBatches to be allocated on a separate dedicated size class, which improves security and may potentially simplify memory reclamation. However in the current state this may cause up to 3% extra memory usage. Subsequent changes should bring this overhead down
llvm-svn: 276377
2016-07-22 01:13:13 +00:00
Kostya Serebryany 35eeea707e [sanitizer] allocator: remove kPopulateSize and only use SizeClassMap::MaxCached; ensure that TransferBatch size is a power of two, refactor TransferBatch creation/destruction into separate functions.
llvm-svn: 276318
2016-07-21 18:47:53 +00:00
Kostya Serebryany 9835a81529 [sanitizers] split sanitizer_allocator.h into a number of smaller .h files; NFC
llvm-svn: 276195
2016-07-20 22:06:41 +00:00