liblzma's API.
The simplest solution is to use setrlimit() if the kernel supports
RLIMIT_AS, which limits the memory usage of the whole process.
For more portable and fine-grained limitting, you can use
- memory limitter functions found from <lzma/memlimit.h>.
+ memory limiter functions found from <lzma/memlimit.h>.
1.2.1. Encoder
1.2.2. Decoder
- A single-threaded decoder should simply use a memory limitter and
+ A single-threaded decoder should simply use a memory limiter and
indicate an error if it runs out of memory.
Memory-limitting with multi-threaded decoding is tricky. The simple
solution is to divide the maximum allowed memory usage with the
maximum allowed threads, and give each Block decoder their own
- independent lzma_memory_limitter. The drawback is that if one Block
+ independent lzma_memory_limiter. The drawback is that if one Block
needs notably more RAM than any other Block, the decoder will run out
of memory when in reality there would be plenty of free RAM.
- An attractive alternative would be using shared lzma_memory_limitter.
+ An attractive alternative would be using shared lzma_memory_limiter.
Depending on the application and the expected type of input, this may
either be the best solution or a source of hard-to-repeat problems.
Consider the following requirements:
- You use at maximum of n threads.
- x(i) is the decoder memory requirements of the Block number i
in an expected input Stream.
- - The memory limitter is set to higher value than the sum of n
+ - The memory limiter is set to higher value than the sum of n
highest values x(i).
(If you are better at explaining the above conditions, please
Most .lzma files have all the Blocks encoded with identical settings,
or at least the memory usage won't vary dramatically. That's why most
multi-threaded decoders probably want to use the simple "separate
- lzma_memory_limitter for each thread" solution, possibly fallbacking
+ lzma_memory_limiter for each thread" solution, possibly fallbacking
to single-threaded mode in case the per-thread memory limits aren't
enough in multi-threaded mode.
creating a denial of service like piping decoded a Data Block to
another process would do.
- At first it would seem that using a memory limitter would prevent
+ At first it would seem that using a memory limiter would prevent
this issue as a side effect. But it does so only if the application
requests liblzma to allocate the Extra Records and provide them to
the application. If Extra Records aren't requested, they aren't
* custom memory allocation functions for use with liblzma.
*
* When using lzma_memory_alloc() and lzma_memory_free(), opaque
- * must point to lzma_memory_limitter structure allocated and
- * initialized with lzma_memory_limitter_create().
+ * must point to lzma_memory_limiter structure allocated and
+ * initialized with lzma_memory_limiter_create().
*
* If you don't need this, you should set it to NULL.
*/
/**
* \file lzma/memlimit.h
- * \brief Memory usage limitter
+ * \brief Memory usage limiter
*
* \author Copyright (C) 1999-2006 Igor Pavlov
* \author Copyright (C) 2007 Lasse Collin
* to these functions can be used in lzma_allocator structure, which makes
* it easy to limit memory usage with liblzma.
*
- * The memory limitter functions are not tied to limitting memory usage
+ * The memory limiter functions are not tied to limitting memory usage
* with liblzma itself. You can use them with anything you like.
*
* In multi-threaded applications, only one thread at once may use the same
/**
- * \brief Gets the number of allocations owned by the memory limitter
+ * \brief Gets the number of allocations owned by the memory limiter
*
* The count does not include the helper structures; if no memory has
* been allocated with lzma_memlimit_alloc() or all memory allocated
/**
- * \brief Frees the memory allocated for and by the memory usage limitter
+ * \brief Frees the memory allocated for and by the memory usage limiter
*
- * \param mem Pointer to memory limitter
+ * \param mem Pointer to memory limiter
* \param free_allocated If this is non-zero, all the memory allocated
* by lzma_memlimit_alloc() using *mem is also
* freed if it hasn't already been freed with
* completely when lzma_code() returns LZMA_STREAM_END. If *footer is still
* NULL, there was no Extra field in the Footer Metadata Block.
*
- * \note If you use lzma_memory_limitter, the Extra Records will be
- * allocated with it, and thus remain in the lzma_memory_limitter
+ * \note If you use lzma_memory_limiter, the Extra Records will be
+ * allocated with it, and thus remain in the lzma_memory_limiter
* even after they get exported to the application via *header
* and *footer pointers.
*/
index.c \
info.c \
init.c \
- memory_limitter.c \
+ memory_limiter.c \
memory_usage.c \
next_coder.c \
raw_common.c \
///////////////////////////////////////////////////////////////////////////////
//
-/// \file memory_limitter.c
+/// \file memory_limiter.c
/// \brief Limitting memory usage
//
// Copyright (C) 2007 Lasse Collin
static void
list_native(listing_handle *handle)
{
- lzma_memory_limitter *limitter
- = lzma_memory_limitter_create(opt_memory);
- if (limitter == NULL) {
+ lzma_memory_limiter *limiter
+ = lzma_memory_limiter_create(opt_memory);
+ if (limiter == NULL) {
errmsg(V_ERROR,
}
lzma_info *info =
lzma_init_decoder();
- lzma_memlimit *mem_limitter = lzma_memlimit_create(mem_limit);
- if (mem_limitter == NULL) {
+ lzma_memlimit *mem_limiter = lzma_memlimit_create(mem_limit);
+ if (mem_limiter == NULL) {
fprintf(stderr, "%s: %s\n", argv0, strerror(ENOMEM));
exit(ERROR);
}
- assert(lzma_memlimit_count(mem_limitter) == 0);
+ assert(lzma_memlimit_count(mem_limiter) == 0);
- allocator.opaque = mem_limitter;
+ allocator.opaque = mem_limiter;
strm.allocator = &allocator;
#ifdef WIN32
// Free the memory only when debugging. Freeing wastes some time,
// but allows detecting possible memory leaks with Valgrind.
lzma_end(&strm);
- assert(lzma_memlimit_count(mem_limitter) == 0);
- lzma_memlimit_end(mem_limitter, false);
+ assert(lzma_memlimit_count(mem_limiter) == 0);
+ lzma_memlimit_end(mem_limiter, false);
#endif
return exit_status;
///////////////////////////////////////////////////////////////////////////////
//
/// \file test_memlimit.c
-/// \brief Tests the memory usage limitter
+/// \brief Tests the memory usage limiter
///
/// \note These tests cannot be done at exact byte count accuracy,
-/// because memory limitter takes into account the memory wasted
+/// because memory limiter takes into account the memory wasted
/// by bookkeeping structures and alignment (padding).
//
// Copyright (C) 2008 Lasse Collin
projects / xz / commitdiff