VMALLOC(3)

NAME
vmalloc - virtual memory allocation

SYNOPSIS

  Regions

  Allocation functions

  Debugging

  Profiling

  Information and statistics

  Malloc-compatible functions

DESCRIPTION
These functions for dynamic storage allocation work in regions of memory. Each region has an allocation method for parceling out blocks of storage and a memory discipline for obtaining raw space. Automatic locking prevents interference by reentrant access to a region.

Pointers to space have type Void_t* where Void_t is #defined as void if possible, otherwise char. Space is counted in type size_t.

  Regions
Regions have type Vmalloc_t. Two predefined regions are pointed to by:

A general-purpose region, with best-fit
    allocation, and Unix memory discipline Vmdcsbrk.

The default region for malloc-compatible functions; initially
    the same as Vmheap.

These functions manipulate regions:

vmopen creates a region with memory discipline disc, allocation method meth, and a setting for control flags. It returns a pointer to the region on success and NULL on failure. The flags, represented by bit values or-ed together, are:

Disable locking and consistency checks, except under method Vmdebug.

Place tracing messages for each allocation event
    on the tracing file established by vmtrace. This turns off VM_TRUST.

VM_DBCHECK, VM_DBABORT
    See Debugging below.

vmclose closes a region and releases all associated memory according to the region's discipline. The first segment obtained from the discipline's memoryf function (see `Disciplines' below) will be the last released. vmclose returns -1 on failure and a non-negative value otherwise.

vmclear frees all allocated blocks in region regardless of methods. It returns -1 on failure and a non-negative value otherwise.

vmcompact releases as much of a region's free space to its discipline's memoryf function as possible. It returns a nonnegative value on success and -1 on failure.

vmset adjusts and queries a region's flags. The indicated flags are turned on if type is nonzero, off if zero. vmset returns the previous value of all flags. Thus, vmset(region,0,0) queries the flags without changing them. In addition to the settable flags, one of VM_MTBEST, VM_MTDEBUG, VM_MTPROFILE, VM_MTPOOL, or VM_MTLAST is returned to indicate the method used in creating the region.

  Allocation functions
vmalloc returns a pointer to a block of the requested size in a region, aligned to the strictest alignment that is suitable for the needs of any basic data type. It returns NULL on failure.

vmalign works like vmalloc, but returns a block aligned to a common multiple of align and the strictest alignment.

vmresize attempts to change the length of the block pointed to by addr to the specified size. If that is impossible and type is not 0, a new block is allocated and the old block is freed. If, in addition, type is positive, the new block is initialized with as much of the old contents as will fit. vmresize returns a pointer to the final block, or NULL on failure. If addr is NULL, vmresize behaves like vmalloc; otherwise, if size is 0, it behaves like vmfree.

vmfree makes the currently allocated block pointed to by addr available for future allocations in its region. If addr is NULL, vmfree does nothing. It returns -1 on error, and nonnegative otherwise.

  Memory disciplines
Memory disciplines have type Vmdisc_t, a structure with these members:

If this value is positive, all size arguments to the
    memoryf function will be multiples of it.

Points to a function to get or release segments of space for the
    region.

If this pointer is not NULL,
    the function it points to is called to announce events in a region.

There are two standard disciplines. In both, round is 0, and exceptf is NULL.

A discipline whose memoryf function gets space from sbrk(2).

A discipline whose memoryf function gets space from the region Vmheap.
    A region with Vmdcheap discipline and Vmlast allocation is good for building throwaway data structures.

A memoryf function returns a pointer to a memory segment on success, and NULL on failure. If csz is 0, the function returns a new segment of size nsz. Otherwise, the function attempts to change the length of the segment pointed to by addr from csz to nsz. If this is successful, memoryf should return addr (even if nsz is 0).

An exceptf function is called for events identified by type, which is coded thus:

A new region is being opened.
    If this region is to manipulate memory already initialized by a previous vmopen call (perhaps by a different process on persistent or shared memory), the exceptf function should return a positive value. In addition, the argument (Void_t**)obj should be used to return the initial segment. If exceptf returns a non-positive value or if *(Void_t**)obj is NULL, the region opening proceeds normally.

The region is being closed.
    If the exceptf function returns a negative value, the close will fail.

An attempt to extend the region by the amount
    (size_t)obj failed. The region is unlocked, so the exceptf function may free blocks. If the function returns a positive value the memory request will be repeated; if zero, the allocation method will again invoke memoryf to get space; if negative, the allocation request will fail.

Address obj, given to vmfree or vmresize,
    does not point to an allocated block from the region.

  Allocation methods
There are five methods, of type Vmethod_t*:

An approximately best-fit allocation strategy.

A strategy for building structures that are only deleted in whole.
    Only the latest allocated block can be freed or resized. This means that after a block a is allocated, all previously allocated blocks are frozen (see also vmclear) and only a can ever be freed or resized.

A strategy for blocks of one size,
    set by the first vmalloc call after vmopen or vmclear.

An allocation strategy with extra-stringent checking and locking
    regardless of the VM_TRUST flag. It is useful for finding misuses of dynamically allocated memory, such as writing beyond the boundary of a block, or freeing a block twice.

An allocation method that records and prints summaries of memory usage.

  Debugging
The method Vmdebug is used to debug common memory violation problems. When a problem is found, a warning message is written to file descriptor 2 (standard error). In addition, if flag VM_DBABORT is on, the program is terminated by calling abort(2). Each message is a line of self-explanatory fields separated by colons. The optional flag -DVMFL, if used during compilation, enables recording of file names and line numbers. The following functions work with method Vmdebug.

vmdbcheck checks an entire region for block boundary violations and other inconsistencies. It calls vmdbwarn when errors are found. If flag VM_DBCHECK is on for the region, vmdbcheck is called at each invocation of vmalloc, vmfree, or vmresize.

vmdbwatch causes address addr to be watched, and reported whenever met in vmalloc, vmresize or vmfree. The watch list has finite size and if it becomes full, watches will be removed in a first-in-first-out fashion. If addr is NULL, all current watches are canceled. vmdbwatch returns the watch bumped out due to an insertion into a full list or NULL otherwise.

vmdbwarn is an internal function that processes warning messages for discovered errors. It can't be called from outside the vmalloc package, but is a good place to plant debugger traps because control goes there at every trouble.

  Profiling
The method Vmprofile is used to profile memory usage. Profiling data are maintained in private memory of a process so Vmprofile should be avoided from regions manipulating persistent or shared memory. The optional flag -DVMFL, if used during compilation, enables recording of file names and line numbers.

vmprofile prints memory usage summary. The summary is restricted to region vm if vm is not NULL; otherwise, it is for all regions created with Vmprofile. Summary records are written to file descriptor fd as lines with colon-separated fields. Here are some of the fields:

n_alloc,n_free:
    Number of allocation and free calls respectively. Note that a resize operation is coded as a free and an allocation.

s_alloc,s_free:
    Total amounts allocated and freed. The difference between these numbers is the amount of space not yet freed.

max_busy, extent:
    These fields are only with the summary record for region. They show the maximum busy space at any time and the extent of the region.

  Information and statistics
vmregion returns the region to which the block pointed to by addr belongs. This works only in regions that allocate with Vmbest, Vmdebug or Vmprofile. If multiple regions manipulate the same segment of memory, vmregion returns the region that causes the creation that memory segment.

vmsegment finds if some segment of memory in region contains the address addr. It returns the address of a found segment or NULL if none found.

vmaddr checks whether addr points to an address within some allocated block of the given region. If not, it returns -1. If so, it returns the offset from the beginning of the block. The function does not work for a Vmlast region except on the latest allocated block.

vmsize returns the size of the allocated block pointed to by addr. It returns -1 if addr does not point to a valid block in the region. Sizes may be padded beyond that requested; in particular no block has size 0. The function does not work for a Vmlast region except on the latest allocated block.

vmstat gathers statistics on the given region and returns that information in the Vmstat_t structure pointed to by statb. A Vmstat_t structure has at least these members:

Bookeeping overhead is counted in extent, but not in s_busy or s_free.

vmtrace establishes file descriptor fd as the trace file and returns the previous value of the trace file descriptor. The trace descriptor is initially invalid. Output is sent to the trace file by successful allocation events when flag VM_TRACE is on.

Tools for analyzing traces are described in mtreplay(1). The trace record for an allocation event is a line with colon-separated fields, four numbers and one string.

old Zero for a fresh allocation; the address argument for freeing and resizing.

new Zero for freeing; the address returned by allocation or resizing.

size
    The size argument for allocation or resizing; the size freed by freeing. Sizes may differ due to padding for alignment.

region
    The address of the affected region.

method
    A string that tells the region's method: best, last, pool, profile, or debug.

  Malloc-compatible functions
Functions in this set work in region Vmregion and provide the behaviors of malloc(3). The functions memalign and valloc are like vmalign with the region argument fixed to Vmregion; valloc further restricts alignment to page boundaries.

The malloc functions are instrumented for run-time debugging, profiling and tracing. When these modes are enable, time and space performance will be affected. For accurate reporting of files and line numbers, code should include vmalloc.h and compile with -DVMFL. The following environment variables drive different modes:

VMETHOD
    This defines the method to use for allocation. Its value should be one of the strings: Vmbest, Vmdebug, Vmprofile, Vmlast, Vmpool. The 'V' can be in lower case.

VMDEBUG
    This is ignored if a method other than Vmdebug has been selected with VMETHOD. VMDEBUG can be any combination of `a', a decimal number and a list of hexadecimal numbers. `a' causes the program to abort on any discovered allocation error. A hexadecimal number starts with either 0x or 0X and defines an address to watch (see vmdbwatch). Any other number is taken to be decimal and defines a period p to check the arena for integrity. The default period is 1, ie, the arena is checked on every call to a malloc function. Other letters not part of the defined set are ignored.

VMPROFILE
    This is ignored if a method other than Vmprofile has been selected by VMETHOD or VMDEBUG. VMPROFILE defines a file name to store profile data. Each instance of the pattern `%p' found in VMPROFILE is transformed to the process id of the running process. If the file cannot be created, file descriptor 2 (standard error) is used for output.

VMTRACE
    If this defines a valid writable file, trace messages of all allocation calls are written to the given file (see vmopen() and vmtrace()). Similar to VMPROFILE, each instance of the pattern `%p' found in VMTRACE is turned to the process id of the running process.

SEE ALSO
mtreplay(1), malloc(3).

AUTHOR
Kiem-Phong Vo, kpv@research.att.com, AT&T Bell Laboratories