SFB 376

SFB 376 - Project A2

The DIstributed VAriables Library

Reference Manual for DIVA (Version 1.0)

This document provides a short description of all DIVA library functions. It is organized as follows.

Initialization and Termination

void diva_init(int *argc, char ***argv)
void diva_term()
[diva_init] initializes the system and synchronizes all processors. This function must be called before any other DIVA function. [*argc] and [*argv] are the number of program arguments and the pointer to the program arguments, recspectively, that are provided by the arguments to main. Note that both values are passed by reference.

[diva_init] reserves local memory for the simulated global memory on every processor. The amount of memory reserved can be controlled by a commandline parameter passed to the program. For example "ccsrun 1221 hello.exe -mem_size 1024" means that 1024 KB will be reserved for the global memory on every processor. If DIVA is running in an MPI environment, and MPI is not yet initialized, [diva_init] also initializes MPI and passes [*argc] and [*argv] to the MPI initialization function.

[diva_term] should be called by every processor at the end of a DIVA session. It synchronizes the complete system and releases all recources allocated by [diva_init], e.g., all allocated system buffers and spawned system threads. The user should ensure that all allcocated global variables and all generated groups (except for [DIVA_ALL] and [DIVA_SINGLE]) of a process have been released before it calls [diva_term].

If the corresponding [diva_init] call has initialized MPI, [diva_term] finalizes MPI.

The example program hello.c illustrates how these functions are used.

Generation and Release of Global Variables

diva_handle diva_create(size_t size, diva_group group)
void diva_free(diva_handle var, diva_group group)
[diva_create] creates a global variable, i.e., it allocates a memory block of [size] bytes in the virtual shared memory.

The function returns a handle for the created variable. This handle can be viewed as logical pointer to the variable. In the following we usually identify a global variable with its handle, i.e., "global variable [var]" means "the global variable associated with handle [var]".

The function has to be called by all processes which are members of the group [group]. For instance, if the group is [DIVA_ALL] then all processes have to call this function simultanously. Note that, for performance reasons, this function does not (necessarily) synchronize the calling processes. Thus, the processes must synchronized by barrier synchronizations before accessing the created variables.

In order to create a global variable by a single processor, this function can be called with group [DIVA_SINGLE]. Of course, then the global variable is known only by one processor. However, this processor can communicate the handle to other processors, which then also can access the respective global variable.

[diva_free] releases the global variable [var]. The function should be called by the processes which are members of [group]. It is in the users responsibility that all pending accesses to the global variable complete before [diva_free] is called.

These functions are illustrated in the example program hello.c.

Blocking Access to Global Variables

void diva_write(diva_handle var, void *buffer)
void diva_read(diva_handle var, void *buffer)
The functions for accessing global variables (diva_read, diva_write, diva_iread, diva_rlock ...) all use a local buffer for transferring data between local and global memory. This buffer must be large enough to hold the content of the respective variable, this means it must contain at least [size] bytes, where [size] is the corresponding parameter passed to [diva_create], during the creation of the variable.

[diva_write] writes the memory block pointed to by [buffer] into the global variable [var]. [diva_read] reads the global variable [var]. The content of [var] is copied into the local buffer pointed to by [buffer].

[diva_write] and [diva_read] are blocking, that means the calling thread is suspended until the respective read or write access is completed.

The example program hello.c illustrates how these function are used.

Nonblocking Access to Global Variables

void diva_iwrite(diva_handle var, void *buffer, diva_request *request)
void diva_iread(diva_handle var, void *buffer, diva_request *request)
void diva_wait(diva_request request)
int diva_test(diva_request request)
[diva_iwrite] and [diva_iread] are the nonblocking variants of [diva_write] and [diva_read]. That is, these functions only initiate a write access. Eventually, the call returns before the local buffer is written to the global variable. A seperate call to [diva_wait] or [diva_test] parameterized with [request] is needed to complete the write access. The local buffer must not be changed before the write access has been completed.

[diva_iwrite] writes the memory block pointed to by [buffer] into the global variable [var]. [diva_iread] reads the global variable [var]. The content of [var] is copied into the local buffer pointed to by [buffer]. A call to [diva_wait] returns when the read or write access corresponding to [request] is completed.

A call to [diva_test] returns 1 if the read or write access corresponding to [request] is completed, otherwise it returns 0.

The example program hello_nblock.c illustrates how the nonblocking access to variables can be used.

Barrier Synchronization

void diva_sync(diva_group group)
[diva_sync] synchronizes all processes in the group [group], i.e., it blocks the calling process until all other processes in the group have called [diva_sync].

For instance, if diva_sync is called with parameter [DIVA_ALL], then each process in the system has to wait at a barrier until all processes have reached this barrier.

The example program hello.c illustrates how these barrier synchronizations work.

Locking of Global Variables

void diva_lock(diva_handle var)
int diva_slock(diva_handle var)
void diva_unlock(diva_handle var)
If a process locks a variable with [diva_lock], all further accesses to the variable from other processes (especially locks from other processes) are suspended, until the variable is released again.

With [diva_slock] (soft lock) the calling process tries to obtain a lock on the respective variable. If the corresponding variable is not locked then this call locks the variable. If the variable is locked by another process the calling process is suspended until the lock is released, but the variable is not (!) locked. If the lock can not be realized, because another process has locked the variable, the function returns 0, otherwise it returns 1.

With [diva_unlock] a locked variable can be released by the process which locked it. All waiting accesses to the variable will be waked up.

The example program hello_lock.c illustrates the lock mechanism.

Group Management

diva_group diva_new_group(diva_group old_group,int color)
void diva_free_group(diva_group group)
The functions [diva_create], [diva_free], and [diva_sync] are called by groups of processes. The DIVA system provides two kind of standard groups: [DIVA_ALL] is the group of all processes, and [DIVA_SINGLE] denotes a group consisting only of one process, i.e., the calling process.

The function [diva_new_group] allows to generate new groups. In particular, it partitions the group associated with [old_group] into disjoint subgroups, one for each value of color. Each of the subgroups contains all processes of the same color.

The function must be called by any process of [old_group]. Besides to the group generation, the function synchronizes all processes in [old_group].

[diva_free_group] releases a group created with [diva_new_group]. The example program hello_groups.c illustrates the generation and release of groups.

Global Operations

int diva_init_func(int data_size,void (*oper)(void* a,void* b))
void diva_exec_func(diva_group group,int func_id, void* operand)
The functions [diva_init_func] and [diva_exec_func] allow to compute global operations in parallel by the members of a group. The operations have to be associative and commutative.

[diva_init_func] announces a global operation to the system and returns a func_id. The parameter [data_size] specifies the size of the operands the operator works on. The second parameter is a pointer to the operator function. This function, which has to be defined by the user, expects two pointers to the input-datablocks. The result of the operation has to be copied to the first datablock.
The user has to make sure that [diva_init_func] has been called on every processor before the respective function is executed via [diva_exec_func].

[diva_exec_func] computes the global operation on the given operands. Every process of the specified group has to call [diva_exec_func], passing a pointer to his operand and the correct func_id to the function. After execution [operand] points to the result of the operation. The example program hello_op.c illustrates the use of [diva_init_func] and [diva_exec_func].

Garbage Collection Mechanism

diva_handle diva_increase_handle_counter(diva_handle var,int increment)
When calling [diva_create] for a global variable, the reference counter of the variable is set to the size of the calling group. Each call to [diva_increase_handle_counter] increases this counter by [increment]. Each call to [diva_free] decreases the reference counter by the size of the calling group. The global variable is released when the reference counter is decreased to 0.

Auxiliary Functions

int diva_nprocs()
int diva_id()
[diva_nprocs] returns the overall number of processes in the system.

 [diva_id] returns the id of the calling process. The processes are always numbered from 0 to [diva_nprocs] - 1.

Harald Räcke, Christof Krick, August 2000