The PagingSystem Service Class

The service class PagingSystem manages the access of files whose entire content may not be stored in memory at the same time. These files are ultimately read and written via the Offline class, but they contain information that must be persistent which either grows during the time that it is being used or is too large to be stored entirely in memory. To work with these files, they are broken into fixed sized blocks, not all of which need to be memory resident at the same time. The methods in this class keep track of which blocks are currently in memory, where those blocks are in memory, and whether or not those blocks have been changed since they were last accessed. In addition, they must decide which file block to remove from memory when the allocated memory block count for the file has been exhausted.

There are three different approaches that can be used -- BlockVector, BlockSequence, and SearchForBlock. The BlockVector approach keeps a linear byte array indexed by the maximum number of blocks to be stored on the file. Each byte simply contains the number of the memory area containing the file block, if that block is in memory, or a zero if that block is not in memory. This approach is by far the fastest, but it requires as many bytes of memory as there are blocks on the file and it requires that the maximum size of the file be known when it is opened or created. The BlockSequence approach uses the MemorySequence class to maintain the memory block numbers that correspond to the file block numbers. The advantage here is that the maximum size of the file need not be known when it is opened or created. The disadvantage is slightly slower performance and the need to potentially create an extremely large memory sequence. The SearchForBlock approach keeps a linear array indexed by the maximum number of blocks allowed in memory. The blocks are always kept in numeric order by their number in the file. Each time a block is accessed, this array is binary searched to determine if the block is currently in memory. This approach is the most general since its memory requirements are independent of the size of the file but this approach is also the slowest.

The names of these paging approaches are in the PagingType enumeration. This enumeration contains two additional entries -- MemoryMapped and MemorySequence. These two approaches retain the entire file content in memory and can obviously only be used when the file size makes this possible. They are both much more efficient then approaches describe above, but do not support persistence. The MemoryMapped approach has the entire file mapped into memory. It can be used to work with a previously created file whose size will not change. The MemorySequence approach stores block pointers in a MemorySequence and therefore allows the information stored to grow; however, there is no actual file backing the information up so it is not persistent.

The notion of semi-permanent pointers pervades this approach to information management. The basic idea is that a page pointer is a valid memory pointer until the block which it contains is swapped out of memory. Thus these pointers can be used safely within local blocks of code where efficiency is needed, even though they cannot be saved and reused over longer stretches of code.

The method PagingSystem_Access

Prototype

The PagingSystem_Access method accesses a block from a paged file. It returns a pointer to the start of a page in the paging system which contains a specified block from the file being managed. Blocks are numbered sequentially starting at 1. Note that passing the method a block number larger than the current block count may force it to add new blocks to the back of the file. This method should only be passed existing block numbers. The method PagingSystem_AddBlock should be used to expand the file. Its parameters are:

The method returns a semi-permanent pointer to the start of the block, or a NULL if something went wrong.

The method PagingSystem_AddBlock

Prototype

The PagingSystem_AddBlock method adds a new block to a paging system and returns a semi-permanent pointer to its memory image. In addition, the number of the block may be returned. Note that this method is the only one to be used to add blocks to the file. The PagingSystemAccess method is only used to access existing blocks. Its parameters are:

If the paging system has a maximum number of blocks specified, then it returns NULL if that maximum has been exceeded; else it returns a semi-permanent byte pointer to the start of the new block. This new block is initialized at zero.

The method PagingSystem_Close

Prototype

The PagingSystem_Close method closes a PagingSystem instance, by returning any memory occupied by it to the operating system. Before doing so it writes any of its blocks that have been changed in memory to the file being controlled by the system. Note that this method does not close the actual file, just as the PagingSystem_Create method does not open the file. Its parameter is:

The method has no return value.

The method PagingSystem_Create

Prototype

The PagingSystem_Create method creates a PagingSystem instance for a file. This instance can then be used to access blocks on the file as stored in memory pages. Its parameters are:

If all goes well this method returns a handle for the control structure that will page the blocks of the file. If there is insufficient memory to allocate the various needed components, then a NULL is returned.