STORAGE - layout of the storage subsystem.
-------


The storage system is two-tiered, consisting of a layer of caching on top
of permanent secondary storage. The two layers are sharply separated to
allow easy modification of the underlying storage mechanism, should that
be necessary. The calling parameters of the cache layer exactly match
the calling parameters of the permanent storage layer, to allow ease of
change.


THE CACHE
---------
The cache is a two level hashed cache of doubly-linked lists. The first
level of the cache contains objects that are "active" - IE: objects that
have been accessed since the last time that cache was reset. This is
used to provide what amounts to locking, since objects on the active
cache chains will never be taken out of memory. The second cache chain
is the old cache chain, and contains objects that have not been recently
accessed. These objects may be replaced memory as newer objects come in
and require space. If the old cache chain, at any time, is entirely
transferred to the active cache, new objects are still be brought into
memory by allocating new cache chain holders. The layout of the cache
makes it somewhat complex but provides effective locking of current
objects-in-use into memory, without requiring locking bits and a second
pass through the cache to free locks after every command is processed.
Whenever an object is accessed, it is moved to the active cache via
pointer-juggling, and is kept there until the cache is reset - which
moves the entire active cache to the head of the old cache - effecting
a very fast LRU. When an old object is to be dropped from the cache
chain, it is taken from the tail of the old cache chain, written to
disk if dirty, and any allocated memory is freed.

The cache width and depth are tuneable parameters - the width of the
cache will influence the speed of accessing cached objects, since
the width is hashed across (based on the numeric component of the
object name). The depth of the cache influences the amount of
memory that will be used, and the amount of time old objects will
persist in the cache. Due to the way that the cache depth may
be dynamically increased under heavy loads, setting the initial cache
depth to a low value will result in poor initial performance, up
to a steady state.


SECONDARY STORAGE - permanent database
-----------------
Secondary storage is accessed through two fuctions in the cache system:
Obj *db_get(name)
and
int db_put(obj,name)
which are solely responsible for storing an object and retrieving it
by name. This is intended to permit alternate storage systems for
architectures with database support or other constraints. The current
implementations of the freeze/thaw routines generate a hash value
from the object number (hashing across a different value than the
cache) and use that to generate a directory name in which the data
is stored. One object is stored in each file, in OIF format, and the
supplementary routines to read and write OIF objects are used to
handle interpreting and saving the objects. In practice this approach
is quite quick, but there is room for improvement. If you find yourself
modifying these routines, a good starting point is to achieve a deeper
understanding of how the OIF and stretchy buffer library
routines work, and to adapt them.


INTERFACES
----------
Currently the following functions are expected to be provided by the
cache layer:

extern	int	cache_init();	/* initialize the cache */
extern	int	cache_sync();	/* update the disk based versions */
extern	Obj	*cache_get();	/* get an object from the cache */
extern	int	cache_put();	/* put an object back into the cache */
extern	int	cache_check();	/* check to see if an object exists */
extern	int	cache_configure(ac,av,who);	/* configure cache params */

Any of the cache functions *MAY* call permanent database functions, if
appropriate - the permanent database functions are call-compaible. IE:
cache_sync()	->WILL call->	db_sync() ->and MAY call-> db_put()
cache_get()	->may call->	db_get()
cache_put()	->may call->	db_put()
cache_check()	->may call->	db_check()
cache_drop()	->WILL call->	db_drop()

The permanent database routines also have a configuration function:
extern	int	db_configure(ac,av,who);	/* configure db params */


mjr. '91