STORAGE - layout of the storage subsystem. ------- The storage system is two-tiered, consisting of a layer of caching on top of permanent secondary storage. The two layers are sharply separated to allow easy modification of the underlying storage mechanism, should that be necessary. The calling parameters of the cache layer exactly match the calling parameters of the permanent storage layer, to allow ease of change. THE CACHE --------- The cache is a two level hashed cache of doubly-linked lists. The first level of the cache contains objects that are "active" - IE: objects that have been accessed since the last time that cache was reset. This is used to provide what amounts to locking, since objects on the active cache chains will never be taken out of memory. The second cache chain is the old cache chain, and contains objects that have not been recently accessed. These objects may be replaced memory as newer objects come in and require space. If the old cache chain, at any time, is entirely transferred to the active cache, new objects are still be brought into memory by allocating new cache chain holders. The layout of the cache makes it somewhat complex but provides effective locking of current objects-in-use into memory, without requiring locking bits and a second pass through the cache to free locks after every command is processed. Whenever an object is accessed, it is moved to the active cache via pointer-juggling, and is kept there until the cache is reset - which moves the entire active cache to the head of the old cache - effecting a very fast LRU. When an old object is to be dropped from the cache chain, it is taken from the tail of the old cache chain, written to disk if dirty, and any allocated memory is freed. The cache width and depth are tuneable parameters - the width of the cache will influence the speed of accessing cached objects, since the width is hashed across (based on the numeric component of the object name). The depth of the cache influences the amount of memory that will be used, and the amount of time old objects will persist in the cache. Due to the way that the cache depth may be dynamically increased under heavy loads, setting the initial cache depth to a low value will result in poor initial performance, up to a steady state. SECONDARY STORAGE - permanent database ----------------- Secondary storage is accessed through two fuctions in the cache system: Obj *db_get(name) and int db_put(obj,name) which are solely responsible for storing an object and retrieving it by name. This is intended to permit alternate storage systems for architectures with database support or other constraints. The current implementations of the freeze/thaw routines generate a hash value from the object number (hashing across a different value than the cache) and use that to generate a directory name in which the data is stored. One object is stored in each file, in OIF format, and the supplementary routines to read and write OIF objects are used to handle interpreting and saving the objects. In practice this approach is quite quick, but there is room for improvement. If you find yourself modifying these routines, a good starting point is to achieve a deeper understanding of how the OIF and stretchy buffer library routines work, and to adapt them. INTERFACES ---------- Currently the following functions are expected to be provided by the cache layer: extern int cache_init(); /* initialize the cache */ extern int cache_sync(); /* update the disk based versions */ extern Obj *cache_get(); /* get an object from the cache */ extern int cache_put(); /* put an object back into the cache */ extern int cache_check(); /* check to see if an object exists */ extern int cache_configure(ac,av,who); /* configure cache params */ Any of the cache functions *MAY* call permanent database functions, if appropriate - the permanent database functions are call-compaible. IE: cache_sync() ->WILL call-> db_sync() ->and MAY call-> db_put() cache_get() ->may call-> db_get() cache_put() ->may call-> db_put() cache_check() ->may call-> db_check() cache_drop() ->WILL call-> db_drop() The permanent database routines also have a configuration function: extern int db_configure(ac,av,who); /* configure db params */ mjr. '91