<!-- MHonArc v2.4.4 -->
<!--X-Subject: Re: [MUD-Dev] ColdStore. Belated response from a developer. -->
<!--X-From-R13: pbyvaNsvryq.zrqvpvar.nqrynvqr.rqh.nh -->
<!--X-Date: Mon, 24 Apr 2000 11:47:00 -0700 -->
<!--X-Message-Id: 20000424084309.265092F02C#sharedtech,eyep.net -->
<!--X-Content-Type: text/plain -->
<!--X-Reference: 20000422001959.76C5B2F02C#sharedtech,eyep.net -->
<!--X-Reference: 7er9bx2571.fsf#zesoi,fer.hr -->
<!--X-Head-End-->
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<title>MUD-Dev message, Re: [MUD-Dev] ColdStore. Belated response from a developer.</title>
<!-- meta name="robots" content="noindex,nofollow" -->
<link rev="made" href="mailto:colin#field,medicine.adelaide.edu.au">
</head>
<body background="/backgrounds/paperback.gif" bgcolor="#ffffff"
text="#000000" link="#0000FF" alink="#FF0000" vlink="#006000">
<font size="+4" color="#804040">
<strong><em>MUD-Dev<br>mailing list archive</em></strong>
</font>
<br>
[ <a href="../">Other Periods</a>
| <a href="../../">Other mailing lists</a>
| <a href="/search.php3">Search</a>
]
<br clear=all><hr>
<!--X-Body-Begin-->
<!--X-User-Header-->
<!--X-User-Header-End-->
<!--X-TopPNI-->
Date:
[ <a href="msg00545.html">Previous</a>
| <a href="msg00544.html">Next</a>
]
Thread:
[ <a href="msg00539.html">Previous</a>
| <a href="msg00500.html">Next</a>
]
Index:
[ <A HREF="author.html#00546">Author</A>
| <A HREF="#00546">Date</A>
| <A HREF="thread.html#00546">Thread</A>
]
<!--X-TopPNI-End-->
<!--X-MsgBody-->
<!--X-Subject-Header-Begin-->
<H1>Re: [MUD-Dev] ColdStore. Belated response from a developer.</H1>
<HR>
<!--X-Subject-Header-End-->
<!--X-Head-of-Message-->
<UL>
<LI><em>To</em>: <A HREF="mailto:mud-dev#kanga,nu">mud-dev#kanga,nu</A></LI>
<LI><em>Subject</em>: Re: [MUD-Dev] ColdStore. Belated response from a developer. </LI>
<LI><em>From</em>: <A HREF="mailto:colin#field,medicine.adelaide.edu.au">colin#field,medicine.adelaide.edu.au</A></LI>
<LI><em>Date</em>: Mon, 24 Apr 2000 18:43:09 +1000</LI>
<LI><em>Reply-To</em>: <A HREF="mailto:mud-dev#kanga,nu">mud-dev#kanga,nu</A></LI>
<LI><em>Sender</em>: <A HREF="mailto:mud-dev-admin#kanga,nu">mud-dev-admin#kanga,nu</A></LI>
</UL>
<!--X-Head-of-Message-End-->
<!--X-Head-Body-Sep-Begin-->
<HR>
<!--X-Head-Body-Sep-End-->
<!--X-Body-of-Message-->
<PRE>
> Miroslav Silovic <silovic#zesoi,fer.hr> writes:
> colin#field,medicine.adelaide.edu.au writes:
>
> > ColdStore compresses small integers too.
>
> Could you post a bit more about ColdStore internals on this list?
ColdStore's a layered system. At layer0 you have a locale-based allocator
which works fairly hard to keep regions allocated to a given locale on a
minimal number of pages (theory being to reduce hard page faults.) At layer1,
you have Yet Another Class Library, with collection objects, BigInts, Strings,
whatever else you believe should be a primitive datatype for whatever you're
building above it. At layer2, are objects designed to support language
interpretation and such.
From layer1 up, the objects inherit from a primitive Data class which exports
a virtual protocol based broadly on Python's API. The idea here is that you
write any useful datatype which you want to use above here, and coerce its
interface to look like Data's, and have immediate access to it from languages
implemented at layer2. Layer1 objects written to date have been optimised
wherever possible to use layer0's allocation tactics to maximise spatial
locality of reference.
Layer1 uses a Slot smartpointer to reference Data instances, maintaining
refcounts and such. It additionally exploits the fact that qvmm longword
aligns all allocations to tag short integers (31bits) and transparently
converts them to pool-allocated Integers when they need to be operated on.
> I was under impression that it used plain virtual memory to talk to
> hardware (thus getting speed).
>
> > I'd have thought the storage between Cold and ColdStore was
> > comparable.
I meant the amount of space between Genesis and ColdStore would be
approximately comparable, unless you've changed the marshalling code from the
original coldmud (and it was pretty optimal.)
> The last version of Cold storage is my work, and one I'm not
> particularily proud of (okay, this is an understatement. In actuality,
> it's a brown paper bag quality code - I want to wear one over my head
> when I think about certain things in there). :)
>
> > ColdStore doesn't set out to make C++ programming safe, because
> > that's impossible. It sets out to provide a toolkit for supporting
> > higher-level interpreted languages in which one would write
> > applications.
>
> The website suggested it was C++ persistent store system, I was
> replying under that assumption.
It is a C++ persistent store, with all the baggage that C++ brings.
> > It's certainly necessary to thoroughly engineer the C++ classes you
> > use and to avoid world-stopping bugs, however Miro's argument could
> > apply to any complex piece of code upon which your world depends,
> > the o/s, the db backend.
>
> 'Don't write buggy code' is not something I'd recommend to a
> programmer and expect gratitude. :)
Again, if there's a bug somewhere in (say) MOO server, or in Genesis server,
or in FreeBSD or Linux or Solaris o/s, it's going to have an adverse impact on
the service run ... ColdStore C++ objects are at that level, below the
application level.
> > One could certainly use the toconstructor() facility for changing
> > the raw layout of objects.
> >
> > This would be painful, however, and one would be well advised to
> > think through the raw layout of lowest level objects before
> > committing them to persistent store.
>
> Assuming that ColdStore is not meant to store C++ objects, but only be
> used with an interpreter to keep it in check, this makes sense.
ColdStore can be used at layer0, to store C++ objects, or at layer1 to mediate
between the storage of C++ objects and the provision of those C++ objects to
an interpreted language. It's really up to the application developer. I'd
probably recommend either the use of an interpreted language, or the use of
Java or something to generate the native code used to manipulate those objects
stored in C++.
> > >- db format is highly non-portable. It's not just architecture dependant,
> > it's C++ compiler dependant (as object layout may change).
> >
> > Certainly. It was a design decision to localise the cost of moving
> > between architectures to the time when one might wish to actually
> > move them, rather than seek to support an abstracted/portable format
> > for which one pays on each access.
> >
> > Many database formats are not immediately/binary portable, usually the
> > databases have to be serialised or marshalled before moving them.
>
> Does ColdStore support this, or is it supposed to be provided by an
> application sitting on top of it?
I've followed this convention in all the layer1 objects I've written, the
toconstructor() method is present in the virtual protocol supported by layer1,
if people want to write objects which they don't want to serialise, that's
their lookout. I probably wouldn't choose to use them, though.
Another alternative presenting itself more recently is the use of the openc++
preprocessor (which we use) to autogenerate serialising/deserialising code.
It's possible, but it's a long way from the top of my priority queue.
> > I was in the process of writing it right out in favour of GC, when
> > someone pointed out to me that refcounting is a great storage
> > optimiser - ColdStore collections and vector types share all
> > substructure, and implement Copy On Write semantics on low level
> > objects.
>
> This is acceptable if you already decided to pay the penalty for
> having an interpreter in the first place.
Sure. Look, I personally would write C++ directly to the store, because I'm
really comfortable with it. I like it. I wrote a lot of it ... On the other
hand, I really think you could do worse than using some interpreted language
to implement the functionality you want, work out what needs to be written
more efficiently, and then sink it lower down, in C++. <-- The usual
scripting language justification.
> > I don't see the problem with using inline assembly for atomic
> > increments, that's what inline assembly is for. It's what we use,
> > nicely wrapped in a Counter class.
>
> *.......* (insert any number of sounds here)
(insert colorful finger gestures here)
> > Maximising data locality by sharing substructure will always tend to
> > increase contention between threads at the same time as it tends to
> > decrease page faults. This is a classic trade-off: throughput
> > vs. responsiveness. Are networks really fast enough, yet, that
> > response-lag is an issue?
> >
> > As long as the time taken to swap a page from disk is much longer
> > than a thread context switch, I'll tend to worry more about locality
> > than thread performance, most particularly since most of large MUD
> > architectures don't even have real multitasking.
>
> Okay, I think this is the central issue.
>
> Yes, textual MUDs are bandwidth-bound. However, this list is not only
> about textual MUDs. Also, I think that bandwidth-bound MUDs are not
> interesting any more (because even MOO, once that memory became cheap,
> CPUs became fast, and codebase became slightly more optimised than it
> used to be, is now bandwidth-bound - so why bother inventing a better
> mousetrap?)
I didn't really mean throughput as in network bandwidth, but more disk swap
bandwidth.
ColdStore arose from an observation that coldmud was reputed to be faster than
MOO even though coldmud had to go through a serialisation/deserialisation
whenever objects were swapped between disk and memory. We concluded that the
disparity was due to an accidental feature of coldmud, that it would unpack
objects into a minimal set of pages in the process address space, while MOO
(seemingly by design) scatters its object across the whole address space.
The analysis lead us to suppose that maximising spatial locality of reference
was a highly desirable performance enhancement.
I guess I'm contending that it doesn't matter how little contention there is
for shared resources, if those resources are scattered all over the disk in
little pieces which have to be reassembled while the threads wait for them to
be swapped back in.
> Now, if you have a predictive client (one that keeps some state, and
> knows how to evolve and display it), you can just send diffs from the
> server, saving LOTS of bandwidth. In that case, the interesting things
> become possible, but the server bottleneck shifts from bandwidth to
> processing. Refer to this list's archive for examples.
> Notice about paging - you have to write to the page each time you
> increase the refcount, and refcount may get increased just by passing
> pointers around.
Writing to a page doesn't necessarily entail writing it back to disk ASAP. If
it does happen to be so in existing kernels, then I guess I'll have to modify
the kernel.
> This is different from GC schemes, where you only
> touch the pages you really want to write to (the tradeoff being the
> need for write barrier, with any decent GC). Then there is robustness
> of GC compared to refcounting (as reported by people involved with
> Mozilla project). Refcounting certainly makes sense if the ammount of
> the code in native language (C++, in this case) is small - but I don't
> like assuming that for a low level system I'd like to see in wide use.
Yes. We're trying to minimise low level code.
> > Additionally, the authors of Texas don't answer email, the build
> > process entails heavy use of a hacked gdb (using STABS to generate
> > object interpretation code for swizzling) and Texas has no support
> > for dynamically loading new objects into a store.
>
> I actually meant to say that Texas is a cool concept whose
> implementation desperately needs a complete rewrite, preferably by a
> team of programmers (rather than a team of computer scientists). :)
Computer scientists, having proven the concept, move on and leave we poor
proggers to pick up the pieces.
Anyway, it occurred to me after discussing your comments with others on the
ColdStore team (skeptopotamus, specifically) that it'd be entirely possible to
use the Texas swizzling concept to provide a much larger second-level store,
with the unswizzled objects relocated to reside in ColdStore (as a kind of
cache.)
What it'd buy you is a much larger address space - Texas object ids are
32bits, but can be mapped onto a much larger store. You'd still take the
performance hit of the segv overhead, and unswizzling, but it'd be an
interesting hybrid.
Perhaps you could do some kind of magical DB backing store and use coldstore
as a general cache ... don't know, I'll jump that (1.75 Gb) barrier when I
come to it.
> To conclude, within the assumptions you decided to work with,
> ColdStore seems cool (modulo my disagreement with use of inline
> assembly). But I tend to work under VERY different assumptions. :)
I'm all for mosaic solutions. May your assumptions lead you to interesting new problems.
Colin.
_______________________________________________
MUD-Dev mailing list
MUD-Dev#kanga,nu
<A HREF="http://www.kanga.nu/lists/listinfo/mud-dev">http://www.kanga.nu/lists/listinfo/mud-dev</A>
</PRE>
<!--X-Body-of-Message-End-->
<!--X-MsgBody-End-->
<!--X-Follow-Ups-->
<HR>
<!--X-Follow-Ups-End-->
<!--X-References-->
<UL><LI><STRONG>References</STRONG>:
<UL>
<LI><STRONG><A NAME="00502" HREF="msg00502.html">[MUD-Dev] ColdStore. Belated response from a developer.</A></STRONG>
<UL><LI><EM>From:</EM> colin#field,medicine.adelaide.edu.au</LI></UL></LI>
<LI><STRONG><A NAME="00531" HREF="msg00531.html">Re: [MUD-Dev] ColdStore. Belated response from a developer.</A></STRONG>
<UL><LI><EM>From:</EM> Miroslav Silovic <silovic#zesoi,fer.hr></LI></UL></LI>
</UL></LI></UL>
<!--X-References-End-->
<!--X-BotPNI-->
<UL>
<LI>Prev by Date:
<STRONG><A HREF="msg00545.html">Re: [MUD-Dev] Simpson's "In-Game Economics of UO"</A></STRONG>
</LI>
<LI>Next by Date:
<STRONG><A HREF="msg00544.html">Re: [MUD-Dev] Alignment</A></STRONG>
</LI>
<LI>Prev by thread:
<STRONG><A HREF="msg00539.html">Re: [MUD-Dev] ColdStore. Belated response from a developer.</A></STRONG>
</LI>
<LI>Next by thread:
<STRONG><A HREF="msg00500.html">[MUD-Dev] shrink wrapped mud development kit (fwd)</A></STRONG>
</LI>
<LI>Index(es):
<UL>
<LI><A HREF="index.html#00546"><STRONG>Date</STRONG></A></LI>
<LI><A HREF="thread.html#00546"><STRONG>Thread</STRONG></A></LI>
</UL>
</LI>
</UL>
<!--X-BotPNI-End-->
<!--X-User-Footer-->
<!--X-User-Footer-End-->
<ul><li>Thread context:
<BLOCKQUOTE><UL>
<LI><strong><A NAME="00523" HREF="msg00523.html">[MUD-Dev] Game Spec</A></strong>,
Jman77777 <a href="mailto:Jman77777#aol,com">Jman77777#aol,com</a>, Sat 22 Apr 2000, 18:11 GMT
<LI><strong><A NAME="00502" HREF="msg00502.html">[MUD-Dev] ColdStore. Belated response from a developer.</A></strong>,
colin <a href="mailto:colin#field,medicine.adelaide.edu.au">colin#field,medicine.adelaide.edu.au</a>, Sat 22 Apr 2000, 04:13 GMT
<UL>
<LI><strong><A NAME="00531" HREF="msg00531.html">Re: [MUD-Dev] ColdStore. Belated response from a developer.</A></strong>,
Miroslav Silovic <a href="mailto:silovic#zesoi,fer.hr">silovic#zesoi,fer.hr</a>, Sun 23 Apr 2000, 21:46 GMT
<UL>
<LI><strong><A NAME="00539" HREF="msg00539.html">Re: [MUD-Dev] ColdStore. Belated response from a developer.</A></strong>,
Jay Carlson <a href="mailto:nop#mitre,org">nop#mitre,org</a>, Mon 24 Apr 2000, 01:20 GMT
</LI>
<LI><strong><A NAME="00546" HREF="msg00546.html">Re: [MUD-Dev] ColdStore. Belated response from a developer.</A></strong>,
colin <a href="mailto:colin#field,medicine.adelaide.edu.au">colin#field,medicine.adelaide.edu.au</a>, Mon 24 Apr 2000, 18:47 GMT
</LI>
</UL>
</LI>
</UL>
</LI>
<LI><strong><A NAME="00500" HREF="msg00500.html">[MUD-Dev] shrink wrapped mud development kit (fwd)</A></strong>,
J C Lawrence <a href="mailto:claw#cp,net">claw#cp,net</a>, Fri 21 Apr 2000, 01:52 GMT
<UL>
<LI><strong><A NAME="00508" HREF="msg00508.html">Re: [MUD-Dev] shrink wrapped mud development kit (fwd)</A></strong>,
Richard Ross <a href="mailto:rross#redhotant,com">rross#redhotant,com</a>, Sat 22 Apr 2000, 04:13 GMT
</LI>
<LI><strong><A NAME="00513" HREF="msg00513.html">Re: [MUD-Dev] shrink wrapped mud development kit (fwd)</A></strong>,
Charles <a href="mailto:caugusti#gladstone,uoregon.edu">caugusti#gladstone,uoregon.edu</a>, Sat 22 Apr 2000, 04:13 GMT
<UL>
<LI><strong><A NAME="00517" HREF="msg00517.html">Re: [MUD-Dev] shrink wrapped mud development kit (fwd)</A></strong>,
Matt Chatterley <a href="mailto:chattemp#ee,port.ac.uk">chattemp#ee,port.ac.uk</a>, Sat 22 Apr 2000, 18:11 GMT
</LI>
</UL>
</LI>
</UL>
</LI>
</UL></BLOCKQUOTE>
</ul>
<hr>
<center>
[ <a href="../">Other Periods</a>
| <a href="../../">Other mailing lists</a>
| <a href="/search.php3">Search</a>
]
</center>
<hr>
</body>
</html>