LPC

Speaking as someone working on one, I have zero doubt that a pure Python MUD running on commodity hardware can easily handle hundreds of simultaneous players. It would not shock me to see one handle thousands running under Linux + Epoll.

@David Haley:
I think you misunderstood. I attempted to make it quite clear that the "Hummer" wasn't the best at everything by referencing the fact that it would be harder to service (alluding to the code maintainability and ease of development issues that arise when using C and, to a lesser extent, C++). In short, I know that no one language is the best at everything, even for this particular problem set. What I'm more interested in is finding out if said dynamic languages will work "good enough" in all cases, where "good enough" simply means that the performance differential is unnoticeable to my end users, even if the number of those users were 10x what I'm realistically expecting. For comparison, Java will work good enough in all cases, even though I could gain marginal performance benefits with C. The performance of a completely Java-written MUD will be sufficiently indistinguishable from a completely (or piecemeal) C-written MUD from the user's point of view, regardless of how big the world is or how many players are in it. I just want to know if this is also true for languages I'm not familiar with, like Python, Perl, Lua, Ruby, etc.

@Barm:
Interesting. I'm assuming you're referring to vanilla CPython? And are you just making an educated guess, or have you had a good number of concurrent players on your MUD with which to judge it by? I'm liking what I'm reading about Python so far (minus indentation blocking… meh), especially its extensive libraries.

Edit: it's = its

If that's your question, then yes, I think it's been stated a few times that it should be fine, especially if you're careful when it matters and you profile if you have to. Your first versions might suffer from performance problems, but that's usually because dynamic languages in general encourage you to program without thinking about performance whereas occasionally you simply have to.

It's unlikely that you'd be doing something so intensive that having a C module would make much of a difference anyhow, but you never know. It's not an option you should remove from the table; a lot of libraries have C implementations and this is usually considered a good thing, and it will be basically transparent to the Python code anyhow.


One thing to point out is that the number of users isn't the only thing you need to worry about. How you model and process your world updates can also make a difference. Some games will keel over with just a few players if you make a sufficiently complex world with involved NPC AI.

This is very true, but I'm just looking for a baseline. If it can't be done at all, there's no need in worrying whether it can be done with a significantly sophisticated AI.

You've intrigued me, though. What do you consider sufficiently complex and involved? Are there any particular design patterns you would suggest staying away from for this reason?

It's hard to tell without concrete examples because so much stuff can be happening, and an awful lot of assumptions matter. A somewhat extreme example might be all NPC behavior deriving from goal-driven planning. If you simulate mobs way off in the middle of nowhere, without players nearby, thinking "oh I'm hungry let me go find a rabbit to kill", chances are that you will quickly overload your server with every orc figuring out what to eat for dinner. Now, you probably wouldn't do something like this if you were designing for scale in the first place, because you would do some combination of other things instead:
(a) not simulate world behavior if there are no players nearby
(b) simulate activity in aggregate, not on an individual basis. (E.g., if there are more orcs than rabbits, and orcs like to eat rabbits, then over time the number of rabbits decreases. Contrast with individual rabbits dying and not coming back, with individual orcs killing them.)
© simulate individual activity on a much simpler basis, with carefully crafted hard-coded behavior rather than involved goal planning

and so forth. Note that you could do a combination of these things, such as aggregate simulation when players aren't nearby, and individual simulation when players are nearby, etc.

Basically, when it comes to things like AI, the naive (read: straightforward) implementations tend to not scale very well. But this is already a relatively advanced topic for MUDs, so I'm not sure how applicable it is here.

It's worth noting that the complexity here is an asymptotic issue, and not something that C++ would be better at than e.g. Python. The speed difference between C++ and dynamic languages is not an asymptotic one, it's a constant-time factor, so if your algorithms are not well designed performance will suffer no matter the language. It's just that with C++ you might be able to get away with it for longer because your AI frames run in 50ms instead of 250ms (just making up numbers here).



As far as design patterns in code are concerned, you get similar themes. If your game is modeled such that every action by every actor requires a complex chain of generic events and event handling, you are likely to suffer as you scale. This is quite similar to the AI example, really, because if players aren't around to observe all this complex behavior, you probably shouldn't be doing it in the first place. (A theme here is that you shouldn't do work that players don't see.)

There are issues with building large world maps, where straightforward representations might work for small worlds but might fall flat on their face for large worlds. If you have a 10,000x10,000 world grid, and you represent every single spot on that grid in memory, you need ~100MB if you assume that every location is stored as a single byte; more realistically you'd need several hundred MB or even a gigabyte, depending on how much stuff you store, how many pointers each location has, and so forth. By contrast, if you only store "interesting" places and use some kind of tiling/compression system to reuse common landscape patterns, you can drastically reduce usage.

But again, it's hard to talk specifics because there are so many assumptions to make. Basically, these are the kinds of issues that cause scaling problems; if you want to Think Big you usually need to do so from the beginning or be willing to rewrite a good chunk of your code.

There's been some discussion of multiple inheritance. While I wouldn't go so far as to say that it's always wrong (except as implemented in C++ where it is truly evil!) it is also a feature which is much less useful than it initially appears. In cases where I find myself wanting access to multiple inheritance, roughly 95% of the time I eventually realise it's because I've cocked up the class hierarchy somewhere else. And it can be a nightmare to maintain.

Another question that's been raised is the performance of dynamic languages. The gap between them and C is often much narrower than you might imagine, especially in cases where the dynamic language is JIT-compiled.

For general programming I am a huge fan of Smalltalk - a language that some people on this forum have probably never even heard of. :sad: It is a very pure OO language. Some dialects are very fast (JIT compilers have been around for Smalltalk for a lot longer than they have for Java or .NET, and have become extremely powerful). It's dynamically-typed (which is a Good Thing, trust me) and intelligently garbage-collected. Being virtual machine based, some dialects are highly cross-platform, and will run on almost any OS or hardware with only a different underlying VM. It also has an extremely simple syntax, and is very highly modifiable; features that, in any other language, would be part of the language itself (e.g. for loops, behaviour of arrays and collections, conditionals), in Smalltalk are instead part of the class library and can be browsed and even rewritten - so you don't have the limitation that you have in (say) .NET of not being able to modify foundation classes. The class hierarchy is also much more sensibly designed than the .NET foundation classes (e.g. you don't have to deal with Lists and Arrays not being polymorphic, or with Streams and Stringbuilders being in different parts of the hierarchy despite doing the same thing). It's also very highly reflective.

If you are wedded to C-like syntax, then Ruby is definitely worth a look. Although the syntax is very different from Smalltalk, the underlying concepts (things like classes and metaclasses) are almost exactly the same.

I'm not sure I'd recommend Smalltalk for MUD purposes, though. MUDs pose some almost unique challenges, in that the behaviour of any given game object is likely to be much more dynamic than is the case in almost any other sort of problem. Consider, for example, a light spell that can be cast on any object and cause it to glow. You might have a LightEmitter class defined somewhere. If an object has had a light spell cast on it then, for the duration of the spell, that object becomesa LightEmitter. This is a classic OO "is-a" relationship, which suggests inheritance; but it isn't the normal kind of inheritance. If you cast a light spell on a sword (an instance of a Sword class), that doesn't mean that the entire Sword class inherits from LightEmitter; only that one specific instance of Sword needs to inherit from it, while all other instances don't. What's more, the inheritance relationship is dynamic over time - this object should inherit from LightEmitter for the duration of the spell, but cease to do so after it lapses.

(You could, of course, define a whole bunch of behaviours to do with emitting light or not up in the ultimate game-object superclass; but that would be horribly bad encapsulation, as 100% of the objects in the game would have to be aware of LightEmitter code, while less than 1% are actually using it at any given moment. Adopting the same approach for many other magical effects would eventually lead to an appallingly complex and bloated GameObject superclass.)

So, to handle this, you need some way of managing inheritance-like behaviour but in a way that is dynamic, and controllable object-by-object rather than class by class. Up to a point you can simulate this by means of composition (in fact you can do it quite nicely in Smalltalk or Ruby by using proxies) but ideally I think you need some kind of language which does this stuff in a more "native" way, perhaps by working in prototypes rather than classes.

One language that I think you should definitely look into is Self.

You should also check out languages that were specifically created for writing MUDs. Cratylus already suggested LPC; ColdC is worth a look too.

Do you say Ruby and not, e.g., Python, because you think that Python doesn't have those features, or because you don't know enough about it to make a claim? (Curious, because Python has pretty developed notions of classes and metaclasses as well.)

Despite being overlooked in the "best language from scratch" discussion, LPC is obviously a good choice if you're more interested in coding your game, rather than the engine it runs on.

For those who don't know, LPC takes a totally different approach from most of the Dikuesque games out there. A Diku implements all game logic in C (or whatever base language you use), and may provide hooks to run tiny scripts that are written in another language, and which can react to specific events. Recently, embedding Lua has become a popular way to accomplish this. The core game logic is still written in the host language.

An LpMUD implements a game driver which handles network I/O, file I/O, and provides a language interpreter which runs code in response to I/O events or timed events. The code in this interpreter is sandboxed, so it is resource controlled and any error in one section of code won't cause cascading errors in unrelated sections. It is also dynamically loaded and updated. The entire game system is written in LPC, meaning it can be changed on the fly by anyone with permission to do so.

The learning curve is a bit steeper, because you have to not only learn the LPC language, but you have to learn the driver/mudlib API and design your system to work within that framework. If you don't mind using an existing system, there are several public mudlibs which can be used as a starting point.

From what I've seen, LPC can be made to do anything you want, provided you are willing to work within certain constraints. Long running computations will have to either be broken into segments so they don't exceed your resource limits, or coded as external-functions that are compiled into the game driver.

I'm in a similar boat to the OP… I have an antique DikuMUD, and I originally looked at a bunch of ways to move it forward. Because I don't have a player base, I eventually decided to make a new game, but I had done some work into converting the data systems from Diku to LPC, and that's still a viable idea.

So, that's perhaps the first question to answer…. do you want to write a game, or do you want to write a game driver?

@shasarak:
Thanks for the interesting post. You touched on a lot of interesting stuff and your example made me rethink my design requirements. Self was… interesting to read about, but I don't think I could write an entire game in it. I don't want to use languages like LPC or ColdC, because I'd like to write the entire game myself, as an exercise in futility creativity.

I hadn't heard of proxies in Ruby before, and at first look, they seem to solve the same problems as modules do, only in a much more obfuscated way? Maybe I'm missing the point, though. Admittedly, I've never written anything in Ruby.

I've already written both, though admittedly, the game is a little on the skinny side ;) I just want to move it forward by moving towards an OO design. Legacy C is so frustrating to maintain.

The problem I have with things like LPC, and the reason I won't consider them, is not because I think they aren't great at what they do (they are), but rather, because they allow you to do whatever you want to do, but (usually) limit you to doing it one particular way. This is the inherent trade-off for specificity. It's easier to use, but not as flexible. I think it's one step too high on the low-to-high-level language scale.

I think "never used before" might be a more accurate statement. I cant think of a single programming book I've read
that hasn't mentioned it (Ruby, Python, Objective-C, ect)

I've played around with Ruby as a project, and I LOVE IT. But I discovered something crazy about myself in the process.
I don't like RAD. I am weird in a way that I like C, I like the hurdles and the challenges of it. I guess because C was very
challenging to learn, and because I grew up playing Diku muds. I'm probably one of the only people that are happier this
way. I love old technology though, in fact, i want to get a 386 and run my mud in C99 standard, just to see if I can do it!

Them's fighting words. :p

I think he didn't use python because it doesn't have c-like syntax.

I'm simply not familiar enough with Python to be able to make any educated comments about it. :sad:

I've never actually done any serious programming in Ruby either, to be perfectly honest, but I have done a fair amount of reading on it by way of research into whether it might be suitable for a particular project, and attended a few lectures about it at conferences. Certainly Ruby really is effectively identical to Smalltalk under the surface (with the single exception of limited support for "mix-in" inheritance in the form of modules). The programming style is rather different, of course, in that Ruby syntax is much more complex than Smalltalk's: in Smalltalk you are always explicitly invoking methods on objects, but in Ruby there is a lot of "syntactic sugar", i.e. ways of writing things which actually are method invocations under the surface but which look as if they aren't, and which you have to learn as language features. Personally I prefer the Smalltalk approach, I think it's far easier to figure out what the code is doing and I like being to look up language features by browsing the class hierarchy. But since most Smalltalk IDEs and compilers are actually written in Smalltalk, and modifiable, it's not hard to add syntactic sugar to a Smalltalk environment if you want to.
:smile:


I said some people, not all. :smile:

This hasn't been a problem in my experiences. In other words, I've gotten more done and learned the language faster. I think you overstate the case that this causes any problem at all for most purposes that are appropriate for a high level language. I wrote with smalltalk as a toy language at university and I can safely say the two have some things in common, but from a presentation to the person actually using the language—the syntax is much different. More different than Ruby vs Python; And under the hood as well.

Proxies have far wider-ranging applications than that. They're useful under any circumstances when it is useful to intercept a method invocation and do some additional processing either before or after it.

So, for example, you could use proxies as a performance profiling tool. Wrap an object up in a proxy, then any message which comes in can be relayed on to the main object, but the proxy can do some logging and recording of timestamps either side of that; this gives you a good measurement of how much time it takes to complete every method invocation on the main object. Or, of course, you could do this only for certain methods, and relay others without any logging.

The Smalltalk application I used to work on (and occasionally still do) uses proxies as part of the object-relational mapper. The dataset it works on is quite large, so you don't want to load the entire database into memory at one time, but you still want an object that represents a database record to have its instance variables pointing directly to other objects representing other records. This is a problem: if one object references another, which references another, and so on, how do you avoid loading the entire DB into memory?

The way this is done is that each DB object is wrapped up in a proxy. So, if you have an object representing (say) a department in an organisation, that bbject has a reference to a parent department (its parentDepartment instance varoable). When the child-department object is being instantiated (from the corresponding DB record), the record contains a foreign key to the parent department. The application looks up that key in a central list of departments (which maps keys to proxies via a weak reference - more on that in a moment). What is returned from that list is always a proxy. If the parent department object has already been loaded into cache, the proxy will contain that; if it hasn't then the proxy will instead contain a very compact representation of a database query which, if executed, will return the data needed to create the parent department object.

The child department's parentDepartment property therefore points to the proxy. If we suppose that the parent dept hasn't yet been fetched into cache, this proxy can continue wrapping up a query for quite some time. At the point when a message is first sent to the parent dept, the proxy intercepts that, fetches the data necessary to create the parent department object, disposes of the wrapped-up query, points itself to the newly-created department object instead, and then relays the message. On subsequent message sends, the message is relayed directly to the department object, without any data fetch.

The proxy also tracks how often it has been referenced. If memory is running low and we need to free up some space, the proxies that are accessed least frequently are told to convert their real objects back into compact queries again. Since the proxy holds the only strong reference to the fully instantiated DB object, the real object will then be garbage-collected. That in turn means that the strong references from that real object to other proxies are removed; and since the central look-up table of keys and proxies uses weak references, many other proxies end up getting garbage collected as well.

The advantage of using proxies here is that only the proxy itself knows whether what it contains is a fully instantiated DB object, or a wrapped-up query; everything else references the proxy, and doesn't need to know what state it is in.

If you are mad enough to not only create your own codebase but actually invent your own scripting language to go with it (so that your MUD driver is acting as a virtual machine for your custom language, in the way that an LP driver does for LPC) then I think there's a lot to be said for a Smalltalk-like approach; not necessarily in terms of syntax, but in terms of language features being implemented as part of the class library. This dramatically reduces the complexity of the compiler. Pretty much all the compiler needs to be able to handle is declaring local variables, assigning variables, referencing instance or global variables, order of evaluation, the basic syntax of a message send, and the invocation of VM primitives. Every other feature of the language can then be added by modifying the classes.

So, for example, the compiler doesn't need to know about the existence of conditionals. Instead you have classes True and False whose singleton instances correspond to boolean values true and false. On class True there is method ifTrue: which takes a block argument, and simply evaluates that block. On class False the ifTrue: method does nothing. On class Integer there might be a method ">" which returns a boolean value (via a VM primitive).

So, in a C-like language, if you have an expression like:

if (x > 0) { //do something }

then the compiler has to handle the "if" construction. The equivalent in Smalltalk is all message-based:

x > 0 ifTrue: ["do something"]

Here you are sending the message > to x with argument 0; that returns a boolean. You then send the boolean the message ifTrue: with a block of code as an argument. The boolean will then either tell the block of code to evaluate itself, or do nothing, depending on whether it is a True or a False.

Similarly, for-loops are implemented on the Number class by using a "while" construction. The Smalltalk syntax is something like:

1 to: 10 do: [:z| Transcript cr; show: z asString]

On the Number class there is a to:do: method defined, which looks like this:

to: aNumber do: aBlock

    ^self to: aNumber by: 1 do: aBlock

So that invokes the to:by:do method, which looks like this:

to: aNumber by: interval do: aBlock

    | current |

    current := self.
    [current = aNumber] whileFalse: 
        [aBlock value: current.
        current := current + interval]

whileFalse: is a method defined on a block, which uses a VM primitive - evaluate the second block until the result of evaluating the first one is true. Similarly, value: is a method defined on a block which uses a VM primitive meaning "evaluate this one-argument block using the argument passed into the value: method as the block's argument".

(Actually the for-loop example is a little more complex than that in real life, as you have to check for silly arguments being passed in, and allow for the numbers and interval being negative, etc. but the principle is the same).

As time passes, of course, you can make the compiler more intelligent so that to:do: loops are actually inlined rather than evaluated as formal message sends; but this is a performance optimisation, not something it needs to do from the word go. Handling things in this way is a good way of getting off the ground, I think.

@shasarak:
Thank you for that explanation. Proxies make more sense to me now, and indeed, they look very useful.

I'm really rather torn between Python and Ruby now. If it comes down to which language I like better on the surface, Ruby wins hands down, because I think indentation blocking is quite a cludge, and the everything-as-an-object-even-literals approach appeals to me quite a bit. I like that Python's libraries seem far more advanced and encompassing, but I'm not trying to write code to run the world here, so I'm not sure it makes much of a difference within the confines of my project.

How are Ruby's collection libraries? I don't really want to be writing my own hash sets and such.

FWIW the proxy approach can be implemented basically the same way in Python.

The indentation thing bothered me at first but I got used to it pretty quickly. Some people feel the same about Ruby's use of sigils everywhere. :shrug: :wink: