26 Sep, 2008, Silenus wrote in the 1st comment:
Votes: 0
I have been working on a new C variant language for mudding applications and just thought I would share and see if I could get some input. So far the syntax is untested since I am still working on building a parser generator in the language I am developing in (gambit scheme). Basically to make C coders at home the syntax will just take from C almost all the expression stuff (except maybe the comma operator) and statements as well. It will contain the standard LPC extensions of arrays and mappings as well. I hope to be able to use {} and [] for these instead of the slightly clumsy ({}), ([]) stuff.

The main differences from traditional C variants is that it will include a number of advance typing features typically seen in functional languages (however these features are not strictly functional). The features are:

tuples, union types, recursive types (via type names) & pattern matching.

I am still debating about what sorts of of OOP stuff to support if any (most likely I will support something) and what sort of module support to have. Mixin modules are capable of emulating classes and higher order stuff like type parameters and are quite appealing but I havent found a method to comfortably handle security in such a setting where data is more distinct from functions (though you can have functions as data).

Any thoughts? Is there space in this day in age for more C like languages for muds?
26 Sep, 2008, David Haley wrote in the 2nd comment:
Votes: 0
Out of curiosity, why? If the answer is for the learning experience, that's a fine reason. But are you trying to fill a void? Do you need features that languages around you don't see?

Writing a language is hard work, especially when you want advanced features like these. Getting the syntax isn't even half the battle, because then you need to worry about interpreting the code. (Hello, memory management and performance…) It's still interesting/fun if you like it, but if (a) you don't need to write your own language and/or (b) you aren't in it for the learning, I'd strongly consider using another language.

To answer your questions:

- emulating C as closely as possible is nice in some sense but can be misleading. Your language will obviously be quite different from C and even C++. Having a different syntax can remind people of that.

- OOP support is basically mandatory IMHO. I don't know what exactly you mean by handling security – there are many things you could mean. What is wrong with the approach taken by other languages that have these properties?

I'd take a look at Scala: it has the features you're looking at, and maybe you could get some inspiration from it.
26 Sep, 2008, Silenus wrote in the 3rd comment:
Votes: 0
Well the main reason is that I have been using LPC for a awhile now and looking at extending the language somewhat perhaps without full backwards compatibility. Basically I have been looking at the language from a typing perspective and trying to see if it is possible to impose a stronger typing framework on the language. I feel that LPC still (after having used it for sometime) doesnt quite do what I want in many cases and that certain features borrowed from the more advanced type systems of modern statically type checked functional programming languages might solve some of these problems for me in a type safe manner.

Unfortunately for the task I want to use it for there aren't many good modern choices. The alternatives given that I would like multiple users to be able to program the system while simultaneously logged in are rather limited. Rather than having to completely redesign the language I decided that using C for the subset of the language which behaves more or less exactly like C/LPC to remain mostly the same with just some slight modifications (primarily in type annotations and declarations for type safety). The only mechanism that is different is the pattern matching which has been incorporated into C like languages before. I am actually familiar with Scala and it perhaps on some level inspired me to embark on this somewhat strange project.

You are correct that syntax isn't even half the battle- but because of a lack of mature parsing tools under Gambit scheme I have sort of left this last. Actually because of how this project has evolved I have an IL already implemented which supports something simiar to a LPC subset and I have also partially implemented the semantic analysis. Strictly speaking it isnt really a compiler but more of a translator which I intend to build which maps the language into gambit scheme. I.e. no stuff that you find in compiler systems yet such as a low level CPS IL and manipulation (though I might add this later if performance turns out to be not so good). I will consider the project mostly a success if I can finish the translation layer and map all the features I would like to see into the IL and then onto scheme and get the syntax parsing phases correct.

The reason why I have been looking at omitting OOP in it's present form is that apparently mixin modules can emulate classes and thus do OOP in some fashion. However perhaps given that designing some like that and making it work takes quite a bit of effort- I might go the more conservative route and support something like MI perhaps with a slightly different syntax. By security I mean mostly preventing information leakage and ensuring data integrity. The main problem if you remove OO is how to decide which entities are actors and which entities are resources in the system. I have just been trying to understand the full implications of such a change.
26 Sep, 2008, David Haley wrote in the 4th comment:
Votes: 0
Are you sure that you can't take an existing language and build upon it? E.g. why does Scala not work for you? If it's just a syntax issue, you could always write a syntax translator or something like that. From past experience, I am personally very wary of setting out to design/implement a new language, so I would look very carefully at what's out there before rolling my own. Perhaps Scala is ruled out because you want people to be writing code in the language while logged in.

If you already have a VM to target and don't have to implement it, that makes your task a lot easier. That's definitely a good thing. :smile:

As for OOP, you don't necessarily have to implement actual classes and so forth. If you look at something like Lua, Self or Javascript, you'll see that a "class" is just a proxy object that responds to messages that the object doesn't know how to respond to. In other words, the object contains its instance variables; if a field (e.g. a method) is not found, it gets looked up in the parent. Inheritance can be implemented by chaining parents and mixins can be implemented by having a list of parents instead of a single parent.

Am I correct that by security you mean not letting objects see private data members in other objects? It seems that you could implement that in several ways without having objects and classes. You could return an object whose data members are stored in a closure, so that only the functions returned in the object have any access at all to the variables. There are several languages that implement their OOP paradigm by using closures, but unfortunately their names are escaping me at the moment…

I'm not sure I fully understand what you mean by needing to distinguish actors from resources. Why is this distinction important?
26 Sep, 2008, Silenus wrote in the 5th comment:
Votes: 0
The requirements of having the system as you indicated (writing code while in the system) makes alot of languages not so appropriate targets unless they support some sort of runtime loading system which can be secured properly. To prevent certain types of calls (i.e. to libraries or to the OS system) in many cases I think you would be forced to have a lang A -> lang A layer to trap the calls in an appropriate manner. So regardless I would have to write some sort of simple translation layer even if it's mostly mapping a language onto itself. Scheme is such a language but I think most mud coders would find the syntax abhorrent for mud programming which is why I have been thinking of designing my own (or really in a sense extending the LPC class or languages with some minor incompatibilities).

With OOP the problem is rather simple to solve. I.e. you can restrict data access in a number of ways i.e. by using something like private scopes and then checking various properties of requests by examining the call stack- like if the calling object should be able to make calls to the current object. In LPC you would further associate a "id" with each of these objects and use this in many cases to determine what privileges the object has. For example if an admin wants to write to a certain file in the system or modify an object in general the security system should always permit such an operation while if the average wizard wanted to do the same thing he/she should not be able to. (I think an example of using closures for data hiding purposes while maintaining local mutable state is any class system built on top of scheme or LISP like CLOS).

I have just been examining and trying to eliminate the possibility of an objectless solution to this problem using modules in the ML/Ocaml vein with the extended ability of having something akin to module inheritance. I think the correct term for this is mixin modules. The problem with this is that since you are no longer dispatching or have an implicit this argument to all calls. It is seems hard (so far for me) to associate privileges or capabilities with anything other than perhaps individual functions. These would be the "actors" in the system and the "resources" would be the variables/ADT's (which can also be functions obviously). I think this "works" but it might be quite foreign for most users.
26 Sep, 2008, quixadhal wrote in the 6th comment:
Votes: 0
One way to accomplish some of this would be to embed a language into your system and hand out different data windows depending on the security needed for the entity that will be using the embedded interpreter. So, you hand your admin an interpreter which has fairly transparent methods and most data elements that can be modified with any degree of safety. You hand your average wizard one that has fewer chunks of data exported to it, and that has restricted methods for things like open_file/etc.

Nothing wrong with writing your own language, but it IS as lot of work, so make sure you really WANT to do it before you get buried under it. :)
26 Sep, 2008, David Haley wrote in the 7th comment:
Votes: 0
Lua makes sandboxing extremely easy to implement. If you want to remove the ability to, say, write to files, just set io.write to nil and poof, that function doesn't exist anymore. Lua lets you load code at runtime. Lua does not implement static type checking, so you'd have to do all that yourself, which might be a deal breaker right there.

For the security issues, it might be worth taking a look at how Java implements its security model; it does a fair bit of stuff with giving limited permissions to untrusted objects.

As for the security model being foreign to users, it might not matter unless they're trying to do something they shouldn't. Ideally a call to a function would "just work" without having to do anything with security, as long as the needed permissions are present.

Quix's idea for having different interpreters is a good one, as long as you can easily share data across interpreters – that would be up to the implementation details of those interpreters.

I'm still not really convinced it's appropriate to be writing a language from scratch. I would need to see a list of key important features, and why they can't be accomplished using whatever means necessary in other languages. Lua is known for being very extensible, and you could (in principle) implement almost any of this stuff on top of it. Lua is not the only such language. It's just that you seem to be wanting to do an awful lot of things, with the end result being a fairly complicated language. I'd think long and hard if this is really what you want to be spending your time on, which would presumably depend on whether the benefits outweigh the costs.

Are you trying to write a language that scripts various aspects of the game, or a language in which to implement core game functionality? My understanding is that LPC games have their fundamental logic implemented in LPC. You might have very different problems on your hands depending on how you answer this question. If you can afford to have your core logic not be editable on the fly, things become a lot easier. While scripting really does want to be editable on the fly, I'm not convinced that the core logic should be. But, well, YMMV as always.
27 Sep, 2008, Silenus wrote in the 8th comment:
Votes: 0
I think having core logic editable in game though not a common feature is quite a boon and is the primary reason I am drawn to the LPC platform. It is also amazingly secure in the sense that you only have to trust the server application and dont require unsecure patching which you might have to manually verify and check if the modules supplied to you are hostile or not. If Lua allows for this sort of thing and includes a security model which allows for different users having different access in game I could probably use Lua and save myself a good deal of effort.

I have briefly read over the Lua specification and from your description it doesnt sound like it supports static compile time type checking. I know certain ppl can live without this "feature" i.e. lack of type annotations but to me given how large mud libraries tend to get if projects like Dead Souls, Lima, Discworld and Tublib (all LP projects) are anything to go by I might really miss it. As for the embedded interpreter idea the problem is still controlling the data windows and I suspect the safest way of doing so is to create a language translation layer which gives you full control which then becomes pretty close to what I am attempting.

Once one get's into static compile time type checking I feel it is perhaps appropriate to go the distance so to speak. I.e. include various powerful features from the advanced typechecking systems you see in languages like Haskell and ML which havent made it into the imperative languages yet. I.e. tuples (python has these), sum types, and pattern matching. I dont think a language with these features is any more complex than say modern Java 1.5+. In fact the specification is probably easier to write and maybe of equal complexity implementationally.

As for the security issue- Java relies on an object based type security model IIRC. The issue for me is is it possible to build a security model which lacks objects per se.

I admit that if the language was simple enough ala a Scheme/Lisp I might consider dropping static type checking- but for programming in the large it might be quite useful- but how to convince the average mud coder to use recursion instead of iteration for common tasks. I know it's more powerful but I think given how most coders are brought up still on imperative languages as opposed to those in the functional school this might mean an embedded Scheme like language driver might have a rather limited audience.
28 Sep, 2008, David Haley wrote in the 9th comment:
Votes: 0
Lua has no security model whatsoever. Their philosophy is to provide mechanisms with which you build your application. The mechanisms they provide are general enough to let you do almost whatever you want, assuming you're willing to put in the effort. That said, it is very easy to sandbox function environments or even the whole interpreter in Lua.

Indeed Lua does not have static-time type checking, but there are projects to add type annotations. I haven't followed them much so I'm not sure what the status is.

Re: language complexity: I fully agree! The thing though is that implementing Java 1.5+ from scratch would be quite the task, and so would be implementing something similar for a different language.

Re: security models. Presumably, yes, if you can control/annotate a function invocation environment. E.g., add extra data to the call stack that lets the interpreter know what this invocation is supposed to be able to do (or not do).

Recursion isn't really more powerful than iteration – you can implement recursion iteratively using a stack-like data structure – it's just that some things are more naturally expressed with recursion. Also, those languages provide iterative features. The real problem for the unfamiliar, I would think, is the completely different syntax.
28 Sep, 2008, Silenus wrote in the 10th comment:
Votes: 0
From your description it might mean that Lua might not work for me. The ability to at least install some sort of security model or have a built in one is quite important to me. As for language complexity. I suspect depending on how you implement the backend of the system constructing a language translation unit though non-trivial is not that difficult. A modern parser generator can eliminate alot of the grunt work and some like ANTLR can even generate the AST directly meaning you start your work at the semantic analysis phase. If you eliminate some of the backend and optimization phases the task of code generation is not so difficult.

Java per 1.5+ has become quite a large language so embedding it into a system as you not would be quite difficult but other languages with similar features are alot smaller (for example self). I think typing the mechanisms I have in mind are in general quite straightforward to implement the language is definitely no bigger or more complex than LPC. The runtime system is where you might have to concentrate your effort- but this might also be true even if you dont write a full language interface.

LISP does provide an iteration mechanism, but Scheme does not. It instead relies on a constant space tail recursion optimization which it mandates as part of the RSRn standard.
28 Sep, 2008, Kelvin wrote in the 11th comment:
Votes: 0
I'm afraid I'm going to have to echo some of the sentiments I've seen already here. Why another language when there are already so many that are well-suited to MU* development? Any potential gain for language stylistics that you might make with a new language would probably be offset by a lack of support, development, documentation, and community.

I'm not trying to be a downer, but this seems like it'd be one of those short-lived projects that won't make it past that "early development" stage.
28 Sep, 2008, David Haley wrote in the 12th comment:
Votes: 0
You can install a security model with Lua if you want to. You'll just have to do it yourself. That's the point of Lua in some sense: they give you bare bones, but they also give you mechanisms to build upon it. Frankly, though, I'd be more concerned about the static typing than the security model.

But I am very curious to know how this turns out for you. Please do post again as you make progress. :smile:
28 Sep, 2008, elanthis wrote in the 13th comment:
Votes: 0
JavaScript2 (still in development) gives you full dynamicism, a hardened security model, and optional static typing. Look at ActionScript3 to get an idea of how it works.

If you're going to implement a new language runtime, you're best off using a well-established language like JavaScript. Then your users have plenty of existing documentation and experience to build off of rather than having to learn something new from scratch.
28 Sep, 2008, kiasyn wrote in the 14th comment:
Votes: 0
Honestly, I don't believe any languages out there are truely suited to MUD scripting.
29 Sep, 2008, Silenus wrote in the 15th comment:
Votes: 0
Well my original proposal was basically a c variant + first class functions + algebraic data types + modules (& objects?). I dont think any existing language has this particular combination since in general imperative languages do not have pattern matching machinery. If I include objects it would resemble Ocaml. If Lua can do security i.e. have the ability to create security barriers around untrusted objects perhaps I could consider it.

However if I went this far, my preference might be for something like Scheme + classes + capabilities.

This would address to some extent elanthis's and Kelvin's comment that for potential users there might be a lack of adequate examples for how the language is properly used. It also address's kiasyn remark since you could trivially embed your own interpreter with a Scheme like syntax by writing an "almost" metacircular layer to handle scripting specifics since any language which needs to do core logic would have to be by it's very nature somewhat general purpose and perhaps ill-suited to scripting objects.

So far I have made some limited but good progress on this. Scheme -> Scheme would be a vast simplification for me it would mean that some of the code I have written (tree parser generator for AST walking) might not be needed since I am primarily doing this for type checking purposes. I will report later if I make more progress.

I haven't managed to find almost any information on JavaScript2. Most of the information I can locate is over 2 years old. Do you have a link?
29 Sep, 2008, David Haley wrote in the 16th comment:
Votes: 0
Why not Kayle? That seems like a pretty big statement… What would a MUD language have in particular?

Silenus said:
Well my original proposal was basically a c variant + first class functions + algebraic data types + modules (& objects?). I dont think any existing language has this particular combination since in general imperative languages do not have pattern matching machinery.

Scala has all that, if perhaps not exactly C syntax.
29 Sep, 2008, Kayle wrote in the 17th comment:
Votes: 0
I didn't say anything. That was Uh.. Kiasyn. >.>
29 Sep, 2008, Silenus wrote in the 18th comment:
Votes: 0
Well actually by modules I meant ML style mixin modules as per some research papers out there. It looks like to me from a cursory glance over the language specification Scala doesnt support these though you are correct that it supports pattern matching and algebraic data types.
29 Sep, 2008, David Haley wrote in the 19th comment:
Votes: 0
Oh, yeah, sorry Kayle, I shouldn't post when I'm so tired… :wink:

As for modules: I think you can achieve more or less the same thing with mixin classes and class methods.
29 Sep, 2008, quixadhal wrote in the 20th comment:
Votes: 0
Out of curiosity, are you talking about string pattern matching (via regular expressions)? If so, I wouldn't let that be a make-or-break decision for a language. While it's convenient to have it built-in, it's also not difficult to add. Even in C, you can use the pcre() library to import perl's regex system.

Also, on the idea of type checking… is there a reason to want a fully compiled language vs. a runtime language which can load and unload code at runtime? I ask, because if you don't insist on type checking everything all at once, there's no reason you couldn't use (or write) a tool to do more exhaustive checking than the language itself does, and simply use that on a module before it's loaded.