12 Mar, 2009, elanthis wrote in the 61st comment:
Votes: 0
Tyche said:
Would it be fair to expect that in particular ZMP, MXP, MCP and Pueblo cannot all be active at the same time?


There is no technical reason why they can't be that I can think. They all use different-enough "envelopes" that they shouldn't conflict on the wire.

Quote
What about MSP, MSP2, MCCP2, other Telnet options?


I implement MCCP2 and various other TELNET and ANSI control codes along with ZMP with no problem. Totally orthogonal.

Quote
I thought these might be rare cases, but with Mushclient being open source and all, it certainly becomes possible to have all the aforementioned protocols implemented in a single client.


It has a lot to do with the server, too. If a server just blindly tries to start using all the protocols at once, bad and weird things may happen where they overlap. Were I to add MCP support to SourceMUD, for example, I would attempt to enable ZMP first, and if that fails, then attempt to enable MCP. I would never try to enable both simultaneously.

Quote
Is there any value to an authentication/session key core package negotiation in ZMP, like MCP does?


No, because such a feature is useless in ZMP.

<long-winded-explanation>

It's fairly useless in MCP, for that matter. If you are trying to use it as protection against man-in-the-middle style message injection attacks than it offers zero real protocol as the snooper can just get the key. I believe the real purpose of the session key in MCP is to provide a "secret" in each message so a malicious player cannot inject MCP commands into any input that gets sent to other players on a server with MCP parsing bugs. ZMP relies on the fact that commands are TELNET subrequests and so should not ever be spoofable by a client, because I naively assumed that all MUDs had proper TELNET handlers. I was obviously mistaken. I believe it is easier to have MUDs fix their TELNET handling than it is to ask them to implement key negotiation. If nothing else, all a MUD has to do is strip the IAC byte out of all player input and it becomes impossible to inject malicious ZMP commands into other players' connections. Compare to MCP, where a MUD would at the very least have to strip #$# from all player input (that isn't part of an MCP message itself) and even then might be vulnerable if the MUD ever prints # or #$ following by arbitrary player input (unlikely, but possible… especially if it happens to concatenate input received at different times). And compare to MXP which uses modes to determine if commands have any meaning or not, but which is likely going to run afoul when a developer wants to make marked-up text with player-provided text, for example when printing a "Greg said blah" line with marked-up player names or even highlighted player speech keywords.

It's worth nothing that if you make it possible for players to inject ZMP commands into the stream that you also are allowing player to inject other arbitrary TELNET commands into the stream, so a player could just as easily be an ass and trick a client into turning MCCP2 on or off without the MUD playing along, or freeze the terminal, or do any number of other bad things.

MUD servers also need to strip out byte 27 to avoid ANSI code injections, especially given that some ANSI codes can easily render a terminal unusable or crash some terminals outright. The terminal title string feature has a history of crashing a lot of popular terminal emulators when a long or unterminated title is provided.

I usually just escape things at output time (255 is after all a completely valid character code in most 8-bit encodings – it's y-umlaut in ISO-8859-1 for example), generally by having a small set of functions for adding data to the output buffer. MUD developers used to using sprintf or the like to mix in a ton of TELNET, ANSI, and text certainly need to completely sanitize data at input time, but my approach just requires me to use functions like:

connection->send_iac(EOR); // sends IAC <byte>
connection->send_negotiate(WILL, ECHO); // sends IAC <type> <opt>
connection->send_color(COLOR_RED); // sends \e

No, because such a feature is useless in ZMP.

<long-winded-explanation>

It's fairly useless in MCP, for that matter. If you are trying to use it as protection against man-in-the-middle style message injection attacks than it offers zero real protocol as the snooper can just get the key. I believe the real purpose of the session key in MCP is to provide a "secret" in each message so a malicious player cannot inject MCP commands into any input that gets sent to other players on a server with MCP parsing bugs. ZMP relies on the fact that commands are TELNET subrequests and so should not ever be spoofable by a client, because I naively assumed that all MUDs had proper TELNET handlers. I was obviously mistaken. I believe it is easier to have MUDs fix their TELNET handling than it is to ask them to implement key negotiation. If nothing else, all a MUD has to do is strip the IAC byte out of all player input and it becomes impossible to inject malicious ZMP commands into other players' connections. Compare to MCP, where a MUD would at the very least have to strip #$# from all player input (that isn't part of an MCP message itself) and even then might be vulnerable if the MUD ever prints # or #$ following by arbitrary player input (unlikely, but possible… especially if it happens to concatenate input received at different times). And compare to MXP which uses modes to determine if commands have any meaning or not, but which is likely going to run afoul when a developer wants to make marked-up text with player-provided text, for example when printing a "Greg said blah" line with marked-up player names or even highlighted player speech keywords.

It's worth nothing that if you make it possible for players to inject ZMP commands into the stream that you also are allowing player to inject other arbitrary TELNET commands into the stream, so a player could just as easily be an ass and trick a client into turning MCCP2 on or off without the MUD playing along, or freeze the terminal, or do any number of other bad things.

MUD servers also need to strip out byte 27 to avoid ANSI code injections, especially given that some ANSI codes can easily render a terminal unusable or crash some terminals outright. The terminal title string feature has a history of crashing a lot of popular terminal emulators when a long or unterminated title is provided.

I usually just escape things at output time (255 is after all a completely valid character code in most 8-bit encodings – it's y-umlaut in ISO-8859-1 for example), generally by having a small set of functions for adding data to the output buffer. MUD developers used to using sprintf or the like to mix in a ton of TELNET, ANSI, and text certainly need to completely sanitize data at input time, but my approach just requires me to use functions like:

connection->send_iac(EOR); // sends IAC <byte>
connection->send_negotiate(WILL, ECHO); // sends IAC <type> <opt>
connection->send_color(COLOR_RED); // sends \e[<color>m
connection->send_zmp(argc, argv); // sends IAC SB ZMP, then each string argument terminated with NUL and with any IAC escaped and NUL stripped, and then IAC SE
connection->send_text("foo bar blagh"); // sends text with IAC escaped to IAC IAC, \e stripped (no way to escape <Esc>, ironically enough), and \n replaced with \r\n

Since the only two places that player-provided input are ever put in the output buffer are both solid and safe, I never ever have to worry about a player injecting anything. I _also_ strip out \e and NUL at input time since they have no valid use. Double the protection is double-mint good!

But yeah, simplistic server code with broken TELNET handling need only strip IAC bytes from player input and ZMP has no need for extra special secret handshake stuff to protect against malicious input.

</long-winded-explanation>
12 Mar, 2009, Tyche wrote in the 62nd comment:
Votes: 0
David Haley said:
Tyche said:
Would it be fair to expect that in particular ZMP, MXP, MCP and Pueblo cannot all be active at the same time?

I guess so, although it would more be a question of "why would the MUD do that?" rather than "must the MUD not do it?".


It's also a question of would the client do it? Mushclient supports Pueblo and MXP. I'm not entirely sure if the client both of them can be on at the same time as I haven't checked.

David Haley said:
Tyche said:
Is there any value to an authentication/session key core package negotiation in ZMP, like MCP does?

I'm not familiar with what MCP does. But being able to auto-send user/password/whatever on request is a nifty feature of MXP that I think ZMP should have.



Not that, an authentication mechanism, although that also might also be a useful package.

What I meant was when MCP is turned on it hands the client an authentication key which is active for the duration of the session. So it's more akin to a session id, like a web cookie. All messages issued from the server contain that id, so a MCP client ignores messages that don't have the authentication key. The notion is to prevent (or reduce the opportunity) of other clients connected to the server spoofing MCP messages. Since ZMP is a two way protocol, the same sort of spoofing might be possible. Then again, it might not.
12 Mar, 2009, Nick Gammon wrote in the 63rd comment:
Votes: 0
David Haley drew my attention to this thread today, so I'll add some comments if you don't mind, based on my experiences with adding some telnet support to MUSHclient, and for Aardwolf.

elanthis said:
What, are they using NUL-terminated buffers instead of keeping a separate buffer length variable?


MUSHclient handles NULs OK, as MCCP input itself will contain NULs. However in text output they are thrown away in the interpretation-of-ANSI stage. However many of the script routines, based as they were on 'const char *' arguments, will not correctly handle strings with imbedded NULs. Lua, however does, and certain script functions were re-implemented in Lua to provide appropriate support (eg. data compression).

elanthis said:
Context menues would be similar to what MXP has: just a way of "marking up" text so that it can be clicked and a menu pops up, and when an item is clicked it sends a command to a server.


I have long been of the opinion that if you are going to mark up text you should mark up the semantics and not the look. In other words, it is much more helpful to a client to say "here is my health, mana and movement points - display them how and where you like", rather than "please display a box 20 x 30 pixels, draw 3 bars in it, and colour them in a certain way".

IMHO this is where MXP fell down - too much of it was concentrated on the colour, boldness, italic, and indeed position of the text.

David Haley said:
My pipe dream is that the MUD sends a semantically rich description of what is going on, and the client renders it as text, images, whatever. Of course this is probably all but impossible in practice if you want to keep it general; it could be done for one game at a time I think.


As usual, I agree with David. :)

Indeed, he expresses what I was trying to say above.

elanthis said:
Most non-toy scripting languages I have seen do not rely on the NUL byte.


I agree, and Lua, which MUSHclient supports now, does not rely on a NUL byte. Again, however, I am hampered by years-old design decisions that let things (eg. client variables) be stored in such a way that the NUL would terminate them.

Anyway, why get hung up on the NUL byte? Can't strings be terminated in some other way in the ZMP protocol?

elanthis said:
Should subwindows be allowed to have dedicated input lines?


From experience, multiple input lines are a pain. Then you have the worry of which one has the focus. And if you press Ctrl+V to paste, which one you are pasting into, and so on.


elanthis said:
Should the server get notified if a user explicitly closes a subwindow?


My view on this is that the server provides information, whether or not the player looks at it is irrelevant. That is like a TV station wanting to be notified if you stop watching a program.

More follows …

Vassi said:
To take the inventory and map example again, closing the windows means that I want it to show in-line instead of separately.


No, it means the player doesn't want to see the inventory/map. If I have a small screen, or simply dislike clutter, then closing an inventory window doesn't mean I want to see it somewhere else. I suppose you could reply "but what if I type 'i', then I want to see it!".

This goes back to the server providing the information (eg. your current inventory) and the client displaying it where, and when, the player wants to. The same for room descriptions, room names, etc. This stuff should be downloaded when it changes (eg. you pick something up, or change rooms). However it can be updated behind the scenes. Then when the player hits an "inventory hotkey" the inventory appears - whether inline, or in a little window with tabs, or a scrolling window, that is an implementation detail.

For example, MUSHclient's miniwindows allow you to update them, even if they are not being drawn. This is entirely based on this idea. For example, a window that displays your room description (exits/inventory) can go ahead and update its contents, however the player may have chosen not to view it.

elanthis said:
Well, assuming you guys do want a terminal control package, what exactly would it need besides the existing color stuff and screen clearing? Is cursor position or line clearing at all useful once you have subwindows, graphical widgets, dedicated prompt display areas, etc.?


My personal view, as you can see, is to not bother with cursor positioning. Have the protocol describe information. Leave its presentation as a client implementation detail.



I don't totally know why all this has to be done by a telnet protocol in the first place. Why not just have lines of information, preferably described with keywords for flexibility and future expansion? Then you can stop worrying about NUL bytes, and interpreting funny sequences like IAC IAC and all that stuff.

As a partial example of what I am talking about, consider the "stats" line that Aardwolf puts out, if required:

{stats}15/11,29/25,41/30,22/15,22/15,24/21


Clients can make a trigger to match that, and then omit it, so the player doesn't see it. The only reason we used telnet on Aardwolf was to handle the tricky situation of conveying information from client to server (in particular) when the player had not necessarily logged in. Thus a command like "show me the {stats} line" in whatever format it might appear, could be mistaken for the player name or password.

However if I was redesigning the stats line, I would use keywords, so it would be more like this:

{stats}hp=15,maxhp=11,mana=29,maxmana=25,move=41,maxmove=30  …


That way you can add new keywords later on without breaking existing code that might be expecting only a certain number of comma-delimited fields.
12 Mar, 2009, Tyche wrote in the 64th comment:
Votes: 0
Ahh… okay some things are falling into place.

I'm looking over some old code and it looks like we used to try Pueblo first, then attempt MXP.
The reason was Pueblo features were better developed on the mud.

As a general rule, while a server could support all of them, I need configurable options in my library that allows a mud server to both disable negotiation completely for unsupported protocols AND a sort of priority scheme of which to negotiate first if many are supported. No matter. ;-)
12 Mar, 2009, Tyche wrote in the 65th comment:
Votes: 0
I had this issue with TeensyMUD and was wondering how others solved it.
How much time does one allow for non-response or letting the negotiation settle down before sending the welcome screen?

I defined an arbitrary counter for each pass of the i/o loop. When I hit 20, I start sending welcome screen and processing in-band communications. Note I still continue negotiation, but the notion here is that I ought to have enough info to run the welcome screen through whatever filters we've agreed on thus far. This doesn't seem an altogether satisfactory solution.
12 Mar, 2009, Vassi wrote in the 66th comment:
Votes: 0
NickGammon said:
I don't totally know why all this has to be done by a telnet protocol in the first place. Why not just have lines of information, preferably described with keywords for flexibility and future expansion? Then you can stop worrying about NUL bytes, and interpreting funny sequences like IAC IAC and all that stuff.

As a partial example of what I am talking about, consider the "stats" line that Aardwolf puts out, if required:

{stats}15/11,29/25,41/30,22/15,22/15,24/21


Clients can make a trigger to match that, and then omit it, so the player doesn't see it. The only reason we used telnet on Aardwolf was to handle the tricky situation of conveying information from client to server (in particular) when the player had not necessarily logged in. Thus a command like "show me the {stats} line" in whatever format it might appear, could be mistaken for the player name or password.

However if I was redesigning the stats line, I would use keywords, so it would be more like this:

{stats}hp=15,maxhp=11,mana=29,maxmana=25,move=41,maxmove=30  …


That way you can add new keywords later on without breaking existing code that might be expecting only a certain number of comma-delimited fields.


I knew a Front End that did that. You enabled 'wizard mode' and then it would periodically spit out a line like:

ABCDEF000010000200003999900002100030

And the client would gag it, then parse it for input. It worked perfect with their custom client, but I had an issue with it for various reasons. Number one, their client sucked. It was ugly and ungainly, the few little widgets and gauges they provided were not good enough to put up with it. So I tried ZMud. I used it to gag the same line and painstakingly figured out how the above line was broken up and what each bit actually meant. At one point - to illustrate my insanity - I used DDE to communicate with mIRC because mIRC scripting had an easy way to convert HEX into decimal and I couldn't figure out how to do it in ZMud. After all that I had working gauges in ZMUD. A little later, someone made a plugin for ZMud that did what I had done, except it only worked if you had the commercial version of the client.

The moral of the story is that I had to manually configure the client I moved to with triggers. You and David have both mentioned the 'one game at a time' thing, but that places the onus on the users. I'm looking at ZMP, and the clients that implement the subsequent packages, to avoid all of that.

To use the Aardwolf client as an example, you released a special download just so that everything was already pre-configured. Are you prepared to do the same for every popular MUD? Will they have to package a download themselves? Wouldn't it be better if Aardwolf could talk to MUSH and configure it upon connect, no extra steps involved. Thats kind of my 'vision' for ZMP.

Edit: I'm not trying to diminish the work at all, because of mini-windows I actually have Aardwolf a try and had a bit of fun doing so. The problem with parsing text is that you will run into an edgecase somewhere - the way you described with needing to communicate something back to the server - which will necessitate a Telnet link of some kind anyway. Why not just start there?
12 Mar, 2009, elanthis wrote in the 67th comment:
Votes: 0
Nick Gammon said:
I agree, and Lua, which MUSHclient supports now, does not rely on a NUL byte. Again, however, I am hampered by years-old design decisions that let things (eg. client variables) be stored in such a way that the NUL would terminate them.

Anyway, why get hung up on the NUL byte? Can't strings be terminated in some other way in the ZMP protocol?


We don't use NUL bytes for strings so much as we do as a separator between arguments. If you put the core ZMP handler in the core C code for the client (it is about 8 lines of code of simple code) you avoid the entire issue. Or if you do pass the raw subrequest data to a script language to do the parsing (what little there is), just pass it to Lua.

A ZMP subrequest looks like this:

IAC SB ZMP "command" NUL "arg1" NUL "arg2" NUL "…" NUL IAC SE

The TELNET code I see usually creates a byte buffer for subrequests, and then calls a routine to process the buffer once the IAC SE bit is read. Most clients only look for the MCCP2 code as of right, some look for environ and the like. All the code really needs to do (simplest case) is:

/* variables we need */
char* argv[32];
char* c = request_buffer;
size_t argc;

/* make sure buffer is not empty and does not end with a NUL */
if (request_buffer_size == 0 || request_buffer[request_buffer_size - 1] == 0)
return error_code;

/* parse args */
for (argc = 0; argc != 32 && c != request_buffer + request_buffer_size + 1; ++argc) {
argv[argc] = c;
c += strlen© + 1;
}

/* send the argument list off to your script language */
invoke_script_function("zmp_handler", argc, argv);


The loop could be made a little more robust looking, but honestly, the NUL byte check and that loop are completely safe. I usually make it look more robust though just to feel warm and fuzzy, which might expand the code out to like 14 lines.

Quote
From experience, multiple input lines are a pain. Then you have the worry of which one has the focus. And if you press Ctrl+V to paste, which one you are pasting into, and so on.


Every GUI toolkit makes that dead easy. If you're writing your GUI widgets from hand it could be a bit of a pain, I suppose.

Quote
I don't totally know why all this has to be done by a telnet protocol in the first place. Why not just have lines of information, preferably described with keywords for flexibility and future expansion? Then you can stop worrying about NUL bytes, and interpreting funny sequences like IAC IAC and all that stuff.

As a partial example of what I am talking about, consider the "stats" line that Aardwolf puts out, if required:

{stats}15/11,29/25,41/30,22/15,22/15,24/21


Because you still have to worry about funny stuff. What happens if a player types in:

say {stats}0/11, 0/25, 0/30, 0/15, 0/15, 0/21

Maybe that doesn't work because the resulting "Foo said" text would leave {stats} off the start of the line, but are you sure there's no other ways for players to get text to show up at the start of a line? Do you just strip { out of all input?

If it's line based, how do you deal with any kind of informational markup, like clickable text? I mean, take this:

One the ground you see \r\n
{action}get get sword, look look sword\r\n
a long sword\r\n
{/action}\r\n
.\r\n

We wouldn't want the line markers on the two sentence fragments to actually inject newlines into the output, but we do want it for the newline after the period.

A line based protocol embedded in a free-form textural protocol like TELNET is a bitch to parse. Hence why I wrote ZMP as a TELNET-based protocol.

To be honest, I'd rather just toss TELNET out the window entirely and go with a pure line-based protocol (which still has to have funny commands for dealing with newlines, but at least then the rules would be clearly defined and logical).

Tyche said:
I had this issue with TeensyMUD and was wondering how others solved it.
How much time does one allow for non-response or letting the negotiation settle down before sending the welcome screen?


I personally just don't bother. I don't make the welcome screen rely on anything fancy.

I have been considering adding an approximately 5 second delay, both to allow for negotiation and for an ugly hacky hacktastic hackish hack – allowing a single port to serve both TELNET and HTTP. If an HTTP request line comes in within those 5 seconds, the connection is treated as HTTP, and otherwise it falls back to TELNET. Would fail whenever a connection opens and then lag occurs before the first real data arrives (or if you're TELNETing to test the HTTP portion and aren't typing fast enough…), but it would make it possible for MUDs stuck on host providers that only allow a single port to get XML-RPC/JSON-RPC integration with their website. I'd default to the feature off and recommend using a second port, naturally) Like I said, hackity hack hack.
12 Mar, 2009, David Haley wrote in the 68th comment:
Votes: 0
Vassi said:
I'm looking at ZMP, and the clients that implement the subsequent packages, to avoid all of that.

I don't think it's a silver bullet, although any kind of standard is a start. But to illustrate that point, note how many times in just this thread we've had to shrug and say that MUDs will have to implement their own packages if they need specific behavior, and then get clients to implement it.

Vassi said:
To use the Aardwolf client as an example, you released a special download just so that everything was already pre-configured. Are you prepared to do the same for every popular MUD? Will they have to package a download themselves?

Well, to be fair, Nick released MUSHclient as open source, so it's not that hard really to package it.
Vassi said:
Wouldn't it be better if Aardwolf could talk to MUSH and configure it upon connect, no extra steps involved. Thats kind of my 'vision' for ZMP.

See, this is what I mean when I say that I think ZMP is being used to mean several things. As it is, ZMP is a transport protocol that happens to let you ask if the other side understands a given message. There are no semantics beyond that.

Yes, it would be wonderful if you could configure the client. Heck, you could conceivably set it up so that it sends Lua code down the line to MUSHclient which would then put it into a plugin and be ready to receive the rest. (Let's put security aside for the moment.) But for this to work, you would need to either have a client so generalized that it can be configured basically however the server wants, or you would need a server intimately aware of how the client works to be able to configure it. Such a malleable client is not a very realistic proposition IMO, and the latter is not really that different from bundling some plugins and distributing them.

Nick Gammon said:
I have long been of the opinion that if you are going to mark up text you should mark up the semantics and not the look. In other words, it is much more helpful to a client to say "here is my health, mana and movement points - display them how and where you like", rather than "please display a box 20 x 30 pixels, draw 3 bars in it, and colour them in a certain way".

I think this is an important thing to think about. We've been talking a lot about how the server can control the client's display, but really, the important information is the marked up game content. Perhaps a subwindow package approach is completely inappropriate for a lot of the use cases I've been giving, as they would be better served by simply informing the client of the new state of parameter X, which it could then decide to render (or not).
12 Mar, 2009, Vassi wrote in the 69th comment:
Votes: 0
DavidHaley said:
Such a malleable client is not a very realistic proposition IMO


This is where we'll just have to agree to disagree.

Perhaps what you meant was that expecting existing, well established, clients to support such a protocol is unrealistic given the extensive redesign it may entail. I can see the validity of that point, but I don't see why that should be a limiting factor.
12 Mar, 2009, David Haley wrote in the 70th comment:
Votes: 0
Vassi said:
Perhaps what you meant was that expecting existing, well established, clients to support such a protocol is unrealistic given the extensive redesign it may entail.

Yes.
Vassi said:
I can see the validity of that point, but I don't see why that should be a limiting factor.

Because they're what we have to work with for the foreseeable future. Unless somebody is stepping up to write this very flexible client, support it etc., we're rather dead before having started.
Besides, I still don't think we need all that for a lot of what we (or at least, to be selfish, I) want. I'm not sure we really need a whole GUI description language here.
12 Mar, 2009, elanthis wrote in the 71st comment:
Votes: 0
David Haley said:
Such a malleable client is not a very realistic proposition IMO


Tell that to web browsers. ;)

Quote
tant thing to think about. We've been talking a lot about how the server can control the client's display, but really, the important information is the marked up game content. Perhaps a subwindow package approach is completely inappropriate for a lot of the use cases I've been giving, as they would be better served by simply informing the client of the new state of parameter X, which it could then decide to render (or not).


That gets back to adding a ton of smarts to the client, though. Now the client has to understand what every X is. In some cases you might want this – and that then becomes a custom package.

Wherever possible, though, we need to rely on generic mechanisms. Subwindows are one such mechanism, one which is compatible with how MUDs today work with very little changes. MUDs are already used to formatting text and sending them to a client to be displayed exactly as the server asked for it to be done. Limiting perhaps, yes, but it's how things work.

If we wanted to go with a purely semantic approach, then we need – for example – and inventory package, that has a billion options to cover the billion different ways MUDs deal with inventory specifics, and then each client needs to know how to deal with inventory and all those specifics.

Things like tree view widgets are very generic and yet still very specific. You can use them for a ton of things (generic) but yet you are forced to follow a limited set of behavior guidelines for how tree views work (specific). You end up being forced to display things like inventory like this:

sword     | 15gc    | 1d4+4    | magic
axe | 16gc | 1d8+2 | plain


and lose the ability to get a potentially nicer layout:

[b]sword[/b] (magic)
1d4+4 15gc
———————————
[b]axe[/b]
1d8+2 16gc


You really can't have it both ways. You can't say that you want clients to lack smarts and tons of specific widgets and at the same time want the server to just convey semantic information without formatting, layout, and behavioral controls.

For rich clients you either have to go the route of HTML+CSS+JavaScript (the server expresses very specific layout and behavior using a nearly semantics-free language) or you go the route of DocBook (express semantics and expect the client to understand how to pleasantly format and display all 500 different elements).

Quote
Vassi said:
I can see the validity of that point, but I don't see why that should be a limiting factor.


Because they're what we have to work with for the foreseeable future. Unless somebody is stepping up to write this very flexible client, support it etc., we're rather dead before having started.
Besides, I still don't think we need all that for a lot of what we (or at least, to be selfish, I) want. I'm not sure we really need a whole GUI description language here.


Vassi's point (I think) is that some MUDs may want to make a client specifically for their MUD, and so may end up using a custom package that no other client supports. That's entirely their call. Obviously they will lose a lot of players who refuse to play a MUD that doesn't support their favorite client. The MUD may gain a lot of players if it has a very awesome custom client that has a ton of features that make the game fun (which aren't applicable to 99% of other MUDs, because it's such a unique MUD).

At the very least, look at classic MUDs versus geometic MUDs. I think a compass widget is almost a necessity for a graphical MUD client, but yet many geometrical MUD designs have no use for (that kind of) compass. They may want a widget that shows bearing and velocity instead of just a list of available cardinal directions. There really isn't much middle ground there. Either they both just have to use subwindows and deal with having (IMO) ugly pure-text compass widgets, or they just have to hope clients bother to implement both, or just keep on doing what they do right now and not rely on havng any kind of special widget available (the widgets make it easier to play the game, they should not be REQUIRED to play the game, IMO).

ZMP needs generic reusable packages that a wide variety of MUDs can use and that a wide variety of clients will happily implement. MUDs should whenever possible rely on nothing more than those common standard packages. That is not a question at all.

However, there is nothing fundamentally wrong with a MUD or a client implementing highly specific packages, so long as (a) they realize what the consequences of being unique are, and (b) most servers/clients operate just fine with the generic packages.

For another Web analogy, look at some of the modern Web technologies. SVG, for example. Super powerful and nifty in Opera, WebKit, or FIrefox, but completely unsupported by IE. A website that wants to be accessible by 99% of users cannot depend on SVG. Some websites that don't mind being inaccessible to 60% of users can go ahead and rely on SVG extensively (and these sites exist, but they're obviously unpopular). Best of all, though, a well-designed site can use SVG if it's supported and fall back to something "lesser" for browsers that don't support it: 99% of users can still use the site just fine with no problems, but 40% of them will get an enhanced experience: perfect!

The neat thing is that a MUD that implements some highly custom packages may end up popularizing those features. Maybe just SourceMUD and Cloud9 will be able to use a vector-based paper doll system to support my very non-standard damage system. Maybe once other MUD developers and client developers see how awesome that is, they'll go "wow we want that!" and start implementing it, and soon it goes from being a highly specialized unique package to being implemented by 50% of the MUDs. Who knows? It would suck if ZMP said "here is what the protocol can do, go invent a whole different protocol if you want to experiment," because then nothing new will ever be invented. Imagine if IP/TCP didn't allow you to create any new protocol over top it that you wanted? This forum wouldn't even exist.

Obviously we still very much need a collection of widely applicable simple packages that almost all MUDs and clients can agree on the usefulness of _right now_. Again, no question about that at all.
12 Mar, 2009, David Haley wrote in the 72nd comment:
Votes: 0
elanthis said:
Tell that to web browsers. ;)

That's not the point though: we're not talking about web browsers. I didn't mean to say that a malleable client is unfeasible in general, but for the foreseeable future I don't see it happening for MUDs.

elanthis said:
You can't say that you want clients to lack smarts and tons of specific widgets and at the same time want the server to just convey semantic information without formatting, layout, and behavioral controls.

What I'm driving at is that there are basically two situations/approaches here:
1- MUDs don't create their own packages, and so the standard package has to be general enough for most MUDs, and be supported enough in all clients.
2- MUDs do create their own packages, which means that it's unrealistic for them to tell the client what widgets to display in a custom package.

Is that distinction clear? I'm doing this quickly so might not have expressed it too well…

elanthis said:
Vassi's point (I think) is that some MUDs may want to make a client specifically for their MUD

This is very true, but at this point we've left the conversation of making a standard package.

elanthis said:
Obviously we still very much need a collection of widely applicable simple packages that almost all MUDs and clients can agree on the usefulness of _right now_. Again, no question about that at all.

I agree, so in the interest of promoting immediately productive discussion, I think we should go for this. You already gave a draft specification of the subwindow package; that works for me. I'll try to come up with an immediate application for Darkstone (it's a fully functional codebase, unlike my "project" that is kind of a mess right now) and we can play with it. I think listing more use cases would be pretty useful as well. The tactical map I've been talking about isn't applicable to Darkstone, for example.
12 Mar, 2009, elanthis wrote in the 73rd comment:
Votes: 0
David Haley said:
I agree, so in the interest of promoting immediately productive discussion, I think we should go for this. You already gave a draft specification of the subwindow package; that works for me. I'll try to come up with an immediate application for Darkstone (it's a fully functional codebase, unlike my "project" that is kind of a mess right now) and we can play with it. I think listing more use cases would be pretty useful as well. The tactical map I've been talking about isn't applicable to Darkstone, for example.


Righty. Were you still planning on adding ZMP to MUSHclient? My promise to make a patch for tbaMUD still stands if you do. (Grabbing a fresh SVN checkout now…)

Any other _actively maintained and popular_ codebases I should think about writing a patch for?
12 Mar, 2009, David Haley wrote in the 74th comment:
Votes: 0
elanthis said:
Any other _actively maintained and popular_ codebases I should think about writing a patch for?

Absolutely most definitely the FUSS family, or at the very least, SmaugFUSS.

elanthis said:
Righty. Were you still planning on adding ZMP to MUSHclient?

Sure, I can give this a go. I've never done anything with intercepting packets, but Nick has that whole thing down already for the Aardwolf stuff so I imagine it can't be that hard to just copy it.
12 Mar, 2009, elanthis wrote in the 75th comment:
Votes: 0
::cries:: both tbaMUD (from Circle) and all the FUSS codebases have a complete lack of real TELNET handling.

Think I need to brush the dust off of libtelnet…
12 Mar, 2009, David Haley wrote in the 76th comment:
Votes: 0
I'm not surprised :sad: The networking code in general in SMAUG (and therefore FUSS) bases tends to be, uh, imperfect.
12 Mar, 2009, elanthis wrote in the 77th comment:
Votes: 0
I have a half-finished private project for a TELNET library I think I'm going to use for this. It implements TELNET as a black-box machine, a lot like how zlib works (which MUD developers may be familiar with from implementing MCCP2).

Each connection would be a struct libtelnet_t somewhere (probably in the descriptor), and a bit of initialization to setup callback functions. You just pass bytes into libtelnet_process(struct libtelnet_t *state, unsigned char byte, void *user_data) and then it invokes the various callbacks as appropriate. e.g., text, negotiate, subrequest, command, error, etc.

Small and compact so it can just be copied into a MUD's codebase without needing to rely on users installing an external library of any kind. And I can just build core ZMP protocol support into it, maybe offer default implementations of the core package, and there we go.
12 Mar, 2009, David Haley wrote in the 78th comment:
Votes: 0
Oh, sounds like a pretty nice project. Would definitely improve telnet support in codebases. (Almost by definition, no? :tongue:)

Might not build ZMP straight into it, though, if anything just so that the telnet lib stays just telnet (so somebody can use it elsewhere).
12 Mar, 2009, Nick Gammon wrote in the 79th comment:
Votes: 0
Vassi said:
To use the Aardwolf client as an example, you released a special download just so that everything was already pre-configured. Are you prepared to do the same for every popular MUD? Will they have to package a download themselves? Wouldn't it be better if Aardwolf could talk to MUSH and configure it upon connect, no extra steps involved.


The download was just an installer (the .exe was the same). Basically what Lasher was trying to do was avoid, especially for newbie players, having to download and install, in the correct locations, about 6 or 10 files that were needed for MUSHclient to smoothly start up. Plus, some registry configuration to make windows (like the minimap) appear in a logical place. More recent versions of MUSHclient use a SQLite database for configuration, so tweaking the registry is now unnecessary.

Now that the registry stuff is gone, all a different MUD admin would have to do is, load up the various packages, plugins etc., get the screen positions right, and then simply zip up the resulting directory and distribute that. I personally would not want to be involved, however I put the mechanism there that I don't have to be.

Vassi said:
it would periodically spit out a line like:

ABCDEF000010000200003999900002100030

And the client would gag it, then parse it for input.


That's why I want to go for key/value pairs.

Vassi said:
The problem with parsing text is that you will run into an edgecase somewhere - the way you described with needing to communicate something back to the server - which will necessitate a Telnet link of some kind anyway. Why not just start there?


I regard, perhaps incorrectly, the telnet negotiation protocols as just that - negotiation. In other words the "do/dont/will/wont you support an option" stuff. Most clients will at least handle that to the extent of not displaying the negotiation on the screen. Smarter ones will do something with it (like MCCP). Once you negotiate "I support ZMP - do you? Yes I do …" then you may as well use something simpler after that.

I may well be in a minority here - recent discussions with Lasher have shown that he wouldn't mind extending the client/server interaction using telnet protocols.

elanthis said:
The loop could be made a little more robust looking, but honestly, the NUL byte check and that loop are completely safe.


Call me a pessimist, but I always wonder when I see something like that, what happens when you get:

IAC SB ZMP <20 Kbytes of data here>

If someone manages to spoof the IAC SB ZMP then basically the other players' clients go "dead".

elanthis said:
Every GUI toolkit makes that dead easy. If you're writing your GUI widgets from hand it could be a bit of a pain, I suppose.


I was thinking of the player's perspective. The toolkit can't decide which input window, if there are multiple ones, s/he really should be typing into.

elanthis said:
Because you still have to worry about funny stuff. What happens if a player types in:

say {stats}0/11, 0/25, 0/30, 0/15, 0/15, 0/21


Yes, you have to worry about that. In Aardwolf the trigger specifically looked for {stats} at the start of the line, and chats and tells always start with something.

elanthis said:
Do you just strip { out of all input?


Well you are going to have to strip out IAC SB ZMP so what is the difference? There has to be some foolproof way of making sure players don't spoof the system. Either you make sure everything that the player can initiate doesn't have, or start with, special characters, or the server strips them out, possibly in the output routine at the last moment.

elanthis said:
If it's line based, how do you deal with any kind of informational markup, like clickable text? I mean, take this:

One the ground you see \r\n
{action}get get sword, look look sword\r\n
a long sword\r\n
{/action}\r\n
.\r\n


That's a very valid point. To be honest I wasn't necessarily strongly recommending a simple line-based system, however suggesting alternatives to peppering everything with IAC and NUL bytes. To illustrate how Aardwolf handles the multi line stuff, they simply have a start and end tag, eg.

{roomdesc}
You are in a large room. Its walls are made of mud.
Inside the room are many uninteresting things.
Also the boss is there eyeing you off.
{/roomdesc}


Now you simply have a "state" in the client. The first tag omits this, and subsequent lines. Then the client collects the intermediate lines. Finally the {/roomdesc} tag (itself omitted) turns the state off, and you are back to displaying output. Virtually any client should be able to implement this.

elanthis said:
To be honest, I'd rather just toss TELNET out the window entirely and go with a pure line-based protocol (which still has to have funny commands for dealing with newlines, but at least then the rules would be clearly defined and logical).


I had this vision of using pure Lua data definitions (as Lua is free, easily imbedded, and simple to use). This gives you so much flexibility you won't know what to do with it <grin>.

Just to take an example, say you want to download to the player his/her/its inventory. The inventory can be described as a table. Then if there are containers in the inventory (bags etc.) then they are sub-tables. And if the bag has a smaller bag, that is another sub-table. The entire data description exists already. Lua lets you imbed newlines as \n for example. You also have string, number, boolean data types.

You don't need to reinvent the wheel to describe data, Lua handles special characters (including 0x00 even), and it is parsed at the client end with a single function call. Admittedly it might take a few more function calls to get all the data back, but heck you are going to have to do some work somewhere.


The other things you might want to think about are delayed data delivery, and caching. I tried to get a working example up here:

http://www.gammon.com.au/forum/?id=8936

The caching part was that the inventory was downloaded occasionally, but displayed all the time (or on request) from the cached copy. Naturally you have issues of refreshing it at appropriate times, but the important thing is that a player who obsessively checks their inventory every 5 seconds does not impose any overhead on the server (as they are just looking at the cache). I think you will find World of Warcraft does that, as the inventory can always be immediately viewed, even if the server is laggy.

The delayed data delivery is illustrated by the mouse-over window (in this case the Aardwolf Breastplate of magic resistance). When you hover the mouse over that line in the inventory tabbed window it initially only knows the basics (name, item type, level). So it fires off a request to the server "give me details about item 1067747/0". The server responds a second or so later with all the stats, resists, flags etc. While that second is not yet up the mouse-over window displays all it can with a message in light gray "downloading stats". Then the window refreshes itself to what you see on that page.

This stuff is then cached, so if you just wave the mouse up and down over the inventory, each item's stats only have be obtained once (and the display is lightning fast).

This is the sort of thing that gives a "graphical MMO" feel to what is basically still just a text-based MUD. It looks good, and has a low overhead for the server.

So, my point is that is isn't enough to say "this is how we will send down an inventory". You also want to consider when you send down parts of the inventory.

elanthis said:
Were you still planning on adding ZMP to MUSHclient?


Despite my misgivings about telnet protocols, that isn't hard to do. Already it supports plugin callbacks for the Aardwolf telnet negotiation, so adding a different one is no big deal. Basically then, a plugin could just get all the stuff inside the IAC SB .. whatever .. IAC SE as a string. Of course with imbedded NULs that won't be helpful for most script languages, although Lua could handle it. I might have to break the resulting string down into a table at the NULs for the other script languages. Sigh. You sure you don't want to use another obscure character like ETX (0x03) (or TAB)? Then at least every language can handle it, and clients that don't handle 0x00 well will cope too.
12 Mar, 2009, elanthis wrote in the 80th comment:
Votes: 0
Nick Gammon said:
Call me a pessimist, but I always wonder when I see something like that, what happens when you get:

IAC SB ZMP <20 Kbytes of data here>


What happens when your line-based MUD server gets 20kb of data before the \r\n? Generally you disconnect the remote end on account of them sending you excessive data.

Quote
If someone manages to spoof the IAC SB ZMP then basically the other players' clients go "dead".


That being the advantage of ZMP – it's very, very, very, very easy tomake sure another client can never ever spoof a TELNET command.

I can make your client go dead in any number of ways if you let players inject the IAC byte into a stream. All I have to do is send IAC SB MCCP2 IAC SE in many cases and a dumb client will start trying to pass all following bytes through zlib. Or I could send just send IAC SB <something>. If you allow players to inject the IAC byte, you're already screwed. If you protect against IAC injection, then ZMP is already safe.

Quote
I was thinking of the player's perspective. The toolkit can't decide which input window, if there are multiple ones, s/he really should be typing into.


Not really a problem in a well-designed GUI. A MUD that pops up 50 windows with their own input is going to be a pain in the ass, obviously. I'm imagining there being maybe 2 windows with input, tops. Mostly just for dedicated chat.

Quote
Well you are going to have to strip out IAC SB ZMP so what is the difference?


Not at all. You don't need to strip out _anything_ to be safe. If your TELNET layer is correctly implemented, there is ZERO danger. Because any time any bytes get sent to the remote end that the code doesn't explicitly intend to be a TELNET command MUST be sure to escape IAC with IAC IAC.

Or you can strip JUST the IAC byte from input and make ZMP safe (as well as TELNET).

Seriously: if it is possible to inject the IAC byte unescaped, it is already possible to **** the other players without even supporting ZMP.

Quote
That's a very valid point. To be honest I wasn't necessarily strongly recommending a simple line-based system, however suggesting alternatives to peppering everything with IAC and NUL bytes. To illustrate how Aardwolf handles the multi line stuff, they simply have a start and end tag, eg.


Meh, I don't see the problem with using TELNET. If your code can't handle that, your code is already broken. You can't handle simple TELNET negotiation. You can't handle NAWS. You can't handle MCCP2. You can't handle anything interesting.

But since your code DOES already handle that, why add a totally second set of characters that you have to escape and parse? Your code already deals with TELNET so it's already set to handle ZMP (unless your code is broken and held together by scotch tape) without requiring any significant work at all.

Quote
The caching part was that the inventory was downloaded occasionally, but displayed all the time (or on request) from the cached copy. Naturally you have issues of refreshing it at appropriate times, but the important thing is that a player who obsessively checks their inventory every 5 seconds does not impose any overhead on the server (as they are just looking at the cache). I think you will find World of Warcraft does that, as the inventory can always be immediately viewed, even if the server is laggy.


That's kind of the point of an inventory subwindow. The MUD sends the text for the window once, and then only sends updates when the inventory changes. A MUD could go so far as to disable the inventory command if subwindows are in use (I wouldn't, but if you're being paranoid about bandwidth usage in a text-based game on 2009's brandband speeds… knock yourself out).

Quote
Despite my misgivings about telnet protocols, that isn't hard to do. Already it supports plugin callbacks for the Aardwolf telnet negotiation, so adding a different one is no big deal. Basically then, a plugin could just get all the stuff inside the IAC SB .. whatever .. IAC SE as a string. Of course with imbedded NULs that won't be helpful for most script languages, although Lua could handle it. I might have to break the resulting string down into a table at the NULs for the other script languages. Sigh. You sure you don't want to use another obscure character like ETX (0x03) (or TAB)? Then at least every language can handle it, and clients that don't handle 0x00 well will cope too.


Given that there are already servers and clients that implement ZMP, it's a bit late now. And to be blunt, I consider any language that can't deal with NUL bytes in a string broken. Even C can do it just fine as my example code illustrated, and it's the language the NUL-terminated string originated from (I think). If you just parse the ZMP in C (I gave the example code) you can just send an array of strings to the crazy script languages and not have to think twice about it. You don't pass MCCP2-compressed data into the script languages, do you?
60.0/108