18 Mar, 2009, elanthis wrote in the 1st comment:
Votes: 0
Tyche had asked what the point of package checking with zmp.check was, which led to me noting that it really kind of pointless.

I plan on updating the spec to remove the requirement of this behavior. It simplifies the implementation of zmp.check greatly, and we can safely assume nobody is using package checking at this point given the lack of official packages. (Though Les let me know that gnome-mud know has preliminary support for the subwindow package!)

Someone (forget who) also mentioned the possibility of a zmp.check-all command that would tell the other side to send a zmp.support command for every supported command. That would cause a small flurry of activity on connect, but nothing that I think is even remotely close to "excessive."

So, does that sound like a plan to everyone?
18 Mar, 2009, David Haley wrote in the 2nd comment:
Votes: 0
I like the idea of being able to check two things:

- an individual command. Do you actually support this command?

- a whole package. Do you support this package?

The answer to the first is trivial (actually it's not), the answer to the second could be one of "not at all", "fully, version x", "somewhat". "Somewhat" could be either literally just "somewhat" or a list of supported commands.

The reason why I said the answer to the first isn't trivial is that as soon as you have versioned packages, you need to ask about supporting a command from a given version. Then you have all kinds of annoyances about supporting the command from package v5, which means you might not support the command from v3 anymore, unless we require that all package APIs be backwards compatible…

So maybe it is wrong to ask for supporting a package, but it's not so evil IMO.
18 Mar, 2009, elanthis wrote in the 3rd comment:
Votes: 0
Well, here's the dealio.

ZMP does not have versioned commands or packages. Supporting those is complicated no matter how you look at it. I come from the school of "do it right the first time," but if you do have to break compatibility, you should do it the same way a shared library does: rename the symbols and library soname so both versions can peacefully coexist. If your mypkg package is so broken that you need to actually change its ocmmands, then you need to publish mypkg2. The commands end up having a different identifier (by being in a technically totally different package) and life goes on.

That then extends to the package checking. You don't need to ask for a package and then deal with different versions. You ask for mypkg2 and mypkg and see which is supported.

The second problem is that ZMP has no possibly way of enforcing either end to fully support a package. You can say "package foo must support commands foo.bar, foo.baz, and foo.bla" but if the client only implements foo.bla, what are you going to do about it? Finding out there is partial support isn't even interesting. Either the command you need is there (in which case you must check the command) or it isn't (in which case a response like "partial foo support" is insufficient).

Ideally, packages get defined once and do not change. I've already broken that by adding one command to the zmp core package (and here I am proposing two more changes), which technically means older implementations became uncompliant, which really isn't cool. I admit to that sucking and it being all my fault. On the upside ZMP apparently has next to no real-world usage yet so I guess making tweaks is practically safe… but I want to get any and all further tweaks done NOW so I don't ever need to do them again.

Oops, tangent. So, point being, if you want to check for a package and just trust that the client really did implement the whole thing (as per the package's spec) then you can just check for any command in the package. If the client supports subwindow.open, then it is assumed that it supports every other subwindow command except maybe for subwindow.set-input, because that one is marked as optional. If the client does not implement all the commands… not much to do about it. Even if you know it only has a partial implementation, you are still forced to either (a) pretend subwindow isn't support at all all, (b) use all the commands and hope the critical ones work, or © check each command individually. A client that doesn't comply with the spec is technically broken and should be considered the same as a client that responds to IAC WILL MCCP2 with IAC DO MCCP2 but then doesn't support MCCP2 – damn broke and not a thing you can do about it.

Checking for a command is easy. if you have a list of commands, or a tree of commands, or a hash table of commands, or whatever, you just call your lookup function and see if it returns a result. If yes, command is there. If not, command is not there. Simple. Checking for a package requires a whole different codepath, though. If you have a list of commands, you have to write a second function that iterates over them and looks for partial matches. If you have a tree or hash you can optimize a little by having the command registration function register the package name in a separate tree/hash, but still, excess code. All for relatively little gain as illustrated.

If you are making a relatively large package of related functionality that you expect implementations may only want to partially include, consider breaking your package into subpackages. e.g., if you add a mymud package with a ton of experimental/custom stuff, you might have a mymud.olc package for your OLC stuff, mymud.chat for your custom chat stuff, mymud.text for your crazy text formatting stuff, etc. Assuming each package is small and self-contained, it reduces the likelihood of a client implementing just part of a package – the mymud.olc package might be totally useless without all 3 or 4 of its commands, for example. Making subpackages doesn't really _do_ anything technically, but it makes the clear boundaries between functionality very clear and apparent and makes the mymud custom package more "self-describing" so you don't really need to spell out "if you implement A, you must implement B and C; and if you implement D, you must implement E and F and G; and if you implement H, you must implement I" and so on.

Summary: unless I add versioning and package schemas and all other kind of crap (which is like going from XML-RPC to SOAP), checking for a package is no more useful than just checking for a command in the package.
18 Mar, 2009, Tyche wrote in the 4th comment:
Votes: 0
elanthis said:
Tyche had asked what the point of package checking with zmp.check was, which led to me noting that it really kind of pointless.

I plan on updating the spec to remove the requirement of this behavior. It simplifies the implementation of zmp.check greatly, and we can safely assume nobody is using package checking at this point given the lack of official packages. (Though Les let me know that gnome-mud know has preliminary support for the subwindow package!)


It's possible that altering the zmp.check package. specification to respond positively ONLY if ALL commands in the package are supported makes it useful again.

elanthis said:
Someone (forget who) also mentioned the possibility of a zmp.check-all command that would tell the other side to send a zmp.support command for every supported command. That would cause a small flurry of activity on connect, but nothing that I think is even remotely close to "excessive."


Me again. I proposed to respond with all commands in a single go, although it doesn't really make a logical difference whether one sends a dozen zmp.support comamnds versus one zmp.support-list command (or whatever it be called).

However if zmp.check package. were useful that should mitigate the problem. Obviously there are only a handful of packages and only a handful of commands, but suppose one defines a windowing package with 10-20 commands. It's far easier to do a zmp.check package. command once, than do a zmp.check for each and every command, or a zmp.check-all and process all the commands returned.
18 Mar, 2009, elanthis wrote in the 5th comment:
Votes: 0
If a window package is defined and it states that all commands must be implemented unless explicitly marked optional, what advantage does a dedicate package check facility offer over checking for one of those commands? If servers are written to check for a package, and that check only responds affirmatively, and a client only partially implements the package, then no server would ever use those features on said client and adding a partial implementation would be pointless. Or am I assuming too much?
18 Mar, 2009, Les wrote in the 6th comment:
Votes: 0
One possible use for checking for package support with zmp.check would be to change it's semantics.

When a server asks the client zmp.check subwindow. for example, rather than the current (useless) behavior the client could send back all the supported commands and unsupported commands in that package.

So for example, on a client that implements all the non-optional bits of the subwindow spec it would look something like this:
server: zmp.check subwindow.
client: zmp.support subwindow.open
client: zmp.support subwindow.close
client: zmp.support subwindow.select
client: zmp.support subwindow.size
client: zmp.nosupport subwindow.input
cilent: zmp.nosupport subwindow.set-input


This way we can keep the current command/package distinction but querying for a package does something useful.
18 Mar, 2009, Tyche wrote in the 7th comment:
Votes: 0
Les said:
When a server asks the client zmp.check subwindow. for example, rather than the current (useless) behavior the client could send back all the supported commands and unsupported commands in that package.

So for example, on a client that implements all the non-optional bits of the subwindow spec it would look something like this:
server: zmp.check subwindow.
client: zmp.support subwindow.open
client: zmp.support subwindow.close
client: zmp.support subwindow.select
client: zmp.support subwindow.size
client: zmp.nosupport subwindow.input
cilent: zmp.nosupport subwindow.set-input


This way we can keep the current command/package distinction but querying for a package does something useful.


Good idea, although I'd restrict the response to just the commands that are supported. Because otherwise it's a chicken and egg problem. How do I know there exists a subwindow.foo command if I haven't bothered implementing it? I can respond to specific command with nosupport.

Then again the reverse is true, I get a positive response for a command I don't know exists. ;-)
18 Mar, 2009, Les wrote in the 8th comment:
Votes: 0
Tyche said:
Good idea, although I'd restrict the response to just the commands that are supported. Because otherwise it's a chicken and egg problem. How do I know there exists a subwindow.foo command if I haven't bothered implementing it? I can respond to specific command with nosupport.

Then again the reverse is true, I get a positive response for a command I don't know exists. ;-)


Yeah you're dead-on regarding sending no-support. I was thinking the client would just keep track of what it doesn't support but it's alot better not to rely on that.

As for your other concern, I'm not sure I see the problem. If you get a positive response for an unknown command then you just ignore it since you don't do anything with that command. Like if for some reason the server decides its not going to bother dealing with subwindow.input, the client can send zmp.support subwindow.input and the server doesn't have to do anything about it (maybe snidely remark 'good for you little client' or something ;) ).

Even if the situation with the server and client is reversed I don't see a problem. The client wants to know what commands are supported by the server for the foo package (it's a totally sweet package). the client implements foo.bar foo.baz and foo.whee. The server sends back those plus an additional one foo.WinLife. The client doesn't support foo.WinLife but it knows that and just ignores it (and therefore loses).

What am I missing here?
19 Mar, 2009, elanthis wrote in the 9th comment:
Votes: 0
I like that proposal better. It would probably be best to send the responses in a single command for implementation sanity reasons: if you need to make sure 5 commands in a package are supported, you aren't going to want to have to receive 5 separate commands and maintain state between each one of them.

Should these be new commands or should I just update the existing ones (in a compatible way). I imagine we could just say that zmp.support should return the list of supported commands in a package after the first argument (which is the pacakge name itself with the trailing dot), and that it must give at least one command in the list. If no commands in the package are supported it does what the spec already says and just responsed with zmp.no-support and the package name.

Alternatively, move the package stuff to a second set of commands. Just a zmp.get-package and a zmp.package-list or something like that.
19 Mar, 2009, Les wrote in the 10th comment:
Votes: 0
elanthis said:
I like that proposal better. It would probably be best to send the responses in a single command for implementation sanity reasons: if you need to make sure 5 commands in a package are supported, you aren't going to want to have to receive 5 separate commands and maintain state between each one of them.

Should these be new commands or should I just update the existing ones (in a compatible way). I imagine we could just say that zmp.support should return the list of supported commands in a package after the first argument (which is the pacakge name itself with the trailing dot), and that it must give at least one command in the list. If no commands in the package are supported it does what the spec already says and just responsed with zmp.no-support and the package name.

Alternatively, move the package stuff to a second set of commands. Just a zmp.get-package and a zmp.package-list or something like that.


I like the first option better precisely because it doesn't add any new commands. It also keeps zmp.check semantics identical to the current spec with the differences localized into zmp.support. I *think* it wouldn't break any unknown and preexisting implementations as a bonus. Since the package name is still the first argument and that's what any existing implementation would check for I think probably they would just ignore the other args.

On a side note, zmp could very well become the 'MUDNET' telnet replacement. You've already reimplemented NAWS and a version of TTYPE in ZMP :tongue:
19 Mar, 2009, elanthis wrote in the 11th comment:
Votes: 0
Technically implementations would become broken on account of them not sending the additional parameters in zmp.support.

Where did I implement TTYPE?
19 Mar, 2009, Les wrote in the 12th comment:
Votes: 0
elanthis said:
Technically implementations would become broken on account of them not sending the additional parameters in zmp.support.


That's true I didn't think of that. Well even if implementations are broken I still like changing the behavior of the current commands over adding new commands.

elanthis said:
Where did I implement TTYPE?


Was just tongue in cheek but I was referring to zmp.ident :)
21 Mar, 2009, Les wrote in the 13th comment:
Votes: 0
Just wanted to ping this topic to see if there's been any further thoughts on it.

elanthis said:
Should these be new commands or should I just update the existing ones (in a compatible way). I imagine we could just say that zmp.support should return the list of supported commands in a package after the first argument (which is the pacakge name itself with the trailing dot), and that it must give at least one command in the list. If no commands in the package are supported it does what the spec already says and just responsed with zmp.no-support and the package name.


So if this is the method adopted then it would work like the following on a client that supports the mandatory parts of the subwindow spec:
server: zmp.check subwindow.
client: zmp.support subwindow. subwindow.open subwindow.close subwindow.size subwindow.select


And if the client doesn't support subwindow. then
server: zmp.check subwindow.
client: zmp.nosupport subwindow.


This seems reasonable to me. Honestly, I don't see the benefit of moving package checking to it's own set of commands when it's so easily used in this manner. Clientside, it's no big deal to check if the last character is '.' for a package check which would be the only reason I can think of to separate them.
21 Mar, 2009, elanthis wrote in the 14th comment:
Votes: 0
That is the proposed behavior. (Note that it's zmp.no-support, with the dash in there.)
21 Mar, 2009, Les wrote in the 15th comment:
Votes: 0
elanthis said:
(Note that it's zmp.no-support, with the dash in there.)


Well I got it right once in my implementation. Good enough for me :wink:
21 Mar, 2009, David Haley wrote in the 16th comment:
Votes: 0
Why do we need a period to tell if it's a package or not? Isn't it implicit? When you look it up in your list of supported things, if you want that it's a package, you respond with the stuff inside it you support. If you find it's a command, you respond just like that. If you don't know, you send back no-support.

I would say that there should be a distinction between required and optional commands in a package. Let's assume that P has commands a, b and c, where b is optional.

Server: support P?
Client: support P

this means that the client supports all the required commands in P.

Server: support P?
Client: support P, P.b

This means that the client supports P.b as well.

The reason I want to push for required vs. optional commands is that for some packages, it simply doesn't make sense to not support some commands. If you implement the subwindow package, for instance, you must support subwindow.select to do anything sensible with it. So I don't think it's useful to say that you explicitly support that command. The server wouldn't really be able to do anything useful if you didn't support it, after all. However, for optional commands, it does make sense to query for them.
22 Mar, 2009, Les wrote in the 17th comment:
Votes: 0
David Haley said:
Why do we need a period to tell if it's a package or not? Isn't it implicit? When you look it up in your list of supported things, if you want that it's a package, you respond with the stuff inside it you support. If you find it's a command, you respond just like that. If you don't know, you send back no-support.


It's implicit because you're a human. It simplifies implementation to know if it's an explicit package check, which is the point of the period.

David Haley said:
I would say that there should be a distinction between required and optional commands in a package. Let's assume that P has commands a, b and c, where b is optional.


I am not sure what benefit this provides. If the client doesn't support subwindow.select then by the spec the server MUST not use it anyway. Adding explicit markers that this is required doesn't change anything.
22 Mar, 2009, elanthis wrote in the 18th comment:
Votes: 0
(very late, very tired, rambling ensues…)

The way you want it to work David requires that every implementation have a full registry of all possible command pertaining to any package, there-for allowing it to actually track which commands it does and does not have.

The implementation of the existing commands and the possibly changes requires an implementation to know absolutely nothing other than the list of commands it supports. It makes it trivial to make code like:

zmp.register("package.foo", closure() { blah });


Instead of needing something like:

zmp.register("package", "2.4+", list {
list { "foo", "required", closure() { blah } },
list { "bar", "optional", nil }
})


Even with that kind of requirement there's no guarantee (or even a strong indication) that the client will tell the truth when you ask if it supports all required commands. If a client has a few entries missing from its "package schema" tables then it is just going to return bogus data.

Checking if a package exists (or if a single required command in a package exists) tells you the same thing for less work: it tells you that the other end attempted to implement the package and that they think the package is usefully complete (otherwise it wouldn't advertise it at all). Checking if an optional command exists (or getting a list of commands in a package) tells you the same thing for less work too because implementations don't need to worry about whether a command is optional or not.

Put simply, if a client says it supports subwindow.open but not subwindow.select or other required commands then the client is broken. it is no different than a client replying IAC DO COMPRESS2 when it has no MCCP2 support. Or claiming the terminal type is VT100 and not supporting cursor control or color commands. Or connecting to a TELNET server and choking on valid TELNET commands.

We just have to assume that if a package is implemented that it is being implemented fully or that it is somehow disabled in the client/server for non-debug builds.

If you REALLY want full package checking then you most certainly do not want that encoded in the protocol. What you want is the proposed behavior of retrieving a list of all commands in a package, and then comparing that list of commands to data file on the requester's side, similar to how XML Schemas work.

Even then you still have to assume that the client implements the commands properly.

Overthinking and worrying about forcing all kinds of strict interface definitions leads people down the same path that created SOAP as a replacement for XML-RPC. Which is a bad place to be. SOAP requires 100x more code, far more effort to use, has an insanely bloated and complex specification, and most software using it still breaks on a frequent basis because nobody understands how to use or even bothers to use all of the interface definition features. In the time it takes just to write a WSDL (only one tiny piece necessary for building a SOAP API) you could have written you entire server API using XML-RPC and just put a README file on your website documenting the interface.

If you're going down the strict package definition path, you end up wanting to know how many arguments the implementation expects each command to receive, the types of those arguments, etc. You end up wanting some kind of interface definition like:

<zmp:interface>
<zmp:package zmp:name="mypackage" zmp:version="2.1">
<zmp:command zmp:name="foo">
<zmp:parameters>
<zmp:parameter zmp:name="identifier" zmp:type="string" zmp:min-length="1" zmp:min-length="20" zmp:match="^[a0z]$"/>
<zmp:parameter zmp:name="some flag" zmp:type="bool"/>
<zmp:parameter-group zmp:name="variables" zmp:min-count="1">
<zmp:parameter zmp:name="key" zmp:type="string" zmp:min-length="1" zmp:match="^\w+$"/>
<zmp:parameter zmp:name="value" zmp:type="mixed"/>
</zmp:parameter-group>
</zmp:parameters>
<zmp:response>
</zmp:response>
</zmp:command>
</zmp:package>
</zmp:interface>


You end up wanting clients to express all that information in their ZMP handlers, and you want servers to parse those definitions, and you expect people to actually maintain those definitions and version packages properly.

And at the end of the day, you still have to let users turn off certain ZMP features or add client-detection or just disable the feature entirely because some broken-ass client says it supports a package but doesn't properly implement it. Some client says it supports subwindow.open but it bombs if the window id is anything other than an integer so your windows with ids like "inventory" get ignored and you end up displaying all inventory in a window that doesn't exist. So sure, maybe we add in actual command responses and errors codes – or just a set of commands for each package so subwindow.open can return a subwindow.wont-open command – and now the spec is even more bloated and you're still stuck hoping that a client isn't reporting success depsite failing internally some how.

So no, requiring any kind of package definition smarts inside a client or server is not something I will support, ever. If you are really worried about a client failing to support a package fully then just make sure your MUD allows the user to turn off the use of any ZMP features, be it all-or-nothing or something more fine grained.
22 Mar, 2009, Tyche wrote in the 19th comment:
Votes: 0
Quote
server: zmp.check subwindow.
client: zmp.support subwindow. subwindow.open subwindow.close subwindow.size subwindow.select

And if the client doesn't support subwindow. then
server: zmp.check subwindow.
client: zmp.no-support subwindow.


That's workable and makes the package check useful again. Thanx
22 Mar, 2009, David Haley wrote in the 20th comment:
Votes: 0
elanthis said:
The way you want it to work David requires that every implementation have a full registry of all possible command pertaining to any package, there-for allowing it to actually track which commands it does and does not have.

Forgive me but I don't see at all why this is a problem. Since specs are "set in stone" – there are no versions! – any implementation knows what it's implementing. How could it possibly not have a list of commands it should be implementing?

You asked yourself:
"If a window package is defined and it states that all commands must be implemented unless explicitly marked optional"
What exactly does this mean, if you are refusing to have implementations know what they are implementing and what they aren't?
0.0/36