Forking this from the discussion of the NakedMud snippets for while it's a helpful discussion its not entirely on topic :) In retrospect I should have done that before I replied, but as they say, hindsight's always 20/20.

The reply, for some context:



Maya/Rudha

I believe that if you keep asking it, it's supposed to keep telling you.

Anyhow, if you believe MUSHclient is doing something against the RFC, you should find the section of the RFC that it violates and post it here (for our edification) and on the MUSHclient forums (so that Nick can fix it). There's not much point just talking about it like this.

The problem is that this goes for both parties in the negotiation, and this is what generates the loop. Removing this from one end (the server), does solve the loop problem, but it also makes the server non-compliant in regards to state changes. The implementation of a proper state machine isn't touched in RFC 854, in fact there are several examples of parts of that RFC that generates loops; one of which I quoted previously; a fact that makes me wonder if you read the whole post before responding.

RFC 1145 was essentially formulated to address these problems, hence me originally saying that I wasn't planning on releasing the TELOPTs module I have until I know that the server follows that RFC. The full text is here, for perusal: [[url=http://www.faqs.org/rfcs/rfc1143.html]li...]] (http://www.faqs.org/rfcs/rfc1143.html)

Of particular interest:



The main important idea behind RFC 1145 is that we should remember the state we are attempting to achieve, instead of simply responding to the requests with a "I can" or "I cant" but rather remembering the state we "want" In all cases where the client doesn't want to have an option, in order to adhere to RFC 854 a server MUST not start using a protocol where the client has responded to the handshake challenge with DONT; pragmatically speaking, this means that any case other than WILL/DO we should not use the protocol.

A state machine is a good thing to have on both ends, for just this reason. Some clients have this, most don't. Some clients which can be called 'sloppier' as you described them, discard WONT messages if it agrees with their current setting for that being set off, or they don't support the feature. This is probably a more correct method as it avoids loops and follows RFC 1145. Specifically they should only respond to challenges where their current state changes. It is probably worthwhile recording data for error cases, however.

The problem with my implementation currently is it seems to have trouble "remembering" it began the negotiation. Its unclear whether RFC 854 requires this, but it's a good idea; if we do not, then we fall into these endless loops with protocols like MSDP because we do not "remember" whether the user is responding to the handshake, or not. If they're responding to the handshake we need to remember not to use the protocol for that socket. Otherwise, it is possible they may re-enable it at a later time with IAC <protocol> WILL/DO. Strictly speaking RFC 854 doesn't require us to differentiate between these scenarios, however, failing to do so may result in loops. We have no real alternative manner to differentiate between the user sending a challenge by changing an option in their client, and a client/server loop.

Maya/Rudha

[edit]: I should have known better than to post those RFC numbers all over the place, given my predisposition to dyslexia.

You receive IAC WILL/DO X. You have three possibilities:

1: You've already got mode X switched on, so don't respond.

2: You've got mode X switched off, so return IAC DO/WILL X and switch mode X on.

3: You don't understand option X, so return IAC DONT/WONT X.

You receive IAC WONT/DONT X. You have three possibilities:

1: You've got mode X switched on, so return IAC DONT/WONT X and switch mode X off.

2: You've already got mode X switched off, so don't respond.

3: You don't understand option X, so return IAC DONT/WONT X.

Thus in your case with MSDP, it should work like this:

Server: Start up the server. MSDP mode is off. Send the "IAC WILL MSDP" request.

Client: Unknown option, reject with "IAC DONT MSDP".

Server: MSDP mode is already off, therefore don't respond.

Right; a state machine is the correct way to deal with that; basically the point people are getting stuck on is my saying that both ends of the communication should be doing this; if you have responded to a response with one state, then if youre following RFC 1145 or my interpretation at least; well it states itself: you shouldnt duplicate that response unless prompted to by the user, or in the case of the server, a background component.

This isn't just being said to "rag" on Mushclient - although I must admit that I am frustrated that its the only client Ive encountered which presents this loop-causing behaviour - its also what the server is doing wrong. Since its not remembering it hasnt finished negotiating its assuming the IAC MSDP DONT is sent to terminate output. That needs to change for it to be adherent to RFC 1145.

What I dont understand is why the Conditional is failing to discard this response as duplicating the current state; poking with gdb confirms DONT MSDP is in the relevant socket index; the conditional should evaluate true.

Maya/Rudha

RFC 1145 is "TCP alternate checksum options". Did you mean RFC 1143, which you mentioned in the previous thread, which is "The Q Method of Implementing TELNET Option Negotiation"? If so, please remember that RFC 1143 is not a protocol specification, nor a recommendation of the IETF - it's an experimental method of implementing a protocol, designed to extend RFC 854.

And as I've said before, RFC 854 states that "all hosts may implement their TELNET processes to be totally unaware of options that are not supported, simply returning a rejection to (i.e., refusing) any option request that cannot be understood". This appears to be what MUSHclient is doing.

RFC 1143 does state that "A TELNET implementation MUST refuse (DONT/WONT) a request to enable an option for which it does not comply with the appropriate protocol specification", but I cannot see any mention of how it should treat a request to disable an unknown option.

However what it does state is that "The implementation MUST NOT automatically respond to the rejection of a request by submitting a new request", and that's exactly what your server is doing. If you can fix that, then MUSHclient should work fine.

If you feel this is a client problem then you should contact Nick Gammon, but as far as I can see MUSHclient does appear to follow the RFC, and you may well encounter other clients with similar behaviour. As was pointed out earlier, MUSHclient tends to follow specs to the letter, while other clients tend to favour robustness (unknown MXP tags and MCCP compression errors being obvious examples of this). IMO this makes MUSHclient a superb choice for testing your protocols, as it'll often reveal problems with your implementation that other clients will try to work around.

Yes thats the RFC I meant, and we're not disagreeing there either - as I originally noted the reason I didnt toss my TELOPTs module in there is because of bugs it ends up behaving in a noncompliant manner.

I think beyond that Ill not comment; Im both unwell and stressed at work lately which in retrospect is starting to carry over to my posts here.

Maya/Rudha

Sorry to hear you're unwell & stressed.

I agree with KaVir's analysis, incidentally. That said, Nick is pretty good about responding to this stuff, and sometimes even helping debug server behavior (if anything to establish that his client is following specs). So even if you believe that it's not the client after all, it can be worth talking to him, if anything to get his interpretation of the RFC.

I think MTH handles this more or less correctly. Might be worth the trouble to look at WickedMud for inspiration, it's public domain so there are no copyright worries. I think (been a while since I worked on it) MTH uses an array of characters with 256 indexes to keep track of the state, so that's 256 bytes of overhead per socket, though I might have added some bit screwing to only use 128 or 64 bytes.

I'll take a look at tt++ responding to both DO and WILL requests when an option is set as enabled. MTH might have the same issue.

It really isn't mushclient that it is at fault and it wasn't my intention to say it was; rather, what I guess I was trying to say is since it's the one client I tested that reacted in that way, it was being problematic in that way. In retrospect I do agree that it is helpful as a debugging tool; but ultimately what it resulted in was my handful of admins and testers complaining at me that the MUD didnt work; this looping behaviour manifests in MushClient in the client giving blank line after blank line, which seems somewhat problematic - I dont think the user should be seeing anything as a result of TELOPT negotiations unless they choose that they want to; such as by selecting #CONFIG DEBUG ON in TT++.

I'll probably send Nick Gammon an email to that regard when I have the time to more thoroughly research the client behaviour. Incidentally, is there a way to make MushClient report the TELNET negotiations such as #CONFIG DEBUG ON does on TT++

Scandum: TT++ seems to correctly not respond to new requests that are sent which agree with the current state; however it would be nice if there was some way for it to distinguish when the server does a copyover as socket UIDs are not going to be identical from before and after a copyover; I suppose I can use NEW_ENVIRON to make the client store the TELOPTs prior to copyover; however I am unsure how supported that is in TT++ or in general. I do know of MUD codebases that use this where they can, though, to do just that.

Maya/Rudha

I've never seen Mush output blank lines during any telopt negotiations I've ever tested, except in the cases when my code was mistakenly sending out a stray CRLF in one of the output wrapper methods. So, my advice would be to dump your low-level output and make sure you're sending nothing but the negs.

As for debugging, I don't know of any way to display telopts on the screen in real-time in Mush, but you can always dump the raw log (which does include them, IIRC) to a file and examine it.

I think it also exposes a plugin API that can be called on each telnet state change, so you could do that – I'd also suggest asking on the forums because somebody has probably done this before (perhaps Nick himself).

Mine did the same, and I also ran into the same issue when testing my protocol snippet in a Diku. However it did it with other clients as well, it wasn't specific to MUSHclient.

There shouldn't be carriage returns in there; furthermore if there was, this behaviour would be seen on other clients too.

# TELOPT characters
EXTEND_ASCII = chr(17) # '%c' % 17
TERMTYPE     = chr(24)   # "%c" % 24
NAWS         = chr(31)   # "%c" % 32
MSDP         = chr(69)   #'E'    # "%c" % 69
MSSP         = chr(70)   #'F'    # "%c" % 70
MSP          = chr(90)   # '%c' % 90
ATCP         = chr(200)  # '%c' % 200

MCCP1        = chr(85)
MCCP2        = chr(86)

# Handshakes
server_will_extend_ascii = telnet.IAC + telnet.WILL + EXTEND_ASCII
server_wont_extend_ascii = telnet.IAC + telnet.WONT + EXTEND_ASCII
client_do_extend_ascii   = telnet.IAC + telnet.DO   + EXTEND_ASCII
client_dont_extend_ascii = telnet.IAC + telnet.DONT + EXTEND_ASCII

server_will_termtype     = telnet.IAC + telnet.WILL + TERMTYPE
server_wont_termtype     = telnet.IAC + telnet.WONT + TERMTYPE
server_do_termtype       = telnet.IAC + telnet.DO   + TERMTYPE
server_dont_termtype     = telnet.IAC + telnet.DONT + TERMTYPE
client_will_termtype     = telnet.IAC + telnet.WILL + TERMTYPE
client_wont_termtype     = telnet.IAC + telnet.WONT + TERMTYPE
client_do_termtype       = telnet.IAC + telnet.DO   + TERMTYPE
client_dont_termtype     = telnet.IAC + telnet.DONT + TERMTYPE

server_do_naws           = telnet.IAC + telnet.DO   + NAWS
server_dont_naws         = telnet.IAC + telnet.DONT + NAWS
client_will_naws         = telnet.IAC + telnet.WILL + NAWS
client_wont_naws         = telnet.IAC + telnet.WONT + NAWS

server_will_msdp         = telnet.IAC + telnet.WILL + MSDP
server_wont_msdp         = telnet.IAC + telnet.WONT + MSDP
client_do_msdp           = telnet.IAC + telnet.DO   + MSDP
client_dont_msdp         = telnet.IAC + telnet.DONT + MSDP

server_will_mssp         = telnet.IAC + telnet.WILL + MSSP
server_wont_mssp         = telnet.IAC + telnet.WONT + MSSP
client_do_mssp           = telnet.IAC + telnet.DO   + MSSP
client_dont_mssp         = telnet.IAC + telnet.DONT + MSSP

server_will_msp          = telnet.IAC + telnet.WILL + MSP
server_wont_msp          = telnet.IAC + telnet.WONT + MSP
client_do_msp            = telnet.IAC + telnet.DO   + MSP
client_dont_msp          = telnet.IAC + telnet.DONT + MSP

server_do_atcp           = telnet.IAC + telnet.WILL + ATCP
server_dont_atcp         = telnet.IAC + telnet.WONT + ATCP
client_will_atcp         = telnet.IAC + telnet.DO   + ATCP
client_wont_atcp         = telnet.IAC + telnet.DONT + ATCP

server_will_mccp1         = telnet.IAC + telnet.WILL + MCCP1
server_wont_mccp1         = telnet.IAC + telnet.WONT + MCCP1
client_do_mccp1           = telnet.IAC + telnet.DO   + MCCP1
client_dont_mccp1         = telnet.IAC + telnet.DONT + MCCP1

server_will_mccp2         = telnet.IAC + telnet.WILL + MCCP2
server_wont_mccp2         = telnet.IAC + telnet.WONT + MCCP2
client_do_mccp2           = telnet.IAC + telnet.DO   + MCCP2
client_dont_mccp2         = telnet.IAC + telnet.DONT + MCCP2

[…]

# ——————————————————————————
# Function: will_negotiation_hook
# We have recieved a connection, so lets see what protocols they support.
def will_negotiation_hook(info):
    sock, = parse_hook(info)
    sock.send_raw(server_will_extend_ascii)
    # sock.send_raw(server_will_termtype)
    sock.send_raw(server_do_termtype)
    # sock.send_raw(server_do_naws)
    sock.send_raw(server_will_mssp)
    sock.send_raw(server_will_msdp)
    sock.send_raw(server_will_msp)
    sock.send_raw(server_do_atcp)
    sock.push_ih(negotiating_wait_handler, acct_wait_negotiating_prompt)

(difference between send and send_raw is that send appends cr to the output while send_raw doesnt)

Maya/Rudha

A question, is there a reliable way to distinguish between IAC DONT being because its unsupported, as opposed to an option simply being disabled at the onset?

Maya/Rudha

[edit] Specifically, Im wondering how "safe" it is to assume that IAC DONT <whatever> at the time of initial negotiation = unsupported.

The client can't detect a copyover as far as I know. In the case of MSDP the server ought to disable MSDP before the copyover, and restart from scratch after the copyover. The same can be done for MCCP though I forgot the specifics. TTYPE can be re-negotiated by disabling it before the copyover (some clients may not support this correctly - another thing I got to double check for tt++) and re-negotiate after the copyover, it's an option to save the TTYPE to file and reload it after the copyover, the same can be done for NAWS, and EOR.

No. IAC DONT is either a response to enable an option or a request to disable an option.
The assumption for most Telnet options is that they start disabled. Although some assume other initial states, like ECHO.

TT++ does not renegotiate TERMTYPE if it thinks it has already negotiated this; this is my problem for many options and just about any client other than MushClient. I would simply save socket states and reload them on copyover, but that seems sketchy; A: NakedMud doesn't reliably synch the socket UIDs up to connections on a copyover which means the data would be given to a bad socket, and B: Many protocols this can be a bad idea for, such as MSDP as you mentioned; though that's easy enough to put in conditionals.

I wonder if NEW-ENVIRON is widely supported enough that you could put client information in a variable on their side which the server could then check and use; that would, minimally, let me put the socket uid number the MUD has assigned that person away, and then use it to identify them to that data on a reconnect.

Maya/Rudha

I know other clients have had problems with this as well. MCCP seems to be the real killer, and you really do need to switch it off before the copyover. But for the other protocols I see no reason why you couldn't just save and restore their state.

I do actually renegotiate for MSDP, but only because I found it easier than saving and loading a list of REPORTABLE variables for each user. I save the TTYPE automatically though, as I want to be able to 'whois' offline players and see what client they're using.