Pretty much, yes. MCCP is agnostic as to what data it's compressing: in fact, in practice for a long time it was only compressing the standard output you'd see in a telnet client. When MUD servers started adding OOB data, it was compressed naturally.

MCCP is perhaps best thought of as a layer between the data the MUD sends, the actual bytes sent over the network, and the data the client receives. So it doesn't really matter if the stream has additional meta-data; MCCP just sits in between and compresses everything the server sends. In that respect, yes, it's just a zlib implementation (in fact it uses zlib compression). The main difference from straight zlib is that it's a telnet protocol and therefore has negotiation etc.

(Maybe that's what you said and I misunderstood; sorry if that's the case. I just wanted to be clear on the meta-data point; MCCP is agnostic as to what the data actually is, it just compresses stuff.)


When I say that it's near-JSON, it's that the ATCP2 spec has this weird allowance for sending list data in non-list form, with the client being expected to add brackets as necessary in some cases. ATCP2 can be sent with strict JSON, but technically speaking the standard (at least from a few months ago, I don't know what Zugg and IRE have done since) allows a looser form of JSON that a standard-compliant parser would not understand. I can dig up the details if you care, but I don't think it's a terribly important point for now at least.

It's not even MUD-specific - it's basically just a zlib implementation (which you've now added) with a simple handshake to establish support. The name MCCP is just because the standard was invented for muds, by mud developers.

Thus my sardonic quip: On TMC you said you'd read about MCCP, yet you dismissed it as highly insufficient, and I've also argued the advantages of compression (without success) in this thread. Now you've effectively gone ahead and implemented it (at least the important part), which is why I interpreted your sudden excitement as tongue-in-cheek and responded in kind.

Also note that MCCP is completely in-band other than a single activation command (which is something I disagree with, but it's too well established to change now).

At the time, I had no clear understanding that MCCP is a compression layer and not a protocol. When I took a brief look at its specs, I didn't get that immediately, either (I thought it had optional compression support, not that that's all it is).

So, no, I don't do tongue-in-cheek if I can help it. I was genuinely excited, but not so much at the bandwidth savings as at the ease of implementing zipped async JSON, which I didn't know modern browsers supported so readily. The best kind of optimization for me is the kind that lets me write two lines of code and then focus on something I actually enjoy. The credit for opening this possibility belongs to this thread and so I gave it.

Backing up for a minute, just to be clear. You said…"Easiest for the MUD would be to set a time stamp whenever a room is modified. When a player enters a room the mud would send the vnum and timestamp…". This certainly works if you are already storing update time on your objects. The "simpler" approach I'm using is just sending the object number and a calculated hash code. We're likely talking about the same amount of data (presuming timestamp and hash code are the same length), just no extra storage or modification is added to each object server side. In other words, after the first send of a message, we only send 8 or so bytes to represent that message every time it's used.

No, zlib streams do not implement cacheing.

It's certainly possible to implement both message cacheing and zlib compression.

I have strong reservations about accepting any of Lasher's claims.

Testing zlib compression on typical mud ...

short: 17, 25 -47.05882352941176
desc : 362, 210 41.988950276243095
help : 951, 352 62.9863301787592
help2: 781, 421 46.094750320102435
aaa : 3456, 40 98.8425925925926 <— untypical repeating letters.

—————

If I published MUDP, it would only confuse the issue.

byte data-type
long long hash-code

I'm sending 9 bytes for a cache request.

If we take desc above as a typical message and assume it gets sent 10 times during a session:
Cache: 443 bytes (362+9+9+9+9+9+9+9+9+9)
Zlib: 2100 bytes (210*10)

Suppose we compress the data sent during the cache-request response.

Zlib+Cache: 291 bytes (210+9+9+9+9+9+9+9+9+9)

Am I right?

It takes about 50 lines of code to convert MSDP data to JSON tables, and probably about 25 to convert MSDP data to Lua tables. Not exactly 2 lines, but it's trivial. The telnet handling itself can be a pain in the ass to get right, but there's MTH in the MudBytes repository which you could take a look at.


Zlib supports run-length encoding, which seems to be a fancy word for finding and caching repetitive sequences in raw data. I think run-length encoding can be disabled, and specifically enabled, which might be the problem with your tests? I'm not sure how to interpret the data you just posted.

While at it, would compression of input be an interesting addition for MCCP3 ?

RLE is local to one message (well, more accurately, a localized stream) whereas caching would typically be across messages or on a whole-message basis; that is the caching that zlib does not support that Tyche is (presumably) referring to. For example, caching could send an entire chunk of 10mb as a single cache identifier, whereas zlib would still be compressing it one way or the other.

No I'm wrong.

With RLE I can get 5 to 1 ratio, sending the same text over and over.
Sending desc 10 times in a row for example:
Zlib: 297 bytes

The only problem is the 256K memory overhead per zlib user on the server.
Obviously that could be fiddled with server side.

You can fix the memory usage by calling deflateInit2, for example:

deflateInit2(stream, Z_BEST_COMPRESSION, Z_DEFLATED, 13, 6, Z_DEFAULT_STRATEGY)

This would use 64K of memory. The 13 (default is 15) stands for widowbits, which I think is the history buffer. At 15 it uses 128K, so 13 is 32K of memory. Assuming it's a literal history buffer, and an average line of 64 characters, we're talking about a history size of about 500 lines. The 6 is the memlevel variable, which is 8 (128K) by default, and seems to mainly affect speed. Zlib has a Z_RLE setting, but from what I've seen it'll give a 50% compression ratios, Z_HUFFMAN_ONLY gives a 45% compression ratio, and Z_DEFAULT_STRATEGY (which I assume is both RLE and HUFFMAN) typically gives a 80% compression ratio.

Here's the automapper embedded inside the app, with all the delights of pinch and 2tap zooming. The overlay mode works, too, but will require some more tweaking for useability.





[quote=[url=/topic-3167-52543#p52543]Scandum[/url]
It takes about 50 lines of code to convert MSDP data to JSON tables, and probably about 25 to convert MSDP data to Lua tables. Not exactly 2 lines, but it's trivial. The telnet handling itself can be a pain in the ass to get right, but there's MTH in the MudBytes repository which you could take a look at.
[/quote]

In the case of my automapper mini-project, I'm writing both server and client. Since my choice of client is a web browser, and since I have no prior server-side work to leverage, JSON + gzip seems clearly the best way to go. It may even be the only way to go–I'm not sure if it's even possible to write a JS decompressor for MSDP that works and performs well inside a browser sandbox.

Now, if you're writing a server-side-only implementation with advanced MUD clients in mind, or if you're writing a MUD client that wants to support MSDP, those are very different scenarios and any amount of code is probably worth it.

That's a cool interface, plamzi and a really cool way to represent your game. The art reminds me of some of the classic turn-based RPGs I used to play.

For the Android OS, I've been using BlowTorch. The developer is very responsive and has reworked the code so it supports more MUDs (meaning some of the issues that Lyanic, ATT_Turan, and Kline(?) saw should be fixed). Recent updates added a lot of nice options, including fullscreen (hurray for more real-estate!), encoding options (UTF-8, etc.), and timers.

Honestly, the best things I like about the app are MCCP support and the ability to add buttons. I made my own D-Pad that, when flipped (click, drag, release), also opens doors. I have a few button sets that I use like menus so i can quickly switch to my "Travel buttons" set, or my "Training/Stats" set. And yeah, plamzi, from the developer's forums it seems like he's found a few annoying bugs with Android's OS (like features that stopped working in 2.2).

Sounds right. My counterpart to BlowTorch for iPhone (MUDMaster) is currently being rated down by users because Apple's latest dev kit broke stuff that only the upcoming iOS update will fix. Of course, when users update their phones and the app starts working again, no-one's going to remember to change their review or infer from what happened that it was really Apple's fault. App Store users generally assume everything is the app developer's fault, including the fact that an app can't stay connected in the background. Apple misadvertises multitasking (there's really no such thing for networked apps) and we devs get to reap the anger of consumers who expect it.

As I've mentioned earlier in this thread, developing for smartphone platforms is extremely challenging even when it comes to such "simple" features as smooth-scrolling text. First you fight to get it working, then you fight to keep it working as multiple yearly OS updates start pouring in. If you like the BlowTorch dev and his app, report any issues you find, together with your device and OS version, and the MUD in question. Your goal should be to help him reproduce and fix the issue. Trust me, he'll need all the help he can get.

And then, you're lucky when you're on an iPhone and only have a few versions of the device to worry about – not the plethora of Android devices you get out there! :smile:

True, I am lucky that way. But on the other hand, Google's motto is "Don't be evil." And Apple's motto is clearly the same, except for the "don't" part.

Haha. Yeah, with Android's freedom comes many hiccups. In BlowTorch, you can import and export settings for editing, but it's difficult if you ever want to share them because of all the different screen sizes and aspect ratios (works just fine between similar models, kinda glad I have a popular one). I made a number of triggers and button sets while I was going through a MUD's tutorial and I thought it'd be really cool to share them or download someone else's, especially if you want to try out a different MUD.

Those are the exact questions he asks and unless you add comments in Android's crash reporter, he only gets what line of code the error occurred on with no way to contact who sent the report. Sorry to hear about the iOS crapping on your code. I wonder if my iPad Touch is still working… I should see if I can play around with MUDMaster on it. I bet developers have a love/hate (mostly hate) relationship with the comment system in the app market places. Like you said, who remembers to re-review an app?

Ah, where to begin…

Maybe it's because I'm lucky, but I've yet to receive a useful crash report through the AppStore. On the other hand, its review pages seem to be dominated either by idiot teenagers (in which case there's hope) or idiots period. Apps used to get classic one-line reviews like "This app sux." and "This app don't works." but the new fashion is to leave reviews like "OMG, I love this game! But ranger is too week, WTF! I give one star til you make ranger ultimat. Comon ppl!"

And no means to respond to reviews or get in touch with your customers. How can you not love it?

Ugh. Reviews like that, especially when only 1 in 100 people who use your app review it, can kill small developers. Android has a comment rating system (helpful, unhelpful, spam), but I'm convinced it doesn't do anything. That's why I usually browse through reviews a bit before downloading apps and try to ignore the obvious idiots. Unfortunately, if I'm scrolling through and see several apps that do the same thing, I'll usually ignore the lowest rated ones.

I wouldn't trust Google just because of their informal motto. They collect all the data they can from every inch of the internet and every device within reach. They work closely with the US government and are as important to the US's internet interests as NASA is to their space interests.

Yes, I feel much safer when unknown small companies do the exact same thing, but provide no useful services to me in the process.

That is the kind of argument I used to make when I was a snot-nosed teenager in school, so I understand the thought. The reality is, the internet is not a secured network. If you don't like anyone and everyone reading, archiving, and exploiting each and every thing you put on it, DON'T PUT YOUR DATA ON THE INTERNET. It really is that simple.