21 Jun, 2009, kiasyn wrote in the 1st comment:

Votes: 0

The MUDBytes MSSP Crawler now supports MSSP via Telnet negotiation and plaintext as specified in the articles section.

21 Jun, 2009, quixadhal wrote in the 2nd comment:

Votes: 0

Out of idle curiosity, I assume if you've gotten the data via the telnet version, you wouldn't attempt to use the plain text version in the same session? I can't imagine anyone implementing BOTH on the same MUD server, but… :)

21 Jun, 2009, kiasyn wrote in the 3rd comment:

Votes: 0

it shouldn't get both sets of data - as once it has processed data it closes off the connection.

25 Jun, 2009, Mabus wrote in the 4th comment:

Votes: 0

Any way to be removed from the crawler?

Not planning on implementing MSSP, and having it check every 1-2 minutes is annoying in my logs.

25 Jun, 2009, Scandum wrote in the 5th comment:

Votes: 0

I could add the "CRAWL DELAY" MSSP variable to tell crawlers your preferred crawl speed. That way you could set it to once every 23 hours instead of 1-2 minutes.

25 Jun, 2009, David Haley wrote in the 6th comment:

Votes: 0

I think that's not exactly what he asked for. A value saying "don't come back" would be closer to what he asked for. :rolleyes:

So having a 'crawl delay' makes sense; I'd suggest it be a value in hours (why is it crawling every few minutes??), with zero meaning "do whatever you want", and -1 meaning "please don't crawl this MUD". (This is a case where -1 makes sense as a "no" answer.)

25 Jun, 2009, Hades_Kane wrote in the 7th comment:

Votes: 0

My logs have quite a bit of spam as well, coming from "static.xlhost.com."

This the same thing?

It stopped eventually… I wonder if my server decided it was spam and blocked it or they altered it?

25 Jun, 2009, Mabus wrote in the 8th comment:

Votes: 0

David Haley said:

I think that's not exactly what he asked for. A value saying "don't come back" would be closer to what he asked for.

What I am asking for is an "opt out" without having to alter my code to get it.

Shouldn't this system be an "opt in" system anyway?

It wasn't until I PMed Samson here that I had any idea what the connection could be. I had already banned the IP from my game internally, and from access to my ports from C-Panel.

This is from my log after I internally banned it from our game:

Quote

20090625.0706.50243 Info MUD Connection from 209.190.9.170: 1200
20090625.0706.50663 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0707.4643 Info MUD Connection from 209.190.9.170: 1200
20090625.0707.4643 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0708.45356 Info MUD Connection from 209.190.9.170: 1200
20090625.0708.45356 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0709.45835 Info MUD Connection from 209.190.9.170: 1200
20090625.0709.45835 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0710.45455 Info MUD Connection from 209.190.9.170: 1200
20090625.0710.47666 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0711.45401 Info MUD Connection from 209.190.9.170: 1200
20090625.0711.45401 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0712.45161 Info MUD Connection from 209.190.9.170: 1200
20090625.0712.45161 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0713.46128 Info MUD Connection from 209.190.9.170: 1200
20090625.0713.46128 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0714.46983 Info MUD Connection from 209.190.9.170: 1200
20090625.0714.46983 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0715.48716 Info MUD Connection from 209.190.9.170: 1200
20090625.0715.48716 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0717.06459 Info MUD Connection from 209.190.9.170: 1200
20090625.0717.06459 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0717.48450 Info MUD Connection from 209.190.9.170: 1200
20090625.0717.48451 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0718.45129 Info MUD Connection from 209.190.9.170: 1200
20090625.0718.45129 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0719.45158 Info MUD Connection from 209.190.9.170: 1200
20090625.0719.45158 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0720.45876 Info MUD Connection from 209.190.9.170: 1200
20090625.0720.45877 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0721.46273 Info MUD Connection from 209.190.9.170: 1200
20090625.0721.46273 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0722.47282 Info MUD Connection from 209.190.9.170: 1200
20090625.0722.47282 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0723.47866 Info MUD Connection from 209.190.9.170: 1200
20090625.0723.47867 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0724.46368 Info MUD Connection from 209.190.9.170: 1200
20090625.0724.46368 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0725.47391 Info MUD Connection from 209.190.9.170: 1200
20090625.0725.47391 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0726.48441 Info MUD Connection from 209.190.9.170: 1200
20090625.0726.48441 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0727.47543 Info MUD Connection from 209.190.9.170: 1200
20090625.0727.47543 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0728.00664 Info MUD Connection from 204.209.44.14: 1200
20090625.0728.47610 Info MUD Connection from 209.190.9.170: 1200
20090625.0728.47610 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0729.4866 Info MUD Connection from 209.190.9.170: 1200
20090625.0729.4866 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0730.46286 Info MUD Connection from 209.190.9.170: 1200
20090625.0730.46287 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0731.46503 Info MUD Connection from 209.190.9.170: 1200
20090625.0731.46504 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0732.47623 Info MUD Connection from 209.190.9.170: 1200
20090625.0732.47623 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0733.46482 Info MUD Connection from 209.190.9.170: 1200
20090625.0733.46482 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0734.52999 Info MUD Connection from 209.190.9.170: 1200
20090625.0734.5300 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0735.47461 Info MUD Connection from 209.190.9.170: 1200
20090625.0735.47462 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0736.47101 Info MUD Connection from 209.190.9.170: 1200
20090625.0736.47102 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0737.51420 Info MUD Connection from 209.190.9.170: 1200
20090625.0737.51420 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0738.50977 Info MUD Connection from 209.190.9.170: 1200
20090625.0738.50978 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0739.50605 Info MUD Connection from 209.190.9.170: 1200
20090625.0739.50606 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0740.48561 Info MUD Connection from 209.190.9.170: 1200
20090625.0740.48562 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0742.03904 Info MUD Connection from 209.190.9.170: 1200
20090625.0742.03905 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0742.52887 Info MUD Connection from 209.190.9.170: 1200
20090625.0742.54502 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0743.47899 Info MUD Connection from 209.190.9.170: 1200
20090625.0743.47899 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0745.00590 Info MUD Connection from 209.190.9.170: 1200
20090625.0745.00590 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0745.50915 Info MUD Connection from 209.190.9.170: 1200
20090625.0745.50916 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0746.51319 Info MUD Connection from 209.190.9.170: 1200
20090625.0746.51319 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0747.53800 Info MUD Connection from 209.190.9.170: 1200
20090625.0747.53800 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0748.48812 Info MUD Connection from 209.190.9.170: 1200
20090625.0748.48812 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0749.55848 Info MUD Connection from 209.190.9.170: 1200
20090625.0749.55849 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0750.57367 Info MUD Connection from 209.190.9.170: 1200
20090625.0750.57367 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0751.55245 Info MUD Connection from 209.190.9.170: 1200
20090625.0751.55245 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0752.52901 Info MUD Connection from 209.190.9.170: 1200
20090625.0752.52901 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0754.09868 Info MUD Connection from 209.190.9.170: 1200
20090625.0754.09868 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0755.01251 Info MUD Connection from 209.190.9.170: 1200
20090625.0755.01251 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0755.56416 Info MUD Connection from 209.190.9.170: 1200
20090625.0755.56417 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0756.55439 Info MUD Connection from 209.190.9.170: 1200
20090625.0756.55439 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0758.10828 Info MUD Connection from 209.190.9.170: 1200
20090625.0758.10828 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0758.57931 Info MUD Connection from 209.190.9.170: 1200
20090625.0758.57932 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0759.58922 Info MUD Connection from 209.190.9.170: 1200
20090625.0759.58922 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0800.48697 Info MUD Connection from 209.190.9.170: 1200
20090625.0800.48698 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0801.57570 Info MUD Connection from 209.190.9.170: 1200
20090625.0801.57571 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0803.0642 Info MUD Connection from 209.190.9.170: 1200
20090625.0803.0642 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0804.01310 Info MUD Connection from 209.190.9.170: 1200
20090625.0804.01311 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0804.56564 Info MUD Connection from 209.190.9.170: 1200
20090625.0804.56564 Info MUD Blocking a connection from 209.190.9.170 on port 1200
20090625.0806.01376 Info MUD Connection from 209.190.9.170: 1200
20090625.0806.01376 Info MUD Blocking a connection from 209.190.9.170 on port 1200

25 Jun, 2009, David Haley wrote in the 9th comment:

Votes: 0

Wow, that's pretty ridiculous. There's no reason a crawler should be doing that.

I don't agree that the crawler should be strictly opt-in; the whole point of crawlers is to, well, crawl around and find stuff automagically. An opt-out option should definitely exist, though.

25 Jun, 2009, Rendelven wrote in the 10th comment:

Votes: 0

The MSSP crawler has done the same on my MUD.

I log when I send MSSP to a client (helps me know which connections are just for that).

I received a barrage of connections from the same IP early this morning.

25 Jun, 2009, Cratylus wrote in the 11th comment:

Votes: 0

I think that being annoyed by log spam is fixed by not looking at
log spam, or avoiding the logging of spam. A mud is an internet
daemon hanging out there for anyone to rub up against it. As with
a web server's log, spam is just something I'd expect.

-Crat
http://lpmuds.net

25 Jun, 2009, Runter wrote in the 12th comment:

Votes: 0

Cratylus said:

I somewhat agree with this statement. If you're not designing your game to be able to handle this type of thing gracefully then you're going to get burned at some point down the road.
And if the entire issue is about log spam instead of some type of performance issue, then make a more graceful logging mechanism for connections.

That being said, It's obvious this crawler doesn't at all need to do that.

25 Jun, 2009, Mabus wrote in the 13th comment:

Votes: 0

While I can understand the supporters of this system defending it I cannot see the spam that we received as beneficial or inevitable. If I would have gotten these connections from any IP but one I am friendly with I would have emailed the ISP about being spammed. Instead I looked up the IP info, saw it was likely coming from here and contacted a staff member for the site.

All I am asking for is an opt-out option, either by specifically stating "I do not want the crawler" to the people running it (a web-based opt out or some such), which I am now doing, or similar to a robots exclusion standard.

I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things.

We just want to opt out of the crawler.

25 Jun, 2009, Runter wrote in the 14th comment:

Votes: 0

Mabus said:

I'm not a supporter of the system. I don't know anything about the system. All I'm saying is you need to guard yourself better. Not just from annoyances, but from actual attacks.
Asking everyone nicely not to spam you doesn't always work.

25 Jun, 2009, Mabus wrote in the 15th comment:

Votes: 0

Runter said:

We log connection info. When spammed we first block internal to the game, then from our area of the server (through C-Panel). What extra layer of security do you advise so we can "guard yourself better"? This isn't a bank, it is a MUD. While I could do a "number of failed connections" check and internally code a ban of the IP, it would still spam my error logs with "Connection refused by server". Perhaps you would like to start a thread describing these added levels of security beyond that?

I am posting about the MSSP crawler, specifically about a possible system to opt out, and that was the issue I was addressing.

25 Jun, 2009, Runter wrote in the 16th comment:

Votes: 0

Well, I'll just say I think the entire issue can be fixed by separating "spam" from your log.
But the point is moot, since you don't want to talk about logging techniques. ;)

25 Jun, 2009, Mabus wrote in the 17th comment:

Votes: 0

Which brings up a simple option:

Why not have the crawler check for a mssp.txt at the MUD website before sending connection info? If it encounters a 404 it doesn't crawl the MUD. Those that want to enable MSSP can simply drop a mssp.txt into the public_html.

Not even sure if this is possible or workable, but I thought to throw a positive idea rather then sound dismissive.

25 Jun, 2009, David Haley wrote in the 18th comment:

Votes: 0

Not all MUDs have websites, so it wouldn't be workable and besides defeats the purpose of the exercise. I maintain that the point of a crawler is to try crawling, and people can opt out if they want to. Think of how Google and other search engines work: you tell them to stop, you don't give them permission to start.

25 Jun, 2009, Mabus wrote in the 19th comment:

Votes: 0

David Haley said:

That is why I brought up robots.txt (the robots exclusion).

25 Jun, 2009, David Haley wrote in the 20th comment:

Votes: 0

Your proposal is not the same as robots.txt for search engines. You proposed that it only crawl if it finds the text file. I say that it should crawl unless it finds a text file saying not to. (Or, let MUDs opt out at the crawler website.)

Random Picks

Trigun Mud

World of Naruto

Malicious Bliss