MUDBytes Community - Viewing Topic: MUDBytes MSSP Crawler

MudBytes

» MUDBytes Community » Coding Discussions » Protocols » MSSP » MUDBytes MSSP Crawler

Mark all Recent Posts

< Newer Topic :: Older Topic >

Topic Locked

Pages: << prev 1, 2, 3, 4, 5 ... next >>

kiasyn Wizard Group: Administrators Posts: 857 Joined: May 15, 2006	#1 Posted Jun 21, 2009, 2:18 am
	The MUDBytes MSSP Crawler now supports MSSP via Telnet negotiation and plaintext as specified in the articles section.
	......................... WWW

quixadhal Wizard Group: Members Posts: 1,256 Joined: Oct 17, 2007	#2 Posted Jun 21, 2009, 5:27 am
	Out of idle curiosity, I assume if you've gotten the data via the telnet version, you wouldn't attempt to use the plain text version in the same session? I can't imagine anyone implementing BOTH on the same MUD server, but... :)
	.........................

kiasyn Wizard Group: Administrators Posts: 857 Joined: May 15, 2006	#3 Posted Jun 21, 2009, 5:44 am
	it shouldn't get both sets of data - as once it has processed data it closes off the connection.
	......................... WWW

Mabus Sorcerer Group: Members Posts: 255 Joined: Aug 24, 2006	#4 Posted Jun 25, 2009, 9:06 am
	Any way to be removed from the crawler? Not planning on implementing MSSP, and having it check every 1-2 minutes is annoying in my logs.
	......................... -Mabus Insomniac Coder WWW AIM

Scandum Wizard Group: Members Posts: 1,105 Joined: Aug 8, 2006	#5 Posted Jun 25, 2009, 9:13 am
	I could add the "CRAWL DELAY" MSSP variable to tell crawlers your preferred crawl speed. That way you could set it to once every 23 hours instead of 1-2 minutes.
	......................... TinTin++ Mud Client for Windows, Linux, and Mac OS X.

David Haley Wizard Group: Members Posts: 5,728 Joined: Jun 30, 2007	#6 Posted Jun 25, 2009, 10:18 am
	I think that's not exactly what he asked for. A value saying "don't come back" would be closer to what he asked for. So having a 'crawl delay' makes sense; I'd suggest it be a value in hours (why is it crawling every few minutes??), with zero meaning "do whatever you want", and -1 meaning "please don't crawl this MUD". (This is a case where -1 makes sense as a "no" answer.)
	......................... -- d.c.h -- BabbleMUD Project (custom codebase) Legends of the Darkstone (head coder) http://david.the-haleys.org ......................... WWW

Hades_Kane Wizard Group: Moderators Posts: 786 Joined: May 31, 2006	#7 Posted Jun 25, 2009, 11:19 am
	My logs have quite a bit of spam as well, coming from "static.xlhost.com." This the same thing? It stopped eventually... I wonder if my server decided it was spam and blocked it or they altered it?
	......................... -Diablos of End of Time, eotmud.com:4000, Final Fantasy themed -- Seeking Builders! For more info visit: http://www.eotmud.com/ WWW AIM

Mabus Sorcerer Group: Members Posts: 255 Joined: Aug 24, 2006	#8 Posted Jun 25, 2009, 1:18 pm
	David Haley said: I think that's not exactly what he asked for. A value saying "don't come back" would be closer to what he asked for. What I am asking for is an "opt out" without having to alter my code to get it. Shouldn't this system be an "opt in" system anyway? It wasn't until I PMed Samson here that I had any idea what the connection could be. I had already banned the IP from my game internally, and from access to my ports from C-Panel. This is from my log after I internally banned it from our game: Quote: 20090625.0706.50243 Info MUD Connection from 209.190.9.170: 1200 20090625.0706.50663 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0707.4643 Info MUD Connection from 209.190.9.170: 1200 20090625.0707.4643 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0708.45356 Info MUD Connection from 209.190.9.170: 1200 20090625.0708.45356 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0709.45835 Info MUD Connection from 209.190.9.170: 1200 20090625.0709.45835 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0710.45455 Info MUD Connection from 209.190.9.170: 1200 20090625.0710.47666 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0711.45401 Info MUD Connection from 209.190.9.170: 1200 20090625.0711.45401 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0712.45161 Info MUD Connection from 209.190.9.170: 1200 20090625.0712.45161 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0713.46128 Info MUD Connection from 209.190.9.170: 1200 20090625.0713.46128 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0714.46983 Info MUD Connection from 209.190.9.170: 1200 20090625.0714.46983 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0715.48716 Info MUD Connection from 209.190.9.170: 1200 20090625.0715.48716 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0717.06459 Info MUD Connection from 209.190.9.170: 1200 20090625.0717.06459 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0717.48450 Info MUD Connection from 209.190.9.170: 1200 20090625.0717.48451 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0718.45129 Info MUD Connection from 209.190.9.170: 1200 20090625.0718.45129 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0719.45158 Info MUD Connection from 209.190.9.170: 1200 20090625.0719.45158 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0720.45876 Info MUD Connection from 209.190.9.170: 1200 20090625.0720.45877 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0721.46273 Info MUD Connection from 209.190.9.170: 1200 20090625.0721.46273 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0722.47282 Info MUD Connection from 209.190.9.170: 1200 20090625.0722.47282 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0723.47866 Info MUD Connection from 209.190.9.170: 1200 20090625.0723.47867 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0724.46368 Info MUD Connection from 209.190.9.170: 1200 20090625.0724.46368 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0725.47391 Info MUD Connection from 209.190.9.170: 1200 20090625.0725.47391 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0726.48441 Info MUD Connection from 209.190.9.170: 1200 20090625.0726.48441 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0727.47543 Info MUD Connection from 209.190.9.170: 1200 20090625.0727.47543 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0728.00664 Info MUD Connection from 204.209.44.14: 1200 20090625.0728.47610 Info MUD Connection from 209.190.9.170: 1200 20090625.0728.47610 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0729.4866 Info MUD Connection from 209.190.9.170: 1200 20090625.0729.4866 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0730.46286 Info MUD Connection from 209.190.9.170: 1200 20090625.0730.46287 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0731.46503 Info MUD Connection from 209.190.9.170: 1200 20090625.0731.46504 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0732.47623 Info MUD Connection from 209.190.9.170: 1200 20090625.0732.47623 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0733.46482 Info MUD Connection from 209.190.9.170: 1200 20090625.0733.46482 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0734.52999 Info MUD Connection from 209.190.9.170: 1200 20090625.0734.5300 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0735.47461 Info MUD Connection from 209.190.9.170: 1200 20090625.0735.47462 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0736.47101 Info MUD Connection from 209.190.9.170: 1200 20090625.0736.47102 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0737.51420 Info MUD Connection from 209.190.9.170: 1200 20090625.0737.51420 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0738.50977 Info MUD Connection from 209.190.9.170: 1200 20090625.0738.50978 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0739.50605 Info MUD Connection from 209.190.9.170: 1200 20090625.0739.50606 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0740.48561 Info MUD Connection from 209.190.9.170: 1200 20090625.0740.48562 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0742.03904 Info MUD Connection from 209.190.9.170: 1200 20090625.0742.03905 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0742.52887 Info MUD Connection from 209.190.9.170: 1200 20090625.0742.54502 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0743.47899 Info MUD Connection from 209.190.9.170: 1200 20090625.0743.47899 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0745.00590 Info MUD Connection from 209.190.9.170: 1200 20090625.0745.00590 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0745.50915 Info MUD Connection from 209.190.9.170: 1200 20090625.0745.50916 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0746.51319 Info MUD Connection from 209.190.9.170: 1200 20090625.0746.51319 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0747.53800 Info MUD Connection from 209.190.9.170: 1200 20090625.0747.53800 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0748.48812 Info MUD Connection from 209.190.9.170: 1200 20090625.0748.48812 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0749.55848 Info MUD Connection from 209.190.9.170: 1200 20090625.0749.55849 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0750.57367 Info MUD Connection from 209.190.9.170: 1200 20090625.0750.57367 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0751.55245 Info MUD Connection from 209.190.9.170: 1200 20090625.0751.55245 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0752.52901 Info MUD Connection from 209.190.9.170: 1200 20090625.0752.52901 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0754.09868 Info MUD Connection from 209.190.9.170: 1200 20090625.0754.09868 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0755.01251 Info MUD Connection from 209.190.9.170: 1200 20090625.0755.01251 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0755.56416 Info MUD Connection from 209.190.9.170: 1200 20090625.0755.56417 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0756.55439 Info MUD Connection from 209.190.9.170: 1200 20090625.0756.55439 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0758.10828 Info MUD Connection from 209.190.9.170: 1200 20090625.0758.10828 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0758.57931 Info MUD Connection from 209.190.9.170: 1200 20090625.0758.57932 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0759.58922 Info MUD Connection from 209.190.9.170: 1200 20090625.0759.58922 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0800.48697 Info MUD Connection from 209.190.9.170: 1200 20090625.0800.48698 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0801.57570 Info MUD Connection from 209.190.9.170: 1200 20090625.0801.57571 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0803.0642 Info MUD Connection from 209.190.9.170: 1200 20090625.0803.0642 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0804.01310 Info MUD Connection from 209.190.9.170: 1200 20090625.0804.01311 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0804.56564 Info MUD Connection from 209.190.9.170: 1200 20090625.0804.56564 Info MUD Blocking a connection from 209.190.9.170 on port 1200 20090625.0806.01376 Info MUD Connection from 209.190.9.170: 1200 20090625.0806.01376 Info MUD Blocking a connection from 209.190.9.170 on port 1200
	......................... -Mabus Insomniac Coder Last edited Jun 25, 2009, 1:21 pm by Mabus WWW AIM

David Haley Wizard Group: Members Posts: 5,728 Joined: Jun 30, 2007	#9 Posted Jun 25, 2009, 1:24 pm
	Wow, that's pretty ridiculous. There's no reason a crawler should be doing that. I don't agree that the crawler should be strictly opt-in; the whole point of crawlers is to, well, crawl around and find stuff automagically. An opt-out option should definitely exist, though.
	......................... -- d.c.h -- BabbleMUD Project (custom codebase) Legends of the Darkstone (head coder) http://david.the-haleys.org ......................... WWW

Rendelven Apprentice Group: Members Posts: 21 Joined: Jan 7, 2009	#10 Posted Jun 25, 2009, 1:51 pm
	The MSSP crawler has done the same on my MUD. I log when I send MSSP to a client (helps me know which connections are just for that). I received a barrage of connections from the same IP early this morning.
	......................... R. Scratch - C++ Codebase: http://www.dragonssquare.com Cruentus Apocalypsis http://cruentus.genesismuds.com

Cratylus Wizard Group: Members Posts: 1,224 Joined: May 22, 2006	#11 Posted Jun 25, 2009, 2:06 pm
	I think that being annoyed by log spam is fixed by not looking at log spam, or avoiding the logging of spam. A mud is an internet daemon hanging out there for anyone to rub up against it. As with a web server's log, spam is just something I'd expect. -Crat http://lpmuds.net
	WWW

Runter Wizard Group: Members Posts: 1,074 Joined: Jun 1, 2006	#12 Posted Jun 25, 2009, 2:09 pm
	Cratylus said: I think that being annoyed by log spam is fixed by not looking at log spam, or avoiding the logging of spam. A mud is an internet daemon hanging out there for anyone to rub up against it. As with a web server's log, spam is just something I'd expect. -Crat http://lpmuds.net I somewhat agree with this statement. If you're not designing your game to be able to handle this type of thing gracefully then you're going to get burned at some point down the road. And if the entire issue is about log spam instead of some type of performance issue, then make a more graceful logging mechanism for connections. That being said, It's obvious this crawler doesn't at all need to do that.
	......................... -Heath For once you have tasted flight Ruby you will walk the earth with your eyes turned skywards, for there you have been and there you will long to return. -- Leonardo Da Vinci Yukihiro Matsumoto AIM Yahoo MSN

Mabus Sorcerer Group: Members Posts: 255 Joined: Aug 24, 2006	#13 Posted Jun 25, 2009, 2:24 pm
	While I can understand the supporters of this system defending it I cannot see the spam that we received as beneficial or inevitable. If I would have gotten these connections from any IP but one I am friendly with I would have emailed the ISP about being spammed. Instead I looked up the IP info, saw it was likely coming from here and contacted a staff member for the site. All I am asking for is an opt-out option, either by specifically stating "I do not want the crawler" to the people running it (a web-based opt out or some such), which I am now doing, or similar to a robots exclusion standard. I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things. We just want to opt out of the crawler.
	......................... -Mabus Insomniac Coder WWW AIM

Runter Wizard Group: Members Posts: 1,074 Joined: Jun 1, 2006	#14 Posted Jun 25, 2009, 2:30 pm
	Mabus said: While I can understand the supporters of this system defending it I cannot see the spam that we received as beneficial or inevitable. If I would have gotten these connections from any IP but one I am friendly with I would have emailed the ISP about being spammed. Instead I looked up the IP info, saw it was likely coming from here and contacted a staff member for the site. All I am asking for is an opt-out option, either by specifically stating "I do not want the crawler" to the people running it (a web-based opt out or some such), which I am now doing, or similar to a robots exclusion standard. I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things. We just want to opt out of the crawler. I'm not a supporter of the system. I don't know anything about the system. All I'm saying is you need to guard yourself better. Not just from annoyances, but from actual attacks. Asking everyone nicely not to spam you doesn't always work.
	......................... -Heath For once you have tasted flight Ruby you will walk the earth with your eyes turned skywards, for there you have been and there you will long to return. -- Leonardo Da Vinci Yukihiro Matsumoto Last edited Jun 25, 2009, 2:31 pm by Runter AIM Yahoo MSN

Mabus Sorcerer Group: Members Posts: 255 Joined: Aug 24, 2006	#15 Posted Jun 25, 2009, 2:41 pm
	Runter said: I'm not a supporter of the system. I don't know anything about the system. All I'm saying is you need to guard yourself better. Not just from annoyances, but from actual attacks. Asking everyone nicely not to spam you doesn't always work. We log connection info. When spammed we first block internal to the game, then from our area of the server (through C-Panel). What extra layer of security do you advise so we can "guard yourself better"? This isn't a bank, it is a MUD. While I could do a "number of failed connections" check and internally code a ban of the IP, it would still spam my error logs with "Connection refused by server". Perhaps you would like to start a thread describing these added levels of security beyond that? I am posting about the MSSP crawler, specifically about a possible system to opt out, and that was the issue I was addressing.
	......................... -Mabus Insomniac Coder WWW AIM

Topic Locked

Pages:<< prev 1, 2, 3, 4, 5 ... next >>

Powered by QSF Portal © 2006-2009 The QSF Portal Development Team
Based on Quicksilver Forums © 2005-2006 The Quicksilver Forums Development Team
Based on MercuryBoard © 2001-2005 The Mercury Development Team
MudBytes skin © 2006-2007 by The MudBytes Team

Feb 9, 2010, 12:03 pm

Valid XHTML 1.1!

Valid CSS!