25 Jun, 2009, Hades_Kane wrote in the 61st comment:
Votes: 0
David Haley said:
Have we read the thread yet, or are we still editorializing without really knowing what we're talking about?


Quote
post 13:
All I am asking for is an opt-out option, either by specifically stating "I do not want the crawler" to the people running it (a web-based opt out or some such), which I am now doing, or similar to a robots exclusion standard.

I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things.

We just want to opt out of the crawler.

Post 15:
Perhaps you would like to start a thread describing these added levels of security beyond that?

I am posting about the MSSP crawler, specifically about a possible system to opt out, and that was the issue I was addressing.

——————————-

16:
Well, I'll just say I think the entire issue can be fixed by separating "spam" from your log.
But the point is moot, since you don't want to talk about logging techniques. ;)

22:
This is why I am suggesting you get used to it, don't look at it, or drop mssp
connections in a logging black hole.

25:
So yes, if your logging system bugs you, you might have to fix it. C'est la vie, I guess. :shrug:
There's an easy solution here, in this case, so we can all be happy for now.

29:
Also. if one additional connection to the login screen every minute may "obstruct the communication between your intended users and your service" there's something seriously wrong with your design.

31:
We're giving you our opinion on how to avoid this problem because we can't solve the root of it for you in the past five seconds.

36:
Excuse me, what's the difference between this and "OMG GOOGLE IS CRAWLING MY WEBSITE!!!!"?

38:
Since eventually there will be more than a few crawlers/clients, it is hoped, and it
can be reasonably expected some will be broken or run by dicks, handling this
spam, if you log at this level, is something that's going to be "up to you" anyway.

40:
You are going to have to come up with some kind of log filtering for this kind of thing. It's the nature of the beast



Aaaand so on…

I think its pretty clear he stated very early in the thread what he was after, and I think its pretty clear no one else gave a damn.

So I repeat:
Then my question is, after he made it clear he isn't interested in hearing "fix your MUD" solutions, why wasn't the issue dropped, or simply left at "I'm sure an Administrator will be along to look at the issue."

It was you that seemed to insinuate that he needed to have this fixed RIGHT NOW as nothing in his posts seemed to indicate that at all to me, as that seems to be the main defense for continuing with the "fix your MUD" suggestions. I think when he posts asking for help and is basically told by the majority of replies "fix your MUD" as the solution, its not unreasonable for a perceived impolite response to follow what I see as an impolite disregard for his wishes that the topic remain about MSSP.
25 Jun, 2009, David Haley wrote in the 62nd comment:
Votes: 0
Quote
I think its pretty clear he stated very early in the thread what he was after, and I think its pretty clear no one else gave a damn.

HK, which part of the statement:
"people agreed with his original complaint and found it legitimate"
do you disagree with?

Quote
It was you that seemed to insinuate that he needed to have this fixed RIGHT NOW

Well, umm, yeah. He said that if he saw a single additional occurrence, he would pursue legal action. That seems pretty right-nowish, impatient and unreasonable to me.
25 Jun, 2009, Hades_Kane wrote in the 63rd comment:
Votes: 0
Quote
But you will never be able to stop a random person from pinging your game in ways you don't like or didn't foresee.


Again, that wasn't what he was trying to address… this was -specifically- about MSSP.

Quote
Anyhow, I'm not sure what your big problem is: you've managed to get everybody so far to agree that your complaint was valid, and yet here you are still terribly unhappy.


See your response above. That's basically where I've been standing. He is clearly not interested in hearing other solutions to this, and stated as such in post 13.

All of this "Drama" would have been easily avoided by a simple "The admins will deal with this in a timely fashion I imagine." It hasn't been him unwilling to drop this, it's been everyone else. Again, he said quite early in the thread if people wanted to discuss security issues, fine, do that somewhere else, but he's asking a -specific- question about a -specific- instance regarding -MSSP-. Even still after that's pointed out, people are still hounding the "Well, what about the other crawlers or attackers!" when that isn't even at issue here.

He's asking about MSSP.
25 Jun, 2009, Hades_Kane wrote in the 64th comment:
Votes: 0
David Haley said:
Quote
I think its pretty clear he stated very early in the thread what he was after, and I think its pretty clear no one else gave a damn.

HK, which part of the statement:
"people agreed with his original complaint and found it legitimate"
do you disagree with?


What part of:
"I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things.

We just want to opt out of the crawler.

Perhaps you would like to start a thread describing these added levels of security beyond that? "

Was an unreasonable request not worthy of respecting?
25 Jun, 2009, David Haley wrote in the 65th comment:
Votes: 0
Quote
Again, that wasn't what he was trying to address… this was -specifically- about MSSP.

Umm. Yes. People agreed that the crawler shouldn't be hammering his MUD, and furthermore, that he should be provided with a means to tell the crawler to stop entirely, without having to implement MSSP at all. And your point is?

Quote
Even still after that's pointed out, people are still hounding the "Well, what about the other crawlers or attackers!" when that isn't even at issue here.

Not to put too fine of a point on it, but, umm, you might want to look at post #23 when you refer to "the other crawlers or attackers", and see who wrote that. :wink:

Quote
(…) Was an unreasonable request not worthy of respecting?

It was respected and agreed with. Your point escapes me.
25 Jun, 2009, Hades_Kane wrote in the 66th comment:
Votes: 0
David Haley said:
Quote
Again, that wasn't what he was trying to address… this was -specifically- about MSSP.

Umm. Yes. People agreed that the crawler shouldn't be hammering his MUD, and furthermore, that he should be provided with a means to tell the crawler to stop entirely, without having to implement MSSP at all. And your point is?


My point is that you were consistently asking him to drop it, yet everyone else seemed unwilling to respect that he wasn't interested in "fix your MUD" suggestions (including you), and -that- is what continued this thread going. If you wanted the issue to be dropped so badly, I don't see why you couldn't respect that he wasn't interested in "fix your MUD" solutions.

Quote
Quote
Even still after that's pointed out, people are still hounding the "Well, what about the other crawlers or attackers!" when that isn't even at issue here.

Not to put too fine of a point on it, but, umm, you might want to look at post #23 when you refer to "the other crawlers or attackers", and see who wrote that. :wink:


I see nothing in post 23 about that:
http://www.mudbytes.net/index.php?a=topi...
What is your point?

Quote
(…) Was an unreasonable request not worthy of respecting?

Quote
It was respected and agreed with. Your point escapes me.


So when in post 13 he said:
"I am not putting down the system, not trying to discuss logging techniques of specific games nor am I wanting to argue about any issue surrounding such things."
And in post 15 when he said:
"Perhaps you would like to start a thread describing these added levels of security beyond that? "

That was respected and agreed with? Then how come I was able to point out numerous other posts following it where people continued to "discuss logging techniques of specific games" and "argue about any issue surrounding such things"? How come I never saw another thread describing these added levels of security beyond that"?
25 Jun, 2009, Runter wrote in the 67th comment:
Votes: 0
Any changes to that crawler isn't going to fix his overarching problem– That the next crawler his mud comes across might ping him 2 times a minute instead of 1.
It would be doing anyone else reading this thread out there a real disservice to just leave it at that.
Nobody has attacked him. This is a forum for discussion and there's plenty of room for legitimate debate.
25 Jun, 2009, Guest wrote in the 68th comment:
Votes: 0
So….. in the time it took me to read through (ok, skim through) you guys have started into a 5th page. In 3 hours. Perhaps a small amount of patience is due here, yes?

Mabus logs initial client connections. There's absolutely nothing wrong with that, though I personally find the practice a bit odd and took steps to not do so on my own MUD.

That connection logging exposed a problem I was not aware of due to not having been the one who wrote the crawler bot. I don't even know where the code is located. Yes, shocker, an admin that doesn't know everything about the site. It happens.

The issue will be dealt with. The complaint is perfectly valid. 1-2 minute intervals is way too short. We're not the Yahoo search bot here. These aren't webpages that absolutely must be indexed 20 minutes ago or you'll lose out on millions in potential advertising dollars. I'm not sure what exactly such a short interval hopes to accomplish. Chances are once every 30 minutes would suffice for whatever purpose there was in mind. This is something that should be dealt with at the crawler level, not another addition to the protocol that would then require all the existing implementations to go back and add more junk.

I also don't think giving us less than a full day to deal with this is reasonable at all. Nor is it really warranted to already be talking about escalating this to the ISP as a DoS attack. That's just absurd. We all have lives here. None of us are making money from this place, so ya'll will just have to be patient and know that it's being handled in due time.

Oh, and yes, an opt-out is certainly warranted. I can't see any valid reason for a crawler to just show up for something like this when it's not wanted. MSSP is not HTTP.
25 Jun, 2009, David Haley wrote in the 69th comment:
Votes: 0
I dunno about you, but I don't have any more time to spend on this ridiculous exercise. So I'll stop with you here. :rolleyes:

EDIT: in case it wasn't clear, that was in response to HK.
25 Jun, 2009, Hades_Kane wrote in the 70th comment:
Votes: 0
Runter said:
Any changes to that crawler isn't going to fix his overarching problem– That the next crawler his mud comes across might ping him 2 times a minute instead of 1.


I'm not debating that, seriously. I understand, and in fact agree.

But he's not interested in fixing his overarching problem, he's stated quite clearly, politely, and early, that he wasn't trying to discuss all that, and recommended another thread be started to discuss that if people wanted to. Considering this, why should anyone be surprised that someone might lose their patience after it continued anyway?

Yes, it's a forum, and yes, people can talk about whatever they want, but I don't see why its such a hard thing to simply understand "ok, he doesn't want to hear these solutions" and then simply not suggest them.

As absurd as it might be, if I posted saying "my leg itches, how can I fix this, but I'm not interested in hearing 'scratch it', what can I do?" well then, I don't want to hear "scratch it" and if that's all anyone replies, should they be surprised when I insist on more than one occasion that's not what I'm after, and they continue, that I might begin to lose patience? Poor analogy, I know, but that's pretty much what happened.
25 Jun, 2009, Hades_Kane wrote in the 71st comment:
Votes: 0
Samson said:
So….. in the time it took me to read through (ok, skim through) you guys have started into a 5th page.


Be careful just skimming a thread, Samson, I hear that isn't taken to very kindly 'round these parts :p
25 Jun, 2009, Hades_Kane wrote in the 72nd comment:
Votes: 0
David Haley said:
I dunno about you, but I don't have any more time to spend on this ridiculous exercise. So I'll stop with you here.


Helped the last part of my work day go by quicker :)

Edited to add:

Oh, but when you do have some more time to spend on it, I would be interested to learn how you think that his wishes not to discuss "fix your MUD" solutions was agreed with and respected, because I'm still quite curious about that.
25 Jun, 2009, Runter wrote in the 73rd comment:
Votes: 0
You disregarded the part about it being a disservice to other people to not mention an actual fix to the problem Even if he doesn't want to hear it. People can read it and make up their own minds instead of him taking offense to a parallel idea.

Again, nobody disagreed with him that it was a problem. The real question is, "How do we fix it?" And many of us tried to give a realistic answer to the problem that applies to more than The Great Mudbytes Cyber Attack of 2009.
25 Jun, 2009, Cratylus wrote in the 74th comment:
Votes: 0
I'm really, really sorry you don't want to hear it, I understand you've
heard it before, but please understand that scratching it actually is
the answer.

Anyway, since Mabus didn't actually start this thread, I'm not sure
what authority you think he has in steering who can talk about what.

I find your contributions normally to be very reasonable and
principled, HK, but today I am having difficulty following you.

In any case, to address the opt-in/out question, I am curious how
this would be done without actually implementing MSSP.

What exactly is being proposed here, technically?

-Crat
http://lpmuds.net
25 Jun, 2009, Hades_Kane wrote in the 75th comment:
Votes: 0
Runter said:
You disregarded the part about it being a disservice to other people to not mention an actual fix to the problem Even if he doesn't want to hear it. People can read it and make up their own minds instead of him taking offense to a parallel idea.


That is a valid point, however I don't recall anyone even mentioning that prior to this. Had someone simply said "I understand YOU might not be interested in hearing these solutions, I think it's valid that other people might be" and then continued, all of this also would have been avoided. As it stands, it simply looks like no one was interested in respecting that he didn't care to fix his overall problem.


Quote
The Great Mudbytes Cyber Attack of 2009.


I think that level of sarcasm is unnecessary.
25 Jun, 2009, David Haley wrote in the 76th comment:
Votes: 0
Finally, a reasonable question. :rolleyes:
Cratylus said:
In any case, to address the opt-in/out question, I am curious how
this would be done without actually implementing MSSP.

What exactly is being proposed here, technically?

I imagine that the website running crawler would have to provide some mechanism for MUDs to opt out. Presumably this would need to be done by the person who registered the listing.

Another option is to have a robots.txt-like entry that lets you opt out, although this has the obvious problem of not working for people without webspace.

Probably a combination of both is appropriate.

Of course this all assumes that the people implementing the crawler actually honor any of this, and dealing with this reasonably is of course the question that spawned some 60 posts.
25 Jun, 2009, Runter wrote in the 77th comment:
Votes: 0
Would it be possible to make it a form when registering a mud to the listings (assuming that is where the crawler gets its destinations) to let them opt out when they create the listing in the first place?

People can update their listings I'm sure.
25 Jun, 2009, David Haley wrote in the 78th comment:
Votes: 0
Presumably. That brings up an interesting point, of course, which is what to do with crawlers that don't get their destinations from listings, or crawlers that scrape listings that might not even know about MSSP – and so the listing wouldn't have the opt-out field.
25 Jun, 2009, Hades_Kane wrote in the 79th comment:
Votes: 0
Cratylus said:
I find your contributions normally to be very reasonable and
principled, HK, but today I am having difficulty following you.


Fair enough :p

Quote
In any case, to address the opt-in/out question, I am curious how
this would be done without actually implementing MSSP.

What exactly is being proposed here, technically?


I think I mentioned it before, but I think the best solution would be to edit your listing directly to determine whether you want to be included. Surely, games not listed won't be crawled, as the point of the crawler is for updated listings, right?
25 Jun, 2009, Runter wrote in the 80th comment:
Votes: 0
David Haley said:
Presumably. That brings up an interesting point, of course, which is what to do with crawlers that don't get their destinations from listings, or crawlers that scrape listings that might not even know about MSSP – and so the listing wouldn't have the opt-out field.


*Inserts pithy comment here.*
60.0/136