17 Jan, 2013, arendjr wrote in the 61st comment:
Votes: 0
Rarva.Riendf said:
Having a database does not mean you have to work with it all the time. Nor push everything into it in real time as well. I work totally in-memory, I occasionaly save stuff on 'some' events, I am pretty sure the filesystem-io is a lot worse than what a performance a database that could also work in memory most of the time could provide.

If you're doing everything in-memory (I'm doing that too), why still use a database?

The reasons for using a database are SQL and indices. If you're doing everything in-memory you're having your own indices and references, and you don't need SQL.

And no, the disk I/O is not better. Whenever you save to the database it will immediately sync to disk. Whatever it's keeping in memory too is useless since you already got your own in-memory representation.
17 Jan, 2013, Rarva.Riendf wrote in the 62nd comment:
Votes: 0
arendjr said:
If you're doing everything in-memory (I'm doing that too), why still use a database?


Because a query can often be a lot easier to read than code parsing files. (and easier to write than the code to do it as well).
At boot and at shutdown.

And no a database will not flush to disk at every second, it depends on when you actually commit. (and even how you configure it as well. You can lower the integrity to improve performance, no need to go write every 1 hp loss you take as an example).

Don't think I am advocating for Database though, I think the old muds can do very well without for the most part especially since they have been designed to work without to start with. But since you have a 'modern' engine, it is funny you dont use one (IMHO)
17 Jan, 2013, arendjr wrote in the 63rd comment:
Votes: 0
Rarva.Riendf said:
Because a query can often be a lot easier to read than code parsing files. (and easier to write than the code to do it as well).
At boot and at shutdown.

Fair enough. Then it largely comes down to a matter of personal preference I guess.

Personally I'm using JSON files to store the data, which given that I had to write JSON import/export anyway for communicating with the web frontend it's not really adding complexity. And not requiring a database setup is actually a way to reduce complexity, in my opinion.

Rarva.Riendf said:
And no a database will not flush to disk at every second, it depends on when you actually commit. (and even how you configure it as well. You can lower the integrity to improve performance, no need to go write every 1 hp loss you take as an example).

Wrapping everything in long-running transactions is something I could never quite wrap my head around… It feels like a misuse of the transaction concept. But then again, I guess it's indeed a pragmatic way to reduce the writes, so why not? :)

The way I do it is by marking the objects that get modified, and after each event on the game thread is completed I copy those modified objects to the sync thread. This way, if I update 7 properties on a game object, I still only have to do one disk write.
17 Jan, 2013, Kelvin wrote in the 64th comment:
Votes: 0
arendjr, the last popular game I ran (Battletech: The Frontier Lands) had a real-time, coordinate based combat system that is a whole lot more complex than what you were saying in a monolithic form. It's important to understand that multi-process != performance. There is no real correlation. I think you'll find that you'll spend *a lot* more time passing data back and forth, and the potential for things like de-syncs or obscure parameter issues (using your pathfinding example) are painful to debug. Depending on which messaging format you use, you may spend a bit more CPU time serializing/de-serializing data, as well. You'll also want to make sure that you gracefully degrade if one of your external "services" fail.

As far as the DB stuff goes, I think the important thing to add (which has already been kind of mentioned) is that you need not use a DB in real-time. I use it *just* for persistence in my game. The game server queries the objects at game startup, then is just write-only. All I/O is async, so saving to it incurs no measurable performance hit, and I only save when an individual object is modified. I'll have some web integration that *will* read/write, and can ensure that no one record gets touched by the game and the MUD at the same time. I also get a number of data integrity and scaling benefits over the flatfile approach, though, again, scaling and performance is not going to be a problem for 98% of us.

You know your game best, but the reasons you have given so far make me think that you are guessing what would be optimal, and it sounds like you're guessing the more complex route first. Very little of what you have said so far is based on actual fact (from using flat files for performance reasons, to needing multiple processes for calculations). This is not a personal attack (quite the contrary, I'm trying to help), but it sounds like you're more complex than you need to be.
17 Jan, 2013, arendjr wrote in the 65th comment:
Votes: 0
@Kelvin: The pathfinding example was not mine. I'm not advocating for a multi-process approach at all. I keep everything in one process, though I do have a few different threads :)
17 Jan, 2013, Kelvin wrote in the 66th comment:
Votes: 0
Fair enough, but the database part is probably the more important of the two. Using a database does not mean sacrificing performance. It's all in how you use it. I keep everything in-memory, but "dump" to the DB instead of dumping to a flat file. Best of both worlds.
17 Jan, 2013, arendjr wrote in the 67th comment:
Votes: 0
Kelvin said:
Fair enough, but the database part is probably the more important of the two. Using a database does not mean sacrificing performance. It's all in how you use it. I keep everything in-memory, but "dump" to the DB instead of dumping to a flat file. Best of both worlds.

(emphasis mine)

Exactly. I made the mistake of assuming that when you're using a database, you would use it instead of an in-memory representation. The in-memory representation is the key to good performance, and you can have it regardless of using a database. It's just that when you have an in-memory representation, and you already got JSON importing/exporting (as in my case), I see very little benefit of using a database over also using JSON for storage.

Data integrity is an advantage indeed, as I had to manually add some auto-correction, for example. The scaling benefits I don't see though. Can you elaborate? If I understand correctly you can't run a second instance against the database because the instances wouldn't see each other's changes anyway (since they only read the database at startup). Nor will it give memory benefits anymore, as you have to keep your own in-memory model anyway. If anything, you now have to share the available memory with the database.

I know scaling and performance are not an issue for most MUDs. What I was trying to counter was the statement that good design or architecture were not that important, and that it would be easy to fix performance bottlenecks as you go. If your architecture is not right you might find yourself in a situation where the only way of fixing certain bottlenecks is a rewrite of a large part of the codebase, something which rarely ends well. I knew from the start I wanted to try to push the boundaries, so I created an architecture that I believe would allow me to do that. My architecture has some complexities (it's multithreaded), but I don't think I made it overly complex. In fact, I stripped the complexity of having a separate database. Using JSON files for storage was not done for performance reasons, it was done to avoid the complexity of a database, which I believe would be overkill given that I do everything in-memory, which in turn was indeed done for performance reasons.

I think we're actually quite well on the same line, but maybe I haven't communicated all my motivations properly :) I started with a high-performance in-memory design, and needed a storage solution. I only needed a very small subset of the features a database would give me, so I consider it a good tradeoff to leave the database out, and handle the storage myself.
17 Jan, 2013, Kelvin wrote in the 68th comment:
Votes: 0
arendjr said:
I know scaling and performance are not an issue for most MUDs. What I was trying to counter was the statement that good design or architecture were not that important, and that it would be easy to fix performance bottlenecks as you go. If your architecture is not right you might find yourself in a situation where the only way of fixing certain bottlenecks is a rewrite of a large part of the codebase, something which rarely ends well


Yes, I think we are fundamentally in agreement. I just saw a few things that made my spidey senses tingle, but honestly I guess they're somewhat off topic and I should have just stayed quiet about them.
18 Jan, 2013, Runter wrote in the 69th comment:
Votes: 0
Who cares about scaling? How many people are you going to have? 10? 100? 500? Everyone is obsessed with scaling without any context of what they need. That's what this entire thread was about debunking, and it seems to have steered itself back to that. Use databases because it's standardized, there's tons of technologies to pick from, and it's easily distributed. And yes, quite unimportantly, many of them scale just fine. They give you basically free what you'll be designing from scratch in your own database. Yes, your json store is a database. Probably one that doesn't have 1/10th the features of a postgres store, or a mongodb store, or a graph based db. Why maintain your own code when there's bulletproof software ready for you to use for free right now?

If you want to start off with a "good design" then build something as simple as possible that separates concerns properly and allows you to later identify bottlenecks and swap out modules. If you do that, and you carefully design that, you'll save yourself a lot of headache building cloverleaf intersections for a few vehicles a day.
18 Jan, 2013, Rarva.Riendf wrote in the 70th comment:
Votes: 0
Runter said:
Who cares about scaling? How many people are you going to have? 10? 100? 500? Everyone is obsessed with scaling without any context of what they need. That's what this entire thread was about debunking, and it seems to have steered itself back to that.


Well I am not :) Hell if I started to actually have a scaling problem it would be good :) That is indeed proof the OP is totally spot on.
Maybe this problem exists because the coders are not experienced enough? Would be interesting to know the rates between professional/pure tinkerer one.
A single very bad pattern in code can cause a lot of problems, and push people to think of complex and extreme solutions.
I remember when I had a problem with my track system that was extermely slow, I started to think about multithreading it, but first I asked here if anyone did it, and seeing that people had no problems with theirs, it became obvious that I was doing something wrong. And indeed found where my problem was.

Multithreadng would have been a solution though, would have worked (but would have raised a whole bunch of problems I would have had to solve, so adding more complexity, thus harder to expand/maintain code).

I think an unexperienced coder will have started to multithread rigt away.

What do you think?
18 Jan, 2013, Runter wrote in the 71st comment:
Votes: 0
It's true that sometimes it's inexperienced programmers. But, I don't think it's so often the case. Actually, I think experienced programmers are perhaps more susceptible to it. I know I personally am. At some point or another we all get caught up in writing code for fun. And it is a lot of fun just solving problems. Sometimes problems we make for ourselves. Actually, I take issue there. It's great to tinker. We just need to be careful to understand the relationship between minimum and maximum goals and why sometimes the minimum goal is preferable for the time being when our larger goal is building something that works.
18 Jan, 2013, Rarva.Riendf wrote in the 72nd comment:
Votes: 0
Great to tinker with a personal project of course, but when you intend to actually release something, do you have the same attitude ? Experience taught me to avoid being to smart for my own good when I code. I sometimes made code that worked beautifully, but could not understand how the hell it worked a week after. Using recursivity, multithreading. Off course I did not commented it enough…as I was tinkering, and not planning any release.
18 Jan, 2013, arendjr wrote in the 73rd comment:
Votes: 0
Runter said:
Use databases because it's standardized

Heh, that's a great way to encourage innovation :D

For the record, databases are standard practice in web development, and they are too when you're developing an enterprise application. Much less so in game development.

Runter said:
(…) and it's easily distributed. And yes, quite unimportantly, many of them scale just fine.

The funny thing is, you're now advocating advantages you not only deem unimportant, but have already nullified by the way MUD engines use databases. Single read at startup, long-running transactions and an additional in-memory representation will limit you to a single client instance anyway.

Runter said:
They give you basically free what you'll be designing from scratch in your own database.

At least in my case that's not true. As I said, I needed JSON importing/exporting anyway, so the from scratch part didn't hold for me.

Runter said:
Yes, your json store is a database.

In common conversation a database is typically a relational database, or at the very least an off the shelf external database. So while your statement is technically correct, I will not adopt this terminology to avoid confusion :)

Runter said:
Probably one that doesn't have 1/10th the features of a postgres store, or a mongodb store, or a graph based db. Why maintain your own code when there's bulletproof software ready for you to use for free right now?

Of course it doesn't have that! Heck, I didn't even want 1/10th of the features! I'll quote myself again:
Quote
I only needed a very small subset of the features a database would give me, so I consider it a good tradeoff to leave the database out, and handle the storage myself.

You make it sound like I did the work of a madman by reinventing the wheel, where in reality it's no more than a tiny bit of glue around the JSON conversion.

Runter said:
It's true that sometimes it's inexperienced programmers. But, I don't think it's so often the case. Actually, I think experienced programmers are perhaps more susceptible to it.

I consider myself to be far from an inexperienced programmer, and yes, there's truth to your latter statement :)

That said, the decision to use a relational database or not is quite fundamental. Not something you easily refactor somewhere down the road. I'll happily acknowledge a database is a fine choice for the majority of MUDs. But I strongly disagree it's always the preferred solution. Not using a database has not given me any headaches yet, has not made my code significantly more complex (in fact, has kept it simpler in many cases), and has given me some real advantages:
- Fewer dependencies
- No need for an external database setup
- A file format that plays nice with version control
18 Jan, 2013, quixadhal wrote in the 74th comment:
Votes: 0
A typical MUD game system tends to be very highly coupled. Each element of the system generally has to be able to access information from a large percentage of the rest of the system, and THAT is why you'll find people telling you it's pointless to make a MUD codebase distributed.

Of course, there are different ways to divide up the problem space, so perhaps you are thinking of something that would work fairly well. I can tell you that trying to divide things up so each actor has their own thread is doomed to failure, mostly because every thread will spend most of its time waiting for resource locks during combat.

However, you could divide things up by region/zone quite nicely. Chat services could certainly be split off, as can I/O for both networking and file/database access.
18 Jan, 2013, Kaz wrote in the 75th comment:
Votes: 0
Jumping in where angels dare to tread…

arendjr said:
Runter said:
Use databases because it's standardized

Heh, that's a great way to encourage innovation :D

For the record, databases are standard practice in web development, and they are too when you're developing an enterprise application. Much less so in game development.


I think you should compare like-with-like. MUDs are the precursors to MMOs (in fact, TVTropes – beware, it will consume your afternoon – refers to RetroMUD as "A relatively small Massively Multiplayer Online Role-Playing Game."*)

World of Warcraft, Guild Wars (1+2), Eve Online, Dungeons and Dragons Online… which of these do you think use databases and which don't?



Of course, the real answer is to develop a facade around your game data so that you can switch the storage and query mechanisms without needing to rewrite your code ;)


* http://tvtropes.org/pmwiki/pmwiki.php/Ma...
18 Jan, 2013, arendjr wrote in the 76th comment:
Votes: 0
Kaz said:
Of course, the real answer is to develop a facade around your game data so that you can switch the storage and query mechanisms without needing to rewrite your code ;)

In theory this sounds good. In practice you will be developing a facade, its interfaces, and possibly a plugin architecture, upfront without knowing if you will ever need this. Worse, you will not know upfront whether the interface you're designing at that time will actually be abstract enough to switch to whatever new system you may want to switch to in the future. This approach will guarantee to add complexity, only for a hypothetical future gain :)

Edit: That's not the same as saying it's a bad idea. The hypothetical gain may still become very real at some point. But I doubt this approach would be advocated for when talking about the minimal viable product :)
18 Jan, 2013, mangan wrote in the 77th comment:
Votes: 0
Compared to the amount of work being put into a game engine, interfaces are hardly going to be any overhead although they will save you time down the road if applied correctly. One key here is to determine which requirements are likely to change in the future, and which aren't. Anything likely to change should be behind interfaces (thus being wrapped as a decoupled module). Your core requirements that determine what you game is may be more coupled to everything else and thus not be behind interfaces, but that is generally because they will not be changing. Base example here is Diku's combat system… I would not suggest to any non-combat game designers that they look at Diku as a starting point for an engine. The things that aren't their own modules (decoupled via interfaces to allow changes within the module) only really have a defense if changing them would alter the core requirements of the engine anyway.

That's just software engineering principles though. In this thread, I would argue that a nice interface to decouple the database be refactored into the system as a step directly after showing the database will work (proof-of-concept). It means that in the future, you could easily modify the database structure or internal requirements as desired without concern for the rest of the system, perhaps going to a database that is convenient for web integration or mobile applications, etc.
19 Jan, 2013, Tyche wrote in the 78th comment:
Votes: 0
arendjr said:
I want someone standing behind the battlements on a high wall. He will have a clear view over the grasslands, and will see people approaching from afar. Except when they're on the back side of a hill. Except when there's fog. The engine will calculate whether there's line of sight. The engine will know the distance, and use it to determine whether you can identify the person, or even see whether is a human at all. The distance will determine if his arrows can reach the person.

I want someone walking on soft shoes that muffle his steps when he hears a conversation. He stops and hears the voices of a man and a woman and they're plotting to kill her husband. The sounds are coming from the little office around the corner. If only they had closed the door, he might not have heard them.

I want someone throwing a granade into a corridor. The corridor is tight and the pressure of the explosion is compressed into two directions. People get knocked off their feet. The sound of the blast will alarm everyone within hundreds of yards. Even outside a soft cracking sound is heard.

These are the things I can actually do with my system of propagating events. However the algorithms involved will often require access to hundreds of objects at once. When an event starts, you don't know in advance where it will end. It literally depends on things like doors or windows being open or closed, presence of fog, line of sight, etc.. This means you cannot prefetch the objects. If I would've used a database, it would result in hundreds or thousands of SQL queries. At this point, performance is down the drain.

Had I used a database for my project, my main distinctive feature would've been impossible to implement.

Most RDBMSs now implement spatial indexing systems. SQLite supports multi-dimensional R-Tree indexing. There's a higher level interface available called SpatialLite that allows you define geometries like rivers, trees, lakes, buildings. Note that by flipping a few configuration parameters in these databases all your data can reside in memory. Given the theme of Kelvin's article, you can save time and meet the game requirements above by using software already available.
19 Jan, 2013, arendjr wrote in the 79th comment:
Votes: 0
Tyche said:
Most RDBMSs now implement spatial indexing systems. SQLite supports multi-dimensional R-Tree indexing. There's a higher level interface available called SpatialLite that allows you define geometries like rivers, trees, lakes, buildings. Note that by flipping a few configuration parameters in these databases all your data can reside in memory.

I know they do, but this would really only be relevent if it would fit my requirements, which it doesn't. In addition we already established in this thread to be using in-memory representations of the data, so doing another SQL query for this at runtime would be really awkward at best.

Tyche said:
Given the theme of Kelvin's article, you can save time and meet the game requirements above by using software already available.

I guess you should've first understood my game requirements before making that claim.
20 Jan, 2013, Tyche wrote in the 80th comment:
Votes: 0
arendjr said:
I know they do, but this would really only be relevent if it would fit my requirements, which it doesn't. In addition we already established in this thread to be using in-memory representations of the data, so doing another SQL query for this at runtime would be really awkward at best.

Presuming the requirements were the three paragraphs I quoted then it certainly does.
Probably not you, specifically, but some other person with the same requirements.
60.0/204