09 Oct, 2011, teqneek wrote in the 1st comment:
Votes: 0
Greetings!

Looking for some input / suggestions / ideas/ work around for an ID system for objects and creatures and methods for storing them. A little overview of my question/idea:

Im at a point where Im starting to load objects and npcs into my game. All data is accessed via a SQL db. table: mud_npcs, mud_objects (->mud_object_modifiers)

All "Entities (objects, creatures, etc)" are derived from base class "Entity", the main game manager houses a global static map of master records of both objects and npcs (to be cloned from to make an instance of that obj/npc) and a map of all existing game entities (when they are created, spawned, etc).

The base class entity defines string uid; Which I was using to store a custom string (which I later parse with, getUID()) that represented a type unique game id:

Type DB ID keyword
O 1 Sword // Object
C 20 Troll // Creature
H 14 Teqneek // Player (Human)

O1Sword, C20Troll, H14Teqneek

Searching and iterating is a breeze using a STL Multimap… The issue I have is.

Multiple instances; I could easily add a number to the end of the ID, eg. O1Sword4 , or I may want 20 trolls for an army, C20Troll18..19..20 etc
but when a creature dies, or an item is destroyed should i be reusing that id, keeping track of use in an array? int idPool[x]; for every master object.

Honestly I dont know, seeking advice!
09 Oct, 2011, Runter wrote in the 2nd comment:
Votes: 0
I don't really understand the point of the composite string. If you're using a database just add fields for type, keyword, and id (should already exist, anyways).

Furthermore, while you can have copies in the database of items it's usually a good idea to share data between instances and the prototype. Because a majority of the time the data will be the same, but I don't really see a purpose in the composite string thing for the uid.
09 Oct, 2011, teqneek wrote in the 3rd comment:
Votes: 0
I guess i dont understand Runter? I do have those fields in the database,…. Combining them ?(when creating an instance of an object or say just initially loading an object/creature) would create a UID that i could pick apart for use that isnt just.. say.. VNUM = 1000 instead i would be O1000Sword (With the ID i can easily use it to determine simply by the ID if its a Object, its DB ID number in the table, and its Keyword (Used for look, get etc standard)

I could easily then also psuedo : multimap.find("O1000Sword2") since all entities are in the same global list. EntityList


While I have looked at UUID type idea, the format it uses doesnt seem very fluid for the system Im trying to use it on. Thats why I am looking for ideas or suggestions on a better way to handle a UID system for a MUD, which also handles multiple instances of the entities

Diku simply loads objects and mobs into a list, and it does various loop cycles to find multiples…. I would kind of like a more definitive referencing value to the instanced entities.
09 Oct, 2011, Runter wrote in the 4th comment:
Votes: 0
Okay, so it's something used to replace vnums for a notation in game? What's the point of including the keyword with the vnum, and the type? This makes it seem even more onerous to me if I were a builder. Because now not only do I have to memorize the vnum, I have to memorize the keyword and the type.

My advice is use two public keys to the resource. One intended for builders to be able to interact with, and one intended to be a definitive singular value (like the vnum only) for the code lookup. It doesn't matter how inefficient the builder key is for lookup, because once resolved internally it should use the vnum. When I designed something like this in the past I went very unorthodox and it worked out well for me. I let builders assign any type of "vtag" they wanted to things. Like "frisky fido", "steel sword", "path through the woods". Then a builder could simply type edit steel sword.

If you're interested I go into more details about the system here:
http://www.mudbytes.net/topic-2735
09 Oct, 2011, teqneek wrote in the 5th comment:
Votes: 0
Thanks for the Link.

As for builders (if i ever have any) it is all done from a Windws based App or the web based App… therefore there is no need to remember the string of the ID, its simply all for server side/code purposes.
09 Oct, 2011, Runter wrote in the 6th comment:
Votes: 0
teqneek said:
Thanks for the Link.

As for builders (if i ever have any) it is all done from a Windws based App or the web based App… therefore there is no need to remember the string of the ID, its simply all for server side/code purposes.


If it's just for the server then just use the unique number or database serial number. What's the point of the composite key you're suggesting if it's used internally? Would it ever be exposed to players or users or imms at all?
10 Oct, 2011, quixadhal wrote in the 7th comment:
Votes: 0
Quote
Type DB ID keyword
O 1 Sword // Object
C 20 Troll // Creature
H 14 Teqneek // Player (Human)

O1Sword, C20Troll, H14Teqneek


What's the point of using vnums (or any hokey numbering scheme) if you don't NEED to do so?

If you want to refer to a thing by a convenient label, why not just make it convenient?

/object/Sword
/creature/Troll
/human/Teqneek

Pathnames have worked just fine for LpMUD's since 1990. But don't let that fool you. There's no particular need or reason to make them real file paths. But presenting them in that way is very natural and easy to read.

So, if someone adds /object/Sword to a treasure chest, it's pretty simple to do a database lookup. Using paths also lets you have more specific divisions, so /object/DreadCastle/Sword and /object/SmurfVillage/Sword are two different swords.

If it's just a unique ID, as Runter said, why bother making some composite thing you have to compose and decompose? Just use an integer and be done.

CREATE TABLE types (
type_id SERIAL PRIMARY KEY NOT NULL,
type_name TEXT NOT NULL
);

INSERT INTO types(type_name) VALUES ('object');
INSERT INTO types(type_name) VALUES ('creature');
INSERT INTO types(type_name) VALUES ('human');

CREATE TABLE things (
obj_id SERIAL PRIMARY KEY NOT NULL,
type_id INTEGER NOT NULL REFERENCES types(type_id),
keyword TEXT UNIQUE NOT NULL
);

INSERT INTO things (type_id, keyword) VALUES (1, 'Sword');
INSERT INTO things (type_id, keyword) VALUES (3, 'Teqneek');
INSERT INTO things (type_id, keyword) VALUES (2, 'Troll');


You then don't have to know or care what numerical values things are. Your building software would load the types table and display a list (droplist) of the text names of each type. You can't accidentally insert a bad type because of the reference, like you could fat-finger a bad composite key. Because type_id and obj_id are both serials (auto-increment integers based on a sequence), you can't accidentally end up with duplicate id's – and the database itself assigns the next unused number.
10 Oct, 2011, teqneek wrote in the 8th comment:
Votes: 0
I guess I never thought of real time access to a DB as a very efficient (performance wise) to go about accessing data. If someone has a different view or experience with that I would love to hear it. There fore the DB doing anything for me other than assigning IDs as objects and creatures are created (for the master table of objects creatures etc) doesnt do much but give me a number.

So if I have a Sword thats an object, and a Sword thats a creature, i would have to loop through my list to find the entity i want based on its type… vs using some kind of Unique identifier to do a lookup on a multimap.

Using just numbers, using STL containers becomes pointless for some of their features (Using a key lookup .find() on a multimap) would not work using a number… i have a troll thats id 1000 as a creature, and a Sword thats id 1000 as a Object… unless i seperate my entities into different lists by type, iterating through the map seems tedious to differentiate between entity types


Another thing I was pondering.. is multiple instances of that said object or creature and tracking them, for easy lookup. As far as I know theres is nothing to uniquely identify an object in a DIKU based mud, from its multiple instances.
10 Oct, 2011, Runter wrote in the 9th comment:
Votes: 0
Caching strategies make accessing the store directly perfectly fine from a performance stand point. 20c web apps generally all work this way and while databases often are the bottleneck, this wouldn't at all be the case for a mud with any sane caching strategy.
10 Oct, 2011, Rarva.Riendf wrote in the 10th comment:
Votes: 0
teqneek said:
I guess I never thought of real time access to a DB as a very efficient (performance wise) to go about accessing data.

Bottlenecks from database come mostly from concurrency, write/read on the same data. Databases, as Runter said, implement caching, so it is mostly like having your objects loaded in memory.
10 Oct, 2011, quixadhal wrote in the 11th comment:
Votes: 0
teqneek said:
There fore the DB doing anything for me other than assigning IDs as objects and creatures are created (for the master table of objects creatures etc) doesnt do much but give me a number.

So if I have a Sword thats an object, and a Sword thats a creature, i would have to loop through my list to find the entity i want based on its type… vs using some kind of Unique identifier to do a lookup on a multimap.


The usual method of handling real-time data requirements it to cache commonly used items and pre-load things as usage suggests you'll need them. A MUD is real-time but uses a tiny fraction of the resources available to it, and any decent RDBMS will likely perform just as well as opening files and pawing through them.

I think you missed the point I tried to make.

There is no functional difference between "O1Sword" and "/object/SmurfTown/Sword", except that the latter is easily readable and obvious. Anywhere you could use one, you could use the other.

While you can't "lookup" the numbers, why would you need to? If you want a unique identifier, a number is perfectly acceptable since anything referring to stuff by that identifier already knows the thing it's accessing. Why would it matter if the keys of your multimap look like "O1Sword", "/object/SmurfTown/Sword", or 36721?

I'm not talking about YOU picking numbers. You have a sword. You add it to the database or flat-file. It's the 137th object you've created so it is ID 137. Later, you add an NPC called "Sword", it's the 750th object you've created so it gets ID 750. There's no confusion or need to "loop through" stuff. If you want to find all the Swords, you look in the map using text keywords and get an array of matching ID's. If you've chosen the pathname system, you'd be iterating through "/object/SmurfTown/Sword" and "/creature/SpookyForest/Sword". If you've chosen the obfuscated label system, it'd be "O137Sword" and "C750Sword". In either case, having the ID values then gets you the object/record/etc you were looking for.

As for instances, the simplest way is to keep a second instance ID, unique to the individual item. If you were using an SQL schema like the one I gave above:

CREATE TABLE things_instances (
obj_id INTEGER REFERENCES things(obj_id) NOT NULL,
instance_id SERIAL NOT NULL,
owner_id INTEGER REFRENCES things(obj_id)
– other transitory state information that varies from one item to the next
);

CREATE UNIQUE INDEX ix_thing_instance ON things_instances(obj_id, instance_id);

INSERT INTO things_instances(obj_id, owner_id) VALUES (1, 2);


That would create a specific Sword owned by the Troll from the previous SQL example. If it were the first instance of a Sword, it would get instance_id 1.

For reference, LPC would use "/object/Sword#1" to refer to the specific clone of a Sword.
11 Oct, 2011, teqneek wrote in the 12th comment:
Votes: 0
Thanks for the responses Quix… the SQL bit is still a bit.. perplexing… alot of these concepts dont allude me, in fact id love to do them that way… but accessing a large table (1000 records? lets say for 1000 loaded instances of a mobile), pushing queries against that quite frequently, (checking a radius for NPCs, querying the db for the instance id of that particular npc etc) seems like a bottleneck? While doing a query isnt much more than parsing a file.. In a flat file system you wouldnt be parsing a file to find an instance id of an npc everytime you needed it. Or perhaps I am misunderstanding your example.

A second note to that.. the reason I found it difficult to conceptulize with the idea I was approaching at the time, was once youve deleted an item, that instance ID is no longer valid, as instances increase your instance ID increases quite rapidly over time, 1 5 to 100 to 500 (after alot of objects or npcs have been created or deleted) at what point would i be running into a problem as the instance ID started getting out of hand.. (even if i used a double to store that ID)

As far as the ID system goes… Perhaps I should redesign my DB a bit, so that there is a general "Things" table, and seperate tables I would JOIN with a basic query.. currently NPCs and Objects are in two seperate tables, which of course would have IDs being duplicated, so a numerical ID for npcs and objects wouldnt currently work until I change the storage method.. but It has me pondering now.
11 Oct, 2011, Runter wrote in the 13th comment:
Votes: 0
I wouldn't worry about your database being a bottleneck until you have 5000 or so concurrent users. And let's be real, most muds (none of them?) ever hit that number. So I'm not sure the database is a bottleneck flies until, you know, it's actually the bottleneck. At which time you can employ strategies to fix it. Like preemptive eager loading and aggressive caching. Realistically the few ms a query takes, especially when combinating queries, isn't going to be noticable. And because of the nature of a query, it's not really going to be inefficient. It's definitely a trade, but I think it's a great one. And it's not one that with proper encapsulation you can't changed easily later to suit your needs if you really ever do get a bottleneck. But bottlenecks are something you test to find. You don't proclaim what is or isn't your bottleneck before you even have a userbase. That's a little silly. Bottlenecks are going to be based on a lot of factors other than just your code and algorithms. For example, some databases are going to perform differently than others. Some hardware is going to perform differently than others. Taking an extreme example, milage using Cassandra as your database is going to be different from sqlite3.
11 Oct, 2011, Tyche wrote in the 14th comment:
Votes: 0
This might be useful…SymbolTable
12 Oct, 2011, David Haley wrote in the 15th comment:
Votes: 0
Quote
but accessing a large table (1000 records? lets say for 1000 loaded instances of a mobile), pushing queries against that quite frequently, (checking a radius for NPCs, querying the db for the instance id of that particular npc etc) seems like a bottleneck?

High-frequency trading systems use databases as storage mechanisms. I'm pretty sure that a MUD would be ok. You just have to make sure that you're using intelligent keys and indexing. Obviously, a linear search over thousands of mobs will be terrible.
12 Oct, 2011, oenone wrote in the 16th comment:
Votes: 0
Well, Google uses flat file databases for their data, because SQL servers would be too slow. But I think this is a very special case and you shouldn't worry about your database being slow.
12 Oct, 2011, teqneek wrote in the 17th comment:
Votes: 0
David Haley said:
Quote
but accessing a large table (1000 records? lets say for 1000 loaded instances of a mobile), pushing queries against that quite frequently, (checking a radius for NPCs, querying the db for the instance id of that particular npc etc) seems like a bottleneck?

High-frequency trading systems use databases as storage mechanisms. I'm pretty sure that a MUD would be ok. You just have to make sure that you're using intelligent keys and indexing. Obviously, a linear search over thousands of mobs will be terrible.


While this may be true… these large trading sites, arent requiring the real time update and instant access required for certain operations in a game.. While alot of their concepts are alot more complex, how a SQL database relates to the web and a real time simulation seem a bit distant in relation.

Most games that use an SQL Database keep all simulation data in memory, and only use the DB for transactional type returns (The only systems ive ever done for real time DB access, are things such as Help files, account / character login lookup, a change list, in game note systems etc.) ie: Things that arent requiring access 10 to 20 times a second to the database. Though trying to do something more in depth for a small scale game (100 users would be even alot of a new MUd now a days heh) could be interesting to play around with in the least.


Tyche, thanks for the link sir, that is a good read.
12 Oct, 2011, Idealiad wrote in the 18th comment:
Votes: 0
To go back to the question of what to do with rapidly increasing IDs – apart from the point that your maximum ID could be very large regardless – some bases like Tiny simply recycle IDs when the prior entity with the ID is deleted.

Maiden Desmodus has an IMO very sensible system where you have a configurable option to show the unique IDs of all items when they're listed, if you need to be absolutely sure of what you're referring to for buy/get/drop etc.
12 Oct, 2011, teqneek wrote in the 19th comment:
Votes: 0
oenone said:
Well, Google uses flat file databases for their data, because SQL servers would be too slow. But I think this is a very special case and you shouldn't worry about your database being slow.


Not really worried about it being .. "slow" per say.. slow to the point where some suggestions here have operations that normally run in memory, are being suggested to use a DB Table. There seems to be alot of reference to web based applications in the database, alot of queries and return to a website take around .005 to .01 for execution times in the query. To use that kind of look up in a game even with proper indexes and caching, and doing that lookup several times a second… seems inefficent. Regardless, the game is going to require a response from the SQL server before it continue.

I do appreciate all the debate and suggestions though, it has me thinking about alot of different paths to take.
12 Oct, 2011, quixadhal wrote in the 20th comment:
Votes: 0
Most games I've seen (outside the tinker-toy realm of MUD's, most of which are written by hobbyists) that use SQL do, in fact, store ALL the game data in SQL. The advantages far outweigh the limitations, but it does require you design for the fact.

You're worrying about a few thousand records, at a lookup speed of a few dozen a second? Even in the 1980's, that was easily doable. I've worked on systems with millions of rows, updating a hundred thousand entries an hour, and had front-end interfaces that would retrieve the data in tenths of a second. The high volume trading systems David mentioned are indeed real-time. You will never convince investors that "a few seconds" is adequate responsiveness, because "a few seconds" might make the difference between selling at $400/share and selling at $150/share.

As for increasing ID numbers. I wouldn't worry about it until and unless you see it as a problem. With a 32-bit integer you have 2 billion to work with. If your maximum number starts getting upwards of a billion, you probably still have some time to think about how to recycle old ones. As an added bonus, if you design your database with the appropriate triggers on updates, you could simply change the ID of an item to a lower-value previously deleted one and have all the data change with it.

Note that being able to do that does mean you have to work out a cache system which respects changes of that nature, so your game code also adjusts accordingly.

Much of the "slowness" in web applications comes from the fact that they often run as one-off scripts which have to connect and tear-down the database connection every time they run. That stuff does indeed take time, but a normal game won't do that. It will connect at bootup and hold the connection open until shutdown.
0.0/51