04 May, 2014, Tyche wrote in the 1st comment:
Votes: 0
quixadhal said:
Tyche said:
Surely you would also have edit all those things even if you use JSON when you are adding a new field or flag to the structures in your Dikumud.
The same would be true when using XML or YAML or SQL or "your own special" format.


The entire point of using a serializer (JSON, XML, whatever) is that you hand it a data structure and it returns a string which can be used to restore that data structure. Even in a non-dynamic language like C, the most you should have to do is save that string along with (maybe) a hint of the kind of structure to allocate when you want to load it back again.

You show an example of a Perl associative array, which by definition is virtually identical to a JSON object.
The problem is JSON doesn't understand classes, structures, many other kinds of hashes and associated arrays, and data types.
Nor does it understand relationships between classes and structures. It presumes a top down nested heirarchy.
Complex applications like muds aren't top down nested heirarchies (like documents).
They're full of cyclic object graphs, sometimes one-way, sometimes bi-directional.
And full serialization isn't really what we want anyway. It wouldn't be a DikuMUD (it'd be a MOO or MUSH then).

For example here's the Merc room…
struct	room_index_data
{
ROOM_INDEX_DATA * next;
CHAR_DATA * people;
OBJ_DATA * contents;
EXTRA_DESCR_DATA * extra_descr;
AREA_DATA * area;
EXIT_DATA * exit [6];
char * name;
char * description;
sh_int vnum;
sh_int room_flags;
sh_int light;
sh_int sector_type;
};


Assuming we want to keep the one area to one file approach (and there are many reasons muds use it), you've got to produce a complete JSON parsable object.
Which means we have an area object, which contains arrays of resets, rooms, objects, shops, mob_progs, etcetera.

So just looking at the room struct…
It's got data we don't want to serialize, next, people, contents, area.
It has references we probably want to descend into and recursively serialize as arrays of objects within the object, extra_desc, exit[].
And when we descend in some records like exit, we'll find link references to rooms that we don't want to follow and instead
convert into vnums.

There is no easy peasy 2 line automated serialization whether the above be a C structure or a C++, Python, Perl, or Ruby class.
You will have to write a custom serialization routines, and for every new field you add to the serialization routine.

In order serialize in Ruby you'd add a to_json and self.create_json method. Similarly in Python. TO_JSON for Perl classes.

Here's some C JSON libraries…
http://sourceforge.net/projects/cjson/
https://github.com/json-c/json-c
http://www.digip.org/jansson/

This is the beginning of how one might dump a room to JSON using Merc and the cjson library.

//             if ( pRoomIndex->area == pArea )
// {
// fprintf( fp, "#%d\n", pRoomIndex->vnum );
// fprintf( fp, "%s~\n", pRoomIndex->name );
// fprintf( fp, "%s~\n", fix_string( pRoomIndex->description ) );
// fprintf( fp, "0 " );
// fprintf( fp, "%d ", pRoomIndex->room_flags );
// fprintf( fp, "%d\n", pRoomIndex->sector_type );

cJson * room;

room=cJSON_CreateObject();
cJSON_AddNumberToObject(room, "vnum", pRoomIndex->vnum);
cJSON_AddStringToObject(room, "name", pRoomIndex->name);
cJSON_AddStringToObject(room, "description", pRoomIndex->description);
cJSON_AddNumberToObject(room, "room_flags", pRoomIndex->room_flags);
cJSON_AddNumberToObject(room, "sector_type", pRoomIndex->sector_type);


// for ( pEd = pRoomIndex->extra_descr; pEd; pEd = pEd->next )
// {
// fprintf( fp, "E\n%s~\n%s~\n", pEd->keyword,
// fix_string( pEd->description ) );
// }

cJson * extras
cJSON_AddItemToObject(room,extras=cJSON_CreateArray());
for ( pEd = pRoomIndex->extra_descr; pEd; pEd = pEd->next )
{
cJson * ex;
cJSON_AddItemToArray(extras,ex=cJSON_CreateObject());
cJSON_AddStringToObject(ex, "keyword", pEd->keyword);
cJSON_AddStringToObject(ex, "description", pEd->description);
}

……..
And somewhere at the end we serialize it:

out=cJSON_Print(area);
cJSON_Delete(area);


You could of course make big architectural changes to your DikuMUD in order make it look more like like a JSON document.
If you think DikuMuds ought to be using JSON rather than their own special serialization, then you might want to show how much better it is.
05 May, 2014, plamzi wrote in the 2nd comment:
Votes: 0
Tyche said:
If you think DikuMuds ought to be using JSON rather than their own special serialization, then you might want to show how much better it is.


The biggest reason to use JSON is not so you can serialize in one line. It's so you can parse in one line, on pretty much any platform, with pretty much 0 hassle.

Even if you decide that your game server will not care about any client beyond the generic MUD clients, many of them support GMCP, which is JSON.

So why invent your own format when you can use a format invented by smarter folks and supported widely? It's not like your own format would be any less work to implement on the server-side. In fact, it's very likely to be more work.
05 May, 2014, quixadhal wrote in the 3rd comment:
Votes: 0
The above, Tyche, is why I suggested an external tool which would perform that by finding struct definitions in your source code and emitting functions which would do all the JSON_addFOO() calls. Perl's version also has a TO_JSON mechanism for objects, which is needed if you want to omit pointless data (like function pointers, or circular references).

And yes, I know such a tool won't get it 100% right… but it WOULD save you from having to write all that boiler-plate code, most of the time. For example, when the only change you're making is adding a new integer field to your room data, a tool could easily do that as part of the make chain. I know I'd much rather rewrite a few dozen lines that the tool gets wrong, than write it all by hand.

As for perl's hashes somehow not being structures… I'd ask you to describe what a structure is, in your mind. Functionally, a structure is a way to organize primative data type collections into an ordered state, where the elements can be accessed by name within code. That sure sounds exactly like an associative array to me.

struct room_index_data
{
ROOM_INDEX_DATA * next;
CHAR_DATA * people;
OBJ_DATA * contents;
EXTRA_DESCR_DATA * extra_descr;
AREA_DATA * area;
EXIT_DATA * exit [6];
char * name;
char * description;
sh_int vnum;
sh_int room_flags;
sh_int light;
sh_int sector_type;
};

my $room_index_data = {
next => undef,
people => [],
contents => [],
extra_descr => [],
area => undef,
exit => [undef,undef,undef,undef,undef,undef],
name => '',
description => '',
vnum => undef,
room_flags => 0,
light => 0,
sector_type => undef,
};


Just like the C version, you'd assign values to the various elements before trying to access them. Sure looks like it does the same job to me. Yes, the "next" element can be assigned a reference to another room_index_data hash.

I also consider it silly to think that full serialization of the game data would somehow make it "not a Diku". Since when does the file format or data structure define the Dikuness of a game?

As planzi said, even if you don't build a tool to make all those serialization functions for you (not needed in dynamic languages which can inspect their own data types), it still saves you from having to write both the saving AND loading code.
05 May, 2014, Tyche wrote in the 4th comment:
Votes: 0
plamzi said:
Tyche said:
If you think DikuMuds ought to be using JSON rather than their own special serialization, then you might want to show how much better it is.


The biggest reason to use JSON is not so you can serialize in one line. It's so you can parse in one line, on pretty much any platform, with pretty much 0 hassle.

Even if you decide that your game server will not care about any client beyond the generic MUD clients, many of them support GMCP, which is JSON.

So why invent your own format when you can use a format invented by smarter folks and supported widely? It's not like your own format would be any less work to implement on the server-side. In fact, it's very likely to be more work.

Sending simple network messages is quite a different use case than representing the contents of one's DikuMud.
Unfortunately GMCP is not JSON.
05 May, 2014, plamzi wrote in the 5th comment:
Votes: 0
Tyche said:
Unfortunately GMCP is not JSON.


As it often happens, you didn't address the substance of my post, and also I have no idea what you mean. If it's parsed by JSON parsers, I call it JSON. If the JSON package is wrapped inside some extra stuff that takes an extra line to remove, I still call it JSON.

If you mean that some implementations like Aardwolf's tack on data packages that don't follow the JSON standard, then heh, who cares, as long as all modern MUD clients with GMCP support can easily parse *any* JSON package you send to them, regardless of whether it follows some kind of GMCP standard or not. This is what we do with our implementation, and it's the only format I know of that I can send to a web app, iOS app, and a client like Mudlet, and parse it easily in all three places.

Challenges like circular references, recursion, and excluding certain members are all very common, and all have common solutions. Maybe in older languages like C and C++ those solutions are not going to be as minimal as in, say, JS, but it's not like coming up with your own format is going to save you from them. In fact, you'll have to also come up with your own solutions instead of googling out some good ones. How is that a plus?
05 May, 2014, Davion wrote in the 6th comment:
Votes: 0
Tyche is definitely not wrong. With my python port of ROM this has been bothering me. I can use things like Pickle for complete serialization, but human readability goes out the door. For using JSON the best I've come up with is creating a dictionary of the values you want to save, then JSONing that. On the other end, you'd load the dictionary, then translate needed values (eg, inroom vs contents). to_JSON is fine for simple cases but for diku's you're never sure if it's a reference or the entire object itself. You can't guess…
05 May, 2014, Tyche wrote in the 7th comment:
Votes: 0
quixadhal said:
The above, Tyche, is why I suggested an external tool which would perform that by finding struct definitions in your source code and emitting functions which would do all the JSON_addFOO() calls. Perl's version also has a TO_JSON mechanism for objects, which is needed if you want to omit pointless data (like function pointers, or circular references).

And yes, I know such a tool won't get it 100% right… but it WOULD save you from having to write all that boiler-plate code, most of the time. For example, when the only change you're making is adding a new integer field to your room data, a tool could easily do that as part of the make chain. I know I'd much rather rewrite a few dozen lines that the tool gets wrong, than write it all by hand.


It is pretty much line for line the same as Diku's own custom serialization. Sorry I don't think writing a tool to read C structures and generate functions to
serialize them is something I'd care to do. Anyone?

quixadhal said:
As for perl's hashes somehow not being structures… I'd ask you to describe what a structure is, in your mind. Functionally, a structure is a way to organize primative data type collections into an ordered state, where the elements can be accessed by name within code. That sure sounds exactly like an associative array to me.


I didn't say perl's hashes were not structures. I said…
Tyche said:
The problem is JSON doesn't understand classes, structures, many other kinds of hashes and associated arrays, and data types.
Nor does it understand relationships between classes and structures. It presumes a top down nested hierarchy.

Substitute 'structs' for 'structures' if that clarifies, though I wasn't referring to Perl's hashes.

Quote
I also consider it silly to think that full serialization of the game data would somehow make it "not a Diku". Since when does the file format or data structure define the Dikuness of a game?

Yeah I said it would make it a MOO or Mush. It's about the contents of the area files not the file format or data structure.
DikuMud zones/areas contain the prototypes of the room/objects/mobiles/shops and the rules/resets to generate instances of objects/mobiles.
Not the instances themselves. That's why when you save an area, especially rooms, partial serialization of the in memory data
is done. On MOOs and MUSHes everything is an instance and everything is saved… full serialization.

quixadhal said:
As planzi said, even if you don't build a tool to make all those serialization functions for you (not needed in dynamic languages which can inspect their own data types), it still saves you from having to write both the saving AND loading code.

A DikuMud written in Python or Ruby is going to have to write saving and loading routines for its classes in order to write an "area" files.
I know of no JSON parser that understands how to convert references to another object from pointers/references to vnums, or which data
is to be persisted and which is temporary game state data.

Here's an example that might get the point across:
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
};

Serialize this using JSON, but don't serialize email or password.
06 May, 2014, plamzi wrote in the 8th comment:
Votes: 0
Tyche said:
A DikuMud written in Python or Ruby is going to have to write saving and loading routines for its classes in order to write an "area" files.
I know of no JSON parser that understands how to convert references to another object from pointers/references to vnums, or which data is to be persisted and which is temporary game state data.

Here's an example that might get the point across:
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
};

Serialize this using JSON, but don't serialize email or password.


Am I the only one here whose logical sensor is in agony from this pseudo-argument? I know of no storage *period* "that understands how to convert references to another object from pointers/references to vnums, or which data is to be persisted and which is temporary game state data" without being told in some form or fashion. So what exactly are you comparing JSON to? Something that can read your mind?

If, on the other hand, you compare JSON to other *existing* or *possible* storage methods, there are plenty of advantages to be had.
06 May, 2014, Davion wrote in the 9th comment:
Votes: 0
Tyche said:
quixadhal said:
As planzi said, even if you don't build a tool to make all those serialization functions for you (not needed in dynamic languages which can inspect their own data types), it still saves you from having to write both the saving AND loading code.

A DikuMud written in Python or Ruby is going to have to write saving and loading routines for its classes in order to write an "area" files.
I know of no JSON parser that understands how to convert references to another object from pointers/references to vnums, or which data
is to be persisted and which is temporary game state data.

Here's an example that might get the point across:
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
};

Serialize this using JSON, but don't serialize email or password.


This is also a problem I was coming across. It keeps bringing me back to looking at ORMs. Djangos in particular would have no problem serializing the data, and even pumping it out to JSON to be transported. I think this is a far better fit for Dikurivatives rather then something like pure JSON (which will not work as a one line print,read solution). Unless you want to store simple data, instead of serialized game objects.
06 May, 2014, plamzi wrote in the 10th comment:
Votes: 0
Davion said:
This is also a problem I was coming across. It keeps bringing me back to looking at ORMs. Djangos in particular would have no problem serializing the data, and even pumping it out to JSON to be transported.


I'm not sure why you think ORM's are the solution to the "problem" Tyche is showing. I'm using an ORM for one of my projects. AFAIK, no ORM is going to know that sometimes you don't want to output the "password" parameter, but sometimes you do. No ORM is going to automatically know that sometimes you may want to ignore a reference to another object, while sometimes you may want to recurse it. When I don't want to output certain parameters in the ORM-based app, I have to either make them non-enumerable, or simply filter them during composing the output. That's the only way, ORM or no-ORM, unless you want to have and maintain two versions of the object, which is obviously even more of a headache.

Also, at least the ORM I am using, requires that you define all relationships and data types for each object in its own language. That's an understandable requirement, because it needs to be able to recreate the storage routines for these if you change the storage medium. But then, if you want to modify your objects, you have to modify those definitions accordingly so it will know how to handle the new data members.

At the end of the day, the benefits that an ORM brings to the table have nothing to do with overcoming the issues listed above. And anyway, knowing the OP, he will probably advocate for writing your own ORM :)
06 May, 2014, quixadhal wrote in the 11th comment:
Votes: 0
Tyche said:
Here's an example that might get the point across:
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
};

Serialize this using JSON, but don't serialize email or password.


#!/usr/bin/perl -w

package Example::Person;

use strict;
use warnings;
use English -no_match_vars;
use JSON;

sub load_file {
my $filename = shift;

return undef if !defined $filename or !-r $filename;
my $data = '';
open FP, "<$filename" or die "Cannot open $filename: $!";
while(<FP>) {
$data .= $_;
}
close FP;
return $data;
}

sub new {
my $class = shift;
my $person = shift;
my $self = {};

if( defined $person ) {
my $filename = "/lib/people/storage/$person.json";
if ( -r $filename ) {
my $data = load_file $filename;
$self = decode_json $data;
} else {
$self->{filename} = $filename;
}
} else {
$self = {
filename => "/lib/people/storage/bsmith.json",
name => "Bob Smith",
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password",
};
}
bless $self, $class;
return $self;
}

sub TO_JSON {
my $self = shift;

my $obj = {};
$obj->{$_} = $self->{$_} foreach grep { ! /^(email|password)$/ } (keys %$self);
return $obj;
}

sub save {
my $self = shift;

die "Cannot save without a valid filename!" if ! defined $self->{filename};

my $filename = $self->{filename};
my $json = JSON->new->allow_blessed(1)->convert_blessed(1);
my $data = $json->encode($self) or die "Invalid JSON conversion: $!";
open FP, ">$filename" or die "Cannot open $filename: $!";
print FP "$data\n";
close FP;
}


How's that?

Note the following… when I create a new person object, I would do so from other code as

my $person = Example::Person->new('jdoe');

This would attempt to load jdoe.json, if the file exists, otherwise create an empty object with that filename. If I didn't pass in a value, it would use a default, but you could also just leave everything undefined.

To save the object, you'd call $person->save();

When you attempt to serialize an object, the JSON library calls the TO_JSON method to get the data to serialize. At that point, you can either just hand back the data, or manipulate it as desired.

Note that the part relevant to our discussion is the TO_JSON code. You'll note that, as per your request, I've omitted the email and password fields. This adds a whopping ONE line of complexity to my saving code for this object. When I decide to add a new field for ICQ numbers, you will note that I don't have to adjust the loading or saving code, just the initialization code in new()…. unless you don't want to save ICQ numbers either, then I'd add it to the list of things to not save.

—-

Now, as to your desire to have "area files". I don't consider that a requirement for it to be a DikuMUD any more than I consider your argument about using objects instead of flat data structures to be valid. If you want to have the same "template" mechanics, just make duplicates of each object rather than re-loading them from disk. Rocket science it is not.

But, if you DID want to bundle a bunch of rooms, npc's, etc into a single file… you don't have to write each thing out to its own file. You can just as easily keep them as strings and write them out en-masse… or better yet, make your area object's serializer also serialize all child objects, who will in turn do the same.

Then your area object's serialization has a field for NPC's that is an array of serialized NPC objects.. each of which probably has an array of serialized ITEM objects for their inventory.
06 May, 2014, Davion wrote in the 12th comment:
Votes: 0
plamzi said:
Davion said:
This is also a problem I was coming across. It keeps bringing me back to looking at ORMs. Djangos in particular would have no problem serializing the data, and even pumping it out to JSON to be transported.


I'm not sure why you think ORM's are the solution to the "problem" Tyche is showing. I'm using an ORM for one of my projects. AFAIK, no ORM is going to know that sometimes you don't want to output the "password" parameter, but sometimes you do. No ORM is going to automatically know that sometimes you may want to ignore a reference to another object, while sometimes you may want to recurse it. When I don't want to output certain parameters in the ORM-based app, I have to either make them non-enumerable, or simply filter them during composing the output. That's the only way, ORM or no-ORM, unless you want to have and maintain two versions of the object, which is obviously even more of a headache.

Also, at least the ORM I am using, requires that you define all relationships and data types for each object in its own language. That's an understandable requirement, because it needs to be able to recreate the storage routines for these if you change the storage medium. But then, if you want to modify your objects, you have to modify those definitions accordingly so it will know how to handle the new data members.

At the end of the day, the benefits that an ORM brings to the table have nothing to do with overcoming the issues listed above. And anyway, knowing the OP, he will probably advocate for writing your own ORM :)


The problem presented involves storage and handling of data on serialization and objectification using JSON. The particular ORM I deal with allows for you to customize the way each individual variable is handled on definition. Should there be special cases like these you can attach hooks that change the behaviour in a particular instance. eg. which widget to use in a form.

Part of the problem as well is distinguishing between a reference and an actual object. The ORM handles that and Django also lets you pump out JSON'd versions of your objects for you. Should you want to use JSON as your file format you'd create fixtures of your objects via Django which are just JSON files and use those. Maybe not using an ORM instead but in conjunction. It might be a bit over the top but you'd need the fine control over each variable like that. We'r
07 May, 2014, Tyche wrote in the 13th comment:
Votes: 0
plamzi said:
As it often happens, you didn't address the substance of my post, and also I have no idea what you mean.


Sending network messages using JSON wasn't the substance of my post. Serialization of a DikuMud was.

plamzi said:
Am I the only one here whose logical sensor is in agony from this pseudo-argument? I know of no storage *period* "that understands how to convert references to another object from pointers/references to vnums, or which data is to be persisted and which is temporary game state data" without being told in some form or fashion. So what exactly are you comparing JSON to? Something that can read your mind?


I agree. And that is exactly the point that prompted me to spawn this thread.
07 May, 2014, Tyche wrote in the 14th comment:
Votes: 0
Davion said:
For using JSON the best I've come up with is creating a dictionary of the values you want to save, then JSONing that. On the other end, you'd load the dictionary, then translate needed values (eg, inroom vs contents). to_JSON is fine for simple cases but for diku's you're never sure if it's a reference or the entire object itself. You can't guess…


There's an advantage in being able to read (and maybe write) existing area file formats, even if you want to also use a database.
The objects and mobiles prototypes will be easier to process because they don't store any temporary game state like the room and exits do.
07 May, 2014, Tyche wrote in the 15th comment:
Votes: 0
quixadhal said:
Tyche said:
Here's an example that might get the point across:
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
};

Serialize this using JSON, but don't serialize email or password.

How's that?

Excellent. Now I'm going to add a new field for a feature.
my $people_data = {
name => "Bob Smith"
address => "123 Main St.",
city => "MyTown",
state => "MyState",
email => "bob@yahoo.com",
password => "password"
ignore_list => $ignore_list
};

I'm going to have to make a decision whether or not to serialize this, and how to do it.
So I'm going to have to modify my application ("mud") to use this feature, present it to an interface to the user ("OLC"),
and modify my save and load code (to_json, to_yaml, to_xml, to_mysql, to_whatever_custom_format).

We know it doesn't make the process of coding changes any different.
So what really are the benefits of converting the DikuMUD file formats to JSON?
07 May, 2014, quixadhal wrote in the 16th comment:
Votes: 0
Having serialization or not isn't going to help you "decide whether or not to serialize this". However, the "how to do it" part is pretty simple.

If $ignore_list is an array-ref (I assume so, since it would be @ignore_list if it were an array), you do… NOTHING.

Yep, that's right. My code, as shown above, will just happily store the array as JSON… it would look something like:

{"name":"Bob Smith","address":"123 Main St.","city":"MyTown","state":"MyState","ignore_list":["entry1","entry2",…]}


If the contents of ignore_list are objects instead of simple types, then THOSE objects would have a TO_JSON that would serialize each one, and then you'd see something like:

{"name":"Bob Smith","address":"123 Main St.","city":"MyTown","state":"MyState","ignore_list":[{"name":"entry1",…},{"name":"entry2",…},…]}


But, the important thing is you don't need to change the TO_JSON of the person class to accomodate that. And if it is just a list of names or id's, there's no code involved at all… JSON deals with scalars, arrays, and mappings without needing to be spoon-fed. At least, dynamic language versions do. I guess C doesn't, because it has no meta-knowledge… it's just pointers and how to read from them, write to them, or call code from them.
07 May, 2014, Kelvin wrote in the 17th comment:
Votes: 0
Tyche said:
We know it doesn't make the process of coding changes any different.
So what really are the benefits of converting the DikuMUD file formats to JSON?


There really aren't many, aside from making them easier for humans to read directly. But really, the area files are meant for consumption by the codebase, so this doesn't even register on my list of cares. Yeah, the Diku format is funky, but of all the things I could spend my time trying to innovate on (for the sake of my players), this is about as bottom-of-the-list as it gets.

We as MU* developers tend to gut/rewrite stuff just for giggles. Doing something like this would fall under that.
07 May, 2014, Tyche wrote in the 18th comment:
Votes: 0
Kelvin said:
Tyche said:
We know it doesn't make the process of coding changes any different.
So what really are the benefits of converting the DikuMUD file formats to JSON?


There really aren't many, aside from making them easier for humans to read directly. But really, the area files are meant for consumption by the codebase, so this doesn't even register on my list of cares. Yeah, the Diku format is funky, but of all the things I could spend my time trying to innovate on (for the sake of my players), this is about as bottom-of-the-list as it gets.

We as MU* developers tend to gut/rewrite stuff just for giggles. Doing something like this would fall under that.


Well this really ->… meant for consumption by the codebase

So one's pragmatic choices for serialization are those mechanisms which are well supported and tightly integrated to what your using.
In Ruby, maybe Marshall or YAML. In Python, maybe Pickle. The obvious advantage is those mechanisms know all about native types.
JSON doesn't understand Python or Ruby classes so you're going to write encode and decode routines anyway.

Worse, using JSON on Ruby presents data loss problems:
irb> JSON::load(JSON::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {"1"=>"c", "one"=>"d"}
irb> YAML::load(YAML::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {1=>"a", :one=>"b", "1"=>"c", "one"=>"d"}
irb> Marshal::load(Marshal::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {1=>"a", :one=>"b", "1"=>"c", "one"=>"d"}
irb(main):009:0> JSON::load(JSON::dump([1, :one, "one"]))
=> [1, "one", "one"]
irb(main):010:0> YAML::load(YAML::dump([1, :one, "one"]))
=> [1, :one, "one"]
irb(main):011:0> Marshal::load(Marshal::dump([1, :one, "one"]))
=> [1, :one, "one"]


Another method is carefully design your architecture to only use types which JSON can handle correctly.
I can't think of many reasons to do that.
07 May, 2014, quixadhal wrote in the 19th comment:
Votes: 0
Tyche said:
Worse, using JSON on Ruby presents data loss problems:
irb> JSON::load(JSON::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {"1"=>"c", "one"=>"d"}
irb> YAML::load(YAML::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {1=>"a", :one=>"b", "1"=>"c", "one"=>"d"}
irb> Marshal::load(Marshal::dump({1=>"a", :one =>"b", "1"=>"c", "one"=>"d"}))
=> {1=>"a", :one=>"b", "1"=>"c", "one"=>"d"}
irb(main):009:0> JSON::load(JSON::dump([1, :one, "one"]))
=> [1, "one", "one"]
irb(main):010:0> YAML::load(YAML::dump([1, :one, "one"]))
=> [1, :one, "one"]
irb(main):011:0> Marshal::load(Marshal::dump([1, :one, "one"]))
=> [1, :one, "one"]


Another method is carefully design your architecture to only use types which JSON can handle correctly.
I can't think of many reasons to do that.


Like you carefully constructed that example to exploit a weakness?

Not sure how common it is for real-world cases of people using both a numerical value AND the equivalent string value as keys in a mapping, besides errors in data processing that let such things through. Seems like a pretty terrible practice to begin with.

I tried to recreate that in perl, to see if it was the specific JSON implementation, however perl sensibly does NOT allow you to have the same value as multiple keys, despite being a different type. Maybe you could force it by overriding the way keys are likely stringified, but it seems like a way to allow problems.

Perl provides several "native" serializers as well. Storable and FreezeThaw are popular, but don't support regexp objects or circular references. Data::Dumper takes the unique approach of creating perl source code which can be eval'd to restore the object.
07 May, 2014, alteraeon wrote in the 20th comment:
Votes: 0
There aren't any. I saw this same thing fifteen years ago at IBM, when XML was supposed to be the magical interchange format that solved all data portability and storage problems. That was no more effective than JSON will be. It's just the fad of the year.

The simple fact of the matter is that conversion and translation of arbitrary in-memory structures to file data and back is complex. We must always remember: for every complex problem, there exists a solution that is simple, easy to implement, easy to understand, and wrong. There is no silver bullet.
0.0/35