Jump to content
Sign in to follow this  
lynkfs

back(s)lash

Recommended Posts

5 lines of code which drove me bonkers.

My brilliant idea was to have a set of classes, where class A has a field which is an array of class B, and class B has a field which is an array of class C etc - and then just stringify the whole setup and store that string in a MySQL database. That would save me from setting up multilayered tables and I could just compress everything in a single base table.

  JB = record
    field1      : string;
    params  : Array of JC;
  end;

  JA= record
    fields : array of JB;
  end;

stringifying this (asm @textdata = JSON.stringify(@data);  where @data refers to JA) gives what I was after.

Except that all fieldnames in the result-string have a '$1' appended ({"field1$1"). No idea why that happens or how to avoid that.

Pushing on, using a while-loop with 'pos' and 'delete'  to get rid of this suffix doesn't work. The Pos and Delete functions don't work on strings resulting from an asm @string operation.

Next best thing : string replace with regex.

In the end I needed to do something like this :

asm
  @textdata = JSON.stringify(@data);
  var r = new RegExp('\\$1', 'g');           //delete all name-postfixes of '$1'
  @res = (@textdata).replace(r, "");
  var r2 = new RegExp('\\"', 'g');           //add backslashes just before double quotes
  @res2 = (@res).replace(r2, '\\"');         //and add mysql backslash character as well
end;

The first replacement operation gets rid of all the $1 suffixes

The second replacement operation sets a \ around text pieces which either have a special character or space in their content, like : font-family: \"Old Standard TT\"

And to make matters worse, found out that MySQL just ignores all backslashes in update statements. Unless these backslashes are preceded with a special character. (In this case another backslash).

Haven't mentioned some other caveats like handling EOL characters

Anyway, got it working, but it might have been easier to set up multi-layer tables after all.

If anyone has a solution for the automatic suffix addition ($1) in stringify operations please let me know.

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

> stringifying this (asm @textdata = JSON.stringify(@data);  where @data refers to JA) gives what I was after.

> Except that all fieldnames in the result-string have a '$1' appended ({"field1$1"). No idea why that happens or how to avoid that.

I would bet on obfuscation mechanism, try declaring all your fields with property:

JB = record
  property field1: string;
  property params: Array of JC;
end;

Share this post


Link to post
Share on other sites

 

uses ECMA.JSON
  
type
  JC = record
     id   : integer; external 'id';
     name : string; external 'name';
     age  : integer; external 'age';
  end;

  JB = record
    field1  : string; external 'field1';
    params  : Array of JC; external 'params';
  end;

  JA= record 
    fields : array of JB; external 'JA';
  end; 

var recA : JA;
    recB : JB;
    recC : JC;

for var k:=0 to 5 do begin
  recC.id   := k;
  recC.name := 'abc'+IntToStr(k);
  recC.age  := 10+k;

  recB.field1 := 'rec'+ IntToStr(k);
  recB.params.Add(recC);

  recA.fields.Add(recB);
end;


{
"JA" : [{
		"field1" : "rec0",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}, {
		"field1" : "rec1",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}, {
		"field1" : "rec2",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}, {
		"field1" : "rec3",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}, {
		"field1" : "rec4",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}, {
		"field1" : "rec5",
		"params" : [{
				"id" : 0,
				"name" : "abc0",
				"age" : 10
			}, {
				"id" : 1,
				"name" : "abc1",
				"age" : 11
			}, {
				"id" : 2,
				"name" : "abc2",
				"age" : 12
			}, {
				"id" : 3,
				"name" : "abc3",
				"age" : 13
			}, {
				"id" : 4,
				"name" : "abc4",
				"age" : 14
			}, {
				"id" : 5,
				"name" : "abc5",
				"age" : 15
			}
		]
	}
]
}

 

Share this post


Link to post
Share on other sites

Strange, no idea how you get the $ postfixing there (which means its a managed variable or object, it has nothing to do with obfuscation -- it just means smart handles the life-cycle of that variable or entity). Are you using inline variables to hold this or properly defined variables? There is a subtle but huge difference.

But as EWB points out, you need to build it up like you would any array. And I would add some functions for each record to simplify managing them. Like "AddParam(bla bla bla)" and stuff like that.

Just a quick test i did now to see what this was about:
 

 JC = record
    id   : integer; external 'id';
    name : string; external 'name';
    age  : integer; external 'age';
  end;

  JB = record
    field1      : string;
    params  : Array of JC;
  end;

  JA = record
    fields : array of JB;
  end;

Then i just wrote some code in a button onClick handler:

procedure Form1.W3Button1OnClick(Sender: TObject)
var
  head: JA;
  row : JB;
begin
  row.field1 := 'first';
  head.fields.add(row);

  asm
    @raw = JSON.stringify(@head);
  end;

  showmessage(raw);
end;

The code above, when it reaches the showmessage call, produces clean JSON. No postfixing. The postfix is added to all Smart "managed" variables, but records should be excluded from that. But (!) JavaScript work by references, so seem to me you managed to fall between two chairs and when you assigned values to the data, JS produces references to the values rather than the values itself (a bit like pointers can behave under COM Delphi coding).

Here is the result I got:

{"fields":[{"field1":"first"}]}

Now, mapping to existing data, which i presume you are doing (?), where you have a wad of JSON you load in, parse to an object structure -- and then i presume you want a ready to go array structure from that?

Well, i did something similar in the memory-filesystem unit if you look there, but as a general rule i read in the data into a variant, then typecast and traverse that, copying over the data i need. Then just null the variant to get the GC to clean up.

I also have a node.js websocket server that sends info as JSON on my network, and designed the structures to be simple and easy to transport. Using a typical envelope record to make identification easy, then attaching the actual message as base64 inside:

type
TNetworkPacket = record
	Identifier: string; //GUID
	ClientId:	string; //GUID
	envelopeId:	string; //ID of the contained type
	envelope: string;	// Contained type, base64 encoded
end;

// Then i can do simple lookups for the handler when i get a package

// Locate a handler for this JSON packet
var handler := __dispatch[envelopeid];

// Did we support this message? Ok, dispatch
if assigned(handler) then
	handler(Socket, Packet.Envelope):

 

Share this post


Link to post
Share on other sites

As to the unwanted suffix error, it doesn't reproduce using a simple code snippet like above, and I can't pinpoint why it did in my codebase.

It was resolved though by EWB's suggestion of using the 'external' keyword.

 

Mapping to existing data :

Quote

Well, i did something similar in the memory-filesystem unit if you look there, but as a general rule i read in the data into a variant, then typecast and traverse that, copying over the data i need. Then just null the variant to get the GC to clean up.

Thanks

Share this post


Link to post
Share on other sites

OK Lennart, brain-teaser here.

example of unwanted suffix :

{"ribbons$1":[{"ribbonname$1":"ribbonname","params$1":[{"paramtype$1":"paramtype"}]}]}

code to reproduce (normal standard latest alpha)

type
  JW3WebParam = class
    paramtype       : string;
    paramcontent    : string;
    strbefore       : string;
    strafter        : string;
  end;

  JW3WebRibbon = class
    ribbonname      : string;
    ribbonid        : integer;
    imagename       : string;
    category        : string;
    params          : Array of JW3WebParam;
  end;

  JW3WebProject = class
    projectname     : string;
    author          : string;
    password        : string;
    webheader       : string;
    webtrailer      : string;
    webjson         : string;
    ribbons         : Array of JW3WebRibbon;
  end;

type
  TForm1 = class(TW3Form)
    procedure W3Button1Click(Sender: TObject);
  private
    {$I 'Form1:intf'}
  protected
    procedure InitializeForm; override;
    procedure InitializeObject; override;
    procedure Resize; override;
    Procedure ShowProblem;
  end;

// record types for stringify purposes
type
  JStrC = record
    paramtype    : string;              //external 'paramtype';
  end;

  JStrB = record
    ribbonname   : string;              //external 'ribbonname';
    params       : Array of JStrC;      //external 'params';
  end;

  JStrA = record
    ribbons      : Array of JStrB;      //external 'ribbons';
  end;

var
  StrC : JStrC;
  StrB : JStrB;
  StrA : JStrA;


implementation

{ TForm1 }

procedure TForm1.W3Button1Click(Sender: TObject);
begin
  ShowProblem;
end;

Procedure TForm1.ShowProblem;
begin

    StrB.ribbonname := 'ribbonname';
    StrC.paramtype  := 'paramtype';
    StrB.params.add(StrC);
    StrA.ribbons.add(StrB);

// Turn object to json text
  var textdata: string := '';
  var res: string := '';

  asm
    @textdata = JSON.stringify(@StrA);
    console.log(@textdata);
  end;

end;


The above code produces the faulty stringyfied string.

Now notice that the first classes are actually not used :

type
  JW3WebParam = class
    paramtype       : string;
    paramcontent    : string;
    strbefore       : string;
    strafter        : string;
  end;

  JW3WebRibbon = class
    ribbonname      : string;
    ribbonid        : integer;
    imagename       : string;
    category        : string;
    params          : Array of JW3WebParam;
  end;

  JW3WebProject = class
    projectname     : string;
    author          : string;
    password        : string;
    webheader       : string;
    webtrailer      : string;
    webjson         : string;
    ribbons         : Array of JW3WebRibbon;
  end;

So delete these from the code, leaving just

type
  TForm1 = class(TW3Form)
    procedure W3Button1Click(Sender: TObject);
  private
    {$I 'Form1:intf'}
  protected
    procedure InitializeForm; override;
    procedure InitializeObject; override;
    procedure Resize; override;
    Procedure ShowProblem;
  end;

// record types for stringify purposes
type
  JStrC = record
    paramtype    : string;              //external 'paramtype';
  end;

  JStrB = record
    ribbonname   : string;              //external 'ribbonname';
    params       : Array of JStrC;      //external 'params';
  end;

  JStrA = record
    ribbons      : Array of JStrB;      //external 'ribbons';
  end;

var
  StrC : JStrC;
  StrB : JStrB;
  StrA : JStrA;


implementation

{ TForm1 }

procedure TForm1.W3Button1Click(Sender: TObject);
begin
  ShowProblem;
end;

Procedure TForm1.ShowProblem;
begin

    StrB.ribbonname := 'ribbonname';
    StrC.paramtype  := 'paramtype';
    StrB.params.add(StrC);
    StrA.ribbons.add(StrB);

// Turn object to json text
  var textdata: string := '';
  var res: string := '';

  asm
    @textdata = JSON.stringify(@StrA);
    console.log(@textdata);
  end;

end;

Executing this gives as output :

{"ribbons":[{"ribbonname":"ribbonname","params":[{"paramtype":"paramtype"}]}]}

Go figure

 

Note : may or may not be related to 

 

 

 

 

Share this post


Link to post
Share on other sites

Its not a "bug", the $ postfix / prefix means its a managed variable/field.
Write a normal streamer or implement a ToJSON() mechanism. The mess of raw JS is what we are trying to get away from :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×