Jump to content
Sign in to follow this  
lynkfs

node ground zero

Recommended Posts

Node is still a new beast to tame, and at times it drives me bonkers.

To get my head around node, I started with a couple of very simple node programs (js) and how these could be translated into object pascal. The exercise also meant to discover which of the RTL node units to use in which circumstances.

The first one is a node program which accesses www.random.org (a site which generates random integers) and prints out the value to the console.

in plain vanilla javascript :

var https = require('https');

//The url is: 'https://www.random.org/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
var options = {
  host: 'www.random.org',
  path: '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
};

callback = function(response) {
  var str = '';

  //another chunk of data has been received, so append it to `str`
  response.on('data', function (chunk) {
    str += chunk;
  });

  //the whole response has been received, so we just print it out here
  response.on('end', function () {
    console.log(str);
  });
}

https.request(options, callback).end();

http and https are modules built into node itself, so no need to npm-install these separately. The code above makes a https request to this particular url. In the callback it catches 2 events : on-'data' and on-'end'. On-'data' is called every time a chunk of data has been read. Not sure what the standard chunk-size is, but it will be more than the size of a single integer, so this event will only occur once. After that the on-'end' event fires. 

and executing "node tryout.js" does indeed produce a random integer

 

Converting this line-by-line to SMS :

start a new node project (tryout.sproj) and delete unit1.

Replace the code in the root unit (tryout) by this :

function RequireModule(id: string): Variant; external 'require';

var https := RequireModule('https');

var options: variant := new JObject;
options.host := 'www.random.org';
options.path := '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new';

var callback: variant := new JObject;
callback := procedure(response: variant)
begin
  var str: string := '';
  response.on('data', procedure(chunk: variant)
    begin
      str+= chunk;
    end);
  response.on('end', lambda
      asm console.log(@str); end;
    end);
end;

https.request(options, callback).end();

and compile. The compiled tryout.js in the output directory reads as

var https,
   options$1,
   callback;
https = require("https");
options$1 = {};
options$1.host = "www.random.org";
options$1.path = "\/integers\/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new";
callback = {};
callback = function (response) {
   var str = "";
   str = "";
   response.on("data",function (chunk) {
      str+=String(chunk);
   });
   response.on("end",function () {
      console.log(str);
   });
};
https.request(options$1,callback).end();

which is close enough and when executed produces the same results.

btw: the compilation output is slightly different when having the 'use main body' in the 'compiler/code generation' project options selected or unselected. However results are the same for both.

The first line

function RequireModule(id: string): Variant; external 'require'; 

is copied from NodeJS.Core.pas so obviously that unit could be included in this particular use-case.

 

Note 1 :

http(s) offers a shortcut for GET calls (http.get...), so the SMS project above could be shortened to

function RequireModule(id: string): Variant; external 'require';  
//or alternatively : uses NodeJS.Core;

var https := RequireModule('https');

var callback: variant := new JObject;
callback := procedure(response: variant)
begin
  var str: string := '';
  response.on('data', procedure(chunk: variant)
    begin
      str+= chunk;
    end);
  response.on('end', lambda  //procedure()
    //begin
      asm console.log(@str); end;
    end);
end;

https.get('https://www.random.org/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new', callback).end();

 

Note 2:

Instead of reading chunks of data and appending these into a string (see the 'data' event above), the response of the http request can also be sent to a stream.

In that case the code is slightly modified and even more simplified :

function RequireModule(id: string): Variant; external 'require';
var https := RequireModule('https');

var callback: variant := new JObject;
callback := procedure(response: variant)
begin
  response.setEncoding('utf8');
  response.on('readable', lambda asm console.log((@response).read()); end; end);
end;

https.get('https://forums.smartmobilestudio.com/topic/4652-node-ground-zero/', callback).end();

 

Note 3 : 

to make it all a bit more SMS-like :

start a standard new Node project and replace the code in Unit1 by :

unit Unit1;

interface

uses nodeBasics;

type
  TNodeProgram = class(TObject)
  public
    procedure   Execute;
    constructor Create; virtual;
    https:      variant;
    procedure   callback(response: variant);
  end;

implementation

constructor TNodeProgram.Create;
begin
  inherited Create;
  https := RequireModule('https');
end;

procedure TNodeProgram.callback(response: variant);
begin
  response.setEncoding('utf8');
  response.on('readable', lambda console.log(response.read()); end);
end;

procedure TNodeProgram.Execute;
begin
  https.get('https://www.random.org/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new', @callback).end();
end;

end.

and for now the bare-bones nodeBasics :

unit nodeBasics;

interface

type
  JConsole = class external 'Console'
  public
    procedure log(data: Variant);
  end;
var Console external 'console': JConsole;

function RequireModule(id: string): Variant; external 'require';

implementation

end.

 

 

 

 

Share this post


Link to post
Share on other sites

A Node http server

In plain vanilla javascript the simplest of all servers :

var http = require('http');
var requestListener = function (req, res) {
  res.writeHead(200);                   //responsecode 200 = 'ok'
  res.end('Hello World!');
}

var server = http.createServer(requestListener);
server.listen(8080);

Execute this (node tryout.js) and point a browser to http://localhost:8080 which will display the first words.

 

In SMS as close to the metal as possible :

function RequireModule(id: string): Variant; external 'require';

var http := RequireModule('http');

var callback: variant := new JObject;
callback := procedure(req, res: variant)
begin
  res.writeHead(200);
  res.end('Hello World!');
end;

var server := http.createServer(callback);
server.listen(8080);

Code above replaces the root-unit code in a standard new Node project. Unit1 can be be removed.

 

and in SMS style

unit Unit1;

interface

uses nodeBasics;

type
  TNodeProgram = class(TObject)
  public
    procedure   Execute;
    constructor Create; virtual;
    http:       variant;
    server:     variant;
    procedure   callback(req,res: variant);
  end;

implementation

constructor TNodeProgram.Create;
begin
  inherited Create;
  http := RequireModule('http');
  server := http.createServer(@callback);

  server.on('request', procedure(req, res: variant)
  begin
    req.setEncoding('utf8');
    req.on('readable', lambda console.log(req.read()); end);
    req.on('end', lambda console.log('DONE'); end);
  end);

end;

procedure TNodeProgram.callback(req,res: variant);
begin
  var media: variant := new JObject;
  asm @media = { "Content-Type": "text/plain" }; end;
  res.writeHead(200,media);
  res.write('PINGING: ');
  res.end('Hello World');
end;

procedure TNodeProgram.Execute;
begin
  server.listen(8080);
end;

end.

New Node project and replace Unit1 code with the above.

Some embellishments : the response code 200 header includes a text/plain encoding type (although default anyway) and the server now listens to incoming requests (and streams these requests to the console).

 

Note 1 :

A slightly different approach where the server is not created by http.createServer but by its constructor. Also some messaging at the onConnection event and an automatic timeOut of the connection after 2 sec

constructor TNodeProgram.Create;
begin
  inherited Create;
  http := RequireModule('http');
  asm @server = new (@http).Server(); end;

  server.on('connection', procedure(client: variant)
  begin
    console.log('Client arrived ' + DateTimeToStr(now));

    client.on('end', lambda
      console.log('Client left ' + DateTimeToStr(now));
    end);
  end);

  server.on('request', procedure(req, res: variant)
  begin
    req.setEncoding('utf8');
    req.on('readable', lambda
      var data : variant := req.read();
      if data then res.&end(data);
    end);
  end);

  server.setTimeout(2000,procedure(client:variant)
  begin
    client.&end();
  end);

end;

 

Note 2 :

The Http Server Example (simplehttserver) in the Featured Demos section is more polished (but not as close to the metal), otherwise pretty similar. 

Share this post


Link to post
Share on other sites

A node https server

The biggest difference between a http server and a https server are the SSL certificates necessary for the latter. To short-cut it all, two demo certificate files are provided below which are ok for local testing purposes. As these are files, they need to be read (synchronously !) from the file system (using package 'fs') prior to firing up the server. 

In plain vanilla javascript the simplest of https server is :

var https = require('https');
var fs = require('fs');

var options = {
  key: fs.readFileSync('key.pem'),
  cert: fs.readFileSync('cert.pem')
};

var a = https.createServer(options, function (req, res) {
  res.writeHead(200);
  res.end("hello world");
}).listen(8000);

Pointing a browser to https://localhost:8000/ gives initially a once-off warning about insecure connections. However clicking 'advanced' will eventually display the first words.

cert.pem

-----BEGIN CERTIFICATE-----
MIIDRjCCAi4CCQDSDjyQOeQzmDANBgkqhkiG9w0BAQsFADBnMQswCQYDVQQGEwJV
UzENMAsGA1UECAwEVXRhaDEOMAwGA1UEBwwFUHJvdm8xIzAhBgNVBAoMGkFDTUUg
U2lnbmluZyBBdXRob3JpdHkgSW5jMRQwEgYDVQQDDAtleGFtcGxlLmNvbTAeFw0x
NjA5MTYxMTEwMDZaFw0xODAxMjkxMTEwMDZaMGMxCzAJBgNVBAYTAlVTMQ0wCwYD
VQQIDARVdGFoMQ4wDAYDVQQHDAVQcm92bzEWMBQGA1UECgwNQUNNRSBUZWNoIElu
YzEdMBsGA1UEAwwUbG9jYWwuZm9vYmFyMzAwMC5jb20wggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDj1LC15whNFxq67LddOLUc6KBEbSiN4wT/lxLfXZ/Y
qBYvFvTsxh/nfJFkXNRF3Mj4kKC12NG29MCOwegTBOl8PQCqa56tpykP8+owpYri
+s5zxEtxkuIoC9Ectsi8hZ6vHrgT1avo9RHewCvvIAGm4Wmkb1w5Q5Y2xU+IIcju
fn/Vw/YKS4R9WNFRIBvIIbmqVFXPcsG46KxufRUgPmQP2iOGT+Vy+KxHFCn0vs2y
nOvzaTRxP6Q4AUGXDy30ZM5589k6r/SccrdWpbKqJ8XJ8J0mILvrbrupvmre8PBt
5mgI+WtcmH7w1LXp7gM1p2MwUoEINUcIbyrkKF41suqRAgMBAAEwDQYJKoZIhvcN
AQELBQADggEBACDweIoT0wksdfitj2PtXb5tAenSKXBtEuKO+x56y67b3np7+WM6
ciNhq5XNMAuZdSusO5Xa9aRSFQywg8hNdn0l02PHiiLi5BloWvGFJAag1dWRclpx
jfD+Eh9JGFRvwXoW10p43Rc9hR9yVXVnp+5aBjjM1sfW8zQp/+S/DJhkPZWMt8H/
hN5kW411VMM9OMEuP2k6TbYX8HM8XPetnIz4X6jTAgGN4+9fj6pTjamkeHf9KmKG
y1lT7zISr5sakKLcwDhXydDrmls2JUNSH3gTWz+9yHgAjJVo7QTDle0JNqyDvkO+
pH3t2LOqyX6xAOIs6KGu22Ys/LmY5dUK1V8=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIDoTCCAomgAwIBAgIJAL4U2wSzq/CnMA0GCSqGSIb3DQEBCwUAMGcxCzAJBgNV
BAYTAlVTMQ0wCwYDVQQIDARVdGFoMQ4wDAYDVQQHDAVQcm92bzEjMCEGA1UECgwa
QUNNRSBTaWduaW5nIEF1dGhvcml0eSBJbmMxFDASBgNVBAMMC2V4YW1wbGUuY29t
MB4XDTE2MDkxNjExMTAwNVoXDTE5MDcwNzExMTAwNVowZzELMAkGA1UEBhMCVVMx
DTALBgNVBAgMBFV0YWgxDjAMBgNVBAcMBVByb3ZvMSMwIQYDVQQKDBpBQ01FIFNp
Z25pbmcgQXV0aG9yaXR5IEluYzEUMBIGA1UEAwwLZXhhbXBsZS5jb20wggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDWZDpGbn2X95qfX6ytG/hMd0t2vtkh
NlwIFNjI//354CrMhzsHSWdk6R5QMVQ6PDUFXK++/vaqyig/y8UW6Pf5nB7nFIZt
4lu9jn0DVXZN6TiAvnkXpFBL3zpQ15hivkmYUX2UfsC469oqPEIU+sl6YoDprwbL
+02239llAZXYfhUC3c5wfEehwCen6mq41OGon4R4daDRzITfZP5bB5mfRUPACWXM
emDD1xlarFqt3owuJKZ1nUnugTntehN55MG4G+FZP19XQQhA5e1gktMgmAzo/5qp
4bVqSH3AZEP1sVi2lKeCeitd7wLkLBnYbFMKAhx10EcTL5APd0VPtIQrAgMBAAGj
UDBOMB0GA1UdDgQWBBSL3xmPmIBW3pcdk4R7Lr0BV4A9iTAfBgNVHSMEGDAWgBSL
3xmPmIBW3pcdk4R7Lr0BV4A9iTAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUA
A4IBAQCD4xwt9L4BHjBNwUbC9SXrIpfBf/CAZsqXqbxmKQ0pIChbJ53K6vBl0HqU
+0vXYHRsn0Vdu1ROjfoCP+CDHj+60A99gGqTS9fKjpTqMx78iq1YQxHOFLmt/GIy
GGKDLyTMvM5ET2rKfUBS3gYqZQNeA6m4AFFkyAGpz3P/4/YEP/y0Ny2wQjFbkaAA
HQDwNQknC+0wRXhxDzq4P4D5QiKXJCbZwG462Arj1qwcmNTiwtbMyhjmTTP1ijXv
tHoRwUNDE5f+r6NpHF/K9fenwkt4z795hF6k3ifqS4SOpPccL/Yj6KXJmXns8TIk
Aa4I51rCF0MV+2NbKsjxNToDS/71
-----END CERTIFICATE-----

key.pem

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA49SwtecITRcauuy3XTi1HOigRG0ojeME/5cS312f2KgWLxb0
7MYf53yRZFzURdzI+JCgtdjRtvTAjsHoEwTpfD0AqmueracpD/PqMKWK4vrOc8RL
cZLiKAvRHLbIvIWerx64E9Wr6PUR3sAr7yABpuFppG9cOUOWNsVPiCHI7n5/1cP2
CkuEfVjRUSAbyCG5qlRVz3LBuOisbn0VID5kD9ojhk/lcvisRxQp9L7Nspzr82k0
cT+kOAFBlw8t9GTOefPZOq/0nHK3VqWyqifFyfCdJiC76267qb5q3vDwbeZoCPlr
XJh+8NS16e4DNadjMFKBCDVHCG8q5CheNbLqkQIDAQABAoIBAEGEAiTtAQgaIseq
Frip+/sKMjw7H7Rdmixdp0GwTK0i+O4lrd0AxF/vBGhWAXztlqVUAcMJTJVOIKls
vW3qtbHSYQSg2gqZzUcXHztJ/3xyHzhPNFq+oGZfr4yQS31a91zoeorJoRvmjXL+
1CN5ksAIhjaRB3Y4J1QsvUgqXvulvcR5CT7J/CKSI0OXNNZH/yvv6UMk2wKgwKch
Ip5xW0LcQu9XyFTxQXHiP0PoVvJLFzqEnfaPZctnkJqvBZ68HfjuAV2vBfQEQGDd
1xa9sSxuAeg7kV0z5Pgs/jV9u6a08qXExTC1NeAR2nYK8lVS/YYm0BoxxY9PhYPh
TSkXoyECgYEA/0ovo5AUAMmN7XYr6/Q26kqAmHUtSInzPOiK4GSqlIASTmQuoF3u
oWzNyqexr5S4yKGLMK1UilDieHX87Vf6bR+K8oXgpbY6nno1rpWXYxe0CQmX+wAu
2BMLIAAqZmBbxpjfp0fTSPAn6ucA8cTtG4wsPcRLTihLPMx2/0IfGr0CgYEA5Hby
wq6NUwKu9st2QmPk3ieFbSqzCb/D3VYNdV6/AuDT87q4b0N5ZaGUknSV3hnzjGjF
h7q3nSNw4syV0h+xXAqQiN+syHy5hJ6g3SQsvG3W/pPxWq1KWKAnujk3mF0+N45f
i+Dm6wC0sQpnuX7IUF+ny1V+Ops6jKaOVnhHtmUCgYEAyhVJnKoii3UBOmX+4qrR
BamwHf8zt2Yr+50Awhwtw5uV4iP/nMZ+bZ4ZUUBpsVmC5J6r9f5fG5Znj3+mlaK9
A8CiVtSPPd4AjQ6ki+yyh6htxXGosvI0IqsAdXZsGdpRC+ZbtKlwoR4qGxJ9duSx
5MqTAtnt2pKe7CPWVr1arekCgYBRy+SOn0AqAEP0SFRG4M0IXM4Aj9EaJHZTwIOB
CjMJLMW19Zwi+d107azr4qHlqxTvqHNQbSFgOVgpW2YonQA7G+0zlCGghkdFnCWs
B0+dBxZ6fy1icbi0kmKm4eVtv7SO34KF6jfC27XVzQvl9eZyIE1LF4jiIsGLqfY8
L0Q9oQKBgQCZNyZQEX0XDHphYTNM9EjkSPQRX/aU9YHSEaBlYKzmPE+ZErXa7A4u
sPJM6tGh/A1px5wMNVcK5t36iIFcgLuPNqu81kgSqRXkP8pkX9fEw450a6YOVTfC
mzjWLxCnNVbONtGfYOpNZWIYNTjLMOYjrZ4WFySagIlA+5s+F1HKUw==
-----END RSA PRIVATE KEY-----

 

Note 1 :

Real SSL certificates need a third option, something like 

var options = {
  key: fs.readFileSync('mysite.key'),
  cert: fs.readFileSync('mysite.com.crt'),
  ca: [fs.readFileSync('gd_bundle.crt')]
};

 

Note 2 :

The SMS code bases for https servers are the same as those in the previous http-server post.

 

Share this post


Link to post
Share on other sites

and to end this post series : a socket-server with a MySql database at the back-end

unit Unit1;

interface

uses nodeBasics;

type
  TNodeProgram = class(TObject)
  public
    procedure   Execute;
    constructor Create; virtual;
    mysql:      variant;
    io:         variant;
    db:         variant;
    // Define global variables
    notes : variant;
    isInitNotes : boolean;
    socketCount : integer;
  end;

implementation

constructor TNodeProgram.Create;
begin
  inherited Create;
  mysql := RequireModule('mysql');
  io    := RequireModule('socket.io').listen(3000);

  // Database connect
  var credentials : variant := new JObject;
  credentials.host     := '***************';
  credentials.user     := '***************';
  credentials.password := '***************';
  credentials.database := '***************';
  db := mysql.createConnection(credentials);

  // Log any errors connected to the db
  db.connect(procedure(err:variant)
  begin
    if (err) then console.log(err);
  end);

  // Initialize global variables
  asm @notes = []; end;    //or TVariant.CreateArray
  isInitNotes := false;
  socketCount := 0;

end;

procedure TNodeProgram.Execute;
begin
//define major event handlers
io.sockets.on('connection', procedure(socket:variant)
begin
  // Socket has connected, increase socket count
  inc(socketCount);

  // Let all sockets know how many are connected
  io.sockets.emit('users connected', socketCount);

  socket.on('disconnect', procedure()
  begin
    // Decrease the socket count on a disconnect, emit
    Dec(socketCount);
    io.sockets.emit('users connected', socketCount);
  end);

  socket.on('new note', procedure(data:variant)
  begin
    // New note added, push to all sockets and insert into db
    notes.push(data);
    io.sockets.emit('new note', data);
    db.query("INSERT INTO notes(note) VALUES ('" + data[0].note + "')");
  end);

  // Check to see if initial query/notes are set
  if isInitNotes = false then
  begin
    // Initial app start, run db query
    db.query('SELECT * FROM notes')
      .on('result', procedure(data:variant)
      begin
        // Push results onto the notes array
        notes.push(data);
      end)
      .on('end', procedure()
      begin
        // Only emit notes after query has been completed
        socket.emit('initial notes', notes);
      end);

    isInitNotes := true;
  end else begin
    // Initial notes already exist, send out
    socket.emit('initial notes', notes);
  end;
end);

end;

end.

 

The client for this server is the same as the client in this post from last year :  https://forums.smartmobilestudio.com/topic/4258-node-mysql-and-socketio/?tab=comments#comment-18069

 

Share this post


Link to post
Share on other sites

I had the idea firmly rooted in my mind that Node is for server-side deployment only.
This happens to be not the case, it is apparently feasible to use Node packages in the browser !

Talking about packages, the largest store of Node packages is NPM, which stores some 500,000 packages or more. (350,000 as per jan 2017)
Not all of those are suitable for use in the browser, but a lot of them are.

The ones which don't work client-side are the ones that require internet access, graphics or files, so this is not a work-around to be able to access databases on the client without going through a server, or enabling to write files client-side. Probably a good thing.

One of the ways to use an existing node package (or a self-written one) client-side is Browserify. This product basically takes a packages and iterates through all its dependencies to other packages and collates them in a single bundle. Which can then be used client-side.

Browserify also has written client-side versions of some of Node's core modules, which are automatically included in a bundle when necessary. HTTP and HTTPS for instance. That still doesn't make it possible to set up a http server within the browser, but other http-functions will work : the first program in this thread (see top of this thread) was a vanilla js node program which accesses www.random.org (a site which generates random integers) and prints out the value to the console.

in plain vanilla javascript :

var https = require('https');

//The url is: 'https://www.random.org/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
var options = {
  host: 'www.random.org',
  path: '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
};

callback = function(response) {
  var str = '';

  //another chunk of data has been received, so append it to `str`
  response.on('data', function (chunk) {
    str += chunk;
  });

  //the whole response has been received, so we just print it out here
  response.on('end', function () {
    console.log(str);
  });
}

https.request(options, callback).end();

Browserifying this package and accessing it from the browser works fine, and can be an alternative to issuing xmlhttprequests

Another example using Node's event mechanism 

// require the core node events module
var EventEmitter = require('events').EventEmitter

//create a new event emitter
var emitter = new EventEmitter

// set up a listener for the event
emitter.on('test', function (message) {
  console.log(message)
})

// emit an event
emitter.emit('test', 'this is an event test')

works fine as well, an alternative to add event listeners etc

It wouldn't be too difficult to Smarten this up either.

I'm having a look at the other 500,000 packages. Daunting but interesting.

 

 

 

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×