Jump to content
Sign in to follow this  
lynkfs

databases and applications

Recommended Posts

Two bobs worth of thoughts on the interface between applications and databases.

Every business today has its data stored in a number of databases. Whether SQL-based, proprietory, ms Office (Excel anyone ?), Hadoop on Google Cloud or whatever.

On the application side apps need access to this data, and more often than not the data they need will reside in more than 1 datastore. The days that every application had its own private database are long gone.

Which means we need something inbetween

middleware.JPG


This middleware has the task of translating application requests into database calls, retrieving the data and making it available in some format the application understands.
(Not ususally done, it could potentially also work the other way around : external changes to data could result in autonomously firing up of applications to process these changes, or at least put them in a queue for later processing).

There are some products available which occupy this space. Conceivably 'ORM's, which talk 'object' on one side and SQL on the other, or products based on GraphQL (a specification opensourced by Facebook which covers some of the above), and possibly other products unknown to me.

How would that work in SMS terms

Middleware
The middleware has to have a 'data'-model at its heart. This can be a traditional data-model, a UML model of sorts, a graph, or some other scheme - as long as it is able to represent data structure and dependencies. 
Since graphs are my new friends, I'm sticking with that for now.

An ubiquitous example is 'the library'.
a library ==> has books ==> have author(s) and books ==> have publisher 
And some variables associated with these nodes : title, ISBN number, publisher etc for books; name, contact info, autobiography details etc. for authors and co-authors.

Databases
Suppose the current situation of this imaginary library is something like this
"Some former regional libraries have been amalgamated into the new current library. However staff at present still has to work with the old separate databases, waiting for a reconciliation effort by the IT-department later this year/cycle/decade".

Applications.
App-01 is meant to solve this problem. It is also meant to decouple application processing and database management as much as possible. 

As an example : one of its functions is to display book and author info based on selecting (partial) titles.
Such a query call could look something like this

{
  book(title: "*smart*") {
    ISBN
    author
  }
}

the middleware would reconcile this with its internal graph model, find out where and how to extract the data fields for this query and spawn calls to one or more book databases. These SQL calls would be something like

select * from books a, authors b where a.author = b.author and a.title like '%smart%'

and the middleware would amalgamate the select cursors into a single result set and return that to App-01, possibly in a format like

{
  book: {
    'A Smart Book' {
      ISBN  : ''
      author: 'Primož Gabrijelcic'
    },
    'The Smart Company' {
      ISBN  : 'coming'
      author: 'Jon Aasenden'
    }
  }
} 

I think this would be feasible. The advantages of such an approach / product would be a certain amount of decoupling of dependencies between application and database.

Share this post


Link to post
Share on other sites

This is taken into account with the framework we are researching / making now.
The objects you use to access data act more as "front-ends" to the mechanisms behind it, and just like the filesystem classes the operations are all async.
This has the benefit of decoupling the consumer-part (i.e controls + bindings) from the producer aspect.
To the consumer parts the database can be resident - or on the other side of the planet, it cares not where the data originates, only that its made available in a format that it can use.

A typical middleware entity would be a node.js server that is connected to X number of databases.
The client will use the framework and connect to the node.js endpoint, but neither the controls or bindings will have any clue about the data originating from a server.
It will simply see a driver and issue calls to it. When the driver (or class) gets the data it will validate and dispatch it via its internal mechanisms.

The nice part about this is that, a datasource can be almost anything. As long as someone implements the 3 core classes, it can be anything from a hardcoded file, local database or remote data server for that matter.

I will go into more detail about these classes when we are a little closer, but you are quite right that old-school 1:1 endpoints is useless in the new paradigm

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×