Jump to content

databases and applications

Recommended Posts

  • Moderators

Two bobs worth of thoughts on the interface between applications and databases.

Every business today has its data stored in a number of databases. Whether SQL-based, proprietory, ms Office (Excel anyone ?), Hadoop on Google Cloud or whatever.

On the application side apps need access to this data, and more often than not the data they need will reside in more than 1 datastore. The days that every application had its own private database are long gone.

Which means we need something inbetween


This middleware has the task of translating application requests into database calls, retrieving the data and making it available in some format the application understands.
(Not ususally done, it could potentially also work the other way around : external changes to data could result in autonomously firing up of applications to process these changes, or at least put them in a queue for later processing).

There are some products available which occupy this space. Conceivably 'ORM's, which talk 'object' on one side and SQL on the other, or products based on GraphQL (a specification opensourced by Facebook which covers some of the above), and possibly other products unknown to me.

How would that work in SMS terms

The middleware has to have a 'data'-model at its heart. This can be a traditional data-model, a UML model of sorts, a graph, or some other scheme - as long as it is able to represent data structure and dependencies. 
Since graphs are my new friends, I'm sticking with that for now.

An ubiquitous example is 'the library'.
a library ==> has books ==> have author(s) and books ==> have publisher 
And some variables associated with these nodes : title, ISBN number, publisher etc for books; name, contact info, autobiography details etc. for authors and co-authors.

Suppose the current situation of this imaginary library is something like this
"Some former regional libraries have been amalgamated into the new current library. However staff at present still has to work with the old separate databases, waiting for a reconciliation effort by the IT-department later this year/cycle/decade".

App-01 is meant to solve this problem. It is also meant to decouple application processing and database management as much as possible. 

As an example : one of its functions is to display book and author info based on selecting (partial) titles.
Such a query call could look something like this

  book(title: "*smart*") {

the middleware would reconcile this with its internal graph model, find out where and how to extract the data fields for this query and spawn calls to one or more book databases. These SQL calls would be something like

select * from books a, authors b where a.author = b.author and a.title like '%smart%'

and the middleware would amalgamate the select cursors into a single result set and return that to App-01, possibly in a format like

  book: {
    'A Smart Book' {
      ISBN  : ''
      author: 'Primož Gabrijelcic'
    'The Smart Company' {
      ISBN  : 'coming'
      author: 'Jon Aasenden'

I think this would be feasible. The advantages of such an approach / product would be a certain amount of decoupling of dependencies between application and database.

Link to post
Share on other sites

This is taken into account with the framework we are researching / making now.
The objects you use to access data act more as "front-ends" to the mechanisms behind it, and just like the filesystem classes the operations are all async.
This has the benefit of decoupling the consumer-part (i.e controls + bindings) from the producer aspect.
To the consumer parts the database can be resident - or on the other side of the planet, it cares not where the data originates, only that its made available in a format that it can use.

A typical middleware entity would be a node.js server that is connected to X number of databases.
The client will use the framework and connect to the node.js endpoint, but neither the controls or bindings will have any clue about the data originating from a server.
It will simply see a driver and issue calls to it. When the driver (or class) gets the data it will validate and dispatch it via its internal mechanisms.

The nice part about this is that, a datasource can be almost anything. As long as someone implements the 3 core classes, it can be anything from a hardcoded file, local database or remote data server for that matter.

I will go into more detail about these classes when we are a little closer, but you are quite right that old-school 1:1 endpoints is useless in the new paradigm

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...