Jump to content
Sign in to follow this  
lynkfs

tensorflow

Recommended Posts

edited

Been a while since I had a look at Googles Tensorflow deep learning library (js version)

In the meantime this library has expanded quite a bit, most notably it enables using pre-built models for a variety of tasks.

This one takes any image and detects up to 90 categories of objects in the image (persons, dogs, traffic lights etc)

Capture.PNG

The library correctly identifies 2 objects of type "person" in the above image (image 620*408 pixels, bbox coordinates x,y,width,height)

Usage is incredibly simple : load the libraries, display an image in the browser and issue a detect command

  var Script := document.createElement('script');
  Script.src := 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs';
  console.log('init tensorflow');
  document.head.appendChild(Script);
  Script.onload := procedure
  begin
    console.log('tensorflow loaded');
    var Script := document.createElement('script');
    Script.src := 'https://cdn.jsdelivr.net/npm/@tensorflow-models/coco-ssd';
    console.log('init coco-ssd');
    document.head.appendChild(Script);
    Script.onload := procedure
    begin
      console.log('coco-ssd loaded');

      var Image1 := TW3Image.Create(Self);
      Image1.SetBounds(0, 0, 620, 408);
      Image1.src := 'puydesancy.jpg';
      var img := Image1.handle;

      asm              //quick and dirty :
        cocoSsd.load().then(model => {
        // detect objects in the image.
          model.detect(@img).then(predictions => {
          console.log('Predictions: ', predictions);
          });
        });
      end;
    end;
  end;

(should have checked for image1.onload). Anyway, demo here. Works from server only due to cors constraints, otherwise results in 'tainted image' errors.

edited

and another random image

Capture2.PNG

 

 

Share this post


Link to post
Share on other sites

It seems a bit slow with JS implementation, I guess it can work realtime with C implmentation.

Anyway, interesting news from Google, they've entered SBC market with a mini computer with already set up libraries for tensorflow and machine learning

https://www.arrow.com/en/research-and-events/articles/tapping-into-the-power-of-google-an-introduction-to-googles-new-line-of-coral-products

Share this post


Link to post
Share on other sites

it is a bit slow

timing for this example :

start:                 0.0849609375ms
init tensorflow  :     0.936767578125ms
tensorflow loaded : 1017.56591796875ms
init coco-ssd :     1017.623779296875ms
coco-ssd loaded :   1421.75390625ms
get image :         1422.751708984375ms
Predictions:       45869.56982421875ms

Firefox has the same sort of response times, Edge doesn't support console timers.

It is basically the time to load the mobilenet model itself from the google servers, which is responsible for the large timegap between get image and the resulting predictions.

GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/model.json"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard5of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard4of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard3of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard2of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard1of5"

It should be possible to cache all of these files, or store them on a SBC :)

 

 

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×