Jump to content

Recommended Posts

  • Moderators

edited

Been a while since I had a look at Googles Tensorflow deep learning library (js version)

In the meantime this library has expanded quite a bit, most notably it enables using pre-built models for a variety of tasks.

This one takes any image and detects up to 90 categories of objects in the image (persons, dogs, traffic lights etc)

Capture.PNG

The library correctly identifies 2 objects of type "person" in the above image (image 620*408 pixels, bbox coordinates x,y,width,height)

Usage is incredibly simple : load the libraries, display an image in the browser and issue a detect command

  var Script := document.createElement('script');
  Script.src := 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs';
  console.log('init tensorflow');
  document.head.appendChild(Script);
  Script.onload := procedure
  begin
    console.log('tensorflow loaded');
    var Script := document.createElement('script');
    Script.src := 'https://cdn.jsdelivr.net/npm/@tensorflow-models/coco-ssd';
    console.log('init coco-ssd');
    document.head.appendChild(Script);
    Script.onload := procedure
    begin
      console.log('coco-ssd loaded');

      var Image1 := TW3Image.Create(Self);
      Image1.SetBounds(0, 0, 620, 408);
      Image1.src := 'puydesancy.jpg';
      var img := Image1.handle;

      asm              //quick and dirty :
        cocoSsd.load().then(model => {
        // detect objects in the image.
          model.detect(@img).then(predictions => {
          console.log('Predictions: ', predictions);
          });
        });
      end;
    end;
  end;

(should have checked for image1.onload). Anyway, demo here. Works from server only due to cors constraints, otherwise results in 'tainted image' errors.

edited

and another random image

Capture2.PNG

 

 

Link to post
Share on other sites

It seems a bit slow with JS implementation, I guess it can work realtime with C implmentation.

Anyway, interesting news from Google, they've entered SBC market with a mini computer with already set up libraries for tensorflow and machine learning

https://www.arrow.com/en/research-and-events/articles/tapping-into-the-power-of-google-an-introduction-to-googles-new-line-of-coral-products

Link to post
Share on other sites
  • Moderators

it is a bit slow

timing for this example :

start:                 0.0849609375ms
init tensorflow  :     0.936767578125ms
tensorflow loaded : 1017.56591796875ms
init coco-ssd :     1017.623779296875ms
coco-ssd loaded :   1421.75390625ms
get image :         1422.751708984375ms
Predictions:       45869.56982421875ms

Firefox has the same sort of response times, Edge doesn't support console timers.

It is basically the time to load the mobilenet model itself from the google servers, which is responsible for the large timegap between get image and the resulting predictions.

GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/model.json"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard5of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard4of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard3of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard2of5"
GET "https://storage.googleapis.com/tfjs-models/savedmodel/ssdlite_mobilenet_v2/group1-shard1of5"

It should be possible to cache all of these files, or store them on a SBC :)

 

 

 

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...