Jump to content
DidierXT

Background uploading files

Recommended Posts

I need to send several binary files to a (RO) server.

The files are located on the user computer or phone.

The connexion is not so good and I need the sending to occur in a background sending queue.

I've found topic about the sending
https://forums.smartmobilestudio.com/topic/4111-upload-image-to-rest-server/

But I wonder if and how I can load the binary from a given file name (\fakepath?).

Or do I have to chain several (low) level Http requests
https://talk.remobjects.com/t/file-upload-using-http-post/6116/2

Please excuse my confusion, but I've read that file management with PhoneGap is somewhat unified with NodeJS. Nice.
But what about file management on web clients ? 
 

Share this post


Link to post
Share on other sites

First, lets look at a couple of facts:

  1. The browser does not have a filesystem (*)
  2. The browser only allow storage to the cache or database
  3. Phonegap does not change the above, but it does remove the limitations on size
  4. In version 3.0 of Smart which is just around the corner, this is all about to change for the better

(*) Most browsers implements a "fake" sandboxed filesystem. We have support for this, but the way it's implemented in the browser makes it extremely hard to port. You end up with a spaghetti mountain of code that leaves much to be desired.

What people do is to store binary data as bin-hex strings in the cache. This has its limitations, and depending on the browser you can store between 5 to 15 megabytes of data there. Again the cache is a sandboxed file, so you never get access to the real files. Its only meant to keep temporary data before you ship it to the server.

The units you want to look at for storing data in the cache are:

  • SmartCL.Storage
  • SmartCL.Storage.Local
  • SmartCL.Storage.Cookie
  • SmartCL.Storage.Session

SmartCl.Storage contains just a baseclass, but you should add that first to have it available (makes code suggestion faster). As the names imply, each of these provides a standard interface for all the browsers storage mechanisms. The one you want is "local". The session storage is just for temporary storage while the page is active. So when you app closes the data is deleted. Only "storage.local" is persistent.

Smart Mobile Studio v3

In the next version we add storage device classes to both web and nodejs. This allows you to store files, examine folders etc. and gives you a full filesystem-api on top of whatever is there. So in the browser we have actually implemented a complete b-tree based storage device from scratch, that saves to the local-storage for you. Under node the same API wraps the exotic and cumbersome node.js filesystem, and allows you to write code that works both in the browser and under node (!)

So I think you will love that :)

Uploading files

I presume you have X number of files you want to upload in the background right? Well, JS is async so you can just upload all of them at once. The uploading will happen in the background and wont block anything (javascript is non-blocking). But for sake of brewity, you can use a web-worker for it:

uses
  System.Types, System.Types.Convert,
  SmartCL.WebWorker;

type

TUploadThread = class(TWebWorkerThread)
private
  procedure InitializeRemobjects;
public
  // This fires when a message is received
  procedure ReceiveMessage(Data: Variant); override;
end;

procedure TUploadThread.Execute;
begin
  InitializeRemobjects();
end;

procedure TUploadThread.InitializeRemobjects;
begin
  // Do login and everything here
  // Remember, a webworker or thread runs in its own context so
  // it will know nothing of your main program
end;

procedure TUploadThread.ReceiveMessage(Data: Variant);
var
  ByteData: TByteArray;
begin
  // Convert data-string to bytes (note, see BytesToString method in same class)
  ByteData := TDataType.StringToBytes(Data);

  // Initialize remobjects for this context, login etc.
  InitializeRemobjects();

  // Do upload here
end;

Personally I would not bother with a web-worker for this. JavaScript is non-blocking, so you can just call upload on all the files, and they will finish as the work is done. Perhaps add a counter mechanism to make sure they all upload - and to catch errors if they dont.

In the above code, once you create it you can use the postmessage to tell the thread what to do -- and the OnMessage() event to catch messages sent back from the worker. This is actually pretty cool because the message stuff works on a lot of tech (for example between frames). So its worth taking a look at.

Hope it helps!

Share this post


Link to post
Share on other sites

Very useful informations, thanks!
I've gone the web-worker path. Indeed the messaging part is full of potential.

Still, how can I get (FileReader?) the content of the selected file?
I don't want the file to be sent directly after selection (on change), only after manual validation of the form.

Share this post


Link to post
Share on other sites

 

I 'm unable to retrieve FileList.
=> Uncaught TypeError: Cannot read property 'slice' of null

Full test project is attached.

uses W3C.DOM, W3C.File;

procedure TForm1.InitializeForm;
begin
  inherited;
  edPhoto.InputType := itFile;
  edPhoto.Handle.ReadyExecute(
    procedure ()
    begin
      w3_setAttrib(edPhoto.Handle, 'accept', 'image/png,image/jpg');
      w3_setAttrib(edPhoto.Handle, 'capture', '');

      // inline callback
      w3_AddEvent(edPhoto.Handle, 'click',
        procedure()
        var filelist : JFileList;
        begin
          var reader := new JFileReader;
          reader.onload :=
            function (e: JEvent): Variant
            begin
              var dataURL := reader.result;
              imgPhoto.LoadFromURL(dataURL);
              result := true;
            end;
          //var filelist := JFileList(event.target.files);
          asm
            @filelist = event.target.files;
          end;
          reader.readAsArrayBuffer(filelist.item(0).slice);
        end,
        FALSE
      );

      // separated callback
      //w3_AddEvent(edPhoto.Handle, 'click', @OnSelectImage, FALSE);
    end
  );
end;

//procedure TForm1.SelectImage(event: Variant);
procedure TForm1.OnSelectImage;
var filelist: JFileList;
begin
  var reader := new JFileReader;
  reader.onload :=
    function (e: JEvent): Variant
    begin
      var dataURL := reader.result;
      imgPhoto.LoadFromURL(dataURL);
      result := true;
    end;
  //var filelist := JFileList(event.target.files);
  asm
    @filelist = event.target.files;
  end;
  reader.readAsArrayBuffer(filelist.item(0).slice);
  // => Uncaught TypeError: Cannot read property 'slice' of null  [line #16957]
end;

 

ImgUploader.zip

Share this post


Link to post
Share on other sites

Here is the JS script I would like to reproduce. 
But seems they're is no event object available...

<input type='file' accept='image/*' onchange='openFile(event)'><br>
<img id='output'>
<script>
  var openFile = function(event) {
    var input = event.target;

    var reader = new FileReader();
    reader.onload = function(){
      var dataURL = reader.result;
      var output = document.getElementById('output');
      output.src = dataURL;
    };
    reader.readAsDataURL(input.files[0]);
  };
</script>

 

Share this post


Link to post
Share on other sites

JavaScript's eventlistener callback has an implicit event object, sort of confusing really.

Smart requires parameters to be named explicitely, which is a good thing

So if you need to access the event object, you can use constructs like these :

edPhoto.handle.onchange := procedure(event: variant)
begin

end;

or

edPhoto.handle.addEventListener('change',procedure(event:variant)
begin

end,false);

which has the same effect (different in that you can specify 'onchange' once only but 'change addeventlisteners' multiple times if necessary)

If you don't need access to the event object, just use W3_AddEvent, or one of the above constructs with or without the 'event:variant' bit 

 

Share this post


Link to post
Share on other sites

as to your other question :

I don't think you can step outside of the sandbox mechanism imposed by all browsers.

So, yes, you can access (image) files in the same directory as your index.html file

and yes, you can access (image) files in subdirectories of that directory, as long as you specify the path to this subdirectory in code

and no, you can't access files anywhere else on the users file system (using a browser)

If that's ok for your purposes, then you can simplify the code to something like

procedure TForm1.InitializeForm;
begin
  inherited;
  edPhoto.InputType := itFile;
  Handle.ReadyExecute(
    procedure ()
    begin
      w3_setAttrib(edPhoto.Handle, 'accept', 'image/png,image/jpg');
      w3_setAttrib(edPhoto.Handle, 'capture', '');

      edPhoto.handle.onchange := procedure()
        begin
          writeln(edPhoto.handle.files[0].name);
          writeln(edPhoto.Text);
          imgPhoto.LoadFromURL('images\' + edPhoto.handle.files[0].name);
        end;

    end
  );
end;

 

Share this post


Link to post
Share on other sites

Thanks Lynkfs,
I've made progress, but I think you have misunderstood my goal.
I want the selected picture (user computer/phone) to be sent to the server.
So no \image directory involved. I just need its binary data...

Anyway, it should be ok, although I still have to use JS for several reasons.

These events are never triggered:
edPhoto.Handle.OnChanged := procedure (event: variant)     // (BTW, no RTL reference to OnChange)
edPhoto.Handle.OnChanged := procedure()
edPhoto.Handle.OnChange := procedure (event: variant) 
edPhoto.Handle.OnChange := procedure()

These ones are working fine:
edPhoto.Handle.addEventListener('change', procedure (event: variant)
edPhoto.Handle.addEventListener('change', procedure ( )

 

So here is a working sample [SOLVED].
It would be nice if we could get rid off the asm block, and simplify a bit the whole process.
 

  edPhoto1.InputType := itFile;
  edPhoto1.Handle.ReadyExecute( procedure ()
    begin
      if w3_getIsMobile
        then w3_setAttrib(edPhoto1.Handle, 'accept', 'image/png,image/jpg')
        else w3_setAttrib(edPhoto1.Handle, 'accept', 'image/*');
      w3_setAttrib(edPhoto1.Handle, 'capture', '');

      edPhoto1.Handle.addEventListener('change', procedure (event: variant)
        begin
          writeln('OnChange');
          writeln(edPhoto1.handle.files[0].name);
          var img : variant := new JObject;
          img := imgPhoto1.Handle.id;
          asm
            var reader = new FileReader();
            reader.onload = function(){
              var data = reader.result;
              var output = document.getElementById(@img);
              output.src = data;
            }
            reader.readAsDataURL(event.target.files[0]);
          end;
        end);

    end);

 

Share this post


Link to post
Share on other sites

yep, get what you intend, interesting approach.

If you want to simplify this more, see if this works for you

Pointers :

- 'handle' is the doorway between object pascal and javascript : Pascal on the left and javascript on the right hand side. So edPhoto.Handle.OnChanged is invalid but both edPhoto.handle.onchange and edPhoto.OnChanged are syntactically correct

- 'event.target' is basically the 'handle' of the control the eventlistener is attached to, in your case the editbox. Since you're after the files object of this input element, you can refer to it by it's handle : edPhoto1.handle.files[0] instead of event.target.handle.files[0]. That also eleminates the requirement to access the event object.

- both the above means you could use the rtl 'OnChanged' handler rather than the javascriptish addeventhandler construct

- afaik javascript objects with a constructor which only allows the 'new' keyword need to be constructed in an asm block, no way around that. FileReader being one of them. Other than that, once you have a reference to that object, and since it is declared as a variant, you can reference all data and functions outside the constructor block. Which also enables more direct access to the image object

- (and to save yourself a mere 9 coding keystrokes, you could make the reader.onload handler anonymous, 'lambda' instead of 'procedure begin')

      edPhoto1.OnChanged := procedure(sender:TObject)
        begin
          var reader: variant;
          asm @reader = new FileReader(); end;
          reader.onload := lambda imgPhoto1.handle.src := reader.result; end;
          reader.readAsDataURL(edPhoto1.handle.files[0]);
        end;

 

I didn't run any tests and can't comment on if this is in line with what you want, but it does compile :)

cheers

 

Edited :

obviously I was wrong in my previous post. Using your method it actually is possible to access (at least images) anywhere on the users filesystem in the browser.

 

Share this post


Link to post
Share on other sites
reader.readAsArrayBuffer(filelist.item(0).slice);

First of all "slice" is not a property, but a function.
Secondly, item/items is an array, so you access them as: filelist.item[0].
There is a variance between browsers, so i dont think "item" is right, but rather "items".

Share this post


Link to post
Share on other sites

How would this sample be extended so that the image uploaded is stored in the (for example) /res folder?

Is that possible? If so how?

 

Share this post


Link to post
Share on other sites

afaik you can access images from anywhere on the users pc, but only store these by autodownload in the default download directory (win10 : c:/users/.../Downloads)

 

procedure TForm1.InitializeForm;
begin
  inherited;
  edPhoto1.InputType := itFile;
  edPhoto1.Handle.ReadyExecute( procedure ()
    begin
      if w3_getIsMobile
        then w3_setAttrib(edPhoto1.Handle, 'accept', 'image/png,image/jpg')
        else w3_setAttrib(edPhoto1.Handle, 'accept', 'image/*');
      w3_setAttrib(edPhoto1.Handle, 'capture', '');

      edPhoto1.OnChanged := procedure(sender:TObject)
        begin
          var reader: variant;
          asm @reader = new FileReader(); end;
          reader.onload := lambda
            imgPhoto1.handle.src := reader.result;
            TW3URLObject.Download(reader.result, 'res/picture.png');
          end;

          reader.readAsDataURL(edPhoto1.handle.files[0]);
        end;


    end);
end;

ps : change the urlobject.download call to "TW3URLObject.Download(reader.result, 'picture.png');"

trying to include a destination other than the standard download directory just renames the output to something like "res_picture.png" (FF) and still stores this in the download directory

ps: doesn't work in ide or from file(chrome). server only

 

edited :

unless your question was how to upload images to a specific directory on the server rather than to a specific directory on the users pc. In that case disregard the above. You'll need an ajax call to a server side script.

 

Share this post


Link to post
Share on other sites

@lynkfs- I liked the ability to upload an image. But I was hoping to store that image server-side. So, yes, I am referring to your edited section. I might use a php script to upload the images - or simply ftp them up when required. I am making a little inhouse app to allow our franchisees to make personalised ads - i.e., image add in their phone number etc and then download the result to their computer. I was simply looking for a way to allow a colleague to upload the images without my help ;)

Share this post


Link to post
Share on other sites

This app is now almost done

https://numberworksnwords.com/adtoolkit/index.html

However, it won't download the resulting image in MS Edge :(


    writeln('About to create image for downloading');
    TW3URLObject.Download( LEncodedData, textbox[0].imagefilename);   // only works in a external browser

    writeln('encoded '+LEncodedData);

The  LENcodedData is there but no image is created when TW3URLObject.Download is called. Works no probs with Chrome and Firefox

 

Any ideas why edge is different and if there is possible to work around?

 

I have attached simple example

testedge.rar

Share this post


Link to post
Share on other sites

amazing app 💥 - looks absolutely wonderful :)

As to Edge, I don't know why these guys (MS) bother at all

In my experience Edge is completely buggy (latest : doesn't even allow localStorage on win10) and is far behind implementing even the simplest of html5 / w3c specs (detail/summary tags, doh) 

Edge is the new IE

(I'll have a look later if there is a workaround)

Cheers

 

Share this post


Link to post
Share on other sites

quick search

1) links which state there is a problem on edge with the anchor tag and download attribute
https://caniuse.com/#feat=download
https://stackoverflow.com/questions/18394871/download-attribute-on-a-tag-not-working-in-ie
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/14603958/

2) link which states that it all should work on edge. However look at the 'known issues' tab
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/7260192/

3) link with a possible solution, which will require some work before it is usable
https://catonrug.blogspot.com/2018/01/anchor-download-attribute-internet-explorer.html

suppose depends on how desperate you are to have it working in Edge

 

Share this post


Link to post
Share on other sites

Cheers Nico, the client app is as simple as I could make it. The creation app (not linked) has a bit more going for it. It is a bit rough but it is easy for my colleague to create new items with a minimum of fuss for franchisees to then customise and use for their social media requirements. It will save us many hours of work not having to customise on demand in Illustrator.

 

I am in no way a fan of Edge - it is just that my years of educating our franchisees in using a real browser haven't paid 100% dividends. I will check out the links. I have insufficient understanding of javascript and find interfacing with SMS mostly hocus pocus. I will see how I get on tomorrow - as for now I am back to painting architraves and window sills :(

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×