Thought for improving the speed of the HTML 5: Will it work?

Bringing your stacks to the web

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

William Jamieson
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 212
Joined: Fri Feb 01, 2013 1:31 am
Location: Palo Alto, CA put williamdjamieso into tEmail / put n@gmail.com after tEmail/ revmail tEmail
Contact:

Thought for improving the speed of the HTML 5: Will it work?

Post by William Jamieson » Wed Sep 16, 2015 9:03 pm

I know that LC is already trying to compress the size of the loading package, but I just had a thought that might be able to help improve the speed for the HTML5 further.

I know that browsers can store certain pieces of information all the way from "Cookies" to Google chrome apps and every in between. Does anybody know if there is a way to store the LC javascript files in one's browser. So that when one initializes LC for the first time, it saves it to their browser, making the load time a lot less. Whether the file package is 9MB, 22MB, or even 1MB (in the future), I figure this could help.

The other idea is to separate the library by tiers. Put all of the most used functions into tier 1 to be downloaded first. Then, for all very specific functionalities we can create different tier packages such as -advanced movement functions, -advanced text functions -advanced media functions -etc And let the browser pull only what is needed.

Please let me know if either of these ideas have any merit to them. Thanks!

-Will

hliljegren
Posts: 108
Joined: Sun Aug 23, 2009 7:48 am
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by hliljegren » Mon Sep 21, 2015 10:37 am

This might be a nice add to the HTML deployment. But if the engine allows for modularisation I guess that is in plan for future releases.

Another way to improve the download speed is to have a fast CDN from where the files are downloaded. If everyone downloaded the engine .js and .mem files from the same URL it would only be downloaded once for all apps! The documentation says that "These files are always the same, and only change when LiveCode is upgraded." so moving them to a CDN will not help the first download but all following downloads (as long as your browser cache is big enough so the files doesn't get thrown away by the browser).
___________________________________
MacBook Pro, 15" 2.6GHz i7 Mac OS X 10.10.4
iMac 27", 3.2 GHz Quad i7, Mac OS 10.10.4
LiveCode 7.0.6 or 8.0dp4

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Tue Sep 22, 2015 10:48 am

Sort of your proposal is already available. Newer browsers have the two large files after the first load cached (and it saves disk space on the server):
  • File standalone.zip
    You can rename it, for example by script, and put it into any folder of your (accessible) file system.
    This has to be 'marked' in the calling html-file, see below.
  • File standalone-community-8.0.0-dp-4.html.mem (6 MByte)
    This must be in the same folder as the calling html-file (the file path is internally derived by LC's output javascript). We should propose to make this changeable.
  • File standalone-community-8.0.0-dp-4.js (11 MByte)
    This file can reside in any folder of your (accessible) file system. The path has to be given in the calling html-file.
Example.
Using all this, I have for me the following solution:

A folder LC-HTML5, local or on the webserver, containing (see explanation below)
  • ONE copy of standalone-community-8.0.0-dp-4.js
  • ONE copy of standalone-community-8.0.0-dp-4.html.mem
  • For each <project>
    a file <project>.html and a folder <project>OUT.
  • The folder <project>OUT contains <project>.livecode and <project>sa.zip
= My input stack is <project>.livecode.
= When LC creates by the HTML5 routine the folder <project>N then
...... a script renames and moves the file "standalone.zip"
...... to "<project>OUT/<project>sa.zip"
= Then the script deletes the folder <project>N (incl. contents)
...... and writes into the folder LC-HTML5 a file <project>.html
...... with the following content
...... (minimal, you have to replace <project>).

Code: Select all

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>My page title here :)</title>
</head>
<body>
  <div>
    <p>My top poetry here</p>
    <canvas style="border: 0px none;"
            id="canvas" oncontextmenu="event.preventDefault()"></canvas>
    <script type="text/javascript">
       var Module = { canvas: document.getElementById('canvas')  };
       Module.livecodeStandalone = "<project>so.zip";
       Module.livecodeStandalonePrefixURL = "./<project>OUT/";
    </script>
    <script async type="text/javascript"
       src="standalone-community-8.0.0-dp-4.js">
    </script>
    <p>My bottom poetry here</p>
 </div>
</body>
</html>
Edit 1: Forgot in the HTML above the call of the canvas. Corrected now, sorry for that.

Note. You may try the improvement in speed by this technique HERE (extern server, that shows some of my examples from the thread "Successful test" in this subforum).

There is also a demo of simultaneus load for identic modules: 3 rows and 2 columns respectively.This is nearly as fast as a loading of the corresponding single modules.

Edit 2: JUST FOR FUN.
[Next three links are from an extern server].
Start with the 2 diff columns display (the right column is the index page)
In the right index page choose the 3 diff rows display (the bottom row is the index page)
In the bottom index page choose the 3 identic rows display.

The frames are adjustable (drag dividers).

Now you have at left Alice at MacPlusPlus,
...... start the autoscroll there (black dot)

In the right column you have
= A countDown (start one)
= A six-month calendar (choose year and moth of your birth)
= Three drawTime modules
...... (click topleft btn of the second to show countDown to next midnight)
...... (click bottomleft btn of the third to show the date: YYMMDD)

The moduls are called from the following (really) minimal HTML,
I call the files <project>X.html

Code: Select all

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="utf-8">
	<title>HTML5: Your project name here</title>
</head>
<body>
    <div>
    <canvas style="border: 0px none;"
            id="canvas" oncontextmenu="event.preventDefault()"></canvas>
	<script type="text/javascript">
		var Module = { canvas: document.getElementById('canvas')  };
		Module.livecodeStandalone = "<project>sa.zip";
	  Module.livecodeStandalonePrefixURL = "./<project>OUT/";
	</script>
	<script async type="text/javascript" src="standalone-community-8.0.0-dp-4.js">
	</script>
	</div>
</body>
</html>
The frameset is called for example by the file "frameThreeRows.html" (3 diff rows display).

Code: Select all

<!DOCTYPE html>
<html>
<frameset rows="25%,45%,*"> 
   <frame src="./countdownX.html">
   <frame src="./calendarX.html">
   <frame src="./">
</frameset>
</html>
Note. The HTML tags "frameset"/"frames" are deprecated in HTML5, but there is no good substituting method, perhaps except using "iframe" (requires some work to have a similar view).
shiftLock happens

peter-b
Posts: 182
Joined: Thu Nov 20, 2014 2:14 pm
Location: LiveCode Ltd.

Re: Thought for improving the speed of the HTML 5: Will it w

Post by peter-b » Thu Sep 24, 2015 9:02 am

[-hh] wrote:
  • File standalone-community-8.0.0-dp-4.html.mem (6 MByte)
    This must be in the same folder as the calling html-file (the file path is internally derived by LC's output javascript). We should propose to make this changeable.
You can change this by setting

Code: Select all

Module.locateFile
to a suitable function. See the Emscripten documentation: https://kripken.github.io/emscripten-si ... locateFile
LiveCode Open Source Team — @PeterTBBrett — peter.brett@livecode.com

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Fri Sep 25, 2015 9:42 am

Thank you very much for this hint. Sadly this acts not only on the *.mem file.
Module.locateFile (excerpt from emscripten's docu)

If set, this method will be called when the runtime needs to load either a file generated by the file packager (this is a generalization of Module.filePackagePrefixURL), or the .mem memory init file. In both cases the function receives the URL, and should return the actual URL. This lets you host file packages or the .mem file on a different location than the current directory (which is the default expectation), for example if you want to host them on a CDN.
I tried *with* that several constellations, all had unpredictable results.
There is only (*without* that) one setting, that worked effectively (the browser caching) on all platforms:

Put all four files always in one and only one folder (that is, the two big files "*.js" and "*.mem" only once for each LC version). This folder can be anywhere where user has access or the server has access. Of course, you have to rename the data file "standalone.zip" for that, LC's standalone builder could do this using the standaloneName.

This seems to be a working solution, also for the future.
shiftLock happens

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Sat Oct 10, 2015 12:03 am

Meanwhile the answer to the question of the original post is

YES, IT WORKS.

My file organizing is as follows.
All four needed files (current versions)

--| standalone-community-8.0.0-dp-7.js (20,5 MB)
--| standalone-community-8.0.0-dp-7.html.mem (10,7 MB)
--| <project>-dp-7.zip (ca. 500 KB) <-- the renamed standalone.zip
--| <project>-dp-7.html (ca. 1 KB) <-- the (renamed) calling html

are in one and only folder on the server, the first two big files only ONCE.

The main browsers (MSIE/MSEdge do not yet work with LC's stuff) have different caching methods. For our purpose I have here this ranking from speed:
Firefox > Safari > Chrome and Opera.
Firefox takes the two big files, after loading two modules in a row, into the RAM and then loads other modules *very* fast.
The other browsers also become faster and faster, but have not the same extreme increase in speed as Firefox.

You may test it it here (from extern server).
First load two different modules in a row.
Then use the frameset of three rows and load by quick clicks three other modules.

If we can use later on a local 'static' storage for the two huge files, this will be faster than LC-server.
Go on LC dev team, go on fast with HTML5! You will win the day!
shiftLock happens

William Jamieson
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 212
Joined: Fri Feb 01, 2013 1:31 am
Location: Palo Alto, CA put williamdjamieso into tEmail / put n@gmail.com after tEmail/ revmail tEmail
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by William Jamieson » Mon Oct 19, 2015 7:54 am

They will win more than the day, they will win the industry...

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by monte » Thu Nov 05, 2015 11:29 pm

@-hh

I haven't tested but reading the docs it appears the standalone js file can be located anywhere just by changing the url and the Module.locateFile function can then be used to locate the .mem file and other data file (which I don't think we use) elsewhere also. If that's correct then we could all use some common CDN and cache the engine for each other.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Fri Nov 06, 2015 12:27 am

The (remaining) problem is indeed the data file ("standalone.zip") and it's location, currently dependent of the mem.file's location. The zip is the filesystem that's visible for the HTML5-"stack" and if (Peter B. says "soon") copy files will be available in the HTML5-standalone builder then the files will go into that zip (containg fonts/text/images etc.)

Because you, the wizard of externals, are visiting HTML5, here a big problem connected to that.
And you could solve it by a cross-platform/cross-browser external?
It's Copy and Paste into the HTML5-canvas or load and save files from the canvas, at least accessing the system clipboard.
As I understand, Emscripten (partially caused by security considerations) doesn't enable such things.
Thus it is currently not possible to paste (or load) for example a large data file into a HTML5-evaluation engine. There are approaches that use flash or OCR from screenshots, but that's not good/exact enough for working with sensible numeric data. This makes many desktop projects useless for HTML5 ...
shiftLock happens

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by FourthWorld » Fri Nov 06, 2015 12:50 am

[-hh] wrote:Because you, the wizard of externals, are visiting HTML5, here a big problem connected to that.
And you could solve it by a cross-platform/cross-browser external?
It's Copy and Paste into the HTML5-canvas or load and save files from the canvas, at least accessing the system clipboard.
As I understand, Emscripten (partially caused by security considerations) doesn't enable such things.
That should be enforced by all browsers. If it's possible to download and execute compiled machine code over the Web that would be a very dangerous security hole in the browser which should be reported and addressed immediately.
This makes many desktop projects useless for HTML5 ...
On the desktop as with mobile, web apps are great for many things, native apps great for others, and the capabilities and benefits of each are different enough that their optimal use-cases are fairly distinct.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Fri Nov 06, 2015 2:17 am

I really don't understand these considerations.

Copying and pasting plain text or upload and download images or similar objects as we do for example here in the forum should be made available. That's what I want. It requires some clear user interaction to do that and uses, if user wishes, the system clipboard and local file system.

So what is the difference between an early javascript editor and an advanced HTML5 editor? There are online-services allowing to upload large dataFiles for evaluation, this evaluation mostly done in php or python on a local server. Why not to an HTML5 app?

May the user take these decisions and not an international thought police.

Accessing the system clipboard means, that the user accesses it, what else?
It's possible with each and every browser (for forms, search fields and url-input etc.).
Everything else, without user interaction, is a criminal act, no matter if one is using a mobile, desktop or web app, isn't it?

[Written in TextEdit on Mac and pasted from my system clipboard].
Last edited by [-hh] on Fri Nov 06, 2015 8:39 pm, edited 1 time in total.
shiftLock happens

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by FourthWorld » Fri Nov 06, 2015 2:51 am

Executing externals and copying data are very different things. Browsers support copy and paste. What issues have you found with the copy and paste commands in H5 export?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Thought for improving the speed of the HTML 5: Will it w

Post by monte » Fri Nov 06, 2015 4:37 am

I experimented with this and it appears to work in terms of delivering the .js and .mem file from elsewhere however because there's no caching of the mem file there's still a sizeable lag while that downloads. If looks like there's a couple of options found here https://kripken.github.io/emscripten-si ... /emcc.html

* Compile with --memory-init-file 0 so there is no mem file. The docs say this is less efficient as it is stored in text in the js, however, it seems unlikely it is less efficient than downloading the mem every refresh.
* Use Module.memoryInitializerRequest to download a cache the mem file

This shows my changes to the file that the standalone builder creates in order to download the two big files from wherever we want:

Code: Select all

    
        locateFile: function( url ) {
            var extension = url.split('.').pop();

            if (extension === "mem")
            {
              return "http://appisle.net/livecode/standalone-community-8.0.0-dp-8.html.mem";
            }
            else
            {
              return url;
            }
        }
      };
      Module.setStatus('Downloading...');
      window.onerror = function(event) {
        // TODO: do not warn on ok events like simulating an infinite loop or exitStatus
        Module.setStatus('Exception thrown, see JavaScript console');
        spinnerElement.style.display = 'none';
        Module.setStatus = function(text) {
          if (text) Module.printErr('[post-exception status] ' + text);
        };
      };
    </script>
    <script async type="text/javascript" src="http://appisle.net/livecode/standalone-community-8.0.0-dp-8.js"></script>
Cheers

Monte
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: Thought for improving the speed of the HTML 5: Will it w

Post by LCMark » Fri Nov 06, 2015 9:13 am

@hh: In order for the emscripten engine to support access to the clipboard, the browsers it runs in have to support a means to manipulate the clipboard from JavaScript. The necessary APIs for this (in HTML5) are still in their infancy it seems.

There is a working draft from W3C here - http://www.w3.org/TR/clipboard-apis/.

Current browser support for clipboard access from JavaScript can be seen here http://caniuse.com/#feat=clipboard - there are lots of differences / limitations at the moment - mostly to do with 'security issues'.

Now, browsers do evolve pretty quickly in terms of implementing the W3C drafts (or their own variants) but there is some hope we could get something to work sooner rather than later - https://www.lucidchart.com/techblog/201 ... avascript/, however we still have to figure out how to get it to work in the context of LiveCode and LiveCode's clipboard features.

So this is not a limitation of emscripten - it is a limitation of current browsers - it is important to remember that the emscripten engine (beyond rendering - as we just rasterize everything ourselves) can *only* do as much as browser's allow and expose through JavaScript APIs and events.

[-hh]
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 2262
Joined: Thu Feb 28, 2013 11:52 pm
Location: Göttingen, DE

Re: Thought for improving the speed of the HTML 5: Will it w

Post by [-hh] » Fri Nov 06, 2015 9:04 pm

@Fourthworld
I misunderstood some of your arguments, sorry. You misunderstood some of mine too?
The discussion is about using very elementary
= copy/paste and get/set clipboardData["..."]
= (or more general) get/put into URL (...)
of LC by user interaction. Of course nobody here wants to have the ability to add SPAM or harmful compiled code to the system clipboard.

@Monte
That's it, summarizing in a javascript the first two posts.
The problem was until now the smaller (6 MByte only) .mem file, requiring that the calling HTML file is in the same folder.

@Mark
Peter already partially explained these problems when I reported "copy/paste" as bug/feature request for HTML5. That's why I thought of an external.
The 'security issues'-handling should simply be exactly the same as for the new general fullClipboardData usage of the desktop LC 8.
_________
Because the above sounds too much critical let me add a final positive statement:

I backed HTML5 support and I'm currently deep in "what is not (yet) working in HTML5".
Nevertheless I didn't hesitate a second to buy an (additional) year using the current HTML5 offer. Many users (especially me) are until now very contented with the HTML5 work and engagement of the LC team.

Already now, at this "experimental state", it's worth to buy the 'commercial' HTML5-standalone-building.
shiftLock happens

Post Reply

Return to “HTML5”