Pre-load RevBrowserCef to shorten launch time [Solved]

[Steve Collins]

A LC standalone I wrote for a widely distributed CDROM requires revBrowserCef (CEF) to play WebM videos via HTML5, a feature I recently added. It works great on Windows and Mac from a hard drive or flash drive. But from a CDROM, load time for just revBrowserCef is a whopping 70 seconds, a life time in computing. The entire 95MB app with the revBrowserCef resource can be copied from the CDROM in just 30 seconds, thats reading and writing, double the work of just loading into memory I figure. I'm suspicious that revBrowserCef takes forever to load from CDROM because it's in 17 pieces in the Package Contents folder, so lots of daisy-chaining pieces together finding and loading. That process is slow from a CD.

I'm thinking preloading revBrowserCef into memory to do its dance from there, instead of the CDROM, would be a big help. Is there a way to preload the 95MB stack (standalone), Package Contents Folder or revBrowserCef? (Preloading revBrowserCef via opening in startup just shifts the disturbingly long startup time to the front end. Video file size or HTML page size/type do not change slow load time)

Thanks,
Steve


Update. Get these many times over on initial launch of revBrowserCef (but they come fast at end of loading, so not direct cause of slowdown):

23 times: 3/14/16 10:41:18.000 PM kernel[0]: Process[23473] crashed: revbrowser-cefpr. Too many corpses being created.
5 times: 3/14/16 10:41:18.994 PM ReportCrash[23468]: Saved crash report for revbrowser-cefprocess EH[23470] version 1 to ...
17 times: 3/14/16 10:41:19.009 PM ReportCrash[23468]: Removing excessive log: file:///Users/stevecollins/Library/Logs/DiagnosticReports/revbrowser...

Consol Message Reports:
http://hermosacyclery.com/CEF_First_Lau ... onsole.pdf
http://hermosacyclery.com/CEF_Second_La ... onsole.jpg

[Steve Collins]

Could I use specialFolderPath(“Temporary”) from a standalone to load a second standalone with CEF on board into the hard disc’s temp folder. I would call the second standalone from that path and run from there. The standalone on a hard disc loads the above troubled CEF in 2-3 seconds, plenty fast.

[LCNeil]

Hi Steve,

Sounds like you are hitting a CD medium performance limitation as 70 seconds is a very long time for your application to launch. One thing you may be able to try, is creating zipping up your application and then using a "Launcher" stack to extract the app locally and then launch the app within the extracted directory.

Something like the following should give you some leads with this process-

Code: Select all

on openCard
   unzipApp
end openCard

command unzipApp
   if the environment is "standalone application" then
      
      put (specialFolderPath("Resources")&"test.zip") into tZip
      
      put (specialFolderPath("Desktop")) into tLoc
      
      put shell ("unzip "&tZip&" -d  "&tLoc) into tTemp
      
      put tTemp
      
      launch (specialFolderPath("Desktop")&"/test.app")
      
      close this stack
   end if
end unzipApp

The above will take an included zip file (via standalone settings copy files), extraxt it via shell command, close the stack and the open the extracted app.

I hope this gives you some leads

Kind Regards,

Neil Roger
--
LiveCode Support Team ~ http://www.livecode.com
--

[Steve Collins]

Code: Select all

on openCard
   unzipApp
end openCard

command unzipApp
   if the environment is "standalone application" then
      put (specialFolderPath("Resources")&"test.zip") into tZip
      put (specialFolderPath("Desktop")) into tLoc
      put shell ("unzip "&tZip&" -d  "&tLoc) into tTemp
      put tTemp
      launch (specialFolderPath("Desktop")&"/test.app")
      close this stack
   end if
end unzipApp

That worked great! Needed a few tweaks. Need to add the single quotes around the single line quote marks around the source file to prevent a Zip based error with terminal on Mac. Cut the application launch time (including CEF launch, the biggest part by far) from 70 seconds to a reasonable 27 seconds, from a CD. (Standalone launch without CEF is like 3-5 seconds from CD). Planning to hide the file copied to the hard disc in the Temporary folder instead of Desktop. If anyone sees a red flag here, please do tell. Needs to work on Windows and Mac OS's, as many versions as are readily in use. Still need to test below on PC. Here is what worked for me in development or standalone modes:

Code: Select all

command unzipApp
   put the filename of this stack into tPath
   -- get the file path for this stack
   set the itemdel to slash
   if the platform = "MacOS" and the environment = "standalone application" then
      -- if this is a standalone application running on a Mac, find the path to the .app file, gets rid of extra app name stuff 
      repeat until last item of tPath contains ".app"
         delete last item of tPath
      end repeat
   end if
   -- remove the last item to get the path to the containing folder
   -- delete last item of tPath
   put "test.zip" into last item of tPath
   -- unzip file and place in computer's temporary folder 
   put (specialFolderPath("Temporary")) into tLoc
   put "/" after tLoc
   answer ("unzip "&"'" &tPath&"'" &" -d " &tLoc)
   put shell ("unzip "&"'" &tPath&"'" &" -d " &tLoc) into tTemp
   put tTemp
   launch (specialFolderPath("Temporary")&"/test.app")
end unzipApp

By the way, for those like me, unzipping a file from a source to another location is faster, sometimes strikingly faster, than just copying an already unzipped file.

[Steve Collins]

Does anyone know what to do about spaces in a file path? Above Resources path gets reworked by Terminal in Mac to absolute paths, when building the shell. It can't do that when spaces are in the path. Replacing spaces with "\ " threw off Terminal. It did not know to convert relative path to absolute.

[Klaus]

Hi Steve,

simply (full) quote the pathname"!

Just tested this in the terminal and worked fine: cd "Calibre Library"
Et voila, "ls" showed the content of folder "Calibre Library"


Best

Klaus

[Steve Collins]

simply (full) quote the pathname"!

Best
Klaus

Fixed everything, including the need to have the single quote marks around the path mentioned above.

[Steve Collins]

simply (full) quote the pathname"!
Best
Klaus

Fixed everything, including the need to have the single quote marks around the path mentioned above.

The above script to unzip to a hard disk destination is working great on Mac, but not in Windows. Getting this path (from flash drive), which looks like the Mac one that is working:

unzip E:/Media/Videos/TestFileWin.zip -d C:/Users/Steve Collins/AppData/Local/Temp/

Getting this error message (Windows):

'unzip' is not recognized as an internal or external command operable program or batch file

Read these:
> unzipping in files in Windows from cmd requires 3rd party software such as 7z or winter
> Back in 2013, that was not possible. MS provided no executable for this.

I see LC has a zip resource. I just zipped the files with StuffIt. Is above using LC's unzip feature or the OS's? Maybe I should use revZipExtractItemToFile?

[LCNeil]

Hi Steve,

Yes, you should also be able to use LiveCodes in-built syntax for this purpose. I used the shell command in my initial script as it required only a single line.

LiveCode revZip commands require a few more lines of script but you do have a little bit more direct control over what you can do with the zip file.

A great lesson to get you started can be found here-

http://lessons.livecode.com/m/2592/l/126625-zip-tool

Kind Regards,

Neil Roger
--
LiveCode Support Team ~ http://www.livecode.com
--

[Steve Collins]

Hi Steve,

Yes, you should also be able to use LiveCodes in-built syntax for this purpose. I used the shell command in my initial script as it required only a single line.

LiveCode revZip commands require a few more lines of script but you do have a little bit more direct control over what you can do with the zip file.

A great lesson to get you started can be found here-

http://lessons.livecode.com/m/2592/l/126625-zip-tool

Kind Regards,

Neil Roger
--
LiveCode Support Team ~ http://www.livecode.com
--

Took a while integrate that sample stack scripts into my stack, but now unzips right to where I want and fast. BUT, I get this message from the MacOS when double clicking the app: The application “MyApplication” can’t be opened. Happens with the sample stack ReppizVer.rev, too. If I unzip the application with DropStuff, app works fine.

Checking the Contents folder of the good and bad app, a folder "_MacOS" is missing from Resources folder in the bad app. It has just an empty folder called Externals. Tried moving "_MacOS" to the bad app and no luck. But that is not causing the problem.

Opening the MacOS folder in the Contents and replacing the "MyApplication" file fixes the bad app. Turns out it's supposed to be a Unix executable but when unzipped via LC it becomes a TextEdit Document, according to Get Info.

The sample zip produces a fine app opened with Stiffit Expander. (My DropStuff is set to "deflate" type, so not the type of zip)

[FourthWorld]

Opening the MacOS folder in the Contents and replacing the "MyApplication" file fixes the bad app. Turns out it's supposed to be a Unix executable but when unzipped via LC it becomes a TextEdit Document, according to Get Info.

Not sure why the executable bit is carried over by other tools but not the Zip libs LC uses, but you can call chmod +x on the file using LC's shell function to turn it back into an executable file.

[Steve Collins]

> Yes but you have to know that revZipEnumerateItems only works until LC 6.7.1

Ugh! On Windows, it just returns the one .exe file inside and skips the rest of the other files in the zip file. I suppose I could use revZipEnumerateItems, store the items and unzip based on that, without getting the list each time.

> put shell ("unzip "&tZip&" -d "&tLoc) into tTemp

Is it safe on Mac to use the Terminal script, that works great for me on El Capitan? I could use the revZip feature on PC.

Thanks,
Steve

[jacque]

You mentioned in our private conversation that the delay only happened when playing the first video, which is why you needed a "preload". I also agree that the delay is extended due to loading from CD, which is slow. So you might take a slightly different approach to solve both issues.

Don't include the videos as part of the app, leave them on the CD separately in a folder. That should make your app very much smaller, and if it's small enough, you may be able to do a straight copy to the temp folder fast enough that you don't need to worry about compression. If you want it to copy even faster, go ahead and zip the app on Mac and unzip it to the temp folder as you're doing now, but only for Mac. You might try zipping the Mac app using OS X's native feature rather than StuffIt. Just right-click the app and choose "Compress MyApp" from the menu. That might preserve the executable bit. Or use Neil's Terminal script if that works better, it should be fine on most Macs.

For Windows, you can use LC's compress and decompress commands. Those work with a single file (a Mac app is a bundle of many files, but a Windows app will work.) LC does not need any other software to do compression, it's built-in. To decompress:

Code: Select all

put url ("binfile:"&<path to compressed app>)into tData
 put decompress(tData) into tData
 put tData into url ("binfile:"& <path to destination>)

In addition to the videos you'll leave on the CD, create a very small "prelaunch" video (just a few seconds) and put that on the CD too. After you copy the app to the temp folder, copy the tiny video there as well. Videos don't compress much, so don't bother. Just make it very small and short.

In an openCard handler on the first card of the app stack, open a browser and load it with the prelaunch video from the temp folder. Do not set the rect of the browser; by default, the browser has no rect and so will be invisible. Put up a "loading" message so the user knows something is happening and wait a few seconds. Then delete the browser and leave the user in control. Alternately, and maybe better, you could make the prelaunch video show a logo or welcome message. If you do that, you could set the browser rect so the video would show, and it would serve as a sort of splash screen.

The real videos still reside on the CD, and your app can play them from there. If it's true that only the first browser load is the problem, then they should play smoothly and you'll avoid the huge overhead of copying all those megabytes to the hard disk.

[Steve Collins]

[quote="jacque"]

For Windows, you can use LC's compress and decompress commands. Those work with a single file (a Mac app is a bundle of many files, but a Windows app will work.) LC does not need any other software to do compression, it's built-in. To decompress:

Code: Select all

put url ("binfile:"&<path to compressed app>)into tData
 put decompress(tData) into tData
 put tData into url ("binfile:"& <path to destination>)

Thanks!

Chasing it down, piece by piece. I've tried a lot of variations and seem to be missing something on the unzip on Windows. I used a zip file and got this message in LC: "line 74 (decompress: string is not compressed data), char 1." This is line 74: put decompress(tData) into tData. I tried DropStuff's TAR Gzip and changed the path's file name to .tgz, and no more error. But no file placed in Temp folder. I'm probably missing a dot or a / somewhere. Here are the scripts and paths:

put url ("binfile:"&tPath)into tData
put decompress(tData) into tData
put tData into url ("binfile:"&tLoc)

tPath = E:/Media/Videos/MyApplication.zip (The windows zip file is on a flash drive nested in folders Media>Videos)
tLoc = C:/Users/Steve Collins/AppData/Local/Temp/ (also tried C:/Users/Steve Collins/AppData/Local/Temp)

My LC text file I'm writing with the tLoc path is getting into the Temp folder fine, so I think my "to" path is good. Also, I need to unzip and place the even larger Externals folder that has the CEF contents, many pieces. I'm guessing I need to download the pieces one at a time and not sure that works with this without a repeat that knows the name of all of the pieces.

tried the tgz with a .zip.

Do I need to zip as Gzip with LC to get the file right?

I'm on Windows 10, got machine about 2 months ago.

[jacque]

Sorry, I didn't address the compression part. It's standard gz but to be sure it's compatible with LC, do the compression in LC too. Use LC's "compress" command. It's the same three lines I posted with only the one command changed.

Compress only works with single files. If you need externals or other supporting files you'll need to do each one separately. But I was suggesting you leave the videos on the flash drive and only copy the app itself.