Livecode execution speed problem

Want to move your code and projects to LiveCode but don't know where to start?

Moderators: Klaus, FourthWorld, heatherlaine, robinmiller

Post Reply
Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Livecode execution speed problem

Post by Simon Knight » Sun Mar 06, 2011 12:39 pm

I am looking for some pointers of how to identify the portions of my Livecode are running slowly. I have just rebuilt a small application that I first wrote in RealBasic. the application processes a binary file decoding certain elements. The decoded elements are written out to a text file. The RealBasic application running in the IDE processes a 70 Mbyte binary file in 38 seconds. My Livecode version is very slow taking 30 seconds to process 10000 bytes (thats 58 hours to process the complete file!). In the past I have found Livecode to be as fast as Realbasic so I hope I can find some way of speeding things up.

best wishes
Simon
best wishes
Skids

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3324
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: Livecode execution speed problem

Post by bn » Sun Mar 06, 2011 1:22 pm

Hi Simon,

it is very difficult to give advice in a generic manner. It all depends on the code. If you want you could send me a sample stack and a sample file (not all of the 70 MB) to look at. No promises as to what improvements could result.

Kind regards

Bernd

Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Livecode execution speed problem

Post by Simon Knight » Sun Mar 06, 2011 1:33 pm

Hi Bernd,

Thanks for your kind offer - I will send you a stack and a small data file. I have been running some tests by calling routines to discover the major bottle neck and have discovered that the routine that I use to extract a 16 bit word is taking 134ms to come up with a result. I should say that I am loading the whole data file into a variable and then reading single characters in with put byte n of theBinaryData into LSB. This method is extremely slow.

best wishes

Simon
best wishes
Skids

Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Livecode execution speed problem

Post by Simon Knight » Sun Mar 06, 2011 9:53 pm

Thanks to Bernd the estimated 56 hours for processing a datafile is down to a real 70 seconds. The main speed problem was caused by passing a copy of the complete data variable to a function that was being called over a million times. Livecode just choked. The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.

Code: Select all

private function ExtractDataword @theBinaryData, BytePTR
There are a few more possible gains to be had by removing calls to functions and placing the function code inside the main loop which I will try in the near future.

Thanks again Bernd,

Simon
best wishes
Skids

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7037
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Livecode execution speed problem

Post by FourthWorld » Mon Mar 07, 2011 1:15 am

If you have any "repeat with" statements try using "repeat for each" and see if that doesn't get you an order of magnitude performance boost.
Richard Gaskin
Community volunteer LiveCode Community Liaison

LiveCode development, training, and consulting services: Fourth World Systems: http://FourthWorld.com
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Livecode execution speed problem

Post by Simon Knight » Mon Mar 07, 2011 12:09 pm

Dear Richard,
Thanks for the tip. My code does not use any repeat structures as they are quite slow. At present I am attempting to reduce the number of calls to the main data variable; I need two bytes but typically they are in LSB MSB in the file and I would rather have them MSB LSB. If I read in two bytes at a time using

Code: Select all

put char 12 to 13 of theData into theDataWord
I save 3 second processing but I will have to change the bit extraction routines to accommodate the LSB MSB structure.

best wishes
Simon
best wishes
Skids

bangkok
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 837
Joined: Fri Aug 15, 2008 7:15 am

Re: Livecode execution speed problem

Post by bangkok » Mon Mar 07, 2011 2:53 pm

Simon Knight wrote: The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.
Simon
Could you elaborate ? I never heard about this "@" tricK.

The gain of time seems, in your case, totally huge !

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Livecode execution speed problem

Post by BvG » Mon Mar 07, 2011 3:27 pm

In general "repeat for each <chunk>" is faster then @, tho depending on task it can reverse.

bangkok: read the docu entry.
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Livecode execution speed problem

Post by Simon Knight » Mon Mar 07, 2011 4:19 pm

Hi, I probably did not explain myself very well, I hope that this explanation makes things clearer.

My code loads a large data file into a variable. The main processing loop needs to step through the data in 66 byte steps. It calculates the next byte position using last byte position plus 66 and then passes the number along with the data variable to a function that extracts two bytes from the requested position. Livecode defaults to passing a copy of any variable defined as a parameter. In my code the passing of such a large variable (circa 70 Mbye) several million times caused it to run very slowly. However if when defining the parameters of a function the parameter name in prefixed with the @ character, Livecode sends a reference to the variable rather than a copy. Using a referenced parameter variable in this way reduced the execution time of my code from many hours down to 70 seconds.

Simon
best wishes
Skids

BvG
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1236
Joined: Sat Apr 08, 2006 1:10 pm
Location: Zurich
Contact:

Re: Livecode execution speed problem

Post by BvG » Mon Mar 07, 2011 8:14 pm

right but it would be even faster to do it this way (you can of course combine them, using a function with @ within the loop) The reason for this to be _vastly_ speedier then using "repeat with x = blah", is that you don't need to seek trough the file every repeat, but let LC handle that for you.

Code: Select all

on mouseUp
  -set up
  local decodedBinary
  --load file
  put url "some/path/that/is/correct" into theData
  repeat for each char theChar in theData
    add one to theSixties
    put theChar after processMe
    if theSixties mod 60 = 0 then
      -- do handling on processMe here, for example:
      put binarydecode("H*",processMe,decodedBinary) into garbageBin
      put decodedBinary & return after theResult
      --end of example handling
      put "" into processMe
    end if
  end repeat
end mouseUp
Various teststacks and stuff:
http://bjoernke.com

Chat with other RunRev developers:
chat.freenode.net:6666 #livecode

Simon Knight
Posts: 478
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Livecode execution speed problem

Post by Simon Knight » Mon Mar 07, 2011 8:22 pm

Hi,
Thanks for the code and method, I will give it a try with my test file. Bernd did a version that runs at the best speed with a time of 35852ms with everything in just a single routine. I have code that uses functions and processes the same data in 48202ms. I will post the speed that a version running your code achieves.

best wishes
Simon
best wishes
Skids

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 6265
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Livecode execution speed problem

Post by dunbarx » Mon Mar 07, 2011 9:53 pm

Bangkok.

Read the dictionary under "@". This symbol determines whether a parameter is passed by value or by reference.

I have a field "field44" with the number 5 in it. Put this in a button script:

Code: Select all

on mouseUp
   put fld "field44" into tParam
   update tParam
   answer tParam && "by normal"
end mouseUp

on update var --NORMAL
   add 1 to var
end update

on update @var --BY REFERENCE
   add 1 to var
end update
Comment out one or the other of the update handlers. In one you will see that a 5 is returned, and in the other a 6.

Craig Newman

Post Reply

Return to “Converting to LiveCode”