Livecode execution speed problem
Moderators: FourthWorld, heatherlaine, Klaus, robinmiller
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Livecode execution speed problem
I am looking for some pointers of how to identify the portions of my Livecode are running slowly. I have just rebuilt a small application that I first wrote in RealBasic. the application processes a binary file decoding certain elements. The decoded elements are written out to a text file. The RealBasic application running in the IDE processes a 70 Mbyte binary file in 38 seconds. My Livecode version is very slow taking 30 seconds to process 10000 bytes (thats 58 hours to process the complete file!). In the past I have found Livecode to be as fast as Realbasic so I hope I can find some way of speeding things up.
best wishes
Simon
best wishes
Simon
best wishes
Skids
Skids
-
- VIP Livecode Opensource Backer
- Posts: 3999
- Joined: Sun Jan 07, 2007 9:12 pm
- Location: Bochum, Germany
Re: Livecode execution speed problem
Hi Simon,
it is very difficult to give advice in a generic manner. It all depends on the code. If you want you could send me a sample stack and a sample file (not all of the 70 MB) to look at. No promises as to what improvements could result.
Kind regards
Bernd
it is very difficult to give advice in a generic manner. It all depends on the code. If you want you could send me a sample stack and a sample file (not all of the 70 MB) to look at. No promises as to what improvements could result.
Kind regards
Bernd
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Livecode execution speed problem
Hi Bernd,
Thanks for your kind offer - I will send you a stack and a small data file. I have been running some tests by calling routines to discover the major bottle neck and have discovered that the routine that I use to extract a 16 bit word is taking 134ms to come up with a result. I should say that I am loading the whole data file into a variable and then reading single characters in with put byte n of theBinaryData into LSB. This method is extremely slow.
best wishes
Simon
Thanks for your kind offer - I will send you a stack and a small data file. I have been running some tests by calling routines to discover the major bottle neck and have discovered that the routine that I use to extract a 16 bit word is taking 134ms to come up with a result. I should say that I am loading the whole data file into a variable and then reading single characters in with put byte n of theBinaryData into LSB. This method is extremely slow.
best wishes
Simon
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Livecode execution speed problem
Thanks to Bernd the estimated 56 hours for processing a datafile is down to a real 70 seconds. The main speed problem was caused by passing a copy of the complete data variable to a function that was being called over a million times. Livecode just choked. The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.
There are a few more possible gains to be had by removing calls to functions and placing the function code inside the main loop which I will try in the near future.
Thanks again Bernd,
Simon
Code: Select all
private function ExtractDataword @theBinaryData, BytePTR
Thanks again Bernd,
Simon
best wishes
Skids
Skids
-
- VIP Livecode Opensource Backer
- Posts: 9823
- Joined: Sat Apr 08, 2006 7:05 am
- Location: Los Angeles
- Contact:
Re: Livecode execution speed problem
If you have any "repeat with" statements try using "repeat for each" and see if that doesn't get you an order of magnitude performance boost.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Livecode execution speed problem
Dear Richard,
Thanks for the tip. My code does not use any repeat structures as they are quite slow. At present I am attempting to reduce the number of calls to the main data variable; I need two bytes but typically they are in LSB MSB in the file and I would rather have them MSB LSB. If I read in two bytes at a time using I save 3 second processing but I will have to change the bit extraction routines to accommodate the LSB MSB structure.
best wishes
Simon
Thanks for the tip. My code does not use any repeat structures as they are quite slow. At present I am attempting to reduce the number of calls to the main data variable; I need two bytes but typically they are in LSB MSB in the file and I would rather have them MSB LSB. If I read in two bytes at a time using
Code: Select all
put char 12 to 13 of theData into theDataWord
best wishes
Simon
best wishes
Skids
Skids
Re: Livecode execution speed problem
Could you elaborate ? I never heard about this "@" tricK.Simon Knight wrote: The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.
Simon
The gain of time seems, in your case, totally huge !
-
- VIP Livecode Opensource Backer
- Posts: 1236
- Joined: Sat Apr 08, 2006 1:10 pm
- Location: Zurich
- Contact:
Re: Livecode execution speed problem
In general "repeat for each <chunk>" is faster then @, tho depending on task it can reverse.
bangkok: read the docu entry.
bangkok: read the docu entry.
Various teststacks and stuff:
http://bjoernke.com
Chat with other RunRev developers:
chat.freenode.net:6666 #livecode
http://bjoernke.com
Chat with other RunRev developers:
chat.freenode.net:6666 #livecode
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Livecode execution speed problem
Hi, I probably did not explain myself very well, I hope that this explanation makes things clearer.
My code loads a large data file into a variable. The main processing loop needs to step through the data in 66 byte steps. It calculates the next byte position using last byte position plus 66 and then passes the number along with the data variable to a function that extracts two bytes from the requested position. Livecode defaults to passing a copy of any variable defined as a parameter. In my code the passing of such a large variable (circa 70 Mbye) several million times caused it to run very slowly. However if when defining the parameters of a function the parameter name in prefixed with the @ character, Livecode sends a reference to the variable rather than a copy. Using a referenced parameter variable in this way reduced the execution time of my code from many hours down to 70 seconds.
Simon
My code loads a large data file into a variable. The main processing loop needs to step through the data in 66 byte steps. It calculates the next byte position using last byte position plus 66 and then passes the number along with the data variable to a function that extracts two bytes from the requested position. Livecode defaults to passing a copy of any variable defined as a parameter. In my code the passing of such a large variable (circa 70 Mbye) several million times caused it to run very slowly. However if when defining the parameters of a function the parameter name in prefixed with the @ character, Livecode sends a reference to the variable rather than a copy. Using a referenced parameter variable in this way reduced the execution time of my code from many hours down to 70 seconds.
Simon
best wishes
Skids
Skids
-
- VIP Livecode Opensource Backer
- Posts: 1236
- Joined: Sat Apr 08, 2006 1:10 pm
- Location: Zurich
- Contact:
Re: Livecode execution speed problem
right but it would be even faster to do it this way (you can of course combine them, using a function with @ within the loop) The reason for this to be _vastly_ speedier then using "repeat with x = blah", is that you don't need to seek trough the file every repeat, but let LC handle that for you.
Code: Select all
on mouseUp
-set up
local decodedBinary
--load file
put url "some/path/that/is/correct" into theData
repeat for each char theChar in theData
add one to theSixties
put theChar after processMe
if theSixties mod 60 = 0 then
-- do handling on processMe here, for example:
put binarydecode("H*",processMe,decodedBinary) into garbageBin
put decodedBinary & return after theResult
--end of example handling
put "" into processMe
end if
end repeat
end mouseUp
Various teststacks and stuff:
http://bjoernke.com
Chat with other RunRev developers:
chat.freenode.net:6666 #livecode
http://bjoernke.com
Chat with other RunRev developers:
chat.freenode.net:6666 #livecode
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Livecode execution speed problem
Hi,
Thanks for the code and method, I will give it a try with my test file. Bernd did a version that runs at the best speed with a time of 35852ms with everything in just a single routine. I have code that uses functions and processes the same data in 48202ms. I will post the speed that a version running your code achieves.
best wishes
Simon
Thanks for the code and method, I will give it a try with my test file. Bernd did a version that runs at the best speed with a time of 35852ms with everything in just a single routine. I have code that uses functions and processes the same data in 48202ms. I will post the speed that a version running your code achieves.
best wishes
Simon
best wishes
Skids
Skids
-
- VIP Livecode Opensource Backer
- Posts: 9648
- Joined: Wed May 06, 2009 2:28 pm
- Location: New York, NY
Re: Livecode execution speed problem
Bangkok.
Read the dictionary under "@". This symbol determines whether a parameter is passed by value or by reference.
I have a field "field44" with the number 5 in it. Put this in a button script:
Comment out one or the other of the update handlers. In one you will see that a 5 is returned, and in the other a 6.
Craig Newman
Read the dictionary under "@". This symbol determines whether a parameter is passed by value or by reference.
I have a field "field44" with the number 5 in it. Put this in a button script:
Code: Select all
on mouseUp
put fld "field44" into tParam
update tParam
answer tParam && "by normal"
end mouseUp
on update var --NORMAL
add 1 to var
end update
on update @var --BY REFERENCE
add 1 to var
end update
Craig Newman