Livecode execution speed problem

[Simon Knight]

I am looking for some pointers of how to identify the portions of my Livecode are running slowly. I have just rebuilt a small application that I first wrote in RealBasic. the application processes a binary file decoding certain elements. The decoded elements are written out to a text file. The RealBasic application running in the IDE processes a 70 Mbyte binary file in 38 seconds. My Livecode version is very slow taking 30 seconds to process 10000 bytes (thats 58 hours to process the complete file!). In the past I have found Livecode to be as fast as Realbasic so I hope I can find some way of speeding things up.

best wishes
Simon

[bn]

Hi Simon,

it is very difficult to give advice in a generic manner. It all depends on the code. If you want you could send me a sample stack and a sample file (not all of the 70 MB) to look at. No promises as to what improvements could result.

Kind regards

Bernd

[Simon Knight]

Hi Bernd,

Thanks for your kind offer - I will send you a stack and a small data file. I have been running some tests by calling routines to discover the major bottle neck and have discovered that the routine that I use to extract a 16 bit word is taking 134ms to come up with a result. I should say that I am loading the whole data file into a variable and then reading single characters in with put byte n of theBinaryData into LSB. This method is extremely slow.

best wishes

Simon

[Simon Knight]

Thanks to Bernd the estimated 56 hours for processing a datafile is down to a real 70 seconds. The main speed problem was caused by passing a copy of the complete data variable to a function that was being called over a million times. Livecode just choked. The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.

Code: Select all

private function ExtractDataword @theBinaryData, BytePTR

There are a few more possible gains to be had by removing calls to functions and placing the function code inside the main loop which I will try in the near future.

Thanks again Bernd,

Simon

[FourthWorld]

If you have any "repeat with" statements try using "repeat for each" and see if that doesn't get you an order of magnitude performance boost.

[Simon Knight]

Dear Richard,
Thanks for the tip. My code does not use any repeat structures as they are quite slow. At present I am attempting to reduce the number of calls to the main data variable; I need two bytes but typically they are in LSB MSB in the file and I would rather have them MSB LSB. If I read in two bytes at a time using

Code: Select all

put char 12 to 13 of theData into theDataWord

I save 3 second processing but I will have to change the bit extraction routines to accommodate the LSB MSB structure.

best wishes
Simon

[bangkok]

The solution was to add the @ character before the parameter name in the function declaration, this makes livecode pass a reference to the data rather than the default of sending a copy. the data is stored in theBinarydata e.g.
Simon

Could you elaborate ? I never heard about this "@" tricK.

The gain of time seems, in your case, totally huge !

[BvG]

In general "repeat for each <chunk>" is faster then @, tho depending on task it can reverse.

bangkok: read the docu entry.

[Simon Knight]

Hi, I probably did not explain myself very well, I hope that this explanation makes things clearer.

My code loads a large data file into a variable. The main processing loop needs to step through the data in 66 byte steps. It calculates the next byte position using last byte position plus 66 and then passes the number along with the data variable to a function that extracts two bytes from the requested position. Livecode defaults to passing a copy of any variable defined as a parameter. In my code the passing of such a large variable (circa 70 Mbye) several million times caused it to run very slowly. However if when defining the parameters of a function the parameter name in prefixed with the @ character, Livecode sends a reference to the variable rather than a copy. Using a referenced parameter variable in this way reduced the execution time of my code from many hours down to 70 seconds.

Simon

[BvG]

right but it would be even faster to do it this way (you can of course combine them, using a function with @ within the loop) The reason for this to be _vastly_ speedier then using "repeat with x = blah", is that you don't need to seek trough the file every repeat, but let LC handle that for you.

Code: Select all

on mouseUp
  -set up
  local decodedBinary
  --load file
  put url "some/path/that/is/correct" into theData
  repeat for each char theChar in theData
    add one to theSixties
    put theChar after processMe
    if theSixties mod 60 = 0 then
      -- do handling on processMe here, for example:
      put binarydecode("H*",processMe,decodedBinary) into garbageBin
      put decodedBinary & return after theResult
      --end of example handling
      put "" into processMe
    end if
  end repeat
end mouseUp

[Simon Knight]

Hi,
Thanks for the code and method, I will give it a try with my test file. Bernd did a version that runs at the best speed with a time of 35852ms with everything in just a single routine. I have code that uses functions and processes the same data in 48202ms. I will post the speed that a version running your code achieves.

best wishes
Simon

[dunbarx]

Bangkok.

Read the dictionary under "@". This symbol determines whether a parameter is passed by value or by reference.

I have a field "field44" with the number 5 in it. Put this in a button script:

Code: Select all

on mouseUp
   put fld "field44" into tParam
   update tParam
   answer tParam && "by normal"
end mouseUp

on update var --NORMAL
   add 1 to var
end update

on update @var --BY REFERENCE
   add 1 to var
end update

Comment out one or the other of the update handlers. In one you will see that a 5 is returned, and in the other a 6.

Craig Newman