msg box not displaying while running

[adventuresofgreg]

Hi: I'm not sure why, but for some reason the msg box is NOT displaying my messages live while scripts are running. The last message appears in the msg box after the script has completed running.

Is there a way to get long scripts to run in the background while I take control of the stack (like scroll through fields, etc)?

Thanks,
Greg

[Janschenkel]

You could insert the following line in an appropriate place in your tight loop:

Code: Select all

wait 0 milliseconds

to give the LiveCode engine breathing room to display the messages. Or you could even insert the following line in the loop:

Code: Select all

wait 0 milliseconds with messages

to allow user interaction, as this will allow other messages to be handled before continuing with the loop. Of course, if you do that, you should make sure to disable potentially disruptive user interface elements, preventing the user from starting the same job or another interfering job until the big job is finished.

See also this blog post of mine for more information on the wait command.

HTH,

Jan Schenkel.

[dunbarx]

Hmm. Something else is going on. The put command should update the message box without issue. If I write:

on mouseUp
repeat 10

wait 10 --just so we can see them
--wait 0 with messages -- as per Jan's advice

put random(99)
end repeat
end mouseUp

I get those numbers in succession in msg.

Craig Newman

[adventuresofgreg]

Thanks - yes the wait for milliseconds with messages works. However, I am using fields to organize data through the process, and to speed up processing, I don't want the field to refresh, so I use "set the screenlock to true". When the screenlock is set to true, you can use fields instead of variables (processing time is the same) as long as you don't refresh the field contents. The advantage of doing this is to periodically observe how the data is being processed - check for bugs, etc. When I use screenlock, then the message box does not update. Is there a way to separate the msg box from screenlock?

[FourthWorld]

Thanks - yes the wait for milliseconds with messages works. However, I am using fields to organize data through the process, and to speed up processing, I don't want the field to refresh, so I use "set the screenlock to true"...

If you really want to speed up processing do it outside of the field. Get the data, work on it in a variable, then put it back when you're done. Using a variable instead of a field will speed things up so much you may not need to worry about indicating progress at all.

[adventuresofgreg]

Thanks - yes the wait for milliseconds with messages works. However, I am using fields to organize data through the process, and to speed up processing, I don't want the field to refresh, so I use "set the screenlock to true"...

If you really want to speed up processing do it outside of the field. Get the data, work on it in a variable, then put it back when you're done. Using a variable instead of a field will speed things up so much you may not need to worry about indicating progress at all.

Hi Richard:

I don't think that is true. When you use fields WITH screenlock ON, it's just as fast as using variables. I think (correct me if I'm wrong) a field IS just a variable, but with the option of displaying the data. Essentially, turning screenlock ON will suppress the display - no? At least this was the result when I ran some tests - the result was the same for a variable as with a screenlocked field.

[FourthWorld]

If you really want to speed up processing do it outside of the field. Get the data, work on it in a variable, then put it back when you're done. Using a variable instead of a field will speed things up so much you may not need to worry about indicating progress at all.

I don't think that is true. When you use fields WITH screenlock ON, it's just as fast as using variables.

Did you test it?

A field seems simple to us because it's easy to use, but it's easy to use because the inherent complexity of the beast is gracefully hidden away deep in the engine.

This light-hearted metaphoric review of relative field performance may be a fun read:
http://lists.runrev.com/pipermail/use-l ... 57144.html

Try this - Make a stack with one button and one field, and put this in the button script:

Code: Select all

on mouseUp
  -- Run the test this many times:
  put 100 into N
  --
  -- Each test will create 100 lines, counting down
  -- from 100 to 1
  --
  -- Test 1: field
  put the millisecs into t
  repeat n
    lock screen
    put empty into fld 1
    repeat with i = 100 down to 1
      put i & " bottles of beer on the wall" & cr after fld 1
    end repeat
  end repeat
  put the millisecs - t into t1
  --
  -- Test 2: variable
  put the millisecs into t
  repeat n
    put empty into tVar
    repeat with i = 100 down to 1
      put i & " bottles of beer on the wall" & cr after tVar
    end repeat
    put tVar into fld 1
  end repeat
  put the millisecs - t into t2
  --
  -- Results:
  put "Field: "& t1 & "  Var: "& t2
end mouseUp

What results do you get from that?


If you get hooked on benchmarking (there are worse hobbies to have) you might like this:

Benchmarking Performance in LiveCode
http://livecodejournal.com/tutorials/be ... vtalk.html

[adventuresofgreg]

If you really want to speed up processing do it outside of the field. Get the data, work on it in a variable, then put it back when you're done. Using a variable instead of a field will speed things up so much you may not need to worry about indicating progress at all.

I don't think that is true. When you use fields WITH screenlock ON, it's just as fast as using variables.

Did you test it?

A field seems simple to us because it's easy to use, but it's easy to use because the inherent complexity of the beast is gracefully hidden away deep in the engine.

This light-hearted metaphoric review of relative field performance may be a fun read:
http://lists.runrev.com/pipermail/use-l ... 57144.html

Try this - Make a stack with one button and one field, and put this in the button script:

Code: Select all

on mouseUp
  -- Run the test this many times:
  put 100 into N
  --
  -- Each test will create 100 lines, counting down
  -- from 100 to 1
  --
  -- Test 1: field
  put the millisecs into t
  repeat n
    lock screen
    put empty into fld 1
    repeat with i = 100 down to 1
      put i & " bottles of beer on the wall" & cr after fld 1
    end repeat
  end repeat
  put the millisecs - t into t1
  --
  -- Test 2: variable
  put the millisecs into t
  repeat n
    put empty into tVar
    repeat with i = 100 down to 1
      put i & " bottles of beer on the wall" & cr after tVar
    end repeat
    put tVar into fld 1
  end repeat
  put the millisecs - t into t2
  --
  -- Results:
  put "Field: "& t1 & "  Var: "& t2
end mouseUp

What results do you get from that?


If you get hooked on benchmarking (there are worse hobbies to have) you might like this:

Benchmarking Performance in LiveCode
http://livecodejournal.com/tutorials/be ... vtalk.html

Hi: Yes, you are correct. I'm sorry, but I didn't describe it right. What I meant, was that I do not use fields to process data, I always move the data from a field into a variable, do some stuff to it, then move it back to a field. Basically, I use fields as global variables to shuffle data between various processes. I did test this - I wrote a script that places results of processes into fields, and one that places the results into variables and they were the same speed - but only with the screenlock turned off. When LiveCard has to redraw the screen showing the data in the fields, it takes considerably longer.

I'm going to try that test again though... You have me thinking now... And thanks for the links - I'll look them over. I have a script that is currently taking HOURS to run and I'm trying everything I can think of to speed it up.

Cheers,
Greg

[adventuresofgreg]

Richard:

I stand WAY corrected. I read over the benchmarking link that you sent, and set up a few my own tests. In every case, avoiding fields as much as possible was faster - even with Lock Screen.

One question that is still bothering me: why is:

repeat for each line myline in holder

so much faster than:

repeat with i=1 to the number of lines of holder

The reason this bothers me, is that when using the fast method, I have no way of referring to some line offset from the current "myline". I've posted about this before and I'm not certain that I am explaining the issue properly. I'll try again.

To keep it simple, we have a list of 10000 lines of random numbers. Lets say that I want to add the 1st line to the 100th line - for all 10000 lines.

If I use the repeat with i=1 to 10000 method, then all I need to do is add line i to line 1+100, then next i.

If I use the repeat for eachline myline, then I can't refer to "myline + 100". I have to start building a second list where I can refer to line x and line x+100, and that slows the process down tremendously.

Thanks
Greg

[FourthWorld]

I stand WAY corrected. I read over the benchmarking link that you sent, and set up a few my own tests. In every case, avoiding fields as much as possible was faster - even with Lock Screen.

I've been using this family of languages for more than 20 years, and in the early days many xTalk dialects were quite slow so I took up the habit of testing various ways to do things to see which was faster. Sometimes the difference is trivial, but when it pays off it can pay off big, freeing up enough clock cycles to add new features like real-time updates.

LiveCode is so fast that many benchmarks don't matter much, but sometimes I find surprising things that make it worth taking a minute or so to test them out.

Glad that was helpful for you.

One question that is still bothering me: why is:

repeat for each line myline in holder

so much faster than:

repeat with i=1 to the number of lines of holder

The reason this bothers me, is that when using the fast method, I have no way of referring to some line offset from the current "myline".

Let me first explain what's happening under the hood so you can better appreciate the speed difference, and then I'll offer a solution for your counter that will hopefully be just what you need while still taking advantage of the speed boost:

When you use this:

Code: Select all

repeat with i = 1 to the number of lines of tData
  DoSomethingWith line i of tData
end repeat

…here's what the engine needs to do each time through the loop:

1. That repeat statement requires the engine to walk through every character in tData, counting the return characters and adding them up, while incrementing the integer in i. After all, the engine has no way to know if tData was changed within the loop, so reexamining it in full is required for every iteration.

2. The second line requires the engine to repeat most of the steps it just did in the repeat statement before it, examining every character and counting returns until it reaches the number of return characters in i.


In contrast, when you use this:

Code: Select all

repeat for each line tLine in tData
  DoSomethingWith tLine
end repeat

…the engine is only counting return characters once for the entire duration of the process:

1. Since this repeat form allows the engine to assume that the data in tData will not be changed in the loop (indeed, it won't even allow changing tData), it makes one pass through the data, examing every character but stopping when it reaches the next return, putting everything from the previous return to the current one into tLine, so that:

2. tLine is already parsed out and ready to use, with no further traversing of tData needed.


If you need a counter in a "repeat for each" loop, you can do this:

Code: Select all

put 0 into i
repeat for each line tLine in tData
   add 1 to i
   DoSomethingWith tLine, i
end repeat

In that setup i can be used to increment a progress indicator or do other things that depend on knowing which line you're currently working with, at the low cost of incrementing an integer so it has no noticeable effect on performance.

If you need to alter the data in tData, you'll find that "put…after" has been well optimized since LiveCode 1.5, so you can rebuild lists on the fly very quickly - a simple example:

Code: Select all

put empty into tNuData
put 0 into i
repeat for each line tLine in tData
  add 1 to i
  put "This is line # " & i &tab& tLine &cr after tNuData
end repeat
delete last char of tNuData -- trailing cr
return tNuData

In your specific scenario, however, I would consider a completely different approach since you need to be able to operate on multiple lines in each iteration:

To keep it simple, we have a list of 10000 lines of random numbers. Lets say that I want to add the 1st line to the 100th line - for all 10000 lines.

If I use the repeat with i=1 to 10000 method, then all I need to do is add line i to line 1+100, then next i.

If I use the repeat for eachline myline, then I can't refer to "myline + 100". I have to start building a second list where I can refer to line x and line x+100, and that slows the process down tremendously.

This seems like an excellent place to use an array for part of that.

Using "repeat for each" will parse data efficiently, sometimes as fast or even faster than some array methods, but is ideally suited for those cases where you'll be working with only one line at a time.

If you need to traverse the data rapidly to various locations, arrays will be hard to beat. While lists must be parsed to count return characters to find a given line, an array uses a more sophisticated hashing scheme to point to slots where the data can be found much more quickly - often by orders of magnitude if you need to jump around through the data frequently.

Arrays are able to do this for the modest cost of a one-time parsing that's done within the engine's well-optimized C++ code using the "split" command.

That said, it occurs to me that that the "split" command can sometimes be a relatively expensive operation, in that it effectively has to parse the entire chunk looking for all returns in order to put each line into the various array elemens.

This leaves the performance-obsessive scripter with a question: Would it be faster to load the array incrementally using the values already parsed in each line?

Being the benchmarking fan that I am, I figured it would be worth a couple minutes to try both methods to see how they compare:

Code: Select all

on mouseUp
  put fld 1 into tData
  --
  -- Test 1: parse up front with split
  --
  put the millisecs into t
  put empty into tNuData1
  put tData into tParsedValuesA
  split tParsedValuesA by cr
  --
  put 0 into i
  repeat for each line tLine in tData
    add 1 to i
    put tLine + tParsedValuesA[i-100] &cr after tNuData1
  end repeat
  delete last char of tNuData1 -- trailing cr
  put the millisecs - t into t1
  --
  -- Test 2: parse on the fly
  --
  put the millisecs into t
  put empty into tNuData2
  put empty into tParsedValuesA
  --
  put 0 into i
  repeat for each line tLine in tData
    add 1 to i
    put tLine into tParsedValuesA[i]
    put tLine + tParsedValuesA[i-100] &cr after tNuData2
  end repeat
  delete last char of tNuData2 -- trailing cr
  put the millisec - t into t2
  --
  -- Show results:
  put "Split : "& t1 &"   On-the-fly: "& t2 &cr&\
      "Results match: "& (tNuData1 = tNuData2)
  -- The last line is just a sanity check to make sure
  -- both methods produce the same result.
end mouseUp

That test shows that even with the expense of using split, it's still faster than loading each array element individually as tData is parsed, by about 30%.

There are probably many ways to speed it up even more, and with any luck Bernd will post a stack illustrating how to do it.

But for an off-the-cuff rough draft it's not bad, parsing 10,000 lines and adding the value of every hundredth line in 43 milliseconds on my 2.6 GHz Mac.

You can probably speed it up even more, but even this rough algo is a few orders of magnitude faster than diving into the field for each operation.

[adventuresofgreg]

Hi Richard:

Thanks for your help. I looked back at my script, and I actually did end up figuring out how to use both "repeat for each line thisline" and taking chunks and forming a new i=1 to whatever list - and yes, it works way faster.

I didn't think of using an array - I'll give that a shot. I don't use arrays much, so I'll have to experiment.

I have a field with 20,000 lines - each line has 31 "," separated numbers

I need to run some math on a moving window of a certain item # of each line (window size changes, and math functions are average, max, min, st dev, etc)

Currently, for each line (using "repeat for each line thisline"), I advance a counter and collect values from x number of previous lines and put them into my moving window. Then on each new line, I delete the first value and add the new value (thereby avoiding having to reassemble the window after every new line).

How would do accomplish this using an array? If I stick every item of every line into an array, is there an easy way to, say compute an average for item 3 of each line for lines 10000 to 15000 using arrays?

[FourthWorld]

Take a look at the script I provided which tests two different array-based methods. The first one is slightly faster, and arguably a little simpler. It should be reasonably straightforward to modify it to parse the items in each line. Since you only have 31 items in each line you could probably walk through them with "repeat for each item..." quickly enough.

And of course if you get stuck we're always here.