Performance drop LC 6.1 --> LC 7.1

[okk]

Hi all,
I wrote a while ago a subtitling/player software in livecode 6.1.2 for creating karaoke versions of famous speeches. (here is a link to the project: http://www.speechkaraoke.org )

I rewrote the software with LC 6.5. but wanted now to update the software to LC 7.1. to work with the new Unicode features and leave QuickTime behind. Originally I used the Enhanced QuickTime External, but have now disabled it.

I noticed a massive performance drop going from LC 6.5. to LC 7.1. It's hard to describe it so I made a screen recording: https://vimeo.com/147021976
In the Karaoke software each word of the speech has a timecode-start and a timecode-end. When the playhead reaches the timecode-start of a word the program is coloring the first letter of that word, waits a short while, colors the next letter and so on. The wait time is calculated in a way that the word is completely colored when the playhead reaches the timecode-end. Now in LC 6.5. this works pretty well as you can see in the video. In LC 7.1 (I also tested with LC 6.7) the same routine is extremely much slower, so that everything goes out of sync. I have no explanation for this. It is the same code, on the same system. Some enlightenment would be appreciated.

Out of curiosity I made a short test running following code in a stack:

Code: Select all

on mouseUp
   put the milliseconds into tcstart
   repeat with i=1 to the number of words of field "subtitle"
      repeat with k=1 to the number of chars of word i of field "subtitle"
         set the foregroundcolor of char k of word i of field "subtitle" to 200,0,0
      end repeat
   end repeat
   put the milliseconds - tcstart into field "time"
end mouseUp

Field "subtitle" contains 3 words with 13 chars each. In LC 6.1.2 this takes about 25 milliseconds, In LC 6.5.2 this takes about 70 milliseconds on my system. In LC 7.1. it takes about 650 milliseconds!!

Thanks for any insights! Oliver

[FourthWorld]

I would be interested in knowing how long it takes in the latest build, v8.0 dp10.

[bn]

Hi Oliver,

your speed comparisons do not take into account that in LC versions prior to 6.7.x and up on a Mac whatever you do to the field in a repeat loop will not update the screen.
No wonder the speed with your test script is so much faster in LC versions before 6.7.x.
If you add a line "wait 0 milliseconds with messages" in your handler in before 6.7.x you will see around 330 milliseconds for your handler becaus wait with messages forces a screen update.
In LC >= 6.7.x with the the line "wait with messages" it takes roughly 670 milliseconds for the same handler.

Obviously the architectural changes with the advent of cocoa have slowed down screen updates. Even when you put 50 times empty into a field it take around 800 milliseconds.

The time lag you see is almost entirely due to screen update time, and not caused by your script. And the improved "repeat with i = " in LC8 dp10 does not speed your script up.

As seen in the video the slower screen update messes your karaoke up (by the way I liked the idea and thought it was very funny to let people make a "famous speech karaoke").

Apparently it is slower in your video then just double time. But then there is a lot more going on at the same time as seen in the video.

Maybe if you just color entire words instead of each character it would be fast enough?

Kind regards
Bernd

[Martin Koob]

Do you have alwaysBuffer set to true on your player?

I found that in 6.7 with the new AVFoundation based player setting alwaysBuffer to true can result in choppy playback.

I have a bug report submitted.

http://quality.livecode.com/show_bug.cgi?id=12811

Martin

[bn]

Hi Oliver,

just played around with your script:

maybe if you colorize 2 chars instead of 1 at a time it would be fast enough. This effectively cuts down the time it takes by half

Code: Select all

on mouseUp
   put the milliseconds into tcstart
   repeat with i=1 to the number of words of field "subtitle"
      repeat with k=2 to the number of chars of word i of field "subtitle" step 2
         set the foregroundcolor of char k - 1 to k of word i of field "subtitle" to 200,0,0
      end repeat
   end repeat
   put the milliseconds - tcstart into field "time"
end mouseUp

if that is not fast enough you could colorize 3 chars at a time

Code: Select all

 repeat with k=3 to the number of chars of word i of field "subtitle" step 3
    set the foregroundcolor of char k - 2 to k of word i of field "subtitle" to 200,0,0
 end repeat

Kind regards
Bernd

[FourthWorld]

Bernd's observation about lockScreen is key. A long-standing bug which made OS X apps behave differently from running the same app on other platforms was fixed with the Cocoa migration in v6.7. Prior to that, even if you didn't set lockScreen to true the window wouldn't update until it hit idle, preventing screen updates as they would appear on other platforms.

LiveCode provides explicit control over showing or not showing screen updates: setting the lockScreen global property to true will prevent redraws until idle.

So for the purposes of accurate testing, it would be helpful to add "set the lockscreen to true" before the loop and then run the stack in both 6.5 and a later version.

To get the best measure of the current state of the engine it would be best to use v8.0 DP10 for that test, as there have also been optimizations to some chunk expressions which bring performance back to v6 levels in some circumstances.

Once that comparison test has been run please share the results here so we can assess how well the optimization efforts have been doing for v8.

After that, finding alternative means of speeding that up will increase performance even more. In addition to the good alternatives discusses here thus far, another option is to update the field object only once by adding the formatting in htmlText.

In general, field operations are relatively slow compared to performing the same operations on the text outside of the field. So whenever you have a time-consuming operation to run on styled text in a field, if it's possible to get the htmlText, modify that, and then set the htmlText of the field to the modified string you'll usually see much faster throughput, since it only needs to update the complex field structures once.

[bn]

Hi Richard,

In addition to the good alternatives discusses here thus far, another option is to update the field object only once by adding the formatting in htmlText.

With what Oliver tries to do neither taking the field into a local variable nor setting the the htmlText of the field has a meaningful impact on speed in his special case. Nor setting the styledText.
I have tried all of that, also for LC8 DP10, no joy.

Even putting 50 times "" into a field takes 830 milliseconds in LC >=6.7. Oliver's simple task does not really add to that. A simple screen update is about 16 ms in LC >= 6.7, and since he colors individual chars he has a screen update on each char. In his special case the rest is trivial

Kind regards
Bernd

[FourthWorld]

Clearly there are nuances of this use-case I don't understand. The outcome appears to me to be that the entire text of the field is turned to red, offering many different ways to attain that, including these three:

Code: Select all

on mouseUp
   -- This test uses the text of JFK's inaugural address in fld "orig" from:
   -- http://www.americanrhetoric.com/speeches/jfkinaugural.htm
   --
   -- test 1: okk code:
   put fld "orig" into fld "subtitle"
   put the milliseconds into tcstart
   repeat with i=1 to the number of words of field "subtitle"
      repeat with k=1 to the number of chars of word i of field "subtitle"
         set the foregroundcolor of char k of word i of field "subtitle" to 200,0,0
      end repeat
   end repeat
   put the milliseconds - tcstart into t1
   --
   -- test 2: okk code with lockscreen:
   put fld "orig" into fld "subtitle"
   put the milliseconds into tcstart
   lock screen
   repeat with i=1 to the number of words of field "subtitle"
      repeat with k=1 to the number of chars of word i of field "subtitle"
         set the foregroundcolor of char k of word i of field "subtitle" to 200,0,0
      end repeat
   end repeat
   unlock screen
   put the milliseconds - tcstart into t2
   --
   -- test 3: set color of all words at once:
   put fld "orig" into fld "subtitle"
   put the milliseconds into tcstart
   put fld "orig" into fld "subtitle"
   set the foregroundcolor of word 1 to -1 of field "subtitle" to 200,0,0
   put the milliseconds - tcstart into t3
   --
   -- display timing results:
   put t1 &cr& t2 &cr& t3 into field "time"
end mouseUp

...for which I get in fld "time":
4736
1074
24

[bn]

Hi Richard,

I agree completely with your points and think your code samples illustrate very well the speed differences one can see using different approaches.

My goal was to steer Oliver to a solution of his problem which I deduced from the short video (about 40 seconds) he linked to

I noticed a massive performance drop going from LC 6.5. to LC 7.1. It's hard to describe it so I made a screen recording: https://vimeo.com/147021976

As far as performance of the LC >= 6.7 series is concerned: Performance has increased a lot and is still increasing, very notably for chunk repeats e.g. (repeat with i = 1 to the number of items) which has been implemented in LC8 DP10.

And I believe performance of LC >= 6.7 is in real applications is by now almost as good as before. Some small changes to the code might be needed. To take advantage of the continuing efforts to optimize speed it is important to test against the latest versions.

Now that we have spend some time on this you might as well have a look at the short video to see why our angle on this thread differed.

Kind regards
Bernd

[FourthWorld]

Unless the code he's using in the app is very different from what he posted here, it would seem to be very dependent on not just a specific version of LC but also a specific version of the OS and the specifics of the hardware it's running on. Upgrade either the OS or the CPU, or even just have an audio segment with longer spans of silence between words, and the result of an algorithm which only colors text at maximum speed will no longer match the audio, since the audio is maintained with a constant playback rate.

This seems like a good case for callbacks, no?

[bn]

This seems like a good case for callbacks, no?

callbacks are extremely useful when working with players. I don't know the original poster's setup and requirements. It seems to be some sort of art project where people perform a karaoke of a famous speech. So I think he is in control of the hardware and software and can adapt without providing a generalized solution

One thing about callbacks: in some recent versions they work and in some they don't.

Just tried in 7.1.1 RC4 (they work) and LC 8.0 DP10 (they don't work)

But Martin Koop who has contributed to this thread has more to say on this. Check for his contributions to the Quality Center how he contributed well, to the quality of the new player and AV-foundation etc.

Kind regards
Bernd

[Martin Koob]

Hi Bernd

I hadn't realized that callbacks were not working in LC8 DP 10.

They were working at the Global Jam. I just checked where the issue started. With DP 6 and DP 7 player callbacks work. They stop working in DP 8 and don't work in DP 10.

I haven't used LC 8 since the Global Jam so missed that regression. I have posted a bug report.

http://quality.livecode.com/show_bug.cgi?id=16515

Thanks Bernd for the heads up on this one.

Martin

[okk]

Hi all, thanks a lot for your insights and support. The script I posted was just a small test to find out, if the colouring of single chars takes really that much longer in LC >= 6.7. I added a "wait 0 milliseconds" to the command, to make sure the screen is drawn after each char is coloured. In LC 6.1. the routine takes now ca. 100 milliseconds, in LC 8 DP 10 about 650. So still a massive difference. I tested this both on a mid 2011 Macbook Pro with OS 10.10.5 and on the latest pumped up Retina i7 iMac with MacOS 10.10.5. The numbers I get are almost the same, so it seems not dependent on processor speed etc. I wasn't able to test on an earlier version of MacOS.

In my case using "lock screen" is of no help, since I exactly want to have the screen redrawn after the color of each char is changed. It's karaoke after all.

For me callbacks did not work; I just was not able to get it work properly. Also, the actual karaoke player software doesn't use any playerobject that could send callback messages. The timing is encoded in the speechtext itself. In the actual program that you see in the screencast I use the "send message in xx milliseconds " command.

Each word of the speech is timecoded in this way (for example): 1000_I_2000 4000_have_6500 7000_a_7500 8000_dream_9000 with the number before the word marking the timcodestart and the second number marking the timecodeend for the colouring of that word to occur. When the karaoke player is started, I start a time counter in milliseconds and the routine for colouring of the next word is called by: "send colornextword to me in (difference between actual time counter and the timecodestart for the next word) milliseconds" Now in LC >=6.7. the colouring of the first words takes already much longer so that the time counter has already moved past the timecodestart of the next word. Not sure if you can follow here. In any case the standalone application I saved from LC 6.1.2 works still very solid on the latest MacOS, so it's not an urgent concern. I am not (yet) looking for a dirty work around, my puzzlement is about what happened with LC 6.7 that my code doesn't work anymore. Can it be related to the changes for Unicode, or could it be related to LC interaction with OS X Yosemite?

Best
Oliver

PS: people in Finland are crazy about karaoke in all forms, whether it is heavy metal karaoke or in our case speech karaoke.

[bn]

Hi Oliver,

I am afraid this increase in time is due to the change to Cocoa. Now a screen refresh is 16 milliseconds whereas it used to be in pre 6.7 between 3 and 12 milliseconds.

But if you think of your problem in terms of framerate, you will notice that you are aiming at framerates that are unnecessary high.

In your example 3 words consisting of 13 chars each is 39 chars to color sequentially. You see you can do that in 100 milliseconds in 6.1 (I see about 300 milliseconds in LC 6.1.3 MacBook Pro mid 2010)

A framerate of 24 (movies) seems enough to trick the eye into perception of motion. 39 chars in 100 milliseconds would give you a framerate of 390/second.

Assuming in your timing the timescale of your audio is 600 then you have for "8000_dream_9000" 1000/600 = 1.6 seconds. you have 5 chars in "dream".

What I am suggesting is that Livecode in its current 16 milliseconds is fast enough to give a good animation (a framerate of 25 would need a fresh screen every 40 milliseconds)

If I were you I would throw 2,3 or 4 characters at the screen each time and see how fluent that look. I am sure you can find a timing that fits your time slots.

Try the examples I posted above an see what happens, whether you really see a significant degradation of animation. After all animation is cheating, isn't it?

Kind regards

Bernd

[okk]

Hi Bernd, thanks for your comment concerning frame rates. I agree with you that anything higher than 20 frames per second is not necessary for our purpose. In fact, I measured some fast speeches, the shortest time for coloring 1 char was about 30 milliseconds, but that is rare. The test script I posted was just about my surprise, that things slowed down from 6.1. to 7.1. In any case, the slower speed would be no issue for our purpose. It's karaoke and people anyway ignore the coloring of the words to a large extend.

But the issue which I demonstrated in the screen recording (see my original posting) was of another magnitude. Here the delay accumulated in such a way that I quickly run out of synch with over a second. Something else went on there. I actually found a preliminary explanation. When I made the controller of the player object invisible, thinks worked almost as intended. I was thinking if the controller of the new player object might be the culprit here? I made another screen recording to demonstrate the effect: https://vimeo.com/147381340

Another - albeit less prominent - factor of the slowdown was the small textfield with a running timecode on the right. It is refreshed every 50 milliseconds which I believed should not be too much of a strain on the performance. But when I set the player object to invisible and disabled the running timecode the coloring of the words were quite precise. So, mystery partially solved but not really... It seems that different processes that need a refresh of the screen (controller of player object, the running timecode field, the coloring of chars, the moving playhead) are accumulating and slowing down everything. Not sure though if it would help to lock / unlock the screen every 20 milliseconds.

Thanks for all the helpful comments from the forum members!
Oliver