Player Object and AlwaysBuffer on = massive delays in script execution

Visuals, audio, animation. Blended, not stirred. If LiveCode is part of your rich media production toolbox, this is the forum for you.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3990
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by bn » Wed Feb 26, 2020 12:30 am

okk wrote:
Tue Feb 25, 2020 10:58 pm
In any case, I fiddled around a bit more and things got more curious. Here is what I tried: I have a stack with just a player object. AlwaysBuffer is OFF. I created a second stack, placed it on top of the first stack, set the blendlevel to 50, so I have a composite image. On this second stack I put my LC controls, my text field etc, I have my game loop on the card of this stack. I start and stop the player located on the first stack from a button on this second stack. Now, everything runs super smooth, the video runs perfectly in the background, my loop is executed every 10 milliseconds, the text is printed almost every 10 ms. I scratch my head here: why is this possible when layering two stacks on top, but not possible in one stack with two layers? Thanks! Oliver
That is a neat trick. I would not have dared that. Even with a 1920 by 1080 video it plays smoothly and messages flow freely.
It is probably like Richard said that in non-buffering LC is not busy at all because the player just passes the video through.
I am glad that this looks like a solution to your problem.

Kind regards
Bernd

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by FourthWorld » Wed Feb 26, 2020 1:03 am

okk wrote:
Tue Feb 25, 2020 10:58 pm
In any case, I fiddled around a bit more and things got more curious. Here is what I tried: I have a stack with just a player object. AlwaysBuffer is OFF. I created a second stack, placed it on top of the first stack, set the blendlevel to 50, so I have a composite image. On this second stack I put my LC controls, my text field etc, I have my game loop on the card of this stack. I start and stop the player located on the first stack from a button on this second stack. Now, everything runs super smooth, the video runs perfectly in the background, my loop is executed every 10 milliseconds, the text is printed almost every 10 ms. I scratch my head here: why is this possible when layering two stacks on top, but not possible in one stack with two layers? Thanks! Oliver
Very clever. I'm impressed.

It makes sense when we consider what in play:

In your original solution, every frame of the video is double-buffered and double-composited before winding up on screen. But in your new masterfully-super-tricky implementation the OS video engine is allowed to run at full speed, LC minds its own business without needing to interfere with that rendering, and the OS window compositor is running in its own thread to handle the overlay of multiple windows.

In short, your solution is excellent because it's a shining example of separation of concerns: it allows each part of the system to do the thing it's optimized to do, each within its own process thread. Well done.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

okk
Posts: 176
Joined: Wed Feb 04, 2015 11:37 am

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by okk » Wed Feb 26, 2020 10:08 am

Hi and thanks. So far so good. I think I can work with this approach, in our use case it might be an option, but it is nevertheless just a half solution. I can only set the blendlevel for the whole stack on top, so there won't be true transparency, like I can't have a text at blendlevel 0 and the stackbackground at blendlevel 100 (for the overlaying stack). Or can I do something like this:

Code: Select all

set the opaque of stack "overlay" to false 
.
Than it would be possible to set the blendlevel for each LC control spearately and if that would be possible the alwaysbuffer / playerobject issue would become kind of almost irrelevant.

But then asking further, why not implement this directly in LC, if the alwaysbuffer of a playerobject is switched on, LC creates a "fake" backgroundstack with the player object. Just a thought.

When pondering this solution, there came 2 more questions and 1 discovery:
1. Is there a way to keep the layer order of two stacks, is there a way to keep the overlaying stack always on top, for example when the overlaying stack is only covering a small area of the background stack?
2. How to keep the location of two stacks in sync, if one stack is moved, the second stack moves as well?
3. It's possible to give your stack odd shapes... I did not know this beforehand. See the windowShape property...

Thanks as always for great contributions and insights!
Best
Oliver

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by bogs » Wed Feb 26, 2020 10:47 am

okk wrote:
Wed Feb 26, 2020 10:08 am
1. Is there a way to keep the layer order of two stacks, is there a way to keep the overlaying stack always on top, for example when the overlaying stack is only covering a small area of the background stack?
2. How to keep the location of two stacks in sync, if one stack is moved, the second stack moves as well?
3. It's possible to give your stack odd shapes... I did not know this beforehand. See the windowShape property...
This demo works on 'nix, I've never tested it on Mac or **Win, but see if it gives you any ideas for -
keep the layer order of two stacks....keep the location of two stacks in sync...give your stack odd shapes
stackMovesStack.zip
Just a matter of time...
(53.25 KiB) Downloaded 301 times
When the stack starts, you only see the shaped stack.
Hover over that with the mouse, and you should see the back stack.
To move grab the title bar of the back stack.
When you move out of the stacks, you should only see the fore stack.

Ignore the sloppy timer code hah :D

*Edit - just tested it on Win and it appears to work there as well.
Image

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by FourthWorld » Wed Feb 26, 2020 4:52 pm

okk wrote:
Tue Feb 25, 2020 10:58 pm
The tip with emebdding the text as track in the mediacontainer was something we looked into in the very beginning, but we couldnt find any solution for our usecase, it was precesily the lack of solutions that made us to code it ourselves in LC. Normal subtitles have a timecodestart and end for the a whole line of text, in our case we time-code each word. A word is colored based on the start and end timecode for that word which again is based on the original speed of the speaker, which can be sometimes quite fast.
Can you describe the process you use to establish your time codes?

I may be naively overlooking something, but so many people use text tracks for this that I find myself fixated on perhaps using LC to assist with the authoring of some XML-based format that a player engine can then use by itself without needing any complex rendering/buffering workarounds.

Or even simpler: does the text truly need to be placed directly on top of the video? Could it be done in a black field with white text spanning the width of the player just below it? That would add just 20px to the vertical layout requirements of the player, and allow you to run with alwaysBuffer off. Extra bonus points that the text would no longer obscure part of the video.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

okk
Posts: 176
Joined: Wed Feb 04, 2015 11:37 am

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by okk » Wed Feb 26, 2020 6:22 pm

Hi bogs, thanks for you stack, that was a quick response! I understood how you move the two stacks in sync. What I did not understood from your sample stack ow you kept the "winshape" stack always on top of the "positioner" stack. I even made the positioner stack larger, but clicking in it did not make the stack disappear behind the positioner stack. Did you somehow trap the "mousedown" message in the positioner stack?

@ Richard: as I wrote earlier I will for now keep the text field with the "karaoke" animation in a separate area of the stack from the player and let the player run witth alwaysbuffer off. It is not ideal but works for now. Still I would like to be able to have LC controls on top of a player object with a full HD Movie with decent script execution time. That possibility to create different kind of interactive video installations was what brought me to LC in the first place.

Our karaoke subtitle file looks like this:

Code: Select all

<?xml version="1.0"?>
<speech>
		<subtitlesequenz>
			<subtitle>
				<word>Now,</word>
				<timecodestart>2375</timecodestart>
				<timecodeend>2463</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>I</word>
			<timecodestart>2550</timecodestart>
			<timecodeend>2729</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>have</word>
			<timecodestart>2907</timecodestart>
			<timecodeend>2973</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>to</word>
			<timecodestart>3038</timecodestart>
			<timecodeend>3087</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>go</word>
			<timecodestart>3135</timecodestart>
			<timecodeend>3185</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>back</word>
			<timecodestart>3235</timecodestart>
			<timecodeend>3283</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>to</word>
			<timecodestart>3330</timecodestart>
			<timecodeend>3421</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>work</word>
			<timecodestart>3512</timecodestart>
			<timecodeend>3611</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>3710</linebreak>
		</subtitle>
		<subtitle>
			<word>on</word>
			<timecodestart>3835</timecodestart>
			<timecodeend>3954</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		<subtitle>
			<word>my</word>
			<timecodestart>4072</timecodestart>
			<timecodeend>4144</timecodeend>
			<fontsize>1</fontsize>
			<decoration>none</decoration>
			<linebreak>none</linebreak>
		</subtitle>
		etc.
	
It is created in a subtitle program made with LC. Basically we listen the speech at really slow speed and hit enter when we hear a word start and end. That returnkey messages are translated into timecode.

Btw. can you recodgnize the speech? Best. Oliver
Last edited by okk on Wed Feb 26, 2020 8:55 pm, edited 1 time in total.

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by FourthWorld » Wed Feb 26, 2020 6:53 pm

okk wrote:
Wed Feb 26, 2020 6:22 pm
Btw. can you recodgnize the speech?
It's either me ranting about not having enough time to make more LC tools, or maybe Bill Clinton's reference to a second-term SOTU?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3990
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by bn » Wed Feb 26, 2020 8:00 pm

Hi Oliver,

I expanded my test stack and added a substack "u1" that floats at the bottom of the player since I set its mode to "palette". It moves with the stack (see card script on moveStack) Stack "u1" has a button to set it to "topLevel" for developing. Otherwise it is difficult to get at a palette stack.
I added two buttons to change the "ink" of the field in stack "u1" either black on white (srcCopy) or white on black "blendDestATop". Whatever suits you better. Also a slider to adjust blendmode of stack "u1". You could set the "decorations" of stack "u1" to none see Properties Inspector.

I figured that with the subtitles in "palette" mode it is the easiest to keep other controls in the main stack and just use the floating substack for subtitles.

Then again I don't know how your stack is supposed to work.

Kind regards
Bernd

Edit
add this to the card of the main stack

Code: Select all

on suspendStack
   hide stack "u1"
end suspendStack

on resumeStack
   show stack "u1"
end resumeStack
Attachments
testCallbacks okk_03.livecode.zip
(2.63 KiB) Downloaded 294 times

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by bogs » Wed Feb 26, 2020 8:56 pm

okk wrote:
Wed Feb 26, 2020 6:22 pm
Hi bogs, thanks for you stack, that was a quick response! I understood how you move the two stacks in sync. What I did not understood from your sample stack ow you kept the "winhsape" stack always on top of the "positioner" stack. I even made the positioner stack larger, but clicking in it did not make any. Did you somehow trap the "mousedown" message in the positioner stack?
Nothing so fancy as that, I simply set the winShape stack 'systemWindow' property, it will stay above all other windows, not just the stack your moving, because I wanted the timer to be on top of anything else on my system.

There are other methods that you can use to make sure when your done moving a stack, the stack you desire stays on top though, short of keeping it on top of everything else.
Image

okk
Posts: 176
Joined: Wed Feb 04, 2015 11:37 am

Re: Player Object and AlwaysBuffer on = massive delays in script execution

Post by okk » Wed Feb 26, 2020 9:26 pm

Thanks Bernd and bogs for this great LC lesson and for your sample stacks. I havent really worked with multiple stacks before. I am optimistic that I can work with this. In the production version I will keep for now player and subtitle field separate, but the two stack method will make it in the next version. If I still can get my "set the opaque of stack "overlay" to false " in the next LC version that would be the perfect world. @Richard, yes, it is the famous Clinton denial speech, the apology is actually even better. Those are very popular speeches in our events. Thanks & Good night.
Oliver

Post Reply

Return to “Multimedia”