frame-byframe with Video Formats

Visuals, audio, animation. Blended, not stirred. If LiveCode is part of your rich media production toolbox, this is the forum for you.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Xero
Posts: 152
Joined: Sat Jun 23, 2018 2:22 pm

frame-byframe with Video Formats

Post by Xero » Thu Aug 02, 2018 8:07 am

I'm trying to script a frame-by-frame function for a video player object. I don't want to use the buttons on the player, as I have other functions such as different playrates that aren't on the player object itself.
I am using the following code on forward frame and back frame buttons:

Code: Select all

on mousedown
play step forward player "Player"
end mousedown
or "play step back player..."
When I use .avi and .wmv files, this functions perfectly well. It steps forwards on frame and stops.
When I use .mov and .mp4 files, this doesn't work. It steps to a frame somewhere in the video and stops, and won't step forwards any more.
Clearly this is something that is unique to file types (Klaus has warned me on this one!!!)
So I am looking at a work around.
I have tried:

Code: Select all

on mousedown
put the currenttime of player "Player" into tTime
put tTime+1 into tTime
set the currenttime of player "Player" to tTime
end moousedown
But no deal... doesn't work on any file types.
I have used this code:

Code: Select all

on mousedown
start player "Player"
wait 0.5 seconds
stop player "Player"
end mousedown
Which in effect shifts the video forwards a fraction of a second and then stops it. This could be refined to make a single frame leap with some maths... This works on all formats, but is a little hit and miss with accuracy, so is not preferred.
The problem with this method is that to go backwards, I need to use "set playrate of player "Player" to -1" instead of "start" command. This doesn't work at all...

Any thoughts on another workaround that may produce a result like a frame-by-frame forwards and backwards that work on .mov and .mp4/ .mpg files?

Thanks in advance.
XdM

Klaus
Posts: 13806
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: frame-byframe with Video Formats

Post by Klaus » Thu Aug 02, 2018 8:17 am

Hi Xero,

use "the timescale of player xyz" to do some math! The timescale is the internal, well timescale
of a movie file and returns the number of "its own time units" per second.

So this will "step" forward half a second:

Code: Select all

...
put the currenttime of player xyz into tNOW
put the timescale of player xyz into tTimescale
put tTimescale/2 into tTimedifference
put tNow + tTimedifference to tNowPlusHalfASecond
set the currenttime of player xyz to tNowPlusHalfASecond
...
Unfortunately: the timescale <> FPS, and we cannot find out the FPS of a movie in LC.
Again, do NOT rely on "step forward/back player xyz"!


Best

Klaus

Xero
Posts: 152
Joined: Sat Jun 23, 2018 2:22 pm

Re: frame-byframe with Video Formats

Post by Xero » Thu Aug 02, 2018 8:21 am

I know... I'm trying to find an alternative to the "play step back" command that doesn't work with all file types.
Is there an alternative to go backwards by one frame? Surely there is!
XdM

capellan
Posts: 654
Joined: Wed Aug 15, 2007 11:09 pm

Re: frame-byframe with Video Formats

Post by capellan » Thu Aug 02, 2018 11:47 pm

Hi Xeno,

Could you use FFMPEG for extracting single video frames and then
show this jpeg within LiveCode?

https://stackoverflow.com/questions/275 ... given-time

Al

Klaus
Posts: 13806
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: frame-byframe with Video Formats

Post by Klaus » Fri Aug 03, 2018 10:43 am

Hi Al,

great tip, but I am not sure this is what Xero had in mind... :D


Best

Klaus

Xero
Posts: 152
Joined: Sat Jun 23, 2018 2:22 pm

Re: frame-byframe with Video Formats

Post by Xero » Fri Aug 03, 2018 3:34 pm

Hey Capellan…
Xeno… ha ha ha… I ain't no alien! :lol:
Sadly, I don't just need a jpeg image, I need to just forward the video one frame, or backward it one frame, without too much jpeg creation etc.
I'm currently working on getting any input videos into a format that I can use. I have a reformatting program (they're dime-a-dozen) that is lightning fast, so there's only a little lag in getting things started in the app.
Does anyone know how I can open another app, put a video through it, and output it while still staying within Livecode? I could use a macro in Windows, or just do it manually...
That way, I have a video in a format that works in my app.
Sadly, my camera and webcam only output .mov and .mp4 files... the two that aren't working! :cry:
XdM

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3990
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: frame-byframe with Video Formats

Post by bn » Sun Aug 05, 2018 11:28 am

Hi Xero,

here is a stack that attempts to implement a step forward and step backward button. It works most of the time.

To determine the frame duration I have to "calibrate" the movie for 1 second. I then take the time units in the timescale and divide by number of frames.

The logic is almost all in the behavior of the player that I set to a button "playerBehavior". Clicking on that button edits the script of that button. Using behaviors makes it portable.

Behaviors are reset on opencard in the card script if the name of your stack changes and does not match the behavior of the player previously set.

Kind regards
Bernd
autoDetectFrameLength_1.livecode.zip
(2.42 KiB) Downloaded 309 times
EDIT: please note that the stack does not work correctly with many video formats and is not reliably able to step through the frames of a movie
Last edited by bn on Mon Aug 06, 2018 8:06 am, edited 1 time in total.

Klaus
Posts: 13806
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: frame-byframe with Video Formats

Post by Klaus » Sun Aug 05, 2018 6:01 pm

Hi Bernd,

very nice stack, but the computed FPS do not match the real FPS of these movies.
At least for the different MP4s I have on my disk.

I posted an enhancement request:
https://quality.livecode.com/show_bug.cgi?id=21463


Best

Klaus

Xero
Posts: 152
Joined: Sat Jun 23, 2018 2:22 pm

Re: frame-byframe with Video Formats

Post by Xero » Mon Aug 06, 2018 7:05 am

Hey Bernd,
It's close... and it's doing something, but it's not doing frame-by frames... at least not with mov and mp4 files. It steps the slider forwards, but the picture still sits where it is...
Thanks anayway!
XdM

bn
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3990
Joined: Sun Jan 07, 2007 9:12 pm
Location: Bochum, Germany

Re: frame-byframe with Video Formats

Post by bn » Mon Aug 06, 2018 8:02 am

Hi Klaus, Xero,

I did not test enough and did not read the dictionary well. I fell afoul of thinking that "currentTimeChanged" was related to a frame change. This assumption did not work out.

I will leave the stack here as a "bad" example.

Kind regards
Bernd

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: frame-byframe with Video Formats

Post by bogs » Mon Aug 06, 2018 12:53 pm

bn wrote:
Mon Aug 06, 2018 8:02 am
I will leave the stack here as a "bad" example.
I think more like "as a start" would be more appropriate Bernd :D
Image

Klaus
Posts: 13806
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: frame-byframe with Video Formats

Post by Klaus » Mon Aug 06, 2018 1:14 pm

bn wrote:
Mon Aug 06, 2018 8:02 am
I fell afoul of thinking that "currentTimeChanged" was related to a frame change. This assumption did not work out.
Unfortunately the dictionary is really misleading here:
currenttimechanged
...
Sent to a player when the user switches to another frame.
...
This is simply not true.

Xero
Posts: 152
Joined: Sat Jun 23, 2018 2:22 pm

Re: frame-byframe with Video Formats

Post by Xero » Mon Aug 06, 2018 4:48 pm

Hey Bernd...
It does actually do stuff, so as Bogs says... it's a start! It just needs refinement (hacking and slashing) to get it to where it needs to be!
We're getting there... even if it is taking the entire power of the greatest Livecode Forum minds to get it done.
BB
XdM

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: frame-byframe with Video Formats

Post by FourthWorld » Mon Aug 06, 2018 7:31 pm

Klaus wrote:
Mon Aug 06, 2018 1:14 pm
Unfortunately the dictionary is really misleading here:
currenttimechanged
...
Sent to a player when the user switches to another frame.
...
This is simply not true.
Unless the player object lost that functionality, it's true but in a very limited sense: when I last wrote something dependent on the currentTimeChanged message, it only sent the message when the user changed the current time, by clicking in the player control below the video, or dragging its thumb control.

The message is not sent as the engine changes frames during playback, requiring me to use a timer to keep other things updated as the video played.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

Klaus
Posts: 13806
Joined: Sat Apr 08, 2006 8:41 am
Location: Germany
Contact:

Re: frame-byframe with Video Formats

Post by Klaus » Mon Aug 06, 2018 7:45 pm

Hm, on my Mac I loaded an MP4 video file (timescale 1000, 25 FPS) into a player and added this script to the player:

Code: Select all

on currentTimeChanged pNewTime
   put CR & pNewTime after msg   
end currentTimeChanged
Then I manually started the video and got this in the messge box until I stopped the player again:
0
132
167
183
211
246
277
301
331
361
...
I think that speaks for itself.


Best

Klaus

Post Reply

Return to “Multimedia”