the microseconds

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, LCMark

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

the microseconds

Post by mwieder » Tue Sep 15, 2020 12:19 am

Twenty years down the road now, "the milliseconds" is no longer granular enough for accurate timing of events.
Profiling code, for instance, has to rely on averaging multiple runs in order to get anything resembling real command timing.

I've implemented "the microseconds" locally. Is there any interest?

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: the microseconds

Post by FourthWorld » Tue Sep 15, 2020 1:51 am

Sure. But I have a vague recollection of a conversation with Mark Waddingham about the accuracy of measurements smaller than milliseconds. I got the impression such things would be approximations, perhaps not all that different from what we could obtain with arithmetic on milliseconds. Do I misremember?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: the microseconds

Post by mwieder » Tue Sep 15, 2020 1:54 am

Apparently. The current code gets the microseconds, then divides by 1000 to return the milliseconds.
Benchmarking, of course, will depend on the overhead of the enclosing code, and that may be what Mark was referring to.

rkriesel
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 118
Joined: Thu Apr 13, 2006 6:25 pm

Re: the microseconds

Post by rkriesel » Tue Sep 15, 2020 1:59 am

Hi, Mark. I share the motivation, but satisfy it with "the long milliseconds," which distinguishes nanoseconds. That maximizes precision, and, with enough test results and additional measures ( median, standard deviation, and many more), accuracy beyond milliseconds. (Disclaimer: I'm not a statistician.)
-- Dick

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: the microseconds

Post by mwieder » Tue Sep 15, 2020 2:13 am

Dick- do you actually have code that returns the nanoseconds? Er... long milliseconds?
The best I could come up with cross-platform is microsecond timing.

Oh... wait... the "long milliseconds" is built in?
Learned something new today.
Would be nice if that were documented.

AxWald
Posts: 578
Joined: Thu Mar 06, 2014 2:57 pm

Re: the microseconds

Post by AxWald » Tue Sep 15, 2020 11:20 am

Hi,
Whow! "Long millisecs" actually yields results in LC 6 already!
But they behave a little bizarre - "put (the long milliseconds)":

6.7.10: They show the fractional part only occasionally, and then it's always ".0002".
9.6/ 64bit: They show no fractional part at all.

On Win 10. Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: the microseconds

Post by mwieder » Tue Sep 15, 2020 4:41 pm

Um. On Windows I think the code needs to use a different call to get more resolution.
That's part of what I implemented. You need to call both QueryPerformanceFrequency and QueryPerformanceCounter to get the microseconds.

Code: Select all

virtual real64_t GetCurrentMicroseconds()
{
     LARGE_INTEGER CurrentTime, Frequency;
 
     QueryPerformanceFrequency(&Frequency); 
     QueryPerformanceCounter(&CurrentTime);
     CurrentTime.QuadPart *= 1000000.0;
     CurrentTime.QuadPart /= CurrentTime.QuadPart;
         return (real64_t)CurrentTime;
}

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: the microseconds

Post by LCMark » Tue Sep 15, 2020 5:46 pm

Twenty years down the road now, "the milliseconds" is no longer granular enough for accurate timing of events.
Profiling code, for instance, has to rely on averaging multiple runs in order to get anything resembling real command timing.
I'm not sure that has changed has it? :D

Multiple (average) runs should always be used to profile code - particularly if the thing you are measuring has such a small timing to be < 1ms.

On multi-user, pre-emptive multi-thread/process systems there's too much going on to be sure that any single run is representative (again particularly if the event being measured only lasts for a tiny amount of time - a single page fault requiring a fetch of a disk page from a hard disc which has spun down can completely obliterate any meaningful results when you are looking at microseconds, for example!).

There's also the overhead of making the time measurement and processing it in a higher level language like LiveCode Script to consider too.

That being said...

In regards to the current notion of 'time' - the core measurement is 'the seconds' - this is the number of seconds since the UNIX epoch (so is intimately tied to the datetime functionality).

This value is computed as a double internally - `the seconds`, `the ticks`, `the millisecs` are that value scaled and floored (so they are integers), the long variants don't floor (so you get fractional parts).

On UNIX-based systems it uses gettimeofday() (which returns seconds + microseconds - but the resolution is system defined - I think you get full microsecond precision, or at least close to full microsecond precision on macOS)

On Windows the time is seeded using _ftime (a C library function which returns the UNIX time), and then it is updated using timeGetTime - which has millisecond precision.

This is, admittedly a rather archaic mechanism and how the engine has always computed MCS_time() on windows. The logic behind it was performance I think. The equivalent 'gettimeofday' in the Windows API used to be really slow whereas timeGetTime has always been super quick.

These days there might well be a better system call to use to get higher-precision timings on windows - but replacing the current mechanism would need to make sure it is indeed as fast or faster than the current method (as MCS_time() - the core method - is used a lot) and also retains parity with the system-clock (i.e. an absolute measurement relative to some well-defined epoch - so it can be shifted to be UNIX time).

NB: QueryPerformanceCounter is certainly fast - but it isn't 'tied' to any external clock - so can only measure relative time, not absolute making it unsuitable for a replacement to MCS_time() (I have used it quite frequently for internal performance benchmarking on windows at the C level as its a really simple API to use - but I'm still careful to ensure I time enough loops of whatever I'm doing to make sure the measurements are useful!).

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: the microseconds

Post by mwieder » Tue Sep 15, 2020 8:35 pm

I'm not sure that has changed has it? :D
That in itself certainly hasn't changed, but a more granular approach to timing can result in orders of magnitude benefits.

As usual, you make several good points here.
But I think benchmarking and profiling are related but different actions with different objectives.

In profiling code the objective is usually to find the areas that can benefit from optimizing, and therefore the relative speed of code with relation to other sections of code is important. You often don't get the chance to run multiple loops if sections of code are executed once only.

The problem there is that single lines of code don't normally take milliseconds to execute, and so you end up with zero milliseconds. Averaging several of these will still give you zero. A sub-millisecond result will enable the averaging of fewer iterations, and even with just one time through will get you into the ballpark. "Accuracy" of timing is always going to be a bit fuzzy - in my profiler I current set a floor of 1 millisecond to avoid the averaging of zeros problem. This gives what I think is unacceptably inaccurate real-world timing but usable relative timing.

As a counterexample, imagine trying to benchmark/profile code if you had only "the seconds" to work with. You'll need several orders of magnitude more iterations to get anything meaningful than if you had "the milliseconds" to work with. You're using a brush that's too big for the canvas.

That said, the "long milliseconds" gives me what I need since I don't really care about Windows.

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9578
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: the microseconds

Post by dunbarx » Tue Sep 15, 2020 10:26 pm

The "Long Milliseconds". Fascinating, and yet undocumented.

I have a friend who would love to use LC to control a homeMade CNC milling machine. He uses assembly language now, because he cannot get LC accurately to know what time it is. The milliseconds don't cut it.

The long milliseconds would. But before I present this possibility to him, has anyone ever figured out a way to determine how accurate they are? Even if I dropped the three least significant digits, yielding only "the microSeconds" (which would be fast enough), it matters not if the accuracy is not there.

What do you run as a standard with which to compare? In other words, if you run a process in LC that takes 1,000,000 long millisecs, say, and then run the standard for the same length of time, how do you read the standard? It does not have to be perfect, just close. I assume there will be losses here and there along the way.

Craig

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: the microseconds

Post by FourthWorld » Wed Sep 16, 2020 1:31 am

Assembly? Why not cut the development time by 90% and use C?

And if that much precision with time is required, does the CNC require specialized hardware (mobo, bus, etc)?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

mrcoollion
Posts: 719
Joined: Thu Sep 11, 2014 1:49 pm
Location: The Netherlands

Re: the microseconds

Post by mrcoollion » Wed Sep 16, 2020 10:32 am

AxWald wrote:
Tue Sep 15, 2020 11:20 am
9.6/ 64bit: They show no fractional part at all.
On Win 10. Have fun!
I made a little stack that compares long milliseconds and milliseconds statement and for what it is worth there is a difference in Windows 10 64b with LC 9.6.1. and sometimes a fractional part is shown as well. However, I do not know how reliable it is.

Regards,

Paul
Milliseconds01.png
Milliseconds01.png (12.01 KiB) Viewed 9344 times
This is the stack.
Milliseconds01.zip
(986 Bytes) Downloaded 286 times

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9578
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: the microseconds

Post by dunbarx » Wed Sep 16, 2020 2:58 pm

@ Richard.

I think he is just comfortable, and old fashioned, using assembly.

@ mrcoolion.

I think I may have misrepresented what I meant about accuracy of timing. Ultimately, in any computer, the system clock rules. If someone wants to control an outside process, like moving a milling head with servo motors a precise distance in a precise time, both the accuracy and resolution of the "output" matter.

In order to use LC for this, and I have been promoting the idea for years, he needed better than millisecond time resolution. That this now seems "internally" possible still begs the issue of how to output control signals, likely through the USB bus, to his custom-built software/hardware interface.

I have been doing this sort of thing for years, using (and touting) an interface made by a company in Germany. But my work has never been nearly as time sensitive as his. That gadget internally can only get down to about one millisec resolution, which has been fine for everything I have ever done.

In matter of fact, his controller requires only that signals be regular and consistent, not necessarily accurate with respect to any other particular timeFrame, either internal or external. Anyway, I will keep thinking about this before I open the discussion with him again.

Craig

mwieder
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 3581
Joined: Mon Jan 22, 2007 7:36 am
Location: Berkeley, CA, US
Contact:

Re: the microseconds

Post by mwieder » Wed Sep 16, 2020 4:27 pm

mrcoollion- your stack isn't really showing anything useful because of the time it takes LC to execute another statement, and more importantly, to update a field on the screen. Locking the screen would help, as would putting the results into variables before the subtraction and *then* updating the screen. But even so, executing two similar commands in sequence is bound to have different microsecond results. while millisecond results will have a higher probability of having the same value.

What differences are you seeing in Win10.64b/9.6.1?

mrcoollion
Posts: 719
Joined: Thu Sep 11, 2014 1:49 pm
Location: The Netherlands

Re: the microseconds

Post by mrcoollion » Wed Sep 16, 2020 7:23 pm

mwieder wrote:
Wed Sep 16, 2020 4:27 pm
mrcoollion- your stack isn't really showing anything useful because of the time it takes LC to execute another statement, and more importantly, to update a field on the screen. Locking the screen would help, as would putting the results into variables before the subtraction and *then* updating the screen. But even so, executing two similar commands in sequence is bound to have different microsecond results. while millisecond results will have a higher probability of having the same value.

What differences are you seeing in Win10.64b/9.6.1?
It was just made to show me that on Windows there is a fractial part the rest was just there to see how fast the engine does those two commands for fun. No serious purposes intended.

Locked

Return to “Engine Contributors”