LC > 5.5.1 performance is really disappointing!

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9801
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: LC > 5.5.1 performance is really disappointing!

Post by FourthWorld » Tue Mar 09, 2021 6:55 pm

adventuresofgreg wrote:
Tue Mar 09, 2021 6:43 pm
...I haven't a clue where the bottle necks are.
In your Plugins menu is GoLiveNet. Open that, and when it's loaded go to LiveNet's Stackfile section. There download "4W Flight Recorder".

That's a tool I wrote to replace LC's Message Watcher, but it not only does that more readably, it also provides a lightweight form of performance profiling.

To use it:

1. Click "Start Recording".
2. Perform whatever action runs the routine you want to understand how to improve.
3. Click "Stop Recording".

In the 4W Flight Recorder window you'll then see a list of messages handled, with an indented list of the command and function calls triggered by the messages.

One of the the right-hand columns will show an approximation of time spend within each routine. For many reasons this can only be an approximation (mostly Heisenberg's Uncertainty Principle, that the act of observing something alters the thing being observed). But the approximation will likely be useful in allowing you to see where the greatest amount of time is being spent.

The output listing can be saved to a file. It might be useful if you Zip it and post it here.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 7210
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: LC > 5.5.1 performance is really disappointing!

Post by jacque » Wed Mar 10, 2021 6:41 pm

Code: Select all

if the number of items of theStDev > 200 then delete item 1 of theStdev
I'm not sure if this will make any difference, but I have a vague recollection that it takes more time to delete something from the front of a list than from the end because everything in the remainder of the list has to be moved up one slot. It's a minor thing but with thousands of repetitions it might add up.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

liveme
Posts: 240
Joined: Thu Aug 27, 2015 5:22 pm
Location: down under

Re: LC > 5.5.1 performance is really disappointing!

Post by liveme » Wed Mar 10, 2021 11:10 pm

Hi Greg... I was going to ask you about that :
-- collecting a group of 200 values and then performing some math on them like stdev, or a average, etc.
How did you come to the chunk size of 200 ...or was it just a guess ?

As for what I've been reading - beside coding some small benchmarks virual test. (data stored in memory not file)
There is a termendous difference in speed processing if data comes from your Cpu L1, L2, L3 memory ...compared to your MB Ram memory.

So, your latest idea to test smaller chunck would most likely help in this process of avoiding to load too much data that is needed from your SSD at some point, read is fast compared to writting.

Any process will be executing faster if it can be loaded as best as possible into your L1 cache memory.
Optimizing for what the L1 can handle is the best !.
So one want to avoid request back any data to a slower component down the road as much as possible, mostly RAM or SSD.

Optimizing execution at this level requires getting to know your own CPU(s) capabilities and how the whole CPU Cache L1-L3 system works. Taking your L1 cache size into account to "code" your processing will give you the best results you can possibly achieve on that machine :)

There is probably specialized tools out there who could help you knowing more precisely how big of a "chunk" would best fit into your "Cpu/cache/ram" according to the data you submit - but meanwhile benchmarking how each process - from little to bigger chuncks - can help.

Now about the "processing order",there is a dif. between processing numbers and characters inside the CPU...so.. It also would be interesting to compare - in your case - the speed between processing one type at a time - if doable - operations like running first :
- processing of all dates, and "write" that (to memory or ssd).
- next...just doing the substraction of the 2 mys..and write that again...

what happen is CPUs have many smart processes and tends to detect what type of operation is being submited to them then optimize that process for execution with the L1 cache...therefore, if the Type AND the Lenght of data, changes often...as many time the Cpu working with Cache will need to adapt to.
There is so much interesting stuff to learn about this ! (not to talk about parallel processing, right ?)

- Have you had a chance to check how good your Cpus cores are involved while you are processing your data ?
Some system will place some Cores on rest, to cool them down, nothing wrong.

*see video at 9:05
https://www.youtube.com/watch?v=IAkj32VPcUE

adventuresofgreg
Posts: 349
Joined: Tue Oct 28, 2008 1:23 am
Contact:

Re: LC > 5.5.1 performance is really disappointing!

Post by adventuresofgreg » Thu Mar 18, 2021 10:12 pm

This is all very useful thanks. I think I can "organize" my process better. Just timing some functions and functions like convert to date and seconds, and random(something) and sort all take time, so if I'm ever calling these functions MORE than a single time, that's probably not very efficient.

Question: How do you store data in Cpu L1,L2, etc vs. Ram ?? Do you mean that smaller chunks of data are more likely available from Cpu vs. Ram? I have noticed that looping routines working on smaller chunks is much faster.

Greg

Post Reply

Return to “Talking LiveCode”