LiveCode Forums.

Posted: **Mon Mar 14, 2022 4:56 pm**

Hello everyone,

I recently developed an application that does some simple things which are the following:

Opens a socket connection to a server

Captures the response of the server, which includes some json foramted data

Stores them in a text field and then saves the content of the text field to a .json file

closes connection

Then runs two .batch files with Launch document command, one that corrects the json format and one that converts json to XML

The above commands run in a loop every second approximately(depends of the time it takes to execute).

I see that in an i7 7700HQ 8core laptop, it takjes approx. 15% of cpu usage.

Does this percent include the load of the 2x batch files when running or it is just the livecode standalone app which uses 15%?

Also what I see, is that because the server doesnt close the connection to keep it alive to send new messages, the command "open socket" blocks the stack untill I set a time-out time.

Thanks in advance

Posted: **Mon Mar 14, 2022 6:09 pm**

mikelpapamike wrote: ↑
Mon Mar 14, 2022 4:56 pm
I see that in an i7 7700HQ 8core laptop, it takjes approx. 15% of cpu usage.

Does this percent include the load of the 2x batch files when running or it is just the livecode standalone app which uses 15%?

The most complete way to answer that would be to make a test build with the socket comms commented out.

Let us know what that shows you.

Posted: **Tue Mar 15, 2022 11:07 am**

I just did a test build without the part of socket connection, just saving out static data of a field to a .json file and then running the 2x batch that correct the json data and convert it to XML.

Running in the same loop of every second approximately the cpu usage is about 3.5% to 4.5%

In task manager I see coming and going as a sub process the windows script host execution so I may assume it is the "launch commands" of batch files?

So the rest ~10-12% of usage comes from the socket part?

Also in another application, I noticed that the livecode standalone used only and maxed out one of the processors and the rest where in idle.

How can I build a standalone app that is multi-threaded?

Posted: **Wed Mar 16, 2022 3:52 am**

mikelpapamike wrote: ↑
Tue Mar 15, 2022 11:07 am
I just did a test build without the part of socket connection, just saving out static data of a field to a .json file and then running the 2x batch that correct the json data and convert it to XML.

Thank you for running the test. Quantifying things is usually helpful for diagnostics.

In task manager I see coming and going as a sub process the windows script host execution so I may assume it is the "launch commands" of batch files?

That seems correct.

So the rest ~10-12% of usage comes from the socket part?

If the only change was commenting the socket comms, the overhead of those comms would appear to account for the difference.

Also in another application, I noticed that the livecode standalone used only and maxed out one of the processors and the rest where in idle.

How can I build a standalone app that is multi-threaded?

What do you want to accomplish with threading?

Have you used threading in other languages?

My questions are in earnest. I've seen a good many requests for threading over the years, but once we examine the goal we find that threading would do little if anything to satisfy the goal in most cases.

Multithreading send to have taken on a sort of folklorish role as a universal panacea. In some cases (like smoothly animating GIFs while other actions are happening) it's exactly what's needed. But not often. And threading adds complexity to code, requiring a certain amount of additional overhead to manage them, both in the engine and by the developer. And threading introduces the potential for race conditions and other side effects single-threaded engines usually avoid. And of course the throughput of any task distributed across threads is still bound by Amdahl's Law, so results are rarely as glamorous as imagined.

To answer your question, like most scripting languages LiveCode is single-threaded.

That said, it has async options for some tasks, including socket I/O, which offload much of the handling to the OS. Sometimes that's useful. Once in a while profoundly useful. Other times it adds complication with little benefit. That's why I ask about goals.

And most code can be optimized. Sometimes a lot. Lean execution can sometimes be dramatic enough to make the thought of threading just go away. We've seen cases in these forums of code optimized by more than two orders of magnitude, sometimes more. Occasionally much more.

And if concurrent execution (the one thing threads actually provide for their tradeoffs) is truly essential, LiveCode can be used to build mutiprocessing solutions, a pool of worker standalones happily churning away on big tasks. The overhead of multiprocessing is not much greater than multithreading on Linux, though multiprocessing on Windows does come at a greater resource cost.

Let's learn more about what you're looking to do, and that will allow us to guide you down the most productive path for what you're working on.

Posted: **Wed Mar 16, 2022 2:54 pm**

Thanks a lot for the detailed information. Yes the only change was the socket connection part that was commented out.

I'm not an Expirienced programmer, not at all. What I do is writing Apps for automation of TV Broadcast On-air Graphics and other automations based on communication with other Broadcast hardware.

My experience is mostly in Flash and ActionScript. Where is the same thing, every Flash application is not multithreaded, but the engine that renders the various flash build is distributing many concurent apps running to multiple threads.

To be honest I wasn't aware of Amdahl's law. So I just thought that if the execution of a code maxes out one thread, then multi-threading will manage to run the whole process in less time.

Also what I noticed is that my code is really slow on livecode whet it involves visual changes of stack fields and tables(For example, I have a simple code that checks tab delimited items in 2 table fields that contain numbers,and if the number is greater than 10,it makes item's background Green.With only 6 to 8 items fullfiling the criteria, it takes 5 to 8 seconds to execute.)

What I want to achieve is, if I can make my code run faster, especially the part of socket connectivity.

Posted: **Wed Mar 16, 2022 6:32 pm**

mikelpapamike wrote: ↑
Wed Mar 16, 2022 2:54 pm
Also what I noticed is that my code is really slow on livecode whet it involves visual changes of stack fields and tables(For example, I have a simple code that checks tab delimited items in 2 table fields that contain numbers,and if the number is greater than 10,it makes item's background Green.With only 6 to 8 items fullfiling the criteria, it takes 5 to 8 seconds to execute.)

Please post either the section of code in question (good) or an example stack that illustrates what you're looking for and the limitation you've encountered (better) and we can almost certainly speed things up, hopefully you the level you're aiming for.

What I want to achieve is, if I can make my code run faster, especially the part of socket connectivity.

The example immediately above doesn't appear to be socket related, which is good for two reasons: it gives us more variety to chew on (folks here love optimization challenges), and sockets are mostly handled outside of LiveCode so we have few opportunities to optimize their tasks.

Most of the time spent with sockets is in transit, generically referred to as "latency". Though electricity travels fast over the net, error-correction eats up a LOT of time (TCP is a strange and wondrous thing).

In LC, some socket based workflows can benefit from using the asynchronous option available, callbacks. See the "with messages" option for socket and URL commands in the Dictionary, and we can explore that more as needed.

But before we spend too much time thinking the logic of how you get the data over the wire, my hunch is that the bigger opportunity is to review what it app is doing with the data once it arrives.

Can you tell us more about how you process the incoming data, or better post the relevant portion here so we can see it?

Posted: **Thu Mar 17, 2022 9:37 pm**

This is the part of the code that i mentioned previously. This code is from an app that plays out Graphics with statistics of basketball match. I want to present visualy to the operator,if one player has some statistic bigger than 10(number) lets say points or field goals which is important for basketball.

So bellow is my code:

Code: Select all

on mouseUp
   set the itemdel to tab
   repeat with x = 1 to the number of lines in fld "DataGridC"
      if item 10 of line x of field "DataGridC" >= "10"
      then
         set the backgroundColor of item 10 of line x of field "DataGridC" to green
      else
         set the backgroundColor of item 10 of line x of field "DataGridC" to white
      end if
   end repeat
   
   
   repeat with x = 1 to the number of lines in fld "DataGridC"
      if item 13 of line x of field "DataGridC" >= "10"
      then
         set the backgroundColor of item 13 of line x of field "DataGridC" to green
      else
         set the backgroundColor of item 13 of line x of field "DataGridC" to white
      end if
   end repeat
   
   repeat with x = 1 to the number of lines in fld "DataGridC"
      if item 15 of line x of field "DataGridC" >= "10"
      then
         set the backgroundColor of item 15 of line x of field "DataGridC" to green
      else
         set the backgroundColor of item 15 of line x of field "DataGridC" to white
      end if
   end repeat
   
   repeat with x = 1 to the number of lines in fld "DataGridC"
      if item 17 of line x of field "DataGridC" >= "10"
      then
         set the backgroundColor of item 17 of line x of field "DataGridC" to green
      else
         set the backgroundColor of item 17 of line x of field "DataGridC" to white
      end if
   end repeat
   
   
   
   //b team
   
   
   
   repeat with x = 1 to the number of lines in fld "DataGridD"
      if item 10 of line x of field "DataGridD" >= "10"
      then
         set the backgroundColor of item 10 of line x of field "DataGridD" to green
      else
         set the backgroundColor of item 10 of line x of field "DataGridD" to white
      end if
   end repeat
   
   
   repeat with x = 1 to the number of lines in fld "DataGridD"
      if item 13 of line x of field "DataGridD" >= "10"
      then
         set the backgroundColor of item 13 of line x of field "DataGridD" to green
      else
         set the backgroundColor of item 13 of line x of field "DataGridD" to white
      end if
   end repeat
   
   repeat with x = 1 to the number of lines in fld "DataGridD"
      if item 15 of line x of field "DataGridD" >= "10"
      then
         set the backgroundColor of item 15 of line x of field "DataGridD" to green
      else
         set the backgroundColor of item 15 of line x of field "DataGridD" to white
      end if
   end repeat
   
   repeat with x = 1 to the number of lines in fld "DataGridD"
      if item 17 of line x of field "DataGridD" >= "10"
      then
         set the backgroundColor of item 17 of line x of field "DataGridD" to green
      else
         set the backgroundColor of item 17 of line x of field "DataGridD" to white
      end if
   end repeat
end mouseUp

To execute this part of code, takes too long, sometimes more than 10 seconds and maxes out the first thread of the cpu.
But the issue is that blocks any other user input as the code is beeing executed. User can not press any button until the execution completes.

This is the table field

https://drive.google.com/file/d/1PQUvbw ... 9B/preview

And this is after the code

https://drive.google.com/file/d/1BvhkB- ... sp=sharing

Posted: **Thu Mar 17, 2022 9:57 pm**

This will be fun. Can you share some sample data we can work with? Offhand I see opportunities for at least an order of magnitude performance gain, but it would help to be able to test on real-world data.

Posted: **Thu Mar 17, 2022 10:16 pm**

You mean the data that are exposed to the table field or a sample stack?

Also I tried using DataGrid instead of table fields to gain it's benefits, but exposing data to Datagrid is so much slower than table fields.

Here are 2 files with sample data exposed to the 2 table fields (tab delimited)

https://drive.google.com/file/d/1xla7YS ... sp=sharing

https://drive.google.com/file/d/1ND3Ns6 ... sp=sharing

Posted: **Fri Mar 18, 2022 4:25 am**

Thanks. Add this after "on mouseUp":

Code: Select all

lock screen

Here that took a 450 ms run down to 4 ms.

We can drop it more if needed.

LiveCode Forums.

CPU Usage of standalone application

CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application

Re: CPU Usage of standalone application