Unexpected behaviour

Got a LiveCode personal license? Are you a beginner, hobbyist or educator that's new to LiveCode? This forum is the place to go for help getting started. Welcome!

Moderators: Klaus, FourthWorld, heatherlaine, kevinmiller

ajperks
Posts: 60
Joined: Sat Sep 06, 2014 3:38 pm

Unexpected behaviour

Post by ajperks » Thu Nov 01, 2018 5:38 pm

I have a piece of Livecode running in a loop.
It is acting as a server for my project. It is running in 9.0 in the development window.
It constantly looks for files that might appear in a folder. The folder is on the same PC as the server and App. The App is running under 9.01 at the same time in a development window.
These files are sent to the folder by the app. Think of a Facebook type post.
If no file found, then the server skips the file processing and loops again.
If a file or files are found, the file processing starts within the loop and when completed, the loop continues.

After a few seconds, the server seems to fail to complete the processing and skips a few lines of the output display field. A table field. The server window in the development screen also greys over like it has crashed. about 15 seconds later it will display the window normally.

However, if I include a wait for 2 seconds, it works perfectly. Full processing and no gray-outs.

Is this something outers have noticed with short loops?
Is this something that will be OK when made into exe and android?

Any advice would be welcome. Thanks.

bogs
Posts: 3814
Joined: Sat Feb 25, 2017 10:45 pm

Re: Unexpected behaviour

Post by bogs » Thu Nov 01, 2018 5:44 pm

A couple of thoughts -
. I believe that Jacque mentioned in a previous post that the field is currently very slow, are you putting the list into a variable then flipping it to the field, or are you putting it directly into the field?

. If waiting 2 seconds to do the loop resolves the issue, why not just put the 2 second wait in? It doesn't seem like a 2 second cycle would kill a conversation taking place in text.
Image

SparkOut
Posts: 2020
Joined: Sun Sep 23, 2007 4:58 pm

Re: Unexpected behaviour

Post by SparkOut » Thu Nov 01, 2018 5:44 pm

Queued up repeats can make the app appear to have hung.
Within a repeat loop include a line

Code: Select all

wait 0 milliseconds with messages
This gives the engine a near zero duration pause for the opportunity to update the screen, poll the keyboard, and other housekeeping processes.

ajperks
Posts: 60
Joined: Sat Sep 06, 2014 3:38 pm

Re: Unexpected behaviour

Post by ajperks » Thu Nov 01, 2018 6:00 pm

Thank you both for the advice.
Regarding the field idea mentioned (Jacque's earlier comment). There will be optimisation issues that need to be managed. That will be one of them. At the moment, I am resorting to simple code that I know will work, so I can progress to other areas of this project. I is possible I will discover a major problem that kills the project and wasting time with optimum code at such an early stage makes no sense. If it all works, then the code, look of the GUI... will all need attention.

The solution to include WAITs is not one I was looking for. Ultimately speed and throughput will be a key factor. Perhaps it will be fine when compiled, or whatever you call it. The WAIT can stay if it has to, but other solutions would be more attractive.
Thank you all.

ajperks
Posts: 60
Joined: Sat Sep 06, 2014 3:38 pm

Re: Unexpected behaviour

Post by ajperks » Thu Nov 01, 2018 6:03 pm

The code

Code: Select all

wait 0 milliseconds with messages
seems to work well.
Good idea.

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 5114
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Unexpected behaviour

Post by jacque » Fri Nov 02, 2018 6:38 pm

If the processing is in a repeat loop that will run forever, you have a risk of an eventual crash or memory depletion. The "wait with messages" is one solution for handlers that exit after some time, but depending on what your loop is doing it may fail in the long term. If that happens, change the script to use the "send <message > in <time >" method which is more stable for a handler that never ends.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 6179
Joined: Wed May 06, 2009 2:28 pm
Location: New York, NY

Re: Unexpected behaviour

Post by dunbarx » Fri Nov 02, 2018 7:05 pm

What Jacque said.

Know that even the tiny amount of time (0 ticks, say,) that the processor is required to, er, process, is enough of a delay to prevent even the longest and most unruly repeat loops not to reach the recursion limit. It can be a useful practice in and of itself.

This issue arises in a repeat loop where user action is the only way to exit. I always check to see if the optionKey is down.

But if the loop itself is not built properly, so that it intrinsically can NEVER exit, then this little trick will only delay the inevitable. But it does indeed provide a measure of forgiveness in many cases.

Craig Newman

ajperks
Posts: 60
Joined: Sat Sep 06, 2014 3:38 pm

Re: Unexpected behaviour

Post by ajperks » Sat Nov 03, 2018 11:30 am

How would you deal with this requirement?

I have the program in a loop because I don't know how else to deal with the case.
Files (Posts from a social app) come in sporadically and are dumped in the folder to be found as the server (server is my term for this element of the project) loops and checks for files.
I can't see how the server can be triggered when files appear, unless in a loop.

AxWald
Posts: 368
Joined: Thu Mar 06, 2014 2:57 pm

Re: Unexpected behaviour

Post by AxWald » Sat Nov 03, 2018 1:06 pm

Hi,
ajperks wrote:
Sat Nov 03, 2018 11:30 am
I can't see how the server can be triggered when files appear, unless in a loop.
2 possibilities:
  • Have your program started by the OS via cron, taskplaner etc. in suitable intervals, do the job once and then terminate itself. Or:
  • Have something as such in your stack script:

    Code: Select all

    constant kTime = 120  -- this is the interval in seconds
    
    on openstack
       doMyJob                      --  initial start
    end openstack
    
    on doMyJob
       if checkFile() then
          --  do whatever is required to do
       end if
       send "doMyJob" to me in kTime seconds  --  start next interval
    end doMyJob
    
    function checkFile
       --  check here if you need to run your worker script
       --  (special file exists, there are files at all, ...)
       --  if yes, then:
       --  put true into myVar
       return myVar
    end checkFile
Both versions have pros and cons, choose wisely ;-)

Have fun!
Livecode programming until the cat hits the fan ...

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 5114
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Unexpected behaviour

Post by jacque » Sat Nov 03, 2018 5:46 pm

Dunbar's comment about exiting a loop is correct but I wasn't thinking about exits, more about memory depletion. Every time through the loop, the handler may (usually?) retain some amount of RAM. After a period of time memory will be exhausted and the app will crash. I learned this 30 years ago when I used an infinite repeat loop to process incoming data. Each time through the loop the handler retained one byte of memory. It reliably crashed every 26 hours or so.

Axwald's scripted solution is what I had in mind, it exits the working handler completely after each iteration and frees up all RAM. I hadn't thought of the cron job solution but that's good too.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

ajperks
Posts: 60
Joined: Sat Sep 06, 2014 3:38 pm

Re: Unexpected behaviour

Post by ajperks » Sat Nov 03, 2018 5:59 pm

I don't know what Cron or Taskplanner is. Also, This server, as I call it, might be on a big commercial thing over which I will have no control/understanding.

Thank you for the code. I appreciate you have put a lot of effort into it. I don't understand it, so It will take me a while to explore it. For now, I need to concentrate on developing the app & server so they work, even if only for short bursts.

What I think I understood was that if the loop can exit, then be triggered by some external means, that would prevent the memory depletion crash. Is that correct? Once out of the loop, the app just sits there, waiting for an event to happen.

Thank you all again.

jacque
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 5114
Joined: Sat Apr 08, 2006 8:31 pm
Location: Minneapolis MN
Contact:

Re: Unexpected behaviour

Post by jacque » Sun Nov 04, 2018 5:35 pm

Yes, you understand it right. LC has the ability to send a command at intervals you specify, which is the "external" event you mentioned, but it isn't really external like a user action, it's triggered by the script itself.

You start the process by calling the handler once, which in the example is done in an openStack handler. The processing handler runs and at the end it calls itself to start again in a certain amount of time:

Code: Select all

send "doMyJob" to me in kTime seconds
kTime here is a constant declared above the handlers, but you can use a number directly if you want, for example: send "doMyJob" to me in 5 seconds

Then in 5 seconds the processing handler will run, call itself again in another 5 seconds, and continue doing that forever until you stop it. This is the most efficient way to do long loops for all sorts of reasons.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com

Lagi Pittas
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 327
Joined: Mon Jun 10, 2013 1:32 pm

Re: Unexpected behaviour

Post by Lagi Pittas » Sun Nov 04, 2018 11:52 pm

The best way of doing this is to use tge calls in the windows api
That exist for this specific purpose.

The problem is that you need to write a widget to
Do this.

I’m throwing this here for the more experienced widget creators ... it is vey specific and could be a good template for other operating system API widgets or a widget toolkit for important system calls. At the moment I don’t have ANY expertise in widget creation and the other resource I need is the time which is a bigger problem at the moment.

https://docs.microsoft.com/en-us/window ... ifications

Lagi

AxWald
Posts: 368
Joined: Thu Mar 06, 2014 2:57 pm

Re: Unexpected behaviour

Post by AxWald » Mon Nov 05, 2018 2:27 pm

Hi,
ajperks wrote:
Sat Nov 03, 2018 5:59 pm
I don't know what Cron or Taskplanner is. Also, This server, as I call it, might be on a big commercial thing over which I will have no control/understanding.
These are core mechanisms that every OS has in one form or another - starting repetitive tasks is base functionality of each OS.
If you can install software on a server, you should be able to have it run, right? Besides, if you want to run your software on a server, you'll want to do it on one you can control yourself sufficiently - you'll need to kill it if it misbehaves, you'll need to restart it sometimes, you'll want to check RAM/ CPU consumption etc.

Running your code on "a big commercial thing over which I will have no control/understanding" looks, for me, like a sure call for disaster ...
ajperks wrote:
Sat Nov 03, 2018 5:59 pm
Thank you for the code. I appreciate you have put a lot of effort into it. I don't understand it, so It will take me a while to explore it.
It's a very simple approach:
  1. Put the work that must be done repeatedly into a single handler (here: "doMyJob").
  2. Make sure this handler, after it has run, calls itself again, after a suitable time (here: "send "doMyJob" to me in kTime seconds")
Anything else (timer value in a constant "kTime", "checkFile" function to determine if there's work to do at all) is just cosmetics. Guess Jacque explained it better already ;-)


Having quite some server side programs out there that are used in commercial environments & are running 24/7/365 with partially heavy load, I'm doing this in 2 ways:
  1. Running as service/ daemon:
    This way the program runs as a system task, independent of any logged in user, and can even be restarted by the system itself should it crash.
    Use case: Mainly socket servers, simple LC standalones that authenticate users, give computed values, and can do simple tasks (log cleanup, restart services, restart server, ...) without need to actually login to the server.
    Precaution: Still I reboot these once in 24h (via taskplaner), because even if they have proved to run flawlessly for months, very rarely they have decided to hang, too. For sure, at the most annoying times ...
  2. Start - work - shutdown programs:
    These are simple programs that do a specific task when started, then quit.
    Use case: Mainly fetching data from web/ FTP/ EMail, converting them & writing them into databases.
    Precaution: I use a "starter" for these that is called in short intervals via taskplaner: This starter program looks in his schedule what actual "worker program" is on queue, then looks if it is running already, in which case it sends a kill command (and an alert email). Then it starts the "worker program", and quits. The "worker program" does its job, and also quits.
You may think this a lot of expenditure - but this method has grown in years of actual work with LC.
Even the most stable standalone will misbehave once in a lifetime, and, thanks Murphy, at the most unappropriated time & causing maximum damage. Especially when it relies on data fetched over the internet where timeouts, transmission failures etc. can occur. Better safe than sorry.
Remember, these programs run somewhere on a server where nobody is watching, and a certain number of people may depend of their function, to earn their food ...


Lagi's suggestion is quite interesting - only that watching a directory is really easy already using plain LC. So it looks a bit of overkill for me.
And besides, the LC versions meeting my requirements of stability don't support widgets ;-)

Have fun!
Livecode programming until the cat hits the fan ...

bogs
Posts: 3814
Joined: Sat Feb 25, 2017 10:45 pm

Re: Unexpected behaviour

Post by bogs » Mon Nov 05, 2018 3:21 pm

'nix based systems
Wikipedia wrote: The software utility cron is a time-based job scheduler in Unix-like computer operating systems. People who set up and maintain software environments use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals. It typically automates system maintenance or administration—though its general-purpose nature makes it useful for things like downloading files from the Internet and downloading email at regular intervals.[1] The origin of the name cron is from the Greek word for time, χρόνος (chronos).[2]

cron is most suitable for scheduling repetitive tasks. Scheduling one-time tasks is often more easily[who?] accomplished using the associated at utility.
Win based systems
dummies.com wrote: Task Scheduler
. Windows 7 and Windows Vista {Edit mine- all Windows back to 3.x actually} sport a feature-rich task scheduler called, amazingly, Task Scheduler. It shows you all the tasks scheduled in Windows — tasks you create yourself as well as system tasks. You can take advantage of this power yourself to set up and run your own tasks, ensuring that all the computer maintenance gets done.
Mac based systems
Chron.com wrote: If you've been manually performing a task on your Mac over and over, you can automate it with Automator and use iCal to schedule when you want the task to run. Apple includes a free copy of Automator on all new Macs running the OS X operating system. The Automator library comes with a variety of actions to help you automate tasks to work with documents, files and folders, email, music, images and more. All new Macs also come with iCal, Apple’s application for scheduling tasks in a calendar.
Note - on all the above systems, there are more than one way to start jobs using a built in system scheduler, but OSX in particular has mixed breeding, so cron is a native source from its 'nix background (from what I understand, though, it is being deprecated on OSX), and Apple is bringing in 'launchd', which is a more complicated (imho) alternative that uses plists to schedule up jobs.

I think that Automator + iCal would probably be the simplest solution outside of Cron on a Mac for your particular problem *if* you decided to go native system on checking for files.

As AxWald points out, there are pros/cons to each method, or you could wind up with a hybrid of both methods, i.e. cron launching your Lc app, Lc app shuts down after running your loop(s), rinse repeat.
Image

Post Reply

Return to “Getting Started with LiveCode - Complete Beginners”