Unexpected behaviour
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller
Unexpected behaviour
I have a piece of Livecode running in a loop.
It is acting as a server for my project. It is running in 9.0 in the development window.
It constantly looks for files that might appear in a folder. The folder is on the same PC as the server and App. The App is running under 9.01 at the same time in a development window.
These files are sent to the folder by the app. Think of a Facebook type post.
If no file found, then the server skips the file processing and loops again.
If a file or files are found, the file processing starts within the loop and when completed, the loop continues.
After a few seconds, the server seems to fail to complete the processing and skips a few lines of the output display field. A table field. The server window in the development screen also greys over like it has crashed. about 15 seconds later it will display the window normally.
However, if I include a wait for 2 seconds, it works perfectly. Full processing and no gray-outs.
Is this something outers have noticed with short loops?
Is this something that will be OK when made into exe and android?
Any advice would be welcome. Thanks.
It is acting as a server for my project. It is running in 9.0 in the development window.
It constantly looks for files that might appear in a folder. The folder is on the same PC as the server and App. The App is running under 9.01 at the same time in a development window.
These files are sent to the folder by the app. Think of a Facebook type post.
If no file found, then the server skips the file processing and loops again.
If a file or files are found, the file processing starts within the loop and when completed, the loop continues.
After a few seconds, the server seems to fail to complete the processing and skips a few lines of the output display field. A table field. The server window in the development screen also greys over like it has crashed. about 15 seconds later it will display the window normally.
However, if I include a wait for 2 seconds, it works perfectly. Full processing and no gray-outs.
Is this something outers have noticed with short loops?
Is this something that will be OK when made into exe and android?
Any advice would be welcome. Thanks.
Re: Unexpected behaviour
A couple of thoughts -
. I believe that Jacque mentioned in a previous post that the field is currently very slow, are you putting the list into a variable then flipping it to the field, or are you putting it directly into the field?
. If waiting 2 seconds to do the loop resolves the issue, why not just put the 2 second wait in? It doesn't seem like a 2 second cycle would kill a conversation taking place in text.
. I believe that Jacque mentioned in a previous post that the field is currently very slow, are you putting the list into a variable then flipping it to the field, or are you putting it directly into the field?
. If waiting 2 seconds to do the loop resolves the issue, why not just put the 2 second wait in? It doesn't seem like a 2 second cycle would kill a conversation taking place in text.
Re: Unexpected behaviour
Queued up repeats can make the app appear to have hung.
Within a repeat loop include a lineThis gives the engine a near zero duration pause for the opportunity to update the screen, poll the keyboard, and other housekeeping processes.
Within a repeat loop include a line
Code: Select all
wait 0 milliseconds with messages
Re: Unexpected behaviour
Thank you both for the advice.
Regarding the field idea mentioned (Jacque's earlier comment). There will be optimisation issues that need to be managed. That will be one of them. At the moment, I am resorting to simple code that I know will work, so I can progress to other areas of this project. I is possible I will discover a major problem that kills the project and wasting time with optimum code at such an early stage makes no sense. If it all works, then the code, look of the GUI... will all need attention.
The solution to include WAITs is not one I was looking for. Ultimately speed and throughput will be a key factor. Perhaps it will be fine when compiled, or whatever you call it. The WAIT can stay if it has to, but other solutions would be more attractive.
Thank you all.
Regarding the field idea mentioned (Jacque's earlier comment). There will be optimisation issues that need to be managed. That will be one of them. At the moment, I am resorting to simple code that I know will work, so I can progress to other areas of this project. I is possible I will discover a major problem that kills the project and wasting time with optimum code at such an early stage makes no sense. If it all works, then the code, look of the GUI... will all need attention.
The solution to include WAITs is not one I was looking for. Ultimately speed and throughput will be a key factor. Perhaps it will be fine when compiled, or whatever you call it. The WAIT can stay if it has to, but other solutions would be more attractive.
Thank you all.
Re: Unexpected behaviour
The code
seems to work well.
Good idea.
Code: Select all
wait 0 milliseconds with messages
Good idea.
-
- VIP Livecode Opensource Backer
- Posts: 7215
- Joined: Sat Apr 08, 2006 8:31 pm
- Location: Minneapolis MN
- Contact:
Re: Unexpected behaviour
If the processing is in a repeat loop that will run forever, you have a risk of an eventual crash or memory depletion. The "wait with messages" is one solution for handlers that exit after some time, but depending on what your loop is doing it may fail in the long term. If that happens, change the script to use the "send <message > in <time >" method which is more stable for a handler that never ends.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
-
- VIP Livecode Opensource Backer
- Posts: 9580
- Joined: Wed May 06, 2009 2:28 pm
- Location: New York, NY
Re: Unexpected behaviour
What Jacque said.
Know that even the tiny amount of time (0 ticks, say,) that the processor is required to, er, process, is enough of a delay to prevent even the longest and most unruly repeat loops not to reach the recursion limit. It can be a useful practice in and of itself.
This issue arises in a repeat loop where user action is the only way to exit. I always check to see if the optionKey is down.
But if the loop itself is not built properly, so that it intrinsically can NEVER exit, then this little trick will only delay the inevitable. But it does indeed provide a measure of forgiveness in many cases.
Craig Newman
Know that even the tiny amount of time (0 ticks, say,) that the processor is required to, er, process, is enough of a delay to prevent even the longest and most unruly repeat loops not to reach the recursion limit. It can be a useful practice in and of itself.
This issue arises in a repeat loop where user action is the only way to exit. I always check to see if the optionKey is down.
But if the loop itself is not built properly, so that it intrinsically can NEVER exit, then this little trick will only delay the inevitable. But it does indeed provide a measure of forgiveness in many cases.
Craig Newman
Re: Unexpected behaviour
How would you deal with this requirement?
I have the program in a loop because I don't know how else to deal with the case.
Files (Posts from a social app) come in sporadically and are dumped in the folder to be found as the server (server is my term for this element of the project) loops and checks for files.
I can't see how the server can be triggered when files appear, unless in a loop.
I have the program in a loop because I don't know how else to deal with the case.
Files (Posts from a social app) come in sporadically and are dumped in the folder to be found as the server (server is my term for this element of the project) loops and checks for files.
I can't see how the server can be triggered when files appear, unless in a loop.
Re: Unexpected behaviour
Hi,
Have fun!
2 possibilities:
- Have your program started by the OS via cron, taskplaner etc. in suitable intervals, do the job once and then terminate itself. Or:
- Have something as such in your stack script:
Code: Select all
constant kTime = 120 -- this is the interval in seconds on openstack doMyJob -- initial start end openstack on doMyJob if checkFile() then -- do whatever is required to do end if send "doMyJob" to me in kTime seconds -- start next interval end doMyJob function checkFile -- check here if you need to run your worker script -- (special file exists, there are files at all, ...) -- if yes, then: -- put true into myVar return myVar end checkFile
Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
-
- VIP Livecode Opensource Backer
- Posts: 7215
- Joined: Sat Apr 08, 2006 8:31 pm
- Location: Minneapolis MN
- Contact:
Re: Unexpected behaviour
Dunbar's comment about exiting a loop is correct but I wasn't thinking about exits, more about memory depletion. Every time through the loop, the handler may (usually?) retain some amount of RAM. After a period of time memory will be exhausted and the app will crash. I learned this 30 years ago when I used an infinite repeat loop to process incoming data. Each time through the loop the handler retained one byte of memory. It reliably crashed every 26 hours or so.
Axwald's scripted solution is what I had in mind, it exits the working handler completely after each iteration and frees up all RAM. I hadn't thought of the cron job solution but that's good too.
Axwald's scripted solution is what I had in mind, it exits the working handler completely after each iteration and frees up all RAM. I hadn't thought of the cron job solution but that's good too.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
Re: Unexpected behaviour
I don't know what Cron or Taskplanner is. Also, This server, as I call it, might be on a big commercial thing over which I will have no control/understanding.
Thank you for the code. I appreciate you have put a lot of effort into it. I don't understand it, so It will take me a while to explore it. For now, I need to concentrate on developing the app & server so they work, even if only for short bursts.
What I think I understood was that if the loop can exit, then be triggered by some external means, that would prevent the memory depletion crash. Is that correct? Once out of the loop, the app just sits there, waiting for an event to happen.
Thank you all again.
Thank you for the code. I appreciate you have put a lot of effort into it. I don't understand it, so It will take me a while to explore it. For now, I need to concentrate on developing the app & server so they work, even if only for short bursts.
What I think I understood was that if the loop can exit, then be triggered by some external means, that would prevent the memory depletion crash. Is that correct? Once out of the loop, the app just sits there, waiting for an event to happen.
Thank you all again.
-
- VIP Livecode Opensource Backer
- Posts: 7215
- Joined: Sat Apr 08, 2006 8:31 pm
- Location: Minneapolis MN
- Contact:
Re: Unexpected behaviour
Yes, you understand it right. LC has the ability to send a command at intervals you specify, which is the "external" event you mentioned, but it isn't really external like a user action, it's triggered by the script itself.
You start the process by calling the handler once, which in the example is done in an openStack handler. The processing handler runs and at the end it calls itself to start again in a certain amount of time:
kTime here is a constant declared above the handlers, but you can use a number directly if you want, for example: send "doMyJob" to me in 5 seconds
Then in 5 seconds the processing handler will run, call itself again in another 5 seconds, and continue doing that forever until you stop it. This is the most efficient way to do long loops for all sorts of reasons.
You start the process by calling the handler once, which in the example is done in an openStack handler. The processing handler runs and at the end it calls itself to start again in a certain amount of time:
Code: Select all
send "doMyJob" to me in kTime seconds
Then in 5 seconds the processing handler will run, call itself again in another 5 seconds, and continue doing that forever until you stop it. This is the most efficient way to do long loops for all sorts of reasons.
Jacqueline Landman Gay | jacque at hyperactivesw dot com
HyperActive Software | http://www.hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
-
- VIP Livecode Opensource Backer
- Posts: 365
- Joined: Mon Jun 10, 2013 1:32 pm
Re: Unexpected behaviour
The best way of doing this is to use tge calls in the windows api
That exist for this specific purpose.
The problem is that you need to write a widget to
Do this.
I’m throwing this here for the more experienced widget creators ... it is vey specific and could be a good template for other operating system API widgets or a widget toolkit for important system calls. At the moment I don’t have ANY expertise in widget creation and the other resource I need is the time which is a bigger problem at the moment.
https://docs.microsoft.com/en-us/window ... ifications
Lagi
That exist for this specific purpose.
The problem is that you need to write a widget to
Do this.
I’m throwing this here for the more experienced widget creators ... it is vey specific and could be a good template for other operating system API widgets or a widget toolkit for important system calls. At the moment I don’t have ANY expertise in widget creation and the other resource I need is the time which is a bigger problem at the moment.
https://docs.microsoft.com/en-us/window ... ifications
Lagi
Re: Unexpected behaviour
Hi,
If you can install software on a server, you should be able to have it run, right? Besides, if you want to run your software on a server, you'll want to do it on one you can control yourself sufficiently - you'll need to kill it if it misbehaves, you'll need to restart it sometimes, you'll want to check RAM/ CPU consumption etc.
Running your code on "a big commercial thing over which I will have no control/understanding" looks, for me, like a sure call for disaster ...
Having quite some server side programs out there that are used in commercial environments & are running 24/7/365 with partially heavy load, I'm doing this in 2 ways:
Even the most stable standalone will misbehave once in a lifetime, and, thanks Murphy, at the most unappropriated time & causing maximum damage. Especially when it relies on data fetched over the internet where timeouts, transmission failures etc. can occur. Better safe than sorry.
Remember, these programs run somewhere on a server where nobody is watching, and a certain number of people may depend of their function, to earn their food ...
Lagi's suggestion is quite interesting - only that watching a directory is really easy already using plain LC. So it looks a bit of overkill for me.
And besides, the LC versions meeting my requirements of stability don't support widgets ;-)
Have fun!
These are core mechanisms that every OS has in one form or another - starting repetitive tasks is base functionality of each OS.
If you can install software on a server, you should be able to have it run, right? Besides, if you want to run your software on a server, you'll want to do it on one you can control yourself sufficiently - you'll need to kill it if it misbehaves, you'll need to restart it sometimes, you'll want to check RAM/ CPU consumption etc.
Running your code on "a big commercial thing over which I will have no control/understanding" looks, for me, like a sure call for disaster ...
It's a very simple approach:
- Put the work that must be done repeatedly into a single handler (here: "doMyJob").
- Make sure this handler, after it has run, calls itself again, after a suitable time (here: "send "doMyJob" to me in kTime seconds")
Having quite some server side programs out there that are used in commercial environments & are running 24/7/365 with partially heavy load, I'm doing this in 2 ways:
- Running as service/ daemon:
This way the program runs as a system task, independent of any logged in user, and can even be restarted by the system itself should it crash.
Use case: Mainly socket servers, simple LC standalones that authenticate users, give computed values, and can do simple tasks (log cleanup, restart services, restart server, ...) without need to actually login to the server.
Precaution: Still I reboot these once in 24h (via taskplaner), because even if they have proved to run flawlessly for months, very rarely they have decided to hang, too. For sure, at the most annoying times ... - Start - work - shutdown programs:
These are simple programs that do a specific task when started, then quit.
Use case: Mainly fetching data from web/ FTP/ EMail, converting them & writing them into databases.
Precaution: I use a "starter" for these that is called in short intervals via taskplaner: This starter program looks in his schedule what actual "worker program" is on queue, then looks if it is running already, in which case it sends a kill command (and an alert email). Then it starts the "worker program", and quits. The "worker program" does its job, and also quits.
Even the most stable standalone will misbehave once in a lifetime, and, thanks Murphy, at the most unappropriated time & causing maximum damage. Especially when it relies on data fetched over the internet where timeouts, transmission failures etc. can occur. Better safe than sorry.
Remember, these programs run somewhere on a server where nobody is watching, and a certain number of people may depend of their function, to earn their food ...
Lagi's suggestion is quite interesting - only that watching a directory is really easy already using plain LC. So it looks a bit of overkill for me.
And besides, the LC versions meeting my requirements of stability don't support widgets ;-)
Have fun!
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
Re: Unexpected behaviour
'nix based systems
I think that Automator + iCal would probably be the simplest solution outside of Cron on a Mac for your particular problem *if* you decided to go native system on checking for files.
As AxWald points out, there are pros/cons to each method, or you could wind up with a hybrid of both methods, i.e. cron launching your Lc app, Lc app shuts down after running your loop(s), rinse repeat.
Win based systemsWikipedia wrote: The software utility cron is a time-based job scheduler in Unix-like computer operating systems. People who set up and maintain software environments use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals. It typically automates system maintenance or administration—though its general-purpose nature makes it useful for things like downloading files from the Internet and downloading email at regular intervals.[1] The origin of the name cron is from the Greek word for time, χρόνος (chronos).[2]
cron is most suitable for scheduling repetitive tasks. Scheduling one-time tasks is often more easily[who?] accomplished using the associated at utility.
Mac based systemsdummies.com wrote: Task Scheduler
. Windows 7 and Windows Vista {Edit mine- all Windows back to 3.x actually} sport a feature-rich task scheduler called, amazingly, Task Scheduler. It shows you all the tasks scheduled in Windows — tasks you create yourself as well as system tasks. You can take advantage of this power yourself to set up and run your own tasks, ensuring that all the computer maintenance gets done.
Note - on all the above systems, there are more than one way to start jobs using a built in system scheduler, but OSX in particular has mixed breeding, so cron is a native source from its 'nix background (from what I understand, though, it is being deprecated on OSX), and Apple is bringing in 'launchd', which is a more complicated (imho) alternative that uses plists to schedule up jobs.Chron.com wrote: If you've been manually performing a task on your Mac over and over, you can automate it with Automator and use iCal to schedule when you want the task to run. Apple includes a free copy of Automator on all new Macs running the OS X operating system. The Automator library comes with a variety of actions to help you automate tasks to work with documents, files and folders, email, music, images and more. All new Macs also come with iCal, Apple’s application for scheduling tasks in a calendar.
I think that Automator + iCal would probably be the simplest solution outside of Cron on a Mac for your particular problem *if* you decided to go native system on checking for files.
As AxWald points out, there are pros/cons to each method, or you could wind up with a hybrid of both methods, i.e. cron launching your Lc app, Lc app shuts down after running your loop(s), rinse repeat.