LiveCode Forums.

Hello all,

I am interested in improving and learning new techniques for managing lengthy processes in Rev.

By a lengthy process I mean a script that may run in isolation take 4-8 hours to run.

I have 6-7 such processes (they function well, but are prototypes that require improvement). Right now they are characterised by repeat constructs that perform various actions on large datasets or in large datagrids. These processes run fine, and well within Rev.

If for some reason I wished to pause/resume, or abort such scripts the keyboard shortcuts for doing so never work. I guess the repeat loop, doesn't give it a chance.

So, I started to pick my way through the RevLive DVD's and having an interest in games I watched Klaus's lectures (Klaus if you are there I really enjoyed your sessions). I started to absorb the piece on triggers and realised that beyond it's importance to game play (keeping the app responsive), I might be able to better control a long process by breaking up the code to various functions and then using 'send' - 'send in time'.

Then again, perhaps then it is better to use local sockets with the added benefit that they can be scripted to behave as background tasks and are threaded to not only allow the application to be responsive during such processes, but also to allow the ability to pause, resume or terminate such a process, and have greater throughput.

Am I on the right track with this? Perhaps there is a better technique? Perhaps one is better than another for database intensive or web enabled apps.

A happy place right now would be to kick off a process (onMouseUp on a button), have on a card somewhere within the interface display some stats a process id, a progress bar etc, and to have the app remain responsive and if at all possible the opportunity to kick off and similarly watch a second process.

Any guidance would be appreciated.

Kind regards, Andrew

Hi Andrew,

well, I was not at RevLive, so who the heck is claiming to be Klaus on these DVDs?

Whatever, I would try to cut the "blocking" repeat loops into small pieces and use "send... in...".

OR you could add a "wait 0 with messages" at the beginning or end of the repeat loop.
This way Rev will be responsive during the loop.

AND I would check a variable or custom property at the beginning of each loop to see, if the loop should stop.

Something like this: And this in a button script or a script that you can evoke via shortcut: You get the picture!

Best

Klaus

Hi Andrew,

you might be interested in this thread that deals with lengthy data
http://n4.nabble.com/Deleting-Data-Woef ... l#a1690021
I don't know what exactly you are doing that takes hours and hours and I realize that large datasets require a lot of time, often in Rev for long repeating tasks there are slight variations that cut down the time needed. I remember a project Garrett was doing. He indexed a 23 GB Wikipedia file extracting the title of the articles. It took him 3 Days in his first version. After some optimizations this was cut down to 30 minutes.

http://forums.livecode.com/phpBB2/viewt ... f=9&t=3690
http://forums.livecode.com/phpBB2/viewt ... f=9&t=3728

Klaus pointed out a way to make a handler more responsive to user input. If you are after a pause/RESUME thing you would have to store the state of the current process, which is possible but involved.
If you just want to make the process more responsive by adding "wait 0 milliseconds with messages" then you might consider to do it every 100th or 1000th time in the repeat loop. Although it says 0 milliseconds it takes up time, and every 1000th time is probably enough (depending on your repeat loop) to make Rev responsive. In my experience the overhead of having an additional counter and do a mod on it plus the conditional is less than doing wait 0 milliseconds everytime around.

If you know your data in advance and how it is structured you might be able to find ways to cut down on computation time. A little benchmarking is very helpful in this setting.

I dont know about the socket approach, did once try to distribute image processing to a stack and 2 standalones, it was not worth it. Again this might be completely different for your needs.

regards
Bernd

Thank you Klaus (I mixed you up with Malte, I mean no offence to anyone, I don't post everyday but I read nearly every post so names stick in my head), and equal thanks to bn,

Apologies for the late reply. I wasn't being lazy, I disappeared for a couple of days (having put it off for 9 months) to clean out my mother's house after she passed away last year. An exhausting, but somehow healing task, trying to sort the things she had collected during her life journey.

Back to the topic, great tips all round, in respect of script control and workflow.

Yes, my first attempt ran for around 3 days as well, so recently 7-12 hours seemed a great speed bump.

Having a web background (mostly PHP) I have used Rev a couple of times as a control panel of sorts to manage PHP CLI (command line interface) scripts. PHP CLI is multithreaded, so I can commonly launch 15-20, instances of the same process from Rev and it works fine and fast. I spoke about that in this earlier topic http://forums.runrev.com/phpBB2/viewtop ... 093#p18093.

I need to devote more time to really investigate the sockets approach - still a WIP for me at present.

So basically right now, I am improving my Rev scripts with the techniques you have both suggested, using PHP CLI from within Rev when I need multithreading, whilst I investigate sockets in the medium term. It feels like progress.

I am still feeling my way which method in a corporate environment is most tolerable from the perspective of network admin, firewalls etc, when using a rev desktop client with web resident data.

Thanks again - kind regards , Andrew