Multi Threadedness ?

Something you want to see in a LiveCode product? Want a new forum set up for a specific topic? Talk about it here.

Moderators: heatherlaine, Klaus, FourthWorld, robinmiller, kevinmiller

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 996
Joined: Thu Apr 11, 2013 11:27 am

Re: Multi Threadedness ?

Post by LCMark » Thu Aug 09, 2018 7:07 am

Haha! @bogs if asking poorly worded questions were on that list I mentioned then I suspect the forums, bugzilla and lists would be exceptionally quite places... Indeed I wouldn't actually know because I'd have had to ban myself by now - many times over!

Anyway don't read too much into what I said - it was directed at no-one, I just thought it a useful opportunity to remind everyone that for all the inhumanity of internet based communication - there are always real people at the other end reading whatever we say.

bogs
Posts: 2189
Joined: Sat Feb 25, 2017 10:45 pm

Re: Multi Threadedness ?

Post by bogs » Thu Aug 09, 2018 7:17 am

@ Richard - 2039? I have a machine for that... :wink:

@ Mark - Well, I didn't mean poorly worded nasty heh, and I do know how hard it is to interpret something in type without the clues you'd pick up in person (although I've seen that fail pretty badly as well).

Anyways, I'll give it a shot, just don't expect it to be coherent. Or Shakespear. I also hate to bring up other languages, but my disjointed understanding of this one means I may not have the right words otherwise.

SO, I came to Lc most recently from RB/Delphi. I'm sure a whole lot of you know one or the other of those two languages. Both are high level languages, and are where I learned about multithreading, so when I am talking about it, it is from that perspective.

You were talking about blocking and non blocking code, which took me a while to find out about (accident, was working on something network related and stumbled upon a blocking and non-blocking example). From what Mark was saying up there, is this similar to so called 'green threading' you find in a lot of other scripting languages? I don't think it is the same as Delphi's take on it, which I am pretty sure was actual multithreading (but I could be mistaken or been lead down the primrose path), and I'm unsure whether RBs take was green threading or not, despite having used code to spin off threads for quite a few years.

As a follow up, why would it be preferable to have a handler that is blocking? I always believed it was preferable to lock something yourself as it were.

And with that, it is bed time for Bonzo. Night all :mrgreen:
Image

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 996
Joined: Thu Apr 11, 2013 11:27 am

Re: Multi Threadedness ?

Post by LCMark » Thu Aug 09, 2018 12:53 pm

@bogs: Richard has probably been using Jacque's time machine stack for years... However, probably incorrectly which is why he still doesn't word things well ;)
... without the clues you'd pick up in person (although I've seen that fail pretty badly as well).
Hehe - tell me about it!
Anyways, I'll give it a shot, just don't expect it to be coherent. Or Shakespear. I also hate to bring up other languages, but my disjointed understanding of this one means I may not have the right words otherwise.
Heh - there's never an issue (in my mind at least) with cross comparisons with other languages - a programming language can't be developed in isolation. There's a long history in programming language design of borrowing, adapting and sometimes just blatantly stealing notions from existing ones; on the flip side it also helps determine what things are generally just a really bad idea!

So - yes - RB/Delph. I have played with RB a bit (years and years ago - in the late 90's I think) its what you expect very similar to VB pre .NET (although admittedly mostly because of its 'traditional' BASIC heritage). So it kind does / works / is really just like that... Just a cross-platform version these days!

Delphi I loved. I quite liked how they mutated Pascal, subtlety, into something which allowed components to be built and then tied together in a very natural way and which fit so well with Window's underlying concepts (I mean, the fact it was a single declaration to wrap an API / COM call and have any errors massaged into an exception was pretty cool in its own right!).

One thing to remember is that Delphi, in particular, was a Windows thing. There aim was not cross-platform - it was to enable people to write Window's software much more easily using all the ideas and notions about such which existed at the time - but in an easier way. They essentially took MFC / ATL (MS's C/C++ based App building frameworks), improved them, wrapped them in explicit syntax in the language so as to eliminate a lot of the cruft you needed to do the same thing in C++ - then bound together a Form-designed based IDE and the ability to easily create independent components which could be shared.

Due to its single platform nature - it meant that things like threading and everything else could be implemented in the way the OS allowed. Out of the 'big 5' OSes which are the main ones which are around today - Windows, Mac, Linux, iOS and Android... Windows stands in a separate class. The other four are all UNIX derived - Windows is not UNIX. It is nothing like UNIX at all in fact. MS created their own entire thing there - whose architecture is fundamentally different from the others. (For better or for worse - I do think 'worse' these days - I don't think I always did).

On Windows UI and Kernel are essentially intertwined - this means threading is more portable amongst threads and as such there's various things you can do on Windows, which you cannot on the other platforms.

So, anyway, sorry for the digression - but - yes - Delphi had true pre-emptive threading. It could do that, in its UI, and across the board - because Windows - at the UI API level could relatively easily. Okay, so yes, you still have to be careful with locks and so forth. However, because Windows handled UI events in a certain way, and tied threads to actual windows in a certain way, and a littany of other things. It worked, it was relatively straightforward, and it worked.

In terms of your question about blocking and non-blocking...

A non-blocking call is one where you initiate the call - and then you ask it to tell you when it is done - in LC terms you get a callback message.

A blocking call is one where you initiate the call, but then nothing else happens until it is completed.

All non-blocking calls can be made blocking with a little bit of scripting:

Code: Select all

local sCallResult
on blockingCall
  put empty into sCallResult
  nonBlockingCall "callMeBack"
  wait until sCallResult is not empty with messages
  return sCallResult
end blockingCall

on callMeBack pNonBlockingResult
  put pNonBlockingResult into sCallResult
end callMeBack
Whether things are better as blocking or non-blocking largely depends on what you are doing.

If something is essentially a 'bit of work which needs to be done, but won't interfere with the users use of a program until it is complete' - then non-blocking is ideal.

On the other hand, if something is 'a bit of work which is required for the user to be able to take the next step' - then blocking is ideal.

In a model where wait is not recursive (which is not currently the case in LiveCode) - then there isn't actually any difference between the two. You can always use blocking - you just have to remember to take into account that other things (which you initiate) might be happening whilst one call is waiting for completion (although you have to do that in the non-blocking model too). Essentially if wait is non-recursive then you don't have to split up a series of blocking handler calls into lots of little handlers tied together 'with message' to make them non-blocking - they just are... Should you want them to be.

In terms of threading - there are two factors in terms of my general 'anti-general-threading'.

The first is purely related to LiveCode code base. It is old, it is large - it was, quite frankly, poorly architected from the start (I have spent a great deal of time and effort trying to improve that!). It was also designed and built for old-style UNIX platforms without any regard for it ever being cross-platform (the UNIXy design goes right down to the core architecture such that it ever had!). Cleaving in multi-threading in any full generality now would be *exceptionally* difficult (by the way, this should also answer @FourthWorld's question about why scripting languages tend not to have general multi-threading - its mainly because most scripting languages are still derived from their original implementation because writing them is incredibly difficult in the first place, and most original implementations of most scripting languages were done without regard for threading!).

The second is that there is no point in adding features to LiveCode (especially ones which come with a huge engineering cost) unless they can benefit *most* users of LiveCode. How many of you actually want to have to deal with all the intricacies of multi-threading, with its locks, semaphores, dead-locks and all that stuff? Surely it is better we choose very high-level and appropriate abstractions which are easy to use, work flawlessly and allow you to do 99% of what you could do in a lower-level language without the huge amount of technical baggage?

I'm pretty sure the combination of non-blocking 'I/O', non-recursive wait, and restricted worker tasks would give LiveCode an arsenal of multi-threading-related capabilities that would put it on a par with any other language-du-jour.

LiveCode's language development and evolution might be slow - however, its important to remember that the language itself has now been around in some form for 31 years... Which is longer than 6 of the languages in the top 8 of TIOBE, the only ones which have been around for longer are C and C++ - and C++ only beat us by 2 years (although given C++ via Cfront was absolutely atrocious - I think we can probably say our language has been for longer than usable C++!). It has stood the test of time, and needs to continue to do so.

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 6151
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: Multi Threadedness ?

Post by FourthWorld » Thu Aug 09, 2018 1:43 pm

Speaking of TIOBE, I see that LiveCode is back on that list again this month:
https://www.tiobe.com/tiobe-index/

This makes 2018 the first year I can recall where LC has been on the TIOBE Index at least half of the months of the year thus far.

Out of the many hundreds of languages in the world, for LC to be in the top 100 by evident popular usage is quite an achievement, something I haven't seen Xojo do in years, and putting it in the same general category as ActionScript, Applescript, Awk, Clojure, Common Lisp, PowerShell, SPSS, and VBScript.

Congrats!
Richard Gaskin
Community volunteer LiveCode Community Liaison

LiveCode development, training, and consulting services: Fourth World Systems: http://FourthWorld.com
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Mikey
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 631
Joined: Fri Jun 27, 2008 9:00 pm

Re: Multi Threadedness ?

Post by Mikey » Thu Aug 09, 2018 3:11 pm

I want to apologize to everyone for being snarky and petulant on this thread since I jumped in. Really, this comes down to one thought.

My company's mission-critical software has to have it. LC doesn't offer it.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 996
Joined: Thu Apr 11, 2013 11:27 am

Re: Multi Threadedness ?

Post by LCMark » Thu Aug 09, 2018 3:34 pm

@Mikey: As I said above none of the issues you listed suggest to me that they would solved by true multi threading... So the best thing to do is to write me and tell me what problems you are having that you need to solve for your mission critical software and maybe I can help.

I strongly doubt adding low-level pre-emptive threading primitives to LiveCode would help (even if we could add such things in a reasonable timeframe - which we simply cannot!).

However such a conversation might lead to additions to the language which not only solve your problem but also might do so in an easier way, and make it easier for every other LiveCode developer!

Sometimes if you have your head stuck in a problem for a long time you get blinkered in terms of possible solutions and taking the time explain to someone else can help find a simpler/better path... Particularly when the person you are explaining to happens to know more than just a little bit about such things 😉

P.S. Without understanding the full outline of your system and the problems - im just one of the blind men trying to work out what an elephant looks like!

Mikey
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 631
Joined: Fri Jun 27, 2008 9:00 pm

Re: Multi Threadedness ?

Post by Mikey » Thu Aug 09, 2018 4:46 pm

Mark,
What you are proposing is the very definition of a workaround. Workarounds aren't good enough for mission-critical systems. They have to just function. They have to recover when things go badly and no one with admin credentials is there to fix it.

The mergGoogle issues are really (really) annoying, but that code isn't mission critical, so it will stay in LC for now. The moment that system becomes mission critical, it's gonna be ported to one of the other RAD tools we have here, all of which have some variant of threads or tasks or processes.

Support for different windows or stacks in different threads isn't exactly the same thing, until there's an answer dialog thrown by one of them. Then it's exactly the same thing. In one of those other tools, one window is stopped. In LC, everything is stopped.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 996
Joined: Thu Apr 11, 2013 11:27 am

Re: Multi Threadedness ?

Post by LCMark » Thu Aug 09, 2018 5:13 pm

@Mikey - no what I'm proposing is no way a work-around. If you think it is then I'm not sure you actually get what a 'very high level' language is about (although I think you must do judging by our interactions in the past, which is why I'm slightly puzzled by your vehemently negative reaction to my proposal!).

Mission-critical systems require that they are correct. That they do not fail - or if they do fail (because, at the end of the day, no system can be made 100% reliable - all hardware is subject to failure rates), they recover in all circumstances without loss of data, or propagation of errors. (Although the latter is usually as much about the design of the system's architecture as it is about anything else, or the language of choice, or whatever concurrency primitives it may or may not have - given that, UNIX systems never used to have threads or locking primitives, or semaphores, and a great many mission-critical systems used to be run on such systems).

I'm sure you know full well what the biggest sort of failure in mission critical systems are - it is human error. Usually not at the operator level - because a mission-critical system should be designed to cope with erroneous human input, or by an immense amount of training, validation, testing and accreditation etc. of the operators - e.g. pilots flying jumbo-jets with 800+ people on board; or air traffic control operatives; or operatives of nuclear power stations.

So, assuming that the human error at the operator level has been mitigated, the only other source of human error in such systems which can lead to catastrophic failures is programmer error in the code that they write which makes the system run.

The way to reduce human error in programming a system is to ensure that the programmers can express the intent of a system at the highest possible level in such a way that the chance of programmatic errors is virtually eliminated.

For example, if you're problem is modal dialogs blocking with wait-with-messages - then we can look at adding a callback variant (which has been on the cards for ages). Unfortunately, the mergGoogle issue sounds like that it is actually the way you are using mergGoogle and the 'recursive wait problem'. I can probably help you fix that in your code - but only if I understand what you are trying to achieve and know what code you are using (I've done so for several other people producing large systems, who have faced similar problems in recent years).

However, as I said before, I really have no idea what you are trying to do - and giving lists of low-level features you think you could use from other programming environments doesn't really help me very much visualize how things fit together as a whole; and unless I can see how features fit together in whole-livecode-language-system-sense, then not a great deal of progress can be made (towards what you want, anyway) [ There are huge constraints in terms of what is and isn't possible with our code-base in any particular period of time - but I've become pretty good at navigating that over the last 15 years ].

At the end of the day I've offered, quite openly, for you to (privately!) let me help see if we can figure out together what high-level patterns we can abstract from your 'mission-critical system' in terms of the concurrency support you have asserted you need for it - and then see how they can be folded into LiveCode in its very-high level way so you can actually ensure your mission-critical system can be just that. You never know, it might be that I end up seeing a much simpler model / architecture which will save you 100's of man hours in coding, bug fixing and maintenance!

If you aren't interested in doing that, then fine - I'll spend that offered time doing something else. It isn't like I don't have enough things on my todo list to be getting on with.

bogs
Posts: 2189
Joined: Sat Feb 25, 2017 10:45 pm

Re: Multi Threadedness ?

Post by bogs » Thu Aug 09, 2018 8:34 pm

I found RB late myself (v5.x, after they fixed a lot of problems from the earlier versions and .net had come out). At the time, even though I wasn't using VB for most of the heavy lifting I was using it for front ends and some of the lighter lifting. RB let me swing that the other way, not only prototyping and front end stuff, but to actual work load stuff that did indeed work across its supported platforms. I stayed with it till 2012, but it became (in my opinion) a much worse environment over those years. I still use 5.5 and 2006 a lot when I hit something here I haven't figured out how to resolve. During all those years, their version of thread management wasn't difficult to master, which is why I brought them up. I'm still not sure it is real multi-threading though.

I came to Delphi from Pascal, which I REALLY loved, and which probably is what draws me to this language. Delphi as you say had a lot of cool things going for it from my point of view. Aside from what you mentioned, being able to write a full program, compile it, and wind up with something almost as small and fast as C but clearly understandable code was .... I dunno, like finding the holy grail. Not to mention being able to access any ocx or com object, being able to use grids while your developing and have them update in real time... well, you know, no surprise I still have every version back to 5 on my machine. I still port code from there to FPC/Lazarus.
LCMark wrote:
Thu Aug 09, 2018 12:53 pm
One thing to remember is that Delphi, in particular, was a Windows thing. There aim was not cross-platform - it was to enable people to write Window's software much more easily using all the ideas and notions about such which existed at the time - but in an easier way.
I know Kylix never panned out the way Borland wanted it to (because I was already a 'nix user by the time it came out), but FPC and Lazarus did eventually fill the cross platform end of Delphis potential, and kept the multi-threading through all platforms.

On this page, they do echo pretty much the concerns you outlined above that any dev needs to be aware of when using multi-threading, but while they mention as you did that on Win is it a without need of any special unit, it can be deployed on other platforms with a (few) includes.
So, anyway, sorry for the digression - but - yes - Delphi had true pre-emptive threading.
I'm not sorry about the digressions, I myself like and am subject to them, and I'd much rather have the fullest set of thoughts you have no matter how disjointed they come out.

Besides, I think we agree on where it is truly useful to have threading available, aside from the 'hackey' thing you mentioned ( I don't know about mobile, since I have nothing to do with that end of programming ), but it looks like I'll have to learn much more to get to using this language in the way I use others to accomplish things like network comms and using multiple processors ( I think I mentioned Blender and it's rendering before ) to speed things up.

I really appreciate the explanation on blocking and non blocking, it made a lot of sense to me. I also can see the point about adding multi-threading to this language. While I think a few could certainly benefit from it (and I am conceited enough to believe myself one of those :P ), I'm certainly not blind to the difficulties you describe in adding it, as opposed to the greater number of users that would -
  1. probably not know it exists or how to use it or
  2. have it cause issues they complain to you about when they fumble it. You know, kind of like I am fumbling the existing tool-sets to deal with things :twisted:
*Edited to add that is quite a generous offer on your part a post or so above this one , I hope Mikey will take advantage of it. It certainly isn't something you see every day.
Last edited by bogs on Thu Aug 09, 2018 8:53 pm, edited 3 times in total.
Image

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 2717
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria
Contact:

Re: Multi Threadedness ?

Post by richmond62 » Thu Aug 09, 2018 8:41 pm

I'm not sorry about the digressions, I myself like and am subject to them
Why apologise?

This forum is full of Multi-threaded threads, and that is one of it charms. 8)

Tangentialism can often bring wonderful ideas to light.

Mikey
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 631
Joined: Fri Jun 27, 2008 9:00 pm

Re: Multi Threadedness ?

Post by Mikey » Thu Aug 09, 2018 9:15 pm

If anyone else is really interested in this topic then I think it should take place out in the open so we're not all doing this privately. I did email Mark privately. It was pretty short and simple, pointing out a few things that we are doing with RAD tools that compete with LC but are in my opinion a similar "level" of abstraction from silicon. All of the features I described originally in this thread are features that we have put to use in applications that are running (and were really easy for us to implement). In the case of one of our mission-critical applications, we have been able to get it to the point that it never, ever goes down to the point that we have to get a nerd involved.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 996
Joined: Thu Apr 11, 2013 11:27 am

Re: Multi Threadedness ?

Post by LCMark » Fri Aug 10, 2018 5:04 pm

More than happy to discuss out in the open - the reason I suggested privately was to give you an opportunity to describe an overview of the system etc. itself so I could understand what it is - oftentimes people are sketchy about discussing what things actually are in the open, and thus resort to list of features (which you basically did, even in a private email).

The reason lists of features usually don't work is because the features that can be implemented are constrained by existing structures and the implementation we currently have. So, usually, a higher-level understanding is required - so an abstract model can be derived which then leads to a better visualization of how such a model could be made easier by mapping to specific language features.

In this case you clearly are taking 4D as a more appropriate target for what your 'mission critical' system (whatever it is) needs, so why are you not using that, if it has all the features you need? As I said above - reducing programmer errors is best done by being able to express your program at the highest level possible.

I have skim read the 4D docs and I really can't see 4D as a competitor to LiveCode - they are apples and oranges as far as I can see - LiveCode is a general RAD/programming system, 4D is a vertical database application RAD tool... You can do the latter in LiveCode, but for domain-specific 'sitting direct on top of database application RAD', 4D would appear to allow a much higher level of expression for that specific kind of thing.

Given that 4D *only* got pre-emptive threads in R16 - which was released in January 2017 - it is an incredibly recent addition. So it isn't as if you've had the option to use those for very long.

Basically - my observations...

4D offers the notion of 'processes' and 'workers'.

Processes (which appear to be 'threads' in 4D speak) are co-operative (non-recursive wait, presumably) or pre-emptive (R16+) where the latter come with quite strict limitations on what you can do. Also pre-emptive processes are limited to server (presumably the 'processes' are run on the server), single-user mode, or compiled mode (presumably because then it can analyse to make sure you aren't breaking the rules).

Co-operative processes are essentially what 'non-recursive wait' would give us; and pre-emptive processes are essentially what 'tasks' would give us.

Workers are quite a neat concept from what I understand of them so far - they wrap the notion of process providing a message queue, and then another process can send a 'message' with arguments to the worker process - queuing the request. This essentially gives a way for one process to share state with another through computations performed. This could perhaps be seen as a generalization of 'send in time' - just where you have multiple pending message queues - one per (worker) process.

The key thing here is 4D processes could actually be implemented by either processes or threads, I suspect which they are implemented depends on context and OS and whether they have the need to display UI. As 4D's process / worker model is so high-level this would be entirely hidden from the developer - this is definitely *not* general multi-threading - it is multi-processing (thread/process distinctions not being a developer concern). It is quite a nice restricted model of IPC and parallel processing...

... Oddly enough not too far removed (well, pretty much the same, in fact!) from my suggestions to date. My suggestions would need to be expanded slightly though - task could be augmented with the message queue idea easily enough; and when I say 'non-blocking I/O' everywhere, then I also mean in open process (which is currently poll-requiring, rather than callback using).

Regardless of whether it is done as true threads or a farm of processes (on UNIX there's very little difference between the two beyond sharing address space) - the effect is the same.

In terms of what we could do now / later...

Now it would be possible to add with message variants of the modal (blocking) dialog popup commands - meaning that if you 'modal as sheet with message' - you won't get into the recursive wait problem.

It would be possible to add with message variants of the process commands - meaning you won't have to poll.

As long as you then did not use get url, or post url - and instead moved to using the Async versions of tsNet, and stopped using 'wait' alongside the already non-blocking mergGoogle - all the issues which appear to come down to incorrect use of 'wait with messages' would be resolved.

Realistically 'tasks' and non-recursive (i.e. co-operative) processes are quite a way off... However, it would be possible to relatively easily build the UI to drive a farm of processes to do the heavy-lifting. This is essentially I imagine how 4D works internally; and I think covers the rather terse descriptions of things you listed in your email.

(I'm pretty sure that a good deal of 4D's multi-processing capabilities could actually be implemented as a well designed script library - based upon open process and elimination of the use of 'wait with messages' in calling code - i.e. without non-recursive wait and workers. So only a few smallish engine changes would be required.)

richmond62
Livecode Opensource Backer
Livecode Opensource Backer
Posts: 2717
Joined: Fri Feb 19, 2010 10:17 am
Location: Bulgaria
Contact:

Re: Multi Threadedness ?

Post by richmond62 » Fri Aug 10, 2018 6:15 pm

Cripes LCMark, when do you get time to practise the piano?
-
Here's a set of songs by Thomas Arne
(the chap who gave that fat slob, Handel and his live-in boyfriend Jimmy Quinn,
a run for their money) sung by a Russian devochka:

https://www.youtube.com/watch?v=jb5WvIzC_eA

"And what, pray tell," thou mayest ask, "hast this to do with Livecode?"

and the answer is, "Everything!"

SparkOut
Posts: 1793
Joined: Sun Sep 23, 2007 4:58 pm

Re: Multi Threadedness ?

Post by SparkOut » Fri Aug 10, 2018 7:56 pm

You put that so well Richmond.

Hats off to LCMark

Mikey
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 631
Joined: Fri Jun 27, 2008 9:00 pm

Re: Multi Threadedness ?

Post by Mikey » Fri Aug 10, 2018 7:56 pm

The reason for going generic in my email to you is because I don't want to spend an hour doing an LCMark on a message explaining what we do until you had a background on the tools we use to do it (what's a "worker"?).

I'm going to ignore Xojo for this conversation, since it's model is cooperative, and I would consider myself much more of a 4D hack than a Xojo/RB hack.

The question as to "why not do everything in 4D" is simple. I like LC better, and it's my money. In the 90's I was a HC fanboi. I became an "expert" in 4D because after grad school I got a job as a consultant. Then HC was allowed to die. When I came back to Erie to the family firm, 4D came with me, so it is one of the tools we started using to build our systems. One of those systems, our material management system, became mission-critical because it did things that none our customers or our customers' customers software could, namely manage the inventory and movement for the entire supply chain without any of them having to pay for another module or hire another person.

We use a lot of 4D. We also use a lot of eloquence. LC is definitely the third-place tool, but it's still my favorite and it seems that fun things happen a lot faster in LC than they happen in most other tools (like mobile, cloud storage, my ability to get someone to write an external to let me drive google sheets).

4D's multitasking didn't begin in 2015. It was first introduced in 4D v.6 in 1996 (so 20+years, not 3 years). Before v2015, 4D was using a simulated tasking model (for lack of a better word - processes were really only single-threaded but they behaved like they were multithreaded, because the 4D engine managed the slices).

Since then, our material management system has been using 4D's "process" management to
[*] Manage multiple simultaneous independent windows. The main functional areas all open in their own separate windows in separate "processes". If you read the 4D docs you will know that 4D can run in both compiled and interpreted mode. Today, in compiled mode, the processes are all independent. In interpreted they are walled gardens but they don't get their own cores. In all cases, it is hard for one process to break another. In case anyone cares, 4D has three types of variables: local (only have scope in the current "script", process (global to the process), and interprocess (global across all processes operating in the 4D instance).
[*] Manage multiple simultaneous processes that do everything from sending emails to printing reports to pulling/pushing data to/from our ERP software, mobiles, and the cloud. We have an internal CRON process running inside of 4D that manages these tasks firing off periodically to do their work, with live rescheduling depending on a variety of events (for example, if you complete a shipment, the emailer is scheduled to fire immediately to clear the email-out queue, instead of waiting until the next time it is scheduled to run). The emailer will not be rescheduled for lower-priority emails.
[*] Mange multi-stage commits due to record-locking issues, distributed hardware, or network connectivity. If you can find them, I wrote a three-part series on multi-stage commits in 4D for "Dimensions" journal that relies heavily on these ideas. (A multi-stage commit is a way to sync multiple independent databases or reconcile multiple transactions in the same table in the event of record locks. The U. S. Navy, for example, used 24-stage commits the last time I checked. In my writeup for Dimensions it will go a little bit further afield as I was showing everyone how to extend the idea to multi-process/task/thread/user setups using triggers, so there is a bit of trigger code in there, too, not just process management code.

Despite the original tasking/processing/threading model not being an independent model, it has been remarkably stable. Geeks don't get called because something broke, and took everything else with it. If something bad happens, the process is either automatically killed or allowed to time out and die, and then the system tries again until it either succeeds or exceeds the threshold for calling for help.

That is the system that most heavily uses the features that I've been proposing. Our other software that is written in 4D also take advantage of those features, but they don't push the envelope so hard.

Did I give enough detail for what we're trying to do? If not, you know how to find me and we can discuss implementation further.

To your comment on 4D's processes, see above. They are "processes" due to 20 years of history in that product. They only recently became truly independent and then only on compiled apps. The first question I was trying to pose in the very beginning of this thread was does it really matter? Do you really need "true" multithreading, or would Xojo's cooperative model, or 4D's 20-year-old supervised model suffice? If they're walled off from each other and they act like they're independent, then it's a feature that you don't have now. I don't really care if they're on their own core or if I can find them in the process monitor. We ain't disassembling a binary here, we're trying to do cool stuff as fast as we can.

[removed last three paragraphs on script library - misread your suggestion on that point]
Last edited by Mikey on Fri Aug 10, 2018 10:21 pm, edited 1 time in total.

Post Reply

Return to “Feature Requests”