External Dev Kit vs LiveCode Builder

LiveCode Builder is a language for extending LiveCode's capabilities, creating new object types as Widgets, and libraries that access lower-level APIs in OSes, applications, and DLLs.

Moderators: LCMark, LCfraser

Post Reply
PaulDaMacMan
Posts: 324
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

External Dev Kit vs LiveCode Builder

Post by PaulDaMacMan » Mon Apr 13, 2020 12:15 am

Thanks to the Stay At Home campaign thingy I now have an Indy license for a year, I don't do any commercial development but I figured maybe I would help a little bit to pay some bills at the LC mothership... but since I did, does this entitle me to some extra support or something? I really would like to get the last couple of pieces worked out in a couple of my projects, at least get a clearer picture of how LCB and the engine work together. Who should I ask? Who has the best over all knowledge of LCB? Is there someone specifically in charge of LCB?

I've been looking at the iOS external SDK / guide and it seems like a lot of it corresponds to bits of LCB/FFI but with certain things missing from LCB. I think these are things that I may need to get some problematic things working in LCB.

It seems LC runs on two threads, one for the main app/UI, and a seperate thread for script execution. That's a separate RunLoop? Blocks were recently added to the Objective C module, are Blocks executed on the main/system as in 'LCRunBlockOnSystemThread''? Is a Block passed to a foreign function 'owned' by the LC engine / main app thread?
Where does the LCB fall in with this setup? Is LCB / FFI in a separate "context" or thread or RunLoop ? What is the RunLoop mode? Are there things that can't be done with LCB/FFI that CAN be done with an External?

From the iOS External Dev Guide:

Code: Select all

 Threading
The engine runs using two threads in a co-operative fashion with only one executing at a time. The first (main/system) thread is where all system related calls are made – for example, manipulating UIViews or any other system-provided classes. The second (engine/script) thread is where code that executes scripts run.

The reason for this separation is the existence of ‘wait’ – the ability to block an executing handler while events are processed. On iOS, in order to process events, the system thread must return to the core event loop which iOS manages; however waiting requires running the event loop from nested calls. To make this work, when the engine thread invokes a wait, the engine jumps to the system thread, jumping back when an event occurs.

As most iOS system classes are UI / RunLoop related, it is critical that they be manipulated on the system thread, rather than the engine thread. To do this there are two choices:

If an external handler does not use any LC API calls that might call script (and thus invoke ‘wait’) then it can be marked as a ‘tail’ call in the idl – for example:
tail function rreMicrophoneIsAvailable
If marked as a ‘tail’ call then the handler will be invoked on the system thread, meaning no extra code is needed.

If an external handler does use LC API calls that might call script (and thus invoke ‘wait’) then ‘tail’ cannot be used. Instead, there are two new LC API calls LCRunOnSystemThread and LCRunBlockOnSystemThread . These allow code to be execute on the system thread, from the engine thread. See the SDK examples for ways this is done.
The reason I'm interested in CFRunLoop, Block ownership, threading, etc. is because of the crashes I keep getting with CoreMIDI and block callbacks. They look like this:

Code: Select all

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x00007fff3fea80c0
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x7fff3fea80c0:
    MALLOC_TINY            00007fff3e800000-00007fff3fe00000 [ 22.0M] rw-/rwx SM=PRV  
--> MALLOC_TINY            00007fff3fe00000-00007fff3ff00000 [ 1024K] rw-/rwx SM=COW  
    MALLOC_TINY            00007fff3ff00000-00007fff40000000 [ 1024K] rw-/rwx SM=PRV  

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   ???                           	0x00007fff3fea80c0 0 + 140734265721024
1   com.apple.audio.midi.CoreMIDI 	0x0000000110f156f4 _XNotify + 74
2   com.apple.audio.midi.CoreMIDI 	0x0000000110f316dd mshMIGPerform + 248
3   com.apple.CoreFoundation      	0x00007fff9a10a7e9 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 41
4   com.apple.CoreFoundation      	0x00007fff9a10a761 __CFRunLoopDoSource1 + 465
5   com.apple.CoreFoundation      	0x00007fff9a1023d5 __CFRunLoopRun + 2389
6   com.apple.CoreFoundation      	0x00007fff9a101824 CFRunLoopRunSpecific + 420
7   com.apple.HIToolbox           	0x00007fff99661ebc RunCurrentEventLoopInMode + 240
8   com.apple.HIToolbox           	0x00007fff99661bf9 ReceiveNextEventCommon + 184
9   com.apple.HIToolbox           	0x00007fff99661b26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
10  com.apple.AppKit              	0x00007fff97bf6a04 _DPSNextEvent + 1120
11  com.apple.AppKit              	0x00007fff983727ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
12  com.runrev.livecode           	0x000000010873fcb4 0x108420000 + 3275956
13  com.runrev.livecode           	0x0000000108731818 0x108420000 + 3217432
....
As soon as the callback block is triggered and the app crashes.

The callback I'm trying to use in this specific crash is MIDINotifyProc which has this note on Apple's dev site:
Discussion
The MIDINotifyProc callback function is invoked on the same thread on which you called the MIDIClientCreate function
I think "The same thread on which you called the MIDIClientCreate" would be whatever thread the LCB FFI library is running in.
Last edited by PaulDaMacMan on Mon Apr 13, 2020 8:53 pm, edited 1 time in total.
My LCB Repos: https://github.com/PaulMcClernan/

PaulDaMacMan
Posts: 324
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

Re: External Dev Kit vs LiveCode Builder

Post by PaulDaMacMan » Mon Apr 13, 2020 3:09 am

Well of course I got it working right after I posted, smdh!
Screen Shot 2020-04-12 at 9.41.37 PM.png
I can see (screenshot) the callback firing and being logged along with the two elements of the MIDINotification struct. Yeah! In fact it seems it's firing 4 times, I guess once for each MIDI source/destination, for each change I make to my setup in Apple's Audio/MIDI setup app.

BUT the way I got it working was by using a pretty much undocumented alternate API call "MIDIClientCreateWithBlock" (as found here: https://developer.apple.com/documentati ... guage=objc) instead of the regular "MIDIClientCreate", unfortunately Apple's Dev site says this call is only available from iOS 9+/macOS 10.11+ which bumps the minimum OS requirement for my library up to a lot newer major OS versions. This "MIDIClientCreateWithBlock" gets rid of the "refCon" (reference context) parameter completely. Maybe that param was indeed the problem area of my code and so this did the trick?

I'd still like to understand more about LC/LCB's threading, context, runLoop modes, LCB vs Externals, etc. But for now at least I'm ready to try moving on to the readProc callback that I need to receive actual MIDI-in data from sources outside of LC.

Probably important to remember: I don't think LCB created messages posted back to LC actually show up in the message watcher window unless there is a script handler for the message somewhere in the message path.
Last edited by PaulDaMacMan on Mon Apr 13, 2020 9:02 pm, edited 1 time in total.
My LCB Repos: https://github.com/PaulMcClernan/

bogs
Posts: 4791
Joined: Sat Feb 25, 2017 10:45 pm

Re: External Dev Kit vs LiveCode Builder

Post by bogs » Mon Apr 13, 2020 10:50 am

Hey Paul, Good to hear you found undocumented stuff, pretty much what I live for doing myself haha :D

I don't have an answer to any of your important questions, but, this ~
... but since I did, does this entitle me to some extra support or something? I really would like to get the last couple of pieces worked out in a couple of my projects, at least get a clearer picture of how LCB and the engine work together. Who should I ask? Who as the best over all knowledge of LCB? Is there someone specifically in charge of LCB?
As Richard (I think) and a few others explain it, and I am sure you know, the forums here are populated by your fellow users (I KNOW you are not asking about the forums, this was for others in general who might ask the same question :) )

For support like what your talking about, I would contact Heather or just simply use the support link at the motherships homepage. The process isn't instantaneous, but it isn't exagerated either. They will look at your question, and get the answer from who they determine knows it I would assume.

Just in case you've never tried it, I've contacted them many times and have never had a paid for version of anything. They have unfailingly contacted me back (depending on the difficulty of the question and whether a holiday was coming up) either the same day or with in a reasonable day or two type period.

Now, if I misunderstood any part of what you were referring too, feel free to say "You IDIOT!!", I'll understand 8)
Image

PaulDaMacMan
Posts: 324
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

Re: External Dev Kit vs LiveCode Builder

Post by PaulDaMacMan » Mon Apr 13, 2020 8:43 pm

bogs wrote:
Mon Apr 13, 2020 10:50 am
Hey Paul, Good to hear you found undocumented stuff, pretty much what I live for doing myself haha :D
I've found a dev blog post talking about the additions to the API which were apparently added to work around problems using CoreMIDI with Swift 1.x
It seems the returned data has also changed, since I noticed neither values of the elements of the struct that my block is receiving are anything remotely close to matching the values that correspond to the constants in the CoreMIDI API... Yet another mystery!
For support like what your talking about, I would contact Heather or just simply use the support link at the motherships homepage. The process isn't instantaneous, but it isn't exagerated either. They will look at your question, and get the answer from who they determine knows it I would assume.

Just in case you've never tried it, I've contacted them many times and have never had a paid for version of anything. They have unfailingly contacted me back (depending on the difficulty of the question and whether a holiday was coming up) either the same day or with in a reasonable day or two type period.
Yeah, I'm probably looking for more lower-level, technical information than most LC scripters are interested in. Even if I don't need to go that low of a level to do what I'm out to do, I'd still like to know. I think it must be possibly create high-priority threads and such, separate from the two threads than LC runs in.

Good to know, the team is that responsive via email. I frequently have gone to the Gitter.im LC web chat room, Ali, Monte, Trevor, etc. are often there, but it's been a bit dead there lately. I'm not getting any answers for these questions.

Thanks for your response either way. Sometimes I feel like... I Talk to The Wind, the wind cannot hear!
My LCB Repos: https://github.com/PaulMcClernan/

Post Reply

Return to “LiveCode Builder”