LiveCode Builder Foreign Function Interface for Beginners

[SirWobbyTheFirst]

Hello everyone,

I’m looking for a beginners guide to utilising LCB to bind to third party library DLLs and SOs and expose functions found within to LiveCode. When I first started with Revolution 2.0, there was a great Hello World tutorial stack that described everything line by line in a way that helped me understand the language. Something like that for LCB would be amazing.

I have a couple ideas for implementing protocols via third party libraries. The current idea is to build a wrapper around the LibUnbound library from the Unbound DNS project to implement support for complex DNS resolution beyond the HostNameToAddress() functions.

Implementing this could open up support for utilising SRV, CNAME, TXT, MX records that could be utilised for network and internet applications. It would also allow using third party DNS servers such as 1.1.1.1.

What do we think?

[SparkOut]

You're right, something like that would be amazing. It's something I would dearly love to see, and although there is some information out there, it's hard to find, limited and quite opaque to someone trying to take first explorations.
I asked a similar question some time ago, https://forums.livecode.com/viewtopic.php?f=93&t=30429 and through great support, I was able to make a wrapper for an experimentation board dll.
This got me through my needs for that project, and I learnt a lot but still have not got a handle on how to break into the understanding of what means what and why. It remains my only successful project in LCB - through lack of time to investigate properly, but there's nothing that's ever looked like a route to try investigating.

[stam]

Such a beginners guide (or preferably guides) is sorely missing.
LCB is potentially powerful but the limited documentation makes it extremely onerous to pick up. Like I've said before, LCB is LC's unwanted love child

Edited to cater for Richmond's sensitivities...

[richmond62]

Who is making off-colour remarks about red hair?

Go away and wash your mouth out with soap.

I have red haired, was always wanted, and loved, and my parents were married.

Funny: you'd never make offensive remarks about people with brown skin: but you wouldn't hesitate to be grossly offensive about people with red hair!

[stam]

Who is making off-colour remarks about red hair?

Go away and wash your mouth out with soap.

I have red haired, was always wanted, and loved, and my parents were married.

Funny: you'd never make offensive remarks about people with brown skin: but you wouldn't hesitate to be grossly offensive about people with red hair!

No offence meant - it's common expression round my parts.
If you're touchy about red hair I apologise.

But please do not equate this with expressions of racism. In no way does the plight of red haired people equate to the hardships of people with brown skin and that you equate these is incredibly crass.

[richmond62]

Having experienced years of physical bullying at school, and knowing many other redheads who have; being told that I am stupid because I'm a redhead, and a "stupid Jock" in England (the second home of European racism), the equation rests.

I am not touchy about my red hair, but I am touchy for the sake of younger red heads who are being bullied about their hair every day.

I am sure that someone round these parts probably feels much the same re remarks about the size of Greek bottoms.

So, stop and think a wee bit; that's all.

[richmond62]

Oh, and while I'm here, I may not be stupid because I have red hair, but I do feel fairly stupid when I look at LCB.

An introductory guide to LCB, or at least as much help as is supplied to learn LiveCode would be wonderful.

The first thing that, IMHO, needs to be made explicitly clear it its relationship with LiveCode:

1. Is LCB a full-blown programming language which can be used completely independently of LC?

2. Is LCB a sort of semi-complete language which can only be used 'hooked on' to LC to extend LC's capabilities?

3. Something somewhere between #1 and #2?

[richmond62]

As a 10 year old I know (my younger son, 18 years ago) managed to produce a functioning calculator ("F*ck off, Dad, I can do it myself.") with ONLY the LC setup, it would be super if there were a way a 10 year old could do something with LCB completely by themselves . . .

. . . I have criticised the Dictionary many times . . .

But it is considerably better than nothing.

[richmond62]

To expand:

If: 1. LCB is a full-blown programming language which can be used completely independently of LC.

Then a self-study pack for starters would really have to follow the pattern of most self-study packs for starters for other programming languages (preferably missing out the vacuous "Hello World" gubbins).

If: LCB is a sort of semi-complete language which can only be used 'hooked on' to LC to extend LC's capabilities

A self-study pack for starters would have to start with LiveCode (and here one of the difficult bits would be to decide on what level of competence the LCB starter should have with LiveCode) and proceed from there.

In an ideal world this would probably have to be worked out with several focus groups:

1. Half a dozen 9-12 year olds who are "wet behind the ears."

2. Half a dozen adults who are "wet behind the ears."

3. Half a dozen adults who have worked with LiveCode for quite some time.

4. Half a dozen adults who are competent in programming languages other than LiveCode.

And, quite obviously, whoever were to conduct these focus groups who have to be competent in LCB already (so, no, Richmond is not offering to do any of these focus groups, for the simple reason that he knows less about LCB than about speaking Cambodian).

Like anything else that does not result in a half-c*ck job this would involve time and money.

[jacque]

LCB is a more x-talk way to write LC externals and thus is linked exclusively to LC. It can't be used to write independent programs.

[SirWobbyTheFirst]

Hi Everyone,

So I’ve been keeping track of this thread along with the other one mentioned and also had a look at a LiveCode Lessons presentation for LCB about working on widgets and creating libraries. Nothing about FFI but I’ve been perusing the GitHub’s of some known widget devs and it seems like it might be easy enough.

I haven’t had a chance to try LCBs FFI functionality yet as I’ve been busy migrating VMs but the API for LibUnbound looks simple enough, so I will start chipping away at it once I’ve finished these migrations.

I’ll upload the source code to GitHub so once I do, I’d appreciate feedback and help as I’ll likely be a bit rough during learning.

Mike

[mwieder]

Peter Brett had a couple of useful blog posts back in the day:

https://blog.peter-b.co.uk/2015/08/live ... e-bit.html
https://blog.peter-b.co.uk/2015/09/usin ... ecode.html
https://blog.peter-b.co.uk/2015/09/foun ... ilder.html

Hope this helps.

[PaulDaMacMan]

I'll break down a C API binding string from that other thread:

private __safe foreign handler _ReadBackAnalogOut (out rBuffer as CInt) returns nothing binds to "c:K8055D>ReadBackAnalogOut!stdcall"

private -- means this handler will be invisible outside of the context of the current Extension module.

__safe -- there is safe and unsafe context, I'm a bit sketchy on this. The docs say: Foreign handlers are always considered unsafe, and thus may only be called from unsafe context - i.e. from within an unsafe handler, or unsafe statement block. The _safe keyword seems to be undocumented, I've only seen it used with C APIs, but the bottom line is foreign handlers that have this keyword can be called outside of a safe/unsafe block.

foreign handler -- as opposed to a regular handler, this let's us know we are binding to some external code

_ReadBackAnalogOut -- the name we'll be using to call this foreign handler, naming conventions here are up to your own discretion

(out rBuffer as CInt) -- inside the parens are declaring what the the foreign handler is expecting to receive (in) or send back (out) or both (inout) when this external code is being called. In this case it's declaring that the foreign code will return a parameter that should be a C signed integer (as CInt) but you might also be able to use LCB's type 'as Number' or any other variable type that's compatible with a C signed int. When I say compatible what I mean is the data can fit, byte size wise, into the type you're declaring. So make sure you're using the correct type. A unsigned C integer (as CUint) can fight a signed int, but will give you the wrong number range. rBuffer is an arbitrarily named variable but the lead 'r' is a commonly used convention for indicating it's a variable that's being return.

returns nothing -- the foreign code will return nothing (null) at least not in the normal way, in the case of this handler the variable you pass for 'rBuffer' is will be filled (out) with the result of the foreign code. With many APIs the return value here will simply be 'returns CBool (or Boolean) that indicates success or failure of the call.

binds to -- Always needed, just indicates the next quoted string is the actual binding string for the foreign handler, that gives libFFI some clues about the code being called.

"c:K8055D>ReadBackAnalogOut!stdcall" --
The binding string starts with "c:" which says the library or API we want is C based. It can be one of c: java: or objc: (and maybe others? js:? And I see mention of Pascal in the docs).
Followed by K8055D>, so K8055D is the library or API name to look for, followed by '>'. The actual foreign code library can be included in platform_cpu specifically named folders in your extension project folder, or the foreign code library may be an API (or .Framework for MacOS) part of the OS or in user installed common shared library folder in the OS. I'm not sure of the exact order that memory or directories are searched for libraries / function symbols. Like if there were two copies of the same library available, I'm not sure if it would load the included copy first or another one in the system one first.
ReadBackAnalogOut -- ReadBackAnalogOut is the name of the actual function we want to use from the foreign code library.
!stdcall is the calling convention, with C on windows this is one of default, stdcall, thiscall, fastcall, cdecl, pascal, register All but default are Win32-only. I've only had to change this once so far, most of the time you probably don't need to worry about it. On Java / Android this can be one of instance, static, or nonvirtual. I haven't done much with Android in some time so I don't know anything about those calling conventions.

Now in some regular handler you then call like _ReadBackAnalogOut( tMyCint ), tMyInt would be an integer variable you've already declared in the calling handler with 'variable tMyInt as CInt' or variable tMyInt as Integer'.
A lot of times the the returned data or object may need further processing to get the data into a format that's useful for our scripting language. For example ObjC's NSColor Red Blue Green components might be floating point numbers between 0.00000 and 1.00000, so we would multiply them by 255 to get to be 8 bit RGB values we can use with our scripts language.

The binding strings look a bit different depending on if you're binding to c:, java:, or objc: foreign code. You can, and in the case of Apple probably need to, mix c: and objc: APIs together in the same module (C based CoreAudio AND Objective C based AVAudio APIs for example)

The foreign code is only resolved when it's called so you won't know if you got the foreign binding strings correct until you actually try to use that foreign handler.

Besides knowing how to bind to the foreign code, you will probably need to learn about the API you're trying to use first. And of course there's bound to be some 'gotchas' along the way too. Like for examples dealing with returned data that's in a different byte order, or maybe you get back a Pointer to data or an Object that exists elsewhere and need to parse the block of data a pointer points to (variable foreign type declarations is a whole other thing to know) or need to query an object for its relevant properties.

Once you start to get the feel of it, it does get a little easier. I've used ChatGPT to type out a lot of binding strings for me nowadays, and most of the time it even makes valid binding strings.

https://github.com/OpenXTalk-org/OpenXT ... -naming.md
https://github.com/OpenXTalk-org/OpenXT ... form-id.md
https://github.com/OpenXTalk-org/OpenXT ... cedence.md
https://github.com/OpenXTalk-org/OpenXT ... ference.md
https://github.com/OpenXTalk-org/OpenXT ... 20Guide.md
https://github.com/OpenXTalk-org/OpenXT ... g%20OXT.md
https://github.com/OpenXTalk-org/OpenXT ... ference.md